beautypg.com

Calculation range and precision, Function calculation input ranges and precision, E-52 – Casio fx-115ES PLUS User Manual

Page 53

background image

E-52

The following shows the two-digit numbers for each of the metric conversion
commands.

01: in

' cm

02: cm

' in

03: ft

' m

04: m

' ft

05: yd

' m

06: m

' yd

07: mile

' km

08: km

' mile

09: n mile

' m

10: m

' n mile

11: acre

' m

2

12: m

2

' acre

13: gal (US)

'R 14: R' gal (US) 15: gal (UK) 'R 16: R' gal (UK)

17: pc

' km

18: km

' pc

19: km/h

' m/s

20: m/s

' km/h

21: oz

' g

22: g

' oz

23: lb

' kg

24: kg

' lb

25: atm

' Pa

26: Pa

' atm

27: mmHg

' Pa 28: Pa ' mmHg

29: hp

' kW

30: kW

' hp

31: kgf/cm

2

' Pa 32: Pa ' kgf/cm

2

33: kgf

m

' J

34: J

' kgf

m

35: lbf/in

2

' kPa 36: kPa ' lbf/in

2

37: °F

' °C

38: °C

' °F

39: J

' cal

40: cal

' J

Conversion formula data is based on the “NIST Special Publication 811
(1995)”.
Note: The J

'cal command performs conversion for values at a temperature

of 15°C.

Calculation Ranges, Number of Digits,
and Precision

The calculation range, number of digits used for internal calculation, and
calculation precision depend on the type of calculation you are performing.

Calculation Range and Precision

Calculation Range

±1 × 10

–99

to

±9.999999999 × 10

99

or 0

Number of Digits for Internal
Calculation

15 digits

Precision

In general,

±1 at the 10th digit for a single

calculation. Precision for exponential
display is

±1 at the least significant digit.

Errors are cumulative in the case of
consecutive calculations.

Function Calculation Input Ranges and Precision

Functions

Input Range

sin

x

DEG

0

|

x

| 9 × 10

9

RAD

0

|

x

| 157079632.7

GRA

0

|

x

| 1 × 10

10

cos

x

DEG

0

|

x

| 9 × 10

9

RAD

0

|

x

| 157079632.7

GRA

0

|

x

| 1 × 10

10

tan

x

DEG

Same as sin

x

, except when

|

x

| = (2

n

–1)

× 90.

RAD

Same as sin

x

, except when

|

x

| = (2

n

–1)

× π/2.

GRA

Same as sin

x

, except when

|

x

| = (2

n

–1)

Ч 100.