beautypg.com

Calculation range and precision, Function calculation input ranges and precision, E-31 – Casio FX-100MS User Manual

Page 32

background image

E-31

The following shows the two-digit numbers for each of the metric
conversion commands.

01: in

' cm

02: cm

' in

03: ft

' m

04: m

' ft

05: yd

' m

06: m

' yd

07: mile

' km

08: km

' mile

09: n mile

' m

10: m

' n mile

11: acre

' m

2

12: m

2

' acre

13: gal (US)

'R

14:

R' gal (US)

15: gal (UK)

'R

16:

R' gal (UK)

17: pc

' km

18: km

' pc

19: km/h

' m/s

20: m/s

' km/h

21: oz

' g

22: g

' oz

23: lb

' kg

24: kg

' lb

25: atm

' Pa

26: Pa

' atm

27: mmHg

' Pa

28: Pa

' mmHg

29: hp

' kW

30: kW

' hp

31: kgf/cm

2

' Pa

32: Pa

' kgf/cm

2

33: kgf

m

' J

34: J

' kgf

m

35: lbf/in

2

' kPa

36: kPa

' lbf/in

2

37: °F

' °C

38: °C

' °F

39: J

' cal

40: cal

' J

Conversion formula data is based on the “NIST Special Publication
811 (2008)”.
Note: The J

' cal command performs conversion for values at a

temperature of 15°C.

Calculation Ranges, Number of
Digits, and Precision

Calculation Range and Precision

Calculation Range

± 1 × 10

–99

to

± 9.999999999 × 10

99

or 0

Number of Digits for
Internal Calculation

15 digits

Precision

In general,

± 1 at the 10th digit for a single

calculation. Precision for exponential
display is

± 1 at the least significant digit.

Errors are cumulative in the case of
consecutive calculations.

Function Calculation Input Ranges and
Precision

Functions

Input Range

sin

x

cos

x

DEG

0

|

x

| 9 × 10

9

RAD

0

|

x

| 157079632.7

GRA

0

|

x

| 1 Ч 10

10

This manual is related to the following products: