Calculation range and precision, Function calculation input ranges and precision – Casio fx-991ES PLUS User Manual
Page 39
E-38
25: atm
Pa
26: Pa
atm
27: mmHg
Pa
28: Pa
mmHg
29: hp
kW
30: kW
hp
31: kgf/cm
2
Pa 32: Pa kgf/cm
2
33: kgf
•
m
J
34: J
kgf
•
m
35: lbf/in
2
kPa
36: kPa
lbf/in
2
37: °F
°C
38: °C
°F
39: J
cal
40: cal
J
Conversion formula data is based on the “NIST Special Publication 811
(1995)”.
Note: The J
cal command performs conversion for values at a temperature
of 15°C.
Calculation Ranges, Number of Digits,
and Precision
The calculation range, number of digits used for internal calculation, and
calculation precision depend on the type of calculation you are performing.
Calculation Range and Precision
Calculation Range
p1 s 10
–99
to
p9.999999999 s 10
99
or 0
Number of Digits for Internal
Calculation
15 digits
Precision
In general,
p1 at the 10th digit for a single
calculation. Precision for exponential
display is
p1 at the least significant digit.
Errors are cumulative in the case of
consecutive calculations.
Function Calculation Input Ranges and Precision
Functions
Input Range
sin
x
DEG
0
\
x
\
9
RAD
0
\
x
\
GRA
0
\
x
\
10
cos
x
DEG
0
\
x
\
9
RAD
0
\
x
\
GRA
0
\
x
\
10
tan
x
DEG
Same as sin
x
, except when
\
x
\ = (2
n
–1)
s
RAD
Same as sin
x
, except when
\
x
\ = (2
n
–1)
s
GRA
Same as sin
x
, except when
\
x
\ = (2
n
–1)
s
sin
–1
x
0
\
x
\
cos
–1
x
tan
–1
x
0
\
x
\
99
sinh
x
0
\
x
\
cosh
x
sinh
–1
x
0
\
x
\
99
cosh
–1
x
1
x
99
tanh
x
0
\
x
\
99