beautypg.com

Calculation range and precision, Function calculation input ranges and precision, E-14 – Casio FX-55 User Manual

Page 15: F=5, 5 6, F=3, 6 7

background image

E-14

l=

F=5,

5
6

l=

5
6

When further simplification is not possible, “Simplified” appears momentarily
on the display and then the calculation result is displayed.

When a divisor of 3 is specified:

18
21

=

6
7

18

' 21 =l 3 =

F=3,

6
7

or 18

' 21 el 3 =

F=3,

6
7

Calculation Ranges, Number of Digits,
and Precision

The calculation range, number of digits used for internal calculation, and
calculation precision depend on the type of calculation you are performing.

Calculation Range and Precision

Calculation Range

±1 × 10

–99

to

±9.999999999 × 10

99

or 0

Number of Digits for Internal
Calculation

15 digits

Precision

In general,

±1 at the 10th digit for a single

calculation. Precision for exponential
display is

±1 at the least significant digit.

Errors are cumulative in the case of
consecutive calculations.

Function Calculation Input Ranges and Precision

Functions

Input Range

10

x

–9.999999999

× 10

99

Ϲ

x

Ϲ 99.99999999

'

x

0

Ϲ

x

Ͻ 1 × 10

100

x

2

|

x

| Ͻ 1 × 10

50

x

–1

|

x

| Ͻ 1 × 10

100

;

x

G 0

°’ ”

|

a

|,

b

,

c

Ͻ 1 × 10

100

; 0

Ϲ

b

,

c

The display seconds value is subject to an error of ±1 at
the second decimal place.

x

y

x

Ͼ 0: –1 × 10

100

Ͻ

y

log

x

Ͻ 100

x

= 0:

y

Ͼ 0

x

Ͻ 0:

y

=

n

,

m

2

n

+1

(

m

,

n

are integers)

However: –1

× 10

100

Ͻ

y

log

|

x

| Ͻ 100

a

b

/

c

Total of integer, numerator, and denominator must be
10 digits or less (including division marks).