beautypg.com

2 calculation algorithms, Calculation algorithms – Metrohm viva 1.0 Manual User Manual

Page 27

background image

■■■■■■■■■■■■■■■■■■■■■■

2 General program functions

viva 1.0

■■■■■■■■

15

2.3.2

Calculation algorithms

Dialog window: Formula editor

Numerical format
The IEEE 754 (1985) standard for binary floating-point arithmetic is imple-
mented in "double precision" (64 Bit) in the software.

Rounding-off process
Measured values and results are rounded off symmetrically (commercial
rounding), i.e., 1, 2, 3, 4 are always rounded down whereas 5, 6, 7, 8,
9
are always rounded up.

Examples

2.33 yields 2.3

2.35 yields 2.4

2.47 yields 2.5

-2.38 yields -2.4

-2.45 yields -2.5

Statistics
The mean value as well as the absolute and relative standard deviation of
results R are calculated using the following formulas:

Mean value

Absolute standard
deviation

Relative standard devi-
ation (in %)

The statistical calculations of the software have been implemented so that
they can be checked to as great a degree as possible by the user. The indi-
vidual values are therefore incorporated in the statistics with full accuracy.

It is not the number of decimal places which is decisive for the accuracy of
the calculations, but rather the number of significant digits of the decimal
numbers displayed. The binary 64-bit numerical format implemented on