Metrohm 840 PC Control 5.0 / Touch Control - Manual (Page 337)

6 Appendix
PC Control / Touch Control
325
Explanation
The statistical calculations in the software have been implemented in
such a way that they are as revisable as possible for the user. This is
why the individual values used in the statistics are in the rounded pres-
entation form selected by the user.
It is not the number of decimal places which is decisive for the accuracy
of the calculations, but rather the number of significant digits of the
decimal numbers displayed. As a result of the binary 32-bit number
format implemented on the basis of the IEEE 754 standard, the decimal
numbers which are produced have 7 reliable significant decimal digits.
You can influence the number of significant digits by selecting the unit
and the number of decimal places. As the results unit to be set some-
times contains both the prefix "milli" and the physical unit itself, the
number of significant digits is altered accordingly in such a case by
three digits.
Example:
The displayed result 1234,56789 mg/L has 7 reliable digits. It is to be
rounded to three decimal places according to the rounding method
given above:
1234,568 mg/L (7 significant digits, 3 of which are
decimal places)
With the unit "g/L" the same result 1.23456789 g/L is also rounded to
three decimal places:
1.235 g/L (4 significant digits, 3 of which are
decimal places)
The number of significant digits has now been reduced by three to four
digits by omitting the prefix "milli".
You will obtain the smallest loss of rounding accuracy by selecting the
application and number format so that the displayed numbers contain
as many pre-decimal places as possible.
A complete recalculation of the statistics using a pocket calculator or a
PC calculation program can produce variations. This is caused by the
different binary number formats used in these instruments. Whereas the
Titrando calculate with binary 32-bit numbers as described above, PC
programs (e. g. MS-Excel) use a different binary format, e. g. IEEE 754
64-bit.
Note!
The losses of accuracy by rounding described above in the range of
the maximum reliable digits are only of theoretical relevance. Most of
the time they are lower by several orders of magnitude than – as an
example – the uncertainties resulting from weighing out the sample.