Yokogawa Button Operated DX2000 - Manual (Page 11)

10
3. Accuracy and Display Resolution
3.1. Measurement and Display Accuracy (Digital Display)
Computing method of the display value accuracy
¾ Input voltage X
¾ Lower limit of display span VL Upper limit of display span VU
¾ Scale lower limit SL Scale upper limit SU
¾ Digital display value Y
1) Converting equation for the input voltage (setting)
• Y = 10 (SU-SL)*(X-VL)/(VU-VL)+SL
2) Measurement accuracy of input voltage X (uses the negative side due to the
LOG characteristic)
• X
err
=X – (measurement accuracy of the voltage range)
3) Hardware error of the display value
• Y’= Y – Y
err
• The difference from the true value Y is hardware error Y’ when display
value Y
err
corresponds to input voltage X
err
.
4) Display accuracy of the mantissa
• Display accuracy = Y’* 1.1 + 1 digit