beautypg.com

2 floating point math, nan, and ±inf – Campbell Scientific CR9000X Measurement and Control System User Manual

Page 76

background image

Section 2. Data Storage and Retrieval

2.2.1.2 Floating Point Math, NAN, and ±INF

Table 2.2-1 lists math expressions, their CRBASIC form, and IEEE
floating point math result loaded into variables declared as FLOAT or
STRING.

TABLE 2.2-1. Math Expressions and CRBASIC Results

Expression CRBASIC

Expression

Result

0 / 0

0 / 0

NAN

-

(1 / 0) - (1 / 0)

NAN

(-1)

-1 ^ (1 / 0)

NAN

0 * (

-

)

0 * (-1 * (1 / 0))

NAN

±

(1 / 0) / (1 / 0)

NAN

1

1 ^ (1 / 0)

NAN

0 *

0 * (1 / 0)

NAN

x / 0

1 / 0

INF

x / -0

1 / -0

INF

-x / 0

-1 / 0

-INF

-x / -0

-1 / -0

-INF

0

(1 / 0) ^ 0

INF

0

0 ^ (1 / 0)

0

0

0

0 ^ 0

1

NAN and ±INF are presented differently depending on the declared
variable data type. Further, they are recorded differently depending on the
final storage data type chosen compounded with the declared variable data
type used as the source.

For example, INF in a variable declared as LONG is represented by the
integer -2147483648. When that variable is used as the source, the final
storage word when sampled as UINT2 is stored as 0. See Table 2.2-2
below.

TABLE 2.2-2. Variable and Final Storage Data Types with NAN and ±INF

Variable

Test

Variable's

Final Storage Data Type & associated stored value

Type Expression Value

FP2

IEEE4 UINT2

STRING BOOL LONG

1 / 0

INF

INF

INF

65535

+INF

TRUE

2,147,483,647

As FLOAT

0 / 0

NAN

NAN

NAN

0

NAN

TRUE -2,147,483,648

1 / 0

2,147,483,647

7999

2.147484E+09

65535 2147483647

TRUE

2,147,483,647

As LONG

0 / 0

-2,147,483,648

-7999

-2.147484E+09

0 -2147483648

TRUE -2,147,483,648

1 / 0

TRUE

-1

-1

65535

-1

TRUE

-1

As

BOOLEAN

0 / 0

TRUE

-1

-1

65535

-1

TRUE

-1

1 / 0

+INF

INF

INF

65535

+INF

TRUE

2,147,483,647

As STRING

0 / 0

NAN

NAN

NAN

0

NAN

TRUE -2,147,483,648

2-4