next up previous contents
Next: Maximum Data Values
Up: CONSTANTS
Previous: Bad Data Values

Machine Precision

Constants with names of the form VAL__EPS$<$T$>$ represent the machine precision when performing calculations using each data type. The machine precision $\epsilon$ is the smallest positive value such that ($1+\epsilon)$ can be represented on the machine and is distinguishable from 1 using the data type in question. The data types of these constants match the quantities they describe, as follows:

Constant Type Value Hexadecimal Pattern
VAL__EPSUB _UBYTE 1 01
VAL__EPSB _BYTE 1 01
VAL__EPSUW _UWORD 1 0001
VAL__EPSW _WORD 1 0001
VAL__EPSI _INTEGER 1 00000001
VAL__EPSR _REAL 1.1920929E$-$7 00003500
VAL__EPSD _DOUBLE 2.7755575615628914E$-$17 0000000000002500


next up previous contents
Next: Maximum Data Values
Up: CONSTANTS
Previous: Bad Data Values

PRIMDAT --- Processing of Primitive Numerical Data
Starlink User Note 39
R F Warren-Smith
28 February 1995
E-mail:ussc@star.rl.ac.uk

Copyright © 2009 Science and Technology Facilities Council