General-purpose applications

In general-purpose applications, the data-quality values are regarded as a set of 8 independent masks, each of which is 1 bit deep. Whether a given pixel is to be included in the processing or not (i.e. whether it ``bad'') is determined by comparing its quality value with a bit pattern stored in a $<$_UBYTE$>$ data object [BADBITS] within the $<$QUALITY$>$ structure. The following logical expression is evaluated:

$$BAD = QUALITY \wedge BADBITS$$

where $\wedge$ is the logical AND operation.

Note that if a [BADBITS] mask is zero (i.e. all false), the corresponding data-quality mask is ignored. This can be used to turn off all 8 data-quality masks and allow inspection or processing of the pixels whatever their status. For a single bit, the above expression has the following truth table:

and the overall logical value of BAD is the OR of the results for all eight bits--just one of which has to be TRUE to make the resulting pixel bad.

An example may clarify this. Assume [BADBITS] is 01001010 (where the bits of the binary number are written with the most significant at the left, and are numbered from the right beginning with zero). For this [BADBITS] value, a pixel with a [QUALITY] value of 10100100 is interpreted as non-bad, because bits 2, 5 and 7, which are set in the data-quality value, are not set in [BADBITS]. However, a [QUALITY] value of 10100110 generates a bad value because bit 1, which is set in the data-quality value, is also set in [BADBITS]. If data object [BADBITS] is not present its value is assumed to be to be 00000000, and general-purpose applications will accept as ``good'' any pixel, irrespective of the corresponding data-quality value.

The rules and conventions for the processing of data-quality values and their associated data, taking into account the possible presence of undefined values, are as follows.

Rules

--

• Undefined pixels stay bad after processing.
• Undefined pixels generated during the processing (other than through data quality), e.g. logarithm of a negative data value, are propagated to the output data value.
• If processing would or might have changed the value of a pixel, had the pixel not been marked as bad through data quality, then it must propagate an undefined pixel. The input quality is propagated. In applications where data value will not have been changed as a result of the processing, the application is permitted either (1) to propagate the original data value and quality or (2) to propagate an undefined pixel.

Conventions

--

• When there is more than one input data array (cf. Propagation), the input or original data quality is deemed to be that associated with the principal data array.

  However, in some cases it is hard to identify a principal data array, or the principal data array does not have quality and one or more of the others does. Therefore, what is best depends on the nature of the application. For example, in the computation of the statistics of corresponding pixels from each of a series of pictures, to produce mean or standard-deviation arrays, it is vital to exclude all bad values from the calculations. A related problem is what the quality of output data arrays should be, and here again programmers must make case-by-case judgements.

• If an undefined pixel is generated during the processing, the data quality of the output data value is nonetheless the same as the input data quality, just as if a good pixel had been generated. (Although a pixel is undefined, you may still need to know, for example, that the pixel was a part of fiducial mark.)
• The original data and quality values remain unchanged.

If a [QUALITY] array is present it is assumed that it is to be used to define bad pixels unless:

• there is a parameter which overrides the default;
• the bit pattern of [BADBITS] is 00000000, or
• [BADBITS] is omitted from the $<$QUALITY$>$ structure.

If [QUALITY] is not present the magic-value method is assumed.

There is no one ideal way of handling data quality in general-purpose routines. Methods will evolve as experience with real applications and data is gained. The main considerations are:

• Disc space and virtual memory--use of full-size work arrays may be unacceptable for large data frames.
• Speed--checks for magic value and data quality in pixel-by-pixel processing loops should be kept to a minimum. For example, the program could first determine whether bad pixels and data quality are present or not, and then call different processing routines for the two cases.