Data structures

In an ideal world you would not need to know how your data are stored. It would be transparent. Starlink applications attempt to achieve this through standard, but extensible, data structures, and the ability to apparently operate on other formats through the so-called `on-the-fly conversion' (see SUN/55).

The official standard data format used by Starlink applications is the NDF (Extensible n-dimensional Data Format, SUN/33). The data in an NDF is stored using HDS  which has numerous advantages, not least that HDS files are portable between operating systems; both have file extension .sdf.

The NDF has been carefully designed to facilitate processing by both general applications like KAPPA and specialist packages. It contains an n-dimensional data array that can store most astronomical data such as spectra, images and spectral-line data cubes. The NDF may also contain other items such as a title, axis labels and units, error and quality arrays, and World Co-ordinate System (WCS) information. There are also places in the NDF, called extensions, to store any ancillary data associated with the data array, even other NDFs.

The NDF format and its components are described more fully in NDF standard components, which includes commands for manipulating the components.

The NDF format permits arrays to have seven dimensions, but some applications only handle one-dimensional and/or two-dimensional data arrays. The data and variance arrays are not constrained to a single data type. Valid types are the HDS numeric primitive types.

Many applications are generic, that is they can work on all or some of these data types directly. This makes these applications faster, since there is no need to make a copy of the data converted to the type supported by the application. If an application is not generic it only processes _REAL data. Look in the Implementation Status in the help or the reference manual. If none is given you can assume that processing will occur in _REAL.

In KAPPA the elements of the data array are often called pixels, even if the NDF is not two-dimensional.