Design Fundamentals

The most fundamental issue (certainly the most fruitful source of controversy) is whether we are trying to design formats which are so comprehensive that any piece of information, however specific to an instrument or processing phase, has a defined location ready to receive it, or whether we are instead trying to design the simplest possible system which can do the job.

The first approach, sometimes called the ``we've thought of everything'' philosophy or TOE, is the one that has been traditionally employed. The designers of such systems have tried to predict everything that might be needed (in their experience as optical spectroscopists, aperture synthesists, X-ray observers, etc.) for the general case of a picture, spectrum, time series, or whatever. These designs generally work well for their inventor, but when others try to use them they find omissions, inconsistencies, ambiguities and limitations, and either have to add new items of their own or use existing items in a non-standard way. Even where a new form of data is expressed in what appears to be the standard way, experience has shown that precise interpretation by different application programs of all the various ancillary items (e.g. exposure time, astrometric parameters, etc.) cannot be relied on, and so these items become little more than comments.

The Starlink designs reject the TOE approach in favour of one where:

• The structures are simple.
• The processing rules for all components are defined.
• There is orderly treatment of extensions.

The third point--the treatment of extensions--is crucial. Most astronomer/programmers feel drawn to the familiar TOE approach, where there is a place to put the $\alpha,\delta$, exposure time, polarimeter setting, relative humidity, feed-horn collimation parameters, etc., and are unhappy that many of the items they wish to include have to be ``demoted'' by being moved into an extension. Alternatively, they are willing to accept the need for extensions, but only for the idiosyncratic data required by other astronomers. It is important to understand that the extensions in the Starlink standard formats are an essential part of the scheme, safe havens where important but specialised items can reside, accessible to programs which understand them, and automatically copied from generation to generation. All extensions should be registered with the Starlink Head of Applications to avoid clashes between different groups of applications. Certain general-purpose extensions will be highly standardised, and will be used by many application packages.

The combination of (i) trying to keep the formats simple and (ii) defining precisely how the different items should be interpreted by application programs has produced a result which has remarkably little evidence of astronomy in it. This should not be regarded as a worry; the astronomical information, relating to astrometry, radiometry, timing, and so on, will reside in standard extensions which will be defined in due course. A byproduct of this conservatism (which came largely from the need to reduce the task to a manageable size) is that the standard structures, and the general-purpose applications which process them, may have uses outside astronomy.