Part III
Reference Material

9 Introduction

This part of the document comprises reference material describing the ACL configuration file and the Tab-Separated Table (TST) format. Most of the material is adapted from Section 2 of Allan Brighton’s Astronomical Catalogue Library User Manual[2].

10 The Configuration File

This section describes the format of the ACL configuration file. When you refer to it you might find it useful to have to hand a copy of either simpleconfig.cfg or some other configuration file.

An ACL configuration file mediates the interaction between a client such as GAIA and a remote server. The configuration file comprises a list of one or more databases, giving details for each. Usually each ‘database’ will be a simple astronomical catalogue. However, other alternatives are possible: archives, name servers, etc. Consequently, in this section the generic term ‘database’ is used to denote each entry. Also, it is individual databases, not servers, which are listed in the configuration file: some servers might give access to more than one database. The details supplied for each database are things like: its URL, the type of queries supported, the name that will be used to describe it to users etc. The client reads the configuration file and the databases listed become the ones that it knows about.

By convention, configuration files have file-type ‘.cfg’. They are ASCII text files which may be created and modified with a text editor. Their basic syntax is as follows.

• Blank lines are ignored.
• Lines beginning with ‘#’ are considered to be comments and are ignored.
• Lines ending with ‘\’ (backslash) are continued on the next line.
• The description for each database comprises several ‘keyword:value’ pairs. Each keyword:value pair occupies a single line, unless continued over more than one line. The individual keywords are described in Section 10.1, below. Most are optional.
• The first keyword of each database entry must be serv_type.
• Subsequent keywords of each database entry can occur in any order.
• Unrecognised keywords are ignored (to permit future extensions).
• Many of the optional keywords correspond to facilities that are not supported by most databases.
• Some of the keywords require a list of values. All such lists have the same basic format as a Tcl² list: usually a list of words or strings enclosed in double-quotes (‘"’) and separated by one or more space characters. Alternatively, curly brackets (‘{ }’) may be used instead of double-quotes.
• If a keyword’s value comprises more than one list then these lists are separated by a colon (‘:’).
• If a keyword’s value includes a variable reference (see Section 10.6, below) then the variable is expressed in Tcl format: in practice it will start with a dollar character (‘$’).

10.1 Keywords

The entry for an individual database in a configuration file comprises some of the following keywords. The entry for the database must begin with the serv_type keyword. Other keywords can occur in any order. The keywords are optional unless otherwise indicated. The keywords are case-sensitive and must be specified entirely in lower case.

serv_type

(mandatory) The type of database. The options permitted are described in Section 10.2, below.

long_name

(mandatory) A one-line description of the database. Typically the client will display it to allow the user to identify the database.

short_name

(mandatory) A short name for the database. Conventionally it has the form:

database@institution

where database is an abbreviation for the database and institution a standardised abbreviation for the institution where the on-line version is located. By convention institution has three or four characters; some common values are listed in Table 1.

---

Abbreviation	Institution
cadc	Canadian Astronomy Data Centre, Dominion Astrophysical Observatory
eso	European Southern Observatory, Garching bei München
lei	Department of Physics and Astronomy, University of Leicester
roe	Royal Observatory Edinburgh

Table 1: Abbreviations for institutions hosting ACL servers

---

url

(mandatory) The URL and query template to access the server. See Section 10.3.

symbol

Defines how objects in the returned table should be plotted. See Section 10.6.

copyright

A copyright notice for the client to display.

search_cols

A list of columns on which range searches are permitted. See Section 10.5.

sort_cols

The columns on which the returned table is sorted.

sort_order

The order into which the returned table is sorted. The permitted values are: increasing and descreasing.

show_cols

By default a client will display all the columns in the returned table. If a show_cols list is specified then by default only the columns in the list will be displayed. Also the order of the list defines the order in which the columns should be displayed.

id_col

The number of a column in the returned table which contains a unique identifier (or ‘name’) for each object in the table. By default the first column of a TST format table contains such an identifer (see Section 11).

ra_col

The number of a column in the returned table which contains the Right Ascension. By default the second column of a TST format table contains the Right Ascension (see Section 11).

dec_col

The number of a column in the returned table which contains the Declination. By default the third column of a TST format table contains Declination (see Section 11).

x_col

The number of a column in the returned table which contains $x$ image pixel coordinates.

y_col

The number of a column in the returned table which contains $y$ image pixel coordinates.

is_tcs

If this keyword is present it should be set to ‘1’. Its presence indicates that the returned table either is in or should be converted to TCS format. TCS format catalogues have fixed column names and rules for converting column names when catalogues are imported from other formats.

help

A URL pointing to a Web page (or pages) describing the database.

backup1

A reserve URL to be used to access the database in the case where the URL specified by the url keyword does not respond.

backup2

A second reserve URL to be used to access the database in the case where the URLs specified by the url and backup1 keywords do not respond.

For keywords id_col, ra_col, dec_col, x_col and y_col the number of a column is defined as its sequence number in the list of column names which defines the columns in a TST table (see Section 11 and Figure 2). The first column is numbered zero. This sequence number is sometimes referred to as the ‘column index’.

10.2 Types of servers

The types of database which may be specified for keyword serv_type are as follows. If an unrecognised type is specified it will be ignored (to permit future extensions).

catalog

The database is a simple catalogue; this is the simplest and most common option.

archive

The database is an archive. This option is similar to ‘catalog’, but the table returned may contain special columns whose values are URLs giving access to ‘bulk data’ (images, spectra, time-series etc) for the selected objects. See Section 11.1 for details of these special columns.

namesvr

The database is a name server. The only type of query permitted is to submit to the server a character string corresponding to the name of an astronomical object. The server returns a TST catalogue with a single row and three columns. The row corresponds to the object named and the three columns are: identifier (name), Right Ascension and Declination. See configuration file simpleconfig.cfg for an example of a namesvr entry.

imagesvr

The database is an image server. See Section 12.

local

The database is a local catalogue in the TST format.

directory

The ‘database’ is another configuration file. This facility allows a tree (or rather network, since recursion is permitted) of linked configuration files to be built up. The url keyword gives the URL of the destination configuration file. The term ‘directory’ comes from making an analogy with an hierachical file system. See configuration file simpleconfig.cfg for an example.

10.3 URL and query specification

The url keyword prescribes how the client should access a remote server to query a database. Following the usual conventions for a CGI gateway it consists of the URL corresponding to the script which constitutes the server, followed by a ‘?’ and then a string defining the query passed to the server. The query string consists of simple characters and ‘tokens’. The tokens start with a ‘%’ character. When a client makes a query the tokens are replaced with values which correspond to the individual query and the resulting string is sent to the server. An example query string for a server which provides circular area searches on a single catalogue might be:

which contains the tokens %ra, %dec and %r2. The client might substitute them to yield:

The format of the query string must correspond to that expected by the server. This restriction aside, considerable freedom is allowed in the format of the query string. See configuration file simpleconfig.cfg for examples. The complete list of the permitted tokens is as follows.

%ra

Right Ascension (see Section 10.4).

%dec

Declination (see Section 10.4).

%x

$x$ image pixel coordinate (see Section 10.4).

%y

$y$ image pixel coordinate (see Section 10.4).

%r

radius of a circular area search (see Section 10.4).

%r1

inner radius (see Section 10.4).

%r2

outer radius (see Section 10.4).

%w

width of a rectangular region (see Section 10.4).

%h

height of a rectangular region (see Section 10.4).

%m1

Minimum magnitude.

%m2

Maximum magnitude.

%m

Maximum magnitude when no minimum is specified.

%n

Maximum number of objects (or rows) to return.

%cols

A list of the columns which the returned table is to contain. If no value is supplied all the columns in the database will be returned. The order of the list corresponds to the order in which the columns are returned.

%id

The name (or identifier) of an object submitted to a namesvr (‘name server’) database.

%mime-type

The HTTP mime-type to be inserted in a HTTP get command.

%sort

A list of columns on which the returned table is to be sorted. If no value is specified the table is returned unsorted.

%sortorder

The order into which the returned table is sorted. The permitted values are increasing and decreasing.

%cond

A string specifying one or more range searches. See Section 10.5.

10.4 Coordinates, units and formats

By default databases are assumed to contain columns of Right Ascension and Declination on which they can be searched. However, some databases do not contain such coordinates, but rather have Cartesian $x,y$ positions. Typically such positions occur in catalogues generated by detecting the objects present in a CCD direct image frame or a digitised photographic plate. The following rules apply.

(1)

By default databases are assumed to be searchable on columns of Right Ascension and Declination. Further, these columns are assumed to be called ‘ra’ and ‘dec’, respectively. Simply include the tokens %ra and %dec in the query.

(2)

If a ‘positional’ search (ie. one finding objects in either a circular area or a region bounded by meridians of Right Ascension and parallels of Declination) is to be made on columns of Right Ascension and Declination then the formats and units of the given values should be:

Right Ascension

sexagesimal or decimal hours,

Declination

sexagesimal or decimal degrees.

If a sexagesimal value is entered than a colon (‘:’) should be used as a separator. The coordinates should be for equinox and epoch J2000.

(3)

If a database is searchable on $x,y$ positions rather than celestial coordinates then the following keywords should be added to its entry in the configuration file:

ra_col:     -1
dec_col:    -1
x_col:      (sequence number of column holding $x$ coordinate)
y_col:      (sequence number of column holding $y$ coordinate)

Setting ra_col and dec_col to -1 indicates that columns of Right Ascension and Declination are not available.

(4)

The units of the $x,y$ positions are simply ‘pixels’, that is, dimensionless numbers. Note however, that they are of type REAL rather than INTEGER; positions can be expressed to a fraction of a pixel.

(5)

The units in which the radius, height and width of circular and ‘rectangular’ positional searches are expressed also vary depending on whether the database contains celestial coordinates or $x,y$ positions. The tokens affected are:

%r %r1 %r2 %w %h

The alternatives for their units are:

in a search on celestial coordinates:

decimal minutes of arc,

in a search on $x,y$ positions:

decimal pixels.

10.5 Range queries

To permit range queries on one or more columns of a database the keyword search_cols should be included in the entry for the database in the configuration file and the %cond keyword should be included in the query part of its url keyword. These items have the following syntax.

search_cols keyword

A list of one or more column names together with text prompts for the minimum and maximum values of the range required.

search_cols:column-name-1 minimum-prompt-1 maximum-prompt-1 :
column-name-2 minimum-prompt-2 maximum-prompt-2 …

The client uses the minimum and maximum prompts when soliciting the extrema of the required range from the user. An example might be

search_cols: mag "Bright limit" "Faint limit" : \
b_v "Minimum colour" "Maximum colour"

%cond token

This token is included in the query part of the url keyword and specifies any range search that is supplied. The syntax of the values substututed into the %cond string is:

column-name-1 = minimum-value-1,maximum-value-1& column-name-2 = minimum-value-2,maximum-value-2 …

When a search is specified column-name-n, minimum-value-n and maximum-value-n are substituted with respectively the column name, minimum value and maximum value. The server should process this string and return only rows that lie within the given ranges. An example of substituted query might be:

mag=1.0,2.0&b_v=0.1,0.3

10.6 Plotting directives

The keyword symbol prescribes how objects in the returned table of values are to be displayed in finding charts, image overlays etc. Considerable flexibility is allowed in the way that objects are plotted. In particular, the two common types of cases which are conventionally used in atlases and finding charts are supported:

• the size of the plotted symbol is scaled according to some property not directly related to the apparent size of the object. For optical star catalogues the magnitude is traditionally used,
• the size and orientation of the plotted symbol bears some relation to the apparent size and orientation of the image. For example, galaxies are often plotted as ellipses with the outline of the ellipse approximating to some isophote.

The symbol keyword only prescribes how objects should be plotted in finding charts and image overlays; it is not appropriate for scatter-plots where the two axes are not celestial coordinates or $x,y$ positions. The syntax of the symbol keyword is:

This triumavirate of items can be repeated an arbitrary number of times, with each occurence being separated by a colon (‘:’). The meaning of each of the three items is as follows.

column-names

is a list of the names of columns in the returned table which appear as variables in symbol-info or size-expr. The names should be separated by spaces. Column names containing embedded spaces or characters which Tcl³ interprets as special should be enclosed in curly brackets (‘{ } ’). Alternatively, names which contain spaces but not any characters which Tcl regards as special may be enclosed in double-quotes (‘" "’).

symbol-info

prescribes the appearance of the plotting symbol. See Section 10.6.1 for details.

size-expr

is an expression which is evaluated to give the size of each symbol. This expression uses Tcl syntax, with some or all of the column names listed in column-names appearing as variables. The expression may optionally be followed by the units of the value computed. The units permitted are:

image

image pixels,

deg J2000

degrees for equinox J2000,

deg B1950

degrees for equinox B1950.

10.6.1 symbol-info

symbol-info prescribes the appearance of the symbol plotted. It is a list comprising the following items:

symbol is mandatory; the other items are optional.

symbol

is the plotting symbol to be used. The values permitted are listed in Table 2. The symbols marked with a bullet (‘$\bullet$’) are typically scaled and rotated to correspond to the appearance of the image; see ratio and angle, below.

colour

is the colour in which the symbol is plotted. It may be any valid X colour name. If colour is omitted then GAIA and SkyCat draw the symbol in black and white, which stand the best chance of being visible when overlaid on a colour image. CURSA will recognise only the restricted set of colours listed in Table 3.

ratio and angle

are respectively the width / height ratio of the object and rotation from north. Both quantities may be given as expressions involving column names.

label

is the label for each object. It may be fixed text, an expression involving column names or a mixture of the two.

condition

is a boolean expression involving column names whose value prescribes whether the object is plotted or not. If the expression evaluates to true (or 1) then the object is plotted; if it evaluates to false (or 0) it is not. By having a list of entries for the symbol keyword, each with a different condition attached to the symbol-info item different symbols can be plotted for each object, depending on the values of its column entries.

10.6.2 Examples

symbol: "" circle 12

 
Plots objects as circles of constant size.

symbol: mag circle 15-$mag

 
Plots objects as circles scaled according to magnitude (column mag).

symbol: mag {circle red} 15-$mag

 
Plots objects as red circles scaled according to magnitude.

symbol: {a b pa} {ellipse red $a/$b $pa} {$a/3600.0 "deg J2000"}

 
Plots objects as red ellipses. The ratio of the ellipse is computed from the semi-major and semi-minor axes (a and b respectively). The orientation is computed from position angle (pa). The image size is computed in degrees from the semi-major axis, which is assumed to be in seconds of arc.

symbol: {mag rv} {circle red "" "" "" $rv>0 } 15-$mag :\

 
{mag rv} {circle blue "" "" "" $rv<=0 } 15-$mag
Plot objects as circles scaled according to magnitude. However, the colour of each circle depends on the radial velocity (rv). Objects with a positive radial velocity are shown in red, those with a negative one in blue.

10.6.3 Inserting private Tcl procedures

If a standard Tcl expression does not allow you to calculate the symbol size as you wish then you may be able to invoke your own Tcl procedure to perform the calculation. This facility is only available if you are using SkyCat as a client and is only likely to be useful with your own catalogues. An example might be:

where you provide Tcl procedure my_plot_proc.

10.7 Checking a configuration file

A Perl script is available to check the validity of configuration files. You can use it to check for errors in any configuration files that you have created or modified. It is included with the examples provided with this document as file:

You can either run it from the examples directory or copy it to a convenient local directory. On non-Starlink systems you may need to edit the first line to correspond to wherever Perl is located on your system.

To use the script simply specify the name of the configuration file, optionally preceded by the appropriate directory specification, on the command line:

For example, to check a copy of the example configuration file supplied with this document (assuming that there is a copy in your current directory):

If the configuration file is valid the checkcfg will display the message:

Conversely, if there are problems with the configuration file then explanatory error messages are displayed. Usually the number of the invalid line in the configuration file is included in the message. Note that checkcfg is mostly checking for syntax errors; it does not, for example, check that any URLs specified are valid.

11 The Tab-Separated Table Format

This section gives a brief description of the tab-separated table (TST) format. ACL servers should return the list of selected objects in this format. Various packages, including GAIA, CURSA and Starbase can also read local files containing catalogues in this format. There are alternative descriptions of it in SUN/214[4], the Starbase FAQ⁴ and the Astronomical Catalogue Library User Manual[2].

TST format files are text files. They are usually generated by a remote server in response to a query from a local client. However, they could equally well be local files created with a text editor. Figure 2 shows a simple example of a tab-separated table. This example is available as file:

The description of the table and the table of values occupy the same file and the description occurs at the start of the file. Most of the description shown in Figure 2 is optional.

The first line of the description is a title. Lines beginning with a ‘#’ are comments which are ignored.

Parameter definitions start with the parameter name, and a colon (‘:’) is appended to the end of the name to identify it as such. The rest of the line contains the parameter value. The name and value are the only information stored for each parameter. The example contains two parameters: EQUINOX and EPOCH.

Any remaining lines in the description (apart from the last two, which immediately precede the table of values) are free text.

The only mandatory items in the description are the two lines immediately before the table of values. The first of these lines is the list of column names. Each name is separated by a single tab character (ASCII code nine; strictly speaking the horizontal tab). In the figure tab characters are shown as ‘<tab>’; obviously a real tab-separated table would contain actual tab characters instead. The name is the only mandatory information stored for each column. The TST format places few restrictions on the column names: they can contain spaces, special and punctuation characters etc. However, it is usually a prudent precaution to restrict column names to contain only alphanumeric and underscore (‘_’) characters and to make the first character alphabetic. If these precautions are observed then fewer problems are likely to occur if the table is subsequently converted to another format or read by a variety of different clients. Remember that if you are writing a server which returns a TST table via the Internet then you do not know which client will be used to access it.

The line immediately after the list of column names indicates the end of the description and the start of the table. It consists solely of dashes and tab characters. By convention there are as many sequences of dashes as there are column names, each sequence is separated by a single tab character and each has the same number of dashes as there are characters in the corresponding column name.

In the table of values each row occupies a single line. Individual fields are separated by a single tab character. The fields occur in the same order as the corresponding column names.

11.1 Special columns of identifiers and celestial coordinates

The TST format has the following additional rules for special columns containing identifiers and celestial coordinates.

(1)

By default, the first three columns in the table are: an identifier (that is, an object name), Right Ascension and Declination. These columns are usually called Id, ra and dec respectively.

(2)

If the first three columns of the table are not an identifier, Right Ascension and Declination then the TST should contain the following three parameters:

id_col
ra_col
dec_col

The value of each parameter should be the number of the appropriate column or, if there is no such column, -1. The column number is the sequence number of the column in the list of column names, starting counting at zero.

(3)

The Right Ascension and the Declination are either both in decimal degrees or the Right Ascension is in sexagesimal hours and the Declination is in sexagesimal degrees. If sexagesimal notation is used then a colon (‘:’) is used as the sexagesimal separator.

(4)

The equinox and epoch are specified using parameters equinox and epoch respectively. The default value if the equinox and epoch are not specified is J2000.0.

11.2 Conventions for tables returned by ACL servers

In addition to the basic TST format the following additional conventions apply to tables returned by ACL servers.

(1)

Keywords specified for a database in its entry in the configuration file are equivalent to, and treated as, parameters in the TST. Thus, id_col, ra_col and dec_col are usually supplied as keywords in the configuration file rather than parameters in the TST returned by the server. If a client writes a table of retrieved objects as a local file in the TST format it should include these keywords as parameters.

(2)

Some types of database, particularly archives (see Section 10.2), may return tables containing columns called more and preview. These columns are optional. However, if present they should be used as follows.

more

a URL giving more information on the object. The destination URL will typically be a conventional page of HTML, suitable for display with a Web browser.

preview

a URL pointing to an image of the object.

11.3 CURSA extensions

Some additional items can be added to the TST header information which provide additional information about the catalogue and allow CURSA[3] to interpret it more precisely. These items begin with a ‘#’ character and thus are TST comments. Consequently a catalogue which contains them remains perfectly standard and valid and can be processed with software other than CURSA. The items are illustrated in Tables 1 and  2 and are as follows.

#column-units:

is followed by a tab-separated list of units for the columns.

#column-types:

is followed by a tab-separated list of data types for the columns. The permitted data types are listed in Table 4.

#column-formats:

is followed by a tab-separated list of display formats for the columns. Fortran-like, FITS-compatible formats are used. This convention facilitates converting tables between the TST and FITS tables formats without loss of information.

In all cases the items are listed in the order in which they occur in the table. There is no tab character between #column-units:, #column-types: and #column-formats: and the following value. Values in the special TST columns of Right Ascension and Declination (that is, those identified by the ra_col and dec_col parameters) are always interpretted using the TST rules for representing angles rather than the CURSA ones.

11.4 MIME type

The first line of information returned by an ACL server should always be the MIME⁵ type:

This line is strictly speaking not part of the TST table and will be used by the client or Web browser to interpret the format of the data which follow.

12 Image Servers

Image servers differ from other types of ACL server in that they return a direct image or ‘pixel map’ of a region of sky rather than a catalogue of objects. The image is returned formatted as a FITS⁶ file which may optionally be compressed.

(1)

Image servers have a configuration file entry similar to those for other types of database. The server type is imagesvr. The region of sky required is specified by giving the central Right Ascension and Declination (or $x,y$ pixel position if celestial coordinates are not available), width and height. The usual tokens (see Section 10.3):

 %ra  %dec  %x  %y  %w  %h

are included in a query string as part of a url keyword in the usual fashion. The tokens have their usual meaning and units. An example configuration file entry for an image server is:

  serv_type:      imagesvr
  long_name:      Digitized Sky at ESO
  short_name:     dss@eso
  url:            http://archive.eso.org/dss/dss\
  ?ra=%ra&dec=%dec&mime-type=%mime-type&x=%w&y=%h
  copyright:      Digitized Sky Survey (c) by AURA, provided online by ESO

(2)

The server parses the query and generates a FITS file corresponding to the region specified. It first returns the MIME type, followed by the FITS file which optionally may be compressed. The MIME types corresponding to the various types of compression are listed in Table 5. For example, the MIME type for a gzipped FITS file is:

Content-type: image/x-gfits

Acknowledgements

The material in Part II is largely based on Section 2 of Allan Brighton’s Astronomical Catalogue Library User Manual[2]. Martin Bly, Allan Brighton, Peter Draper and Mike Read provided useful comments on the draft version of the document. Any mistakes, of course, are my own.

Bibliography