Statistics computed for individual columns

Statistics can be computed for one or more individual columns. They can be computed from either all the rows in the catalogue or just the subset of rows comprising a selection which has been created previously. Obviously, only non-null rows are used in the calculations. Statistics can be displayed for columns of any data type, though for CHARACTER and LOGICAL columns the only quantity which can be determined is the number of non-null rows.

For each chosen column its name, data type and the number of non-null rows (that is, the number of rows used in the calculation) are displayed and the statistics listed in Table  are computed. Though all these quantities are standard statistics there is a remarkable amount of muddle and confusion over their definitions, with textbooks giving divers differing formulæ. For completeness, and to avoid any possible ambiguity, the definitions used in xcatview and catview are given below. These formulæ follow the CRC Standard Mathematical Tables[4] except for the definition of skewness which is taken from Wall[30].

Table: Statistics computed for columns

Minimum

Maximum

Total range

First quartile

Third quartile

Interquartile range

Median

Mean

Mode (approximate)

Standard deviation

Skewness

Kurtosis

In the following the set of rows for which statistics are computed is called the `current selection' and it contains $n$ non-null rows. $x_{i}$ is the value of the column for the $i$th non-null row in the current selection. The definitions of the various statistics are then as follows.

• The minimum and maximum are (obviously) simply the smallest and largest values in the current selection and the total range is simply the positive difference between these two values.

• If the column is sorted into ascending order then the $j$th quartile, $Q_{j}$, is the value of element $j(n + 1)/4$, where $j = 1$, 2 or 3. Depending on the value $n$, there may not be an element which corresponds exactly to a given quartile. In this case the value is computed by averaging the two nearest elements.

  The interquartile range is simply the positive difference between $Q_{1}$and $Q_{3}$.

• The median is simply the second quartile ($j = 2$). The mean has its usual definition: the sum of all the values divided by the number of values.

  The value computed for the mode is not exact. Indeed it is not obvious that the mode is defined for ungrouped data. Rather, the value given is computed from the empirical relation:

  
  

  $${\rm mode} = {\rm mean} - 3({\rm mean} - {\rm median})$$ (1)

  
  

• The standard deviation, $s$, is defined as:

  
  

  $$s = \sqrt{ \frac{1}{(n - 1)} \sum_{i=1}^{n} (x_{i} - {\rm mean})^2 }$$ (2)

  
  

• The skewness and kurtosis are defined in terms of moments. The $k$th moment, $u_{k}$, is defined as

  
  

  $$u_{k} = \frac{1}{n} \sum_{i=1}^{n} (x_{i} - {\rm mean})^{k}$$ (3)

  
  

  then

  
  

  $${\rm skewness} = u_{3}^{2} / u_{2}^{3}$$ (4)

  
  

  and

  
  

  $${\rm kurtosis} = u_{4} / u_{2}^{2}$$ (5)

  
  

  The expected values for the skewness and kurtosis are:

  • skewness = 0 for a symmetrical distribution,

  • kurtosis = 3 for a normal (or Gaussian) distribution.