Overview

There are two sorts of minimisation, one is to minimise any old function, and one to minimise a sum of squares. The first is more general, the least-squares routines are presumably more efficient. For the programmer the differences are as follows:

• For a least-squares fit, your merit function is a vector of residuals between measurements and current model guess. For a general minimisation your merit function is a scalar, basically you have to add up the squared residuals within your merit function.
• Perhaps more important is the amount of workspace you have to provide to the fit algorithm. In least-squares fits this scales with n*m where n is the number of parameters to be fitted and m is the number of residuals to be added up (e.g. channels in a spectrum). m is quite large and depends on the data set at hand. The general minimisation does in principle not know that there is a spectrum with m pixels, and its workspace scales with the square of n, which is more or less constant for any given application.
• Another difference is that least-squares fits tend to return the Jacobi matrix, the derivatives of each fit parameter with respect to each measurement. This is quite valuable if you want to know the variances of the fit parameters and the covariances between them. With the general minimisation you would have to work out the Hesse matrix and invert it. Which means you have to be able to work out the derivatives of you merit function w.r.t. to each fit parameter.

Two other issues in choosing minimisation algorithms are whether only the merit function can be provided or also first derivatives, and whether the fit parameters have to be constrained or not. It appears that Starlink applications use mainly unconstrained fits or at most simple bounds, i.e. hard constant limits on parameters. This library contains function-only algorithms, and also the SUMSL algorithm which requires both functions and gradients. The only constrainable algorithm is the simulated annealing SIMANN.

• PDA_UNCMND is a general unconstrained minimisation using function values only. A quasi-Newton algorithm with line search is used.
• PDA_LMDIF/PDA_LMDIF1 is an unconstrained least-squares minimisation using residuals only (no derivatives). A modified Levenberg-Marquardt algorithm is used, the Jacobian is calculated by a forward-difference approximation.
• SIMANN is a simulated annealing algorithm. It uses function values only and can be used for non-smooth functions as well. It should also have a fair chance of getting out of local minima and going on to find the global minimum.
• SUMSL is a general unconstrained minimisation using function values and gradients. A trusted regions algorithm is used.
• SUBPLEX is a generalisation and improvement on the Simplex algorithm. It should be very robust with any function to minimise. But it should also be rather inefficient.