Radial basis and support vector machine algorithms for approximating discrete data
The aim of this thesis is to demonstrate how the versatility of radial basis functions
can be used to construct algorithms for approximating discrete sets of scattered data.
In many cases, these algorithms have been constructed by blending together existing
methods or by extending algorithms that exploit certain properties of a particular
basis function to include certain radial functions. In the later chapters, we shall see
that methods which currently use radial basis functions can be made more efficient
by considering a change to the existing methods of solution.
In chapter one we introduce radial basis functions (RBFs) and show how they
can be used to construct interpolation and approximation models. We examine the
uniqueness properties of the interpolation scheme for two specific functions and review
some of the methods currently being used to determine the type of function to use
and how to choose the number and location of centres. We describe three methods
for choosing centres based on data clustering techniques and compare the accuracy of
an approximation using two of these schemes. We show through a numerical example
how greater accuracy can be achieved by combining these two schemes intelligently
to construct a new, hybrid method. Problems that currently exist, for a particular
clustering algorithm, when dealing with domain boundaries and which are not covered
in great detail in the literature are highlighted and a new method is proposed.
We conclude the chapter with an investigation into point distributions on the
sphere. Radial basis functions are increasingly being used as a tool for approximating
both discrete data and known functions on the sphere. Much of the current research
focuses on constructing optimum point distributions for approximations using spherical
harmonics. In this section we compare and evaluate these point distributions for RBF approximations and contrast the accuracy of the spherical harmonics with
results obtained using the multiquadric function.
In chapter two we develop an algorithm for surface approximation by combining
the works of Mason & Bennell , and Clenshaw & Hayes . Here, the well
known method for constructing tensor products on rectangular grids is combined
with an algorithm for approximating data collected along curved paths. The method
developed in the literature for separable Chebyshev polynomials is extended to include
the Gaussian radial function. Since the centres of the Gaussians can be distinct from
the data points, we suggest a method for constructing a suitable set of centres to
enable the efficiency of the two methods to be preserved. Possibilities for further
efficiency using parallel processing are also discussed.
We conclude the chapter by reviewing the Gram-Schmidt method and show how
the use of orthogonal functions results in a numerically stable computation for evaluating
the model parameters. The local support of the Gaussian function is investigated
and the method of Mason & Crampton  is explained for constructing
orthogonalised Gaussian functions.
Chapter three introduces a relatively new topic in data approximation called
support vector machines (SVMs). The motivation behind using SVMs for constructing
regression models to corrupted data is addressed and the use of RBF kernels to
map data into feature space is explained. We show how the regression model is formulated
and discuss currently used methods of solution. The flexibility of SVMs to
adapt to different types and level of noise is demonstrated through some numerical
examples. We make use of the techniques developed in SVM regression to show how
the algorithm described in chapter two can be extended. Here we make use of SVMs
in the early stages of the algorithm to remove the need for further consideration of
noise. We complete the discussion of SVMs by explaining their use in the field of
data classification through a simple pattern recognition example.Chapter four focuses on a new approach to the solution of an SVM. The new
approach taken is one of constructing an entirely linear objective function. This is
achieved by changing the regularisation term. We show, in detail, how the changes
made to the existing framework affects the construction of the model. We describe
the solution method and explain how advantage can be taken of the new linear structure.
To determine the model parameters, we show how the solution, in the form of
a simplex tableau, can be found extremely efficiently by recognising certain relationships
between variables that allow us to employ Lei's algorithm. Examples that show
SVM approximants to noisy data for both curves and surfaces are given together with
a comparison between Lei's algorithm and a standard simplex solution method. We
finish the section by highlighting the link between support vectors and radial basis
function centres. The sparsity produced by the method in the coefficient vector is
The new linearised approach to constructing SVM regression models is used in a
new algorithm developed to construct planar curves that model the path of fault lines
in a surface. Part of a detection algorithm proposed by Gutzmer & Iske  is used
to determine points that lie close to a fault line. The new approach is then to model
the fault line by constructing an SVM regression curve. The chapter concludes with
some examples and remarks.
The thesis concludes with Chapter five in which we summarise the main points
discussed and point to possibilities for extending the work presented.