Spiking Neural Networks (SNNs) are considered to be the third generation of neural networks, and have been proven to be just as powerful as the more established artificial neural networks from previous generations. The study of SNNs is motivated by their close resemblance to biological neural networks. However, their applicability in real world applications has been limited due to the lack of effective and efficient training methods. This is particularly relevant for training large networks and on-line learning applications. This thesis, which aims to develop an algorithm for SNNs that can conduct online learning, initially reviews the various types of spiking neural models to explore neuronal function and various kinds of encoding schemes. It is desirable employing a more abstract model when large networks are developed and online learning is applied. The description of the state of the art is followed with respect to learning algorithms in SNN s across a range of applications, and existing research challenges in online learning algorithms are identified. First, this thesis presents a new evolving RBF-like learning algorithm for SNNs. Both the structure and weights of the SNN are learned dynamically through a combination of unsupervised and supervised learning paradigms. Second, this thesis presents an enhanced Rank-Order based learning algorithm, called SpikeTemp, for SNNs with a dynamically adaptive structure, which is more effective and efficient than a Rank-Order based learning algorithm. A further algorithm, SpikeComp that has a more compact neural structure than SpikeTemp is then developed. This has a similar network topology to SpikeTemp, however, differing learning strategies are proposed to reduce the neuron count in the output layer. Finally an advanced RBF-like dynamically Evolving Spiking Neural Classifier (ESNC) is presented. After training, each output neuron represents a unique class. These developed learning algorithms have adaptive structures and show a comparable learning performance to other existing SNN s algorithms and standard classical methods after only one (or more in SpikeComp) presentation of the training inputs. The thesis concludes with possible future directions for this research.