Title:
|
Hybridising evolution and temporal difference learning
|
This work investigates combinations of two different nature-inspired machine learning
algorithms - Evolutionary Algorithms and Temporal Difference Learning. Both algorithms are introduced along with a survey of previous work in the field. A variety of
ways of hybridising the two algorithms are considered, falling into two main categories
- those where both algorithms operate on the same set of parameters, and those where
evolution searches for beneficial parameters to aid Temporal Difference Learning.
These potential approaches to hybridisation are explored by applying them to three
different problem domains, all loosely linked by the theme of games. The Mountain
Car task is a common reinforcement learning benchmark that has been shown to be
potentially problematic for neural networks. Ms. Pac-Man is a classic arcade game with
a complex virtual environment, and Othello is a popular two-player zero sum board
game.
Results show that simple hybridisation approaches often do not improve performance,
which can be dependent on many factors of the individual algorithms. However, results
have also shown that these factors can be successfully tuned by evolution.
The main contributions of this thesis are an analysis of the factors that can affect
individual algorithm performance, and demonstration of some novel approaches to hybridisation. These consist of use of Evolution Strategies to tune Temporal Difference
Learning parameters on multiple problem domains, and evolution of n-tuple configurations for Othello board evaluation. In the latter case, a level of performance was achieved
that was competitive with the state of the art.
|