This work investigates combinations of two different nature-inspired machine learning algorithms - Evolutionary Algorithms and Temporal Difference Learning. Both algorithms are introduced along with a survey of previous work in the field. A variety of ways of hybridising the two algorithms are considered, falling into two main categories - those where both algorithms operate on the same set of parameters, and those where evolution searches for beneficial parameters to aid Temporal Difference Learning. These potential approaches to hybridisation are explored by applying them to three different problem domains, all loosely linked by the theme of games. The Mountain Car task is a common reinforcement learning benchmark that has been shown to be potentially problematic for neural networks. Ms. Pac-Man is a classic arcade game with a complex virtual environment, and Othello is a popular two-player zero sum board game. Results show that simple hybridisation approaches often do not improve performance, which can be dependent on many factors of the individual algorithms. However, results have also shown that these factors can be successfully tuned by evolution. The main contributions of this thesis are an analysis of the factors that can affect individual algorithm performance, and demonstration of some novel approaches to hybridisation. These consist of use of Evolution Strategies to tune Temporal Difference Learning parameters on multiple problem domains, and evolution of n-tuple configurations for Othello board evaluation. In the latter case, a level of performance was achieved that was competitive with the state of the art.