Learning classifier systems for decision making in continuous-valued domains
This thesis investigates Learning Classifier System architectures for decision making
in continuous-valued domains.
The information contained in continuous-valued domains is not always conveniently
expressed using the ternary representation typically used by Learning Classifier
Systems and an interval-based representation is a natural choice. Two intervalbased
representations recently proposed are analysed, together with their associated
operators. Evidence of considerable representational and operator bias is found. A
new interval-based representation is proposed that is more straightforward than the
previous ones and its bias is analysed.
Learning Classifier Systems are compared for online environments that consist of
real-valued states and which require every action made by the agent to count towards
its performance. Two Learning Classifier System architecture are considered , XCS
and ZCS. An interval representation is used for the rule conditions and a roultte wh
is used for action selection. The performance of these two Learning Classifier system
architectures is investigated on a set of abstract environments with both deterministic
and stochastic reward functions. Although XCS clearly delivers superior performance
in the deterministic environments tested, the simple ZCS architectur is found to
be robust and able to equal or exceed the performance of XCS in the stochastic
environments tested, especially those with more demanding characteristics,
Aspects of the algorithm and parameter set of ZCS are studied on problems with
real-valued states and a Boolean action space. Increased performance is found to
result from the use of an update algorithm based on that of NewBoole, an earlier
strength-based Learning Classifier System. A new operator, specialize, is introduced
and found to be effective in combatting over general classifiers. The modified algorithm
and parameter set is tested on several variants of three real-valued test problems The resulting Learning Classifier System is applied to simulated Foreign Exchange
trading using an experimental setup and data previously presented in the literature.
Results show that a simple Learning Classifier System is able to achieve a positive
excess return in simulated trading.