Evolutionary algorithms are a competent nature-inspired approach for complex computational problem solving. One recent development is Grammatical Evolution, a grammar-based evolutionary algorithm which uses genotypes of variable length binary strings and a unique genotype-to-phenotype mapping process based on a BNF grammar definition describing the output language that is able to create valid individuals of an arbitrary structure or programming language. This study surveys Grammatical Evolution, identifies its most important issues, investigates the competence of the algorithm in a series of agent-oriented benchmark problems, provides experimental results which cast doubt about its effectiveness and efficiency on problems involving the evolution of the behaviour of an agent, and presents Constituent Grammatical Evolution (CGE), a new innovative evolutionary automatic programming algorithm. CGE extends Grammatical Evolution by incorporating the concepts of constituent genes and conditional behaviour-switching. It builds from elementary and more complex building blocks a control program which dictates the behaviour of an agent and it is applicable to the class of problems where the subject of search is the behaviour of an agent in a given environment. Experimental results show that the new algorithm significantly improves Grammatical Evolution in all problems it has been benchmarked. Additionally, the investigation undertaken in this work required the development of a series of tools which are presented and described in detail. These tools provide an extendable open source and publicly available framework for experimentation in the area of evolutionary algorithms and their application in agent-oriented environments and complex systems.