Geometric Nelder-Mead Algorithm on the Space of Genetic Programs

The Nelder-Mead Algorithm (NMA) is a close relative of Particle Swarm Optimization (PSO) and Differential Evolution (DE). In recent work, PSO, DE and NMA have been generalized using a formal geometric framework that treats solution representations in a uniform way. These formal algorithms can be used as templates to derive rigorously specific PSO, DE and NMA for both continuous and combinatorial spaces retaining the same geometric interpretation of the search dynamics of the original algorithms across representations. In previous work, a geometric NMA has been derived for the binary string representation and permutation representation. Furthermore, PSO and DE have already been derived for the space of genetic programs. In this paper, we continue this line of research and derive formally a specific NMA for the space of genetic programs. The result is a Nelder-Mead Algorithm searching the space of genetic programs by acting directly on their tree representation. We present initial experimental results for the new algorithm. The challenge tackled in the present work compared with earlier work is that the pair NMA and genetic programs is the most complex considered so far. This combination raises a number of issues and casts light on how algorithmic features can interact with representation features to give rise to a highly peculiar search behaviour.