Written in 2018, this was an exploration into Evolutionary Algorithms applied to Reinforcement Learning tasks (specifically Atari games). It is a fork of d9w/CGP.jl (Dennis Wilson, Apache 2.0); my work centered on the Atari reinforcement-learning experiments rather than the core CGP framework.

Overview

Standard Cartesian Genetic Programming (CGP) relies heavily on mutation. The upstream library hybridizes CGP with NEAT (NeuroEvolution of Augmenting Topologies) concepts to protect topological innovation through speciation.

My goal in forking it was to evolve graph-based programs that could learn Atari control policies using gradient-free optimization.

Features

The upstream framework provides the CGP machinery this project builds on:

  • Graph-based Crossover: Crossover operators such as subgraph_crossover and aligned_node_crossover that handle the destructive nature of mating graph structures.
  • Speciation: A NEAT-inspired compatibility-distance metric (cgpneat.jl) to maintain population diversity and prevent premature convergence.
  • Active Gene Tracking: Differentiates between “active” nodes (those contributing to output) and “junk DNA,” focusing mutation on phenotypic changes.

My own contribution was the Atari reinforcement-learning layer on top of this: experiment variants (action_atari.jl, original_atari.jl, manual_atari.jl, play_atari.jl, param_sweep.jl), custom fitness and scoring functions, early-stopping and completion-percentage logging, multithreading and pmap multiprocessing attempts (reverted to single-thread), and config tuning to match a reference paper’s hyperparameters.

Usage

The library provides a Julia API for defining CGP graphs, configuring evolutionary parameters, and running the evolutionary loop against custom environments.

Results

Looking back, this codebase captures a transitional moment where I was moving from scripting to library design.

  • The Ambition: Getting CGP graphs to learn Atari policies under the mixed-type regime (RGB-array inputs, scalar action outputs) was an ambitious undertaking for my software engineering skills at the time.
  • The “Legacy” Code: The project relies on the now-deprecated Julia v0.6 and uses eval(parse(...)) patterns for configuration (a significant performance anti-pattern in modern Julia).
  • The Lesson: It taught me the difficulty of designing genetic operators that respect topological constraints, a lesson that informs my current understanding of optimization in structured spaces.