Modelling Natural Systems & Reinforcement Learning

Three systems, one method

A nonlinear oscillator, a population of predators and prey, and a learning agent. On the face of it these have nothing to do with each other. In practice they are the same exercise done three times: write down the rule that takes the system from one step to the next, run it forward, and then ask what the long-run behaviour actually is. Python and NumPy throughout.

The Duffing oscillator

The Duffing oscillator is a nonlinear system, and it is treated here as one: not a formula to be solved once, but a system whose behaviour and stability have to be characterised by running it and seeing what it settles into. The rule that generates it is short. What the rule produces is not.

In the oscillator and the predator-prey model, the rules are given and the behaviour is the unknown.

Predator and prey

The predator-prey model is agent-based rather than a pair of coupled equations, which is a different claim about where the dynamics come from. An agent-based model describes no population directly. It describes individuals and their rules, and whatever population-level behaviour appears, cycles, collapse or something steady, has to emerge from those rules rather than be written into them. That is the reason to build it this way, and the reason it is harder to reason about.

Tabular Q-learning

The third piece inverts the other two. In the oscillator and the predator-prey model, the rules are given and the behaviour is the unknown. In Q-learning the behaviour is the target and the rule is what gets learned, from reward alone. Tabular is the plain version of that: no network, no approximation, just a value for every state and action, updated until it stops moving.