Teaching an agent to play Flappy Bird (and solve other classic control problems) with Deep Q-Learning, built with PyTorch and Gymnasium.
The main agent (agent.py) is environment-agnostic: it reads a named hyperparameter set from hyperparameters.yml, so the same code trains on Flappy Bird, CartPole, or any other Gymnasium environment with a discrete action space and a flat observation vector.
| Flappy Bird | CartPole | Taxi |
|---|---|---|
![]() |
![]() |
![]() |
Each plot shows the mean reward over the last 100 episodes (left) and the epsilon decay schedule (right). Trained model weights are included in runs/.
The agent implements the core ideas from the DQN paper:
- Q-network (
dqn.py) — a small fully-connected network that maps an observation to a Q-value per action. - Experience replay (
experience_replay.py) — transitions are stored in a fixed-size buffer and sampled in random mini-batches, decorrelating the training data. - Target network — a periodically-synced copy of the policy network provides stable Q-value targets.
- Epsilon-greedy exploration — the agent starts fully random and gradually shifts to exploiting its learned policy as epsilon decays.
Training runs indefinitely, saving the model whenever a new best episode reward is reached and refreshing the training graph every 10 seconds. Stop it manually once you're happy with the results.
Requires Python 3.10+.
python -m venv .venv
.venv\Scripts\activate # Windows (source .venv/bin/activate on Linux/macOS)
pip install -r requirements.txtTrain an agent, passing the name of a hyperparameter set from hyperparameters.yml:
python agent.py flappybird1 --train
python agent.py cartpole1 --trainWatch a trained agent play (loads the saved model from runs/ and renders the environment):
python agent.py flappybird1Logs, training graphs, and model checkpoints are written to runs/<set-name>.{log,png,pt}. To experiment, add a new named set to hyperparameters.yml and pass its name on the command line.
| File | Description |
|---|---|
agent.py |
Main DQN agent — training and evaluation loop |
dqn.py |
Q-network definition |
experience_replay.py |
Replay memory buffer |
hyperparameters.yml |
Named hyperparameter sets per environment |
taxi_agent.py |
DQN variant for Taxi-v3 (one-hot encodes the discrete state) |
taxi_dql.py |
Earlier standalone DQN implementation for Taxi-v3 |
taxi_q.py |
Tabular Q-learning baseline for Taxi-v3 |
runs/ |
Training logs, graphs, and saved models |
OldResources/ |
Earlier iterations kept for reference |


