Breakout

The Breakout demo is the classic paddle-and-ball game. A paddle slides along the bottom to keep a bouncing ball in play, and the ball smashes a wall of bricks at the top. The genetic algorithm evolves a small neural network to control the paddle. Early on the paddle barely moves and the ball slips past at once. After a few generations the paddle starts shadowing the ball and the rallies stretch out.

It lives alongside the other live-agent games (Flappy Bird, Snake, the Self-Driving Car, the Dino Runner, Pong, Gem Collector and Frog Crossing) under the same Live Agent demo. Use the tabs at the top of the page to switch between them.

The Problem¶

The network has one job: keep the ball alive by getting the paddle under it. It reads seven inputs: the ball's position and velocity, the paddle's position and speed, and how far the paddle is from where the ball will land. It picks one of three moves: push left, ease off, or push right. The largest output wins. The paddle has momentum, so a push speeds it up over a few frames and easing off brings it to a stop, the way a hand on a controller moves.

The network never sees the bricks. They break on their own whenever the ball reaches them, so the wall comes down as a side effect of good paddle play, not something the network has to aim at. Keeping the agent's job this simple is what makes it reliable to learn while the screen still fills with breaking bricks.

Fitness rewards how long the ball stays in play, with a bonus for every brick broken and every paddle hit. Each time the paddle turns around costs a little, so between two equally safe networks the GA prefers the one that moves with purpose instead of wobbling around the ball. A miss ends the run. The ball speeds up a little with each paddle hit, so long rallies get gradually harder, and clearing the whole wall drops in a fresh one so a strong policy just keeps going.

How a Run Looks¶

Click Start. The current best network plays a run live, and a panel in the corner shows the three move scores each step with the chosen one highlighted. The chart on the right tracks the best score for each cycle. The launch angle changes each cycle, so the agent has to track the ball rather than repeat one sequence.

Generation zero mostly loses the ball almost immediately. Within a few cycles the paddle begins to track the ball across the court and the rallies stretch out, until the best paddle keeps the ball in play for the whole two-minute game. Every paddle is tested on two serves, each as long as the game you watch, so a champion has already proved itself for the full length. In our tests (24 runs, 20 cycles each), half the runs had a champion last the full two minutes by the eighth cycle, and in the last five cycles 116 of 120 champions did.

Playing It Yourself¶

Switch the mode toggle from Agent to You to take over. Slide the paddle with the left and right arrow keys, or drag the joystick over the game left or right. The paddle keeps sliding for as long as you hold. The joystick appears wherever you touch or click the game, so you can play with your thumb in whichever spot is comfortable, and it stays there for next time. When the game ends, tap the game (or press Space, or Play again) to start a new round straight away. There is no step limit in You mode, so the game lasts until the ball gets past you. Switch back to Agent at any time and the evolved network picks up from where you left off.

Settings Worth Trying¶

Open Settings to tune the genetic algorithm for the next run:

• Population size. More candidates per generation means more variety to select from, at the cost of a slower cycle.
• Generations per cycle. How much evolving happens between each live run you watch. The default is 2, so the improvement shows over the cycles; raise it for a stronger player sooner.
• Mutation rate. How much the networks are randomly perturbed each generation. A little keeps the search exploring; too much stops it from settling on a good paddle.
• Max steps / episode. How long each training game lasts: 3600 steps (two minutes) by default, the same as the game you watch. A shorter training game makes each cycle faster, but a paddle is only tested for that long.

Breakout records and shares like the other demos. See Sharing and recording for how to capture a clip of a long rally.