> |

Deep Learning for playing Video Games (Academic)

Aug, 2016 - Dec, 17

< Back

Academic project carried out with Dr. Akshi Kumar during junior year at Delhi Technological University.

Motivation

As my 2nd year ended, I had gained skills in app development, algorithms & data structures and more. I wanted to explore machine learning next.

As I explored the field, I came across Deepmind's (Google) paper 'Playing Atari with Deep Reinforcement Learning'. They used Deep Neural Nets to play multiple video games. It was fascinating because the same algorithm could play multiple games.

I decided to replicate this paper as part of my academic project, and compare it with other reinforcement learning algorithms.

Project

I selected Flappy Bird as the game to evaluate the algorithms on. Falppy bird hype had begun to dwindle by 2015. But, I was still obsessed as I had recently made a flappy bird clone for Github's Game Jam (check it out !).

Modelling the game as an MDP for Q learning

Modelling the game is straightforward:

  • The screen of the game can be modelled as a 2 dimensional grid, with bird at position (x,y) and velocity (vx,vy).
  • The pipe openings can are at x1,y1.
  • The objective of the game is to avoid collisions with the pipe and the ground.
  • The agent only has two options to tap or not tap. Depending on this the state of the game changes.
  • The gravity is constantly changing the game state by pulling the bird downwards.

Using these parameters we can model the game as a Markov decision process, and use Q learning (dynamic programming/ table approach) to learn the best possible move (tap or not tap) at each state of the game. The policy or the Q function table can also be approximated with a neural network which takes the game state as input and outputs the best move.

On the other hand, in Deepmind's approach, modelling is not needed. The deep neural net takes the some number of video frames as input to predict the action required (Previous frames are needed to set context on the velocity of the bird).

The agents in action

As part of this project I also made a contribution to PyGamePlayer an open source library to create game playing agents for games built using PyGame framework.

< Back | Top