How to Build a Python Bot That Can Play Web Games

In this tutorial we’ll explore the ins and outs of building a Computer Vision-based game bot in Python, which will be able to to play the popular Flash game Sushi Go Round. You can use the techniques taught in this tutorial to create bots for automatically testing your own web games.

---

Final Result Preview

Let’s take a look at the final result we will be working towards:

---

Prerequisites

This tutorial, and all the code within it, requires that a few additional Python libraries be installed. They provide a nice Python wrapping to a bunch of low-level C code which greatly eases the process and speed of bot scripting.
Some of the code and libraries are Windows-specific. There may be Mac or Linux equivalents, but we won’t be covering them in this tutorial.
You’ll need to download and install the following libraries:

• The Python Imaging Library
• Numpy
• PyWin
All of the above have self installers; Running them will automatically install the modules into your \lib\site-packages directory and, in theory, adjust your pythonPath accordingly. However in practice this doesn’t always happen. Should you begin receiving any Import Error messages after installation, you’ll probably need to manually adjust your Environment Variables. More information on adjusting Path Variables may be found here.

The final tool we’ll need is a decent paint program. I suggest Paint.NET as an excellent free option, but any program with rulers that display their measurements in pixels can be used.
We’ll use a few games as examples along the way

---

Introduction

This tutorial is written to gave a basic introduction to the process of building bots that play browser-based games. The approach we’re going to take is likely slightly different than what most would expect when they think of a bot. Rather than making a program that sits between the client and server injecting code (like a Quake or C/S bot), our bot will sit purely on the ‘outside’. We’ll rely on Computer Vision-esque techniques and Windows API calls to gather needed information and generate movement.
With this approach we lose a bit of refined detail and control, but make up for it in shortened dev time and ease of use. Automating a specific game function can be done in a few short lines of code, and a full-fledged, start-to-finish bot (for a simple game) can be cranked out in a few hours.
The joys of this fast approach are such that once you get familiar with what the computer can easily ‘see’, you’ll begin to view games slightly differently. A good example is found in puzzle games. A common construct involves exploiting human speed limitations to force you into a less than optimal solution. It’s fun (and pretty easy) to ‘break’ these games by scripting in movements that could never be accomplished by a human.
These bots are also very useful for testing simple games – unlike a human, a bot won’t get bored playing the same scenario over and over again.
Source code for all of the tutorial examples, as well as for one of the completed example bots, may be found here.
Have fun!

---

Step 1: Create a New Python Project

In a new folder, right-click and select New > Text Document [...]
Read more: How to Build a Python Bot That Can Play Web Games