Showing posts with label tracking. Show all posts
Showing posts with label tracking. Show all posts

Friday, March 23, 2012

How to Build a Python Bot That Can Play Web Games

In this tutorial we’ll explore the ins and outs of building a Computer Vision-based game bot in Python, which will be able to to play the popular Flash game Sushi Go Round. You can use the techniques taught in this tutorial to create bots for automatically testing your own web games.

Final Result Preview

Let’s take a look at the final result we will be working towards:

Prerequisites

This tutorial, and all the code within it, requires that a few additional Python libraries be installed. They provide a nice Python wrapping to a bunch of low-level C code which greatly eases the process and speed of bot scripting.
Some of the code and libraries are Windows-specific. There may be Mac or Linux equivalents, but we won’t be covering them in this tutorial.
You’ll need to download and install the following libraries:
  • The Python Imaging Library
  • Numpy
  • PyWin
  • All of the above have self installers; Running them will automatically install the modules into your \lib\site-packages directory and, in theory, adjust your pythonPath accordingly. However in practice this doesn’t always happen. Should you begin receiving any Import Error messages after installation, you’ll probably need to manually adjust your Environment Variables. More information on adjusting Path Variables may be found here.
The final tool we’ll need is a decent paint program. I suggest Paint.NET as an excellent free option, but any program with rulers that display their measurements in pixels can be used.
We’ll use a few games as examples along the way

Introduction

This tutorial is written to gave a basic introduction to the process of building bots that play browser-based games. The approach we’re going to take is likely slightly different than what most would expect when they think of a bot. Rather than making a program that sits between the client and server injecting code (like a Quake or C/S bot), our bot will sit purely on the ‘outside’. We’ll rely on Computer Vision-esque techniques and Windows API calls to gather needed information and generate movement.
With this approach we lose a bit of refined detail and control, but make up for it in shortened dev time and ease of use. Automating a specific game function can be done in a few short lines of code, and a full-fledged, start-to-finish bot (for a simple game) can be cranked out in a few hours.
The joys of this fast approach are such that once you get familiar with what the computer can easily ‘see’, you’ll begin to view games slightly differently. A good example is found in puzzle games. A common construct involves exploiting human speed limitations to force you into a less than optimal solution. It’s fun (and pretty easy) to ‘break’ these games by scripting in movements that could never be accomplished by a human.
These bots are also very useful for testing simple games – unlike a human, a bot won’t get bored playing the same scenario over and over again.
Source code for all of the tutorial examples, as well as for one of the completed example bots, may be found here.
Have fun!

Step 1: Create a New Python Project

In a new folder, right-click and select New > Text Document [...]
Read more: How to Build a Python Bot That Can Play Web Games

Friday, April 15, 2011

RGB Wheel Tracker

Hi dear readers, I've worked on a simple RGB recognition algorithm which is nicely accurate.

Let me show you a little demonstration :

RGB Wheel Tracker from didier.brun on Vimeo.

Algorithm

The algorithm is based on a paper I've read a few years a go, but I can not find the link. The idea is ready simple and require a simple pass on the image.

The first phase is to compare the proportional amount of red, green and blue from 3 relative points, and calculate a score for each pixel.

The second phase is then to blur the result, apply a threshold on the image.

The last phase is to find the blob with the biggest size.

This algorithm is applied for each orientation pattern according to this logic :

  • Iterate the marker searching on each orientation, one by frame rendered (top red, top blue, top green, top red, top blue, ...)
  • If a marker is found, keep the searching on the corresponding orientation only
  • When the marker is lost, iterate on the 3 orientation (point 1) until a marker is found

A sample image :

RGB Wheel Tracker sample image

The algorithm [...]

Read more: RGB Wheel Tracker

Monday, March 28, 2011

Link: Face-tracking and 3D Scene

Link: Face-tracking and 3D Scene:

3D Scene and Face tracking demo

I may be slow on the uptake here, but Mr.doob has some awesome HTML5 and Flash Player showcase pieces over at mrdoob.com. It’s hard to pick a favorite, because they’re all darn impressive, but I’m a fan of the Face-tracking and 3D Scene demo (which I’ve screenshotted above).  It’s based in Flash, because it has to access the computers’ webcam, and is a pretty neat concept.

If it didn’t require the whole webcam-access notification message that makes the whole effect so obvious, I’d definitely want to slide this into loads of projects, just for the hell of it!

"