Sikuli, or how I improved my Personal Hygiene using Test Automation

Written by Ithai

Test Automation is the practice of running scripted tests automatically as opposed to running the same tests manually.

While working on Fly ‘n’ Write, I’ve written a small set of C++ unit-tests to verify critical parts of my code. This worked pretty well and increased my confidence in my builds, but something was missing.

By definition, unit tests don’t usually cover user interaction. To test even the most basic playing scenarios in Fly ‘n’ Write, I had to repeat a set of basic tests manually, day after day… same tests… over and over again.

This was getting to be boring, but the worst part is that as the project evolved, it simply consumed too much of my (precious) time.

Enter Sikuli

Sikuli is a screen automation technology. Sikuli scripts use commands such as find, click, doubleClick and hover to emulate user actions on the screen. User actions are applied using what Sikuli calls PSMRL, an interchangeable parameter that can hold a screen pattern to detect, a string path for an image to detect, a match object from a previous find operation, a region or a screen location.


Using Sikuli, I was able to test Fly ‘n’ Write, an OpenGL/ES app, from the user standpoint.

I’ve ended up with a set of scripts that run automatically for 30 minutes each day. That’s 30 minutes of extra time I can use for - say taking a shower, brush my teeth and shave == profit.

Installing Sikuli

The Sikuli IDE and tests runner is a Java application (grrr…) and I had to install JRE, openCV and Tesseract in my Arch Linux machine. Once I had these dependancies installed I invoked the setup jar:

    java -jar sikuli-setup.jar

In the setup options box, I chose to install the IDE:

Sikuli Setup

Shortly after I had Sikuli installed and I was able to run the IDE:

Sikuli IDE

Building my first script

To stay organized, I created a dedicated folder to hold my Sikuli tests. Sikuli store each test in a separate folder, holding the script as well as some other required metadata.

On the right hand of the Sikuli IDE you can see a commands palette. This is very helpful during your first steps with the tool. Open your game window and choose click to record your first testing step. Sikuli will ask you select a region from the screen to click on. Once you do so, you will see the new command in your script.

Here is how I recorded my interaction with Fly ‘n’ Write:

I finally ended up with my first home screen test:

Basic Sikuli Test

It can get complicated

The more user scenarios I needed to test, the more I learned of what Sikuli can do. I learned how to verify state using find and wait, how to control the mouse and how to reuse code.

More Tests

The whole process took no more than a couple of hours! and I finally ended up with a basic sanity test-suite I can use and run on each build day:

Tests Suite

To run the suite I use:

    ./runIDE -r ~/workspace/fly-n-write/sikuli/main.sikuli

Having said so..

Your mileage with Sikuli may vary. For some games it may be very difficult to use it effectively, but for many projects Sikuli can help automate a significant amount of tests and reduce the workload from indie developers and small teams. As for me, I’m one cleaner, better looking programmer :)