Travis CI and GUI Testing

Writing computer emulators in my spare time is actually something I really enjoy doing. The first emulator I wrote was a simple Chip 8 emulator in C. While that was a good learning experience, debugging it was frustrating. Simple unit test frameworks like MinUnit work well for C code, but writing more extensive tests can be time consuming, especially if you dive right into a project first, without following some type of Test Driven Development(TDD) philosophy. So, my second cut of the emulator was born – this time a Chip 8 emulator written in Python with the goodness of a unit test framework behind the language, and TDD on my mind.

Once I got the bulk of the new emulator up and running with unit tests, most of my changes were style tweaks in order to satisfy Pylint, as well as to correct some of the comments in the code. I started to grow tired of running all of my unit tests each time I made a change to the codebase since it was unlikely that my changes would impact previously tested and corrected code. So, I turned to the power of the web to help me offload the burden of locally running the full suite of tests by looking for a good Continuous Integration (CI) service.

Finding a Continuous Integration Service

While Jenkins, Bamboo and TeamCity are all good CI servers that I have used in the past (Jenkins and Teamcity are two services with a zero cost to set up and start using), they all required a local install. While I’m not against setting up my own services, I wanted something I could hook into with little maintenance on my end. In other words, if I was running on a laptop or netbook, I wanted the power to simply make my changes and push to a remote repository and have the unit tests run automatically. To address this need, I quickly found Travis CI, a continuous integration service that is freely available for open source projects.

Talking to Travis CI

Modifying my GitHub repo to make use of Travis CI was as simple as logging into Travis with my GitHub credentials, letting it sync up, and flipping the switch on Travis to tell it to scan my repos when changed via a commit hook. From there, all I needed was a simple YAML configuration file (.travis.yml) in my project’s root directory to configure it for Travis. Here was the configuration file:

language: python
python:
  - 2.7
script:
  - nosetests --with-coverage --cover-package=chip8

I simply added it to my git repository, committed the changes, and pushed it to master:

git add .travis.yml
git commit -m "Travis YAML file"
git push origin master

Then, I watched on Travis as the changes were picked up on the repo, and my project was built with the unit tests.

Testing GUI Applications

While Travis CI was really good, I quickly ran into a problem – I had unit tests for my Chip 8 screen class that needed to write to an actually display while they ran. After scouring the docs, I quickly found that Travis CI supports GUI testing through the use of the X Virtual Framebuffer. All you need to do is put the display port in your configuration, and start the X Virtual Framebuffer:

env:
  - DISPLAY=:99.0
before_install:
  - sh -e /etc/init.d/xvfb start

All together, the configuration file is:

language: python
python:
  - 2.7
env:
  - DISPLAY=:99.0
before_install:
  - sh -e /etc/init.d/xvfb start
script:
  - nosetests --with-coverage --cover-package=chip8

Problem solved! Now I can run the full range of tests on my projects each time I push to my repo. Travis CI even emails me when I’ve broken a build. Travis CI also offers a service called Travis Pro for private repos if you have code that you need to keep closed source.

Conclusion

Travis CI is really easy to use, and for open source projects, offers simple configuration and flexibility. If you are looking for continuous integration, and don’t want to have to turn to tools such as Jenkins or TeamCity, then give Travis CI a shot. With a few simple changes, you can even test your GUI application using Travis.