Our Buildbot Configuration

Setting up buildbot is a serious pain: it's pretty complicated, and a lot of fiddling is needed. The documentation is pretty good but we figure examples are always welcome... so here you go!

Below are the segments of our buildbot configuration that we think are noteworthy, or at least example-worthy. Specifically, we discuss how to put the "force build" option behind a password; how to use locks to prevent resource contention; how to drive builds based on check-ins; and how to start buildbot on boot.

Before reading this, you should definitely check out Grig's article on setting up buildbot.

Our master configuration file is available here.

Keeping the "Force Build" Option Private

If you go to our buildbot status page, you'll note that there's no "force build" button. We've put those under a separate, password-protected page, here. (Yes, it is password protected ;).

If you're going to have a public buildbot status page, this is pretty obviously what you want: a public page that doesn't let random users force builds, and a private page that developers can use to make a particular buildbot slave do the build. The way we did this is with the Apache ProxyPass and ProxyPassReverse directives. This lets you password protect the URLs as well as putting them under a specific virtual domain -- something that you can't do with buildbot directly, I think.

The setup for this is pretty easy. First, create two separate buildbot Waterfall instances, both bound to the 127.0.0.1 interface; on the public one, set 'allowForce' to False to disable the 'force' buttons. (You'll need buildbot 0.7.2 or above in order to use the 'interface' config option.)

public = html.Waterfall(http_port="tcp:12344:interface=127.0.0.1", allowForce=False)
private = html.Waterfall(http_port="tcp:12345:interface=127.0.0.1")

c['status'] = [public, private]

This binds the status pages to a port that cannot be accessed externally.

Now, configure your Apache2 virtual hosts file like so:

<VirtualHost 69.55.232.78:80>
ServerName agile.idyll.org

## public

<Location "/buildbot/">
Satisfy any
Allow from all
</Location>

ProxyPass /buildbot/ http://localhost:12344/
ProxyPassReverse /buildbot/ http://localhost:12344/

## private

<Location "/buildbot_admin/">
AuthType Basic
AuthName "Buildbot Admin"
AuthUserFile /some/path/to/htpassword/file
Require valid-user
</Location>

ProxyPass /buildbot_admin/ http://localhost:12345/
ProxyPassReverse /buildbot_admin/ http://localhost:12345/

</VirtualHost>

(You'll need to load mod_proxy for this to work.)

The ProxyPass and ProxyPassReverse directives tell Apache to forward requests on appropriately & rewrite URLs backwards, respectively.

Locking System Resources for each Build Slave

Early on in the buildbot process, we discovered that tests sometimes interfered with each other. In particular, our twill tests involved starting up a Web server that bound to a port, and if the twill tests were started in parallel then one test would end up talking to the other Web server. Not good. Another situation arose later on where Firefox (used to run the Selenium tests) could not execute in parallel on two different VNC servers without some profile magic.

Luckily, buildbot has both master (global) and slave (local) locks for just this situation! Basically just define everything as you would normally, and then add in the locks at the individual command level.

from buildbot import locks

port_lock = locks.SlaveLock("ports")

unit_tests23 = s(UnitTests,
                 command="/usr/bin/python2.3 bin/run-timed-unit-tests",
                 locks=[port_lock])

unit_tests23 = s(UnitTests,
                 command="/usr/bin/python2.3 bin/run-timed-unit-tests",
                 locks=[port_lock])


f23 = factory.BuildFactory([..., unit_tests23, ...])

f24 = factory.BuildFactory([..., unit_tests24, ...])

c['builders'] = [
    dict(..., slavename='vallista', ..., factory=f23),
    dict(..., slavename='vallista', ..., factory=f24)
]

Choose a master lock if you have something that should locked across all build processes, and a slave lock if it's only something that should be locked on a single slave.

(Note here that 'slavename' is the same across the two builders; this is what targets the lock.)

Driving Builds upon Subversion Check-in

It's great to be able to tell the build slaves to run immediately after a check-in. This is pretty easy to do, once you figure out what port to connect to ;).

In your master.cfg file, add the following:

from buildbot.changes.pb import PBChangeSource

c['sources'] = [ PBChangeSource() ]

Then, in your 'hooks/post-commit' file, call:

/path/to/svn_buildbot.py --repository "$REPOS" --revision "$REV" --bbserver localhost --bbport 9989

(The 'svn_buildbot.py' file is in the buildbot /contrib/ directory.)

The only tricky part is that the port number '9989' is the one that's configured with this command:

c['slavePortnum'] = 9989

That is, the port used to talk to slaves is also the port used by the change notifier, which makes a certain amount of sense.

Starting Buildbot on Boot

Put the following in your build slave account's crontab and, once per reboot, the given command will be executed.

PATH=/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games:/usr/bin/X11
@reboot /usr/bin/buildbot start /path/to/buildslave/config

(This was honestly new to me -- I'd never seen this crontab extension before!)

Note that you don't need to set PATH explicitly unless you have something special on there; in our case, '/usr/bin/X11' wasn't added by default, so 'vncserver' wasn't working.

Running Buildbot on Various Platforms

Read Grig's post for details/gotchas about running buildbot on Windows, Solaris and AIX.