Flomosa Company Blog

Finally found time to clean up some of the Flomosa build environment. None of this really aids in the “when will it be released” department, but I think it’s cool none-the-less. Previously, in my Hudson build jobs for my unit tests, code coverage, pep8 and pylint, I had the actual commands in the job itself. I spent the last few days creating a Makefile so I can do things like:

• gmake xunit.xml
• gmake coverage.xml
• gmake pep8.txt
• gmake lint.txt

(gmake because Hudson is running on FreeBSD)

My next step is to automate the deployment of code up to Google AppEngine if the Hudson build is successful.

I spent a long time searching around google for information on this topic, but didn’t come up with too many hits, so I figured I’d outline what we’re doing at Flomosa.  I wanted a way to be able to run a testing suite against a Python-developed AppEngine program outside of the development environment.

I found a few posts on the topic, but nothing related to AppEngine. Then I came across PubSubhubbub and saw how their testing suite was set up and figured I could leverage some of their code to configuring the necessary stubs to simulate the datastore and taskqueues.

The tests are created using the standard unittest libraries and can be executed by a run_tests.py script on the command line. Using the HandlerTestBase from the PubSubhubbub example, the request handler classes can be executed as if they were being run inside the dev_appserver. The fix_paths() function from the example is vital to this all working correctly. After that is ran, you can run this code to set up the stubs for the development environment:

root_path = os.path.realpath(os.sep.join([os.path.dirname(__file__), '..']))
dev_appserver.SetupStubs(
TEST_APP_ID,
root_path=root_path,
login_url='',
datastore_path=None,
blobstore_path=tempfile.mkdtemp(suffix='blobstore_stub'),
require_indexes=require_indexes,
clear_datastore=False,
disable_task_running=False
dev_appserver_index.SetupIndexes(TEST_APP_ID, root_path)

I also found you don’t even need to specify a datastore_path as it will store each test run in memory.

We’re using Hudson as our continuous integration server and I have the following jobs configured:

• flomosa – runs the test suite
• flomosa-coverage – runs the code through coverage.py
• flomosa-pep8 – runs the code through pep8
• flomosa-pylint – runs the code through pylint

Similar to the github process described above, code is developed in any branch the developer wants. When they are comfortable with their changes and the local unittests pass, the code is merged into the pending branch on github. Hudson will checkout the pending branch from github and merge it into the master branch in the Hudson job workspace. The code will be merged and the run_tests.py script is executed to run the tests (without having AppEngine running). If the tests pass, the following happens:

1. Tests results are published in Hudson
2. The master branch on Hudson is pushed back to the master branch on github (no changes are ever directly committed to the master branch on github)
3. A build tag is created and pushed to github
4. The downstream jobs are executed (coverage, pep8 and pylint)

It actually all seems to work very well. Hopefully this post will be of some use to someone as it took me awhile to piece this all together.