I worked with the Cloud Control Panel (aka Reach) team this summer as an intern at the Rackspace Blacksburg office. The Cloud Control Panel is a handy web interface to the OpenStack-based Rackspace Cloud APIs. It allows the user to provision a server, allocate storage, create database instances and much more, all with the click of a button. (Well, maybe more than one button, but you get the point.)
I quickly assimilated into the Platypus (formerly Narwhal) sub-team pretty quickly and began working on user stories from day one. I was impressed by the degree of adoption of extreme programming (XP) practices in Reach. I think its emphasis on customer requirements fits well with Rackspace’s motto of Fanatical Support. As part of this approach, I experienced pair programming; test-driven development with unit, integration and acceptance tests; continuous integration; code reviews based on well defined coding standards; a sustainable pace of development; and small releases that helped me release to production on the first day at work!
As this was my first time with continuous integration (CI), I wanted to learn more about it. I believe that the best way to learn is through hands-on experience, so I picked up a story that involved experimenting with the CI infrastructure.
Reach hosts code on Github and uses Jenkins to build, test and deploy code. The team follows a branch-review-test-merge model. When a developer wants to add some code to the main codebase, they first create a new branch, make their changes on that branch, push the branch to Github and then open a pull request. On this pull request, fellow teammates provide code review comments. The developer must either clarify the comments with proper justification or fix the corresponding issue in their code. The developer then makes sure that their new code passes all the tests. Finally, after all tests have passed and all review comments have been resolved, the developer can merge their branch with the master branch. This model forms the backbone of continuous integration in Reach, as it enables merging all developer working copies into the shared main codebase several times a day.
Jenkins comes into a play at the test phase. The Jenkins setup consists of developer created jobs. Each job is a combination of
Several jobs can be combined to form a hierarchy of jobs. The original hierarchy of the job that ran all the tests for a branch in the aforementioned model was:
The most obvious approach was to break the job down into independent components that could be run in parallel. This involved figuring out the dependencies between the various parts of the master job and configuring Jenkins to understand the new hierarchy. Jenkins is a plugin driven ecosystem and I stumbled upon many useful plugins during the course of this task, most notably the Join plugin, the Copy Artifact plugin and the Parameterized Trigger plugin.
The final hierarchy that I came up with was:
The parallel speedup in this case is 15/6 = 2.5. This means that the new job is 2.5 times faster than the old one. Now the developer has to only wait for 12 minutes instead of 21!
Technical fun aside, the best part about interning at Rackspace was that I could pick any task I wanted and work on it with full developer freedom. The team went out of its way to make sure I had access to the entire infrastructure and any resources I needed. The only thing required from my side was enthusiasm, determination and the courage to take on a new challenge.