Paul Julius > Blog

Cup of Coffee Metric for Continuous Integration

September 14th, 2009

How much can you learn in the time it takes to get a cup of coffee?

How much can you learn in the time it takes to get a cup of coffee?

What happens in the time it takes to get a cup of coffee can tell you a lot about the quality of your continuous integration system. Try something out for me. Commit some code. Leave your desk to get a cup of coffee, or some other tasty beverage. When you get back to your desk, do you know whether your changes were any good? No? Yes? I hope so, and so should you. It’s called the cup of coffee metric.

The idea is simple. In the about the amount of time it takes to get a cup of coffee you should be able to get information about the quality of your changes. Look at it this way. You just committed some work. You need a short break before you move on to your next task. You don’t want to wait too long, or you might find yourself surfing the web instead of writing code. So, leave your desk. Get a coffee. Take a bathroom break. See what’s hot around the water cooler. Once refreshed, return to your desk. You don’t want to move on until you make sure your last commit didn’t cause any immediate problems. A good CI system will support this behavior by giving you some feedback in about that much time.

The level of information may not be complete. Enterprise build systems are notorious for the length of their builds. Enterprise systems are also notorious for waiting to provide any feedback until the end of the build cycle. That sucks. A little information would be better than none. It might only be an initial piece of feedback, like “Your changes compiled!” If that’s all your CI system can get done in cup of coffee time, that’s better than nothing. Your CI system might tell you everything is going smoothly, only to return later to tell you something broke. No worries, sometimes that will be the case. Hopefully not always.

As a build engineer, I am going to try my darnedest to provide you with as much information as possible while you are getting your refreshment. That’s how I keep my team humming along. I might even use a “build pipeline” to help facilitate this sort of thing, but that’s an implementation detail. I am always asking myself, “What’s the most I can possibly discover about a commit within about 5 or 10 minutes from the time a developer presses the button?”

This idea came from Jeffrey Fredrick‘s real life experiences. He and I co-present on various topics. We talk about CI a lot at CITCON, of course. Invariably we discuss the cup of coffee metric when talking about evaluating CI systems. I stole the idea from him for use in my consulting. (Most of my good ideas come from our community. Thank you internet!) With his permission, I figured it would make sense to write it up for the world. So, here you go.

Enterprise CI Culture Maturity Model

September 14th, 2009

Last Friday, Jeffrey Fredrick posted my “guest” article on the Anthill Pro blog. The article reflects on a culture addendum to Urbancode’s Enterprise CI Maturity Model.

I use the maturity model. It has worked well for me. It works especially well as a communication tool with my clients. I was feeling, though, like something was missing. The aspects listed on the original maturity model were thing focused. Here’s an example, Historical Reports are listed on the Intermediate level for Reporting. Excellent point, and I agree. But that’s a thing. One could publish historical reports until they are blue in the face. If nobody looks at them, then what’s the point?

OK, so most of the maturity model implies some action, I know. I wanted to call that out a little more explicitly.

In fact, I wanted to bring out the People side of this equation. People over process, remember? Actually, People over Automation tools, should also be true. So, that’s what I did. I tried to pull together some People aspects that will help you gauge your company’s success. I hope it helps. As Jeffrey noted when he emailed the CITCON mailing list, he looks forward to seeing the maturity model continue to evolve. I do, too. I think it has the potential to level the playing field a bit and mediate the “tool wars”. Those wars cloud the real mission – working software.

Please read the full post. Feel free to comment there. I will be interested to see where we go from here.

Feature Branches are Poor Man’s Modular Architecture

September 3rd, 2009

As dusk falls on some beautiful architecture.

Melbourne skyline, as dusk falls on some beautiful architecture.

Martin Fowler just published a much anticipated (by me at least) blog article on Feature Branching and how it relates to Continuous Integration:
http://martinfowler.com/bliki/FeatureBranch.html

I really enjoyed the article and recommend it to anyone considering using a distributed version control system (like Git), or even the non-distributed ones that promote branching, ClearCase and RTC’s scm come to mind.

As a consultant, I have run into the branching nightmare scenario time and time again. In fact, I spend a lot of effort moving clients to mainline development. So, though feature branching and distributed version control system’s prolific support for it may seem new it has been around for a long time. In the past though it was typically born from necessity, not forward thinking choice. Usually I hear something like, “We had to create [feature] branches because the developers kept breaking everything.” Can you guess the result? The developers still broke everything, but all the problems were saved until the end of the project. After a hard code freeze there would be 6 weeks of branch merging, aka: Integration Hell.

I thought we (as an industry) figured out a better way back in `99 with the Extreme Programming white book – Continuous Integration! If it hurts, do it more often, not less. Don’t save up all the pain for one big blast. Spread it out, in tiny bits, over the course of the project.

So, why are feature branches still a consideration? Because continuously integrated code bases require some architectural craftiness. In truth, it can be challenging. Architecting big enterprise systems provides enough challenges on its own. “Now you CI guys want us to make the application modular?” (Actual quote)

Yep, that’s right. At the recent “How to be really awesome at CI” panel at Agile 2009, Tom Sulston chaired a panel including myself and several other folks with deep CI experience. A question was asked about how to achieve the “check-in every hour” mantra I was pushing, without breaking everything in the codebase. Yes, feature branches were mentioned. Feature hiding was mentioned. Modularization (componentization) was mentioned. There are lots of great ways to achieve this that don’t require you to put your team back into Integration Hell territory.

Fowler, in the aforementioned article, lands on:

I much prefer designing the software in such a way that makes it easy to enable or disable features through configuration changes. My colleague Paul Hammant calls this Branch by Abstraction. This requires you to put some thought into what needs to be modularized and how to control that variation, but we’ve the result to be far less messy [than] relying on the VCS.

I agree.

Refactoring CruiseControl Configuration Files (From Agile 2009 Demo)

September 1st, 2009

Building a stable CI system requires some crafty architecture, just like building a functional central business district.

Building a stable CI system requires some crafty architecture, just like building a functional central business district.

At Agile 2009, Tom Sulston refereed a “cage match” with all the Continuous Integration tool vendors available. That included me representing CruiseControl. Each vendor had 10 minutes to share a bit about their tool and take questions from the audience. I got to go first!

I decided to focus on a couple characteristics that distinguish CruiseControl. The things I chose were project pre-configuration and a usage pattern for supporting multiple projects. Here’s a textual version of the demonstration that I gave at Agile 2009.

Let’s start with the sample CruiseControl configuration that comes with the binary distribution when you download it.


<cruisecontrol>
    <project name="connectfour">

        <listeners>
            <currentbuildstatuslistener file="logs/${project.name}/status.txt"/>
        </listeners>

        <bootstrappers>
            <antbootstrapper anthome="apache-ant-1.7.0" 
                     buildfile="projects/${project.name}/build.xml" target="clean" />
        </bootstrappers>

        <modificationset quietperiod="30">
            <!-- touch any file in connectfour project to trigger a build -->
            <filesystem folder="projects/${project.name}"/>
        </modificationset>

        <schedule interval="300">
            <ant anthome="apache-ant-1.7.0" 
                buildfile="projects/${project.name}/build.xml"/>
        </schedule>

        <log>
            <merge dir="projects/${project.name}/target/test-results"/>
        </log>

        <publishers>
            <onsuccess>
                <artifactspublisher dest="artifacts/${project.name}" 
                       file="projects/${project.name}/target/${project.name}.jar"/>
            </onsuccess>
        </publishers>

    </project>
</cruisecontrol>

That works great to have a little sample project running out of the box. Now, I want to add another project. In this case, I am going to use HttpUnit. I checked it out from subversion in my projects directory. After that, I need to configure CruiseControl to run the build for it. I’ll start by copying and pasting the project configuration used for the connectfour project. I only need to change one thing: the location of the final jar file. Instead of the jar being built into the target directory, it goes into the lib directory. So now, our configuration file contains the following after the end of the connectfour project definition.


    ...
    <project name="httpunit">
    
            <listeners>
                <currentbuildstatuslistener 
                    file="logs/${project.name}/status.txt"/>
            </listeners>
    
            <bootstrappers>
                <antbootstrapper anthome="apache-ant-1.7.0" 
                    buildfile="projects/${project.name}/build.xml" 
                    target="clean" />
            </bootstrappers>
    
            <modificationset quietperiod="30">
                <filesystem folder="projects/${project.name}"/>
            </modificationset>
    
            <schedule interval="300">
                <ant anthome="apache-ant-1.7.0" 
                    buildfile="projects/${project.name}/build.xml" />
            </schedule>
    
            <log>
                <merge dir="projects/${project.name}/target/test-results"/>
            </log>
    
            <publishers>
                <onsuccess>
                    <artifactspublisher dest="artifacts/${project.name}" 
                        file="projects/${project.name}/lib/${project.name}.jar"/>
                </onsuccess>
            </publishers>    
    </project>
</cruisecontrol>

That will work. Both the connectfour and httpunit projects will build in CruiseControl. Looking at the full configuration file, though, makes me feel a little ill. The file contains a ton of duplication. As noted, the only thing that differs between the two is the location of the project distributable, i.e. the jar file. Let’s refactor that a bit to pull out some of the duplication.

For CruiseControl, we can take advantage of the plugin preconfiguration features. Projects are plugins. So, we can extract all the common elements to a project “template” that all our projects use.

Within the individual projects we can override the elements that are unique to that project. Sort of like creating a base class in object oriented programming. Subclasses extend the base class and override the methods for which they need to define explicit behavior.

Here’s the full configuration file after the refactoring:


<cruisecontrol>
    <plugin name="project">
        <bootstrappers>
            <antbootstrapper anthome="apache-ant-1.7.0" 
                buildfile="projects/${project.name}/build.xml" 
                target="clean" />
        </bootstrappers>

        <modificationset quietperiod="30">
            <filesystem folder="projects/${project.name}"/>
        </modificationset>

        <schedule interval="300">
            <ant anthome="apache-ant-1.7.0" 
                buildfile="projects/${project.name}/build.xml"/>
        </schedule>

        <log>
            <merge dir="projects/${project.name}/target/test-results"/>
        </log>
        
        <!-- Publishers are up to the individual projects for now. -->
        
    </plugin>
    
    <project name="connectfour">
        <publishers>
            <onsuccess>
                <artifactspublisher dest="artifacts/${project.name}" 
                    file="projects/${project.name}/target/${project.name}.jar"/>
            </onsuccess>
        </publishers>
    </project>
    
    <project name="httpunit">
        <publishers>
            <onsuccess>
                <artifactspublisher dest="artifacts/${project.name}" 
                    file="projects/${project.name}/lib/${project.name}.jar"/>
            </onsuccess>
        </publishers>    
    </project>
    
</cruisecontrol>

That’s a lot better. Now the projects share a common template. They each define their own publishers, which allows them to override the location of the jar file.

One more step. The project definition could be extracted from this central configuration file and included with the files that the project owns. I use the include.projects plugin for CruiseControl.

I like to create a file right next to my build file, e.g. build.xml in the case of Ant. Then the project, or component if you prefer, team can take ownership for their project definition. As a CruiseControl administrator, all I have to do is point to their configuration to include their build in a particular CruiseControl instance. Here’s what my main configuration file looks like after that change:


<cruisecontrol>
    <plugin name="project">
        <bootstrappers>
            <antbootstrapper anthome="apache-ant-1.7.0" 
                buildfile="projects/${project.name}/build.xml" 
                target="clean" />
        </bootstrappers>

        <modificationset quietperiod="30">
            <filesystem folder="projects/${project.name}"/>
        </modificationset>

        <schedule interval="300">
            <ant anthome="apache-ant-1.7.0" 
                buildfile="projects/${project.name}/build.xml"/>
        </schedule>

        <log>
            <merge dir="projects/${project.name}/target/test-results"/>
        </log>
        
        <!-- Publishers are up to the individual projects for now. -->
        
    </plugin>
    
    <include.projects file="projects/connectfour/cruisecontrol.config.xml" />
    
    <include.projects file="projects/httpunit/cruisecontrol.config.xml" />
    
</cruisecontrol>

Then, the project team gets to own the cruisecontrol.config.xml file. The one for httpunit looks like the following:


<cruisecontrol>
    <project name="httpunit">
        <publishers>
            <onsuccess>
                <artifactspublisher dest="artifacts/${project.name}" 
                    file="projects/${project.name}/lib/${project.name}.jar"/>
            </onsuccess>
        </publishers>    
    </project>
</cruisecontrol>

The httpunit team can version that file as an artifact important to their project, just like the build file.

As a CruiseControl administrator at a big company, I might have tens or hundreds of different projects that I want to include in my server. I also might want to include the same project in lots of different CI servers. Extracting the project specific definitions allows me to do this with ease!

CITCON Paris 2009 Almost Here

September 1st, 2009

CITCON Paris, the European instance of the Continuous Integration and Testing Conference, is just around the corner. Our registrations filled quickly and we had to put quite a few late registrants on the waiting list. The committee has begun the task of sorting through the waiting list now. We hope to accommodate as many as possible.

For those attending, I’ll see you in Paris.

If you can’t make it to this event, keep an eye on the CITCON website for 2010 events in North America, New Zealand/Australia and Europe.