DRY your CruiseControl.NET Configuration

Don’t Repeat Yourself (DRY) is one of the principles of good software development. The idea is that there should ideally be one and only one “source of knowledge” for a particular fact or calculation in a system. Basically it comes down to not copying-and-pasting code around or duplicating code if at all possible. The advantages of this are many.

Advantages of DRY

  • There will be less code to maintain
  • If a bug is found, it should only have to be fixed in one place
  • If an algorithm or process is changed, it only needs to be changed in one place
  • More of the code should become reusable because as you do this you will parameterize methods to make them flexible for more cases

If it’s good for code isn’t it good for other things like configuration? Why yes it is.

Using CruiseControl.NET Configuration Builder

The Configuration Preprocessor allows you to define string properties and full blocks of XML to use for substitution and replacement. To start using the Configuration Preprocessor, you add xmlns:cb=”urn:ccnet.config.builder”, an xml namespace, to your document to tell the config parser that you plan to do this.

From there you can define a simple property like:


Or you can make it a full block of XML:

http://svn.example.com/svn/$(client)/$(project)/trunk D:\Builds-Net\projects\$(client)\$(project)\trunk
svn.exe
true


Defining Reusable Blocks

Using these ideas I wanted to come up with a templated approach that would allow me to share configuration among multiple projects. That way, if I added new statistics or change the layout of my build server, I would only have to change it in a single place. Thus keeping things DRY. It also encourages more consistency across multiple projects making things easier to understand.

So, I started defining some reusable blocks in the main ccnet.config file which you can see below. The exact details will depend on your configuration of course.

Full Example of config.xml



http://svn.example.com/svn/$(client)/$(project)/trunk D:\Builds-Net\projects\$(client)\$(project)\trunk
svn.exe
true



C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\MSBuild.exe
D:\Builds-Net\projects\$(client)\$(project)\trunk build.proj $(build-args)
$(build-targets)
600
D:\Program Files\CruiseControl.NET\server\Rodemeyer.MsBuildToCCNet.dll



C:\WINDOWS\Microsoft.NET\Framework\v3.5\MSBuild.exe
D:\Builds-Net\projects\$(client)\$(project)\trunk build.proj $(build-args)
$(build-targets)
600
D:\Program Files\CruiseControl.NET\server\Rodemeyer.MsBuildToCCNet.dll





D:\Builds-Net\projects\$(client)\$(project)\trunk\*.Test.xml
D:\Builds-Net\projects\$(client)\$(project)\trunk\*.CoverageMerge.xml
D:\Builds-Net\projects\$(client)\$(project)\trunk\*.CoverageSummary.xml
D:\Builds-Net\projects\$(client)\$(project)\trunk\*.FxCop.xml




D:\Builds-Net\projects\$(client)\$(project)\logs
















At the end of the file you can see the cb:include references. Those are one-line includes to include the configuration of each project. This makes things easier to manage, I think, because you only have to look at the individual project configuration.

Using Reusable Blocks in Individual Configuration Files

From there I need to make use of those defined blocks in in individual file. The first thing I needed to do was to set the parameters that I had defined as simple string replacements in the reusable blocks. Normally you would do that with cb:define as I showed above. But the trick is that you can only have one property with a given name defined. If you include multiple project configurations that doesn’t work. What does work is using cb:scope definitions. This allows for a value to be defined only within a specific scope.



...

From there you just need to start including the blocks that you defined in the main ccnet.confg within the scope block.

Full Example of Project Configuration













As you can see, the only one I didn’t template out was the email block because that depends on the developers working on each project.

Have fun bringing simplicity and consistency to your Cruise Control.NET configuration!

For the full details see the CruiseControl.NET Configuration Preprocessor documentation.

Tracking Project Metrics

How do you track the health of your software projects? Ideally you could come up with few, easy to collect metrics and have your Continuous Integration system generate the information and maybe even graph it over time. What we track is going to be based on a set of beliefs and assumptions, so I think we should make that clear.

My Software Metrics Beliefs and Assumptions

  • The simplest code that solves the problem is the best. Simple does not mean rote or repetitive. It means well designed, well abstracted, well factored.
  • Unit Testing improves the quality of code.
  • Overly complex code, code that is not well factored, “big” code is hard to unit test.
  • The metrics need to be easy to interpret and easy to gather or I won’t do it.

Based on those beliefs and assumptions we have defined the kinds of things we care about. We want simple, small classes and methods. We want classes that fit the Single Responsibility Principle. We want unit test coverage. And we want to know when we deviate from those things.

Inevitably this won’t tell you the whole picture of a project. Some deviation is inevitable as well (we’re not perfect). But this is giving us a picture into a project that would let us look at “hot spots” and determine if they are things we want to deal with. It will never tell you if the system does what a user really wants. It will never fully tell you if a project will be successful.

The Metrics I Came Up With

  • Unit Test Coverage – How many code paths are being exercised in our tests.
  • Cyclomatic Complexity – Number of methods over 10, 20, 40
  • Lines of Code – General size information
  • Methods over a certain size – Number of methods over 15, 30, 45 lines
  • Classes over a certain size – Number of classes over 150, 300, 600 lines
  • Afferent/Efferent Coupling – Dead code and code that does too much

Tools

I’m currently thinking mostly about .NET projects because at work we do a lot of small to mid-size projects using .NET. Many of the tools already exist and are Open Source in the Java world and many Java Continuous Integration servers support calculating some of those metrics for you already. So you’re probably in pretty good shape if you’re doing Java development.

FxCop

Out-of-the-box FxCop doesn’t really give you many of these things. But it does provide you a way that you can fairly easily your own rules using their API.

For Example to calculate Cyclomatic Complexity you can implement a class that extends BaseIntrospectionRule and overrides the VistBranch and VisitSwitchInstruction.


public override void VisitBranch(Branch branch)
{
if (branch.Condition != null)
{
Level ++;
}
base.VisitBranch(branch);
}

public override void VisitSwitchInstruction(SwitchInstruction switchInstruction)
{
Level += switchInstruction.Targets.Count;
base.VisitSwitchInstruction(switchInstruction);
}

Coverage

PartCover is an open source code coverage tool for .NET. It does the job but does not have what I’d call any “wow” factor.

NCover is a proprietary coverage tool. High on wow factor, but expensive. NCover was once free and open source and gained a good following, but they switched and closed it and made the decision to charge $150 – $300 a license for it depending on the version.

(I lament the state of the .NET ecosystem with regard to tooling. Either an OK open source version or an expensive commercial version is not a real choice. There are so many good options in the Unit Testing space, but not in the coverage space.)

NDepend Mini-Review

Disclosure: I received a free copy of NDepend from the creator. I would like to think that didn’t influence this at all, but wanted to make that known.

One of the tools that I’ve been looking at for .NET projects is NDepend. It seems to cover all of the cases that I mentioned above except for the Code Coverage case (although it integrates with NCover, but I haven’t looked at that). It has a really cool query languages, that looks a lot like SQL, that lets you customize any of the existing metrics that it tracks or write your own (it comes with so many that it seems like I just customize). It comes with so many metrics that in practice it can be seem quite overwhelming. I think the right thing to do for most projects is to pick the handful that you care about and limit it to that.

NDepend comes with NAnt and MSBuild tasks that will let you integrate it into your build automation. It also comes with an XSL stylesheet to integrate the NDepend output into CruiseControl.NET for reporting purposes.

Some things you might run into:

  • C# support is the primary language (e.g. No support for code level Cyclomatic Complexity for non-C# language, but still supports IL level CC). Only a problem if you are forced to do VB.NET of course, or use another CLR language.
  • A single license is $400 and 20 licenses or more are over $200 each. So price could be a concern to many people. I’m thinking it might make the most sense to run this on a CI server, so in that case you would probably only need a small number of licenses.
  • It integrates with code coverage tools, but currently only NCover. See the previous comments about the cost of NCover combined with the cost of NDepend if price is an issue.

What Else?

What does everyone else think? What do you care about and what are the right metrics for keeping track of the health of a software project?

Pragmatic Project Automation

In a previous post, I wrote a review of Pragmatic Version Control Using Subversion which covers effectively using version control systems on software development projects. This entry covers another book by the Pragmatic Programmers in their Pragmatic Starter Series.

Pragmatic Project Automation: How to Build, Deploy, and Monitor Java Apps

Pragmatic Project Automation by Mike Clark

Index:

  1. Introduction
  2. One-Step Builds
  3. Scheduled Builds
  4. Push-Button Releases
  5. Installation and Deployment
  6. Monitoring

Pragmatic Project Automation shares a lot with the rest of the Pragmatic Starter Series. As the name implies these books are intended to get you started down the path without necessarily showing you the entire way. The book is relatively short coming in at a slim 152 pages. This makes it a quick read and allows it to cover the topic from a high level. It sells the idea of automation and shows how to do automation in all aspects of software development. It shows examples everything from the simplest scripts using Unix shell and Windows tools all the way up to purpose-build programs that will make automation very easy for larger and more complex projects.

Chapter 1 – Introduction

The introduction lays the groundwork for automation. It categorizes automation into 3 types:

  • Scheduled Automation
  • Triggered Automation
  • Commanded Automation

Commanded Automation is probably the most familiar to most Java developers. This is the idea that you run a small script or an Ant build. This kind of automation is user driven and run when they want to do a certain task. Scheduled Automation, as it’s name implies, is something that is run based on time while Triggered Automation is automation that happens in response to a change in environment such as checking in code to a version control system. This groudwork is built upon during the rest of the book.

Chapter 2 – One-Step Builds

This is the most common kind of automation from what I’ve seen. This chapter covers building source code using a script. Being a Pragmatic book, it starts with a simple example using a shell script. It then does a good job of building a good template for an Ant build. One of the interesting things in this chapter is that it discusses using Groovy, a Java based scripting language, that has built-in support for Ant scripting. In many ways this Groovy example is very interesting because it’s a real scripting language so you can easily create your own custom code without the overhead of creating an Ant task. Maybe JRuby will catch on here (hint, hint JRuby folks) as I like the Ruby syntax a lot more than Groovy.

Chapter 3 – Scheduled Builds

Chapter 3 starts with the easiest way of doing a scheduled build, using cron on Unix or at on Windows to periodically run a build script. I really like the fact that the authors mentioned this because I really think it lowers the barrier to people getting into automation. There’s no need to over engineer a solution if you have simple needs. Of course this simple technique is fairly limited so they don’t stop there.

This chapter also covers using CruiseControl for doing both Scheduled and Triggered builds. It walks through the setup of your first project in CruiseControl. I’m a huge fan of Continuous Integration because it’s an always on backup for the processes that your team wants to enforce.

Chapter 4 – Push-Button Releases

Doing a build which you do everyday, hopefully multiple times a day, is one thing, but how do you move code from source control to QA, a staging server or a production server? Well you automate it of course! This chapter covers creating a script that you can use to turn your code into a form you can send to a customer. The simple examples are just creating the proper artifacts and creating archives like Zip or Tar files. It also talks about a number of processes including using automation to manage branching and tagging in your version control system. The branching and tagging automation not only makes the “release manager’s” life easier, it really helps to create a consistent process that means you’ll always be able to find and recreate any version in the future.

Chapter 5 – Installation and Deployment

This chapter covers 2 slightly unrelated (to me) but interesting ideas. The most obvious based on the name is creating an installer. It discusses everything from a simple Zip file “install” to using a full-blown installation system. The book uses the Open Source Nullsoft Scripted Installer (NSIS) as an example of building a Windows install. I’ve never used NSIS before so I can’t speak for its strengths or weaknesses, but it is used in a lot of Open Source projects. The introduction is very brief, so plan on doing some other reading if you want to use Nullsoft.

Chapter 6 – Monitoring

To wrap the book up, we get the Make chapter of the book. Now that you have triggered and scheduled builds running and applications in production, how do you know when there is a problem? This chapter talks about a number of good ideas starting with the easy ones of email and cell phones but taking some unanticipated turns into things like RSS feeds as well.

Some of the more interesting ideas where using custom applications to drive visual feedback monitors. Two of the more interesting examples where using Lava Lamps and Ambient Orbs to signal when a build is broken. The Lava Lamps would sit in the office and turn on/off red or green Lava Lamps based on the build status. Even more interesting is the Ambient Orb. It can glow in multiple colors. It communicates over a national wireless network like the pager network. So now you can take your magic orb with you and it will notify you when there’s a problem with the build.

The only problem with this chapter is that it really doesn’t have any implementations for these cool ideas. It leaves a lot up to the imagination.

Conclusion

The book is 152 pages, as I said above, which means that it is by no means a comprehensive reference to any of the topics covered. If you are already sold on the idea of automation, this would be good ammunition to convince other people on a team or in management to support the idea. If you are completely unfamiliar with the topic, then this book will offer you a good basis for getting started. If you are already familiar with the topic and are looking to expand on what you know or get ideas for more ways to automate then you probably won’t get a lot out of this book and should probably look elsewhere.

Pragmatic Project Automation: How to Build, Deploy, and Monitor Java Apps

Other Related

Ant
Cruise Control
Java Development with Ant

Continuous Integration Revisited

I had a chance to install and play with JetBrain’s new Team City beta today. Team City at its most basic is a Continuous Integration server. Continuous Integration (CI) systems are used to help manage a team’s software development process. Every time a developer checks in code, the CI server will check out the code from the source control system, compile the code, run unit test and perform static analysis and code coverage. This is a great way to back up your development process to ensure that you always have a working source tree and development is never held up because of mistakes.

There are other CI systems available, for example, in the past, I’ve used and liked CruiseControl. It is a very capable system that uses Ant build files to compile and test your code. It’s also Open Source and free.

I want to look at a couple of things that I really liked about Team City that make it shine.

Team City Configuration

The first part is the installation and configuration. It’s dead simple.

Team City is distributed as a WAR file, so if you already have a Servlet container like Tomcat up and running it’s as easy as dropping the WAR file into the webapps directory and deploying it through the management interface.

After the installation all of the configuration of a project can be done directly through the web interface, which is very nice compared to hand editing XML files that is often the case in other CI systems like CruiseControl.

The only gotcha is that in addition to the Team City itself, you have to install one or more Build Agents (this is really a feature, more on that later). I skipped this step at first and of course nothing would compile. Directly from the web application you can run either a Java Web Start or a Windows Installer application that will install a Build Agent on the computer. In my case I used the same computer where I was installing Team City – in a more intensive environment, it might make sense to install it on a different machine.

Multiple Build Agents

While it’s an extra step at the beginning, having multiple Build Agents is a really great idea. You can use many machines to build your projects spreading the load around to different machines. This is especially great if you have big project with lots of unit tests (you do have lots of unit tests right?) or just a bunch of projects and configurations that you would like to build. As software developers know, compiling code and running tests is a pretty intensive activity. Spreading the load around means that you can do more concurrent builds. This is a great way to decouple where things get compiled and tested from the centralized management application. This way you don’t have to have multiple instances of the CI system running on different machines if you want to load balance them.

Multi-Platform

Not only is the server itself multi-platform, thanks to it being written in Java, but Team City can compile and test both Java and .NET code. It can use a number of different configurations for compiling and testing. In Java, it can currently use Ant, Maven and Idea build files. On the .NET side of things, it can use MSBuild files and Visual Studio 2005 Solution files.

In addition to compiling code for these platforms, it can use either JUnit or NUnit for running Unit Tests as well. I really like this aspect of it because it offers shops that do more than one language a way to use the same systems for all of their development. At work we do both .NET development and Java development for our clients, so this makes a lot of sense for us. At least in the consulting world, this is a big win.

Delayed Commits

Team City has IDE integration built in. Currently it only supports Idea (why would you need anything else?) but they have already announced that for the 1.5 release they will support Eclipse, Netbeans and Visual Studio. There are a number of features that you need to have the Plugin installed to use, so for the short term Team City will be an OK solution, but not a great solution for non-Idea users.

Delayed Commits is quite possibly the best idea in Team City. It basically allows you to “commit” your changes to the server, but only have them checked into the source control repository if they compile and test correctly. No longer do you have to wait around for the CI server to report back that your checkin was successful before going to lunch or going home for the night. You can now Delay Commit your code and if everything works out it will be checked in for all the other developers to pull down, if not no one will be the wiser and you will fix any issues when you get back to your workstation.

Buttery

The UI is also very nice. They effectively use AJAX techniques to have a really nice user experience.

While this product is in Beta still and I haven’t had a chance to use it in anger yet, I think it’s a very promising tool. JetBrains has a great track record in delivering excellent products. I’m a huge fan of Idea and I can’t do .NET development without ReSharper. I think they have another hit on their hands with Team City. I haven’t heard anything about pricing yet, but the JetBrains stuff seems to usually be affordable.

As an aside, I have no intention of taking anything away from CruiseControl. It is still a great application as well (did I mention it was free).

(Jet Brains people, Yes, I’m willing to take paid endorsements :))