Would you like any toppings on your build agent, Madam? No thanks!

I have long been of the opinion that as far as possible, builds should be able to run on build agents without having to install any software on them. [Note – where I work we use TeamCity for continuous integration, and thus I use the following terminology: Build Server = the machine that coordinates the builds; Build Agent = the machines that actually run the builds.] Any necessary library files should be stored within the source control system and checked out alongside the source code. My two main arguments for this position for this have, until this Friday, been:

  • Build agents are often shared between several different products. One product’s dependencies may be incompatible with another’s, thus stopping the two products from being built on the same machine.
  • If the build agent has all necessary dependencies installed on it, it may not be obvious what actually needs to be installed on the production server. This happened at my workplace on one occasion – our application depended on ASP.NET MVC2 being globally installed, but this dependency wasn’t noticed because the library was present on the developer machines, the test environments and the build server. Result: panic and chaos on deployment.

But now I’ve added another reason to the list. Last Friday I was engaged in the task of setting up a build agent for a lot of products that have been built over the last three or so years, and that led me to the conclusion that:

  • It is very, very faffy to install old versions of products.

Here’s how I reached this conclusion. The first build I tried running on the new agent was a plain-ish application whose development had started about a year ago. It had been written in .NET 4.0 and consisted of an ASP.NET MVC3 website and a SQL Server 2008 database. The first run of the build failed, complaining about the lack of a Visual Studio 2010 file. I tutted the long-suffering tut of an Architect, but really I couldn’t claim to be surprised. The developers responsible for the suite of projects destined for the server had argued strongly that they couldn’t share the build agent with anyone else because of their very peculiar requirements, so I had every reason to suppose that the builds would be highly dependent on the environment. So I ran Visual Studio 2010 Ultimate Web Installer. I then installed Visual Studio 2010 SP1, which turned out to be a mistake, but I hope you’ll think it’s an understandable one. Visual Studio started – great! But the build still didn’t work, seemingly because of the absence of some MVC 3 files. It seemed that Visual Studio SP1 bundled ASP.NET MVC 4 but not 3. No problem – I would just install MVC 3.


I tried installing the general package available on the ASP.NET downloads page, and when that didn’t work, the targeted MVC3 download. Neither worked and the second one redirected me to a Microsoft Knowledge Base article which, rather than saying anything related to the problem in question, was a homepage for Visual Studio 2010 Express for Web, so I and installed that. This didn’t solve the problem. So I Googled and found a Microsoft support page which said that it was because Nuget 2.0 (part of VS2010 SP1) prevented the installation of MVC3. I went through a similarly frustrating process when trying to install SQL Server 2008 Express – I had to delete the %userprofile%\local settings\application data\Microsoft_Corporation because of some incompatibility with components released after SQL Server 2008.

By implication, all this faff also has to take place on the servers or workstations on which the software is destined to run in production, which means it’s multiplied by the number of machines it’s installed on. Granted, they will be managed by people greatly more adept at system administration than I. But I still conclude that a bit of up-front effort in at least making sure your library assemblies are under source control, your projects reference the local copy rather than the version in the GAC and you’ve specified ‘Copy Local’ for each reference will greatly ease your deployments.

Tasks which don’t take place in production, such as compilation, unit test running, code inspection and packaging, are a slightly different matter because the argument that avoiding dependencies on the build server saves time for the deployment teams is no longer valid. However, unless you’ve got a snazzy setup in which the configuration of all machines is automatically kept in sync with some template, time will still be saved in setting up and maintaining build agents and developer work stations.

One problem in this area is Microsoft’s lack of support for running build tools on vanilla machines. Want to package up your website for Azure on the build server? Microsoft’s advice is to install the Windows Azure Authoring Tools and the Windows Azure Tools. In fact, as the installers just put a set of DLLs and MSBuild XML files in your Program Files directory, it isn’t actually necessary for the tools to be installed on a machine to be able to use them. Judicious copying of dlls and tweaking of MSBuild files allows you to create the package without installing  anything. But this feels like a murky practice, a bit like converting a bedroom into a bathroom without removing all the floor-level electrical sockets. MSBuild XML files aren’t very nice to edit, and I think this is because MSBuild XML is a declarative language which, as part of the trend to automate more and more deployment, has started being used to do more and more procedural things. There’s a danger that it’ll become a write-only language.

So my architectural guidance is as follows:

  • There is no excuse for making any project references to DLLs in the GAC, except for the standard ones which come with the .NET Framework. Always store the DLLs in your source control system alongside your project, make your references point to the checked-out DLL, and set  ‘Copy Local’ to true.
  • If you’re Microsoft or anyone else publishing a tool which could be used on the build server, please let us know how we can run it from the command line using only copied files.
  • If you find yourself needing to edit MSBuild files in order to get build tools to work on vanilla servers, document the changes you make. Weigh up carefully the risk of having to repeat the process because you move onto a later version of the tool with the time and complexity saved by not having to install the tool on the build server.