Hudson and TFS2008 For Continuous Integration

At work we’re taking some time to get a proper CI environment up and running. Since we also have a Java product, Hudson was already set up and being used. I had used Hudson in the past for doing continuous integration of IronRuby under Mono on Linux. Getting our build up and running was pretty much a breeze thanks to the TFS, Gallio, MSBuild, and NCover plugins. Once that was set up and running, it was easy enough to subscribe to the CheckinEvent on TFS and have it ping a defined URL on Hudson to kick off the build. Yay! Proper Continuous Integration!

But then something unexpected started to happen. At two hour intervals, the build would kick off, even if not check in had occurred! Some digging into the Windows Event Log, lead to what seemed to be the culprit. Every two hours TFS syncs users and groups with Active Directory. For some reason this raises a CheckinEvent. So the cause was discovered, now how to stop it? A quick searching on [SearchEngine] revealed this tidbit:

you can now take advantage of the ability to modify the periodic AD/GSS sync period. To do so, you need to manually modify a Web.config file on the Application Tier that by deafult will appear under the folder "%PROGRAMFILES%\Microsoft Visual Studio 2008 Team Foundation Server\Web Services\services". DISCLAIMER: I do not guarrantee that this will not cause your computer to eat all of the data it has ever come into contact with, possible with fire. This confers no warranties and no rights, your mileage may vary, use at your own risk, void where prohibited, etc. [Note: still my blog]. You’ll need to add the following two lines to the key/value pair section at the top of this file, followed by resetting IIS to make sure our application pool picks up the changes:

<add key="IdentityUpdatePeriod" value="01:00:00" />
<add key="IdentityUpdateInitial" value="01:00:00" />

Note that I’ve set both the initial update (i.e. the delay after startup that the first periodic sync happens) and the delay between periodic syncs to 1 hour. You can increase this to multiple hours, or decrease it to a matter of minutes.

So after making those changes, I noticed that the Events were no longer being logged every two hours, but the CheckinEvent was still getting raised at that interval! What could be going on? Well after some (read: A LOT) of [SearchEngine]ing, it turns out, TFS will automatically try resending "failed" alerts every two hours for ~12 hours. Since the CheckinEvent subscription is expecting a SOAP webservice, and not just a random url, it thinks that the notification "failed" when it doesn’t get the response it expects. Causing it to try resending to the same url over, and over, and over again at two hour intervals.

So what we need is a very very simple web service to take the SOAP message then turn around and ping a url. This way TFS gets the response it expects and acknowledges that the notification was successfully sent.

All you have to do is implement the ‘Notify’ method that TFS expects to be able to call:

        [SoapDocumentMethod("http://schemas.microsoft.com/TeamFoundation/2005/06/Services/Notification/03/Notify",
        RequestNamespace = "http://schemas.microsoft.com/TeamFoundation/2005/06/Services/Notification/03")]
        [WebMethod]
        public void Notify(string eventXml)
        {
                try
                {
                    System.Net.WebRequest req = System.Net.HttpWebRequest.Create("http://www.example.org/hudson/job/AutomatedBuild/build?token=TOKEN");
                    req.GetResponse();
                }
                catch
                {
                }
        }

I have put together a simple service that will do this for multiple URLs defined in the Web.config. Feel free to get it here at GitHub and save yourself the frustration I encountered.

MbUnit, NCover, and TeamBuild Living Together, Mass Hysteria!

 

Recently I’ve been asked to make a case for using something other then MSTest, one of the arguments against using an xUnit framework was the lack of integration to Team Foundation Server.  Well, I took it upon myself to show that MbUnit and NCover could perform just as well in a TeamBuild environment.  Here’s how to get it all working.

First you’ll want to get MbUnit, NCover (1.5.8), NCoverExplorer (1.4.0.7) along with it’s Extras (1.4.0.5),  Note that NCover and NCoverExplorer are no longer free, but their past version were and are still available free of charge, just without official support.  They work quite well though!

Next you’ll need to edit the proj file for your unit tests, near the top of the file, enter some UsingTask directives for NCoverExplorer:

<UsingTask TaskName="NCoverExplorer.MSBuildTasks.NCoverExplorer" AssemblyFile="..\libs\NCoverExplorer.MSBuildTasks.dll"/>
<UsingTask TaskName="NCoverExplorer.MSBuildTasks.NCover" AssemblyFile="..\libs\NCoverExplorer.MSBuildTasks.dll"/>
<UsingTask TaskName="NCoverExplorer.MSBuildTasks.NUnitProject" AssemblyFile="..\libs\NCoverExplorer.MSBuildTasks.dll"/>

As you can see, I’m using a relative path to the MSBuildTask assemblies, you can put in whatever path you need, but there are good arguments for keeping your testing frameworks in the same project directory structure as your unit tests.

Near the bottom uncomment out the <Target> section with the name of "AfterBuild", this is were we’re going to tell our build system to run our tests and coverage analysis.

Remember we’re going to be doing TeamBuild, but we’d still like it to work outside that environment for local use.  So let’s add a couple of property groups that set up some variables for us depending on our build environment:

<PropertyGroup>
	<CoveragePath  Condition="'$(DropLocation)' != ''">$(DropLocation)\$(BuildNumber)\$(Configuration)\CodeCoverage</CoveragePath>
	<CoveragePath  Condition="'$(DropLocation)' == ''">$(TargetDir)\CodeCoverage</CoveragePath>
	<TestPath Condition="'$(DropLocation)' != ''">$(DropLocation)\$(BuildNumber)\$(Configuration)\TestResults</TestPath>
	<TestPath Condition="'$(DropLocation)' == ''">$(TargetDir)\TestResults</TestPath>
</PropertyGroup>

<PropertyGroup>
	<CoverageFile>$(CoveragePath)\Coverage.xml</CoverageFile>
	<TestReport>TestResults{0}{1}</TestReport>
</PropertyGroup> 

 

As you can see here we check to see if DropLocation is set and if it is or isn’t determines where we’re going to place our test and coverage reports. Next we’ll simply add some tasks to setup directories and actually run NCover and NCover explorer:

<MakeDir Condition="!Exists('$(CoveragePath)')" Directories="$(CoveragePath)" />
<MakeDir Condition="!Exists('$(TestPath)')" Directories="$(TestPath)" />

<NCover CoverageFile="$(CoverageFile)" CommandLineExe="C:\Program Files\MbUnit\mbunit.cons.exe" CommandLineArgs='"$(TargetPath)" /rf:"$(TestPath)" /rt:HTML /rnf:"$(TestReport)"'  />

<NCoverExplorer
	ToolPath="C:\Program Files\NCoverExplorer"
	OutputDir="$(CoveragePath)"
	XmlReportName="CoverageSummary.xml"
	HtmlReportName="CoverageSummary.html"
	CoverageFiles="$(CoverageFile)"
	ShowExcluded="False"
	MinimumCoverage="75"
	SatisfactoryCoverage="80"
	Exclusions="Assembly=.*Tests.*"
/>

And that’s it really, when you build via a TeamBuild, local MSBuild, or even from Visual Studio, you’ll have two extra directories in your output location, CodeCoverage and TestResults, in there you’ll find nicely formatted HTML reports informing you of tests and code coverage.  If any tests fail then the build will fail, if code coverage doesn’t meet the minimum your build will fail.  NCoverExplorer’s build task has an option called "FailMinimum", if you set this to false then the build will succeed even if your code coverage doesn’t meet the "minimum".  With TFS 2008′s "build on check-in" feature, you start to see possibilities here.  With the NCover and MbUnit reports having xml equivalents the reporting/analytic possibilities are endless.