The NET build process has been recently changed to use MSBuild tool instead of NAnt. In this document will be presented the workflow for building .NET components at TopCoder by making use of MSBuild and Visual Studion IDE facilities.

The master build files that are supposed to be used as templates in building .NET component can be found in the Master Build Files SVN Repository or in Master Build files archive if you can't access the SVN. The repository can evolve through time so please update your template build files when you get notified about this.

Overview

Generally the principles of the build process are:

Build scripts that require no component specific customization.
All component specific configuration is external to the build script.
Standard version rules are used for all components in a fashion that will prevent broken builds.

1. Setup the System Environment

In this step will be installed the major system components of .NET and required for building most .NET based products. Please note that there are many versions of .NET available. TopCoder certifies components for the environment specified in the component's Requirements Specification. If another environment is used then the component should be re-certified. MSBuild provides a lot of flexibility, these instructions provide an intended process to be followed, other processes may also yield working results.

1.1. Install .NET and MSBuild

Download and install the .NET framework that you need. As TopCoder doesn't support anymore .NET 1.x assemblies as target platform it is preferred to install one of the frameworks above .NET framework 2.0

Once you install one of these frameworks you can notice in subdirectories of C:\Windows\Microsoft.NET\Framework correspondent to the framework you have installed that MSBuild.exe is there too. The Microsoft Build Engine (MSBuild) is the new build platform for Microsoft Visual Studio and TopCoder .NET components. MSBuild is completely transparent with regards to how it processes and builds software, enabling developers to orchestrate and build products in build lab environments where Visual Studio is not installed. You can find a complete overview of this build tool here.

1.2. Install MSBuild Community Tasks

The MSBuild Community Tasks Project is an open source project for MSBuild tasks like zip,nunit, and other useful tasks. It can be downloaded from here.

2. Setup a TopCoder Build Environment

This step will install the required libraries that are standard to all TopCoder components. NUnit is used as a testing harness to run unit, accuracy, failure, and stress tests packaged with each component. NCover is used to provide a test coverage report for each TopCoder .NET component. FxCop is an application that analyzes managed code assemblies and reports information about the assemblies, such as possible design, localization, performance, and security improvements. The TopCoder Code Documenter is a command line utility that writes API documentation in a manner similar to JavaDoc for Java and NDoc for C#/.NET 1.0 which is designed as a replacement for nDoc, which is no longer maintained.
A standard properties file is used to provide environment reference configuration for the build scripts.

2.1. Install NUnit

Install one of the recent versions of NUnit.

2.2. Install NCover

Install the 1.5.8 version of NCover. This is a freeware version of the tool. The versions above this one are commercial.

2.3 Install FxCop

Install the 1.35 version of FxCop application from here. You can find an overview of tool here.

2.4 Install Code Documenter

Install the latest version of the TopCoder Code Documenter tool. This is a TopCoder tool so any feedback regarding it on the project's page would be appreciated.

2.5. Designate Source and Library Directories

TopCoder relies on a top level source directory which contains all the NET components as sub directories. The exact name of the .NET component directory is flexible. An example of this directory might be: c:\code\topcoder\NET-components\

TopCoder components build to a standard location specified by the build configuration, this ensures that the binary dll for a given component version is in a unique location on the build system. Additionally it ensures that components which depend on that binary will find it. And example of this directory might be: c:\code\lib\topcoder-net\

2.6 Setup the common .targets files

The common .targets files to be considered in the build process are :

Global.targets
Common.targets
Distribution.targets

The location for these files is specified in the .csproj files of each component. They are supposed to be placed in the parent directory (..) relative to the component's base directory which is to be built.

   |+ component_directory
         |+ conf
         |+ docs
         |+ log
         |+ src
         |+ test_files
         |- Build.dependencies
         |- Build.version
         |- Component.sln
         |- Component Sources.csproj
         |- Component Tests.csproj
         |- README.txt
   |- Global.targets
   |- Common.targets
   |- Distribution.targets

2.6.1. Setup TopCoder Global Build Properties

The build scripts depend on a common build properties file in addition to those which are packaged with the component. The Global.targets file contains the overall build properties for the specific integration box, including:

The TopCoder Library directory
- The TopCoder integration environment uses ext_bin as the parent for all third party libraries.
The environment install location of NCover,NUnit,Code Documenter and FxCop for all components.
The location of third party libraries (as needed for components on a case by case basis)

Standard build parameters

Notice bellow that the Global.targets file is a MSBuild compliant project file.

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <!-- Reference to NUnit -->
    <NUnit>C:\Program Files\NUnit 2.4.3\.net-2.0\bin\nunit.framework.dll</NUnit>
    <NUnitDir>C:\Program Files\NUnit 2.4.3\.net-2.0\bin</NUnitDir>
    <NUnitConsole>C:\Program Files\NUnit 2.4.3\.net-2.0\bin\nunit-console.exe</NUnitConsole>
    <NUnitSummaryTransform>C:\Program Files\NUnit 2.4.3\.net-2.0\doc\files\Summary.xslt</NUnitSummaryTransform>

    <!-- Reference to NCover -->
    <NCoverHome>C:\Program Files\NCover</NCoverHome>
    <NCoverConsole>$(NCoverHome)\NCover.Console.exe</NCoverConsole>
    <NCoverStyle>$(NCoverHome)\Coverage.xsl</NCoverStyle>

    <!-- Reference to TC Code Documenter-->
    <TCDocHome>c:\Code Documenter</TCDocHome>
    <TCDoc>$(TCDocHome)\CodeDocumenter.exe</TCDoc>

    <!-- Reference to FXCop -->
    <FxCopPath>C:\Program Files\Microsoft FxCop 1.35\</FxCopPath>
    <FxCopExecutable>FxCopCmd.exe</FxCopExecutable>
    <FxCopTransform>$(FxCopPath)\Xml\FxCopReport.xsl</FxCopTransform>

    <!-- Library repositories -->
    <ext_bin>..\tcs\bin\third_party</ext_bin>
    <tcs_bin>..\tcs\bin\tcs</tcs_bin>
    <hermes_bin>..\tcs\bin\hermes</hermes_bin>
  </PropertyGroup>
</Project>

2.6.2. Common.targets and Distribution.targets

These two files, as well as other template build files for .NET components, are to be retrieved from the Master Build Files SVN Repository.

The Common.targets file contains all the common targets that should be called when building a TopCoder component. Bellow will be presented the targets made available by this file:

Build
- compiles .cs sources into the /build directory
This first target presented here are not really defined in Common.targets file, but it is imported from $(MSBuildBinPath)\Microsoft.CSHARP.Targets file which is imported in the Common.targets file.
CleanSolution
- deletes all compiled code in /build and the content of /log directory.
Test (to be runned from .csproj for the tests of the component)
- runs test cases and outputs nunit results into /log, depends on 'Build'
- NCoverReport (to be runned from .csproj for the tests of the component)
  - generates test coverage reports for the component's tests into /log. This executable can be found in the NCover directory.
FxCop (to be runned from .csproj for the sources of the component)
- generates reports information about the assemblies, such as possible design, localization, performance, and security improvements.
Doc (to be runned from .csproj for the sources of the component)
- generates API documentation for the component in a javadoc manner.
Dist
- generates a binary distribution for deployment, depends on 'Build'

The Distribution.targets file contains distribution related targets and is suppossed to be used when packaging the design/development distribution or submission or when creating a catalog distribution for the component.
Bellow will be presendted the targets made available by this file:

DesignDist
- builds a .zip file which must contain the files needed in a design competition as starting point for designing the component.
DevDist
- builds a .zip file which must contain the files needed in a development competition as starting point for developing the component.
DesignSubmission
- builds a .zip file containing the design submission of the competitor
DevSubmission
- builds a .zip file containing the development submission of the competitor
DistTcs
- copies the assembly built from the sources of the component and also the distribution .zip file for the component, to the component's path $(target_bin) directory.
DeployLib
- copies the assembly built from the sources of the component to the component's path $(target_bin) directory (light version of DistTcs target).
PackageDocs
- builds an archive in the $(BuildDir) which contains the TC docs for the components along with the UML design images for the component.

3. Build the Component

From the command line the following ant targets are most useful:

msbuild "Component Sources.csproj" /t:Build - build the component source code
msbuild "Component Tests.csproj" /t:Build - build the component source&tests code
msbuild "Component Tests.csproj" /t:Test - runs the unit tests
msbuild "Component Tests.csproj" /t:NCoverReport - generates test coverage reports
msbuild "Component Sources.csproj" /t:Docs - generates the TC docs for the API of the component
msbuild "Component Sources.csproj" /t:FxCop - generates the FxCop report about the design problems in the assembly
msbuild "Component Sources.csproj" /t:DesignSubmission - build the .zip file containing the design submission of the competitor
msbuild "Component Sources.csproj" /t:DevSubmission - build the .zip file containing the development submission of the competitor

msbuild "Component Sources.csproj" /t:DistTcs - moves the runtime .dll as well as the .zip file containing the distribution of the component to the library directory.

The library directory component will be $(tcs_bin) for a generic component, but can also be the library directory for the client's components in case if a custom component is built.

Note that the target names to be executed by MSBuild are not case sensitive. You can also group several targets in one msbuild command by separating them witha semi-colon.
e.g. msbuild "Component Sources.csproj" /t:Build;Test;NCoverReport;DistTcs

In the chapters bellow is made a in-depth presentation of the NET component file structure and of the content of the build files which has a target audience the people who are supposed to make tweaks on the build files for their submissions.

If you are just using the distribution of the component and don't need to get into these details you can consider that you've finished the tutorial.

4. NET component file structure

The component to build should have following files and directories structure after the build is done :

README.txt - instructions for setting up this component for build
Component.sln - Visual Studio solution file which integrates the two projects for the component (source + sources&test cases). The name of this file will need to be modified to the name of the component.

Microsoft Visual Studio Solution File, Format Version 9.00
\# Visual Studio 2005
Project("\{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC\}") = "Component Sources", "Component Sources.csproj", "\{1E6A139B-BDCE-4EDB-9087-54B2C3037828\}"
EndProject
Project("\{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC\}") = "Component Tests", "Component Tests.csproj", "\{1C7B15F8-F1D0-4298-A4C4-42CD224DAF72\}"
EndProject
Global
	GlobalSection(SolutionConfigurationPlatforms) = preSolution
		Debug|Any CPU = Debug|Any CPU
	EndGlobalSection
	GlobalSection(ProjectConfigurationPlatforms) = postSolution
		\{1E6A139B-BDCE-4EDB-9087-54B2C3037828\}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
		\{1E6A139B-BDCE-4EDB-9087-54B2C3037828\}.Debug|Any CPU.Build.0 = Debug|Any CPU
		\{1C7B15F8-F1D0-4298-A4C4-42CD224DAF72\}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
		\{1C7B15F8-F1D0-4298-A4C4-42CD224DAF72\}.Debug|Any CPU.Build.0 = Debug|Any CPU
	EndGlobalSection
	GlobalSection(SolutionProperties) = preSolution
		HideSolutionNode = FALSE
	EndGlobalSection
EndGlobal

Microsoft Visual Studio Solution File, Format Version 10.00
\# Visual Studio 2008
Project("\{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC\}") = "Component Name Sources", "Component Sources.csproj", "\{1E6A139B-BDCE-4EDB-9087-54B2C3037828\}"
EndProject
Project("\{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC\}") = "Component Name Tests", "Component Tests.csproj", "\{1C7B15F8-F1D0-4298-A4C4-42CD224DAF72\}"
EndProject
Global
	GlobalSection(SolutionConfigurationPlatforms) = preSolution
		Debug|Any CPU = Debug|Any CPU
		Release|Any CPU = Release|Any CPU
	EndGlobalSection
	GlobalSection(ProjectConfigurationPlatforms) = postSolution
		\{1E6A139B-BDCE-4EDB-9087-54B2C3037828\}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
		\{1E6A139B-BDCE-4EDB-9087-54B2C3037828\}.Debug|Any CPU.Build.0 = Debug|Any CPU
		\{1E6A139B-BDCE-4EDB-9087-54B2C3037828\}.Release|Any CPU.ActiveCfg = Debug|Any CPU
		\{1E6A139B-BDCE-4EDB-9087-54B2C3037828\}.Release|Any CPU.Build.0 = Debug|Any CPU
		\{1C7B15F8-F1D0-4298-A4C4-42CD224DAF72\}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
		\{1C7B15F8-F1D0-4298-A4C4-42CD224DAF72\}.Debug|Any CPU.Build.0 = Debug|Any CPU
		\{1C7B15F8-F1D0-4298-A4C4-42CD224DAF72\}.Release|Any CPU.ActiveCfg = Debug|Any CPU
		\{1C7B15F8-F1D0-4298-A4C4-42CD224DAF72\}.Release|Any CPU.Build.0 = Debug|Any CPU
	EndGlobalSection
	GlobalSection(SolutionProperties) = preSolution
		HideSolutionNode = FALSE
	EndGlobalSection
EndGlobal

Component Sources.csproj - Visual Studio project file which can be used for building in a transparent way the component either by using VS2005 or VS2008 or an msbuild call from the command prompt and which is supposed to integrate the sources & documents for the component, along with the referenced assemblies (and in some cases embedded resources for the component).
Component Tests.csproj - Visual Studio project file which can be used for building in a transparent way the component either by using VS/2005 or VS2008 or an msbuild call from the command prompt and which is supposed to integrate the sources & test cases & test files and documents, along with the referenced assemblies for compiling/testing the component (and in some cases embedded resources for the component).
Please note that the content of the files :

Component Sources.csproj

Component Tests.csproj
are not supposed to be modified as they don't require component specific modification.
Build.version - specific component version information, including version number and component name
- Component versions are specified as major.minor.micro.build - the public version is major.minor.micro and is used by components to find their dependencies. The full version is used for internal management and non-software changes, such as updating documentation.Whenever a modification is done in svn for a specific public version for a component (major.minor.micro) the value for component.version.build is incremented.
  Let's say we're having a specific component A with version to be retrieved from the catalog - 1.0.1 . If there are small changes made on the component which don't need a public version incremented (like 1.1.3) released for the component, the only thing which changes regarding the component version will be the value for component.version.build property which will be always incremented.
  So component A could have version numbers like these ones: 1.0.0.1 ; 1.0.1.1 ; 1.0.1.2 ; 1.0.1.3 ; 1.1.0.1 .

Template Build.version file

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
    <!-- Component Name and Version Information -->
    <ComponentName>Component Name</ComponentName>
    <ComponentDistFileName>component_name</ComponentDistFileName>
    <ComponentPackage>TopCoder.Package</ComponentPackage>
    <ComponentPackageDir>TopCoder\Package</ComponentPackageDir>
    <ComponentVersionMajor>1</ComponentVersionMajor>
    <ComponentVersionMinor>0</ComponentVersionMinor>
    <ComponentVersionMicro>0</ComponentVersionMicro>
    <ComponentVersionBuild>1</ComponentVersionBuild>
  </PropertyGroup>
</Project>

Build.dependencies - specific component information regarding the sources/embedded resources to be added for compilation, referenced assemblies in the sources/test project.The new build process aims to minimize the modifications needed when customizing the build files for a component. This is a reason why in Build.dependencies file should be modified only the part between <Choose> tags.

Custom.targets - this is an optional file and it will be used ONLY in case of custom testing scenarios or when the component neeeds some exta targets for building. Basically in this file can be redefined existing targets and also can be added new targets in case they are needed.

Custom.targets file for ASP.NET Localization component

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

  <!--
     The TestSetup target is redefined (because it has been declared in Common.targets file)
     in order to copy the output assemblies to the test_files\accuracy\WebTests\Accuracy1\bin\ and
     test_files\accuracy\WebTests\Accuracy1\bin\ directories before running the tests on the component.
    -->
  <Target Name="TestSetup"
          DependsOnTargets="Build">

    <Copy SourceFiles="$(BuildClassDir)\$(ComponentPackage).dll" DestinationFolder="$(TestFilesDir)\Accuracy\WebTests\Accuracy1\bin" />
    <Copy SourceFiles="$(BuildClassDir)\$(ComponentPackage).dll" DestinationFolder="$(TestFilesDir)\Accuracy\WebTests\Accuracy2\bin" />
  </Target>

</Project>

conf - configuration files for the component.

docs - contains the documentation for the component including Requirements Specification, Component Specification, a .tcuml/.zuml file containing UML design of the component and .gif images generated for the UML design.

Typically this directory needs to include a fileset of documents having this format :

<fileset dir="${docsdir}">
    <include name="${distfilename}_Class_Diagram*"/>
    <include name="${distfilename}_Use_Case_Diagram*"/>
    <include name="${distfilename}_Sequence_Diagram*"/>
    <include name="${distfilename}_Requirements_Specification.pdf"/>
    <include name="${distfilename}_Requirements_Specification.rtf"/>
    <include name="${distfilename}_Component_Specification.pdf"/>
    <include name="${distfilename}_Component_Specification.rtf"/>
    <include name="${distfilename}.tcuml"/>
</fileset>

Note that ${distfilename} represents the component distribution file name.
The documents belonging, if it is the case, to a previous version of the component need to be removed from docs/ directory when adding in svn the new version. Be sure to have the files from docs/ directory matching EXACTLY the format specified earlier.
To help out what I'm trying to say here I will add an image on how the structure for docs/ directory should look like.

src - the directory containing source files of the component
- In src/main directory will be found an AssemblyInfo.cs file. This file contains informations about the component including :title, description, version, etc. Be sure to update the version of the assembly when a rebuild operation is made and also if you are handling a generic component (belonging to the TopCoder Catalog and not to a client) to uncomment the line :
```
//[assembly: AssemblyKeyFile(@"..\tcs\bin\TopCoder.snk")]
```
test_files - file resources needed for testing
- Please note that runtime .dll files that can be found in this directory are for the purposes of compiling and running the test code, not for any other use, including production. Component's that are required for the production code must be downloaded from TopCoder Software Catalog.
  In these cases please add the content of the test_files directory to the Component Tests project in Build.dependencies file in this manner:
```
 <Content Include="$(TestFilesDir)\**\*" Exclude="$(TestFilesDir)\**\.svn\**;$(TestFilesDir)\**\*.dll">
          <Visible>true</Visible>
 </Content>
```

4.1. Build.dependencies file presentation

Beside Build.version, this is the only file that is supposed to be modified in the build process for a component.
Bellow will be presented the content of a common dependencies file.

Build.dependencies file for Configuration API component

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <!--Directory structure of the component.-->
    <BuildDir>build</BuildDir>
    <BuildClassDir>$(BuildDir)\classes</BuildClassDir>
    <ConfigDir>conf</ConfigDir>
    <DocsDir>docs</DocsDir>
    <SrcDir>src</SrcDir>
    <TestFilesDir>test_files</TestFilesDir>
    <TestLogDir>log</TestLogDir>

    <!--Src directory structure.-->
    <DotNetSrc>$(SrcDir)\csharp</DotNetSrc>
    <DotNetMain>$(DotNetSrc)\main</DotNetMain>
    <DotNetTests>$(DotNetSrc)\tests</DotNetTests>
    <OutputPath>$(BuildClassDir)</OutputPath>

    <!-- Properties related to assembly building. -->
    <OutputType>Library</OutputType>
    <TargetFrameworkVersion>v2.0</TargetFrameworkVersion>
  </PropertyGroup>
  <ItemGroup>
    <!-- Content present in Component Source & Component Tests project -->
    <Content Include="$(ConfigDir)\**\*" Exclude="$(ConfigDir)\**\.svn\**">
      <Visible>true</Visible>
    </Content>
    <Content Include="$(DocsDir)\**\*" Exclude="$(DocsDir)\**\.svn\**">
      <Visible>true</Visible>
    </Content>
    <Content Include="README.txt">
      <Visible>true</Visible>
    </Content>
    <Folder Include="$(ConfigDir)\">
      <Visible>true</Visible>
    </Folder>
  </ItemGroup>
  <!-- This item group is dedicated for declaring the libraries referenced by both Component Sources
       and Component Tests projects.
   -->
  <ItemGroup>
    <Reference Include="System" />
    <Reference Include="System.Xml" />

    <!--TopCoder Library References.-->
    <Reference Include="TopCoder.Util.Collection.Set">
      <SpecificVersion>False</SpecificVersion>
      <HintPath>$(tcs_bin)\set_utility\1.0.0\TopCoder.Util.Collection.Set.dll</HintPath>
    </Reference>

  </ItemGroup>
  <Choose>
    <When Condition="$(ProjectKind)=='ComponentSources'">
      <!--Component Sources and referenced assemblies-->
      <PropertyGroup>
        <AssemblyName>$(ComponentPackage)</AssemblyName>
        <RootNamespace>$(ComponentPackage)</RootNamespace>
      </PropertyGroup>
      <ItemGroup>
        <!-- The Component Sources project contain only the .cs files from $(DotNetMain) directory. -->
        <Compile Include="$(DotNetMain)\**\*.cs">
          <Visible>true</Visible>
        </Compile>
      </ItemGroup>
    </When>
    <When Condition="$(ProjectKind)=='ComponentTests'">
      <!--Component test sources and referenced assemblies-->
      <PropertyGroup>
        <AssemblyName>$(ComponentPackage).Test</AssemblyName>
        <RootNamespace>$(ComponentPackage)</RootNamespace>
      </PropertyGroup>
      <ItemGroup>
        <Compile Include="$(DotNetSrc)\**\*.cs">
          <Visible>true</Visible>
        </Compile>

        <!-- Content specific for the Component Tests project. -->
        <Content Include="$(TestFilesDir)\**\*" Exclude="$(TestFilesDir)\**\.svn\**">
          <Visible>true</Visible>
        </Content>
        <Folder Include="$(TestFilesDir)\">
          <Visible>true</Visible>
        </Folder>
        <Content Include="$(TestLogDir)\**\*">
          <Visible>true</Visible>
        </Content>
        <Folder Include="$(TestLogDir)\">
          <Visible>true</Visible>
        </Folder>
      </ItemGroup>
      <ItemGroup>
        <!-- Libraries referenced only by Component Tests project. -->
        <Reference Include="nunit.framework">
          <SpecificVersion>False</SpecificVersion>
          <HintPath>$(NUnit)</HintPath>
        </Reference>
      </ItemGroup>
    </When>
  </Choose>
</Project>

Next will be described the way in which libraries referenced by a component should be added in the Build.dependencies file.

The assemblies belonging to GAC are referenced simply :

    <Reference Include="System" />
    <Reference Include="System.Xml" />

The TopCoder libraries are referenced in the following manner :

    <Reference Include="TopCoder.Assembly.Name">
      <SpecificVersion>False</SpecificVersion>
      <HintPath>$(tcs_bin)\component_name\component_version\TopCoder.Assembly.Name.dll</HintPath>
    </Reference>

The third party libraries are referenced in teh following manner :

    <Reference Include="Third.Party.Assembly.Name">
      <SpecificVersion>False</SpecificVersion>
      <HintPath>$(ext_bin)\component_name\component_version\Third.Party.Assembly.Name.dll</HintPath>
    </Reference>

Some of the components present in the TopCoder .NET component catalog integrate into the assembly resulted from compiling the sources, resource files. Embedding resource files is pretty straightforward and can be done by specifying the resource files in Build.dependencies file.

- Specifying an .xsd resource file with logical name composed of $(ComponentPackage) and its filename:

- Specifying an .xsd resource file with the same logical name as its filename:

You can also encounter assemblies which need a configuration file for Component Tests assembly. The solution for specifying this configuration file is presented in the image bellow:

Another thing to take into account is that for some newer components (with target versions like NETFX 3.0 or NETFX 3.5) is that in the Build.dependencies you will have to manually modify the TargetFrameworkVersion property with appropriate values like v3.0 or v3.5

Another note about the changes you can make in Build.dependencies file is about the OutputType library (see the OutputType element in the picture above). This property can have one of the following values :

Library (default value)
Exe
WinExe

Normally you are supposed to change this property only if you want a different kind of output from the Build target.

4.2. Custom testing scenarios

4.2.1. Copying configuration files to specific locations before testing

If some assemblies need more configuration files copied int the place where the compile output assembly is stored you can use a tweak like the one from ASP.NET Localization component (redefine TestSetup target in Custom.targets file).

Custom.targets file for ASP.NET Localization component

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

  <Target Name="TestSetup"
          DependsOnTargets="Build">

    <Copy SourceFiles="$(BuildClassDir)\$(ComponentPackage).dll" DestinationFolder="$(TestFilesDir)\Accuracy\WebTests\Accuracy1\bin" />
    <Copy SourceFiles="$(BuildClassDir)\$(ComponentPackage).dll" DestinationFolder="$(TestFilesDir)\Accuracy\WebTests\Accuracy2\bin" />
  </Target>

</Project>

Custom.targets file for Entity-Attribute-Value Manager component

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

  <ItemGroup>
     <MappingFiles Include="$(TestFilesDir)\mappings\*.hbm.xml" Exclude="$(TestFilesDir)\mappings\**\.svn\**"/>
  </ItemGroup>

  <Target Name="TestSetup"
          DependsOnTargets="Build">
    <!-- Copying the mapping files for hibernate to $(BuildClassDir). -->
    <Copy SourceFiles="@(MappingFiles)"
          DestinationFiles="@(MappingFiles->'$(BuildClassDir)\%(Filename)%(Extension)')" />
  </Target>
</Project>

4.2.2. Performing initialization/cleanup operations when testing against a database

One of the most common types of components are the ones which are interacting with databases.Here will be detailed how to handle this kind of scenario. As an example will be used the component Data Access Interface which can be found in TopCoder SVN repository here.
Bellow you can see the content of the Custom.targets file for this component:

Custom.targets file for Data Access Interface component

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

  <PropertyGroup>
      <ConnectionStringSqlServer>Initial Catalog=tempdb;Data Source=(local);Uid=sa;Pwd=topcoder;</ConnectionStringSqlServer>
  </PropertyGroup>

  <Target Name="TestSetup"
          DependsOnTargets="Build">          
      <ExecuteDDL ConnectionString="$(ConnectionStringSqlServer)" Files="test_files\ddl_for_sqlServer.sql" ContinueOnError="false" />
      
      <Exec Command="sqlplus system/topcoder @&quot;$(MSBuildProjectDirectory)\test_files\ddl_for_oracle.sql&quot;"/>
  </Target>

  <Target Name="TestTearDown"
          DependsOnTargets="Build">          
      <ExecuteDDL ConnectionString="$(ConnectionStringSqlServer)" Files="test_files\sql_server_teardown.sql" ContinueOnError="false" />
      
      <Exec Command="sqlplus system/topcoder @&quot;$(MSBuildProjectDirectory)\test_files\oracle_teardown.sql&quot;"/> 
  </Target>

</Project>

As you see above, there are multiple files referenced in the TestSetup and TestTeardown targets.For adding you comprehend better what is done for initializing/cleaning up the database environment for this component will be presented a part of the content of these files.

ddl_for_sqlServer.sql

IF EXISTS (SELECT name FROM master.dbo.sysdatabases WHERE name = N'TCNET-8595405')
	drop database [TCNET-8595405]
GO

create database [TCNET-8595405]
GO

use [TCNET-8595405]
GO

-- Content of ddl_for_sqlserver.sql
-- ...........

ddl_for_oracle.sql

create user TCNET8595405 identified by TCNET8595405 default tablespace
  users temporary tablespace temp;

grant dba to TCNET8595405;

alter user TCNET8595405 quota unlimited on users;

connect TCNET8595405/TCNET8595405

-- Content of ddl_for_oracle.sql
-- .......

sql_server_teardown.sql

IF EXISTS (SELECT name FROM master.dbo.sysdatabases WHERE name = N'TCNET-8595405')
	drop database [TCNET-8595405];
GO

oracle_teardown.sql

drop user TCNET8595405 cascade;

So to summarize when encountering components which need db initialization/cleanup you must :

create the .sql scripts for initialization/cleanup of the database. Note that the database name is not a common name, it is the build key (TCNET-8595405 or TCNET8595405) of the Bamboo plan for the component(this way we ensure that it is unique and doesn't collide with the databases created by some other components). If the component requires more databases for testing, use the build key as a prefix for the name of the databases.
add a Custom.targets file in which should be overriden TestSetup and TestTeardown targets. In TestSetup you must run the scripts used for initializing the database schema & its tables. In TestTeardown target you have to drop the database created in TestSetup.

Note that for performing db operations on MS Sql Server db engine you can use ExecuteDDL msbuild community task. For performing db operations on Oracle db engine you can use sqlplus utility which comes with Oracle installation.

4.3. Write down the detailed instructions on how the testing environment should be setup

The README.txt file (in the root directory of the component) needs to contain the steps to run through the tests. This file should be added/updated in svn.
Here let's take impact_report_manager component for example:

README.txt file

Installation Requirements
  -------------------------
  Microsoft .NET Framework 2.0  <http://msdn.microsoft.com/netframework/productinfo>
  MSBuild Community Tasks 1.3 <http://msbuildtasks.tigris.org/>
  NUnit 2.4  <http://sourceforge.net/projects/nunit>
  NCover 1.5.8 <http://www.ncover.com/download/discontinued>
  FxCop 1.35 <http://code.msdn.microsoft.com/Release/ProjectReleases.aspx?ProjectName=codeanalysis&ReleaseId=553>
  Code Documenter 1.0.1 (http://www.topcoder.com/wiki/display/projects/Code+Documenter/)


  TopCoder Software Environment Configuration
  -------------------------------------------
  TopCoder Software has defined a directory structure to promote component reuse
  and facilitate Component Based Development.

  Steps to setting up your environment:
  1- Designate a directory on your file system to be used as your TCS_HOME.
     (i.e. c:\tcs or /etc/home/user/tcs)
  2- All TopCoder Software components are distributed with MSBuild
     based build scripts and NUnit (http://nunit.org) based test cases.
     Please properly install and configure these tools before working with TopCoder
     Software components.

  Component directory Structure
  ------------------------------------------
  The directory layout for each component is set up as follows:
  /build               - binary related files required to run the components
  /conf                - configuration data required by the component
  /docs                - component documentation
  /docs/coverage       - test coverage reports in html format
  /log                 - log files generated from test suite execution
  /src                 - source code for the component
  /test_files          - source code for the component test cases


  TopCoder Software MSBuild Targets
  ------------------------------------------
  Each component comes with 2 .csproj (one including only the sources and one including the sources&tests  build scripts of the component) with the following targets:
  Build
       - compiles .cs sources into the /build directory
  Test (to be runned from .csproj for the tests of the component)
       - runs test cases and outputs nunit results into /log,
         depends on 'Build'
  NCoverReport (to be runned from .csproj for the tests of the component)
       - generates test coverage reports for the component's tests into /log.
         This executable can be found in the NCover directory.
  FxCop (to be runned from .csproj for the sources of the component)
       - generates reports information about the assemblies, such as possible design,
         localization, performance, and security improvements.
  Doc (to be runned from .csproj for the sources of the component)
       - generates API documentation for the component in a javadoc manner.
  Dist
       - generates a binary distribution for deployment, depends on 'Build'
  CleanSolution
       - deletes all compiled code in /build and the content of /log directory.

  A target can be executed in the following manner:
       msbuild "Component Sources.csproj" /t:TARGET_NAME
  or
       msbuild "Component Tests.csproj" /t:TARGET_NAME
  where TARGET_NAME is one of the targets enumerated earlier.


  Steps to run for setting up testing environment for the component:
  ------------------------------------------

      1. Initialize your Oracle and SqlServer services.
      2. Execute "test_files\Drop_Tables.sql" to clean up the database environment, you can also use client tools like "PLSQL Developer" to do this job.
      3. Execute the following sql scripts to set up the databases.
           - test_files\Impact_Report_Manager_DDL_Updated.sql
           - test_files\Init_Tables.sql
           - test_files\ID_Generator_for_Oracle.sql
      4. Update the following configuration files, you need to modify the connection string for databases, like "User Id=system;Password=dwb;Data Source=ORCL;".
           - test_files\impact_report_manager.xml
           - test_files\invalid_configuration.xml
           - test_files\AccuracyTests\config.xml
           - test_files\Failure\impact_report_manager.xml
           - test_files\Stress\common.xml

  ---------------------------------------------------------------------------------------------------------------
  Thanks for using TopCoder Software components!

  The TopCoder Software Team
  service@topcodersoftware.com

4.4. Configure the build environment.

You should read the component specification first, make sure in what environment the component will be run. It may include the following issues:
Platform - Windows 2003, Windows XP, Windows Vista, etc
Compiling Target (TargetFrameworkVersion property in Build.dependencies file) - Microsoft .NET runtime environment 2.0/3.0/3.5, etc
Dependencies - Including TopCoder or third party components, required versions of these components should be used.

Especially for custom components, for example if the tests compiled successfully under Oracle 10.2, but failed under Oracle 10.1, and the required version is the latter, you should report it to PM. It is very important to respect the constraints imposed by the Component Specification regarding the Environment Requirements.

4.5. External libraries directory structure.

TopCoder is not allowed to re-distribute the 3rd party libraries on which some of the components are dependent (like junit for example). This is the reason why attached to document you will find here a listing of TopCoder's ${ext_libdir} directory. Please structure your ${ext_libdir} directory in the same way as TopCoder has structured these libraries to avoid having problems when integrating the component on the Bamboo server.Note that there will be other libraries added, when needed, to the external libraries directory on the Bamboo machine, so please check out from time to time if anything has updated on the 3rd party directory listing.

5. Commit the modifications made on the component after the build is done to SVN

Please make sure that every change made to the component (from file additions to modifications or remove of useless files) to appear in the Commit dialog.
You should log the modifications you made when updating the svn, including trivial things like "Useless files are removed". Normally when the building of a component is done you should put a comment like "Build x.x.x.x successful" where x.x.x.x is the version for the component.

At the end of this process, the SVN file structure of the component to be built will look like this :

Be sure always to check if the file structure is similar to what can be see in the image presented earlier.
The file Custom.targets can be present in this file structure when, as it was previously stated, a custom test scenario is made or custom targets are needed in the build script.
In the case one or more of the conf/, test_files/ directories are empty you don't need to add them to svn as they are useless in this case for the component.

When you encounter a problem in building a component because of a missing third-party library, please contact the component manager and ivern in order to have the problem solved.

Please feel free to comment any of the details presented in this tutorial if you feel that it can be polished to be easier to understand or if there is more information needed.