 |
The NET build process has been recently changed to use MSBuild tool instead of NAnt. In this document will be presented the workflow for building .NET components at TopCoder by making use of MSBuild and Visual Studion IDE facilities. |
 |
The master build files that are supposed to be used as templates in building .NET component in VS 2008 format can be found in the VS_2008_Format Master Build Files SVN Repository . The repository can evolve through time so please update your template build files when you get notified about this. |
Overview
Generally the principles of the build process are:
- Build scripts that require minimum component specific customization.
- All component specific configuration is external to the build script.
- Standard version rules are used for all components in a fashion that will prevent broken builds.
1. Setup the System Environment
In this step will be installed the major system components of .NET and required for building most .NET based products. Please note that there are many versions of .NET available. TopCoder certifies components for the environment specified in the component's Requirements Specification. If another environment is used then the component should be re-certified. MSBuild provides a lot of flexibility, these instructions provide an intended process to be followed, other processes may also yield working results.
1.1. Install .NET and MSBuild
- Download
and install the .NET framework that you need. As TopCoder doesn't support anymore .NET 1.x assemblies as target platform it is preferred to install one of the frameworks above .NET framework 2.0
Once you install one of these frameworks you can notice in subdirectories of C:\Windows\Microsoft.NET\Framework correspondent to the framework you have installed that MSBuild.exe is there too. The Microsoft Build Engine (MSBuild) is the new build platform for Microsoft Visual Studio and TopCoder .NET components. MSBuild is completely transparent with regards to how it processes and builds software, enabling developers to orchestrate and build products in build lab environments where Visual Studio is not installed. You can find a complete overview of this build tool here
.
1.2. Install MSBuild Community Tasks
The MSBuild Community Tasks Project is an open source project for MSBuild tasks like zip,nunit, and other useful tasks. It can be downloaded from here
.
2. Setup a TopCoder Build Environment
This step will install the required libraries that are standard to all TopCoder components. NUnit is used as a testing harness to run unit, accuracy, failure, and stress tests packaged with each component. NCover is used to provide a test coverage report for each TopCoder .NET component. FxCop is an application that analyzes managed code assemblies and reports information about the assemblies, such as possible design, localization, performance, and security improvements. The TopCoder Code Documenter is a command line utility that writes API documentation in a manner similar to JavaDoc for Java and NDoc for C#/.NET 1.0 which is designed as a replacement for nDoc, which is no longer maintained.
A standard properties file is used to provide environment reference configuration for the build scripts.
2.1. Install NUnit
Install one of the recent versions of NUnit
.
2.2. Install NCover
Install the 1.5.8 version of NCover
. This is a freeware version of the tool. The versions above this one are commercial.
2.3 Install FxCop
Install the 1.35 version of FxCop application from here
. You can find an overview of tool here
.
2.4 Install Code Documenter
Install the latest version of the TopCoder Code Documenter tool
. This is a TopCoder tool so any feedback regarding it on the project's page would be appreciated.
2.5. Designate Source and Library Directories
TopCoder relies on a top level source directory which contains all the NET components as sub directories. The exact name of the .NET component directory is flexible. An example of this directory might be: c:\code\topcoder\NET-components\
TopCoder components build to a standard location specified by the build configuration, this ensures that the binary dll for a given component version is in a unique location on the build system. Additionally it ensures that components which depend on that binary will find it. And example of this directory might be: c:\code\lib\topcoder-net\
2.6 Setup the common targets files
The common .targets files to be considered in the build process are :
- Global.targets
- Common.targets
- Distribution.targets
- Component Name.proj
The location for these files is specified in the .csproj files of each component. Global.targets file is supposed to be placed in the parent directory (..) relative to the component's base directory which is to be built and the files Common.targets, as well as Distribution.targets and Component Name.proj (which should be renamed to match the component name -same as the .sln file), will be placed in the basedir of the component.
2.6.1. Setup TopCoder Global Build Properties
The build scripts depend on a common build properties file in addition to those which are packaged with the component. The Global.targets file contains the overall build properties for the specific integration box, including:
- The TopCoder Library directory
- The TopCoder integration environment uses ext_bin as the parent for all third party libraries.
- The environment install location of NCover,NUnit,Code Documenter and FxCop for all components.
- The location of third party libraries (as needed for components on a case by case basis)
- Standard build parameters
 | Notice bellow that the Global.targets file is a MSBuild compliant project file. |
2.6.2. Common.targets file
This file, as well as other template build files for .NET components, are to be retrieved from the VS_2008_Format Master Build Files SVN Repository
.
The Common.targets file contains all the common targets that should be called when building a project (a .csproj file) from the component solution.
Bellow will be presented the targets made available by this file:
- Build
- compiles .cs sources into the ../build directory
 | This first target presented here are not really defined in Common.targets file, but it is imported from $(MSBuildBinPath)\Microsoft.CSHARP.Targets file which is imported in the Common.targets file. |
- Clean
- deletes all compiled code from ../build concerning the project
- Test
- runs test cases and outputs nunit results into ../log, depends on 'Build'
- NCoverReport
- generates test coverage reports for the component's tests into ../log/coverage. This executable can be found in the NCover directory.
- FxCop
- generates reports information about the assemblies, such as possible design, localization, performance, and security improvements.
2.6.3. Component Name.proj and Common.targets files
The Component Name.proj file is similar somehow with the Common.targets file because it handles tasks that concern compiling/testing .csproj files. The difference between them is that Component Name.proj contains all the common targets that should be called when building the entire solution and not only a project of the solution.
Bellow will be presented the targets made available by this file:
- Build
- compiles .cs sources for every project of the solution into the build/ directory
- Clean
- deletes all compiled code from build/ and removes also log/ directory
- Test
- runs test cases on each of the test project of the solution and outputs nunit results into log/, depends on 'Build'
- NCoverReport
- generates test coverage reports for the each of the component's test projects into log/coverage/. This executable can be found in the NCover directory.
- FxCop
- generates reports information about the component source projects, such as possible design, localization, performance, and security improvements.
- Doc
- generates API documentation for the component source projects in a javadoc manner.
- Dist
- generates a binary distribution for deployment containing only the component source assemblies as well as their dependencies, depends on 'Build'
The Distribution.targets file contains distribution related targets and is suppossed to be used when packaging the design/development distribution or submission or when creating a catalog distribution for the component. It is used by Component Name.proj file.
Bellow will be presendted the targets made available by this file:
- DesignDist
- builds a .zip file which must contain the files needed in a design competition as starting point for designing the component.
- DevDist
- builds a .zip file which must contain the files needed in a development competition as starting point for developing the component.
- DesignSubmission
- builds a .zip file containing the design submission of the competitor
- DevSubmission
- builds a .zip file containing the development submission of the competitor
- DistTcs
- copies the component source assemblies built from the sources of the component and also the distribution .zip file for the component, to the component's path $(target_bin) directory.
- DeployLib
- copies the component source assemblies built from the sources of the component to the component's path $(target_bin) directory (light version of DistTcs target).
- PackageDocs
- builds an archive in the $(BuildDir) which contains the TC docs for the components along with the UML design images for the component.
3. Build the Component
From the command line the following ant targets are most useful:
- msbuild "Component Name.proj" /t:Build - build the component solution assemblies (sources & tests)
- msbuild "Component Name.proj" /t:Clean - clean the build/ and log/ directories
- msbuild "Component Name.proj" /t:Test - runs the unit tests on each of the component solution test projects
- msbuild "Component Name.proj" /t:NCoverReport - generates test coverage reports for each of the component solution test projects
- msbuild "Component Name.proj" /t:Doc - generates the TC docs for the API of the component source assemblies from the solution
- msbuild "Component Name.proj" /t:FxCop - generates the FxCop report about the design problems in the component source assemblies from the solution
- msbuild "Component Name.proj" /t:DesignSubmission - build the .zip file containing the design submission of the competitor
- msbuild "Component Name.proj" /t:DevSubmission - build the .zip file containing the development submission of the competitor
- msbuild "Component Name.proj" /t:DistTcs - moves the runtime .dll files (built from the component source projects of the solution) as well as the .zip file containing the distribution of the component to the library directory.
- The library directory component will be $(tcs_bin) for a generic component, but can also be the library directory for the client's components in case if a custom component is built.
 | Note that "Component Name" should be changed with the actual name of the component (the one from Build.version file) and also that the target names to be executed by MSBuild are not case sensitive. You can also group several targets in one msbuild command by separating them witha semi-colon.
e.g. msbuild "Component Name.proj" /t:Build;Test;NCoverReport;DistTcs |
 |
In the chapters bellow is made a in-depth presentation of the NET component file structure and of the content of the build files which has a target audience the people who are supposed to make tweaks on the build files for their submissions.
If you are just using the distribution of the component and don't need to get into these details you can consider that you've finished the tutorial. |
4. NET component file structure
The component to build should have following files and directories structure after the build is done :
- README.txt - instructions for setting up this component for build
- Component Name.sln - Visual Studio solution file which integrates the two projects for the component (source + sources&test cases). The name of this file will need to be modified to the name of the component. VS IDE will take care of creating this file and since only its name is to be changed there's no reason to go into details on its content.
- Component Name.proj - contains msbuild targets used to do building operations on the entire solution.
- Distribution.targets - contains msbuild targets dedicated for packaging the component (this file is used by Component Name.proj file).
- Common.targets - contains msbuild targets to be used when building separate projects from the solution.
 |
For more details on Component Name.proj, Distribution.targets and Common.targets files please see section 2.6 of this document. |
- Build.version - specific component version information, including version number and component name
- Component versions are specified as major.minor.micro.build - the public version is major.minor.micro and is used by components to find their dependencies. The full version is used for internal management and non-software changes, such as updating documentation.Whenever a modification is done in svn for a specific public version for a component (major.minor.micro) the value for component.version.build is incremented.
Let's say we're having a specific component A with version to be retrieved from the catalog - 1.0.1 . If there are small changes made on the component which don't need a public version incremented (like 1.1.3) released for the component, the only thing which changes regarding the component version will be the value for component.version.build property which will be always incremented.
So component A could have version numbers like these ones: 1.0.0.1 ; 1.0.1.1 ; 1.0.1.2 ; 1.0.1.3 ; 1.1.0.1 .
- conf - configuration files for the component.
- docs - contains the documentation for the component including Requirements Specification, Component Specification, a .tcuml/.zuml file containing UML design of the component and .gif images generated for the UML design.
- Typically this directory needs to include a fileset of documents having this format :
Note that ${distfilename} represents the component distribution file name.
The documents belonging, if it is the case, to a previous version of the component need to be removed from docs/ directory when adding in svn the new version. Be sure to have the files from docs/ directory matching EXACTLY the format specified earlier.
- test_files - file resources needed for testing
- lib - library files used in compiling & testing the component.
- Please note that runtime .dll files that can be found in this directory are for the purposes of compiling and running the test code, not for any other use, including production. Component's that are required for the production code must be downloaded from TopCoder Software Catalog
.
4.1. Build file structure of the projects from the solution
Each project of the solution has a dedicated directory in which are placed the .cs source files, embedded resources and other files necessary to be included in the output assembly.
The build file structure of the project is relatively simple. It is composed out of the :
- the .csproj file with the same name as the output assembly
- Build.dependencies file - contains informations regarding the assembly name, output path, output type, files to be compiled, embedded resources, etc.
4.1.1 Presentation of the .csproj file content
Basically what you need to do to convert a .csproj file built by VS IDE to the building format required by the current process consists in a few simple steps:
1. From the first property group retrieve the properties:
- ProjectGuid
- ProjectTypeGuids(optional)
- RootNamespace and AssemblyName
- OutputType
and then save them somewhere.
2. Copy the content of the item groups dedicated for the references, compile items, embedded resources to the Build.dependencies file and make use of RootNamespace, AssemblyName and OutputType properties saved earlier.
3. Replace the content of the .csproj file with the content presented bellow. The only change you have to make in this step is to replace the value of ProjectGuid and ProjectTypeGuids properties with the ones you saved in step 1.
In some situations you may have to add to the .csproj file some other include statements like the following :
In the Attachments
you can find some samples of .csproj files and a sample of Build.dependencies file.
4.2. Write down the detailed instructions on how the testing environment should be setup
The README.txt file (in the root directory of the component) needs to contain the steps to run through the tests. This file should be added/updated in svn.
Here let's take NORM Configuration Entities component for example:
4.3. Configure the build environment.
You should read the component specification first, make sure in what environment the component will be run. It may include the following issues:
Platform - Windows 2003, Windows XP, Windows Vista, etc
Compiling Target (TargetFrameworkVersion property in Build.dependencies file) - Microsoft .NET runtime environment 2.0/3.0/3.5, etc
Dependencies - Including TopCoder or third party components, required versions of these components should be used.
Especially for custom components, for example if the tests compiled successfully under Oracle 10.2, but failed under Oracle 10.1, and the required version is the latter, you should report it to PM. It is very important to respect the constraints imposed by the Component Specification regarding the Environment Requirements.
4.4. External libraries directory structure.
TopCoder is not allowed to re-distribute the 3rd party libraries on which some of the components are dependent (like junit for example). This is the reason why attached to document you will find here
a listing of TopCoder's ${ext_libdir} directory. Please structure your ${ext_libdir} directory in the same way as TopCoder has structured these libraries to avoid having problems when integrating the component on the Bamboo server.Note that there will be other libraries added, when needed, to the external libraries directory on the Bamboo machine, so please check out from time to time if anything has updated on the 3rd party directory listing
.
5. Commit the modifications made on the component after the build is done to SVN
Please make sure that every change made to the component (from file additions to modifications or remove of useless files) to appear in the Commit dialog.
You should log the modifications you made when updating the svn, including trivial things like "Useless files are removed". Normally when the building of a component is done you should put a comment like "Build x.x.x.x successful" where x.x.x.x is the version for the component.
At the end of this process, the SVN file structure of the component to be built will look like this :

Be sure always to check if the file structure is similar to what can be see in the image presented earlier.
When you encounter a problem in building a component because of a missing third-party library, please contact the component manager
and ivern
in order to have the problem solved.
Please feel free to comment any of the details presented in this tutorial if you feel that it can be polished to be easier to understand or if there is more information needed.