Register Now
Member Count: 176,913 - November 21, 2008  [Get Time]
Login
Dashboard > TopCoder Competitions > ... > Component Build Process > .Net Component Build Process - with NAnt
TopCoder Competitions View a printable version of the current page.  
.Net Component Build Process - with NAnt
Added by durin , last edited by marius_neo on Jun 24, 2008  (view change)
Labels: 
(None)

The NET build process is being currently improved internally at TopCoder to use MSBuild tool instead of NAnt, but until it will be done the workflow presented here must be followed for building .net components.

Overview

Generally the principles of the build process are:

  • Standard version rules are used for all components in a fashion that will prevent broken builds.
  • Third party libraries can be easily adjusted for all components in build environment.

1.  Setup the System Environment

This will install the major system components of .NET and NAnt required for building most .NET based products. Please note that there are many versions of .NET and NAnt available. TopCoder certifies components for the environment specified in the component's Requirements Specification. If another environment is used then the component should be re-certified. NAnt provides a lot of flexibility, these instructions provide an intended process to be followed, other processes may also yield working results.


1.1.  Install .NET

  • Download and install the .NET framework that you need.


1.2.  Install NAnt

  • Download and install NAnt.
  • Add into PATH environment variable the installation directory path for Nant


2.  Setup a TopCoder Build Environment

This step will install the required libraries that are standard to all TopCoder components. NUnit is used as a testing harness to run unit, accuracy, failure, and stress tests packaged with each component. NCover is used to provide a test coverage report. A standard properties file is used to provide environment reference configuration for the build scripts.


2.1.  Install NUnit

Download NUnit.


2.2.  Install NCover

Download the 1.5.8 version of NCover.


2.3.  Designate Source and Library Directories

TopCoder relies on a top level source directory which contains all the Java components as sub directories. The exact name of the .NET component directory is flexible. An example of this directory might be: c:\code\topcoder\NET-components\

TopCoder components build to a standard location specified by the build configuration, this ensures that the binary dll for a given component version is in a unique location on the build system. Additionally it ensures that components which depend on that binary will find it. And example of this directory might be: c:\code\lib\topcoder-net\


2.4.  Setup TopCoder Global Build Properties

The build scripts depend on a common build properties file in addition to those which are packaged with the component. This file describes the overall build properties for the specific integration box, including:

  • The TopCoder Library directory
    • The TopCoder integration environment uses ext_libdir as the parent for all third party libraries, however it's not necessary to setup your environment in this fashion, by having each individual jar or dir property point to the exact path.
  • The environment install location of NCover and NUnit for all components
  • The location of third party libraries (as needed for components on a case by case basis)
  • Standard build parameters
topcoder_global.build
<?xml version="1.0"?>
<!--
    Global settings file for TopCoder Software .NET component builds.  You should place this file in the parent directory
    of the component sources.  By default, these settings will override the same settings in the component build file.
 -->
<!--
    This file will only work with newer TopCoder Software NAnt build files.  If you wish to use it with an older component,
    you will need to add the following line *below* all the lines that set the properties listed in this file.

    <include buildfile="../topcoder_global.build" failonerror="false" />
 -->
<project>
    <!-- NUnit reference. It is safer to use the properties declared in default.build file since the target platform for the components may differ
          so the values for these properties should change too. -->
    <!--
    <property name="Nunit" value="C:\Program Files\NUnit-Net-2.0 2.2.8\bin\nunit.framework.dll"/>
    <property name="NunitConsole" value="C:\Program Files\NUnit-Net-2.0 2.2.8\bin\nunit-console.exe"/>

    -->

    <!-- NDoc reference. -->
    <property name="NDoc" value="C:\Program Files\NDoc\bin\.net-1.1\NDocConsole.exe"/>

    <!-- Base directory for all third party components. -->
    <property name="ext_bin" value="..\tcs\bin"/>

    <!-- Base directory for all TCS components. -->
    <property name="tcs_bin" value="..\tcs\bin\tcs"/>

    <!-- Base directory where the distributable versions of the custom components made for Hermes client should reside.-->
    <property name="hermes_bin" value="..\tcs\bin\hermes"/>
</project>

3.  NET component file structure

The component to build should have following files and directories structure after the build is done :

  • README.txt - instructions for setting up this component for build
  • default.build - ant build script
    • There is a model default.build you can use when building the components. You only need to modify the part where the dependency components needed are declared and used in compile, compile_tests, copy_dependencies targets and also the part which refers to component information:
      Window Size Manager component information
        <property name="component" value="Window Size Manager" />
        <property name="distfilename" value="window_size_manager" />
        <property name="namespace" value="TopCoder\WPF\SizeManager" />
        <property name="namespace_file" value="TopCoder.WPF.SizeManager" />
      
        <property name="component_version" value="1.0" />
      

Note in the model default.build file how the dependency libraries are referenced:

Window Size Manager dependency declaration
  <!-- 3rd party-->
  <property name="netfx_v3_bin" value="c:/Program Files/Reference Assemblies/Microsoft/Framework/v3.0"/>
  <property name="netfx_v3.5_bin" value="c:/Program Files/Reference Assemblies/Microsoft/Framework/v3.5"/>

  <property name="presentation_core.dll" value="${netfx_v3_bin}/PresentationCore.dll" />
  <property name="presentation_framework.dll" value="${netfx_v3_bin}/PresentationFramework.dll" />
  <property name="windows_base.dll" value="${netfx_v3_bin}/WindowsBase.dll" />
  <property name="system_windows_presentation.dll" value="${netfx_v3.5_bin}/System.Windows.Presentation.dll" />

  <!-- ZIPS -->
  <property name="configuration_manager.dll" value="${tcs_bin}\configuration_manager\2.0.1\TopCoder.Util.ConfigurationManager.dll" />
  <property name="configuration_api.dll" value="${tcs_bin}\configuration_api\1.0\TopCoder.Configuration.dll" />
  <property name="file_based_configuration.dll" value="${tcs_bin}\file_based_configuration\1.0.1\TopCoder.Configuration.File.dll" />
  <property name="connection_factory.dll" value="${tcs_bin}\connection_factory\1.0\TopCoder.Data.ConnectionFactory.dll" />
  <property name="object_factory.dll" value="${tcs_bin}\object_factory\1.2\TopCoder.Util.ObjectFactory.dll"/>
  <property name="object_factory_configuration_api_plugin.dll" value="${tcs_bin}\object_factory_configuration_api_plugin\1.1\TopCoder.Util.ObjectFactory.Configuration.dll"/>
  <property name="self-documenting_exception.dll" value="${tcs_bin}\self-documenting_exception\1.0.1\TopCoder.Util.ExceptionManager.SDE.dll"/>
  <property name="exception_manager.dll" value="${tcs_bin}\exception_manager\1.0.1\TopCoder.Util.ExceptionManager.dll"/>
  <property name="simple_file_database.dll" value="${tcs_bin}\simple_file_database\1.0\TopCoder.DB.FileStore.dll"/>
  <property name="simple_cache.dll" value="${tcs_bin}\simple_cache\2.0\TopCoder.Cache.dll"/>
  <property name="random_string_generator.dll" value="${tcs_bin}\random_string_generator\1.0\TopCoder.String.Random.dll"/>
  <property name="linked_list.dll" value="${tcs_bin}\linked_list\1.0\TopCoder.Util.Collection.List.LinkedList.dll"/>
  <property name="set_utility.dll" value="${tcs_bin}\set_utility\1.0\TopCoder.Util.Collection.Set.dll"/>

Notice that the format used for referencing the dependencies : ${component.repository}/${component.name}/${component.version}/${component.dll.name}

Also note that the libraries are referenced ONLY by their name, so don't use :

    • component.name-component.version.dll names
    • dll1, dll2, dll3 or any other name that is not meaningful

Apart the netfx libraries, the libraries will be placed either in the repository :

    • ${tcs_libdir} directory, if they are TopCoder libraries
    • ${ext_libdir}\third_party, if they are 3rd party libraries

Example:

<!-- TopCoder dependency -->
<property name="set_utility.dll" value="${tcs_bin}\set_utility\1.0\TopCoder.Util.Collection.Set.dll"/>

<!-- 3rd party dependency -->
<property name="log4net.dll" value="${ext_bin}\third_party\log4net\1.2.9\log4net.dll"/>

This referencing scheme must be respected for all the libraries either if they belong to ${tcs_libdir} / ${ext_libdir}

  • default_dist.build - the ant build script which will be found in the distribution of the component
    • This build file should be the same as default.build file with the only difference that the text marked between
      <!-- ************************************************************** -->
      <!-- ***** REMOVE EVERYTHING BELOW HERE FOR THE DISTRIBUTION ***** -->
      <!-- ************************************************************** -->
      and
      <!-- ************************************************************** -->
      <!-- ******END REMOVE EVERYTHING ********************************* -->
      <!-- ************************************************************** -->
      

      , including the comments, should be removed, You can make a diff between default.build and default_dist.build files before commiting the files for verifying that their upper parts are the same.

  • conf - configuration files for the component.
  • docs - contains the documentation for the component including Requirements Specification, Component Specification, a .tcuml/.zuml file containing UML design of the component and .gif images generated for the UML design.
    • Typically this directory needs to include a fileset of documents having this format :
      <fileset dir="${docsdir}">
          <include name="${distfilename}_Class_Diagram*"/>
          <include name="${distfilename}_Use_Case_Diagram*"/>
          <include name="${distfilename}_Sequence_Diagram*"/>
          <include name="${distfilename}_Requirements_Specification.pdf"/>
          <include name="${distfilename}_Requirements_Specification.rtf"/>
          <include name="${distfilename}_Component_Specification.pdf"/>
          <include name="${distfilename}_Component_Specification.rtf"/>
          <include name="${distfilename}.tcuml"/>
      </fileset>
      

      Note that ${distfilename} represents the component distribution file name.
      The documents belonging, if it is the case, to a previous version of the component need to be removed from docs/ directory when adding in svn the new version. Be sure to have the files from docs/ directory matching EXACTLY the format specified earlier.
      To help out what I'm trying to say here I will add an image on how the structure for docs/ directory should look like.

  • src - the directory containing source files of the component
    • In src/main directory will be found an AssemblyInfo.cs file. This file contains informations about the component including :title, description, version, etc. Be sure to update the version of the assembly when a rebuild operation is made and also if you are handling a generic component (belonging to the TopCoder Catalog and not to a client) to uncomment the line :
      //[assembly: AssemblyKeyFile(@"..\tcs\bin\TopCoder.snk")]
      
  • test_files - file resources needed for testing
    • test_readme.txt - file containing detailed instructions on how to setup the test environment and how to run the tests.
    • Please note that runtime .dll files that can be found in this directory are for the purposes of compiling and running the test code, not for any other use, including production. Component's that are required for the production code must be downloaded from TopCoder Software Catalog.


3.1.  Configure the environment.

You should read the component specification first, make sure in what environment the component will be run. It may include the following issues:
Platform - Windows 2003, Windows XP, Windows Vista, etc
Compiling Target - Microsoft .NET runtime environment 1.1/2.0/3.0/3.5, etc
Dependencies - Including TopCoder or third party components, required versions of these components should be used.

Especially for custom components, for example if the tests compiled successfully under Oracle 10.2, but failed under Oracle 10.1, and the required version is the latter, you should report it to PM. It is very important to respect the constraints imposed by the Component Specification regarding the Environment Requirements.


3.2.  Remove useless files

Remove auto-generated files like :

  • conf/putYourConfigFilesHere.txt
  • test_files/putYourTestFilesHere.txt
  • .txt files from ${srcmain}
  • Thumbs.db (this files can be added by mistake in the submission, but they are useless)

Remove also the files which are not really needed by the distribution of the component. I mean here files like .txt files containing details about the final fixes, .log files.
Remove all mocked files, use real dependencies, you can find the dependencies from http://software.topcoder.com/. If the dependencies have not been built yet, report it to PM. If the component could not compile and execute successfully using the real dependencies, report it to PM.


3.3.  Deploy Dependent Components

In the file default.build there is a listing for each TopCoder component and version that is a dependency.

The runtime jars for each of these components must be deployed to the TopCoder library location for the build to work. The pattern is ${tcs_bin}/"component name"/"component version"/"component dll name".dll. For example using the above value for tcs_bin, the current version (1.0.1) of wcf_base component would be located at: c:/code/lib/topcoder-net/wcf_base/1.0.1/TopCoder.Services.WCF.dll

TCS dependencies
    <!-- ZIPS -->
    <property name="configuration_manager.dll" value="${tcs_bin}\configuration_manager\2.0.1\TopCoder.Util.ConfigurationManager.dll" />
    <property name="configuration_api.dll" value="${tcs_bin}\configuration_api\1.0\TopCoder.Configuration.dll" />
    <property name="object_factory.dll" value="${tcs_bin}\object_factory\1.2.1\TopCoder.Util.ObjectFactory.dll"/>
    <property name="object_factory_configuration_api_plugin.dll" value="${tcs_bin}\object_factory_configuration_api_plugin\1.1\TopCoder.Util.ObjectFactory.Configuration.dll"/>
    <property name="logging_wrapper.dll" value="${tcs_bin}\logging_wrapper\2.0.1\TopCoder.LoggingWrapper.dll"/>
    <property name="self_documenting_exception.dll" value="${tcs_bin}\self-documenting_exception\1.0.1\TopCoder.Util.ExceptionManager.SDE.dll"/>
    <property name="exception_manager.dll" value="${tcs_bin}\exception_manager\1.0.1\TopCoder.Util.ExceptionManager.dll"/>
    <property name="set_utility.dll" value="${tcs_bin}\set_utility\1.0\TopCoder.Util.Collection.Set.dll"/>

These .dll libraries can either be downloaded and saved in the correct location - or by running the build for each dependent component.


3.4.  Setup Third Party Libraries

Each third party library is listed as property in the default.build file. For example:

Third party Dependencies
    <!-- 3rd party-->
    <property name="service_model.dll" value="C:/Program Files/Reference Assemblies/Microsoft/Framework/v3.0/System.ServiceModel.dll" />
    <property name="serialization.dll" value="C:/Program Files/Reference Assemblies/Microsoft/Framework/v3.0/System.Runtime.Serialization.dll" />
    <property name="log4net" value="${ext_bin}\third_party\log4net\1.2.9\log4net.dll"/>


3.5.  Write down the detailed instructions on how the testing environment should be setup

Write a test_readme.txt (in test_files/ directory) for every build which contains the steps to run through the tests. Also, code coverage tools should be used to generate a report about the component. These files should all be updated into svn.
Here let's take impact_report_manager component for example:

test_readme.txt file
Steps to run through the tests:
1. Initialize your Oracle and SqlServer services.
2. Execute "test_files\Drop_Tables.sql" to clean up the database environment, you can also use client tools like "PLSQL Developer" to do this job.
3. Execute the following sql scripts to set up the databases.
   - test_files\Impact_Report_Manager_DDL_Updated.sql
   - test_files\Init_Tables.sql
   - test_files\ID_Generator_for_Oracle.sql
4. Update the following configuration files, you need to modify the connection string for databases, like "User Id=system;Password=dwb;Data Source=ORCL;".
   - test_files\impact_report_manager.xml
   - test_files\invalid_configuration.xml
   - test_files\AccuracyTests\config.xml
   - test_files\Failure\impact_report_manager.xml
   - test_files\Stress\common.xml

4.  Build the Component

From the command line the following ant targets are most useful:

  • nant compile - build the component code
  • nant compile_tests - builds the test code
  • nant test - runs the unit tests
  • nant nunitreport - builds the NUnit reports
  • nant ncoverreport - builds the test coverage reports
  • nant dist_tcs - moves the runtime .dll and the distributable .zip to the library directory for the component.
    • The library directory component will be ${tcs_bin} for a generic component, but can also be the library directory for the client's components in case if a custom component is built. In this case the build file must be modified to point to ${client_name_bin}
      where "client_name" is the name of the client and client_name_bin is a property defined in topcoder_global.build file or in the default.build file.

Be sure to compile/test the component under the required target platform from the component specification.
So when you'll have a component with .netfx 1.1 target platform execute:

Commands to execute under .netfx 1.1
nant -t:net-1.1 target_name

where "target_name" would be one of the targets specified earlier.


5.  Commit the modifications made on the component after the build is done to SVN

Please make sure that every change made to the component (from file additions to modifications or remove of useless files) to appear in the Commit dialog.
You should log the modifications you made when updating the svn, including trivial things like "Useless files are removed". Normally when the building of a component is done you should put a comment like "Build x.x.x.x successful" where x.x.x.x is the version for the component.

At the end of this process, the SVN file structure of the component to be built will look like this :

Please feel free to comment any of the details presented in this tutorial if you feel that it can be polished to be easier to understand or if there is more information needed.