Challenge Overview
Project Overview
As you might have already known, TopCoder is now part of Appirio, creating the world's largest professional development and design commuity. TopCoder and CloudSpokes will eventually be merged together, and one of the very first steps is to merge the user accounts of two platforms.
For this contest, we are going to implement a loader that will load CloudSpokes contest data into TopCoder's contest related tables.
Competition Task Overview
The purpose of this contest is to implement a loader to load CloudSpokes contest data into TopCoder's contest related tables.
This is a 96h contest, make sure you meet the timeline!
Source & Destination
The source from which to load CloudSpokes contest data will be an Excel spreadsheet. The destination should be TopCoder's contest related tables, we provided mapping file from their data to TC's data but you still need to analyze the provided references to figure out which exact tables to insert data to. We'll also provide sample contest data from CloudSpokes.
Notes:
1. Columns not in the mapping file can be ignored.
2. We don't need any billing related setting on contest or project.
3. We have user_permission_grant which is used for project permission, so if a contest has a project (TC Direct Project), and when we add resources (manager/observer), we need to add them to project (give full access).
Scope
For this contest, we're not targeting to import the data into all related tables, instead we're focusing on the comp related tables, we'll leave the project related tables for next phase.
At a high level, this will at least include but not limited to these tables:
- comp_versions: set phase_id to 112
- comp_catalog: root_category_id can be hardcoded to 9926572, set status_id to 102
- comp_categories
- comp_technology
- comp_version_dates: phase_id should be hardcoded to 112, posting_date should be hardcoded to 1976-06-05, other ***_date columns shold be hardcoded to 2000-02-01, set status_id to 301 and level_id to 100
- technology_types
The easist way to find out which exact tables are affected is to create a new contest in your VM, and then check for new records in all tables. You must make sure all comp related tables are correctly populated by this contest. And in the next contest we'll add the project tables too.
Existing Loaders
This new loader needs to follow the existing loader (such as https://coder.topcoder.com/internal/web_module/trunk/src/main/com/topcoder/utilities/dwload/TCLoadTCS.java), which has a script and xml config file.
More existing loaders can be found at: https://coder.topcoder.com/internal/web_module/trunk/scripts/dwload
We will also provide CloudSpokes User Data Loader for your reference.
Errors
Unexpected errors should not cause the whole loading process to fail, instead rows/records that caused errors should be logged and the loader should continue with the remaining records.
Conflicts
If a contest is already migrated (there is a project, project_info with the challenge id, and the project status is not deleted), then we will not migrate that challenge, and we'll write that to the conflicts report.
Transaction
Do NOT use one big transaction for all data, commit a transation for every 50 contests.
Progress
While loading the data, the tool should show some progress (in console).
Report
After the loader is done with loading the data, we need to see a report of the results (for example: how many records are imported, how many fail and which exact ones fail, etc...)
Logging
The tool should log detailed debug/warn/error info during the load process.
Command Line
The loader should be implemented as a command line application using Java, and it should support at least the following command line options:
- Conflicts File: the path of the file to write conflict records.
- Source Excel File: the path of the source Excel file. This is a required parameter.
- Output Directory: the path of the folder to which the report will be written. This is a required parameter.
- Help: this option should print the usage of the command line application.
References
public SoftwareCompetition createSoftwareContest(TCSubject tcSubject, SoftwareCompetition contest, long tcDirectProjectId, Date multiRoundEndDate, Date endDate) which calls private void createUpdateAssetDTO(TCSubject tcSubject, SoftwareCompetition contest) for the component related tables.
2. https://coder.topcoder.com/tcs/clients/cronos/components/catalog_entities/branches/cockpit_trunk and https://coder.topcoder.com/tcs/clients/cronos/components/catalog_services/branches/cockpit_trunk
These two components mainly write to the following tables:
comp_versions, comp_catalog, comp_technology, comp_version_dates
SVN
Please email support@topcoder.com to request access to direct trunk and any other SVN paths listed above and below, please include the contest title/link and your user name in the email.
Trunk: https://coder.topcoder.com/internal/web_module/trunk/src/main/com/topcoder/utilities/dwload/cloudspokes
DB Trunk: https://coder.topcoder.com/internal/database/scripts/trunk
TC Web: https://coder.topcoder.com/internal/web_module/trunk
VM
VM specific information is found here: http://www.topcoder.com/wiki/display/docs/VM+Image+2.5.
Upon registration as a submitter or reviewer you will need to request a VM based on the new TopCoder Cockpit/Direct image. To request your image, please use the forum.
Before requesting your VM, you need to ensure that you have an SSH key created and in your member profile. Instructions to do so are here:http://www.topcoder.com/wiki/display/projects/Generate+SSH+Key, and instructions to connect afterwards are here: http://www.topcoder.com/wiki/display/projects/Connect+Using+SSH+Key.
There is a TC Site Resource page that contains documentation / instructions for the VM instance you will be issued upon request. It is located here:http://www.topcoder.com/wiki/display/projects/Cockpit%2CDirect+VM
Technology Overview
- Java 1.5
- Excel
- Apache POI
- JSP
- Struts 2
- Redhat Linux
- JBoss 4.0.2
- Informix 11
Final Submission Guidelines
Submission Deliverables
- Source code which are necessary to meet all the requirements stated above.
- The competitor should provide a deployment document. This document should include a step-by-step guide on how to use and verify the tool.
- Test data to verify your submission.
Final Submission
For each member, the final submission should be uploaded to the Online Review Tool.