Project Overview

Our forests, fisheries and wildlife are subject to many pressures, but illegal activities are especially damaging due to their scale and scope.  WWF has developed a methodology to identify products moving through global trade routes that are suspected as illegal by comparing import and export data.  The first step in addressing this challenge is gathering the data itself.

This challenge is part of the HPE Living Progress Challenge Blitz Program (Secure top placements in the leaderboard to grab additional cash prizes)

Challenge Task Overview
In this challenge, we’re going to ask you to write a Python script which pulls data from the UNComtrade web service related to seafood.   Each request will generate a csv file which will then be loaded to our Vertica instance, but this challenge is just about getting the raw csv files.

The endpoint for the API is found here:

http://comtrade.un.org/api/get

The full documentation for the API can be found here:
http://comtrade.un.org/data/Doc/API

What we want to do is perform the following request:
http://comtrade.un.org/api/get?max=50000&type=C&freq=A&px=HS&ps=<year>&r=<reporter>&p=all&rg=all&cc=<commodity_code>

We need the script to iterate through each of the possible values for each of the parameters: year, reporter, commodity code.

The years are 2003-2015

The reporters can be found in the reporter.json document attached to this challenge.  You should enter the id for the parameter.

The commodity codes can be found in the commodities.json document attached to this challenge.

Other requirements:

Your application should not execute a request more than one request per second per the usage guidelines.

The app should save the csv files generated in an “data” folder with the following naming convention - <year>_<reporter>_<cc>.csv

You do not need to save/write empty files.

Platforms:  Linux

Technologies:  Python

Final Submission Guidelines

Submission Deliverables

• Python script that covers all requirements.
  A complete and detailed deployment document explaining how to deploy the application including configuration information.

 

Final Submission

For each member, the final submission should be uploaded via the challenge detail page on topcoder.com.