Challenge Overview
NASA needs to integrate the ION Disruption Tolerant Networking (DTN) implementation of Bundle Protocol (BP) with Microsoft Outlook and Microsoft Exchange Server to support the transfer of astronaut email to/from the International Space Station (ISS). Communications are delayed and frequently disrupted due to ISS structural blockage and Tracking and Data Relay Satellite System (TDRSS) handovers. The combination of delay and disruptions causes Microsoft Outlook to frequently have problems and become unusable, particularly when sending emails with large attachments, such as pictures or videos. See more at: http://www.topcoder.com/dtn/astronaut-email
TopCoder created and deployed a DTNTUN network interface that can transmit TCP/IP over DTN using the Bundle Protocol. We need to test and find bugs in the DTN tunnel.
There is a deployed system, but we will provide EC2 servers for bug hunters. It is needed because you will have to introduce delays and failures to test the ISS environment.
We are mainly interested in bugs that may occur with the following scenarios:
- Outlook connection:
- What happens when network is disrupted?
- Can we send an email and it is kept in outlook outbox until network is restablished?
- Does outlook become unresponsive?
- Large email attachments:
- What happens when network is disrupted in the middle of a file transfer?
- Is the file retransmitted after network is restablished?
- Does outlook become unresponsive when network is disrupted while sending and/or receiving a large email?
- Calendars and tasks
- Disrupt communication (simulated using ION-DTN owltsim)
- Packet loss (simulated using ION-DTN owltsim)
The customer supplied also a very specific list of tests (which might overlap with the above list):
- We should have a test case to demonstrate that non-DTN email on the ground can operate in parallel with DTN email. This would involve one ground user sending an email to another ground user while other users are sending email to a space user through the DTN proxy, but all are using the same Exchange server. This would require multiple users on the ground.
- I would add a test case where a large email is sent but the client loses connection to the proxy before the message is completely sent (similar to a wireless connection dropout). When the connection is reestablished, the message should be sent successfully.
- I would also add a test case where the outlook client is closed before the message leaves the outbox. When the outlook client reopens, the message should be sent successfully.
- I would add a test case with significant rate mismatches between the space clients/space proxy and between the space proxy/ground proxy, with the lower rate between the space/ground proxies. This is analogous to the ISS scenario. The ISS onboard network is 100 Mbps, but space link rate can vary significantly and is much less at times. The BP rate can be controlled in the ionrc configuration file in bytes/sec and there are other rate-limiting options depending on the convergence layer. We could try space link rates of 500 kbps, 1 Mbps, 5 Mbps, 10 Mbps and 50 Mbps. ISS is also planning to upgrade the onboard network to 1 Gbps, so the rate mismatch will be greater in the future.
- For the test cases that involve injecting delay, I would specify a range of times to test. Something like (0.5, 1.5, 3, 10, 30, 60, 120, 1200, 1800) seconds for the one-way light time.
- Similarly, for the test cases the involve injecting errors, I would specify a range. Something like error rates of (1e-4, 1e-3, 1e-2, 1e-1, 1.5e-1, 2.0e-1)
The guidelines for this contest are given below:
- As issues are identified they need to be logged in Jira: https://apps.topcoder.com/bugs/browse/DTNX
- Issues must include clear descriptions, test cases and steps to reproduce and expected vs. actual results in order to be counted.
- Please provide logs or files used to test.
- First competitor to find an issue gets credit, duplicates will not be counted.
- Reviewer will accept, reject or mark the issues as duplicate.
- Please DO take a look at the reported bugs, duplicated bugs cost your work time and the reviewer's time.
Important Notice:
You must also be the first person to report the issue and submit it while submission phase is open. Any issues before and after the submission phase will NOT be counted.
Additional Resources
Some of the additional resources helpful for the project are:
- DTN TUN submission and source code: explains how to setup EC2 and OpenVPN as well as outlook client.
- Test Scenarios
Contest Prize Eligibility
The submitter with the most accepted bugs will win the contest.
Final Submission Guidelines
Bug Report Process
Bug Report Format
For each report of a limitation or bug, we need the following information:
- Steps to reproduce, including any needed information
- Screen shots (if applicable)
- Configuration files used in each endpoint
- Expected results after the bug is fixed
- Current results, before the bug is fixed
Important Notice:
- If you do not properly document your bug reports, they will likely be rejected due to lack of information or documentation. Also, make sure your bug reports are reasonably general.
- If you submit the same bug that is seen in multiple screens, for instance, you will likely only get credit for the original bug report. The others will all be closed as duplicates.
IMPORTANT:
Unlike other bug hunts, we will also reward $500,00 for a second place.
A bonus of $500 for the first person to hit 100 defects (only 1 person will be eligible for the bonus)Ticket Logging and Test Scenarios Report
You will log your tickets here: https://apps.topcoder.com/bugs/browse/DTNX
Scoring
The Scoring guidelines followed for the contest are given below:
- For scoring, the submitter with the most accepted bugs will win.
- For submitters who submit but don't take first placement, if they submit bugs that aren't covered in the first place submission, they will receive $10 for each unique bug reported up to a maximum of $500,00.
Important Notice:
If two submitters submit the same bug report, the submitter who submitted the report first into Github Issues will get credit for the bug. The second submitter will not.
Tips
Some of the tips helpful for the contest are:
- Submitting what is obviously the same issue multiple times with small variations will only annoy the reviewer that has to sort through all the issues and will only count as one issue anyway. If it's less obvious if it is the same issue or not, use your best judgment and the reviewers will do the same.
- Put an eye on the issues being submitted by other members to minimize the time you may be spending on duplicate efforts. Knowing what has already been reported will allow you to better focus your time on finding yet undiscovered issues.
- Double check your steps to reproduce and test cases to make sure they are clear. Make sure your steps include creation of any necessary data.
Environment Setup
- You should request EC2 VMs on the forum
- You should request access to Jira on the forum
- ION-DTN source code: http://sourceforge.net/projects/ion-dtn
- There are a large number of regression tests in ion-open-source/tests that could be copied and adapted to test delay, retransmission and disrupcy. For example, the ltp-retransmission test.
- Look at owltsim tool and adjust DTN configuration files to test different scenarios
- Both issue-324-ltp-files and issue-364-dtpc are good examples of how to include owltsim in a test scenario. In both cases the one-way light time is only 1 second and the "channel noise" rate is fairly low (like 5% datagram loss), but these parameters are easy to adjust.
- The "limbo" test is a good example of using duct blocking to implement link disruption.
Submission Deliverables
You need report your issues in Jira: https://apps.topcoder.com/bugs/browse/DTNX
Please submit a report file containing the overall tests you performed (positive and negative). Also, list the bug reports you submitted.