TopCoder’s big data biomedics solution performs hundreds of times faster with a higher degree of accuracy than original solution
Highlight
- Execution time of TopCoder’s algorithm was multiple magnitudes better than the original solution.
- TopCoder members spent an average of 22 hours working on the problem.
- The results achieved by contest participants in only 14 days improved significantly upon the existing solutions available to academic researchers, decreasing processing time by up to three orders of magnitude with accuracy reaching the theoretical maximum performance level.
Challenges
Cutting-edge computational work is consistently under-resourced. To counteract this challenge, it traditionally requires deep pockets or access to talented coders and algorithmists who are willing to spend an exorbitant amount of time on the project.
Harvard Medical School was facing this exact challenge for a complex DNA sequencing alignment challenge. This was a real-world problem, where the limitations of existing tools severely constrained the ability to pioneer new advances in medical knowledge.
Solutions
Harvard Medical School enlisted TopCoder to help find a faster, more accurate solution for a tool that calculates the edit distance between a query DNA string and the original DNA string. This is critical for making high-precision, high-throughput readouts of the immune system. Prior to the TopCoder contest, the best known solution, MegaBLAST, processed 100,000 sequences to a high degree of accuracy, yet required 2,000 seconds to execute. A full-time, Harvard resource that spent a year on this unique problem was able to produce an improved outcome, reducing the computational time to 400 seconds.
With $6,000 in total prize money, 733 registrants and 122 members submitting working algorithms, TopCoder provided HMS with a winning solution that performed hundreds of times faster and at a higher degree of accuracy, reducing the time to execution to just over 16 seconds.
Benefits
Harvard Medical School enlisted TopCoder to help find a faster, more accurate solution for a tool that calculates the edit distance between a query DNA string and the original DNA string. This is critical for making high-precision, high-throughput readouts of the immune system. Prior to the TopCoder contest, the best known solution, MegaBLAST, processed 100,000 sequences to a high degree of accuracy, yet required 2,000 seconds to execute.
A full-time, Harvard resource that spent a year on this unique problem was able to produce an improved outcome, reducing the computational time to 400 seconds. With $6,000 in total prize money, 733 registrants and 122 members submitting working algorithms, TopCoder provided HMS with a winning solution that performed hundreds of times faster and at a higher degree of accuracy, reducing the time to execution to just over 16 seconds.
