Research Publications Intelligence
Accelerating pharma market insights using public data and precision scraping
Research Publications Intelligence
Accelerating pharma market insights using public data and precision scraping
The Challenge
A Top 500 pharma company needed a scalable and reliable way to collect actionable competitive intelligence from a vast pool of publicly available research publications. The goal was to identify how 13 major pharmaceutical competitors were investing in key areas like therapeutic innovation, disease research, and data science strategies.
This required finding the right data sources, extracting relevant content, identifying competitor mentions, and capturing it all in a structured, machine-readable format. The biggest hurdles were data quality, trustworthiness of sources, balanced coverage across all competitors, and the ability to replicate the process through clean, documented code.
The Solution
Through Topcoder’s Innovation Challenge model, participants were tasked with identifying trustworthy sources, scraping research content, and tagging it with competitor-specific insights.
The solution included a structured dataset of over 120 submissions formatted in JSONL, featuring metadata, publication dates, and relevance snippets for 13 major pharmaceutical competitors. Submissions were built using Docker and scored based on the depth of insights, coverage consistency across all competitors, and the ability to surface emerging trends in therapeutic innovation and strategy.
Challenge we ran:
9
Challenges
77
Participants
127
Submissions
The Impact
The customer gained a reproducible and scalable framework for competitive intelligence, reducing the time and manual effort typically required for this kind of research. The final dataset helped uncover patterns in competitor activity, spotlighted emerging research trends, and enabled faster, data-backed decision-making.
By outsourcing the data extraction task to Topcoder’s global community, the customer accessed a broader pool of technical expertise, which resulted in a more balanced and comprehensive result set than a traditional in-house effort might have achieved.
Achieve high-quality outcomes with
Topcoder.
Achieve high-quality outcomes with Topcoder.