Key Information

The challenge is finished.

Challenge Overview

A previous challenge has implemented a set of REST APIs for handling video assets, including storing them and managing them (create, retrieve, update, delete).  This challenge will add HLS validation to the scrapers that read HLS / M3U8 streams:

* NewsMax TV 
* CBS News
* Fox News

It will also update the BBC News scraper to pull MPEG DASH / MPD streams instead of HLS.

Existing API

The existing Node application and deployment details are in Gitlab, and the URL to the repository can be found in the forum.

HLS Validator

Some of the streams don't currently fit with the HLS standard, particularly the CBS News Live TV stream - it doesn't seem to match the M3U8 / HLS spec exactly.  We need to add some basic validation to the scrapers that parse out HLS streams to ensure the streams we add to the data store are valid.


When we are scraping HLS streams for Fox News, News Max TV, or CBS News, we want to validate the HLS streams.  The implementation is at your discretion, but here are some possibile implementation tips:



The validation should be able to be turned on or off via a configuration flag that can be set in the environment and using "heroku config:set", named "VALIDATE_HLS_STREAMS".  The configuration value will either be true or false.


The client has asked if we can update the BBC scraper to parse out MPEG/DASH streams instead of HLS.  These streams have an "mpd" extension, like:���

This challenge needs to ensure that the BBC scraper parses out the MPD / DASH stream URLs when available.  It can default back to HLS when no DASH videos are found.

Make sure the README is updated with verification information about the new features and configuration information so they can be easily added.

Unit tests

As with the other scrapers, unit tests are required for these new chanages.

Heroku deploy

Make sure the Heroku deployment information is up-to-date and that you keep the package.json up to date as well.  Don't expect the deployment to be anything other than "npm install" / "npm start" locally and "git push heroku master" for Heroku deployment.

Submission format

Your submission should be provided as a Git patch file against the commit hash mentioned in the forum.  MAKE SURE TO TEST YOUR PATCH FILE!


Final Submission Guidelines

Please see above


2017 TopCoder(R) Open


Final Review:

Community Review Board


User Sign-Off


ID: 30055172