Private Edge -Human Parameters Detection ML & Web Dev Challenge

Challenge Overview

Topcoder Private Edge Challenge Series

Welcome to the Topcoder Private Edge Challenge Series! We are running a series of challenges leading up to the 2021 Topcoder Open (TCO21) to educate and introduce you to the Private Edge Platform, API’s and Services. Learn all about the Challenge Series - jump in, register on the Private Edge Developer Platform, ask questions and have fun creating! Each Topcoder challenge in the series will have its own unique objectives, so join in at any time and realize that the more you learn about Private Edge and 5G, the more it will help you during TCO21 upcoming events.

Technology Stack

Use whatever stack you prefer, just create a fully functioning app. Here are some machine learning library links to potentially get you started:

• Python Gender & Age Recognition
• MediaPipe - Face / Hand / Object / etc - Android / iOS / Python / etc
• GoLang face detection
• Python face detection
• NodeJS face detection
• Python action recognition

Note on the use of existing libraries - the above libraries are basic starting point recommendation and might or might not be useful for your solution. The contestants are expected to take a call on which libraries to use based on their own discretion, and they are welcome to use other libraries or build custom built solutions.

Project Background

This challenge is part of a series of challenges to create an ML app that interfaces seamlessly with Private Edge’s services. The first demo use-case was built starting with a data science challenge to develop the image classification model, followed by a web development challenge to connect the model to the user via a web interface.

In the second use case, we are looking to build: an ML app that pulls images from a webcam and identifies people’s age, gender, and sentiment. These detected parameters will then be used to run a demo ad that is appropriate for the type of family.

Challenge Objectives

The primary requirements of the challenge for contestants are:

• Detect humans in an image: After locating a human, a bounding box around that image should be extracted, and its image coordinates should be included in the image’s output parameter object.

• Identify the age, gender and sentiment of each human in the image: existing or custom-built tools can be used to detect the age, gender and sentiment of each human in the image. Contestants are free to use multiple tools or detectors, as is using a separate detector for age, gender and sentiment respectively.

• Create the web front-end: The frontend of the web app needs to display the results based on the returned JSON from the backend.

• Create the web backend to receive images, perform machine learning operations and render results: The backend should be able to accept continuous images from a webcam and perform machine learning operations on these images and render the results in JSON format to the frontend.

• Must create an account at developer.alefedge.com and use the video enablement API to play a demo ad after the paramters of the humans in the picture is complete: In other words, the app must use the Alef Video Enablement API to play an ad depending on the type of human detected. The video can be somewhat related to the detected features. As a fall back, you can play a locally hosted ad if the edge hosted ad is not detected. More resources to assist in this process can be found in the forum shortly after the challenge launch.

Note on 'Ease of deployment'- Code needs to be able to install easily using preferably a single line of command. The code should run on both PC and Linux.

Sequential Review - The review of this challenge will be performed in a sequential manner, i.e. first the data science aspect will be judged, followed by the web development aspect. Submissions that do not fulfill the data science requirements will not undergo web development code evaluation.

Challenge Details

Data Science Part

As mentioned above, the data science requirement is to create a function get_people_parameters(list_of_images) -> list_of_parameter_objects , which should be able to take in a list of image paths or image links as input and return a list of objects, with each object containing the detected parameters in its corresponding input image. Contestants are free to explore existing services or tools (with appropriate licensing) that can help with the detection of people in an image and to identify their age, gender and sentiment.

The primary requirements for the data science part are: 1. Detect humans in an image. 2. Identify the age, gender and sentiment of each human in the image 3. Encapsulate these detectors into a script, which contains the function get_people_parameters(list_of_images) ->; list_of_parameter_object.

Here, list_of_images is the list of N local image paths, or links to images, while list_of_parameter_objects is a list containing N objects (called parameter objects here), with each parameter object corresponding to an image in the list_of_images.

In each parameter object in turn should of the form:

{
    "human_found": True/False,
    "list_of_humans": [
        {
            "bounding_box_coords" : ((bounding_box_coordinates)),
            "age": ((age)),
            "gender": ((gender)),
            "sentiment": ((sentiment))
        },
        {
            "bounding_box_coords" : ((bounding_box_coordinates)),
            "age": ((age)),
            "gender": ((gender)),
            "sentiment": ((sentiment))
        }
    ]
}

Here, “human_found” is True if the list_of_humans list contains more than 0 objects, else it should be set to False. Within the list_of_humans property, a separate object should be inserted for each detected human. For each human, their “bounding_box_coords”, “age”, “gender” and “sentiment” should be mentioned.

Demonstration Notebook or testing script - Note that along with the script, the submission should also include a Jupyter notebook or a normal script called demonstration.ipynb or demonstration.<your_file_extention>. It should contain a demo of the functions capabilities (the script and the function should be imported to this notebook via the import syntax). Here, it should be possible to change the test images folder.

The notebook/script’s code should go through all the images in the target images folder (configurable), and should generate the list_of_parameter_objects from the function get_people_parameters in a json file named output.json. This will be used during the review to verify the capabilities of the detectors and the script.

Web Development Part

• Create the web front-end to allow multiple images to be uploaded by the user and to display the backend generated results: The frontend of the web app needs to be built to allow a user to push images to the backend, and to display the parameter results received from the backend via a clean interface. The contestants are free to decide the overall look and design of the page, but in terms of aesthetics, the UI should be clean and the elements on the page should be well-placed and well-aligned.

• Create the web backend to receive images, perform machine learning operations and render results: A backend needs to be built, which primarily encapsulates the get_people_parameters function. The backend should be able to accept images from a webcam, perform machine learning operations on these images using the function discussed above, and render the results in JSON format to the UI, which in this demo will be used by the frontend discussed above.

• The backend primarily has two components: the handler sub-component and the image Machine-learning sub-component:
  • Handler sub-component - This component of the backend should take the uploaded images, store them in a temporary folder, and after all the analysis has been performed on them, these images should be deleted. These images should also be deleted in case any exception is detected during code execution. The list of local paths from this temporary folder then needs to be passed to the machine-learning component, which in turn will return the list of classes. These results should then be returned to the client that made the API call.
  • Machine Learning sub-component - This will primarily consist of the get_people_parameters*** function (discussed in the data science section above) and all the ancillary code required for its execution. The handler component should be the only endpoint that is exposed for frontend calls, and this handler component will in turn interact internally with the Machine-learning sub-component.
  • On efficiency - an attempt should be made to keep the backend code as efficient as possible, because its speed will directly impact the overall feel of the web app.

Video Ad using AlefEdge Video Enablement API

As discussed, the submitter must create an account at developer.alefedge.com and use the video enablement API to play a demo ad after the paramters of the humans in the picture is complete. In other words, the app must use the Alef Video Enablement API to play an ad depending on the type of human detected. The video can be somewhat related to the detected features. As a fall back, you can play a locally hosted ad if the edge hosted ad is not detected. More resources to assist in this process can be found in the forum shortly after the challenge launch.

Here, the app should basically play a demo video based onn the features of the detected ad. For example, if the detected features are Male, 15, happy - maybe a video advertisement of clothing for male teenagers can be shown.

Video mapping should be configurable - It should be possible to add videos later and it should be possible to set video links/files based on the parameter triplet, i.e. {Gender, Age, Sentiment}. For the purpose of ads, different age ranges can be defined and a video can be set for each. For instance, the age ranges can be 0 to 5, 5 to 12, 12 to 18, 18 to 30, 30 to 45, 45 to 60, 60+. Let's say there are 7 possible age ranges, 2 possible genders (contestants are free to consider more as they see fit), and let's say 5 sentiment types. So in principle, there could be 7 x 2 x 5 = 70 total {Gender, Age, Sentiment} parameter triplet, and hence 70 possible unique videos (one for each triplet).

But it is safe to assume that a lot of videos can be repeated across multiple triplets. For example, in case it's an ad of a movie (a trailer), it can be shown to multiple age ranges, across multiple genders and multiple sentiments. Hence for the sake of this challenge, only a few demo ad videos should be enough, although it should be possible to configure and add more videos if needed.

Review Criteria

Sequential review - As discussed, the review of this challenge will be performed in a sequential manner, i.e. first the data science aspect will be judged followed by the web development aspect. Submissions that do not fulfill the data science requirements will not undergo web development code evaluation.

Data Science Review Criteria

The final score will be a combination of objective score and a subjective code review score.

1. Objective Scoring (80% weightage) - Broadly, the final objective score will be calculated by taking the geometric mean of the submission performances in bounding box overlap correct, age prediction error, gender match accuracy and sentiment match accuracy. Based on the final number of submissions, more specific details of the objective scoring criteria will be discussed in the forum. The exact scoring metric for each individual score component will be shared in the forum shortly after the launch of the challenge.

2. Subjective Scoring (20% weightage) - In the subjective scoring, the quality of the code, code organisation, documentation quality, notebook quality etc will be taken into consideration.

Note - the submission can be disqualified if the required script with the function get_people_parameters(list_of_images) -> list_of_parameter_objects* is not found in the submission. Furthermore, absence of the Jupyter notebook or providing no option to configure its test data folder can also lead to the submission’s disqualification.

Review platform - The review will be performed on a Windows 10 64 bit machine, with an Nvidia GTX 1650Ti GPU available. As discussed earlier, the code should be cross-platform, and hence the contestants should ensure that their deployment instructions do work on Windows.

Web Development Review Criteria For this part, the review will be performed similar to the parameters tested in the standard Topcoder Code review scorecard, such as fulfillment of individual requirements, code quality, comments, documentation, etc.

What To Submit

• Source code - All the source code (in any language of your choice)
• Testing script - the model testing script/notebook
• Documentation - Instructions to deploy the solution from scratch.
• Verification Instructions - The documentation should contain detailed instructions
• Video (Optional) - Although not mandatory, contestants can optionally include a video demonstrating their script and submission meets all the requirements. The video can be uploaded to YouTube, Google Drive, Dropbox or elsewhere and the link can be included in the submission archive.

Private Edge Topcoder Challenge Series

As part of the Topcoder Private Edge Challenge series, we are excited to hear and learn from Topcoder Community members about your experience with this challenge, future challenges, and the Private Edge platform. The winners of this challenge will have an opportunity to share their experience or be showcased as a top developer in Private Edge challenges. Topcoder may reach out to the winner, where the winners can participate in additional marketing and promotional activities related to the Private Edge Challenge Series.