Titan Eye - Operator Behaviour Compliance DS Code Challenges, Part 2 - Mobile Phone and Truck Movement Detection






    The challenge is finished.
    Show Deadlinesicon-arrow-up

    Challenge Overview

    Your task in this challenge is to analyze videos showing the driver's cabin of a garbage collecting truck. You have to create a tool that will help the client to determine whether the driver's behaviour is compliant to the company's code of conduct: nobody is allowed to smoke inside the cabin and the driver is not allowed to use his mobile phone while truck is moving.


    Current Scope of this Challenge is limited to “Finding if driver is using mobile phone in cabin” and “finding if truck is moving or not” and you may develop this on top of the solution of the previous challenge (part 1) that aimed at detecting the presence of the driver and whether he is smoking. The two best scoring submissions to part 1 are available from the contest forum.


    More specifically, we need a tool that:

    • Takes a video as input.

    • Creates a text file (events.txt) as output that contains time stamped events of several kinds. An event means that one or more of the following parameters change. All parameters are booleans with possible values {yes, no}.

      • driver: the presence of the driver, 'yes' means the driver is present in the cabin. (Partially solved by the solution of the previous challenge (Part 1))

      • driver_smoking: whether the driver is smoking. (Partially solved by previous challenge)

      • driver_phone: whether the driver is using his mobile phone.

      • moving: the truck's moving state, 'yes' corresponds to moving, 'no' corresponds to stopped.


    A note on phone usage: as long as the truck is moving and driver holds the phone in his hands, he is non-compliant to standards. But using the phone in a hands-free way (e.g. listening to music using earphones) is compliant. The 'driver_phone' annotations were created using this rule in mind.


    In events.txt you have to report all events when any of these parameters change. In case of using mobile phone it is not possible to tell exactly when these events start and end, different human annotators are very likely to set these timestamp differently. However there should be good agreement on the fact whether the driver was using mobile phone at all in a longer period of time, say, in the past minute. (The same is true for smoking but this is not in our current scope.) Because of this your tool's performance will not be judged on whether it reports these events at the exact right time, but it should be able to detect the presence or absence of them. See the Evaluation section for details on this.

    Data files

    Very Important: Participants are supposed to delete the Video contents from their personal device after the challenge is over and this data is not allowed to be shared with public as well as with any other person or systems.


    Raw video files and corresponding annotations are available from the challenge forum after registration. (The videos are the same as in the case of the previous challenge.) Some videos feature normal daily operations, others show simulated scenes where people follow prescribed behaviour patterns. (Note that some of the simulated videos show unrealistic scenes like people using various objects instead of a real mobile phone like phone case, hard drives etc. You may use these videos for training but we will use more realistic videos for evaluation.)


    Please study the annotation format carefully, it is expected that your tool creates the events.txt output file in the exact same format:

    • A comma-separated text file.

    • First element of each line is time in mm:ss format, from the beginning of the video.

    • The rest of the elements are either:

      • driver:{yes,no}

      • driver_smoking:{yes,no}

      • driver_phone:{yes,no}           

      • moving:{yes,no}     

    • The first line should describe the initial state of the video at time 00:00, it should list all the parameters in their initial state, e.g.:

    • The rest of the lines will typically contain a single parameter change, e.g. if the driver starts using his mobile phone at 1 min 30 sec:

    • There may be multiple elements present for the same time stamp, e.g. if the driver enters the cabin while smoking at 1 min 30 sec:

    • The published annotation files contain additional parameters (passenger_count, p_smoking, p_phone), these describe the presence and behaviour of the passengers in the cabin. These can be ignored in the current project and need not be present in your output files.


    Based on these low level parameters described above the compliance can be calculated in one of the following ways: at any point of time non-compliance is detected if any of the following was true in the previous 15 seconds:

    1. 'driver_smoking' is 'yes',

    2. 'driver_phone' and 'moving' are both 'yes' at the same time,

    3. either a) or b) is true.


    In the previous challenge method a) was used to detect non-compliant behaviour. In the eventual end system of the client method c) will be used. However, for evaluation purposes of the current challenge we will use both a) and b), and calculate compliance separately for smoking and phone usage.


    Your submitted tools will be evaluated by running them on new videos. The compliance states calculated using the output of your tool will be compared against compliance states calculated from hand made annotations. To measure submission quality we will use the F-score metric  which considers both the recall and precision of the algorithm, i.e. it awards high recall (catching a high number of non-compliant cases) and also high precision (not making too many false alarms).


    For calculating the F-score we shall use 1-second long time windows and count True Positives (number of seconds when both your tool and the expected annotation reports non-compliance), False Positives (you report non-compliance but it is not expected) and False Negatives (there is non-compliance but your tool doesn't detect it).


    To be eligible for prizes

    • you need to achieve an F-score of at least 70% using method b), and also

    • you need to achieve an F-score of at least 55% using method a). (This corresponds to the best performance from the previous contest.)


    The prizes are as follows: 1st place: $3000, 2nd place: $2000. No prizes will be paid if the above 2 criteria are not met.

    In addition to the main prizes  

    • we'll pay a $1000 bonus to the winning submission over 1st prize, if it achieves a 85% F-score using method b) (mobile phone detection)

    • we'll pay a $1000 bonus to the winning submission over 1st prize, if it achieves a 75% F-score using method a) (that is, it significantly improves smoking detection).

    Final Submission Guidelines

    The submission package should include the following items.

    • A document in PDF / Word format to describe your approach.

      • It should be minimum of 2 pages.

      • It should be written in English. You’re not being judged on your facility with English. We won’t deduct points for spelling mistakes or grammatical issues.

      • Leveraging charts, diagrams or tables to explain your ideas is encouraged to complement your proposal.

    • The output (2 txt files per video) your tool generates on the following videos:

      • ch06_20180603131655_20180603133847.mp4

      • Mobile Device and Smoking.mp4

    • Your implementation and deployment guide.

      • We need a step-by-step instruction about how to run your code, including description of all dependencies.

      • The code may be implemented in any programming language, however the stakeholders of this contest have a strong preference to Python or R.

    • Your source code and build scripts.

    • All dependent 3rd party libraries and tools. (Alternatively pointers to them if their size is large.)

    • If your solution includes prebuilt models (like neural network weights) then make sure we can run your tool without having to run training first. So either include or link to your hosted model files. But make sure that we can reproduce your training process as well.

    • Code requirements.

      • Your code may use open source libraries as long as they are free to use by the client in a commercial deployment. Apache2, BSD and MIT licenses are permitted. GPL and LGPL are not permitted. All other libraries require permission. To request permission, post your request to the challenge Forum.

      • The eventual end system of the client will run in real time. This means that processing a video should not take longer than the running time of the video.

      • The end system will NOT contain a GPU, you must make sure that the target performance can be achieved using a CPU-only solution.

      • Your solution must process the video sequentially and must not look ahead into future video frames.

      • Your tool will eventually be run in the client's MS Azure environment, you must make sure that this will be possible to do. This requires that you check that none of the libraries that you are using has any known problem in Azure.

    Reliability Rating and Bonus

    For challenges that have a reliability bonus, the bonus depends on the reliability rating at the moment of registration for that project. A participant with no previous projects is considered to have no reliability rating, and therefore gets no bonus. Reliability bonus does not apply to Digital Run winnings. Since reliability rating is based on the past 15 projects, it can only have 15 discrete values.
    Read more.


    Final Review:

    Community Review Board


    User Sign-Off


    Review Scorecard