June 22, 2016 Topcoder News: Scorecards and Bug Hunts
For some time, we have realized through our forums and conversations with CAB, that there have been inconsistencies when it came to processes either related to reviewing or simply the execution of a challenge.
A few weeks ago, an email was sent to all copilots and reviewers in an effort to unify the thoughts of how some challenges should be run and how certain sections of the scorecards are to be interpreted. We are sharing this with the entire community because we feel that transparency is a key to success and you will always be able to refer to this blog post if you seem to be running into these specific scenarios.
Scorecard – Grammatical Errors
This has been a hot topic for some time as some members wanted to see it removed and some felt it necessary. The description of this scorecard section states:
“Grammatical errors, spelling mistakes, formatting and copy-paste mistakes are to be scored here. Also pay attention to typos in file names.”
Here is the final word: Spelling and grammatical errors should be judged appropriately when it comes to file, class, and function names. Reviewers should not deduct points if these errors occur in comments or anywhere else in code. These grammatical errors however should be fixed by the winner in final review.
In our email, we had notified challenge reviewers that it had been reported that some bugs in a Bug Hunt challenge may have been accepted even though the name and/or description of the bug was similar to another, which would cause a duplicate. It has to be understood by the submitters however that bugs do get submitted at a fast rate and there are typically many of them for the reviewers to review in a short amount of time. With that being said, we have informed the reviewers to make a conscientious effort when accepting a new bug.
CAB has also brought to our attention that members have felt that the Bug Hunts are unfair because they all seem to be released around the same time of day. The issue here is that because we have members from all over the world, some feel that they are at a disadvantage if they are unable to submit and join early.
In order to address this, we have asked copilots to make an effort and mix up the release times of Bug Hunt challenges. Not only that, but copilots should launch Bug Hunt challenges 24 hours before it goes live with just a title for the challenge (no challenge description). Members will not be able to register for this challenge during this time window, but it will allow everyone time to plan and decide whether they want to participate. When the challenge launches 24 hours later, everyone will be on the same playing field. Everyone must understand however that sometimes customers need bug hunts to begin as soon as possible and in those situations, copilots are not required to adhere to the guidelines described in this paragraph.
Thank you for taking the time to review this announcement! We feel that efforts such as this will create less conflicts between members and reviewers as well as creating a sense of fairness in the Topcoder environment. If you have any specific comments or questions, feel free to visit this thread and discuss!