 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
|
|
|
|||||||||
| |||||||||||
|
|||||||||||
| |||||||||||
|
|||||||||||
|
CDDC 2 - What the reviewers saw
05.04.06 7:25 PM  By dplass TopCoder Member [Before I start, I'd like to give an update from the CDDC Round 1. In the forums, duner was wondering what the outcome of the round was. As it turned out, the relative placement of the design entries was the same -- nicka81 in first, adic second, followed by Pops in third. However, for development, the ordering did in fact change. After appeals, visualage came in first (he had been third), sindu in second (as he was before) and colau was in third, having dropped from first. I guess the "lesson learned" really shoujld've been, "It's not over until the For the second set of CDDC appeals, I decided to take a different tack - from the reviewer's point of view. I went backstage to the war room and observed, and interacted with, the three development reviewers, cucu, argolite and UFP2161, who were resolving appeals for both the Java and the .NET (C#) implementations of the Testing Framework component. Note: I will be concentrating on development only for this blog entry. The way the appeals process works mechanically is almost identical for a reviewer as for a developer or designer. When an appeal is filed, an email goes to the appropriate reviewer, who opens (or refreshes) the Project Submit & Review page (a.k.a. "View My Open Projects"). In the list of appeals, all the appeals for a single projects are grouped together (and a reviewer can only see the appeals to his own review, not the other reviewers.) The reviewer can look at the appeal, and (new for the TCO) accept or deny it, as well as write a comment back. Of course, they are free to change the number of points assigned to that review question, and/or change an item from recommended to required (or vice versa.) For the sake of the TCO, when an appeal is accepted or rejected, it doesn't necessarily mean that the point score for that question has gone up or down, but whether or not the reviewer agreed with the appeal or not. Looking at the "big board" for development, sindu started out in 1st place, followed by biotrail and traugust. However, the scores were very close — less than two points from first (98.49) to third (96.07). Behind the scenes, the three reviewers felt it was a fairly straightforward component, much less difficult than yesterday's component (Document Indexer Persistence) or tomorrow's (Bread Crumb Trail). As a result, UFP2161 said it would be a "light day". This obviously makes the reviewer's job easier, even if there is a large volume of appeals, they can often be resolved quickly. Yesterday I noticed that the developers all submitted appeals for the reviews by UFP2161 and cucu before touching argolite's. He noticed this, and said that it was 45 minutes until his first appeal yesterday! Today, though, they came through right away and steadily for all 3 reviewers, which meant constant work. An added source of pressure is that for a normal component compeitition, reviewers have 24 hours to resolve appeals, but here the time is much more limited. A technique that argolite said he uses is to do the easy ones quickly, and "batch up" the harder, more detailed responses for the quieter times so he can spend enough time to write a coherent thoughtful response. Possibly the most amusing appeal (maybe in the history of TopCoder?) I saw was one where the developer had forgotten to remove an unused private method, which caused him to lose a point for that item. He wrote in his appeal "Arrrgh. Silly me. This is not an appeal. Just complaining so I feel better. Thank you." To which cucu replied "Thank you for writing an appeal that doesn't make me think or go digging in code." Who says reviewing can't be fun?! While I was in the war room during all this, all of a sudden we heard this teeny, squeeky voice saying something like "Welcome to the TopCoder Open 2006 Algorithm Wildcard Round." Turns out, ntrefz was testing the sound system (which you should not touch, ever, no matter what happens, or who tells you to, by the way) and, oops, had the wrong speed setting on the playback. jmpld40 thought it would be funny to really use it (as a joke) in one of the rounds. I'm not sure which is funnier — using that munchkin voice, or the booming Darth Vader voice scaring the bejebus out of us, interrupting the algorithm rounds with "Attention please! There are two minutes left in the coding phase."!) Wish you were here, — dplass |
    |
