Judging & Scoring

We spent a lot of time thinking about how to keep Node.js Knockout judging fun, fair, and exciting. Here's how it’s going to work:

Voting

There will be two groups of voters: expert judges and the general public.

All voting will take place at the same time: between 03:00 GMT on Monday, August 30 and 0:00 GMT on Friday, September 3.

Expert Voting

We have an amazing and diverse panel of expert judges who are volunteering their time to provide thoughtful feedback on your weekend’s work.

Each expert judge will be asked to review an initial batch of 5-10 entries. After they do their initial batch, judges will be encouraged to review a handful more of the top entries that pique their interest. Experts will not be allowed to vote for people who they know.

In order to ensure all expert judges are anchoring similarly, they will be asked to grade along a curve with a mean of 3.5.

The hope is that this will allow every entry to get at least 2-3 thoughtful expert reviews and that the best entries will get enough reviews that the law of large numbers ensures fairness.

Public Voting

Any member of the general public may vote. You are encouraged to get as many people as possible to vote for your entry. Voting will be very easy (yes, your mom will be able to do it).

General public voters must meet the following requirements:

They need a valid e-mail address (we will ask them to verify their votes by e-mail).

We will implement a number of mechanisms to detect and to eliminate ballot stuffing, but the honor code applies (see the cheating section below).

Evaluation

Entries will be judged on a 1-5 scale across 4 dimensions:

Utility - Is the site offering a service you'd use again and again?
Design - How good does it look and feel to use?
Innovation - How original is the idea and execution?
Completeness - How "fully baked" is the site? Are there bugs or dead ends?

Calculation

A team's score will be determined by adding the expert judge score and the public score for each dimension.

In addition, there will be a popularity component derived from each entry's public vote count.

The overall score will be the average of all 5 components (utility, design, innovation, completeness and popularity).

Prizes

There will be the following 7 prize categories:

Overall - the best overall score
Solo - the best overall score for a one-person team
Utility - the highest utility score
Design - the highest design score
Innovation - the highest innovation score
Completeness - the highest completeness score
Popularity - the entry with the most public votes

A team may only win one prize and will earn the highest prize (in the above rank order) for which it qualifies, opening its spot in the running for lower prizes to the next most qualified team.

Cheating

This is a competition for fun. The honor code applies: we’re trusting you to do the honorable thing.

We will be implementing a zero tolerance policy towards cheating. If you are provably caught cheating, you will be disqualified immediately, be banned from all future events, and generally be thought of as a bad person by all of the organizers and potentially many, many, more people. Trust us, it's not worth the risk.

Closing Thoughts

There's no system of voting that will make everybody happy. Our decisions are based on lessons from dozens of other competitions and we believe that they're going to make for a great Knockout.

But it’s important to remember at the end of the day, even if you don’t win, you’ll still have had a great time, have built something awesome, and have gotten a ton of exposure for your idea.

Interested in judging? Contact us.