DISQUS

John Schultz · 5 months ago

Aaron, saw this come through on Twitter, and realize it's an old post that is still very much relevant today. First off, you're right on the mark!! That said, my experiences in Minnesota may be of assistance to some associations.

We currently have five outside judges (typically NAA reps, auction school reps, champion auctioneers from other states, and sometimes general public) judge the contest. All scores are kept (i.e. the high or the low is never discarded). The judges score on paper score sheets that are collected after each contestant and provided to two official scorers. These scorers are sequestered and one person adds the scores with a calculator and the other inputs them into a spreadsheet that auto calculates the scores. The two scores are compared for accuracy. Any differences result in a recount until the scores match. This method has proven to be an inexpensive method to score the contest, while providing near instant tabulation and results. Not to mention relieving the otherwise strenuous task of manually scoring 40 score sheets.

As far as dropping scores, several years ago I compiled all the scores for the last 5 years. I ran three analysis on the scores: keep all scores, drop lowest score, and drop highest score. In all three methods, the champion did not change. Our five finalists changed on a few occasions (5 finalists, sometimes the 5 and 6 would flip flop), but never the champion. In fact, as I reviewed the scoring habits of judges, I noticed that some judges just score lower but are consistently across the board lower (a few years ago we had a judge that didn't score anyone above a 70, and no lower than a 50). Allowing each judge to establish their own standard deviation makes for a fairer contest than tossing scores. If they're good enough to be trusted as our judge, their scoring methods certainly should be trusted.

Aaron Traffas · 5 months ago

Thanks for the comment, John. It's good to hear about some actual data regarding the subject. It sounds like you've got things figured out with both foreign judges and electronic tabulation.