Scoring and Ranking¶

Each registration pair completed by each participant is scored in four separate categories as described on the evaluation page The purpose of this page is to explain how those scores are combined into averaged scores and rankings.

Ranking - General¶

In all cases a low score is better than a high score. Therefore scores of 2.1, 3.5, 6.7, 8.2 will be ranked 1, 2, 3, 4 respectively. Where scores are equal, rankings are averaged. E.g. scores of 2.1, 3.5, 3.5, 8.2 will be ranked 1, 2.5, 2.5, 4 respectively. Similarly scores of 3.5, 3.5, 3.5, 8.2 will be ranked 2, 2, 2, 4.

Ranking in individual categories¶

For each scan-pair and each scoring category, the scores of a participant are compared with the scores of all other participants on the same scan-pair and in the same category. Rankings per scan-pair per category are then calculated. These rankings are shown on the per-category tables (last column) on individual participant results pages. If a participant did not complete a particular registration then no scores are available and the maximum ranking possible is applied. (If several participants did not complete the same registration then the ranking will be averaged among them). The average ranking per category for a participant is then calculated by averaging over the rankings for all scans. This is shown in the bottom right cell of the per-category tables on individual participant results pages. Note that this average ranking is NOT directly connected to the average score in this category (See "Average scores in individual categories").

Average Rankings Overall¶

An average ranking overall is determined for each participant by averaging the rankings from each of the four scoring categories. This average overall ranking determines the final placement of the candidate in the challenge. This ranking is shown in the bottom right cell of the Summary table on each participant page, and also on the main results page for all participants.

Average scores in individual categories¶

The last row of the per-category tables on individual participant pages shows the average scores in each category for the participant. If one or more scan-pairs were not registered by the participant then the average scores will be calculated based only on the scan-pairs that WERE completed. This means that a participant may have a relatively good average score (for example if he completed just one registration but did it very well) and yet have a very poor ranking (because maximum ranking will be applied for each scan-pair that was not registered). There is no average score overall since there is no clear way to average scores from different categories.