I scored more and lost: how the set system works

Part of the complete guide: How to improve your archery score

The board says 4-6 and the scorecard says 142-136. The archer who lost put six more points on the target than their opponent, shot 30, 27, 28, 30, and 27 across the five sets against 26, 28, 29, 25, and 28, and goes home eliminated. Nothing strange happened and nobody miscounted. The system worked exactly as it is supposed to.

Understanding why means looking at how the arithmetic of a match is built, which since 2010 has not been the arithmetic almost everyone still carries in their head. From there you reach the questions that actually matter: how often something like this happens, whether it happens more to the strong or to the weak, and whether the format does what the federation said it wanted when it adopted it.

The figures that follow come from work I published in the Lab, where the calculation is set out in full and the code to redo it can be downloaded.

How a set match is counted

The rule is simple to state. The match is broken into sets of three arrows; whoever scores higher in a set takes two set points, and if the set is tied they take one each; the first archer to six set points wins the match. Since set points accumulate and never come back, the match ends the instant one of the two touches six, which can happen in three sets, four, or five, meaning after nine, twelve, or fifteen arrows.

Before 2009 the counting worked another way, the one recurve no longer uses but compound still does: fifteen arrows each, add them all up, highest total wins. They are two ways of reading exactly the same arrows, and on one identical competition they can return two different verdicts. The set system entered individual recurve in 2009 and became the format of the whole senior circuit the following year, and in that first season it was not yet uniform, because the earlier elimination rounds shot sets of six arrows and three-arrow sets applied only from the quarterfinals on.

Two reasons were given. Under the old system a contest could be essentially lost after a single bad end, because one disastrous arrow was enough to make the total unrecoverable, and the set format was adopted to stop that, as well as to reward consistency and to keep the match open to the finish. Whether it got what it wanted is a question that can be answered with a number, and I come to it shortly.

Where the leftover points go

The mechanism that produced the scorecard at the top of this article sits entirely in one property of the rule: inside a set, winning by one point is worth exactly as much as winning by ten. The margin is wiped out at the boundary of the set and does not carry over, so a set won 30-25 and a set won 28-27 are worth the same thing, namely two set points.

How much gets thrown away this way can be measured. In the band where the format weighs most, meaning when an archer averaging 692 points meets one averaging 675, a set that is won is won by 1.76 points on average, and in more than half of cases by two or more. Between two identical archers the mean margin falls to 1.41. That information, repeated at every set, is what cumulative scoring keeps and the set system discards.

In the same pair the stronger archer takes the individual set 55.5% of the time, ties it 24.7% of the time, and loses it 19.8%. The fact that one set in four ends level matters more than it looks, because a tied end distributes one point each and therefore separates nobody: it is a stretch of match that runs by without producing any information useful for deciding who is the better archer.

What counting this way costs

The question the essay asks is not whether the set system is fair, but how often it points to the more precise of the two archers, and how it behaves against the old count on the same arrows. The difference can be calculated exactly, without simulating competitions, and the result is that the set format does cost precision, but not much.

At the point where the gap is widest, an archer at 692 against one at 675, cumulative scoring identifies the more precise archer with probability 0.870 and the set system with 0.847. Translated into matches, out of a hundred contests the cumulative count gets it right 87 times and the set system 85: two matches in a hundred change hands, one in forty-three.

The cost is not the same at every gap, though, and the shape of the curve says something that frees a coach from a common misconception.

The disadvantage, then, is not greatest between near-level archers, as you might suppose, but in the middle. The reason, once stated, is almost obvious: when two archers are nearly identical no format in the world can do better than a coin toss, because the contest really is a toss-up and settling it at random is not a measurement error; and when one is clearly stronger, every rule recognises it without effort. Only in the middle band does the choice of format have anything to lose.

The absolute level of the two contenders matters as well, and it matters in the convenient direction. The peak cost is 1.99 percentage points between archers at 700, rises to 2.32 between archers at 692, to 2.63 between those at 680, and to 2.97 between those at 660: between a world-class match and a good national-level match the cost of the format goes from barely two points to almost three. At the top almost every arrow finishes in the ten, sets resemble each other, and the two arithmetics converge, which means the format costs least exactly where it is used most.

The match ending early is not the problem

An explanation you often hear puts the cost down to brevity: a set match can close in nine arrows instead of fifteen, and fewer arrows mean less information. In the reference pair the average duration is 4.268 sets, meaning 12.80 arrows rather than 15, so a few arrows genuinely do go unshot.

The calculation dismantles that explanation decisively. Force the two archers to play all five sets even when one has already reached six, and the probability of winning does not change by a thousandth: the two versions return the same value to the last digit the machine can write. The reason is arithmetic, because five sets distribute ten set points in total, and the loser never gets beyond four set points against the six or more of the winner. The arrows the set system leaves unshot are exactly the ones that could not have changed the verdict.

There is a second piece of evidence, and it runs against intuition. If instead of shortening the match you lengthen it, the gap between the two formats grows: with a single set it is exactly zero, because there is no internal boundary to cross and the two arithmetics coincide; with three sets it is 1.84 percentage points, with five it is 2.32, and it keeps rising to around six sets before coming back down, because at twenty-one sets both formats are close to infallible and there is no room left for a difference. The cost, in other words, is born at the boundaries between one set and the next, and the more boundaries there are the more margin gets thrown away.

How often the lower total wins

The scorecard at the top of this article has a name and a frequency. Between two essentially level top-flight archers it happens in 7.1% of matches counting the arrows actually shot, and in 7.6% if you imagine all fifteen being shot: about one match in fourteen.

The frequency changes with the gap, and it changes in the way you would expect.

Here is the thing worth taking away, because it runs exactly counter to the indignation the scorecard produces. The reversal is most frequent precisely where the format betrays nobody: it grows the closer the two archers are, and its maximum falls near equality, where the cost in precision is zero. Between two identical archers reversals are very common indeed, one in fourteen, and not one of them is an error, because there is no more precise archer to betray and whoever wins, the verdict is legitimate. As the gap widens the reversals thin out, down to one in sixty-nine.

The reason is geometric. For the winner to have posted the lower total, the two totals have to be close enough to swap places, and that happens when the two archers resemble each other. When one is much stronger their total sits a long way from the other's, and chance is no longer enough to make them cross.

Six matches in a row: what happens to a whole tournament

A match is a match, but a competition is made of six rounds of single elimination, and winning takes surviving all of them. So the question can be asked again at tournament scale: how often does the favourite reach gold under one format and how often under the other, and where does the probability lost along the way end up?

Across a six-round bracket the direction never changes, in any of the fields of competitors built for the test: cumulative scoring takes the favourite to gold more often than the set system does, by a difference running from 3.3 to 4.2 percentage points. In a field whose level is spread out regularly the favourite wins gold 46.7% of the time counting by points and 42.5% counting by sets.

The most useful thing, though, is the comparison between that difference and another factor. If instead of the format you change the composition of the field, keeping the number of competitors the same but varying how close together they are, the favourite's probability of gold moves from 31% to 58%: twenty-seven percentage points against the four that the arithmetic is worth. Who else is in the competition weighs almost seven times more than the way the points are counted, and that cuts a fifteen-year-old argument down to size.

One last question remains, the one nobody usually asks: the four points the favourite loses by moving to the set system, where do they go? A tournament awards a gold medal either way, so that probability does not vanish, it moves. The top seed loses 4.2 of it, the group from the second to the eighth seed gains 1.9, and the group from the ninth to the thirty-second gains 2.3. The set format transfers probability from the favourite to the outsiders, and transfers more of it to the distant group than to the one immediately below.

Consistency, which the format was meant to reward

What remains is to settle up with the original promise. The federation wanted two things: to stop one unlucky arrow deciding a match, and to reward consistency. It got the first, and the calculations confirm it, because the format confines the damage of an arrow inside the set where it lands. On the second the calculations say the opposite.

The measurement is made by setting two archers with the same expected score against each other, one of whom shoots steadily while the other occasionally places a ruinous arrow. Under cumulative scoring the steady archer wins 48.5% of matches; under the set system that falls to 45.4%, and the direction does not change at the top or further down. About three percentage points of advantage therefore pass from the archer who shoots consistently to the archer who occasionally throws one away, and it is the direct consequence of the protection the format offers: isolating the catastrophic arrow inside the set where it lands means protecting whoever shoots it.

That same property is the one spectators like, because a format that protects the archer who errs produces more open matches, less predictable outcomes, and more frequent upsets, which is the third thing the federation said it wanted. On that front the set system works, and works for exactly the reason it measures worse: they are the same property under two different names, and which of the two to use is not something a calculation can decide.

What the calculation does not look at

All these numbers come out of a model in which an archer is the width of their group and nothing else, arrows are independent of one another, and the form of the day does not exist. No real archer's score enters the calculation, and the archers are synthetic profiles defined by their expected score, so the conclusion is about the competition format rather than about a population of athletes.

On one point the model turned out to be fragile in an interesting way, and the essay declares it. Allow that 5% of arrows are anomalous, meaning drawn from a dispersion three times wider than normal, and the cost of the set system falls from 2.32 to 0.68 percentage points, less than a third. The reason is that a thrown-away arrow does more damage to a cumulative total than to a set format, so the advantage of the cumulative count shrinks precisely when anomalous arrows really exist. How frequent they are among real archers nobody knows, because establishing it would take the coordinates of the impacts rather than the rings alone.

On other fronts the result holds up well. Changing the shape of the group, moving from a circular dispersion to a vertical ellipse or displacing the centre by three centimetres, moves the cost by a few hundredths of a percentage point, from 2.32 to 2.34 and 2.36: the verdict does not depend on the geometry of the group. Making the centre denser does matter, because taking the share of Xs from 22% to 30% at unchanged score raises the cost from 2.32 to 3.13.

What to do with it, on the shooting line

The most immediate practical consequence concerns the Monday after a defeat. If you lost a set match having outscored your opponent, and your opponent was at a level similar to yours, that scorecard is worth little as a technical diagnosis, because it describes the situation in which the format reverses most often and in which no format could do better. Rebuilding your technique on the strength of that result means correcting something that may not have gone wrong.

The scorecard becomes informative when the gap is wide, because reversals are rare there: losing on sets to a much weaker archer having outscored them is an event that happens about once in sixty-nine, and it is worth working out what happened in the individual sets. The same logic applies in reverse to the winner: a win on sets against an equal is far less telling than it looks about the relative merit of the two.

There is finally a consequence for preparation, which follows from the set probabilities above. If a quarter of ends finish level and the mean margin by which a set is won is under two points, then in the set format ends decided by a single point matter enormously, and one arrow recovered in the right set is worth three recovered in a set already lost. It is why consistency end by end weighs more in a set match than it did under the old count, even though the format, as we have seen, rewards the consistent archer less over the match as a whole.

Where this goes next

The calculation about the format is only half the story, and not the bigger half. The instrument that decides who meets whom, namely the qualification round, carries a far larger uncertainty than the effect discussed here: between two neighbours on a ranking the typical gap is four tenths of a point, and over a gap like that 72 arrows point to the better archer barely more than half the time. I have written about it at length in the article on how many points it takes to say one archer is better than another.

The essay carries the full calculation, the cost curve gap by gap and level by level, the robustness tests on the shape of the group, the chapter on 64-archer tournaments, and the code package with which anyone can redo the sums and check whether the published numbers are the ones the calculation really produces.

Questions I get asked most

How does the set system work in archery?
The match is broken into sets of three arrows. Whoever scores higher in a set takes two set points, if the set is tied they take one each, and the first archer to six set points wins the match. The match therefore closes in three, four, or five sets, meaning nine, twelve, or fifteen arrows. The sum of the arrows does not count: the sets count.

Why did I score more and still lose the match?
Because inside a set winning by one point is worth as much as winning by ten, and the margin is wiped out at the set boundary. An archer who wins the close sets and loses the wide ones can finish the match ahead while having scored fewer points. Between two level top-flight archers it happens in about one match in fourteen, and further down the levels it passes one match in ten.

Is the set system less precise than cumulative scoring?
Yes, but only slightly. At the worst point, between an archer at 692 and one at 675, out of a hundred matches the cumulative count identifies the more precise archer 87 times and the set system 85: two matches in a hundred change hands. The cost is zero between level archers, greatest at intermediate gaps, and falls again once the gap is wide.

Does the set system really reward consistency?
No, the calculations say the opposite. Between two archers with the same expected score, one steady and one who occasionally places a ruinous arrow, the steady archer wins 48.5% of matches under cumulative scoring and 45.4% under the set system. The format isolates the catastrophic arrow inside the set where it lands, and so shelters the archer who shoots it.