Score v9 as first shipped (1 September 2026): three fixed-harness boards and a 30% evaluator cap

The ModelCap Score supplies the measured point estimate to the v4 Index: a weighted combination of a model's results across four evidence families, general, coding, agent and reasoning, each fed by named public boards with named shares. This first version of Score v9 shipped on 1 September 2026. It changed what the coding, agent and reasoning families were fed by, and it added a structural limit the score had never had.

Why v9 was made

When Claude Fable 5.1 launched, the two products of that family sat a few tenths of a point apart on the board while the independent evaluations that had already measured both showed a clear gap: 65.7 against 62.1 on the Artificial Analysis Intelligence Index, 91.4 against 84.6 on a fixed-harness run of Terminal-Bench 2.1. The gap was invisible because the specialist families were scored only on boards that had not measured the new release, and in two cases had stopped publishing altogether. A frontier release was waiting on upstream boards that, for a closed model, sometimes never arrive. That story is told in the dormancy note.

Artificial Analysis publishes per-evaluation results alongside its index. Those results are third-party measurements of a model the publisher does not control, which is the class of evidence the score already admitted for the index itself. The first v9 admitted three of them, one per specialist family.

Family sources in the first v9

Family weights were unchanged. The three new boards were percentage boards, pass rates from 0 to 100, so they took the same 80/20 split between placement and raw achievement that SWE-bench and ARC already used. Each fed only the family it measured: a terminal pass rate says nothing about reasoning.

The organisation share cap

Admitting four Artificial Analysis boards raised a question the score had never had to answer: how much configured source weight should one evaluator be allowed? The first v9 answered it explicitly. Every entry in the source plan named the organisation that runs the board, an organisation's share was the sum over its boards of family weight times source share, and no organisation other than the Arena anchor could exceed 30% of that configured total.

The cap was asserted when the scoring module loaded. A plan entry without an organisation, or a non-anchor organisation above 0.30, threw before any score was computed.

The cap constrained the configured source plan. It did not guarantee a maximum change in a model's score or rank if a source changed or disappeared. Family scores normalize over the supported observations actually available for that model. If only one general-family source has measured it, that source can supply the whole observed general estimate despite having a smaller nominal share. Changes in evidence support can also change the Index interval.

Also in the first v9

• Terminal-Bench 2.1 Terminus 2 was retired from scoring, and the served Terminal-Bench 4.0 board was carried as display evidence with the vendor harness and effort named on each row, never scored. Score v10 keeps Terminal-Bench 2.1 (Terminus 2) in the coding plan, where it has been dormant since 7 September 2026.
• The Artificial Analysis evaluation boards date no individual row, so each model-and-board pair keeps the fetch instant that first showed it as its publication time, advancing only when the value or placement moves. A board 223 models wide is a field of 223, not the whole index.
• A board the payload no longer carries publishes as an unavailable source with zero rows; retained rows keep scoring until dormancy, and the lifecycle alerts make the loss visible long before that.
• Release authority stopped demoting a lab's only checkpoint for being stored in FP8 or MXFP4; twelve first-party products got their board seats back. That change is still in force and has its own note.

The current score section of the methodology describes Score v10, and the method changelog there lists every score version.