Why SWE-bench, BFCL and Terminal-Bench went dormant on the board, and what happened next

Independent benchmarks are the only thing that makes a ModelCap rank “measured”, and independent benchmarks have lives of their own. Maintainers move on, harnesses change, a board gets replaced by a newer version under the same URL. The Index has a rule for that, and in the summer of 2026 it fired on three boards at once. This note explains the rule, what happened to each board, and what happened next.

The dormancy rule

A board whose newest row in our corpus is more than ninety days old is dormant: it is excluded from scoring for every model at once until it publishes again. Its rows stay in the dataset and on model pages but stop counting, so a dying board cannot keep scoring a lucky subset of models. Ninety days is long enough to ride out a maintainer's holiday and short enough that a board which has stopped for good does not keep gatekeeping a family for a year.

What happened to each board

SWE-bench's official bash-only leaderboard simply stopped taking submissions in February; the row dates on our board show that honestly, and the benchmark page has said “February 2026” ever since. BFCL went quiet in the spring. Both are ordinary dormancy.

Terminal-Bench is the interesting one. In late August tbench.ai began serving its 4.0 board under every version URL, including the 2.1 one our fetcher read. Our parser expects a leaderboard with the pinned board's title, so it returned zero usable rows from 26 August onward and the source went to a stale-cache alert three days later. That failure was the correct behaviour: the 4.0 board is a different thing. Every row on it measures a model through a vendor's own agent, Claude Code, Codex or Grok Build, at an effort level the vendor chose. Those rows compare products, not models, and two rows for the same model can differ by ten points depending on whose harness ran it.

Why it mattered more than it looked

Coding carries 12 of the 100 points of weight and agent carries 5, so on paper three quiet boards should barely move a rank. In practice they made the specialist families hollow at the frontier: a new release could be measured on the general boards within days and then wait indefinitely for a coding or agent board that, for a closed model, might never arrive. On the day Claude Fable 5.1 launched, that was the reason it sat a few tenths of a point from its predecessor while the independent evaluations that had measured both showed a clear gap. The launch is written up in its own note.

What happened next

• On 1 September a first version of Score v9 admitted the Artificial Analysis runs of Terminal-Bench 2.1, Tau2-Bench Telecom and Humanity's Last Exam, one per specialist family, and capped any evaluator other than the Arena anchor at 30%. Neither survived the 4 September revision: those runs are reference evidence on model pages and feed no family, and there is no cap. Nothing has replaced the dormant boards in scoring. The first version is recorded in the Score v9 note.
• Terminal-Bench 2.1 (Terminus 2) stays in the coding plan but has been dormant since 7 September 2026. The served Terminal-Bench 4.0 board is read from the page's embedded payload and shown as display evidence with the harness and effort named in each row's configuration, never scored. The fetch fails closed if the served board's name changes again.
• Source lifecycle alerts now warn fourteen days before a board's dormancy horizon and when a family's live coverage drops, and a source whose fetch has failed for three days escalates to critical. A board dying quietly is what this summer taught us to watch for.

The current board list, with each board's family, weight and newest publication date, is on the methodology page, and each board has its own leaderboard page under benchmarks.