The refresh that retired its own number one

ModelCap has a “one row per product family” rule. When a publisher ships a successor, the older release drops off the current board and keeps only its archive rank, so the table does not fill up with a lab's entire back catalogue. On 1 September that rule met a closed API launch it had never been designed for, and for about ninety minutes the board was wrong in the most visible way possible. This is the write-up.

Timeline (UTC)

1. 18:03 OpenRouter lists anthropic/claude-fable-5.1. Discovery sees it within a cycle.
2. 18:21 The next full refresh admits it. The family resolver places both rows in the family anthropic::claude fable, sees that the newer sibling is being served, and marks Claude Fable 5, measured on seven boards and ranked #1, as superseded. Claude Fable 5.1 has no model card, no Hub repository and no independent benchmark yet, so it opens on the corpus prior for closed products: #53.
3. 18:21 to about 19:47 The board shows the wrong model at the top and no Claude Fable at all in the top fifty. No alert fires.
4. 19:37 Artificial Analysis lists the model. The next cycle measures it at 84.3 on one board with confidence 20, #1, with Claude Fable 5 back at #2 on 84.0.
5. 19:47 and 20:14 Two releases cut over with the generic fixes below. An interim hotfix that hard-coded the two Fable rows as concurrent had gone out first; it worked for this pair and for nothing else.

Why it happened

Supersession was keyed on serving: if a newer family member was available from a provider, it replaced the older one. That is a fine rule for open-weight releases, which arrive with a model card and a comparison table that gives them a start rank within the same refresh. It is the wrong rule for a closed API product, which arrives with nothing but a price and a context window. The refresh did exactly what it was told: it promoted the served successor and demoted the measured incumbent, then ranked the successor on the only thing it had, a prior.

The alert that should have paged did not, because the launch watchdog only watched releases whose release authority was official, meaning an official Hugging Face namespace. Closed API products carry the authority not-applicable and were invisible to it. A live audit that afternoon found four more families where the same silent retirement had already happened: Tencent's HY3, Inception's Mercury 2, Kwaipilot's KAT Coder Pro V2 and the dated GPT-4o routes.

What changed

• Supersession is evidence-gated. A served successor with no benchmark evidence of its own cannot retire the newest served sibling that has some. Until the successor is measured, both rows are current and the board shows two rows from one family. That is now expected for the days between a closed launch and its first Arena or Artificial Analysis listing.
• A succession start rank. Index version 4.7 adds a channel between the last peer-based placement and the corpus prior: when a launch has an older measured sibling in its family, it opens three points beneath that sibling's lower bound and point estimate, with confidence capped at 45 or half the predecessor's, labelled Modeled · succession. A Fable 5.1 with no evidence at all now opens near #2, not #53.
• Two new alerts. successor-start-on-prior pages when a watched launch opens on a prior despite having a measured predecessor; measured-incumbent-retired pages when a family keeps no current measured member. The watch list now includes served closed products, not only official Hub releases.
• Same-cycle admission. A candidate that first appears in a discovery cycle can trigger a full refresh after five minutes instead of waiting for the next scheduled one, bounded to newly-seen candidates so a stuck one cannot make every cycle full.

The lesson we kept

Every rule on the board that decides who replaces whom has to be keyed on evidence, never on availability. Availability tells you a product exists; only a measurement tells you what it is. The related rule for a successor that is measured but still thinly, the succession floor, is described in its own note, and the launch itself in the Fable 5.1 note. The current start-rank contract is in the methodology.