One seat per product: open-weight twins, FP8-native checkpoints, and the twelve models that lost their place

Identity is the part of a leaderboard nobody sees and everybody depends on. A ranking is only meaningful if each row is one product, the product the evidence was measured on, and if that product appears once. Publishers ship the same model three or four ways, so the board spends a lot of code deciding which repository, route and alias are the same thing. This note covers two rules that work and one that did not.

Open-weight twins of API products

A publisher may ship one product twice: a closed API row with no Hub id and an open checkpoint with one. Qwen's 3.8 Max and its 2.4T A95B checkpoint are the canonical example. Nothing structural links them, so both would take a seat and the lab would be counted twice. Two rules close the gap.

• Declared fold. When the catalogue description carries an exact same-publisher link preceded by an equivalence phrase, “open-weight variant of”, “same model as”, “fast-mode variant of”, “identical to”, the row folds under that product like a reviewed registry entry. “Based on”, “update to”, “distilled from” and “successor to” never fold; they describe a different model. The fold is a pure function of persisted fields, so a recompute agrees with the sync that wrote it.
• Sibling proposals. Same-publisher pairs that both stay canonical while a licence slug, a model-card sentence or a description phrase says they are one product are written to a proposals file and raised as an alert. A human adds the pair to the equivalent or distinct registry; the proposal never folds on its own.

Direction follows the declaration: the open checkpoint folds under the API product, which keeps its evidence and its seat and does not inherit the open-weights flag.

The rule that unseated twelve products

The release-authority code decides whether an official-namespace repository is the product or a distribution build of it: a GGUF, an AWQ, a community requant. A distribution build never takes a seat; the product it repackages keeps it. Until v9 the test was a pattern over the repository's tags, and the pattern included fp8 and mxfp4. That was correct when FP8 meant “a smaller copy beside the real one”. It stopped being correct when labs began shipping the only checkpoint natively in FP8 or MXFP4. The product itself was labelled a distribution build, its canonical flag was cleared, and no other row existed to keep the family's seat. The public page read “No Index position · visible for discovery” for a model that Arena and Artificial Analysis had measured.

Twelve current measured products, forty-one rows in total, silently off the board. Nothing alerted because from the code's point of view nothing had failed: each row had been classified, correctly by its own rule, as something that should not have a seat.

The fix

The v9 release inverts the test. A storage-dtype tag on a clean-named official repository is not packaging. Only a packaging-named repository corroborated by a declared parent or a packaging tag is a distribution build: the name has to say “this is a repack” and something else has to agree. A lab's sole checkpoint is the product regardless of the dtype it is stored in; a repack beside an unquantized sibling is still a repack. Twelve products regained their seats when the registry change was replayed offline, and the derived data, evidence ledger and benchmark plan were regenerated together, because an identity change touches all three.

The current rules are in the release-authority section of the methodology. If you publish a model and believe it is folded or classified wrongly, the contact page says what evidence settles it.