How Sybil reads a profile

Shown on every match and on a live lobby. It answers one narrow question: based on the ten people in this game and the heroes they are on, which side is on more familiar ground?

It is called game advantage rather than draft advantage on purpose. Nothing about the draft as players normally mean it is in here — no lane matchups, no counter-picks, no composition, no bans. Two teams of identical players on identical heroes score identically no matter how the picks line up against each other.

How it is calculated

1. 1. Take each player's adjusted win rate on the hero they picked, computed from their games before this match started so the game being described is never part of its own evidence.
2. 2. Clamp each one to 30–70%, so a single hero record cannot run away with a number that is only ever one fifth of a side.
3. 3. Average across all five players on each side.
4. 4. Combine the two averages with Bradley–Terry: convert each to log-odds, take the difference, convert back — plus a fixed term for Radiant's advantage.

P(radiant) = σ( logit(avg_radiant) − logit(avg_dire) + 0.13 )

That last term is the only fitted number in the model, and it is not about the players at all. Radiant wins a bit over half of all Dota games — easier Roshan, better warding angles, a friendlier jungle — so two evenly matched sides read as roughly 53% Radiant, not 50%. That is not a bug in the display; it is what an even game is actually worth.

It sits at the match level rather than inside anyone's adjusted win rate, because a player's history is split roughly evenly between the two sides and is therefore already side-neutral. The advantage belongs to the game being played, not to the people playing it.

Priors: the hero, not a coin flip

Each player's record is shrunk toward the hero's own win rate in the bracket this match was played at, not toward 50%. That matters most where it is needed most: a random lobby is mostly people with a handful of games on what they picked, and shrinking those toward a coin flip claims an unfamiliar hero is an even proposition whether the hero wins 44% or 53%.

A player we cannot read at all — private profile, anonymous slot, or simply no games on that hero — counts at the hero's bracket rate too. We still know which hero they locked in, and that is a great deal more than nothing. Leaving them out of the average entirely, which this used to do, is badly wrong: a side with four private profiles and one visible player on 60% became a 60% side, and then "outdrafted" a fully readable team of ordinary players. A flat 50% fixed that but threw the hero away. Those rows show an uncoloured figure — it is the hero's number, not a reading of the person.

The bracket is taken from the medals on the scoreboard, so all ten players are measured against one field. Using each player's own bracket would compare an Archon's Lion to Archon Lions and an Immortal's to Immortal ones, inside a single game where they are on the same map against each other.

What it is worth

Backtested over 1,340 ranked matches drawn from thirteen accounts, with every player's record recomputed from games before each match started:

Those first two rows are the trap. 53.7% looks like a win over a coin flip — until you notice Radiant won 53.4% of the same matches. Radiant has a real side advantage in Dota, so always guess Radiant scores 53.4% while knowing nothing at all. Measured against that baseline rather than against 50%, the model's accuracy adds 0.3 points: nothing.

The AUC is the honest measure, because it does not care about the base rate — it asks whether a match Radiant won scores higher than one they lost. At 0.567 across 1,340 matches it is more than four standard errors above chance, and it holds at 0.568 on the subset where at least eight of ten players are readable. So there is real signal here. It is weak — 0.567 means the ranking is right about 57% of the time instead of 50% — but it is not noise.

Those figures are from before the side term existed, and they are exactly why it now does. Without it the model treated 50% as neutral when the true neutral is Radiant's base rate, so every prediction leaned a couple of points toward Dire — visible in the old calibration, where matches rated 45–50% were in fact won by Radiant 51.9% of the time.

Both fixes were tested out of sample rather than trusted: the side term was measured on half the matches and every variant scored on the other half, which it had never seen.

The last row is what ships. Every variant is scored on the same matches, so the comparisons are paired and firmer than the individual intervals suggest. Over the full 1,338 matches that configuration runs at 56.4% against a 53.4% baseline, with AUC 0.570 and Brier 0.2458 — the ranking signal the AUC always showed is now actually reaching the prediction.

Caveats that remain: every account came from one bracket and one player's peer group, matchmaking already balances teams by rating, and adjusted win rates are shrunk hard toward the middle. The figure on a match page is still best read as a description of the two rosters — but "it predicts nothing" would now be the wrong conclusion.