New to these reports? Start here
- Dotted-underlined words have a plain-English definition — hover or tap them. Every term is also on the glossary page.
- "Null" means we found nothing, not that something broke. Most reports here are negative results, on purpose — knowing an idea doesn't work is the point.
- Two questions get asked separately. First, is the effect real? Second, is it already priced into the betting odds? An effect can be completely real and still useless to bet on.
- A "calibration" row is a self-check. It runs the same method on something already known to be true. If that fails, the whole report is unreliable — so it's reported alongside the findings.
- If a confidence interval includes zero, the real effect might be nothing at all, so no claim gets made.
The Close Band Can Be Improved (+2.5pp, ~half the ECR gap)
Date: 2026-07-20 · Harness: close_band_experiment.py ·
Sample: 12,510 close pairs of 56,046, 2025 weeks 5–17 · Seed: 20260719
The target
The start/sit audit isolated the only band that matters: pairs projected within 2 points of each other. There the engine managed 54.6% — a coin flip, and indistinguishable from a 4-game average — while ECRexpert consensus rankingFantasyPros' averaged ranking across many fantasy analysts. A strong benchmark, and hard to beat. reached 60.7%. Everything above a 5-point gap is decided by who the players are, not by any model.
Result
| Ranker | Close-band accuracy |
|---|---|
| naive trailing-4 | 53.6% [52.2, 55.4] |
| baseline projection | 54.8% [54.0, 55.4] |
| + snap / depth | 57.3% [56.1, 58.7] |
| + snap / depth + injury | 57.3% [56.0, 58.7] |
| ECR (expert consensus) | 60.0% [58.6, 61.6] |
+2.51pp over baseline, 95% CI95% confidence intervalThe range the true value is plausibly in. If this range includes zero, we cannot rule out that the real effect is nothing at all. [+0.98, +4.34] — excludes zero.
That closes roughly half the 5.2pp gap to expert consensus, on precisely the decisions that were near-worthless before.
Injury reports add nothing — and the reason is instructive
Adding the official weekly injury report (report status + practice participation) moved the number by +0.02pp. Not noise around a small effect; essentially exactly zero.
The cause is visible in the coverage: only 74 of 2,438 player-weeks (3%) carry an injury designation at all. The scored population is players who were projected and actually recorded a stat line. Players who are Out or Doubtful mostly do not play, so they never enter the sample. The injury report's main information — this player will not play — is already expressed by his absence.
This kills an intuitive hypothesis cheaply. "Experts know who's hurt" looked like the obvious explanation for the ECR edge; among players who actually suit up, it explains none of it. Whatever the remaining 2.7pp is, it is not the injury report.
The same feature that failed the MAE gate
Snap/depth is the identical signal that was rejected twice for the projection engine — as a post-hoc multiplier and as a volume-model input, both worsening MAEmean absolute errorAverage size of the miss, ignoring direction. If a projection is off by 3 one week and -5 the next, the MAE is 4. Lower is better.. Here it produces the largest improvement yet measured on the metric that describes the actual product decision.
That is not a contradiction, it is the whole lesson stated three ways:
| Objective | Snap/depth effect |
|---|---|
| Per-stat MAE (engine headline) | worse (0–1 of 4 categories improved) |
| Ranking (SpearmanSpearman correlationA measure of whether two rankings agree, from -1 (opposite) through 0 (unrelated) to +1 (identical). Cares about order, not exact values.) | +0.0185, CI excludes zero |
| Close-band start/sit | +2.51pp, CI excludes zero |
A signal can be genuinely useful for ordering under uncertainty and useless for point accuracy. Judging it on the engine's headline metric would have discarded the best start/sit improvement available.
Ceiling and honest bounds
- ECR at 60.0% is the practical ceiling, not 80%. Two players projected within 2 points have heavily overlapping outcome distributions; most of what separates them on Sunday is variance. Headroom is ~5pp, and we just took half.
- One season, 11 scored weeks, 12,510 pairs. The CI excludes zero but the point estimate will move.
- Features and λ were fixed a priori; no search was run.
Recommendation
Ship the snap/depth layer as a start/sit ordering surface, not inside the
projection engine. It re-ranks players the projection places close together,
which is exactly where it helps and exactly where the engine currently offers
nothing over a 4-game average. Keep snap_share_weight and snap_rate_volume
at 0.0 in the engine.
The residual 2.7pp to ECR is now a sharper question than before: it is not injuries, and it is not role. The remaining candidates are matchup/game-script judgement and genuine analyst insight — both harder and neither cheap.