Does NFL Preseason Performance Predict Regular-Season Outcomes?

One-off study, not a model, not production. Question from [[preseason_signal_todo]]: does preseason performance carry real signal for regular-season outcomes, for teams, rookies, and vets separately? Short answer: no, on all three counts — one technically-significant-but-trivial team-level effect, and two clean nulls for players. Filed as an honest negative next to [[fantasy_mae_v4]] and [[nfl_prop_modeling]]'s other results in this vein.

TL;DR

The last row is the important calibrationcalibrationA deliberate sanity check on the method itself: run it on something already known to be true. If it fails to detect the known thing, the method is broken and its other results mean nothing. check: this project's own data pipeline easily finds r=0.77 when a real, strong relationship exists (a good player last year is a good player this year). Preseason performance doesn't come close to that for any population tested.

Data availability (worth knowing for next time)

nflverse/nflreadpy has ZERO preseason coverage — confirmed empirically, not assumed: load_player_stats, load_pbp, and load_schedules were all pulled with seasons=True (full 1999-2026 range) and every one shows only REG/POST/playoff game types. No PRE rows exist anywhere in this data source, for any season. This is a hard wall, not a filter to work around.

ESPN's public API does carry preseason data (seasontype=1 in their scoreboard and game-summary endpoints) — final scores AND full player box scores (passing/rushing/receiving lines), going back at least to 2015. This is the same family of hidden JSON API already used elsewhere in this repo (the /today unified view's ESPN adapter).

⚠ Gotcha worth remembering: ESPN's edge (Akamai) returned a 403 to both urllib and requests from this server — confirmed it's a TLS-fingerprint block, not a header/User-Agent issue, since the exact same User-Agent string that 403s under requests gets a clean 200 under curl. Fix used here: shell out to curl via subprocess instead of a Python HTTP client. This is a different failure mode than the existing _espn.py adapter's "browser User-Agent avoids intermittent blocks from cloud IPs" comment — that one's about IP reputation, this one's about TLS handshake fingerprinting. Both are real, and both can bite independently.

Methodology

• Team-level: pulled every ESPN preseason game 2015-2025 (pull_preseason_team_results.py, 488 games; 2020 correctly shows 0 games — COVID canceled the entire preseason, a good sanity check that the scrape is real). Computed each team's preseason win% per season, correlated against that team's real regular-season win% the same year (from nflreadpy.load_schedules, which DOES have full regular-season coverage). 320 team-seasons after requiring ≥2 preseason games and ≥10 regular-season games.
• Rookie-level: pulled full preseason box scores 2019/2021-2025 (2020 skipped, no games; pull_preseason_boxscores.py, 293 games, 12,710 individual stat lines). Aggregated each player's preseason stat line into PPRpoints per receptionA fantasy scoring format that awards a point for every catch, which raises the value of high-volume receivers. points via this project's own scoring.stat_line_points (the same function draft_sim uses), identified that season's rookie class via nflreadpy.load_rosters' years_exp == 0 for QB/RB/WR/TE, and joined to real regular-season PPG from scoring.season_fantasy_summary (requiring ≥4 real games played, i.e. they actually made a roster and got a role). 410 rookies.
• Veteran-level: identical pipeline, years_exp >= 2 instead of == 0. 1,427 player-seasons. Also computed the prior-season PPG → this-season PPG baseline correlationcorrelationHow closely two things move together, from -1 (opposite) through 0 (unrelated) to +1 (in lockstep). It does not by itself mean one causes the other. as a calibration check (same players, same join logic, swap the predictor) — this is the "does this pipeline even find real signal when it's there" sanity check, and it clearly does (r=0.77).
• All player-name joins use this project's own norm_name() (draft_sim/pool.py), same cross-source matching already used throughout the fantasy/draft-sim code.

Notable illustrative cases (rookies)

The top-10 rookie preseason-PPG performers are a good gut-check for why this is null — mostly stat-padding against backup defenses that means nothing:

Two real hits (Stevenson, Worthy) out of ten torrid preseason performers; the rest cratered to irrelevance. That ratio is consistent with the r≈0 finding, not an exception to it.

A confound worth flagging (not overclaiming a "negative predictor")

Preseason volume (total points, games played) came back weakly negative against real production for rookies (r=-0.11, p=0.03; r=-0.10, p=0.04 for games played) — this is almost certainly a selection artifact, not a causal signal: fringe roster players who are genuine camp-battle question marks get extended preseason run specifically because their role is uncertain, while locked-in high-value rookies are often held out of preseason entirely to avoid injury risk. "Played a lot of preseason" is confounded with "wasn't a lock," not a useful negative predictor on its own. Flagging this explicitly so it doesn't get miscited later as "preseason usage hurts rookies" — it doesn't; it's just correlated with a different underlying cause.

Caveats / what this does NOT rule out

• No opponent/game-script control. A rookie's preseason line against a team's 2nd/3rd-string defense in week 3 isn't adjusted for who was on the field opposite them, or which preseason week it was (week 1 skews toward more starter snaps than week 3-4 in most franchises' modern usage patterns).
• Single-season, not multi-year trajectory. This tests "this preseason → this regular season," not "does a rookie's preseason predict where his career eventually goes" (a slower, harder question).
• PPG, not role/snap-share. Real fantasy-relevant signal might live in snap-share or target-share trajectories rather than points — untested here.
• Small-ish samples per position (QB n=44, TE n=75 for rookies) mean the position-specific breakdowns are underpoweredunderpoweredNot enough data to detect an effect even if it is really there. An underpowered null means "we could not tell", which is very different from "there is nothing there". to detect anything but a fairly large effect; only the pooled n=410/1,427 tests are well-powered.

Bottom line for draft_sim / redraft decisions

Don't weight preseason production in rankings or the draft tool. ECRexpert consensus rankingFantasyPros' averaged ranking across many fantasy analysts. A strong benchmark, and hard to beat. (already the pool's ranking source, confirmed better than this project's own engine in FINDINGS_VS_ECR.md) almost certainly already prices in whatever real preseason signal exists via beat-reporter/analyst attention — this study found no additional signal in the raw box score numbers themselves, for either population.

Addendum 2026-08-16: is there ANY modeling value in preseason data?

Follow-up question after the above: production is null, but preseason is also the literal mechanism coaches use to decide roster/depth-chart opportunity — so does preseason snap-share/role (not points) predict Week 1 role? Conceptually more defensible than what was tested above. In practice: untestable for free, dead end, not just this study's gap.

Checked directly (not just docs): - nflreadpy.load_snap_counts() and .load_depth_charts() — same REG/POST- only blackout as every other nflverse loader used above. The preseason gap is systematic across the whole data source, not one missing endpoint. - ESPN's game-summary API (which DOES carry preseason stat lines) has no snap-count field at all in its box score — checked the raw response directly (passing/rushing/receiving/fumbles/defense/returns/kicking only). - Pro Football Reference's own snap-count pages appear regular-season only (presumably why nflverse's PFR ingestion is REG-only too). - PFF and RotoWire DO publish preseason snap/usage numbers, but only as prose recap articles per week — no structured feed, no historical archive, and PFF's real structured data is paywalled. - ESPN also exposes a depth-chart endpoint, but it's LIVE-STATE only — no way to query what a team's depth chart looked like during a past preseason, so it can't answer a historical question even in principle.

Commercial APIs (Sportradar: preseason coverage since 2015) exist but aren't worth paying for on a hobby question. Filed closed — don't re-investigate this angle without a new, currently-nonexistent free data source.