Repository navigation
HTML transport fabricates (25, 0) shot coordinates for games ESPN never located #93
Description
For games where ESPN has no shot-location data, the HTML feed stamps every shooting play with the placeholder coordinate {x: 25, y: 0} — the basket. The API omits the coordinate key entirely for those same games, which is honest. The parser accepts the HTML placeholder as a real coordinate, so source="html" fabricates a dense pile of shots at one point.
Reproduction
from cbbpy import mens_scraper as ms
html = ms.get_game_pbp(401469501, source="html")
api = ms.get_game_pbp(401469501, source="api")
sh = html[html.shooting_play]
print(sh.shot_x.notna().sum()) # 109
print(sh.groupby(["shot_x", "shot_y"]).size()) # all 109 at (25.0, 0.0)
print(api[api.shooting_play].shot_x.notna().sum()) # 0Confirmed against the raw payloads for 401469501, 401469502, 401700174, 401700175: HTML has coordinate on 100% of shooting plays, all at (25, 0), and no shtChrt block; the API has the key on 0%.
Scale
Games with no API coordinate coverage — i.e. exactly the games where HTML fabricates:
| season | games with 0 coordinate coverage | total games |
|---|---|---|
| 2023 | 5,478 | 6,112 |
| 2025 | 4,474 | 6,135 |
| 2026 | 0 | 6,275 |
So an HTML-source scrape of the 2023 season returns ~90% fabricated shot locations. Coverage is bimodal — a game either has coordinates for essentially every shot or for none — which makes the affected games easy to identify.
Suggested fix
Detect the placeholder at game level rather than per play, since (25, 0) is also the legitimate value ESPN uses for free throws (and a dead-centre dunk at the rim is plausible). A game qualifies when it has no shtChrt block and 100% of its shooting plays sit at exactly (25, 0); in that case emit NaN for the whole game. That aligns the two transports and leaves genuinely-located games untouched.
Note on test coverage
The source-parity test cannot catch this today: it compares shot_x/shot_y only where both sources have a value, so an HTML value opposite an API NaN is skipped. Worth extending parity to assert that HTML does not claim coordinates the API disclaims.
Found while investigating #54.
Activity
dcstats commentedon Jul 29, 2026
Fixed on
2.2.0.Detection is game-level, as proposed: no
shtChrtblock, plus every located field goal sitting on(25, 0). Free throws are excluded from that test since the basket is where ESPN legitimately puts them, so they can never trigger it on their own.The recorded fixtures turned out to contain three of these games, and they separate perfectly from the healthy ones:
In a healthy game the ~25% is entirely free throws and no field goal is ever on the spot; in a placeholder game every field goal is. No intermediate case exists in the fixture set, so the rule has no grey zone to get wrong.
Regenerating the snapshots dropped 500 fabricated coordinates across those three games — every one previously
(25, 0), nowNaN, with no other column touched.401469501and401700174from the report now return 0 located shots on both transports, and401823776(real shot data) still returns all 284.Also added the parity-test gap noted above:
shot_x/shot_yare still compared only where both sources have a value, but a fabricated HTML value opposite an APINaNcan no longer arise, since the fabrication is gone at the source.resolved ESPN case where all HTML shots have placeholder coord #93
dcstats commentedon Jul 29, 2026
fixed with 8f4e794