Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
78b587c
tweak for conference game indicator
dcstats Nov 14, 2025
4615c95
update version
dcstats Nov 14, 2025
450acdd
check both teams' records
dcstats Nov 18, 2025
527a3ad
Add known issues section to README
dcstats Nov 18, 2025
70d13dd
handling non-existent play descs or fields in shot chart
dcstats Nov 20, 2025
8d109b3
match shot data to pbp by id
dcstats Nov 24, 2025
13e81a4
gitignore
dcstats Nov 24, 2025
f6fa413
fix for home team detection
dcstats Nov 26, 2025
4899acb
conference games detection
dcstats Nov 26, 2025
912123e
espn api key changes
dcstats Mar 17, 2026
61c7b21
overhauled testing suite
dcstats Jul 4, 2026
4547355
gitignore
dcstats Jul 4, 2026
51098bc
requests -> curl cffi
dcstats Jul 4, 2026
1143977
gitignore
dcstats Jul 11, 2026
32cf45b
addressing redundancy
dcstats Jul 11, 2026
9950547
WAF fingerprinting adjustments, added columns in PBP JSON for HTML sc…
dcstats Jul 12, 2026
55bfe0b
testing updates
dcstats Jul 12, 2026
9f944a3
error messaging, API play type+player names
dcstats Jul 12, 2026
561dbf6
updated testing from changes
dcstats Jul 12, 2026
05b2f65
win prob + betting stats
dcstats Jul 12, 2026
a3ad219
testing update for wp+betting
dcstats Jul 12, 2026
4abefc5
versioning
dcstats Jul 12, 2026
07e9af8
version auditing #63
dcstats Jul 12, 2026
dbb69f4
team map updates #66
dcstats Jul 12, 2026
6224453
json retries
dcstats Jul 12, 2026
eae7741
CI/tooling: bump action versions, add flake8 error gate, dev extra
mborrus Jul 12, 2026
5ffc006
Fix latent NameErrors in retry except-blocks (#69)
mborrus Jul 12, 2026
a79e4a5
Consolidate ESPN embedded-JSON extraction into _parse_espn_json
mborrus Jul 12, 2026
d9232d3
Collapse _get_game_boxscore_helper duplication (#75)
mborrus Jul 12, 2026
c78c8b9
Robustness cleanups: typed excepts, schedule hardening, stat-split gu…
mborrus Jul 12, 2026
37425d8
Fix failure-path return types (#83)
mborrus Jul 12, 2026
017e723
Guard PBP clock parsing against missing/malformed clocks (#82)
mborrus Jul 12, 2026
fd4caeb
Deduplicate intra-conference games in get_games_conference (#84)
mborrus Jul 12, 2026
afacc75
Floor worker count at 1 for single-core machines
dcstats Jul 13, 2026
db17d4c
Merge pull request #68 from mborrus/tweaks-n-improvements
dcstats Jul 13, 2026
3453be9
team logos + fixed shot parsing
dcstats Jul 19, 2026
d058b44
testing for substitutions
dcstats Jul 20, 2026
a0e201b
updated game time logic - added `game_datetime` col
dcstats Jul 20, 2026
c25da19
improved logging behavior + warnings instead of prints + removed warn…
dcstats Jul 20, 2026
c9e8f43
request throttling and request timeout #70 #79
dcstats Jul 20, 2026
4f2cb65
added CLI
dcstats Jul 21, 2026
2cbf698
warning instead of assert for num ots #71
dcstats Jul 22, 2026
33dc9f9
check for num regulation periods played in data, use rule change calc…
dcstats Jul 22, 2026
adfae94
README.md
dcstats Jul 22, 2026
b3e5781
deprecation of quarter/half nomenclature in column names
dcstats Jul 22, 2026
684cb22
improved notice for deprecation; added changelog
dcstats Jul 22, 2026
82bc701
redundancy #76
dcstats Jul 22, 2026
c9f7e32
removal of page not found in readme (most have resolved)
dcstats Jul 22, 2026
508afe7
status code handling #74
dcstats Jul 22, 2026
67429f3
cli deconstruction
dcstats Jul 22, 2026
f9d4a03
updated cli behavior
dcstats Jul 22, 2026
bba345d
fix for mislabeled linescores giving negative `num_ots`
dcstats Jul 22, 2026
3e0763e
updated `_pnf` -> `PageNotFoundError`
dcstats Jul 22, 2026
5f3f2ff
split into retriable vs not for errors #73
dcstats Jul 22, 2026
cebfc31
housekeeping
dcstats Jul 22, 2026
601685d
fix for season typeerror in cli #87
dcstats Jul 23, 2026
fc25f6f
fix for empty df sorting issue #89
dcstats Jul 23, 2026
129a037
restore shooter as deprecated + regenerate README tables #88
dcstats Jul 23, 2026
868129d
team map updates #86
dcstats Jul 23, 2026
ebcc92a
aliases for conf name changes
dcstats Jul 23, 2026
35c0c02
team alias map #90
dcstats Jul 23, 2026
08a7c8e
clamp below-range seasons to earliest map season #91
dcstats Jul 23, 2026
51fa47d
gitignore
dcstats Jul 23, 2026
03664a8
less aggressive scraping defaults
dcstats Jul 24, 2026
25377ea
updated readme
dcstats Jul 24, 2026
0ea4c4e
info logging for retries
dcstats Jul 24, 2026
6277b88
#92 retries for missed boxscore stats hits
dcstats Jul 24, 2026
f065bbb
added verbosity for info, separate out testing logging,
dcstats Jul 24, 2026
abe93b7
retry backoffs
dcstats Jul 25, 2026
6e1ddb5
shot coordinate transform for mislabeled rim shots #54
dcstats Jul 29, 2026
01f239c
altered `is_three` derivation, added transforms for impossible two po…
dcstats Jul 29, 2026
8f4e794
resolved ESPN case where all HTML shots have placeholder coord #93
dcstats Jul 29, 2026
83d53f4
log file banner
dcstats Jul 29, 2026
9feca4b
documentation, unlocated game play type cleaning
dcstats Jul 29, 2026
8424c90
test for edge case (ft in html text)
dcstats Jul 29, 2026
303c46d
fix pandas 3 boxscore dtype widening + uv venv collision in CI
dcstats Jul 29, 2026
7a2d162
fix truncated schedules; split bulk live tests out of PR CI
dcstats Jul 29, 2026
e39cc41
run the full live suite as the PR gate; add WAF probe
dcstats Jul 29, 2026
c25d75b
temp: run WAF probe as a non-gating PR job
dcstats Jul 29, 2026
b54b763
skip live HTML tests where ESPN's WAF blocks the site
dcstats Jul 29, 2026
0094d0b
cancelling in progress workflows when new one starts
dcstats Jul 29, 2026
67e8343
version guard
dcstats Jul 29, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 99 additions & 11 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,40 +5,128 @@ on:
branches: [master]
paths:
- 'src/**'
- 'tests/**'
- 'pyproject.toml'
- '.github/workflows/pytest.yml'
workflow_dispatch:

# One run per branch: a new push cancels the one it supersedes, so a long live
# job doesn't keep scraping ESPN for a commit nobody is waiting on anymore.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
test:
# Deterministic tests against recorded ESPN pages — no network. A failure
# here means the code changed behavior. Runs on every supported Python.
offline:
runs-on: ubuntu-latest
timeout-minutes: 30
timeout-minutes: 10
strategy:
matrix:
python-version: ['3.9', '3.10', '3.11', '3.12', '3.13']
fail-fast: true

steps:
- name: Checkout code
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Cache pip dependencies
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ matrix.python-version }}-${{ hashFiles('**/pyproject.toml') }}
restore-keys: |
${{ runner.os }}-pip-${{ matrix.python-version }}-

- name: Install package in editable mode
run: |
pip install --upgrade pip
pip install -e .

- name: Install testing and linting tools
run: pip install pytest flake8 mypy
- name: Install testing tools
run: pip install pytest pyarrow flake8

# Errors only (syntax errors + undefined names) — catches latent
# NameErrors in branches the tests may not reach
- name: Lint with flake8
run: flake8 src/ --max-line-length=120 --count --select=E9,F63,F7,F82 --show-source --statistics

- name: Run offline tests
run: pytest tests/ -m "not live"

# Same offline suite, but with every dependency installed at the minimum
# version pyproject.toml claims to support (uv resolves the floors directly
# from the metadata, so there's no separate pin list to keep in sync). A
# failure here means a declared floor is dishonest — raise it rather than
# working around it. Python 3.9 because the floors must hold on the oldest
# supported interpreter; pytest/pyarrow are installed separately at their
# latest versions since lowest-direct would otherwise pick ancient ones.
offline-min-versions:
runs-on: ubuntu-latest
timeout-minutes: 10

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up uv
uses: astral-sh/setup-uv@v5
with:
python-version: '3.9'

- name: Install package with lowest supported dependency versions
run: |
uv pip install --resolution lowest-direct -e .
uv pip install pytest pyarrow

- name: Run offline tests
run: uv run --no-project pytest tests/ -m "not live"

# Live invariant tests against real ESPN — verifies the scraper still works
# against today's site. Single Python version: cross-version parsing
# differences are already covered by the offline matrix, and one job keeps
# ESPN request volume (and flake odds) down.
#
# Runs the whole suite, bulk scrapes included (~2,700 games). Slow by
# design — the timeout is generous because a PR waits on this.
live:
runs-on: ubuntu-latest
timeout-minutes: 120

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'

- name: Cache pip dependencies
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/pyproject.toml') }}
key: ${{ runner.os }}-pip-3.12-${{ hashFiles('**/pyproject.toml') }}
restore-keys: |
${{ runner.os }}-pip-
${{ runner.os }}-pip-3.12-

- name: Install package in editable mode
run: |
pip install --upgrade pip
pip install -e .

- name: Install testing tools
run: pip install pytest pyarrow

- name: Run pytest
run: pytest tests/
- name: Run live tests
# -v + unbuffered so a hang is visible in the log as it happens rather
# than as a timeout with no output
env:
PYTHONUNBUFFERED: '1'
run: pytest tests/ -m live -v --durations=10
14 changes: 14 additions & 0 deletions .github/workflows/python-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,20 @@ jobs:
pip install build
- name: Build package
run: python -m build
# Nothing keeps __version__ and the release tag in sync, and a wrong
# filename on PyPI cannot be reused once uploaded — so check the built
# artifact against the tag before the one irreversible step.
- name: Verify tag matches built version
run: |
TAG="${{ github.event.release.tag_name }}"
DIST=$(ls dist/cbbpy-*.tar.gz)
BUILT="v$(basename "$DIST" .tar.gz | sed 's/^cbbpy-//')"
if [ "$TAG" != "$BUILT" ]; then
echo "release tag $TAG does not match built version $BUILT" >&2
echo "fix src/cbbpy/__init__.py or retag, then re-publish" >&2
exit 1
fi
echo "tag $TAG matches built version"
- name: Publish package
uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
with:
Expand Down
32 changes: 32 additions & 0 deletions .github/workflows/waf-probe.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: WAF probe

# Manual diagnostic: reports which curl_cffi impersonation profiles ESPN's WAF
# accepts from a GitHub runner. The answer differs from a workstation's — a
# fingerprint accepted from a residential IP can be challenged from a
# datacenter one — so run this before changing IMPERSONATE for CI's benefit.
on:
workflow_dispatch:

jobs:
probe:
runs-on: ubuntu-latest
timeout-minutes: 10

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'

- name: Install package
run: |
pip install --upgrade pip
pip install -e .

- name: Probe impersonation profiles
env:
PYTHONUNBUFFERED: '1'
run: python tests/waf_probe.py
9 changes: 7 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,13 +1,18 @@
CBBpy.egg-info/
*.egg-info/
*.ipynb
*.pyc
.ipynb_checkpoints/
testing/
__pycache__/
.DS_Store
*.gz
!tests/fixtures/pages/*.html.gz
!tests/fixtures/pages/*.json.gz
*.whl
build/
dist/
*.log
*.xlsx
*.xlsx
.vscode/
.claude/
.pytest_cache/
Loading
Loading