Skip to content

Day-ahead price forecasting + walk-forward backtest - #3

Merged
pisethTang merged 3 commits into
mainfrom
feature/forecast-backtest
Oct 12, 2026
Merged

pisethTang merged 3 commits into
mainfrom
feature/forecast-backtest

Conversation

@pisethTang

Copy link
Copy Markdown
Owner

Summary

  • forecasting/ package: SA1 price history from AEMO NEMWeb (2019-2026); naive, LightGBM and PyTorch LSTM day-ahead models; walk-forward backtest (3-month validation, then frozen; 365 test days; retrain every 30 days)
  • Forecasts go through the unchanged PuLP LP and are scored at actual prices
  • Results: LSTM 61.8%, LightGBM 49.5%, naive 27.2% of perfect-foresight revenue (results/REPORT.md)
  • API: GET /api/v1/backtest/days, GET /api/v1/backtest/day/{date} (/simulate unchanged)
  • React: new Backtest page (#/backtest) with synced price/SoC/profit charts and capture-rate bars; Live/Backtest tabs
  • Tests: leakage checks (features and fit+predict) and realised-profit tests; CI installs the forecast extra

Reproduce

uv sync --extra forecast && uv run python -m forecasting.backtest

tangpiseth1 and others added 3 commits October 12, 2026 16:22
- forecasting/: AEMO NEMWeb SA1 history (2019-2026, 30-min AEST) cached to
  data/sa1_prices.parquet; naive, LightGBM and PyTorch LSTM day-ahead models;
  validation-then-frozen walk-forward test (365 days, retrain every 30 days)
  scored by feeding forecasts into the unchanged LP and settling at actual prices
- results/: backtest.json, REPORT.md, typical-day chart, per-day JSON
  (LSTM 61.8%, LightGBM 49.5%, naive 27.2% of perfect-foresight revenue)
- tests: leakage tests (features and fit+predict) and realised-profit tests
- API: GET /api/v1/backtest/days and /api/v1/backtest/day/{date}
- React: Backtest page (#/backtest) with synced price/SoC/profit charts,
  capture-rate bars and best/worst/spike day pickers; live page unchanged
  apart from a link
- forecasting deps in an optional "forecast" extra (CPU torch); CI installs it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Live/Backtest tab switch on both pages; capture-rate chart moved to top so it no longer jumps off-screen
- Prev/next day buttons, icon jump buttons, zoom toggle button, red losses, #/backtest/<date> deep links
- README: backtest results table and worst-day (2026-05-18) walkthrough with screenshot
Resolve backend-tests.yml conflict: keep main's comment cleanup, keep --extra forecast so CI can run the forecasting tests.
@pisethTang
pisethTang merged commit 5563861 into main Oct 12, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants