Day-ahead price forecasting + walk-forward backtest

.github/workflows/backend-tests.yml

@@ -24,4 +24,4 @@ jobs:
     - name: Run Tests
       env:
         OPENELECTRICITY_API_KEY: ${{ secrets.OPENELECTRICITY_API_KEY }}
-      run: uv run pytest
+      run: uv run --extra forecast pytest

.gitignore

@@ -2,4 +2,6 @@
 /**/.pytest_cache
 .venv/
 .env
-*.svg
+*.svg
+# forecasting: raw NEMWeb downloads are re-fetchable; the processed parquet is committed
+data/raw/

README.md

@@ -119,6 +119,56 @@ Live spot prices are fetched from AEMO via the Open Electricity API:
 
 Implementation: [main.py](main.py) - `run_live_simulation()`
 
+## Day-Ahead Forecasting Backtest
+
+The live optimizer schedules against prices that have already happened (perfect foresight). The [forecasting/](forecasting/) package measures how much of that revenue a real operator could capture by scheduling on **forecast** prices instead.
+
+- **Data:** SA1 prices from AEMO NEMWeb from 2019 onwards, averaged to 30 minutes in AEST and cached in `data/sa1_prices.parquet`.
+- **Models:** a naive baseline (same half-hour yesterday), LightGBM (lagged prices and calendar features) and a PyTorch LSTM (the past 7 days of prices plus calendar features). Each model forecasts all 48 prices of day D using only data up to the end of D-1. A pytest test checks for leakage.
+- **Evaluation:** walk-forward. Hyperparameters are chosen on a 3-month validation window. The final 12 months are the test period, with models retrained every 30 days. Each day, the unchanged LP is run on the forecast, and the resulting schedule is scored at actual prices.
+
+### Results
+
+The test period ran from 12 Oct 2025 to 11 Oct 2026 (365 days). The battery was 5 MWh / 2.5 MW / 90% efficiency. With perfect foresight it would have earned **$560,314**.
+
+| Model | Capture rate | Realised revenue (AUD) | MAE (AUD/MWh) | RMSE (AUD/MWh) | Days with a loss |
+|---|---:|---:|---:|---:|---:|
+| Naive (yesterday) | 27.2% | 152,353 | 80.0 | 584.1 | 85 |
+| LightGBM | 49.5% | 277,312 | 57.8 | 417.2 | 32 |
+| **LSTM** | **61.8%** | **346,394** | **51.2** | **413.9** | **28** |
+
+*Capture rate = total realised revenue ÷ total perfect-foresight revenue over all test days.*
+
+- **The LSTM won.** It captured 62% of perfect-foresight revenue, against 49% for LightGBM and 27% for the naive baseline. It also had the lowest forecast error. It won every full quarter of the test year (49–68%) and only tied LightGBM in the 11-day partial quarter at the end.
+- **Both ML models more than doubled the naive baseline's revenue** and had about a third as many losing days (28–32 vs 85).
+- **Revenue depends on timing, not average accuracy.** South Australian revenue is dominated by a few extreme price spikes, so capture rates vary a lot from quarter to quarter. Getting the evening peak right matters far more than being accurate overnight.
+
+### When forecasts go wrong
+
+![Backtest page on the worst test day](images/backtest_worst_day.png)
+
+The Backtest page on the **worst test day, 18 May 2026**:
+
+- **What happened:** prices were unusually high from morning to early afternoon (averaging $214/MWh from 06:00 to 14:00), then fell to about $105/MWh over the evening.
+- **What the models expected:** all three forecasts assumed a normal day, with a cheaper middle of the day and the usual 17:30 evening peak.
+- **Result:** every model charged during the expensive daytime and discharged into the cheap evening.
+
+| | Perfect foresight | Naive | LightGBM | LSTM |
+|---|---:|---:|---:|---:|
+| Profit on 18 May 2026 | +$391 | −$649 | −$490 | −$360 |
+
+The LSTM lost the least, but it still lost money. A schedule fixed at the end of the previous day can't react when the day's pattern flips. The fix would be intraday re-optimisation using AEMO pre-dispatch forecasts (see [results/REPORT.md](results/REPORT.md) for the full limitations).
+
+To reproduce (results go to `results/`, see [results/REPORT.md](results/REPORT.md)):
+
+```bash
+uv sync --extra forecast
+uv run python -m forecasting.data       # optional: refresh the AEMO price cache
+uv run python -m forecasting.backtest   # validation -> freeze -> walk-forward test -> report
+```
+
+The API serves the results at `GET /api/v1/backtest/days` and `GET /api/v1/backtest/day/{date}`. The React app shows them on the **Backtest** page (`#/backtest`).
+
 ## Current Limitations
 
 ### Single Market Region

demo_script.txt

@@ -0,0 +1,149 @@
+OptiGrid Demo Script
+====================
+
+Before you start
+----------------
+- Open optigrid.seth-tang.me in a clean browser tab.
+- Make sure you are on the live site, not a local screenshot.
+- Be ready for the backend to take 5-10 seconds on the first run because the Render service has to wake up.
+
+
+1. Opening pitch (30 seconds)
+-----------------------------
+Say:
+"This is OptiGrid, a battery dispatch optimizer. It takes live wholesale electricity prices from AEMO -- the Australian Energy Market Operator -- and figures out the best times for a battery to charge and discharge to make the most money."
+
+Say:
+"It is a classic energy arbitrage problem: buy electricity when it is cheap, store it, and sell it back when it is expensive."
+
+
+2. Walk through the UI (1 minute)
+---------------------------------
+Point to the left sidebar.
+
+Say:
+"On the left we configure the battery. Right now it is set to 5 MWh of capacity and 2.5 MW max power output."
+
+Explain the two sliders:
+
+- Battery Capacity (5 MWh): the size of the energy tank. Maximum amount of electricity it can hold.
+- Max Power Output (2.5 MW): the size of the pipe. Fastest rate it can charge or discharge.
+
+Say:
+"Think of it like a water tank. Capacity is how big the tank is. Max power is how fast the tap can fill or empty it."
+
+Then point to the three cards on the right.
+
+Say:
+"These three cards will show us the solver status, projected profit, and how much energy we moved through the battery."
+
+
+3. Run the optimization (1 minute)
+----------------------------------
+Click Run Optimization.
+
+Say:
+"When I click this, the frontend sends the battery specs to a FastAPI backend. The backend fetches live AEMO spot prices from the Open Electricity API, resamples them to 30-minute intervals, and runs a linear programming solver to find the optimal schedule."
+
+Wait for the loading screen to finish.
+
+Say:
+"This takes a few seconds because the backend service is waking up and fetching fresh market data."
+
+Once it finishes, the cards update:
+
+- Status: Optimal
+- Projected Profit: e.g. $946.75
+- Energy Cycled: e.g. 23.2 MWh
+
+Say:
+"Status Optimal means the solver found a feasible schedule. Projected profit is the total arbitrage profit over the next 24 hours. Energy cycled tells us how much energy we pulled from the grid into the battery."
+
+
+4. Read the chart (2 minutes)
+-----------------------------
+Point to the chart.
+
+Say:
+"The green line is the spot price in Australian dollars per megawatt-hour. The blue bars are the battery's State of Charge -- how full the battery is at each 30-minute interval."
+
+Then explain the pattern.
+
+Say:
+"The basic strategy is simple: charge when prices are low or negative, discharge when prices are high."
+
+Use the 02:00 example.
+
+Say:
+"At 02:00 the price is negative -- about minus fifteen dollars per MWh. That means the market is paying us to take electricity. So the solver charges the battery as fast as it can. That is why the blue bar jumps up."
+
+Say:
+"At 01:30 the price is over a hundred dollars per MWh, so the battery is empty -- it sold everything it had."
+
+If the employer asks why the chart starts at 02:00:
+
+Say:
+"The chart starts at 02:00 AEST because the Open Electricity API returns a fixed 24-hour window of AEMO data starting at that time. It is not aligned to when I clicked the button. In production I would make the datetime timezone-aware so the window matches the current time."
+
+
+5. Explain the math briefly (1 minute)
+--------------------------------------
+If they want to know how it works under the hood.
+
+Say:
+"The optimizer is a linear program built with PuLP. The decision variables are charge, discharge, and state of charge for each 30-minute interval."
+
+Say:
+"The objective is to maximize profit: sum of (discharge minus charge) times price times half an hour."
+
+Say:
+"The constraints are physical: you cannot charge faster than 2.5 MW, you cannot store more than 5 MWh, and the state of charge has to balance from one interval to the next."
+
+
+6. Mention the architecture (1 minute)
+--------------------------------------
+Say:
+"The stack is React with Tailwind on the frontend, FastAPI on the backend, and PuLP for the optimization. The backend pulls live prices from AEMO via the Open Electricity API. Everything is containerized and deployed through GitHub Actions to Render for the backend and S3 plus CloudFront for the frontend."
+
+
+7. Be honest about limitations (1 minute)
+-----------------------------------------
+Say:
+"A few things to keep in mind. This version assumes perfect foresight of prices, which is fine for a demo but not realistic. In production you would use price forecasts and Model Predictive Control."
+
+Say:
+"It also only optimizes one region, SA1, and ignores things like battery degradation, transmission losses, and FCAS revenue. Those would be the next features to add."
+
+
+8. Clarify what this dashboard is
+---------------------------------
+If they ask whether this controls a real battery.
+
+Say:
+"This is a planning and visualization dashboard. The solver produces a recommended dispatch schedule. In a real deployment that schedule would be sent as setpoints to the battery's control system or SCADA. The battery itself, through its inverter and battery management system, would actually execute it."
+
+
+9. Closing line
+---------------
+Say:
+"So in short: this tool reads live market prices, runs an optimizer, and shows you when to charge and discharge a battery to maximize arbitrage profit."
+
+
+Quick demo checklist
+--------------------
+1. Open optigrid.seth-tang.me
+2. Introduce the project
+3. Explain capacity vs max power
+4. Click Run Optimization
+5. Explain Status, Profit, Energy Cycled
+6. Walk through the chart
+7. Mention the stack
+8. Mention limitations
+9. Clarify dashboard vs real BESS control
+
+
+If something goes wrong
+-----------------------
+- Chart stays empty: wait a bit, then click Run Optimization again. The backend may still be cold-starting.
+- Profit looks weird: prices change every day, so the number will be different each time you run it.
+- Chart starts at 02:00: that is expected because of the API timezone handling. Use it as a chance to talk about production improvements.