Skip to content

About

FlareCraft is an enterprise-grade Space Weather Risk & Intelligence Platform. It ingests live solar and geomagnetic telemetry from public space agencies, forecasts space weather hazards using machine learning, and translates raw physical forecasts into asset-specific operational risks and actionable mitigation steps.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Repository files navigation

FlareCraft

A space weather risk platform. It ingests live solar and geomagnetic data from public agency feeds, forecasts geomagnetic conditions with a trained GRU quantile model, and translates each forecast into asset-specific operational impact.

The output is never "Kp will reach 7." The output is:

DEMO-NODE-01 sees 0.63 V/km geoelectric field at 112 nT/min, relative GIC index 100/100 — reduce transformer loading to 80% and stage a spare before 2026-07-25T15:00Z. Probability 0.93, high confidence, cost of inaction $13,800,000.

The impact translation is the product. The physics is free from NOAA and GFZ.


Architecture

        NOAA SWPC            NASA/GSFC OMNI2          GFZ Potsdam
     RTSW · GOES · Kp        hourly 1995-2026      definitive Kp 1932-
        alerts.json           (training data)        (Bartels history)
            │                       │                      │
            ▼                       │                      │
   ┌─────────────────┐              │                      │
   │  apps/ingest    │              │                      │
   │  APScheduler    │◄─────────────┴──────────────────────┘
   │ 4 jobs @60/300s │   ml/data/download · kp_history_job
   └────────┬────────┘
            │ idempotent upsert on (source, metric, time_tag)
            ▼
   ┌─────────────────────────────────────────────┐
   │  observations · swpc_alerts · assets ·      │
   │  forecasts · risk_assessments               │
   │  SQLite (dev) / PostgreSQL + TimescaleDB    │
   └────────┬────────────────────────────────────┘
            │
   ┌────────▼────────┐      ┌──────────────────────────────┐
   │  apps/api       │      │  ml/                         │
   │  FastAPI        │◄─────┤  GRU quantile forecaster     │
   │                 │ ONNX │  train → eval → geomag_gru   │
   │  /health /ready │      └──────────────────────────────┘
   │  /status        │──► System & Feed Status Page
   │  /conditions    │──► Hazard
   │  /forecast      │──► Quantile Forecast (p10, p50, p90)
   │  /risk          │──► Risk = Hazard × Exposure × Vulnerability × Criticality
   │  /briefing      │──► Gemini 2.5 Flash (narration ONLY)
   │  /v1/*          │──► API Key Auth & Per-Key Rate Limiting
   └────────┬────────┘
            ▼
   ┌─────────────────┐
   │  apps/web       │  Next.js 15 · uPlot · 17 routes (including /status)
   └─────────────────┘

Where prediction happens, and where it does not. The GRU produces every forecast value. The vulnerability curves produce every physical consequence. Gemini produces only prose — a numeric guard rejects any narration containing a number absent from its input, and the model card panel is labelled as the forecaster, separately from the "AI briefing" label. That separation is deliberate.


Production Backtest Numbers (Including Bad Cases)

The GRU forecaster was evaluated across held-out test splits (2024–2025) and four major historical storm windows cut out of training data.

Overall Performance Summary

Metric / Horizon t+1h t+3h t+4h t+6h Baseline Benchmark
FlareCraft GRU MAE 0.52 0.64 0.70 0.75 Target forecast accuracy
Persistence MAE 0.22 0.61 0.74 0.81 Beats GRU at t+1h; loses from t+4h out
27-Day Recurrence MAE 1.15 1.18 1.21 1.25 Beaten by GRU at all horizons
Climatology MAE 1.34 1.34 1.34 1.34 Beaten by GRU at all horizons
80% Quantile Coverage 81.6% 81.1% 80.4% 79.8% Well-calibrated 80% uncertainty band

Stated Limitations & Unfavourable Cases

  1. Short-Horizon Persistence Defeat (t+1h):
    • Persistence beats the GRU model at t+1h (0.22 vs 0.52 MAE). Because Kp is a 3-hourly index, Kp one hour ahead is often literally the same number as Kp now. Persistence is a free copy at short lead times. The GRU wins from t+4h out, where operational lead time actually matters.
  2. Severe Storm Calibration Collapse (Halloween 2003 Storm):
    • During the extreme Halloween 2003 storm (peak Kp 9.0), mean error increased to 1.3 Kp and 80-quantile empirical coverage collapsed from 81.6% down to 0.55. Kp 9 events are rare in training data and the quantile heads regress toward the mean.

For a full breakdown of model weaknesses, see LIMITATIONS.md.


Data Sources and Attribution

source used for licence / attribution
NOAA SWPC — RTSW mag & plasma, GOES X-ray, planetary Kp, alerts live conditions, nowcast US Government work, public domain
NASA/GSFC OMNI2 hourly, via SPDF Model training data 1995–2026 Courtesy of NASA/GSFC Space Physics Data Facility; OMNI compiled by King & Papitashvili
GFZ Potsdam definitive Kp Bartels rotation history back to 1932 CC BY 4.0. Matzka et al. (2021), The geomagnetic Kp index and derived indices of geomagnetic activity, Space Weather, doi:10.1029/2020SW002641
NRLMSISE-00 thermospheric density anchors Picone, Hedin, Drob & Aikin (2002), JGR 107(A12)
King-Hele (1987) orbital decay relation Satellite Orbits in an Atmosphere
Viljanen & Pirjola (1994) GIC transformer heating curve see vulnerability.py docstrings
Google Gemini 2.5 Flash narration only, never prediction free tier; posture in 05_CLAUDE.md

Setup & Running the Stack

1. Requirements

  • Python 3.12+ with virtualenv
  • Node.js 20+ with npm / pnpm

2. Backend & Ingest Setup

# Set PYTHONPATH to include apps/api
set PYTHONPATH=apps/api

# Run local development backend & ingest workers
python run_dev.py

FastAPI server runs on http://localhost:8000. Key endpoints:

  • GET /health — Liveness probe
  • GET /ready — Readiness probe (verifies database)
  • GET /status — Detailed system component status
  • GET /v1/api-key/verify — Verifies API keys & rate limit status (X-API-Key: demo-key)

3. Frontend Web App Setup

npm --prefix apps/web run dev

Open http://localhost:3000/now or the public status page at http://localhost:3000/status.

4. Running Tests & Quality Checks

# Python unit tests & health/security tests
python -m pytest tests/

# Frontend Vitest unit tests
npx vitest run --dir apps/web

# Playwright smoke test across all 17 routes
npx playwright test

# Production build test
npm --prefix apps/web run build

5. Git Pre-Commit Secret Scanner

Install the pre-commit hook to block accidental secret commits:

bash scripts/install-hooks.sh

Security & Reliability Controls

  • Structured JSON Logging: Every log line is formatted as JSON with timestamp, level, logger, message, and request_id.
  • Request ID Threading: X-Request-ID is generated/threaded through every layer from Next.js fetch calls to FastAPI middleware.
  • Sentry Integration: Integrated on both backend (FastAPI) and frontend (Next.js).
  • API Keys & Rate Limiting: Per-key sliding window rate limits on public /v1/* endpoints with X-RateLimit-* headers.
  • CORS Locking: Restricted to known production/local origins.
  • CI/CD Pipeline: GitHub Actions workflow (.github/workflows/ci.yml) runs type-checks, ESLint, Pytest, Vitest, production builds, and Playwright smoke tests, blocking PR merge on failure.

About

FlareCraft is an enterprise-grade Space Weather Risk & Intelligence Platform. It ingests live solar and geomagnetic telemetry from public space agencies, forecasts space weather hazards using machine learning, and translates raw physical forecasts into asset-specific operational risks and actionable mitigation steps.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages