fix: Volatility (24h) N/A — replace broken CryptoCompare source

[protostatis]

Summary

The Volatility (24h) card on the dashboard shows N/A in production because volatility_24h is always NULL in the confounders table.

Root cause: RegimeCollector.calculate_volatility pulled hourly prices from CryptoCompare's histohour endpoint, which now requires an API key (HTTP 401). The failed request was caught and stored as NULL, which flows to get_volatility_state returning "Unknown" and the frontend rendering N/A.

Changes

• New data source (no API key): CoinGecko market_chart (hourly) as primary, with Coinbase public candles as fallback. Both verified reachable from the environment; Binance was rejected (geo-blocked, HTTP 451) and CryptoCompare needs a paid key.
• Fixed the realized-vol formula: previously annualized an already-windowed std (std * sqrt(24*365)), inflating values to ~50–148%; now uses std * sqrt(n) for the window, yielding a sane ~1–6% 24h realized vol that matches the existing get_volatility_state thresholds (<2 Low, <4 Moderate, <6 High).
• Tests: added tests/test_regime_volatility.py covering primary path, Coinbase fallback, both-fail, insufficient-data, and formula scale (5 tests).

Verification

• New tests pass; full suite green (64 passed).
• Live check: BTC 24h realized vol now computes to ~1.95% (was N/A).

Test plan

• CI passes (pytest / ruff / mypy)
• After deploy, dashboard Volatility (24h) card shows a percentage instead of N/A
• confounders.volatility_24h populates in prod DB

🤖 Generated with OpenCode

[protostatis]

Sky's Code Review

Replaces broken CryptoCompare hourly-price source (now requires API key, returns 401) with CoinGecko as primary and Coinbase as fallback for the volatility calculation. Also corrects the realized-vol formula from annualized (inflated ~50-148%) to window-scaled (~1-6%). Tests cover primary, fallback, both-fail, insufficient-data, and formula scale. Changes are clean and well-structured.

Verdict: Approve

Comments

• The formula fix from annualized (std * sqrt(24*365)) to window-scaled (std * sqrt(n)) is correct and well-explained. The new values (~1-6% 24h realized vol) align with the existing get_volatility_state thresholds (<2 Low, <4 Moderate, <6 High), which would have been unreachable with the old annualized formula.
• No security concerns: no secrets/tokens exposed, no user input flows to shell commands or URL construction unsafely, hardcoded constants for API endpoints.
• The PR description is thorough — root cause, data source evaluation (Binance rejected for geo-blocking, CryptoCompare needs paid key), and live verification all documented.

---

Reviewed by Sky — Unchained Sky engineering agent

Inline Comments (could not attach to lines)

crypto_sentiment_crawler/confounders/regime.py:155 — Both except Exception blocks in _fetch_hourly_closes silently swallow all errors (including TypeError, json.JSONDecodeError, etc.). For a fallback pattern this is acceptable, but consider logging at warning level (not debug) for the primary source failure since it indicates a real degradation that operators should notice in logs. The fallback can stay at debug.

crypto_sentiment_crawler/confounders/regime.py:165 — The CoinGecko free/demo API has aggressive rate limits (~10-30 req/min). Consider adding a brief comment noting this, since the fallback to Coinbase exists specifically for this reason. If this endpoint is called for multiple coins (BTC, ETH, SOL) in quick succession, rate-limiting is likely. No code change needed — the fallback handles it — but a comment would help future maintainers.

crypto_sentiment_crawler/confounders/regime.py:181 — Coinbase candles endpoint has a default limit of 300 candles and returns an empty array (not an error) when no data exists for the time range. The if candles: guard handles the empty case, but the code will silently return an empty list if Coinbase returns [], which then falls through to the 'Insufficient price data' error at the caller. This is correct behavior — just noting it for clarity.

crypto_sentiment_crawler/confounders/regime.py:199 — The days calculation max(1, (hours + 1) // 24 + 1) for hours=24 gives days=2, which requests ~48 hourly data points from CoinGecko. This is more than the 25 needed but harmless — the extra closes just extend the return series. Works correctly.

tests/test_regime_volatility.py:62 — Good use of deterministic oscillation (math.sin(i/3.0)) for reproducible test data. Consider adding a test for Coinbase returning an empty array [] (distinct from a 4xx/5xx error) to verify that path is also handled correctly.