Skip to content

feat: impact × uncertainty signal map in the prospective demo - #7

Open
picaultj wants to merge 1 commit into
mainfrom
feat/signal-impact-uncertainty-map
Open

feat: impact × uncertainty signal map in the prospective demo#7
picaultj wants to merge 1 commit into
mainfrom
feat/signal-impact-uncertainty-map

Conversation

@picaultj

Copy link
Copy Markdown
Owner

Summary

Implements "better characterize weak signals (axes impact vs. uncertainty)" — the fourth idea in upstream issue rte-france/BERTrend#59 — as a foresight-style signal map in the prospective demo's signal-analysis view.

Approach (heuristic, data-derived)

The prospective demo loads signals from saved parquet files (scalar metrics only, no popularity time series), so both axes are derived from the metrics already present — no LLM call, no schema change, and it works retroactively on existing models:

  • Impact = normalized Latest_Popularity (magnitude of attention)
  • Uncertainty = 1 − normalized(evidence), where evidence blends Docs_Count and Source_Diversity — a thinly-supported topic (few documents / few sources) is treated as more uncertain

Both axes are min-max normalized to [0, 1] across the model's combined signal set. (A future upgrade could replace/augment these with LLM-scored impact — see the earlier design discussion; this PR is the low-risk first slice.)

Changes

  • signal_characterization.py (new): Streamlit-free helpers
    • compute_signal_characterization(dfs_topics) → per-topic dataframe with impact / uncertainty
    • plot_signal_characterization(char_df, ...) → Plotly quadrant scatter (x=uncertainty, y=impact, color by signal type, size by document count, dashed median lines at 0.5)
  • dashboard_signals.py: renders the map in an expander above the signal tables.
  • i18n_translations.py: FR/EN strings (signal_map_title, signal_map_help, impact_axis, uncertainty_axis).
  • tests/apps/test_signal_characterization.py (new): unit tests.

Notes / limitations

  • The impact/uncertainty formulas are heuristics flagged for calibration; the caption explains what the axes mean to users.
  • Uncertainty here reflects evidence volume, not popularity volatility, because the prospective demo's stored dataframes don't carry the popularity time series (that would be available in the live weak-signals demo, a possible follow-up).

Tests

tests/apps/test_signal_characterization.py: 9 passed — normalization bounds, higher-popularity→higher-impact, more-evidence→lower-uncertainty, signal-type labelling, constant-metric midpoint, missing-column handling, and figure construction.


Generated by Claude Code

Adds a foresight-style characterization of signals (issue rte-france#59, 'better
characterize weak signals - axes impact vs uncertainty') to the signal-analysis
view of the prospective demo.

A heuristic, data-derived approach (no LLM call, works on existing models):
- impact = normalized Latest_Popularity
- uncertainty = 1 - normalized(evidence), evidence blending Docs_Count and
  Source_Diversity (a thinly-supported topic is more uncertain)
Both axes min-max normalized across the model's signals.

- New Streamlit-free helpers compute_signal_characterization() and
  plot_signal_characterization() (signal_characterization.py), unit-tested.
- dashboard_signals renders the quadrant scatter in an expander (color by
  signal type, size by document count, median quadrant lines).
- Added FR/EN translations.
@picaultj
picaultj force-pushed the feat/signal-impact-uncertainty-map branch from 7515c4b to 4c3b0cd Compare August 18, 2026 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant