Skip to content

Add processed-payload telemetry and prompt duplicate shadow#53

Open
SecretSettler wants to merge 1 commit into
feat/modular-context-opportunity-analyzerfrom
feat/actual-token-prompt-shadow
Open

Add processed-payload telemetry and prompt duplicate shadow#53
SecretSettler wants to merge 1 commit into
feat/modular-context-opportunity-analyzerfrom
feat/actual-token-prompt-shadow

Conversation

@SecretSettler

Copy link
Copy Markdown
Member

Summary

  • Add metadata-only before/after processed-payload char counters to ContextPilot telemetry
  • Add opt-in tokenizer-based before/after token counters; tokenizer measurement is off by default and no fake token numbers are emitted when unavailable
  • Label legacy chars/4 values as derived, not actual tokens, across savings script/docs/reports
  • Add advisory-only system/skill prompt duplicate shadow section to the Hermes opportunity analyzer
  • No new replacement, summarization, routing, or prompt rewriting behavior

Verification

  • python3 -m pytest tests/test_hermes_plugin_patch.py tests/test_hermes_context_opportunity_analyzer.py tests/test_contextpilot_savings.py tests/test_contextpilot_savings_skill.py tests/test_hermes_contextpilot_monitor.py -q
  • python3 -m pytest -q
  • python3 scripts/analyze_hermes_context_opportunities.py --since-hours 24 --out-dir /tmp/contextpilot_actual_prompt_check --date actual_prompt_check --top-n 5
  • python3 scripts/contextpilot_savings.py --since-hours 24
  • Claude Code read-only final review: APPROVE, no blockers

Stacked on PR #52 (modular analyzer refactor).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant