Skip to content

docs(eval): rewrite README around B1/B2/B3 extracted modules - #20

Merged
nyashkn merged 1 commit into
mainfrom
docs/eval-readme-b4
Jun 9, 2026
Merged

nyashkn merged 1 commit into
mainfrom
docs/eval-readme-b4

Conversation

@nyashkn

@nyashkn nyashkn commented Jun 9, 2026

Copy link
Copy Markdown
Owner

Summary

  • Replace the old Stage-1/2 local-container recipe in `eval/README.md` with a public-surface reference for the now-extracted modules (B1+B2+B3 of the eval-harness extraction).
  • Document `insight_kit.platform.eval.score`, `insight_kit.integrations.omp.session_parser`, `insight_kit.platform.observability.lmnr`, and `insight_kit.narrative.analyst_prompt.render_analyst_prompt` with their exact public surface.
  • Add `growth_insights` as the reference consumer — git+ssh dep pin (`rev = 6f0ade3`), BuildKit `--mount=type=ssh` Dockerfile pattern, `founder_context` kwarg injection, telemetry wiring at `deploy/eval/lmnr_harness.py`.

Constraints surfaced

  • Never touch the shared `LMNR_PROJECT_API_KEY` Infisical secret (kuze blast radius across `kg_rust` + `kg_rust_embed_first`, 25+ consumers).
  • gRPC `LMNR_BASE_URL` is scheme+host with NO port in the URL string — port goes in `LMNR_GRPC_PORT`.
  • Telemetry must never fail the eval run.
  • Pin `insight-kit` by SHA, not by path (Docker build context).

Dropped

  • Stage-1/2 dev history.
  • 0.7778 score-regression incident specifics (now resolved).
  • Blast-radius narrative (resolved by dedicated key).

Test plan

  • Render `eval/README.md` on GitHub — verify tables, code fences, and headings.
  • Verify all referenced symbols exist in the modules under `src/insight_kit/`.
  • Cross-check the `growth_insights` snippets against `deploy/eval/Dockerfile`, `deploy/eval/lmnr_harness.py`, and `src/growth_insights/analyst/runner.py` on `growth_insights@main`.

🤖 Generated with Claude Code

Replace the old Stage-1/2 local-container recipe with a public-surface
reference for the now-extracted modules:

- insight_kit.platform.eval.score — 9-check score gate, T-suffix regex,
  query_path-honouring citation check
- insight_kit.integrations.omp.session_parser — omp 15.2.4 parser
- insight_kit.platform.observability.lmnr — init_from_env, RunIdentity,
  emit_session_spans, score_check_span, read_capped
- insight_kit.narrative.analyst_prompt.render_analyst_prompt — founder_context
  kwarg

Add growth_insights as the reference consumer:
- git+ssh dep pin (rev pinned to current main sha)
- BuildKit --mount=type=ssh + ssh-keyscan github.com Dockerfile pattern
- founder_context injection at AnalystRun.build_prompt()
- Telemetry wiring at deploy/eval/lmnr_harness.py

Constraints called out: never touch shared LMNR_PROJECT_API_KEY (kuze 25+
consumers), gRPC base_url scheme+host with no port, telemetry must never
fail the run, pin insight-kit by sha not path.

Drops: Stage-1/2 dev history, 0.7778 incident specifics, blast-radius
narrative.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@nyashkn
nyashkn merged commit ad127a0 into main Jun 9, 2026
1 check passed
@nyashkn
nyashkn deleted the docs/eval-readme-b4 branch June 9, 2026 00:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant