feat(eval): evaluate the dashboard-summary skill via the /summary endpoint#1646

Open

romrak wants to merge 9 commits into

rr/summary-endpoint-eval

Commits on Jun 4, 2026

feat(eval): add dashboard_summary test kind via /summary endpoint

Roman Rakus
and
claude
committed
fix(eval): resolve active LLM provider setting by type, not fixed id

Roman Rakus
and
claude
committed
feat(eval): robust summary scoring + multi-case example dataset

Roman Rakus
and
claude
committed
Merge remote-tracking branch 'origin/master' into rr/summary-endpoint-eval
Roman Rakus
committed
fix(eval): set provider_type in resolve_and_activate default path

Roman Rakus
and
claude
committed

Commits on Jun 5, 2026

feat(eval): carry summary_input through the Langfuse dataset source

Roman Rakus
and
claude
committed
chore(eval): drop example summary dataset (now lives in Langfuse)

Roman Rakus
and
claude
committed
fix(eval): return SummaryInput from the Langfuse source helper

Roman Rakus
and
claude
committed
feat(eval): set Langfuse trace version to the model id

Roman Rakus
and
claude
committed