feat: add skill opinion outcome performance statistics - #2
Open
ObVious55 wants to merge 8 commits into
Open
Conversation
🤖 自动审查报告
📁 修改的文件
|
* feat: add skill opinion outcome evaluation core * feat: add skill opinion outcome evaluation core * fix: unify expected-start resolution * changelog * fix: validate persisted daily start sessions * fix: preserve legacy local backtest windows
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Type
Background And Problem
#1904 P2 要求 Skill 权重只能基于真实、可归因的 outcome 样本进行校准;样本不足时必须保持 neutral / observational,不能通过其他 skill、horizon、engine version 或非 evaluated 记录拼接样本量。
父 PR #2116 已建立 individual SkillAgent outcome 的计算和持久化核心,但明确未提供表现统计、样本充足度、排名或权重调整。
本 stacked PR 在 ZhuLinsen#2116 的持久化事实之上增加只读表现统计:
skill_id + horizon + engine_version独立聚合;evaluated >= 30时输出表现指标;null;Scope Of Change
当前描述基于:
ObVious55:1904-p2-skill-opinion-outcomes-v294c2c6906882471331bcb66405151025c4ed29fbTODO: 提交后补充Repository
src/repositories/skill_opinion_outcome_repo.pySkillOpinionPerformanceBucket;skill_id + horizon + engine_version聚合;Service
src/services/skill_opinion_performance_service.pySkillOpinionPerformanceService;sample_status=observational,所有表现指标为null;hit + miss作为分母;evaluated + observational + unable作为分母,排除 pending;Tests
tests/test_skill_opinion_outcome_stats.pynull;Documentation
docs/multi-strategy-contract.mddocs/CHANGELOG.md[Unreleased]扁平列表中记录只读 Skill Opinion 表现统计能力。Issue Link
Refs #1904
Stacked on #2116.
本 PR 只实现 P2 的只读表现统计和样本充足度契约,不声称完成保守权重接入,也不关闭整个 Issue。
Verification Commands And Results
Executed local verification
结果:
103 passed, 3 warnings in 68.18s。结果:通过,无输出。
结果:通过,无输出。
git diff --check结果:已跟踪文件通过,无 whitespace error;Git 仅提示部分文件未来可能执行 LF → CRLF 转换。
Full-suite note
曾启动:
该命令选择了 5056 个测试。本地运行至约 10% 时,
tests/test_agent_backend_status_service.py附近出现与本机 Codex app-server / protocol 环境相关的失败,随后停止执行;该命令未完成,不能记为通过。CI status
当前分支尚未 push,PR 尚未创建,因此当前 Head CI 不可用:
ai-governance:TODObackend-gate:TODOdocker-build:TODOweb-gate:TODO(按改动路径预计不触发,仍以实际 CI 为准)Visual Evidence (if applicable)
不适用。
本 PR 只修改后端 repository/service、测试和专题文档,不修改报告格式、报告渲染、Web UI、Web 设置或桌面端界面。
Compatibility And Risk
BacktestService.get_skill_summary()、AgentMemory、SkillAggregator或最终投资建议。Rollback Plan
最小回滚方式:
本 PR 不涉及数据库迁移、数据写入入口或运行时配置,因此无需额外的数据和配置回滚。父 PR ZhuLinsen#2116 的 outcome 数据可以继续保留。
EXTRACT_PROMPT Change (if applicable)
不适用:本 PR 未修改
src/services/image_stock_extractor.py或EXTRACT_PROMPT。Checklist
docs/CHANGELOG.md