Codex/finance nontraining sft v1 20261017

research/finance_program_v1/.gitignore

@@ -0,0 +1,16 @@
+__pycache__/
+*.py[cod]
+.pytest_cache/
+*.safetensors
+*.bin
+*.pt
+*.tar.gz
+.env*
+runs/
+formal/
+trial/
+exports/
+manifest.json
+data/planner200.jsonl
+data/train-labels.json
+

research/finance_program_v1/README.md

@@ -0,0 +1,60 @@
+# Finance Program：non-training 与第一版 Planner SFT
+
+本目录保留 non-training、教师轨迹采集、第一版学生 Planner SFT 的代码，以及完整金融 **Val422** 的同接口配对评估结果。它是独立研究快照，不替换仓库原应用。
+
+## 完整 Val422 结果
+
+两组均为 Qwen3.5-9B，在相同 F66 Program/Skill 框架、输入与执行预算下比较：
+
+- **ProgSkill / `base`**：未微调 Planner，仍调用 Skill，**不是直接回答的纯 Base-LLM**。
+- **DCG-SFT / `sft`**：只加载第一版 Planner SFT A；所有模型型 Skill 继续使用原始基座并禁用 adapter。
+
+共 **422 个唯一配对、844 条正式轨迹**。失败与未终止均保留在每组 422 的分母内，双方无缺失。
+
+| 指标 | ProgSkill：未微调 Planner | DCG-SFT：第一版 Planner SFT | 提升 |
+| --- | ---: | ---: | ---: |
+| 类别正确率 | 79/422，18.72% | **171/422，40.52%** | +21.80 个百分点 |
+| 等级正确率 | 111/422，26.30% | **230/422，54.50%** | +28.20 个百分点 |
+| 类别与等级联合正确率 | 79/422，18.72% | **171/422，40.52%** | +21.80 个百分点 |
+| 合法终止率 | 191/422，45.26% | **420/422，99.53%** | +54.27 个百分点 |
+| 首次 Top-5 命中率 | 177/422，41.94% | **251/422，59.48%** | +17.54 个百分点 |
+| 最终 Top-5 命中率 | 177/422，41.94% | **251/422，59.48%** | +17.54 个百分点 |
+| 任次 Top-5 命中率 | 177/422，41.94% | **251/422，59.48%** | +17.54 个百分点 |
+| 类别 Macro-F1：8 个有支持类别 | 0.18845 | **0.32218** | +0.13373 |
+| 类别 Macro-F1：固定 12 类 | 0.12563 | **0.21479** | +0.08916 |
+| 未合法终止 | 231 | **2** | 减少 229 条 |
+| 缺失轨迹 | 0 | 0 | 无缺失 |
+
+逐 ID 配对联合正确性：**99 条改善、7 条退步，净增加 92 条正确结果**。三种 Top-5 口径分别计算，本轮数值恰好相同。合法终止率不等于业务正确率。
+
+| 调用与成本（422 条/组） | ProgSkill | DCG-SFT | 相对减少 |
+| --- | ---: | ---: | ---: |
+| 模型调用总数 | 2,897 | 1,695 | 41.49% |
+| 平均模型调用数/来源 | 6.86 | 4.02 | 41.49% |
+| 输入 token 总数 | 30,563,818 | 11,128,130 | 63.59% |
+| 输出 token 总数 | 319,905 | 93,618 | 70.74% |
+| 平均来源执行耗时 | 34.03 秒 | 11.03 秒 | 67.58% |
+
+耗时为各来源执行 receipt 的均值，包含失败来源，不是整轮墙钟或服务时延承诺。原已验证部分精确复用，其余来源唯一补充，完整集合无重叠遗漏；没有按结果重跑或筛除。原、新两个阶段墙钟分别保存于 JSON，不能当作同一次完整运行的墙钟测量。
+
+## 代码与结果目录
+
+| 目录 | 内容 |
+| --- | --- |
+| `nontraining/` | 历史 non-training 执行及审计代码；历史接口不作为本表的严格配对对照 |
+| `teacher/` | **百炼 `qwen3.8-2.4t-a95b`** 的教师轨迹采集代码，不是 9B 教师模型 |
+| `sft/` | 第一版 Qwen3.5-9B Planner-only LoRA，200 个训练来源 |
+| `evaluation/` | 原同接口评估、模型路由和控制器回放代码快照；示例数据为合成测试 fixture，不是完整真实验证数据 |
+| `trajectories/` | 教师 200 条、完整验证 `base` 422 条与 `sft` 422 条，共 **1,044 条脱敏公开轨迹** |
+| `tools/`、`publication_tests/` | 轨迹脱敏工具与隐私回归测试 |
+| `results/` | [完整指标](results/evidence-summary.json)、[轨迹清单与 SHA](results/trajectory-export.json)、[模型角色](results/model-roles.json)、[验证说明](results/publication-verification.json) |
+
+## 验证与隐私边界
+
+所有服务器阶段及完整回放成功退出；本机从各自解包目录再次完整回放，并逐 SHA 核验原始证据全部成员及 844 份正式轨迹。逐 ID 指标、配对统计和两种 Macro-F1 已独立重算。聚合 JSON 保留证据 SHA 依赖，但不发布私有目录和来源 ID。
+
+公开轨迹只保留 Program、实际步骤状态、最终类别/等级与原 trace SHA；业务元数据、原始消息、推理文本和语义证据已脱敏。两组学生使用一致的顺序公开编号；无私有 ID 映射。它们**不是训练数据，也不是完整控制器回放输入**。教师轨迹保持不变。
+
+这是开发验证集、单 seed，仅 8/12 类有支持，且来源可能共享表；不是独立 Test 或跨领域泛化结论。教育与钢铁数据集不在本轮结果内。该更新只替换结果展示和学生公开轨迹，不修改实验代码、权重或训练设置。
+
+真实数据集、gold、权重、凭证和实验服务器地址均不上传。clone/import 不会启动任何训练、GPU 或 API 作业。复现说明见 [REPRODUCE.md](REPRODUCE.md)。

research/finance_program_v1/REPRODUCE.md

@@ -0,0 +1,46 @@
+# Reproduction and publication boundaries
+
+## CPU tests
+
+Run stages in separate processes from their own directories; names such as `gate` and `io_utils` are intentionally local:
+
+```powershell
+cd nontraining
+python -B -m pytest tests -q -p no:cacheprovider
+cd ../sft
+python -B -m unittest discover -s tests -v
+cd ../evaluation
+python -B -m unittest discover -s tests -v
+cd ..
+python -B -m unittest discover -s publication_tests -v
+```
+
+Python 3.11+ is expected. The original publication checks used Python 3.13 with Torch 2.9 CPU operations. The actual 9B experiment used Torch 2.9 / Transformers 5.10.4 / PEFT 0.21 / Accelerate 1.15. These CPU tests do not download models or start GPU or teacher API work. The result update does not alter the original experiment code or its synthetic fixtures.
+
+`evaluation/data/study.json` is a fictional field/category fixture, **not the real Val422 dataset**. The private-data prompt integration test and private transport/deployment/download helpers are excluded. Adapter fixtures contain architecture/configuration only, with a placeholder base path.
+
+## Full financial Val422 results
+
+The current [aggregate result](results/evidence-summary.json) covers all 422 unique sources and 844 formal trajectories, comparing an untrained Planner and the accepted first SFT A under the same F66 Program/Skill interface. Both arms call Skills; `base` is not a direct-answer Base-LLM. Model Skills always use the original base without adapter; only the SFT Planner enables the accepted LoRA.
+
+The already verified subset was reused byte-for-byte and all remaining sources were run once under the same frozen protocol. Original smoke trajectories are excluded from formal metrics. The source union is exact and disjoint; failures and nontermination remain in each fixed 422 denominator. Labels were used only locally after generation and replay, never uploaded or inserted into model messages. Separate generation-stage walls are recorded, not represented as one full-run wall measurement.
+
+All remote stages, controller/tokenizer/routing replay and archive steps exited successfully. On Windows the original and supplementary archives were safely extracted and all listed member hashes rechecked; full controller replay ran from each extracted root. The local replay does not claim a local real tokenizer: it is bound to the same-SHA remote real-tokenizer proof. All 844 formal trace hashes, per-ID metrics, paired counts and Macro-F1 were independently rechecked before publication.
+
+Full numerical reproduction requires controlled input study/registry, exact source partition and order, local-only gold, full raw traces, original model revision and accepted first LoRA. These are not public. The `evaluation/` directory retains the original evaluation/replay code snapshot; it is not a self-contained launcher with private full-dataset orchestration or production data. The public JSON and trajectory views are a result release, not a replacement deployment bundle.
+
+For controlled real execution, create a new owned configuration/manifest with authorized assets and the prescribed resource/identity checks. Do not overwrite immutable runs, silently change budgets/labels or infer permission for GPU/API work from this publication.
+
+## Teacher and student trajectories
+
+The current release contains **1,044 lossy projections**: unchanged teacher 200, matched full-validation base 422 and SFT 422. The teacher is **Bailian `qwen3.8-2.4t-a95b`**, not a locally trained 9B teacher. The first student uses 200 teacher-selected sources for Planner-only Program+EOS SFT; Skill output text and evaluation labels are not supervision.
+
+Both student files align on public IDs `val-0001` through `val-0422`. No private source-ID mapping is published. The original trace SHA identifies controlled evidence; it is not the SHA of the redacted row. [trajectory-export.json](results/trajectory-export.json) records public-file sizes, row counts and SHA values, including the unchanged teacher file.
+
+Actual Program structure, attempts, execution statuses and final classification/grade are retained. Prompts, raw reasoning, source metadata and semantic observation values are omitted. Literal arguments are replaced by `[REDACTED]` and flagged; unparseable replies retain status/digest, not raw text. No failed row is removed. This format cannot independently establish semantic correctness and must not be used as training data or full replay input.
+
+The prior partial-validation result tables and standalone partial trajectory files are no longer part of the current branch tree. Original raw evidence and its cryptographic provenance remain preserved privately; the first SFT weights, teacher trajectories and experiment source code are unchanged.
+
+## Interpretation
+
+Val422 is a development set with eight supported classes out of twelve, one seed and potentially correlated table sources. It is not held-out Test or independent generalization. Supported-class and fixed-twelve-class Macro-F1 are reported separately; legal termination is not business correctness. Historical non-training interfaces are not interchangeable with this matched F66 comparison.

research/finance_program_v1/evaluation/archive.py

@@ -0,0 +1,21 @@
+"""After writer exit, snapshot all frozen code, logs, inputs, calls, routes and traces."""
+import tarfile
+from io_utils import ROOT,verify,put,read,sha
+
+
+def main():
+    verify();read(ROOT/'runs/exit.json')
+    out=ROOT/'exports/final-01';out.mkdir(parents=True,exist_ok=False)
+    files=sorted(p for p in ROOT.rglob('*') if p.is_file() and '__pycache__' not in p.parts
+        and 'exports' not in p.relative_to(ROOT).parts and p.relative_to(ROOT).as_posix()!='runs/watch.log')
+    entries=[{'path':p.relative_to(ROOT).as_posix(),'sha256':sha(p),'bytes':p.stat().st_size} for p in files]
+    put(out/'members.json',entries)
+    archive=out/'student-val80-evidence.tar.gz'
+    with archive.open('xb') as stream,tarfile.open(fileobj=stream,mode='w:gz') as tar:
+        for p in files:tar.add(p,arcname=p.relative_to(ROOT).as_posix(),recursive=False)
+        tar.add(out/'members.json',arcname='archive-members.json',recursive=False)
+    put(out/'archive.json',{'archive_sha256':sha(archive),'bytes':archive.stat().st_size,
+        'members':len(entries)+1,'members_sha256':sha(out/'members.json'),'exit':read(ROOT/'runs/exit.json')})
+
+
+if __name__=='__main__':main()

research/finance_program_v1/evaluation/backend.py

@@ -0,0 +1,79 @@
+"""Full-context frozen local generation with measured Planner-only LoRA routing."""
+from copy import deepcopy
+from pathlib import Path
+import time
+from io_utils import ROOT, read, put, sha
+from gate import enabled_for,eos_options,validate_adapter_files
+from local_smoke import Backend,encode_messages,generation_metadata
+
+
+def layer_state(model,disabled):
+    layers=[m for _,m in model.named_modules() if hasattr(m,'lora_A') and hasattr(m,'disable_adapters')]
+    if len(layers)!=40 or any(bool(m.disable_adapters)!=disabled or list(m.active_adapters)!=['default'] for m in layers):
+        raise RuntimeError('actual LoRA layer state differs from requested route')
+    return {'layer_count':len(layers),'disabled':disabled}
+
+
+class RoutedBackend(Backend):
+    def __init__(self,model,adapter,anchor,cohort):
+        if read(Path(model)/'model-manifest.json')!=read(ROOT/'model-identity.json'):
+            raise ValueError('base model manifest differs from frozen original identity')
+        validate_adapter_files(adapter,anchor)
+        super().__init__(model,ROOT/'runs'/(cohort+'-backend-init'))
+        from peft import PeftModel
+        validate_adapter_files(adapter,anchor)
+        self.model=PeftModel.from_pretrained(self.model,adapter,is_trainable=False,autocast_adapter_dtype=False)
+        validate_adapter_files(adapter,anchor)
+        self.model.to(dtype=self.torch.float16);self.model.requires_grad_(False);self.model.eval()
+        self.parameter_devices=sorted({str(p.device) for p in self.model.parameters()})
+        self.parameter_dtypes=sorted({str(p.dtype) for p in self.model.parameters() if p.is_floating_point()})
+        if self.parameter_devices!=['cuda:0'] or self.parameter_dtypes!=['torch.float16']:
+            raise RuntimeError('entire model and adapter must be FP16 cuda:0')
+        self.options['eos_token_id']=eos_options(self.model.generation_config.eos_token_id,self.tokenizer.eos_token_id)
+        if self.options['eos_token_id']!=[248044,248046]:raise ValueError('model/tokenizer EOS identity differs')
+        self.system=read(ROOT/'data/protocol.json')['planner_system'];self.anchor=anchor
+        self.model.base_model.disable_adapter_layers();layer_state(self.model,True)
+        put(ROOT/'runs'/(cohort+'-backend-loaded.json'),{'adapter':str(adapter),'adapter_sha256':anchor['adapter_sha256'],
+            'base_revision':self.identity['revision'],'parameter_devices':self.parameter_devices,
+            'parameter_dtypes':self.parameter_dtypes,'generation':self.options,'targets':40})
+
+    def start_source(self,folder,arm):
+        self.arm=arm;self.call_dir=Path(folder)/'calls';self.call_dir.mkdir(parents=True,exist_ok=False)
+        self.calls=[];self.failures=[]
+
+    def generate(self,messages):
+        enabled=enabled_for(self.arm,messages,self.system)
+        call=self.call_dir/('call-%04d'%(len(self.calls)+1));call.mkdir();self.calls.append(str(call))
+        self.last_generation_metadata={};started=time.perf_counter()
+        q={'messages':deepcopy(messages),'generation':self.options};put(call/'request.json',q)
+        x={'messages':deepcopy(messages),'raw_text':None,'input_tokens':None,'output_tokens':None,
+            'generated_ids':[],'input_devices':[],'parameter_devices':self.parameter_devices,
+            'parameter_dtypes':self.parameter_dtypes,'generation':self.options,'truncation_at_budget':False,'error':None}
+        route={'arm':self.arm,'enabled':enabled,'layer_count':40,'entered_disabled':None,'restored_disabled':None,
+            'adapter_sha256':self.anchor['adapter_sha256']}
+        try:
+            if enabled:self.model.base_model.enable_adapter_layers()
+            else:self.model.base_model.disable_adapter_layers()
+            try:
+                state=layer_state(self.model,not enabled);route['entered_disabled']=state['disabled']
+                encoded=encode_messages(self.tokenizer,messages)
+                x['input_tokens']=encoded['input_ids'].shape[1]
+                encoded={k:v.to('cuda:0') for k,v in encoded.items()};x['input_devices']=['cuda:0']
+                self.torch.cuda.synchronize(0)
+                with self.torch.inference_mode():output=self.model.generate(**encoded,**self.options)
+                self.torch.cuda.synchronize(0)
+                ids=output[0,x['input_tokens']:].tolist();x['generated_ids']=ids;x['output_tokens']=len(ids)
+                x['truncation_at_budget']=len(ids)>=512
+                x['raw_text']=self.tokenizer.decode(ids,skip_special_tokens=True,clean_up_tokenization_spaces=False)
+                return x['raw_text']
+            finally:
+                self.model.base_model.disable_adapter_layers()
+        except Exception as exc:
+            x['error']=type(exc).__name__+': '+str(exc);self.failures.append(x['error']);raise
+        finally:
+            route['restored_disabled']=layer_state(self.model,True)['disabled']
+            x['seconds']=time.perf_counter()-started
+            self.last_generation_metadata=generation_metadata(x)
+            put(call/'result.json',x)
+            route.update(request_sha256=sha(call/'request.json'),result_sha256=sha(call/'result.json'))
+            put(call/'route.json',route)