Repository navigation
Conversation
Signed-off-by: Virtue039 <virtue039@qq.com>
3 of 4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A model can use a grounded statement subject while omitting its declaration from
e. Three recorded identity-bench replies do this for 李娜; extraction currently drops the statements even when the base already has one matching named entity. This is a coverage defect within the existing open extraction format (related to #1136).What changes
Recover an omitted subject only from a uniquely located, single-sentence quote with one subject owner and one occurrence of its name. Require one active existing exact-name/known-as holder in the same base, with generic-name alternatives also preventing an ambiguous recovery. Preserve the ordinary profile/grey-zone rules and typed-holder threshold. The lookup does not create a subject, reclassify it, or queue a review.
Keep recovered references in a subject-only map. Existing declared/described references, alias declarations, unlisted-object literal fallback, pushed statements and memory confirmation retain their current paths. Repeated or conflicting quotes, invalidated aliases, merged/foreign/described holders and longer declared names have controls. Retaining a statement can still materialize its explicitly declared described object and affect later model work; total entities/calls remain measured outcomes.
How it was checked
Same-session measurement and limits
Baseline
1142792979b61272d92cf322cbfd2f86615f6cb3, candidatef908a102942a726497af861e9aa882b1031e717b; fixedqwen3.7-flash-2026-07-15, thinking off, BGE-M3 revision5617a9f61b028005a4858fdac845db406aefb181. Two rounds, two independent forward/reverse runs per side each round, stock corpus/scorer/settings unchanged.16 eligible primary KBs; 20 experimental KBs actually run. One proxy/provider timeout and host sleep contaminating two observations caused three originals plus one matched counterpart to be excluded. Whole pairs were replaced once under a recorded infrastructure-only rule, irrespective of scores. Every original result and cost is retained. Two registration-created empty General bases are separate and had no documents/entities/model calls. A long replacement label was rejected before measurement and then shortened; that failed setup created no KB and made no provider call.
No accuracy improvement is claimed: the F1 change is small and reverses direction between rounds, while false merges and human work increase. Across all 20 experimental KBs there were 1,273 chat attempts, 1,272 successful calls and 1,726,727 reported successful-request tokens; the failed request's actual billed tokens are unknown. The 17-token connectivity check is separate.
Attribution: 120 same-state/same-reply controls call each commit's production parser/value cleanup/
apply_open_extraction. 117 are identical; three retain the correct omitted 李娜 statement and its evidence/qualifiers, with two materializing an explicitly declared described object. No review record differs. These controls use reconstructed fixed bench KB settings and synthetic workspace roots, retain existing graph IDs, normalize newly allocated IDs/timestamps, and do not replay async alignment/adjudication.There are 24 new paired error observations (9 false merges, 8 false splits, 7 missing anchors). The first-error document's fixed-state write control is identical on both commits in all 24. The extraction request is identical in all 24, with different provider replies in 23. For example, the awards response sometimes lists one named 张伟 for both people and sometimes lists the finance director and engineer separately; the one-name reply binds identically on both commits. The extra human item is a 李娜 pair whose existing investigation returns
different(0.5)and remains pending underescalate_unsure; this is not a formal different decision. All 12 write controls for that KB are identical.These isolate the changed write branch; they do not exclude all indirect async effects of newly retained facts. Keep draft. The source-reference design (#1454), adjudication policy (#1193) and additional recall channel remain separate work; #1136 remains open.
Primary per-KB measurements, missing anchors and call costs
[ { "round": 1, "repeat": 1, "order": "forward", "side": "baseline", "kb": "01a181f0-946b-7aa0-9e15-cedfda10aca7", "source_label": "baseline-r1-b1", "f1": 0.667, "pairs": { "tp": 6, "fp": 0, "fn": 6, "tn": 93 }, "false_merges": 0, "false_splits": 6, "covered_anchors": 15, "total_anchors": 21, "coverage": 0.7142857142857143, "missing": [ "d00-inst", "d00-zw", "d02-p1-lead", "d06-p2-lead", "d06-p2-depth", "d08-p2" ], "split_in_doc": [], "entities": 26, "pending_human_reviews": 0, "pending_reviews": [], "chat_calls": 58, "chat_attempts": 58, "failed_calls": 0, "failed_request_snapshots": [], "valid_for_clean_comparison": true, "settle_timeout": false, "unknown_tokens_on_failed_requests": false, "prompt_tokens": 76888, "completion_tokens": 4346, "total_tokens": 81234, "unfinished_documents": 0, "bad_jobs": [] }, { "round": 1, "repeat": 1, "order": "forward", "side": "candidate", "kb": "01a181f0-946b-7c30-92cf-e4b56b4283a5", "source_label": "candidate-r1-b1", "f1": 0.692, "pairs": { "tp": 9, "fp": 3, "fn": 5, "tn": 103 }, "false_merges": 3, "false_splits": 5, "covered_anchors": 16, "total_anchors": 21, "coverage": 0.7619047619047619, "missing": [ "d00-inst", "d02-p1-lead", "d06-p2-lead", "d06-p2-depth", "d08-p2" ], "split_in_doc": [], "entities": 29, "pending_human_reviews": 0, "pending_reviews": [], "chat_calls": 59, "chat_attempts": 59, "failed_calls": 0, "failed_request_snapshots": [], "valid_for_clean_comparison": true, "settle_timeout": false, "unknown_tokens_on_failed_requests": false, "prompt_tokens": 78742, "completion_tokens": 4455, "total_tokens": 83197, "unfinished_documents": 0, "bad_jobs": [] }, { "round": 1, "repeat": 1, "order": "reverse", "side": "baseline", "kb": "01a181f3-9211-7110-87df-97d9e1ff7a75", "source_label": "baseline-r1-b1", "f1": 0.538, "pairs": { "tp": 7, "fp": 3, "fn": 9, "tn": 134 }, "false_merges": 3, "false_splits": 9, "covered_anchors": 18, "total_anchors": 21, "coverage": 0.8571428571428571, "missing": [ "d03-p1", "d06-p2-lead", "d06-p2-depth" ], "split_in_doc": [], "entities": 28, "pending_human_reviews": 0, "pending_reviews": [], "chat_calls": 63, "chat_attempts": 63, "failed_calls": 0, "failed_request_snapshots": [], "valid_for_clean_comparison": true, "settle_timeout": false, "unknown_tokens_on_failed_requests": false, "prompt_tokens": 77054, "completion_tokens": 4581, "total_tokens": 81635, "unfinished_documents": 0, "bad_jobs": [] }, { "round": 1, "repeat": 1, "order": "reverse", "side": "candidate", "kb": "01a181f3-61ab-7d63-bc99-50a1e92ccb6f", "source_label": "candidate-r1-b1", "f1": 0.526, "pairs": { "tp": 5, "fp": 2, "fn": 7, "tn": 106 }, "false_merges": 2, "false_splits": 7, "covered_anchors": 16, "total_anchors": 21, "coverage": 0.7619047619047619, "missing": [ "d00-inst", "d00-zw", "d02-p1-lead", "d06-p2-lead", "d08-p2" ], "split_in_doc": [], "entities": 27, "pending_human_reviews": 0, "pending_reviews": [], "chat_calls": 68, "chat_attempts": 68, "failed_calls": 0, "failed_request_snapshots": [], "valid_for_clean_comparison": true, "settle_timeout": false, "unknown_tokens_on_failed_requests": false, "prompt_tokens": 84233, "completion_tokens": 4514, "total_tokens": 88747, "unfinished_documents": 0, "bad_jobs": [] }, { "round": 1, "repeat": 2, "order": "forward", "side": "baseline", "kb": "01a1822c-7d21-7301-9c69-2545dbcc40cd", "source_label": "replacement-r1-b2-forward-baseline", "f1": 0.533, "pairs": { "tp": 4, "fp": 3, "fn": 4, "tn": 67 }, "false_merges": 3, "false_splits": 4, "covered_anchors": 13, "total_anchors": 21, "coverage": 0.6190476190476191, "missing": [ "d00-inst", "d02-p1-lead", "d03-p1", "d06-ln", "d06-p2-lead", "d06-p2-depth", "d08-p2", "d09-p1" ], "split_in_doc": [], "entities": 25, "pending_human_reviews": 0, "pending_reviews": [], "chat_calls": 57, "chat_attempts": 57, "failed_calls": 0, "failed_request_snapshots": [], "valid_for_clean_comparison": true, "settle_timeout": false, "unknown_tokens_on_failed_requests": false, "prompt_tokens": 75488, "completion_tokens": 4029, "total_tokens": 79517, "unfinished_documents": 0, "bad_jobs": [] }, { "round": 1, "repeat": 2, "order": "forward", "side": "candidate", "kb": "01a1822c-7d24-79c0-852a-d7517ea1ab3b", "source_label": "replacement-r1-b2-forward-candidate", "f1": 0.583, "pairs": { "tp": 7, "fp": 3, "fn": 7, "tn": 103 }, "false_merges": 3, "false_splits": 7, "covered_anchors": 16, "total_anchors": 21, "coverage": 0.7619047619047619, "missing": [ "d00-inst", "d02-p1-lead", "d06-p2-lead", "d06-p2-depth", "d08-p2" ], "split_in_doc": [], "entities": 28, "pending_human_reviews": 0, "pending_reviews": [], "chat_calls": 61, "chat_attempts": 61, "failed_calls": 0, "failed_request_snapshots": [], "valid_for_clean_comparison": true, "settle_timeout": false, "unknown_tokens_on_failed_requests": false, "prompt_tokens": 82200, "completion_tokens": 4509, "total_tokens": 86709, "unfinished_documents": 0, "bad_jobs": [] }, { "round": 1, "repeat": 2, "order": "reverse", "side": "baseline", "kb": "01a181fb-43f8-78b1-a632-c95bc5cda9c2", "source_label": "baseline-r1-b2", "f1": 0.583, "pairs": { "tp": 7, "fp": 3, "fn": 7, "tn": 119 }, "false_merges": 3, "false_splits": 7, "covered_anchors": 17, "total_anchors": 21, "coverage": 0.8095238095238095, "missing": [ "d00-inst", "d02-p1-lead", "d06-p2-lead", "d06-p2-depth" ], "split_in_doc": [], "entities": 29, "pending_human_reviews": 0, "pending_reviews": [], "chat_calls": 66, "chat_attempts": 66, "failed_calls": 0, "failed_request_snapshots": [], "valid_for_clean_comparison": true, "settle_timeout": false, "unknown_tokens_on_failed_requests": false, "prompt_tokens": 82061, "completion_tokens": 4583, "total_tokens": 86644, "unfinished_documents": 0, "bad_jobs": [] }, { "round": 1, "repeat": 2, "order": "reverse", "side": "candidate", "kb": "01a181fe-9bf2-7f10-949b-8093de718b0d", "source_label": "candidate-r1-b2", "f1": 0.571, "pairs": { "tp": 6, "fp": 3, "fn": 6, "tn": 90 }, "false_merges": 3, "false_splits": 6, "covered_anchors": 15, "total_anchors": 21, "coverage": 0.7142857142857143, "missing": [ "d00-inst", "d02-p1-lead", "d03-p1", "d06-p2-lead", "d06-p2-depth", "d08-p2" ], "split_in_doc": [], "entities": 27, "pending_human_reviews": 0, "pending_reviews": [], "chat_calls": 62, "chat_attempts": 62, "failed_calls": 0, "failed_request_snapshots": [], "valid_for_clean_comparison": true, "settle_timeout": false, "unknown_tokens_on_failed_requests": false, "prompt_tokens": 81077, "completion_tokens": 4180, "total_tokens": 85257, "unfinished_documents": 0, "bad_jobs": [] }, { "round": 2, "repeat": 1, "order": "forward", "side": "baseline", "kb": "01a18230-72e3-7691-b9dc-eee2a004bd3c", "source_label": "replacement-r2-b1-forward-baseline", "f1": 0.583, "pairs": { "tp": 7, "fp": 3, "fn": 7, "tn": 103 }, "false_merges": 3, "false_splits": 7, "covered_anchors": 16, "total_anchors": 21, "coverage": 0.7619047619047619, "missing": [ "d00-inst", "d02-p1-lead", "d06-p2-lead", "d06-p2-depth", "d08-p2" ], "split_in_doc": [], "entities": 27, "pending_human_reviews": 0, "pending_reviews": [], "chat_calls": 58, "chat_attempts": 58, "failed_calls": 0, "failed_request_snapshots": [], "valid_for_clean_comparison": true, "settle_timeout": false, "unknown_tokens_on_failed_requests": false, "prompt_tokens": 75114, "completion_tokens": 4257, "total_tokens": 79371, "unfinished_documents": 0, "bad_jobs": [] }, { "round": 2, "repeat": 1, "order": "forward", "side": "candidate", "kb": "01a18230-72e2-7a30-aa99-53c1d4254bf3", "source_label": "replacement-r2-b1-forward-candidate", "f1": 0.545, "pairs": { "tp": 6, "fp": 3, "fn": 7, "tn": 120 }, "false_merges": 3, "false_splits": 7, "covered_anchors": 17, "total_anchors": 21, "coverage": 0.8095238095238095, "missing": [ "d00-inst", "d02-p1-lead", "d03-p1", "d06-p2-lead" ], "split_in_doc": [], "entities": 30, "pending_human_reviews": 0, "pending_reviews": [], "chat_calls": 68, "chat_attempts": 68, "failed_calls": 0, "failed_request_snapshots": [], "valid_for_clean_comparison": true, "settle_timeout": false, "unknown_tokens_on_failed_requests": false, "prompt_tokens": 83921, "completion_tokens": 4499, "total_tokens": 88420, "unfinished_documents": 0, "bad_jobs": [] }, { "round": 2, "repeat": 1, "order": "reverse", "side": "baseline", "kb": "01a1821e-92fd-7480-adae-3fb6573f8eab", "source_label": "baseline-r2-b1", "f1": 0.5, "pairs": { "tp": 4, "fp": 0, "fn": 8, "tn": 93 }, "false_merges": 0, "false_splits": 8, "covered_anchors": 15, "total_anchors": 21, "coverage": 0.7142857142857143, "missing": [ "d00-inst", "d02-p1-lead", "d06-p2-lead", "d06-p2-depth", "d07-zw-fin", "d08-p2" ], "split_in_doc": [], "entities": 29, "pending_human_reviews": 0, "pending_reviews": [], "chat_calls": 61, "chat_attempts": 61, "failed_calls": 0, "failed_request_snapshots": [], "valid_for_clean_comparison": true, "settle_timeout": false, "unknown_tokens_on_failed_requests": false, "prompt_tokens": 76097, "completion_tokens": 4389, "total_tokens": 80486, "unfinished_documents": 0, "bad_jobs": [] }, { "round": 2, "repeat": 1, "order": "reverse", "side": "candidate", "kb": "01a1821e-cff5-7dc2-b8d4-4a8e502fb364", "source_label": "candidate-r2-b1", "f1": 0.583, "pairs": { "tp": 7, "fp": 3, "fn": 7, "tn": 119 }, "false_merges": 3, "false_splits": 7, "covered_anchors": 17, "total_anchors": 21, "coverage": 0.8095238095238095, "missing": [ "d00-inst", "d02-p1-lead", "d06-p2-lead", "d06-p2-depth" ], "split_in_doc": [], "entities": 27, "pending_human_reviews": 0, "pending_reviews": [], "chat_calls": 70, "chat_attempts": 70, "failed_calls": 0, "failed_request_snapshots": [], "valid_for_clean_comparison": true, "settle_timeout": false, "unknown_tokens_on_failed_requests": false, "prompt_tokens": 93122, "completion_tokens": 4874, "total_tokens": 97996, "unfinished_documents": 0, "bad_jobs": [] }, { "round": 2, "repeat": 2, "order": "forward", "side": "baseline", "kb": "01a18222-de77-7c73-b772-5bef23532afc", "source_label": "baseline-r2-b2", "f1": 0.667, "pairs": { "tp": 7, "fp": 0, "fn": 7, "tn": 106 }, "false_merges": 0, "false_splits": 7, "covered_anchors": 16, "total_anchors": 21, "coverage": 0.7619047619047619, "missing": [ "d00-inst", "d02-p1-lead", "d06-p2-lead", "d06-p2-depth", "d08-p2" ], "split_in_doc": [], "entities": 28, "pending_human_reviews": 0, "pending_reviews": [], "chat_calls": 66, "chat_attempts": 66, "failed_calls": 0, "failed_request_snapshots": [], "valid_for_clean_comparison": true, "settle_timeout": false, "unknown_tokens_on_failed_requests": false, "prompt_tokens": 84139, "completion_tokens": 4582, "total_tokens": 88721, "unfinished_documents": 0, "bad_jobs": [] }, { "round": 2, "repeat": 2, "order": "forward", "side": "candidate", "kb": "01a18222-de6e-7090-af88-dbae49944330", "source_label": "candidate-r2-b2", "f1": 0.692, "pairs": { "tp": 9, "fp": 3, "fn": 5, "tn": 103 }, "false_merges": 3, "false_splits": 5, "covered_anchors": 16, "total_anchors": 21, "coverage": 0.7619047619047619, "missing": [ "d00-inst", "d02-p1-lead", "d06-p2-lead", "d06-p2-depth", "d08-p2" ], "split_in_doc": [], "entities": 26, "pending_human_reviews": 0, "pending_reviews": [], "chat_calls": 64, "chat_attempts": 64, "failed_calls": 0, "failed_request_snapshots": [], "valid_for_clean_comparison": true, "settle_timeout": false, "unknown_tokens_on_failed_requests": false, "prompt_tokens": 81855, "completion_tokens": 4429, "total_tokens": 86284, "unfinished_documents": 0, "bad_jobs": [] }, { "round": 2, "repeat": 2, "order": "reverse", "side": "baseline", "kb": "01a18226-d44d-7a12-ad8d-52faeed4de6e", "source_label": "baseline-r2-b2", "f1": 0.6, "pairs": { "tp": 6, "fp": 3, "fn": 5, "tn": 77 }, "false_merges": 3, "false_splits": 5, "covered_anchors": 14, "total_anchors": 21, "coverage": 0.6666666666666666, "missing": [ "d00-inst", "d02-p1-lead", "d03-p1", "d06-p2-lead", "d06-p2-depth", "d08-p2", "d09-p1" ], "split_in_doc": [], "entities": 27, "pending_human_reviews": 0, "pending_reviews": [], "chat_calls": 58, "chat_attempts": 58, "failed_calls": 0, "failed_request_snapshots": [], "valid_for_clean_comparison": true, "settle_timeout": false, "unknown_tokens_on_failed_requests": false, "prompt_tokens": 74076, "completion_tokens": 4421, "total_tokens": 78497, "unfinished_documents": 0, "bad_jobs": [] }, { "round": 2, "repeat": 2, "order": "reverse", "side": "candidate", "kb": "01a18226-3ff0-7dd0-9368-b16627ad3378", "source_label": "candidate-r2-b2", "f1": 0.5, "pairs": { "tp": 5, "fp": 3, "fn": 7, "tn": 105 }, "false_merges": 3, "false_splits": 7, "covered_anchors": 16, "total_anchors": 21, "coverage": 0.7619047619047619, "missing": [ "d00-inst", "d02-p1-lead", "d03-p1", "d06-p2-lead", "d08-p2" ], "split_in_doc": [], "entities": 30, "pending_human_reviews": 1, "pending_reviews": [ { "stage": "human", "reason": "escalate_unsure", "n": 1 } ], "chat_calls": 72, "chat_attempts": 72, "failed_calls": 0, "failed_request_snapshots": [], "valid_for_clean_comparison": true, "settle_timeout": false, "unknown_tokens_on_failed_requests": false, "prompt_tokens": 95809, "completion_tokens": 4977, "total_tokens": 100786, "unfinished_documents": 0, "bad_jobs": [] } ]Before review