Repository navigation
Conversation
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
| def _remaining_seconds(deadline: float) -> float: | ||
| """Return what is left of a preparation's embedding budget; an exhausted budget times out at once.""" | ||
|
|
||
| return max(deadline - asyncio.get_running_loop().time(), 0.0) |
There was a problem hiding this comment.
[P2] 预算耗尽后的回退仍记为 inference_timeout，与真实的 provider 超时不可区分
_remaining_seconds 把已耗尽的预算钳成 0.0，下游 asyncio.timeout(0.0) 在第一次 await 处立即抛 TimeoutError，_fallback_reason 因此给出 inference_timeout。改动前每次检索各有独立预算，memory.search.embedding_fallback 的 error_code=inference_timeout 必然对应一次真实的 provider 等待；改动后同一 preparation 里第二次及之后的检索一次都没有等待，仍落同一条事件。
实测（probes/probe_fallback_codes.py，直接复用本 PR tests/e2e/test_recall_embedding_timeout.py 的夹具：recall_embedding_timeout_seconds=0.2、provider delay 5 s、当前 Scope + 2 个 context reference）：
status ready reason inference_timeout
waits [0.2012, 0.0001, 0.0]
fallback error_codes ['inference_timeout', 'inference_timeout', 'inference_timeout']
后两条检索的实际等待是 0.1 ms 与 0 s，事件与 PreparedContext.reason 仍写 inference_timeout。
影响：按 error_code 统计时，"预算被前一次检索吃掉"会被计成 provider 超时，把排查引向 embedding 服务延迟而不是 POWERCONTEXT_SERVER_RUNTIME_RECALL_EMBEDDING_TIMEOUT_SECONDS；Scope 越多伪超时越多（N 个 Scope 最多 N-1 条），正好出现在这个 PR 想优化的多 Scope 场景。
建议：为"预算已耗尽"给出独立取值（例如 embedding_budget_exhausted），或在该事件里补 effective_timeout_seconds / 剩余预算字段，让事件能区分"provider 慢"与"本次未等待"；并补一条断言后两次检索不进入 provider 等待的用例——当前 sum(waits) < budget * 1.5 无法区分这两种成因。
There was a problem hiding this comment.
复核 head 4afbcd60（test(context): pin one provider call and one fallback event per spent budget）：本条 P2 已闭合。
- 预算耗尽后的检索现在走
allow_embedding=allow_embedding and embedding_budget.remaining > 0，不再以 0 等待调用 provider；test_recall_embedding_timeout.py的断言由len(embedding.waits) == 3收紧为== 1，并新增(record,) = _fallbacks(caplog)+_assert_fallback_record(record, "inference_timeout")，即整次准备只留一条真实超时的回退事件。 - exact head 树在本机（Windows）跑
tests/e2e/test_recall_embedding_timeout.py与tests/e2e/test_recall_sufficiency_gate.py：45 passed。 - 变异验证（
probes/mutant-src，把两处and embedding_budget.remaining > 0去掉、其余不变）：test_one_budget_bounds_the_embedding_waits_of_a_preparation_across_scopes失败，assert 3 == 1，waits=[0.2103, 2.2e-05, 1.7e-05]且三条Memory search fell back to lexical retrieval⇒ 新断言确实保护了这次修改。 - 预算语义也随之从「墙钟截止」改成「只计等待」，
recall_embedding_timeout_seconds默认 0.25 → 0.5 已在config.py、application.py与docs/{en,zh}/docs/operate/configuration.md（0.5）、prepare-context-text.md（500 ms）同步，未发现残留 0.25。
仍有一处需要一并修掉的文档事实错误（同一根因的另一面，因此写在本线程而不是另开线程）：docs/en/docs/operate/configuration.md:173（docs/zh/docs/operate/configuration.md:160 同句）现在写「A search that finds the budget spent continues with lexical matching only and logs memory.search.embedding_fallback or topic_memory.search.embedding_fallback」，但预算已用尽的检索在 atomic_memory.py 的 _resolve_query_embedding 里于 if not allow_embedding and capabilities.fts: return "text", None, None, 0, None 直接返回，不经过 _log_embedding_fallback；本 PR 自己的用例断言三次检索只有一条回退事件，正是反证。建议把该句改成「只有耗尽预算的那次检索记录回退事件，其后的检索静默改用词法」，或者为「预算已用尽」补一个独立 error_code，二者选一，文档与实现必须一致。
recall_embedding_timeout_seconds applied to each search, so a preparation that searched its Scope, context references and Topic Memory waited a full budget for each of them when the provider was slow. Measured against a stalled provider, a three-Scope preparation took 1.06 s at the 0.25 s default, past the 1 s prepare request of Kimi Code, ZCode and DSH. The budget now covers the whole preparation. Each search reports how long it waited for its query embedding and only that wait is charged, so database reads, Experience recall and reranking between two searches do not take a later search's share. A search that finds the budget spent skips its embedding and matches lexically; the timeout that spent it has already recorded the preparation's degradation cause. That bound lets the default rise to 0.5 s, in RuntimeConfig and in BuiltinRuntime. With text-embedding-3-small through OpenRouter (p50 0.24 s), 0.25 s lost the semantic channel on 20 of 60 measured preparations of the oceanbase#2039 Hook prompt, and 0.5 s on 1 of 60.
ddc14b0 to
5d96bcb
Compare
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
… budget Once the first search of a preparation spends the embedding budget, the later searches must not call the provider or log another fallback event, or memory.search.embedding_fallback would count waits that never reached the provider as provider timeouts. Summing the waits could not tell a skipped embedding from a very short one.
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
Which issue or RFC does this PR close?
Refs #2039. This addresses the default value. The recall-statistics item stays open there: it needs a decision in #2218 on where the degradation cause lives.
Rationale for this change
recall_embedding_timeout_seconds(default 0.25 s) is applied to each search inside a context preparation. A preparation searches its Scope, each context reference and Topic Memory in turn. When the provider is slow, each of those searches waits a full budget for the same query embedding, so the waits add up. That leaves two problems:text-embedding-3-smallthrough OpenRouter. A Hook query is a whole prompt, which usually fails the lexical admission floor, so a lost embedding usually meansstatus: "empty".The fix makes the budget the preparation's: the total wait for query embeddings is bounded however many Scopes and families are searched. With that bound in place, the default can be sized to a real provider and still fit a one-second Hook request.
What changes are included in this PR?
ScopedContextApplication._prepare_buildcreates one embedding budget before the first search and passes it through the recall rounds. Each Atomic Memory and Topic Memory search may wait for its query embedding only as long as the budget has left, and afterwards the budget is charged with the time that search actually waited. The searches report that wait (AtomicMemorySearchPage.embedding_wait_seconds,TopicMemorySearchResult.embedding_wait_seconds; internal, excluded from equality and serialization respectively). Only embedding waits are charged, so database reads, Experience recall and reranking between two searches do not take a later search's share.reasonandmemory.search.embedding_fallback/topic_memory.search.embedding_fallbackbehave as before. The budget can only be spent by such a timeout, so a later search that finds it spent skips its embedding request and matches lexically without losing the recorded cause. Expansion rounds already reuse round zero's outcome and are unchanged.RuntimeConfigand in theBuiltinRuntimeconstructor.operate/configuration.mdandworkflows/prepare-context-text.mddescribe the shared budget, what is charged to it, and the 1 s host request timeouts it has to fit.Limitation: Topic Memory reuses the Atomic Memory query vector only when both use the same embedding model and the query fits Topic search (64 distinct terms, 8,192 characters). Otherwise Topic needs a second embedding round trip, which now has to fit in what Atomic Memory left of the same budget. With a 0.3 s provider and the 0.5 s default, that second round trip times out and Topic falls back to FTS. Issuing the two embeddings concurrently would remove this, but is a larger change than this PR.
Are there any user-facing changes?
Yes, no API or persisted-format change:
POWERCONTEXT_SERVER_RUNTIME_RECALL_EMBEDDING_TIMEOUT_SECONDSfor the old per-search meaning now gets that value as the preparation's total. A per-search value tuned for N Scopes would need scaling by N to keep the old worst case. The point of the change is that the old worst case did not fit the Hook request.How was this change tested?
tests/e2e/test_recall_embedding_timeout.py::test_one_budget_bounds_the_embedding_waits_of_a_preparation_across_scopes. It runs a Scope with two context references and a query embedding that never answers. Before the fix every search waits a full budget:[0.201, 0.201, 0.200], so the test fails. After the fix only the first search calls the provider and waits within the budget, the later searches skip their embedding, and exactly onememory.search.embedding_fallbackevent (inference_timeout) is logged, so the event counts only the real provider timeout. Lexical recall andreason: "inference_timeout"are kept.tests/e2e/test_recall_sufficiency_gate.py::test_work_between_embedding_waits_does_not_spend_the_preparation_budget. Atomic Memory and Topic Memory each embed the query in 0.15 s, with a 0.3 s Experience recall between them and a 0.4 s budget. Topic Memory must still get its embedding. Charging elapsed time instead of waits leaves Topic nothing, and the test fails.tests/e2e/test_recall_sufficiency_gate.py: three_recall_roundwrappers that spell out its keyword arguments now passembedding_budgetthrough;_seed_topic_memoriescan publish vectors for a runtime with an embedding model.tests/builtin/runtime/test_recall_embedding_timeout_config.pyasserts the 0.5 s default.make check;make unit-test(5825 passed, 243 skipped).powercontext server runwith SQLite and enforced auth,openai/text-embedding-3-smallthrough OpenRouter, the bug:prepare_contextabandons the query embedding after 0.25 s, so Hook recall silently degrades to lexical search with a remote embedding provider #2039 long query, each Scope holding the bug:prepare_contextabandons the query embedding after 0.25 s, so Hook recall silently degrades to lexical search with a remote embedding provider #2039 fact. Settings were interleaved per round, with a Server restart for each setting.baseis558b8262.Default sweep on
base, single Scope, 3 rounds × 20 (provider p50 0.24 s, p90 0.32 s, p99 1.25 s over 60 direct calls):readyThe degraded preparations take about twice the budget, because Atomic Memory and Topic Memory each wait a full budget. Beyond 0.5 s the extra budget only covers the provider's occasional 1.2 s tail, which a one-second Hook request cannot wait for anyway.
Fix vs base, all three settings interleaved per round:
base, 0.25 s per searchbase, 0.5 s per searchready/ slowestready/ slowestAll of C is
empty: with no embedding, the long query fails the lexical floor on every build. Scenario C measures how long the Host waits.The trade-off is visible in B.
baseat 0.5 s per search gets four morereadyresults, because each further Scope retries the embedding and a provider answering after 0.5 s still lands on a later search. That retry costs up to one budget per Scope: B's slowest preparation already takes 1.00 s, at the Kimi Code / ZCode / DSH deadline, and C takes 2.06 s. This PR keeps the bound instead; every miss in B ends at 0.54 s withreason: "inference_timeout".Not validated: a real Kimi Code, ZCode or DSH session against the slow provider. Their 1 s / 4 s budgets were read from the integration sources, not exercised.
Follow-ups, not in this PR:
AI usage statement
Code, tests, measurements and this description were produced with Claude Code (Claude Opus 5.5) and reviewed by me.