Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/en/docs/operate/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ still occupies an HTTP slot until it finishes; reserve other slots for concurren
| `POWERCONTEXT_SERVER_RUNTIME_RECALL_GATE_ALLOW_WITH_RERANK` | `false` | Whether expansion may still run when `MEMORY_RERANK_ENABLED` is set; by default reranking ends the expansion |
| `POWERCONTEXT_SERVER_RUNTIME_RECALL_FTS_WEIGHT` | `1.0` | Relative FTS contribution in hybrid Atomic Memory and Topic Memory RRF; `0` excludes FTS-only matches from hybrid results |
| `POWERCONTEXT_SERVER_RUNTIME_RECALL_VECTOR_WEIGHT` | `1.0` | Relative vector contribution in hybrid Atomic Memory and Topic Memory RRF; `0` excludes vector-only matches from hybrid results |
| `POWERCONTEXT_SERVER_RUNTIME_RECALL_EMBEDDING_TIMEOUT_SECONDS` | `0.25` | Seconds each Atomic Memory or Topic Memory search inside a context preparation waits for the query embedding, at most `60`. A preparation searches the Scope and each of its context references in turn, so the waits add up. When a budget expires, that search continues with lexical matching only and logs `memory.search.embedding_fallback` or `topic_memory.search.embedding_fallback`. A remote embedding provider needs more than the default; keep the summed waits inside the host's Hook request timeout, 3 seconds on most hosts |
| `POWERCONTEXT_SERVER_RUNTIME_RECALL_EMBEDDING_TIMEOUT_SECONDS` | `0.5` | Seconds one context preparation waits for query embeddings in total, at most `60`. The Atomic Memory searches of the Scope and each of its context references and the Topic Memory search share this budget, and only the time spent waiting for an embedding is charged to it: each search waits only for what the earlier ones left, so the preparation's wait stays within it however many Scopes are searched, while database reads and reranking between the searches do not use it up. A search that finds the budget spent continues with lexical matching only and logs `memory.search.embedding_fallback` or `topic_memory.search.embedding_fallback`. The default covers a typical remote embedding round trip; keep it below the prepare request timeout of the hosts in use, the shortest default being 1 second on Kimi Code, ZCode, and DSH |
| `POWERCONTEXT_SERVER_RUNTIME_MEMORY_SCHEDULE_SECONDS` | unset | Memory automatic admission interval; `SCHEDULE_SECONDS` remains a compatibility alias |
| `POWERCONTEXT_SERVER_RUNTIME_TOPIC_MEMORY_SCHEDULE_SECONDS` | unset | Topic Memory automatic admission interval; unset disables new automatic admission |
| `POWERCONTEXT_SERVER_RUNTIME_TOPIC_MEMORY_SOURCE_WINDOW_LIMIT` | `10` | Maximum Sources per Topic Memory Window, capped at 100; one Scope invocation can finish several Windows |
Expand Down
9 changes: 6 additions & 3 deletions docs/en/docs/workflows/prepare-context-text.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,9 +102,12 @@ asyncio.run(export_context())
Default prepare requests without `assembly` also recall Topic Memory from the current Scope when available.
With explicit `assembly`, select `topic-memory` to include it. `assembly: {}` still selects only Memory and Experience.

Topic recall skips query Embedding when the Scope has no current Topics. During context preparation, each Topic
query Embedding attempt has a 250 ms budget and falls back to FTS on timeout. The same prepare request reuses that FTS
choice in recall-gate expansion rounds; a subsequent request can try Embedding again. A slow optional provider does not
Topic recall skips query Embedding when the Scope has no current Topics. During context preparation, Topic recall
reuses the Memory query vector when both use the same embedding model and the query fits Topic search (64 distinct
terms and 8,192 characters); otherwise it embeds its own query. That attempt waits only for what Memory recall left of
the preparation's `recall_embedding_timeout_seconds` budget (500 ms by default), so a second embedding round trip has
to fit in the same budget, and falls back to FTS when the budget runs out. The same prepare request reuses that
FTS choice in recall-gate expansion rounds; a subsequent request can try Embedding again. A slow optional provider does not
hold up already retrieved Memory. Direct Topic search keeps the configured provider timeout.

Entries retain retrieval order within a family. Existing Memory reranking remains authoritative. Rank can have gaps
Expand Down
2 changes: 1 addition & 1 deletion docs/zh/docs/operate/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ MCP 使用 stateless Streamable HTTP,每次工具执行都采用当前请求
| `POWERCONTEXT_SERVER_RUNTIME_RECALL_GATE_ALLOW_WITH_RERANK` | `false` | 已启用 `MEMORY_RERANK_ENABLED` 时是否仍允许追加搜索;默认在 rerank 之后不再追加 |
| `POWERCONTEXT_SERVER_RUNTIME_RECALL_FTS_WEIGHT` | `1.0` | hybrid Atomic Memory 和 Topic Memory RRF 中 FTS 的相对贡献;`0` 会从 hybrid 结果中排除仅命中 FTS 的候选 |
| `POWERCONTEXT_SERVER_RUNTIME_RECALL_VECTOR_WEIGHT` | `1.0` | hybrid Atomic Memory 和 Topic Memory RRF 中 vector 的相对贡献;`0` 会从 hybrid 结果中排除仅命中 vector 的候选 |
| `POWERCONTEXT_SERVER_RUNTIME_RECALL_EMBEDDING_TIMEOUT_SECONDS` | `0.25` | 一次上下文准备里每次 Atomic Memory 或 Topic Memory 检索等待查询 embedding 的秒数,上限 `60`。准备会依次检索该 Scope 及其每个 context reference,等待时间会累加。某次等待超时后,该次检索只用词法匹配继续,并记录 `memory.search.embedding_fallback` 或 `topic_memory.search.embedding_fallback`。远程 embedding 提供商需要比默认值更长的时间;累计等待应控制在宿主 Hook 请求超时之内,多数宿主为 3 秒 |
| `POWERCONTEXT_SERVER_RUNTIME_RECALL_EMBEDDING_TIMEOUT_SECONDS` | `0.5` | 一次上下文准备等待查询 embedding 的总秒数,上限 `60`。该 Scope 及其每个 context reference 的 Atomic Memory 检索与 Topic Memory 检索共用这一预算,且只有等待 embedding 的时间计入:每次检索只等待前面检索剩下的时间,所以无论检索多少个 Scope,准备的等待都不超过它,而检索之间的数据库读取和重排不会占用它。发现预算已用完的检索只用词法匹配继续,并记录 `memory.search.embedding_fallback` 或 `topic_memory.search.embedding_fallback`。默认值覆盖一次典型的远程 embedding 往返;应小于所用宿主的 prepare 请求超时,其中默认值最短的是 Kimi Code、ZCode 和 DSH 的 1 秒 |
| `POWERCONTEXT_SERVER_RUNTIME_MEMORY_SCHEDULE_SECONDS` | 未设置 | Memory 自动准入间隔;`SCHEDULE_SECONDS` 保留为兼容别名 |
| `POWERCONTEXT_SERVER_RUNTIME_TOPIC_MEMORY_SCHEDULE_SECONDS` | 未设置 | Topic Memory 自动准入间隔;未设置时不接纳新的自动调用 |
| `POWERCONTEXT_SERVER_RUNTIME_TOPIC_MEMORY_SOURCE_WINDOW_LIMIT` | `10` | 每个 Topic Memory Window 的 Source 数量上限,硬上限为 100;一次 Scope 调用可完成多个 Window |
Expand Down
6 changes: 4 additions & 2 deletions docs/zh/docs/workflows/prepare-context-text.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,8 +99,10 @@ asyncio.run(export_context())
省略 `assembly` 的默认 prepare 请求还会召回当前 Scope 中可用的 Topic Memory。
显式 `assembly` 通过选择 `topic-memory` 包含主题记忆;`assembly: {}` 仍只选择 Memory 和 Experience。

当前 Scope 没有 Topic 时,主题召回会跳过查询 Embedding。组装上下文时,每次 Topic 查询 Embedding 最多等待
250 毫秒,超时后回退到 FTS。同一次 prepare 的 recall gate 扩展轮次复用这个选择,后续请求可以重新尝试 Embedding,
当前 Scope 没有 Topic 时,主题召回会跳过查询 Embedding。组装上下文时,若 Topic 与 Memory 使用同一个 Embedding 模型,
且查询在 Topic 检索的限制之内(64 个不同词项、8,192 个字符),主题召回复用 Memory 的查询向量;否则单独嵌入查询。
这次嵌入只等待 Memory 召回用剩的本次准备 `recall_embedding_timeout_seconds` 预算(默认 500 毫秒),所以第二次
Embedding 往返也要落在同一预算内,预算用完后回退到 FTS。同一次 prepare 的 recall gate 扩展轮次复用这个选择,后续请求可以重新尝试 Embedding,
避免缓慢的可选模型重复阻塞已召回的 Memory。直接检索 Topic 仍使用配置的模型超时。

同一类别内保留召回顺序,包括已有 Memory reranker 的排序。前面的候选因预算无法装入时,rank 可能不连续。
Expand Down
2 changes: 2 additions & 0 deletions src/powercontext/builtin/artifacts/topic_memory/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,8 @@ class TopicMemorySearchResult(BaseModel):
artifacts: tuple[TopicMemory, ...] = Field(default=(), exclude=True)
#: Set only when a wanted query embedding was abandoned, never when FTS was the plan.
embedding_fallback: str | None = Field(default=None, exclude=True)
#: Seconds spent waiting for the query embedding, so a context preparation can share one budget across searches.
embedding_wait_seconds: float = Field(default=0.0, exclude=True)


class PublishedTopicMemory(BaseModel):
Expand Down
39 changes: 34 additions & 5 deletions src/powercontext/builtin/runtime/application.py
Original file line number Diff line number Diff line change
Expand Up @@ -469,6 +469,21 @@ def _degradation(current: str | None, observed: str | None) -> str | None:
return _TIMEOUT_REASON if observed == _TIMEOUT_REASON else (current or observed)


@dataclass
class _EmbeddingBudget:
"""What is left of one context preparation's budget for waiting on query embeddings.

Only the waits are charged, so database reads, Experience recall and reranking between two searches do not take a
later search's share. A spent budget is reached only through an embedding that timed out, which already recorded
the preparation's degradation cause, so a later search can skip its embedding without losing that cause.
"""

remaining: float

def spend(self, seconds: float) -> None:
self.remaining = max(self.remaining - seconds, 0.0)


@dataclass(frozen=True)
class _RecallRoundOutcome:
"""All observable results produced by one bounded recall pass."""
Expand Down Expand Up @@ -1140,6 +1155,9 @@ async def _prepare_build(
request.bootstrap_receipt_id,
)

# One budget covers every query embedding this preparation waits for. Each search gets what the earlier ones
# left, so the waits cannot add up past it however many Scopes and families are searched.
embedding_budget = _EmbeddingBudget(self._runtime.recall_embedding_timeout_seconds)
round_zero = await self._recall_round(
request,
scope_ids,
Expand All @@ -1148,6 +1166,7 @@ async def _prepare_build(
admission=None,
reuse=reuse,
topic_reuse=topic_reuse,
embedding_budget=embedding_budget,
excluded_memory=injected_memory_keys,
)
memory_candidates = list(round_zero.memory)
Expand Down Expand Up @@ -1182,6 +1201,7 @@ async def _prepare_build(
profile_candidates=profile_candidates,
reuse=reuse,
topic_reuse=topic_reuse,
embedding_budget=embedding_budget,
round_zero=round_zero,
excluded_memory=injected_memory_keys,
)
Expand Down Expand Up @@ -1310,6 +1330,7 @@ async def _gated_recall_effort( # noqa: C901 - the bounded expansion loop is in
profile_candidates: Sequence[PreparedProfileCandidate],
reuse: dict[str, MemoryQueryEmbedding | None],
topic_reuse: dict[str, MemoryQueryEmbedding | None],
embedding_budget: _EmbeddingBudget,
round_zero: _RecallRoundOutcome,
excluded_memory: frozenset[tuple[object, ...]],
) -> tuple[
Expand Down Expand Up @@ -1397,6 +1418,7 @@ async def _gated_recall_effort( # noqa: C901 - the bounded expansion loop is in
admission=plan.admission,
reuse=reuse,
topic_reuse=topic_reuse,
embedding_budget=embedding_budget,
excluded_memory=excluded_memory,
)
memory_ranks: dict[Any, int] = {}
Expand Down Expand Up @@ -1543,6 +1565,7 @@ async def _recall_round(
admission: AdmissionFloor | None,
reuse: dict[str, MemoryQueryEmbedding | None],
topic_reuse: dict[str, MemoryQueryEmbedding | None],
embedding_budget: _EmbeddingBudget,
excluded_memory: frozenset[tuple[object, ...]],
) -> _RecallRoundOutcome:
memory_candidates: list[PreparedMemoryCandidates] = []
Expand All @@ -1567,6 +1590,7 @@ async def _recall_round(
admission=admission,
reuse=reuse.get(scope_id, shared_embedding),
allow_embedding=scope_id not in reuse or reuse[scope_id] is not None,
embedding_budget=embedding_budget,
)
memory_recoverable |= outcome.memory_recoverable
degradation = _degradation(degradation, outcome.memory_embedding_fallback)
Expand Down Expand Up @@ -1612,6 +1636,7 @@ async def _recall_round(
admission=admission,
reuse=topic_embedding,
allow_embedding=self.scope_id not in topic_reuse or topic_reuse[self.scope_id] is not None,
embedding_budget=embedding_budget,
)
if TOPIC_MEMORY_FAMILY in families
else TopicMemoryRecallOutcome()
Expand Down Expand Up @@ -1639,6 +1664,7 @@ async def _recall_scope(
admission: AdmissionFloor | None,
reuse: MemoryQueryEmbedding | None,
allow_embedding: bool = True,
embedding_budget: _EmbeddingBudget,
) -> _ScopeRecallOutcome:
async with self._runtime._scoped_operation(scope_id, embedding_purpose=ModelUsagePurpose.MEMORY_RECALL):
with self._runtime._stage(
Expand All @@ -1661,12 +1687,13 @@ async def _recall_scope(
mode="auto",
admission=admission,
query_embedding=reuse,
embedding_timeout_seconds=self._runtime.recall_embedding_timeout_seconds,
allow_embedding=allow_embedding,
embedding_timeout_seconds=embedding_budget.remaining,
allow_embedding=allow_embedding and embedding_budget.remaining > 0,
recovery_admission=None if policy is None else policy.round2_admission,
context=_PREPARE_ATOMIC_CONTEXT.get(),
_trace=False,
)
embedding_budget.spend(result.embedding_wait_seconds)
memory_hits = result.hits
memory_admission = result.admission
memory_recoverable = result.recoverable
Expand Down Expand Up @@ -1734,6 +1761,7 @@ async def _topic_memory_hits(
admission: AdmissionFloor | None,
reuse: MemoryQueryEmbedding | None,
allow_embedding: bool,
embedding_budget: _EmbeddingBudget,
) -> TopicMemoryRecallOutcome:
configured = self._runtime._topic_memory_search is not None
bounded_query = bounded_topic_memory_query(query)
Expand All @@ -1752,11 +1780,12 @@ async def _topic_memory_hits(
SearchTopicMemoryRequest(query=bounded_query, limit=limit),
admission=admission,
query_embedding=reuse,
embedding_timeout_seconds=self._runtime.recall_embedding_timeout_seconds,
allow_embedding=allow_embedding,
embedding_timeout_seconds=embedding_budget.remaining,
allow_embedding=allow_embedding and embedding_budget.remaining > 0,
)
)
)
embedding_budget.spend(result.embedding_wait_seconds)
if span is not None:
span.set_attributes({"powercontext.topic_memory.search.result_count": len(result.hits)})
return TopicMemoryRecallOutcome(
Expand Down Expand Up @@ -3830,7 +3859,7 @@ def __init__(
source_window_limit: int = 100,
context_assembly_max_entries: int = 8,
recall_sufficiency_policy: RecallSufficiencyPolicy | None = None,
recall_embedding_timeout_seconds: float = 0.25,
recall_embedding_timeout_seconds: float = 0.5,
scope_cache_size: int = DEFAULT_SCOPE_CACHE_SIZE,
scope_evictor: ScopeEvictor | None = None,
scope_cache_observer: ScopeCacheObserver | None = None,
Expand Down
8 changes: 6 additions & 2 deletions src/powercontext/builtin/runtime/atomic_memory.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
import logging
from collections.abc import Sequence
from contextlib import asynccontextmanager, nullcontext
from dataclasses import dataclass, replace
from dataclasses import dataclass, field, replace
from hashlib import sha256
from time import perf_counter
from typing import cast
Expand Down Expand Up @@ -237,6 +237,8 @@ class AtomicMemorySearchPage:
#: Set only when the query embedding was abandoned for lexical retrieval, so a
#: caller can tell "no vector channel was ever wanted" apart from "it was lost".
embedding_fallback: str | None = None
#: Seconds spent waiting for the query embedding, so a context preparation can share one budget across searches.
embedding_wait_seconds: float = field(default=0.0, compare=False)


class AtomicMemoryApplication:
Expand Down Expand Up @@ -541,6 +543,7 @@ async def search( # noqa: C901
raise InvalidBaseAccessRequestError("mode", "must be auto, text, vector, or hybrid")
if kind is not None:
AtomicMemoryContent(kind=kind, text="validation")
started = asyncio.get_running_loop().time()
mode, vector, query_embedding, embedding_calls, embedding_fallback = await self._resolve_query_embedding(
query,
mode,
Expand All @@ -549,6 +552,7 @@ async def search( # noqa: C901
allow_embedding,
public_search=artifact_request is not None,
)
embedding_wait_seconds = asyncio.get_running_loop().time() - started
channel_weights = application.recall_channel_weights if mode == "hybrid" else None
profile = application.index.capabilities.embedding_profile if vector is not None else None
# Inference finishes before the business snapshot opens. Builtin policy
Expand Down Expand Up @@ -641,7 +645,7 @@ async def search( # noqa: C901
)
)
page = replace(page, matches=tuple(matches))
return replace(page, recoverable=recoverable)
return replace(page, recoverable=recoverable, embedding_wait_seconds=embedding_wait_seconds)

async def _validate_search_hits(self, connection, hits, *, verify_vectors) -> tuple[AtomicMemory, ...]:
"""Verify only returned candidates against one bounded authority snapshot.
Expand Down
11 changes: 6 additions & 5 deletions src/powercontext/builtin/runtime/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -180,11 +180,12 @@ def validate_memory_capacity_order(self) -> RuntimeConfig:
recall_gate_allow_with_rerank: bool = False
recall_fts_weight: float = Field(default=1.0, ge=0.0, allow_inf_nan=False)
recall_vector_weight: float = Field(default=1.0, ge=0.0, allow_inf_nan=False)
# Budget for embedding the query in each Atomic Memory or Topic Memory search of a context preparation. A
# preparation searches the Scope and each context reference in turn, so the waits add up inside the Hook
# request; when a budget expires, that search continues with lexical matching only. A remote embedding
# provider needs more than the default.
recall_embedding_timeout_seconds: float = Field(default=0.25, gt=0.0, le=60.0, allow_inf_nan=False)
# Budget for waiting on query embeddings in one context preparation. The Atomic Memory searches of the Scope and
# each context reference and the Topic Memory search share it, and only their waits are charged, so database reads
# and reranking in between do not take a later search's share; a search that finds it spent continues with lexical
# matching only. The default covers a typical remote embedding round trip and leaves the rest of a one-second Hook
# request to the other work.
recall_embedding_timeout_seconds: float = Field(default=0.5, gt=0.0, le=60.0, allow_inf_nan=False)
profile_schedule_enabled: bool = False
profile_cron: str = "0 2 * * *"
profile_timezone: str = "Asia/Shanghai"
Expand Down
Loading
Loading