Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,9 @@ SAG_JEV_API_KEY=
SAG_JEV_RANKING_TIMEOUT_SECONDS=30
# error:失败即报错;fallback_to_score:允许粗排结果并显示降级提示。
SAG_JEV_FAILURE_POLICY=error
# 可选 SAG2 证据检索(zleap-sag2,SAG_SEARCH_STRATEGY=sag2):共用上面的 JEV 连接(必须有 Key),
# 只读直搜 SQLite / PostgreSQL 关系库,不依赖向量库;失败不回退,SAG_JEV_FAILURE_POLICY 对其无效。
# 调优参数见 apps/api/.env.example 的 SAG_SAG2_* 说明。

# 浏览器访问后端的地址(构建期内联;修改后需重建 web 镜像)。
NEXT_PUBLIC_API_BASE=http://localhost:8000
Expand Down
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@

## Unreleased

- 探索页新增「对比」:同一问题在 2–3 种检索模式下并排检索,显示耗时、召回数、SAG2 的 JEV 用量与共同命中的片段,可选 LLM 评审,并可一键按某一模式正式检索。
- SAG2 只读引擎改在专用事件循环线程运行:大范围检索构建内存语料时不再阻塞 API 事件循环;整次超时会同步取消引擎侧工作。
- 问答页可为当前轮次选择检索模式:输入框工具栏新增与探索页一致的模式控件,默认显示将实际使用的模式及其来源(人格默认 / 信源默认 / 全局),用户选择后对后续轮次生效;付费 JEV 模式缺少 API Key 时在开始生成前拒绝并给出设置入口。回答的活动时间线显示本轮实际模式与来源。
- 信源可设置「默认检索模式」:创建或编辑信源时选择快速、精确、JEV 精排或 SAG2 证据检索,并以通俗说明与付费提示帮助选择;JEV 模式缺少 API Key 时可在对话框内直接配置。搜索与问答在所选范围内信源一致时自动采用该默认(显式选择与 Agent 人格优先),结果 `stats.strategy_origin` 标明来源。不改变文档入库流程。
- 新增可选 SAG2 证据检索模式(`sag2`):接入 `zleap-sag2` 0.1.0,只读直搜 SQLite / PostgreSQL 关系库,BM25 召回后由 JEV 服务多轮评分,不依赖向量库,文档入库后即可检索;与 JEV 精排共用服务配置,失败不回退,首期不支持 MySQL / OceanBase。
- JEV 为付费服务:选择 JEV 精排或 SAG2 证据检索前必须填写 API Key;仅保存服务地址不再足以启用模式,连接测试也要求 Key。探索页灰置项直接给出原因与设置入口,SAG2 结果显示本次评分用量。
- 后端中文分词依赖由 `jieba-py` 改回 `jieba` 0.42.1,与 `zleap-sag2` 共用一份词典;分词结果不变。

## v1.9.2 · 2026-10-22

- 新增结构化输出模式设置和自动降级,兼容不支持严格结构化输出的模型;改善口语化提问的查询分析与语义结果保留(#248)。
Expand Down
13 changes: 9 additions & 4 deletions README-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ SAG 不是传统 RAG 与 GraphRAG 的融合,而是一套替代二者的原创
| 能力 | 解决的问题 |
| --- | --- |
| 知识导入 | 文件与网页信源、文档解析、分块、向量化、事件/实体抽取、后台处理 |
| 检索 | 跨信源或指定信源检索,支持快速(`vector` / `multi_es_fast`)、精确(`multi`)与可选 JEV 精排(`multi_jev`)模式 |
| 检索 | 跨信源或指定信源检索,支持快速(`vector` / `multi_es_fast`)、精确(`multi`)、可选 JEV 精排(`multi_jev`)与 SAG2 证据检索(`sag2`)模式 |
| 原文溯源 | 每条检索结果和引用都能打开对应的原文块 |
| 知识图谱 | 查看事件、实体及其可查询的关联关系 |
| Agent 对话 | 基于指定信源进行多轮问答,并提供可点击引用 |
Expand Down Expand Up @@ -566,12 +566,17 @@ config = EngineConfig.from_env()
| 快速(multi_es-fast) | `multi_es_fast` | BM25 实体召回与事件多跳检索,跳过 LLM 精排 |
| 精确 | `multi` | 结合实体关系与 LLM 精排,结果更完整 |
| JEV 精排 | `multi_jev` | 召回与图扩展后,通过独立 JEV 服务为候选证据评分 |
| SAG2 证据检索 | `sag2` | 由 [zleap-sag2](https://pypi.org/project/zleap-sag2/) 只读直搜关系库:BM25 召回后用 JEV 服务多轮评分,不依赖向量库 |

这里列出的是应用/API 策略名称;应用将 `multi` 映射到 Python 引擎的 `full_expand`,将 `multi_es_fast` 映射到 `pruned_expand_rff`。可用性取决于存储能力:`multi_es_fast` 需要词法检索支持。
这里列出的是应用/API 策略名称;应用将 `multi` 映射到 Python 引擎的 `full_expand`,将 `multi_es_fast` 映射到 `pruned_expand_rff`,`sag2` 由独立的 `zleap-sag2` 引擎执行。可用性取决于存储能力:`multi_es_fast` 与 `multi_jev` 需要词法检索支持,`sag2` 需要 SQLite 或 PostgreSQL 关系库。

在“设置 → 知识库 → 检索”配置 302.AI、TypeSafe 官方或自定义 JEV 服务(默认 `jev-latest`,独立 API Key),再选择 **JEV 精排**。仅保存服务不会启用;默认限时 30 秒、失败报错,可选择粗排降级。
每个信源可在创建或编辑时设置**默认检索模式**;未设置则跟随全局默认。探索页与问答页的输入框都提供模式控件,可为当前查询或当前轮次临时切换;探索页还可开启「对比」,用同一问题并排比较 2–3 种模式的耗时、召回与用量,并一键采用其中一种。搜索与问答按“本次显式选择 → Agent 人格 → 范围内信源的共同默认 → 全局默认”解析模式,范围内信源默认不一致时按全局默认处理;结果 `stats.strategy_origin` 标明来源。该设置只影响检索,不改变文档入库流程。

支持 LanceDB、Elasticsearch,暂不支持 pgvector;PostgreSQL 作为关系库不受影响。未启用时沿用原检索策略,已有文档无需重新入库。效果与成本见[更新日志](CHANGELOG.md#jev-reranking)。
在“设置 → 知识库 → 检索”配置 302.AI、TypeSafe 官方或自定义 JEV 服务(默认 `jev-latest`,独立 API Key),再选择 **JEV 精排** 或 **SAG2 证据检索**;创建信源时选择 JEV 模式也可在对话框内直接完成该配置。JEV 是付费服务:仅保存服务不会启用,选择任一 JEV 模式前必须填写 API Key。JEV 精排默认限时 30 秒、失败报错,可选择粗排降级。

**SAG2 证据检索**与 JEV 精排共用同一服务配置,但不经向量库:它直接只读 SAG 的 SQLite / PostgreSQL 关系库(首期不支持 MySQL / OceanBase),文档分块入库后即可检索,不要求事件抽取完成,pgvector 单库部署也可使用。每次检索会扫描所选信源的全部段落并多轮调用 JEV,用量高于 JEV 精排;失败直接报错,不支持粗排降级。候选数、超时、并发与范围上限等调优参数见 `apps/api/.env.example` 的 `SAG_SAG2_*`。使用 PostgreSQL 时,连接用户需能在目标 schema 创建并读写 `sag2_state`、`sag2_search_sessions` 两张辅助表,并对 `sag2_state` 执行 `LOCK TABLE`;SAG 的业务表保持只读。

JEV 精排支持 LanceDB、Elasticsearch,暂不支持 pgvector;PostgreSQL 作为关系库不受影响。未启用时沿用原检索策略,已有文档无需重新入库。效果与成本见[更新日志](CHANGELOG.md#jev-reranking)。

#### 存储后端

Expand Down
13 changes: 9 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ Upload a document once. SAG parses it, splits it into chunks, embeds it, extract
| Capability | What it gives you |
| --- | --- |
| Knowledge ingestion | File and web sources, document parsing, chunking, embedding, event/entity extraction, background processing |
| Search | Cross-source or source-scoped retrieval with Fast (`vector` / `multi_es_fast`), Precise (`multi`), and optional JEV reranking (`multi_jev`) modes |
| Search | Cross-source or source-scoped retrieval with Fast (`vector` / `multi_es_fast`), Precise (`multi`), optional JEV reranking (`multi_jev`), and SAG2 evidence retrieval (`sag2`) modes |
| Source tracing | Open any result or citation back to the exact original chunk |
| Knowledge graph | Inspect events, entities, and their queryable associations |
| Agent chat | Multi-turn answers grounded in selected sources, with clickable citations |
Expand Down Expand Up @@ -564,12 +564,17 @@ Pipeline request and result types are available from `zleap.sag.pipeline`: `Pars
| Fast (multi_es-fast) | `multi_es_fast` | BM25 entity recall and event multi-hop retrieval, skipping LLM reranking |
| Precise | `multi` | Combines entity relationships with LLM reranking for more complete results |
| JEV reranking | `multi_jev` | Scores recalled and graph-expanded evidence through an independent JEV service |
| SAG2 evidence retrieval | `sag2` | Read-only search by [zleap-sag2](https://pypi.org/project/zleap-sag2/) over the relational store: BM25 recall scored over several JEV rounds, no vector store involved |

These are the application/API strategy names; the application maps `multi` to the Python engine's `full_expand` and `multi_es_fast` to `pruned_expand_rff`. Availability depends on storage capabilities: `multi_es_fast` requires lexical search support.
These are the application/API strategy names; the application maps `multi` to the Python engine's `full_expand` and `multi_es_fast` to `pruned_expand_rff`, while `sag2` runs on the separate `zleap-sag2` engine. Availability depends on storage capabilities: `multi_es_fast` and `multi_jev` require lexical search support; `sag2` requires a SQLite or PostgreSQL relational store.

Configure 302.AI, TypeSafe, or a custom JEV service in **Settings → Knowledge → Retrieval** (default model: `jev-latest`, separate API key), then select **JEV reranking**. Saving the service alone does not enable it. The default deadline is 30 seconds; failures return an error unless coarse fallback is enabled.
Each source can carry a **default retrieval mode**, chosen when the source is created or edited; unset means follow the global default. Both the explore page and the chat composer offer a mode control for a one-off switch on the current query or turn; the explore page can also run a side-by-side comparison of two or three modes for the same question (latency, hits, usage) and adopt one with a click. Search and chat resolve the mode as explicit request → agent persona → the shared default of every source in scope → global default, falling back to the global default when sources in scope disagree; `stats.strategy_origin` reports which one applied. The setting only affects retrieval and never changes document ingestion.

Supported on LanceDB and Elasticsearch; pgvector is currently unsupported. PostgreSQL relational storage can use JEV with a supported vector backend. Existing retrieval settings remain until JEV is selected; documents do not need reindexing. See the [benchmark results and cost estimates](CHANGELOG.md#jev-reranking).
Configure 302.AI, TypeSafe, or a custom JEV service in **Settings → Knowledge → Retrieval** (default model: `jev-latest`, separate API key), then select **JEV reranking** or **SAG2 evidence retrieval**; choosing a JEV mode while creating a source offers the same configuration inline. JEV is a paid service: saving the service alone does not enable it, and an API key is required before either JEV mode can be selected. JEV reranking has a 30-second default deadline; failures return an error unless coarse fallback is enabled.

**SAG2 evidence retrieval** shares the JEV service configuration but bypasses the vector store: it reads SAG's SQLite or PostgreSQL relational store directly (MySQL and OceanBase are not supported yet), documents become searchable as soon as they are chunked and stored, event extraction is not required, and pgvector single-database deployments can use it. Each search scans every passage of the selected sources and calls JEV over several rounds, so it costs more than JEV reranking; failures are reported as errors, with no coarse fallback. Tuning knobs for candidates, timeouts, concurrency and scope limits are documented as `SAG_SAG2_*` in `apps/api/.env.example`. On PostgreSQL the connecting role must be able to create and use the two auxiliary tables `sag2_state` and `sag2_search_sessions` in the target schema and to `LOCK TABLE` `sag2_state`; SAG's own tables stay read-only.

JEV reranking is supported on LanceDB and Elasticsearch; pgvector is currently unsupported. PostgreSQL relational storage can use JEV with a supported vector backend. Existing retrieval settings remain until a JEV mode is selected; documents do not need reindexing. See the [benchmark results and cost estimates](CHANGELOG.md#jev-reranking).

#### Storage backends

Expand Down
14 changes: 14 additions & 0 deletions apps/api/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,20 @@ SAG_JEV_API_KEY=
SAG_JEV_RANKING_TIMEOUT_SECONDS=30
SAG_JEV_FAILURE_POLICY=error

# 可选 SAG2 证据检索(zleap-sag2);选择 SAG_SEARCH_STRATEGY=sag2,或在设置页选择模式。
# 共用上面的 JEV 服务地址、模型与 API Key(付费,选择任一 JEV 模式都必须填写 Key)。
# 只读直搜 SQLite / PostgreSQL 关系库,不依赖向量库也不要求抽取完成;失败不回退,
# SAG_JEV_FAILURE_POLICY 对其无效。以下调优参数首期只经环境变量设置,默认值已控制用量。
# SAG_SAG2_CANDIDATE_K=50 # BM25 候选数(zleap-sag2 默认 100)
# SAG_SAG2_SEED_K=10 # 每轮扩展种子数
# SAG_SAG2_MAX_HOPS=2 # 扩展轮数 0–2
# SAG_SAG2_EXPANSION_TERM_LIMIT=20 # 每轮扩展词数
# SAG_SAG2_SEARCH_TIMEOUT_SECONDS=90 # 整次检索上限(含并发排队、扫描与全部评分轮次;设置发布期间的等待另计)
# SAG_SAG2_DECISION_TIMEOUT_SECONDS=60 # 单次决策请求超时
# SAG_SAG2_DECISION_MAX_CONCURRENCY=20 # 决策请求并发
# SAG_SAG2_MAX_CONCURRENT_SEARCHES=2 # 服务端同时执行的 SAG2 检索数
# SAG_SAG2_MAX_SCOPE_CHUNKS=20000 # 所选信源段落行数上限,0 不限制

# ── 知识宇宙镜头阈值与浏览器预算 ──────────────────────
SAG_UNIVERSE_LOD_ORBIT_PX=72
SAG_UNIVERSE_LOD_NEAR_PX=180
Expand Down
5 changes: 4 additions & 1 deletion apps/api/packaging/sag-api.spec
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ for package in (
# OpenAI, Anthropic, and Gemini, so freezing every LiteLLM module would pull
# unrelated server/CLI dependencies into the desktop sidecar.
datas += collect_data_files("litellm")
# zleap-sag2 的 Unicode 排序在运行时读取 pyuca 的 allkeys-*.txt,冻结包需显式收集。
datas += collect_data_files("pyuca")
datas += collect_data_files("certifi")
for package in (
"litellm.litellm_core_utils",
Expand All @@ -72,9 +74,10 @@ for package in (
hiddenimports += collect_submodules(package)

for package in (
"jieba-py",
"jieba",
"sag-api",
"zleap-sag",
"zleap-sag2",
"lancedb",
"litellm",
"markitdown",
Expand Down
3 changes: 2 additions & 1 deletion apps/api/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ dependencies = [
"litellm==1.103.0",
"orjson>=3.11.6,<4",
"zleap-sag==0.15.0",
"zleap-sag2==0.1.0",
"alembic>=1.13",
"trafilatura>=1.8",
"markitdown[docx,pdf,pptx,xls,xlsx]>=0.1.6,<0.2",
Expand All @@ -31,7 +32,7 @@ dependencies = [
"mcp>=1.28,<2",
"octx[vectors]==0.1.6",
"tzdata>=2024.1",
"jieba-py>=0.46.12,<0.47",
"jieba==0.42.1",
]

[project.optional-dependencies]
Expand Down
5 changes: 5 additions & 0 deletions apps/api/sag_api/api/v1/agents.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
from sse_starlette.sse import EventSourceResponse

from sag_agent import AgentRuntime, EventType, RunHandle
from sag_api.core.config import settings
from sag_api.core.db import SessionLocal, get_session
from sag_api.core.deps import (
get_agent_runtime,
Expand Down Expand Up @@ -351,6 +352,9 @@ async def ask(
thread = await svc.get_thread(session, agent.id, thread_id)
if not llm.configured:
raise ConfigurationError("尚未配置 LLM,无法生成回答")
if body.strategy is not None:
# 本轮显式选择的模式必须可用(付费 JEV 模式需已配置 Key);在流开始前就拒绝。
EngineManager.require_strategy_available(settings, body.strategy)
plan = await svc.prepare_ask(
session,
agent=agent,
Expand All @@ -359,6 +363,7 @@ async def ask(
attachments=body.attachments,
source_ids=body.source_ids,
llm=llm,
strategy=body.strategy,
)
web_enabled = body.effective_web_enabled

Expand Down
5 changes: 3 additions & 2 deletions apps/api/sag_api/api/v1/dify.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
ServiceUnavailableError,
ValidationError,
)
from sag_api.enums import JEV_SCORED_STRATEGIES
from sag_api.sag import EngineManager
from sag_api.schemas.dify import (
DifyRetrievalRecord,
Expand Down Expand Up @@ -104,11 +105,11 @@ async def retrieval(
**({
"score_source": section.metadata.get("score_source", "unknown"),
"ranking_fallback_reason": (outcome.stats.get("ranking") or {}).get("fallback_reason"),
} if strategy == "multi_jev" else {}),
} if strategy in JEV_SCORED_STRATEGIES else {}),
},
)
for section in outcome.sections
if section.score >= body.retrieval_setting.score_threshold
and (strategy != "multi_jev" or fallback_used or section.metadata.get("score_source") == "jev")
and (strategy not in JEV_SCORED_STRATEGIES or fallback_used or section.metadata.get("score_source") == "jev")
]
return DifyRetrievalResponse(records=records)
10 changes: 7 additions & 3 deletions apps/api/sag_api/api/v1/search.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
from sag_api.core.errors import ApiError, ValidationError
from sag_api.core.logging import get_logger
from sag_api.db.models import Source, User
from sag_api.enums import JEV_SCORED_STRATEGIES
from sag_api.generation import LLMClient
from sag_api.sag import EngineManager, RetrievedSection, SearchOutcome
from sag_api.schemas.insight import EntityOut, GraphRelationOut
Expand All @@ -43,6 +44,7 @@
event_scores_from_evidence,
published_document_inputs,
recall_event_scores,
resolve_search_strategy,
retrieve_relevant_sections,
stream_synthesize_search_answer,
synthesize_search_answer,
Expand Down Expand Up @@ -169,16 +171,18 @@ async def _retrieve_search_evidence(
body: SearchRequest | GlobalSearchRequest,
) -> tuple[SearchOutcome, EventScoreMap, bool]:
configuration = getattr(engine_manager, "_settings", settings)
strategy = body.strategy or configuration.search_strategy
# Resolve once for the evidence-shape decision below; the retrieval service
# resolves the same inputs again and records where the mode came from.
strategy = resolve_search_strategy(requested=body.strategy, sources=sources, configuration=configuration).strategy
retrieval = retrieve_relevant_sections(
engine_manager,
sources,
body.query,
strategy=strategy,
strategy=body.strategy,
top_k=body.top_k,
include_ranked_candidates=body.include_ranked_candidates,
)
if strategy == "multi_jev":
if strategy in JEV_SCORED_STRATEGIES:
outcome = await retrieval
return outcome, event_scores_from_evidence(outcome.sections), True
outcome, scores = await asyncio.gather(
Expand Down
Loading
Loading