bug(mcp): "current" and "supported" send how-to questions to release-notes-only search #289

Area

mcp

What happened

pick_search_plan() in intent_router.py sends any query containing "current" or "supported" down the temporal path, even when the question has nothing to do with releases (L21-L24, L112-L121).

The temporal plan:

1. searches release notes only (doc_type == "release")
2. sorts the hits by release date instead of relevance
3. skips the cross-encoder reranker (milvus_search.py L243-L245)

So "How do I get the current run ID inside a KFP component?" is answered from the newest release notes that share a keyword, not from the docs page that explains it. The unfiltered fallback only runs when the filtered search returns nothing.

the router sees the LLM's keyword query, not the user's sentence, and the docs system prompt tells the model to keep "current" in that query (docs-system-message.txt L9).

Two eval cases contain these words: fid-parallelfor-v2 and dbg-external-mysql. The staging eval can't flag this: its gates check tool routing and groundedness, and the judge counts "not found" as grounded (staging_eval.py L199). Could a maintainer check the Langfuse traces for those two cases to confirm which plan they got?

Expected behavior

"current" and "supported" should trigger the temporal plan only when they are about a version or release, e.g. "What is the current version of Kubeflow Pipelines?" (fid-current-kfp) or "supported Kubernetes versions". Other queries should fall through to the normal rules (exact / conceptual / general).

Proposed fix (open to another approach, since the router is maintainer-owned):

• keep "latest", "newest", "most recent" unchanged
• treat "current" / "supported" as temporal only within a few words of "version(s)" / "release(s)"
• add maybe regression test for both

I have this written and tested locally: 6 new tests fail on main and pass with the change, the full suite passes (172 passed, 69 skipped), and ruff is clean. Across the 66 questions in flo_eval_dataset.json, only the two cases above change route. Happy to open a PR if this direction looks right.

Related: #272 (router tests rewrite), #280 (also changes intent_router.py; the two changes apply together cleanly).

Reproduction

No cluster needed. Local, Windows, Python 3.11.9 (same as CI), pip install -r requirements-test.txt, main @ 145409a. From docs-agent-mcp/mcp-server:

from intent_router import pick_search_plan

questions = [
    "Is dsl.ParallelFor supported in KFP v2?",
    "Is running Kubeflow Pipelines with an external MySQL supported, and how is it configured?",
    "How do I get the current run ID inside a KFP component?",
    "Which hyperparameter search algorithms are supported by Katib?",
    "What is the latest Kubeflow release?",
    "What is the current version of Kubeflow Pipelines?",
]
for q in questions:
    p = pick_search_plan(q)
    print(f"{p.intent:<9} {p.filter_expr or '-':<22} {q}")

Output:

temporal  doc_type == "release"  Is dsl.ParallelFor supported in KFP v2?
temporal  doc_type == "release"  Is running Kubeflow Pipelines with an external MySQL supported, and how is it configured?
temporal  doc_type == "release"  How do I get the current run ID inside a KFP component?
temporal  doc_type == "release"  Which hyperparameter search algorithms are supported by Katib?
temporal  doc_type == "release"  What is the latest Kubeflow release?
temporal  doc_type == "release"  What is the current version of Kubeflow Pipelines?

Only the last two are release questions; all six get the release-notes-only plan.

Regression tests from my branch, run against unchanged main (6 fail as expected):

Suspected files

docs-agent-mcp/mcp-server/intent_router.py