ROUTING.md

Intent Recognition & Tiered Router System (ROUTING.md)

Architecture Overview

The routing node (routerAgentNode) in the LangGraph architecture utilizes a four-tier cascade routing design (Tiered Routing Cascade) to minimize response latency and LLM Token expenditure while maintaining high classification precision.

                       +-------------------------------+
                       |      Incoming User Message    |
                       +---------------+---------------+
                                       |
                                       v
                       +-------------------------------+
                       | Tier 1: Rule Short-Circuit    |
                       | hasUnprocessedFile?           |
                       +---------------+---------------+
                                       |
                              +--------+--------+
                              |                 |
                           (Yes)               (No)
                              |                 |
                              v                 v
                   +-------------------+   +-------------------------------+
                   | Route: kbAgent    |   | Tier 2: Keyword Short-Circuit |
                   | source: "rule"    |   | matchKeywordRoute()           |
                   +-------------------+   +---------------+---------------+
                                                           |
                                                  +--------+--------+
                                                  |                 |
                                               (Match)           (Miss)
                                                  |                 |
                                                  v                 v
                                       +-------------------+   +-------------------------------+
                                       | Target Agent      |   | Tier 3: Clef-flash System-One |
                                       | source: "keyword" |   | Decision Model (Confidence>0.7|
                                       +-------------------+   +---------------+---------------+
                                                                               |
                                                                      +--------+--------+
                                                                      |                 |
                                                                   (Match)           (Miss)
                                                                      |                 |
                                                                      v                 v
                                                           +-------------------+   +-------------------------------+
                                                           | Target Agent      |   | Tier 4: LLM Fallback Classifier|
                                                           | source: "decision"|   | withStructuredOutput          |
                                                           +-------------------+   +---------------+---------------+
                                                                                                   |
                                                                                                   v
                                                                                       +-------------------+
                                                                                       | Target Agent      |
                                                                                       | source: "llm"     |
                                                                                       +-------------------+

---

Tiered Routing Layers

Tier 1: Rule Short-Circuit (source: "rule")

• Trigger: Evaluates if the current conversation history contains any unprocessed file/document upload (hasUnprocessedFile).
• Target Agent: kbAgent
• Latency: < 1ms (Zero I/O, Zero LLM overhead)

Tier 2: Priority Keyword Short-Circuit (source: "keyword")

• Trigger: Evaluates the trailing user message against priority-ordered keyword and regex rule groups via lib/router/keywords.ts.
• Priority Groups:
  1. codeAgent: Matches Markdown code blocks (```ts), code writing/refactoring phrases, runtime stack traces, and data format conversions (JSON -> CSV).
  2. cryptoAgent: Matches unambiguous tickers (BTC, ETH, USDT, SOL), cryptocurrency terms, and wallet/swap intent.
  3. weatherAgent: Matches meteorological terms (weather, forecast, celsius, air quality, AQI).
• Target Agents: codeAgent | cryptoAgent | weatherAgent
• Metadata: Includes source: "keyword" and the exact matched rule string (matchedKey).
• Latency: < 1ms

Tier 3: System-One Decision Model (source: "decision")

• Trigger: Triggered when Tier 1 (Rules) and Tier 2 (Keywords) miss, and decision model configuration is available.
• Provider Resolution Hierarchy:
  1. Route A (DB Provider Priority): Queries Postgres provider table for any enabled provider with a model flagged with kind: ["decision"]. Extracts baseUrl, decrypted Bearer key, and model name.
  2. Route B (Generic Env Fallback): If no DB decision model is configured, reads generic environment variables:
     • DECISION_MODEL_BASE_URL (Cloudflare Workers AI endpoint, AI Gateway URL, or private inference proxy)
     • DECISION_MODEL_API_KEY (Standard Bearer token)
     • DECISION_MODEL_NAME (Defaults to @cf/cloudflare/clef-flash)
  3. Route C (Chat Fallback): If neither Route A nor Route B exists, Tier 3 is skipped and execution proceeds directly to Tier 4 with source: "llm".
• Mechanism: Calls Cloudflare Workers AI @cf/cloudflare/clef-flash (or Jev-compatible System-One decision model) using typed question schemas (questions.route: { type: "choice", instructions, criteria }). Instead of autoregressive text generation, it evaluates probabilities across allowable options directly against the input state.
• Criteria Definition:
  • weatherAgent: Meteorological conditions, temperature, rain, forecast, air quality.
  • cryptoAgent: Cryptocurrency, tokens, market prices, BTC, ETH, swap, wallet.
  • codeAgent: Programming code, scripts, algorithms, data conversion, precise math formulas.
  • chatAgent: General conversation, translation, explanation, knowledge Q&A, writing, fallback.
• Confidence Gate: Requires calibrated confidence confidence >= 0.70. If confidence drops below 0.70 or the decision call encounters network errors, the router gracefully falls back to Tier 4 without interrupting the user.
• Latency & Resource Profile:
  • Latency: 150ms - 300ms
  • Output tokens: 0 (Zero token generation waste)
  • Input tokens: Typically ~230 tokens (only user message + question criteria)

Tier 4: LLM Structured Output Fallback (source: "llm")

• Trigger: Triggered when Tiers 1-3 pass without a conclusive match, or when the decision model is unconfigured / below confidence threshold.
• Mechanism: Invokes structured decision model with withStructuredOutput(InvokeRouteDecisionSchema) for deep semantic intent classification.
• Target Agents: weatherAgent | cryptoAgent | codeAgent | chatAgent (default fallback)
• Latency: 500ms - 2.5s

---

Observability & Credit Metering

Observability Tracing

Every invocation of the decision model is captured by CapturingHandler and surfaced in the Observability waterfall:

• ▼ System Message: Full classification instructions and the complete list of candidate sub-agents with their respective criteria.
• ▼ Human Message: The user's input prompt.
• ▼ Ai Message: Complete decision result containing choice, confidence, and the full probabilities breakdown.
• FIELDS Sidebar: Structured key-value display of choice, source: "decision", and confidence.
• Diagnostic Fallback Tracing: When a fallback occurs, the trace records status: "fallback_to_chat" along with the explicit reason (e.g. low_confidence (0.65 < 0.70) or clef_request_null_or_error).

Credit Accounting

When a user session is active (userId present):

• Clef decision model executions are recorded in credit_usage_log with agentName: "routerAgent" and modelName: "@cf/cloudflare/clef-flash".
• Credits are deducted using the provider's configured rate (computeCredits(usage, rate)). If the provider rates are set to 0, the usage is logged for audit without billing the user.

---

Routing Decision Schema (RouterDecision)

export const RouteDecisionSchema = z.object({
  next: z.enum(["weatherAgent", "chatAgent", "cryptoAgent", "codeAgent", "kbAgent"]),
  source: z.enum(["rule", "keyword", "decision", "llm"]).optional(),
  matchedKey: z.string().optional(),
  confidence: z.number().optional(),
  probabilities: z.record(z.string(), z.number()).optional(),
});

---

Source Files & File Tree

langgraph-app/
├── backend/
│   └── node/
│       └── router-agent-node.ts      # 4-tier intent classification node
├── lib/
│   ├── provider/
│   │   ├── decision.ts               # Generic System-One / Jev / Clef decision client
│   │   ├── model-registry.ts         # Multi-pool model registry (kind: ["decision"])
│   │   └── schema.ts                 # Provider & ModelKind type definitions
│   └── router/
│       └── keywords.ts               # Tier 2 priority keyword rules & match engine
├── tests/
│   ├── backend/
│   │   └── node/
│   │       └── router-agent-node.test.ts  # Node cascading logic & observability tests
│   └── lib/
│       ├── provider/
│       │   └── decision.test.ts      # Decision model client unit tests
│       └── router/
│           └── keywords.test.ts      # Regex & keyword precedence tests
└── docs/
    └── ROUTING.md                    # Architecture & operational specification

• Routing Node: backend/node/router-agent-node.ts
• Decision Client (System-One / Jev / Clef): lib/provider/decision.ts
• Keyword Matching Engine: lib/router/keywords.ts
• Documentation: docs/ROUTING.md
• Unit Tests:
  • tests/lib/provider/decision.test.ts
  • tests/lib/router/keywords.test.ts
  • tests/backend/node/router-agent-node.test.ts