Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
b72d51f
✨ feat: Add the Classification Port
danny-avila Sep 24, 2026
e0f654e
feat: credential functions and a ClickHouse gateway preset for the cl…
danny-avila Sep 26, 2026
e695493
feat: support Laya and strict chat classification
lia-librechat Sep 27, 2026
d46ae34
fix: address classifier review findings
lia-librechat Sep 27, 2026
f886154
fix: fail closed on classifier traces and deadline expiry
lia-librechat Sep 27, 2026
1c99d4d
Merge remote-tracking branch 'origin/main' into lia/classification-mo…
lia-librechat Sep 28, 2026
0ac0cf9
style: Format Classification HTTP Adapter
lia-librechat Sep 28, 2026
953216c
fix: Validate Classifier Answers and Provider Metadata
lia-librechat Sep 29, 2026
5fccb73
fix: Guard Boolean Criteria Snapshots
lia-librechat Sep 29, 2026
0a20d22
Merge remote-tracking branch 'origin/main' into lia/finish-classifica…
lia-librechat Sep 30, 2026
7411317
🛂 fix: Validate Classification Criteria and Measured Choices
lia-librechat Sep 30, 2026
f38061e
🛡️ fix: Fail Closed on Unverified Classification Adapters
lia-librechat Sep 30, 2026
7659e4a
🧭 refactor: Name the Shared Port Typed Decisions
lia-librechat Oct 1, 2026
acfeeb4
Merge remote-tracking branch 'origin/main' into lia/typed-decisions
lia-librechat Oct 1, 2026
11502ed
🎨 style: Format Typed Decision Exports
lia-librechat Oct 1, 2026
47d67a3
🛡️ fix: Sanitize Decision Question Preparation Failures
lia-librechat Oct 1, 2026
23cf6d7
🎨 style: Flatten Dynamic Decision Test Fixtures
lia-librechat Oct 1, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions src/decisions/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Typed Decisions

A `DecisionModel` asks typed boolean, choice, or score questions about one state. It is not a chat-model subclass. Construct one decision model per provider and tenant and pass signals, labels, state, and questions per call. This SDK does not cache credentials or host inference weights.

## Backends

- **Jev and Laya:** `createDecisionModel(settings, credential?, options?)` uses the same System One HTTP adapter. `decisionPreset('laya')` supplies only the dialect and optional-auth setting. Supply the full server `/v1/systemone` endpoint in `settings.baseURL`; omitting `settings.model` lets Laya route to a checkpoint. A supplied key is still sent as a bearer token. Jev, gateway, OpenRouter, Cloudflare, and generic HTTP settings require a credential unless `requiresAuth: false` is explicitly set.
- **Strict structured chat:** `createStructuredChatDecisionModel({ model, modelId, method: 'jsonSchema' | 'functionCalling' })` injects an already configured LangChain chat model. The verified adapters are OpenAI (including Azure through the same implementation) in either strict mode and Anthropic in strict `functionCalling` mode. Bedrock and other unverified adapters fail with `unsupported_mode` before invocation, even if `withStructuredOutput` silently accepts strict options. Unsupported modes and score questions fail rather than falling back to JSON-mode prompting or asking a model for expected-value scores. Up to 32 compatible questions (128 choice options total) share one model invocation; larger batches fail explicitly.

```ts
const laya = createDecisionModel(
{
...decisionPreset('laya'),
baseURL: 'http://localhost:8000/v1/systemone',
},
undefined,
{ providerId: 'laya' }
);

const answer = (
await laya.decide({
state: 'Please refund the duplicate charge.',
questions: { refund: booleanQuestion('Did the user request a refund?') },
})
).answers.refund;

if (answer?.type === 'boolean' && answer.probability !== null) {
// Apply only a threshold evaluated for this backend, checkpoint, and use case.
}
```

A System One boolean answer has a measured `probability: number`. A structured-chat boolean is an **unmeasured** `{ decision: boolean, probability: null }`. Choice distributions are `null` if unmeasured; present distributions must cover the rubric and sum to one within rounding tolerance. A measured choice must select an option with maximal probability; ties are valid. Boolean criteria may be omitted, a string, or an object containing only optional `true` and `false` text descriptions. Invalid criteria fail locally before credential minting or provider invocation. A missing HTTP answer is `undefined`; a missing strict-schema answer or any malformed answer throws `DecisionError`. Usage is `null` when unknown, not zero. The `confidence` field on Jev and Laya choice/score answers has different mathematical meanings and is **not** a portable threshold. Score is a System One expected rubric level, not an LLM-selected ordinal level. Measured score maps use consecutive numeric levels starting at zero, including when the exported parser is called without an expected question.

Use the request's `signal` and `timeoutMs` to bound a call. HTTP timeouts cover credential minting, retries, response reading, and backoff. Credential minters run once per call, plus one refresh after a 401; the refreshed key persists across retries. HTTP redirects fail rather than forwarding bearer keys. SDK errors and the `onAnswered(label, elapsedMs)` hook contain no response content or credentials; the caller supplies the label and must not put secrets in it.

The structured-chat adapter sends the original state to the provider. Its prompt carries a marker so Langfuse can drop the **entire state and question text** from generation inputs when any tool-output redaction policy is active. Selectively removing identifiable tool fields would still leak a private result quoted in free-form state or instructions. With no active redaction policy, prompts remain visible. Configure provider-side logging separately. No defaults should change until quality, calibration, latency, and cost have been evaluated for each backend and use case.

## API Naming

`DecisionModel.decide()` is the provider-neutral contract. System One names the Jev/Laya HTTP dialect, not the strict-chat adapter. This API is introduced in this unreleased PR, so there are no classification compatibility aliases. LibreChat configuration keys and downstream migrations are separate changes.
92 changes: 92 additions & 0 deletions src/decisions/deadline.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
import { DecisionError } from './types';

export type AwaitWithinDeadline = <T>(task: Promise<T>) => Promise<T>;

/** Bounds even non-cooperative credential minters, fetches, body reads, and model calls. */
export async function withDecisionDeadline<T>(
provider: string,
timeoutMs: number,
callerSignal: AbortSignal | undefined,
operation: (signal: AbortSignal, waitFor: AwaitWithinDeadline) => Promise<T>
): Promise<T> {
if (
!Number.isSafeInteger(timeoutMs) ||
timeoutMs <= 0 ||
timeoutMs > 3_600_000
) {
throw new DecisionError('bad_request', 'invalid decision model timeout', {
provider,
});
}
if (callerSignal?.aborted === true) {
throw new DecisionError('aborted', 'caller aborted the request', {
provider,
});
}

const expiresAt = performance.now() + timeoutMs;
const deadline = new AbortController();
const signal = callerSignal
? AbortSignal.any([callerSignal, deadline.signal])
: deadline.signal;
const timer = setTimeout(() => deadline.abort(), timeoutMs);
const expired = (): boolean => {
if (performance.now() >= expiresAt) {
deadline.abort();
}
return signal.aborted;
};
const abortError = (): DecisionError => {
if (callerSignal?.aborted === true) {
return new DecisionError('aborted', 'caller aborted the request', {
provider,
});
}
return new DecisionError('timeout', 'decision model deadline exceeded', {
provider,
});
};

const waitFor: AwaitWithinDeadline = <U>(task: Promise<U>): Promise<U> =>
new Promise<U>((resolve, reject) => {
if (expired()) {
void task.catch(() => {});
reject(abortError());
return;
}
const onAbort = (): void => {
signal.removeEventListener('abort', onAbort);
reject(abortError());
};
signal.addEventListener('abort', onAbort, { once: true });
if (expired()) {
onAbort();
}
task.then(
(value) => {
signal.removeEventListener('abort', onAbort);
if (expired()) {
reject(abortError());
} else {
resolve(value);
}
},
(error: unknown) => {
signal.removeEventListener('abort', onAbort);
reject(expired() ? abortError() : error);
}
);
});

try {
return await waitFor(operation(signal, waitFor));
} catch (error) {
if (expired()) {
throw abortError();
}
throw error;
} finally {
expired();
clearTimeout(timer);
}
}
Loading
Loading