Skip to content

System One: serve Tinfoil's decision models - #1259

Merged
AnthonyRonning merged 2 commits into
masterfrom
codex-sysone-tf-maple
Oct 24, 2026
Merged

AnthonyRonning merged 2 commits into
masterfrom
codex-sysone-tf-maple

Conversation

@AnthonyRonning

@AnthonyRonning AnthonyRonning commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

POST /v1/systemone now also serves Tinfoil's decision models: decision-small, decision-medium and decision-large (vLLM-SR's d3 family in Tinfoil's attested enclaves). A request picks one with model. Request and response fields are unchanged, and glm-5-3-flash stays the default with its behavior unchanged.

  • Decision models answer every question of a request with one upstream POST /v1/decisions, sent through the in-process attested Tinfoil client. web/system_one/decisions.rs translates the System One shape both ways, following d3's own reading of a System One question:
    • a noul becomes a predicate; a noul with criteria becomes a boolean choice with the criteria as option descriptions, false first;
    • choice and score keep their labels, descriptions and level order;
    • non-string values are sent as JSON with Python separators, the way d3 renders them.
  • Answers keep the System One shapes. Probabilities come back in option order with TypeSafe's confidence. temperature is 1 unless the request sets one; an explicit value is applied as softmax(ln p / T). option_mass is 1, because a classification head only scores the offered options (vLLM reports the same label_mass).
  • GLM-5.3-Flash is untouched: same prompt, calibration, limits, errors and billing. The shared upstream plumbing (permit, health probe, send, usage, error classification) moved from SystemOneCall::send_read into Upstream::send, which both engines use, without changing what it does.

Per-model contract

glm-5-3-flash (default) decision-*
images up to 4 422 system_one_images_unsupported (new)
Options per choice 2–255 2–26 (422 system_one_bad_option_count, message names the limit)
Score levels 2–10, any text 2–10, distinct non-blank text, as d3 requires
State + one question 32k + 16k tokens also within the model's 32,768-token context, else 413 system_one_context_exceeded (new)
Default temperature calibrated for the option count 1
option_mass measured 1
usage.requests / output_tokens one per question / one per question 1 / 0
Upstream Continuum, masked logprob read Tinfoil /v1/decisions

system_one_unsupported_model now lists the four model ids. Unknown models still return 422.

Integrations

  • Models. model_config: a DecisionModel table, Pro-only. These models never resolve as completion models and appear in neither chat catalog, so chat, Responses and the Research model picker do not see them.
  • Routes and health. provider_registry: one Tinfoil route per decision model, kept apart from completion routing. inference/health: those routes are seeded, so probes, circuit breaking and capacity gates cover them.
  • Billing.
    • One usage event per call: provider tinfoil, model = the public id, input tokens as Tinfoil counts them (the state is counted once per question).
    • Needs MaplePrivacyLabs/maple-billing-server#135 deployed first. Without its price rows, the billing server prices an unknown Tinfoil model at its Kimi K2.5 fallback rate.
  • SDKs (Rust and TypeScript). Doc comments and READMEs only. The types are unchanged, since every answer still carries temperature and option_mass.
  • Proxy. README only. /v1/systemone was already forwarded, and model is opaque to the proxy.
  • Maple Agent. Registers the three decision models as text-only Pi classifier models next to glm-5-3-flash.

Validation

Unit and component gates

  • Backend (services/opensecret): cargo fmt --all -- --check, cargo clippy --locked --all-targets --all-features -- -D warnings, and cargo test --locked --all-features: 893 passed, 0 failed, 34 ignored (the ignored set is the usual disposable-DB and live-provider tests).
  • New backend tests:
    • the decision-model table and its Pro access, kept out of both catalogs and out of completion resolution;
    • one Tinfoil route per decision model;
    • health seeding of those routes;
    • in system_one::decisions: request lowering, answer reading (shape, refusal, value and order checks), usage, the temperature override, per-model validation, and a loopback test of the wire request through the Tinfoil transport.
  • Maple Agent: provider tests 30 passed (including decision-model registration); pi-ai System One tests 5 passed.
  • Hook: the repository pre-commit gates passed on both commits: TypeScript SDK (format, build, tests), Rust SDK (fmt, clippy, 123 tests, docs), OpenSecret (fmt, clippy, 893 tests), and Maple Agent (fmt, git-metadata test, workspace clippy, workspace tests: 2,026 passed).

Local stack, end to end

  • Setup:
    • maple-dev-env workspace, with the backend from this branch, local billing and Continuum proxy, and the real Continuum and Tinfoil upstreams;
    • requests through the released maple-proxy 0.4.2, unchanged, with Free and Pro fixture API keys;
    • 15 cases paced at about 8 per minute.
Case Result
GLM, model omitted and explicit 200; calibrated temperatures (2.48 / 2.41), measured option_mass, requests: 3, output_tokens: 3
decision-small / -medium / -large, same 3 questions 200 in 0.51 / 0.37 / 0.69 s; temperature 1, option_mass 1, requests: 1, output_tokens: 0
noul with criteria, structured instructions and descriptions 200; the criteria move the answer (0.84 plain against 0.04 described)
64 questions on decision-medium 200 in 0.57 s, one upstream request
temperature: 2 on decision-medium 200; applied and echoed
images on a decision model 422 system_one_images_unsupported
27 options on a decision model 422 system_one_bad_option_count (limit 26)
~32k-token state on a decision model 413 system_one_context_exceeded
256 options on GLM 422 system_one_bad_option_count (limit 255, unchanged)
unknown model 422 system_one_unsupported_model
Free plan with API credits, decision model and GLM 403 model_not_available_on_plan
  • Usage events: the backend log shows one event per decision call: provider tinfoil, model decision-*, Tinfoil's input tokens (329 for 3 questions, 6,774 for 64), 0 output. GLM still emits one per question.

Before this change

  • Live probes of Tinfoil's /v1/decisions through the attested router established its limits: text only, 26 options, 10 levels, 64 questions, 32,768-token prompt per question, strict schema.

  • Paced eval on SST-2 and AG News (100 each) through this translation, accuracy:

    Model SST-2 AG News
    glm-5-3-flash (current System One) 0.95 0.88
    decision-small 0.93 0.86
    decision-medium 0.92 0.88
    decision-large 0.96 0.91

    decision-large also has the best log loss on both.

Not covered here: deployed Dev or Prod behavior, and billing's pricing of these events in a deployed environment (local billing does not consume usage events; maple-billing-server#135 has the pricing tests).

Rollout

  1. Merge and deploy maple-billing-server#135.
  2. Merge this PR and deploy the backend.
  3. Merge MaplePrivacyLabs/maple-marketing#167 (docs, draft, stacked on feat: enable Gemma 3 27B model with starter plan access #110).

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

AnthonyRonning and others added 2 commits October 24, 2026 18:21
`POST /v1/systemone` now also answers with Tinfoil's decision models
(`decision-small`, `decision-medium`, `decision-large`: vLLM-SR's d3 family in
Tinfoil's attested enclaves), selected by `model`. Request and response fields
are unchanged, and `glm-5-3-flash` stays the default with the same prompt,
calibration, limits, errors and billing.

A decision-model request is one upstream `POST /v1/decisions` through the
in-process attested Tinfoil client. `web/system_one/decisions.rs` translates
the System One shape both ways, following d3's own reading of a question: a
`noul` is a predicate, or a false-first boolean choice described by its
criteria; choices and scores keep their labels, descriptions and order;
non-string values are sent as JSON with Python separators. Answers keep their
shapes, with TypeSafe's confidence, temperature 1 unless the request sets one
(applied as softmax(ln p / T)) and an option_mass of 1, since a
classification head only scores the offered options.

Decision models read text only (images: 422
`system_one_images_unsupported`), take 2 to 26 options per choice and need
distinct non-blank score levels, and the state plus one question must fit the
model's 32,768-token context (413 `system_one_context_exceeded`). They are
Pro-only, in neither chat catalog, and have one Tinfoil route each, which
route health tracks. Usage is published once per call.

The upstream send, probe, usage and error handling moved from
`SystemOneCall::send_read` into `Upstream::send`, shared by both engines with
no change in what it does. SDK and proxy docs describe the models; the SDK
types are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pi's classifier registry now lists `decision-small`, `decision-medium` and
`decision-large` next to `glm-5-3-flash`, on the same System One API and
account transport. They judge text only and take a 32,768-token context per
question.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AnthonyRonning
AnthonyRonning marked this pull request as ready for review October 24, 2026 21:12
@AnthonyRonning
AnthonyRonning merged commit 1c8f01b into master Oct 24, 2026
53 checks passed
@AnthonyRonning
AnthonyRonning deleted the codex-sysone-tf-maple branch October 26, 2026 03:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant