Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,15 @@ jobs:
- run: agent-ultra doctor
- run: agent-ultra demo
- run: agent-ultra classify "rm -rf build"

leak-gate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install -e .
# Scan the whole tree plus the head commit message. No whitelist:
# a hit fails the build; the fix is to change the offending text.
- run: agent-ultra-leakgate . --message "$(git log -1 --pretty=%B)"
51 changes: 50 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,55 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]

### Added
- **ultracode** (`agent_ultra.ultracode`) — deterministic multi-agent workflow
engine. Workflows are plain Python modules (`META` + `async def run(wf)`) that
fan work across bounded agents via `wf.agent` (one model call, optional
JSON-Schema validation with feedback retries), `wf.parallel` (barrier) and
`wf.pipeline` (no barrier), under hard `wf.budget` call/token ceilings, with
broker-gated `wf.run_check`. Every run writes a typed event stream
(`events.jsonl`), a replayable journal, per-agent artifacts, and a checksummed
receipt (`verify_receipt`). **Resume** replays completed calls from the
journal and spends budget only on new ones (0 calls when unchanged). A
terminal-safe status card renders from the journal (plain ASCII unless stdout
is a proven interactive UTF terminal; never crashes the run). CLI:
`agent-ultra ultracode run|list|status|resume`, with `--mock` for a fully
offline, keyless route. Bundled examples: `smoke` and `review`
(finders -> skeptic votes -> synthesis). `doctor` and `demo` now cover it; see
[docs/ultracode.md](docs/ultracode.md). 14 offline tests.
- **Receipts bus** (`agent_ultra.receipts_bus`). A unified, authenticated
receipt index over SQLite (WAL, default auto-checkpoint). Each receipt carries
two hashes over its canonical body: `receipt_sha256` (integrity) and a keyed
`receipt_hmac` (authenticity) — kept **separate**, so a hand-authored envelope
with a valid sha256 but no HMAC is `authentic=False` and rejected by
enforce-mode consumers. `kind`/`actor`/`verdict` are closed enums validated at
write time. A failed read raises `BusUnavailable` (never an empty list).
`resolve` + `binds` implement the claim-binding rule (cited / verify-command
with scope+freshness / task_ref; command-, manual-, repair-, route_health-kind
never complete; a verifier candidate binds only on `claim_sha256` match; weak
evidence yields a distinct `ledger-weak` clause). `append_audit` /
`verify_audit` maintain a hash-chained `gate_audit`. New CLI:
`agent-ultra receipts list|show|verify|why|attest` (`attest` is
interactive-only and writes `kind=manual`, which never satisfies completion).
- **Verifier** (`agent_ultra.verifier`). Refute-first claim verification with an
injected check-runner and judge (no network, no host commands by default).
Engine re-check wins when a verify command is declared (exit 0 confirms,
non-zero refutes); otherwise a refute-first `judge` runs, defaulting to
refuted on insufficient evidence; with neither channel, refuted. Emits a
`kind=verifier` receipt stamped with `claim_sha256`; a refuted verdict maps to
a non-acceptable receipt verdict so it can never satisfy a gate. `Verifier`
wraps it with a configurable escalation budget (per-session/per-window caps +
dedup on identical `claim_sha256`).
- **Leak gate** (`agent_ultra.leakgate`, `agent-ultra-leakgate`). Scans the
whole tree (and a commit message / PR body) against a base64 denylist decoded
at runtime, so the denylist file never contains plaintext. Case-insensitive
except a whole-word case-sensitive collision token; path separators
normalised; no file whitelist. Wired into CI as a dedicated job.

### Changed
- Renamed the optional memory adapter to `agent_ultra.adapters.external_memory`
(`ExternalMemoryHooks`) and made the worker-layer leakage test consume the
shared base64 denylist, so no host-specific identifier remains in the tree.

- **Structural PANEL enforcement** (`agent_ultra.panel_receipt`). A phase
labelled PANEL is no longer proof — only a valid panel execution receipt with
real executed lenses satisfies it. The loop writes
Expand Down Expand Up @@ -95,7 +144,7 @@ First public release. Portable, stdlib-only core with thin optional adapters.
- **Memory hooks** (`agent_ultra.memory`): five generic events, all no-ops by
default; no memory system required.
- **Adapters** (all optional, none imported by the core): generic CLI,
LiteLLM, Docker sandbox, Mneme-style memory, Hermes-style, Ktisis-style.
LiteLLM, Docker sandbox, external-memory, Hermes-style, Ktisis-style.
- **Install layer**: `agent-ultra doctor` / `init` / `demo`, one-command
installers for PowerShell / bash / CMD, `config.example.yaml`, `.env.example`,
`INSTALL.md` with an AI-agent handoff prompt, and a troubleshooting guide.
Expand Down
42 changes: 39 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,21 +106,56 @@ agent-ultra doctor --live # probe the endpoint
agent-ultra panel "Is this change production-safe?" --evidence-dir ./src
```

## ultracode — deterministic multi-agent workflows

Where the panel debates one question, **ultracode** runs a *script* that fans
work across many bounded agents and proves what happened. A workflow is a plain
Python module — `META` + `async def run(wf)` — using `wf.agent` (one model
call, optionally schema-validated), `wf.parallel` (barrier), `wf.pipeline` (no
barrier), `wf.budget` (hard call/token ceilings), and `wf.run_check`
(broker-gated host commands). Every run writes a replayable **journal** and a
checksummed **receipt**; a terminal status card renders from the journal, so a
model's own text can never fake progress.

```console
$ agent-ultra ultracode run smoke --mock # offline, no API key
ULTRACODE
Run: 20260707T201444-smoke-18eb5f
Name: smoke
Phase: Chain
Agents: 4/4 complete
Failed: 0
Blocked: 0
Progress: [####]
Status: COMPLETE
Result: {"pings": ["pong-one", "pong-two"], "chained": [...], "calls_spent": 4}
Resume: agent-ultra ultracode resume 20260707T201444-smoke-18eb5f

$ agent-ultra ultracode resume 20260707T201444-smoke-18eb5f --mock
Status: COMPLETE
Result: {..., "calls_spent": 0} # every call replayed from the journal
```

Commands: `ultracode run <workflow>` · `list` · `status` · `resume <run_id>`.
Bundled workflows: **smoke** (fan-out + pipeline) and **review** (finders →
skeptic votes → synthesis). Full guide: [docs/ultracode.md](docs/ultracode.md).

## What's in the box

| module | what it does |
|--------|--------------|
| **panel** | Adversarial review: parallel critic *lenses* → steelman → judge cross-exam → synthesis. Verdicts: `real_now` / `real_later` / `theoretical` / `wrong`. Panel agents are roles, not models — one healthy route runs a whole panel. |
| **ultra_loop** | build → test → panel → classify → fix → re-test → re-panel → **ship gate**. Red tests stop before the panel; low-context panels are refused; only proof-gated work ships. |
| **ultracode** | Deterministic multi-agent workflows: `META` + `async run(wf)` scripts fan work across bounded agents (`parallel`/`pipeline`) under hard budgets, with a resumable journal, a checksummed receipt, and a terminal-safe status card. Fan-out → journal → resume → receipt → status. |
| **broker** | Every model-authored command classified SAFE / ELEVATED / DANGEROUS, ledgered, and executed only if its tier allows. DANGEROUS with no approval path is **denied by default**. |
| **proof** | "Done" requires recorded evidence. Accepted findings become gates; `assert_shippable()` raises on unsupported completion claims. |
| **evidence** | Bounded source gathering with secret redaction; low-context detection. |
| **routes** | Health-probed model routes with degradation: dead routes fall through, mixed mode collapses to single, zero routes fails loudly. |
| **artifacts** | Uniform JSON + Markdown run records and JSONL ledgers for every run. |
| **memory** | Five generic write-back hooks (`on_panel_decision`, `on_finding_accepted`, `on_command_run`, `on_task_complete`, `on_lesson_learned`). No memory system required. |

Adapters (all optional): generic CLI, LiteLLM, Docker sandbox, memory
(Mneme-style), Hermes-style and Ktisis-style runtimes.
Adapters (all optional): generic CLI, LiteLLM, Docker sandbox, external
memory, Hermes-style and Ktisis-style runtimes.

### Workers: router (default) vs Deep Agents (optional)

Expand Down Expand Up @@ -205,7 +240,8 @@ Full guide: [docs/adapter-guide.md](docs/adapter-guide.md).
## Docs

[architecture](docs/architecture.md) · [panel](docs/panel.md) ·
[ULTRA loop](docs/ultra-loop.md) · [command broker](docs/command-broker.md) ·
[ULTRA loop](docs/ultra-loop.md) · [ultracode](docs/ultracode.md) ·
[command broker](docs/command-broker.md) ·
[proof gates](docs/proof-gates.md) · [adapter guide](docs/adapter-guide.md) ·
[security](docs/security.md) · [troubleshooting](docs/troubleshooting.md) ·
[releasing](docs/releasing.md) · [INSTALL](INSTALL.md) ·
Expand Down
2 changes: 1 addition & 1 deletion docs/adapter-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ For the ULTRA loop, provide `builder(workspace, task) -> str` and
| generic CLI | `adapters.cli` | run panel/ultra/classify from the shell |
| LiteLLM | `adapters.litellm_routes` | build a pool over a LiteLLM proxy |
| Docker sandbox | `adapters.docker_sandbox` | `sandbox_argv` for the broker |
| memory (Mneme-style) | `adapters.mneme_memory` | wire hooks to a memory system |
| external memory | `adapters.external_memory` | wire hooks to a memory system |
| Hermes | `adapters.hermes` | env-driven pool + broker for Hermes-style agents |
| Ktisis | `adapters.ktisis` | env-driven ULTRA loop for Ktisis-style build agents |

Expand Down
2 changes: 1 addition & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ core imports an adapter; adapters may make runtime-specific assumptions.
cli ── generic shell entrypoint (zero third-party deps)
litellm_routes ── build a pool over a LiteLLM proxy
docker_sandbox ── run non-approved commands in a container
mneme_memory ── wire hooks to a memory system
external_memory ── wire hooks to a memory system
hermes / ktisis ── env-driven wiring for those runtimes
```

Expand Down
85 changes: 85 additions & 0 deletions docs/receipts-bus.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# Receipts bus

A unified, **authenticated** receipt index. Every unit of agent work can leave
a signed receipt — a record of what was done, what it claimed, and what
evidence backs the claim. The bus stores receipts, verifies them, and resolves
a claim to the receipts that actually *bind* to it.

```python
from agent_ultra.receipts_bus import ReceiptsBus, Candidate

bus = ReceiptsBus(db_path="receipts.db", key_path="install.key")

env = bus.append(kind="panel", actor="engine_a", verdict="shipped",
session_id="s1", workspace="/repo")

bus.verify(env["receipt_id"])
# -> {ok, authentic, integrity, refs_ok, receipt}
```

## Authenticity is separate from integrity

Two hashes cover the canonical body (the envelope minus its two hash fields,
`json.dumps(sort_keys=True)`):

- **`receipt_sha256`** — SHA-256. Integrity only: proves the body was not
altered.
- **`receipt_hmac`** — HMAC-SHA256 keyed by a per-install key read from
`key_path`. This is the authenticity control.

A hand-authored envelope with a correct `receipt_sha256` but no/invalid HMAC is
`authentic=False`, and any enforce-mode consumer must reject it (`ok=False`).

> **Key custody.** The host is responsible for fencing `key_path` from
> untrusted writers. Honest residual: a same-user process that can *read* the
> key can still forge an authentic receipt — closing that needs an
> out-of-process signer, which is out of scope for v1.

## A failed read is never empty

The store is SQLite in WAL mode with the **default** auto-checkpoint left in
place. A read that fails after a bounded retry raises `BusUnavailable` — it
never degrades to `[]`, because a caller cannot tell "no receipts" from "the
store was unreachable" and would wrongly conclude a claim is unsupported.

## Binding rule

`resolve(claim_ctx)` returns origin-annotated `Candidate`s scoped by
session / workspace / window (optionally federating external sources via
injected readers). `binds(claim_ctx, candidate) -> (bool, clause)` decides
which candidates back a **completion** claim:

- an explicitly **cited** receipt id, or
- a canonical **verify-command** match with scope coverage **and** freshness, or
- a **task_ref** match.

Never binds: a `command`-kind candidate to a completion claim alone; a
`verifier`-kind candidate unless its `claim_sha256` equals the claim's;
`manual` / `repair` / `route_health` to completion; anything with a
non-acceptable verdict; anything unauthenticated. Weak-but-present evidence
(uncovered / stale) returns the distinct `ledger-weak` clause — not a pass.

## Gate audit (hash chain)

```python
bus.append_audit("ship", "enforce", "allow", "engine_a", "tests green")
bus.verify_audit("ship") # -> {ok, first_broken}
bus.get_audit_event(event_id) # explain one row
```

Each row carries `prev_sha256` + `row_sha256`; `verify_audit` recomputes the
chain and reports the first broken link.

## CLI

```
agent-ultra receipts list [--session S] [--workspace W]
agent-ultra receipts show RECEIPT_ID
agent-ultra receipts verify RECEIPT_ID
agent-ultra receipts why [--claim-sha ...] [--verify-command ...] [--cited ...]
agent-ultra receipts attest # interactive only; writes kind=manual
```

`attest` is the only creation path outside engine code. It refuses
non-interactive use and writes a `kind=manual` receipt — and manual receipts
never satisfy a completion gate.
Loading
Loading