Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
f226ce1
feat!: authority-boundary runtime, fact-derived admission, durable hi…
GCdePaula Aug 22, 2026
f1b4b07
docs: distill the corpus — living docs timeless, history in the register
GCdePaula Aug 23, 2026
1fcb9aa
fix(egress): make the WS feed's I11 filter unconstructible-to-violate
GCdePaula Aug 24, 2026
02a2b34
refactor(run)!: consume config in prepare; one drain loop; collapse t…
GCdePaula Aug 24, 2026
5233880
feat(l1): redact the submitter key at the process edge (SubmitterKey)
GCdePaula Aug 24, 2026
143a290
fix(runtime): simplify fee oracle lifecycle and close review gaps
GCdePaula Aug 25, 2026
7a072de
test(recovery): align E2E clock models
GCdePaula Aug 26, 2026
6918b04
docs(review): record the 2026-09-03 branch stock-take
GCdePaula Sep 4, 2026
b7a9dad
fix(ci): stop styling logs off-TTY; assert the danger exit class, not…
GCdePaula Sep 4, 2026
4c4d2b8
docs: make the runtime prose match what the types enforce
GCdePaula Sep 4, 2026
f9a6890
docs: finish the review-codename sweep
GCdePaula Sep 4, 2026
141fb9d
chore(storage): gate the remaining test-only surface
GCdePaula Sep 4, 2026
394d42a
fix(recovery): log the flush's finality-wait retry as a warning
GCdePaula Sep 4, 2026
f9a6acf
docs(fee): say what fixed_mul's truncation relies on
GCdePaula Sep 4, 2026
b651057
fix(ingress): keep the application's error detail out of the /tx 500 …
GCdePaula Sep 4, 2026
130858f
refactor(app): drop the panicking ApplicationProgress constructor
GCdePaula Sep 4, 2026
2920eae
docs(review): record wave 1 of the stock-take follow-ups
GCdePaula Sep 4, 2026
58ba849
refactor(runtime): containment writes nothing durable; the bracket wr…
GCdePaula Sep 4, 2026
15d9fa7
refactor(storage): the finalized lease carries its inclusion block no…
GCdePaula Sep 4, 2026
99402ab
fix(recovery): the EnsureOpenTip phase names its causes and enforces …
GCdePaula Sep 4, 2026
2d84670
refactor(runtime): workers that only need to stop take the notificati…
GCdePaula Sep 4, 2026
737134f
fix(commands): classify the key file by kind and give every recovery …
GCdePaula Sep 4, 2026
c244153
test(commands): table-drive the exit-code contract and pin two wire v…
GCdePaula Sep 5, 2026
4eb29fd
docs(review): record wave 3 of the stock-take follow-ups
GCdePaula Sep 5, 2026
35d71b6
docs: say only what the code enforces, and give four facts one home
GCdePaula Sep 5, 2026
a510b06
docs: the register owns the rejected alternatives; the ADR keeps its …
GCdePaula Sep 5, 2026
6010ae2
docs: AGENTS.md becomes a map; the invariants register owns the write…
GCdePaula Sep 5, 2026
6ca15ae
docs(review): collapse the six stub ledgers into a review history table
GCdePaula Sep 5, 2026
0837c57
docs: module docs explain rather than defend; the abort bound has one…
GCdePaula Sep 5, 2026
9bbba7e
docs(review): record wave 4 of the stock-take follow-ups
GCdePaula Sep 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ sequencer.db-shm
sequencer.db-wal
/out/
examples/canonical-app/out/
/.DS_Store
.DS_Store
.vscode/
soljson-latest.js
**/states/
Expand Down
183 changes: 120 additions & 63 deletions AGENTS.md

Large diffs are not rendered by default.

11 changes: 8 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,20 +43,25 @@ Rust edition 2024 / Axum API / SQLite (rusqlite, WAL) / EIP-712 signing / SSZ en

`sequencer/src/` is organized by writer role; `storage/<role>.rs` holds each role's storage half.

- `runtime/` — bootstrap, config, shutdown, shared clock.
- `commands/` — the operator command brackets (`run/` plus its worker
supervisor, `setup/`, `flush`) and their command-scoped `config` and
`error` taxonomy (incl. exit-code projection).
- `runtime/` — the runtime authority capabilities, consumed crate-wide:
the exclusive process lock and the runtime scope/shutdown machinery.
- `ingress/` — public write path: `api.rs` (`POST /tx`) + `inclusion_lane/` (hot path).
- `egress/` — internal read path: `api/` (WS subscribe + health) + `l2_tx_feed/`.
- `l1/` — reader, submitter, fee oracle, provider, partition helper.
- `recovery/` — startup preemptive-recovery procedure, runtime danger detector, mempool flusher.
- `storage/` — SQLite persistence, split per writer role.
- `http.rs` — shared HTTP error type + `axum::serve` orchestration.
- `http.rs` — shared HTTP error type + `axum::serve` orchestration; `clock.rs` — the crate-wide wall clock.

## Before You Start Real Work

- **[`AGENTS.md`](AGENTS.md)** — mission, requirements, invariants, duality, recovery, conventions, rules.
- **[`docs/protocol/`](docs/protocol/)** — the authoritative protocol contracts: [`scheduler-semantics.md`](docs/protocol/scheduler-semantics.md) (canonical acceptance algorithm) and [`application-contract.md`](docs/protocol/application-contract.md) (the `Application` FFI trait). Read before touching the scheduler, the gold frontier, the fold, or an `Application` impl.
- **[`docs/invariants.md`](docs/invariants.md)** — cross-module invariants register + the fail-loud check policy. Check it before changing anything it lists as load-bearing.
- **[`docs/review/`](docs/review/)** — dated correctness-review ledgers: known-open findings, settled designs, work packages. Check for open findings in code you're about to touch.
- **[`docs/review/register.md`](docs/review/register.md)** — the review register: open findings, settled decisions, refuted proposals (do-not-re-propose). Check it for open findings in code you're about to touch, and before proposing a mechanism or simplification.
- **[`docs/plans/`](docs/plans/)** — the [authority-boundary ADR](docs/plans/2026-08-authority-boundary-adr.md), active coordination tracks, and in-flight design handoffs. Check before starting work that might belong to a track.
- **[`docs/threat-model/README.md`](docs/threat-model/README.md)** — trust boundaries and in-scope threats.
- **[`docs/recovery/README.md`](docs/recovery/README.md)** — preemptive recovery design + TLA+ proofs.
- **[`docs/snapshots/lifecycle.md`](docs/snapshots/lifecycle.md)** — snapshot lifecycle design + invariants (take/promote/GC, crash-safety). Read before touching the inclusion lane's safe-frontier/snapshot path.
Expand Down
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

46 changes: 33 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,10 @@ A sequencer for Cartesi app-specific rollups. Provides low-latency soft confirma

Rollup applications need fast transaction confirmations. Waiting for L1 finality on every user action (minutes) makes interactive applications impractical. The sequencer bridges this gap: it accepts signed user operations, immediately confirms them (soft confirmation), and asynchronously posts batches to L1. The application sees these batches posted on chain.

The core guarantee: **the off-chain sequencer and the rollup's on-chain scheduler produce identical execution order.** Users get instant feedback while the system converges to L1 truth.
The protocol objective is that, under supported honest operation, **the
off-chain sequencer predicts the same execution order the rollup's on-chain
scheduler later produces.** Soft confirmations are optimistic and may be
invalidated by the recovery cases below; L1 remains canonical truth.

## Two Chains Synchronizing

Expand All @@ -22,7 +25,10 @@ Sequencer (off-chain) Scheduler (on-chain)
user_ops=[D] execute D
```

When things go well, the sequencer's chain and the scheduler's view converge. When they don't — batches arrive stale on L1 — the sequencer detects the divergence and recovers.
When things go well, the sequencer's chain and the scheduler's view converge.
When batches are becoming stale on L1, the sequencer detects the doomed suffix
and runs standard recovery. Terminal canonical divergence is the distinct
content-identity case below.

## Trust Model

Expand All @@ -34,13 +40,25 @@ The sequencer is a **centralized, single-writer** system. It cannot steal funds

**Direct inputs** (L1 → L2 messages, used for deposits) bypass the sequencer entirely. They are posted directly to L1 and are **uncensorable** by the sequencer — the scheduler drains them at every `safe_block` boundary. A censoring sequencer can delay when a direct input is executed (up to `MAX_WAIT_BLOCKS`, ~4h), but cannot prevent it.

During normal operation the sequencer advances logical frame time after five
newly-safe blocks have accumulated. That clock tick drains every covered direct
before later user ops and may also create an empty-direct frame to improve the
application-visible clock. Safe-head publication is best effort: if the node
exposes a multi-block jump, the sequencer creates one frame at the observed tip
and never fabricates intermediate frames.

Soft confirmations are an **optimistic prediction**: the sequencer also
cross-checks every batch the scheduler accepts on L1 against the batch it
sealed locally (a content-identity check), and refuses to operate further the
moment they differ. Detection happens when the divergent batch reaches L1
*safe* finality, so soft confirmations issued inside that window (~2 L1
epochs) can be built on already-diverged state — an inherent, bounded
property of the optimistic model.
cross-checks every at/above-anchor batch its off-chain scheduler simulation
accepts on L1 against the batch it sealed locally (a content-identity check).
When a foreign or byte-different landing reaches L1 *safe* finality and the
input reader ingests it, the same transaction records canonical divergence and
freezes the accepted frontier; the runtime stops when it next observes that
fact, and every later boot refuses until an operator performs cockroach
recovery. User-op chunks committed before runtime observation may still
acknowledge and be rolled back. This check is a narrow zombie/foreign-batch
backstop, not proof that arbitrary application or scheduler divergence cannot
exist, and it does not replace the watchdog. The mechanism and its bounds are
recorded in [`docs/invariants.md`](docs/invariants.md) (I9 and I15).

The third case is handled by the recovery subsystem. Batches that are too old when they reach L1 (`inclusion_block − safe_block ≥ MAX_WAIT_BLOCKS`) are skipped by the scheduler. This "staleness" poisons the nonce counter: all subsequent batches become unreachable regardless of their individual freshness. The sequencer detects this via a danger-zone threshold, preemptively goes offline, flushes the L1 mempool, and cascade-invalidates the doomed chain. See [`docs/recovery/`](docs/recovery/) for the full design, TLA+ formal verification, and design history.

Expand All @@ -51,8 +69,8 @@ The sequencer trusts its own code is bug-free. Recovery means recovery from live
The sequencer is designed to handle:

- **L1 provider outages** — workers retry with exponential backoff. The inclusion lane and API continue operating locally. A wall-clock fallback detects when an outage pushes batches into the danger zone.
- **Process crashes** — recovery runs at startup. All recovery state is derived from SQLite (atomic transactions) and L1 safe state. No external coordination needed.
- **Extended downtime** — on restart, the sequencer syncs to the current L1 safe head, flushes if needed, and recovers.
- **Process crashes** — no operator action is needed: every boot derives any required recovery from SQLite and L1 safe state through the startup reducer, never assuming the previous exit was clean. A terminal death best-effort records its cause in the `terminal_faults` black box, which travels with the data directory for postmortems.
- **Extended downtime** — startup syncs to the current L1 safe head, flushes if needed, and recovers before admission; restart policy is the exit-code contract (a terminal exit means: do not restart, page an operator — the one manual remedy is a fresh-directory `setup --recovery` after canonical divergence).
- **Adversarial L1 mempool** — block builders and private mempools are treated as adversarial. The recovery flusher consumes every pending nonce slot with a no-op so delayed "zombie" submissions cannot land later.

## Interfaces
Expand Down Expand Up @@ -93,7 +111,9 @@ cargo run -p wallet-sequencer -- run
```

A third subcommand, **`flush-mempool`**, settles the batch-submitter wallet
nonce on demand (keyed operator tool).
nonce on demand (keyed operator tool). It is flush-only: it requires a
completed setup and no canonical divergence, and it never performs
Sync/Cascade or launches runtime workers.

`setup` requires: `CARTESI_SEQUENCER_BLOCKCHAIN_HTTP_ENDPOINT`, `CARTESI_SEQUENCER_BLOCKCHAIN_ID`, `CARTESI_SEQUENCER_APP_ADDRESS`, `CARTESI_SEQUENCER_BATCH_SUBMITTER_ADDRESS`.
`run` requires: `CARTESI_SEQUENCER_BLOCKCHAIN_HTTP_ENDPOINT`, `CARTESI_SEQUENCER_AUTH_PRIVATE_KEY` (or `_FILE`); it refuses to boot until `setup` has completed.
Expand All @@ -110,7 +130,7 @@ environment:
CARTESI_SEQUENCER_ALLOW_INSECURE_RPC: "true"
```

Process exit codes follow the R4 orchestrator contract: `0` clean shutdown, `10` restart (expect a recovery boot), `20` transient refusal (retry with backoff), `30` terminal (operator required — e.g. setup not complete, identity mismatch, canonical divergence), `1`/`101` unclassified/panic.
Process exit codes follow the orchestrator exit-code contract: `0` clean shutdown, `10` restart (expect a recovery boot), `20` transient refusal (retry with backoff), `30` terminal (operator required — e.g. setup not complete, identity mismatch, canonical divergence, persistent storage/application invariant failure), `40` a previous instance left work past the checkpoint (wipe the data directory and run `setup --recovery`), and `1` for an unclassified operational failure. Panics inside the command harness or supervised workers are projected to `30` under the fail-loud self-trust policy; `101` remains possible only before the harness can contain the command (for example, process/runtime initialization), and a contained terminal fault that cannot drain within the abort bound exits by `abort()` (SIGABRT, status 134), which supervisors must treat as terminal class. The constants live in `sequencer/src/commands/error.rs`; supervisor recipes are in [`docs/watchdog/operator-deployment.md`](docs/watchdog/operator-deployment.md).

Fixed protocol identity (EIP-712):

Expand Down Expand Up @@ -209,7 +229,7 @@ released even on client disconnect.
- `user_ops`: included user operations
- `sequenced_l2_txs`: append-only ordered replay rows (`UserOp` xor `DirectInput`); inserting into `user_ops` also appends the corresponding replay row via trigger `trg_sequence_user_op`
- `safe_inputs`: direct-input payload stream
- `batch_policy`: singleton knobs and constants for DA-style batch sizing and fee derivation; `batch_policy_derived` exposes `recommended_fee` and `batch_size_target`. Setup writes the first `log_gas_price` (and freshness stamp) for both Fixed and Uniswap modes; Uniswap then refreshes via the setup-pinned WETH/fee-token TWAP source. `log_slack = log(10)` applies the 10× safety margin in log space. Fixed local pricing has no oracle worker. Fees are app-token smallest units — initially USDC (6 decimals) for the wallet prototype — not a protocol-level USDC invariant.
- `batch_policy`: singleton knobs and constants for DA-style batch sizing and fee derivation; `batch_policy_derived` exposes `recommended_fee` and `batch_size_target`. A batch closes on whichever fires first: the derived `batch_size_target` byte budget or the `max_batch_open` wall-clock deadline (an inclusion-lane setting, `CARTESI_SEQUENCER_MAX_BATCH_OPEN_SECONDS`, not a `batch_policy` column). Setup writes the first `log_gas_price` (and observation stamp) for both Fixed and Uniswap modes, failing if the initial Uniswap quote cannot be read. Fixed local pricing has no oracle worker; Uniswap starts from the persisted price and refreshes lazily via the setup-pinned WETH/fee-token TWAP source, retaining that price across transient source failures. `log_slack = log(10)` applies the 10× safety margin in log space. Fees are app-token smallest units — initially USDC (6 decimals) for the wallet prototype — not a protocol-level USDC invariant.

## Project Layout

Expand Down
Loading