Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .agents/skills/redacted-circuit-maintenance/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,9 @@ Each section below is a rule plus the incident that produced it (2026-09-27, PR
"serialized keys and/or value are too large".
- **Browser limits.** The mid tier (2^20) proves in about 25 s in a browser. The large tier (2^22)
needs about 7 GB and is CLI-only (native bb via bb.js 5 in Node).
- Measure proving memory with `/usr/bin/time -l <bb> prove ...` ("peak memory footprint"). Sampling
`footprint`/`vmmap`/RSS of pgrep children read ~2 GB for XL while the real peak is 13.7 GB (large
7.4 GB): the sampler caught the wrong process or missed the short MSM spikes.
- `nargo test` inside `src/circuit/vendor/zkemail` resolves the parent package ("Selected package
`email_mask` was not found"). Copy the vendor dir elsewhere to run the lib tests.

Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ $R verify "https://redacted.zk.email/verify?id=…" # proof check + DKIM k
```

- It uses the same circuits as the site: pinned by commit, and each artifact and verification key is checked against its sha256 before use.
- It proves emails with bodies up to **48,000 bytes**. The "large" tier is CLI-only; bodies over 8,448 bytes need about 7 GB of RAM and about a minute on a recent laptop.
- It proves emails with bodies up to **100,352 bytes**. The "large" and "xl" tiers are CLI-only. Bodies over 8,448 bytes need about 7 GB of RAM and about a minute on a recent laptop, and bodies over 48,000 bytes need about 14 GB (a 16 GB+ machine) and a couple of minutes.
- `publish` sends only `{publicInputs, proof}`. The raw email and the mask arrays are never sent.
- `verify` also checks that the proof's DKIM public-key hash matches the key published in DNS (or in [archive.zk.email](https://archive.zk.email)) for the revealed `d=`/`s=`. The web verify page doesn't do that check yet.
- Agents that fetch the site find the skill through `/llms.txt`, a `<link rel="alternate">` to `SKILL.md`, and a static block in `index.html`. Humans see it in the "For AI agents" section on the home page.
Expand Down Expand Up @@ -94,6 +94,7 @@ selects the smallest circuit that fits your email:
| `email_mask_{1024,2048}_small` | 1024 / 2048-bit | 2 KB | 4 KB | browser or CLI |
| `email_mask_{1024,2048}_mid` | 1024 / 2048-bit | 2 KB | 8.4 KB | browser or CLI |
| `email_mask_{1024,2048}_large` | 1024 / 2048-bit | 4 KB | 48,000 B | CLI only (~7 GB RAM, ~1 min) |
| `email_mask_{1024,2048}_xl` | 1024 / 2048-bit | 4 KB | 100,352 B | CLI only (~14 GB RAM (16 GB+ machine), ~2–4 min) |

Verification uses precomputed verification keys (`src/circuit/target/vk/`), so it takes seconds in
the browser for every tier.
Expand Down
2 changes: 1 addition & 1 deletion public/llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Flow:
1. Get the raw email. Gmail MCP: call `get_message` with `messageFormat: "RAW"`. Gmail API: `format=raw`. Browser: Gmail "Show original" → "Download original".
2. `node ~/.redacted-prover/redacted.mjs inspect email.eml` checks DKIM and shows the signed, canonical content.
3. `... prove email.eml --dry-run [--only-headers from,subject,date] [--hide-body --reveal "text"] [--hide "secret"]` previews exactly what becomes public. Confirm it with the user.
4. `... prove email.eml <same flags> --publish` proves locally and uploads only the proof and its public outputs. It prints `https://redacted.zk.email/verify?id=<uuid>`. Size limits: canonical body up to 49,152 bytes and signed header up to 4,096 bytes. Bodies over 8,448 bytes use the CLI-only "large" tier (~1 min, ~7 GB RAM).
4. `... prove email.eml <same flags> --publish` proves locally and uploads only the proof and its public outputs. It prints `https://redacted.zk.email/verify?id=<uuid>`. Size limits: canonical body up to 100,352 bytes and signed header up to 4,096 bytes. Bodies over 8,448 bytes use the CLI-only "large" tier (~1 min, ~7 GB RAM), and bodies over 48,000 bytes the CLI-only "xl" tier (~2–4 min, ~14 GB RAM, needs a 16 GB+ machine).
5. `... verify <verify-url>` checks a proof locally and checks that its DKIM key hashes (modulus and redc) match the sender's DNS key (or archive.zk.email). Exit code 4 flags a legacy v1 proof, made before the 2026 soundness fixes. Don't rely on it.

Privacy: the .eml is never sent anywhere. The published proof contains only the bytes you left unmasked. By default the values of to/cc/bcc/reply-to/list-post/list-unsubscribe are hidden, along with DKIM bh=.
Expand Down
18 changes: 12 additions & 6 deletions skills/redacted-email-proof/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,12 @@ the smallest tier that fits (RSA 1024- or 2048-bit keys):
| small | 2,048 B | 4,096 B | seconds, ~2 GB RAM |
| mid | 2,048 B | 8,448 B | seconds, ~3 GB RAM |
| large | 4,096 B | 48,000 B | ~1 min on 18 cores (longer on fewer), **~7 GB RAM** |
| xl | 4,096 B | 100,352 B | ~2–4 min on 18 cores (longer on fewer), **~14 GB RAM** (16 GB+ machine) |

The large-tier figures were measured on a 34 KB PayPal receipt: 58 s, 6.6 GB peak for the native `bb`.
The large-tier figures were measured on a 34 KB PayPal receipt: 58 s, 6.6 GB peak for the native
`bb`. XL figures were measured on 87.6 KB and 94.9 KB receipts (loaded machine; faster idle). Peak
memory is the kernel's lifetime physical footprint of the `bb` process (`/usr/bin/time -l`): large
7.4 GB, XL 13.7 GB (18-core M-series Mac, 2026-09-29).

**Quoted-printable emails** (most HTML email: `=3D`, `=E2=82=AC`, soft line breaks) are shown
**decoded** by `inspect`, the preview, `verify` and the website. `--hide` and `--reveal` match
Expand All @@ -68,8 +72,8 @@ DKIM signatures from a domain other than `From:` (ESPs like ccsend.com, subdomai
email.airbnb.com) and `c=simple` header canonicalization are both supported. `verify` prints the
signing `d=` and warns when it isn't the `From:` domain.

The website can prove small and mid in a browser. **Large is CLI-only**, but its proofs verify
on the website like any other.
The website can prove small and mid in a browser. **Large and XL are CLI-only**, but their proofs
verify on the website like any other (XL in about 7 s).

## 3. Decide with the user what to reveal, then preview it

Expand Down Expand Up @@ -151,8 +155,10 @@ it to the sender's domain.

- `DKIM verification failed`: the file isn't the original raw message (see step 1), or the key was
rotated and isn't in the archive. Try a newer email.
- `canonical body is N bytes`: the email is longer than the large tier (48,000-byte body,
- `canonical body is N bytes`: the email is longer than the XL tier (100,352-byte body,
4,096-byte header). Pick a shorter or plain-text email.
- Out of memory: the large tier needs about 7 GB free. Close other programs or set
`REDACTED_THREADS=4`.
- Out of memory: the large tier needs about 7 GB free and XL about 14 GB (in practice a 16 GB+ machine). Close other programs or
set `REDACTED_THREADS=4`.
- `bb did not start in time`: bb.js waits only 5 s for the native prover to start. The CLI retries
automatically; if it keeps failing, the machine is overloaded, so try again when it's idle.
- Point the CLI at another deployment with `REDACTED_SITE` / `REDACTED_API`.
35 changes: 31 additions & 4 deletions skills/redacted-email-proof/scripts/redacted.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ const API = (process.env.REDACTED_API || `${SITE}/api`).replace(/\/$/, "");
// v1 = the original beta.5 / bb 0.84 circuits. Kept ONLY to verify links created before the upgrade,
// and every v1 proof is reported as legacy: v1 published bytes past the signed length (a prover
// could append unsigned text), and its 2048-bit key hash doesn't bind redc (zkemail.nr PR #62).
const CIRCUIT_REF = "f7fd8f459f8ffa112599bc20bc947d6a0167c114";
const CIRCUIT_REF = "530980ca2869121605bffb13d0d3cf381dadd9b9";
const GH_RAW = (ref) => `https://raw.githubusercontent.com/zkemail/Redacted/${ref}/src/circuit`;
const V2 = {
version: 2,
Expand All @@ -51,6 +51,9 @@ const V2 = {
{ name: "email_mask_1024_large", keyBits: 1024, maxHeaderLength: 4096, maxBodyLength: 48000, file: "email_mask_1024_large.json",
sha256: "bf5897eb4ba364925d391f63fe639e032e4e14d8aea52a9e8db96088b59f15c8",
vkSha256: "ae0335801e676bfc396b868849afae8f6d849178e3a17ece6559819137577a6a" },
{ name: "email_mask_1024_xl", keyBits: 1024, maxHeaderLength: 4096, maxBodyLength: 100352, file: "email_mask_1024_xl.json",
sha256: "4cf398ca2b9b3fc01c17738a1450dc8246419d57e7680c2670823a175b96ca4a",
vkSha256: "e43d369fd72f7b2d651ed1f8e6c87103a8547de2c3ab20d7e599ec91f0c2d4a8" },
{ name: "email_mask_2048_small", keyBits: 2048, maxHeaderLength: 2048, maxBodyLength: 4096, file: "email_mask_2048_small.json",
sha256: "cd6d5ad5131ba0139ae7ffabf394c369c03e2f64ef621206453b8dd4eda60694",
vkSha256: "088a51ad9d8560fe0aa6e42f68ec4c16a70d83c4165806b87e5816784658b649" },
Expand All @@ -60,6 +63,9 @@ const V2 = {
{ name: "email_mask_2048_large", keyBits: 2048, maxHeaderLength: 4096, maxBodyLength: 48000, file: "email_mask_2048_large.json",
sha256: "8022a74af7eb7145565d6bb8612b9ddbd41a321b556d1e26f45295a213ddea3e",
vkSha256: "c2dac85da0771ccc1b8112f51dc1a057558be46007eaa862edc59169c2ba3657" },
{ name: "email_mask_2048_xl", keyBits: 2048, maxHeaderLength: 4096, maxBodyLength: 100352, file: "email_mask_2048_xl.json",
sha256: "31b7e7db5008745f3c4eea4abe1b4724319079cf4d69ce2b5cd4d5f64919ff90",
vkSha256: "d4f20b3ec18cc317e9ff8e2fad5c1984e49ef92431f39428c4f11c9dc8ce746b" },
],
};
const V1 = {
Expand Down Expand Up @@ -242,6 +248,22 @@ function pickCircuit(keyBits, headerLen, bodyLen) {

const sha256hex = (b) => createHash("sha256").update(b).digest("hex");

// REASON: bb.js 5's native backend waits a hardcoded 5 s for the spawned `bb` to open its socket
// ("Timeout waiting for bb socket connection"). On a busy or slow machine that fails spuriously
// (seen at load average ~250 while proving the XL tier), so retry a few times with backoff
// before giving up.
async function newBarretenberg(bb, opts, attempts = 4) {
for (let i = 1; ; i++) {
try {
return await bb.Barretenberg.new(opts);
} catch (e) {
if (i >= attempts || !/socket/i.test(String(e?.message))) throw e;
log(`bb did not start in time (attempt ${i}/${attempts}); retrying…`);
await new Promise((r) => setTimeout(r, 2000 * i));
}
}
}

// Download (once) and cache a pinned artifact; refuses anything whose sha256 doesn't match.
async function fetchPinned(set, relPath, sha256, label) {
const dir = path.join(process.env.REDACTED_CACHE || path.join(HERE, ".circuits"), set.ref.slice(0, 12));
Expand Down Expand Up @@ -753,16 +775,21 @@ async function cmdProve(file, opts) {
inputs.pubkey.redc = redcLimbsV2(bignum, dkim.publicKey, dkim.modulusLength);
const threads = Number(process.env.REDACTED_THREADS) || Math.max(1, os.cpus().length);
const large = c.maxBodyLength > 8448;
const xl = c.maxBodyLength > 48000;
log(`Generating witness…`);
const t0 = Date.now();
const { witness } = await new noir.Noir(circuit).execute(inputs);
log(
`Proving with UltraHonk (${threads} threads; ` +
(large ? `large tier: ~1 min on 16+ cores, longer on fewer, ~7 GB RAM)…` : `typically under a minute, ~2 GB RAM)…`),
(xl
? `XL tier: ~2-4 min on 16+ cores, longer on fewer, ~14 GB RAM, needs a 16 GB+ machine)…`
: large
? `large tier: ~1 min on 16+ cores, longer on fewer, ~7 GB RAM)…`
: `typically under a minute, ~2 GB RAM)…`),
);
// In Node, bb.js 5 runs the native bb binary it ships (no 4 GB WebAssembly memory cap), which is
// what makes the 48 KB-body tier provable at all.
const api = await bb.Barretenberg.new({ threads });
const api = await newBarretenberg(bb, { threads });
let proof;
try {
proof = await new bb.UltraHonkBackend(circuit.bytecode, api).generateProof(witness);
Expand Down Expand Up @@ -846,7 +873,7 @@ async function verifyWith(c, proof) {
try {
if (c.set.version === 2) {
// v2: VK-only verification — no circuit download, milliseconds even for the large tier
const api = await bb.Barretenberg.new({ threads: 1 });
const api = await newBarretenberg(bb, { threads: 1 });
try {
return await new bb.UltraHonkVerifierBackend(api).verifyProof({ ...proof, verificationKey: await loadVk(c) });
} finally {
Expand Down
16 changes: 16 additions & 0 deletions src/circuit-configs.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,14 @@
"keyBits": 1024,
"browser": false
},
{
"name": "email_mask_1024_xl",
"outputFile": "email_mask_1024_xl.json",
"maxHeaderLength": 4096,
"maxBodyLength": 100352,
"keyBits": 1024,
"browser": false
},
{
"name": "email_mask_2048_small",
"outputFile": "email_mask_2048_small.json",
Expand All @@ -47,6 +55,14 @@
"maxBodyLength": 48000,
"keyBits": 2048,
"browser": false
},
{
"name": "email_mask_2048_xl",
"outputFile": "email_mask_2048_xl.json",
"maxHeaderLength": 4096,
"maxBodyLength": 100352,
"keyBits": 2048,
"browser": false
}
],
"legacy": [
Expand Down
1 change: 1 addition & 0 deletions src/circuit/target/email_mask_1024_xl.json

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions src/circuit/target/email_mask_2048_xl.json

Large diffs are not rendered by default.

Binary file added src/circuit/target/vk/email_mask_1024_xl.vk
Binary file not shown.
Binary file added src/circuit/target/vk/email_mask_2048_xl.vk
Binary file not shown.
2 changes: 1 addition & 1 deletion src/lib.ts
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ const publicInputCount = (c: CircuitConfig) =>
prefixLength(c.version) + c.maxHeaderLength + c.maxBodyLength;

// Browser-provable v2 circuits. Vite code-splits each JSON, so only the selected one downloads.
// NOTE: the large tier is intentionally absent. Proving its 2^22-gate circuit needs ~7 GB,
// NOTE: the large (2^22) and xl (2^23) tiers are intentionally absent. Proving them needs ~7.4 GB (large) and ~13.7 GB (xl),
// beyond a browser's 4 GB WebAssembly memory. Those emails are proved with the CLI
// (skills/redacted-email-proof) and verified here from the VK alone.
const V2_LOADERS: Record<string, () => Promise<unknown>> = {
Expand Down
Loading