Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .github/workflows/pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ name: Pages
on:
push:
branches: [main]
paths: ["site/**", "bindings/**", "src/**", "Cargo.*", ".github/workflows/pages.yml"]
paths: ["site/**", "bindings/**", "src/**", "data/**", "Cargo.*", ".github/workflows/pages.yml"]
workflow_dispatch:

permissions:
Expand All @@ -27,6 +27,10 @@ jobs:
version=$(cargo metadata --format-version 1 --locked | jq -r '.packages[] | select(.name == "wasm-bindgen") | .version')
cargo install wasm-bindgen-cli --version "$version" --locked
wasm-bindgen --target web --out-dir site/pkg target/wasm32-unknown-unknown/release/setoff_engine_wasm.wasm
- name: Add the mainnet-day sample
run: |
mkdir -p site/samples
cp data/mainnet-2026-10-21.csv site/samples/mainnet-day.csv
- uses: actions/configure-pages@v6
- uses: actions/upload-pages-artifact@v5
with:
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
target/
site/pkg/
bindings/wasm/pkg-node/
site/samples/
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,13 @@ All notable changes to this project are documented here. The format follows

### Added

- A real mainnet day: `tools/mainnet_day.py` builds a netting window from one
UTC day of USDC payments between institutions on Stellar, and
`examples/mainnet_day.rs` nets it. On 2026-10-21, 336 payments between 10
institutions settle in 9 transfers ([method and limits](docs/mainnet-day.md)).
- Simulator: **A real mainnet day** loads that day, showing results in USDC,
and **Import CSV** reads `id,debtor,creditor,asset,amount` files in the page,
reporting malformed lines by number while loading the rest.
- `setoff-engine-wasm` (`bindings/wasm`): the netting engine for JavaScript,
with per-row validation errors and both plan strategies.
- A [netting simulator](https://setoff-org.github.io/setoff-engine/) on
Expand Down
4 changes: 4 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ criterion = { version = "0.8", default-features = false }
name = "net"
harness = false

[[example]]
name = "mainnet_day"
required-features = ["csv"]

[lints.rust]
unsafe_code = "forbid"
missing_docs = "warn"
Expand Down
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,13 @@ What the reference cases in [`tests/vectors`](tests/vectors) show:
| 200 pseudo-random, USDC | 145 | 715.48 → 141.83 | 80.2% | 11 |
| Fan-in to one merchant | 5 | 150 → 150 | 0% | 5 |

And on real data: every USDC payment between 10 institutions (exchanges,
Circle and others) on Stellar mainnet on 2026-10-21, 336 payments and
55.5 million USDC, would have settled in **9 transfers instead of 336**. Value
moved fell only 3.9% that day, because the flows were mostly one-way; the
gain is 37 times fewer transactions to sign, fee and reconcile.
[How it was measured, and its limits](docs/mainnet-day.md).

## Install

```sh
Expand Down
337 changes: 337 additions & 0 deletions data/mainnet-2026-10-21.csv

Large diffs are not rendered by default.

82 changes: 82 additions & 0 deletions docs/mainnet-day.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# One real mainnet day, netted

What would SetOff have changed for institutions settling USDC with each other
on Stellar? This runs the engine on every USDC payment made between known
institutions on one real day, 2026-10-21 (UTC).

## Result

336 payments between 10 institutions, 55,532,924.24 USDC in total.

| Settlement | USDC moved | Saved | Transfers |
|---|---:|---:|---:|
| Gross (every payment on its own, as happened) | 55,532,924.24 | | 336 |
| Bilateral (each pair settles its difference) | 53,505,367.08 | 3.7% | 13 |
| Multilateral (SetOff: each settles its net position) | 53,355,736.73 | 3.9% | **9** |

**336 payments become 9 transfers, 37 times fewer.** The value saved is
small on this day because the flows were largely one-way: most of them
were exchanges and Circle (`centre.io`) moving USDC in a single direction,
which netting cannot offset. The saving is in operations: nine transactions
to sign, fee, submit and reconcile instead of 336, and nine places a
settlement can fail instead of 336.

The largest net debtors, which is what each would actually have paid:

| Institution | Net USDC owed |
|---|---:|
| binance.com | 48,919,022.28 |
| kraken.com | 4,340,681.00 |
| changelly.com | 60,000.00 |
| bitstamp.net | 26,028.45 |
| hitbtc.com | 10,005.00 |

The more two-way a corridor is, the more value netting saves too. The
[simulator](https://setoff-org.github.io/setoff-engine/) loads this day with
**A real mainnet day**, and takes your own CSV with **Import CSV**.

## How it was built

[`tools/mainnet_day.py`](../tools/mainnet_day.py) (Python standard library only):

1. **Institutions.** Every classic account that stellar.expert's public
[directory](https://stellar.expert/directory) tags as an exchange, anchor,
custodian or wallet: 480 accounts, grouped by organization (home domain,
else name) into 229 institutions. An institution's accounts are one
participant, so moving funds between its own accounts is not an
obligation.
2. **Payments.** For each account, Horizon's payments for that UTC day.
Kept: successful `payment` operations in USDC issued by Circle
(`GA5Z…KZVN`), and path payments with USDC on both legs, whose two ends
belong to *different* institutions. Each operation is counted once,
though it is seen from both ends.
3. **Obligations.** One row per payment in the engine's CSV format, with the
operation id as the row id, so every row can be looked up on a block
explorer (`https://stellar.expert/explorer/public/op/<id>`). Amounts are
stroops (10⁻⁷ USDC), exact.

The file is [`data/mainnet-2026-10-21.csv`](../data/mainnet-2026-10-21.csv).

## Limits

- **A lower bound.** Three very busy accounts (binance.com, ultracapital.xyz
and aps.money) have more payments that day than the script reads per
account (12,000 operations). Their payments with the other institutions are
still captured from the other end, but payments between two of those three
can be missing.
- **The directory is not complete.** Institutions it does not list, and
their payments, are not included.
- **Observed payments, not obligations.** On chain, these payments settled
one by one. Treating a day's payments as one netting window shows what
settling the net would have changed, not what the institutions owed each
other at any moment.

## Reproduce

```sh
python3 tools/mainnet_day.py 2026-10-21 > data/mainnet-2026-10-21.csv
cargo run --example mainnet_day --features csv -- data/mainnet-2026-10-21.csv
```

Any other day works the same way. Fetching takes a while within Horizon's
rate limits.
59 changes: 59 additions & 0 deletions examples/mainnet_day.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
//! Nets one real day of mainnet payments between institutions and reports
//! what settling the net instead of every payment would have saved.
//!
//! python3 tools/mainnet_day.py 2026-10-21 > data/mainnet-2026-10-21.csv
//! cargo run --example mainnet_day --features csv -- data/mainnet-2026-10-21.csv

#![allow(
clippy::unwrap_used,
clippy::expect_used,
clippy::arithmetic_side_effects,
clippy::cast_precision_loss,
missing_docs
)]

use std::collections::BTreeSet;

use setoff_engine::{compare, net, participants, read_csv};

fn main() {
let path = std::env::args().nth(1).expect("usage: mainnet_day <obligations.csv>");
let obligations = read_csv(std::fs::File::open(&path).expect("open the CSV")).expect("read the CSV");
let netting = net(&obligations).expect("net the obligations");
let units = |v: i128| v as f64 / 1e7;

let institutions: BTreeSet<&str> =
obligations.iter().flat_map(|o| [o.debtor.as_str(), o.creditor.as_str()]).collect();
println!("{path}");
println!("{} payments between {} institutions\n", obligations.len(), institutions.len());

for c in compare(&obligations).unwrap() {
let payments = obligations.iter().filter(|o| o.asset == c.asset).count();
let saved = |v: i128| 100.0 * (1.0 - v as f64 / c.gross as f64);
println!("{} moved {:>18} transfers", c.asset, "saved");
println!(" gross {:>16.2} {:>8} {payments:>9}", units(c.gross), "");
println!(
" bilateral {:>16.2} {:>7.1}% {:>9}",
units(c.bilateral),
saved(c.bilateral),
c.bilateral_transfers
);
println!(
" multilateral {:>16.2} {:>7.1}% {:>9}",
units(c.multilateral),
saved(c.multilateral),
c.multilateral_transfers
);
}
debug_assert_eq!(
netting.transfers.len(),
compare(&obligations).unwrap().iter().map(|c| c.multilateral_transfers).sum::<usize>()
);

let mut debtors: Vec<_> = participants(&obligations).unwrap().into_iter().filter(|p| p.collateral > 0).collect();
debtors.sort_by_key(|p| std::cmp::Reverse(p.collateral));
println!("\nlargest net debtors (what each would actually have to pay)");
for p in debtors.iter().take(10) {
println!(" {:<28} {:>16.2} {}", p.participant, units(p.collateral), p.asset);
}
}
117 changes: 111 additions & 6 deletions site/app.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,17 @@ const strategyEl = document.getElementById("strategy");
let rows = [];
let ready = false;

const fmt = (s) => BigInt(s).toLocaleString("en-US");
// Amounts are integers in each asset's smallest unit. By default they are
// shown as typed; a preset that knows an asset's decimals (the mainnet day:
// USDC in stroops, 7) sets them here so the page shows real USDC.
let decimals = {};
const fmt = (s, asset) => {
const v = BigInt(s), d = decimals[asset] || 0;
if (!d) return v.toLocaleString("en-US");
const neg = v < 0n, a = neg ? -v : v, base = 10n ** BigInt(d);
const frac = (a % base).toString().padStart(d, "0").slice(0, 2);
return `${neg ? "-" : ""}${(a / base).toLocaleString("en-US")}.${frac}`;
};
const clean = (s) => String(s).trim().replace(/[,\s_]/g, "");

function randomRows(n) {
Expand All @@ -34,7 +44,8 @@ function randomRows(n) {
return out;
}

function load(list) {
function load(list, units = {}) {
decimals = units;
rows = list.map(([debtor, creditor, asset, amount]) => ({ debtor, creditor, asset, amount }));
render();
}
Expand Down Expand Up @@ -142,20 +153,20 @@ function renderResult(n, count) {
const w = g === 0n ? 0 : Number((s * 1000n) / g) / 10;
html += `<div class="asset"><h3>${esc(a.asset)}</h3>
<div class="bars">
<div class="bar"><span>Owed ${fmt(g)}</span><i style="width:100%"></i></div>
<div class="bar"><span>Moves ${fmt(s)}</span><i class="settled" style="width:${Math.max(w, s > 0n ? 1 : 0)}%"></i></div>
<div class="bar"><span>Owed ${fmt(g, a.asset)}</span><i style="width:100%"></i></div>
<div class="bar"><span>Moves ${fmt(s, a.asset)}</span><i class="settled" style="width:${Math.max(w, s > 0n ? 1 : 0)}%"></i></div>
</div></div>`;
}
html += `<h3>Settlement plan</h3><ol class="plan">`;
html += transfers === 0
? `<li class="none">Nothing moves: every position nets to zero.</li>`
: n.transfers.map((t) => `<li><b>${esc(t.from)}</b> pays <b>${esc(t.to)}</b> ${fmt(t.amount)} ${esc(t.asset)}</li>`).join("");
: n.transfers.map((t) => `<li><b>${esc(t.from)}</b> pays <b>${esc(t.to)}</b> ${fmt(t.amount, t.asset)} ${esc(t.asset)}</li>`).join("");
html += `</ol>`;
if (n.positions.length) {
html += `<h3>Net positions</h3><table class="positions"><thead><tr><th>Participant</th><th>Asset</th><th class="num">Net</th></tr></thead><tbody>`;
html += n.positions.map((p) => {
const v = BigInt(p.net);
return `<tr><td>${esc(p.participant)}</td><td>${esc(p.asset)}</td><td class="num ${v > 0n ? "pos" : "neg"}">${v > 0n ? "+" : ""}${fmt(p.net)}</td></tr>`;
return `<tr><td>${esc(p.participant)}</td><td>${esc(p.asset)}</td><td class="num ${v > 0n ? "pos" : "neg"}">${v > 0n ? "+" : ""}${fmt(p.net, p.asset)}</td></tr>`;
}).join("");
html += `</tbody></table>`;
}
Expand All @@ -170,9 +181,103 @@ function esc(s) {
function encode() { return btoa(unescape(encodeURIComponent(JSON.stringify(rows.map((r) => [r.debtor, r.creditor, r.asset, r.amount]))))); }
function decode(h) { try { return JSON.parse(decodeURIComponent(escape(atob(h)))); } catch { return null; } }

const mainnetNote = document.getElementById("mainnet-note");
document.querySelectorAll("[data-example]").forEach((b) => b.addEventListener("click", () => {
mainnetNote.hidden = true;
csvErrors.hidden = true;
load(b.dataset.example === "random" ? randomRows(40) : EXAMPLES[b.dataset.example]);
}));

// CSV in the format `setoff net` and the engine's read_csv take:
// id,debtor,creditor,asset,amount. Quoted fields are allowed. Returns the
// rows and one message per line that could not be read; good lines load.
const CSV_HEADER = ["id", "debtor", "creditor", "asset", "amount"];
function parseCSV(text) {
const records = [];
let field = "", record = [], quoted = false, line = 1, start = 1;
const endField = () => { record.push(field); field = ""; };
const endRecord = () => { endField(); records.push({ line: start, fields: record }); record = []; start = line; };
for (let i = 0; i < text.length; i++) {
const ch = text[i];
if (quoted) {
if (ch === '"' && text[i + 1] === '"') { field += '"'; i++; }
else if (ch === '"') quoted = false;
else { if (ch === "\n") line++; field += ch; }
} else if (ch === '"') quoted = true;
else if (ch === ",") endField();
else if (ch === "\n") { line++; endRecord(); }
else if (ch !== "\r") field += ch;
}
if (field !== "" || record.length) endRecord();
const rows = [], errors = [];
const [head, ...body] = records.filter((r) => r.fields.some((f) => f.trim() !== ""));
if (!head || head.fields.map((f) => f.trim()).join(",") !== CSV_HEADER.join(",")) {
errors.push(`line 1: the header must be ${CSV_HEADER.join(",")}`);
return { rows, errors };
}
for (const { line: n, fields } of body) {
if (fields.length !== CSV_HEADER.length) { errors.push(`line ${n}: expected ${CSV_HEADER.length} fields, found ${fields.length}`); continue; }
const [, debtor, creditor, asset, amount] = fields.map((f) => f.trim());
if (!/^(0|[1-9][0-9]*)$/.test(amount)) { errors.push(`line ${n}: amount ${JSON.stringify(amount)} is not a whole number`); continue; }
rows.push([debtor, creditor, asset, amount]);
}
return { rows, errors };
}

const csvErrors = document.getElementById("csv-errors");
function showCSVErrors(errors, name) {
csvErrors.hidden = errors.length === 0;
if (!errors.length) return;
const shown = errors.slice(0, 20).map((e) => `<li>${esc(e)}</li>`).join("");
const more = errors.length > 20 ? `<li>…and ${errors.length - 20} more</li>` : "";
csvErrors.innerHTML = `<strong>${errors.length} line${errors.length === 1 ? "" : "s"} of ${esc(name)} could not be read and ${errors.length === 1 ? "was" : "were"} skipped:</strong><ul>${shown}${more}</ul>`;
}

document.getElementById("csv-file").addEventListener("change", async (e) => {
const file = e.target.files[0];
e.target.value = ""; // the same file can be chosen again after a fix
if (!file) return;
mainnetNote.hidden = true;
try {
const { rows: list, errors } = parseCSV(await file.text());
showCSVErrors(errors, file.name);
if (list.length) {
load(list);
history.replaceState(null, "", location.pathname);
} else if (!errors.length) {
showCSVErrors(["the file has a header but no obligations"], file.name);
}
} catch (err) {
showCSVErrors([`could not read the file: ${err.message}`], file.name);
}
});

// One UTC day of real USDC payments between institutions on Stellar mainnet,
// built by tools/mainnet_day.py. Amounts stay exact, in stroops (10^-7 USDC).
const MAINNET = { file: "samples/mainnet-day.csv", day: "2026-10-21" };
const mainnetBtn = document.getElementById("mainnet");
mainnetBtn.addEventListener("click", async () => {
mainnetBtn.disabled = true;
mainnetNote.hidden = false;
csvErrors.hidden = true;
mainnetNote.textContent = "Loading the day's payments…";
try {
const res = await fetch(MAINNET.file);
if (!res.ok) throw new Error(`HTTP ${res.status}`);
const { rows: list, errors } = parseCSV(await res.text());
if (errors.length) throw new Error(errors[0]);
load(list, { USDC: 7 });
mainnetNote.innerHTML = `${list.length.toLocaleString("en-US")} real USDC payments made on ${MAINNET.day} (UTC) between institutions
that <a href="https://stellar.expert/directory">stellar.expert's directory</a> lists as exchanges, anchors, custodians or wallets,
grouped by institution. The table holds amounts in stroops (10<sup>−7</sup> USDC), exactly as recorded on chain; results are shown in USDC.
<a href="https://github.com/SetOff-Org/setoff-engine/blob/main/docs/mainnet-day.md">How this was built</a>.`;
history.replaceState(null, "", location.pathname);
} catch (e) {
mainnetNote.textContent = `The mainnet sample couldn't be loaded (${e.message}). Check your connection and try again.`;
} finally {
mainnetBtn.disabled = false;
}
});
document.getElementById("add").addEventListener("click", () => {
rows.push({ debtor: "", creditor: "", asset: rows.at(-1)?.asset || "USDC", amount: "" });
render();
Expand Down
4 changes: 4 additions & 0 deletions site/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,12 @@ <h2 id="obligations-h">Obligations</h2>
<button type="button" data-example="cycle">Three-way cycle</button>
<button type="button" data-example="merchant">Fan-in to a merchant</button>
<button type="button" data-example="random">Random 40</button>
<button type="button" id="mainnet" class="primary">A real mainnet day</button>
<label class="file-btn">Import CSV<input type="file" id="csv-file" accept=".csv,text/csv"></label>
</div>
</div>
<p class="hint" id="mainnet-note" hidden></p>
<div class="error-box" id="csv-errors" role="alert" hidden></div>
<p class="hint">Amounts are whole units of the asset. Each row is one obligation: the debtor owes the creditor.</p>
<table class="obligations">
<thead><tr><th scope="col">Debtor</th><th scope="col">Creditor</th><th scope="col">Asset</th><th scope="col" class="num">Amount</th><th scope="col"><span class="sr">Remove</span></th></tr></thead>
Expand Down
6 changes: 6 additions & 0 deletions site/style.css
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,12 @@ button, select, input { font: inherit; color: inherit; }
button { border: 1px solid var(--line); background: var(--panel); border-radius: 8px; padding: 6px 12px; cursor: pointer; }
button:hover { border-color: var(--accent); }
button.primary { background: var(--accent); color: var(--accent-ink); border-color: var(--accent); font-weight: 600; }
.file-btn { border: 1px solid var(--line); background: var(--panel); border-radius: 8px; padding: 6px 12px; cursor: pointer; color: var(--ink); }
.file-btn:hover { border-color: var(--accent); }
.file-btn:focus-within { outline: 3px solid var(--accent); outline-offset: 2px; }
.file-btn input { position: absolute; width: 1px; height: 1px; opacity: 0; }
#csv-errors { margin: 8px 0; }
#csv-errors ul { margin: 6px 0 0; padding-left: 20px; }
:focus-visible { outline: 3px solid var(--accent); outline-offset: 2px; }
table { width: 100%; border-collapse: collapse; }
.obligations th { text-align: left; font-size: 13px; color: var(--muted); font-weight: 600; padding: 6px 4px; }
Expand Down
Loading
Loading