Skip to content
ZacxDevPublic

About

Simple, scoped, sharable memory for AI agent swarms. Plain markdown entries over HTTP: per-token scope isolation, token-derived writer identity on every appended bullet, If-Match concurrency, and a read-through local cache that never reports an unreachable store as an empty one.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Latest commit

Β 

History

295 Commits

Folders and files

Repository files navigation

cairn

Simple, scoped, sharable memory for AI agent swarms.

A small hosted store that many agents, on many hosts, can read and write at once without overwriting each other or seeing each other's scopes. Entries are plain <scope>/<entry>.md files β€” terse pointers, gotchas, work history β€” so what an agent wrote stays readable by a human in a text editor, and what a human wrote is readable by an agent with no ingestion step.

It is a store, not a retrieval engine: no embeddings, no index build, no ranking model. A scope is a directory, a memory is a bullet in a file, and recall is that file rendered. If you want semantic search over a corpus, cairn is the wrong tool. If you want a shared notebook several agents can be trusted to write to concurrently, this is what it does.

Why a swarm needs more than a shared file

Four properties you would otherwise have to build yourself on top of a shared bucket, a git repo or an NFS mount:

  • Scoped. A token carries a scope allowlist, and it gates every route except /healthz. A scope outside it answers exactly what a scope that never existed answers, so an agent cannot enumerate the store by probing for errors. Give each agent, or each task, its own token and its own slice. ⚠ One thing is still shared, and it is the reason this is not tenant isolation: X-Store-Snapshot and the freshness line that opens every body carry a store-wide entry-files= count, over scopes the caller cannot name. Two parties on one pod can each watch the other's count move. Stated in server/server.py, and repeated here rather than left to be found.

  • Attributed β€” a record of who wrote what, not a security boundary. cairn append requires --session (no default, no env fallback) and renders - YYYY-MM-DD: <text> [cairn: <identity>/<session>], so a swarm's memory carries which agent believed a thing and on which run. Read at its real width:

    • the <identity> is taken from the token on POST /bullets, and a body-supplied actor is never read β€” so on that route it cannot be forged;
    • the <session> is supplied by the caller. It is correlation data, not an identity claim: the server validates its shape and never its ownership, so one agent can name another's session β€” and a session id is printed in the recall of the entry it wrote to, readable by anyone who can read the scope (--ref <entry>, or --limit <n> for the whole scope; the default digest prints one body out of N, so it is not the way to enumerate them), so they are not secrets either;
    • put and create write your bytes verbatim, trailer included, and the server does not check it. Enforcing that was considered and declined (server/server.py).

    πŸ”΄ And there is no append-only credential to close the gap with. The verb set is closed at three β€” read, write, admin β€” with append deliberately absent, because all three write verbs mutate a file through one path and a fourth verb would be spelled rather than structural (internal/control). Any token that can append can also PUT. So treat a trailer as a cooperative record for debugging and recall, never as evidence of authorship.

  • Concurrency-safe, and safe to retry. PUT requires If-Match (428 without it; * refused), and create lands through a hard link, so EEXIST is decided by the kernel rather than by a check-then-write. Two agents racing a create cannot both win β€” the loser gets exit 9 (it already exists, do not retry), which is deliberately not the 8 a lost put update gets (re-sync, re-derive, re-apply): the server answers the same 412 for both and the remedies are opposites, so conflating them retries forever. An append is recognised by content hash, so a re-POST after a timeout is idempotent and reports duplicate rather than writing twice β€” the failure mode a retrying agent actually has.

  • Honest about staleness. Each host keeps a stamped read-through replica, so recall does not depend on the network. A read names which of four states produced it β€” live, cached (with age and revision), scope-empty, or πŸ”΄ store-unreachable, no cache, which exits 3 and must never be read as the third: scope-empty and an unreachable store both "print no entries" and one of them is a lie. ⚠ Exit 3 is therefore wider than that one state. A read whose cache exists but cannot be fully read β€” a scope directory or an entry file at mode 000 β€” exits 3 with index entry unreadable: under <root> (…) β€” the store was not fully read. πŸ”΄ The exit code, not the output, is what says whether the report is complete. πŸ”΄ WHAT YOU SEE DEPENDS ON THE VERB, AND THIS PARAGRAPH USED TO BE WRONG FOR TWO OF THE THREE. It said such a read "prints ⚠ cairn: cached β€” … and then exits 3", which sends a caller hunting for a banner that is never emitted. Measured on BOTH clients over a mode-000 scope directory and a mode-000 entry file β€” twelve runs, the two clients agreeing byte for byte in every one, each run naming ONE scope with --scope:

    verb exit stdout the cached banner
    recall 3 0 bytes not printed, on either stream
    search 3 0 bytes not printed, on either stream
    validate 3 0 bytes for the single scope those runs named β€” see below on stderr

    πŸ”΄ THE validate STDOUT CELL IS NOT A GENERAL CLAIM. validate prints its per-scope line INSIDE its loop, so every scope processed before the unreadable one has already emitted output by the time the raise happens; 0 bytes is true only when the failing scope is the FIRST one validated, which is what all twelve runs above measured because each named a single --scope. Re-measured with no --scope at all, one cache holding a readable scope of 9 entries plus an unreadable one, validate --no-sync, both clients byte-identical: the unreadable scope sorting after β†’ exit 3 with 2,030 bytes of stdout, first line cairn: <scope>: 9 of 9 entry file(s) parse, 0 malformed; sorting before β†’ exit 3 with 0 bytes. So the honest rule is that at exit 3 stdout is a PARTIAL report whose length depends on how far the walk got, and the code is what says it is partial β€” a supervisor told "3 β‡’ 0 bytes" either stops reading stdout or reads a non-empty stdout at 3 as corruption. ⚠ One shape neither measurement covered: search --all-scopes over a partially-unreadable store β€” that flag refuses --cache, so it was not built. Unmeasured, and not asserted either way.

    The banner column's cause is ordering rather than policy β€” the reads print their banner after the reader returns, so the raise pre-empts it, while validate prints its banner before it loads anything. For a read, the ABSENCE of the banner is the signal: a cached banner is not a promise of 0, and no banner at all is not a promise that nothing was attempted. Where a banner IS printed it says which of the four states the sync reached. Orthogonally a read names the scope's status, so a cached read can still report scope-absent (no such scope here β€” also what a scope your token cannot see looks like) or scope-unreadable. An agent that cannot tell "nothing is there" from "I could not look" will act on the difference.

Quickstart β€” two agents, one store

# Each agent host: point at the pod and its own token.
mkdir -p ~/.config/subsystem-store
cat > ~/.config/subsystem-store/env <<'EOF'
CAIRN_URL=https://store.example.invalid
CAIRN_TOKEN=<this agent's token>
EOF

# Agent A records a finding, attributed to its own session.
cairn append --scope alpha-index --ref widget-cfg \
  --text 'the retry budget is per-connection, not per-request' \
  --session "$AGENT_SESSION_ID"
# β†’ cairn: appended instance=personal scope=alpha-index ref=widget-cfg revision=<sha>
#   (`duplicate` instead of `appended` if that exact bullet was already there)

# Agent B, on another host, picks it up.
cairn sync                              # refresh the local replica
cairn recall --scope alpha-index        # the scope's digest, with A's bullet and its session id
cairn search 'retry budget'             # or find the hunk by text

# A third agent, whose token does not name alpha-index:
cairn recall --scope alpha-index        # `scope-absent`, exit 0 β€” byte-identical
                                        # to a scope that was never created

cairn doctor answers pod reachability, credential, cache stamp, counts, scope visibility and which store this host resolves β€” in one call, with --json for a supervising process.

Installing and building with nix

nix run   github:ZacxDev/cairn -- doctor       # the default client β€” the GO one β€” uninstalled
nix build github:ZacxDev/cairn#cairn-go        # the Go client, by name
nix build github:ZacxDev/cairn#cairn           # the PYTHON client β€” no longer the default
nix build github:ZacxDev/cairn#server-image    # the PYTHON pod image β€” built, published, NOT deployed
nix build github:ZacxDev/cairn#server-image-go # the GO pod image β€” this is what the cluster runs
nix build github:ZacxDev/cairn#cairn-ui        # the BROWSER surface β€” published, and DEPLOYED

Consumers pin this flake as an input. The version is the git revision β€” never written down by hand β€” so a built cairn cannot disagree with the code in it.

πŸ”΄ The default client is now the Go one β€” what changed for you

packages.default and apps.default were the Python client. They are now the Go one, landed by #50 β€” the anchor to check a pin against, because "now" cannot tell you whether the flip sits between the revision you are on and the one you are moving to. So nix run github:ZacxDev/cairn and nix profile install github:ZacxDev/cairn both execute cmd/cairn rather than the Python script.

Two behaviours change, both stated here rather than left to be discovered: cairn -verbs and cairn -exit-codes now answer β€” exit 0 with a table on stdout, where the Python client exits 2 with argparse's usage: for the same argv, which is a single-dash token succeeding where the default previously refused; and argparse's exact wording is gone from --help and from usage errors, so anything parsing that prose needs re-reading. The exit codes themselves are identical and gated. ⚠ Whether -verbs/-exit-codes become documented public surface or move behind a gate is a P8 decision that has not been taken β€” they are the Go client's internal ledgers, and today they answer.

Nothing was deleted, and cairn is the opt-out β€” in BOTH consumption modes. The Python client is still built and is still the oracle the Go one is measured against. From the CLI that is a #fragment:

nix build github:ZacxDev/cairn#cairn           # the Python client, explicitly
nix run   github:ZacxDev/cairn#cairn -- doctor

…but the consumer this flip actually lands on pins this flake as an input and reaches an attribute, where a #fragment is not a spelling you can use:

{
  inputs.cairn.url = "github:ZacxDev/cairn";   # or pin a rev/ref

  outputs = { self, nixpkgs, cairn, ... }:
    let system = "x86_64-linux"; in {
      # `cairn.packages.${system}.default` is now the GO client.
      # Take the PYTHON client by name instead:
      packages.${system}.my-cairn = cairn.packages.${system}.cairn;
      # …and its runnable form, if you re-export an app:
      apps.${system}.my-cairn = cairn.apps.${system}.cairn;
    };
}

Both spellings are gated rather than promised: checks.default-is-the-go-client pins the flip itself and both opt-out spellings, at evaluation time, building neither client. ⚠ It is described here in one sentence and in full beside the code it guards (flake.nix), which is the only description worth trusting β€” this one is hand-written and nothing asserts it. The record of why the flip was taken on an operator decision rather than on a green gate lives in tests/parity/README.md residual 7, and nowhere else on purpose. CHANGELOG.md indexes this and every later contract change by PR and sha, so a consumer can tell whether one sits between the revision they are pinned to and the one they are moving to.

πŸ”΄ The environment variables are now CAIRN_*

Every SUBSYSTEM_STORE_* variable has a CAIRN_* name. Both work. Within one source the new name wins: the old one is read only when the new one is unset or blank there. An old name that is present β€” including when the new one shadows it β€” prints one line per process on stderr naming its replacement. Nothing breaks on the day you upgrade, and nothing silently half-migrates.

set this instead of what it is
CAIRN_URL SUBSYSTEM_STORE_URL client: the pod's base URL
CAIRN_TOKEN SUBSYSTEM_STORE_TOKEN client: the bearer token Β· pod: the token-SET fallback when no token file is readable
CAIRN_CONFIG SUBSYSTEM_STORE_CONFIG client: where the config file lives
CAIRN_TOKEN_FILE SUBSYSTEM_STORE_TOKEN_FILE pod and cairn-ui: the token file
CAIRN_STORE_ROOT SUBSYSTEM_STORE_ROOT pod and cairn-ui: the store root
CAIRN_LISTEN_HOST SUBSYSTEM_STORE_HOST pod: the listen address
CAIRN_PORT SUBSYSTEM_STORE_PORT pod: the listen port
CAIRN_TRUSTED_PROXIES SUBSYSTEM_STORE_TRUSTED_PROXIES pod: the peer allowlist that makes CF-Connecting-IP readable
CAIRN_MAX_FAILURES SUBSYSTEM_STORE_MAX_FAILURES pod: failed auths before a lockout
CAIRN_FAILURE_WINDOW_S SUBSYSTEM_STORE_FAILURE_WINDOW_S pod: the window they must fall inside
CAIRN_LOCKOUT_S SUBSYSTEM_STORE_LOCKOUT_S pod: the lockout duration

⚠ Two rows are not the mechanical prefix swap, and copying the pattern instead of the table will break a pod. CAIRN_HOST was already taken β€” it is the human-readable machine label that appears in rendered output β€” so the pod's listen address is CAIRN_LISTEN_HOST. And CAIRN_ROOT would read as a sibling of the client-side CAIRN_MIRROR_ROOT when it is the pod's store root and not a client-side name at all, so the pod's store root is CAIRN_STORE_ROOT.

The config file's KEYS count too. ~/.config/subsystem-store/env and instances/<alias>.env accept either spelling with the same precedence, and a deprecated key there warns with a line that names the file rather than a $VAR, because that is where you have to go to change it.

πŸ”΄ BUT NEW-BEATS-OLD IS A RULE WITHIN ONE SOURCE, AND THE SOURCES COMPOSE THE OTHER WAY ROUND. READ THIS BEFORE MIGRATING A CONFIG FILE. For the default instance the whole environment is consulted first, and only if it yields nothing is the file read β€” so an old name exported beats a new name in the file:

~/.config/subsystem-store/env:   CAIRN_URL=https://new.example.invalid
environment:                     SUBSYSTEM_STORE_URL=https://old.example.invalid
β†’ the client uses old.example.invalid

Migrating the file alone therefore changes nothing while the old variable is still exported, and the deprecation line β€” which describes only its own source β€” will not tell you so. unset SUBSYSTEM_STORE_URL (and SUBSYSTEM_STORE_TOKEN) in the same change, or export the new names too. ⚠ For a non-default instance the environment is not consulted at all, so there the file is the only thing that decides.

πŸ”΄ The pod images set NO store variable, in either spelling β€” so in a container there is no image default for your env: to argue with. server/Dockerfile and packages.server-image/server-image-go used to bake SUBSYSTEM_STORE_ROOT=/data, SUBSYSTEM_STORE_PORT=8102 and SUBSYSTEM_STORE_TOKEN_FILE=/run/secrets/subsystem-store/token. Every one of those values was identical to the default the server already falls back to, so they configured nothing β€” while tripping the deprecation sweep at every pod start with three lines no manifest could clear, because the sweep reads the whole process environment and an image ENV is part of it. They are gone. What this means for you:

  • Either spelling works in a container, and neither is shadowed. Set CAIRN_STORE_ROOT or SUBSYSTEM_STORE_ROOT in your Deployment; whichever you set is what the pod reads. Set neither and it resolves /data, port 8102, token /run/secrets/subsystem-store/token β€” the same three values the image used to state.
  • Migrating your Deployment to CAIRN_* now silences the warnings. It did not before: the image's own ENV kept emitting them regardless of what your manifest said.
  • ⚠ docker inspect no longer documents the store root, port or token path. That is the accepted cost, and the startup line replaces two of the three, not all three. Both pods print listening on <host>:<port> store=<root> token-ids=… …, so the port and the store root are readable from the log of a running container. token-ids= is the credential FINGERPRINTS, not the file they came from β€” the token PATH appears in no startup line on either implementation. Where it is observable: the Go server's cairn-server -h, which prints (default "/run/secrets/subsystem-store/token"); the oracle's --help does not print its default at all. Both also emit the path to stderr in one case only β€” token file <path> absent; falling back to $CAIRN_TOKEN β€” which is a failure notice, not documentation. What keeps the three from drifting is tests/test_flake_image_matches_dockerfile.py, which pins them against both implementations' code defaults.

When the old names stop being read: when the Python client (packages.cairn) is retired, which is this arc's P8 milestone. Not a date β€” there is no semver here to hang one on (flake.nix sets version = self.shortRev), and a milestone is something you can check.

⚠ That table is the RENAME ledger, not every variable this project reads. Names with no older spelling are not in it and cannot be β€” CAIRN_UI_*, CAIRN_SUPABASE_REDIRECT_URL, and the open family CAIRN_REF_BASE_<SYSTEM> in the section below, whose suffix comes out of a store file and so could not be enumerated by any table.

πŸ”΄ AND THE LEDGER NOW HOLDS A SECOND RENAME, WHICH IS WHY THE BULLET ABOVE NO LONGER SAYS CAIRN_SUPABASE_* WHOLESALE. The pod-side JWT verifier's seven settings are CAIRN_OIDC_*:

now was
CAIRN_OIDC_JWKS_URL CAIRN_SUPABASE_JWKS_URL
CAIRN_OIDC_ISSUER CAIRN_SUPABASE_ISSUER
CAIRN_OIDC_AUDIENCE CAIRN_SUPABASE_AUDIENCE
CAIRN_OIDC_PROVIDER CAIRN_SUPABASE_PROVIDER
CAIRN_OIDC_REQUIRE_ROLE CAIRN_SUPABASE_REQUIRE_ROLE
CAIRN_OIDC_LEEWAY CAIRN_SUPABASE_LEEWAY
CAIRN_OIDC_MAX_AGE CAIRN_SUPABASE_MAX_AGE

Same rules as the table above: both spellings work, the new one wins, an old one that is set warns once per process naming its replacement, and the window closes when packages.cairn is retired (P8). The verifier never required a particular provider β€” it is an RFC 7519 verifier over a JWKS URL, with a required issuer and audience and a closed asymmetric algorithm set β€” so the old prefix asserted a dependency that was not there, and you can point it at Keycloak, Authentik, Dex, Zitadel, Okta, Auth0 or Supabase/GoTrue alike. The operator contract β€” every setting, the refusals, the algorithm set, two worked non-Supabase examples, and the limit (OIDC buys authentication only; grants are still yours) β€” is internal/identity/README.md Β§ Wiring any OIDC provider.

⚠ Two CAIRN_SUPABASE_* names are deliberately NOT renamed and one more is not a variable at all. CAIRN_SUPABASE_REDIRECT_URL configures cairn-ui's GoTrue PKCE sign-in flow and is honestly named for what it talks to; CAIRN_SUPABASE_JWT_SECRET and CAIRN_SUPABASE_JWT_SECRET_FILE are retired and setting either is a startup refusal, not an alias.

πŸ”΄ An entry can be TAGGED, and both readers narrow by tag

A new optional front-matter key, tags:. A sequence, like aliases:; every tag is folded the way a ref is (lowercased, everything outside [a-z0-9.-] to -, runs collapsed), deduped and sorted. An entry with no tags: behaves exactly as it did, and a tags: file loads on a reader that has never heard of the key, reporting no tags rather than refusing the file.

---
service: rollout-runbook
scope: alpha-notes
tags: [Marketing, project-xyz]     # folds to `marketing`, `project-xyz`
---

πŸ”΄ ONE THING CAN BREAK, AND IT IS NOT "NOTHING" β€” tags: USED TO BE AN IGNORED KEY. Before this change the loaders named no tags: at all, so any value there loaded and was discarded; it was the probe key the ignore-rule test used, precisely because nothing read it. It is read now, and the parser carries four refusals. πŸ”΄ WHICH OF THEM A FILE CAN ACTUALLY REACH IS A DIFFERENT QUESTION, AND THE TABLE ANSWERS THAT ONE β€” measured on both implementations over real files rather than reasoned from the code, and they agree sentence for sentence:

what you may already have written what it does now
tags: marketing β€” a bare scalar REFUSED: `tags:` must be a list, not a bare string β€” write `tags: [<name>]`
tags: ["!!!"] β€” a flow item that folds to nothing REFUSED: tag '!!!' normalizes to the empty string
tags: [" "] β€” a blank flow item REFUSED, same sentence: whitespace folds away too
tags: {a: b} or tags: {} β€” inline braces REFUSED, but by the first row's sentence and not by the "got <type>" one: front matter holds only strings and lists of strings, so {a: b} parses as the string {a: b} and is a bare scalar
tags: with an indented a: b under it β€” a block mapping NOT refused. It parses as the empty string, which the or () rule makes an ABSENT key, so the entry loads carrying no tags
tags: with a - item that is blank β€” the block list NOT refused. The block-list scanner drops an empty item before the tag loop ever sees it, so tags: followed by a bare - and then - alpha loads carrying alpha. Only the flow spelling of a blank item reaches the refusal above

⚠ So the fourth refusal β€” `tags:` must be a list, got <type> β€” is unreachable from a file, and that is stated rather than left to be discovered. It guards the PROGRAMMATIC loader (from_mapping / EntryFromMapping), which is also the writer's validate pass, where a caller really can hand it a dict.

A refused entry is MALFORMED, and malformed is wider than "not indexed". It leaves the index, so --ref, --search and every rendered listing lose it β€” it is named in the report's malformed block rather than vanishing silently β€” and it also becomes unwritable: append and put resolve their target through that same index, so they answer 404 ref-unknown for a file that is sitting right there. Its bytes are still shipped by /snapshot, which walks the store directly and builds no index, so a synced cache carries the file and each client classifies it malformed for itself.

πŸ”΄ How likely that is, measured rather than guessed β€” and it is a measurement, not a guarantee. Across three stores on one machine β€” two live client caches and one frozen pre-cutover mirror β€” 0 of 591 entries carried a tags: or tag: front-matter key at this anchor. The reader doing the counting was checked against a positive control in the same run: it found 5 front-matter keys on a sample entry, so a zero from it is a zero it could have contradicted. That says nothing about your store. The check is grep -rl '^tags:' <your cache root>; the remedy is to make the value a list.

πŸ”΄ The vocabulary is CLOSED on the WRITE path and OPEN to every READER, and the asymmetry is the design rather than a gap. The declared set is exactly three terms β€” infra, product, tooling β€” and both servers refuse a PUT whose front matter carries anything else: 422 with X-Store-Status: entry-shape and a body naming the three. Nothing lands.

The axis is the technical domain of the work. It is deliberately not "who the work is for": a fourth term client-work shipped for one round and was removed, because infra / product / tooling answer what kind of work while client-work answers whose, and a closed set mixing two axes under a one-tag-per-entry rule makes both unassertable β€” the same objection this project already records against folding the category axis into kind:. The "whose" question is answered by the scope name, which every entry already carries.

The READER accepts any folded tag, and it has to. A vocabulary refusal in the loader would make the entry malformed, and the paragraph above is what that costs: out of the index, out of --ref, out of --search and unwritable, because the write routes resolve their target through that index. Closing the vocabulary in the reader would therefore take every entry that already carries an off-vocabulary tag and make it unreadable and unrepairable in the same stroke β€” a store-wide outage caused by the guard rather than by the data. So --tag/?tag= will name any token you like, an entry written before the closure keeps loading and keeps serving, and the rendered non-finding still says a typo looks exactly like an honest zero.

Widening or narrowing the set is a change in SIX files β€” two declarations and four expectations β€” and the count is measured rather than counted by eye. Derived by adding a fourth term to both declarations and reading what went red:

# file what is in it which gate reds
1 internal/write/tagvocab.go tagVocabulary β€” the deployed Go pod's declaration β€” (this is the change)
2 lib/entry_shape.py TAG_VOCABULARY β€” the Python oracle's β€” (this is the change)
3 internal/write/tagvocab_test.go two literals: the refusal sentence and the term list go test ./internal/write/
4 internal/api/tagvocab_test.go the 422 body as served, including its unprocessable: prefix go test ./internal/api/
5 tests/test_tag_vocabulary.py two literals: DECLARED and DECLARED_REFUSAL pytest
6 tests/conformance/golden/put-*-tag-*-the-vocabulary.json the recorded refusal bytes β€” regenerate with python3 tests/conformance/suite.py generate, never hand-edit (a golden's body is checked against its own recorded digest) three gates: tests/conformance/run_go.sh, python3 tests/conformance/suite.py run, and pytest tests/test_conformance_suite.py

So: eight hand-typed literals across four test files, plus three goldens. Every one is a literal on purpose β€” a test that derived its expectation from the declaration would assert x == x and stay green through exactly this edit.

⚠ And two more files carry the terms as PROSE, which no gate covers: this README's list above, and CHANGELOG.md's row. Those are the ones that go stale silently, so edit them in the same commit.

⚠ The count above says "six files", and the earlier draft of this paragraph said "four places" β€” wrong in the direction that makes the change look cheap. That is the identical mistake lib/entry_shape.py already records about its own per-host-cache sentence ("THIS COMMENT SAID 'a four-place change' AND THAT WAS WRONG BY 2Γ—"), two files from where the undercount was written. Re-derive rather than trusting the number:

# add a term to both declarations, then read what fails
go test ./... ; python3 -m pytest tests -q

tests/test_tag_vocabulary.py is the cheap red that says which side moved; tests/conformance/ is the one that compares the two servers' refusal bytes over the wire.

It is deliberately not the kind: enum, which stays its own closed four-value set (service/process/org/doc). kind: says what SHAPE of thing the entry describes and the tag axis says which technical domain it belongs to; an entry carries one value from each, neither set refines the other, and putting two axes in one closed set makes both unassertable.

⚠ Which surfaces the gate actually covers, stated rather than left to be assumed. Both servers' PUT routes are gated β€” that is every path by which an entry's front matter can be written through cairn, including cairn put and cairn create, because both clients send the body and relay the server's answer rather than validating locally. POST …/bullets (cairn append) is not gated and must not be: it never touches front matter, and gating it would refuse appends to entries that predate the closure. The browser surface has no entry-write route at all β€” internal/ui declares /share, /unshare, the session pair, /invite, /invite/revoke and the OAuth pair, and nothing that creates, replaces or appends to an entry β€” so "the browser is gated" is true only vacuously, and no browser-side check was built.

The five things worth knowing if you script this:

  • A new read filter, on both clients and both read routes. cairn recall --tag <name> and cairn search … --tag <name>; over HTTP, GET /api/v1/recall/<scope>?tag=… and GET /api/v1/search/<scope>?q=…&tag=…. It composes with everything β€” --ref, --ref-to, --all-scopes, --mode, --page.
  • πŸ”΄ It takes ONE tag. --tag is scalar and last-wins, like every other value-bearing flag on these clients, and ?tag= is last-wins like every other parameter on these routes β€” so --tag a --tag b and ?tag=a&tag=b both mean b, exactly as ?limit=1&limit=2 means 2. There is no way to ask for two categories at once, and that is deliberate: a repeatable --tag with AND semantics was built and then dropped, because nobody had asked for the repetition and the choice between AND and OR had no author. Ask again if you want it β€” with the semantics you want named.
  • An operand that folds to nothing is a 400, not an empty result. --tag '' and --tag '!!' are refused by a message naming the fold, on both clients (exit 2) and both routes. That is the opposite of ?ref=, which narrows on an empty value: a ref names an ENTRY, so "nothing is called that" is a fact worth reporting, while a tag that folds away names no category at all and answering "no entry carries this" would hide the cause.
  • A new X-Store-Status value: tag-absent, answered 200 and exit 0. It is a non-finding and not an error, reached two ways β€” nothing in the scope carries the tag, or --ref/?ref= named an entry that does not while others do. The rendered tag: line carries both counts (N of M entries … carry it), so a filtered index cannot be mistaken for a whole one. No exit code changed: cairn -exit-codes prints the same table it did.
  • The query side folds the same way the file side does, so --tag Marketing finds an entry written tags: [marketing] and vice versa. A tag you can write is one you can ask for.

On the browse pages the entry and scope pages render an entry's tags, each a link to /scopes?tag=<name> β€” a query parameter on the scope list, never a path segment. (The scope list was the root until the root became a hub; /?tag= and /?q= still answer, with a 303 to /scopes carrying the query.) That listing shows every visible entry carrying the tag, across scopes, and reports how many entries it LOOKED at so an empty answer is legible.

πŸ”΄ And ?q= and ?tag= COMPOSE on the scope list, the same way they compose on the pod. /scopes?q=lease&tag=marketing is one card: a search over the entries carrying marketing and nothing else, narrowed by the same report.Search call GET /api/v1/search/<scope>?q=…&tag=… makes. ⚠ This changed what a URL that already worked answers β€” the page used to render two independent cards, a search over everything beside a listing of everything tagged, neither narrowing the other. If you link to a two-parameter browse URL, it now returns the intersection. The card's summary names both operands and reports how many entries the tag left out, so a composed zero says why it is zero; the search box carries the tag along in a hidden control, so refining the words keeps the filter and emptying the box returns to the plain tag listing.

⚠ The recall report gains a tags: a, b line under a printed entry's header, beside the existing refs: line, and a tag: … header line only when the filter was sent β€” so no output moves for a store with no tags in it.

πŸ”΄ An entry's refs are refs:, and both readers answer "what references this?"

The front-matter key is refs:. tasks: and task: are accepted spellings and stay accepted β€” nothing warns, nothing is rewritten, and an entry carrying either loads with exactly the refs it always did. refs: was chosen because the key now carries repos, PRs, docs and dashboards rather than only work-tracker items, so tasks: named a subset of what it holds. Where both appear on one entry, refs: is the one that is read.

Nothing you have to do. The three things worth knowing if you script this:

  • A new read filter, on both clients and both read routes. cairn recall --ref-to <system>:<id> and cairn search … --ref-to <system>:<id>; over HTTP, GET /api/v1/recall/<scope>?ref-to=… and GET /api/v1/search/<scope>?q=…&ref-to=…. It is spelled ref-to and not ref because ?ref= already means "address one entry" on that route. A malformed operand is a 400 naming the parameter, from the same parser an entry's own refs: item goes through. ⚠ # must be percent-encoded as %23 in a URL, or the operand arrives truncated at the fragment delimiter.
  • A new X-Store-Status value: ref-to-absent, answered 200 with a body that says what was read. It is a non-finding and not an error, and it is reached two ways β€” no entry in the scope carries the ref, or --ref/?ref= named an entry that does not carry it while others do. The rendered ref-to: line carries both counts (N of M entries … reference it), so a filtered index cannot be mistaken for a whole one.
  • CAIRN_REF_BASE_<SYSTEM> turns a ref into a link, with no network call. github: and clickup: resolve from built-in public hosts. Every other system resolves through a variable you set β€” the system half upper-cased with - folded to _, so refs: [tracker:1234] links only when CAIRN_REF_BASE_TRACKER=https://tracker.example.invalid/t is exported to the process that renders it, giving …/t/1234. Set nothing and the ref renders as plain text, exactly as it did before. πŸ”΄ It is read by whatever process RENDERS, which for the browse pages is cairn-ui and not your shell; and a base whose scheme is not http(s) β€” javascript: included β€” has its link withheld while the ref still renders as plain text, the same thing an unregistered system does. See internal/store/refurl.go for why that is the safe direction rather than a limitation.

On the browse pages the entry section that used to be headed Tasks is now Refs, and each ref is a link when it resolves. The text renderer now agrees: a printed entry's body carries refs: … and its index row carries πŸ”— N ref(s). ⚠ That sentence read "the text renderer's label is still tasks:" until an operator ruled on it; both rendered words moved together with a regenerated reader fixture and conformance corpus, because the recall report's bytes are compared byte-for-byte against the other implementation's by two gates. What did NOT move: the JSON payload's key is still "tasks" β€” --json consumers are unaffected, deliberately, because renaming a machine-readable key breaks a presence check silently where renaming a human-read label breaks nobody.

The client β€” cairn

you want run
refresh the local cache from the pod cairn sync
a scope's digest cairn recall --scope X (--ref R, --list, --limit N, --page N)
find a hunk by text cairn search 'query' (--all-scopes)
what the cache actually holds β€” the ENTRY files, never a scope's README.md cairn ls-entries
which sessions wrote attributed bullets in a scope, with coverage counts (Go client only) cairn sessions --scope X (or --repo P)
which registered arcs touched a scope, declared or inferred (Go client only; asks the pod) cairn arcs --scope X (or --repo P)
one registered arc: status, tooling coverage, members (Go client only; asks the pod) cairn arc-show --slug S (home from --scope/--repo)
register or update an arc from a JSON payload (Go client only) cairn arc-register --slug S --from F (home from --scope/--repo)
the post-write check: parse, dropped lines, marker reachability cairn validate
one call of diagnostics cairn doctor (--json, --no-sync)
append one dated, attributed bullet cairn append --scope S --ref R --text '…' --session ID
replace a whole entry behind If-Match cairn put --scope S --ref R --file F
create a new entry (refuses to overwrite) cairn create --scope S --ref R --file F
the scope→instance table, graded cairn routes (--check)

cairn sessions β€” who wrote here, and what that answer cannot see

cairn sessions --scope X (and the pod's GET /api/v1/sessions/X) lists every session id that signed a surviving ## Nuance / work-history bullet with a [cairn: <actor>/<session>] trailer, per session: the actor(s) as written, the bullet count and the date span. Both surfaces call ONE renderer, so the report bytes are identical; only the line above them differs (the CLI's cache banner, the pod's snapshot stamp). It exists on the Go client and pod only β€” the Python oracle has no twin. Every answer prints its coverage, because most bullets carry no trailer: attributed: K of N (the Nβˆ’K are NOT listed), what was scanned or rejected, that reads are not recorded, and that trailers are self-reported (put/create write bytes verbatim, and the session id is the writer's own word). scope-absent, scope-empty, no-attributed-writes and scope-unreadable are four different statements and render differently; a scope you may not read answers exactly like one that does not exist. Exit codes are the read set below β€” 0 for every answer, 3 when nothing in the scope could be scanned.

cairn arcs, cairn arc-show, cairn arc-register β€” the arc registry

An arc is one effort tracked by one handoff doc in one repo, keyed by (home scope, slug). Arcs are REGISTERED β€” pushed by the operator tooling at each handoff β€” never derived, and they live in an append-only journal the pod keeps outside the store tree: cairn-server -arc-journal <file> (or $CAIRN_ARC_JOURNAL), with no default. Unset, every arc route answers registrations-unconfigured, the designed off state; and the pod refuses to start if the symlink-resolved path is inside -store, because every directory at the store root is enumerated as a scope. Registrations are kept forever (no compaction in this phase); the latest valid record per arc is what is shown, and a torn final line or an unreadable one is skipped without poisoning the rest.

cairn arc-register --repo . --slug my-topic --from arc.json   # PUT  /api/v1/arc/<home>/my-topic
cairn arc-show     --repo . --slug my-topic                   # GET  /api/v1/arc/<home>/my-topic
cairn arcs         --scope alpha-notes                        # GET  /api/v1/arcs/alpha-notes

The payload (schema: 1) carries status (open | closed; omit it and it is unknown, which is never shown or counted as open), closing_kind, declared_scopes (the home is always declared), writers_measured/readers_measured, commits_total/commits_unstamped, reported_at and members (session, role ∈ originated | earliest-stamped | wrote | resumed, first_seen). Unknown fields are refused; a member whose session id could never match a write trailer is not stored and is counted back as unjoinable=N. The pod stamps registered_by (the credential's identity) and registered_at (its clock). A push that did not measure a leg keeps the members an earlier push measured for it, marked carried. Registering needs the write verb on the home scope and on every declared scope (a scope you may not write answers like one that does not exist); the bare legacy row may read arcs and may not register one.

Visibility: an arc exists for you only when its HOME scope is readable to you; its other scopes are listed only if readable too, and hidden ones are omitted, not counted. arcs --scope X lists an arc as declared when it names X, or inferred when one of its member sessions wrote an attributed bullet in X without the arc declaring it β€” a join over self-reported trailers, labelled as such. All three verbs print the pod's body verbatim (registrations are not in the cache) and use the existing exit codes: reads 0 for every answer, 3 if the pod did not answer; arc-register 0 registered or unchanged, 6 refused (including a pod with no journal), 7 did not happen.

cairn arcs --check β€” the arc orphan check

cairn arcs --check asks the pod (GET /api/v1/arcs/<scope>?check=1) whether the arc registry holds anything it should not. It is a flag on arcs, not a verb and not a doctor check: doctor is about this host and this credential, and its stdout is byte-compared against the Python oracle.

cairn arcs --check --repo .                       # arcs HOMED in this repo's scope
cairn arcs --check --all-scopes --scope alpha-notes   # every arc visible on that scope's pod

Findings (exit 9): a journal-damaged journal (a complete record that did not parse, or a torn tail β€” an arc registered only by that record is invisible everywhere); an arc whose home-scope-absent (its home is readable to you by grant and does not exist in the store); and a declared-scope-absent (the same for a declared scope). Not findings, printed as coverage instead: a member session with no attributed write in any scope you can read (members come from commits and transcripts, so this is the normal state), and a writing session in no arc (arcs are registered only from now on). Turning either into a finding would hold the check non-zero forever.

Exit codes are doctor's β€” 0 nothing wrong measured, 9 a finding, 10 could not look (no -arc-journal on the pod, a journal it cannot read, a pod that did not answer, or a pod too old to know ?check=1, which answers a listing that the client refuses to read as a pass). No new exit code exists for it: cairn -exit-codes is unchanged. Usage errors are still 2, and a scope no instance serves is still 11. Only arcs whose HOME you can read are checked, and a declared scope you cannot read is neither checked nor counted β€” the check answers a scope you may not read exactly as one that does not exist. ⚠ One pod per run: the scope picks the instance, and a multi-instance host is not fanned out.

tests/arcs/e2e.sh is the end-to-end check over both real binaries (--self-test proves it goes RED on a sabotaged build).

Exit codes, because the caller is usually a program

Read outcomes and write outcomes are disjoint, so a supervisor cannot read silence as success. 0 content was served (live, cached, or a genuinely empty scope) Β· 2 usage Β· 3 nothing was read β€” the store was unreachable with no cache, or the cache is there and could not be fully read (a mode-000 scope directory or entry file), which is a permissions problem on this host's cache rather than an outage to retry: fix the mode, or cairn sync to replace the cache. ⚠ This clause used to say the state is "a content defect cairn validate diagnoses", and that remedy is a dead end β€” measured on both clients, cairn validate over exactly that state exits 3 itself with 0 bytes on stdout, so it diagnoses nothing the failing read did not already say. The one thing it adds is the PATH, which every verb already names in the same sentence. Β· 4 sync did not refresh though a usable cache survived Β· 5 the archive was refused Β· 6 the store refused the write, change the request Β· 7 the write did not happen and a retry is the right response Β· 8 precondition failed, re-sync and re-apply Β· 9 create only, the entry already exists β€” do not retry, and do not conflate it with 8 Β· 11 no instance could be decided for this scope. doctor prints its own set on every run, where 10 means no problem measured, but at least one check could not look β€” which is not a clean bill of health.

Writes never degrade. There is no cache to answer from and no such thing as a stale write, so an unreachable store is a refused write at 7, never a queued or local one.

An index row now warns when ONE entry is a whole read's worth of tokens

An index row ends in badges, and there is a new one. An entry with more than 30 ## Nuance / work-history bullets gets ⚠ OVER 30 nuance β€” prune or split:

  example-entry  151 nuance   internal   πŸ”΄ 3 OPEN   ⚠ OVER 30 nuance β€” prune or split

Why it exists. A digest prints the index for every entry and one featured entry's body in full, so a single oversized entry sets the cost of the whole read. Measured on a real store: a digest came to ~50K tokens and 97.6% of it was one body, because that entry had 151 nuance bullets. The row already printed 151 nuance β€” a number with nothing to compare it against. The badge supplies the comparison by naming the bar.

Where 30 comes from. The bullet count is a proxy for what a featured render costs, so the threshold was set from the cost curve rather than picked: over one live store of 331 entries the median nuance section is 2.5 KB at 0–9 bullets, 22 KB at 20–29 and 47 KB at 30–39 β€” and the smallest body above 30 is 29 KB (~7.4K tokens) against 14.6 KB at a ceiling of 25. It fires on ~5% of rows, against 41% for the πŸ”΄ N OPEN badge already beside it; a badge that fires on everything trains its own bypass.

It is advisory. Nothing refuses, no exit code moved, no new status value, and an entry at or under the ceiling renders byte-identical to before. ⚠ If you PARSE an index row, note that a row may now carry one more badge, and that it is appended after πŸ”— N refs β€” the existing badges keep their order. The threshold is one named constant per renderer (NuanceBulletCeiling / NUANCE_BULLET_CEILING), pinned equal by tests/test_nuance_bullet_ceiling.py.

Every printed bullet now carries an opaque citation id

Inside a printed body, each line that opens a top-level bullet gains a trailing [cb:xxxxxxxx] β€” 8 lowercase hex:

    ## Pointers
      - apps/example/values.yaml β€” the chart values [cb:710f5a96]
    ## Nuance / work-history
      - 2000-01-02: OPEN: the retry budget is still unbounded. [cb:3cc70d82]
        a continuation line, which belongs to the bullet above it.

Both ids above are the real values for those exact bytes, and the second one is worth a second look: it is taken over the bullet's two lines, so the same opening line with no continuation under it answers a different id. A bullet is its wrapped prose, not its first line.

Why it exists. "Was this recalled bullet actually used?" had no answer. Every available proxy is a spelled one, walkable by rewording, and the obvious one is saturated by a mandate rather than by use β€” a downstream consumer's resume flow requires its report to echo what it recalled, so "a printed ref reappears later" fires on 451 of 454 runs (99.3%) and measures compliance. A token that exists nowhere else in a corpus turns that question into a match.

All four surfaced sections, not just the journal. ## What it is, ## Pointers, ## Nuance / work-history and ## Requirements are all annotated. Measured over a real 344-entry store, ## Pointers carries 1,589 of the 4,978 ids a full read would print β€” 32% of them β€” so annotating only the journal would have annotated the section where the signal is not. (That total is one reading of a live store and will not re-derive; the nuance section alone moved by 8 bullets between two measurements an hour apart.)

What the id is a function of. sha256 over the bullet's own lines joined with \n, first 8 hex. It is not scoped: two byte-identical bullets in different entries or scopes get the same id, deterministically. It moves when the bullet does β€” including on a trailing-whitespace change, a change of Unicode form, adding or removing an OPEN:/RESOLVED <sha>: marker, and the [cairn: actor/session] write trailer, which lives inside the opening line and makes an id partly a function of the writing session. Line terminators are normalised, so a CRLF file and an LF file name the same bullet with the same id.

The cost, measured rather than estimated. 14 bytes per annotated line. On a real local cache: the default digest read of one scope went 9,181 β†’ 9,279 bytes (+98 B, +1.07%, 7 tokens), and a full --limit 100 read of the same scope went 150,569 β†’ 152,935 (+2,366 B, +1.57%, 169 tokens). Across four scopes the full-read cost ranged +1.57% to +2.61%; the percentage is highest where bullets are shortest. --list is unchanged β€” the index prints no bodies, so it carries no tokens.

⚠ If you PARSE a printed body, a bullet's opening line now has a suffix. Lines that open no bullet are untouched, and not every bullet line carries an id: prose, blanks, an indented dash (a continuation), a dash inside a code fence, and text before a section's first bullet all print in full with no token. A missing id is never evidence that a bullet was not printed β€” the body is still emitted line for line, verbatim.

⚠ If you echo a recalled bullet VERBATIM back into a write, nothing changes: both pods strip the token (and the attribution, in any order) before taking a bullet's content hash, so a re-POST of an annotated line is still recognised as the same bullet and answers duplicate without writing. The token does count against the 2,000-character submitted-text cap.

πŸ”΄ But that is a claim about the HASH, not about the bytes on disk β€” and a token you quote INSIDE new prose is PERMANENT. The strip only reads a trailing token; a [cb:…] in mid-sentence is prose, and the write path stores the submitted text verbatim. Measured end to end against the pod's own functions:

READ 1   - 2000-01-02: the retry budget is still unbounded. [cb:7a54575e]
SUBMIT   building on [cb:7a54575e], the budget is now capped at 5 retries.
STORED   - 2000-01-05: building on [cb:7a54575e], the budget is now capped … [cairn: …]
READ 2   - 2000-01-05: building on [cb:7a54575e], … [cairn: …] [cb:58ff27d9]

Two consequences, and neither is hypothetical once anyone cites a bullet this way:

  1. A consumer parsing the annotation must take the LAST [cb:…] on a line β€” and must NOT read a token's PRESENCE as evidence that the line opens a bullet. Both halves are needed and the second was missing from the first draft of this rule. A line that already contains a token renders with two, and a first-match regex reads the quoted one β€” the id of a different bullet. But the mirror image is worse, because a last-match regex is still wrong on it: a quoted token can end a line that carries no id at all β€” prose, a wrapped bullet's continuation line, an indented dash, or a fenced sample β€” and all four of those shapes are enumerated above as unannotated. Measured twice, on two different bodies: a strict last-match scraper agrees with citation_ids_by_start_line on every annotated line and misattributes exactly one line per unannotated shape β€” four of them β€” each to a different bullet's id, which is precisely the harm the first half exists to prevent. πŸ”΄ The four is STRUCTURAL and the denominator is not: there are four unannotated shapes, so the count is four on a 9-line body and four on a 14-line one. Do not quote a ratio here; quote the shapes. Both renderers produce identical bytes here, so nothing downstream can detect the difference by comparing the two; the rule is the contract. πŸ”΄ Which is why the real advice is not to scrape at all. Derive ids from the body with CitationIDsByStartLine / citation_ids_by_start_line, which answers None for every unannotated shape by construction. A scraper has to re-decide what opens a bullet, and that decision is the one this repo already records a parser defect in.
  2. "A token that exists nowhere else in a corpus" stops holding after the first such write. That sentence is this feature's founding rationale β€” it is what makes "was this bullet used?" a match rather than a guess β€” and a citation written into stored prose puts the token in the corpus permanently, where a later search cannot tell a use from a quotation. This is the same saturation-by-echo that disqualified the previous proxy at 451/454 = 99.3%, arriving by a slower route. The feature is therefore weaker than the rationale above states, by an amount that grows with use, and nothing in either implementation bounds it.

A sync that is already current does not re-download

GET /api/v1/snapshot carries an ETag β€” "sha256:<64 hex>" over the uncompressed tar β€” and both clients store it beside the cache in .sync-etag and offer it back as If-None-Match on the next sync. A pod that has nothing new answers 304 with no body, and the client keeps the cache it already has:

$ cairn sync            # first run
cairn: live β€” fetched from https://store.example.invalid just now β€” 7 entries, snapshot seeded=…
$ cairn sync            # nothing changed since
cairn: live β€” already current at https://store.example.invalid β€” not modified, snapshot seeded=…

Read that line as its own thing. It is still the live state β€” the pod was reached and answered β€” and it is neither fetched … just now (bytes arrived) nor ⚠ … SERVED FROM CACHE (the pod could not be reached). Exit code, cache and subsequent reads are unchanged.

⚠ What it buys, exactly: the client's download and its extraction. The server still walks the store and builds the archive to compute the digest, so this is bandwidth and client CPU, not pod CPU. Nothing is written on a 304, so a later cached banner still dates the last download β€” it under-states freshness rather than over-stating it.

You need nothing for this. An older client against a new pod simply sends no validator; a new client against an older pod stores none.

One instance, or several

The client is configured by ~/.config/subsystem-store/env β€” CAIRN_URL and CAIRN_TOKEN, environment variables of the same name winning (and the old SUBSYSTEM_STORE_* spellings still accepted in both places, see the section The environment variables are now CAIRN_* above) β€” and that instance is called personal. A second instance is an additive file at ~/.config/subsystem-store/instances/<alias>.env with its own cache root (~/.cache/subsystem-store-<alias>) and its own sync stamp; the default instance's cache root does not move. Which instance a scope belongs to is decided by a table you supply β€” ~/.config/subsystem-store/routes.json, or $CAIRN_ROUTES, a flat JSON object of "scope": "alias" β€” because this program knows how to route and nothing about who routes where.

πŸ”΄ With more than one instance, an unregistered scope refuses at exit 11, naming the scope. It never falls back: a write that lands in a store nobody reads is found days later, if at all, and a refusal costs one error message. cairn routes --check grades the table in both directions β€” a scope with no entry, and an entry naming an alias that does not exist. ⚠ It does not fail on a table entry naming a scope that holds no entries: that prints as a note, because a snapshot ships entry files rather than directories, so a stale entry and one pre-registered before its first write look identical from here. Do not gate CI on --check expecting it to catch that case. With one instance configured, every read prints what it always did. Two exceptions at any instance count: the write verbs name their instance unconditionally β€” "where did that bullet go" is a question about a durable record, asked later, by someone who no longer has the terminal β€” and a table entry naming an alias this host has no config for still refuses. ⚠ If you parse cairn: appended scope=… ref=… positionally, that field is new β€” it sits between the status and scope=. The binding rules for anyone editing a routing path are in lib/README.md.

The server

route what
GET /healthz unauthenticated; body is exactly ok\n
GET /api/v1/recall/{scope} rendered digest (?mode=&ref=&limit=&page=)
GET /api/v1/search/{scope}?q=… search (?threshold=&max_hits=&context=&all_scopes=)
GET /api/v1/snapshot[?scope=] gzipped tar of the entry files β€” the sync payload
GET /api/v1/sessions/{scope} which sessions wrote attributed bullets, with coverage β€” Go pod only, same renderer as cairn sessions
GET /api/v1/arcs/{scope} registered arcs that touched the scope, declared or inferred; with ?check=1 (&all_scopes=1) the orphan check, X-Store-Exit on doctor's 0/9/10 β€” Go pod only
GET /api/v1/arc/{home}/{slug} one registered arc β€” Go pod only; arc-unregistered when it is not visible to you
PUT /api/v1/arc/{home}/{slug} register or update one arc (write verb on every declared scope; 409 registrations-unconfigured with no -arc-journal) β€” Go pod only
GET /api/v1/sources/{scope} the scope's declared code sources (git:<host>/<repo-path>[//<subpath>]@<branch>, first is primary), read from $CAIRN_SOURCE_JOURNAL β€” an environment variable with no flag, a file OUTSIDE the store root, read-only to the pod; sources-unconfigured when unset, 503 store-unreachable when it cannot be read β€” Go pod only
POST /api/v1/entry/{scope}/{ref}/bullets append ONE attributed bullet (the actor comes from the token, never the body)
PUT /api/v1/entry/{scope}/{ref} whole-file replace via If-Match, or create via If-None-Match: *

Auth is a token set β€” one row per line, <token> for legacy or <token> <identity> <scope>,<scope> for a mapped row β€” rotated by overlap and hot-reloadable with SIGHUP, so onboarding an agent does not restart the pod. The full contract, plus the operating runbook (seeding, byte-identity verification, rotation, rate limiting), is server/README.md.

The browser surface β€” cairn-ui

There is one. It carries the entries page, a browser sign-in with server-side revocable sessions, the share flow, and a GitHub sign-in through the operator's GoTrue.

⚠ THE ROUTE COUNT AND THE PHASE COUNT ARE DELIBERATELY NOT WRITTEN HERE, AND THE DELETION IS THE FIX. This paragraph said "three phases" and "Seven routes" and then enumerated them β€” and stayed at seven through the change that added three (POST /sign-in/github, GET /sign-in/github/callback, GET /static/app.css). cmd/cairn-ui's own package comment identifies this exact hazard, fixed it there, and left the literal seven standing in the file the fix was about. A number in prose beside a ledger the program DERIVES is a second spelling that only ever goes stale, so: cairn-ui prints its own count at startup, and ui.DeclaredRoutes() is the list. What is worth stating instead is the SHAPE, which does not move when a row is added: GET / is the entries page; the sign-in rows and the stylesheet answer without a session, because they are how you get one; the share rows and POST /sign-out are behind it; and /healthz is answered before the chain and is outside the ledger entirely.

πŸ”΄ cairn-ui is a SINGLE-REPLICA surface, for TWO reasons and not one. The session table is the loud one: each replica holds its own sessions, so a second replica without shared storage signs users out on a random fraction of requests. The quieter one is the control-plane cache β€” a share recorded on replica A is not served by B until B's cache refreshes, which is what the replica-honesty notice on every share page exists to say. Putting only the session file on shared storage buys two replicas that keep people signed in and silently disagree about who can see what. Multi-replica is a later arc, not a configuration.

⚠ AND THE CLAIM THAT NOTHING DEPLOYED IT IS RETRACTED. This paragraph read "And nothing deploys it: packages.ui-image builds an image, but nothing publishes that image and there is no manifest in this repository β€” it is built and run by hand." All three clauses are false. .github/workflows/publish-image.yml pushes the UI image to its own ghcr package and then proves it pullable with no credentials; a manifest deploys it from the operator's GitOps repository; and the surface is live on a public hostname. "No manifest in this repository" was never evidence about what is running β€” that is the inference this sweep exists to kill, and it stood in four places at once.

nix build github:ZacxDev/cairn#cairn-ui
./result/bin/cairn-ui -store <store root> -token-file <token file> -port 8103

⚠ A credential is required, and off-cluster you must say where it is. -token-file defaults to the pod's secret mount (/run/secrets/subsystem-store/token), so on a machine without one the binary exits 78 and serves nothing. Three ways to supply it, all measured: -token-file <path>; CAIRN_TOKEN_FILE=<path> with no flag; or -token-file= (explicitly empty) plus CAIRN_TOKEN=<row>, which is the env fallback the binary's own refusal names. Single-dash flags: this uses Go's stdlib flag, not the client's --long style. -h lists eight β€” -store (CAIRN_STORE_ROOT), -host (CAIRN_UI_HOST), -port (CAIRN_UI_PORT), -token-file (CAIRN_TOKEN_FILE), -session-file (CAIRN_UI_SESSION_FILE), -session-ttl (CAIRN_UI_SESSION_TTL), -control-journal (CAIRN_UI_CONTROL_JOURNAL) and -db-dsn (CAIRN_UI_DB_DSN) β€” and every default is env-resolved, so what -h prints depends on your environment. It reads the store from disk rather than over HTTP, and authenticates against the same token file as the pod.

πŸ”΄ -db-dsn MOVES THE SESSION TABLE AS WELL AS ARMING INVITATIONS, AND THE FIRST START WITH ONE SIGNS EVERY OPEN BROWSER OUT ONCE. It is a PostgreSQL connection string for this surface's mutable state, which is both tables internal/pgstore holds: sessions and invitations. With it set, -session-file is ignored β€” the binary says so on startup β€” and the invite flow works; without it, sessions stay on that file and every invite route tells its reader the deployment holds no invitation store. The connection is made, pinged and migrated at startup, under a 10 s bound, and a failure is a refusal to start (exit 78) rather than a surface that passes its health check and fails every request. ⚠ Prefer the environment variable to the flag: a DSN carries a password and a flag value is in argv. Nothing in this program echoes the value β€” not the startup line, not any refusal.

πŸ”΄ "Env-resolved" does NOT mean "the variable and the flag are interchangeable", and CAIRN_UI_CONTROL_JOURNAL is where that matters. An unset variable and an explicitly EMPTY one both mean "no control journal" β€” the shape a manifest that emits every variable with an empty default produces. A value that reduces to nothing but whitespace is refused at 78, naming the variable, rather than read as unset: read as unset this surface would come up on the token-file projection, which confers admin on nobody, so every scope page answers 404 and no share can be recorded while /healthz still answers 200. That is the same ruling the pod makes for its own CAIRN_CONTROL_JOURNAL. The flag path refuses the same value for a different reason β€” -control-journal ' ' has always failed its stat β€” so the two arrival paths agree for every value that does not name an existing file, which they did not between 1659663 and 68cf955.

⚠ THEY DO NOT AGREE ON A VALUE THAT DOES, AND THE FLAG IS THE LENIENT SIDE β€” MEASURED, NOT INFERRED. With a control journal in the working directory whose filename is literally three spaces, -control-journal ' ' serves, announcing sharing writable (control journal ), while CAIRN_UI_CONTROL_JOURNAL=' ' exits 78 naming the variable. The flag never meets the blank policy at all β€” it meets openAuthority's stat, which is a question about the filesystem and not about the spelling β€” so "the two paths refuse the same set" is true of every value anyone would type and false in general. It is left open, and the reason is a judgement rather than an argument from the mechanism: the fix is available and obvious β€” run the same blank policy over the flag's value too β€” and it was not taken because the direction is safe (the flag is the LENIENT side, so nothing is refused that should be served), because naming a file with nothing but whitespace is not a thing an operator does, and because a check on the flag would refuse a path the filesystem resolves. Nothing gates it; the agreement test's own docstring says which values it covers.

The other six variables above are resolved by internal/envalias, which reads a whitespace-only value as absent and silently takes the code default; tests/conformance/README.md measures what each one does with one.

πŸ”΄ Two backends are absent, for two different reasons, and conflating them is the misreading to avoid. Supabase is simply not wired yet and returns with the phase that builds a sign-in flow. The trusted-header backend is refused on principle and is not coming back here: a browser surface exists to be publicly reachable, and that backend lets anyone who can open a socket to it be any user at full authority. AuthBackends takes one parameter so neither can be passed; TestTheUIChainHasNoTrustedHeaderMember pins the exclusion.

Sharing a scope with somebody

GET /share lists the scopes your credential may administer; GET /share?scope=<id> is one scope's page. It answers three things: who has access to this, which of those you can take back, and a form to share it with somebody.

πŸ”΄ "Who has access to this" is computed from the authority, not from the list of shares. Access arrives two ways β€” a share, or membership of the project that owns the scope β€” so a page that listed only shares would under-report every project member, and in the direction that tells you your notes are more private than they are. The two lists are shown separately because only shares can be revoked: somebody who reaches a scope through project membership keeps it after every share is taken back, and the page says so.

⚠ You can only share with people and projects you already share a project with. That is deliberate β€” a picker listing every user would turn admin on one scope into a directory of everyone in the deployment β€” and it means reaching anybody else needs an invite. ⚠ The clause that stood here said an invite "does not exist yet", and it is corrected rather than deleted because it was true across three changes and is the sentence somebody would act on. The invite flow exists: GET/POST /invite, POST /invite/revoke and a public GET /join, over the invite store -db-dsn configures. It is absent on a deployment with no -db-dsn, where the invite pages say so and their writes answer 501 β€” so "does not exist yet" is now a statement about a CONFIGURATION rather than about this repository.

πŸ”΄ THE SHARE FLOW NEEDS -control-journal <path>, AND NOT ONLY TO WRITE. Without one the authority is the token file, which has no shares to record and confers admin on nobody β€” so no scope is administrable, GET /share renders "No scope is administrable by this credential", and a scope page answers 404 to every caller. The page says on every load that this deployment cannot record a share, so the cause is visible rather than discovered at a click; but read the limit at its real width β€” without a journal the share flow has nothing to show, not merely nothing to change.

⚠ An earlier draft of this paragraph claimed "the share pages still answer who can see this" without one. That is false, and it was measured false rather than argued: with a token-file authority, 0 scopes are administrable across every principal in the projection and the scope page is a 404. The retraction is kept because the sentence was plausible enough to survive writing it.

Every share page carries a notice about what this surface can and cannot promise: it is one replica's answer from a cached copy of the authority, another reader gains or loses access when their own cache refreshes rather than the instant you click, and revoking stops future syncs without recalling entries already copied onto somebody's machine. That notice is pinned whole by a test, so it cannot be quietly reworded into a stronger promise.

It is the only package here that links a third-party module (gomponents, for HTML), and the serving path is still stdlib-only β€” no package the pod or the CLI links reaches it. What replaced vendorHash = null, and how that is measured rather than asked for, is stated once in internal/depspolicy's package doc; how this page escapes entry text is stated once in internal/ui/README.md. Both are pointers on purpose: those claims have one home each and a correction belongs there.

Layout

path what
cairn the Python client CLI, and the ORACLE the Go one is measured against
lib/ the Python reader: cache resolution, recall rendering, scope/ref resolution, doctor
server/ the ORACLE pod: server.py, Dockerfile, seed.sh, verify-byte-identity.sh β€” no longer what runs
cmd/cairn-server, internal/api the Go port of the server β€” passes the corpus, and the deployed pod
cmd/cairn, internal/client the Go port of the CLIENT, and the default β€” diffed against the Python one by tests/parity/, which declares both its residuals and the rows that compare only the exit code
internal/report the ONE renderer, shared by the pod and the CLI
cmd/cairn-ui, internal/ui the BROWSER surface β€” pages, sign-in (credential form and GitHub through the operator's GoTrue), the share flow, gomponents; published, and DEPLOYED
internal/depspolicy the allowlist and import ban that replaced vendorHash = null β€” the serving path is still stdlib-only, and this is what measures it
tests/ the suites, plus leakscan.py, the HTTP conformance corpus, the server dual-run gate and the client parity gate
flake.nix both clients (default is the Go one, #cairn the Python one), the server image, the Go server, and the checks

How the agreement is measured, and why two of everything is alive

server/server.py is the oracle; cmd/cairn-server is a stdlib-only Go port of it, and it is what the cluster runs β€” this line read "deployed by nothing" until the cutover. Rewriting the server alone would have left two renderers in two languages that must agree byte-for-byte forever, with drift arriving as "a different order that reads as a stale cache" β€” no error, no missing entry. So both land on the same internal/report: one renderer, two consumers β€” the pod and the CLI. ⚠ The browser surface is not one of them. internal/ui imports internal/report nowhere; it reads internal/store and renders HTML through its own code, so the page is a second renderer producing a different medium, and nothing compares the two. That was a forecast ("a future UI") until cairn-ui shipped; it is now a measured exception, and whether the page should ever route through internal/report is open. ⚠ And one-renderer becomes a property only when the Python renderer is deleted at P8; until then it is a discipline, and these three instruments are what enforce it:

instrument what it compares read it
tests/conformance/ the served HTTP contract, as golden fixtures generated from the oracle, replayed against either server the corpus and its declared blind spots
tests/dualrun/ both servers over one store β€” every route, scope, entry and principal, and the snapshot's uncompressed tar byte for byte why gzip identity is unattainable, and the PAX-header divergences a tree comparison cannot see
tests/parity/ both clients, one pod, one cache root, identical argv, across every verb in the table above the declared residuals, the P8 retirement ledger, and which rows compare only the exit code

⚠ Counts are deliberately not quoted here. Every one of them went stale in this file at least once while the gates moved, and nothing asserted on them; the numbers live beside the gates that produce them, and the CI floors live in .github/workflows/ci.yml. Re-derive rather than trust β€” grep -c 'Case(' tests/parity/harness.py for the parity case count, and the compare= distribution beside it for how many rows compare output text rather than only the exit code.

⚠ And a green gate is not evidence until its controls have been watched to work. The parity gate's first full run reported 72 PASS / 0 FAIL while every request was refused and no cache was ever written β€” two clients failing identically compare equal. Both harnesses now refuse to vouch (exit 2, which is not "failed") unless a pre-flight succeeds, and both ship a --self-test that proves the differ can go red.

Development

pytest tests/                         # the Python suite
go vet ./... && go test ./...         # the Go port's own guards
tests/conformance/run_go.sh           # the corpus against the Go server
python3 tests/conformance/suite.py run  # …and against the oracle: must stay at 0 failures
python3 tests/parity/harness.py       # both CLIENTS over one cache root: the P2 gate
python3 tests/parity/harness.py --self-test  # prove the differ can go RED
python3 tests/reader_fixtures.py generate  # re-record the renderer's bytes FROM the oracle
python3 tests/leakscan.py             # the leak gate β€” runs in CI on every commit
python3 tests/leakscan.py --self-test # prove the gate is an instrument

This repository is public and was extracted from a private one: never commit hostnames, private IPs, real project/scope names, dated incident narration, or captured text β€” fixtures are synthetic. tests/leakscan.py runs in CI on every commit. The full rules agents work under live in AGENTS.md.

Several sessions and agents routinely work this repository at once, through linked worktrees of one clone, so changes are made in a worktree and never in the base clone β€” a commit onto the branch a peer left checked out is the silent failure, because git log afterwards shows exactly what you expect. claudedocs/working-in-parallel.md is the recipe and the reason behind each rule; .claude/hooks/base-clone-write-guard.py refuses the write rather than relying on anyone having read it. That hook fires only when the clone actually has linked worktrees, so a fresh clone never sees it, and it fails open by design β€” it is a guard against a routine mistake, not a security boundary.

Naming

The project is cairn. Identifiers that are not environment variables still read subsystem_store β€” the lib/ module names, ~/.config/subsystem-store/, /run/secrets/subsystem-store/, the subsystem-recall CLI alias. Those are accepted aliases, kept so existing deployments don't need a coordinated cutover. The environment variables have been renamed; see the migration note below.

About

Simple, scoped, sharable memory for AI agent swarms. Plain markdown entries over HTTP: per-token scope isolation, token-derived writer identity on every appended bullet, If-Match concurrency, and a read-through local cache that never reports an unreachable store as an empty one.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages