Simple, scoped, sharable memory for AI agent swarms.
A small hosted store that many agents, on many hosts, can read and write at
once without overwriting each other or seeing each other's scopes. Entries are
plain <scope>/<entry>.md files β terse pointers, gotchas, work history β so
what an agent wrote stays readable by a human in a text editor, and what a human
wrote is readable by an agent with no ingestion step.
It is a store, not a retrieval engine: no embeddings, no index build, no ranking model. A scope is a directory, a memory is a bullet in a file, and recall is that file rendered. If you want semantic search over a corpus, cairn is the wrong tool. If you want a shared notebook several agents can be trusted to write to concurrently, this is what it does.
Four properties you would otherwise have to build yourself on top of a shared bucket, a git repo or an NFS mount:
-
Scoped. A token carries a scope allowlist, and it gates every route except
/healthz. A scope outside it answers exactly what a scope that never existed answers, so an agent cannot enumerate the store by probing for errors. Give each agent, or each task, its own token and its own slice. β One thing is still shared, and it is the reason this is not tenant isolation:X-Store-Snapshotand the freshness line that opens every body carry a store-wideentry-files=count, over scopes the caller cannot name. Two parties on one pod can each watch the other's count move. Stated inserver/server.py, and repeated here rather than left to be found. -
Attributed β a record of who wrote what, not a security boundary.
cairn appendrequires--session(no default, no env fallback) and renders- YYYY-MM-DD: <text> [cairn: <identity>/<session>], so a swarm's memory carries which agent believed a thing and on which run. Read at its real width:- the
<identity>is taken from the token onPOST /bullets, and a body-suppliedactoris never read β so on that route it cannot be forged; - the
<session>is supplied by the caller. It is correlation data, not an identity claim: the server validates its shape and never its ownership, so one agent can name another's session β and a session id is printed in the recall of the entry it wrote to, readable by anyone who can read the scope (--ref <entry>, or--limit <n>for the whole scope; the default digest prints one body out of N, so it is not the way to enumerate them), so they are not secrets either; putandcreatewrite your bytes verbatim, trailer included, and the server does not check it. Enforcing that was considered and declined (server/server.py).
π΄ And there is no append-only credential to close the gap with. The verb set is closed at three β
read,write,adminβ withappenddeliberately absent, because all three write verbs mutate a file through one path and a fourth verb would be spelled rather than structural (internal/control). Any token that can append can alsoPUT. So treat a trailer as a cooperative record for debugging and recall, never as evidence of authorship. - the
-
Concurrency-safe, and safe to retry.
PUTrequiresIf-Match(428without it;*refused), andcreatelands through a hard link, soEEXISTis decided by the kernel rather than by a check-then-write. Two agents racing a create cannot both win β the loser gets exit9(it already exists, do not retry), which is deliberately not the8a lostputupdate gets (re-sync, re-derive, re-apply): the server answers the same412for both and the remedies are opposites, so conflating them retries forever. An append is recognised by content hash, so a re-POST after a timeout is idempotent and reportsduplicaterather than writing twice β the failure mode a retrying agent actually has. -
Honest about staleness. Each host keeps a stamped read-through replica, so recall does not depend on the network. A read names which of four states produced it β
live,cached(with age and revision),scope-empty, or π΄store-unreachable, no cache, which exits3and must never be read as the third:scope-emptyand an unreachable store both "print no entries" and one of them is a lie. β Exit3is therefore wider than that one state. A read whose cache exists but cannot be fully read β a scope directory or an entry file at mode000β exits3withindex entry unreadable: under <root> (β¦) β the store was not fully read. π΄ The exit code, not the output, is what says whether the report is complete. π΄ WHAT YOU SEE DEPENDS ON THE VERB, AND THIS PARAGRAPH USED TO BE WRONG FOR TWO OF THE THREE. It said such a read "printsβ cairn: cached β β¦and then exits3", which sends a caller hunting for a banner that is never emitted. Measured on BOTH clients over a mode-000scope directory and a mode-000entry file β twelve runs, the two clients agreeing byte for byte in every one, each run naming ONE scope with--scope:verb exit stdout the cachedbannerrecall30 bytes not printed, on either stream search30 bytes not printed, on either stream validate30 bytes for the single scope those runs named β see below on stderr π΄ THE
validateSTDOUT CELL IS NOT A GENERAL CLAIM.validateprints its per-scope line INSIDE its loop, so every scope processed before the unreadable one has already emitted output by the time the raise happens;0 bytesis true only when the failing scope is the FIRST one validated, which is what all twelve runs above measured because each named a single--scope. Re-measured with no--scopeat all, one cache holding a readable scope of 9 entries plus an unreadable one,validate --no-sync, both clients byte-identical: the unreadable scope sorting after β exit3with 2,030 bytes of stdout, first linecairn: <scope>: 9 of 9 entry file(s) parse, 0 malformed; sorting before β exit3with 0 bytes. So the honest rule is that at exit3stdout is a PARTIAL report whose length depends on how far the walk got, and the code is what says it is partial β a supervisor told "3 β 0 bytes" either stops reading stdout or reads a non-empty stdout at 3 as corruption. β One shape neither measurement covered:search --all-scopesover a partially-unreadable store β that flag refuses--cache, so it was not built. Unmeasured, and not asserted either way.The banner column's cause is ordering rather than policy β the reads print their banner after the reader returns, so the raise pre-empts it, while
validateprints its banner before it loads anything. For a read, the ABSENCE of the banner is the signal: acachedbanner is not a promise of0, and no banner at all is not a promise that nothing was attempted. Where a banner IS printed it says which of the four states the sync reached. Orthogonally a read names the scope's status, so acachedread can still reportscope-absent(no such scope here β also what a scope your token cannot see looks like) orscope-unreadable. An agent that cannot tell "nothing is there" from "I could not look" will act on the difference.
# Each agent host: point at the pod and its own token.
mkdir -p ~/.config/subsystem-store
cat > ~/.config/subsystem-store/env <<'EOF'
CAIRN_URL=https://store.example.invalid
CAIRN_TOKEN=<this agent's token>
EOF
# Agent A records a finding, attributed to its own session.
cairn append --scope alpha-index --ref widget-cfg \
--text 'the retry budget is per-connection, not per-request' \
--session "$AGENT_SESSION_ID"
# β cairn: appended instance=personal scope=alpha-index ref=widget-cfg revision=<sha>
# (`duplicate` instead of `appended` if that exact bullet was already there)
# Agent B, on another host, picks it up.
cairn sync # refresh the local replica
cairn recall --scope alpha-index # the scope's digest, with A's bullet and its session id
cairn search 'retry budget' # or find the hunk by text
# A third agent, whose token does not name alpha-index:
cairn recall --scope alpha-index # `scope-absent`, exit 0 β byte-identical
# to a scope that was never createdcairn doctor answers pod reachability, credential, cache stamp, counts, scope
visibility and which store this host resolves β in one call, with --json for a
supervising process.
nix run github:ZacxDev/cairn -- doctor # the default client β the GO one β uninstalled
nix build github:ZacxDev/cairn#cairn-go # the Go client, by name
nix build github:ZacxDev/cairn#cairn # the PYTHON client β no longer the default
nix build github:ZacxDev/cairn#server-image # the PYTHON pod image β built, published, NOT deployed
nix build github:ZacxDev/cairn#server-image-go # the GO pod image β this is what the cluster runs
nix build github:ZacxDev/cairn#cairn-ui # the BROWSER surface β published, and DEPLOYEDConsumers pin this flake as an input. The version is the git revision β
never written down by hand β so a built cairn cannot disagree with the code
in it.
packages.default and apps.default were the Python client. They are now the
Go one, landed by #50 β the anchor
to check a pin against, because "now" cannot tell you whether the flip sits between the
revision you are on and the one you are moving to. So nix run github:ZacxDev/cairn
and nix profile install github:ZacxDev/cairn both execute cmd/cairn rather than the
Python script.
Two behaviours change, both stated here rather than left to be discovered:
cairn -verbs and cairn -exit-codes now answer β exit 0 with a table on
stdout, where the Python client exits 2 with argparse's usage: for the same
argv, which is a single-dash token succeeding where the default previously
refused; and argparse's exact wording is gone from --help and from usage
errors, so anything parsing that prose needs re-reading. The exit codes
themselves are identical and gated. β Whether -verbs/-exit-codes become
documented public surface or move behind a gate is a P8 decision that has not
been taken β they are the Go client's internal ledgers, and today they answer.
Nothing was deleted, and cairn is the opt-out β in BOTH consumption modes. The
Python client is still built and is still the oracle the Go one is measured
against. From the CLI that is a #fragment:
nix build github:ZacxDev/cairn#cairn # the Python client, explicitly
nix run github:ZacxDev/cairn#cairn -- doctorβ¦but the consumer this flip actually lands on pins this flake as an input and
reaches an attribute, where a #fragment is not a spelling you can use:
{
inputs.cairn.url = "github:ZacxDev/cairn"; # or pin a rev/ref
outputs = { self, nixpkgs, cairn, ... }:
let system = "x86_64-linux"; in {
# `cairn.packages.${system}.default` is now the GO client.
# Take the PYTHON client by name instead:
packages.${system}.my-cairn = cairn.packages.${system}.cairn;
# β¦and its runnable form, if you re-export an app:
apps.${system}.my-cairn = cairn.apps.${system}.cairn;
};
}Both spellings are gated rather than promised: checks.default-is-the-go-client pins
the flip itself and both opt-out spellings, at evaluation time, building neither client.
β It is described here in one sentence and in full beside the code it guards
(flake.nix), which is the only description worth trusting β this one is hand-written
and nothing asserts it. The record of why the flip was taken on an operator decision
rather than on a green gate lives in tests/parity/README.md
residual 7, and nowhere else on purpose. CHANGELOG.md indexes this
and every later contract change by PR and sha, so a consumer can tell whether one sits
between the revision they are pinned to and the one they are moving to.
Every SUBSYSTEM_STORE_* variable has a CAIRN_* name. Both work. Within one
source the new name wins: the old one is read only when the new one is unset or blank
there. An old name that is present β including when the new one shadows it β prints
one line per process on stderr naming its replacement. Nothing breaks on the day you
upgrade, and nothing silently half-migrates.
| set this | instead of | what it is |
|---|---|---|
CAIRN_URL |
SUBSYSTEM_STORE_URL |
client: the pod's base URL |
CAIRN_TOKEN |
SUBSYSTEM_STORE_TOKEN |
client: the bearer token Β· pod: the token-SET fallback when no token file is readable |
CAIRN_CONFIG |
SUBSYSTEM_STORE_CONFIG |
client: where the config file lives |
CAIRN_TOKEN_FILE |
SUBSYSTEM_STORE_TOKEN_FILE |
pod and cairn-ui: the token file |
CAIRN_STORE_ROOT |
SUBSYSTEM_STORE_ROOT |
pod and cairn-ui: the store root |
CAIRN_LISTEN_HOST |
SUBSYSTEM_STORE_HOST |
pod: the listen address |
CAIRN_PORT |
SUBSYSTEM_STORE_PORT |
pod: the listen port |
CAIRN_TRUSTED_PROXIES |
SUBSYSTEM_STORE_TRUSTED_PROXIES |
pod: the peer allowlist that makes CF-Connecting-IP readable |
CAIRN_MAX_FAILURES |
SUBSYSTEM_STORE_MAX_FAILURES |
pod: failed auths before a lockout |
CAIRN_FAILURE_WINDOW_S |
SUBSYSTEM_STORE_FAILURE_WINDOW_S |
pod: the window they must fall inside |
CAIRN_LOCKOUT_S |
SUBSYSTEM_STORE_LOCKOUT_S |
pod: the lockout duration |
β Two rows are not the mechanical prefix swap, and copying the pattern instead of the
table will break a pod. CAIRN_HOST was already taken β it is the human-readable
machine label that appears in rendered output β so the pod's listen address is
CAIRN_LISTEN_HOST. And CAIRN_ROOT would read as a sibling of the client-side
CAIRN_MIRROR_ROOT when it is the pod's store root and not a client-side name at all,
so the pod's store root is CAIRN_STORE_ROOT.
The config file's KEYS count too. ~/.config/subsystem-store/env and
instances/<alias>.env accept either spelling with the same precedence, and a deprecated
key there warns with a line that names the file rather than a $VAR, because that is
where you have to go to change it.
π΄ BUT NEW-BEATS-OLD IS A RULE WITHIN ONE SOURCE, AND THE SOURCES COMPOSE THE OTHER WAY ROUND. READ THIS BEFORE MIGRATING A CONFIG FILE. For the default instance the whole environment is consulted first, and only if it yields nothing is the file read β so an old name exported beats a new name in the file:
~/.config/subsystem-store/env: CAIRN_URL=https://new.example.invalid
environment: SUBSYSTEM_STORE_URL=https://old.example.invalid
β the client uses old.example.invalid
Migrating the file alone therefore changes nothing while the old variable is still
exported, and the deprecation line β which describes only its own source β will not tell
you so. unset SUBSYSTEM_STORE_URL (and SUBSYSTEM_STORE_TOKEN) in the same change, or
export the new names too. β For a non-default instance the environment is not
consulted at all, so there the file is the only thing that decides.
π΄ The pod images set NO store variable, in either spelling β so in a container there
is no image default for your env: to argue with. server/Dockerfile and
packages.server-image/server-image-go used to bake SUBSYSTEM_STORE_ROOT=/data,
SUBSYSTEM_STORE_PORT=8102 and
SUBSYSTEM_STORE_TOKEN_FILE=/run/secrets/subsystem-store/token. Every one of those
values was identical to the default the server already falls back to, so they configured
nothing β while tripping the deprecation sweep at every pod start with three lines no
manifest could clear, because the sweep reads the whole process environment and an
image ENV is part of it. They are gone. What this means for you:
- Either spelling works in a container, and neither is shadowed. Set
CAIRN_STORE_ROOTorSUBSYSTEM_STORE_ROOTin your Deployment; whichever you set is what the pod reads. Set neither and it resolves/data, port8102, token/run/secrets/subsystem-store/tokenβ the same three values the image used to state. - Migrating your Deployment to
CAIRN_*now silences the warnings. It did not before: the image's ownENVkept emitting them regardless of what your manifest said. - β
docker inspectno longer documents the store root, port or token path. That is the accepted cost, and the startup line replaces two of the three, not all three. Both pods printlistening on <host>:<port> store=<root> token-ids=β¦ β¦, so the port and the store root are readable from the log of a running container.token-ids=is the credential FINGERPRINTS, not the file they came from β the token PATH appears in no startup line on either implementation. Where it is observable: the Go server'scairn-server -h, which prints(default "/run/secrets/subsystem-store/token"); the oracle's--helpdoes not print its default at all. Both also emit the path to stderr in one case only βtoken file <path> absent; falling back to $CAIRN_TOKENβ which is a failure notice, not documentation. What keeps the three from drifting istests/test_flake_image_matches_dockerfile.py, which pins them against both implementations' code defaults.
When the old names stop being read: when the Python client (packages.cairn) is
retired, which is this arc's P8 milestone. Not a date β there is no semver here to hang
one on (flake.nix sets version = self.shortRev), and a milestone is something you can
check.
β That table is the RENAME ledger, not every variable this project reads. Names with no
older spelling are not in it and cannot be β CAIRN_UI_*, CAIRN_SUPABASE_REDIRECT_URL, and
the open family CAIRN_REF_BASE_<SYSTEM> in the section below, whose suffix comes out of a
store file and so could not be enumerated by any table.
π΄ AND THE LEDGER NOW HOLDS A SECOND RENAME, WHICH IS WHY THE BULLET ABOVE NO LONGER SAYS
CAIRN_SUPABASE_* WHOLESALE. The pod-side JWT verifier's seven settings are
CAIRN_OIDC_*:
| now | was |
|---|---|
CAIRN_OIDC_JWKS_URL |
CAIRN_SUPABASE_JWKS_URL |
CAIRN_OIDC_ISSUER |
CAIRN_SUPABASE_ISSUER |
CAIRN_OIDC_AUDIENCE |
CAIRN_SUPABASE_AUDIENCE |
CAIRN_OIDC_PROVIDER |
CAIRN_SUPABASE_PROVIDER |
CAIRN_OIDC_REQUIRE_ROLE |
CAIRN_SUPABASE_REQUIRE_ROLE |
CAIRN_OIDC_LEEWAY |
CAIRN_SUPABASE_LEEWAY |
CAIRN_OIDC_MAX_AGE |
CAIRN_SUPABASE_MAX_AGE |
Same rules as the table above: both spellings work, the new one wins, an old one that is
set warns once per process naming its replacement, and the window closes when
packages.cairn is retired (P8). The verifier never required a particular provider β it is
an RFC 7519 verifier over a JWKS URL, with a required issuer and audience and a closed
asymmetric algorithm set β so the old prefix asserted a dependency that was not there, and
you can point it at Keycloak, Authentik, Dex, Zitadel, Okta, Auth0 or Supabase/GoTrue
alike. The operator contract β every setting, the refusals, the algorithm set, two worked
non-Supabase examples, and the limit (OIDC buys authentication only; grants are still
yours) β is internal/identity/README.md Β§ Wiring any OIDC
provider.
β Two CAIRN_SUPABASE_* names are deliberately NOT renamed and one more is not a variable
at all. CAIRN_SUPABASE_REDIRECT_URL configures cairn-ui's GoTrue PKCE sign-in flow and
is honestly named for what it talks to; CAIRN_SUPABASE_JWT_SECRET and
CAIRN_SUPABASE_JWT_SECRET_FILE are retired and setting either is a startup refusal, not
an alias.
A new optional front-matter key, tags:. A sequence, like aliases:; every tag is folded
the way a ref is (lowercased, everything outside [a-z0-9.-] to -, runs collapsed), deduped
and sorted. An entry with no tags: behaves exactly as it did, and a tags: file loads on a
reader that has never heard of the key, reporting no tags rather than refusing the file.
---
service: rollout-runbook
scope: alpha-notes
tags: [Marketing, project-xyz] # folds to `marketing`, `project-xyz`
---π΄ ONE THING CAN BREAK, AND IT IS NOT "NOTHING" β tags: USED TO BE AN IGNORED KEY. Before
this change the loaders named no tags: at all, so any value there loaded and was discarded; it
was the probe key the ignore-rule test used, precisely because nothing read it. It is read now,
and the parser carries four refusals. π΄ WHICH OF THEM A FILE CAN ACTUALLY REACH IS A
DIFFERENT QUESTION, AND THE TABLE ANSWERS THAT ONE β measured on both implementations over real
files rather than reasoned from the code, and they agree sentence for sentence:
| what you may already have written | what it does now |
|---|---|
tags: marketing β a bare scalar |
REFUSED: `tags:` must be a list, not a bare string β write `tags: [<name>]` |
tags: ["!!!"] β a flow item that folds to nothing |
REFUSED: tag '!!!' normalizes to the empty string |
tags: [" "] β a blank flow item |
REFUSED, same sentence: whitespace folds away too |
tags: {a: b} or tags: {} β inline braces |
REFUSED, but by the first row's sentence and not by the "got <type>" one: front matter holds only strings and lists of strings, so {a: b} parses as the string {a: b} and is a bare scalar |
tags: with an indented a: b under it β a block mapping |
NOT refused. It parses as the empty string, which the or () rule makes an ABSENT key, so the entry loads carrying no tags |
tags: with a - item that is blank β the block list |
NOT refused. The block-list scanner drops an empty item before the tag loop ever sees it, so tags: followed by a bare - and then - alpha loads carrying alpha. Only the flow spelling of a blank item reaches the refusal above |
β So the fourth refusal β `tags:` must be a list, got <type> β is unreachable from a
file, and that is stated rather than left to be discovered. It guards the PROGRAMMATIC loader
(from_mapping / EntryFromMapping), which is also the writer's validate pass, where a caller
really can hand it a dict.
A refused entry is MALFORMED, and malformed is wider than "not indexed". It leaves the index,
so --ref, --search and every rendered listing lose it β it is named in the report's
malformed block rather than vanishing silently β and it also becomes unwritable: append and
put resolve their target through that same index, so they answer 404 ref-unknown for a file
that is sitting right there. Its bytes are still shipped by /snapshot, which walks the store
directly and builds no index, so a synced cache carries the file and each client classifies it
malformed for itself.
π΄ How likely that is, measured rather than guessed β and it is a measurement, not a guarantee.
Across three stores on one machine β two live client caches and one frozen pre-cutover mirror β
0 of 591 entries carried a tags: or tag: front-matter key at this anchor. The reader doing
the counting was checked against a positive control in the same run: it found 5 front-matter keys
on a sample entry, so a zero from it is a zero it could have contradicted. That says nothing about
your store. The check is grep -rl '^tags:' <your cache root>; the remedy is to make the value
a list.
π΄ The vocabulary is CLOSED on the WRITE path and OPEN to every READER, and the asymmetry is
the design rather than a gap. The declared set is exactly three terms β infra,
product, tooling β and both servers refuse a PUT whose front matter carries
anything else: 422 with X-Store-Status: entry-shape and a body naming the three. Nothing
lands.
The axis is the technical domain of the work. It is deliberately not "who the work is
for": a fourth term client-work shipped for one round and was removed, because infra /
product / tooling answer what kind of work while client-work answers whose, and a
closed set mixing two axes under a one-tag-per-entry rule makes both unassertable β the same
objection this project already records against folding the category axis into kind:. The
"whose" question is answered by the scope name, which every entry already carries.
The READER accepts any folded tag, and it has to. A vocabulary refusal in the loader would make
the entry malformed, and the paragraph above is what that costs: out of the index, out of
--ref, out of --search and unwritable, because the write routes resolve their target
through that index. Closing the vocabulary in the reader would therefore take every entry that
already carries an off-vocabulary tag and make it unreadable and unrepairable in the same stroke
β a store-wide outage caused by the guard rather than by the data. So --tag/?tag= will name
any token you like, an entry written before the closure keeps loading and keeps serving, and the
rendered non-finding still says a typo looks exactly like an honest zero.
Widening or narrowing the set is a change in SIX files β two declarations and four expectations β and the count is measured rather than counted by eye. Derived by adding a fourth term to both declarations and reading what went red:
| # | file | what is in it | which gate reds |
|---|---|---|---|
| 1 | internal/write/tagvocab.go |
tagVocabulary β the deployed Go pod's declaration |
β (this is the change) |
| 2 | lib/entry_shape.py |
TAG_VOCABULARY β the Python oracle's |
β (this is the change) |
| 3 | internal/write/tagvocab_test.go |
two literals: the refusal sentence and the term list | go test ./internal/write/ |
| 4 | internal/api/tagvocab_test.go |
the 422 body as served, including its unprocessable: prefix |
go test ./internal/api/ |
| 5 | tests/test_tag_vocabulary.py |
two literals: DECLARED and DECLARED_REFUSAL |
pytest |
| 6 | tests/conformance/golden/put-*-tag-*-the-vocabulary.json |
the recorded refusal bytes β regenerate with python3 tests/conformance/suite.py generate, never hand-edit (a golden's body is checked against its own recorded digest) |
three gates: tests/conformance/run_go.sh, python3 tests/conformance/suite.py run, and pytest tests/test_conformance_suite.py |
So: eight hand-typed literals across four test files, plus three goldens. Every one is a
literal on purpose β a test that derived its expectation from the declaration would assert
x == x and stay green through exactly this edit.
β And two more files carry the terms as PROSE, which no gate covers: this README's list
above, and CHANGELOG.md's row. Those are the ones that go stale silently, so edit them in the
same commit.
β The count above says "six files", and the earlier draft of this paragraph said "four
places" β wrong in the direction that makes the change look cheap. That is the identical
mistake lib/entry_shape.py already records about its own per-host-cache sentence ("THIS
COMMENT SAID 'a four-place change' AND THAT WAS WRONG BY 2Γ"), two files from where the
undercount was written. Re-derive rather than trusting the number:
# add a term to both declarations, then read what fails
go test ./... ; python3 -m pytest tests -qtests/test_tag_vocabulary.py is the cheap red that says which side moved;
tests/conformance/ is the one that compares the two servers' refusal bytes over the wire.
It is deliberately not the kind: enum, which stays its own closed four-value set
(service/process/org/doc). kind: says what SHAPE of thing the entry describes and the
tag axis says which technical domain it belongs to; an entry carries one value from each, neither
set refines the other, and putting two axes in one closed set makes both unassertable.
β Which surfaces the gate actually covers, stated rather than left to be assumed. Both
servers' PUT routes are gated β that is every path by which an entry's front matter can be
written through cairn, including cairn put and cairn create, because both clients send the
body and relay the server's answer rather than validating locally. POST β¦/bullets (cairn append) is not gated and must not be: it never touches front matter, and gating it would
refuse appends to entries that predate the closure. The browser surface has no entry-write
route at all β internal/ui declares /share, /unshare, the session pair, /invite,
/invite/revoke and the OAuth pair, and nothing that creates, replaces or appends to an entry
β so "the browser is gated" is true only vacuously, and no browser-side check was built.
The five things worth knowing if you script this:
- A new read filter, on both clients and both read routes.
cairn recall --tag <name>andcairn search β¦ --tag <name>; over HTTP,GET /api/v1/recall/<scope>?tag=β¦andGET /api/v1/search/<scope>?q=β¦&tag=β¦. It composes with everything β--ref,--ref-to,--all-scopes,--mode,--page. - π΄ It takes ONE tag.
--tagis scalar and last-wins, like every other value-bearing flag on these clients, and?tag=is last-wins like every other parameter on these routes β so--tag a --tag band?tag=a&tag=bboth meanb, exactly as?limit=1&limit=2means 2. There is no way to ask for two categories at once, and that is deliberate: a repeatable--tagwith AND semantics was built and then dropped, because nobody had asked for the repetition and the choice between AND and OR had no author. Ask again if you want it β with the semantics you want named. - An operand that folds to nothing is a 400, not an empty result.
--tag ''and--tag '!!'are refused by a message naming the fold, on both clients (exit 2) and both routes. That is the opposite of?ref=, which narrows on an empty value: a ref names an ENTRY, so "nothing is called that" is a fact worth reporting, while a tag that folds away names no category at all and answering "no entry carries this" would hide the cause. - A new
X-Store-Statusvalue:tag-absent, answered 200 and exit 0. It is a non-finding and not an error, reached two ways β nothing in the scope carries the tag, or--ref/?ref=named an entry that does not while others do. The renderedtag:line carries both counts (N of M entries β¦ carry it), so a filtered index cannot be mistaken for a whole one. No exit code changed:cairn -exit-codesprints the same table it did. - The query side folds the same way the file side does, so
--tag Marketingfinds an entry writtentags: [marketing]and vice versa. A tag you can write is one you can ask for.
On the browse pages the entry and scope pages render an entry's tags, each a link to
/scopes?tag=<name> β a query parameter on the scope list, never a path segment. (The scope
list was the root until the root became a hub; /?tag= and /?q= still answer, with a 303 to
/scopes carrying the query.) That listing shows every visible entry carrying the tag, across
scopes, and reports how many entries it LOOKED at so an empty answer is legible.
π΄ And ?q= and ?tag= COMPOSE on the scope list, the same way they compose on the pod.
/scopes?q=lease&tag=marketing is one card: a search over the entries carrying marketing and
nothing else, narrowed by the same report.Search call GET /api/v1/search/<scope>?q=β¦&tag=β¦
makes. β This changed what a URL that already worked answers β the page used to render two
independent cards, a search over everything beside a listing of everything tagged, neither
narrowing the other. If you link to a two-parameter browse URL, it now returns the intersection.
The card's summary names both operands and reports how many entries the tag left out, so a
composed zero says why it is zero; the search box carries the tag along in a hidden control, so
refining the words keeps the filter and emptying the box returns to the plain tag listing.
β The recall report gains a tags: a, b line under a printed entry's header, beside the
existing refs: line, and a tag: β¦ header line only when the filter was sent β so no
output moves for a store with no tags in it.
The front-matter key is refs:. tasks: and task: are accepted spellings and stay
accepted β nothing warns, nothing is rewritten, and an entry carrying either loads with
exactly the refs it always did. refs: was chosen because the key now carries repos, PRs,
docs and dashboards rather than only work-tracker items, so tasks: named a subset of what
it holds. Where both appear on one entry, refs: is the one that is read.
Nothing you have to do. The three things worth knowing if you script this:
- A new read filter, on both clients and both read routes.
cairn recall --ref-to <system>:<id>andcairn search β¦ --ref-to <system>:<id>; over HTTP,GET /api/v1/recall/<scope>?ref-to=β¦andGET /api/v1/search/<scope>?q=β¦&ref-to=β¦. It is spelledref-toand notrefbecause?ref=already means "address one entry" on that route. A malformed operand is a 400 naming the parameter, from the same parser an entry's ownrefs:item goes through. β#must be percent-encoded as%23in a URL, or the operand arrives truncated at the fragment delimiter. - A new
X-Store-Statusvalue:ref-to-absent, answered 200 with a body that says what was read. It is a non-finding and not an error, and it is reached two ways β no entry in the scope carries the ref, or--ref/?ref=named an entry that does not carry it while others do. The renderedref-to:line carries both counts (N of M entries β¦ reference it), so a filtered index cannot be mistaken for a whole one. CAIRN_REF_BASE_<SYSTEM>turns a ref into a link, with no network call.github:andclickup:resolve from built-in public hosts. Every other system resolves through a variable you set β the system half upper-cased with-folded to_, sorefs: [tracker:1234]links only whenCAIRN_REF_BASE_TRACKER=https://tracker.example.invalid/tis exported to the process that renders it, givingβ¦/t/1234. Set nothing and the ref renders as plain text, exactly as it did before. π΄ It is read by whatever process RENDERS, which for the browse pages iscairn-uiand not your shell; and a base whose scheme is nothttp(s)βjavascript:included β has its link withheld while the ref still renders as plain text, the same thing an unregistered system does. Seeinternal/store/refurl.gofor why that is the safe direction rather than a limitation.
On the browse pages the entry section that used to be headed Tasks is now Refs, and
each ref is a link when it resolves. The text renderer now agrees: a printed entry's body
carries refs: β¦ and its index row carries π N ref(s). β That sentence read "the text
renderer's label is still tasks:" until an operator ruled on it; both rendered words moved
together with a regenerated reader fixture and conformance corpus, because the recall report's
bytes are compared byte-for-byte against the other implementation's by two gates. What did
NOT move: the JSON payload's key is still "tasks" β --json consumers are unaffected,
deliberately, because renaming a machine-readable key breaks a presence check silently where
renaming a human-read label breaks nobody.
| you want | run |
|---|---|
| refresh the local cache from the pod | cairn sync |
| a scope's digest | cairn recall --scope X (--ref R, --list, --limit N, --page N) |
| find a hunk by text | cairn search 'query' (--all-scopes) |
what the cache actually holds β the ENTRY files, never a scope's README.md |
cairn ls-entries |
| which sessions wrote attributed bullets in a scope, with coverage counts (Go client only) | cairn sessions --scope X (or --repo P) |
which registered arcs touched a scope, declared or inferred (Go client only; asks the pod) |
cairn arcs --scope X (or --repo P) |
| one registered arc: status, tooling coverage, members (Go client only; asks the pod) | cairn arc-show --slug S (home from --scope/--repo) |
| register or update an arc from a JSON payload (Go client only) | cairn arc-register --slug S --from F (home from --scope/--repo) |
| the post-write check: parse, dropped lines, marker reachability | cairn validate |
| one call of diagnostics | cairn doctor (--json, --no-sync) |
| append one dated, attributed bullet | cairn append --scope S --ref R --text 'β¦' --session ID |
replace a whole entry behind If-Match |
cairn put --scope S --ref R --file F |
| create a new entry (refuses to overwrite) | cairn create --scope S --ref R --file F |
| the scopeβinstance table, graded | cairn routes (--check) |
cairn sessions --scope X (and the pod's GET /api/v1/sessions/X) lists every session id that
signed a surviving ## Nuance / work-history bullet with a [cairn: <actor>/<session>] trailer,
per session: the actor(s) as written, the bullet count and the date span. Both surfaces call ONE
renderer, so the report bytes are identical; only the line above them differs (the CLI's cache
banner, the pod's snapshot stamp). It exists on the Go client and pod only β the Python oracle
has no twin. Every answer prints its coverage, because most bullets carry no trailer:
attributed: K of N (the NβK are NOT listed), what was scanned or rejected, that reads are not
recorded, and that trailers are self-reported (put/create write bytes verbatim, and the
session id is the writer's own word). scope-absent, scope-empty, no-attributed-writes and
scope-unreadable are four different statements and render differently; a scope you may not read
answers exactly like one that does not exist. Exit codes are the read set below β 0 for every
answer, 3 when nothing in the scope could be scanned.
An arc is one effort tracked by one handoff doc in one repo, keyed by (home scope, slug).
Arcs are REGISTERED β pushed by the operator tooling at each handoff β never derived, and they live
in an append-only journal the pod keeps outside the store tree: cairn-server -arc-journal <file> (or $CAIRN_ARC_JOURNAL), with no default. Unset, every arc route answers
registrations-unconfigured, the designed off state; and the pod refuses to start if the
symlink-resolved path is inside -store, because every directory at the store root is enumerated
as a scope. Registrations are kept forever (no compaction in this phase); the latest valid record
per arc is what is shown, and a torn final line or an unreadable one is skipped without poisoning
the rest.
cairn arc-register --repo . --slug my-topic --from arc.json # PUT /api/v1/arc/<home>/my-topic
cairn arc-show --repo . --slug my-topic # GET /api/v1/arc/<home>/my-topic
cairn arcs --scope alpha-notes # GET /api/v1/arcs/alpha-notesThe payload (schema: 1) carries status (open | closed; omit it and it is unknown, which
is never shown or counted as open), closing_kind, declared_scopes (the home is always
declared), writers_measured/readers_measured, commits_total/commits_unstamped,
reported_at and members (session, role β originated | earliest-stamped | wrote | resumed,
first_seen). Unknown fields are refused; a member whose session id could never match a write
trailer is not stored and is counted back as unjoinable=N. The pod stamps registered_by (the
credential's identity) and registered_at (its clock). A push that did not measure a leg keeps the
members an earlier push measured for it, marked carried. Registering needs the write verb on
the home scope and on every declared scope (a scope you may not write answers like one that does
not exist); the bare legacy row may read arcs and may not register one.
Visibility: an arc exists for you only when its HOME scope is readable to you; its other scopes
are listed only if readable too, and hidden ones are omitted, not counted. arcs --scope X lists an
arc as declared when it names X, or inferred when one of its member sessions wrote an
attributed bullet in X without the arc declaring it β a join over self-reported trailers, labelled
as such. All three verbs print the pod's body verbatim (registrations are not in the cache) and use
the existing exit codes: reads 0 for every answer, 3 if the pod did not answer; arc-register 0
registered or unchanged, 6 refused (including a pod with no journal), 7 did not happen.
cairn arcs --check asks the pod (GET /api/v1/arcs/<scope>?check=1) whether the arc registry
holds anything it should not. It is a flag on arcs, not a verb and not a doctor check: doctor
is about this host and this credential, and its stdout is byte-compared against the Python oracle.
cairn arcs --check --repo . # arcs HOMED in this repo's scope
cairn arcs --check --all-scopes --scope alpha-notes # every arc visible on that scope's podFindings (exit 9): a journal-damaged journal (a complete record that did not parse, or a
torn tail β an arc registered only by that record is invisible everywhere); an arc whose
home-scope-absent (its home is readable to you by grant and does not exist in the store); and a
declared-scope-absent (the same for a declared scope). Not findings, printed as coverage
instead: a member session with no attributed write in any scope you can read (members come from
commits and transcripts, so this is the normal state), and a writing session in no arc (arcs are
registered only from now on). Turning either into a finding would hold the check non-zero forever.
Exit codes are doctor's β 0 nothing wrong measured, 9 a finding, 10 could not look (no
-arc-journal on the pod, a journal it cannot read, a pod that did not answer, or a pod too old to
know ?check=1, which answers a listing that the client refuses to read as a pass). No new exit
code exists for it: cairn -exit-codes is unchanged. Usage errors are still 2, and a scope no
instance serves is still 11. Only arcs whose HOME you can read are checked, and a declared scope
you cannot read is neither checked nor counted β the check answers a scope you may not read exactly
as one that does not exist. β One pod per run: the scope picks the instance, and a multi-instance
host is not fanned out.
tests/arcs/e2e.sh is the end-to-end check over both real binaries (--self-test proves it goes RED
on a sabotaged build).
Read outcomes and write outcomes are disjoint, so a supervisor cannot read
silence as success. 0 content was served (live, cached, or a genuinely empty
scope) Β· 2 usage Β· 3 nothing was read β the store was unreachable with no
cache, or the cache is there and could not be fully read (a mode-000 scope
directory or entry file), which is a permissions problem on this host's cache
rather than an outage to retry: fix the mode, or cairn sync to replace the
cache. β This clause used to say the state is "a content defect cairn validate diagnoses", and that remedy is a dead end β measured on both
clients, cairn validate over exactly that state exits 3 itself with 0
bytes on stdout, so it diagnoses nothing the failing read did not already say.
The one thing it adds is the PATH, which every verb already names in the same
sentence. Β· 4 sync did
not refresh though a usable cache survived Β· 5 the archive was refused Β· 6 the
store refused the write, change the request Β· 7 the write did not happen and
a retry is the right response Β· 8 precondition failed, re-sync and re-apply Β·
9 create only, the entry already exists β do not retry, and do not
conflate it with 8 Β· 11 no instance could be decided for this scope. doctor
prints its own set on every run, where 10 means no problem measured, but at
least one check could not look β which is not a clean bill of health.
Writes never degrade. There is no cache to answer from and no such thing as a
stale write, so an unreachable store is a refused write at 7, never a queued or
local one.
An index row ends in badges, and there is a new one. An entry with more than 30
## Nuance / work-history bullets gets β OVER 30 nuance β prune or split:
example-entry 151 nuance internal π΄ 3 OPEN β OVER 30 nuance β prune or split
Why it exists. A digest prints the index for every entry and one featured
entry's body in full, so a single oversized entry sets the cost of the whole read.
Measured on a real store: a digest came to ~50K tokens and 97.6% of it was one
body, because that entry had 151 nuance bullets. The row already printed 151 nuance β a number with nothing to compare it against. The badge supplies the
comparison by naming the bar.
Where 30 comes from. The bullet count is a proxy for what a featured render
costs, so the threshold was set from the cost curve rather than picked: over one
live store of 331 entries the median nuance section is 2.5 KB at 0β9 bullets,
22 KB at 20β29 and 47 KB at 30β39 β and the smallest body above 30 is 29 KB
(~7.4K tokens) against 14.6 KB at a ceiling of 25. It fires on ~5% of rows,
against 41% for the π΄ N OPEN badge already beside it; a badge that fires on
everything trains its own bypass.
It is advisory. Nothing refuses, no exit code moved, no new status value, and
an entry at or under the ceiling renders byte-identical to before. β If you
PARSE an index row, note that a row may now carry one more badge, and that it is
appended after π N refs β the existing badges keep their order. The
threshold is one named constant per renderer (NuanceBulletCeiling /
NUANCE_BULLET_CEILING), pinned equal by tests/test_nuance_bullet_ceiling.py.
Inside a printed body, each line that opens a top-level bullet gains a trailing
[cb:xxxxxxxx] β 8 lowercase hex:
## Pointers
- apps/example/values.yaml β the chart values [cb:710f5a96]
## Nuance / work-history
- 2000-01-02: OPEN: the retry budget is still unbounded. [cb:3cc70d82]
a continuation line, which belongs to the bullet above it.
Both ids above are the real values for those exact bytes, and the second one is worth a second look: it is taken over the bullet's two lines, so the same opening line with no continuation under it answers a different id. A bullet is its wrapped prose, not its first line.
Why it exists. "Was this recalled bullet actually used?" had no answer. Every available proxy is a spelled one, walkable by rewording, and the obvious one is saturated by a mandate rather than by use β a downstream consumer's resume flow requires its report to echo what it recalled, so "a printed ref reappears later" fires on 451 of 454 runs (99.3%) and measures compliance. A token that exists nowhere else in a corpus turns that question into a match.
All four surfaced sections, not just the journal. ## What it is,
## Pointers, ## Nuance / work-history and ## Requirements are all annotated.
Measured over a real 344-entry store, ## Pointers carries 1,589 of the 4,978
ids a full read would print β 32% of them β so annotating only the journal would
have annotated the section where the signal is not. (That total is one reading of a
live store and will not re-derive; the nuance section alone moved by 8 bullets
between two measurements an hour apart.)
What the id is a function of. sha256 over the bullet's own lines joined with
\n, first 8 hex. It is not scoped: two byte-identical bullets in different
entries or scopes get the same id, deterministically. It moves when the bullet
does β including on a trailing-whitespace change, a change of Unicode form, adding
or removing an OPEN:/RESOLVED <sha>: marker, and the [cairn: actor/session]
write trailer, which lives inside the opening line and makes an id partly a function
of the writing session. Line terminators are normalised, so a CRLF file and an LF
file name the same bullet with the same id.
The cost, measured rather than estimated. 14 bytes per annotated line. On a real
local cache: the default digest read of one scope went 9,181 β 9,279 bytes
(+98 B, +1.07%, 7 tokens), and a full --limit 100 read of the same scope went
150,569 β 152,935 (+2,366 B, +1.57%, 169 tokens). Across four scopes the full-read
cost ranged +1.57% to +2.61%; the percentage is highest where bullets are
shortest. --list is unchanged β the index prints no bodies, so it carries no
tokens.
β If you PARSE a printed body, a bullet's opening line now has a suffix. Lines that open no bullet are untouched, and not every bullet line carries an id: prose, blanks, an indented dash (a continuation), a dash inside a code fence, and text before a section's first bullet all print in full with no token. A missing id is never evidence that a bullet was not printed β the body is still emitted line for line, verbatim.
β If you echo a recalled bullet VERBATIM back into a write, nothing changes: both
pods strip the token (and the attribution, in any order) before taking a bullet's
content hash, so a re-POST of an annotated line is still recognised as the same bullet
and answers duplicate without writing. The token does count against the
2,000-character submitted-text cap.
π΄ But that is a claim about the HASH, not about the bytes on disk β and a token you
quote INSIDE new prose is PERMANENT. The strip only reads a trailing token; a
[cb:β¦] in mid-sentence is prose, and the write path stores the submitted text
verbatim. Measured end to end against the pod's own functions:
READ 1 - 2000-01-02: the retry budget is still unbounded. [cb:7a54575e]
SUBMIT building on [cb:7a54575e], the budget is now capped at 5 retries.
STORED - 2000-01-05: building on [cb:7a54575e], the budget is now capped β¦ [cairn: β¦]
READ 2 - 2000-01-05: building on [cb:7a54575e], β¦ [cairn: β¦] [cb:58ff27d9]
Two consequences, and neither is hypothetical once anyone cites a bullet this way:
- A consumer parsing the annotation must take the LAST
[cb:β¦]on a line β and must NOT read a token's PRESENCE as evidence that the line opens a bullet. Both halves are needed and the second was missing from the first draft of this rule. A line that already contains a token renders with two, and a first-match regex reads the quoted one β the id of a different bullet. But the mirror image is worse, because a last-match regex is still wrong on it: a quoted token can end a line that carries no id at all β prose, a wrapped bullet's continuation line, an indented dash, or a fenced sample β and all four of those shapes are enumerated above as unannotated. Measured twice, on two different bodies: a strict last-match scraper agrees withcitation_ids_by_start_lineon every annotated line and misattributes exactly one line per unannotated shape β four of them β each to a different bullet's id, which is precisely the harm the first half exists to prevent. π΄ The four is STRUCTURAL and the denominator is not: there are four unannotated shapes, so the count is four on a 9-line body and four on a 14-line one. Do not quote a ratio here; quote the shapes. Both renderers produce identical bytes here, so nothing downstream can detect the difference by comparing the two; the rule is the contract. π΄ Which is why the real advice is not to scrape at all. Derive ids from the body withCitationIDsByStartLine/citation_ids_by_start_line, which answersNonefor every unannotated shape by construction. A scraper has to re-decide what opens a bullet, and that decision is the one this repo already records a parser defect in. - "A token that exists nowhere else in a corpus" stops holding after the first such write. That sentence is this feature's founding rationale β it is what makes "was this bullet used?" a match rather than a guess β and a citation written into stored prose puts the token in the corpus permanently, where a later search cannot tell a use from a quotation. This is the same saturation-by-echo that disqualified the previous proxy at 451/454 = 99.3%, arriving by a slower route. The feature is therefore weaker than the rationale above states, by an amount that grows with use, and nothing in either implementation bounds it.
GET /api/v1/snapshot carries an ETag β "sha256:<64 hex>" over the
uncompressed tar β and both clients store it beside the cache in .sync-etag
and offer it back as If-None-Match on the next sync. A pod that has nothing new
answers 304 with no body, and the client keeps the cache it already has:
$ cairn sync # first run
cairn: live β fetched from https://store.example.invalid just now β 7 entries, snapshot seeded=β¦
$ cairn sync # nothing changed since
cairn: live β already current at https://store.example.invalid β not modified, snapshot seeded=β¦
Read that line as its own thing. It is still the live state β the pod was
reached and answered β and it is neither fetched β¦ just now (bytes arrived) nor
β β¦ SERVED FROM CACHE (the pod could not be reached). Exit code, cache and
subsequent reads are unchanged.
β What it buys, exactly: the client's download and its extraction. The server
still walks the store and builds the archive to compute the digest, so this is
bandwidth and client CPU, not pod CPU. Nothing is written on a 304, so a later
cached banner still dates the last download β it under-states freshness
rather than over-stating it.
You need nothing for this. An older client against a new pod simply sends no validator; a new client against an older pod stores none.
The client is configured by ~/.config/subsystem-store/env β
CAIRN_URL and CAIRN_TOKEN, environment variables of the same name winning
(and the old SUBSYSTEM_STORE_* spellings still accepted in both places, see
the section The environment variables are now CAIRN_* above) β
and that instance is called personal. A second instance is
an additive file at ~/.config/subsystem-store/instances/<alias>.env with its
own cache root (~/.cache/subsystem-store-<alias>) and its own sync stamp; the
default instance's cache root does not move. Which instance a scope belongs to is
decided by a table you supply β
~/.config/subsystem-store/routes.json, or $CAIRN_ROUTES, a flat JSON object of
"scope": "alias" β because this program knows how to route and nothing about who
routes where.
π΄ With more than one instance, an unregistered scope refuses at exit 11, naming
the scope. It never falls back: a write that lands in a store nobody reads is
found days later, if at all, and a refusal costs one error message.
cairn routes --check grades the table in both directions β a scope with no
entry, and an entry naming an alias that does not exist. β It does not fail on
a table entry naming a scope that holds no entries: that prints as a note,
because a snapshot ships entry files rather than directories, so a stale entry
and one pre-registered before its first write look identical from here. Do not
gate CI on --check expecting it to catch that case. With one instance
configured, every read prints what it always did. Two exceptions at any instance
count: the write verbs name their instance unconditionally β "where did that
bullet go" is a question about a durable record, asked later, by someone who no
longer has the terminal β and a table entry naming an alias this host has no config
for still refuses. β If you parse cairn: appended scope=β¦ ref=β¦ positionally,
that field is new β it sits between the status and scope=. The binding rules for
anyone editing a routing path are in lib/README.md.
| route | what |
|---|---|
GET /healthz |
unauthenticated; body is exactly ok\n |
GET /api/v1/recall/{scope} |
rendered digest (?mode=&ref=&limit=&page=) |
GET /api/v1/search/{scope}?q=β¦ |
search (?threshold=&max_hits=&context=&all_scopes=) |
GET /api/v1/snapshot[?scope=] |
gzipped tar of the entry files β the sync payload |
GET /api/v1/sessions/{scope} |
which sessions wrote attributed bullets, with coverage β Go pod only, same renderer as cairn sessions |
GET /api/v1/arcs/{scope} |
registered arcs that touched the scope, declared or inferred; with ?check=1 (&all_scopes=1) the orphan check, X-Store-Exit on doctor's 0/9/10 β Go pod only |
GET /api/v1/arc/{home}/{slug} |
one registered arc β Go pod only; arc-unregistered when it is not visible to you |
PUT /api/v1/arc/{home}/{slug} |
register or update one arc (write verb on every declared scope; 409 registrations-unconfigured with no -arc-journal) β Go pod only |
GET /api/v1/sources/{scope} |
the scope's declared code sources (git:<host>/<repo-path>[//<subpath>]@<branch>, first is primary), read from $CAIRN_SOURCE_JOURNAL β an environment variable with no flag, a file OUTSIDE the store root, read-only to the pod; sources-unconfigured when unset, 503 store-unreachable when it cannot be read β Go pod only |
POST /api/v1/entry/{scope}/{ref}/bullets |
append ONE attributed bullet (the actor comes from the token, never the body) |
PUT /api/v1/entry/{scope}/{ref} |
whole-file replace via If-Match, or create via If-None-Match: * |
Auth is a token set β one row per line, <token> for legacy or
<token> <identity> <scope>,<scope> for a mapped row β rotated by overlap and
hot-reloadable with SIGHUP, so onboarding an agent does not restart the pod. The
full contract, plus the operating runbook (seeding, byte-identity verification,
rotation, rate limiting), is server/README.md.
There is one. It carries the entries page, a browser sign-in with server-side revocable sessions, the share flow, and a GitHub sign-in through the operator's GoTrue.
β THE ROUTE COUNT AND THE PHASE COUNT ARE DELIBERATELY NOT WRITTEN HERE, AND THE DELETION IS
THE FIX. This paragraph said "three phases" and "Seven routes" and then enumerated
them β and stayed at seven through the change that added three (POST /sign-in/github,
GET /sign-in/github/callback, GET /static/app.css). cmd/cairn-ui's own package comment
identifies this exact hazard, fixed it there, and left the literal seven standing in the file
the fix was about. A number in prose beside a ledger the program DERIVES is a second spelling
that only ever goes stale, so: cairn-ui prints its own count at startup, and
ui.DeclaredRoutes() is the list. What is worth stating instead is the SHAPE, which does not
move when a row is added: GET / is the entries page; the sign-in rows and the stylesheet
answer without a session, because they are how you get one; the share rows and POST /sign-out
are behind it; and /healthz is answered before the chain and is outside the ledger entirely.
π΄ cairn-ui is a SINGLE-REPLICA surface, for TWO reasons and not one. The
session table is the loud one: each replica holds its own sessions, so a second
replica without shared storage signs users out on a random fraction of requests.
The quieter one is the control-plane cache β a share recorded on replica A is
not served by B until B's cache refreshes, which is what the replica-honesty notice
on every share page exists to say. Putting only the session file on shared storage
buys two replicas that keep people signed in and silently disagree about who can
see what. Multi-replica is a later arc, not a configuration.
β AND THE CLAIM THAT NOTHING DEPLOYED IT IS RETRACTED. This paragraph read "And nothing
deploys it: packages.ui-image builds an image, but nothing publishes that image and there is
no manifest in this repository β it is built and run by hand." All three clauses are false.
.github/workflows/publish-image.yml pushes the UI image to its own ghcr package and then
proves it pullable with no credentials; a manifest deploys it from the operator's GitOps
repository; and the surface is live on a public hostname. "No manifest in this repository"
was never evidence about what is running β that is the inference this sweep exists to kill, and
it stood in four places at once.
nix build github:ZacxDev/cairn#cairn-ui
./result/bin/cairn-ui -store <store root> -token-file <token file> -port 8103β A credential is required, and off-cluster you must say where it is.
-token-file defaults to the pod's secret mount
(/run/secrets/subsystem-store/token), so on a machine without one the binary
exits 78 and serves nothing. Three ways to supply it, all measured:
-token-file <path>; CAIRN_TOKEN_FILE=<path> with no flag; or
-token-file= (explicitly empty) plus CAIRN_TOKEN=<row>, which is the
env fallback the binary's own refusal names. Single-dash flags: this uses Go's
stdlib flag, not the client's --long style. -h lists eight β -store
(CAIRN_STORE_ROOT), -host (CAIRN_UI_HOST), -port (CAIRN_UI_PORT),
-token-file (CAIRN_TOKEN_FILE), -session-file
(CAIRN_UI_SESSION_FILE), -session-ttl (CAIRN_UI_SESSION_TTL),
-control-journal (CAIRN_UI_CONTROL_JOURNAL) and -db-dsn
(CAIRN_UI_DB_DSN) β and every default is env-resolved,
so what -h prints depends on your environment. It reads the store from disk
rather than over HTTP, and authenticates against the same token file as the pod.
π΄ -db-dsn MOVES THE SESSION TABLE AS WELL AS ARMING INVITATIONS, AND THE FIRST
START WITH ONE SIGNS EVERY OPEN BROWSER OUT ONCE. It is a PostgreSQL connection
string for this surface's mutable state, which is both tables internal/pgstore
holds: sessions and invitations. With it set, -session-file is ignored β the binary
says so on startup β and the invite flow works; without it, sessions stay on that file
and every invite route tells its reader the deployment holds no invitation store. The
connection is made, pinged and migrated at startup, under a 10 s bound, and a
failure is a refusal to start (exit 78) rather than a surface that passes its health
check and fails every request. β Prefer the environment variable to the flag: a DSN
carries a password and a flag value is in argv. Nothing in this program echoes the
value β not the startup line, not any refusal.
π΄ "Env-resolved" does NOT mean "the variable and the flag are interchangeable", and
CAIRN_UI_CONTROL_JOURNAL is where that matters. An unset variable and an explicitly
EMPTY one both mean "no control journal" β the shape a manifest that emits every variable
with an empty default produces. A value that reduces to nothing but whitespace is
refused at 78, naming the variable, rather than read as unset: read as unset this
surface would come up on the token-file projection, which confers admin on nobody, so
every scope page answers 404 and no share can be recorded while /healthz still answers
200. That is the same ruling the pod makes for its own CAIRN_CONTROL_JOURNAL. The flag
path refuses the same value for a different reason β -control-journal ' ' has always
failed its stat β so the two arrival paths agree for every value that does not name an
existing file, which they did not between 1659663 and 68cf955.
β THEY DO NOT AGREE ON A VALUE THAT DOES, AND THE FLAG IS THE LENIENT SIDE β MEASURED,
NOT INFERRED. With a control journal in the working directory whose filename is literally
three spaces, -control-journal ' ' serves, announcing
sharing writable (control journal ), while CAIRN_UI_CONTROL_JOURNAL=' ' exits
78 naming the variable. The flag never meets the blank policy at all β it meets
openAuthority's stat, which is a question about the filesystem and not about the
spelling β so "the two paths refuse the same set" is true of every value anyone would
type and false in general. It is left open, and the reason is a judgement rather than an
argument from the mechanism: the fix is available and obvious β run the same blank policy
over the flag's value too β and it was not taken because the direction is safe (the flag is
the LENIENT side, so nothing is refused that should be served), because naming a file with
nothing but whitespace is not a thing an operator does, and because a check on the flag
would refuse a path the filesystem resolves. Nothing gates it; the agreement test's own
docstring says which values it covers.
The other six variables above are resolved by
internal/envalias, which reads a whitespace-only value as absent and silently takes
the code default; tests/conformance/README.md measures what each one does with one.
π΄ Two backends are absent, for two different reasons, and conflating them is the
misreading to avoid. Supabase is simply not wired yet and returns with the
phase that builds a sign-in flow. The trusted-header backend is refused on
principle and is not coming back here: a browser surface exists to be publicly
reachable, and that backend lets anyone who can open a socket to it be any user
at full authority. AuthBackends takes one parameter so neither can be passed;
TestTheUIChainHasNoTrustedHeaderMember pins the exclusion.
GET /share lists the scopes your credential may administer; GET /share?scope=<id>
is one scope's page. It answers three things: who has access to this, which of
those you can take back, and a form to share it with somebody.
π΄ "Who has access to this" is computed from the authority, not from the list of shares. Access arrives two ways β a share, or membership of the project that owns the scope β so a page that listed only shares would under-report every project member, and in the direction that tells you your notes are more private than they are. The two lists are shown separately because only shares can be revoked: somebody who reaches a scope through project membership keeps it after every share is taken back, and the page says so.
β You can only share with people and projects you already share a project with.
That is deliberate β a picker listing every user would turn admin on one scope into
a directory of everyone in the deployment β and it means reaching anybody else needs
an invite. β The clause that stood here said an invite "does not exist yet", and it
is corrected rather than deleted because it was true across three changes and is the
sentence somebody would act on. The invite flow exists: GET/POST /invite,
POST /invite/revoke and a public GET /join, over the invite store -db-dsn
configures. It is absent on a deployment with no -db-dsn, where the invite pages
say so and their writes answer 501 β so "does not exist yet" is now a statement about a
CONFIGURATION rather than about this repository.
π΄ THE SHARE FLOW NEEDS -control-journal <path>, AND NOT ONLY TO WRITE. Without one
the authority is the token file, which has no shares to record and confers admin on
nobody β so no scope is administrable, GET /share renders "No scope is administrable
by this credential", and a scope page answers 404 to every caller. The page says on
every load that this deployment cannot record a share, so the cause is visible rather
than discovered at a click; but read the limit at its real width β without a journal the
share flow has nothing to show, not merely nothing to change.
β An earlier draft of this paragraph claimed "the share pages still answer who can see this" without one. That is false, and it was measured false rather than argued: with a token-file authority, 0 scopes are administrable across every principal in the projection and the scope page is a 404. The retraction is kept because the sentence was plausible enough to survive writing it.
Every share page carries a notice about what this surface can and cannot promise: it is one replica's answer from a cached copy of the authority, another reader gains or loses access when their own cache refreshes rather than the instant you click, and revoking stops future syncs without recalling entries already copied onto somebody's machine. That notice is pinned whole by a test, so it cannot be quietly reworded into a stronger promise.
It is the only package here that links a third-party module (gomponents, for
HTML), and the serving path is still stdlib-only β no package the pod or the
CLI links reaches it. What replaced vendorHash = null, and how that is measured
rather than asked for, is stated once in internal/depspolicy's package doc; how
this page escapes entry text is stated once in
internal/ui/README.md. Both are pointers on purpose:
those claims have one home each and a correction belongs there.
| path | what |
|---|---|
cairn |
the Python client CLI, and the ORACLE the Go one is measured against |
lib/ |
the Python reader: cache resolution, recall rendering, scope/ref resolution, doctor |
server/ |
the ORACLE pod: server.py, Dockerfile, seed.sh, verify-byte-identity.sh β no longer what runs |
cmd/cairn-server, internal/api |
the Go port of the server β passes the corpus, and the deployed pod |
cmd/cairn, internal/client |
the Go port of the CLIENT, and the default β diffed against the Python one by tests/parity/, which declares both its residuals and the rows that compare only the exit code |
internal/report |
the ONE renderer, shared by the pod and the CLI |
cmd/cairn-ui, internal/ui |
the BROWSER surface β pages, sign-in (credential form and GitHub through the operator's GoTrue), the share flow, gomponents; published, and DEPLOYED |
internal/depspolicy |
the allowlist and import ban that replaced vendorHash = null β the serving path is still stdlib-only, and this is what measures it |
tests/ |
the suites, plus leakscan.py, the HTTP conformance corpus, the server dual-run gate and the client parity gate |
flake.nix |
both clients (default is the Go one, #cairn the Python one), the server image, the Go server, and the checks |
server/server.py is the oracle; cmd/cairn-server is a stdlib-only Go port of
it, and it is what the cluster runs β this line read "deployed by nothing" until
the cutover. Rewriting the server alone would have left two renderers in
two languages that must agree byte-for-byte forever, with drift arriving as "a
different order that reads as a stale cache" β no error, no missing entry. So both
land on the same internal/report: one renderer, two consumers β the pod and the CLI.
β The browser surface is not one of them. internal/ui imports internal/report
nowhere; it reads internal/store and renders HTML through its own code, so the page
is a second renderer producing a different medium, and nothing compares the two. That
was a forecast ("a future UI") until cairn-ui shipped; it is now a measured exception,
and whether the page should ever route through internal/report is open. β And
one-renderer becomes a property only when the Python renderer is deleted at P8; until
then it is a discipline, and these three instruments are what enforce it:
| instrument | what it compares | read it |
|---|---|---|
tests/conformance/ |
the served HTTP contract, as golden fixtures generated from the oracle, replayed against either server | the corpus and its declared blind spots |
tests/dualrun/ |
both servers over one store β every route, scope, entry and principal, and the snapshot's uncompressed tar byte for byte | why gzip identity is unattainable, and the PAX-header divergences a tree comparison cannot see |
tests/parity/ |
both clients, one pod, one cache root, identical argv, across every verb in the table above | the declared residuals, the P8 retirement ledger, and which rows compare only the exit code |
β Counts are deliberately not quoted here. Every one of them went stale in this
file at least once while the gates moved, and nothing asserted on them; the numbers
live beside the gates that produce them, and the CI floors live in
.github/workflows/ci.yml. Re-derive rather than trust β
grep -c 'Case(' tests/parity/harness.py for the parity case count, and the
compare= distribution beside it for how many rows compare output text rather than
only the exit code.
β And a green gate is not evidence until its controls have been watched to work.
The parity gate's first full run reported 72 PASS / 0 FAIL while every request was
refused and no cache was ever written β two clients failing identically compare equal.
Both harnesses now refuse to vouch (exit 2, which is not "failed") unless a
pre-flight succeeds, and both ship a --self-test that proves the differ can go red.
pytest tests/ # the Python suite
go vet ./... && go test ./... # the Go port's own guards
tests/conformance/run_go.sh # the corpus against the Go server
python3 tests/conformance/suite.py run # β¦and against the oracle: must stay at 0 failures
python3 tests/parity/harness.py # both CLIENTS over one cache root: the P2 gate
python3 tests/parity/harness.py --self-test # prove the differ can go RED
python3 tests/reader_fixtures.py generate # re-record the renderer's bytes FROM the oracle
python3 tests/leakscan.py # the leak gate β runs in CI on every commit
python3 tests/leakscan.py --self-test # prove the gate is an instrumentThis repository is public and was extracted from a private one: never commit
hostnames, private IPs, real project/scope names, dated incident narration, or
captured text β fixtures are synthetic. tests/leakscan.py runs in CI on every
commit. The full rules agents work under live in AGENTS.md.
Several sessions and agents routinely work this repository at once, through
linked worktrees of one clone, so changes are made in a worktree and never in
the base clone β a commit onto the branch a peer left checked out is the silent
failure, because git log afterwards shows exactly what you expect.
claudedocs/working-in-parallel.md is the
recipe and the reason behind each rule; .claude/hooks/base-clone-write-guard.py
refuses the write rather than relying on anyone having read it. That hook fires
only when the clone actually has linked worktrees, so a fresh clone never sees
it, and it fails open by design β it is a guard against a routine mistake, not a
security boundary.
The project is cairn. Identifiers that are not environment variables still read
subsystem_store β the lib/ module names, ~/.config/subsystem-store/,
/run/secrets/subsystem-store/, the subsystem-recall CLI alias. Those are accepted
aliases, kept so existing deployments don't need a coordinated cutover. The
environment variables have been renamed; see the migration note below.