Skip to content

feat(services): sealed, proven service discovery for a configured node - #694

Closed
bahdotsh wants to merge 2 commits into
feat/app-identifiers-from-app-keyfrom
feat/app-discovery-membership
Closed

bahdotsh wants to merge 2 commits into
feat/app-identifiers-from-app-keyfrom
feat/app-discovery-membership

Conversation

@bahdotsh

@bahdotsh bahdotsh commented Nov 4, 2026

Copy link
Copy Markdown
Member

Step 7c of the shared-network plan (ADR 0037, "Service discovery"). Stacked on #693.

On a configured node, service discovery stops naming services in the clear. A provider answers only queries proven by a member of its own application. This replaces the __SVC_DISC* exemption that #689 left in the inbound gate.

Wire

  • Sealed bodies: query and response bodies are sealed under HKDF(K_app, salt, "offline-app-svc-v1") as salt(16) || ChaCha20-Poly1305(nonce 0, no AAD).
    • The cleartext service_id is the sentinel __sealed. No ServiceId can be that value, and register_service refuses it.
    • Query plaintext: {issued_at, identity_key, service_id?}.
    • Response plaintext: {provider, identity_key, service_id, version, capabilities, hop_count}.
  • Proofs: each body carries an end-to-end proof (kinds 4 and 5) in a member_proof payload field. It is not in frame metadata, because every forwarder builds a new frame. The proof binds u16be(len(query_id)) || query_id || originator(21) || sealed, which no hop changes. The hop budget is not bound, because every forwarder rewrites it.
  • Identity key: the sender's identity key travels sealed, because the frame's control key is the last forwarder's. The verifier checks that derive_address(identity_key) matches the address the proof speaks for.
  • Unconfigured frames are byte for byte unchanged (skip_serializing_if).
  • Vectors: a discovery block in app-membership-v1.vectors.json.

Rules

  • Freshness: |now - issued_at| <= 60 s. One answer per query_id, held in an answered map until issued_at + 60. At its cap the map refuses rather than evicts, so the failure mode is a lost result, never a double answer.
  • No crossing between modes: a configured provider never answers a cleartext query, and a configured originator never reports a cleartext response.
  • Key without a certificate: a configured node with the key but no certificate sends 189 random bytes as its proof.
  • Behaviour change from feat(protocol): a member answers only the peers its application admitted #689: acknowledgements now go out for discovery carriage. That covers a query, a response bound for someone else, and a proven response that ends here. Without it, a proven provider retries its answer to expiry, and which forwarded queries get acknowledged would tell an observer which pairs are members. A configured node therefore acknowledges a stranger's query, as an unconfigured node does. Please look at this one in review.

Tests

  • Services: 12 new tests over a fake sealer.
  • Engine (tests/membership_discovery.rs): 4 tests on a line of nodes, run over the real paths.
    • Two members are three hops apart. The nodes in between, one from another app and one unconfigured, carry the frames and register the same service id, yet never see it in the clear, never answer and never report.
    • A node with the key but no certificate gets no answer.
    • A replayed query is not answered.
    • A response whose proof names another provider is not reported.
  • Mutation checks:
    • Turning off the response rule makes a_forged_answer_to_a_member_is_not_acknowledged fail.
    • Skipping the query proof check makes the stranger tests fail.

Rebase

Rebased onto #693.

  • The sealed constants, label list and vector-block conflicts were merged by keeping both sides.
  • The vectors were regenerated, and generate.py --check passes.
  • A duplicate membership_configured (both stacks added it) was removed.
  • Run after the rebase:
    • cargo clippy --workspace -- -D warnings: pass.
    • cargo test --workspace --lib: pass, except the local-only no_shipped_file_names_a_relay, whose offenders are all under a stale .claude/worktrees.
    • Uniffi 319 tests: pass.
    • Rustdoc and fmt: pass.
    • Bare-metal clippy for sealed on all four targets: pass.

Known

  • Threat model R9 still says discovery gossip travels in the clear. The docs PR at the top of the stack rewrites it.
  • A node on an older release forwarding a configured query drops the unknown sealed/member_proof fields, so configured discovery does not cross older nodes.

CI runs only once the stack below merges into main.

Step 7c of the shared network plan (ADR 0037, "Service discovery"), the
services half.

- The sealed layer gains the discovery constants, in their one home
  (ADR 0022): the body key label `offline-app-svc-v1`, the 16-byte salt,
  the sentinel `__sealed` a configured node writes in the cleartext
  `service_id` (a service id may not begin with `__`, so no provider can
  register it), and `discovery_bound_body`, the bytes a discovery proof
  binds: `u16be(len(query_id)) || query_id || originator(21) || sealed`.
  Pinned by a new `discovery` block in the membership vectors, and the
  discovery query vector now binds that body. no_std clean.
- `DiscoverySealer`: what the engine lends a configured node (the
  application key, the certificate, the identity key, a clock), so this
  crate links no cipher and knows nothing of MLS.
- With a sealer, a query carries only `query_id`, `originator` and the
  hop budget in the clear; the service asked for, `issued_at` and the
  originator's identity key travel sealed, with a proof of kind 4. A
  provider answers only a proven query, only while `|now - issued_at|`
  is within the 60 s lifetime, once per `query_id`, addressed to the
  originator. An originator reports only a response whose sealed
  provider is its sender and whose proof verifies. Forwarding is
  unchanged either way, and the sealed fields ride through every
  forwarder, configured or not.
- Without a sealer every frame is byte for byte what it was.

The proof travels in a `member_proof` payload field, not frame metadata:
every forwarder builds a new frame, so metadata would not survive a hop.
…ation's rules

Step 7c of the shared network plan (ADR 0037, "Service discovery"), the
engine half.

- `MembershipKeys`: this node's membership borrowed apart from the rest
  of the engine, the one place the end-to-end proof is made and checked
  for session-opening frames and discovery alike. It implements
  `DiscoverySealer`: the body under `K_app` with a fresh salt, the proof
  of kinds 4 and 5, and an identity key carried in a sealed body that
  must derive to the address it speaks for. A node with the key and no
  certificate sends random bytes in the proof's place.
- With a publisher key set, discovery fans out to the mesh neighbours
  only, never to `known_peers`, which membership makes an admitted set.
- The inbound gate no longer passes every `__SVC_DISC` frame: a query,
  and a response on its way to someone else, pass whoever sent them; a
  response that ends here passes only from an admitted sender or with
  its provider's proof. The acknowledgement point follows the same rule,
  so a member acknowledges gossip from anyone (no ack pattern marks its
  fellow members on the link) and a proven answer is not resent to its
  expiry.
- The control-message chapter states the configured node's discovery
  wire: the sentinel, the sealed body, the bound body, where the proof
  travels and why, and the freshness and once rules.

Tests over a line of nodes on the real send, relay and receive paths:
two members three hops apart find each other's service through a
member of another application and an unconfigured node, which carry
both frames, never see the service id, and neither answer nor report;
a stranger's query is forwarded and never answered; a forged answer to
a member gets no event and no acknowledgement; an unconfigured
originator reports as before. Disabling the response gate or the query
proof check fails one each.
@bahdotsh bahdotsh closed this Nov 4, 2026
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 4, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant