Skip to content

Put the whole lab on one wire, so a recording is one file - #78

Merged
aenawi merged 1 commit into
developfrom
hud-feed
Sep 12, 2026
Merged

aenawi merged 1 commit into
developfrom
hud-feed

Conversation

@aenawi

@aenawi aenawi commented Sep 12, 2026

Copy link
Copy Markdown
Owner

Target

  • This is going into develop: the normal case for a fix, a chapter, a
    lab change, a typo
  • This is a release: develop → main, shipping to the published guide
  • This is a hotfix: straight into main, because published advice is
    currently locking readers out or leaving them exposed

What this changes

The live half of this server was four SSE endpoints — status, tcpdump,
logs, stats — and each consumer opened the one it wanted and parsed it for
itself. Now there is one, /api/stream/hud, carrying every live thing in one
envelope: {seq, at, type, data}. Ten types, listed in an opening hello so a
consumer can ask rather than assume.

Two things are on it that were never on a stream. Every Result the API returns
is published as verdict (action, set, preset) or flow (probe), with the id of
what ran travelling beside it. And note now says which source it came from.

No issue — this was asked for directly. It is the second of a planned series;
what is deliberately not in it is listed at the bottom.

Kind

  • Prose: clarity, typo, dead link
  • The advice was wrong, unsafe, or out of date
  • Lab or sandbox: code, checks, a new scenario
  • A divergence between the lab and the sandbox, closed or newly documented
  • Tests
  • Something else

If this changes advice or a command

Not advice, but it removes four endpoints and adds one, so everything below was
run rather than assumed.

OS: macOS 15 (Darwin 25.6.0), Docker 29.7.2 (build a7dcaa6)
tailscale: 1.102.4 in the containers; Headscale as pinned in the compose file
Go: golang:1.26-alpine, as pinned in lab/control/Dockerfile
Browser: Chromium via CDP, headless
Measured on: 2026-09-12, against the lab running on this machine

What I ran, and what it printed.

The four are gone and the one answers.

$ docker compose up -d --build --no-deps control
 Container control Started

$ for p in status tcpdump logs stats hud; do
    curl -s -o /dev/null -m 2 -w '%{http_code}\n' .../api/stream/$p; done
  /api/stream/status   -> 404
  /api/stream/tcpdump  -> 404
  /api/stream/logs     -> 404
  /api/stream/stats    -> 404
  /api/stream/hud      -> 200

One wire, with a real attack on it. Recording ?capture=evil-box while
POST /api/action {"id":"sniff"} ran from outside the recorder:

events : 904   seq 1 -> 904   gaps: none
  packet   : 880
  netcheck : 7
  stat     : 7
  status   : 4
  state    : 3
  hello    : 1
  note     : 1
  verdict  : 1

hello.sources : [state status netcheck stat verdict flow packet]
hello.capture : evil-box
note          : capture | evil-box # tcpdump -l -n -X -i eth1 'udp port 41641'
verdict       : id=sniff kind=action ok=true rung=2

A probe on the same wire arrives as flow, not verdict:

$ curl -s -X POST .../api/probe -d '{"from":"lab-ubuntu","to":"lab-vps","port":"22"}'
{"ok":true,"rung":5,"rule":"direct — both NATs held a mapping open and the punch
 completed: pong from lab-vps (100.71.4.3) via 203.0.113.11:41641 in 1ms", …}

on the feed:  {"seq":10,"type":"flow","data":{"id":"probe","kind":"probe",
              "result":{…rung 5…}}}

Every command the README now documents. All four, verbatim:

1. bare feed
   {"seq":1,…,"type":"hello","data":{"pollMs":3000,"sources":["state","status",
    "netcheck","stat","verdict","flow"],"types":[…ten…]}}
   {"seq":2,…,"type":"status","data":{"text":"lab-ubuntu $ tailscale status\n…"}}

2. ?capture=evil-box&iface=eth0&filter=icmp   (the overrides are honoured)
   note -> {"line":"evil-box # tcpdump -l -n -X -i eth0 'icmp'","source":"capture"}

3. ?logs=lab-ubuntu&logsAll=1
   note -> logs | lab-ubuntu # tail -f /var/log/tailscaled.log
   log lines: 40                              (unfiltered, as logsAll asks)

4. the attack.jsonl recipe
   $ curl -sN '…?capture=evil-box' | sed -n 's/^data: //p' > attack.jsonl
   lines: 1138      every line valid json: 1138      seq contiguous 1..n: true

In the page, driving it. Headless Chromium, the real buttons:

  on load            one connection      /api/stream/hud
                     opens 1, errors 0, dropped 0
                     hello.types         all ten, read off the server

  cap-on + logs-on   /api/stream/hud?capture=evil-box&logs=lab-ubuntu
  in one turn        opens 2             ← one reconnect for two toggles
                     cap-out first line  # evil-box # tcpdump … 'udp port 41641'
                     logs-out first line # lab-ubuntu # tail -f …tailscaled.log
                     cap has log banner  false     ← the note filter holding
                     logs has tcpdump    false
                     108 packet lines, 27 log lines

  both off           /api/stream/hud     ← released, in one reconnect (opens 3)

  click "sniff"      /api/stream/hud?capture=evil-box   ← the set-piece asked
                     hud verdict         "HELD · WireGuard transport data —
                                          sealed for a key evil-box does not have"
                     page verdict        "rung 2 of 5 — evil-box captured 120
                                          frames. The marker sent in the clear
                                          appears 1 time(s)…"

  click "replay"     /api/stream/hud     ← the previous shot's capture released

  console errors     none
                     (one pre-existing headless WebGL ReadPixels perf warning)
  • I checked whether any other chapter makes the same claim — the four
    endpoints were named in two places, lab/README.md's file tree and the
    API table in lab/design/five-gates.html; both moved with this
  • If this step can lock someone out of a machine, the warning sits before the step — n/a
  • The mechanism is explained, not just the command — lab/README.md gains a
    The feed section: the envelope, the ten types, what is always on and
    what is asked for, the drop policy, and the .jsonl recipe

Checks

  • make -C lab check passes — "Everything held."
  • gofmt is clean (make check fails on any output at all)
  • New source files carry an SPDX-License-Identifier header —
    feed.go, feed_test.go, ui/feed.js
  • ui/feed.js is registered in checks/syntax.mjs, so it is parse-checked
    like the rest: "All 10 files parse" (it was 9)

Nine new Go tests, none of which need a container: the hub's fan-out, that
unsubscribing twice is safe, that a reader which stops reading loses events
rather than blocking /api/action
, the envelope's field names pinned against
a literal, that a verdict carries the id of what ran and the whole Result,
that FeedTypes covers what the wire emits, that emit gives up on a cancelled
context, and that a note says which source it came from.

What those tests deliberately do not claim is anything needing Docker —
whether tailscale status comes back, whether tcpdump finds an interface. Those
are in the measurements above instead, because a test asserting them would be
asserting something it did not observe.

If this touches the guide's pages

It does not. Everything here is under lab/. assets/ and nav.js are
untouched.

If this touches the lab or sandbox engine

The engine is untouched: no probe, no attack, no rung and no Result changed.
The feed carries Result whole and adds nothing to it — a feed that summarised
it would be a second engine. Ticked because they were checked, not assumed:

  • Nothing is drawn that was not measured — the capture set-piece still draws
    one particle per real frame; it now asks the feed for the capture instead
    of opening its own EventSource, and the frames are the same frames
  • A failed attack does not render as a green tick — sniff above reads
    HELD at rung 2, unchanged
  • danger still outranks ok
  • Time is not faked — at is the server's own time.Now() per event, and it
    is there so a replay does not have to invent the spacing
  • The engine still returns the rung that decided the outcome, not a bare
    pass/fail — and the feed publishes that same Result, rung included
  • If a score moved, the comparison table in lab/README.md moved with it —
    no score moved; the 27 board-rule tests pass unchanged

Licence

  • I have the right to submit this, and I am fine with it going out under
    CC BY-SA 4.0 (guide) or GPL-3.0-or-later (code), per the file I touched

Not in this change

Kept out on purpose, one idea per branch:

  • Widgets and slots. Cards docked around the board, declared by a theme's
    manifest rather than by index.html. This is the branch that unblocks it —
    window.LabFeed is the seam a widget reads — and it is the next one.
  • Record and replay. attack.jsonl above is a real recording and nothing
    yet plays it back. The player, and the question of what a replay is allowed to
    drive, is its own branch.
  • The board reading verdicts off the feed. It still draws from the POST
    reply. Moving it would put the feed in the drawing path, and there is no
    reason to risk that in the branch that introduces the feed.
  • netcheck is sent every poll. It is diffed like status and state, but
    its text contains timings that change every run, so the diff never holds and a
    recording carries one every three seconds. Pre-existing — the old status
    stream did the same — and a real thing to fix, but not by this branch.

The live half of this server was four SSE endpoints — status, tcpdump, logs,
stats — and each consumer opened the one it wanted and parsed it for itself.
Four places that know how to read the lab, which is four things a widget
somebody else writes has to get right before it can draw anything, and four
files a recording would have to be stitched out of.

Now there is /api/stream/hud and one envelope: {seq, at, type, data}. Ten types,
listed in an opening hello so a consumer can ask rather than assume. Seq and the
timestamp are both there for replay — the order, and how long apart — and every
data: line is complete on its own, so a `.jsonl` of them is the recording with
no framing to reconstruct.

Two things carried that were not on a stream before. Every Result the API
returns is published as `verdict` (action, set, preset) or `flow` (probe), with
the id of what ran travelling beside it, because the set-piece a replay has to
pick is chosen by id and cannot be inferred from a Result's shape. The board
still draws from the POST reply, so the feed is not in the drawing path and
cannot break it; what it adds is that something which did not press the button
can see what the button did.

The two sources that cost a process inside a container are asked for in the URL
rather than turned on by a POST — ?capture=evil-box, ?logs=lab-ubuntu. No
subscription state on the server, and a recording whose URL says what was being
watched.

A note now says which source it came from. On four streams it could only have
come from the stream it arrived on; on one wire, a capture pane headed
`tail -f /var/log/tailscaled.log` is what dropping that field gets you.

On the page, ui/feed.js owns the one EventSource and the four call sites that
used to open their own now share it: on() to listen, want() to ask for an
expensive source — reference counted, so two things wanting the capture is one
capture — and status() for what the connection is doing. A classic script loaded
before app.js, because the readouts read the lab from their first line.

Nothing looks different. Measured against the running lab: 904 events on one
wire with no gaps in seq, the sniff verdict and a probe's flow both landing on
it, each text pane getting its own banner and not the other's, two toggles in
one turn costing one reconnect, and no console errors.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant