[cekees]

Groundwork for proteus#86, which keeps the .ymf current during a run. ymf merges first — proteus CI installs ymf from @main.

The problem

A writer producing one timestep at a time had to rewrite the whole document to record it: O(steps) per step, O(steps²) over a run. Measured on a one-collection archive — 0.6 ms at 10 steps, 6.8 ms at 100, 35 ms at 500, so a 500-step run spends 8.6 s in it and a 5,000-step run would spend minutes.

That requirement is inherited from XML, and YAML does not have it. A stream of documents separated by --- is valid after any complete document.

The change

• append_step(path, collection, step) — opens in append mode, writes one record, flushes. Optional fsync.
• read_ymf folds appended records back into their collections, so both shapes read identically: one document (what write_ymf produces, and what a finished run leaves), or that document followed by one per step (what a run in progress, or a killed one, leaves).

A record names its collection rather than indexing the header's list, because collections are created lazily — a model that first archives at step 40 adds one then, and an index agreed at step 0 would be wrong by then.

The part that needed care

A truncated YAML document usually still parses. Cutting inside a mapping leaves a shorter mapping, so completeness cannot be judged by parsing — an early draft here read a half-written step back as a real one, silently.

So each record ends with ..., YAML's explicit document-end marker, written last. Anything after the final marker is a partial write, dropped with a RuntimeWarning. Damage earlier in the file is raised rather than truncated, because treating it as a killed writer would hide real corruption.

Tests (10 new; suite 471 passed, 6 skipped)

• appending produces the same domain as writing the whole document;
• a collection appearing after the header;
• a kill at four cut points (mid-mapping, just after ---, a bare ---, and during the very first step before any marker exists) — each loses only the step in flight;
• a complete archive raises no warning, under simplefilter("error");
• corruption mid-file raises rather than silently truncating;
• the cost itself, asserted on bytes written per step — deterministic where timing is not. The spread is bounded at ≤8 bytes (the time value's text is wider at 10.0 than 0.0) while the file grows past 10× that, so the write does not grow with what the archive holds.

🤖 Generated with Claude Code

[cekees]

Noted on top of this (commit docs/archive-format.md), because this PR entrenches the current nesting:

Time and space may be nested the wrong way round. The document has one temporal collection per function space. That is XDMF's shape — a Temporal GridCollection belongs to a grid type. The transpose, one time collection with the spaces inside each instant, is likely the better model.

The objection is operator splitting, and it does not hold: a fractional-step calculation has synchronization points where every space is known, and those are the logical archive points anyway; where it does not, there are rational ways to write every space at the finest-grained step. More to the point, the archiving step is a separate choice from the numerical step — writing every space at every step it was approximated at is usually not affordable — so the archive already samples at instants, which is exactly what the transposed shape describes.

Relevant to this PR specifically: append_step has to name a collection in every record only because collections appear lazily as models first archive. Under the transposed shape that problem does not exist — one record per instant, no names needed. So this PR's design is partly a workaround for the nesting, and would get simpler if the nesting were inverted.

Not done here: it is a format version, it changes both ends of the reader and writer, and ymf2xmf would transpose on the way out.