Skip to content

feat(emit): report phase timings on stderr, so the sweep can be attributed - #23

Merged
chris-dare-dev merged 1 commit into
mainfrom
agent/emit-phase-timings
Sep 19, 2026
Merged

chris-dare-dev merged 1 commit into
mainfrom
agent/emit-phase-timings

Conversation

@chris-dare-dev

Copy link
Copy Markdown
Owner

The emitter is 3.85 min of a 20.7 min CI job on derived-alg-geo-lean, and it runs twice per job. Nothing said which part. Three plausible culprits were available and all three turned out to be wrong.

What the guesses were, and what they measure

guess reality
collectAxioms per constant is quadratic No. Since the exportedAxiomsExt extension, an imported constant's axiom closure is a binary search over pre-computed per-module entries — and this emitter's environment is entirely imported. axioms_ms 0 of 547.
Json.pretty 120 on a 38 MB document is expensive No. Timed standalone against that exact artifact at v4.32.1: pretty 120 206 ms, compress 204 ms. A 2 ms difference, 0.09% of the step.
— What is there: on this package's own 331-row library, pp_ms 422 and deps_ms 116 of a 547 ms sweep, inside an import_ms of 1825.

Two JSON lines on stderr, built with Json.mkObj so they are valid by construction:

mfc-emit-sweep:  {"rows":331,"sweep_ms":547,"sort_ms":0,"axioms_ms":0,"attrs_ms":5,
                  "pp_ms":422,"deps_ms":116,"cites_ms":0,"pp_chars":157926,
                  "axiom_refs":181,"dep_edges":725,"cite_refs":0}
mfc-emit-phases: {"modules":2275,"import_ms":1825,"emit_ms":692,"render_ms":0,
                  "write_ms":17,"chars":375969}

stderr and nowhere else

A timing field inside emission/1.0 would rotate emission_sha256, break the two-sweep determinism gate the first time a clock ticked differently between runs, and put a measurement of the machine inside a record about the repository. Telemetry and evidence do not share a file.

Forcing is the whole difficulty

A timer that does not force reads zero. ppExpr's result is lazy — the first draft of this reported the entire sweep at single-digit milliseconds and attributed the cost to Json.pretty in the driver, which is merely where the thunks were being demanded.

Every counter is paired with a size the report carries: String.length walks the UTF-8 bytes (utf8ByteSize would not — O(1) off the header), and Array.size demands the array qsort had to build. Reporting the sizes is also what stops the compiler eliminating the force as dead code. The forcing cost stays inside the number it forces.

The artifact does not move — verified, not asserted

Built at origin/main, emitted; rebuilt with this change, emitted again:

  • mfc_emit_selftest (12 constants, over a library this file is not part of): byte-identical modulo emitted_at.
  • mfc_emit_self (over MathFormalContract, which does contain this file): 37 constants added, 0 removed — every one a Timings projection or _default. Of the 294 in common, 13 differ, and all 13 are declarations in this file whose range moved.

No statement was reordered to fit a timer in. findDocString? stays where it was and contributes two disjoint windows to attrs_ms; moving it to make one clean window would have been a behaviour change hiding inside a measurement.

What this unblocks

Bumping the pin in a topic repo gives a per-run attribution of the emit step in the CI log, which is the input both to deciding whether parallelising the sweep is worth it and to the seconds{} field of a per-merge metrics row.

deps_ms is already suspicious at 21% of a 331-row sweep: sortedNames calls Name.toString inside the qsort comparator, so it rebuilds both strings on every comparison, and a def sorts three times (type, value, then the concatenation). That is a follow-up, with numbers, not a guess folded in here.

🤖 Generated with Claude Code

…buted

The emitter is 3.85 min of a 20.7 min CI job on derived-alg-geo-lean, twice
over, and nothing said which part. Three guesses were available and all three
were wrong in the same direction -- toward whatever was easiest to blame.

  * `collectAxioms` per constant looked quadratic. It is not: since the
    `exportedAxiomsExt` extension, an IMPORTED constant's axiom closure is a
    binary search over pre-computed per-module entries, and the emitter's
    environment is entirely imported. Measured here: `axioms_ms` 0 of 547.
  * `Json.pretty 120` on a 38 MB document looked expensive. It is not.
    Measured standalone against that exact artifact at v4.32.1, `pretty 120`
    takes 206 ms and `compress` 204 ms -- a 2 ms difference, 0.09% of the step.
  * What is actually there, on this package's own 331-row library:
    `pp_ms` 422 and `deps_ms` 116 of a 547 ms sweep, inside an `import_ms` of
    1825. Pretty-printing and dependency extraction, and on a small library the
    import dominates both.

So: measure first. Two JSON lines on stderr, `mfc-emit-sweep:` and
`mfc-emit-phases:`, built with `Json.mkObj` so they are valid by construction.

STDERR AND NOWHERE ELSE. A timing field inside `emission/1.0` would rotate
`emission_sha256`, break the two-sweep determinism gate the first time a clock
ticked differently between runs, and put a measurement of the MACHINE inside a
record about the REPOSITORY. Telemetry and evidence do not share a file.

FORCING IS THE WHOLE DIFFICULTY, and a timer that does not force reads zero.
`ppExpr`'s result is lazy: the first draft of this reported the entire sweep at
single-digit milliseconds and attributed the cost to `Json.pretty` in the
driver, which is merely where the thunks were being demanded. Every counter is
therefore paired with a size that the report carries -- `String.length` walks
the UTF-8 bytes (`utf8ByteSize` would not, being O(1) off the header), and
`Array.size` demands the array `qsort` had to build. Reporting the sizes is
also what stops the compiler eliminating the force as dead code, and the
forcing cost is left inside the number it forces rather than excluded.

THE ARTIFACT DOES NOT MOVE, verified rather than asserted. Built at
origin/main, emitted, rebuilt with this change, emitted again:

  * `mfc_emit_selftest` (12 constants, over a library this file is not part
    of): byte-identical modulo `emitted_at`.
  * `mfc_emit_self` (over `MathFormalContract`, which DOES contain this file):
    37 constants added, 0 removed, and every one of the 37 is a `Timings`
    projection or `_default`; of the 294 in common, 13 differ and all 13 are
    declarations in this file whose `range` moved. Nothing else changed.

No statement was reordered to fit a timer in. `findDocString?` stays where it
was and contributes two disjoint windows to `attrs_ms`, because moving it to
make one window would have been a behaviour change hiding inside a measurement.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chris-dare-dev
chris-dare-dev merged commit d365624 into main Sep 19, 2026
5 checks passed
@chris-dare-dev
chris-dare-dev deleted the agent/emit-phase-timings branch September 19, 2026 22:47
chris-dare-dev added a commit that referenced this pull request Sep 20, 2026
…#22)

derived-alg-geo-lean's combined emission died with `mfc-emit: maximum recursion
depth has been reached`. Deeply nested declaration values are finite input Lean
has already accepted; traversing and pretty-printing them exceeds
`Core.Context`'s default depth of 1000.

THE FIRST TWO ATTEMPTS LOOKED CORRECT AND DID NOTHING, and that is the part
worth keeping. Setting `maxRecDepth := 10000` and then `:= 0` on the emitter's
`Core.Context` had no effect on the call that overflows, because `emitOne`
renders every type and every def body inside `withOptions (fun _ => ppOpts)`,
and `Core.withOptions` rebuilds the field from the options it is handed:

    maxRecDepth := maxRecDepth.get options

`ppOpts` deliberately discards ambient options so a caller's `set_option`
cannot reach the emitted text, and that discarding restored the OPTION default
for exactly the `PrettyPrinter.ppExpr` call that needed the raised one.
Measured rather than assumed: `maxRecDepth.get ({} : Options)` is 512, while
the `Core.Context` field it replaces defaults to 1000. Entering the block did
not merely fail to raise the depth -- it lowered it.

The evidence that the second commit did not work is direct rather than
inferred: a derived-alg-geo-lean CI run whose log shows `MathFormalContract:
checking out revision 'b90d0d97'`, built fresh, still died the same way.

So `ppOpts` carries `maxRecDepth` too, set through the registered option's own
`set` rather than a quoted key, so it uses the same accessor `Core.withOptions`
reads it back with and the two cannot drift on a typo. The context-level
setting stays: it covers the traversal outside the pretty-printing block.
`ppOptsJson` gains the matching entry, because that record exists so the
artifact cannot claim a setting the emitter did not use; the emission schema
already admits integer `pp_options` values, so no schema change is needed.

Zero is Lean's documented no-limit sentinel and `withIncRecDepth` special-cases
it. Any finite bound would make input size a completeness ceiling. This removes
Lean's logical recursion guard and can still exhaust the native stack, but a
failure stays confined to this fresh emitter process rather than producing a
partial artifact.

REBASED ONTO THE TIMING INSTRUMENTATION (#23), and the order matters. That
change forces delaboration inside `rowJson` so the sweep's cost can be
attributed, which means the forced work runs under `ppOpts`'s depth -- the
exact setting this commit fixes. Landing these in the other order builds clean,
passes this package's own tests, and fails again at Mathlib scale.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant