Skip to content

Reduce SmallUndecidableGroups compilation memory - #714

Open
Vilin97 wants to merge 1 commit into
mainfrom
vilin/optimize-small-groups-20261023
Open

Vilin97 wants to merge 1 commit into
mainfrom
vilin/optimize-small-groups-20261023

Conversation

@Vilin97

@Vilin97 Vilin97 commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator

SmallUndecidableGroups repeatedly elaborates concrete HNN-model compositions and expands nested substitution words. This factors the common group arguments and separates independent public foundations from the model proofs. In controlled clean-project builds, median whole-build peak PSS falls 12.55% and simultaneous process-tree RSS falls 18.93%. Median wall time is 4.60% slower on the shared VM; the requested stopping target is met through RAM usage.

Selected project: SmallUndecidableGroups, originally 30,021 lines / 87 modules. Larger candidates including QuantumParallelRepetition and EhrhartVolumeInequality already had optimization PRs in progress. This project had no competing optimizer, and its baseline exposed substantial HNN elaboration costs: ZStage took 13 s, YStage 8.7 s, and Compression 6 s; the complete baseline reached about 2.1 GiB largest-child RSS.

  • Share subgroup-generator and HNN extensionality criteria, including transport through a model equivalence and the cyclic-base case.
  • Reuse one conjugated-word evaluation lemma for six generator identities, keeping the substituted words abstract.
  • Replace repeated inverse-image and composition calculations with direct equivalence and homomorphism equalities.
  • Move independent public foundations into Host/YStage/Definitions.lean and Host/ZStage/Definitions.lean. Existing modules publicly re-export them. Every original private declaration remains in its original module. The generated project Imports index includes both new files.

Base: edbe5bd56f476cc2965564676b66199f281ab714
Head: 36398932203f17bcb66f682af63191ee3adf4b28

Origin/main was fetched and verified unchanged immediately before committing. Uses main's existing Lean/Mathlib 4.35.0-rc3 pins, Mathlib c55e6e786f49471c72fbddbec5415808896aec1e. The project card, root index, pinned configuration and repository gates match base.

Invariant audit. Before editing, a Lean environment inspector imported all project modules with importAll := true and extensions disabled. It recorded exact expression DAGs for every constant's fully elaborated type and every non-theorem value, universe parameters, declaration kind, safety/partial flags, reducibility hints and inductive/constructor/recursor metadata. Only theorem proof values and expression source metadata are omitted. An independent environment-ownership walk checks complete coverage, including private and generated constants.

All 5,728 original inventory names match exactly, covering all 5,726 original project-owned constants: zero removals or type/value/universe/safety differences. The inventory has 5,774 base rows because some generated constants occur in multiple module memberships; it also records two imported generated equation helpers. Head has 5,779 rows / 5,733 names and covers all 5,731 owned constants. The five additions are three general extensionality lemmas, one generated proof helper and the private conjugated-word lemma. Expected ownership annotations change for moved public foundations. Every difference was inspected; generated constructor metadata stays in its original module. The post-measurement clean head passes the same audit.

Canonical original-declaration identity projection, excluding module ownership, is byte-identical: SHA-256 d4870f337af6a1460aefe8dfe922de335e1966daf42de41fd65dacdca28d9736. Base full-manifest SHA-256: 644a1d83d17d806683f27d70df950f3d6e755a88d5b34ea27a5ee9815270aeca; head: aaeb65a0394de1215e5aec8686c3e71c1774bd46fce8760de2c0c1e4d443b5b1.

Controlled measurements. Six sequential builds in base/head, head/base, base/head order on the Azure VM. Each deletes only selected-project Lean/IR artifacts, compiles all 87 base or 89 head modules, retains identical cached pinned dependencies, and verifies source SHA-256 inventories before and after. No other compiler or profiler from this task overlaps a measured build.

LEAN_NUM_THREADS=2 /usr/bin/time -v \
  lake --no-cache build --wfail LeanPool.SmallUndecidableGroups.Imports

A separate sampler follows descendants created by every parent thread and reads Linux smaps_rollup with a 50 ms delay between polls. It records the maximum simultaneous sum of RSS and PSS. PSS apportions shared resident pages. GNU time reports largest-child RSS separately. All memory values below are KiB.

Run Wall s User CPU s Largest-child RSS Tree RSS Tree PSS
base 1 115.14 202.32 2,208,692 4,054,852 2,832,472
head 1 120.84 199.96 2,035,840 3,232,772 2,504,414
head 2 116.48 198.16 2,049,688 3,377,108 2,523,841
base 2 114.39 202.13 2,187,624 3,922,608 2,886,066
base 3 115.92 202.87 2,176,532 3,987,876 2,893,417
head 3 120.44 199.51 2,066,128 3,139,172 2,541,931
base median 115.14 202.32 2,187,624 3,987,876 2,886,066
head median 120.44 199.51 2,049,688 3,232,772 2,523,841

Observed median reductions: 12.55% tree PSS, 18.93% tree RSS, 6.31% largest-child RSS and 1.39% user CPU. Wall time increases 115.14 → 120.44 s. This VM is shared, with no CPU isolation or OS-cache flushing; memory peaks depend partly on parallel module overlap. These are selected-project measurements, with no whole-pool speedup claim.

Validation.

  • Six warning-failing clean builds, plus the coherent final head and fixed +LeanPool root build.
  • Strict post-build declaration-manifest comparison and complete ownership audit; measured source inventories match the committed tree.
  • Unmodified scripts/ci/lint-project.lean over all 89 modules / 5,731 declarations, including slow Mathlib/Batteries linters: passed.
  • lake exe lint-style LeanPool.SmallUndecidableGroups.Imports: passed.
  • Unmodified python -m lean_pool.quality --repo <project-scoped validation view>: passed, including allowed axioms, compiled option/soundness audits, card, reachability, headers and size checks. The view contains this project's complete sources/card and unchanged pinned configuration/root, using its built artifacts.
  • lake exe mk_all --module --check on that view and git diff --check: passed.
  • lake shake --force --keep-public LeanPool.SmallUndecidableGroups.Imports: exit 1 with byte-identical pre-existing redundant-import advisories compared with base. Existing public coverage is preserved.

Hosted whole-pool CI and independent PR review remain pending. Open for review only; this cycle does not request queue entry, auto-merge or merge.

@greptile-apps

greptile-apps Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Advisory cross-file review against Lean Pool's supervisory rubric. This is not by itself a significance, source-verification, or merge verdict.

RetriggerView in Greptile

[Medium impact] No blocking defect was found in the changed proofs or module split.

Summary

Factors repeated group arguments into shared lemmas and moves public foundations into two smaller modules.

  • Path-rotation proofs reuse shared checks for HNN model maps.
  • Compression checks six conjugation identities without expanding their words.
  • Y- and Z-stage foundations now compile in their own files.

Reviews (1) · Last reviewed commit: "Reduce SmallUndecidableGroups build memo..." · Reviewed by Greptile

@github-actions

Copy link
Copy Markdown
Contributor

Proof profile (new / modified Lean files)

lake build wall time (changed modules): 49.21 s (= 0.82 min) — user 81.30 s, sys 10.33 s.

This build covers the added modules and their dependency cones on top of the measurement environment; modified files are measured without rebuilding the pool.

Statement changes

✅ No statements changed — all 368 tracked declarations across 5 modified files kept their type; only proofs / definition bodies changed.

Also: 4 declarations added, 85 removed. A reorganization moves declarations between files, which shows up here as an add in one file and a remove in another.

➕ Declarations added (4)
  • Undecidability.HNNLemmas.closure_hom_ext — LeanPool/SmallUndecidableGroups/GroupTheory/HNNLemmas.lean
  • Undecidability.HNNLemmas.cyclic_hom_ext_of_equiv — LeanPool/SmallUndecidableGroups/GroupTheory/HNNLemmas.lean
  • Undecidability.HNNLemmas.hom_ext_of_equiv — LeanPool/SmallUndecidableGroups/GroupTheory/HNNLemmas.lean
  • Undecidability.Host.Compression.eval_conjugated_word — LeanPool/SmallUndecidableGroups/Host/Compression.lean
➖ Declarations removed (85)
  • Undecidability.HostYStage.CRestricted — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.CRestrictedA — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.CRestrictedB — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.CSubgroup — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.FreeInput — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.Gamma — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.HMinus — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.HPlus — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.c — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.cRestrictedPhi — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.cRestrictedRangeEquiv — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.c_mem_HMinus — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.c_mem_HPlus — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.coreC — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.coreD — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.coreE — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.coreStable — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.d — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.d3_mem_J3 — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.d_mem_HMinus — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.d_mem_HPlus — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.e — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.e3_mem_J3 — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.e_mem_HMinus — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.e_mem_HPlus — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.k — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.k_mem_HPlus — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.minusC — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.minusD — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.minusE — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.minusS1 — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.minusT — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.plusC — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.plusD — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.plusE — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.plusK — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.plusS2 — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.s1 — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.s1_mem_HMinus — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.s2 — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.s2_mem_HPlus — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.stable3_mem_J3 — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.t — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.t_mem_HMinus — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.twistGenerator — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostYStage.untwistGenerator — LeanPool/SmallUndecidableGroups/Host/YStage.lean
  • Undecidability.HostZStage.Y — LeanPool/SmallUndecidableGroups/Host/ZStage.lean
  • Undecidability.HostZStage.cY — LeanPool/SmallUndecidableGroups/Host/ZStage.lean
  • Undecidability.HostZStage.dY — LeanPool/SmallUndecidableGroups/Host/ZStage.lean
  • Undecidability.HostZStage.tY — LeanPool/SmallUndecidableGroups/Host/ZStage.lean
  • … +35 more

Statements are compared textually against the merge base (comments and whitespace ignored; by proof terms embedded in a type are treated as proof-irrelevant), the same method as physlib's /check-golf. Anonymous instances and examples are not tracked.

Compile cost of modified files — base → head

Heartbeats: 361,984 → 316,643 (-45,341, -12.5%). Commands: 119 → 123.

Per-file wall sum: 67.4 s → 44.1 s (-34.5%). Wall clock is measured under parallel load and is noisy; heartbeats are deterministic and are the regression signal.

Files: 2 compared — 1 cheaper, 1 costlier, 0 within noise (threshold: |Δ heartbeats| ≥ 1,000 and ≥ 1%).

Both sides are elaborated against the merge-base build of the pool — sound when only proofs and definition bodies change; a file whose statements changed may fail there and is flagged ⚠. Heartbeats come from disjoint root command traces with trace.profiler.useHeartbeats and synchronous elaboration, converted to maxHeartbeats units. Failed measurements are not classified as cheaper or costlier. Δ lines counts added/deleted lines from this PR diff.

File Δ lines HB base HB head Δ HB Δ HB % Wall base (s) Wall head (s)
LeanPool/SmallUndecidableGroups/Host/Compression.lean +16/-6 162,548 111,624 -50,923 -31.3% 41.21 18.54
LeanPool/SmallUndecidableGroups/GroupTheory/HNNLemmas.lean +44/-0 199,437 205,019 +5,582 +2.8% 26.15 25.56
Total +60/-6 361,984 316,643 -45,341 -12.5% 67.4 44.1

New files — absolute profile

Total heartbeats: 34,371 maxHeartbeats units across 2 valid files (524 added LOC).

Sum of lean --profile: 5125.3 ms (= 5.13 s) across 2 valid files. Import-excluded time: 2849.3 ms (= 2.85 s).

Count-heartbeats wall-clock total: 6.99 s. Repeated import cost inside lean --profile: 2276.0 ms (= 2.28 s).

Heartbeat values sum disjoint root command traces with trace.profiler.useHeartbeats and synchronous elaboration, converted from raw allocations to maxHeartbeats units by dividing by 1000. They include command elaboration and kernel checking, exclude imports, and include profiler overhead; they are not comparable to the deprecated linter's totals. Per-file wall clocks are measured under parallel load and are noisier than heartbeats.

LOC counts added lines in the profiled Lean files from this PR diff.

File LOC Heartbeats (maxHB) Count wall (s) lean --profile (s) Without import (s) Import (s) Commands Errors
LeanPool/SmallUndecidableGroups/Host/ZStage/Definitions.lean 250 24,439 4.00 2.74 1.87 0.88 62 0
LeanPool/SmallUndecidableGroups/Host/YStage/Definitions.lean 194 9,932 2.99 2.38 0.98 1.40 63 0
LeanPool/SmallUndecidableGroups/Host/YStage.lean 1 — — — — — 0 1
LeanPool/SmallUndecidableGroups/Host/ZStage.lean 77 — — — — — 0 1
LeanPool/SmallUndecidableGroups/Imports.lean 2 — — — — — 0 1
Total 524 34,371 6.99 5.13 2.85 2.28 125 3

Aggregate phase totals

Phase Time
import 2276.0 ms (= 2.28 s)
attribute application 852.0 ms (= 0.85 s)
interpretation 576.0 ms (= 0.58 s)
simp 426.1 ms (= 0.43 s)
elaboration 353.0 ms (= 0.35 s)
typeclass inference 205.0 ms (= 0.20 s)
type checking 128.4 ms (= 0.13 s)
tacticAnalysis 70.6 ms (= 0.07 s)
initialization 61.0 ms (= 0.06 s)
linting 44.9 ms (= 0.04 s)
process pre-definitions 42.1 ms (= 0.04 s)
parsing 32.1 ms (= 0.03 s)

Slowest changed modules (from lake build)

Changed module Lake time
LeanPool.SmallUndecidableGroups.GroupTheory.HNNLemmas 6.70 s
LeanPool.SmallUndecidableGroups.Host.YStage 6.70 s
LeanPool.SmallUndecidableGroups.Host.ZStage.Definitions 2.10 s
LeanPool.SmallUndecidableGroups.Host.YStage.Definitions 1.80 s
Per-file `lean --profile` output

LeanPool/SmallUndecidableGroups/Host/YStage/Definitions.lean

import took 1.4s
cumulative profiling times:
	attribute application 235ms
	congr simp thm 0.897ms
	elaboration 176ms
	fix level params 0.915ms
	import 1.4s
	initialization 31.2ms
	instantiate metavars 0.294ms
	interpretation 301ms
	let-to-have transformation 0.72ms
	linting 22.5ms
	module linting 0.00194ms
	overlappingInstancesLinter 7.42ms
	parsing 16.5ms
	process pre-definitions 19ms
	share common exprs 3.31ms
	simp 18.1ms
	tactic execution 0.608ms
	tacticAnalysis 34.9ms
	type checking 54.5ms
	typeclass inference 60ms
real 2.72
user 1.79
sys 0.97

LeanPool/SmallUndecidableGroups/Host/ZStage/Definitions.lean

import took 876ms
attribute application took 219ms
attribute application took 168ms
cumulative profiling times:
	attribute application 617ms
	congr simp thm 1.92ms
	elaboration 177ms
	fix level params 1.14ms
	import 876ms
	initialization 29.8ms
	instantiate metavars 0.331ms
	interpretation 275ms
	let-to-have transformation 0.973ms
	linting 22.4ms
	module linting 0.00139ms
	overlappingInstancesLinter 7.45ms
	parsing 15.6ms
	process pre-definitions 23.1ms
	share common exprs 5.4ms
	simp 408ms
	tactic execution 26.7ms
	tacticAnalysis 35.7ms
	type checking 73.9ms
	typeclass inference 145ms
real 2.67
user 2.68
sys 0.88

Advisory only — never blocks merge. Full log uploaded as the proof-profile artifact.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant