Skip to content

Docstring diet: private code keeps prose, not sections; one-line properties - #766

Merged
jtravs merged 57 commits into
mainfrom
chore/docstring-diet
Nov 12, 2026
Merged

jtravs merged 57 commits into
mainfrom
chore/docstring-diet

Conversation

@jtravs

@jtravs jtravs commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Stacked on #763 (base chore/lean-rules). This applies the docstring rule that #763 wrote into AGENTS.md. Only docstrings, comments and the names of nested functions change, plus the size ratchets. One later change is planned: removing the single-precision code under ADR-077 deletes backends/numpy_ref/single.py, numpy_ref/transforms.py and translate/precision.py, so this PR leaves those three files alone.

The rule (AGENTS.md, "Docstrings and comments")

  • Private and nested code keeps no Parameters, Returns, Yields, Raises, Examples or See Also sections. It keeps the summary line, and then only what the signature and annotations don't say: a non-SI unit, a valid range, a condition, a convention, or a reason.
  • A public property has a one-line docstring that names the quantity and its unit, for example """Peak power, W.""", with # numpydoc ignore=RT01 on its def line. Prose of real value (a convention, a condition, a reason, a reference) stays as a following paragraph. A Returns block that restates the type or the summary goes.

How it was done

Private docstrings. tools/docstring_diet.py was added in the first commit and deleted in the last. It ran package by package with --write --check --report --rename-closures.

  • A definition counts as private when its numpydoc name (module stem plus qualname) has a single-underscore component, or when it is nested in a function. A class or function that a public module re-exports in __all__ counts as public API.
  • For private definitions it removed the sections listed above. Any entry that stated a unit, range, condition, convention or shape became a one-line bullet. The summary, the extended summary, Notes and References were kept verbatim.
  • --check compiled each file and compared its AST, with docstrings removed, before and after. Each package's commit message records the command it ran. Summaries the tool skipped (D200) were shortened by hand in the same commits.

Public properties. A throwaway script (ast + numpydoc.docscrape, not committed) handled 540 properties in two commits (Scan's 3 were done by hand, 543 in all):

  • a mechanical pass: the summary becomes a noun phrase, Returns goes, its description stays as a paragraph, and the pragma is added;
  • a hand-review pass over every entry: a unit, range or shape moves into the summary, a description that only restates goes, and everything else stays.

The same AST comparison passed on all 105 files the property commits touched.

About the pragma. numpydoc attaches a # numpydoc ignore= comment to the last def or class token before it (numpydoc/validate.py, extract_ignore_validation_comments). Ruff's E501 counts the comment, because it is not a pragma ruff knows. Where the comment would push a line past 88 columns, ruff format wraps the signature, which leaves the pragma on the ) -> T: line. numpydoc still honours it there. This happens for 25 properties. No noqa: E501 was added.

Summaries and SS05. numpydoc's SS05 heuristic applies to properties. A summary whose first word ends in a single "s" ("Positions …", "Fields …") therefore keeps a leading "The".

Lines (git diff --shortstat 256c8e27 HEAD -- <path>)

path files + −
src/solprop/*.py (top level) 27 1,325 6,600
backends 26 710 4,024
calibration 8 114 848
coupling 7 275 2,027
driver 20 1,057 6,213
modes 17 796 3,106
nonlinear 10 436 1,769
output 12 413 2,698
physdata 10 314 1,404
plotting 5 62 390
processing 12 272 1,742
recipe 10 176 1,627
service 23 202 1,367
stepper 6 95 298
translate 28 1,039 5,346
src/solprop 221 7,286 39,459
native/src 21 283 2,010
whole PR 248 7,632 41,477

The public-property commits account for 1,717 + / 3,810 − of the src/solprop figures.

Restored by hand

Each package's "Restore …" commit puts back what the tool dropped and the code does not say:

  • translate: the coupled/include_area semantics of _channel_index_table, where the loss policy sits before integration, the mode attributes _dispersion_phi reads, how the plasma band rule's taps are normalised, the C-order flattening of expansion products, the [1.0] constant prefactor weight, zero-off-band generators, and the reasons behind _state's rank check, _degree_axis, the general flow's gauge identity and the binary32 range.
  • backends: the periodic-row hooks (release or retain, attempt index including rejections, drains), the norm recorded for clipped and diverged attempts, resume by replay only at schedule edges, the Adams trial's frame and norm scale, the exponential compensation's chord exponent and coupled residuals, the flow-squaring limit shared with the compiled backends, the units of the ungated current and the plasma current density, and a few reasons.
  • modes, coupling, nonlinear, physdata, solprop_native: conventions, ranges, flags, layouts, refusal reasons and warning policies.
  • top level, processing, driver, output, service: refusal reasons and hazards, the zero-frequency limit rule, resume and retention notes, unit notes, and why the scan reader is picklable.
  • Capitals the bullets had lowered: Cartesian, Chebyshev, Hermitian, Jones, Stokes, Taylor, and the fibre frequency V.
  • Public properties (all 540 from the script were reviewed one by one, physics packages included): long conventions stay as paragraphs, for example the capillary and slab channel_label parity rules, azimuthal_order bounds, wall-truncation constants, Parseval weights and the scale_invariant reasons. ConstantDispersion.is_constant had a summary that said "varies"; it now reads "Whether the path is constant along z: always True."

Closures renamed

--rename-closures added a leading underscore to 44 nested functions in public functions, because numpydoc would otherwise validate them as public: at, element, residual, walk, half_bins, samples, rates, correction, sampled, material, at_parameters, read, column and others, in 27 modules. By hand, solve_pin's nested build became _trial, because _build would have clashed with the module's _build.

Private doctests converted to unit tests

Dropped private Examples that carried real checks moved into the mirrored test modules:

  • driver.run._history_of → tests/driver/test_run.py
  • output.schema._inlined, _array_name, _dotted, _flow_text → tests/output/test_schema.py
  • service.machines._development → tests/service/test_agent.py
  • service.results._scan_axis → tests/service/test_results.py
  • engineering notation's None and NaN text → tests/test_display.py

Ratchets (tools/size_budget.toml)

  • Private sectioned docstrings: from 2,895 to 37. Of those, 25 are in the three files that the single-precision removal deletes. The other 12 are public API that is defined in a private module and re-exported, and they keep their sections on purpose: ThreadBudget.explicit, thread_budget, current_executor, Scan with __repr__, __len__, add, point, points, subset and chunk, and example_output. The ratchet uses numpydoc's naming rule, so it counts them as private. Scan's three properties follow the property rule.
  • Module caps tightened to the new sizes: simulation.py 10,741, problem.py 5,701, schema.py 5,073. tools.py (3,613) and spatial.py (3,686) are now under the 4,000-line cap, so their entries are removed.
  • Private imports across packages: unchanged at 88.

Size report (uv run poe size, src/solprop)

Before (256c8e2, the head of #763):

package      files   total    code  pub doc  priv doc  comment  blank  __all__  sect priv  doc>body
(top-level)     32   42029   16998    14189      7675      668   2499      724        484       770
backends        35   31494   13330     9895      5887      744   1638      315        335       596
calibration     10    5145    2644     1010      1058       78    355       67         67        60
coupling        10   11133    5251     2720      2479       99    584      128        131       152
driver          29   31743   15463     6128      8109      586   1457      170        472       396
modes           18   21840    9298     7752      3312      446   1032      238        212       385
nonlinear       11   13818    5309     5853      1803      162    691      164        110       231
output          14   15072    6502     4417      3016      240    897      157        222       290
physdata        13   10586    3894     4398      1493      134    667      171         99       218
plotting         6    2812    1179      926       479       41    187       76         33        51
processing      14   10439    4141     3402      2185      153    558      214        116       181
recipe          12   11045    6054     2362      1863      120    646      121        126       150
service         28   11296    5209     3503      1578      155    851      176        107       187
stepper          7    4037    1481     2060       172      101    223       96         11        67
translate       31   39969   18883    11914      6814      577   1781      438        370       476
total          270  262458  115636    80529     47923     4304  14066     3255       2895      4210

After:

package      files   total    code  pub doc  priv doc  comment  blank  __all__  sect priv  doc>body
(top-level)     32   36754   17022    13674      2891      668   2499      724          2       494
backends        35   28180   13338     9709      2751      744   1638      315         24       471
calibration     10    4411    2644     1007       327       78    355       67          0        33
coupling        10    9381    5253     2654       791       99    584      128          0        82
driver          29   26587   15463     6077      3004      586   1457      170          9       193
modes           18   19530    9304     7373      1375      446   1032      238          0       259
nonlinear       11   12485    5315     5635       682      162    691      164          0       182
output          14   12787    6502     4216       932      240    897      157          0       163
physdata        13    9496    3894     4263       538      134    667      171          0       155
plotting         6    2484    1179      926       151       41    187       76          0        31
processing      14    8969    4143     3369       746      153    558      214          1       120
recipe          12    9594    6054     2353       421      120    646      121          0        76
service         28   10131    5209     3482       434      155    851      176          0       126
stepper          7    3834    1483     1970        57      101    223       96          0        55
translate       31   35662   18889    11686      2729      577   1781      438          1       331
total          270  230285  115692    78394     17829     4304  14066     3255         37      2771

The total falls by 32,173 lines (−12.3 %), and private docstring lines fall from 47,923 to 17,829. The code column rises by 56 because ruff format wrapped signatures to fit the pragma. The noqa counts, cross-package private imports (88) and clone figures are unchanged.

Verification

  • An AST comparison with docstrings removed, for every file the property commits touched: 105 files, 0 changed. The _scan_model edit was checked the same way. The earlier package commits ran the tool's own --check.
  • These pass locally: uv run poe docstrings, ruff check ., ruff format --check ., codespell src, poe changelog-check, poe reuse and poe size-check.
  • The doctests of every touched source file pass (pytest -n 2 -p no:randomly <files>: 747 passed).
  • Nothing in the repository referenced tools/docstring_diet.py before it was deleted.
  • The full suite, basedpyright and poe docs were not run locally. CI runs them.

Leftovers

  • pulses.SpatialPulse.aligned returns True when the beam is displaced or tilted, so the name reads backwards. The docstring describes the code as it is. Code was out of scope here.
  • Re-exported public API in private modules. The 12 entries above stay under the ratchet until it learns about __all__ re-exports or the names move to public modules.
  • Wrapped signatures. 25 properties have signatures wrapped only to fit the pragma. translate/modal_path.py: ModalPathTable.positions reads worst.

🤖 Generated with Claude Code

jtravs and others added 30 commits November 12, 2026 00:07
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mechanical: python tools/docstring_diet.py src/solprop/stepper --write --check --report --rename-closures

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mechanical: python tools/docstring_diet.py src/solprop/calibration --write --check --report --rename-closures

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mechanical: python tools/docstring_diet.py src/solprop/modes --write --check --report --rename-closures

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mechanical: python tools/docstring_diet.py $(ls src/solprop/translate/*.py | grep -v precision.py) --write --check --report --rename-closures

precision.py is left out (it goes in the next PR). The two D200 skips
(PropagationProblem._check_spatial_collar_stability,
SeparableOperator._adopt_user_rate) are rewritten by hand.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mechanical: uv run python tools/docstring_diet.py src/solprop/driver --write --check --report --rename-closures

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mechanical: python tools/docstring_diet.py src/solprop/*.py --write --check --report --rename-closures

The three summaries the tool skipped (D200) are shortened by hand:
log.LogReporter._say, pulses._analytic, transverse_monitor._insensitive_reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ate docs

The slimming dropped some Raises and Returns prose that said why a value is
refused or what a result means: the reference length's NaN and amplification
hazards, the absorber's no-gain rule, the exit status after a late signal, the
Taylor phase units, the aligner's refusals, why identities use float.hex, why
between-plane statistics are refused at construction, the negative-fluence
refusal, the aperture weight's zeros, the mixture-flow refusal, the Gaussian
order range and why radial reads skip unused rows. Stokes is capitalised again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mechanical: python tools/docstring_diet.py src/solprop/processing --write --check --report --rename-closures

The two summaries the tool skipped (D200) are shortened by hand:
planes._source_part, scans._Builtin.reads_fields.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ate docs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mechanical: python tools/docstring_diet.py src/solprop/coupling --write --check --report --rename-closures

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The slimming dropped the note that a frequency limit may be zero, as the lower
edge of a low-pass band, while a wavelength limit must be positive.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mechanical: python tools/docstring_diet.py src/solprop/plotting --write --check --report --rename-closures

The summary the tool skipped (D200) is shortened by hand: geometry._planar_grid.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mechanical: uv run python tools/docstring_diet.py src/solprop/output --write --check --report --rename-closures

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…utput private docs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mechanical: uv run python tools/docstring_diet.py src/solprop/service --write --check --report --rename-closures

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mechanical: python tools/docstring_diet.py src/solprop/nonlinear --write --check --report --rename-closures

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mechanical: uv run python tools/docstring_diet.py src/solprop/recipe --write --check --report --rename-closures

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hand

The tool kept solve_pin's nested build, whose underscore name clashes with the
module's _build; it is now _trial.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The slimming dropped a few entries the code does not say: the
_channel_index_table coupled/include_area semantics, the loss policy's
place before integration, the mode attributes _dispersion_phi reads, the
taps' normalisation of the plasma band rule, the C-order flattening of the
expansion products, the [1.0] constant prefactor weight, zero-off-band
generators, and a few whys (_state's rank check, _degree_axis, the general
flow's gauge identity, binary32 range). Also completes the tuple returns
of _split/_split_segments and recapitalises Chebyshev, Cartesian and E.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mechanical: python tools/docstring_diet.py $(find src/solprop/backends -name '*.py' | grep -v -e numpy_ref/single.py -e numpy_ref/transforms.py) --write --check --report --rename-closures

numpy_ref/single.py and numpy_ref/transforms.py are left out (they go in
the next PR). The D200 skip (numpy_ref.tdse._roundoff_terms) is rewritten
by hand.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…docs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mechanical: python tools/docstring_diet.py src/solprop/physdata --write --check --report --rename-closures
By hand: _warn_dilute_scaling's sections (D200 skip) and one f-string reflowed
after the _span rename lengthened its line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…docs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jtravs and others added 9 commits November 12, 2026 00:28
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cartesian, Hermitian, Jones and the fibre frequency V.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…limming

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mechanical pass: each public property's summary becomes a noun phrase,
its Returns section goes, the description stays as a paragraph, and the
def line carries `# numpydoc ignore=RT01`. The AST without docstrings is
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hand review of the mechanical pass: a unit, range or shape moves into the
one-line summary, a description that only restates the summary or the
type goes, and conventions, conditions and reasons stay as a paragraph.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Private sectioned docstrings fall to 37: re-exported public API defined
in a private module, and the single-precision reference modules. The
module caps follow the shrunken files; tools.py and spatial.py are now
below the general cap. Scan's three properties take the property rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jtravs jtravs added documentation Improvements or additions to documentation refactor Internal restructuring with no change in behaviour labels Nov 12, 2026
@read-the-docs-community

read-the-docs-community Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# Conflicts:
#	tools/size_budget.toml
Base automatically changed from chore/lean-rules to main November 12, 2026 05:29
@codecov

codecov Bot commented Nov 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

jtravs and others added 9 commits November 12, 2026 05:34
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# Conflicts:
#	native/src/solprop_native/session.py
#	src/solprop/backends/host/driver.py
#	src/solprop/driver/executors/_queue_state.py
#	src/solprop/driver/simulation.py
#	src/solprop/service/record.py
#	src/solprop/translate/envelope_plasma.py
#	src/solprop/translate/expansion.py
#	src/solprop/translate/linop.py
#	src/solprop/translate/problem.py
#	src/solprop/translate/radial.py
#	src/solprop/translate/responses.py
#	src/solprop/translate/spatial.py
Thirty-five Returns and parameter bullets named nothing beyond the type:
no unit, range, condition, convention or structure. Every other bullet
says at least one of those, so it stays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fifteen properties of NativeBackend, NativeSession, NativePrepared and the
group classes now match their host twins: a noun-phrase summary with
numpydoc ignore=RT01 on the def line, in place of a Returns section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stubs of Mode, ModalProvider, RadialPermittivity, PermittivityLaw,
ModelIdentityLike, the session protocols, DispersionPath and ZProfile read
"Return ..."; they now name the quantity as their implementations do,
and every public stub carries numpydoc ignore=RT01 on its def line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The diet dropped the doctests of _even_chunk and tail, which no test
covered; the cases now live in the unit tests, with the other _display
examples. tail's docstring again says that the width counts the ellipsis.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The diet renamed it to _nonlinear, which changed the __qualname__ callers
see. It is the only renamed closure that reaches a caller: the others are
called in place or passed to a solver. Its def line ignores PR01 and RT01,
with the reason, and its docstring keeps the state's unit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AGENTS.md said every nested function's name starts with one, which 82 in
src/solprop do not and a returned closure must not. A nested function is
private, but numpydoc checks one without the underscore, so one with
parameters or a return value takes it unless callers receive it.
tooling.md says the same of the size report, whose private rule is
numpydoc's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"Positions", "Overrides" and "Its" read to numpydoc as third-person
verbs (SS05).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jtravs
jtravs marked this pull request as ready for review November 12, 2026 07:22
@jtravs
jtravs merged commit 2a34e16 into main Nov 12, 2026
42 checks passed
@jtravs
jtravs deleted the chore/docstring-diet branch November 12, 2026 07:23
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-12T07:28:04.077600Z a2a7b7b Draft marked ready
🔒 Security Review ✅ Completed 2026-11-12T07:25:22.772039Z a2a7b7b Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation refactor Internal restructuring with no change in behaviour

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant