Skip to content

feat(autodiff): expand an all-axis mean into matmuls against constants - #132

Open
AlekseiChirkovVention wants to merge 2 commits into
chore/generic-lowering-reference-consumerfrom
feat/matmul-based-mean-expansion
Open

feat(autodiff): expand an all-axis mean into matmuls against constants#132
AlekseiChirkovVention wants to merge 2 commits into
chore/generic-lowering-reference-consumerfrom
feat/matmul-based-mean-expansion

Conversation

@AlekseiChirkovVention

Copy link
Copy Markdown
Collaborator

Rewrite a supported all-axis rank-2 mean into matmuls against generated constants, so a
backend restricted to add, multiply, and matmul can lower the graph with no reduction handler
at all.

Goal

A sum over both axes of a rank-2 tensor is a matmul against a column of ones followed by a
matmul against a row of ones; the mean is that sum scaled by the reciprocal element count.
expand_mean_graph performs that rewrite, in both the rank-preserving form
(keepdims=True, five nodes) and the rank-reducing form (keepdims=False, six).

The rewrite is opt-in by being called. An artifact never passed through it lowers exactly
as it does today, and no existing entry point gained a flag, a keyword, or a branch.

Acceptance criteria

  • keepdims=True emits exactly the five specified nodes with the specified operators, operand
    lists, parameters, and declared types; the terminal node carries the mean's output value id
    and declared type.
  • keepdims=False emits exactly six; the sixth is a real reshape to rank 0, and the fifth
    declares [1, 1].
  • Every emitted node's declared type equals the shape computed independently from its
    operands' declared shapes and the operation's own rule.
  • Every emitted node is a constant, a matmul, a multiply, or — rank-reducing tier only — a
    reshape; every emitted matmul has exactly two operands.
  • Boundaries are preserved and no pre-existing value id is removed.
  • The input graph is unmutated, and equal graphs expand to equal graphs including identifiers
    and order.
  • An unsupported mean raises its categorized error naming the node and the clause it failed,
    and never yields a partially rewritten graph.
  • Unrelated nodes are carried through identically and the relative order of untouched nodes is
    preserved.
  • Minted identifiers come from a reserved namespace and a collision with an existing
    identifier is rejected.

Test evidence

Level: unit. Tests first: the failing set landed in d3f98d0, before the pass existed;
b8cbb98 made it pass. The test file is byte-identical between the two commits.

py/tests/test_autodiff_mean_expansion.py — 20 test functions, 51 cases, all passing. The
focused autodiff suite moved 741 → 792, exactly the 51 new items, so no pre-existing assertion
moved. Broad non-integration suite: 1043 passed, 9 failed, 1 skipped — the 9 are pre-existing
and unrelated to this branch (7 require the unbuilt local backend, 2 are style guards on a file
this branch does not touch), and their count is unchanged across the series.

Design notes for the reviewer

Declared types are derived, never copied. Every emitted node's declared type is computed
from its operands — matmuls through the shared matmul shape and dtype-compatibility helpers,
the scale from its operand, the reshape from its operand's dtype and the target shape, the
constant from its own descriptor. The validated mean's own declared type is deliberately never
read during emission. An earlier iteration of this design was rejected for emitting a node
whose declared shape its operation could not produce, and deriving rather than copying is what
structurally prevents that class of defect from recurring.

For the same reason the [1, 1] → scalar transition is a real reshape node, not an
elementwise multiply relabelled with a different rank. A scalar and a [1, 1] value are
different shapes and are never treated as interchangeable.

Nothing is emitted until every candidate is validated. All means in the graph are checked
first; the identifier minter is not even constructed until that completes. A rejected graph can
therefore never come back partially rewritten, by any path rather than merely in the cases the
tests cover.

The audit test recomputes shapes by hand. It writes out the matmul, multiply, and reshape
rules itself rather than calling the helpers the implementation uses, so a bug in a shared
helper cannot hide behind an audit that shares it.

Stack position

Stack position 3 of 8 — based on chore/generic-lowering-reference-consumer.

Cannot merge into main before feat/generated-constant-operator and
chore/generic-lowering-reference-consumer.

Known follow-ups, deliberately not fixed here

A structurally malformed hand-built artifact — a boundary entry that is not a pair, or a
declared type that is not a mapping — can still surface a bare builtin exception rather than a
categorized one. Unreachable from anything the tracer or the derivative transform produces. The
full failure matrix is a later branch in this series, and the same guard belongs there rather
than in three places.

Pin the rewrite of an all-axis rank-2 mean into two matmuls against generated
constants and a reciprocal-count scale, in both the rank-preserving and the
rank-reducing form, together with the eight-clause supported-mean predicate and
the category each failing clause reports.

The truthful-shape audit recomputes every emitted node's result from its
operands using rules written out in the test, rather than calling the shape
helpers the implementation uses. A declared shape must agree with an
independent computation, not merely with the pass itself -- an earlier design
was rejected for emitting a node whose declared shape its operation could not
produce, and that defect would survive an audit that shared the helper.
A backend restricted to add, multiply, and matmul has no reduction primitive to
map a mean onto. Rewrite a supported all-axis rank-2 mean into a row-sum matmul
against a column of generated ones, a column-sum matmul against a row of ones,
and a scale by the reciprocal element count, so such a backend can lower the
graph with no reduction handler at all.

The rewrite is opt-in by being called: an artifact never passed through it
lowers exactly as before, and no existing entry point gains a flag.

Every emitted node declares the dtype and shape its operation actually
produces, derived from its operands rather than copied. Where the mean declared
a rank-0 result, the rank change is performed by a real reshape node rather
than by an elementwise node claiming a shape it cannot produce, so a declared
shape is never a claim the graph does not honour.

Every candidate is validated before any node is emitted, so an unsupported mean
fails with a categorized error naming the node and the clause it failed, and
never yields a partially rewritten graph. Minted identifiers come from a
reserved, documented namespace and are checked against the artifact.
@AlekseiChirkovVention
AlekseiChirkovVention marked this pull request as ready for review August 28, 2026 06:18
@AlekseiChirkovVention AlekseiChirkovVention self-assigned this Aug 28, 2026
Comment thread py/tinychain/autodiff/expansion.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants