Skip to content

chore(autodiff): add a generic lowering reference consumer for tests - #131

Open
AlekseiChirkovVention wants to merge 2 commits into
feat/generated-constant-operatorfrom
chore/generic-lowering-reference-consumer
Open

chore(autodiff): add a generic lowering reference consumer for tests#131
AlekseiChirkovVention wants to merge 2 commits into
feat/generated-constant-operatorfrom
chore/generic-lowering-reference-consumer

Conversation

@AlekseiChirkovVention

Copy link
Copy Markdown
Collaborator

Add the single shared test-only backend that two features need to prove the lowering seam is
generic and that an expanded artifact computes what the unexpanded one does.

Goal

Create one generic reference consumer in the test tree, over the one dense-array execution
semantics that already exists, rather than letting each feature grow its own.

It supplies a handler registry limited to constants, matmul, and multiply; an opt-in variant
that adds trivial reshape; a reduction-capable control registry; and a wrapper that records
every handler invocation in call order.

Acceptance criteria

  • Exactly one shared generic reference consumer exists in the test tree afterwards, and the
    narrow existing proof of the lowering seam is left untouched.
  • The dense-array dispatcher executes a constant node to an array of the descriptor's shape,
    dtype, and value.
  • The limited-operation registry resolves handlers for exactly constants, matmul, and
    multiply, and rejects reshape, mean, broadcast, and division.
  • The opt-in variant additionally resolves reshape, and its handler rejects a reshape that
    changes the element count.
  • The control registry lowers mean, broadcast, and division to the same values the dispatcher
    produces for the same nodes.
  • The recording registry records every handler invocation in call order.

Test evidence

Level: unit. Tests first: the failing set landed in e4b2b05, before the module it
imports existed; 7946050 made it pass. The test file is byte-identical between the two.

py/tests/test_autodiff_reference_consumer_smoke.py — 16 test functions, 19 cases, all
passing. Full autodiff suite: 741 passed, 2 skipped. Broad non-integration suite: 992 passed,
9 failed, 1 skipped — the 9 are pre-existing and unrelated to this branch (7 require the
unbuilt local backend, 2 are style guards on a file this branch does not touch).

Design notes for the reviewer

Two properties here are load-bearing for later branches in this series, and both are easy to
get subtly wrong:

Every handler delegates to the existing dispatcher instead of restating operator meaning.
That is what keeps one execution semantics behind both the node-level executor and the
lowering-level registry. If the registries recomputed matmul or mean themselves, the two could
drift into agreeing on a wrong answer, and the equivalence proof that depends on this control
would quietly become worthless. The trivial-reshape handler is the one exception, and it only
adds an element-count check before delegating.

The invocation recorder records at handler call time, not at lookup time. A later branch
proves that a backend missing a required handler is rejected before any handler runs, by
asserting this record stayed empty. A lookup-time recorder would log entries in exactly the
case the proof needs empty, and that test would then pass by accident.

The constant-node dtype mapping is explicit and total over the two differentiable dtypes; an
unmapped dtype raises rather than widening to a default, for the same reason — a silent
widening would let a later numerical assertion pass for the wrong reason.

Stack position

Stack position 2 of 8 — based on feat/generated-constant-operator.

Cannot merge into main before feat/generated-constant-operator, whose constant operator
and descriptor reader this branch consumes.

Pin the shared test-only backend two features depend on: a handler registry
limited to constants, matmul, and multiply; an opt-in trivial-reshape variant;
a reduction-capable control registry; and a registry that records every handler
invocation.

The control registry is asserted against the existing dense-array dispatcher
rather than against hand-written expectations, so one execution semantics backs
both the node-level executor and the lowering-level registry and the two cannot
drift into agreeing on a wrong answer.
Two features need a backend that proves the lowering seam is generic and that
an expanded artifact computes what the unexpanded one does. Add one shared
test-only consumer rather than letting each grow its own.

It supplies a registry limited to constants, matmul, and multiply; an opt-in
variant adding trivial reshape; a reduction-capable control registry; and a
wrapper recording handler invocations in call order, which stays empty when
lowering fails closed before any handler runs.

Every handler delegates to the existing dense-array dispatcher, so one
execution semantics backs both the node-level executor and the lowering-level
registry and the two cannot drift into agreeing on a wrong answer. The
dispatcher itself gains constant-node execution, reading the node through the
shared descriptor reader. Its dtype mapping is explicit and total: an unmapped
dtype fails rather than widening to a guess.
@AlekseiChirkovVention
AlekseiChirkovVention marked this pull request as ready for review August 28, 2026 06:18
@AlekseiChirkovVention AlekseiChirkovVention self-assigned this Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants