Differentiation and canonical ordering visit each shared subexpression once

[ofloveandhate]

Fixes #505.

A System's first evaluation builds its Jacobian, and for systems whose functions share subexpressions heavily -- a determinant built by cofactor expansion, the minors deflation appends -- that cost time proportional to the expanded tree rather than the DAG: 24.6 s for a 7 x 7 determinant of quadratics in three variables. Two things were proportional to the expansion, and a third to the square of the DAG.

Differentiation had no memo. SumOperator::Differentiate and MultOperator::Differentiate differentiated each operand, so a subexpression reached by many paths was differentiated once per path. Node::Differentiate is now non-virtual and keeps each derivative for the length of the outermost call; a node type's rule is DifferentiateImpl. A node::DerivativeScope extends the memo across a loop, and both Jacobian builds open one, so functions that share subexpressions differentiate each once among them.

Degree(v) walked the expansion. MultiDegree has been memoized per call since #418, but a unary operator's multidegree is computed from its operand's Degree(v), which was not, so every construction above a negation walked the whole expansion beneath it.

Building a sum or product walked its operands. Canonical ordering asks each operand its variables, its multidegree and, on a tie, its canonical encoding, and each answer was a walk of the operand's DAG -- O(DAG) per construction, O(DAG^2) to build an expression. Each node now keeps a degree record, made once from its operands' records: its variables, its degree and multidegree in each variable it mentions, its total degree, and one answer for every variable it does not mention. Degree, MultiDegree and GatherVariables read it.

The tie-break is a content key. The full canonical encoding cannot be made from the operands' encodings -- its back-references number the whole tree -- so even computed once per node it walked each node's DAG. Operands that tie on multidegree are now ordered by a content key: the SHA-256 of the node's shallow canonical encoding, its own kind, payload and operands in the canonical format with each operand written as @ and that operand's key. A Merkle hash, made once per node from the operands' keys.

The answer for an unmentioned variable is not always 0 -- a power with a negative constant exponent reports -1 in every variable -- so the record computes it, by asking the degree rules about a variable that is in no expression (default-constructed, never interned). The rules compare a variable only by identity with the variables a subtree mentions, so that one answer is every unmentioned variable's. Records and keys are published once by atomic compare-and-swap, since systems compile lazily on worker threads.

Measured

First evaluation of a k x k determinant of dense quadratics in three variables, sub-minors shared:

k	before	after
5	0.42 s	0.009 s
6	3.1 s	0.016 s
7	24.6 s	0.042 s

The growth per step is now about that of the DAG.

In a cellular decomposition of the Barth sextic (bertini_real's algorithm on b2), each of 50 singular critical points gets a deflated system of 7 functions in 6 variables, evaluated once to check a refined point. Those first evaluations took 121 s in all (2.5 s at most) and take 0.6 s (21 ms at most).

Identity

Where operands tie on multidegree their canonical order changes, so the system encoding version is b2sysenc/3, registered for 4.0.1, and every system digest moves: records written by 4.0.0 are asked again rather than recalled. The golden fixture is regenerated, as is the pinned seed-42 homotopy digest. The fixture gains tied_terms, a recipe whose operands tie: no recipe had a tie, so the registry could not see a change to the tie-break. The old and new tie-breaks order its products differently. The degree record changes no order.

Interface

Differentiate and Degree(v) are non-virtual; node types override DifferentiateImpl and DegreeImpl. A Python subclass of Node still overrides Differentiate and Degree by name. Node::Variables() and Node::ContentKey() are new.

Tests

• C++: a 26-level chain whose tree has 2^26 copies of x differentiates in 67 ms (about 9 minutes before, extrapolated) and reports its degrees in 2 ms; a DerivativeScope gives the derivatives of separate calls; an unmentioned variable gets the rules' answer, -1 for a negative constant power and 0 for a polynomial.
• The C++ suite passes with BERTINI_NUM_THREADS unset and =1; the Python suite passes.

ADR-0073 records the decision and what not to undo; ADR-0059's tie-break key is marked replaced by it. CHANGELOG entries are under 4.0.1.

🤖 Generated with Claude Code

[ofloveandhate]

End to end, on the cellular decomposition of the Barth sextic (seed 1042, 11 CPUs): 418.6 s on 4.0.0 and 319.3 s with this branch, with the same result (3926 faces, Euler characteristic 10). Stage 2, which sharpens the critical points and deflates the 50 singular ones, went from 86.5 s to 69.1 s; the slice curves and boundary curves, which find and refine critical points on every slice, from 178.5 s to 149.5 s and 51.8 s to 37.9 s.

[ofloveandhate]

this was super fun to do. it was a glorious weekend of making my math dreams come true.