repsel: deforestation hides the record-producer shape from the Ptr<Shape> element rule

#7034 §3 (PR #7149) proves an element-shape-proven local array from the
shape const a: C[] = []; … a.push(new C(…)); … and then promotes the pushed
producer and every in-bounds a[i] binding to Ptr<Shape>.

It does not fire on benchmarks/app-patterns/kernels/batch.ts — the workload
the representation exists for — and the reason is not any of its five
conjuncts. perry-transform/src/deforest has already rewritten the producer
before the analysis runs. --trace hir --focus buildRows on main:

- buildRows (params: 2, …)
    param n (id=14): Number
    param __deforest_out (id=44): Array(Named("Row"))
    [0] For { … body: [Let { id: 17, name: "row", init: New { class_name: "Row", … } },
                       Expr(ArrayPush { array_id: 44, value: LocalGet(17) })] }

const rows: Row[] = [] and return rows are gone; the array is a
parameter, and a parameter array has no provenance (the caller could pass
anything, and its array may be aliased). So row's push is still an escape.

This matters more than one workload. Deforestation's documented trigger
(deforest/mod.rs) is exactly

function f(…) { const out = []; /* pushes */ return out; }

— the canonical record-producing function, and item 7 of its transform list
rewrites even non-consumer call sites to const all = []; f(args, all);. So
every producer of that shape in a module where deforestation fires is
invisible to Ptr<Shape>'s element rule, and the two passes are working
against each other: deforestation removes an allocation, the element rule
removes a guard diamond, and today you cannot have both.

Options, in ascending order of scope

Order the passes. Run the element analysis on pre-deforestation HIR and
carry the facts forward. Cheapest if fact identity survives the rewrite;
needs checking that local ids do.
Teach the rule about __deforest_out. The parameter is
compiler-synthesized and the transform knows where the array is allocated
(it inserts the allocation at the call site). That provenance could be
recorded as a fact on the parameter. This is the honest fix but it means
trusting a transform-inserted parameter, which needs its own soundness
argument — in particular the caller's array must satisfy E1/E3 at the call
site, and there may be several call sites.
Generalise to array parameters via repsel row 1: scoping Ptr<Shape> beyond locals — measured, promotion is 0 on the motivating workload; build --opt-report (#6952) first #7034 §1's clone-and-route, with
containment replaced by all-call-sites-agree. Subsumes (2) and is the
general answer, but it is the call argument position and much larger.

Measure first: how many modules in the corpus have deforestation firing on a
function whose local array would otherwise satisfy E1-E5? That number decides
between (1) and (3).

The premise still reproduces exactly, the mechanism is worth 7.2% on the shape it bites — and it bites 0 of 61 real workloads, including `batch.ts`

Measured on main @ 514ff7c9a, macOS arm64. One compiler with a throwaway
PERRY_DEFOREST=0 knob at collect_modules.rs:1914 (keyed into
BUILD_CACHE_ENV_VARS), --profile perry-dev, PERRY_RUNTIME_DIR pinned, the
.as identical across every hop. Instrument: /usr/bin/time -l instructions
retired, interleaved reps, outputs compared.

1. The premise is intact

--print-hir on const rows = buildRows(N):

- buildRows (params: 2, …)
    param __deforest_out (id=14): Array(Named("Row"))

and --opt-report then denies the producer:
local 'row' … ptr-shape rule 2 (containment). With PERRY_DEFOREST=0 on the
same source, row -> Ptr<Shape> is selected. deforest/detect.rs:206-208 and
ptr_shape_elements.rs:276 (E1: !*mutable && items.is_empty()) are both
unchanged. Nothing here is stale.

2. On the canonical shape the selection deforestation costs is not consumed

Producer + const r = A[j] consumer, 20k × 200:

	selected	CONSUMED
deforest ON	1	1
deforest OFF	2	1

21,471,590,455 vs 21,495,817,068 instructions (medians of 5, interleaved) —
+0.11%, noise. row has no field access in the producer, so its promotion
emits nothing. This is #7170 §6.1's "a promotion goes unconsumed three ways",
here at the producer end.

3. The shape where it DOES cost, isolated 2×2

Same producer plus four field writes / five field reads on row before the
push. --release compiler and --release runtime archives, medians of 3,
G instructions:

	Ptr<Shape> ON	Ptr<Shape> OFF
deforest ON	31.938	31.941
deforest OFF	29.299	31.756

So Ptr<Shape> is worth 7.7% on this shape, deforestation destroys 100%
of it (31.938 vs 31.941 — the promotion is worth exactly nothing once
deforestation has run), and deforestation itself costs +9.0% in the
full-optimization configuration (31.938 vs 29.299). The mechanism this issue
describes is real and now quantified. (--profile perry-dev artifacts gave
7.2% / 8.5% — same conclusion.)

4. The number this issue asked for

Measure first: how many modules in the corpus have deforestation firing on a
function whose local array would otherwise satisfy E1-E5? That number decides
between (1) and (3).

Measured directly — --opt-report selected/CONSUMED plus cmp of the
--no-link objects, deforest ON vs OFF, over 61 real workloads (30
benchmarks/suite, 12 benchmarks/app-patterns/kernels, 19 programs from the
beat-scriptc sweep corpus):

objects differ on 3: batch, shapes, bench_numeric_array_numeric
deforest costs a selection on 1: batch (2 → 3 selected; the row is
local 'row' -> Ptr<Shape> (in function buildRows))
deforest costs a CONSUMED promotion on 0

batch.ts — the workload this issue was filed against — is in that list, and
timed on --release artifacts it is 1,780,938,481 vs 1,781,380,423
instructions (5 interleaved reps, identical checksum: 1601043.5000):
0.02%. The promotion deforestation
blocks there is row in buildRows, which has no field access, exactly as in
§2.

5. What has changed since this was filed, and why the read side is fine

The read half of batch.ts is no longer waiting on the element rule.
stmt/element_shape_loop.rs (#7480/#7496 runtime element-shape invariant +
versioned loop clone, consumers #7701/#7702/#7771) reads arr[i].field
guard-free through a runtime invariant, and its own header says the
const r = arr[counter] form is "the form a parameter array reaches the
clone through". Confirmed: on the deforested probe the consumer still reports
Ptr<Shape> selected and consumed for the element read. Deforestation
removes the array's static provenance; it does not remove the read-side win.

Recommendation

Close, with the trigger recorded so a future workload can reopen it:
deforestation costs a consumed Ptr<Shape> promotion exactly when the
deforested producer performs field traffic on the pushed local before pushing
it. That shape is worth 7.2% when it occurs and occurs in 0 of 61 programs
here. Options (1)/(2)/(3) should not be scheduled off batch.ts; the element
rule already fires there and the thing it loses emits nothing.

Side finding, NOT this issue (filing separately)

shapes.ts reports Ptr<Shape> 0/0 in both arms and yet deforestation
costs it 5.4%: 1,289,016,130 vs 1,222,424,487 instructions (--release,
medians of 5, interleaved, identical output 1431180 1463160 1176000 320000040000 48000 24000 144000). The deforested producer is build(n) at
shapes.ts:118. That is deforestation being a net loss on its own, on a program
the beat-scriptc campaign is actively measuring — a separate question from
representation selection.

Unmeasured: x86-64; --profile dist; dependency JS (deforestation requires a same-module
producer/consumer pair, which the CJS corpus does not present); and I did not
attempt any of the three fixes.

Closing with the trigger recorded, per the measurement above.

The premise reproduces exactly and the mechanism is real (7.7% on a producer
that does field traffic on the pushed local before pushing it, isolated 2×2 on
--release artifacts). It fires on 0 of 61 real workloads — including
batch.ts, the workload this issue was filed against, where deforestation costs
one Ptr<Shape> selection that is not consumed and the timed difference is
0.02%.

Reopen when a workload appears whose deforested producer touches the pushed
local's fields before the push. Until then, options (1)/(2)/(3) have no
measured payoff, and the read side is already served on the deforested shape by
the runtime element-shape invariant (stmt/element_shape_loop.rs), which post-
dates this issue and works on parameter arrays.

The separate finding — deforestation costing 5.4% on shapes.ts with no repsel
interaction at all — is #8104.

Navigation Menu

Repository navigation

repsel: deforestation hides the record-producer shape from the Ptr<Shape> element rule #7150

Options, in ascending order of scope

The premise still reproduces exactly, the mechanism is worth 7.2% on the shape it bites — and it bites 0 of 61 real workloads, including `batch.ts`

1. The premise is intact

2. On the canonical shape the selection deforestation costs is not consumed

3. The shape where it DOES cost, isolated 2×2

4. The number this issue asked for

5. What has changed since this was filed, and why the read side is fine

Recommendation

Side finding, NOT this issue (filing separately)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

repsel: deforestation hides the record-producer shape from the Ptr<Shape> element rule #7150

Description

Options, in ascending order of scope

Activity

proggeramlug commented on Aug 14, 2026

The premise still reproduces exactly, the mechanism is worth 7.2% on the shape it bites — and it bites 0 of 61 real workloads, including batch.ts

1. The premise is intact

2. On the canonical shape the selection deforestation costs is not consumed

3. The shape where it DOES cost, isolated 2×2

4. The number this issue asked for

5. What has changed since this was filed, and why the read side is fine

Recommendation

Side finding, NOT this issue (filing separately)

proggeramlug commented on Aug 14, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

The premise still reproduces exactly, the mechanism is worth 7.2% on the shape it bites — and it bites 0 of 61 real workloads, including `batch.ts`