Skip to content

feat: ResNet-18 end to end, Conv+BN folding, preload anonymous weights - #5

Merged
asjad2401 merged 4 commits into
mainfrom
feat/resnet18-e2e
Oct 5, 2026
Merged

asjad2401 merged 4 commits into
mainfrom
feat/resnet18-e2e

Conversation

@asjad2401

Copy link
Copy Markdown
Owner

Stacked on #4 (which is stacked on #3) — merge those first.

Summary

ResNet-18 now runs end to end and matches ONNX Runtime across four real exports:

Model Exporter Opset Max logit diff Top-1 = ORT
Model Zoo resnet18-v1-7 MXNet 8 6.2e-6 ✓
Model Zoo resnet18-v2-7 MXNet 7 8.1e-6 ✓
torchvision resnet18 torch.onnx legacy 17 4.8e-6 ✓
torchvision resnet18 torch.onnx dynamo 18 5.0e-6 ✓

(batch 1 and 4, pretrained weights)

Changes

  • ReduceMean / ReduceSum / ReduceMax / ReduceMin — the dynamo exporter writes global average pooling as ReduceMean over H, W.
  • Pre-load anonymous static weights. onnx-ir passes many initializers (including every conv weight) as unnamed ValueSource::Static args. These were skipped at load and converted from TensorData on every forward pass. They're now keyed by data id (static::{id}) and loaded once. ~10% faster at batch 1 across the board.
  • Fold Conv → BatchNorm at load time. W' = W·scale, b' = b·scale + offset, BN node removed. Only when the conv output feeds nothing but that BN (and isn't a graph output). Unfoldable BNs keep the existing mul+add path.
  • BN fast path only for 4D inputs — the [1,C,1,1] pre-fused params turned a rank-2 [N,C] BN into a 4D output.
  • Linear/Gemm bias now uses ctx.resolve, so it benefits from pre-loading too.

Performance (CPU, Apple M-series 10 cores, median of 10)

Model batch 1 before → after batch 8 before → after ORT batch 8
Model Zoo v1 25.2 → 22.4 ms 279 → 103 ms 118 ms
Model Zoo v2 25.2 → 22.6 ms 292 → 186 ms 119 ms
torch legacy 24.7 → 22.6 ms 107 → 105 ms 119 ms
torch dynamo 28.2 → 22.8 ms 113 → 106 ms 119 ms

ORT is ~14.8 ms at batch 1. v2 still trails at batch 8 because its pre-activation BNs (after Add/Relu, not Conv) can't be folded and hit a slow path in burn-flex: a [1,C,1,1] broadcast multiply over [8,64,112,112] takes ~24 ms vs ~3.6 ms same-shape (numpy: 0.9 ms). Worth reporting upstream.

Testing

  • tests/test_resnet.py (new): downloads + caches Model Zoo models, exports torchvision models if torch is installed, compares vs ORT; --bench prints timings.
  • New CI job resnet18 end-to-end (CPU torch, models cached via actions/cache). Not added to required checks — your call.
  • tests/test_ops.py: BN folding cases (folded / no-bias / conv output reused / conv output is graph output / BN after Relu / rank-2 BN) with node-count assertions, plus reductions. 43 checks pass.
  • test_bridge.py, compare_ort.py, cargo fmt --check, clippy -D warnings clean.

Note: torch.export specializes size-1 dims, so a dynamo export traced with batch 1 bakes [1, 512] into the final Reshape. The test exports with batch 2.

🤖 Generated with Claude Code

asjad2401 and others added 4 commits October 5, 2026 18:51
- ResNet-18 now verified against ONNX Runtime on Model Zoo v1-7/v2-7 and
  torchvision legacy + dynamo exports (tests/test_resnet.py, new CI job).
- Add ReduceMean/Sum/Max/Min; the dynamo exporter emits ReduceMean for
  global average pooling.
- Pre-load static weights that onnx-ir passes as anonymous constants
  (keyed by data id). Previously every conv weight was converted from
  TensorData on each forward pass.
- Fold BatchNormalization into a directly preceding Conv2d at load time
  when the conv output has no other consumer. Works around slow [1,C,1,1]
  broadcasting in burn-flex: Model Zoo v1 batch 8 279 -> 103 ms.
- Only use the pre-fused BN fast path for 4D inputs; rank-2 BN was being
  broadcast to a 4D output.
- Linear/Gemm bias goes through ctx.resolve (uses pre-loaded weights).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@asjad2401
asjad2401 changed the base branch from fix/error-handling to main October 5, 2026 14:44
@asjad2401
asjad2401 merged commit 62ed15f into main Oct 5, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant