Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions docs/own-net-auditor.md
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,22 @@ dotnet-gcdump) гоняет сценарий N× на стенде и выдаё
Часть уже покрыта own-check (subscription/region-escape) и WpfAnalyzers (3 902
wpf-freezable) — пак расширяет, не дублирует.

**Карта покрытия — `docs/wpf-audit-coverage.md`:** каждый смелл × техника детекта
(static-C# / static-XAML / runtime-heap / runtime-trace) × статус (have / small-rule /
gap) × маппинг на Avalonia-оракул. Два недоспеценных слоя зафиксированы там же:
**XAML-анализатор** (binding-path, virtualization-off, nested ScrollViewer, heavy
DataTemplate, ElementName/RelativeSource) и **runtime-trace коллектор** (binding-error
log, notification storms). Freezable — чисто-WPF, на оракуле не воспроизводится.

**Data-duplication / строки — `docs/string-canonicalization.md`:** дублированный
`System.String` как доминирующий случай оси data-duplication, под net472. Тот же каркас,
что и везде: **анализатор** (heap-отчёт wasted bytes + Roslyn birth-sites — кладётся в
фазу 5) → **точечный мутатор** (AdaptiveStringCanonicalizer + Cecil/ReJIT/Harmony только
на approved hot sites, только новые строки) → пере-замер. Переписывание всего heap по
ссылкам **отвергнуто** (ломает `ReferenceEquals`/`lock(string)`/identity-кэши/unsafe —
ровно поэтому GC-string-dedup так и не въехал в dotnet/runtime). Анализатор —
framework-агностичен, тестируется на оракуле здесь.

## 5. Что брать готовое

| Слой | Инструмент |
Expand Down
108 changes: 108 additions & 0 deletions docs/string-canonicalization.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
# String canonicalization under .NET 4.7.2 — design note

The data-duplication axis (`docs/wpf-audit-coverage.md`), taken to its most valuable special
case: **duplicated `System.String`**. In a long-running WPF/DevExpress LOB app, bounded-vocabulary
strings — status codes, column names, enum-ish labels, dictionary values rehydrated per row/DTO —
get allocated thousands of times. Same bytes, different references, all retained. This note
captures the whole "CLR string surgery" branch so nothing is lost, and frames it in the project's
own terms: **analyzer first, targeted mutator second, never a heap rewrite.**

This is the same methodology as the rest of Own.NET — *measure → confirm → targeted fix → re-measure*
(the phase-2 ratchet + phase-5 runtime-confirm loop), just pointed at strings. The mutator half is
a fix-arm-style tiered intervention, kept strictly opt-in.

---

## Phase A — Analyzer (folds into Own.NET, build-free to consume)

Objective measurements before anyone touches a pointer. Two sources:

**heap (primary).** ClrMD / dotnet-gcdump heap walk after a scenario → enumerate every
`System.String`, group by content, and report per value:
- `count` — live instances,
- `total_bytes` — Σ object size,
- `wasted_bytes` — `(count − 1) × size` if canonicalized to one,
- `retention` — the dominators / GC-root paths keeping them alive,
- `long_lived_only` — filter to gen2 / pinned, ignore transients.

This is the phase-5 collector extended with a string lens — it emits the same kind of artifact
(`strings.json`, shape mirrors `runtime.json`'s `retained[]`) that the Python side correlates and
the dashboard renders. Top-N by `wasted_bytes` is the actionable output.

**C# (birth sites).** A Roslyn pass that flags *where* repeated long-lived strings are born —
`StringBuilder.ToString()`, `string.Concat`, DTO ctors/factories, parsers/deserialization,
setters/hydration code — so the report points at call sites, not just values.

The analyzer alone is the 80% win: it replaces intuition with **top-20 values by wasted bytes +
their birth sites + retention roots**, which is exactly the report you want before deciding whether
canonicalization is even worth it.

## Phase B — Mutator (separate, gated, net472-specific track)

Only for **approved hot sites**, **bounded-vocabulary** values, **new strings only**. Never the
existing heap. The menu, cheapest→heaviest:

1. **`AdaptiveStringCanonicalizer`** (the policy core, not a mechanism) — *not* `String.Intern`.
Own dictionary with evictable / weak semantics where possible, a `count` threshold and an
`estimated-saved-bytes` threshold, metrics built in. `String.Intern` is rejected as the
default: its pool is process-lifetime and never collected, so it trades duplication for an
unreclaimable leak.
2. **Build-time IL weaving** (Mono.Cecil / Fody) — insert `value = Canonicalizer.Canonicalize(value)`
after the approved string-producing calls. Pros: no GC, no object-graph rewrite, trivially
revertible, works on Framework. The pragmatic default for the mutator.
3. **CLR Profiling API** (`ICorProfilerCallback`) — the grown-up runtime path for net472, in two
modes:
- *observe*: `ObjectAllocated` for allocation stats, `ObjectReferences`/`RootReferences2` to
build the retention graph (overlaps Phase A's heap source),
- *scalpel*: `SetILFunctionBody` + `RequestReJIT` (ReJIT is available on 4.5+) to rewrite
specific methods so new strings route through the canonicalizer.
Most powerful, heaviest to operate.
4. **Harmony** (`Lib.Harmony`) prefix/postfix/transpiler on managed string factories — dirty but
fast for a one-evening PoC of a single pipeline. Coverage is poor and it's easy to introduce
behavioural schizophrenia; not for broad surgery.

## Explicitly rejected — whole-heap reference rewrite

The seductive version ("build the object graph, pick a canonical instance per value, mass-rebind
every reference, sync with the GC") is **out of scope, on purpose.** It breaks the world:
- `ReferenceEquals(a, b)` and any identity-keyed cache (`ConditionalWeakTable`, identity hash);
- `lock(someString)` (monitor uses object identity / sync block);
- `unsafe` code that mutates or pins strings on the assumption they're private;
- compatibility becomes sticky and miserable.

This is precisely why the runtime's own GC string-dedup design (dedup thread, old generations,
weak handles, CAS on reference swap, no stack refs) **never shipped** in dotnet/runtime — it works
in environments where strings don't expose identity the way .NET's do. For net472 this is GC/runtime
territory, not a library, and "truly patch the CLR without it later being agony in prod" is the
opening line of a bad on-call legend. We don't go there.

## Staged PoC plan

1. Small profiler/ClrMD report of duplicated strings (Phase A heap source).
2. Top-20 values by wasted bytes + their top call sites / retention roots (Phase A C# source).
3. Proof-of-concept `AdaptiveStringCanonicalizer` with metrics.
4. IL weaving (Cecil) **or** Harmony at 2–3 approved sites only.
5. Re-measure — wasted bytes before/after. Feedback loop, not a one-shot.

That simulates runtime string dedupe exactly where it pays, without becoming a garbage collector
for an afternoon.

## net472 specifics

- **Profiling API**: `ICorProfilerCallback2/3`; enable via `COR_ENABLE_PROFILING` + profiler GUID.
Heavyweight, but the only first-class retention-graph source in-process.
- **ReJIT**: `RequestReJIT` / `SetILFunctionBody` are available since 4.5 — usable on 4.7.2.
- **Cecil/Fody**: build-time IL rewrite works on Framework assemblies; easiest to ship and revert.
- **ClrMD** (`Microsoft.Diagnostics.Runtime`): attach to a live process or open a dump, enumerate
the heap, group strings — the analyzer's backbone.
- **`String.Intern`**: exists, but its pool is never collected → memory can't be reclaimed; use the
adaptive canonicalizer instead.

## Where this is testable *here* (Avalonia oracle)

String duplication is pure CLR/heap, so the **entire Phase A analyzer is framework-agnostic and
testable on the Avalonia oracle** (or any .NET app we build in this container) — group strings,
report wasted bytes, find birth sites. The build-time Cecil weaving path is cross-platform too, so
even a slice of Phase B can be exercised here. The Profiling-API/ReJIT scalpel is the one piece that
stays a stand exercise. So this branch isn't STS-only — most of it validates on the oracle, same as
phases 3–5.
93 changes: 93 additions & 0 deletions docs/wpf-audit-coverage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# WPF audit pack — coverage matrix (Own.NET Auditor §4)

What the auditor can detect for a WPF/DevExpress LOB app like STS, *by detection technique*, and
how honestly each maps onto an **Avalonia** oracle (the cross-platform stand-in we can build and
run here — see `docs/own-net-auditor.md`). The point of this doc is to draw the line between
"we've got this", "small rule away", and "whole surface we haven't built yet" — so nothing on
the wishlist quietly falls through.

## Detection techniques

| Tag | Technique | Where it runs |
|---|---|---|
| **C#** | Roslyn syntax/semantic analyzer over `.cs` | stand (graph/analyzer extractor) |
| **XAML** | analyzer over `.xaml`/`.axaml` (it's just XML — parseable even in Python) | stand or CI |
| **heap** | ClrMD / dotnet-gcdump heap walk after a scenario (phase 5 `runtime.json`) | stand |
| **trace** | runtime listeners: WPF binding-error trace, Dispatcher/notification counters | stand |

The governing principle is the phase-5 one, extended from leaks to perf/correctness:
**suspect statically, confirm at runtime.** A disabled-virtualization attribute is a *suspicion*
(XAML); 100k live containers for 100k rows is the *confirmation* (heap).

## Matrix

| Smell | Technique | Status | Avalonia |
|---|---|---|---|
| **Event leaks** | | | |
| `+=` without `-=` | C# + **heap** | ✅ core (own-check OWN001) + phase-5 confirm | ✅ 1:1 |
| static-event subscription from instance | C# + **heap** | ✅ + phase-5 confirm | ✅ 1:1 |
| `DependencyPropertyDescriptor.AddValueChanged` (roots in a static table forever) | C# | ✅ small rule | ❌ WPF-only (Avalonia: `GetObservable().Subscribe()` un-disposed — same category, diff API) |
| `CollectionChanged` / `PropertyChanged` subscription leak | C# + **heap** | ✅ | ✅ 1:1 |
| `CommandManager.RequerySuggested` static-event | C# | ✅ small rule | ❌ WPF-only |
| **Timers** | | | |
| `DispatcherTimer` / `Timers.Timer` / `Threading.Timer` no Stop/Dispose | C# + **heap** | ✅ small rule + confirm | ✅ 1:1 (`Avalonia.Threading.DispatcherTimer`) |
| **WPF binding** | | | |
| binding path doesn't exist | **trace** (reliable) / XAML (heuristic) | ⚠️ **no XAML analyzer, no trace collector** | ✅ Avalonia logs `LogArea.Binding` errors → trace maps |
| binding to an expensive property | XAML + heuristic | ⚠️ no XAML analyzer | ~ |
| binding errors via runtime log | **trace** | ⚠️ **no collector** (cheap + high value) | ✅ |
| `ElementName` / `RelativeSource` hell (complexity) | XAML | ⚠️ no XAML analyzer | ✅ (same XAML dialect) |
| **PropertyChanged hell** | | | |
| fat setter (one set → pile of `OnPropertyChanged`) | C# | ✅ Roslyn rule | ✅ 1:1 |
| cascading notifications | C# (call graph) + **trace** | ⚠️ partial; storms need trace | ✅ |
| duplicate / repeated notifications | **trace** | ⚠️ no collector | ✅ |
| UI-thread notification storms | **trace** (Dispatcher counters) | ⚠️ no collector | ✅ |
| **Virtualization** | | | |
| `IsVirtualizing=False` / `VirtualizationMode=Standard` (no recycling) | XAML + **heap** | ⚠️ XAML gap; **heap confirm falls out of phase 5** | ✅ (`VirtualizingStackPanel`/`ItemsRepeater`) |
| non-virtualizing `ItemsPanel` (StackPanel swap) | XAML + **heap** | ⚠️ XAML gap; heap confirm ✅ | ✅ |
| nested `ScrollViewer` / `CanContentScroll=False` kills virtualization | XAML | ⚠️ XAML gap | ✅ |
| heavy `DataTemplate` (element/nesting count) | XAML | ⚠️ XAML gap | ✅ |
| containers not recycled (live count ≫ visible rows) | **heap** | ✅ phase-5 native | ✅ 1:1 |
| DevExpress `GridControl` virtualization config | XAML (vendor) | ❌ vendor-only | ❌ no DevExpress |
| **Freezable** | | | |
| `Brush`/`Geometry`/`ImageSource` not `Freeze()`d | C#/XAML + **heap** | ⚠️ small rule | ❌ **Freezable is a WPF-only concept** |
| repeated immutable resources instead of static/shared | XAML + **heap** (dup count) | ⚠️ | ~ (heap-dup maps; Freeze doesn't) |
| **Data duplication** | | | |
| immutable reference data multiplied across VM/DTO | **heap** (N copies of a should-be-singleton) | ✅ phase-5 native | ✅ 1:1 |
| large collections copied needlessly | **heap** + C# (`.ToList()` in hot paths) | ⚠️ partial | ✅ |
| **duplicated strings** (the dominant case) | **heap** + C# birth-sites | → see [`string-canonicalization.md`](string-canonicalization.md) | ✅ 1:1 (pure CLR) |

## The two surfaces we haven't built (the honest gaps)

1. **XAML analyzer.** A large slice of the wishlist lives in `.xaml`, not `.cs`: binding paths,
`ElementName`/`RelativeSource` complexity, **virtualization-disabled patterns**, nested
`ScrollViewer`, heavy `DataTemplate`, Freezable-in-resources. We have a Roslyn (C#) extractor
but nothing reads XAML. Biggest gap — and technically cheap: XAML is XML, rules are tree
patterns. Avalonia `.axaml` is the same dialect, so it maps.
2. **Runtime-trace collector.** Beyond the heap snapshot there's a second runtime signal class —
the WPF binding-error trace (`System.Windows.Data Error: 40 …`) and UI-thread notification
counters. The heap collector doesn't see these; a small ETW/trace listener (or, on Avalonia,
a `Logger` sink) would. Cheap, high value for binding correctness + storms.

One concept — **Freezable** — is WPF-only and won't exercise on the Avalonia oracle at all; that
tail is validated only on STS.

## Avalonia mappability — summary

What the oracle exercises splits by what's already built — to keep this honest against the gaps
above:

- **Today** (framework-agnostic core, on techniques we have — Roslyn graph + CLR heap): all event
leaks, timers, fat-setter, virtualization *heap-confirmed* (container count ≫ visible rows),
data duplication, string duplication.
- **Once the two gap surfaces exist** (XAML analyzer + runtime-trace collector): heavy templates,
virtualization-off *patterns*, binding-path / `ElementName` complexity, and binding errors —
Avalonia's `.axaml` is the same XAML dialect and it logs binding errors too, so the oracle will
validate these the moment those surfaces land. (Planned, not current.)
- **Never** on the oracle (WPF/vendor-only tail, STS only): Freezable, DevExpress controls,
`CommandManager.RequerySuggested`, `DependencyPropertyDescriptor.AddValueChanged`.

So ~70–80% of the surface is oracle-reachable in total; the "today" slice is the
framework-agnostic core, the rest unlocks with the XAML analyzer and the trace collector.

Crucially, the framework-agnostic core — Roslyn graph, CLR heap — behaves identically, so phases
3–5 are validated on *real* data the moment we run them against a leaking Avalonia app.
Loading