fix: canonical DELETE scanner + key-only single-index delete (W1)

D2 profiling (env-gated phase timers) showed TryScanCanonicalDml's DELETE branch never consumed the whitespace/WHERE keyword after the table name, so every canonical 'DELETE ... WHERE col = literal' fell back to the regex path - DeleteMultipleKeys/B1 were dead code in the benchmark harness.

- Fix scanner: DELETE now consumes whitespace + WHERE + whitespace before the WHERE column; canonical batch deletes route through the structured DeleteMultipleKeys path.

- Diagnostics: Database.CanonicalDeleteStatementsParsed counter + CanonicalBatchDelete_EngagesStructuredPath regression test.

- W1: in DeleteMultipleKeys, when the table has no PK tree and exactly one registered hash index (no B-tree manager) whose key is the condition value, delete positions are recorded with the known key - no per-row engine.Read/decode.

Profile attribution (DELETE 10K, docs): read+decode 62-82ms, commit markers ~50ms, core 23-56ms, index 5-10ms. Per-row read remains while the delete core must remove each auto-rowid PK entry - next lever is batch PK stale/lazy-rebuild.

docs/performance/EXECUTION_PLAN_UPDATE_DELETE.md: combined execution plan + D2 findings.

Author: MPCoreDeveloper

docs/performance/EXECUTION_PLAN_UPDATE_DELETE.md

@@ -0,0 +1,90 @@
+# Uitvoerplan — UPDATE/DELETE-achterstand (combined)
+
+**Datum:** 2026-09-03
+**Bron:** eigen metingen/root-causes (sessie: PR #367–#370) + `PERFORMANCE_DEEP_DIVE.md` (Grok/xAI, second opinion).
+**Status:** actief. Werkwijze: meet eerst (D2), bouw daarna (A1/A2), valideer met median-of-runs + full suite + CI.
+
+## 0. Wat al klaar is (deze sessie)
+| PR | Wat | Effect |
+|---|---|---|
+| #367 | In-place tombstones (directe deletes) | DELETE duurzaam in O(delete), geen flush-rewrite |
+| #368 | **Commit-time tombstones** (transactionele/batch deletes) | **SQL DELETE 0,82 s → 0,24 s** (~12K → ~41-58K ops/s) — de grote sprong |
+| #369 | C4 (batch markers + evict-dedup) + B3 (structured delete, geen dubbele parse) | veilig; neutraal binnen ruis op benchmark |
+| #370 | B1 (key-only decode: alleen PK + hash-indexkolommen) | veilig; neutraal binnen ruis op small-row benchmark |
+
+**Root cause (niet in Grok-doc):** `ExecuteBatchSQL` draait elke batch in een storage-transactie; zonder #368 deed batch-DELETE nog steeds de #366 full-file compactie (~690 ms in `tableFlushLoop`). Daardoor waren eerdere “winst”-metingen niet-duurzaam/logisch-only.
+
+## 1. Baselines & doel
+| | SCDB Direct | SCDB SQL | SQLite | LiteDB |
+|---|---:|---:|---:|---:|
+| INSERT | ~130-187K | ~80-112K | ~130-150K | ~72K |
+| READ | ~127K | ~71K | ~95K | ~15K |
+| UPDATE | ~40-49K | ~34-40K | ~240-275K | ~10K |
+| DELETE | ~55-58K | ~41K | ~325-375K | ~14K |
+
+Cijfers variëren per machine/run (±10-20%). Verdict (beide analyses): niet verloren; **UPDATE/DELETE ~3x achter** is structureel maar aanpakbaar. INSERT/READ + analytics/vector/encryptie zijn al sterke punten.
+
+## 2. Bottlenecks (gecombineerd)
+1. AppendOnly versie-appends: UPDATE/DELETE = pread+pwrite per rij + indexonderhoud (SQLite: in-place leaf + WAL-batch).
+2. Batch-DELETE zat onterecht op flush-compactie → **opgelost (#368)**.
+3. Per-rij punt-I/O + B-tree/hash per operatie (C4/B3/B1 raken de rand, niet de kern — gemeten neutraal).
+4. Volledige (de)serialisatie op in-place paden met leidende variabele-lengte kolommen (deels geraakt door B1).
+5. Contiguïteit wordt alleen voor fixed-width benut (B8/B9); de benchmark-docs-tabel is variabele-lengte met fysiek oplopende posities.
+6. Grove `rwLock` + per-op index-locks.
+7. WAL/fsync-duurzaamheid (bevestigd met split-flush-meting: fsync-tail ~0,5-0,7 s op het oude pad; weg door #368).
+8. Managed/GC + Dictionary/boxing in non-Direct paden.
+9. AES-GCM per record (toggle: `NoEncryptMode`).
+
+## 3. Werkwijze per stap (vóór elke bouw: meten)
+- **D2-first:** profile UPDATE/DELETE hot paths (`DeleteMultipleKeys`/`UpdateMultiple`/`DeleteRecordsCore`/commit-tombstones) met `dotnet-trace` of env-gated fase-timers; bepaal of de tijd zit in resolutie (hash-lookups), per-rij `engine.Read`+decode, indexonderhoud (B-tree/hash), of de commit-markers.
+- Bouw alleen wat het profiel aanwijst. Elke stap: eigen branch → median-of-N benchmark → full suite + 4 CI-filter-suites → doc-update.
+
+## 4. Uitvoeringsfasen
+### Fase A — Quick wins (P0)
+- **A1:** contiguous variabele-lengte single-pass voor UPDATE/DELETE (prefix-walk over oplopende posities; 1 range-read, markers/patches in 1 schrijfpassage). Doel: docs-DELETE/UPDATE ~80-120K. Alleen na D2-bevestiging dat range-I/O de bottleneck is.
+- **A2:** in-place UPDATE waar de slot-lengte gelijk blijft (geen append/stale-versie) voor niet-fastPatch-gevallen.
+- **A3:** batched index re-point per index over de hele batch (deels aanwezig).
+
+### Fase B — Structuur (P1, kern ~3x-achterstand)
+- **B1:** `FixedWidthTable`/in-place page-engine volwassen (vaste offsets, slot-free-space, tombstone+vacuum). Doel: ~120-200K ops/s → ~80-110% van SQLite.
+- **B2:** PageBased als aanbevolen OLTP-engine + storage-engine selector (OLTP→PageBased+FixedWidth; analytics/eventsourcing→AppendOnly/Columnar).
+- **B3:** source-generated typed/ref-struct accessors (geen Dictionary op hot paths) + `PreparedCommand` met herbruikbare buffers.
+- **B4:** fijnmaziger locking (per-page/per-index) + optimistische concurrency.
+
+### Fase C — Platform (P2, .NET 11)
+- **C1:** Native AOT + `Span<T>` + Runtime Async; SIMD lane-API's/AVX-VNNI-512 voor index-lookups.
+- **C2:** optionele “SQLite-compat mode” (NoEncrypt + fixed-width default).
+
+### Fase D — Hygiëne & observability (doorlopend)
+- **D1:** median-of-N + warm-up in de comparative-harness.
+
+## 7. D2-bevindingen (env-gated fase-timers, 2026-09-03)
+DELETE 10K op de docs-tabel (comparative, Release) — attributie in de structured batch path:
+
+| Fase | SQL | Direct |
+|---|---:|---:|
+| per-rij read+decode (`engine.Read`+`DeserializeDeleteKeyRow`) | 62-82 ms | ~66 ms |
+| commit-tombstones (markers, pread+pwrite per rij) | ~50 ms | ~48-50 ms |
+| delete core (index-onderhoud, B-tree/hash) | 23-56 ms | ~32 ms |
+| hash-index lookup | 5-10 ms | ~5 ms |
+
+**Ontdekte bug (gefixed):** `TryScanCanonicalDml`'s DELETE-tak consumente nooit de whitespace/het `WHERE`-keyword na de tabelnaam → elke canonieke `DELETE ... WHERE col = literal` viel terug op de regex-path → `DeleteMultipleKeys` + B1/W1 (PR #369/#370) waren **dead code in de benchmark-harness**. Fix + diagnostische teller `Database.CanonicalDeleteStatementsParsed` + regressietest `CanonicalBatchDelete_EngagesStructuredPath` (deze PR).
+
+**Gevolg voor de attributie:** de per-rij read blijft nodig zolang de delete-core de auto-`rowid`-PK-waarde per rij moet wissen (gate-onderzoek: docs heeft PK-achtige index >1 geregistreerd). Grootste resterende hefbomen, nu met cijfers onderbouwd:
+1. **PK-onderhoud vervangen door één stale/lazy-rebuild** na een grote delete-batch (i.p.v. per-rij `Index.Delete`) → verwijdert ~30-50% van `core` én maakt de per-rij read overbodig (grootste winst op deze workload).
+2. **Marker-batching over een range-read** (commit-tombstones ~50 ms → enkele ms) als vervolg op C4.
+3. W1 (key-only, geen read) vuurt alleen bij een tabel met exact één geregistreerde index en zonder PK-tree — daar direct ~60-80 ms winst per 10K deletes.
+
+- **D2:** `dotnet-trace`/fase-timers op `UpdateAffectedRows`, `DeleteMultipleKeys`, `DeleteRecordsCore`, commit-tombstones.
+- **D3:** AOT/R2R expliciet als aparte meet-as (we meten nu managed JIT + tiered PGO).
+- **D4:** `docs/manual/performance.md` + dit document bijwerken na elke stap.
+
+## 5. Beslisboom (storage-engine)
+- Veel UPDATE/DELETE (OLTP) → **PageBased + FixedWidth** (in-place).
+- Veel appends/analytics/eventsourcing → **AppendOnly/Columnar** (blijft dominant).
+- Pure-throughput scenario’s → `NoEncryptMode`; encryptie blijft default-differentiator.
+
+## 6. Volgende acties
+1. D2-profiel op de huidige DELETE/UPDATE hot paths.
+2. A1/A2 implementeren op basis van het profiel (branch bovenop #370).
+3. Fase B (fixed-width in-place + PageBased default) als aparte roadmap-track.

src/SharpCoreDB/DataStructures/Table.CRUD.cs

@@ -3101,6 +3101,30 @@ internal void DeleteMultipleKeys(List<(string Column, string Literal)> condition
                             var key = ParseValueForHashLookup(value, this.ColumnTypes[colIdx]);
                             if (key != null)
                             {
+                                // W1: when this is the ONLY index the delete core must maintain (no
+                                // PK tree, a single loaded hash index whose key is the condition
+                                // value itself, no secondary B-tree manager), the position can be
+                                // removed with the already-known key — no per-row engine.Read or row
+                                // decode is needed for index cleanup.
+                                bool keyOnlyRow =
+                                    this.PrimaryKeyIndex < 0 &&
+                                    this.hashIndexes.Count == 1 &&
+                                    _btreeManager is null &&
+                                    this.registeredIndexes.Count == 1;
+
+                                if (keyOnlyRow)
+                                {
+                                    foreach (var pos in hashIndex.LookupPositionsUnsafe(key))
+                                    {
+                                        recordsToDelete.Add((pos, new Dictionary<string, object>(1)
+                                        {
+                                            [col] = key
+                                        }));
+                                    }
+
+                                    continue;
+                                }
+
                                 foreach (var pos in hashIndex.LookupPositionsUnsafe(key))
                                 {
                                     var data = engine.Read(Name, pos);

src/SharpCoreDB/Database/Core/Database.Core.cs

@@ -32,6 +32,11 @@ namespace SharpCoreDB;
 /// </summary>
 public partial class Database : IDatabase, IDisposable, IAsyncDisposable
 {
+    // Diagnostics: number of DELETE statements routed through the canonical structured batch path
+    // (proves the scanner/batch fast paths are engaged; 0 means everything fell back to regex).
+    private static long _canonicalDeleteStatements;
+    public long CanonicalDeleteStatementsParsed => Interlocked.Read(ref _canonicalDeleteStatements);
+
     private readonly IServiceProvider _serviceProvider;
     private readonly IStorage storage;
     private readonly IUserService userService;

src/SharpCoreDB/Database/Execution/Database.Batch.cs

@@ -616,6 +616,18 @@ private static bool IsInsertStatement(string sql)
                 return false;
             }
         }
+        else
+        {
+            // DELETE: consume the whitespace after the table name, then the WHERE keyword, then
+            // the whitespace before the WHERE column. (Missing before — every canonical DELETE fell
+            // back to the regex path, bypassing the structured batch-DELETE fast paths.)
+            if (!TryConsumeWhitespace(s, ref i) ||
+                !TryConsumeKeyword(s, ref i, "WHERE") ||
+                !TryConsumeWhitespace(s, ref i))
+            {
+                return false;
+            }
+        }
 
         // WHERE <col> = <literal> (to end of statement)
         if (!TryReadSimpleIdent(s, ref i, out var whereColSpan) || whereColSpan.IsEmpty)
@@ -944,6 +956,7 @@ private bool TryParseDeleteForBatch(string sql, out string tableName, out string
             whereLiteral = whereValRaw;
             // Where stays empty for canonical statements — the table layer reconstructs it only
             // when a fallback actually needs it (B3: no per-statement string allocation).
+            System.Threading.Interlocked.Increment(ref _canonicalDeleteStatements);
             return true;
         }
 

tests/SharpCoreDB.Tests/FixedWidthBulkDeleteTests.cs

@@ -231,6 +231,31 @@ public void DeletesPersistAcrossReopen_AfterFlushCompaction()
         (db as IDisposable)?.Dispose();
     }
 
+    [Fact]
+    public void CanonicalBatchDelete_EngagesStructuredPath()
+    {
+        // Regression: TryScanCanonicalDml's DELETE branch did not consume whitespace/WHERE after
+        // the table name, so every canonical `DELETE ... WHERE col = literal` fell back to the
+        // regex path and bypassed the structured batch-DELETE fast paths (DeleteMultipleKeys).
+        IDatabase? db = CreateDb();
+        try
+        {
+            db.ExecuteSQL("CREATE TABLE docs (id INTEGER PRIMARY KEY, name TEXT, score REAL)");
+            InsertDocs(db, 1, 50);
+            db.Flush();
+
+            var before = ((SharpCoreDB.Database)db).CanonicalDeleteStatementsParsed;
+            db.ExecuteBatchSQL(BuildDeletes(1, 10));
+            db.Flush();
+
+            Assert.True(
+                ((SharpCoreDB.Database)db).CanonicalDeleteStatementsParsed >= before + 10,
+                "canonical DELETE statements must flow through the structured batch path");
+            Assert.Equal(40, db.ExecuteQuery("SELECT id FROM docs").Count);
+        }
+        finally { (db as IDisposable)?.Dispose(); }
+    }
+
     [Fact]
     public void BatchDelete_CommitTimeTombstones_SurviveReopenWithoutExplicitFlush()
     {