fix(#344): compressed single-file reopen crash - propagate Compressed block flag on rewrites + read row-cache via ReadBlockAsync

A table row-cache block that was rewritten while it already existed (auto-flush as JSON grows past the compression threshold, or grow/realloc) was stored compressed WITHOUT the per-block Compressed flag, so on reopen the raw Brotli/GZip bytes were handed to the JSON parser (JsonException '0x0B is an invalid start of a value'). WriteBlockAsync now recomputes the flag for every write, and SingleFileTable.EnsureCacheLoaded reads through ReadBlockAsync (transparent decrypt+decompress). Regression tests added for compression and compression+encryption reopen+SELECT.

Author: MPCoreDeveloper

ROADMAP.md

@@ -2,7 +2,7 @@
 
 <div align="center">
 
-**Last updated: v2.0.0-preview.2 · Maintained by [@MPCoreDeveloper](https://github.com/MPCoreDeveloper)**
+**Last updated: v2.0.0-preview.3 · Maintained by [@MPCoreDeveloper](https://github.com/MPCoreDeveloper)**
 
 </div>
 

docs/CHANGELOG.md

@@ -71,6 +71,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
     publishes with `PublishAot=true` and runs exit 0 including the single-file full-vacuum path.
   - **Follow-up:** full VACUUM now reloads the block registry / FSM / WAL after the file swap so
     in-memory offsets match the compacted file (fixes stale-offset writes after vacuum).
+- **Issue #344 — compressed single-file (`.scdb`) databases crashed on reopen + SELECT with
+  `JsonException: '0x0B' is an invalid start of a value`**:
+  - `WriteBlockAsync` only stamped the per-block `Compressed` flag for brand-new blocks; a table
+    row-cache block that was rewritten while it already existed (auto-flush as the JSON grows past
+    the compression threshold, or a grow/realloc) was stored compressed **without** the flag, so on
+    reopen the raw Brotli/GZip bytes were handed to the JSON parser;
+  - `SingleFileTable.EnsureCacheLoaded` now reads the row-cache block through `ReadBlockAsync`
+    (which decrypts and decompresses transparently) instead of `GetReadStream` (which returns raw
+    on-disk bytes when encryption is off);
+  - regression tests `DatabaseFactory_Compression_GrowingTable_ReopenSelectShouldSurvive` and
+    `DatabaseFactory_CompressionPlusEncryption_GrowingTable_ReopenSelectShouldSurvive` cover the
+    reopen + SELECT path for compression alone and compression + encryption.
 
 ## [2.0.0] - 2026-08-28
 

docs/INDEX.md

@@ -1,6 +1,6 @@
 # SharpCoreDB Documentation Index
 
-**Status:** Active documentation set (`v2.0.0-preview.2`)
+**Status:** Active documentation set (`v2.0.0-preview.3`)
 
 ## 0. Manual (start here)
 

src/SharpCoreDB/NuGet.README.md

@@ -1,12 +1,12 @@
-# SharpCoreDB v2.0.0-preview.2 - Performance-First Database Engine
+# SharpCoreDB v2.0.0-preview.3 - Performance-First Database Engine
 
 **High-Performance Embedded AND Networked Database for .NET 10**
 
 SharpCoreDB is a modern, encrypted, file-based database engine with SQL support, built for production applications. Now available as both embedded database and network server.
 
 [![SonarCloud Quality Gate](https://img.shields.io/sonar/quality_gate/MPCoreDeveloper_SharpCoreDB?server=https%3A%2F%2Fsonarcloud.io&logo=sonarcloud)](https://sonarcloud.io/dashboard?id=MPCoreDeveloper_SharpCoreDB)
 
-## What's New in v2.0.0-preview.2
+## What's New in v2.0.0-preview.3
 
 ### 🚀 Performance-first engine
 - v2.0 closes the v1.x benchmark gap: point reads **beat SQLite** on the default engine, batch INSERTs
@@ -203,7 +203,7 @@ We welcome contributions! Check the repository for contribution guidelines.
 
 ---
 
-**Latest Version:** 2.0.0-preview.2 (August 30, 2026)  
+**Latest Version:** 2.0.0-preview.3 (August 30, 2026)  
 **Target:** .NET 10 / C# 14  
 **Tests:** 1,600+ (100% passing)  
 **Status:** ✅ Preview 2 — performance-first v2.0 line

src/SharpCoreDB/SingleFileTable.cs

@@ -655,19 +655,18 @@ private void EnsureCacheLoaded()
                 return;
             }
 
-            using var stream = _storageProvider.GetReadStream(_dataBlockName);
-            if (stream is null)
+            // Read through ReadBlockAsync so encryption AND block-level compression (#344)
+            // are both transparently handled. GetReadStream returns the raw on-disk bytes
+            // when encryption is off, which would hand compressed data (Brotli/GZip marker
+            // bytes) to the JSON parser on reopen — breaking SELECT after reopen.
+            var jsonBytes = _storageProvider.ReadBlockAsync(_dataBlockName, CancellationToken.None).GetAwaiter().GetResult();
+            if (jsonBytes is null || jsonBytes.Length == 0)
             {
                 _rowCache = [];
                 _cacheLoaded = true;
                 return;
             }
 
-            // Read the stream into bytes and trim trailing null bytes
-            using var memoryStream = new MemoryStream();
-            stream.CopyTo(memoryStream);
-            var jsonBytes = memoryStream.ToArray();
-            
             // Trim trailing null bytes
             var endIndex = jsonBytes.Length;
             while (endIndex > 0 && jsonBytes[endIndex - 1] == 0)

src/SharpCoreDB/Storage/SingleFileStorageProvider.cs

@@ -794,7 +794,19 @@ public async Task WriteBlockAsync(string blockName, ReadOnlyMemory<byte> data, C
         {
             // Calculate required pages
             var requiredPages = (data.Length + _header.PageSize - 1) / _header.PageSize;
-            
+
+            // ✅ Compression (#344): the Compressed flag must reflect the state of THIS write.
+            // A previous write may have stored the block compressed or uncompressed; the flag is
+            // recomputed here so an updated (rewritten/grown) block always carries the correct
+            // marker. The earlier code only set the flag for brand-new blocks, so an existing
+            // block that got compressed lost its Compressed bit and reopen read raw Brotli/GZip
+            // bytes as JSON (JsonException "invalid start of a value").
+            uint flags = (uint)BlockFlags.Dirty;
+            if (isCompressed)
+            {
+                flags |= (uint)BlockFlags.Compressed;
+            }
+
             ulong offset;
             BlockEntry entry;
 
@@ -806,14 +818,14 @@ public async Task WriteBlockAsync(string blockName, ReadOnlyMemory<byte> data, C
                 {
                     // Fits in existing space
                     offset = existingEntry.Offset;
-                    entry = existingEntry with { Length = (ulong)data.Length, Flags = existingEntry.Flags | (uint)BlockFlags.Dirty };
+                    entry = existingEntry with { Length = (ulong)data.Length, Flags = flags };
                 }
                 else
                 {
                     // Need more space: free old, allocate new
                     _freeSpaceManager.FreePages(existingEntry.Offset, (int)existingPages);
                     offset = _freeSpaceManager.AllocatePages(requiredPages);
-                    entry = existingEntry with { Offset = offset, Length = (ulong)data.Length, Flags = (uint)BlockFlags.Dirty };
+                    entry = existingEntry with { Offset = offset, Length = (ulong)data.Length, Flags = flags };
                 }
             }
             else
@@ -835,13 +847,6 @@ public async Task WriteBlockAsync(string blockName, ReadOnlyMemory<byte> data, C
                     offset = registryEnd;
                 }
 
-                // ✅ Compression: set the Compressed flag if this block was compressed.
-                var flags = (uint)BlockFlags.Dirty;
-                if (isCompressed)
-                {
-                    flags |= (uint)BlockFlags.Compressed;
-                }
-
                 entry = new BlockEntry
                 {
                     BlockType = (uint)Scdb.BlockType.TableData,

tests/SharpCoreDB.Tests/Storage/SingleFileCompressionTests.cs

@@ -449,6 +449,65 @@ public void DatabaseFactory_WithCompression_ShouldRoundtripTableData()
         DisposeDatabase(db2);
     }
 
+    [Fact]
+    public void DatabaseFactory_Compression_GrowingTable_ReopenSelectShouldSurvive()
+    {
+        // REGRESSION (#344): a table whose row-cache block is rewritten while it already
+        // exists lost its Compressed flag, so reopening and running SELECT parsed raw
+        // Brotli bytes as JSON (JsonException "'0x0B' is an invalid start of a value").
+        var factory = BuildFactory();
+        var options = CreateCompressedOptions(BlockCompressionMode.Brotli);
+
+        // Auto-flush rewrites the row-cache block as it grows past the compression
+        // threshold — exercising the existing-block rewrite path that lost the flag.
+        var db = factory.CreateWithOptions(_testDbPath, "unused", options);
+        db.ExecuteSQL("CREATE TABLE t (id INTEGER PRIMARY KEY, name TEXT, payload TEXT)");
+        for (int i = 0; i < 500; i++)
+        {
+            db.ExecuteSQL($"INSERT INTO t VALUES ({i}, 'user{i}', '{new string('x', 32)}')");
+        }
+        db.Flush();
+        db.ForceSave();
+        DisposeDatabase(db);
+
+        // Reopen and SELECT must survive with the data intact.
+        var db2 = factory.CreateWithOptions(_testDbPath, "unused", options);
+        var results = db2.ExecuteQuery("SELECT * FROM t ORDER BY id");
+        Assert.Equal(500, results.Count);
+        Assert.Equal("user42", results[42]["name"]?.ToString());
+        Assert.Equal("user499", results[499]["name"]?.ToString());
+        DisposeDatabase(db2);
+    }
+
+    [Fact]
+    public void DatabaseFactory_CompressionPlusEncryption_GrowingTable_ReopenSelectShouldSurvive()
+    {
+        // Same regression as above but with full at-rest encryption combined with
+        // block compression (compression + encryption on the existing-block path).
+        var factory = BuildFactory();
+        var options = CreateCompressedOptions(BlockCompressionMode.Brotli);
+        options.EnableEncryption = true;
+        options.EncryptionKey = RandomNumberGenerator.GetBytes(32);
+
+        var db = factory.CreateWithOptions(_testDbPath, "unused", options);
+        db.ExecuteSQL("CREATE TABLE t (id INTEGER PRIMARY KEY, name TEXT, payload TEXT)");
+        for (int i = 0; i < 500; i++)
+        {
+            db.ExecuteSQL($"INSERT INTO t VALUES ({i}, 'user{i}', '{new string('y', 32)}')");
+        }
+        db.Flush();
+        db.ForceSave();
+        DisposeDatabase(db);
+
+        var db2 = factory.CreateWithOptions(_testDbPath, "unused", options);
+        var results = db2.ExecuteQuery("SELECT * FROM t ORDER BY id");
+        Assert.Equal(500, results.Count);
+        Assert.Equal("user7", results[7]["name"]?.ToString());
+        Assert.Equal("user499", results[499]["name"]?.ToString());
+        DisposeDatabase(db2);
+    }
+
+
     [Fact]
     public void DatabaseFactory_WrongCompressionModeOnReopen_ShouldThrow()
     {