Reduce custom list sizing overhead and stabilize expiration tests

.agents/skills/foundatio/SKILL.md

@@ -317,7 +317,7 @@ public class OrderServiceTests : TestLoggerBase
 - **JobWithLockBase vs manual locking**: Use `JobWithLockBase` when the entire run must be single-instance (leader election). Use manual `ILockProvider.AcquireAsync` inside `JobBase` for finer-grained locking within a job.
 - **JobContext.RenewLockAsync**: Call in long-running jobs (both `JobBase` and `QueueJobBase`) to prevent lock expiration mid-processing.
 - **In-memory expired entries**: Writes treat an expired entry as missing before maintenance sweeps it (as Redis does): increments and `SetIfHigher`/`SetIfLower` start fresh, `ReplaceAsync`, `SetExpirationAsync`, `RemoveIfEqualAsync` and `ReplaceIfEqualAsync` miss. The write reclaims the expired entry and raises `ItemExpired` once. Successful live conditional removal and empty-list removal free memory without that event. Size-rejected updates preserve value, TTL, and tracked size; this also applies when a throwing custom calculator produces an oversized fallback estimate. `ReplaceIfEqualAsync` also rejects a replacement that expires during preparation or a retry, preserving a still-live entry.
-- **Custom `SizeCalculator` and lists**: runs while a per-key-stripe lock may be held during contended list writes, so it must not call back into the cache. By default it receives a `Dictionary<T, DateTime?>` copy of the whole list on each write. Interface-based calculators that never mutate inputs can opt in via `SizeCalculator(calculateSize, supportsReadOnlyCollections: true)` to avoid that copy for immutable lists; both ordinary dictionaries and read-only views must be supported. Built-in `WithDynamicSizing`/`WithFixedSizing` already avoid the copy.
+- **Custom `SizeCalculator` and lists**: runs while a per-key-stripe lock may be held during contended list writes, so it must not call back into the cache. By default it receives a `Dictionary<T, DateTime?>` copy of the whole list on each changed write; disabling cloning does not remove that allocation. Interface-based calculators that never mutate inputs can opt in via `SizeCalculator(calculateSize, supportsReadOnlyCollections: true)` to avoid that copy for immutable lists; both ordinary dictionaries and read-only views must be supported. Built-in `WithDynamicSizing`/`WithFixedSizing` already avoid the copy.
 - **In-memory numeric TTLs**: `SetIfHigherAsync` and `SetIfLowerAsync` apply the requested expiration even when the numeric value is unchanged. A zero return value does not mean the TTL was preserved; `null` clears it. Size-rejected updates remain unchanged.
 - **In-memory lists and cloning**: List writes preserve supported dictionary types and comparers; copying an unsupported `IDictionary` throws `NotSupportedException` without changing the entry. Batch large updates because cloning, custom sizing, and raw dictionary access can require whole-list copies. `Items` honors `CloneValues` without affecting LRU. Cloned dictionary reads preserve incremental list writes; uncloned dictionary reads expose a shared copy and require subsequent writers to copy it. Collection snapshots do not isolate shared mutable elements when cloning is disabled. See the [caching guide](https://foundatio.readthedocs.io/guide/caching.html#in-memory-update-behavior) for details. Adding existing list items still reapplies their resulting expiration, even when the list contents are unchanged.
 - **Register as singletons**: All infrastructure services (`ICacheClient`, `IMessageBus`, `IQueue<T>`, `IFileStorage`, `ILockProvider`) maintain internal state and connections -- always register as singletons.

docs/guide/caching.md

@@ -281,15 +281,15 @@ Successful removal of a live matching value, or removal of the last list value,
 - List size limits (`MaxEntrySize`, a negative `SizeCalculator` result) apply to the whole resulting list on every `ListAddAsync` and `ListRemoveAsync`, so a removal that leaves the list over the limit is rejected and returns `0`.
 - A rejected `ListAddAsync` on an expired key still reclaims the expired entry and raises `ItemExpired`, and every `ListAddAsync` call counts as a write in `Writes`, including rejected ones.
 - Adding items reapplies the resulting list expiration, even when every supplied item is already present with the same TTL. This can replace a key expiration previously set with `SetExpirationAsync`.
-- List item expirations read back with `DateTimeKind.Utc`, whatever kind was passed in.
+- Raw dictionary list expirations may change to `DateTimeKind.Utc` when an incremental update builds the snapshot index; preserving `DateTime.Kind` is not guaranteed.
 :::
 
 List updates preserve the collection structure of previously returned snapshots. They preserve the comparers of `Dictionary`, `SortedDictionary`, `SortedList`, `ConcurrentDictionary`, and, on .NET 9 or later, `OrderedDictionary`. Other `IDictionary` implementations throw `NotSupportedException` when an update requires copying them; the original entry remains unchanged.
 
 `Items` returns cached values without changing eviction order. Both `Items` and ordinary reads honor `CloneValues`. With cloning disabled, mutable payloads and list elements can remain shared: a collection snapshot is not a deep copy. Do not mutate shared objects concurrently with cache operations.
 
 ::: tip Large list updates
-Batch values into a single `ListAddAsync` or `ListRemoveAsync` call. Lists over 32 items are stored as immutable snapshots, so a small change copies only the parts it touches, including with `WithDynamicSizing` or `WithFixedSizing`. A custom `SizeCalculator` receives a `Dictionary<T, DateTime?>` copy of the whole list on each write by default. Calculators that use collection interfaces can opt in to read-only views with `SizeCalculator(calculateSize, supportsReadOnlyCollections: true)` (or `SizeCalculatorSupportsReadOnlyCollections` on the options). They must accept ordinary dictionaries and read-only dictionary views and must not modify either. This avoids the sizing copy for immutable lists; it does not change cache reads, element cloning, or the cost of enumerating all items. Sampling `IReadOnlyDictionary.Keys` or `.Values` enumerates lazily. Lists that shrink substantially rebuild their sparse storage on a subsequent write; that occasional full copy prevents ongoing costs from remaining tied to the historical peak size. Prefer `WithFixedSizing` for a constant estimate. With `CloneValues` on, snapshots apply to lists of strings and of value types without reference fields; other lists are copied on each write. Raw dictionary access (`GetAsync<IDictionary<T, DateTime?>>`) with cloning disabled materializes a shared dictionary and makes subsequent writes copy it. With cloning enabled, dictionary reads and `Items` return independent copies without disabling incremental writes. Benchmark representative list sizes and read/write patterns before choosing these options; atomic updates do not make every workload allocation-free.
+Batch values into a single `ListAddAsync` or `ListRemoveAsync` call. Lists over 32 items are stored as immutable snapshots, so a small change copies only the parts it touches, including with `WithDynamicSizing` or `WithFixedSizing`. A custom `SizeCalculator` receives a `Dictionary<T, DateTime?>` copy of the whole list on each changed write by default. This remains an allocation proportional to list size even with cloning disabled. Calculators that use collection interfaces can opt in to read-only views with `SizeCalculator(calculateSize, supportsReadOnlyCollections: true)` (or `SizeCalculatorSupportsReadOnlyCollections` on the options). They must accept ordinary dictionaries and read-only dictionary views and must not modify either. This avoids the sizing copy for immutable lists; it does not change cache reads, element cloning, or the cost of enumerating all items. Sampling `IReadOnlyDictionary.Keys` or `.Values` enumerates lazily. Lists that shrink substantially rebuild their sparse storage on a subsequent write; that occasional full copy prevents ongoing costs from remaining tied to the historical peak size. Prefer `WithFixedSizing` for a constant estimate. With `CloneValues` on, snapshots apply to lists of strings and of value types without reference fields; other lists are copied on each write. Raw dictionary access (`GetAsync<IDictionary<T, DateTime?>>`) with cloning disabled materializes a shared dictionary and makes subsequent writes copy it. With cloning enabled, dictionary reads and `Items` return independent copies without disabling incremental writes. Benchmark representative list sizes and read/write patterns before choosing these options; atomic updates do not make every workload allocation-free.
 :::
 
 ### HybridCacheClient

src/Foundatio.TestHarness/Caching/CacheClientTestsBase.cs

@@ -1274,25 +1274,22 @@ public virtual async Task ListAddAsync_WithExpiration_SetsExpirationCorrectly()
             Assert.NotNull(expiration);
             Assert.InRange(expiration.Value, TimeSpan.FromMinutes(59), TimeSpan.FromHours(1));
 
-            // Multiple expirations expire individual items - test staggered expiration in one pass
+            // Keep the other item permanent so scheduler delays cannot expire both before the assertion.
             const string key = "list:staggered-expiration";
+            Assert.Equal(1, await cache.ListAddAsync(key, [3]));
             Assert.Equal(1, await cache.ListAddAsync(key, [2], TimeSpan.FromMilliseconds(50)));
-            Assert.Equal(1, await cache.ListAddAsync(key, [3], TimeSpan.FromMilliseconds(150)));
 
+            await Task.Delay(200, TestCancellationToken);
             var cacheValue = await cache.GetListAsync<int>(key);
             Assert.True(cacheValue.HasValue);
-            Assert.Equal(2, cacheValue.Value.Count);
-
-            // Wait for first item to expire
-            await Task.Delay(75);
-            cacheValue = await cache.GetListAsync<int>(key);
-            Assert.True(cacheValue.HasValue);
             Assert.Single(cacheValue.Value);
             Assert.Contains(3, cacheValue.Value);
-            Assert.DoesNotContain(2, cacheValue.Value); // Explicit verification item 2 expired
+            Assert.DoesNotContain(2, cacheValue.Value);
 
-            // Wait for second item to expire
-            await Task.Delay(100, TestCancellationToken);
+            // Verify whole-key expiration on a new list, without relying on shortening a permanent key's TTL.
+            Assert.Equal(1, await cache.ListRemoveAsync(key, [3]));
+            Assert.Equal(1, await cache.ListAddAsync(key, [3], TimeSpan.FromMilliseconds(50)));
+            await Task.Delay(200, TestCancellationToken);
             Assert.False(await cache.ExistsAsync(key));
 
             // Null expiration: should succeed and remove expiration

src/Foundatio/Caching/InMemoryCacheClient.cs

@@ -682,6 +682,16 @@ private long ListAdd<T>(string key, Dictionary<T, DateTime?> items, DateTime? ex
         {
             if (current?.StoredValue is ListSnapshot<T> { IsSourceOfTruth: true } stored)
             {
+                if (_hasSizeCalculator && !_sizesListsInPlace)
+                {
+                    var values = stored.CopyAndUpdate(items.Keys, expiresAt, _timeProvider.GetUtcNow().UtcDateTime, remove: false, out _);
+                    if (values is null)
+                        return (current.WithExpiration(stored.ExpiresAtUtc), (long)items.Count);
+
+                    long replacementSize = CalculateEntrySize(values);
+                    return replacementSize < 0 ? (current, 0L) : (WithListValues(current, values, replacementSize), (long)items.Count);
+                }
+
                 var list = stored.Update(items.Keys, expiresAt, _timeProvider.GetUtcNow().UtcDateTime, remove: false, out _);
                 if (ReferenceEquals(list, stored))
                     return (current.WithExpiration(list.ExpiresAtUtc), (long)items.Count);
@@ -733,23 +743,25 @@ private static class ListItemCloning<T>
     }
 
     /// <summary>
-    /// Sizes a shared list without copying when the calculator supports read-only collections. Other calculators
-    /// keep receiving a <see cref="Dictionary{TKey,TValue}"/>, since they may cast to it.
+    /// Sizes a shared list without copying. Calculators requiring a concrete dictionary use CopyAndUpdate instead.
     /// </summary>
     private long CalculateListSize<T>(ListSnapshot<T> list) where T : notnull
     {
         if (!_hasSizeCalculator)
             return 0;
 
-        return CalculateEntrySize(_sizesListsInPlace ? list.AsReadOnlyDictionary() : list.ToDictionary());
+        return CalculateEntrySize(list.AsReadOnlyDictionary());
     }
 
     private CacheEntry WithListValues<T>(CacheEntry current, IDictionary<T, DateTime?> dictionary, long size) where T : notnull
     {
-        object value = CanShareListValues<T>() && dictionary is Dictionary<T, DateTime?> values && values.Count > ListSnapshot<T>.BlockSize
-            ? new ListSnapshot<T>(values)
-            : dictionary;
-        return current.WithValue(value, GetListExpiration(dictionary), size);
+        if (CanShareListValues<T>() && dictionary is Dictionary<T, DateTime?> values && values.Count > ListSnapshot<T>.BlockSize)
+        {
+            var list = new ListSnapshot<T>(values);
+            return current.WithValue(list, list.ExpiresAtUtc, size);
+        }
+
+        return current.WithValue(dictionary, GetListExpiration(dictionary), size);
     }
 
     /// <summary>
@@ -812,6 +824,18 @@ private long ListRemove<T>(string key, HashSet<T> items) where T : notnull
         {
             if (current?.StoredValue is ListSnapshot<T> { IsSourceOfTruth: true } list)
             {
+                if (_hasSizeCalculator && !_sizesListsInPlace)
+                {
+                    var values = list.CopyAndUpdate(items, null, _timeProvider.GetUtcNow().UtcDateTime, remove: true, out long copiedRemovedCount);
+                    if (values is null)
+                        return (current, 0L);
+                    if (values.Count == 0)
+                        return (null, copiedRemovedCount);
+
+                    long replacementSize = CalculateEntrySize(values);
+                    return replacementSize < 0 ? (current, 0L) : (WithListValues(current, values, replacementSize), copiedRemovedCount);
+                }
+
                 var replacement = list.Update(items, null, _timeProvider.GetUtcNow().UtcDateTime, remove: true, out long count);
                 if (ReferenceEquals(list, replacement))
                     return (current, 0L);

src/Foundatio/Caching/InMemoryListSnapshot.cs

@@ -158,16 +158,8 @@ public ListSnapshot<T> Update(ICollection<T> items, DateTime? expiresAtUtc, Date
         bool compact = Volatile.Read(ref _layout) is { } layout && Count < layout.NextSlot / 4;
         if (batch || compact || (!IsIndexed && !_fromIncrementalChange))
         {
-            var dictionary = ToDictionary();
-            if (prune)
-                foreach (var key in dictionary.Where(pair => pair.Value?.Ticks < utcNowTicks).Select(pair => pair.Key).ToArray())
-                    dictionary.Remove(key);
-            if (remove)
-                removed = items.Count(dictionary.Remove);
-            else
-                foreach (var key in items)
-                    dictionary[key] = expiresAtUtc;
-            return new ListSnapshot<T>(dictionary, fromIncrementalChange: !batch);
+            var dictionary = CopyAndUpdate(items, expiresAtUtc, utcNow, remove, out removed);
+            return dictionary is null ? this : new ListSnapshot<T>(dictionary, fromIncrementalChange: !batch);
         }
 
         var update = new LayoutUpdate(GetLayout(), _comparer, Count);
@@ -191,6 +183,29 @@ public ListSnapshot<T> Update(ICollection<T> items, DateTime? expiresAtUtc, Date
         return update.Changed ? new ListSnapshot<T>(update.Build(), _comparer, update.Count) : this;
     }
 
+    // Custom calculators requiring a concrete dictionary can update that copy directly, without building an index
+    // that would immediately be flattened again for sizing. An unchanged list needs no copy unless pruning is due.
+    public Dictionary<T, DateTime?>? CopyAndUpdate(ICollection<T> items, DateTime? expiresAtUtc, DateTime utcNow, bool remove, out long removed)
+    {
+        removed = 0;
+        bool prune = _minExpiresAtUtcTicks < utcNow.Ticks;
+        if (!prune && (remove
+            ? !items.Any(ContainsKey)
+            : items.All(key => TryGetExpiresAtUtc(key, out var current) && current == expiresAtUtc)))
+            return null;
+
+        var dictionary = ToDictionary();
+        if (prune)
+            foreach (var key in dictionary.Where(pair => pair.Value < utcNow).Select(pair => pair.Key).ToArray())
+                dictionary.Remove(key);
+        if (remove)
+            removed = items.Count(dictionary.Remove);
+        else
+            foreach (var key in items)
+                dictionary[key] = expiresAtUtc;
+        return dictionary;
+    }
+
     private bool ContainsKey(T key) => TryGetExpiresAtUtc(key, out _);
 
     private bool TryGetExpiresAtUtc(T key, out DateTime? expiresAtUtc)