Skip to content

Memoize.function: fix cross-closure cache collision - #3

Merged
yannrichet-asnr merged 1 commit into
masterfrom
fix-memoize-cross-closure-collision
Sep 4, 2026
Merged

yannrichet-asnr merged 1 commit into
masterfrom
fix-memoize-cross-closure-collision

Conversation

@yannrichet

Copy link
Copy Markdown
Contributor

Bug

Memoize.function(fun)'s cache (R.cache::loadCache/saveCache) keys purely on the call arguments -- not on which fun is being wrapped. Two independent Memoize.function(fun) wrappers that happen to share the default cache suffix and are called with the same argument value will collide: the second wrapper silently gets served the first wrapper's cached result instead of ever evaluating its own fun.

Reproduced:

make_fun <- function(offset) function(x) x + offset
f1 <- Memoize.function(make_fun(3000))
f2 <- Memoize.function(make_fun(4000))
f1(5)  # 3005 (correct)
f2(5)  # 3005 -- WRONG, should be 4005. Silently returned f1's cached value.

Where this actually bites

This is exactly the shape produced by wrapping a fresh closure in a loop over several models/datasets -- e.g. cristalgo's EGRI/EGRIw/WEGRI robust-inversion algorithms, which do mesh_exsets(f = Memoize.function(fi), ...) once per seed inside a for (seed in ...) loop, where fi closes over that seed's own fitted Kriging/WarpKriging model. Since root()'s bisection deterministically probes similar midpoints over the same fixed input domain across seeds, a later seed's fi(xi) can be silently served an earlier seed's stale cached value at the same xi.

root() assumes f is a single, self-consistent function throughout its bisection; a corrupted value breaks that invariant. root() also has no working recursion-depth guard (max.rec defaults to NA, so isTRUE(rec > max.rec) never fires), so a single bad value is enough to make it recurse until R's node/pointer-protection stack overflows -- observed in practice as a Slurm job hanging 80+ minutes with zero progress on one seed (while 9 other seeds using the identical code completed normally in under a minute each). Confirmed by reproducing the exact hung state and showing that (a) the underlying model's predict() and the wrapped criterion function are both fast and well-behaved in isolation, (b) the same call through mesh_exsets() reproduces the hang, and (c) removing only the Memoize.function() wrapper (or, as here, fixing it) makes the identical call complete normally.

Fix

Each call to Memoize.function() now gets its own private cache namespace (.memoize_id, generated once per wrapping via tempfile() and folded into the cache key), so independent wrappings can never collide regardless of argument overlap. The documented single-wrapper behavior (F <- Memoize.function(f); F(5); F(6); F(5) -- second F(5) hits cache) is unchanged; verified with identical() before/after.

🤖 Generated with Claude Code

https://claude.ai/code/session_01A5w4ZLUZpoHhtsbGyWcVtc

loadCache()/saveCache() key on the raw call arguments only, so two
different Memoize.function(fun) wrappers sharing the default cache
`suffix` can collide whenever they're called with the same argument
value -- one wrapper silently gets served *another* wrapper's cached
result instead of evaluating its own `fun`.

Reproduced:
  make_fun <- function(offset) function(x) x + offset
  f1 <- Memoize.function(make_fun(3000))
  f2 <- Memoize.function(make_fun(4000))
  f1(5)  # 3005 (correct)
  f2(5)  # 3005 (WRONG: should be 4005 -- served f1's cached value)

This is exactly the shape that arises from wrapping a fresh closure in a
loop over several models/datasets (e.g. cristalgo's EGRI/EGRIw/WEGRI
robust-inversion algorithms: `mesh_exsets(f = Memoize.function(fi), ...)`
inside a per-seed loop, where `fi` closes over that seed's own fitted
model). A stale cross-seed value fed into a root-finder that assumes it is
bisecting one consistent function silently corrupts its bisection
invariants; DiceView:::root() also has no working recursion-depth guard
(`max.rec` defaults to NA), so a single corrupted value is enough to make
it recurse until R's node stack overflows.

Fix: each call to Memoize.function() now gets its own private cache
namespace (`.memoize_id`, folded into the cache key), so independent
wrappings can never collide regardless of argument overlap, while a single
wrapper's own repeated-argument caching (the documented example:
`F <- Memoize.function(f); F(5); F(6); F(5)`) is unchanged.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A5w4ZLUZpoHhtsbGyWcVtc
@yannrichet-asnr
yannrichet-asnr merged commit 1d44405 into master Sep 4, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants