Skip to content

fix(libsy): preserve retained sessions on cache updates - #1190

Open
Yushu-Li wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
Yushu-Li:fix/retained-routing-capacity
Open

Yushu-Li wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
Yushu-Li:fix/retained-routing-capacity

Conversation

@Yushu-Li

@Yushu-Li Yushu-Li commented Oct 31, 2026 •

Copy link
Copy Markdown

What

I fixed retained-routing eviction in AffinityRouter and the composite router's TierSetter. The shared eviction helper now receives the incoming identity and makes room only when that identity is new.

Why

With classify_trigger = "user_turn", an existing session gets a new retained decision when the user speaks. At the 4096-entry limit, both maps previously evicted an unrelated session before overwriting the existing entry, leaving only 4095 entries.

The lost affinity can send that other session's tool continuation back through the judge. A composite route can instead lose its retained tier and use its default. Updating an existing identity needs no additional slot; the cache limit and eviction behavior for new identities stay the same.

Notes for reviewers

  • Two regressions fill the maps, update an existing identity other than the one eviction would remove, and verify that the updated and unrelated sessions survive. The affinity test also checks the retained targets on tool continuations. Both tests fail before the fix with 4095 != 4096, and pass after it.
  • The existing capacity test still verifies that a new identity at the limit causes eviction.
  • No configuration or public API changes. The shared capacity constant is visible within the crate so both test modules use the actual limit.

Local checks passed on macOS arm64 with Rust 1.99.0 and Python 3.12:

  • cargo fmt --all --check
  • cargo clippy --workspace --all-targets --locked -- -D warnings
  • cargo test --workspace --locked: 1131 passed, one existing ignored test.
  • ruff check . and mypy switchyard
  • Built the Python extension with maturin develop --locked --uv, then ran pytest tests/ -v -m "not integration": 157 passed, two integration tests deselected.

The local Rust binding checks used a PyO3 library-path override for the relocated Python runtime. No repository changes were needed for that setup.

I used AI assistance for implementation, regression tests, and local validation. No live model calls or paid services were used.

Summary by CodeRabbit

  • Bug Fixes
    • Updating an existing session when the assignment limit is reached no longer removes another session’s assignment.
    • The assignment map stays within its capacity, the updated session receives its new tier, and other sessions retain their existing tiers.

Signed-off-by: Yushu Li <eeyushuli@gmail.com>
@Yushu-Li
Yushu-Li requested a review from a team as a code owner October 31, 2026 00:17
@coderabbitai

coderabbitai Bot commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 1faddb4 and f8b23e6.

Files selected for processing: 2
  • crates/libsy/src/algorithms/composite.rs
  • crates/libsy/src/algorithms/util/affinity.rs

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Walkthrough

The eviction helper now receives the identity being inserted or updated. At capacity, it evicts an assignment only when that identity is not already present. Regression tests check that updating an existing assignment preserves the map capacity and other assignments.

Changes

Assignment eviction

Layer / File(s) Summary
Identity-aware eviction
crates/libsy/src/algorithms/util/affinity.rs, crates/libsy/src/algorithms/composite.rs
evict_if_full now checks whether the incoming identity already exists before evicting. Call sites pass the identity. Regression tests verify that updates preserve map capacity and other assignments.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: Minimal · up to f8b23

The change preserves retained sessions during updates. No merge-blocking issue is identified.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: preserving retained sessions when updating the cache.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 5 functions across 2 files.

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

A rabbit checks the map at night
An old identity stays in sight
No neighbor hops away
The count holds through the day
New sessions find their place just right

Comment @coderabbitai help to get the list of available commands.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant