fix!: unify LLM stream close behavior#465
Conversation
Signed-off-by: Will Killian <wkillian@nvidia.com>
github-actions
Bot
added
size:M
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughStreaming LLM plumbing now supports cooperative cancellation and explicit completion signaling across Rust, Python, Node.js, FFI, and Go. Stream wrappers expose idempotent close operations, wait for producer cleanup, and finalize partial responses. ChangesStreaming lifecycle coordination
Estimated code review effort: 4 (Complex) | ~60 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
crates/python/src/py_callable.rs (1)
228-235: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winEnsure receiver shutdown and errors finalize the Python iterator.
crates/python/src/py_callable.rs#L228-L235: race error delivery against cancellation and callclose_async_iterbefore every terminal exit.crates/python/tests/coverage/py_callable_coverage_tests.rs#L387-L399: use an observableaclose()implementation and assert receiver shutdown invokes it.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/python/src/py_callable.rs` around lines 228 - 235, Update the receiver loop in py_callable.rs around the terminal branches to race error delivery against cancellation, call close_async_iter before every terminal exit including receiver shutdown and send failures, and preserve the existing break behavior. In crates/python/tests/coverage/py_callable_coverage_tests.rs lines 387-399, replace the iterator’s no-op close behavior with an observable aclose() implementation and assert that receiver shutdown invokes it.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/python/tests/coverage/py_callable_coverage_tests.rs`:
- Around line 387-399: Update the test around forward_async_iter to give the
Python iterator an observable aclose() method, then assert that aclose() is
invoked after the receiver is dropped. Keep the existing dropped-receiver setup
and await behavior, but verify the iterator cleanup side effect rather than only
confirming that forwarding returns.
---
Outside diff comments:
In `@crates/python/src/py_callable.rs`:
- Around line 228-235: Update the receiver loop in py_callable.rs around the
terminal branches to race error delivery against cancellation, call
close_async_iter before every terminal exit including receiver shutdown and send
failures, and preserve the existing break behavior. In
crates/python/tests/coverage/py_callable_coverage_tests.rs lines 387-399,
replace the iterator’s no-op close behavior with an observable aclose()
implementation and assert that receiver shutdown invokes it.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: d9a757f0-b569-48c9-a917-277eb89e1846
📒 Files selected for processing (10)
crates/python/src/py_api/mod.rscrates/python/src/py_callable.rscrates/python/src/py_types/core.rscrates/python/tests/coverage/coverage_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/python/tests/coverage/py_types_coverage_tests.rspython/nemo_relay/_native.pyipython/nemo_relay/llm.pypython/tests/test_llm.py
📜 Review details
⏰ Context from checks skipped due to timeout. (2)
- GitHub Check: Check / Run
- GitHub Check: Preview docs
🧰 Additional context used
📓 Path-based instructions (18)
{crates/python/src/py_api/mod.rs,python/nemo_relay/**/*.py,python/nemo_relay/**/*.pyi,go/nemo_relay/**/*.go,crates/node/src/api/**/*.rs}
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Update the language-native bindings for every exposed surface in Python, Go, and Node.js.
Files:
python/nemo_relay/_native.pyipython/nemo_relay/llm.pycrates/python/src/py_api/mod.rs
{python/nemo_relay/**/*.py,python/nemo_relay/**/*.pyi,go/nemo_relay/**/*.go}
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Update language wrapper helpers such as Python wrapper modules, Python type stubs, and Go shorthand packages when the new behavior belongs in those helper layers.
Files:
python/nemo_relay/_native.pyipython/nemo_relay/llm.py
**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*: Format changed files with the language-native formatter before the final lint/test pass.
If dynamic plugin behavior changed, usemaintain-dynamic-pluginsand include the native SDK, worker protocol, Python SDK, docs, packaging, and Codecov surfaces in the validation plan.
If code changes alter APIs, bindings, commands, paths, packaging behavior, observability/adaptive semantics, or documented best practices, update any dependent maintainer or consumer skills in the same branch.
During iteration, preferuv run pre-commit run --files <changed files...>.
Before review or handoff, runuv run pre-commit run --all-files.
Files:
python/nemo_relay/_native.pyicrates/python/tests/coverage/coverage_tests.rspython/nemo_relay/llm.pycrates/python/tests/coverage/py_callable_coverage_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/python/tests/coverage/py_types_coverage_tests.rspython/tests/test_llm.pycrates/python/src/py_api/mod.rscrates/python/src/py_types/core.rscrates/python/src/py_callable.rs
python/nemo_relay/**/*
⚙️ CodeRabbit configuration file
python/nemo_relay/**/*: Review Python wrapper changes for typed API consistency, contextvars-based scope isolation, async behavior, and parity with the native extension.
Stubs and runtime implementations should stay aligned.
Files:
python/nemo_relay/_native.pyipython/nemo_relay/llm.py
**/*.rs
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
**/*.rs: Any Rust change must runjust test-rust
Any Rust change must runcargo fmt --all
Any Rust change must runcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allfor all FFI work since it is Rust work
Runjust test-rustto validate FFI changes
Runcargo clippy --workspace --all-targets -- -D warningsto enforce strict linting on FFI workWhen Rust files changed as part of Go work, also run
cargo fmt --all,just test-rust, andcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allwhen Rust files are changed as part of Node work
Runcargo clippy --workspace --all-targets -- -D warningswhen Rust files are changed as part of Node work
Runjust test-rustwhen Rust files are changed as part of Node workWhen changing the core Rust runtime or Rust-facing API surface, format Rust code with
cargo fmt(rustfmt defaults), keepcargo clippy -- -D warningsclean, and satisfycargo deny checkperdeny.toml.
**/*.rs: If any Rust code changed, always runjust test-rust.
If any Rust code changed, also runcargo fmt --all.
If any Rust code changed, also runcargo clippy --workspace --all-targets -- -D warnings.
For Rust changes headed for review, runcargo fmt --allandcargo clippy --workspace --all-targets -- -D warningseven if relying on pre-commit.
Files:
crates/python/tests/coverage/coverage_tests.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/python/tests/coverage/py_types_coverage_tests.rscrates/python/src/py_api/mod.rscrates/python/src/py_types/core.rscrates/python/src/py_callable.rs
**/*.{rs,py}
📄 CodeRabbit inference engine (AGENTS.md)
Follow binding naming conventions in Rust and Python: use
snake_case.
Files:
crates/python/tests/coverage/coverage_tests.rspython/nemo_relay/llm.pycrates/python/tests/coverage/py_callable_coverage_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/python/tests/coverage/py_types_coverage_tests.rspython/tests/test_llm.pycrates/python/src/py_api/mod.rscrates/python/src/py_types/core.rscrates/python/src/py_callable.rs
**/*.{rs,py,js,mjs,cjs,ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{rs,py,js,mjs,cjs,ts,tsx}: UseJson = serde_json::Valuein Rust-facing runtime APIs where the existing code expects JSON payloads.
UseResult<T>withFlowErrorin core runtime paths, and keep errors explicit and binding-appropriate at the wrapper layer.
Keep async behavior on the existing tokio-based model; bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.
Files:
crates/python/tests/coverage/coverage_tests.rspython/nemo_relay/llm.pycrates/python/tests/coverage/py_callable_coverage_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/python/tests/coverage/py_types_coverage_tests.rspython/tests/test_llm.pycrates/python/src/py_api/mod.rscrates/python/src/py_types/core.rscrates/python/src/py_callable.rs
**/*.{rs,py,go,js,ts,c,h}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Use language-appropriate naming conventions: Rust
snake_case, C FFI exports prefixednemo_relay_, GoPascalCase, Node.jscamelCase, and Pythonsnake_case.
Files:
crates/python/tests/coverage/coverage_tests.rspython/nemo_relay/llm.pycrates/python/tests/coverage/py_callable_coverage_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/python/tests/coverage/py_types_coverage_tests.rspython/tests/test_llm.pycrates/python/src/py_api/mod.rscrates/python/src/py_types/core.rscrates/python/src/py_callable.rs
**/*.{rs,go,js,ts}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Add the SPDX license header to all Rust, Go, JavaScript, and TypeScript source files using the corresponding
//comment form.
Files:
crates/python/tests/coverage/coverage_tests.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/python/tests/coverage/py_types_coverage_tests.rscrates/python/src/py_api/mod.rscrates/python/src/py_types/core.rscrates/python/src/py_callable.rs
**/*.{rs,py,go,js,ts}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If a language surface changed, always run that language's test target even when Rust core did not change.
Files:
crates/python/tests/coverage/coverage_tests.rspython/nemo_relay/llm.pycrates/python/tests/coverage/py_callable_coverage_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/python/tests/coverage/py_types_coverage_tests.rspython/tests/test_llm.pycrates/python/src/py_api/mod.rscrates/python/src/py_types/core.rscrates/python/src/py_callable.rs
**/*.{rs,py,js,ts,tsx,go,java,kt,swift}
📄 CodeRabbit inference engine (.agents/skills/add-middleware/SKILL.md)
Add tests covering registration and duplicate names, deregistration and missing names, priority ordering, callback failure policy, scope-local inheritance and cleanup, event payload semantics, immutable mark and scope fields, and parity across affected bindings.
Files:
crates/python/tests/coverage/coverage_tests.rspython/nemo_relay/llm.pycrates/python/tests/coverage/py_callable_coverage_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/python/tests/coverage/py_types_coverage_tests.rspython/tests/test_llm.pycrates/python/src/py_api/mod.rscrates/python/src/py_types/core.rscrates/python/src/py_callable.rs
crates/{python,ffi,node}/**/*
⚙️ CodeRabbit configuration file
crates/{python,ffi,node}/**/*: Treat binding changes as public API changes. Check for parity with the other language bindings, FFI ownership/lifetime safety,
callback error propagation, stable type conversion, and consistent async/stream semantics.
Flag changes that update one binding without corresponding tests or documentation for the same surface elsewhere.
Files:
crates/python/tests/coverage/coverage_tests.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/python/tests/coverage/py_types_coverage_tests.rscrates/python/src/py_api/mod.rscrates/python/src/py_types/core.rscrates/python/src/py_callable.rs
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}
⚙️ CodeRabbit configuration file
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.
Files:
crates/python/tests/coverage/coverage_tests.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/python/tests/coverage/py_types_coverage_tests.rspython/tests/test_llm.py
python/nemo_relay/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Python wrapper modules live under
python/nemo_relay/, and the native extension is built fromcrates/pythonwithmaturin.
Files:
python/nemo_relay/llm.py
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
**/*.py: When changing the Python wrapper package, tests, or docs tooling, lint with Ruff (E,F,W,I), format with Ruff formatter (120-character lines, double quotes), and passtytype checking.
Add the SPDX license header to all Python source files using the#comment form.
Files:
python/nemo_relay/llm.pypython/tests/test_llm.py
{crates/**/src/**/*.rs,python/**/*.py}
📄 CodeRabbit inference engine (.agents/skills/maintain-dynamic-plugins/SKILL.md)
Do not add tests under
src; Rust tests belong in cratetests/trees, and Python SDK tests belong underpython/tests.
Files:
python/nemo_relay/llm.pypython/tests/test_llm.pycrates/python/src/py_api/mod.rscrates/python/src/py_types/core.rscrates/python/src/py_callable.rs
**/*.{py,go,js,ts}
📄 CodeRabbit inference engine (.agents/skills/maintain-observability/SKILL.md)
Keep Python, Go, and Node.js config objects and subscriber/exporter methods aligned so all bindings expose the same logical knobs and semantics.
Files:
python/nemo_relay/llm.pypython/tests/test_llm.py
python/tests/**/*.py
📄 CodeRabbit inference engine (.agents/skills/test-python-binding/SKILL.md)
python/tests/**/*.py: Pytest is used to run tests.
Do not add@pytest.mark.asyncioto any test; async tests are automatically detected and run by the async runner.
Do not add a-> Nonereturn type annotation to test functions.
When mocking a class, do not define a new class; useunittest.mock.MagicMockorunittest.mock.AsyncMock, with thespecconstructor argument when necessary.
Name mocked classes with themockprefix, notfake.
Prefer pytest fixtures over helper methods.
Do not repeat fixtures; if a fixture is needed in multiple test files, place it in aconftest.pyfile.
When creating a fixture, use@pytest.fixture(name="<fixture_name>"[, scope="<scope>"])and define the fixture function asdef <fixture_name>_fixture() -> <return_type>:; only specifyscopewhen it is notfunction.
Preferpytest.mark.parametrizeover creating individual tests for different input types.
Files:
python/tests/test_llm.py
🪛 Ruff (0.15.21)
python/tests/test_llm.py
[warning] 520-520: Missing return type annotation for private function stream_func
(ANN202)
[warning] 520-520: Unused function argument: request
(ARG001)
[warning] 531-531: Unused lambda argument: chunk
(ARG005)
[warning] 532-532: Prefer dict over useless lambda
Replace with lambda with dict
(PIE807)
🔇 Additional comments (10)
crates/python/tests/coverage/coverage_tests.rs (1)
223-226: LGTM!crates/python/tests/coverage/py_api_coverage_tests.rs (1)
822-825: LGTM!crates/python/tests/coverage/py_types_coverage_tests.rs (1)
789-796: LGTM!Also applies to: 826-833, 851-858
python/nemo_relay/_native.pyi (1)
936-938: LGTM!python/nemo_relay/llm.py (1)
317-319: LGTM!python/tests/test_llm.py (1)
6-6: LGTM!Also applies to: 516-540
crates/python/src/py_callable.rs (1)
26-26: LGTM!Also applies to: 35-38, 126-227, 241-259, 270-278, 474-486
crates/python/tests/coverage/py_callable_coverage_tests.rs (1)
357-385: LGTM!crates/python/src/py_api/mod.rs (1)
103-126: LGTM!Also applies to: 882-894
crates/python/src/py_types/core.rs (1)
4-5: LGTM!Also applies to: 28-31, 40-80
ericevans-nv
left a comment
ericevans-nv
left a comment
There was a problem hiding this comment.
The new stream-close API is a valuable improvement, but the closure acknowledgment is currently nondeterministic. Repeating the new regression test caused stream.aclose() to return while the underlying generator’s finally block had not completed; run 18 of a 20-run loop timed out waiting for cleanup.
Please make aclose() wait for the full downstream Python-generator cleanup—not only the outer forwarding task’s cancellation—and ensure cleanup failures can be represented appropriately. Once that lifecycle is deterministic, the existing regression should pass consistently under repetition.
ericevans-nv
left a comment
ericevans-nv
left a comment
There was a problem hiding this comment.
Approved. The intermittent close-acknowledgment behavior identified in review is a non-critical reliability concern and can be handled as follow-up work.
Signed-off-by: Will Killian <wkillian@nvidia.com>
github-actions
Bot
added
size:XL
willkill07
changed the title
Signed-off-by: Will Killian <wkillian@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 8
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
crates/ffi/src/callable.rs (1)
496-557: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winExplicitly close the
nextstream before the trampoline returns.Lines 529–533 consume one item and then drop the managed stream, bypassing asynchronous producer cleanup. Always call
stream.close().await, including when polling fails, and propagate cleanup failure according to the callback error policy.Proposed fix
let mut stream = next(request).await?; -match stream.next().await { - Some(item) => item, - None => Ok(Json::Null), +let item = stream.next().await.transpose(); +let close_result = stream.close().await; +match (item, close_result) { + (Err(error), _) => Err(error), + (Ok(_), Err(error)) => Err(error), + (Ok(Some(json)), Ok(())) => Ok(json), + (Ok(None), Ok(())) => Ok(Json::Null), }Add a close-tracking callback regression to verify cleanup runs once.
As per path instructions, binding reviews must check “FFI ownership/lifetime safety, callback error propagation, stable type conversion, and consistent async/stream semantics.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/ffi/src/callable.rs` around lines 496 - 557, Update llm_stream_next_trampoline so the stream returned by next(request) is explicitly closed before the trampoline returns, including when polling the first item fails. Ensure close() is awaited on every path and propagate any cleanup error using the existing callback error policy; add a regression test with a close-tracking callback verifying cleanup runs exactly once.Source: Path instructions
crates/node/src/callable.rs (1)
784-814: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winClose the managed stream when forwarding aborts on an item error.
Line 802 returns immediately through
item?, dropping theLlmJsonStreamwithout awaiting producer cleanup. Capture the item error, callstream.close().await, and propagate the appropriate item/cleanup error before returning.As per path instructions, “Check ... callback error propagation ... and consistent async/stream semantics.” As per coding guidelines, “keep errors explicit and binding-appropriate at the wrapper layer.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/node/src/callable.rs` around lines 784 - 814, Update the item-collection loop in the `next_stream` callback to handle stream item errors explicitly instead of propagating directly through `item?`. On an item error, await `stream.close()`, then return the appropriate item or cleanup error according to the existing error precedence; preserve successful collection behavior and ensure the managed `LlmStream` is closed before returning.Sources: Coding guidelines, Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/core/src/api/runtime/callbacks.rs`:
- Around line 244-247: Update the default stream close implementation and its
owning state to store the wrapped stream in an Option, then take and drop it
during close before returning. Ensure subsequent polls observe the closed state
and add a drop-tracking regression verifying resource cleanup completes before
close().await returns, covering the related constructors and close path around
the stream wrapper.
In `@crates/core/src/plugins/nemo_guardrails/python.rs`:
- Line 294: Update the guarded-stream construction around LlmJsonStream::new to
use a closeable wrapper for the spawned producer, carrying cancellation and
task-completion state. Ensure closing the returned stream cancels and joins the
producer task and awaits provider_stream.close(), preserving callback and future
lifetimes and preventing the task from remaining blocked on chunk_tx.
In `@crates/core/src/stream.rs`:
- Line 78: Replace the String-based close_result cache with a typed, replayable
FlowError representation. Update the close-failure caching and repeated-call
replay logic around close_result so the original FlowError variant and
binding-facing status are preserved, including the first close result, instead
of reconstructing every failure as FlowError::Internal.
In `@crates/node/src/api/mod.rs`:
- Around line 371-375: Update cancel_stream_channel and the
NodePushStream::close path to actively invoke and await the JavaScript async
iterator’s return() before waiting for closed cleanup. Preserve the existing
cancellation flag and ensure close still waits for producer termination and
removes the stream channel registration.
In `@crates/python/src/py_callable.rs`:
- Around line 223-226: Update the cancellation branch around next_value.await to
inspect its result before calling close_async_iter, propagating any
non-cancellation error while preserving the existing handling for expected
cancellation. Ensure a producer failure is returned instead of being discarded
when cleanup completes successfully.
In `@crates/switchyard/src/component.rs`:
- Around line 800-802: Replace LlmJsonStream::new with
LlmJsonStream::from_closeable in the stream adapters around the committed
construction and the corresponding ranges near 1670 and 1691, preserving the
nested closeable stream. Propagate the nested stream’s close error from each
adapter instead of discarding it, so early close cancels and awaits the
provider/worker stream.
In `@go/nemo_relay/stream.go`:
- Around line 124-141: Synchronize LlmStream.Next and LlmStream.Close with
coordinated locking and in-flight-call tracking around native handle access.
Ensure Close atomically claims cleanup, waits for active cgo calls to finish,
then frees the handle and clears callback/finalizer state exactly once;
concurrent Next calls must reject or stop using the stream after closure without
dereferencing a stale ptr.
- Around line 87-89: Update the runtime.SetFinalizer callback for LlmStream so
it performs only non-blocking resource cleanup and does not call Close() or
trigger s.finalizer(). Keep stream aggregation and user callback execution
exclusively in explicit LlmStream.Close(), while ensuring the finalizer still
releases underlying stream resources without waiting for producer shutdown.
---
Outside diff comments:
In `@crates/ffi/src/callable.rs`:
- Around line 496-557: Update llm_stream_next_trampoline so the stream returned
by next(request) is explicitly closed before the trampoline returns, including
when polling the first item fails. Ensure close() is awaited on every path and
propagate any cleanup error using the existing callback error policy; add a
regression test with a close-tracking callback verifying cleanup runs exactly
once.
In `@crates/node/src/callable.rs`:
- Around line 784-814: Update the item-collection loop in the `next_stream`
callback to handle stream item errors explicitly instead of propagating directly
through `item?`. On an item error, await `stream.close()`, then return the
appropriate item or cleanup error according to the existing error precedence;
preserve successful collection behavior and ensure the managed `LlmStream` is
closed before returning.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: e845a467-4c96-47fd-99f3-3bc32ba7cf3f
📒 Files selected for processing (54)
crates/adaptive/tests/integration/runtime_integration_tests.rscrates/adaptive/tests/unit/acg_component_tests.rscrates/adaptive/tests/unit/runtime_features_tests.rscrates/cli/src/gateway/mod.rscrates/cli/tests/coverage/shared/gateway_tests.rscrates/core/src/api/llm.rscrates/core/src/api/runtime.rscrates/core/src/api/runtime/callbacks.rscrates/core/src/plugin/dynamic/native.rscrates/core/src/plugin/dynamic/worker.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/plugins/nemo_guardrails/remote.rscrates/core/src/stream.rscrates/core/tests/integration/api_surface_tests.rscrates/core/tests/integration/middleware_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/tests/integration/pipeline_tests.rscrates/core/tests/integration/stream_tests.rscrates/core/tests/integration/worker_plugin_tests.rscrates/core/tests/unit/dynamic_worker_tests.rscrates/core/tests/unit/llm_api_tests.rscrates/core/tests/unit/native_plugin_tests.rscrates/core/tests/unit/plugin_tests.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/core/tests/unit/plugins/nemo_guardrails/remote_tests.rscrates/ffi/nemo_relay.hcrates/ffi/src/api/llm.rscrates/ffi/src/callable.rscrates/ffi/tests/integration/callable_extra_tests.rscrates/ffi/tests/integration/main.rscrates/ffi/tests/unit/api/coverage_sweeps_tests.rscrates/ffi/tests/unit/api/execution_tests.rscrates/ffi/tests/unit/callable_tests.rscrates/node/src/api/mod.rscrates/node/src/callable.rscrates/node/src/stream.rscrates/node/tests/typed_tests.mjscrates/node/typed.jscrates/python/src/py_api/mod.rscrates/python/src/py_callable.rscrates/python/src/py_types/core.rscrates/python/tests/coverage/coverage_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/python/tests/coverage/py_types_coverage_tests.rscrates/switchyard/src/component.rscrates/switchyard/tests/unit/component_tests.rsdocs/instrument-applications/code-examples.mdxdocs/resources/glossary.mdxgo/nemo_relay/llm_test.gogo/nemo_relay/nemo_relay.gogo/nemo_relay/stream.gopython/nemo_relay/llm.pypython/tests/test_llm.py
📜 Review details
⏰ Context from checks skipped due to timeout. (2)
- GitHub Check: Check / Run
- GitHub Check: Preview docs
🧰 Additional context used
📓 Path-based instructions (48)
**/*.mdx
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
MDX top-of-file SPDX comments must use {/* ... */} delimiters instead of HTML comment delimiters (Must-Fix)
In MDX files, top-of-file comments must use JSX comment delimiters (
{/*to open and*/}to close); do not use HTML comments for MDX SPDX headers
Files:
docs/resources/glossary.mdxdocs/instrument-applications/code-examples.mdx
**/*.{md,mdx}
📄 CodeRabbit inference engine (AGENTS.md)
Update
README.md,fern/, package READMEs, and binding-support notes when public behavior, package names, examples, or supported bindings change.
**/*.{md,mdx}: Prefer the documented public API, not internal shortcuts
Keep package names, repo references, and build commands current
Keep release-process and release-notes guidance in repo-maintainer docs such asRELEASING.md, not as user-facing docs pages orCHANGELOG.md
Keep stable user-facing wrappers atscripts/root in docs and examples; only point at namespaced helper paths when documenting internal maintenance work
When detailed dynamic plugin guides exist, keep Rust native plugin examples, Python worker plugin examples, andgrpc-v1protocol details on separate pagesIf links in documentation change, run
just docs-linkcheck.
Files:
docs/resources/glossary.mdxdocs/instrument-applications/code-examples.mdx
**/*.{md,markdown,mdx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Add the SPDX license header to all Markdown/MDX documentation files using the HTML comment block form.
Files:
docs/resources/glossary.mdxdocs/instrument-applications/code-examples.mdx
{docs,examples}/**/*
📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Update docs and examples.
Files:
docs/resources/glossary.mdxdocs/instrument-applications/code-examples.mdx
**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*: Format changed files with the language-native formatter before the final lint/test pass.
If dynamic plugin behavior changed, usemaintain-dynamic-pluginsand include the native SDK, worker protocol, Python SDK, docs, packaging, and Codecov surfaces in the validation plan.
If code changes alter APIs, bindings, commands, paths, packaging behavior, observability/adaptive semantics, or documented best practices, update any dependent maintainer or consumer skills in the same branch.
During iteration, preferuv run pre-commit run --files <changed files...>.
Before review or handoff, runuv run pre-commit run --all-files.
Files:
docs/resources/glossary.mdxcrates/core/src/plugins/nemo_guardrails/remote.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/api/runtime.rsgo/nemo_relay/nemo_relay.gocrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/ffi/tests/integration/callable_extra_tests.rspython/nemo_relay/llm.pycrates/ffi/tests/unit/api/execution_tests.rscrates/ffi/tests/unit/api/coverage_sweeps_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/cli/src/gateway/mod.rscrates/adaptive/tests/unit/acg_component_tests.rscrates/ffi/tests/integration/main.rscrates/node/typed.jscrates/ffi/nemo_relay.hcrates/core/src/api/llm.rscrates/node/tests/typed_tests.mjsgo/nemo_relay/stream.gocrates/core/src/plugin/dynamic/worker.rscrates/ffi/tests/unit/callable_tests.rspython/tests/test_llm.pycrates/core/src/plugin/dynamic/native.rsgo/nemo_relay/llm_test.gocrates/core/tests/integration/worker_plugin_tests.rscrates/python/src/py_types/core.rscrates/python/tests/coverage/py_types_coverage_tests.rsdocs/instrument-applications/code-examples.mdxcrates/adaptive/tests/integration/runtime_integration_tests.rscrates/python/tests/coverage/coverage_tests.rscrates/core/src/api/runtime/callbacks.rscrates/adaptive/tests/unit/runtime_features_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/cli/tests/coverage/shared/gateway_tests.rscrates/core/tests/unit/plugins/nemo_guardrails/remote_tests.rscrates/node/src/callable.rscrates/core/tests/integration/pipeline_tests.rscrates/ffi/src/callable.rscrates/ffi/src/api/llm.rscrates/core/tests/unit/llm_api_tests.rscrates/switchyard/src/component.rscrates/core/tests/integration/api_surface_tests.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/node/src/stream.rscrates/core/tests/unit/dynamic_worker_tests.rscrates/core/tests/unit/plugin_tests.rscrates/core/tests/unit/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rscrates/core/src/stream.rscrates/python/src/py_api/mod.rscrates/node/src/api/mod.rscrates/switchyard/tests/unit/component_tests.rscrates/python/src/py_callable.rscrates/core/tests/integration/stream_tests.rs
docs/**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If documentation examples or commands under
docs/change, run the targeted docs checks appropriate to the change.
Files:
docs/resources/glossary.mdxdocs/instrument-applications/code-examples.mdx
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}: Review documentation for technical accuracy against the current API, command correctness, and consistency across language bindings.
Flag stale examples, missing SPDX headers where required, and instructions that no longer match CI or pre-commit behavior.
Files:
docs/resources/glossary.mdxdocs/instrument-applications/code-examples.mdx
**/*.rs
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
**/*.rs: Any Rust change must runjust test-rust
Any Rust change must runcargo fmt --all
Any Rust change must runcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allfor all FFI work since it is Rust work
Runjust test-rustto validate FFI changes
Runcargo clippy --workspace --all-targets -- -D warningsto enforce strict linting on FFI workWhen Rust files changed as part of Go work, also run
cargo fmt --all,just test-rust, andcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allwhen Rust files are changed as part of Node work
Runcargo clippy --workspace --all-targets -- -D warningswhen Rust files are changed as part of Node work
Runjust test-rustwhen Rust files are changed as part of Node workWhen changing the core Rust runtime or Rust-facing API surface, format Rust code with
cargo fmt(rustfmt defaults), keepcargo clippy -- -D warningsclean, and satisfycargo deny checkperdeny.toml.
**/*.rs: If any Rust code changed, always runjust test-rust.
If any Rust code changed, also runcargo fmt --all.
If any Rust code changed, also runcargo clippy --workspace --all-targets -- -D warnings.
For Rust changes headed for review, runcargo fmt --allandcargo clippy --workspace --all-targets -- -D warningseven if relying on pre-commit.
Files:
crates/core/src/plugins/nemo_guardrails/remote.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/api/runtime.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/ffi/tests/integration/callable_extra_tests.rscrates/ffi/tests/unit/api/execution_tests.rscrates/ffi/tests/unit/api/coverage_sweeps_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/cli/src/gateway/mod.rscrates/adaptive/tests/unit/acg_component_tests.rscrates/ffi/tests/integration/main.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/worker.rscrates/ffi/tests/unit/callable_tests.rscrates/core/src/plugin/dynamic/native.rscrates/core/tests/integration/worker_plugin_tests.rscrates/python/src/py_types/core.rscrates/python/tests/coverage/py_types_coverage_tests.rscrates/adaptive/tests/integration/runtime_integration_tests.rscrates/python/tests/coverage/coverage_tests.rscrates/core/src/api/runtime/callbacks.rscrates/adaptive/tests/unit/runtime_features_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/cli/tests/coverage/shared/gateway_tests.rscrates/core/tests/unit/plugins/nemo_guardrails/remote_tests.rscrates/node/src/callable.rscrates/core/tests/integration/pipeline_tests.rscrates/ffi/src/callable.rscrates/ffi/src/api/llm.rscrates/core/tests/unit/llm_api_tests.rscrates/switchyard/src/component.rscrates/core/tests/integration/api_surface_tests.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/node/src/stream.rscrates/core/tests/unit/dynamic_worker_tests.rscrates/core/tests/unit/plugin_tests.rscrates/core/tests/unit/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rscrates/core/src/stream.rscrates/python/src/py_api/mod.rscrates/node/src/api/mod.rscrates/switchyard/tests/unit/component_tests.rscrates/python/src/py_callable.rscrates/core/tests/integration/stream_tests.rs
{crates/core,crates/adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Changes to
crates/coreorcrates/adaptivemust run the full language matrix
Files:
crates/core/src/plugins/nemo_guardrails/remote.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/api/runtime.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/adaptive/tests/unit/acg_component_tests.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/worker.rscrates/core/src/plugin/dynamic/native.rscrates/core/tests/integration/worker_plugin_tests.rscrates/adaptive/tests/integration/runtime_integration_tests.rscrates/core/src/api/runtime/callbacks.rscrates/adaptive/tests/unit/runtime_features_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/tests/unit/plugins/nemo_guardrails/remote_tests.rscrates/core/tests/integration/pipeline_tests.rscrates/core/tests/unit/llm_api_tests.rscrates/core/tests/integration/api_surface_tests.rscrates/core/tests/unit/dynamic_worker_tests.rscrates/core/tests/unit/plugin_tests.rscrates/core/tests/unit/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rscrates/core/src/stream.rscrates/core/tests/integration/stream_tests.rs
crates/core/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)
If the change touched
crates/coreor shared runtime semantics, also usevalidate-changefor broader validation
Files:
crates/core/src/plugins/nemo_guardrails/remote.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/api/runtime.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/worker.rscrates/core/src/plugin/dynamic/native.rscrates/core/tests/integration/worker_plugin_tests.rscrates/core/src/api/runtime/callbacks.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/tests/unit/plugins/nemo_guardrails/remote_tests.rscrates/core/tests/integration/pipeline_tests.rscrates/core/tests/unit/llm_api_tests.rscrates/core/tests/integration/api_surface_tests.rscrates/core/tests/unit/dynamic_worker_tests.rscrates/core/tests/unit/plugin_tests.rscrates/core/tests/unit/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rscrates/core/src/stream.rscrates/core/tests/integration/stream_tests.rs
**/*.{rs,py}
📄 CodeRabbit inference engine (AGENTS.md)
Follow binding naming conventions in Rust and Python: use
snake_case.
Files:
crates/core/src/plugins/nemo_guardrails/remote.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/api/runtime.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/ffi/tests/integration/callable_extra_tests.rspython/nemo_relay/llm.pycrates/ffi/tests/unit/api/execution_tests.rscrates/ffi/tests/unit/api/coverage_sweeps_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/cli/src/gateway/mod.rscrates/adaptive/tests/unit/acg_component_tests.rscrates/ffi/tests/integration/main.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/worker.rscrates/ffi/tests/unit/callable_tests.rspython/tests/test_llm.pycrates/core/src/plugin/dynamic/native.rscrates/core/tests/integration/worker_plugin_tests.rscrates/python/src/py_types/core.rscrates/python/tests/coverage/py_types_coverage_tests.rscrates/adaptive/tests/integration/runtime_integration_tests.rscrates/python/tests/coverage/coverage_tests.rscrates/core/src/api/runtime/callbacks.rscrates/adaptive/tests/unit/runtime_features_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/cli/tests/coverage/shared/gateway_tests.rscrates/core/tests/unit/plugins/nemo_guardrails/remote_tests.rscrates/node/src/callable.rscrates/core/tests/integration/pipeline_tests.rscrates/ffi/src/callable.rscrates/ffi/src/api/llm.rscrates/core/tests/unit/llm_api_tests.rscrates/switchyard/src/component.rscrates/core/tests/integration/api_surface_tests.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/node/src/stream.rscrates/core/tests/unit/dynamic_worker_tests.rscrates/core/tests/unit/plugin_tests.rscrates/core/tests/unit/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rscrates/core/src/stream.rscrates/python/src/py_api/mod.rscrates/node/src/api/mod.rscrates/switchyard/tests/unit/component_tests.rscrates/python/src/py_callable.rscrates/core/tests/integration/stream_tests.rs
**/*.{rs,py,js,mjs,cjs,ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{rs,py,js,mjs,cjs,ts,tsx}: UseJson = serde_json::Valuein Rust-facing runtime APIs where the existing code expects JSON payloads.
UseResult<T>withFlowErrorin core runtime paths, and keep errors explicit and binding-appropriate at the wrapper layer.
Keep async behavior on the existing tokio-based model; bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.
Files:
crates/core/src/plugins/nemo_guardrails/remote.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/api/runtime.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/ffi/tests/integration/callable_extra_tests.rspython/nemo_relay/llm.pycrates/ffi/tests/unit/api/execution_tests.rscrates/ffi/tests/unit/api/coverage_sweeps_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/cli/src/gateway/mod.rscrates/adaptive/tests/unit/acg_component_tests.rscrates/ffi/tests/integration/main.rscrates/node/typed.jscrates/core/src/api/llm.rscrates/node/tests/typed_tests.mjscrates/core/src/plugin/dynamic/worker.rscrates/ffi/tests/unit/callable_tests.rspython/tests/test_llm.pycrates/core/src/plugin/dynamic/native.rscrates/core/tests/integration/worker_plugin_tests.rscrates/python/src/py_types/core.rscrates/python/tests/coverage/py_types_coverage_tests.rscrates/adaptive/tests/integration/runtime_integration_tests.rscrates/python/tests/coverage/coverage_tests.rscrates/core/src/api/runtime/callbacks.rscrates/adaptive/tests/unit/runtime_features_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/cli/tests/coverage/shared/gateway_tests.rscrates/core/tests/unit/plugins/nemo_guardrails/remote_tests.rscrates/node/src/callable.rscrates/core/tests/integration/pipeline_tests.rscrates/ffi/src/callable.rscrates/ffi/src/api/llm.rscrates/core/tests/unit/llm_api_tests.rscrates/switchyard/src/component.rscrates/core/tests/integration/api_surface_tests.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/node/src/stream.rscrates/core/tests/unit/dynamic_worker_tests.rscrates/core/tests/unit/plugin_tests.rscrates/core/tests/unit/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rscrates/core/src/stream.rscrates/python/src/py_api/mod.rscrates/node/src/api/mod.rscrates/switchyard/tests/unit/component_tests.rscrates/python/src/py_callable.rscrates/core/tests/integration/stream_tests.rs
**/*.{rs,py,go,js,ts,c,h}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Use language-appropriate naming conventions: Rust
snake_case, C FFI exports prefixednemo_relay_, GoPascalCase, Node.jscamelCase, and Pythonsnake_case.
Files:
crates/core/src/plugins/nemo_guardrails/remote.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/api/runtime.rsgo/nemo_relay/nemo_relay.gocrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/ffi/tests/integration/callable_extra_tests.rspython/nemo_relay/llm.pycrates/ffi/tests/unit/api/execution_tests.rscrates/ffi/tests/unit/api/coverage_sweeps_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/cli/src/gateway/mod.rscrates/adaptive/tests/unit/acg_component_tests.rscrates/ffi/tests/integration/main.rscrates/node/typed.jscrates/ffi/nemo_relay.hcrates/core/src/api/llm.rsgo/nemo_relay/stream.gocrates/core/src/plugin/dynamic/worker.rscrates/ffi/tests/unit/callable_tests.rspython/tests/test_llm.pycrates/core/src/plugin/dynamic/native.rsgo/nemo_relay/llm_test.gocrates/core/tests/integration/worker_plugin_tests.rscrates/python/src/py_types/core.rscrates/python/tests/coverage/py_types_coverage_tests.rscrates/adaptive/tests/integration/runtime_integration_tests.rscrates/python/tests/coverage/coverage_tests.rscrates/core/src/api/runtime/callbacks.rscrates/adaptive/tests/unit/runtime_features_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/cli/tests/coverage/shared/gateway_tests.rscrates/core/tests/unit/plugins/nemo_guardrails/remote_tests.rscrates/node/src/callable.rscrates/core/tests/integration/pipeline_tests.rscrates/ffi/src/callable.rscrates/ffi/src/api/llm.rscrates/core/tests/unit/llm_api_tests.rscrates/switchyard/src/component.rscrates/core/tests/integration/api_surface_tests.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/node/src/stream.rscrates/core/tests/unit/dynamic_worker_tests.rscrates/core/tests/unit/plugin_tests.rscrates/core/tests/unit/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rscrates/core/src/stream.rscrates/python/src/py_api/mod.rscrates/node/src/api/mod.rscrates/switchyard/tests/unit/component_tests.rscrates/python/src/py_callable.rscrates/core/tests/integration/stream_tests.rs
**/*.{rs,go,js,ts}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Add the SPDX license header to all Rust, Go, JavaScript, and TypeScript source files using the corresponding
//comment form.
Files:
crates/core/src/plugins/nemo_guardrails/remote.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/api/runtime.rsgo/nemo_relay/nemo_relay.gocrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/ffi/tests/integration/callable_extra_tests.rscrates/ffi/tests/unit/api/execution_tests.rscrates/ffi/tests/unit/api/coverage_sweeps_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/cli/src/gateway/mod.rscrates/adaptive/tests/unit/acg_component_tests.rscrates/ffi/tests/integration/main.rscrates/node/typed.jscrates/core/src/api/llm.rsgo/nemo_relay/stream.gocrates/core/src/plugin/dynamic/worker.rscrates/ffi/tests/unit/callable_tests.rscrates/core/src/plugin/dynamic/native.rsgo/nemo_relay/llm_test.gocrates/core/tests/integration/worker_plugin_tests.rscrates/python/src/py_types/core.rscrates/python/tests/coverage/py_types_coverage_tests.rscrates/adaptive/tests/integration/runtime_integration_tests.rscrates/python/tests/coverage/coverage_tests.rscrates/core/src/api/runtime/callbacks.rscrates/adaptive/tests/unit/runtime_features_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/cli/tests/coverage/shared/gateway_tests.rscrates/core/tests/unit/plugins/nemo_guardrails/remote_tests.rscrates/node/src/callable.rscrates/core/tests/integration/pipeline_tests.rscrates/ffi/src/callable.rscrates/ffi/src/api/llm.rscrates/core/tests/unit/llm_api_tests.rscrates/switchyard/src/component.rscrates/core/tests/integration/api_surface_tests.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/node/src/stream.rscrates/core/tests/unit/dynamic_worker_tests.rscrates/core/tests/unit/plugin_tests.rscrates/core/tests/unit/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rscrates/core/src/stream.rscrates/python/src/py_api/mod.rscrates/node/src/api/mod.rscrates/switchyard/tests/unit/component_tests.rscrates/python/src/py_callable.rscrates/core/tests/integration/stream_tests.rs
{crates/**/src/**/*.rs,python/**/*.py}
📄 CodeRabbit inference engine (.agents/skills/maintain-dynamic-plugins/SKILL.md)
Do not add tests under
src; Rust tests belong in cratetests/trees, and Python SDK tests belong underpython/tests.
Files:
crates/core/src/plugins/nemo_guardrails/remote.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/api/runtime.rspython/nemo_relay/llm.pycrates/cli/src/gateway/mod.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/worker.rspython/tests/test_llm.pycrates/core/src/plugin/dynamic/native.rscrates/python/src/py_types/core.rscrates/core/src/api/runtime/callbacks.rscrates/node/src/callable.rscrates/ffi/src/callable.rscrates/ffi/src/api/llm.rscrates/switchyard/src/component.rscrates/node/src/stream.rscrates/core/src/stream.rscrates/python/src/py_api/mod.rscrates/node/src/api/mod.rscrates/python/src/py_callable.rs
crates/{core,adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If
crates/coreorcrates/adaptivechanged, run the full validation matrix across Rust, Python, Go, and Node.js.
Files:
crates/core/src/plugins/nemo_guardrails/remote.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/api/runtime.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/adaptive/tests/unit/acg_component_tests.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/worker.rscrates/core/src/plugin/dynamic/native.rscrates/core/tests/integration/worker_plugin_tests.rscrates/adaptive/tests/integration/runtime_integration_tests.rscrates/core/src/api/runtime/callbacks.rscrates/adaptive/tests/unit/runtime_features_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/tests/unit/plugins/nemo_guardrails/remote_tests.rscrates/core/tests/integration/pipeline_tests.rscrates/core/tests/unit/llm_api_tests.rscrates/core/tests/integration/api_surface_tests.rscrates/core/tests/unit/dynamic_worker_tests.rscrates/core/tests/unit/plugin_tests.rscrates/core/tests/unit/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rscrates/core/src/stream.rscrates/core/tests/integration/stream_tests.rs
**/*.{rs,py,go,js,ts}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If a language surface changed, always run that language's test target even when Rust core did not change.
Files:
crates/core/src/plugins/nemo_guardrails/remote.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/api/runtime.rsgo/nemo_relay/nemo_relay.gocrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/ffi/tests/integration/callable_extra_tests.rspython/nemo_relay/llm.pycrates/ffi/tests/unit/api/execution_tests.rscrates/ffi/tests/unit/api/coverage_sweeps_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/cli/src/gateway/mod.rscrates/adaptive/tests/unit/acg_component_tests.rscrates/ffi/tests/integration/main.rscrates/node/typed.jscrates/core/src/api/llm.rsgo/nemo_relay/stream.gocrates/core/src/plugin/dynamic/worker.rscrates/ffi/tests/unit/callable_tests.rspython/tests/test_llm.pycrates/core/src/plugin/dynamic/native.rsgo/nemo_relay/llm_test.gocrates/core/tests/integration/worker_plugin_tests.rscrates/python/src/py_types/core.rscrates/python/tests/coverage/py_types_coverage_tests.rscrates/adaptive/tests/integration/runtime_integration_tests.rscrates/python/tests/coverage/coverage_tests.rscrates/core/src/api/runtime/callbacks.rscrates/adaptive/tests/unit/runtime_features_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/cli/tests/coverage/shared/gateway_tests.rscrates/core/tests/unit/plugins/nemo_guardrails/remote_tests.rscrates/node/src/callable.rscrates/core/tests/integration/pipeline_tests.rscrates/ffi/src/callable.rscrates/ffi/src/api/llm.rscrates/core/tests/unit/llm_api_tests.rscrates/switchyard/src/component.rscrates/core/tests/integration/api_surface_tests.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/node/src/stream.rscrates/core/tests/unit/dynamic_worker_tests.rscrates/core/tests/unit/plugin_tests.rscrates/core/tests/unit/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rscrates/core/src/stream.rscrates/python/src/py_api/mod.rscrates/node/src/api/mod.rscrates/switchyard/tests/unit/component_tests.rscrates/python/src/py_callable.rscrates/core/tests/integration/stream_tests.rs
**/*.{rs,py,js,ts,tsx,go,java,kt,swift}
📄 CodeRabbit inference engine (.agents/skills/add-middleware/SKILL.md)
Add tests covering registration and duplicate names, deregistration and missing names, priority ordering, callback failure policy, scope-local inheritance and cleanup, event payload semantics, immutable mark and scope fields, and parity across affected bindings.
Files:
crates/core/src/plugins/nemo_guardrails/remote.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/api/runtime.rsgo/nemo_relay/nemo_relay.gocrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/ffi/tests/integration/callable_extra_tests.rspython/nemo_relay/llm.pycrates/ffi/tests/unit/api/execution_tests.rscrates/ffi/tests/unit/api/coverage_sweeps_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/cli/src/gateway/mod.rscrates/adaptive/tests/unit/acg_component_tests.rscrates/ffi/tests/integration/main.rscrates/node/typed.jscrates/core/src/api/llm.rsgo/nemo_relay/stream.gocrates/core/src/plugin/dynamic/worker.rscrates/ffi/tests/unit/callable_tests.rspython/tests/test_llm.pycrates/core/src/plugin/dynamic/native.rsgo/nemo_relay/llm_test.gocrates/core/tests/integration/worker_plugin_tests.rscrates/python/src/py_types/core.rscrates/python/tests/coverage/py_types_coverage_tests.rscrates/adaptive/tests/integration/runtime_integration_tests.rscrates/python/tests/coverage/coverage_tests.rscrates/core/src/api/runtime/callbacks.rscrates/adaptive/tests/unit/runtime_features_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/cli/tests/coverage/shared/gateway_tests.rscrates/core/tests/unit/plugins/nemo_guardrails/remote_tests.rscrates/node/src/callable.rscrates/core/tests/integration/pipeline_tests.rscrates/ffi/src/callable.rscrates/ffi/src/api/llm.rscrates/core/tests/unit/llm_api_tests.rscrates/switchyard/src/component.rscrates/core/tests/integration/api_surface_tests.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/node/src/stream.rscrates/core/tests/unit/dynamic_worker_tests.rscrates/core/tests/unit/plugin_tests.rscrates/core/tests/unit/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rscrates/core/src/stream.rscrates/python/src/py_api/mod.rscrates/node/src/api/mod.rscrates/switchyard/tests/unit/component_tests.rscrates/python/src/py_callable.rscrates/core/tests/integration/stream_tests.rs
crates/{core,adaptive}/**/*.rs
⚙️ CodeRabbit configuration file
crates/{core,adaptive}/**/*.rs: Review the Rust runtime for async correctness, scope isolation, middleware ordering, and event lifecycle regressions.
Pay close attention to task-local/thread-local scope propagation, callback lifetimes, stream finalization, and root_uuid isolation.
Public API changes should preserve existing behavior unless tests and docs show the intended migration path.
Files:
crates/core/src/plugins/nemo_guardrails/remote.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/api/runtime.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/adaptive/tests/unit/acg_component_tests.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/worker.rscrates/core/src/plugin/dynamic/native.rscrates/core/tests/integration/worker_plugin_tests.rscrates/adaptive/tests/integration/runtime_integration_tests.rscrates/core/src/api/runtime/callbacks.rscrates/adaptive/tests/unit/runtime_features_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/tests/unit/plugins/nemo_guardrails/remote_tests.rscrates/core/tests/integration/pipeline_tests.rscrates/core/tests/unit/llm_api_tests.rscrates/core/tests/integration/api_surface_tests.rscrates/core/tests/unit/dynamic_worker_tests.rscrates/core/tests/unit/plugin_tests.rscrates/core/tests/unit/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rscrates/core/src/stream.rscrates/core/tests/integration/stream_tests.rs
crates/core/src/{api/**/*.rs,api/runtime/**/*.rs,codec/**/*.rs,json.rs}
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Implement the new or changed public runtime behavior first in the Rust core, especially under
crates/core/src/api/and related core modules such ascrates/core/src/api/runtime/,crates/core/src/codec/, andcrates/core/src/json.rs.
Files:
crates/core/src/api/runtime.rscrates/core/src/api/llm.rscrates/core/src/api/runtime/callbacks.rs
crates/core/src/api/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/add-middleware/SKILL.md)
Preserve the documented pipeline order: conditional guardrails, request intercepts, request sanitization, execution intercepts, and response sanitization for tool and LLM execution; specialized sanitization, event creation, and dispatch for mark and scope events.
Files:
crates/core/src/api/runtime.rscrates/core/src/api/llm.rscrates/core/src/api/runtime/callbacks.rs
go/nemo_relay/**/*.go
📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)
go/nemo_relay/**/*.go: Format changed Go packages withcd go/nemo_relay && go fmt ./...
Run Go tests withjust test-goto build and test the NeMo Relay Go binding
Usejust build-gowhen you want an explicit build-only pass or need the artifact for other work
Usejust ci=true test-gowhen you need the CI-style coverage and JUnit path
On macOS, setDYLD_LIBRARY_PATHto the../../target/releasedirectory before running the rawgo testcommand directlyUse
PascalCasefor public Go APIs.
Files:
go/nemo_relay/nemo_relay.gogo/nemo_relay/stream.gogo/nemo_relay/llm_test.go
**/*.go
📄 CodeRabbit inference engine (CONTRIBUTING.md)
When changing the experimental Go binding, format Go code with
gofmtand keepgo vet ./...passing.
Files:
go/nemo_relay/nemo_relay.gogo/nemo_relay/stream.gogo/nemo_relay/llm_test.go
{crates/python/src/py_api/mod.rs,python/nemo_relay/**/*.py,python/nemo_relay/**/*.pyi,go/nemo_relay/**/*.go,crates/node/src/api/**/*.rs}
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Update the language-native bindings for every exposed surface in Python, Go, and Node.js.
Files:
go/nemo_relay/nemo_relay.gopython/nemo_relay/llm.pygo/nemo_relay/stream.gogo/nemo_relay/llm_test.gocrates/python/src/py_api/mod.rscrates/node/src/api/mod.rs
{python/nemo_relay/**/*.py,python/nemo_relay/**/*.pyi,go/nemo_relay/**/*.go}
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Update language wrapper helpers such as Python wrapper modules, Python type stubs, and Go shorthand packages when the new behavior belongs in those helper layers.
Files:
go/nemo_relay/nemo_relay.gopython/nemo_relay/llm.pygo/nemo_relay/stream.gogo/nemo_relay/llm_test.go
**/*.{py,go,js,ts}
📄 CodeRabbit inference engine (.agents/skills/maintain-observability/SKILL.md)
Keep Python, Go, and Node.js config objects and subscriber/exporter methods aligned so all bindings expose the same logical knobs and semantics.
Files:
go/nemo_relay/nemo_relay.gopython/nemo_relay/llm.pycrates/node/typed.jsgo/nemo_relay/stream.gopython/tests/test_llm.pygo/nemo_relay/llm_test.go
go/nemo_relay/**
📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Keep shared plugin helpers in
go/nemo_relayaligned with plugin registration, composition, and lifecycle behavior.
Files:
go/nemo_relay/nemo_relay.gogo/nemo_relay/stream.gogo/nemo_relay/llm_test.go
go/nemo_relay/**/*
⚙️ CodeRabbit configuration file
go/nemo_relay/**/*: Review Go binding changes for cgo memory ownership, race safety, callback cleanup, idiomatic exported APIs, and parity with Rust/FFI behavior.
Any API change should include focused Go tests and consider race-test behavior.
Files:
go/nemo_relay/nemo_relay.gogo/nemo_relay/stream.gogo/nemo_relay/llm_test.go
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}
⚙️ CodeRabbit configuration file
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.
Files:
crates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/ffi/tests/integration/callable_extra_tests.rscrates/ffi/tests/unit/api/execution_tests.rscrates/ffi/tests/unit/api/coverage_sweeps_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/adaptive/tests/unit/acg_component_tests.rscrates/ffi/tests/integration/main.rscrates/node/tests/typed_tests.mjscrates/ffi/tests/unit/callable_tests.rspython/tests/test_llm.pygo/nemo_relay/llm_test.gocrates/core/tests/integration/worker_plugin_tests.rscrates/python/tests/coverage/py_types_coverage_tests.rscrates/adaptive/tests/integration/runtime_integration_tests.rscrates/python/tests/coverage/coverage_tests.rscrates/adaptive/tests/unit/runtime_features_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/cli/tests/coverage/shared/gateway_tests.rscrates/core/tests/unit/plugins/nemo_guardrails/remote_tests.rscrates/core/tests/integration/pipeline_tests.rscrates/core/tests/unit/llm_api_tests.rscrates/core/tests/integration/api_surface_tests.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/core/tests/unit/dynamic_worker_tests.rscrates/core/tests/unit/plugin_tests.rscrates/core/tests/unit/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rscrates/switchyard/tests/unit/component_tests.rscrates/core/tests/integration/stream_tests.rs
crates/ffi/**
📄 CodeRabbit inference engine (.agents/skills/test-ffi-surface/SKILL.md)
Rebuild the FFI crate in release mode so the shared library and header stay in sync when making changes to crates/ffi
Files:
crates/ffi/tests/integration/callable_extra_tests.rscrates/ffi/tests/unit/api/execution_tests.rscrates/ffi/tests/unit/api/coverage_sweeps_tests.rscrates/ffi/tests/integration/main.rscrates/ffi/nemo_relay.hcrates/ffi/tests/unit/callable_tests.rscrates/ffi/src/callable.rscrates/ffi/src/api/llm.rs
crates/ffi/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)
If the change touched
crates/ffi, also usetest-ffi-surfacefor validationUse C FFI export names prefixed with
nemo_relay_in the raw C FFI layer.
Files:
crates/ffi/tests/integration/callable_extra_tests.rscrates/ffi/tests/unit/api/execution_tests.rscrates/ffi/tests/unit/api/coverage_sweeps_tests.rscrates/ffi/tests/integration/main.rscrates/ffi/tests/unit/callable_tests.rscrates/ffi/src/callable.rscrates/ffi/src/api/llm.rs
crates/{python,ffi,node}/**/*
⚙️ CodeRabbit configuration file
crates/{python,ffi,node}/**/*: Treat binding changes as public API changes. Check for parity with the other language bindings, FFI ownership/lifetime safety,
callback error propagation, stable type conversion, and consistent async/stream semantics.
Flag changes that update one binding without corresponding tests or documentation for the same surface elsewhere.
Files:
crates/ffi/tests/integration/callable_extra_tests.rscrates/ffi/tests/unit/api/execution_tests.rscrates/ffi/tests/unit/api/coverage_sweeps_tests.rscrates/python/tests/coverage/py_api_coverage_tests.rscrates/ffi/tests/integration/main.rscrates/node/typed.jscrates/ffi/nemo_relay.hcrates/node/tests/typed_tests.mjscrates/ffi/tests/unit/callable_tests.rscrates/python/src/py_types/core.rscrates/python/tests/coverage/py_types_coverage_tests.rscrates/python/tests/coverage/coverage_tests.rscrates/node/src/callable.rscrates/ffi/src/callable.rscrates/ffi/src/api/llm.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/node/src/stream.rscrates/python/src/py_api/mod.rscrates/node/src/api/mod.rscrates/python/src/py_callable.rs
python/nemo_relay/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Python wrapper modules live under
python/nemo_relay/, and the native extension is built fromcrates/pythonwithmaturin.
Files:
python/nemo_relay/llm.py
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
**/*.py: When changing the Python wrapper package, tests, or docs tooling, lint with Ruff (E,F,W,I), format with Ruff formatter (120-character lines, double quotes), and passtytype checking.
Add the SPDX license header to all Python source files using the#comment form.
Files:
python/nemo_relay/llm.pypython/tests/test_llm.py
python/nemo_relay/**/*
⚙️ CodeRabbit configuration file
python/nemo_relay/**/*: Review Python wrapper changes for typed API consistency, contextvars-based scope isolation, async behavior, and parity with the native extension.
Stubs and runtime implementations should stay aligned.
Files:
python/nemo_relay/llm.py
crates/adaptive/**
📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Keep
crates/adaptivealigned with the canonical adaptive config schema, built-in section helpers, plugin lifecycle, and validation/report behavior.
Files:
crates/adaptive/tests/unit/acg_component_tests.rscrates/adaptive/tests/integration/runtime_integration_tests.rscrates/adaptive/tests/unit/runtime_features_tests.rs
crates/node/**/*.{js,ts,jsx,tsx,json}
📄 CodeRabbit inference engine (.agents/skills/test-node-binding/SKILL.md)
Format changed Node files with
npm run format --workspace=nemo-relay-node
Files:
crates/node/typed.js
crates/node/**/*.{js,mjs,cjs,ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Use
camelCasefor Node.js public APIs.
Files:
crates/node/typed.jscrates/node/tests/typed_tests.mjs
crates/ffi/nemo_relay.h
📄 CodeRabbit inference engine (.agents/skills/test-ffi-surface/SKILL.md)
Check the generated header diff when any exported symbol or type changed in the FFI surface
Update generated or generated-from-build surfaces such as
crates/ffi/nemo_relay.hthrough the proper build step.
Files:
crates/ffi/nemo_relay.h
{crates/ffi/src/api/*.rs,crates/ffi/nemo_relay.h}
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Add or update the shared C/FFI surface in the relevant
crates/ffi/src/api/*.rsmodule, re-export it throughcrates/ffi/src/api/mod.rs, and keep the generatedcrates/ffi/nemo_relay.hheader correct.
Files:
crates/ffi/nemo_relay.hcrates/ffi/src/api/llm.rs
crates/core/src/api/{tool,llm,shared,scope}.rs
📄 CodeRabbit inference engine (.agents/skills/add-middleware/SKILL.md)
Wire the new middleware chain into the appropriate lifecycle owner and pipeline stage: tool and LLM execution paths use
tool.rsorllm.rs; shared mark and scope event sanitization usesshared.rsand is called fromscope.rs.
Files:
crates/core/src/api/llm.rs
{crates/core/src/plugin/dynamic/**,crates/plugin/**,crates/worker/**,crates/worker-proto/**,crates/types/**,python/plugin/**,examples/rust-native-plugin/**,examples/python-grpc-worker-plugin/**,docs/build-plugins/**}
📄 CodeRabbit inference engine (.agents/skills/maintain-dynamic-plugins/SKILL.md)
Keep the stable boundary explicit: native plugins cross a C ABI, and worker plugins cross
grpc-v1.
Files:
crates/core/src/plugin/dynamic/worker.rscrates/core/src/plugin/dynamic/native.rs
{crates/core/src/plugin/dynamic/**/*.rs,examples/rust-native-plugin/**/*.rs}
📄 CodeRabbit inference engine (.agents/skills/maintain-dynamic-plugins/SKILL.md)
Do not pass Rust runtime types, trait objects, futures, or allocator-owned strings across the native dynamic-library boundary.
Files:
crates/core/src/plugin/dynamic/worker.rscrates/core/src/plugin/dynamic/native.rs
{crates/core/src/plugin/dynamic/**,examples/rust-native-plugin/**,examples/python-grpc-worker-plugin/**,docs/build-plugins/**}
📄 CodeRabbit inference engine (.agents/skills/maintain-dynamic-plugins/SKILL.md)
Native and worker plugins are trusted extensions; document that native plugins are in-process and unsandboxed, and worker plugins provide process isolation but not a security sandbox.
Files:
crates/core/src/plugin/dynamic/worker.rscrates/core/src/plugin/dynamic/native.rs
{crates/core/src/plugin/dynamic/**/*.rs,crates/plugin/**/*.rs,crates/worker/**/*.rs,crates/worker-proto/**/*.rs,python/plugin/**/*.py}
📄 CodeRabbit inference engine (.agents/skills/maintain-dynamic-plugins/SKILL.md)
Manifest validation must cover kind, compatibility, load contract, integrity, capability mismatch, and disabled-plugin behavior.
Files:
crates/core/src/plugin/dynamic/worker.rscrates/core/src/plugin/dynamic/native.rs
crates/core/src/plugin/dynamic/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/maintain-dynamic-plugins/SKILL.md)
The native loader must keep libraries alive until registered callbacks are cleared and must deregister plugin kinds before unload.
Files:
crates/core/src/plugin/dynamic/worker.rscrates/core/src/plugin/dynamic/native.rs
python/tests/**/*.py
📄 CodeRabbit inference engine (.agents/skills/test-python-binding/SKILL.md)
python/tests/**/*.py: Pytest is used to run tests.
Do not add@pytest.mark.asyncioto any test; async tests are automatically detected and run by the async runner.
Do not add a-> Nonereturn type annotation to test functions.
When mocking a class, do not define a new class; useunittest.mock.MagicMockorunittest.mock.AsyncMock, with thespecconstructor argument when necessary.
Name mocked classes with themockprefix, notfake.
Prefer pytest fixtures over helper methods.
Do not repeat fixtures; if a fixture is needed in multiple test files, place it in aconftest.pyfile.
When creating a fixture, use@pytest.fixture(name="<fixture_name>"[, scope="<scope>"])and define the fixture function asdef <fixture_name>_fixture() -> <return_type>:; only specifyscopewhen it is notfunction.
Preferpytest.mark.parametrizeover creating individual tests for different input types.
Files:
python/tests/test_llm.py
crates/core/src/api/runtime/callbacks.rs
📄 CodeRabbit inference engine (.agents/skills/add-middleware/SKILL.md)
Define or reuse the middleware callback type alias in
crates/core/src/api/runtime/callbacks.rs, using the appropriate callable signature and thread-safety bounds.
Files:
crates/core/src/api/runtime/callbacks.rs
willkill07
changed the title
Signed-off-by: Will Killian <wkillian@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
go/nemo_relay/stream.go (1)
141-161: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winCache and replay native close errors.
After the first failing close marks the stream closed, every later caller receives
nil. Store the cleanup error onLlmStreamand return it from repeated and concurrentClose()calls.As per path instructions, binding changes require parity with Rust/FFI behavior.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@go/nemo_relay/stream.go` around lines 141 - 161, Update LlmStream.Close to cache the native cleanup error on the stream before marking it closed, then return that cached error on repeated or concurrent calls after the first close attempt. Add or reuse a synchronized error field on LlmStream, preserve successful nil results, and mirror the same close-error replay behavior in the corresponding Rust/FFI binding implementation.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/core/src/plugins/nemo_guardrails/python.rs`:
- Line 1075: Update the cleanup watch channel and its producer/consumer flow to
store and propagate Option<FlowResult<()>> directly instead of Option<Result<(),
String>>. In the symbols around the closed receiver and the associated cleanup
handling, remove stringification and FlowError::Internal reconstruction so the
original FlowError variant is preserved.
- Around line 1093-1108: Ensure every wrapper becomes locally exhausted when
close completes: in crates/core/src/plugins/nemo_guardrails/python.rs lines
1093-1108, discard queued receiver chunks and mark the wrapper closed; in
crates/python/src/py_callable.rs lines 300-315, discard queued Python values and
reject later polling; in crates/switchyard/src/component.rs lines 1684-1687,
clear PrefixedStream.first; and in crates/switchyard/src/component.rs lines
1771-1774, clear TranslatedStream.buffered and set upstream_finished.
In `@crates/node/typed.js`:
- Around line 283-303: Update the stream.close wrapper around iteratorReady and
the bound close function so that when both iterator.return?.() and native
close() reject, neither error is silently discarded. Aggregate both failures,
such as with AggregateError, or otherwise give the native close error precedence
while preserving the existing behavior when only one operation fails.
In `@crates/python/src/py_callable.rs`:
- Around line 167-174: Restrict the CancelledError-to-Ok(None) handling in the
visible py_callable error path to Relay-initiated cancellation only. Preserve
PyStopAsyncIteration as normal EOF, but let user-raised asyncio.CancelledError
propagate through the existing error handling so aclose() is not skipped; use
the surrounding cancellation state or explicit cancel-path symbol already
present in py_callable.rs to distinguish the cases.
In `@go/nemo_relay/stream.go`:
- Around line 119-120: Refactor the locking in the stream methods around
nemo_relay_stream_next and the collector/finalizer callbacks: do not hold s.mu
during blocking cgo calls or user callbacks. Add explicit stream state and
in-flight-call tracking so Close can cancel active Next calls, release the mutex
before callbacks, and safely wait for or reject operations during shutdown;
ensure callback cleanup and shared state remain race-safe.
---
Outside diff comments:
In `@go/nemo_relay/stream.go`:
- Around line 141-161: Update LlmStream.Close to cache the native cleanup error
on the stream before marking it closed, then return that cached error on
repeated or concurrent calls after the first close attempt. Add or reuse a
synchronized error field on LlmStream, preserve successful nil results, and
mirror the same close-error replay behavior in the corresponding Rust/FFI
binding implementation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 0ca5da1f-5500-4a2d-a02a-d96f088b158f
📒 Files selected for processing (13)
crates/core/src/api/runtime/callbacks.rscrates/core/src/error.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/stream.rscrates/core/tests/integration/stream_tests.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/node/typed.jscrates/python/src/py_callable.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/switchyard/src/component.rscrates/switchyard/tests/unit/component_tests.rsgo/nemo_relay/llm_test.gogo/nemo_relay/stream.go
📜 Review details
⏰ Context from checks skipped due to timeout. (2)
- GitHub Check: Check / Run
- GitHub Check: Preview docs
🧰 Additional context used
📓 Path-based instructions (27)
**/*.rs
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
**/*.rs: Any Rust change must runjust test-rust
Any Rust change must runcargo fmt --all
Any Rust change must runcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allfor all FFI work since it is Rust work
Runjust test-rustto validate FFI changes
Runcargo clippy --workspace --all-targets -- -D warningsto enforce strict linting on FFI workWhen Rust files changed as part of Go work, also run
cargo fmt --all,just test-rust, andcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allwhen Rust files are changed as part of Node work
Runcargo clippy --workspace --all-targets -- -D warningswhen Rust files are changed as part of Node work
Runjust test-rustwhen Rust files are changed as part of Node workWhen changing the core Rust runtime or Rust-facing API surface, format Rust code with
cargo fmt(rustfmt defaults), keepcargo clippy -- -D warningsclean, and satisfycargo deny checkperdeny.toml.
**/*.rs: If any Rust code changed, always runjust test-rust.
If any Rust code changed, also runcargo fmt --all.
If any Rust code changed, also runcargo clippy --workspace --all-targets -- -D warnings.
For Rust changes headed for review, runcargo fmt --allandcargo clippy --workspace --all-targets -- -D warningseven if relying on pre-commit.
Files:
crates/core/src/error.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/switchyard/tests/unit/component_tests.rscrates/switchyard/src/component.rscrates/core/src/stream.rscrates/core/src/api/runtime/callbacks.rscrates/core/tests/integration/stream_tests.rscrates/python/src/py_callable.rs
{crates/core,crates/adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Changes to
crates/coreorcrates/adaptivemust run the full language matrix
Files:
crates/core/src/error.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/stream.rscrates/core/src/api/runtime/callbacks.rscrates/core/tests/integration/stream_tests.rs
crates/core/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)
If the change touched
crates/coreor shared runtime semantics, also usevalidate-changefor broader validation
Files:
crates/core/src/error.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/stream.rscrates/core/src/api/runtime/callbacks.rscrates/core/tests/integration/stream_tests.rs
**/*.{rs,py}
📄 CodeRabbit inference engine (AGENTS.md)
Follow binding naming conventions in Rust and Python: use
snake_case.
Files:
crates/core/src/error.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/switchyard/tests/unit/component_tests.rscrates/switchyard/src/component.rscrates/core/src/stream.rscrates/core/src/api/runtime/callbacks.rscrates/core/tests/integration/stream_tests.rscrates/python/src/py_callable.rs
**/*.{rs,py,js,mjs,cjs,ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{rs,py,js,mjs,cjs,ts,tsx}: UseJson = serde_json::Valuein Rust-facing runtime APIs where the existing code expects JSON payloads.
UseResult<T>withFlowErrorin core runtime paths, and keep errors explicit and binding-appropriate at the wrapper layer.
Keep async behavior on the existing tokio-based model; bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.
Files:
crates/core/src/error.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/node/typed.jscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/switchyard/tests/unit/component_tests.rscrates/switchyard/src/component.rscrates/core/src/stream.rscrates/core/src/api/runtime/callbacks.rscrates/core/tests/integration/stream_tests.rscrates/python/src/py_callable.rs
**/*.{rs,py,go,js,ts,c,h}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Use language-appropriate naming conventions: Rust
snake_case, C FFI exports prefixednemo_relay_, GoPascalCase, Node.jscamelCase, and Pythonsnake_case.
Files:
crates/core/src/error.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/node/typed.jsgo/nemo_relay/llm_test.gocrates/python/tests/coverage/py_callable_coverage_tests.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/switchyard/tests/unit/component_tests.rscrates/switchyard/src/component.rscrates/core/src/stream.rscrates/core/src/api/runtime/callbacks.rscrates/core/tests/integration/stream_tests.rsgo/nemo_relay/stream.gocrates/python/src/py_callable.rs
**/*.{rs,go,js,ts}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Add the SPDX license header to all Rust, Go, JavaScript, and TypeScript source files using the corresponding
//comment form.
Files:
crates/core/src/error.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/node/typed.jsgo/nemo_relay/llm_test.gocrates/python/tests/coverage/py_callable_coverage_tests.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/switchyard/tests/unit/component_tests.rscrates/switchyard/src/component.rscrates/core/src/stream.rscrates/core/src/api/runtime/callbacks.rscrates/core/tests/integration/stream_tests.rsgo/nemo_relay/stream.gocrates/python/src/py_callable.rs
{crates/**/src/**/*.rs,python/**/*.py}
📄 CodeRabbit inference engine (.agents/skills/maintain-dynamic-plugins/SKILL.md)
Do not add tests under
src; Rust tests belong in cratetests/trees, and Python SDK tests belong underpython/tests.
Files:
crates/core/src/error.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/switchyard/src/component.rscrates/core/src/stream.rscrates/core/src/api/runtime/callbacks.rscrates/python/src/py_callable.rs
**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*: Format changed files with the language-native formatter before the final lint/test pass.
If dynamic plugin behavior changed, usemaintain-dynamic-pluginsand include the native SDK, worker protocol, Python SDK, docs, packaging, and Codecov surfaces in the validation plan.
If code changes alter APIs, bindings, commands, paths, packaging behavior, observability/adaptive semantics, or documented best practices, update any dependent maintainer or consumer skills in the same branch.
During iteration, preferuv run pre-commit run --files <changed files...>.
Before review or handoff, runuv run pre-commit run --all-files.
Files:
crates/core/src/error.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/node/typed.jsgo/nemo_relay/llm_test.gocrates/python/tests/coverage/py_callable_coverage_tests.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/switchyard/tests/unit/component_tests.rscrates/switchyard/src/component.rscrates/core/src/stream.rscrates/core/src/api/runtime/callbacks.rscrates/core/tests/integration/stream_tests.rsgo/nemo_relay/stream.gocrates/python/src/py_callable.rs
crates/{core,adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If
crates/coreorcrates/adaptivechanged, run the full validation matrix across Rust, Python, Go, and Node.js.
Files:
crates/core/src/error.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/stream.rscrates/core/src/api/runtime/callbacks.rscrates/core/tests/integration/stream_tests.rs
**/*.{rs,py,go,js,ts}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If a language surface changed, always run that language's test target even when Rust core did not change.
Files:
crates/core/src/error.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/node/typed.jsgo/nemo_relay/llm_test.gocrates/python/tests/coverage/py_callable_coverage_tests.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/switchyard/tests/unit/component_tests.rscrates/switchyard/src/component.rscrates/core/src/stream.rscrates/core/src/api/runtime/callbacks.rscrates/core/tests/integration/stream_tests.rsgo/nemo_relay/stream.gocrates/python/src/py_callable.rs
**/*.{rs,py,js,ts,tsx,go,java,kt,swift}
📄 CodeRabbit inference engine (.agents/skills/add-middleware/SKILL.md)
Add tests covering registration and duplicate names, deregistration and missing names, priority ordering, callback failure policy, scope-local inheritance and cleanup, event payload semantics, immutable mark and scope fields, and parity across affected bindings.
Files:
crates/core/src/error.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/node/typed.jsgo/nemo_relay/llm_test.gocrates/python/tests/coverage/py_callable_coverage_tests.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/switchyard/tests/unit/component_tests.rscrates/switchyard/src/component.rscrates/core/src/stream.rscrates/core/src/api/runtime/callbacks.rscrates/core/tests/integration/stream_tests.rsgo/nemo_relay/stream.gocrates/python/src/py_callable.rs
crates/{core,adaptive}/**/*.rs
⚙️ CodeRabbit configuration file
crates/{core,adaptive}/**/*.rs: Review the Rust runtime for async correctness, scope isolation, middleware ordering, and event lifecycle regressions.
Pay close attention to task-local/thread-local scope propagation, callback lifetimes, stream finalization, and root_uuid isolation.
Public API changes should preserve existing behavior unless tests and docs show the intended migration path.
Files:
crates/core/src/error.rscrates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rscrates/core/src/plugins/nemo_guardrails/python.rscrates/core/src/stream.rscrates/core/src/api/runtime/callbacks.rscrates/core/tests/integration/stream_tests.rs
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}
⚙️ CodeRabbit configuration file
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.
Files:
crates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rsgo/nemo_relay/llm_test.gocrates/python/tests/coverage/py_callable_coverage_tests.rscrates/switchyard/tests/unit/component_tests.rscrates/core/tests/integration/stream_tests.rs
crates/node/**/*.{js,ts,jsx,tsx,json}
📄 CodeRabbit inference engine (.agents/skills/test-node-binding/SKILL.md)
Format changed Node files with
npm run format --workspace=nemo-relay-node
Files:
crates/node/typed.js
crates/node/**/*.{js,mjs,cjs,ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Use
camelCasefor Node.js public APIs.
Files:
crates/node/typed.js
**/*.{py,go,js,ts}
📄 CodeRabbit inference engine (.agents/skills/maintain-observability/SKILL.md)
Keep Python, Go, and Node.js config objects and subscriber/exporter methods aligned so all bindings expose the same logical knobs and semantics.
Files:
crates/node/typed.jsgo/nemo_relay/llm_test.gogo/nemo_relay/stream.go
crates/{python,ffi,node}/**/*
⚙️ CodeRabbit configuration file
crates/{python,ffi,node}/**/*: Treat binding changes as public API changes. Check for parity with the other language bindings, FFI ownership/lifetime safety,
callback error propagation, stable type conversion, and consistent async/stream semantics.
Flag changes that update one binding without corresponding tests or documentation for the same surface elsewhere.
Files:
crates/node/typed.jscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/python/src/py_callable.rs
go/nemo_relay/**/*.go
📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)
go/nemo_relay/**/*.go: Format changed Go packages withcd go/nemo_relay && go fmt ./...
Run Go tests withjust test-goto build and test the NeMo Relay Go binding
Usejust build-gowhen you want an explicit build-only pass or need the artifact for other work
Usejust ci=true test-gowhen you need the CI-style coverage and JUnit path
On macOS, setDYLD_LIBRARY_PATHto the../../target/releasedirectory before running the rawgo testcommand directlyUse
PascalCasefor public Go APIs.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
**/*.go
📄 CodeRabbit inference engine (CONTRIBUTING.md)
When changing the experimental Go binding, format Go code with
gofmtand keepgo vet ./...passing.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
{crates/python/src/py_api/mod.rs,python/nemo_relay/**/*.py,python/nemo_relay/**/*.pyi,go/nemo_relay/**/*.go,crates/node/src/api/**/*.rs}
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Update the language-native bindings for every exposed surface in Python, Go, and Node.js.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
{python/nemo_relay/**/*.py,python/nemo_relay/**/*.pyi,go/nemo_relay/**/*.go}
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Update language wrapper helpers such as Python wrapper modules, Python type stubs, and Go shorthand packages when the new behavior belongs in those helper layers.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
go/nemo_relay/**
📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Keep shared plugin helpers in
go/nemo_relayaligned with plugin registration, composition, and lifecycle behavior.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
go/nemo_relay/**/*
⚙️ CodeRabbit configuration file
go/nemo_relay/**/*: Review Go binding changes for cgo memory ownership, race safety, callback cleanup, idiomatic exported APIs, and parity with Rust/FFI behavior.
Any API change should include focused Go tests and consider race-test behavior.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
crates/core/src/{api/**/*.rs,api/runtime/**/*.rs,codec/**/*.rs,json.rs}
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Implement the new or changed public runtime behavior first in the Rust core, especially under
crates/core/src/api/and related core modules such ascrates/core/src/api/runtime/,crates/core/src/codec/, andcrates/core/src/json.rs.
Files:
crates/core/src/api/runtime/callbacks.rs
crates/core/src/api/runtime/callbacks.rs
📄 CodeRabbit inference engine (.agents/skills/add-middleware/SKILL.md)
Define or reuse the middleware callback type alias in
crates/core/src/api/runtime/callbacks.rs, using the appropriate callable signature and thread-safety bounds.
Files:
crates/core/src/api/runtime/callbacks.rs
crates/core/src/api/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/add-middleware/SKILL.md)
Preserve the documented pipeline order: conditional guardrails, request intercepts, request sanitization, execution intercepts, and response sanitization for tool and LLM execution; specialized sanitization, event creation, and dispatch for mark and scope events.
Files:
crates/core/src/api/runtime/callbacks.rs
🔇 Additional comments (13)
crates/node/typed.js (1)
198-198: LGTM!Also applies to: 214-224, 220-224, 235-253, 268-268
crates/core/src/api/runtime/callbacks.rs (1)
260-260: LGTM!Also applies to: 296-296, 307-320
crates/core/src/error.rs (1)
85-85: LGTM!crates/core/src/plugins/nemo_guardrails/python.rs (1)
7-27: LGTM!Also applies to: 287-307, 1112-1120, 1122-1180
crates/core/src/stream.rs (1)
78-78: LGTM!Also applies to: 319-319, 348-362
crates/core/tests/integration/stream_tests.rs (1)
64-104: LGTM!Also applies to: 214-256
crates/core/tests/unit/plugins/nemo_guardrails/local_python_tests.rs (1)
467-469: LGTM!Also applies to: 486-497
crates/python/src/py_callable.rs (1)
231-236: LGTM!Also applies to: 525-548
crates/python/tests/coverage/py_callable_coverage_tests.rs (1)
284-315: LGTM!Also applies to: 371-428
go/nemo_relay/stream.go (1)
89-106: LGTM!go/nemo_relay/llm_test.go (1)
993-1023: LGTM!crates/switchyard/src/component.rs (1)
6-19: LGTM!Also applies to: 802-805, 1690-1725, 1777-1790
crates/switchyard/tests/unit/component_tests.rs (1)
6-80: LGTM!Also applies to: 787-801
Signed-off-by: Will Killian <wkillian@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
crates/core/src/plugins/nemo_guardrails/python.rs (1)
1171-1179: 🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy liftFinish monitor cleanup before reporting the stream closed.
The cancellation path aborts
monitorwithout awaiting it, whilemonitor_guardrails_streamperformsworker.forget_stream()only during normal execution. Explicit close can therefore return before monitor termination and leave the worker stream registered. Guarantee deregistration on cancellation and await the aborted task before publishingclosed.As per path instructions, review this runtime for “async correctness” and “stream finalization.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/core/src/plugins/nemo_guardrails/python.rs` around lines 1171 - 1179, Update monitor cleanup in monitor_guardrails_stream so the cancellation path aborts and awaits the monitor task, ensuring its worker stream deregistration completes before close is published. Preserve the existing normal-path send_stream_monitor_error handling, and move or defer closed.send_replace until all monitor cleanup has finished.Source: Path instructions
crates/python/src/py_callable.rs (1)
248-253: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winPreserve both task and
aclose()failures.If awaiting the cancelled task and
close_async_iter()both fail, theErr(error)branch silently discards the cleanup error. Aggregate both failures, or otherwise include theaclose()failure in the returnedFlowError.As per coding guidelines, “Keep errors explicit and binding-appropriate at the wrapper layer.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/python/src/py_callable.rs` around lines 248 - 253, Update the async iterator cleanup match around next_result and close_result to preserve failures from both the cancelled task and close_async_iter. When both await operations return errors, aggregate them or include the close failure in the returned FlowError; keep successful cleanup and single-error behavior explicit and binding-appropriate.Source: Coding guidelines
crates/node/typed.js (1)
242-250: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winInvoke
iterator.return()through one memoized cleanup operation.Early native close can make
pushStreamChunk()fail whilestream.close()concurrently callsreturn(), and the producer’sfinallycalls it again. Custom iterators may perform cleanup repeatedly or reject concurrent calls. Share one cleanup promise across all three paths and await that same result.As per path instructions, binding changes must preserve “callback error propagation” and “consistent async/stream semantics.”
Also applies to: 283-288
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/node/typed.js` around lines 242 - 250, Memoize iterator cleanup in the stream producer around the iterator-return handling so the early pushStreamChunk failure path, stream.close path, and producer finally block all invoke one shared cleanup operation and await the same promise. Update the relevant cleanup helper and callers near resolveIterator, preserving callback error propagation and consistent async/stream semantics.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@go/nemo_relay/stream.go`:
- Around line 142-154: Update the stream completion flow around finishNext and
collector invocation so collector execution remains marked as active until the
callback returns. Defer Close/finalizer cleanup and collector clearing until
that work completes, while allowing a collector-initiated Close to proceed
safely without deadlocking or losing the callback. Apply the same lifecycle and
race-safety handling to the related callback path.
- Around line 218-233: Update the close state transition in the stream close
method so closeErr is assigned before setting closed and releasing the mutex,
ensuring concurrent Close calls observe the native cleanup error. Preserve the
existing cleanup, finalizer invocation, and closeDone signaling order otherwise.
---
Outside diff comments:
In `@crates/core/src/plugins/nemo_guardrails/python.rs`:
- Around line 1171-1179: Update monitor cleanup in monitor_guardrails_stream so
the cancellation path aborts and awaits the monitor task, ensuring its worker
stream deregistration completes before close is published. Preserve the existing
normal-path send_stream_monitor_error handling, and move or defer
closed.send_replace until all monitor cleanup has finished.
In `@crates/node/typed.js`:
- Around line 242-250: Memoize iterator cleanup in the stream producer around
the iterator-return handling so the early pushStreamChunk failure path,
stream.close path, and producer finally block all invoke one shared cleanup
operation and await the same promise. Update the relevant cleanup helper and
callers near resolveIterator, preserving callback error propagation and
consistent async/stream semantics.
In `@crates/python/src/py_callable.rs`:
- Around line 248-253: Update the async iterator cleanup match around
next_result and close_result to preserve failures from both the cancelled task
and close_async_iter. When both await operations return errors, aggregate them
or include the close failure in the returned FlowError; keep successful cleanup
and single-error behavior explicit and binding-appropriate.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: ef32fdec-b9e4-45c2-9742-9c2379f3cb97
📒 Files selected for processing (9)
crates/core/src/plugins/nemo_guardrails/python.rscrates/node/typed.jscrates/python/src/py_callable.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/switchyard/src/component.rscrates/switchyard/tests/unit/component_tests.rsgo/nemo_relay/llm_test.gogo/nemo_relay/stream.gopython/nemo_relay/llm.py
📜 Review details
⏰ Context from checks skipped due to timeout. (2)
- GitHub Check: Check / Run
- GitHub Check: Preview docs
🧰 Additional context used
📓 Path-based instructions (27)
**/*.{rs,py}
📄 CodeRabbit inference engine (AGENTS.md)
Follow binding naming conventions in Rust and Python: use
snake_case.
Files:
python/nemo_relay/llm.pycrates/core/src/plugins/nemo_guardrails/python.rscrates/switchyard/src/component.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/switchyard/tests/unit/component_tests.rscrates/python/src/py_callable.rs
**/*.{rs,py,js,mjs,cjs,ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{rs,py,js,mjs,cjs,ts,tsx}: UseJson = serde_json::Valuein Rust-facing runtime APIs where the existing code expects JSON payloads.
UseResult<T>withFlowErrorin core runtime paths, and keep errors explicit and binding-appropriate at the wrapper layer.
Keep async behavior on the existing tokio-based model; bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.
Files:
python/nemo_relay/llm.pycrates/core/src/plugins/nemo_guardrails/python.rscrates/switchyard/src/component.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/node/typed.jscrates/switchyard/tests/unit/component_tests.rscrates/python/src/py_callable.rs
python/nemo_relay/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Python wrapper modules live under
python/nemo_relay/, and the native extension is built fromcrates/pythonwithmaturin.
Files:
python/nemo_relay/llm.py
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
**/*.py: When changing the Python wrapper package, tests, or docs tooling, lint with Ruff (E,F,W,I), format with Ruff formatter (120-character lines, double quotes), and passtytype checking.
Add the SPDX license header to all Python source files using the#comment form.
Files:
python/nemo_relay/llm.py
**/*.{rs,py,go,js,ts,c,h}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Use language-appropriate naming conventions: Rust
snake_case, C FFI exports prefixednemo_relay_, GoPascalCase, Node.jscamelCase, and Pythonsnake_case.
Files:
python/nemo_relay/llm.pycrates/core/src/plugins/nemo_guardrails/python.rscrates/switchyard/src/component.rsgo/nemo_relay/llm_test.gocrates/python/tests/coverage/py_callable_coverage_tests.rscrates/node/typed.jscrates/switchyard/tests/unit/component_tests.rsgo/nemo_relay/stream.gocrates/python/src/py_callable.rs
{crates/python/src/py_api/mod.rs,python/nemo_relay/**/*.py,python/nemo_relay/**/*.pyi,go/nemo_relay/**/*.go,crates/node/src/api/**/*.rs}
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Update the language-native bindings for every exposed surface in Python, Go, and Node.js.
Files:
python/nemo_relay/llm.pygo/nemo_relay/llm_test.gogo/nemo_relay/stream.go
{python/nemo_relay/**/*.py,python/nemo_relay/**/*.pyi,go/nemo_relay/**/*.go}
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Update language wrapper helpers such as Python wrapper modules, Python type stubs, and Go shorthand packages when the new behavior belongs in those helper layers.
Files:
python/nemo_relay/llm.pygo/nemo_relay/llm_test.gogo/nemo_relay/stream.go
{crates/**/src/**/*.rs,python/**/*.py}
📄 CodeRabbit inference engine (.agents/skills/maintain-dynamic-plugins/SKILL.md)
Do not add tests under
src; Rust tests belong in cratetests/trees, and Python SDK tests belong underpython/tests.
Files:
python/nemo_relay/llm.pycrates/core/src/plugins/nemo_guardrails/python.rscrates/switchyard/src/component.rscrates/python/src/py_callable.rs
**/*.{py,go,js,ts}
📄 CodeRabbit inference engine (.agents/skills/maintain-observability/SKILL.md)
Keep Python, Go, and Node.js config objects and subscriber/exporter methods aligned so all bindings expose the same logical knobs and semantics.
Files:
python/nemo_relay/llm.pygo/nemo_relay/llm_test.gocrates/node/typed.jsgo/nemo_relay/stream.go
**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*: Format changed files with the language-native formatter before the final lint/test pass.
If dynamic plugin behavior changed, usemaintain-dynamic-pluginsand include the native SDK, worker protocol, Python SDK, docs, packaging, and Codecov surfaces in the validation plan.
If code changes alter APIs, bindings, commands, paths, packaging behavior, observability/adaptive semantics, or documented best practices, update any dependent maintainer or consumer skills in the same branch.
During iteration, preferuv run pre-commit run --files <changed files...>.
Before review or handoff, runuv run pre-commit run --all-files.
Files:
python/nemo_relay/llm.pycrates/core/src/plugins/nemo_guardrails/python.rscrates/switchyard/src/component.rsgo/nemo_relay/llm_test.gocrates/python/tests/coverage/py_callable_coverage_tests.rscrates/node/typed.jscrates/switchyard/tests/unit/component_tests.rsgo/nemo_relay/stream.gocrates/python/src/py_callable.rs
**/*.{rs,py,go,js,ts}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If a language surface changed, always run that language's test target even when Rust core did not change.
Files:
python/nemo_relay/llm.pycrates/core/src/plugins/nemo_guardrails/python.rscrates/switchyard/src/component.rsgo/nemo_relay/llm_test.gocrates/python/tests/coverage/py_callable_coverage_tests.rscrates/node/typed.jscrates/switchyard/tests/unit/component_tests.rsgo/nemo_relay/stream.gocrates/python/src/py_callable.rs
**/*.{rs,py,js,ts,tsx,go,java,kt,swift}
📄 CodeRabbit inference engine (.agents/skills/add-middleware/SKILL.md)
Add tests covering registration and duplicate names, deregistration and missing names, priority ordering, callback failure policy, scope-local inheritance and cleanup, event payload semantics, immutable mark and scope fields, and parity across affected bindings.
Files:
python/nemo_relay/llm.pycrates/core/src/plugins/nemo_guardrails/python.rscrates/switchyard/src/component.rsgo/nemo_relay/llm_test.gocrates/python/tests/coverage/py_callable_coverage_tests.rscrates/node/typed.jscrates/switchyard/tests/unit/component_tests.rsgo/nemo_relay/stream.gocrates/python/src/py_callable.rs
python/nemo_relay/**/*
⚙️ CodeRabbit configuration file
python/nemo_relay/**/*: Review Python wrapper changes for typed API consistency, contextvars-based scope isolation, async behavior, and parity with the native extension.
Stubs and runtime implementations should stay aligned.
Files:
python/nemo_relay/llm.py
**/*.rs
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
**/*.rs: Any Rust change must runjust test-rust
Any Rust change must runcargo fmt --all
Any Rust change must runcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allfor all FFI work since it is Rust work
Runjust test-rustto validate FFI changes
Runcargo clippy --workspace --all-targets -- -D warningsto enforce strict linting on FFI workWhen Rust files changed as part of Go work, also run
cargo fmt --all,just test-rust, andcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allwhen Rust files are changed as part of Node work
Runcargo clippy --workspace --all-targets -- -D warningswhen Rust files are changed as part of Node work
Runjust test-rustwhen Rust files are changed as part of Node workWhen changing the core Rust runtime or Rust-facing API surface, format Rust code with
cargo fmt(rustfmt defaults), keepcargo clippy -- -D warningsclean, and satisfycargo deny checkperdeny.toml.
**/*.rs: If any Rust code changed, always runjust test-rust.
If any Rust code changed, also runcargo fmt --all.
If any Rust code changed, also runcargo clippy --workspace --all-targets -- -D warnings.
For Rust changes headed for review, runcargo fmt --allandcargo clippy --workspace --all-targets -- -D warningseven if relying on pre-commit.
Files:
crates/core/src/plugins/nemo_guardrails/python.rscrates/switchyard/src/component.rscrates/python/tests/coverage/py_callable_coverage_tests.rscrates/switchyard/tests/unit/component_tests.rscrates/python/src/py_callable.rs
{crates/core,crates/adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Changes to
crates/coreorcrates/adaptivemust run the full language matrix
Files:
crates/core/src/plugins/nemo_guardrails/python.rs
crates/core/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)
If the change touched
crates/coreor shared runtime semantics, also usevalidate-changefor broader validation
Files:
crates/core/src/plugins/nemo_guardrails/python.rs
**/*.{rs,go,js,ts}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Add the SPDX license header to all Rust, Go, JavaScript, and TypeScript source files using the corresponding
//comment form.
Files:
crates/core/src/plugins/nemo_guardrails/python.rscrates/switchyard/src/component.rsgo/nemo_relay/llm_test.gocrates/python/tests/coverage/py_callable_coverage_tests.rscrates/node/typed.jscrates/switchyard/tests/unit/component_tests.rsgo/nemo_relay/stream.gocrates/python/src/py_callable.rs
crates/{core,adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If
crates/coreorcrates/adaptivechanged, run the full validation matrix across Rust, Python, Go, and Node.js.
Files:
crates/core/src/plugins/nemo_guardrails/python.rs
crates/{core,adaptive}/**/*.rs
⚙️ CodeRabbit configuration file
crates/{core,adaptive}/**/*.rs: Review the Rust runtime for async correctness, scope isolation, middleware ordering, and event lifecycle regressions.
Pay close attention to task-local/thread-local scope propagation, callback lifetimes, stream finalization, and root_uuid isolation.
Public API changes should preserve existing behavior unless tests and docs show the intended migration path.
Files:
crates/core/src/plugins/nemo_guardrails/python.rs
go/nemo_relay/**/*.go
📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)
go/nemo_relay/**/*.go: Format changed Go packages withcd go/nemo_relay && go fmt ./...
Run Go tests withjust test-goto build and test the NeMo Relay Go binding
Usejust build-gowhen you want an explicit build-only pass or need the artifact for other work
Usejust ci=true test-gowhen you need the CI-style coverage and JUnit path
On macOS, setDYLD_LIBRARY_PATHto the../../target/releasedirectory before running the rawgo testcommand directlyUse
PascalCasefor public Go APIs.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
**/*.go
📄 CodeRabbit inference engine (CONTRIBUTING.md)
When changing the experimental Go binding, format Go code with
gofmtand keepgo vet ./...passing.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
go/nemo_relay/**
📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Keep shared plugin helpers in
go/nemo_relayaligned with plugin registration, composition, and lifecycle behavior.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
go/nemo_relay/**/*
⚙️ CodeRabbit configuration file
go/nemo_relay/**/*: Review Go binding changes for cgo memory ownership, race safety, callback cleanup, idiomatic exported APIs, and parity with Rust/FFI behavior.
Any API change should include focused Go tests and consider race-test behavior.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}
⚙️ CodeRabbit configuration file
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.
Files:
go/nemo_relay/llm_test.gocrates/python/tests/coverage/py_callable_coverage_tests.rscrates/switchyard/tests/unit/component_tests.rs
crates/{python,ffi,node}/**/*
⚙️ CodeRabbit configuration file
crates/{python,ffi,node}/**/*: Treat binding changes as public API changes. Check for parity with the other language bindings, FFI ownership/lifetime safety,
callback error propagation, stable type conversion, and consistent async/stream semantics.
Flag changes that update one binding without corresponding tests or documentation for the same surface elsewhere.
Files:
crates/python/tests/coverage/py_callable_coverage_tests.rscrates/node/typed.jscrates/python/src/py_callable.rs
crates/node/**/*.{js,ts,jsx,tsx,json}
📄 CodeRabbit inference engine (.agents/skills/test-node-binding/SKILL.md)
Format changed Node files with
npm run format --workspace=nemo-relay-node
Files:
crates/node/typed.js
crates/node/**/*.{js,mjs,cjs,ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Use
camelCasefor Node.js public APIs.
Files:
crates/node/typed.js
🔇 Additional comments (8)
python/nemo_relay/llm.py (1)
36-36: LGTM!Also applies to: 123-123, 174-174, 262-262, 322-324, 363-363, 396-396, 415-415
crates/switchyard/src/component.rs (1)
802-805: LGTM!Also applies to: 1684-1689, 1773-1780
crates/switchyard/tests/unit/component_tests.rs (1)
32-50: LGTM!Also applies to: 788-801
crates/python/src/py_callable.rs (1)
154-197: LGTM!Also applies to: 229-247, 254-338, 547-570
crates/python/tests/coverage/py_callable_coverage_tests.rs (1)
263-324: LGTM!Also applies to: 367-441, 581-583
crates/core/src/plugins/nemo_guardrails/python.rs (1)
287-307: LGTM!Also applies to: 1072-1107, 1111-1170
crates/node/typed.js (1)
198-241: LGTM!Also applies to: 251-282, 289-311
go/nemo_relay/llm_test.go (1)
977-1023: LGTM!Also applies to: 1025-1049, 1051-1081
Signed-off-by: Will Killian <wkillian@nvidia.com>
Signed-off-by: Will Killian <wkillian@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@go/nemo_relay/stream.go`:
- Around line 211-217: Update stream.go’s closure path around finishClose to
distinguish collector-reentrant Close calls from external Close calls: retain
asynchronous cleanup only for the reentrant collector case, while external
callers synchronously wait for finishClose and closeDone before returning. Add
coverage in llm_test.go at the specified range by invoking external Close
concurrently and asserting it remains blocked until the collector and finalizer
complete.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: cc3dcb1d-7168-408a-88e2-a826b9eabdff
📒 Files selected for processing (5)
docs/about-nemo-relay/release-notes/highlights.mdxdocs/about-nemo-relay/release-notes/index.mdxdocs/about-nemo-relay/release-notes/known-issues.mdxgo/nemo_relay/llm_test.gogo/nemo_relay/stream.go
📜 Review details
⚠️ CI failures not shown inline (1)
Commit Status: codecov/project/Go Binding: codecov/project/Go Binding
Conclusion: failure
94.17% (target 95.00%)
🧰 Additional context used
📓 Path-based instructions (21)
**/*.mdx
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
MDX top-of-file SPDX comments must use {/* ... */} delimiters instead of HTML comment delimiters (Must-Fix)
In MDX files, top-of-file comments must use JSX comment delimiters (
{/*to open and*/}to close); do not use HTML comments for MDX SPDX headers
Files:
docs/about-nemo-relay/release-notes/highlights.mdxdocs/about-nemo-relay/release-notes/index.mdxdocs/about-nemo-relay/release-notes/known-issues.mdx
**/*.{md,mdx}
📄 CodeRabbit inference engine (AGENTS.md)
Update
README.md,fern/, package READMEs, and binding-support notes when public behavior, package names, examples, or supported bindings change.
**/*.{md,mdx}: Prefer the documented public API, not internal shortcuts
Keep package names, repo references, and build commands current
Keep release-process and release-notes guidance in repo-maintainer docs such asRELEASING.md, not as user-facing docs pages orCHANGELOG.md
Keep stable user-facing wrappers atscripts/root in docs and examples; only point at namespaced helper paths when documenting internal maintenance work
When detailed dynamic plugin guides exist, keep Rust native plugin examples, Python worker plugin examples, andgrpc-v1protocol details on separate pagesIf links in documentation change, run
just docs-linkcheck.
Files:
docs/about-nemo-relay/release-notes/highlights.mdxdocs/about-nemo-relay/release-notes/index.mdxdocs/about-nemo-relay/release-notes/known-issues.mdx
**/*.{md,markdown,mdx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Add the SPDX license header to all Markdown/MDX documentation files using the HTML comment block form.
Files:
docs/about-nemo-relay/release-notes/highlights.mdxdocs/about-nemo-relay/release-notes/index.mdxdocs/about-nemo-relay/release-notes/known-issues.mdx
docs/about-nemo-relay/release-notes/{index,highlights,known-issues}.mdx
📄 CodeRabbit inference engine (.agents/skills/draft-release-notes/SKILL.md)
docs/about-nemo-relay/release-notes/{index,highlights,known-issues}.mdx: Update onlydocs/about-nemo-relay/release-notes/index.mdx,docs/about-nemo-relay/release-notes/highlights.mdx, anddocs/about-nemo-relay/release-notes/known-issues.mdxunless the release changes their route or entry points.
Preserve the existing MDX front matter and the JSX SPDX comment in the release-notes pages.
Files:
docs/about-nemo-relay/release-notes/highlights.mdxdocs/about-nemo-relay/release-notes/index.mdxdocs/about-nemo-relay/release-notes/known-issues.mdx
{docs,examples}/**/*
📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Update docs and examples.
Files:
docs/about-nemo-relay/release-notes/highlights.mdxdocs/about-nemo-relay/release-notes/index.mdxdocs/about-nemo-relay/release-notes/known-issues.mdx
**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*: Format changed files with the language-native formatter before the final lint/test pass.
If dynamic plugin behavior changed, usemaintain-dynamic-pluginsand include the native SDK, worker protocol, Python SDK, docs, packaging, and Codecov surfaces in the validation plan.
If code changes alter APIs, bindings, commands, paths, packaging behavior, observability/adaptive semantics, or documented best practices, update any dependent maintainer or consumer skills in the same branch.
During iteration, preferuv run pre-commit run --files <changed files...>.
Before review or handoff, runuv run pre-commit run --all-files.
Files:
docs/about-nemo-relay/release-notes/highlights.mdxdocs/about-nemo-relay/release-notes/index.mdxdocs/about-nemo-relay/release-notes/known-issues.mdxgo/nemo_relay/llm_test.gogo/nemo_relay/stream.go
docs/**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If documentation examples or commands under
docs/change, run the targeted docs checks appropriate to the change.
Files:
docs/about-nemo-relay/release-notes/highlights.mdxdocs/about-nemo-relay/release-notes/index.mdxdocs/about-nemo-relay/release-notes/known-issues.mdx
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}: Review documentation for technical accuracy against the current API, command correctness, and consistency across language bindings.
Flag stale examples, missing SPDX headers where required, and instructions that no longer match CI or pre-commit behavior.
Files:
docs/about-nemo-relay/release-notes/highlights.mdxdocs/about-nemo-relay/release-notes/index.mdxdocs/about-nemo-relay/release-notes/known-issues.mdx
docs/about-nemo-relay/release-notes/known-issues.mdx
📄 CodeRabbit inference engine (.agents/skills/draft-release-notes/SKILL.md)
docs/about-nemo-relay/release-notes/known-issues.mdx: Inknown-issues.mdx, preserve every prior fixed-item bullet under release-labeled subsections; do not summarize, deduplicate, or omit it.
Inknown-issues.mdx, keep the complete fixed-item history recorded in earlier release-note pages.
Files:
docs/about-nemo-relay/release-notes/known-issues.mdx
go/nemo_relay/**/*.go
📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)
go/nemo_relay/**/*.go: Format changed Go packages withcd go/nemo_relay && go fmt ./...
Run Go tests withjust test-goto build and test the NeMo Relay Go binding
Usejust build-gowhen you want an explicit build-only pass or need the artifact for other work
Usejust ci=true test-gowhen you need the CI-style coverage and JUnit path
On macOS, setDYLD_LIBRARY_PATHto the../../target/releasedirectory before running the rawgo testcommand directlyUse
PascalCasefor public Go APIs.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
**/*.go
📄 CodeRabbit inference engine (CONTRIBUTING.md)
When changing the experimental Go binding, format Go code with
gofmtand keepgo vet ./...passing.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
**/*.{rs,py,go,js,ts,c,h}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Use language-appropriate naming conventions: Rust
snake_case, C FFI exports prefixednemo_relay_, GoPascalCase, Node.jscamelCase, and Pythonsnake_case.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
**/*.{rs,go,js,ts}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Add the SPDX license header to all Rust, Go, JavaScript, and TypeScript source files using the corresponding
//comment form.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
{crates/python/src/py_api/mod.rs,python/nemo_relay/**/*.py,python/nemo_relay/**/*.pyi,go/nemo_relay/**/*.go,crates/node/src/api/**/*.rs}
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Update the language-native bindings for every exposed surface in Python, Go, and Node.js.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
{python/nemo_relay/**/*.py,python/nemo_relay/**/*.pyi,go/nemo_relay/**/*.go}
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Update language wrapper helpers such as Python wrapper modules, Python type stubs, and Go shorthand packages when the new behavior belongs in those helper layers.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
**/*.{py,go,js,ts}
📄 CodeRabbit inference engine (.agents/skills/maintain-observability/SKILL.md)
Keep Python, Go, and Node.js config objects and subscriber/exporter methods aligned so all bindings expose the same logical knobs and semantics.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
go/nemo_relay/**
📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Keep shared plugin helpers in
go/nemo_relayaligned with plugin registration, composition, and lifecycle behavior.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
**/*.{rs,py,go,js,ts}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If a language surface changed, always run that language's test target even when Rust core did not change.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
**/*.{rs,py,js,ts,tsx,go,java,kt,swift}
📄 CodeRabbit inference engine (.agents/skills/add-middleware/SKILL.md)
Add tests covering registration and duplicate names, deregistration and missing names, priority ordering, callback failure policy, scope-local inheritance and cleanup, event payload semantics, immutable mark and scope fields, and parity across affected bindings.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
go/nemo_relay/**/*
⚙️ CodeRabbit configuration file
go/nemo_relay/**/*: Review Go binding changes for cgo memory ownership, race safety, callback cleanup, idiomatic exported APIs, and parity with Rust/FFI behavior.
Any API change should include focused Go tests and consider race-test behavior.
Files:
go/nemo_relay/llm_test.gogo/nemo_relay/stream.go
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}
⚙️ CodeRabbit configuration file
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.
Files:
go/nemo_relay/llm_test.go
🔇 Additional comments (5)
docs/about-nemo-relay/release-notes/highlights.mdx (1)
122-124: LGTM!docs/about-nemo-relay/release-notes/index.mdx (1)
36-39: LGTM!Also applies to: 76-79
docs/about-nemo-relay/release-notes/known-issues.mdx (1)
84-105: LGTM!go/nemo_relay/llm_test.go (1)
977-991: LGTM!Also applies to: 993-1023, 1025-1049, 1101-1131
go/nemo_relay/stream.go (1)
13-13: LGTM!Also applies to: 23-23, 49-63, 95-113, 124-168, 183-210, 224-250
Signed-off-by: Will Killian <wkillian@nvidia.com>
Signed-off-by: Will Killian <wkillian@nvidia.com>
|
/merge |
4 participants
Warning
BREAKING CHANGE — Rust and Go stream APIs:
LlmJsonStreamis no longer aPin<Box<dyn Stream<...>>>alias; Rust callers that construct it directly or rely on the aliased boxed-stream type must migrate toLlmJsonStream::neworLlmJsonStream::from_closeable. GoLlmStream.Closenow returns anerror, so code that depends on its previousClose()method type must update. Early explicit close now finalizes the partial response and emits an interrupted lifecycle end event.Overview
Make explicit close behavior for managed LLM streams consistent across Rust, Python, Node, Go, and the C FFI.
Details
await stream.aclose()while making it wait for async-generator cancellation andaclose()cleanup.await stream.close(), Gostream.Close() error, and Cnemo_relay_stream_close.Where should the reviewer start?
Start with
crates/core/src/api/runtime/callbacks.rsandcrates/core/src/stream.rsfor the shared close contract, then review the Python, Node, FFI, and Go bridge implementations. Seedocs/instrument-applications/code-examples.mdxfor consumer usage.Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)
Summary by CodeRabbit
aclose), Node.js (close), Go (Close() error), and C (nemo_relay_stream_close), all with idempotent behavior and observable completion.