[Enhancement]: Explore configuration-driven Python function auto-instrumentation

[ericevans-nv]

Affected area

• Python binding
• Middleware or guardrails
• Plugins
• Framework integrations
• Documentation or examples

Problem or opportunity

Relay does not currently provide a configuration-driven way to select functions in an existing Python application and apply Relay middleware to their execution automatically.

The expected capability is for users to identify supported application functions and middleware policies through Relay configuration. When the application runs with Relay instrumentation enabled, Relay would connect the selected functions to its middleware runtime.

For example, an application may already define and use:

async def send_email(recipient, subject, body):
    ...

A user could identify that function in Relay configuration and apply middleware that inspects or modifies its inputs, controls its execution, and inspects or modifies its output. The function implementation and existing call sites would remain unchanged.

The investigation will determine which Python functions can be supported reliably, what setup is required, and how this capability should integrate with Relay.

Proposed enhancement

Investigate and prototype an optional Python auto-instrumentation capability that applies Relay middleware to selected application functions without requiring changes to the application's source code.

Users should be able to:

1. Identify supported application functions in Relay configuration.
2. Associate those functions with configured Relay middleware.
3. Launch the application with Relay instrumentation enabled.
4. Verify that the selected function calls are intercepted.
5. Apply existing Relay middleware behavior to function inputs, execution, and outputs.

The instrumentation capability should establish the connection between the selected application functions and Relay's middleware runtime. The application's original functions and call sites should remain unchanged.

Runtime contract and binding impact

This enhancement is scoped to the Python binding. A broader cross-language contract and binding-specific feasibility are being investigated separately.

Instrumented calls should preserve Relay's existing managed execution ordering and middleware semantics. Auto-instrumentation should establish the managed invocation boundary, not create a second middleware engine.

When no auto-instrumentation configuration is present, Relay and application behavior must remain unchanged.

When the application is launched normally without Relay instrumentation, its behavior must remain unchanged.

When instrumentation is enabled, configured functions should use Relay's existing managed tool execution and middleware semantics, including input handling, execution control, output handling, scopes, and lifecycle events.

Functions that are not configured should continue normally. Instrumentation should preserve the behavior of supported Python functions and clearly report targets that cannot be instrumented. Existing Relay privacy and sensitive-data handling requirements continue to apply.

Other language bindings are outside this issue's implementation scope.

Acceptance criteria

• The supported configuration and setup for Python function instrumentation are defined.
• A proof of concept applies Relay middleware to a configured function in an unmodified Python application.
• The proof of concept supports Relay's existing input, execution-control, output, scope, and lifecycle behavior.
• Unconfigured functions and applications launched without instrumentation retain their existing behavior.
• Unsupported or unresolved function targets produce clear diagnostics, and known Python instrumentation limitations are documented.
• The investigation documents the recommended architecture, security requirements, supported scope, and next implementation steps.

[willkill07]

1. Functions are exposed to agents as tools.
2. Supported agent framework integrations should wrap the tool call execution boundary.
3. Tool call execution boundaries do support all possible tool intercepts and guardrails.

What am I missing here? As far as I know, the existing pipeline should be the only way to gate/modify tool execution and ensure cross-language compatibility. It's not random function middleware we want to provide. It's tool call middleware.

[willkill07]

Relay can execute the registered middleware just fine, but that doesn't mean that Relay should dynamically inject/replace/monkeypatch arbitrary functions based on a runtime configuration. Not only is that incredibly dangerous, but would probably violate several security policies resulting in an assigned CVE.

Additionally, such a feature would only be limited to languages that support such a capability, providing an inconsistent experience across consumers of the library.

Any application requiring dynamic runtime behavior changes with NeMo Relay can consider monkey-patching a function with a call to nemo_relay.tool.execute (Python) or the JavaScript/TypeScript equivalent.