Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/proto-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,15 +27,15 @@ jobs:
env:
PROTO_CHECK_DIR: /tmp/proto_check
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Install protoc
run: |
sudo apt-get update
sudo apt-get install -y protobuf-compiler

- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: '3.12'

Expand Down Expand Up @@ -94,7 +94,7 @@ jobs:
name: Rust build and test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@6bed0761d98439e5a578e2877258200ad565ba87 # stable
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/rust-core-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
name: Rust Kernel Tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: dtolnay/rust-toolchain@6bed0761d98439e5a578e2877258200ad565ba87 # stable
with:
toolchain: stable
Expand All @@ -30,12 +30,12 @@ jobs:
name: Node.js Binding Tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: dtolnay/rust-toolchain@6bed0761d98439e5a578e2877258200ad565ba87 # stable
with:
toolchain: stable
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: '24'
- name: Build native module
Expand All @@ -56,12 +56,12 @@ jobs:
matrix:
python-version: ['3.11', '3.12', '3.13']
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: dtolnay/rust-toolchain@6bed0761d98439e5a578e2877258200ad565ba87 # stable
with:
toolchain: stable
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}
- name: Create venv and build
Expand Down
289 changes: 267 additions & 22 deletions .github/workflows/rust-core-wheels.yml

Large diffs are not rendered by default.

23 changes: 23 additions & 0 deletions CONTRACTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -235,6 +235,29 @@ but do not prevent the remaining callables from running.
Source reference: `bindings/python/src/coordinator/mod.rs::cleanup()` and
`bindings/python/src/coordinator/capabilities.rs::register_cleanup()`.

### Rust-backed Python Session Cleanup

For the PyO3 `RustSession`, `cleanup()` claims and then awaits `session:end` at most
once per successfully initialized lifetime, before registered resource callbacks run.
A successful later `initialize()` resets that claim. Cancellation while a claimed
terminal dispatch is in progress may abort it: handlers not yet reached may never
receive the event, and a later cleanup does not replay it. Cancellation during
resource callbacks may interrupt remaining teardown without allowing a second
terminal attempt. The caller or host retaining and awaiting the cleanup task owns any
deadline or abandonment decision; waiter cancellation does not guarantee callbacks
drain.

Handler cancellation is requested asynchronously. An immediate retry may run
resource callbacks before a prior terminal handler observes cancellation; the
cancellation path has no handler-drain or ordering guarantee.

Uninitialized or partially initialized sessions still run registered resource
callbacks. Callbacks run in reverse registration order, tolerate errors, and may run
again on repeated cleanup, so they must be idempotent. Error tolerance does not make
cleanup cancellation-proof. Hooks and cleanup callbacks must not recursively await
`session.cleanup()`, because the outer cleanup is awaiting them and would form an
await-dependency cycle; this is a caller contract, not a runtime-enforced guard.

### `on_session_ready(coordinator)` — Optional

```python
Expand Down
43 changes: 16 additions & 27 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

44 changes: 41 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,11 +87,22 @@ The kernel provides **capabilities** without **decisions**:

### For consumers

Core `2.0.1` combines the patched PyO3 dependency family, lifecycle contract
work, and updated Actions pins. It is available only when the exact version is
on PyPI, a non-draft GitHub release is published, and that release includes
the matching qualification receipt. Branch and pull-request artifacts are not
published release artifacts.

Install an available, qualified version as a binary-only dependency:

```bash
pip install amplifier-core
python -m pip install --only-binary=:all: amplifier-core==<qualified-release>
```

This installs a pre-built wheel with the Rust kernel included. No Rust toolchain required.
Verify the official release's automated evidence archive and `SHA256SUMS`
before adoption. Follow [Native artifact qualification](docs/NATIVE_ARTIFACTS.md)
to match the installed extension and wheel to its build receipt. Do not rebuild
native Core as part of every application release when a qualified wheel exists.

For complete Amplifier installation and usage:
**-> https://github.com/microsoft/amplifier**
Expand Down Expand Up @@ -122,7 +133,33 @@ python scripts/generate_grpc_stubs.py

See [docs/RUST_CORE_TESTING.md](docs/RUST_CORE_TESTING.md) for the full development setup guide.

**Build dependencies**: Rust 1.70+, maturin
**Build dependencies**: current stable Rust and maturin, using the locked
dependencies. The resolved graph currently has a Wasmtime-driven Rust floor of
at least 1.92 (PyO3 alone has a 1.83 floor); the project's actual MSRV has not
been independently validated. This is guidance, not a new minimum-Rust CI
requirement.

## Core 2.0.1 native-artifact and lifecycle release

Version `2.0.1` updates PyO3 to `0.29.2`,
`pyo3-async-runtimes` to `0.29.0`, and `pyo3-log` to `0.13.4`, outside the
affected ranges for GHSA-36hh-v3qg-5jq4, GHSA-chgr-c6px-7xpp,
RustSec-2026-0176, and RustSec-2026-0177.

The binding keeps `abi3-py311`, `multiple-pymethods`, and
the deprecated `generate-import-lib` feature for compatibility; Windows
qualification is required before any future linkage migration to `raw-dylib`.
The lifecycle work from [PR #114](https://github.com/microsoft/amplifier-core/pull/114)
and the PyO3 remediation from
[PR #115](https://github.com/microsoft/amplifier-core/pull/115) retain their
separate attribution while shipping together in `2.0.1`.

The Rust-backed Python session attempts `session:end` at most once per
initialized lifetime; cancellation can interrupt delivery, and the host owns
cleanup completion. See [CONTRACTS.md](CONTRACTS.md) for the authoritative
lifecycle and registration-ownership contracts. See
[Native Artifact Qualification](docs/NATIVE_ARTIFACTS.md) for evidence and
adoption requirements.

## Core Concepts

Expand Down Expand Up @@ -248,6 +285,7 @@ For complete module development guide:
- [Module Source Protocol](docs/MODULE_SOURCE_PROTOCOL.md) - Custom module loading
- [Rust Core Testing](docs/RUST_CORE_TESTING.md) - Development setup and testing guide
- [Rust Core Limitations](docs/RUST_CORE_LIMITATIONS.md) - Known limitations
- [Native Artifact Qualification](docs/NATIVE_ARTIFACTS.md) - release evidence and consumer verification

**Philosophy**:

Expand Down
8 changes: 4 additions & 4 deletions bindings/python/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "amplifier-core-py"
version = "2.0.0"
version = "2.0.1"
edition = "2021"
description = "PyO3 bridge for amplifier-core Rust kernel"
license = "MIT"
Expand All @@ -16,9 +16,9 @@ wasm = ["amplifier-core/wasm"]

[dependencies]
amplifier-core = { path = "../../crates/amplifier-core" }
pyo3 = { version = "0.28.2", features = ["generate-import-lib", "multiple-pymethods", "abi3-py311"] }
pyo3-async-runtimes = { version = "0.28", features = ["tokio-runtime"] }
pyo3-log = "0.13"
pyo3 = { version = "0.29.2", features = ["generate-import-lib", "multiple-pymethods", "abi3-py311"] }
pyo3-async-runtimes = { version = "0.29.0", features = ["tokio-runtime"] }
pyo3-log = "0.13.4"
log = "0.4"
prost = "0.13"
serde_json = "1"
Expand Down
61 changes: 55 additions & 6 deletions bindings/python/src/bridges.rs
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,32 @@ pub(crate) struct PyHookHandlerBridge {
unsafe impl Send for PyHookHandlerBridge {}
unsafe impl Sync for PyHookHandlerBridge {}

/// into_future_with_locals does not cancel its Python task when the Rust
/// receiver is dropped. Keep exact ownership so cancellation cannot orphan a
/// hook callback or affect an unrelated task on the same event loop.
struct OwnedHookTask {
owner: Py<PyAny>,
event_loop: Py<PyAny>,
cancel_on_drop: bool,
}

impl Drop for OwnedHookTask {
fn drop(&mut self) {
if self.cancel_on_drop {
let result = Python::try_attach(|py| -> PyResult<()> {
let cancel = self.owner.bind(py).getattr("cancel")?;
self.event_loop
.bind(py)
.call_method1("call_soon_threadsafe", (cancel,))?;
Ok(())
});
if let Some(Err(error)) = result {
log::warn!("Could not cancel owned Python hook task: {error}");
}
}
}
}

impl HookHandler for PyHookHandlerBridge {
fn handle(
&self,
Expand Down Expand Up @@ -100,7 +126,7 @@ impl HookHandler for PyHookHandlerBridge {
// fall back to the registration loop when a native callback (such as
// a blocking WASM host import) has lost Tokio task-local state.
let py_result = if is_coro {
let future = Python::try_attach(|py| {
let (future, mut task) = Python::try_attach(|py| -> PyResult<_> {
let locals = pyo3_async_runtimes::tokio::get_current_locals(py)
.ok()
.or_else(|| self.fallback_locals.clone())
Expand All @@ -109,10 +135,31 @@ impl HookHandler for PyHookHandlerBridge {
"No running Python event loop available for coroutine conversion",
)
})?;
pyo3_async_runtimes::into_future_with_locals(
&locals,
py_result_or_coro.into_bound(py),
)
let owner = py
.import("amplifier_core._async_compat")?
.getattr("_OwnedHookTask")?
.call1((py_result_or_coro.bind(py),))?;
let runner = owner.call_method0("run")?;
let future =
match pyo3_async_runtimes::into_future_with_locals(&locals, runner.clone())
{
Ok(future) => future,
Err(error) => {
// Scheduling failed (for example, the loop closed).
// Neither coroutine was adopted by a Python task.
let _ = runner.call_method0("close");
let _ = owner.call_method0("cancel");
return Err(error);
}
};
Ok((
future,
OwnedHookTask {
owner: owner.unbind(),
event_loop: locals.event_loop(py).unbind(),
cancel_on_drop: true,
},
))
})
.ok_or_else(|| HookError::HandlerFailed {
message: "Failed to attach to Python runtime for coroutine conversion"
Expand All @@ -127,7 +174,9 @@ impl HookHandler for PyHookHandlerBridge {
// Await OUTSIDE the GIL — drives the Python coroutine on the
// current task's loop, or its registration loop after native
// re-entry has lost task-local state.
future.await.map_err(|e| HookError::HandlerFailed {
let result = future.await;
task.cancel_on_drop = false;
result.map_err(|e| HookError::HandlerFailed {
message: format!("Python async handler error: {e}"),
handler_name: None,
})?
Expand Down
Loading
Loading