Add WAL provider support to Drasi.create

CHANGELOG.md

@@ -7,6 +7,20 @@ and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.ht
 
 ## [Unreleased]
 
+### Added
+
+- `Drasi.create` and `Drasi.from_config` accept
+  `wal_provider={"path": ...}`. The built-in redb provider stores one WAL file
+  per durable transient source in that directory; `kind` defaults to `redb`.
+- `add_python_source(..., durability=True)` persists changes before acknowledging
+  them and replays them after a fresh-process restart. The async and blocking
+  APIs expose the same option.
+
+### Changed
+
+- Volatile Python sources now report that they cannot replay changes. Persistent
+  queries must use a durable Python source with a configured WAL provider.
+
 ## [0.3.0] - 2026-10-01
 
 Built against drasi-lib 0.9. This is a minor release because two behaviour

Cargo.toml

@@ -41,6 +41,7 @@ drasi-core = "=0.5.10"
 drasi-host-sdk = { version = "=0.11.3", features = ["watcher", "registry"] }
 drasi-plugin-sdk = "=0.11.3"
 drasi-state-store-redb = "=0.2.8"
+drasi-wal-redb = "=0.2.10"
 drasi-index-rocksdb = { version = "=0.6.4", optional = true, features = ["plugin-descriptor"] }
 
 anyhow = "1"

README.md

@@ -163,8 +163,12 @@ drasi = await Drasi.create(
     "app",
     state_store={"kind": "redb", "path": "state.redb"},
     index_store={"kind": "rocksdb", "path": "index"},
+    wal_provider={"path": "wal"},
 )
 
+await drasi.start()
+await drasi.add_python_source("orders", durability=True)
+
 
 async def handle(event):
     await write_somewhere(event)  # if this raises, the event is retried

docs/api-audit.md

@@ -25,7 +25,7 @@ one genuine advantage — streaming as async iterators — has been carried over
 | Components | **loaded at runtime** from `ghcr.io/drasi-project` | **linked at compile time**, one PyPI package each |
 | PyPI packages | 1 | 23 + workspace root |
 | Engine methods (at audit) | 37 | 27 |
-| Errors | 8 classes, 26 codes | 1 class, no codes |
+| Errors | 8 classes, 30 codes | 1 class, no codes |
 | Streaming | none | async iterators |
 | Python | ≥3.10 (abi3) | ≥3.11 |
 
@@ -185,6 +185,7 @@ At audit: 30 of 48 methods (62%). **Now 48 of 48 (100%).**
 | Secret store | `secrets`, and the resolver plugins call back through | ✅ |
 | State store | redb | ✅ |
 | Index store | RocksDB, behind a Cargo feature | ✅ |
+| WAL provider | redb, for durable transient Python sources | ✅ |
 | Identity | password and token | ✅ |
 
 **21 methods exist here that Node does not have**: the eight streaming
@@ -198,17 +199,17 @@ iterators (`query_results`, `query_events`, `source_events`,
 
 ## 4. Gap analysis vs the `drasi-lib` engine
 
-Comparing against `drasi-lib` 0.8.9 `src/lib_core_ops/`, the engine also offers,
-unexposed by *either* prototype:
+Comparing against `drasi-lib` 0.8.9 `src/lib_core_ops/`, the engine also offers
+the following surface. The WAL-provider row is now closed:
 
-| Engine capability | Notes |
-| --- | --- |
-| `get_query_config` / `get_query_info` | inspect a running query |
-| `get_source_info` / `get_reaction_info` | |
-| `snapshot_configuration` / `get_current_config` | round-trip config to disk |
-| `subscribe_all_component_events` | |
-| Query tuning: `with_middleware`, `with_recovery_policy`, `with_storage_backend`, `with_outbox_capacity`, `dispatch_mode` | `Query` builder options; only `joins` and `language` are exposed today |
-| WAL providers | |
+| Engine capability | Notes | Status |
+| --- | --- | --- |
+| `get_query_config` / `get_query_info` | inspect a running query | |
+| `get_source_info` / `get_reaction_info` | | |
+| `snapshot_configuration` / `get_current_config` | round-trip config to disk | |
+| `subscribe_all_component_events` | | |
+| Query tuning: `with_middleware`, `with_recovery_policy`, `with_storage_backend`, `with_outbox_capacity`, `dispatch_mode` | `Query` builder options; only `joins` and `language` are exposed today | |
+| WAL providers | redb provider plus per-Python-source durability opt-in | ✅ |
 
 These are lower priority than the Node parity gaps — they are engine surface
 neither binding has needed yet — but `snapshot_configuration` pairs naturally

python/drasi/_drasi.pyi

@@ -44,6 +44,7 @@ from .types import (
     SourceSchema,
     SourceSubscription,
     StateStore,
+    WalProvider,
 )
 
 # Directory arguments reach Rust as a PathBuf, which accepts anything
@@ -103,6 +104,7 @@ class Drasi:
         secrets: Mapping[str, str] | None = None,
         state_store: StateStore | None = None,
         index_store: IndexStore | None = None,
+        wal_provider: WalProvider | None = None,
         identity: Identity | None = None,
         plugins_dir: StrPath | None = None,
     ) -> Awaitable[Drasi]: ...
@@ -164,7 +166,9 @@ class Drasi:
     def wait_for_query(self, id: str, *, timeout: float = 30.0) -> Awaitable[None]: ...
 
     # components defined in Python
-    def add_python_source(self, id: str, *, auto_start: bool = True) -> Awaitable[None]: ...
+    def add_python_source(
+        self, id: str, *, auto_start: bool = True, durability: bool = False
+    ) -> Awaitable[None]: ...
     def push_change(self, source_id: str, change: SourceChange) -> Awaitable[None]: ...
     def add_python_reaction(
         self,

python/drasi/sync.py

@@ -71,6 +71,7 @@
     SourceSchema,
     SourceSubscription,
     StateStore,
+    WalProvider,
 )
 
 StrPath = str | PathLike[str]
@@ -168,6 +169,7 @@ def create(
         secrets: Mapping[str, str] | None = None,
         state_store: StateStore | None = None,
         index_store: IndexStore | None = None,
+        wal_provider: WalProvider | None = None,
         identity: Identity | None = None,
         plugins_dir: StrPath | None = None,
     ) -> Drasi:
@@ -178,6 +180,7 @@ def create(
                     secrets=secrets,
                     state_store=state_store,
                     index_store=index_store,
+                    wal_provider=wal_provider,
                     identity=identity,
                     plugins_dir=plugins_dir,
                 )
@@ -291,8 +294,12 @@ def list_queries(self) -> list[tuple[str, str]]:
 
     # ------------------------------------------- components defined in Python
 
-    def add_python_source(self, id: str, *, auto_start: bool = True) -> None:
-        _Loop.run(lambda: self._inner.add_python_source(id, auto_start=auto_start))
+    def add_python_source(
+        self, id: str, *, auto_start: bool = True, durability: bool = False
+    ) -> None:
+        _Loop.run(
+            lambda: self._inner.add_python_source(id, auto_start=auto_start, durability=durability)
+        )
 
     def push_change(self, source_id: str, change: SourceChange) -> None:
         _Loop.run(lambda: self._inner.push_change(source_id, change))

python/drasi/types.py

@@ -67,6 +67,7 @@
     "SourceSchema",
     "SourceSubscription",
     "StateStore",
+    "WalProvider",
 ]
 
 QueryLanguage = Literal["cypher", "gql"]
@@ -103,6 +104,16 @@ class IndexStore(TypedDict, total=False):
     direct_io: bool
 
 
+class _WalProviderRequired(TypedDict):
+    path: str
+
+
+class WalProvider(_WalProviderRequired, total=False):
+    """Directory containing one redb WAL file per durable transient source."""
+
+    kind: Literal["redb"]
+
+
 class Identity(TypedDict, total=False):
     """Credentials handed to plugins that ask for them.
 
@@ -446,6 +457,7 @@ class DrasiConfig(TypedDict, total=False):
     secrets: Mapping[str, str]
     state_store: StateStore
     index_store: IndexStore
+    wal_provider: WalProvider
     identity: Identity
     plugins_dir: str
     sources: Sequence[SourceConfig]

scripts/generate_api_reference.py

@@ -62,8 +62,14 @@
 itself and a `Stream` becomes an ordinary iterator — see [the blocking
 API](../guides/blocking-api/).
 
-The shapes referenced here (`SourceChange`, `QueryResultEvent`, `StateStore` and the
-rest) are declared in `drasi.types`, described in the [typing guide](../guides/typing/).
+The shapes referenced here (`SourceChange`, `QueryResultEvent`, `StateStore`,
+`WalProvider` and the rest) are declared in `drasi.types`, described in the
+[typing guide](../guides/typing/).
+
+`wal_provider={"path": ...}` configures a directory. The redb provider creates one
+WAL file per durable transient source inside it. This is Drasi's local
+transient-source write-ahead log; it is not a PostgreSQL replication slot and does
+not create, manage or resume one.
 
 {{% alert title="Generated from the shipped stubs" color="info" %}}
 The tables below are generated from `python/drasi/_drasi.pyi`, the same file your type