M1 consolidated: PRs #8, #9, #12, #15 with review#31
Open
jean-neiverth wants to merge 23 commits into
Open
Conversation
Adds a `[workspace.dependencies]` table to the root manifest
consolidating every dep used by 2+ crates across the full nullis-
shepherd stack (anyhow, thiserror, tokio, futures, serde, serde_json,
tracing, tracing-subscriber, strum, alloy-*, cowprotocol, reqwest,
wit-bindgen, clap). Per-crate manifests inherit with `dep.workspace
= true`, and may add features per call site via `dep = { workspace
= true, features = ["extra"] }`. Single-consumer deps (wasmtime,
toml, redb, getrandom, url, hex, axum, rand, ...) stay per-crate.
Adds `[workspace.lints]` with light-touch defaults: `dbg_macro` and
`todo` denied via clippy, `unsafe_op_in_unsafe_fn` warned via rust.
`unsafe_code = deny` cannot be applied workspace-wide because every
wit-bindgen guest module emits an `unsafe extern "C"` shim.
Also pre-declares `auto_impl` and `derive_more` in the workspace deps
table so future `Arc<dyn Trait>` boundaries and newtype-heavy crates
can opt in without touching the root manifest.
The version-drift failure mode (cowprotocol pinned to `1.0.0-alpha`
in nexum-engine but `1.0.0-alpha.3` in shepherd-sdk, flagged in the
2026-06-25 audit) is now impossible by construction: every consumer
inherits the single workspace pin.
Audit reference: milestone-rubric-grant-audit-2026-06-25.md, judgment
calls 1 + 3.
Replaces the `std::env::args().skip(1)` walker with a `#[derive(clap:: Parser)]` struct so the engine binary picks up `--help`, `--version`, proper argument validation, and structured error reporting for free. The positional surface is preserved one-for-one (`<wasm-path> [manifest-path]`); behaviour for callers that already pass two paths is identical. Help output now documents each argument inline rather than hiding the usage in an anyhow message that only fires on misuse. `clap.workspace = true` consumes the workspace dep added in the prior commit; no new direct version pin in this crate. Audit reference: milestone-rubric-grant-audit-2026-06-25.md, judgment call 2.
…irection A casual reader of `07-rpc-namespace-design.md` hitting the file top or the "Method Allowlisting" subsection could plausibly walk away believing the 0.2 runtime gates RPC methods on a read-only allowlist and intercepts signing methods to delegate them to the identity backend. The shipped host implementation does neither: `chain::request` forwards any method string through to the configured alloy provider. Adds an explicit `Status: Future direction (0.3+ target)` callout both at the file top and right above the "Method Allowlisting" subsection so the gap between design intent and shipped behaviour is visible without having to scroll the design narrative end-to-end. Audit reference: milestone-rubric-grant-audit-2026-06-25.md, judgment call 4.
Adds the dependencies the 0.2 host backends need: - cowprotocol (1.0.0-alpha) for the cow-api submission path (OrderBookApi, OrderCreation, OrderUid, Chain). - alloy-provider / -rpc-client / -transport-ws / -primitives (1.5) for the chain JSON-RPC dispatch. The reqwest feature on alloy-provider engages connect_http; the pubsub/ws features back eth_subscribe-class methods. - redb (2) for local-store. Same crate cowprotocol's own watch-tower picked, so the dep tree does not bifurcate when both are used in the same workspace. - reqwest (0.12, rustls-tls) — direct, so the import survives any future cowprotocol feature rearrangement. - tracing + tracing-subscriber (env-filter + fmt) — replaces the 0.1 eprintln! debug log so the engine can drop into a structured log pipeline without re-instrumenting every host call. - thiserror (2) — typed error enums in each backend. - tempfile + wiremock as dev-deps for the host backend tests. Adds engine.example.toml documenting the [engine] state_dir + per- chain RPC URLs the chain backend reads at boot; data/ is now ignored so a local run does not leave the redb file in tree.
Replaces the 0.2 Unsupported stubs with working backends. Each
capability lives in its own host submodule so the trait impls in
main.rs stay thin (dispatch + project the backend's typed error
onto HostError).
cow_api::submit_order
- Parses the guest's bytes as JSON cowprotocol::OrderCreation.
- Dispatches via cowprotocol::OrderBookApi::post_order.
- Returns the assigned OrderUid as a 0x-prefixed hex string.
cow_api::request
- REST passthrough. The base URL is whichever URL the pool's
OrderBookApi client carries — so OrderBookApi::new_with_base_url
overrides (staging, wiremock) flow through transparently.
- Method/path validated host-side; orderbook 4xx/5xx bodies are
surfaced verbatim so the guest can decode {errorType,description}.
chain::request
- Raw JSON-RPC dispatch over an alloy DynProvider opened from
engine.toml at boot. WebSocket URLs engage pubsub (eth_subscribe);
HTTP URLs use the HTTP transport. Params are passed as
serde_json::RawValue so alloy does not re-encode.
- request-batch falls back to per-call dispatch (same shape as the
earlier stub but now backed by real RPC).
local_store
- redb file under engine_config.engine.state_dir.
- Single shared table. Per-module namespacing is enforced
host-side via [len:u8][module_name][raw_key] prefix on every
key. list_keys strips the prefix before returning to the guest.
logging
- Routes through tracing::event! tagged with module=<namespace>.
- Engine boot installs an EnvFilter-based subscriber; RUST_LOG
overrides the engine.toml log_level.
identity / remote-store / messaging / http stay at Unsupported per
the 0.2 roadmap (keystore / Swarm / Waku land in 0.3).
Tests (14, all green):
- cow_orderbook: pool default chains, unknown-chain typing, REST
GET passthrough, relative-path resolution, unknown-method
rejection, submit_order round-trip — last three under wiremock
so the full HTTP path is exercised without hitting api.cow.fi.
- provider_pool: empty pool surfaces UnknownChain.
- local_store: roundtrip, namespace isolation, delete, list_keys
prefix-stripping, empty-namespace rejection.
End-to-end against modules/example: example.wasm loads under the
new wiring, logs init + on_event through the tracing pipeline.
… death (BLEU-813-817)
…er-pool, supervisor (BLEU-821)
…interfaces (BLEU-819)
…ed_crate_dependencies, drop redundant map_err)
PR #9 specific: - main: warn + return when block/log streams end (WebSocket dropped) - supervisor: simplify dispatch_block by extracting chain_id before move - supervisor: temp_local_store returns (TempDir, LocalStore) instead of leaking - README: correct engine.toml chain syntax to [chains.<id>] with rpc_url Rebased from PR #8: - local_store_redb: table.range() instead of iter() for O(matching) keys - provider_pool: dedupe method clone on the success path - main: hex_encode writes into the pre-allocated buffer - cow_orderbook: drop blank line nit - manifest: collapse nested if and use ? operator (clippy) - alloy_rpc_client / alloy_transport(_ws) imports as _ to satisfy unused_crate_dependencies.
Move the manifest.rs monolith into a directory module with four focused submodules (types, load, capabilities, error). Includes the Subscription enum and the four PR #9 tests for subscription parsing. Behaviour unchanged - pure code motion.
main.rs went from 739 lines of mixed bootstrap + 8 Host trait impls +
CLI parser + event loop to ~125 lines of pure orchestration. New
layout:
- bindings.rs: wasmtime::component::bindgen!() moved out so other
modules can name the generated types.
- cli.rs: Cli struct + manual parser.
- host/state.rs: HostState + WasiView impl.
- host/error.rs: unimplemented / internal_error / hex_encode helpers.
- host/impls/{chain,cow_api,identity,local_store,remote_store,messaging,
logging,clock,random,http,types}.rs: one Host trait impl per file.
- runtime/limits.rs: DEFAULT_FUEL_PER_EVENT + DEFAULT_MEMORY_LIMIT.
- runtime/event_loop.rs: open_block_streams, open_log_streams, run,
wait_for_shutdown_signal, TaggedBlockStream, TaggedLogStream.
Adding a new capability is now a single new file under host/impls/
rather than a 60-80 line diff in main.rs.
local_store_redb.rs was 89% tests, cow_orderbook.rs was 60%, and supervisor.rs was 32% (205 lines absolute). Promote each to a directory module with the test suite living in a sibling tests.rs so impl-side diffs stop competing with test churn for attention.
Port the five missing review refinements from nullislabs PRs #8 and #9 (mfw78 feedback, Jun 15-19) that had not been cherry-picked into this branch: 1. ModuleStore — cached per-module keccak256 prefix. Hashing happens once in LocalStore::module(name); every subsequent get/set/delete/ list_keys concatenates without rehashing. HostState now carries ModuleStore directly instead of (LocalStore, module_namespace). 2. Configurable [limits] in engine.toml — ModuleLimits with optional fuel_per_event and memory_bytes fields that resolve against built-in defaults (1B fuel, 64 MiB). Replaces the hardcoded constants in runtime/limits.rs. 3. alloy bump 1.5 → 1.8 for provider/transport/rpc crates; alloy-primitives stays at 1.6 (its own release cadence). 4. hex_encode → alloy_primitives::hex::encode. Removes the hand-rolled write!-loop hex encoder now that alloy is in the dep graph. 5. ProviderPool::empty() strict #[cfg(test)] gate (was cfg_attr allow(dead_code)). 6. manifest/error.rs converted to thiserror::Error derives with #[from] on Io/Toml variants. 7. extract_host replaced with url::Url::parse + host_str(), inheriting RFC 3986 handling of user-info, port, IDNA, IPv6 brackets. Validated: cargo fmt --check clean, clippy -D warnings clean, cargo test -p nexum-engine 42/42 passed.
jean-neiverth
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch consolidates the features from PRs #8, #9, #12, and #15 into a single branch, with all late-cycle review feedback (mfw78, lgahdl) applied on top.
Included PRs
Review feedback applied
The following refinements from mfw78's review (Jun 15–19) are included, which were pending on the individual PR branches:
ModuleStore— cached per-module keccak256 prefix. Hashing happens once inLocalStore::module(name); every subsequentget/set/delete/list_keysconcatenates without rehashing.HostStatecarriesModuleStoredirectly instead of(LocalStore, module_namespace).Configurable
[limits]inengine.toml—ModuleLimitswith optionalfuel_per_eventandmemory_bytesfields that resolve against built-in defaults (1B fuel, 64 MiB memory). Replaces hardcoded constants.alloy bump 1.5 → 1.8 for provider/transport/rpc crates;
alloy-primitivesstays at 1.6 (its own release cadence).hex_encode→alloy_primitives::hex::encode— removes the hand-rolled write-loop hex encoder.ProviderPool::empty()strict#[cfg(test)]— wascfg_attr(not(test), allow(dead_code)).manifest/error.rsthiserror — converted to#[derive(thiserror::Error)]with#[from]onIo/Tomlvariants.extract_host→url::Url::parse— replaces hand-rolled URL parser withurl::Url::parse+host_str(), inheriting RFC 3986 handling.Additional commits (not in the original PRs)
chore(workspace): hoist [workspace.dependencies] + [workspace.lints]— deduplicate dependency versions across workspace cratesfeat(nexum-engine): migrate CLI from hand-rolled parser to clap— applied before mfw78's review requested the samedocs(07-rpc-namespace-design): mark allowlist enforcement as future directionValidation