Repository navigation
Release gate: v0.123-dev → v0.123 (waits for cross-repo E2E) - #1670
Open
MuncleUscles wants to merge 67 commits into
Open
MuncleUscles wants to merge 67 commits into
MuncleUscles wants to merge 67 commits into
Conversation
* fix: surface stuck-head alert context * fix: avoid finalization claim deadlock * ci: narrow release workflow secrets * ci: keep codecov upload non-blocking * test: cover recovery storm health summary * test: cover no-progress health scan * test: cover no-progress health fallbacks * test: cover detailed consensus health endpoint
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…1669) * fix(fees): user-facing money correctness + canonical RPC alignment Money correctness (designer rulings — all user-visible): - appeal bonds now settle: bond × 3/2 credited to the appellant on successful appeal, explicit forfeiture on failure, 1:1 return on cancel; payouts credited to the recorded appellant (not the tx sender), idempotent across re-settles. Previously every bond was silently destroyed. - zero-value submitAppeal no longer bypasses min-bond validation (gasless mode unaffected) - unused pre-funded rotations refunded: rounds bill min(funded, actual LEADER_ROTATION events) + 1 leader slots instead of always rotations[i]+1 - LEADER_TIMEOUT rounds charge the leader allocation at 50% (extensible per-outcome multiplier table) - round indexing no longer counts LEADER_ROTATION events as consensus rounds (shared helper backend/consensus/history.py) — fixes settlement ladders and appeal-bond quotes when appealRounds > 0 Canonical RPC alignment (cross-backend receipt schema, superset rule): - gen_getTransactionStatus returns {status, statusCode} with canonical v0.6 ITransactions ordinals (0-14, incl. VALIDATORS_TIMEOUT=12, LEADER_TIMEOUT=13) - transaction payloads expose top-level txExecutionResult(Name) and a canonical fees object (deposit/userValue/distribution/locked/consumed as decimal strings; null when gasless) CI: GenVM precompile cache key now uses the actual GENVM_TAG (env or .env), failing loud when absent — the Dockerfile grep had been yielding an empty key since GENVM_TAG lost its default. * fix: keep transaction value an int; soften GenVM cache-key guard; black - top-level 'value' returns to plain int per the numeric-columns contract (tests/db-sqlalchemy/test_numeric_types.py) — the blanket _json_safe_numbers stringification from the fee-accounting merge broke it for values > 2^53; the breakage predates this PR but was masked by CI path filters until now. Big-int-safe consumers read the canonical decimal-string fees object instead. - GENVM_TAG cache-key step: warn + 'untagged' sentinel instead of hard-fail — the load-test and integration jobs run prebuilt images (--no-build) and legitimately have no tag; hard-failing broke them while the warning keeps the degradation visible. - black: 2 files reformatted (pre-commit gate). * style: restore calldata.py to pinned black 24.4.2 formatting Local black 25.1.0 had reformatted it differently from CI's pre-commit pin; the file was never part of this PR's functional changes.
* fix(ci): default GENVM_TAG for standalone PR builds docker-compose hard-requires GENVM_TAG since 963f935, but no PR workflow sets it and .env.example ships it empty, so any run that reaches the buildx bake step fails with 'GENVM_TAG is required' (first surfaced on #1672). Default to the GENVM_TAG repo variable, falling back to the current GenVM release, exported both to .env and the job env so bake and compose interpolation agree. * fix(ci): pin default GENVM_TAG to v0.3.0-rc3 (v0.6 fee host protocol) The Load Tests state-integrity step timed out with the Counter deploy stuck in PROPOSING (run 27292725606). Stack logs show every GenVM run failing instantly: genvm manager /genvm/run failed: 500 {'error': 'unknown variant `execution_budget_per_round`, expected one of: Internal, External'} The studio v0.6 fee code sends message_fee_allocation nodes in the GenVM v0.3.0-rc3 schema (MessageFeeAllocationNode: message_type/ parent_index + flat MessageFeeParams). GenVM v0.3.0-rc4 replaced that with the tagged Internal/External MessageAllocationNodeParams enum, so the rc4 manager rejects the payload, the leader burns its full 600s slot, and deploys end in LEADER_TIMEOUT (or sit in PROPOSING until the test gives up). Default the CI GenVM tag to v0.3.0-rc3 until the host protocol is migrated to the rc4 schema. vars.GENVM_TAG still takes precedence.
…ocol adaptation (#1674) * fix(ci): default GENVM_TAG for standalone PR builds docker-compose hard-requires GENVM_TAG since 963f935, but no PR workflow sets it and .env.example ships it empty, so any run that reaches the buildx bake step fails with 'GENVM_TAG is required' (first surfaced on #1672). Default to the GENVM_TAG repo variable, falling back to the current GenVM release, exported both to .env and the job env so bake and compose interpolation agree. * fix(ci): pin default GENVM_TAG to v0.3.0-rc3 (v0.6 fee host protocol) The Load Tests state-integrity step timed out with the Counter deploy stuck in PROPOSING (run 27292725606). Stack logs show every GenVM run failing instantly: genvm manager /genvm/run failed: 500 {'error': 'unknown variant `execution_budget_per_round`, expected one of: Internal, External'} The studio v0.6 fee code sends message_fee_allocation nodes in the GenVM v0.3.0-rc3 schema (MessageFeeAllocationNode: message_type/ parent_index + flat MessageFeeParams). GenVM v0.3.0-rc4 replaced that with the tagged Internal/External MessageAllocationNodeParams enum, so the rc4 manager rejects the payload, the leader burns its full 600s slot, and deploys end in LEADER_TIMEOUT (or sit in PROPOSING until the test gives up). Default the CI GenVM tag to v0.3.0-rc3 until the host protocol is migrated to the rc4 schema. vars.GENVM_TAG still takes precedence. * feat(docker): add GENVM_REF source-build path for GenVM images GENVM_TAG (release tarball) and GENVM_REF (git ref/SHA, built from source) are now mutually exclusive build args; exactly one must be set. The source path adds a nixos/nix builder stage that replicates genvm's own release recipe (.github/workflows/release.yaml + support/ci/pipelines/build.sh): patch flake-config.json with the resolved SHA, prefetch pinned deps, nix build manager-<arch>-linux / executor-<arch>-linux / runners-all, and merge the outputs into the same /genvm layout the tarballs produce. Prebuilt runner tars are preloaded from genvm's public GCS cache (fixed-output derivations) so runners-all rarely compiles anything. The tarball path is unchanged; post-install runs with --default-download false only for source builds (no release to download the executor from). * feat(ci): support vars.GENVM_REF — resolve to SHA, scope buildx cache by it When the GENVM_REF repo variable is set (e.g. 'main'), the workflows resolve it to a commit SHA via the GitHub API, pass GENVM_REF=<sha> as the build arg instead of GENVM_TAG, key the genvm-precompile cache on ref-<sha>, and move the buildx gha cache into a scope=genvm-<sha> namespace so image layers invalidate exactly when genvm moves. Behavior with the variable unset is unchanged (GENVM_TAG path). * feat(genvm): adapt host protocol to GenVM main (CON-549 fee schema) Studio spoke the rc3-era GenVM host protocol; genvm main (rc4+, #310) restructured the message-fee surface and the manager rejects our /genvm/run payload with 500 "unknown variant 'execution_budget_per_round', expected one of: Internal, External". Request (message_fee_allocation): flat parent-pointer nodes with message_type InternalAccepted/InternalFinalized/External and a 4-field fee_params dict become a nested tree ('children') with 'on' accepted|finalized and an externally tagged params enum — {'Internal': {7 fields incl. max_price_gen_per_time_unit, storage_fee_max_gas_price, receipt_fee_max_gas_price}} or {'External': {gas_limit, max_gas_price}}. The three price caps are sourced from Studio's fees_distribution (they are not part of the canonical per-node 5-field feeParams ABI). Non-root allocations are now sent as children of their roots so GenVM can emit correct subtrees (genvm executor/crates/common/src/domain/fees.rs, modules/implementation/src/manager/run.rs, tests/runner/origin/fees.py). Response (emissions): PostMessage/DeployContract/EthSend now carry fee_params as a struct and an ABI-encoded allocation 'subtree' (8-field internal params ABI). These are translated back at the GenVM boundary to Studio's canonical surfaces: 5-field internal / 2-field external feeParams bytes and the allocationSubtree node list. Studio's canonical fee ABI and fees.consumed reporting are unchanged. backend/node/genvm/origin/* is re-synced with genvm main's tests/runner/origin (new fees.py vendored module). * fix(fees): restore Mode-1 child bucket seeding; skip phase check for external allocations Two regressions from b88e059 ('enforce studio fee execution floor', June 4) that broke TestMessageFeeModes (added with the original v0.6 fee accounting, #1649): - create_child_fee_accounting zeroed the child's message-fee bucket when the message carried no allocation subtree, refunding the declared-budget remainder instead. Mode-1 children have no subtree by definition but must still receive the remainder as their own message bucket (FeeBudgetTransferred custody semantics). The leaf-refund expectation test from b88e059 is updated to the seeding semantics. - fill_message_fee_payload_from_allocation phase-checked (onAcceptance) external allocations; external messages have no accepted/finalized lifecycle (GenVM's matches_external ignores 'on' for the same reason). * fix(docker): disable nix syscall filtering in the genvm builder stage nix's seccomp BPF filter fails to load under qemu/Rosetta emulation ('unable to load seccomp BPF program: Invalid argument'), which blocks cross-platform local source builds (e.g. linux/amd64 on Apple Silicon). Harmless on native hosts — the nixos/nix image already runs with the build sandbox disabled. * feat(docker): GENVM_BUILD_CORES knob for the genvm source build Caps nix 'cores' for the genvm builder stage (default 0 = all). A full-parallel rust build can OOM memory-constrained or emulated builders (observed: cargo -j10 under Rosetta in a 16GB VM kills the nix daemon). * feat(ci): bind GenVM via checked-in third_party/genvm/version, env override Replaces the repo-variable mechanism: the GenVM binding is now a versioned file (same convention as genlayer-node), flipped to the release tag at release cut. Resolution order in CI: 1. GENVM_TAG / GENVM_REF already in the environment (e2e matrix / release train) wins; both set is a hard error. 2. Otherwise third_party/genvm/version: ^v[0-9] = release tag (tarball path), anything else = git ref (source build, resolved to a SHA for exact cache invalidation). Missing/empty file is a hard error rather than a guess. The file says 'main' on v0.123-dev, so PR CI builds genvm from source and validates the binding with no out-of-band setup. * fix(docker): derive genvm executor version from the bound ref's manifest genvm's patch-manifest requires the executor version to extend a version the committed manifest knows (base-match or previous-patch lookup); v0.0.0-dev<sha> matched nothing and failed the manager/executor builds ('Could not find any previous version for tag'). Derive the highest vX.Y.Z from modules/install/data/manifest.yaml of the ref being built and tag the source build vX.Y.Z-dev<sha> (verified against genvm's support/scripts/patch-and-check-manifest.py). * fix(docker): preserve exec bits when merging genvm source-build outputs cp --no-preserve=mode stripped the nix-store exec bits, so the base stage failed with '/genvm/bin/post-install.py: Permission denied' (exit 126) after an otherwise successful source build on CI. Preserve modes, then chmod -R u+w so post-install can write venvs/cache dirs. Also correct the builder comment: arm64 builds natively (the flake target follows TARGETARCH; runners are universal wasm from the GCS cache either way). * fix(docker): native arm64 genvm source builds + binding parse hardening - Work around genvm upstream bug: support/rust.nix patchelf's its rust toolchain with a hardcoded x86-64 ELF loader, breaking native aarch64-linux builds ('.cargo-wrapped: cannot execute'). On arm64 the builder rewrites it to ld-linux-aarch64.so.1, guarded to skip cleanly once fixed upstream and to fail loudly if the line multiplies. - nixos/nix ships no sed: install gnused alongside python3. - Harden the workflow binding parse: CRLF/whitespace trim + optional quoted values in third_party/genvm/version. Validated locally: full backend image builds natively on Apple Silicon (genvm stage 7m20s, GENVM_BUILD_CORES=6, no OOM); executor reports genvm v0.3.0-dev<sha>-aarch64-linux-release. (Implementation reviewed via codex.) * fix(docker): skip lief bin-patch for source-built genvm post-install's lief RPATH rewrite SIGABRTs on the zig/musl-linked amd64 executor built from source ('Can't read got entries' / 'Node not found. Can't resize the section'). Source-built outputs already carry $ORIGIN-relative rpaths from genvm's own nix patch-rpath, so the lief pass is redundant on this path; bin-check still verifies the binaries execute. Tarball path unchanged. * fix(genvm): retry transient 'modules not running' 500 from the manager Studio stops+restarts the genvm llm/web modules on every validator change; a /genvm/run landing in that window gets 500 'modules are required but not running' from the manager. The check exists in rc3 identically, but genvm main's module restart window is wide enough that mock-validator integration tests (which reconfigure validators right before each deploy) hit it near-deterministically — txs park in PROPOSING until timeout (run 27311560490). Route only this transient 500 (plus the sibling 'not all are running') into the existing GENVM_MANAGER_RUN_RETRIES retry loop via a dedicated GenVMManagerRetryableError; every other non-200 keeps failing fast (e.g. protocol errors must not be retried). Retry stats record the reason. (Implemented via codex; +2 parametrized regression tests.) * fix(genvm): match GenVM main's 32-byte U256 fuel framing in host protocol GenVM main widened fuel values to U256 in the host wire protocol; base_host.py still used rc3's 8-byte framing in two places: - REMAINING_FUEL_AS_GEN reply: we sent 8 bytes, the executor reads 32 (executor/src/host/mod.rs remaining_fuel_as_gen at v0.3.0-devef04f7f). The executor blocked on the missing 24 bytes while our host loop waited for the next method - a silent read-read deadlock. It sits in the exec_prompt path (wasi/genlayer_sdk.rs fetches remaining fuel right before the LLM module call), so every nondet execution parked in PROPOSING until LEADER_TIMEOUT while deterministic contracts were unaffected (runs 27311560490, 27314907925). - CONSUME_FUEL request: executor writes a 32-byte U256, we read 8, desyncing the stream whenever it is invoked. Locally wizard_of_coin went from a 10-minute hang to passing in ~11s. * fix(consensus): don't kill workers on transient NO_PROVIDER_FOR_PROMPT leader errors The llm module reports NO_PROVIDER_FOR_PROMPT as fatal, but in Studio it is transient/environmental: every validator change stops+restarts the llm module, so a leader run claimed inside that window (or on a stack without a real API key) sees a provider table lacking its node address. The fatal-leader circuit breaker stopped the worker in that case; with mock-validator tests reconfiguring validators before every test, all workers died one by one and the queue stalled silently (health: worker_task_died). The tx already has a bounded retry budget (recovery cycles escalate to CANCELED), so keep the worker alive and back off briefly (GENVM_TRANSIENT_FATAL_BACKOFF_S, default 3s) before resetting so retries don't burn through their cycles inside the same restart window. * test: modernize integration contracts to GenVM v0.3 SDK The stack now source-builds GenVM from main, where the floating runner aliases py-genlayer:test / :latest / py-genlayer-multi:test resolve to the genvm-main runner whose SDK no longer star-exports gl - every fixture still on an alias died at import with NameError: name 'gl' is not defined (and the upgrade suite cascaded into Contract-not-found). - Pin all alias fixtures to the v0.3.0-rc3 runner already used by every green contract (py-genlayer:1jb45aa8..., py-genlayer-multi:06zyvrl...): 8 icontracts fixtures, 5 inline upgrade-test fixtures, and multi_file_contract/runner.json. Bodies were already valid v0.3 SDK. - log_indexer: repin py-lib-genlayer-embeddings from the vanished 09h0i209... hash (invalid_contract) to 0bmbm3cy..., whose VecDB takes an explicit Distance type parameter - annotate with EuclideanDistanceSquared. - Allow overriding the integration RPC endpoint via TEST_JSONRPC_URL (request helper, conftest rate-limit probe, upgrade tests, utf8 roundtrip test). The previous hardcoded localhost:4000 made the suite untestable against a non-default stack; CI behavior is unchanged as the variable is unset there. Contract repins implemented via codex; verified against a local stack built from this branch (mock validators): 29/31 previously-failing tests pass; genvm_smoke needs a real LLM key (CI has one) and log_indexer is slow-but-valid locally (schema verified via RPC). * style: black-format worker.py backoff hunk * fix(genvm): discard storage changes reported by readonly executions GenVM main reports result_storage_changes for readonly (view) executions too - e.g. genlayer-embeddings VecDB lazily initializes its cover-tree storage on first access, which can happen inside a view knn call. provide_result applied those changes into the readonly state proxy and tripped 'assert not self.readonly' in storage_write, surfacing as '-32603 Internal error' on every gen_call to such a contract (test_log_indexer). View-execution storage changes are ephemeral VM-side effects: skip applying them. * test: upgrade integration contracts to genvm main SDK Supersedes the rc3 repin (098bbb8): instead of pinning fixtures backwards, every integration contract now uses GenVM main's SDK idiom, pinned to main's concrete runner hashes (no test/latest aliases): - py-genlayer:1zr6nqk597d97kg0dyxg0shhrykx5v02zjgnyrajapy4wlqvfvwh - py-genlayer-multi:0jw6688wgnbqghr2wwg2mqkf2ryb5if7w6bp5zcwbq6zlks5ybg4 - py-lib-genlayer-embeddings:0wcvi35grdr47ynkckzriz5sjn5080w2njk7v2cqx3xpp6p1989y Idiom changes (mirroring genvm's own tests, e.g. tests/cases/semi-stable/llm/woc.py): - import genlayer as gl + from genlayer import * (gl is no longer star-exported); class X(gl.contract.Contract) - gl.contract.get_at / gl.contract.deploy (were gl.get_contract_at / gl.deploy_contract); raise gl.vm.UserError (gl.Rollback is gone) - eq_principle.prompt_comparative principle arg is positional-only - @Allow from genlayer.storage (was @allow_storage) - exec_prompt(..., response_format='text') - embeddings VecDB takes an explicit Distance type param Backward-compat canary: examples/contracts/user_storage.py deliberately stays on the rc3 runner (py-genlayer:1jb45aa8..., old idiom) so test_user_storage keeps proving old-runner contracts execute on the genvm-main executor. (The executor parses the whole leading comment block as version+runner JSON, so the canary note sits after a blank line.) gltest 0.24.0 only discovers contracts with the legacy gl.Contract base; tests/integration/gltest_compat.py (activated in conftest) extends the AST matcher to accept both bases until the genlayer-test pin moves past 0.29.2 (upstream main already matches gl.contract.Contract). log_indexer specifics: the main embeddings runner pairs with the main py-genlayer (the old 09h0... hash no longer exists); similarity values recalibrated for its EuclideanDistanceSquared metric; update_log now locates entries by plain iteration because cover-tree knn from a write method currently trips GenVM main's deterministic-mode float trap (wasm_trap DeterministicMode) - views still exercise knn. Bulk rewrite implemented via codex; verified against a local stack built from this branch: previously-failing set + upgrade suite + canary all green locally except genvm_smoke (needs a real LLM key; CI has one). * style: appease autoflake on star-imports; skip genvm-lint for main-runner contracts - autoflake (pre-commit) removes 'from genlayer import *' where no star-exported name is used - accepted for the 7 contracts that only use the gl namespace. - Released genvm-linter (<=0.11.0) cannot load the genvm-main SDK layout ('No module named genlayer.py'); the lint workflow now skips contracts pinned to the main runner hash until a compatible linter ships (genvm-linter dxp-694), and keeps linting rc3-era contracts (canary, load/direct fixtures). * test(unit): cover host fuel-width framing, transient-fatal backoff, readonly storage discard Targeted unit tests for the Sonar new-code coverage gate (PR #1674): - test_host_loop_fuel_widths: drives host_loop over a socketpair and pins the GenVM main wire framing - CONSUME_FUEL reads a 32-byte LE U256 (value > 2**64) and REMAINING_FUEL_AS_GEN replies OK + exactly 32 LE bytes. Regression tests for the read-read deadlock fixed in dfdee12. - test_worker_transient_fatal: NO_PROVIDER_FOR_PROMPT fatal leader errors reset the tx, back off (3s default), and keep the worker alive; other fatal causes still stop the worker; validator-side errors are left to consensus. - test_provide_result_readonly: provide_result discards result storage changes for readonly (view) executions and applies them for writable ones (49db980). Also exclude examples/contracts/** from Sonar coverage: they are GenVM intelligent-contract fixtures executed inside the GenVM, never under the backend test runner (18 of the 38 'uncovered new lines' were contract fixture lines). Full tests/unit suite in-container: 903 passed, 1 skipped.
…d shape) (#1672) The canonical fees.consumed object now reports how many protocol time units the execution actually used, per round and in total: - leaderTimeunitsUsed / validatorTimeunitsUsed: decimal-string totals across all rounds (validator total includes the leader's validator-function execution) - perRound: [{round, consensusRound, leaderTimeunits, validatorTimeunits, maxValidatorTimeunits}] - maxValidatorTimeunits is the single largest validator execution that round, the number to compare against the per-validator validatorTimeunitsAllocation Unit mapping (documented at backend/consensus/history.py): 1 time unit == 1 second of GenVM wall-clock runtime (consensus spec glossary; Studio's DEFAULT_INITIAL_TIME_UNITS_ALLOCATION = 10*60). Consumption is derived from each receipt's measured processing_time (ms), rounded UP per execution - never raw seconds. Computation is a pure helper (time_unit_consumption) over the persisted consensus_history round entries: leader-rotation events fold into the next completed round, with a fallback to consensus_data when no history rounds exist. Backward compatible: new keys only; gasless transactions still return fees: null. This is the reference shape for the node to mirror 1:1; consumer: gltest --fee-profile (genlayer-testing-suite #94) fee-profile suggestions for leader/validator time-unit allocations.
…1677) ad77f6b changed the response to {status, statusCode} — a breaking change for deployed apps that parse the string (Rally polls it on every transaction). The string shape is a compatibility contract; the dict moves to a new additive method, gen_getTransactionStatusDetails. No in-tree consumer used the dict.
* test(ci): gasless-mode integration coverage Rally's hosted studio runs with all three price envs at zero (GENLAYER_STUDIO_GEN_PER_TIME_UNIT / STORAGE_UNIT_PRICE / RECEIPT_GAS_PRICE). Only unit tests covered that configuration; no integration/CI job ever ran a zero-price stack, so fee-path changes could silently break the hosted deployment. - tests/integration/test_gasless_mode.py (@pytest.mark.gasless): pins sim_getFeeConfig enabled=false + zero policy prices, fee-less deploy/write/read via the SDK, canonical receipt fees: null, the sim_estimateTransactionFees surface (the RPC genlayer-js estimateTransactionFees hits), and the tolerant behavior that explicit fee params on a gasless stack are accepted (required deposit is 0), not rejected. - pytest.ini: register the gasless marker and exclude it from the default sweep (same convention as error_handling). - backend_integration_tests_pr.yml: new test-gasless job reusing the bake/compose pipeline with the three price envs forced to 0, a readiness assert on sim_getFeeConfig, and a serial -m gasless run. Buildx cache is read-only in this job so it rides the main test job's cache without racing its writes. Verified locally against a zero-price compose stack (arm64, GENVM_REF source build): 2 passed. * test(ci): appease Sonar hotspots in the gasless job - SHA-pin docker/setup-buildx-action and docker/bake-action (S7637) - drop the explicit http:// scheme from the local readiness curl (S5332); curl defaults to http and the endpoint is runner-local only * refactor(fees): reduce canonical fee helper complexity * test(ci): stabilize gasless and oracle CI
Prevent Kubernetes liveness from restarting workers before the configured transaction timeout can own stuck transaction recovery. Validated by CI, load test, and focused worker health unit tests.
* test: reproduce serialized receipt metrics bug * fix: read serialized receipt execution results
Consolidate Studio backend and consensus-worker images onto a common GenVM runtime and service base so AWS E2E pulls each large runtime layer once.
genvm-linter 0.11.0 (latest release) does not bundle the py-genlayer:9b8kjyda2... runner that every v0.123-dev example contract now pins, so 'Lint Intelligent Contracts' fails with "filename 'runners/py-genlayer/9b/8kjy....tar' not found" on any PR touching contracts. Extend the existing unsupported-runner skip filter (same mechanism already used for the genvm-main runner) until a linter release bundles the new SDK (genvm-linter dxp-694).
…y, wiping contract state on successful validator appeal (#1724) * fix(consensus): return contract_snapshot and consensus_history from claim_next_appeal claim_next_appeal's raw UPDATE ... RETURNING omitted the transactions.contract_snapshot and transactions.consensus_history columns, so every appeal transaction was rebuilt via Transaction.from_dict with contract_snapshot=None and an empty consensus_history. On a successful validator appeal, process_validator_appeal then fell into its "no snapshot" branch and restored accepted_state = {} — for non-deploy write transactions this wiped the contract's accepted state including the code slot, after which the re-run failed with invalid_contract absent_runner_comment and subsequent appeals self-confirmed the error. The missing consensus_history additionally let appeal committees re-draw already-used validators, and appeal validators verified against post-execution state instead of the pre-execution snapshot. Two changes: - worker.py: add both columns to the RETURNING list and the returned appeal_data dict (same raw-JSONB shape as _parse_transaction_data). - base.py: defense in depth in process_validator_appeal — for non-deploy transactions with a missing snapshot, re-fetch it via transactions_processor.get_transaction_by_hash (which hydrates archived snapshots); if it is still unavailable, log an error and skip the state restore instead of writing {}. Deploy rollback behavior is unchanged. Adds unit tests covering the claim dict shape, the RETURNING clause, Transaction.from_dict hydration of a claimed appeal, and all four restore branches of the appeal rollback. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(consensus): generate claim queries from a shared column manifest Review follow-ups for this PR, plus the architectural fix for the bug class it exposed (hand-maintained RETURNING lists + silent Transaction.from_dict defaults = omissions indistinguishable from absent values): - worker.py: all three claim queries (transaction, finalization, appeal) now generate their RETURNING clause and row->dict conversion from one shared column manifest, so query and dict can never drift apart. Heavy state columns (contract_snapshot has been observed at tens of MB) are an explicit opt-in group: appeals include them, pending/finalization deliberately omit them. - base.py: PendingState now refreshes the transaction from the DB before computing pre-effects; previously decide_pending_pre received claim-built appeal flags that silently default to False for re-processed appeals (same drift class, latent since the params are currently unused). Renamed previous_contact_state -> previous_contract_state, and the skipped-restore path now emits a consensus_event ERROR so operators see it, not just the log. - tests/unit/consensus/test_claim_column_manifest.py: pins each claim's SQL to its manifest, pins the appeal claim to the state columns (incident regression), and statically extracts every key Transaction.from_dict consumes to enforce that each claim either returns the column or explicitly allowlists the omission with a rationale - a new from_dict field now fails tests until a conscious per-claim decision is made. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Edgars <edgars@genlayerlabs.com>
* feat: support v0.2 v0.3 interop * fix(genvm): accept a base64 consumed_result from the manager
Ports the per-contract daily snapshot storage quota and actual snapshot-size accounting correction to v0.123-dev.
…keys to v0.123 (#1743) * feat(rate-limit): meter cheap reads separately and expose usage headers (#1741) Rate limiting was method-agnostic: the middleware checked the quota before the JSON-RPC body was parsed, so `gen_getContractCode` — a single indexed read — cost exactly what a write fanning out to LLM validators cost. That forces the limits to be sized for the expensive case, which starves the cheap read traffic that batch tooling generates. Reads that never enter the GenVM now meter into their own bucket at RATE_LIMIT_READ_MULTIPLIER (default 10x) of the tier limits. The standard bucket keeps its key shape and numbers, so limits in flight at deploy time carry over rather than resetting. The allowlist is maintained by hand and deliberately conservative. Two methods that read like lookups are excluded because they build a Node backed by a GenVMManager (gen_getContractSchema, gen_getContractSchemaForCode), and eth_call is excluded for the same reason despite appearing in the DISABLE_INFO_LOGS_ENDPOINTS env list that otherwise looks like the natural source for this. Anything ambiguous — unparseable, oversized, or a batch with one expensive member — is charged to the stricter bucket. Responses now carry X-RateLimit-Limit/Remaining/Reset/Window/Bucket for the window closest to exhaustion, so clients can pace themselves instead of discovering the ceiling by hitting it. Reset is the time until the oldest entry ages out, which is the honest answer for a sliding window. These are listed in the CORS expose_headers, without which browsers hide them from JS. Finally, gen_getContractCode no longer loads the whole contract state to read one slot out of it. ContractSnapshot pulls the entire `data` JSONB — every storage slot the contract owns — which for a contract holding a large vector store is a substantial fetch and deserialize on a call that batch tooling polls hard. The slot is now extracted in SQL. Postgres still detoasts the JSONB server-side, so this narrows transfer and parse cost rather than eliminating the read; legacy and undeployed row shapes defer to the original path to keep their error semantics intact. The Lua script gains a test that actually executes it under a stubbed redis.call. It was previously covered only through mocked evalsha, so an error in the script body would have surfaced first in production, on every /api request. * feat(rate-limit): accept API keys in the URL path (#1742) Keys could only be supplied as an X-API-Key header, which makes Studio unusable from most of the EVM toolchain. MetaMask's "Add network" takes a URL and has no header field at all; the same is true of foundry.toml, --rpc-url, and most viem/ethers setups. Every major provider puts the key in the path for this reason (Alchemy /v2/<key>, Infura /v3/<key>). This is not hypothetical: an integrator on a shared Studio key told us they would have to stand up a backend proxy "for them to be able to use it properly as most of the use comes from testing deployments through metamask". That proxy's whole job would be turning a URL into a header, and a shared proxy is also what pushes teams onto one shared key instead of one key per person. POST /api/{api_key} now routes to the same JSON-RPC handler, with the header still accepted for server-side callers. The path wins when both are present, since it is what the caller typed rather than something an intermediary may have injected. Only `glk_`-prefixed segments count as keys: any other segment falls through to anonymous rather than erroring, while a mistyped real key is still treated as a key so it fails loudly as invalid instead of silently dropping to anonymous limits. The middleware's path gate now covers every path the route can match, not just key-shaped ones. A path served by the route but missed by the gate would be an unlimited, unauthenticated RPC endpoint, so a test pins that correspondence. Keys in URLs get written down in places headers are not, and two of those were ours: uvicorn access logging is on, and Sentry runs with send_default_pii and full trace sampling, so request URLs leave the process on every transaction, not only on errors. api_key_redaction scrubs keys from both — a logging filter covering uvicorn's args-based access line, and before_send/before_send_transaction hooks that walk the whole event rather than known fields. Without this, path keys would be a downgrade on header-only rather than an improvement. * feat(rate-limit): classify the v0.123-only RPC methods The ported allowlist predates three methods that exist only on this branch. sim_getFeeConfig and gen_getTransactionStatusDetails are cheap reads — a config lookup and a transactions_processor read respectively, neither touching the GenVM. sim_estimateTransactionFees is not. It takes validators_manager and genvm_manager and runs the contract through sim_call to measure fees, so it costs a full execution. Worth flagging because eth_estimateGas *is* allowlisted — that one returns a hardcoded constant. The names are nearly identical and the costs are not, which is the same shape of trap as eth_call and the two schema methods already documented here.
* fix(fees): align Studio with consensus v0.6 economics Mirror cap-priced round quotes, configured ladder semantics, appeal profit reserves, exact 2.5x rewards, typed appeal funding, immutable top-up schedules, live appeal overlay pricing, latest appeal ABIs, decision guards, and surplus refunds. Expose the canonical quote over RPC so cross-stack tests no longer maintain a second Studio formula. * test(fees): derive settlement deposit from quote * chore(genvm): replace dead source pin * test(fees): cover timeout rotation multiplier * fix(fees): align studio with consensus v0.6 * fix(hardhat): restore reproducible deployment profile * fix(consensus): match v0.6 round lifecycle * fix(consensus): bind appeals to exact decisions * feat(rpc): expose v0.6 decision lifecycle * fix(fees): complete v0.6 runtime parity * fix(consensus): pin appeal selection authority * fix(consensus): make appeal admission failure value-safe * fix(consensus): preserve paid appeals across worker failures * fix(consensus): make internal message effects replay-safe * fix(consensus): reject unregistered internal recipients * fix(consensus): recover message effects atomically * fix(rpc): keep pre-fee consensus entrypoints decodable * fix(rpc): serve a name-unique consensus ABI * fix(consensus): treat a rollup-free deployment as a valid emission path * chore(genvm): repin to the merged leader-data commit on v0.6-dev The pin inherited from v0.123-dev references the pre-merge feature branch feat/leader-data-to-genvm, whose branch was deleted after squash-merge; the commit is unreachable from any ref, so the source build's clone cannot read it (fatal: unable to read tree). The same work is v0.6-dev head 3c96be6de (feat(manager): carry opaque leader public data, #29), which also contains the previous pin d07f21da. v0.123-dev itself needs the same repin. * fix(consensus): carry the parent rotation schedule onto claims * fix(consensus): bind message effects to protocol recipients * fix(consensus): align child admission with v0.6 * fix(consensus): complete v0.6 studio parity * fix(consensus): report exact creation slots * fix(fees): close remaining consensus handoff gaps Pin the complete activation policy and funding overlay, preserve pool-aware contribution and external remainder ownership, reject invalid allocation and contribution states, and refund activated child value to its actual funder. Honor exact initial committees and fee-plan-sized leader replays, including receipt-less Undetermined finalization when capacity is unavailable. Add the shared simulator quote oracle plus focused lifecycle, settlement, and PostgreSQL atomicity coverage. * fix(rpc): decode submissions without a rollup connection * fix(tooling): qualify the v0.6 fee train Align Studio lifecycle/status RPCs with the node v0.6 consumer interface, pin the exact fee-aware SDK and test-harness revisions, and make integration/load callers submit the canonical SDK fee estimate. Preserve legacy status-string compatibility for existing Studio consumers. * fix(consensus): close latest v0.6 handoff gaps * fix(consensus): unwind descendant appeals on rewind * fix(ghost): scope bundled factory salts by sender * test(ghost): cover parent-scoped salt reuse * fix(fees): derive exact transaction message allocations * test(integration): submit canonical fee envelopes * fix(studio): close v0.123 correctness gaps * fix(rpc): hide unexpected server details * fix(studio): let fee estimation measure full execution * fix(fees): align Studio with GenVM fee buckets Treat GenVM v0.3 bucket 0 as the shared execution reservoir, bucket 1 as message fees, and buckets 2/3 as byte counters. Reconstruct the exact GenVM receipt meter to derive Consensus storage/event-write fees, and size recommendations for both chargeable execution and the enforced shared reservoir. * fix(fees): use canonical allocation wildcard * chore(deps): pin corrected fee SDK heads * chore(deps): repin restacked fee SDK heads * fix(fees): match internal deploy allocations * fix(consensus): bind salted children to GenVM addresses * fix(fees): fund discovered child execution * fix(rpc): expose mined envelope reverts * chore(deps): repin receipt-aware JS SDK * chore(deps): repin topup-safe JS SDK * fix(fees): align envelope and shared allocations * chore(deps): repin message event SDK * fix(validators): replace pools atomically * test(fees): lock fresh appeal capacity Cover the two topology boundaries behind validator appeals: a five-validator pool is exhausted by the initial committee, while twelve identities provide the full fresh seven-seat first jury and preserve the exact Consensus quote. * fix(release): harden studio parity boundaries * test(ci): isolate fee default regression * test(ci): fund gltest default submissions * test(ci): fund gltest write defaults gltest owns separate fee argument adapters for deploys and writes. Bridge both so their public None defaults are treated as omitted by the fee-aware integration fixture, while explicit direct SDK fees=None stays gasless. Add a dependency-free regression covering both entry points. --------- Co-authored-by: kirilaa <kirilantevski@gmail.com>
Add a preview-only Studio RC release path, protect Docker latest, and display the deployed version.
Release image builds do not pass GenVM overrides. Define those optional arguments as empty in both Docker stages so nounset shells can select the checked-in GenVM pin, and cover the no-build-args workflow path with a fast parser regression.
Use the immutable GenVM v0.6.0-rc3 release for distributed backend image builds. Add a fast policy regression so a source branch/SHA cannot reach the release workflow without its required Nix closure.
* fix: normalize faucet funding quantities * test: use active chain id for ghost address
* fix(ci): repin genlayer-py to a compatible version
3 of 4 tasks
Route legacy v0.2 contracts to the retained executor and preserve executor-specific code layouts through schema, deploy simulations, finalized execution, reads, and upgrades. Validated combined candidate tree 7150e21 over v0.123-dev@0737a52de4be3db0ea33336d99f68b5e3e14ce40. Full E2E 35199189091: 174 passed, 107/107 features covered, scope=all and stack=all. Combined backend unit suite: 1,631 passed. Single-PR landing under the owner's explicit admin-merge authorization; no cross-repository PR dependencies. Preview release only.
Price the proposed receipt wrapper plus the 64-byte terminal reserve when quoting and validating the minimum execution budget. Keep the legacy 512-byte receipt estimator separate. Cover the canonical v0.6 quote, alternate wrapper sizes, exact admission boundary, and locked top-up formula. Validated with 1,635 Studio unit tests and repository pre-commit hooks.
* fix(genvm): forward flat fee allocation inputs 🐛 Re-vendor the fee allocation types and replace recursive children with checked direct-child budgets and host-encoded FlatArrays subtrees. Preserve committed fee bytes, ordering, roots and original allowances. Pin GenVM v0.6.0-rc7, the release cut from the fee-allocation branch. * test(genvm): pin the reroute test to the rc9 executor line ✅ GenVM v0.6.0-rc7 ships executor v0.3.0-rc9; the missing v0.3.0-rc7 directory left the pinned deploy stuck in PROPOSING
* chore(genvm): pin descendant-grants genvm and refresh runner hashes ⬆️ * feat(fees): emulate descendant grants of balance-funded messages ✨🐛 Studio also read the emission budget from `declaredBudget`, a key the manager never sends (`message_fee`), so every balance-funded child was declared with budget 0. * chore(genvm): pin genvm v0.6.0-rc9 and refresh runner hashes ⬆️ * fix(fees): preserve descendant funding and grant restrictions Inherit the parent fee depositor and admitted grant rules when materializing descendants. Preserve metering for zero-budget grant descendants and wait through all triggered transaction levels in the integration test. Validation: 1669 unit tests passed, 7 skipped; Black and diff checks passed. Live integration was not rerun. * refactor(fees): treat omitted descendant grants as closed Remove the legacy grant flag and its permissive fee validation and rotation rules. Omitted grants now use the same validation and accounting as explicit Closed grants, with surplus retained in primary escrow. Validation: 1674 unit tests passed, 7 skipped; Black and diff checks passed. Independent review found no actionable issues. * test(fees): run the descendant wait test with the integration suite It imports the integration module, which needs gltest; the unit job does not install it Claude-Session: https://claude.ai/code/session_01N97Hvs14ckrioBoYJTD4Te * fix(docker): default the nix netrc secret to an empty file in the repo buildx bake refuses to read /dev/null without --allow=fs.read=/dev/null, which compose cannot pass, so builds with buildx 0.38 failed Claude-Session: https://claude.ai/code/session_01N97Hvs14ckrioBoYJTD4Te
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Standing release-gate PR:
v0.123-dev→v0.123This is a long-lived gate PR, not a normal feature PR. Do not merge ad-hoc.
Purpose
v0.123-devis the active integration branch for the v0.6 fee work. Individual PRs merge intov0.123-devand only need to pass normal repo CI — they are not blocked on the full cross-repo release train being green.v0.123-devis promoted tov0.123. It waits for cross-repo E2E (the release train) to go green before it is merged.v0.123-dev.How to use
v0.123".(Mirrors genlayerlabs/genlayer-consensus#1070.)