Skip to content

fix(fees): align Studio with Consensus v0.6 economics - #1748

Merged
MuncleUscles merged 56 commits into
v0.123-devfrom
fix/v0.123-fee-consensus-parity
Sep 2, 2026
Merged

MuncleUscles merged 56 commits into
v0.123-devfrom
fix/v0.123-fee-consensus-parity

Conversation

@MuncleUscles

@MuncleUscles MuncleUscles commented Aug 27, 2026 •

Copy link
Copy Markdown
Member

Delivery context

Depends-On: genlayerlabs/genlayer-consensus#1526
Depends-On: genlayerlabs/genlayer-js#214
Depends-On: genlayerlabs/genlayer-py#110

Cross-stack qualification: genlayerlabs/genlayer-e2e#756; harness #748 is merged.
Node destination interface: genlayerlabs/genlayer-node#1800; E2E timeout qualification: genlayerlabs/genlayer-node#1818.
Independent economics oracle: genlayerlabs/genlayer-fee-distribution-simulator#30.
Consensus oracle enforcement: genlayerlabs/genlayer-consensus#1388.
Formal lifecycle and selection reference: genlayerlabs/od-consensus-tla#39.

Consensus is authoritative for deployed runtime behavior. The simulator independently checks economics; TLA+ and the written protocol specification supply lifecycle, input-domain, appeal-window, and appeal-selection invariants.

Problem and outcome

Studio had formula, lifecycle, authority, submission, and failure-recovery differences from the Consensus v0.6 train. This PR aligns implemented Studio behavior—not only displayed quotes—with current Consensus economics and closes applicable defects from the Consensus handoff campaign.

The result covers submission, activation, top-up, appeal, execution, settlement, cancellation, expiry, exact committee and rotation ladders, timeout and decision-bound lifecycle, message allocation and replay, contribution ownership and refunds, storage and receipt pricing, funding overlays, and failure recovery.

The leader-timeout appeal bond deliberately retains the configured source-round committee cost. For five validators it is 100 + 5 × 200 = 1,100; removing the timed-out leader changes replay membership, not the independent source-round cost used by the bond quote.

Important implementation details

  • Activation freezes the complete economic policy and selection authority used by the transaction.
  • Validator-pool replacement is one database transaction, one event, and one GenVM reload. Registry fingerprints suppress duplicate delayed events and force a repair before selection when the cache is partially stale, not only when it is empty.
  • Quotes and actual selection share the same frozen, live, and excluded validator set; consumed identities do not re-enter later appeals.
  • Exact committee requests do not silently shrink. Capacity failure materializes the same receipt-less Undetermined outcome as Consensus.
  • Deposit quotes use the submitted max price, combined-overlay gross-up, absolute and saturated round ladder, every configured rotation, and every successful-appellant reserve slot.
  • Fee mutations lock and re-read state. Top-ups, reservations, cancellation, settlement, refunds, and message effects are replay-safe and single-spend.
  • Appeal and finalization calls require the active decision ID. Studio exposes canonical gen_getTransactionLifecycle and gen_estimateLatestAppealCharge RPCs.
  • Signed Studio EVM envelopes now retain standard mined status: 0 failure semantics and expose the local revert reason as additive receipt metadata. Envelope-only lifecycle calls are also returned by eth_getTransactionByHash, which lets standard viem receipt polling reach the durable receipt. The JS and Python train heads wait for these receipts, so rejected writes and top-ups cannot masquerade as successful submissions.
  • gen_getTransactionStatus supports the Node v0.6 object request and response while preserving the legacy string response for existing Studio consumers.
  • A transaction-specific two-pass estimate discovers exact root internal-message recipient and call-key allocations, then reruns on a fresh snapshot with those budgets metered.
  • Internal deployments canonicalize GenVM's abbreviated 0x recipient to address(0) before allocation matching, so discovered deploy budgets remain usable during reveal.
  • Salted internal deployments are materialized at the exact address GenVM returned to the parent. This closes the Studio symptoms of GAP-016 while preserving the current runner-visible identity; the EVM GhostFactory/GenVM formula mismatch remains an explicit upstream protocol-integration item, not a false cross-runtime parity claim.
  • Fee-estimation measurement leaves exact message budgets enforced but temporarily unmeters only the parent GenVM execution bucket, allowing storage/receipt consumption to complete and become part of the returned recommendation instead of failing at the startup floor. Duplicate children are grouped under one cumulative allocation, while each occurrence receives only its canonical minimum plus child subtree instead of the full aggregate ceiling. useBalance children are excluded from the sender-funded allocation tree while remaining subject to the global cap.
  • The pinned GenVM v0.3 executor uses bucket 0 as one enforced execution reservoir, bucket 1 for message fees, and buckets 2/3 as nondeterministic-output and submitted-message byte counters. Studio now interprets that layout exactly, reconstructs the executor's receipt-meter contribution from its fee config and byte counters, hands only the remaining storage/event-write fee to Consensus, and sizes recommendations to satisfy both Consensus chargeable execution and the stricter shared GenVM reservoir.
  • Frontend estimation and submission share the JS SDK's exact ABI serialization, including keyword arguments.
  • Studio, the SDKs, and E2E now reserve keccak256("") as the allocation wildcard while retaining bytes32(0) for deploys and unnamed calls.
  • Malformed or infrastructure-failed estimates block submission; only the canonical -32000 execution failed path receives the intentional default-fee fallback.
  • Integration and load callers use the canonical transaction-specific SDK fee estimate rather than bypassing fee-enabled admission.

Consensus handoff audit

All 151 unique non-governance findings and controls in the consolidated Consensus handoff #1492 were inspected, including the cross-repository boundary controls from #1527 and #1528. Applicable fee, lifecycle, queue, appeal, message, and identity behavior is implemented and regression-covered. Solidity-only governance, staking, proxy-upgrade, EVM gas-bound pagination, and fixed-width arithmetic findings were classified as non-applicable to Studio rather than copied mechanically.

The Studio release-readiness audit additionally absorbed the applicable fixes from #1735, #1736, #1737, and #1746: zero-stake selection now raises the consensus-domain error, RNG construction is per call, native balance transfers use atomic SQL mutations, and method-call decoding handles complete RLP headers plus both leader-only values. Shared VRF constants and errors were separated to remove the circular import present in the source proposals. The valid error-disclosure half of #1739 is also included: unexpected exceptions are logged server-side and return only a stable generic JSON-RPC error. Wildcard CORS without credentials remains intentional for a public RPC used by arbitrary browser dapps; the proposed mandatory allowlist would break that supported surface.

The final audit added three forward-correctness fixes:

Those three Consensus PRs are expected-red handoffs and are not production fixes in #1526. Studio therefore implements the specified optimal behavior ahead of the currently deployed Solidity path for those cases; this is explicit forward correctness, not an accidental parity claim.

The release-review hardening pass also closes every verified behavioral finding: finalization repair obeys retry backoff; stale terminal leases cannot strand later work; zero-value appeal aborts persist rollback; nullable legacy nonces hash as zero; raw encoded allocation subtrees remain hashable but are not traversed; quote and admission share one appeal-eligibility computation; legacy envelope migration prefers dated rows; explicit integration fees=None remains gasless; frontend and backend failed-appeal defaults agree; malformed Solidity tuple offsets return the protocol error; and default Docker builds honor the pinned GenVM source ref.

Validation

  • Fresh appeal-capacity boundary regression: a five-validator pool is correctly exhausted by the initial committee; a 12-validator pool supplies the complete fresh seven-seat first jury and preserves the exact quote. Full test_studio_fees.py: 467 passing; pre-commit passes.

  • Backend unit suite: 1,553 passing; the 12 Unix-socket cases blocked by the desktop sandbox pass unchanged at host level. The explicit-fees=None regression now imports only a dependency-free shared helper, matching the unit-only CI dependency boundary.

  • The gltest compatibility bridge now treats gltest's default (fees=None, fee_value=None) pair as omitted in both its deployment and write adapters, so fee-enabled integration submissions obtain the canonical Studio estimate while direct SDK fees=None remains explicitly gasless. Four dependency-free unit regressions cover omission semantics, forwarding, both transaction entry points, bridge installation, and idempotence.

  • Validator session/snapshot regression slice: 31 passing, covering atomic replacement, request-scoped sessions, duplicate-event suppression, and partial non-empty cache repair. A local RPC smoke confirmed one replacement event and no redundant reload.

  • Isolated PostgreSQL suite: 235 passing and 1 expected xfail, including the real migration chain and a downgrade/upgrade duplicate-envelope backfill regression.

  • Atomic-transfer focused suite: 3 passing against the finalized source image.

  • Clean-process VRF import: passing without import-order preconditions.

  • Frontend unit suite: 218 passing across 33 files.

  • Hardhat suite: 22 passing, including malformed fee-aware offset rejection with the protocol custom error.

  • Frontend typecheck and production build: passing.

  • Immutable JS SDK pin resolves to 197 sim fe fix contract state and contract methods refresh after depl… #214 bc1fc320; its 184-test suite, type declarations, lint, build, and registry-drift check pass, and Studio's npm ci --dry-run validates the exact repin. Python Remove unused env variables #110 65865a55 passes the 205 release-relevant unit tests, including schedule-free ordinary top-ups.

  • Black check on all touched Python files: passing.

  • Prettier check on touched frontend files: passing.

  • ESLint on touched frontend files: no errors; 62 pre-existing warnings.

  • Repository pre-commit hooks and git diff --check: passing.

  • Fee/RPC regression slice, including internal deployment, salted address, envelope lookup, and shared-allocation accounting: 476 passing.

  • Shared-bucket regression covers an actual storage-writing estimate, proves the recommendation funds GenVM's enforced reservoir, and proves the Consensus payload receives only exact chargeable receipt plus storage/event-write consumption.

  • Shared E2E v0.5 and v0.6 typechecks against exact JS SDK 197 sim fe fix contract state and contract methods refresh after depl… #214: passing.

  • Shared E2E focused message-discovery scenario: all 23 steps resolve in dry-run.

  • 054_fee_consensus_parity.feature runs the same canonical economics, appeal-funding, replay, and child-effect scenarios against Studio and dev-env/Consensus.

  • 051_fee_messages_internal.feature submits the exact fee envelope returned by Studio's allocation discovery and verifies real child execution and budget consumption.

Exact review cut

Release gate and risk

Studio currently pins immutable SDK commits so the prerelease can be qualified before coordinated package publication. Replace those pins with coordinated prerelease versions only when they contain these exact behaviors.

The default Studio Docker build now interprets the immutable branch:commit binding in third_party/genvm/version as source mode. Train images must prepare the sandboxed runner closure for exact source 3c96be6defe30fd9843d47bedc0c66bec324d0fa, or explicitly select a separately qualified release with GENVM_TAG.

Automatic allocation discovery is exact for root messages observed in the representative execution. Descendant allocations still require a manually supplied nested allocation tree when a caller needs to reserve those budgets explicitly; this is a documented capability boundary, not a formula mismatch.

Highest-risk surfaces are persisted fee accounting, asynchronous recovery, decision identity, and message replay. Rollback is the Studio commit stack on this PR; no external migration has been applied by publishing the branch.

Mirror cap-priced round quotes, configured ladder semantics, appeal profit reserves, exact 2.5x rewards, typed appeal funding, immutable top-up schedules, live appeal overlay pricing, latest appeal ABIs, decision guards, and surplus refunds. Expose the canonical quote over RPC so cross-stack tests no longer maintain a second Studio formula.
@MuncleUscles MuncleUscles self-assigned this Aug 27, 2026
@coderabbitai

coderabbitai Bot commented Aug 27, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

This change aligns Studio with updated protocol transaction flow. It adds fee-aware and lifecycle RPC handling, durable queue ordering, EVM envelope tracking, appeal decision metadata, salted GhostFactory address support, internal-message replay handling, new simulator fee helpers, and matching contract, worker, database, configuration, and UI updates.

Changes

Protocol v0.6 lifecycle, ordering, and fee handling

Layer / File(s) Summary
Environment and tooling defaults
.env.example, docker-compose.yml, README.md, .github/scripts/*, .github/workflows/*, requirements.test.txt, frontend/package.json, hardhat/package.json, hardhat/hardhat.config.js, .gitignore, third_party/genvm/version
Fee defaults, queue-cap defaults, and GhostFactory environment values were updated. Hardhat is documented as an optional legacy bridge. Test dependency installation now uses commit-pinned requirements, and frontend, Hardhat, and GenVM pins were updated.
Contract and protocol surface
backend/protocol_rpc/ghost_factory.py, backend/rollup/default_contracts/consensus_main.py, backend/rollup/consensus_service.py, backend/protocol_rpc/transactions_parser.py, backend/protocol_rpc/types.py, backend/node/genvm/base.py, backend/node/types.py, backend/rollup/web3_pool.py, hardhat/contracts/v2_contracts/*, hardhat/contracts/v2_contracts/interfaces/*, hardhat/contracts/v2_contracts/ghost_contracts/*, hardhat/test/consensus/*, hardhat/test/deploy/*, hardhat/test/fees/*, tests/unit/test_transactions_parser.py, tests/unit/test_ghost_factory.py, tests/unit/test_node_state_proxy_metrics.py, tests/unit/test_consensus_service_message_replay.py, tests/consensus/test_payable_scenarios.py
The contract and parser surface now supports fee-aware selectors, exact-decision appeal and finalize calls, salted ghost deployment, internal-message replay storage, public ConsensusMain addressing, offline ABI decoding, widened internal fee params, and preserved use_balance and malformed allocation payload data.
Persistence, ordering, and accounting
backend/database_handler/migration/versions/*, backend/database_handler/models.py, backend/database_handler/chain_snapshot.py, backend/database_handler/accounts_manager.py, backend/database_handler/transactions_processor.py, tests/db-sqlalchemy/*, tests/unit/test_transactions_processor_improvements.py
Database changes add the evm_envelopes ledger and durable queue_order. Transaction processing, account mutations, refunds, top-ups, appeal admission snapshots, recipient-scoped slots, finalization-head checks, and queue counts now use row locking, deterministic ordering, and envelope-backed nonce tracking.
RPC lifecycle, receipts, and appeal UI
backend/protocol_rpc/endpoints.py, backend/protocol_rpc/rpc_methods.py, backend/protocol_rpc/fastapi_endpoint_generator.py, backend/protocol_rpc/health.py, frontend/src/components/Simulator/TransactionItem.vue, frontend/test/unit/components/*, tests/integration/conftest.py, tests/load/*, tests/fixtures/fee_simulator_appeal_quotes.json, tests/unit/test_rpc_methods.py, tests/unit/test_simulator_sessions.py, tests/unit/test_consensus_contract_soft_not_found.py, tests/db-sqlalchemy/test_health_orphan_detection.py
RPC admission now validates chain ID, classifies protocol and lifecycle calls, records reverted signed envelopes, exposes lifecycle and appeal-charge queries, adds fee simulation methods, synthesizes receipts from envelopes, and uses appeal deadlines in health checks. The simulator UI now gates appeals with stored decision deadlines, dispatch-repair state, and terminal validator appeals.
Consensus history, decisions, and worker recovery
backend/consensus/types.py, backend/consensus/utils.py, backend/consensus/history.py, backend/consensus/decisions.py, backend/consensus/worker.py, tests/unit/consensus/*, tests/unit/test_consensus_voting.py, tests/unit/test_contract_not_found_handling.py, tests/unit/test_timeunit_consumption.py, tests/unit/test_worker_transient_fatal.py
Consensus handling now uses canonical result and vote codes, logical fee rounds, persisted decision metadata, deterministic-violation voting, preserved appeal validator history, queue-order-based claims, acceptance-dispatch repair before finalization, protected appeal retries, and recovery from stored appeal snapshots.

Estimated code review effort: 5 (Critical) | ~120 minutes

Merge Risk: 🟠 High · up to ab5f9

The PR changes public transaction admission and fee settlement, but the current implementation can accept an unverified sender identity, which may misattribute transactions and fees; related deployment and recovery paths also contain concrete failure cases. Merge should be blocked until the sender authorization and material runtime/deployment issues are addressed.

Suggested reviewers: kirilaa, cgmello

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant RPC as protocol_rpc.endpoints
  participant Parser as TransactionParser
  participant TP as TransactionsProcessor
  participant CS as ConsensusService

  Client->>RPC: send_raw_transaction(raw tx)
  RPC->>Parser: decode_signed_transaction(raw tx)
  Parser-->>RPC: DecodedRollupTransaction
  RPC->>TP: begin_evm_envelope(hash, sender, nonce)
  alt lifecycle call
    RPC->>TP: apply top-up / admit appeal / validate finalize
    TP-->>RPC: target tx state updated
  else protocol submission
    RPC->>TP: lock queues and reserve storage
    RPC->>TP: insert_transaction(..., commit=False)
    RPC->>CS: add_transaction(..., calldata)
    CS-->>RPC: tx id / emitted recipients
  end
  RPC->>TP: record_evm_envelope(result)
  RPC-->>Client: envelope hash or tx id
Loading
🚥 Pre-merge checks | ✅ 2 | ❌ 3

❌ Failed checks (3 warnings)

Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning The directly linked issue [#1477] requires updating docker/bake-action from v5 to v6. The reviewable changes do not show this workflow update. The description also uses [#1477] for unrelated Consensus… Update the relevant workflow to use docker/bake-action v6 and include the change in the reviewable diff, or link the PR to the issue that actually covers the Consensus v0.6 Studio alignment. Correct the issue references in the description.
Out of Scope Changes check ⚠️ Warning The linked issue [#1477] covers one dependency action update, but the PR includes extensive unrelated backend, frontend, Solidity, database, RPC, SDK, configuration, and test changes for Consensus v0.… Split the Consensus v0.6 implementation into a separate PR or update the linked issues to explicitly cover that scope. Keep this PR limited to the docker/bake-action v6 update.
Docstring Coverage ⚠️ Warning Docstring coverage is 40.12% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 324 functions across 50 files. (29 skippe… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise, uses conventional commit format, and accurately summarizes the primary change: aligning Studio with Consensus v0.6 economics.
Description check ✅ Passed The description is comprehensive and covers the change scope, motivation, implementation decisions, testing, risks, dependencies, and release guidance. It does not use every template heading, and it o…
Full details: Linked Issues check

Explanation

The directly linked issue [#1477] requires updating docker/bake-action from v5 to v6. The reviewable changes do not show this workflow update. The description also uses [#1477] for unrelated Consensus behavior, which does not match the linked issue.

Full details: Out of Scope Changes check

Explanation

The linked issue [#1477] covers one dependency action update, but the PR includes extensive unrelated backend, frontend, Solidity, database, RPC, SDK, configuration, and test changes for Consensus v0.6 alignment.

Full details: Docstring Coverage

Explanation

Docstring coverage is 40.12% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 324 functions across 50 files. (29 skipped: 19 unsupported, 10 over the file limit.)

Full details: Description check

Explanation

The description is comprehensive and covers the change scope, motivation, implementation decisions, testing, risks, dependencies, and release guidance. It does not use every template heading, and it omits the explicit issue reference and checklist, but it provides the required information in equivalent sections.

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/v0.123-fee-consensus-parity

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

MuncleUscles and others added 15 commits August 28, 2026 05:11
The branch's pin replaced a dead source ref; v0.123-dev has since moved to
feat/leader-data-to-genvm:201dc510c together with its vendored leader public
data codec (#1747), so the base side is the one the vendored code expects.
The pin inherited from v0.123-dev references the pre-merge feature branch
feat/leader-data-to-genvm, whose branch was deleted after squash-merge; the
commit is unreachable from any ref, so the source build's clone cannot read
it (fatal: unable to read tree). The same work is v0.6-dev head 3c96be6de
(feat(manager): carry opaque leader public data, #29), which also contains
the previous pin d07f21da. v0.123-dev itself needs the same repin.
Cover the two topology boundaries behind validator appeals: a five-validator pool is exhausted by the initial committee, while twelve identities provide the full fresh seven-seat first jury and preserve the exact Consensus quote.
gltest owns separate fee argument adapters for deploys and writes. Bridge both so their public None defaults are treated as omitted by the fee-aware integration fixture, while explicit direct SDK fees=None stays gasless. Add a dependency-free regression covering both entry points.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants