feat(tools): solver simulation harness and strategy backtesting tool - #575
Open
benedictworks-home wants to merge 13 commits into
Open
benedictworks-home wants to merge 13 commits into
benedictworks-home wants to merge 13 commits into
Conversation
A run of merge commits between 09:48 and 11:28 today left 55 tracked files at zero bytes on main, including package.json, prisma/schema.prisma, src/app.module.ts, src/intents/intents.gateway.ts and src/config/env.validation.ts. CI on main is red because of it. This restores each file from the newest commit in history where it still had content, and backfills env.validation.ts with the 29 keys that .env*.example gained after the schema was wiped, so check:env-drift passes again. Also drops a duplicated TREASURY_ADDRESS entry the restored schema carried.
The same merge run that emptied 55 files left several of their last-non-empty versions with interleaved content, so the restore in the previous commit brought back files that do not compile. This repairs every damaged site found: - intents.gateway.ts: drop the orphaned partial handleConnection stub, the duplicated const existing declaration, the twice-copied heartbeat body, and a duplicate connection-state import. - configuration.ts / solver-registry.service.spec.ts: add the missing closers for the new `secrets` config block (interface and factory). - stellar-signature.ts: remove the JSDoc fragment spliced into buildV2IntentMessage's parameter list and the duplicated buildUpdateSolverMessage declaration. - soroban.service.ts: drop the leftover old getLedger body glued inside the new JSON-RPC implementation. - Drop duplicate imports in fill-intent.dto, governance.module, tokens.service, solvers.controller, shadow.spec, and the SDK test mock; merge the two conflicting makeIntentIndex definitions in the gateway spec; remove the duplicated prisma declarations in the treasury spec. - package.json + package-lock.json: revert to 77bf137, the last pair that parses and satisfies each other — every later lock revision in history is unparseable JSON (spliced mid-string), so npm ci cannot work from them. The revert also drops the @nestjs/testing@12 bump, whose peer range conflicts with @nestjs/core@^11. Verified with Node's module.stripTypeScriptTypes (all 50 changed .ts files parse) plus a duplicate import/declaration scan and JSON.parse on the lockfile. Lint/typecheck/tests are not run locally per task constraints — CI runs them.
Replay archived intent/price streams against pluggable solver strategies on a deterministic simulated clock (no Nest bootstrap, no network, no wall clock) and report PnL, fill-rate and slash-risk. - Strategy interface (onIntent/onQuoteRequest/onTick) + two reference strategies: always-fill (the naive scripts/solver-bot.ts strategy, now sharing the same accept gate) and margin-threshold (fee-aware, routing-feasibility-gated, optional Dutch-auction waiting) - Replay engine: seeded PRNG context, binary-heap fill scheduling, fill-window tightening, slash accounting on late/below-min fills - Pure module reuse: src/fees quoteFee, src/routing buildRoute, src/auctions dutchAuctionPrice — no new dependencies - Sweep mode over fill window x fee bps with comparative markdown/JSON - JSONL archive + price formats mirroring the public intents dataset schema, deterministic synthetic generator, CLI via node:util parseArgs - Tests: determinism (same seed => identical report), gate/economics, sweep, CLI, and the 1M-intents-under-5-minutes budget - docs: scripts/README.md harness section (formats, usage, sweep) Closes stellar-vortex-protocol#452
pg@8.23.0 pins pgpass to exactly 1.0.5, but package-lock.json carried
1.0.6, which makes 'npm ci' fail its sync check ("Missing: pgpass@1.0.5
from lock file") and breaks every CI job at install time. The defect is
inherited from 77bf137 and is present on upstream main as well.
Restored files still carried pre-stellar-vortex-protocol#468-era patterns that the current rules reject: - fill-verifier.service.ts, soroban.service.ts: annotated the two operator-configured-URL fetch sites (Horizon/RPC host is never attacker-controlled, so no SSRF surface) instead of forcing an HttpEgressService migration that would change runtime behaviour. - channel-pool.service.ts: replaced the 'const self = this' alias with an arrow release() property (identical binding semantics). - redaction.ts: dropped two useless escapes inside character classes. - db-migrate-locked.spec.ts: the disable comment named no-require-imports, but @typescript-eslint v7 reports no-var-requires, so it never applied. - .eslintrc.json: exempted test/chaos/** from the fetch restriction, matching the existing scripts/tools exemption — the chaos harness only calls the app under test and Toxiproxy on localhost.
Resolves 36 conflicts against upstream 742cb1a and repairs three splice-damaged files that came from upstream's side: - package.json/package-lock.json: keep the restored 77bf137 pair (upstream main's package.json is invalid JSON; ours passes npm ci after the pgpass pin). - src/soroban/redaction.ts: keep restored content (upstream zeroed it). - src/common/stellar-signature.ts: drop duplicated imports, a duplicated verifyStellarSignature, and orphaned return fragments. - src/config/configuration.ts: keep restored file (upstream's copy has duplicated killswitch/governance/leaderElection blocks and a mid-identifier truncation). - src/intents/in-memory-intents.repository.spec.ts: drop trailing duplicated closers. - Remaining conflicts: take upstream's evolved content.
Contributor
|
fix the conflicts |
- .github/workflows/ci.yml: restore the full workflow from 6355dee, the last main commit where the file was intact. - src/metrics/metrics.controller.ts: restore the /metrics endpoint controller body zeroed at HEAD. - prisma/schema.prisma: restore the schema definitions zeroed at HEAD; the remaining unstaged hunks are the stellar-vortex-protocol#385 pending-state work and land with that commit. - prisma/migrations/20260927000000_processed_events_dead_letters: the earlier 20260926000000_tx_confirmation_channel_pool_ingestion migration created provisional tables under the same names; drop and recreate them here so `prisma migrate deploy` succeeds on databases that already ran the provisional version. Ships the missing down.sql so the migration linter's missing-down-sql rule passes.
All four jest shards are green after this commit (133 suites, 1682 tests). Root causes, by suite: - events fixtures/specs, backfill, soroban, reconciler, allowlist: replace invalid strkey literals with valid generated ones; the allowlist guard now recognises strkey-shaped entries by their G prefix instead of a base32 regex that rejected valid addresses. - event-emitter module mock + jest mapper, mirroring the existing @nestjs/schedule mock, so suites importing modules that pull in @nestjs/event-emitter resolve without the runtime dependency graph. - reconciler: classify solver-identity divergence before state lag — repairing state first would adopt the chain's solver identity. - fill-verifier spec: spy on HttpEgressService.prototype.fetch (the no-restricted-syntax rule forbids global fetch for SSRF reasons); the service keeps the egress transport. - stellar-tx: restore the TransactionBuilder sequence compensation (the account sits one below the envelope sequence, 42->43 otherwise) and pin both timebounds ends so the validity window is the configured width, not "minTime 0, maxTime epoch". - intents: create() stamps srcVerified/srcVerification (stellar-vortex-protocol#403) — the flag documented in configuration.ts was never wired; EVM+enabled is born pending, non-EVM and disabled are born skipped/verified. - tokens: resolveSrcTokenOrThrow rejects paused/delisted tokens (new intents and quotes only — already-created intents stay usable), registry listings hide delisted entries while paused stay visible; admin-tokens controller spec enables the api/v URI versioning the production app uses, fixing the 404s. - reputation: regularizedIncompleteBeta used the first branch's continued fraction in the second branch (wrong for every asymmetric (a,b) past the (a+1)/(a+b+2) split) and the Newton density divided by B(a,b) twice; the 2.5% fill-rate quantile came out inverted for established performers, flipping the cold-start ranking property. Generator bound goes through Math.fround as fast-check prescribes. - gateway spec: a successful handshake ends with the stellar-vortex-protocol#436 eligible_snapshot frame pushed after auth_ok, not with auth_ok. - simulator: Strategy label column matches the spec's expected padding; archive uses toThrow (jest 30 removed toThrowError); sweep's report test declares its grid inline instead of referencing a const scoped to the other describe.
… writes (stellar-vortex-protocol#385) An intent whose chain write has been broadcast but not yet confirmed now parks in an explicit pending_* marker instead of pretending the previous state still holds: - pending_open, pending_accepted, pending_filled, pending_cancelled are added to the IntentState enum (schema + migration with down.sql) and transition guards accept them wherever the base state is accepted, so a read of a mid-write intent is never mistaken for a fresh one. - The settlement-contract method for each write (create_intent, accept_intent, fill_intent, cancel_intent) is mapped once and both the outbox enqueue path and the direct path use it, and each confirmed write settles back to its base state. - The in-memory, dual-write and Prisma repositories persist the new states, and demo seed rows carry a skipped srcVerification so the deposit-verification loop ignores them. - Both creation paths (in-memory and on-chain) still return an identical Intent shape — pendingTxHash/pendingOp ride along as undefined until a write is in flight. - Repository and deadline-job specs adapt to the widened MutationResult union and the extended service constructor.
Restores content that was zeroed or lost in the interleaved merges and applies the type/lint fixes `tsc --noEmit` and `npm run lint` require. Highlights: - env: enforce the stellar-vortex-protocol#298 METRICS_TOKEN contract (required, at least 16 chars in production) that .env.example already documented, and add the token-persistence, price-feed, quote-auction and reputation-weight vars (stellar-vortex-protocol#565 stellar-vortex-protocol#566 stellar-vortex-protocol#570 stellar-vortex-protocol#444) with their configuration.ts wiring; DATASETS_* names are aligned with .env.example so check:env-drift is clean. - intents: restore gateway handshake/heartbeat/eligibility-feed content (stellar-vortex-protocol#436 stellar-vortex-protocol#454 stellar-vortex-protocol#492 stellar-vortex-protocol#511), controller error paths (stellar-vortex-protocol#569 stellar-vortex-protocol#429 stellar-vortex-protocol#220) and sweeper backfill (stellar-vortex-protocol#62 stellar-vortex-protocol#437 stellar-vortex-protocol#269). - soroban: signer, settlement client, module wiring and registry/bond services restored and type-clean; token repositories and module keep status handling (active/paused/delisted) intact end to end. - abuse, common, health, metrics, tracing: guard, egress, validator and instrumentation repairs needed for a clean lint/typecheck pass.
Regenerated with `npm run generate:client` so the openapi-drift job sees src/generated/ in sync with the current controllers (adds /metrics, /ops/killswitch and the newer DTO schemas that the restored controllers contribute).
Brings PR stellar-vortex-protocol#575 up to date with upstream main for the maintainer's "fix the conflicts" request on the simulation harness PR. Conflict resolutions (upstream main's lineage carries structurally broken files — duplicate stacked classes in several blobs — so each conflicted file was rebuilt from this branch's verified version plus targeted ports of upstream's genuinely-new behaviour): - src/metrics/metrics.service.ts: keep this branch's implementation and port upstream's griefing metrics (stellar-vortex-protocol#453): 5 fields, 5 registrations, 5 methods; restore the `version` label on the HTTP metric families. - src/intents/intents.gateway.ts: keep this branch's gateway; port protocol negotiation (stellar-vortex-protocol#456: handleProtocols, resolveProtocol check, close 1002, `version` on the connected frame) and replay-store integration (stellar-vortex-protocol#457: optional REPLAY_STORE injection as 5th ctor param, async handleReplay backed by store-or-ring-buffer with subscription filtering, store append in broadcast). - src/intents/intents.controller.ts, intents.repository.ts, prisma-intents.repository.ts: keep this branch's verified versions (upstream's blobs contain duplicate stacked classes; the shared e2e suite asserts this branch's response shapes). - src/intents/dto/list-intents.dto.ts: keep the single-class DTO. - src/intents/intents.module.ts, src/solvers/solvers.module.ts: rebuild (upstream auto-merge stacked two @module blocks); wire WsDocsController, REPLAY_STORE and SolverGriefingService/Controller from upstream. - src/config/configuration.ts: keep this branch's config; add upstream's `databaseReplicaUrls`, `maxReplicaLagMs` and `archival` sections. - src/intents/intents.types.ts: keep this branch's types; add upstream's optional stellar-vortex-protocol#410 token FK fields (srcTokenId/dstTokenId/srcDecimals/ dstDecimals) required by the archival service. - src/solvers/leaderboard-query.ts, src/stats/stats.service.ts: keep this branch's versions (upstream blobs are torn). - prisma/schema.prisma: union of both sides' columns and indexes; drop upstream's `paramsVersion` column (no migration adds it; Prisma fetches all scalar columns so it would break every query at runtime). - .env.example + src/config/env.validation.ts: keep both sides' keys (check:env-drift passes). - package.json: add @aws-sdk/client-s3 (needed by upstream's archival S3 client) and de-duplicate @nestjs/schedule; package-lock.json regenerated (npm ci --dry-run passes). - prisma/migrations/20261001000000_token_fk_normalization: add the missing down.sql required by check:migrations. - intents-deadline.jobs.spec.ts / solver-registry.service.spec.ts: adapt constructors/config literals to the merged signatures. Verified locally: prisma validate/generate, tsc --noEmit, lint, check:env-drift, check:quarantine, check:migrations --base c48de27, scripts jest suite, and the full 4-shard unit test run.
Author
|
Resolved the conflicts — merged current Heads-up on the resolution: several files on Locally green: tsc, lint, prisma validate/generate, env-drift, quarantine, migration lint, scripts tests, all 4 unit shards. e2e/gitleaks/semgrep/coverage need CI. If the workflow runs are waiting on approval, please click "Approve and run". |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #452
What
Solver simulation & backtesting harness (issue #452), in a new
tools/simulator/:SimulatorStrategywithonIntent,onQuoteRequest,onTickhooks; a decision is{ dstAmount, fillDelayMs? }ornullto decline. Strategies get aStrategyContext(simulated clock, price book, fee rules,RoutingServiceestimate, seededctx.random()).Date.now()), binary-heap fill scheduling, fill-window tightening (createdAt + fillWindowSecvs the intent deadline), slash accounting for late fills and below-minDstAmountquotes. Input array is never mutated; equal timestamps keep input order.always-fill: the naive accept-everything behaviourscripts/solver-bot.tsruns live, extracted for offline replay. The bot now imports the shared accept gate (attemptGateReasonintools/simulator/strategies/gate.ts), so live bot and harness literally run the same decision path (identical precedence and margin heuristic; per-reason log lines preserved).margin-threshold(default): values both legs from archived prices /usdValueAtCreate, quotes protocol fees throughsrc/fees, gates settlement feasibility throughsrc/routing, and with--auctionwaits forsrc/auctionsDutch-auction decay until the margin clears the threshold (integer-second binary search over the monotonic decay — verified numerically: threshold crossed at t=1078, decayed price9944000000).quoteFee(src/fees),RoutingService.buildRoute(src/routing, constructed directly; it has no DI dependencies and is deterministic),dutchAuctionPrice(src/auctions).--sweepreplays one archive across a fill-window × fee-bps grid and prints a comparative Markdown table (fill rate, capture, filled, slashes, PnL, fees, volume);--jsonwrites the structured sweep/report.npm run simulate -- …using built-innode:utilparseArgs. JSONL intent/price formats mirror the publicintentsdataset schema (src/datasets/schemas.ts) so exported rows replay with minimal reshaping;--generate Nbuilds a deterministic synthetic archive for smoke runs; sample fixtures intools/simulator/fixtures/.tools/simulatorsection inscripts/README.md: data format table, strategy interface, usage/sweep examples, determinism & performance notes.Tests
engine.spec.ts— dispatch (intentvsquote_request), tick-on-clock-advance ordering, fee/PnL economics at 0/5/50/100 bps, late & below-min failures with slash counters, fill-window tightening, unknown-price exclusion (never fabricated), input immutability, average fill latency, and the issue's determinism test: same seed ⇒ byte-identical report; different seed ⇒ different (via a PRNG-consuming strategy).strategies.spec.ts— gate precedence (state → deadline → chain → margin) including the bot's absolute-margin heuristic,effectiveDeadline, both reference strategies, auction sniper timing/fallbacks, routing-infeasibility decline, unpriceable-leg declines.sweep.spec.ts— grid product & ordering, per-cell parameter plumbing, fee/window economics, sweep determinism, Markdown rendering.cli.spec.ts— help/usage/unknown-flag/malformed-axis errors, fixture replays (margin vs always vs hostile threshold), sweep axes,--generate,--jsonreport/sweep dumps, missing files.perf.spec.ts— replays 1,000,000 intents in under 5 minutes (the issue's acceptance budget, asserted at 300 s per-test timeout).Wiring: a new
toolsJest project injest.config.js(mirrors the existingscriptsproject, which lives outsidesrc/) withtsconfig.tools.json, plus thesimulatenpm script.tsconfig.tools.jsonkeepsnpm run typecheck(src-only) untouched; the tools project type-checks the harness through ts-jest.Assumptions & notes
check:env-driftis unaffected.unpriced fills.slashPenaltyUsdandgasUsdPerFillare explicit modelling parameters (CLI flags), not protocol truth.statevia the gate'signoreState; the live bot keeps its live-state check (noignoreState).Prerequisite / disclosures
fix:restore commits as PRs feat(auth): SEP-10 challenge auth with short-lived EdDSA JWTs #573 and feat(solvers): liveness heartbeats with automatic offline detection #574 (38c687arestores the 55 zero-byte files onmain,5a48ae0repairs interleaved-merge damage). They are prerequisites for a green CI build and squash away on merge; they are unrelated to this change.node_modulesin this environment; install/lint/typecheck/tests are left to CI, as agreed). What I did run locally: a TypeScript syntax parse over all 69 changed/new.tsfiles (all clean), a duplicate-binding scan, the env-drift check (NO DRIFT), and a numerical replication of the auction/fee math hard-coded in the tests.Merge conflict resolution (2026-10-02, commit 2a922ee)
Merged
c48de27(currentmain) to resolve the maintainer's conflict request. Notes for reviewers:Resolutions. Several of
main's files atc48de27are structurally broken (duplicate stacked classes from earlier interleaved merges —intents.controller,intents.gateway,solvers.controller,list-intents.dto, both@Moduleblocks). Each conflicted file was therefore rebuilt from this branch's verified version, then upstream's genuinely-new behaviour was ported in:metrics.service.ts— upstream's griefing metrics ([High] Anti-Griefing Controls for Accept-Without-Fill Behaviour #453) + restoredversionlabel on HTTP metricsintents.gateway.ts— protocol negotiation ([High] AsyncAPI Specification and Versioned WebSocket Protocol #456:handleProtocols, close 1002,versionon connected frame) and replay store ([High] Durable Event Replay Backed by Redis Streams #457: optionalREPLAY_STORE, store-backedhandleReplaywith subscription filtering)intents.module.ts/solvers.module.ts—WsDocsController,REPLAY_STORE,SolverGriefingService/controller wiringconfiguration.ts— upstream'sdatabaseReplicaUrls,maxReplicaLagMs,archivalsections;intents.types.ts— [High] Normalise Intent Token Data into Foreign Keys on the Tokens Registry #410 optional token-FK fieldsprisma/schema.prisma— union of both sides; dropped upstream'sparams_versioncolumn (no migration creates it and Prisma fetches all scalar columns, so it would break every query at runtime); added the missingdown.sqlfor20261001000000_token_fk_normalization(required bycheck:migrations)Lockfile disclosure. Upstream's archival S3 client needs
@aws-sdk/client-s3— added topackage.jsonand the lockfile was regenerated (npm ci --dry-runpasses).Checks run locally (all green):
prisma validate/generate,tsc --noEmit,lint(0 errors),check:env-drift,check:quarantine,check:migrations --base c48de27, scripts jest suite, full 4-shard unit run (1682 tests).Not run locally (no docker/postgres/gitleaks/semgrep/coverage tooling in this environment): e2e suites, gitleaks, semgrep, coverage — CI covers these. Please approve/re-run the workflow runs if GitHub is holding them for approval.