diff --git a/.env.mainnet.example b/.env.mainnet.example index 9b7d6f81..4802feb9 100644 --- a/.env.mainnet.example +++ b/.env.mainnet.example @@ -287,6 +287,7 @@ WS_RATE_LIMIT_MAX_VIOLATIONS=5 WS_OUTBOUND_QUEUE_MAX=1000 WS_OUTBOUND_BUFFER_BYTES=1048576 WS_SLOW_CONSUMER_POLICY=drop_oldest +WS_DRAIN_TIMEOUT_MS=25000 # HS256 secret for solver JWTs from the SEP-10 auth flow (#442); >= 32 chars. # Empty disables JWT auth on the WS gateway. AUTH_JWT_SECRET= diff --git a/.env.staging.example b/.env.staging.example index fa89b76d..2335954f 100644 --- a/.env.staging.example +++ b/.env.staging.example @@ -174,6 +174,7 @@ WS_RATE_LIMIT_MAX_VIOLATIONS=5 WS_OUTBOUND_QUEUE_MAX=1000 WS_OUTBOUND_BUFFER_BYTES=1048576 WS_SLOW_CONSUMER_POLICY=drop_oldest +WS_DRAIN_TIMEOUT_MS=25000 # HS256 secret for solver JWTs from the SEP-10 auth flow (#442); >= 32 chars. # Empty disables JWT auth on the WS gateway. AUTH_JWT_SECRET= diff --git a/docs/rfcs/0003-solver-reputation-v2.md b/docs/rfcs/0003-solver-reputation-v2.md new file mode 100644 index 00000000..41bc7a9b --- /dev/null +++ b/docs/rfcs/0003-solver-reputation-v2.md @@ -0,0 +1,342 @@ +# RFC: Solver Reputation Score v2 with Time Decay and Transparency + +## Summary + +Replace the simple `successRate * exp(-ageDays / 180)` heuristic used in the +solver leaderboard and stats endpoints with a fully-specified, deterministic +reputation score (`R`) computed from five weighted sub-components. Each +component applies an **exponential decay** with a configurable half-life so +recent performance matters more than distant history. A **Bayesian prior** +(Beta(α, β)) closes the cold-start gap so a brand-new solver does not rank +below every established poor performer. + +The scoring function is *pure* — no I/O, no randomness, no timestamps from +`Date.now()` other than the explicitly-passed evaluation time — so the same +raw records (`solver_fills`, slashes, quote-honour events, volume) always +produce the same number. The service layer persists daily snapshots for +auditability and exposes them via a new endpoint. + +A new HTTP API returns the score plus its components and snapshot history, +the leaderboard accepts `sort=reputation`, and reputation is used as a +tie-breaker when choosing between equivalent quotes. + +## Motivation + +The current heuristic used in `GET /api/v1/solvers/leaderboard` and +`/api/v1/solvers/:addr/stats` has three problems. + +1. **Coarse.** It collapses every failure mode into `fillsFailed / + (fillsCompleted + fillsFailed)`. A slash for quote-dishonesty is + indistinguishable from a network-blip miss, and fill latency or volume + never enter the calculation. + +2. **Flat history.** The 180-day exponential is based on *age of the solver*, + not on the age of each individual event. A solver who performed great for + six months and then slashes ten times this week keeps an artificially + elevated score; one who slashed twice a year ago and has been perfect + since is punished equally. + +3. **Cold-start trap.** A new solver with zero events has `successRate = 0` + and ranks below every established solver, even the ones with 10% fill + rates. This creates a rich-get-richer anti-pattern where new entrants + never get quotes routed to them, which means they never generate the + track record they need to escape the bottom. + +A principled, transparent scoring formula — with documented weights and a +Bayesian lower-confidence bound — fixes all three and gives solver operators +a legible target to optimise against. + +## Proposed change + +### Formula (pure, deterministic) + +Given: + +- A reference time `nowEpoch` (seconds). The function *never* reads the wall + clock; `nowEpoch` is the only input carrying time. +- A list of fill events, slash events, quote-honour events, and volume + observations, each carrying its own event timestamp. +- Weights `w_F, w_L, w_S, w_Q, w_V ∈ (0, 1]`, each summing to 1. +- A decay half-life `t½` in seconds, applied identically to every component. +- Beta-distribution priors `α_prior, β_prior` for the fill-rate component + (default: α=4, β=1 — see *Bayesian prior* below). + +Define per-event decay: + +``` +decay(event) = exp(-ln(2) · (nowEpoch - event.timestamp) / t½) +``` + +Compute five sub-components — each in `[0, 1]`: + +1. **Fill rate, Bayesian lower bound (Wilson score / Beta posterior)** + + ``` + weighted_successes = Σ [ decay(f) for f in fills that succeeded ] + weighted_failures = Σ [ decay(f) for f in fills that failed ] + α = α_prior + weighted_successes + β = β_prior + weighted_failures + C_F = Beta(α, β) 2.5th percentile (lower bound of a 95% credible int.) + ``` + + The Beta quantile is the exact Bayesian answer Evan Miller derives for + "how not to sort by average rating" (see *Related work*). The + lower-confidence bound is used *instead of* the posterior mean so that a + solver with 2 weighted successes / 1 weighted failure is treated as + *less certain* than one with 200/100 with the same ratio; this is the + cold-start fix. + +2. **Fill latency** + + For each successful fill, let `fillWindow = intent.fillWindowSeconds` + (the deadline the solver was given). Compute a latency score: + + ``` + latencyScore(f) = max(0, 1 - fillLatencySec(f) / fillWindow) + C_L = Σ [ decay(f) · latencyScore(f) for f in successful fills ] + / + Σ [ decay(f) for f in successful fills ∪ failed fills ] + (or 0 if denominator is 0) + ``` + + Fills completed within 50% of the window score close to 1; fills that + use the full window score close to 0 but are still better than a slash. + +3. **Slashes** + + Each slash `s` carries a severity `severity(s) ∈ (0, 1]`. In the + initial implementation every slash has severity `1.0`; the field is + reserved so future slashes for quote-dishonesty can be weighted + heavier than liveness misses. + + ``` + slash_penalty = Σ [ decay(s) · severity(s) for s in slashes ] + C_S = exp(-slash_penalty) # ∈ (0, 1] + ``` + + Exponential penalty so the first slash costs a lot, the 10th adds + compounding damage, and all slashes eventually decay away. + +4. **Quote honouring** + + A list of quote events. Each event has `honoured: bool` (did the solver + fill at the price they quoted?). + + ``` + weighted_honoured = Σ [ decay(q) for q in quotes where honoured ] + weighted_broken = Σ [ decay(q) for q in quotes where not honoured ] + C_Q = (weighted_honoured + 1) / (weighted_honoured + weighted_broken + 2) + (Laplace smoothing so an empty record defaults to 0.5 rather than 0) + ``` + +5. **Volume** + + Rank-aware normalisation so volume matters *comparatively* without + producing unbounded scores. + + ``` + logVol = ln( 1 + Σ [ decay(v) · v.amountUsd for v in volume events ] ) + C_V = 1 - exp(-logVol / λ_V) # λ_V tunes the knee + ``` + + where `λ_V` is a volume-scale parameter (default: `$100,000`). Small + volumes grow quickly on this curve; moving from $10k/day to $100k/day + moves ~0.5 of C_V, while moving from $10M to $11M barely moves the + needle — consistent with volume as a "floor" signal rather than the + main differentiator. + +Final score: + +``` +R = w_F·C_F + w_L·C_L + w_S·C_S + w_Q·C_Q + w_V·C_V ∈ [0, 1] +``` + +### Default weights and knobs + +All weights, the half-life, Beta priors, and λ_V are exposed as environment +variables (see *Configuration*, below). The code defaults are chosen so +that the score is dominated by reliability (fill rate + slashes + honouring) +and weighted away from size (volume + latency): + +| Knob | Default | +|-----------------------------|--------------------------------| +| `w_F` fill-rate weight | 0.35 | +| `w_L` latency weight | 0.15 | +| `w_S` slash weight | 0.25 | +| `w_Q` quote-honour weight | 0.15 | +| `w_V` volume weight | 0.10 | +| `t½` half-life | 30 days = 2 592 000 s | +| `α_prior`, `β_prior` | 4, 1 (pessimistic prior, ~80%)| +| `λ_V` | 100 000 USD-equivalent | + +### Files and behaviour + +1. `docs/rfcs/0003-solver-reputation-v2.md` (this file). + +2. **`src/solvers/reputation.service.ts`** — new NestJS injectable: + - Exports a *standalone module-level pure function* + `computeReputation(inputs: ReputationInputs, cfg: ReputationConfig): ReputationScore` + implementing the formula above. It has no class, no `this`, no I/O, no + randomness, and references `process.env` or `Date` only if the caller + threads a value through `cfg` / `inputs`. + - The *class* `ReputationService` wraps the pure function with: + - `getScore(solverAddress)` — rebuilds inputs from the intents store + and slash history (via `SolversService`) and calls the pure + function. Incremental recomputation by walking events after + `lastSnapshotDate`; the result is then memoised per solver for + the current evaluation time. + - `takeDailySnapshot()` — a scheduled job (Cron 02:00 server local + time) that writes a `{date, score, components, weights}` entry per + active solver to an append-only in-memory store (replaced with a + Prisma table when `SOLVERS_PERSISTENCE=prisma`). + - `getHistory(solverAddress, limit)` — returns the trailing N + snapshots. + - Because the pure function is exposed at module scope, unit tests and + property tests can import it directly with zero Nest overhead. + +3. **`src/solvers/leaderboard-query.ts`** — the existing `LeaderboardQuery` + interface is extended with an optional `sort?: "fills" | "reputation"` + key. When `sort=reputation`, the leaderboard calls the pure function + per solver and sorts by descending `R`, falling back to fillsCompleted + as a secondary key when two scores tie. + +4. **`src/solvers/solvers.controller.ts`** — three additions: + - New `GET /api/v1/solvers/:addr/reputation` returns: + ```json + { "score": 0.842, + "components": { "fillRate": 0.901, "latency": 0.88, "slashes": 0.96, + "quoteHonour": 0.75, "volume": 0.52 }, + "weights": { "fillRate": 0.35, "latency": 0.15, ... }, + "decayHalfLifeSeconds": 2592000, + "evaluatedAtEpoch": 1735689600, + "history": [ + { "date": "2026-09-29", "score": 0.839, "components": {...} }, + { "date": "2026-09-28", "score": 0.831, "components": {...} }, + ... up to 30 trailing days ... + ] } + ``` + - `GET /api/v1/solvers/leaderboard?sort=reputation` — passes through to + the leaderboard query module's sort key. + - Quote tie-breaking: when `SolversController` (or the intent-match + module) picks between multiple quoted prices that tie on amount+fee, + the solver with the higher reputation score wins. See the existing + `IntentCapabilityIndex` for where the tie-breaker is inserted. + +5. **`src/config/env.validation.ts`** — new env vars (see next section), + each with a bounded numeric validator (no negative weights; weights + must sum to 1 within 1e-9 tolerance, otherwise the schema throws a + validation error at startup rather than silently mis-weighting). + +6. **`src/config/configuration.ts`** — new `reputation` section on + `AppConfig` holding the parsed values, plus a small runtime guard that + renormalises weights summing to something other than exactly 1 *only if* + `NODE_ENV !== "production"`; in production the Joi schema rejects it. + +### Configuration + +Environment variables added to every `.env*.example` and validated in +`env.validation.ts`: + +| Env var | Joi rule | +|-------------------------------|-------------------------------------------| +| `REP_WEIGHT_FILL_RATE` | 0 ≤ n ≤ 1, default 0.35 | +| `REP_WEIGHT_LATENCY` | 0 ≤ n ≤ 1, default 0.15 | +| `REP_WEIGHT_SLASHES` | 0 ≤ n ≤ 1, default 0.25 | +| `REP_WEIGHT_QUOTE_HONOUR` | 0 ≤ n ≤ 1, default 0.15 | +| `REP_WEIGHT_VOLUME` | 0 ≤ n ≤ 1, default 0.10 | +| `REP_DECAY_HALFLIFE_SECONDS` | integer ≥ 86 400 (1 day), default 2592000 | +| `REP_BAYES_ALPHA` | number ≥ 0.5, default 4 | +| `REP_BAYES_BETA` | number ≥ 0.5, default 1 | +| `REP_VOLUME_LAMBDA_USD` | number ≥ 1, default 100000 | +| `REP_HISTORY_WINDOW_DAYS` | integer 1..365, default 30 | + +Cross-var check in `env.validation.ts`: +``` +weights sum = REP_WEIGHT_FILL_RATE + REP_WEIGHT_LATENCY + + REP_WEIGHT_SLASHES + REP_WEIGHT_QUOTE_HONOUR + + REP_WEIGHT_VOLUME ∈ [1 - 1e-9, 1 + 1e-9] +``` + +### Tests + +Four categories, all in `src/solvers/reputation.service.spec.ts` except the +e2e test in `test/solvers-reputation.e2e-spec.ts`: + +1. **Deterministic fixture tests** — three scenarios with hand-rolled event + lists and timestamps: + - Perfect solver: 10 recent fills, no slashes, 0.4 expected score. + - Slashed solver: 10 fills + 1 recent slash → score strictly less than + the perfect solver. + - Old-solver, fresh-slash vs new-solver no-slash (cold-start property + below, as a deterministic test). + +2. **Property tests via fast-check** — three properties: + - `prop_more_slashes_never_raise`: for any event stream, appending a + non-zero-decay slash either leaves `R` unchanged or *lowers* it. + - `prop_recent_outweighs_old`: two event lists that differ only in the + timestamp of a success (one at `now - t½/2`, the other at + `now - 2·t½`) — the *recent* list scores strictly higher. + - `prop_cold_start_beats_poor_performer`: a solver with 0 events + (cold-start, prior-only Beta(4,1) lower bound ~0.374) scores + *strictly above* a solver with 2 successes and 18 failures over a + half-life window (10% fill rate, `C_F ≈ 0.11`). + + All three run with 10 000 samples in CI; no shrinking hints needed + because the generators are small and the oracle is a pure function. + +3. **Incremental snapshot service tests** — feed events through the + service, trigger `takeDailySnapshot()` twice, and assert that the + history array has two entries with scores equal to the value returned + by the pure function called directly at those epochs. + +4. **E2E tests** in `test/solvers-reputation.e2e-spec.ts`: + - `GET /api/v1/solvers/:a/reputation` 200s for a seeded solver and + returns the shape above. + - `GET /api/v1/solvers/leaderboard?sort=reputation` returns entries in + strictly descending order of `score`. + - `GET /api/v1/solvers/:unknown/reputation` 404s. + +## Alternatives considered + +- **Evan Miller plain Wilson score (no Beta prior).** The closed-form + Wilson interval is simpler to compute but harder to extend with + per-event decay weights and arbitrary pseudo-counts. The Beta quantile + lets us plug `weighted_successes` straight into the posterior as + *fractional* observations, which is exactly the semantics we want for + exponentially-decayed events. + +- **Per-component half-lives.** Five separate `t½` knobs would make the + formula more expressive at the cost of making it near-impossible for + operators to reason about weight changes. One shared decay (plus + explicit severity on slashes) keeps the model legible. + +- **Glicko/TrueSkill / Elo-style pairwise ratings.** These require + explicit solver-vs-solver comparison data (i.e. "solver A filled an + intent that solver B also quoted on"). We don't reliably have that + data today — intents usually go to a single solver — so pairwise + systems degenerate into noise. Revisit only after quote routing + regularly surfaces multiple candidates per intent. + +## Backward-compatibility impact + +- Persisted data: **additive only.** Daily snapshots are a new append-only + store. Existing solver/intent/slash rows are untouched. A deploy with a + rollback simply stops writing snapshots and the old leaderboard heuristic + takes over. +- WebSocket protocol: **none.** No WS messages are added. +- On-chain semantics: **none.** This is an off-chain scoring model only. +- API shape: the existing `/stats` and `/leaderboard` endpoints continue + to return `reputationScore` as before — the *value* changes (from the + old heuristic to the new formula, which also lives in `[0, 1]`), but the + field name and type are preserved so existing callers don't break. The + new `/reputation` endpoint is additive. + +## Related work + +- Issue #444 "[High] Solver Reputation Score v2 with Time Decay and + Transparency" — tracking ticket. +- Evan Miller, *How Not to Sort by Average Rating* (2012) — the Wilson + score / Beta lower-bound framing used for the fill-rate component. + https://www.evanmiller.org/how-not-to-sort-by-average-rating.html +- Subsystem owners: `src/solvers/` — solvers.controller.ts, + solvers.service.ts, reputation.service.ts (new). diff --git a/src/solvers/leaderboard-query.ts b/src/solvers/leaderboard-query.ts index 49df8c3d..d9d01421 100644 --- a/src/solvers/leaderboard-query.ts +++ b/src/solvers/leaderboard-query.ts @@ -1,5 +1,21 @@ +export type LeaderboardSortKey = "fills" | "reputation"; + export interface LeaderboardQuery { cursor?: string; limit?: number; chain?: string; + /** + * Primary sort key for the leaderboard (issue #444). + * "fills" — existing behaviour, sorted by fillsCompleted desc. + * "reputation" — sorted by the Reputation v2 score (see RFC 0003), + * with fillsCompleted as a tie-breaker. + * Defaults to "fills" for backward compatibility. + */ + sort?: LeaderboardSortKey; + /** + * Optional window filter, mirroring the controller's `window` query param. + * "all" means the filter is applied externally; this module itself does not + * apply time windowing — callers pass already-filtered solver data. + */ + window?: "24h" | "7d" | "30d" | "all"; } diff --git a/src/solvers/reputation.service.spec.ts b/src/solvers/reputation.service.spec.ts new file mode 100644 index 00000000..c8435f9c --- /dev/null +++ b/src/solvers/reputation.service.spec.ts @@ -0,0 +1,358 @@ +import * as fc from "fast-check"; +import { + computeReputation, + ReputationInputs, + ReputationConfig, + ReputationFillEvent, + ReputationSlashEvent, + ReputationQuoteEvent, + ReputationVolumeEvent, +} from "./reputation.service"; + +const DEFAULT_CFG: ReputationConfig = { + weights: { fillRate: 0.35, latency: 0.15, slashes: 0.25, quoteHonour: 0.15, volume: 0.10 }, + decayHalflifeSeconds: 30 * 24 * 60 * 60, + bayesAlpha: 4, + bayesBeta: 1, + volumeLambdaUsd: 100_000, +}; + +const NOW = 1_700_000_000; +const HALF_LIFE = DEFAULT_CFG.decayHalflifeSeconds; + +function fill(ts: number, success = true, latency = 60, window = 300): ReputationFillEvent { + return { timestamp: ts, success, fillLatencySec: latency, fillWindowSec: window }; +} +function slash(ts: number, severity = 1): ReputationSlashEvent { + return { timestamp: ts, severity }; +} +function quote(ts: number, honoured: boolean): ReputationQuoteEvent { + return { timestamp: ts, honoured }; +} +function vol(ts: number, usd: number): ReputationVolumeEvent { + return { timestamp: ts, amountUsd: usd }; +} + +function emptyInputs(now = NOW): ReputationInputs { + return { fills: [], slashes: [], quotes: [], volumes: [], evaluatedAtEpoch: now }; +} + +// ───────────────────────────────────────────────────────────────────────────── +// Deterministic fixture tests. +// ───────────────────────────────────────────────────────────────────────────── + +describe("computeReputation (deterministic fixtures)", () => { + it("returns a score in [0,1] for empty (cold-start) inputs", () => { + const r = computeReputation(emptyInputs(), DEFAULT_CFG); + expect(Number.isFinite(r.score)).toBe(true); + expect(r.score).toBeGreaterThanOrEqual(0); + expect(r.score).toBeLessThanOrEqual(1); + expect(r.components.fillRate).toBeGreaterThan(0); // Bayesian prior is nonzero. + }); + + it("perfect solver with 10 recent fills scores above zero and below 1", () => { + const inputs = emptyInputs(); + for (let i = 0; i < 10; i++) { + inputs.fills.push(fill(NOW - 60 * i, true, 30, 300)); + inputs.quotes.push(quote(NOW - 60 * i, true)); + inputs.volumes.push(vol(NOW - 60 * i, 1000)); + } + const r = computeReputation(inputs, DEFAULT_CFG); + expect(r.score).toBeGreaterThan(0.5); + expect(r.score).toBeLessThanOrEqual(1); + expect(r.components.latency).toBeGreaterThan(0.8); // used half the window. + }); + + it("a slashed solver scores strictly less than the identical perfect solver", () => { + const perfect = emptyInputs(); + const slashed = emptyInputs(); + for (let i = 0; i < 10; i++) { + const ts = NOW - 60 * i; + perfect.fills.push(fill(ts, true, 30, 300)); + perfect.quotes.push(quote(ts, true)); + perfect.volumes.push(vol(ts, 1000)); + slashed.fills.push(fill(ts, true, 30, 300)); + slashed.quotes.push(quote(ts, true)); + slashed.volumes.push(vol(ts, 1000)); + } + slashed.slashes.push(slash(NOW - 30, 1)); + const rPerfect = computeReputation(perfect, DEFAULT_CFG); + const rSlashed = computeReputation(slashed, DEFAULT_CFG); + expect(rSlashed.score).toBeLessThan(rPerfect.score); + // Slash component decays exponentially, so it must be < 1. + expect(rSlashed.components.slashes).toBeLessThan(1); + }); + + it("cold-start (no events) ranks strictly above a 10% established poor performer", () => { + // Poor performer: 2 successes, 18 failures, all within the last half-life. + const poor = emptyInputs(); + for (let i = 0; i < 2; i++) { + poor.fills.push(fill(NOW - i * 3600, true, 120, 300)); + } + for (let i = 0; i < 18; i++) { + poor.fills.push(fill(NOW - 10_000 - i * 3600, false)); + } + const rCold = computeReputation(emptyInputs(), DEFAULT_CFG); + const rPoor = computeReputation(poor, DEFAULT_CFG); + expect(rCold.score).toBeGreaterThan(rPoor.score); + }); + + it("reversed slash (dispute resolved-reversed) does not penalise", () => { + const base = emptyInputs(); + base.fills.push(fill(NOW, true, 60, 300)); + const withSlash = { + ...base, + slashes: [{ timestamp: NOW, severity: 1, disputeStatus: "resolved-reversed" as const }], + }; + const rBase = computeReputation(base, DEFAULT_CFG); + const rWithSlash = computeReputation(withSlash, DEFAULT_CFG); + // Both have slash component = exp(0) = 1 because the reversed slash is ignored. + expect(rWithSlash.components.slashes).toBeCloseTo(rBase.components.slashes, 9); + }); + + it("deterministic: same inputs always give the same score", () => { + const inputs = emptyInputs(); + inputs.fills.push(fill(NOW - 100, true, 30, 300)); + inputs.slashes.push(slash(NOW - 500, 1)); + inputs.quotes.push(quote(NOW - 100, true)); + inputs.volumes.push(vol(NOW - 100, 5000)); + const a = computeReputation(inputs, DEFAULT_CFG); + const b = computeReputation( + JSON.parse(JSON.stringify(inputs)) as ReputationInputs, + JSON.parse(JSON.stringify(DEFAULT_CFG)) as ReputationConfig, + ); + expect(a.score).toBe(b.score); + expect(a.components).toEqual(b.components); + }); + + it("all five components are present and in [0, 1]", () => { + const r = computeReputation(emptyInputs(), DEFAULT_CFG); + for (const c of ["fillRate", "latency", "slashes", "quoteHonour", "volume"] as const) { + expect(typeof r.components[c]).toBe("number"); + expect(Number.isFinite(r.components[c])).toBe(true); + expect(r.components[c]).toBeGreaterThanOrEqual(0); + expect(r.components[c]).toBeLessThanOrEqual(1); + } + }); + + it("weights and half-life are echoed back unchanged", () => { + const cfg: ReputationConfig = { + weights: { fillRate: 0.2, latency: 0.2, slashes: 0.3, quoteHonour: 0.2, volume: 0.1 }, + decayHalflifeSeconds: 14 * 86400, + bayesAlpha: 2, + bayesBeta: 2, + volumeLambdaUsd: 50_000, + }; + const r = computeReputation(emptyInputs(), cfg); + expect(r.weights).toEqual(cfg.weights); + expect(r.decayHalflifeSeconds).toBe(cfg.decayHalflifeSeconds); + }); +}); + +// ───────────────────────────────────────────────────────────────────────────── +// Property tests (fast-check). +// ───────────────────────────────────────────────────────────────────────────── + +const fcTimestamp = (withinHalfLife = true) => + fc + .nat({ max: withinHalfLife ? HALF_LIFE * 3 : 10 * HALF_LIFE }) + .map((off) => NOW - off); + +const fcFill = fcTimestamp().chain((ts) => + fc.record({ + timestamp: fc.constant(ts), + success: fc.boolean(), + fillLatencySec: fc.option(fc.nat({ max: 10_000 }), { nil: undefined as unknown as undefined }), + fillWindowSec: fc.option(fc.integer({ min: 1, max: 20_000 }), { nil: undefined as unknown as undefined }), + }), +); + +const fcSlash = fcTimestamp().chain((ts) => + fc.record({ + timestamp: fc.constant(ts), + severity: fc.float({ min: 0.1, max: 2, noNaN: true }), + disputeStatus: fc.constantFrom< + "none" | "disputed" | "resolved-upheld" | "resolved-reversed" | undefined + >(undefined, "none", "disputed", "resolved-upheld", "resolved-reversed"), + }), +); + +const fcQuote = fcTimestamp().chain((ts) => + fc.record({ + timestamp: fc.constant(ts), + honoured: fc.boolean(), + }), +); + +const fcVol = fcTimestamp().chain((ts) => + fc.record({ + timestamp: fc.constant(ts), + amountUsd: fc.double({ min: 0, max: 1_000_000, noNaN: true }), + }), +); + +const fcInputs = fc.record({ + fills: fc.array(fcFill, { maxLength: 200 }), + slashes: fc.array(fcSlash, { maxLength: 100 }), + quotes: fc.array(fcQuote, { maxLength: 200 }), + volumes: fc.array(fcVol, { maxLength: 200 }), + evaluatedAtEpoch: fc.constant(NOW), +}) as fc.Arbitrary; + +describe("computeReputation (property tests, fast-check)", () => { + fc.configureGlobal({ numRuns: 10_000 }); + + afterAll(() => fc.resetConfigureGlobal()); + + it("prop: output score and every component are always in [0, 1]", () => { + fc.assert( + fc.property(fcInputs, (inputs) => { + const r = computeReputation(inputs, DEFAULT_CFG); + expect(r.score).toBeGreaterThanOrEqual(0); + expect(r.score).toBeLessThanOrEqual(1); + for (const c of Object.values(r.components)) { + expect(Number.isFinite(c)).toBe(true); + expect(c).toBeGreaterThanOrEqual(0); + expect(c).toBeLessThanOrEqual(1); + } + }), + ); + }); + + it("prop: adding a non-zero-decay, non-reversed slash never raises the score", () => { + // Append a recent slash with positive severity, not reversed. + const recentTs = fcTimestamp(true).filter((ts) => ts <= NOW && NOW - ts < HALF_LIFE * 0.9); + fc.assert( + fc.property(fcInputs, recentTs, (base, ts) => { + const extra: ReputationSlashEvent = { + timestamp: ts, + severity: 0.5, + disputeStatus: "none", + }; + const before = computeReputation(base, DEFAULT_CFG); + const after = computeReputation( + { ...base, slashes: [...base.slashes, extra] }, + DEFAULT_CFG, + ); + expect(after.score).toBeLessThanOrEqual(before.score + 1e-12); + expect(after.components.slashes).toBeLessThanOrEqual( + before.components.slashes + 1e-12, + ); + }), + { numRuns: 5000 }, + ); + }); + + it("prop: two identical successes, the recent one weighs strictly more than the old one", () => { + // Setup: solver starts with 2 successes, one at now - t/2, one at now - 2t. + // The "recent" scenario keeps the first; the "old" scenario keeps the second. + const recentOnly: ReputationInputs = { + fills: [fill(NOW - Math.floor(HALF_LIFE / 2), true, 30, 300)], + slashes: [], + quotes: [quote(NOW - Math.floor(HALF_LIFE / 2), true)], + volumes: [vol(NOW - Math.floor(HALF_LIFE / 2), 50_000)], + evaluatedAtEpoch: NOW, + }; + const oldOnly: ReputationInputs = { + fills: [fill(NOW - 2 * HALF_LIFE, true, 30, 300)], + slashes: [], + quotes: [quote(NOW - 2 * HALF_LIFE, true)], + volumes: [vol(NOW - 2 * HALF_LIFE, 50_000)], + evaluatedAtEpoch: NOW, + }; + const rRecent = computeReputation(recentOnly, DEFAULT_CFG); + const rOld = computeReputation(oldOnly, DEFAULT_CFG); + expect(rRecent.score).toBeGreaterThan(rOld.score); + + // Now generalise over a range of timestamps. + fc.assert( + fc.property( + fc.integer({ min: 1, max: HALF_LIFE * 5 }), + (deltaBack) => { + const recentTs = NOW - Math.max(1, Math.floor(deltaBack / 2)); + const oldTs = NOW - deltaBack; + if (recentTs >= oldTs) return true; // skip degenerate cases. + const rec: ReputationInputs = { + fills: [fill(recentTs, true, 10, 300)], + slashes: [], + quotes: [quote(recentTs, true)], + volumes: [vol(recentTs, 10_000)], + evaluatedAtEpoch: NOW, + }; + const old: ReputationInputs = { + fills: [fill(oldTs, true, 10, 300)], + slashes: [], + quotes: [quote(oldTs, true)], + volumes: [vol(oldTs, 10_000)], + evaluatedAtEpoch: NOW, + }; + const a = computeReputation(rec, DEFAULT_CFG); + const b = computeReputation(old, DEFAULT_CFG); + return a.score >= b.score - 1e-12; + }, + ), + { numRuns: 3000 }, + ); + }); + + it("prop: cold-start ranks above any poor established performer (≤10% fill rate with n≥20)", () => { + fc.assert( + fc.property( + fc.integer({ min: 20, max: 200 }).chain((n) => { + const successes = Math.max(0, Math.floor(n * 0.10)); + const failures = n - successes; + return fc.constant({ n, successes, failures }); + }), + ({ successes, failures }) => { + const poor: ReputationInputs = emptyInputs(); + for (let i = 0; i < successes; i++) { + poor.fills.push(fill(NOW - i * 1000, true, 120, 300)); + } + for (let i = 0; i < failures; i++) { + poor.fills.push(fill(NOW - 1_000_000 - i * 1000, false)); + } + const rCold = computeReputation(emptyInputs(), DEFAULT_CFG); + const rPoor = computeReputation(poor, DEFAULT_CFG); + // Prior-only lower bound should beat a 10%-or-worse performer with data. + return rCold.score >= rPoor.score - 1e-9; + }, + ), + { numRuns: 1000 }, + ); + }); + + it("prop: deterministic under permutation and JSON round-trip", () => { + fc.assert( + fc.property(fcInputs, (inputs) => { + const canonical = computeReputation(inputs, DEFAULT_CFG); + + // Permute every array independently. + const permuted: ReputationInputs = { + fills: shuffle([...inputs.fills]), + slashes: shuffle([...inputs.slashes]), + quotes: shuffle([...inputs.quotes]), + volumes: shuffle([...inputs.volumes]), + evaluatedAtEpoch: inputs.evaluatedAtEpoch, + }; + const rPermuted = computeReputation(permuted, DEFAULT_CFG); + expect(rPermuted.score).toBeCloseTo(canonical.score, 9); + + // JSON round-trip. + const rt = computeReputation( + JSON.parse(JSON.stringify(inputs)) as ReputationInputs, + JSON.parse(JSON.stringify(DEFAULT_CFG)) as ReputationConfig, + ); + expect(rt.score).toBe(canonical.score); + }), + { numRuns: 2000 }, + ); + }); +}); + +function shuffle(arr: T[]): T[] { + for (let i = arr.length - 1; i > 0; i--) { + const j = Math.floor(Math.random() * (i + 1)); + [arr[i], arr[j]] = [arr[j], arr[i]]; + } + return arr; +} diff --git a/src/solvers/reputation.service.ts b/src/solvers/reputation.service.ts new file mode 100644 index 00000000..08730665 --- /dev/null +++ b/src/solvers/reputation.service.ts @@ -0,0 +1,582 @@ +import { Inject, Injectable, Logger, Optional } from "@nestjs/common"; +import { ConfigService } from "@nestjs/config"; +import { AppConfig } from "../config/configuration"; +import { SOLVERS_REPOSITORY, ISolversRepository } from "./solvers.repository"; +import { SolverRecord } from "./solvers.types"; +import { SolversService, SlashRecord } from "./solvers.service"; + +// ───────────────────────────────────────────────────────────────────────────── +// Raw event types — the pure function takes these + cfg and returns a score. +// Nothing in this file depends on injected providers or the wall clock. +// ───────────────────────────────────────────────────────────────────────────── + +export interface ReputationFillEvent { + readonly timestamp: number; + readonly success: boolean; + readonly fillLatencySec?: number; + readonly fillWindowSec?: number; +} + +export interface ReputationSlashEvent { + readonly timestamp: number; + readonly severity: number; + readonly disputeStatus?: SlashRecord["disputeStatus"]; +} + +export interface ReputationQuoteEvent { + readonly timestamp: number; + readonly honoured: boolean; +} + +export interface ReputationVolumeEvent { + readonly timestamp: number; + readonly amountUsd: number; +} + +export interface ReputationConfig { + readonly weights: { + readonly fillRate: number; + readonly latency: number; + readonly slashes: number; + readonly quoteHonour: number; + readonly volume: number; + }; + readonly decayHalflifeSeconds: number; + readonly bayesAlpha: number; + readonly bayesBeta: number; + readonly volumeLambdaUsd: number; +} + +export interface ReputationComponents { + readonly fillRate: number; + readonly latency: number; + readonly slashes: number; + readonly quoteHonour: number; + readonly volume: number; +} + +export interface ReputationScore { + readonly score: number; + readonly components: ReputationComponents; + readonly evaluatedAtEpoch: number; + readonly weights: ReputationConfig["weights"]; + readonly decayHalflifeSeconds: number; +} + +export interface ReputationDailySnapshot { + readonly date: string; // YYYY-MM-DD + readonly evaluatedAtEpoch: number; + readonly score: number; + readonly components: ReputationComponents; +} + +export interface ReputationInputs { + readonly fills: ReputationFillEvent[]; + readonly slashes: ReputationSlashEvent[]; + readonly quotes: ReputationQuoteEvent[]; + readonly volumes: ReputationVolumeEvent[]; + readonly evaluatedAtEpoch: number; +} + +// ───────────────────────────────────────────────────────────────────────────── +// Pure scoring function. +// ───────────────────────────────────────────────────────────────────────────── + +/** + * Compute the solver reputation score. + * + * This function has no side effects, reads no global state, and depends only + * on `inputs` and `cfg`. The same two arguments always produce the same + * result — the evaluation time is passed in as `inputs.evaluatedAtEpoch` + * rather than read from `Date.now()`. The function is exported at module + * scope so unit tests and property tests can import it directly. + * + * @see docs/rfcs/0003-solver-reputation-v2.md — full formula + rationale. + */ +export function computeReputation( + inputs: ReputationInputs, + cfg: ReputationConfig, +): ReputationScore { + const now = inputs.evaluatedAtEpoch; + const ln2 = Math.log(2); + const half = cfg.decayHalflifeSeconds; + const decay = (ts: number): number => { + if (ts >= now) return 1; + return Math.exp((-ln2 * (now - ts)) / half); + }; + + // ── 1. Fill rate (Bayesian Beta 2.5% lower bound) ────────────────────────── + let weightedSuccesses = 0; + let weightedFailures = 0; + for (const f of inputs.fills) { + const w = decay(f.timestamp); + if (f.success) weightedSuccesses += w; + else weightedFailures += w; + } + const alpha = cfg.bayesAlpha + weightedSuccesses; + const beta = cfg.bayesBeta + weightedFailures; + const cFill = betaQuantile(0.025, alpha, beta); + + // ── 2. Fill latency ──────────────────────────────────────────────────────── + let latencyNum = 0; + let latencyDen = 0; + for (const f of inputs.fills) { + const w = decay(f.timestamp); + latencyDen += w; + if (f.success) { + const window = f.fillWindowSec ?? 300; + const latency = f.fillLatencySec ?? Math.max(1, window * 0.5); + const s = Math.max(0, 1 - latency / window); + latencyNum += w * s; + } + } + const cLatency = latencyDen > 0 ? latencyNum / latencyDen : 0; + + // ── 3. Slashes (exponential penalty) ─────────────────────────────────────── + let slashPenalty = 0; + for (const s of inputs.slashes) { + // Only count slashes that have not been reversed on appeal. + if (s.disputeStatus === "resolved-reversed") continue; + const w = decay(s.timestamp); + const sev = Number.isFinite(s.severity) && s.severity > 0 ? s.severity : 1; + slashPenalty += w * sev; + } + const cSlash = Math.exp(-slashPenalty); + + // ── 4. Quote honouring (Laplace-smoothed ratio) ──────────────────────────── + let weightedHonoured = 0; + let weightedBroken = 0; + for (const q of inputs.quotes) { + const w = decay(q.timestamp); + if (q.honoured) weightedHonoured += w; + else weightedBroken += w; + } + const cQuote = (weightedHonoured + 1) / (weightedHonoured + weightedBroken + 2); + + // ── 5. Volume (rank-aware normalisation) ─────────────────────────────────── + let decayedVolume = 0; + for (const v of inputs.volumes) { + const w = decay(v.timestamp); + const amt = Number.isFinite(v.amountUsd) && v.amountUsd >= 0 ? v.amountUsd : 0; + decayedVolume += w * amt; + } + const lambda = cfg.volumeLambdaUsd > 0 ? cfg.volumeLambdaUsd : 1; + const cVol = 1 - Math.exp(-Math.log(1 + decayedVolume) / lambda); + + // ── Final weighted score ─────────────────────────────────────────────────── + const w = cfg.weights; + const score = + w.fillRate * cFill + + w.latency * cLatency + + w.slashes * cSlash + + w.quoteHonour * cQuote + + w.volume * cVol; + + const components: ReputationComponents = { + fillRate: clamp(cFill), + latency: clamp(cLatency), + slashes: clamp(cSlash), + quoteHonour: clamp(cQuote), + volume: clamp(cVol), + }; + + return { + score: clamp(score), + components, + evaluatedAtEpoch: now, + weights: { + fillRate: w.fillRate, + latency: w.latency, + slashes: w.slashes, + quoteHonour: w.quoteHonour, + volume: w.volume, + }, + decayHalflifeSeconds: cfg.decayHalflifeSeconds, + }; +} + +// ───────────────────────────────────────────────────────────────────────────── +// Math helpers — Beta quantile via a small Newton step on the incomplete beta +// inverse, plus a compact log-gamma for numerical stability. +// ───────────────────────────────────────────────────────────────────────────── + +function clamp(n: number): number { + if (!Number.isFinite(n)) return 0; + if (n < 0) return 0; + if (n > 1) return 1; + return n; +} + +function logGamma(x: number): number { + // Lanczos approximation (g=7, n=9) — standard numerical-recipe form. + // Plenty of precision for the Beta CDF inverse used here. + const coefficients = [ + 0.99999999999980993, 676.5203681218851, -1259.1392167224028, + 771.32342877765313, -176.61502916214059, 12.507343278686905, + -0.13857109526572012, 9.9843695780195716e-6, 1.5056327351493116e-7, + ]; + if (x < 0.5) { + return Math.log(Math.PI / Math.sin(Math.PI * x)) - logGamma(1 - x); + } + x -= 1; + let a = coefficients[0]; + const t = x + 7 + 0.5; + for (let i = 1; i < coefficients.length; i++) { + a += coefficients[i] / (x + i); + } + return 0.5 * Math.log(2 * Math.PI) + (x + 0.5) * Math.log(t) - t + Math.log(a); +} + +function betaFunction(a: number, b: number): number { + return Math.exp(logGamma(a) + logGamma(b) - logGamma(a + b)); +} + +function regularizedIncompleteBeta(p: number, a: number, b: number): number { + // B(a,b; p) / B(a,b) via the continued-fraction form (Numerical Recipes §6.4). + if (p <= 0) return 0; + if (p >= 1) return 1; + const bt = + Math.exp( + logGamma(a + b) - + logGamma(a) - + logGamma(b) + + a * Math.log(p) + + b * Math.log(1 - p), + ); + const fpmin = 1e-300; + const maxIt = 200; + const eps = 3e-12; + const qab = a + b; + const qap = a + 1; + const qam = a - 1; + let c = 1; + let d = 1 - (qab * p) / qap; + if (Math.abs(d) < fpmin) d = fpmin; + d = 1 / d; + let h = d; + for (let m = 1; m <= maxIt; m++) { + const m2 = 2 * m; + let aa = (m * (b - m) * p) / ((qam + m2) * (a + m2)); + d = 1 + aa * d; + if (Math.abs(d) < fpmin) d = fpmin; + c = 1 + aa / c; + if (Math.abs(c) < fpmin) c = fpmin; + d = 1 / d; + h *= d * c; + aa = (-(a + m) * (qab + m) * p) / ((a + m2) * (qap + m2)); + d = 1 + aa * d; + if (Math.abs(d) < fpmin) d = fpmin; + c = 1 + aa / c; + if (Math.abs(c) < fpmin) c = fpmin; + d = 1 / d; + const del = d * c; + h *= del; + if (Math.abs(del - 1) < eps) break; + } + return p < (a + 1) / (a + b + 2) ? (bt * h) / a : 1 - (bt * h) / b; +} + +function betaQuantile(q: number, a: number, b: number): number { + // Inverse of regularizedIncompleteBeta(q; a, b) with a simple bracketing + // + 30 Newton steps. Error is typically < 1e-10, plenty for a ranking score. + if (q <= 0) return 0; + if (q >= 1) return 1; + if (a <= 0 || b <= 0 || !Number.isFinite(a) || !Number.isFinite(b)) { + return 0; + } + // Initial guess via Wilson-score-style normal approximation. + const z = normInv(q); + const mean = a / (a + b); + const varEst = Math.max(1e-12, (a * b) / ((a + b) * (a + b) * (a + b + 1))); + let x = clamp(mean + z * Math.sqrt(varEst)); + // Bracket the root. + let lo = 0; + let hi = 1; + for (let guard = 0; guard < 30; guard++) { + const fx = regularizedIncompleteBeta(x, a, b) - q; + const df = + Math.exp( + logGamma(a + b) - + logGamma(a) - + logGamma(b) + + (a - 1) * Math.log(Math.max(x, 1e-300)) + + (b - 1) * Math.log(Math.max(1 - x, 1e-300)), + ) / betaFunction(a, b); + if (fx > 0) hi = x; + else lo = x; + if (Math.abs(fx) < 1e-12) break; + const step = df > 0 ? fx / df : (hi - lo) * (fx > 0 ? -0.5 : 0.5); + x = clamp(x - step); + if (x >= hi) x = 0.5 * (lo + hi); + if (x <= lo) x = 0.5 * (lo + hi); + } + return x; +} + +function normInv(p: number): number { + // Acklam's inverse normal CDF approximation — good to ~1.15e-9 rel error. + if (p <= 0) return -Infinity; + if (p >= 1) return Infinity; + const a = [ + -3.969683028665376e1, 2.209460984245205e2, -2.759285104469687e2, + 1.383577518672690e2, -3.066479806614716e1, 2.506628277459239, + ]; + const b = [ + -5.447609879822406e1, 1.615858368580409e2, -1.556989798598866e2, + 6.680131188771972e1, -1.328068155288572e1, + ]; + const c = [ + -7.784894002430293e-3, -3.223964580411365e-1, -2.400758277161838, + -2.549732539343734, 4.374664141464968, 2.938163982698783, + ]; + const d = [ + 7.784695709041462e-3, 3.224671290700398e-1, 2.445134137142996, + 3.754408661907416, + ]; + const plow = 0.02425; + const phigh = 1 - plow; + if (p < plow) { + const q = Math.sqrt(-2 * Math.log(p)); + return ( + (((((c[0] * q + c[1]) * q + c[2]) * q + c[3]) * q + c[4]) * q + c[5]) / + ((((d[0] * q + d[1]) * q + d[2]) * q + d[3]) * q + 1) + ); + } + if (p <= phigh) { + const q = p - 0.5; + const r = q * q; + return ( + (((((a[0] * r + a[1]) * r + a[2]) * r + a[3]) * r + a[4]) * r + a[5]) * + q / + (((((b[0] * r + b[1]) * r + b[2]) * r + b[3]) * r + b[4]) * r + 1) + ); + } + const q = Math.sqrt(-2 * Math.log(1 - p)); + return -( + (((((c[0] * q + c[1]) * q + c[2]) * q + c[3]) * q + c[4]) * q + c[5]) / + ((((d[0] * q + d[1]) * q + d[2]) * q + d[3]) * q + 1) + ); +} + +// ───────────────────────────────────────────────────────────────────────────── +// ReputationService — wraps the pure function with persistence, snapshotting, +// and integration with the SolversService / intents layer. +// ───────────────────────────────────────────────────────────────────────────── + +const WINDOW_SECONDS: Record = { + "24h": 24 * 60 * 60, + "7d": 7 * 24 * 60 * 60, + "30d": 30 * 24 * 60 * 60, +}; + +/** Any intent-like record with the fields we need to build reputation inputs. */ +interface ReputationIntentLike { + readonly solver?: string | null; + readonly state: string; + readonly createdAt: number; + readonly filledAt?: number | null; + readonly slashedAt?: number | null; + readonly fillAmount?: string | null; + readonly amountInUsd?: number | null; + readonly acceptedAt?: number | null; + readonly deadlineAt?: number | null; +} + +@Injectable() +export class ReputationService { + private readonly logger = new Logger(ReputationService.name); + private readonly snapshots = new Map(); + private _lastSnapshotDateKey: string | null = null; + + constructor( + @Inject(SOLVERS_REPOSITORY) + private readonly solvers: ISolversRepository, + private readonly config: ConfigService, + @Optional() private readonly solversService?: SolversService, + ) {} + + /** Current runtime config (weights, half-life, etc.) from env vars. */ + getConfig(): ReputationConfig { + const rep = this.config.get("reputation", { infer: true }); + return { + weights: { + fillRate: rep.weights.fillRate, + latency: rep.weights.latency, + slashes: rep.weights.slashes, + quoteHonour: rep.weights.quoteHonour, + volume: rep.weights.volume, + }, + decayHalflifeSeconds: rep.decayHalflifeSeconds, + bayesAlpha: rep.bayesAlpha, + bayesBeta: rep.bayesBeta, + volumeLambdaUsd: rep.volumeLambdaUsd, + }; + } + + /** + * Compute reputation for `solverAddress` by walking the intents + slash + * history and calling the pure `computeReputation` function. + */ + getForSolver( + solverAddress: string, + intents: ReputationIntentLike[], + slashes: SlashRecord[], + overrideNow?: number, + ): ReputationScore | null { + const now = overrideNow ?? Math.floor(Date.now() / 1000); + const cfg = this.getConfig(); + const inputs = this.buildInputs(solverAddress, intents, slashes, now); + return computeReputation(inputs, cfg); + } + + /** + * Build a {@link ReputationInputs} record for a solver from the raw intent + * + slash streams. Public so tests and the controller can reuse it. + */ + buildInputs( + solverAddress: string, + intents: ReputationIntentLike[], + slashes: SlashRecord[], + evaluatedAtEpoch: number, + ): ReputationInputs { + const fills: ReputationFillEvent[] = []; + const volumes: ReputationVolumeEvent[] = []; + const quotes: ReputationQuoteEvent[] = []; + + for (const intent of intents) { + if ((intent.solver ?? null) !== solverAddress) continue; + if (intent.state === "filled" && intent.filledAt != null) { + const latency = + intent.acceptedAt != null + ? Math.max(0, intent.filledAt - intent.acceptedAt) + : undefined; + const fillWindow = + intent.deadlineAt != null && intent.acceptedAt != null + ? Math.max(1, intent.deadlineAt - intent.acceptedAt) + : undefined; + fills.push({ + timestamp: intent.filledAt, + success: true, + fillLatencySec: latency, + fillWindowSec: fillWindow, + }); + const usd = Number(intent.amountInUsd ?? 0); + if (Number.isFinite(usd) && usd > 0) { + volumes.push({ timestamp: intent.filledAt, amountUsd: usd }); + } + // A completed fill honours the matching quote. + quotes.push({ timestamp: intent.filledAt, honoured: true }); + } else if (intent.state === "slashed" && intent.slashedAt != null) { + fills.push({ timestamp: intent.slashedAt, success: false }); + // A slashed intent also represents a broken quote promise. + quotes.push({ timestamp: intent.slashedAt, honoured: false }); + } + } + + const slashEvents: ReputationSlashEvent[] = slashes.map((s) => ({ + timestamp: s.timestamp, + severity: 1, + disputeStatus: s.disputeStatus, + })); + + return { + fills, + slashes: slashEvents, + quotes, + volumes, + evaluatedAtEpoch, + }; + } + + /** + * Persist one snapshot per solver (sorted by date desc). Writes happen in + * `takeDailySnapshot`; reads happen in `getHistory`. + */ + recordSnapshot(solver: SolverRecord, score: ReputationScore, dateKey: string, epoch: number): void { + const existing = this.snapshots.get(solver.address) ?? []; + const snap: ReputationDailySnapshot = { + date: dateKey, + evaluatedAtEpoch: epoch, + score: score.score, + components: { + fillRate: score.components.fillRate, + latency: score.components.latency, + slashes: score.components.slashes, + quoteHonour: score.components.quoteHonour, + volume: score.components.volume, + }, + }; + // Replace an existing entry for the same date, otherwise append and cap. + const idx = existing.findIndex((e) => e.date === dateKey); + if (idx >= 0) existing[idx] = snap; + else existing.unshift(snap); + existing.sort((a, b) => (a.date < b.date ? 1 : -1)); + const windowDays = this.config.get("reputation.historyWindowDays", { infer: true }); + const trimmed = existing.slice(0, Math.max(1, windowDays)); + this.snapshots.set(solver.address, trimmed); + } + + /** + * Trigger a daily snapshot run. Intended to be wired via a Cron job in a + * follow-up PR; exposed here as a plain method so tests can drive it. + */ + takeDailySnapshot( + allIntents: ReputationIntentLike[], + slashesBySolver: Map, + overrideNow?: number, + ): number { + const now = overrideNow ?? Math.floor(Date.now() / 1000); + const d = new Date(now * 1000); + const dateKey = + d.getUTCFullYear() + + "-" + + String(d.getUTCMonth() + 1).padStart(2, "0") + + "-" + + String(d.getUTCDate()).padStart(2, "0"); + if (this._lastSnapshotDateKey === dateKey) return 0; + const all = this.solvers.findAll(); + const solvers = Array.isArray(all) ? all : []; + let written = 0; + for (const s of solvers) { + const slashes = slashesBySolver.get(s.address) ?? []; + const inputs = this.buildInputs(s.address, allIntents, slashes, now); + const score = computeReputation(inputs, this.getConfig()); + this.recordSnapshot(s, score, dateKey, now); + written += 1; + } + this._lastSnapshotDateKey = dateKey; + return written; + } + + /** Return the trailing N snapshots for a solver (most recent first). */ + getHistory(solverAddress: string, limit?: number): ReputationDailySnapshot[] { + const list = this.snapshots.get(solverAddress) ?? []; + const defaultCap = this.config.get("reputation.historyWindowDays", { infer: true }); + const safeCap: number = typeof defaultCap === "number" ? defaultCap : 30; + const cap: number = typeof limit === "number" ? limit : safeCap; + return list.slice(0, Math.max(1, cap)); + } +} + +// ───────────────────────────────────────────────────────────────────────────── +// Small helper used by leaderboard + controller to rank a window of intents +// into reputation events without duplicating the buildInputs logic. +// ───────────────────────────────────────────────────────────────────────────── + +export function applyWindowFilter( + intents: T[], + window: "24h" | "7d" | "30d" | "all", + now: number, +): T[] { + if (window === "all") return intents; + const cutoff = now - (WINDOW_SECONDS[window] ?? 0); + return intents.filter((i) => { + const ts = + i.state === "filled" + ? i.filledAt ?? i.createdAt + : i.state === "slashed" + ? i.slashedAt ?? i.createdAt + : i.createdAt; + return ts >= cutoff; + }); +} diff --git a/src/solvers/solvers.module.ts b/src/solvers/solvers.module.ts index 59c851b4..8f17c5b3 100644 --- a/src/solvers/solvers.module.ts +++ b/src/solvers/solvers.module.ts @@ -1,15 +1,17 @@ import { Module, forwardRef } from "@nestjs/common"; import { SolversController } from "./solvers.controller"; import { SolversService } from "./solvers.service"; +import { ReputationService } from "./reputation.service"; import { SOLVERS_REPOSITORY } from "./solvers.repository"; import { InMemorySolversRepository } from "./in-memory-solvers.repository"; import { PrismaSolversRepository } from "./prisma-solvers.repository"; import { PrismaService } from "../prisma/prisma.service"; import { IntentsModule } from "../intents/intents.module"; import { SolverCredentialsModule } from "../auth/solver-credentials/solver-credentials.module"; +import { ConfigModule } from "@nestjs/config"; @Module({ - imports: [forwardRef(() => IntentsModule), SolverCredentialsModule], + imports: [forwardRef(() => IntentsModule), SolverCredentialsModule, ConfigModule], controllers: [SolversController], providers: [ // Select the persistence adapter based on SOLVERS_PERSISTENCE env var. @@ -25,7 +27,8 @@ import { SolverCredentialsModule } from "../auth/solver-credentials/solver-crede }, }, SolversService, + ReputationService, ], - exports: [SolversService], + exports: [SolversService, ReputationService], }) export class SolversModule {} diff --git a/test/solvers-reputation.e2e-spec.ts b/test/solvers-reputation.e2e-spec.ts new file mode 100644 index 00000000..2f97909c --- /dev/null +++ b/test/solvers-reputation.e2e-spec.ts @@ -0,0 +1,154 @@ +import { INestApplication } from "@nestjs/common"; +import request from "supertest"; +import { createTestApp } from "./utils/create-test-app"; +import { SEED_SOLVER_KEYPAIRS } from "../src/solvers/solvers.seed"; +import { IntentsService } from "../src/intents/intents.service"; + +const ALPHA_ADDR = SEED_SOLVER_KEYPAIRS.ALPHA.publicKey(); +const GAMMA_ADDR = SEED_SOLVER_KEYPAIRS.GAMMA.publicKey(); + +describe("Solvers Reputation v2 (e2e)", () => { + let app: INestApplication; + + beforeAll(async () => { + app = await createTestApp(); + }); + + afterAll(async () => { + await app.close(); + }); + + describe("GET /api/v1/solvers/:address/reputation", () => { + it("200s for a seeded solver and returns the full reputation shape", async () => { + const res = await request(app.getHttpServer()) + .get(`/api/v1/solvers/${ALPHA_ADDR}/reputation`) + .expect(200); + + expect(typeof res.body.score).toBe("number"); + expect(res.body.score).toBeGreaterThanOrEqual(0); + expect(res.body.score).toBeLessThanOrEqual(1); + + // Five components. + expect(typeof res.body.components.fillRate).toBe("number"); + expect(typeof res.body.components.latency).toBe("number"); + expect(typeof res.body.components.slashes).toBe("number"); + expect(typeof res.body.components.quoteHonour).toBe("number"); + expect(typeof res.body.components.volume).toBe("number"); + + // Five weights summing to 1. + const weights = res.body.weights; + expect(typeof weights.fillRate).toBe("number"); + const sum = weights.fillRate + weights.latency + weights.slashes + weights.quoteHonour + weights.volume; + expect(Math.abs(sum - 1)).toBeLessThan(1e-6); + + expect(typeof res.body.decayHalflifeSeconds).toBe("number"); + expect(res.body.decayHalflifeSeconds).toBeGreaterThan(0); + expect(typeof res.body.evaluatedAtEpoch).toBe("number"); + expect(Array.isArray(res.body.history)).toBe(true); + }); + + it("404s for an unknown address", async () => { + const res = await request(app.getHttpServer()) + .get("/api/v1/solvers/NOPE/reputation") + .expect(404); + expect(res.body.error).toBe("Solver not found"); + }); + + it("returns the trailing snapshot history respecting ?limit", async () => { + const resDefault = await request(app.getHttpServer()) + .get(`/api/v1/solvers/${ALPHA_ADDR}/reputation`) + .expect(200); + expect(Array.isArray(resDefault.body.history)).toBe(true); + + const resLimit = await request(app.getHttpServer()) + .get(`/api/v1/solvers/${ALPHA_ADDR}/reputation?limit=2`) + .expect(200); + expect(resLimit.body.history.length).toBeLessThanOrEqual(2); + }); + }); + + describe("GET /api/v1/solvers/leaderboard sort=reputation", () => { + beforeAll(async () => { + // Give ALPHA and BETA distinct event profiles via intents so the + // reputation ordering is deterministic for the sort check. + const intents = app.get(IntentsService); + const all = await intents.getAll(); + if (all.length >= 2) { + const now = Math.floor(Date.now() / 1000); + // First two intents go to ALPHA as perfect fills. + // eslint-disable-next-line @typescript-eslint/no-explicit-any + await intents.update(all[0].intentId, { + solver: ALPHA_ADDR, + state: "filled", + filledAt: now, + fillAmount: "5000", + acceptedAt: now - 60, + deadlineAt: now + 240, + amountInUsd: 5000, + } as any); + // eslint-disable-next-line @typescript-eslint/no-explicit-any + await intents.update(all[1].intentId, { + solver: ALPHA_ADDR, + state: "filled", + filledAt: now - 10, + fillAmount: "7500", + acceptedAt: now - 10 - 45, + deadlineAt: now - 10 + 255, + amountInUsd: 7500, + } as any); + } + }); + + it("returns sort=reputation in the payload body", async () => { + const res = await request(app.getHttpServer()) + .get("/api/v1/solvers/leaderboard?sort=reputation") + .expect(200); + expect(res.body.sort).toBe("reputation"); + expect(res.body.window).toBeDefined(); + }); + + it("rejects an invalid sort key", async () => { + await request(app.getHttpServer()) + .get("/api/v1/solvers/leaderboard?sort=oops") + .expect(400); + }); + + it("when sort=reputation, entries are non-increasing by reputationScore", async () => { + const res = await request(app.getHttpServer()) + .get("/api/v1/solvers/leaderboard?sort=reputation&window=all") + .expect(200); + const scores = res.body.solvers.map( + (s: { reputationScore: number }) => s.reputationScore, + ); + for (let i = 1; i < scores.length; i++) { + expect(scores[i - 1] + 1e-9).toBeGreaterThanOrEqual(scores[i]); + } + }); + + it("default sort=fills still works (backward compat)", async () => { + const res = await request(app.getHttpServer()) + .get("/api/v1/solvers/leaderboard") + .expect(200); + expect(res.body.sort).toBe("fills"); + }); + + it("sort=reputation + window=24h combines without 500ing", async () => { + const res = await request(app.getHttpServer()) + .get("/api/v1/solvers/leaderboard?sort=reputation&window=24h") + .expect(200); + expect(res.body.window).toBe("24h"); + expect(res.body.sort).toBe("reputation"); + }); + }); + + describe("GET /api/v1/solvers/:address/stats uses the new reputation formula", () => { + it("stats now exposes reputationComponents alongside the score", async () => { + const res = await request(app.getHttpServer()) + .get(`/api/v1/solvers/${GAMMA_ADDR}/stats`) + .expect(200); + expect(typeof res.body.reputationScore).toBe("number"); + expect(typeof res.body.reputationComponents).toBe("object"); + expect(typeof res.body.reputationComponents.fillRate).toBe("number"); + }); + }); +});