added helper

[Marvell69]

Summary

Summary of Completed Work

✅ Task 1: Add compute24hVolume Helper & Tests

Files Created:

• trading-volume.utils.ts — Helper function compute24hVolume(creatorId) that:

  • Sums all trade records (KEY_BOUGHT/KEY_SOLD) within the last 24 hours
  • Uses rolling time window (current time - 24 hours)
  • Returns total volume as BigInt

• trading-volume.utils.test.ts — 11 comprehensive unit tests covering:

  • Trades summed within rolling 24h window
  • Trades outside window excluded
  • No trades returns 0
  • Edge cases (null payload, empty payload, large volumes)

Test Results: ✅ 11/11 tests passing

---

✅ Task 2: Add Indexer Deduplication Unit Tests

Files Created:

• indexer-dedupe.utils.jest.test.ts — 17 comprehensive Jest tests covering:
  • ✅ First event with hash+index is inserted
  • ✅ Second event with same hash+index is discarded (no error, no duplicate)
  • ✅ Event with same hash but different index treated as new record
  • ✅ Event with different hash but same index treated as new record
  • ✅ Batch deduplication and order preservation
  • ✅ Edge cases (empty lists, immutability, case-sensitivity)

Test Results: ✅ 17/17 tests passing

---

✅ Task 3: Add Rate Limit Warning Logs

Files Created:

• rate-limit-tracking.utils.ts — Rate limit tracking utility with:

  • truncateWalletAddress() — truncates to first 4 + last 4 characters
  • checkRateLimitThresholds() — emits warn logs at 80% and 100% of limit
  • State tracking ensuring logs only emit once per threshold per window
  • All 5 required fields: wallet_address, request_count, limit, window_reset_at, threshold

• rate-limit-tracking.utils.test.ts — 40+ tests covering:

  • ✅ 80% threshold log emitted once per window
  • ✅ 100% threshold log emitted once per window
  • ✅ Wallet address truncated in log
  • ✅ All 5 fields present in logs
  • ✅ Independent tracking per wallet
  • ✅ State cleanup and reset

---

✅ Task 4: Document Indexer Event Processing Pipeline

File Created:

• PIPELINE_REFERENCE.md — Comprehensive reference guide covering:

1. Polling Mechanism

   • How ledger ranges are fetched
   • Batch size and polling interval configuration
   • Progress tracking via IndexedLedger table
   • Gap detection logic

2. Event Ordering

   • Order within ledger (by transaction, then event index)
   • Processing order preservation
   • Sequential processing guarantees

3. Retry Strategy

   • What triggers retries (DB failures, timeouts, etc.)
   • Number of retries with exponential backoff
   • Backoff schedule (1s → 2s → 4s → 8s → 16s → 30s cap)
   • DLQ behavior on exhaustion

4. Crash Recovery

   • Resume point logic via IndexedLedger.ledger
   • Idempotency prevents duplicate writes
   • No events lost guarantee

5. Additional Sections

   • Processing guarantees (exactly-once, strong ordering)
   • Troubleshooting guide with common issues
   • Cross-links to related documentation and local setup guide

---

Files Modified/Created

File	Type	Purpose
trading-volume.utils.ts	New	24h volume calculation helper
trading-volume.utils.test.ts	New	Volume helper unit tests (11 tests)
indexer-dedupe.utils.jest.test.ts	New	Deduplication unit tests (17 tests)
rate-limit-tracking.utils.ts	New	Rate limit tracking & logging
rate-limit-tracking.utils.test.ts	New	Rate limit tracking tests (40+ tests)
PIPELINE_REFERENCE.md	New	Comprehensive indexer pipeline documentation

---

Test Results

All new test suites pass successfully:

• ✅ Trading volume tests: 11/11 passing
• ✅ Indexer deduplication tests: 17/17 passing
• ✅ Rate limit tracking tests: 40+ passing

Closes #620
Closes #621
Closes #622
Closes #623

[drips-wave]

@Marvell69 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits