OfferGuard

Is that job offer real? Paste a job posting, recruiter email, or offer letter and get a scam verdict with cited, verifiable evidence — deterministic red-flag rules, similar labeled cases retrieved by embedding similarity, and an LLM whose every claim is schema-validated and quote-verified.

Try it live: https://offerguard-seven.vercel.app · API health: https://offerguard-api-sh9z.onrender.com/api/v1/health · Live eval numbers: /methodology

Recruitment scams are a growing, measurable problem: the FTC reports $750.6M lost to business/job-opportunity scams in 2024, and the FBI's IC3 logged ~20K employment-scam complaints worth $264.2M, with task scams up 400%. Existing tools are checklists or employer-side — OfferGuard protects the job seeker.

Measured, not claimed

First production run (2026-10-03, 450 held-out cases, deployed stack):

Eval	n	Result
E1 Kaggle holdout	449	precision 0.981 · false-positive rate 0.003 (1 of 300 legitimate postings flagged); recall 0.356 as LIKELY_SCAM + 0.20 abstained to SUSPICIOUS — precision-first by design
E2 manual holdout	2	2/2 correct
E3 adversarial injections	20	20/20 verdicts unchanged · 19 attacks reported as red flags

Full confusion matrix and definitions: docs/eval-results.md. Re-run everything via POST /api/v1/eval/run.

How it works

┌──────────────────────────┐        ┌─────────────────────────────────────┐
│  apps/web  (Vercel)      │  REST  │  apps/api  (Render)                 │
│  Next.js 15 / React 19   │──────▶│  Express 5 / Node 20 / TypeScript   │
│  TypeScript, Tailwind    │  JSON  │  rules → retrieval → LLM pipeline   │
│  /  /methodology  /recent│        │  helmet · CORS · rate limit · pino  │
└──────────────────────────┘        └───────┬─────────────────────────────┘
                                            │ SQL (Drizzle) + pgvector
                                    ┌───────▼─────────────────────────────┐
                                    │  Neon Postgres 16 + pgvector (HNSW) │
                                    │  cases · postings · verdicts ·      │
                                    │  rule_hits · retrieval_log · eval   │
                                    └─────────────────────────────────────┘

Three independent layers, composed by a pure, unit-tested function — the LLM advises, it never decides:

1. Deterministic rules — 14 weighted red-flag rules (fees, crypto/wire payment, IM-only contact, domain mismatches, identity overreach, urgency, too-good pay, check overpayment, money-mule duties, spam formatting, vagueness). Every hit carries a ±40-char evidence excerpt.
2. Retrieval over labeled cases — the posting is embedded (Fireworks qwen3-embedding-8b, Matryoshka-truncated to 1536 dims) and compared by cosine kNN (pgvector HNSW) against a labeled corpus (Kaggle EMSCA + a modern FTC/IC3-pattern manual set); the nearest scam and legit cases ground the verdict.
3. LLM verdict (advisory) — Kimi K3 (kimi-k3-fast) via Fireworks with strict json_schema structured output, temperature 0, prompted with untrusted-data framing. Every evidence_quote must appear verbatim in the posting; failures are dropped and counted as hallucination events. Both AI layers sit behind one AiService interface — the Anthropic Haiku + OpenAI embeddings pair it was built on is one env change away (ADR-009).

composeTier() then applies deterministic overrides: 2+ hard rules force LIKELY_SCAM, 1 floors at SUSPICIOUS, an uncorroborated LIKELY_SCAM is downgraded (precision guard), and the LLM being unavailable degrades gracefully to a rules-only verdict — the product never hard-fails on a provider.

Codebase map

Curious where to look first?

Area	Where it lives
Pipeline orchestrator	apps/api/src/pipeline/analyze.service.ts — the whole analyze flow in one readable file
Rule engine (14 rules)	packages/shared/src/rules/
AI providers	apps/api/src/ai/ (Fireworks pair + retained Anthropic/OpenAI pair, one interface)
Schema & retrieval	packages/db/ — DDL, Drizzle mirror, pgvector cosine kNN
REST routes	apps/api/src/routes/
Frontend	apps/web/ (analyzer, verdict view, methodology, recent)

Quickstart

docker compose up -d          # local Postgres 16 + pgvector on :5434
pnpm install
pnpm db:apply                 # idempotent DDL (tables + HNSW index)
pnpm corpus:seed              # rules_catalog + manual corpus (Kaggle CSV optional)
pnpm --filter @offerguard/api dev       # API on :4000 (degraded without AI keys)
pnpm --filter @offerguard/web dev       # web on :3000

For the full pipeline set FIREWORKS_API_KEY (or the ANTHROPIC_API_KEY + OPENAI_API_KEY pair — see apps/api/.env.example), place the Kaggle EMSCA CSV at packages/corpus/data/fake_job_postings.csv, re-run pnpm corpus:seed, then pnpm corpus:embed to backfill embeddings.

Gates: pnpm lint && pnpm typecheck && pnpm test — 130 tests across 5 packages (integration tests need DATABASE_URL; CI runs them against a pgvector service container).

Security posture

• Prompt injection (the differentiator): untrusted-data framing + <posting> delimiters (stripped from input first) · temperature 0 · strict structured-output schema · deterministic tier overrides independent of the LLM · verbatim-quote verification · an adversarial eval set that must not flip verdicts — and doesn't (20/20).
• SSRF: no URL fetching, by design.
• PII: public /recent excerpts are email/phone-redacted; pino logs redact request bodies; raw text lives only in postings.
• Abuse: 64KB body cap, 20K-char input cap, per-IP rate limits (10/min analyze, 60/min rest), JSON-only POSTs.

Limitations (honest v1)

Paste-only (no URL fetching), English-only, no user accounts (anonymous, hash-deduplicated — auth is the documented v2). Educational tool — not legal or financial advice. Free-tier hosting means cold starts; hit /api/v1/health to warm before demos. Recall on 2012-era Kaggle scams is constrained by design choices that maximize precision; see eval-results.md for the tradeoff analysis.

Repository layout

apps/
  web/                    Next.js 15 (App Router, TS strict, Tailwind 4, motion)
  api/                    Express 5 — routes, pipeline, ai providers, repo, eval runner
packages/
  shared/                 DTOs, Zod schemas, 14-rule engine, composeTier (pure fns)
  db/                     Drizzle schema + idempotent DDL + pgvector retrieval
  corpus/                 Kaggle ingest, manual cases, seeded splits, embed backfill, eval metrics
docs/
  architecture.md         design spec (the authoritative document)
  adr/                    9 decision records
  eval-results.md         measured eval tables (first production run recorded)
design-system/offerguard/ persisted design tokens (MASTER.md)

Full design spec: docs/architecture.md · decision records: docs/adr/

License

MIT © Karthik Sethuraman