Tuition Post Filter

A webhook service that takes tuition posts (pasted text), uses an LLM to extract structured fields, applies my filtering rules in plain code, and stores everything in Postgres.

Target filter: grade 5+, salary 8000+ BDT, English medium, in Notun Bazar / Baridhara / Badda.

Flow

POST /webhook -> verify secret -> validate input -> duplicate check (hash) -> LLM extracts fields (retry once) -> validate LLM output -> apply rules (rules.js) -> save as match / no_match / needs_review, or failed with the error

Design decisions

• LLM extracts, code decides. Filtering rules are deterministic and testable (node test-rules.js).
• LLM output is validated before use, since it can be malformed or wrong.
• Incomplete posts go to needs_review instead of being dropped.
• Failures are stored with the error message, so nothing is silently lost.
• Duplicates are skipped via a hash of the normalized text, before spending an LLM call.
• Row Level Security is on, so the table is not exposed through Supabase's public API.
• Intake is manual (paste or forward a post) to stay within platform rules for private groups.

Setup

1. npm install
2. Run schema.sql in the Supabase SQL editor
3. Copy .env.example to .env and fill it in
4. node test-rules.js (rules test, no keys needed)
5. node extract.js (tests extraction on samples.json)
6. node server.js

Endpoints (send header x-webhook-secret)

• POST /webhook with { "text": "..." }
• GET /posts?status=match (also no_match, needs_review, failed)

Improvements I'd make next

• Background queue so the webhook responds immediately
• Exponential backoff and alerting on repeated failures
• Telegram bot for intake and match notifications
• Replace rejectUnauthorized: false with Supabase's CA certificate for production

Sample data in this repo is fake.