Skip to content

INTERVIEW.md

Latest commit

 

History

History
64 lines (49 loc) · 3.13 KB

INTERVIEW.md

File metadata and controls

64 lines (49 loc) · 3.13 KB
Raw

Interview talk track (≈30 seconds)

The pitch

AdaptLoop closes the MLOps loop on top of my gateway. Production LLM signals land on Redpanda, Bytewax aggregates them into Postgres, we curate a holdout-safe Parquet set, fine-tune a LoRA adapter, gate promotion with lag and eval checks, register it in MLflow, route base_model + adapter_id back into LLMOps, and canary-watch Staging with automatic rollback if error rate or p95 blows up.

Resume arc: domain product → graph/knowledge AI → inference platform → closed-loop adaptation.

Live demo (what to run)

make interview

That script:

  1. Prints this talk track
  2. Shows Redis lag / aggregates
  3. Promotes an existing run to Staging (or uses smoke_lora)
  4. Injects a canary fault and shows auto-rollback

Full rebuild (2–4 min): make demo

Whiteboard (draw this)

[LLMOps traces]──┐
[Synthetic EPC]──┼──► Redpanda ──► Bytewax ──► Redis + Postgres
                 │                              │
                 │                              ▼
                 │                         MinIO Parquet
                 │                         (train / holdout / review)
                 │                              │
                 │                              ▼
                 │                         LoRA (PEFT) ──► MLflow
                 │                              │
                 └◄── adapter_routes ◄── promote gate ◄── canary

Four failure modes (expect these questions)

Question Answer
How do you stop train/serve skew? One shared normalization + chat template module; golden-file unit tests; same hash in curation and training.
What if production labels are poisoned? Pydantic validation; needs_review samples never enter train.parquet — they go to review_queue. Holdout never trains.
Won’t lag make you promote on stale data? Promote gate reads Redis produced − processed and refuses if lag > N.
What if the new adapter hurts live traffic? Staging + Redis SLO canary → auto-rollback; gateway also sticky-hashes canary_percent of alias traffic to Staging.
Do you only use exact match? No — holdout scores exact match, token F1, and an offline LLM-judge rubric (optional API judge).

Proof points to open on screen

Artifact Where
Event contract docs/EVENT_CONTRACT.md
SQL examples SELECT source, count(*) FROM training_examples GROUP BY source
Holdout split .artifacts/datasets/*/manifest.json
Adapter .artifacts/runs/*/adapter or MinIO adaptloop-checkpoints
Registry http://localhost:5001 — Staging / Archived after canary
Gateway route GET /v1/admin/adapter-routes (LLMOps) + X-Adapter-Id header

Scope honesty (sounds senior)

  • One domain (epc_qa), LoRA only — no full pretrain.
  • Smoke demos use a tiny HF model on CPU; production path is Qwen2.5-3B.
  • Gateway remaps alias → base model and echoes adapter_id; applying LoRA weights needs a custom inference backend (called out as follow-on).