AdaptLoop

AdaptLoop — Event-driven ML adaptation platform: Redpanda ingest of production LLM traces, Bytewax stream aggregation, PostgreSQL/pgvector curation, LoRA fine-tuning with MLflow registry, and lag/canary-gated adapter promotion into a multi-tenant inference gateway.

Closes the loop across a portfolio stack: EPCdash (domain) → AEVAR (knowledge AI) → LLMOps (inference gateway) → AdaptLoop (continuous adaptation).

signals → stream → curated dataset → LoRA → gated promote → canary → rollback

Why it exists

Most “fine-tune notebooks” never meet production traffic. Most gateways never learn from their own traces. AdaptLoop is the missing MLOps loop: ingest live LLM signals, curate safely, train LoRA adapters, gate promotion, and route adapters back into the gateway — with interview-grade failure modes designed in.

Failure mode	Defense
Training/serving skew	Shared normalization + golden tests
Feedback poisoning	Schema validation; needs_review → review queue, never train
Stale promotion under lag	Lag-aware promote gate (produced − processed)
Canary regression	Staging watch on error_rate / p95 → auto-rollback
Bad generations	Offline LLM-judge rubric (optional API judge) in holdout eval
Risky full cutover	Gateway canary_percent sticky-hash split Staging vs Production

Quick demo (interview)

cp .env.example .env
make install          # once — Python 3.12 required (Bytewax)
make demo             # full loop; injects canary fault on purpose
make interview        # 30s talk track + prove rollback on existing run

Details: docs/DEMO.md · docs/INTERVIEW.md · docs/ARCHITECTURE.md

UIs while demoing: MLflow http://localhost:5001 · MinIO http://localhost:9001 (adaptloop / adaptloopsecret) · Redpanda Console http://localhost:8085

Stack

Layer	Choice
Event bus	Redpanda (Kafka API)
Stream processing	Bytewax
Hot state	Redis
Cold store	PostgreSQL + pgvector
Artifacts	MinIO (Parquet, adapters)
Training	HuggingFace + PEFT LoRA (Qwen2.5-3B; smoke uses tiny HF model)
Jobs	arq
Registry	MLflow stages
Promotion	Lag + eval + LLM-judge gates + canary rollback
Gateway	LLMOps adapter_routes (base_model + adapter_id + canary_percent)

Layout

adaptloop/
  schemas/events.py           # LLMSignalEvent contract
  normalization/              # train/serve skew defense
  producers/                  # gateway bridge + synthetic EPC
  consumers/                  # Bytewax → Redis + Postgres
  curation/                   # Parquet export + holdout + review queue
  training/                   # LoRA + holdout eval
  workers/                    # arq run_lora_finetune
  registry/                   # MLflow
  canary/                     # Staging watch + auto-rollback
  api/                        # promotion FastAPI + CLI
  integration/                # LLMOps admin client

Domain

EPC terminology Q&A (portfolio tie to EPCdash). One adapter domain in v1 — no multi-model scope creep.

Docs

• Architecture — system diagram + planes
• Event contract — llm.signals.v1
• Demo guide — make demo walkthrough
• Interview talk track — 30-second narrative + Q&A