Train prompt-conditioned LoRA generators with behavioral supervision. Instead of only matching LoRA weights via MSE (as in DnD), the generator is trained to match delta activations: the difference in hidden states between base and adapted models.
L = λ_w * L_weight + λ_d * L_delta
Δ(M, B) = v(M) - v(B)
v(M) = E[h^(L)_M(p, last_token)] for p in P (probe texts)
The generator learns to produce LoRA weights that induce the same behavioral shift as the teacher adapters, not just numerically similar weights.
Computing Δ for a generated LoRA needs forward passes through the base model on every probe, which is expensive. Delta-guided training adds a cheap delta head that predicts Δ directly from the condition embeddings, and grounds it against the real Δ periodically:
Text Prompt ──► [Text Encoder] ──► condition
│
[LoRA Generator]
│ │
LoRA weights [Delta Head]
│ │
(every N steps) δ_predicted
▼ │
δ_computed ◄────┘
L = λ_pred * MSE(δ_pred, δ_teacher) every step
+ λ_computed * MSE(δ_computed, δ_teacher) ┐ every N steps
+ λ_consistency * MSE(δ_pred, δ_computed) ┘
This cuts base-model forward passes by roughly N× while keeping the generated weights grounded in real behavior.
- DnD's tokenize→detokenize roundtrip keeps LoRA weight error < 1e-3.
- Delta norms are ~0.1 and cluster by domain in t-SNE.
- All LoRA tensors receive gradients through
FunctionalLoRAhooks. - Delta-only and delta-guided training both converge.
Text Prompts
↓
[PretrainedTextEncoder] → condition embeddings (384-dim, MiniLM-L6-v2)
↓
[LoRAGenerator] → LoRA weights (A, B matrices per layer)
↓
[FunctionalLoRA] → apply LoRA via forward hooks (no weight cloning, gradient-enabled)
↓
[compute_delta_*] → Δ_pred via probe texts
↓
[MultiTaskLoss / DeltaGuidedLoss]
uv venv && source .venv/bin/activate
uv pip install -r dnd_repo/requirements.txt
uv pip install peft trl datasets sentence-transformers pandas scikit-learn matplotlib seabornAll scripts run from the repo root as modules and take --help.
# 1. Train teacher adapters (non-overlapping data batches) and compute their deltas
python -m scripts.train_lora_adapters --tasks arc_e boolq piqa
# 2. Train a generator
python -m scripts.train_generator --loss weight # raw (A, B) MSE baseline
python -m scripts.train_generator --loss delta_w # MSE on B@A*scaling (gauge-invariant)
python -m scripts.train_generator --loss multitask # weights + probe deltas
python -m scripts.train_generator --loss delta_only # probe deltas only
python -m scripts.train_delta_guided # dual-head generator, 3-part loss
# 3. Ablation: delta_guided vs multitask vs weight_only, 3 seeds each
python -m scripts.run_ablations --evalOutputs go to outputs/<run>/ (config.json, results.json, training_curves.png, checkpoints). Task accuracy is evaluated when data/{task}_eval.json files exist.
llgbm/
├── probes.py # create_generic_probes(), create_domain_probes()
├── delta.py # compute_base_activation(), compute_adapter_delta(), DeltaCache
├── dataset.py # RealAdapterDataset, DeltaAugmentedDataset
├── functional.py # FunctionalLoRA, compute_delta_differentiable()
├── losses.py # WeightLoss, DeltaWLoss, MultiTaskLoss, DeltaOnlyLoss, DeltaGuidedLoss
├── training.py # TrainingConfig, train(), evaluate()
├── generator.py # LoRAGenerator, LoRAGeneratorWithDeltaHead
├── text_encoder.py # PretrainedTextEncoder, TrainableTextEncoder
├── evaluation.py # compute_accuracy_with_lora_batched(), compute_base_accuracy()
└── ablations.py # AblationConfig, run_ablations()
scripts/ # entry points (see Usage)
results/ # saved ablation results
| Config | Qwen2.5-0.5B-Instruct (default) | Qwen2.5-1.5B (--large_model) |
|---|---|---|
| Layers | 24 | 28 |
| Hidden | 896 | 1536 |
| Intermediate | 4864 | 8960 |
| LoRA rank / alpha | 16 / 32 | 16 / 32 |
LoRA targets: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
data/
├── ARC-e_train.json, BoolQ_train.json, ... # {prompt, response[, system]} lists
└── {task}_eval.json # optional, for task accuracy
checkpoints/
├── manifest.json # adapter index
├── {task}/{task}_{idx:03d}/adapter/
│ ├── adapter_model.safetensors
│ ├── prompts.json
│ └── metrics.json
└── deltas/
├── delta_manifest.json
├── base_activation.npy
└── {adapter}_delta.npy
- Drag-and-Drop LLMs (DnD): base framework
- Qwen2.5 series: target model architecture