dataset-curation

Open source SDK for building multimodal data-quality, processing, enrichment, and curation pipelines for robotics and Physical AI.

Content for workshops on computer vision @ HPI's AI Service Center

HyperView curates datasets and provides model introspection in hyperbolic and Euclidean geometries.

Implementation of TSDS: Data Selection for Task-Specific Model Finetuning. An optimal-transport framework for selecting domain-specific and task-specific training data to improve LLM finetuning and instruction tuning.

Harnessing Large Language Models for Curated Code Reviews

Turn internet-scale video into egocentric and exocentric training data, curated and annotated by agents on the Memories.ai datalake

MedPMC: a high-fidelity medical image–text curation framework from biomedical literature for multimodal foundation models, with released pipelines, curated data, benchmarks, and pretrained models.

Image description/tagging tool

Annotate any LeRobot dataset with a swarm of agents. Turn raw demos into training-ready data for robotic foundation models.

[ACL 2024 (Findings)] ICC: Quantifying Image Caption Concreteness for Multimodal Dataset Curation

Semantic pipes for your terminal - PDFs, images, audio, video, and text through Unix verbs that understand

This is my personal home rig for serious LLM experimentation. I built it to test models head-to-head, create custom evaluation rubrics, automatically improve prompts based on the previous run’s results, and generate high-quality synthetic training data. Everything runs locally first (Ollama by default), with optional cloud support. logged locally.

NAACL 2025 | How Can We Diagnose and Treat Bias in Large Language Models for Clinical Decision-Making?

An image deduplication GUI, made for image generation models dataset deduplication using CLIP.

AI-assisted curation and organization for large media datasets

Dynamic cluster-based data sampling for efficient and long-tail-aware vision-language model pre-training.

Turn real traffic/traces into a versioned, deduped, coverage-tracked LLM regression suite — a curation accelerator with a human-confirmed golden gate.

Your dataset discovery and curation buddy.

Multi-axis VLM-as-judge data curation for robot demonstrations. Five specialist critics (visual / kinematic / task / strategy / safety) score every LeRobot episode with rationale and timestamp evidence. Model-agnostic — any OpenAI-compatible vision endpoint.

Pipeline for querying and turning NASA's ADS publications metadata into curated, analysis-ready datasets, topic maps, and citation networks.