This repository is the publishable snapshot of the PatchEval MVP, repository-memory pipeline, and recorded experiments as of 2026-08-16.
src/- repository-memory, PatchEval task, comprehension, behavioral evidence, and judgment modules.scripts/- reproducible data preparation, memory construction, prompt generation, and evaluation commands.tests/- unit and integration tests for the current pipeline.data/processed/- frozen MVP task inputs and derived labels/manifests.data/oracle/- Oracle patch mappings and patch records.data/memory/- Repository Structure Memory, Security Episode Memory, and Security Behavior Memory v2.data/reports/- intermediate reports, evidence packets, ablation artifacts, and experiment summaries.data/results/- recorded model and evaluation outputs.data/releases/- exported release artifacts.ptach_dataset/- source candidate-patch outputs and manual marker files. The directory name is preserved for source compatibility.docs/and资料/- design documents, experiment plans, and presentation/research materials.
The current candidate-patch labels are experiment snapshots. They are retained for reproducibility and comparison, but should not be treated as final ground truth until the planned manual reassessment is complete. Oracle records are also retained as source artifacts; some historical Oracle extractions may be incomplete and must be checked against their official commits before use as executable patch inputs.
The maintained implementation is project-owned. Model calls go through
src/patcheval_baseline/llm_provider.py, a small OpenAI-compatible
/chat/completions client. It accepts a provider-config JSON path or the
DEEPSEEK_API_KEY/DEEPSEEK_BASE_URL (or OpenAI equivalents) environment
variables; credentials are never committed.
The local Git clones under data/repos/ are intentionally excluded from this repository because they are approximately 1.26 GB and are environment-specific. Repository Memory artifacts and base-commit metadata are included. A fresh environment can reconstruct the clones with the repository URLs and base commits in the task and memory manifests.
From the repository root, the portable test suite is:
python -m pytest -q -k "not commit_mapping"The commit-mapping integration tests additionally require a working local Git installation and an isolated Git configuration. API credentials are never stored in this repository; model runs require the corresponding environment variables to be configured locally.