Skip to content

feat: merge selected audits into benchmark results - #1225

Merged
yakew7 merged 1 commit into
yakew7:mainfrom
dxnnystyles:feat/1203-selective-benchmark-merge
Oct 29, 2026
Merged

yakew7 merged 1 commit into
yakew7:mainfrom
dxnnystyles:feat/1203-selective-benchmark-merge

Conversation

@dxnnystyles

Copy link
Copy Markdown
Contributor

Summary

adds --merge so selected audits can be rerun into existing benchmark results without replacing unrelated audit rows. preserves prior provenance runs, appends the new run, refreshes file hashes, and rolls back on ordinary write failures and catchable interruptions.

Type

  • Audit
  • Explainer
  • Bug fix
  • Other

Audit checklist

N/A — no fairness audit added or changed.

Explainer checklist

N/A — no explainer added or changed.

Linked issue

Closes #1203

Verification

  • Focused merge/provenance tests: 23 passed, including CLI, plots, hashes, rollback, and simulated KeyboardInterrupt recovery.
  • make test-fast: 1,226 passed, 33 skipped, 27 deselected.
  • make lint: passed.
  • git diff --check: passed.
  • Slow tests were not run; hosted CI will provide separate results.

Limitations

the replacement of multiple output files isn't fully atomic. forced termination, power loss, or rollback failure can leave partial results. provenance hash validation detects inconsistent CSV hashes and blocks further merging, but it isn't automatic recovery; restoring a complete known-good results/provenance set or rerunning into a fresh output directory may be necessary.

@vercel

vercel Bot commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

@dxnnystyles is attempting to deploy a commit to the yashkewlani2020-gmailcom's projects Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions

Copy link
Copy Markdown

@yakew7 @Shreyash0712 @ahmdkaml - new PR touching a path you own, please review.

@yakew7
yakew7 merged commit c970f27 into yakew7:main Oct 29, 2026
19 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

faircode benchmark cannot rerun a subset of audits into an existing results/ directory

2 participants