Skip to content

fix(recall): stabilize equal-score search ranks - #293

Merged
dacorvo merged 3 commits into
huggingface:mainfrom
voropaevv:fix/stable-recall-search-ties
Nov 12, 2026
Merged

dacorvo merged 3 commits into
huggingface:mainfrom
voropaevv:fix/stable-recall-search-ties

Conversation

@voropaevv

Copy link
Copy Markdown
Contributor

Summary

Reciprocal rank fusion uses each hit's position in the vector and BM25 results. Equal search scores can arrive in either order and change which hit ranks first.

Sort the returned vector hits by distance and the BM25 hits by descending score, using row ID to break ties in both lists before fusion. The regression swaps tied rows across batches and checks that the top fused hit stays the same. Lance still determines which rows make it into each search's candidate set.

Fixes #292.

Verification

  • The tie regression failed on the parent and passed with this change.
  • Formatting, Clippy for the default and ONNX builds, and the local test suite passed on macOS arm64. The suite skipped seven authenticated Hub scenarios as expected without HF_FUNES_TEST_TOKEN.
  • The public memory at the revision in recall: copies of one passage swap places from run to run #292 produced one output per setting across 12 fresh CLI processes each at -k 30 and -k 8. The parent was also stable in this local sample, so the regression controls the input order directly.

Developed with Codex assistance.

voropaevv and others added 3 commits November 12, 2026 09:01
Lance returns each search best first, so only the order within a run
of equal scores is left to settle. Sorting those runs by row id needs
no score direction, so the comparator each search passed goes away.

The test picked its comparator by column name, apart from the one each
search passed, so a flipped comparator in fts_candidates would still
have passed it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nearest and full_text project their own score column, and
search_scores reads it back from either one. recall named Lance's
_distance and _score columns, which belong with the Lance mechanics,
like _rowid behind row_ids. So hit_cols and the two searches in recall
are back to what main has.

The projection is still needed. Lance 13 adds the score column on its
own today, but warns that it will stop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dacorvo
dacorvo merged commit 9fe7ddb into huggingface:main Nov 12, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

recall: copies of one passage swap places from run to run

2 participants