Skip to content

feat: add high-performance HNSW RaBitQ index - #1

Draft
CLiqing wants to merge 2 commits into
mainfrom
codex/pr-hnsw-rabitq
Draft

feat: add high-performance HNSW RaBitQ index#1
CLiqing wants to merge 2 commits into
mainfrom
codex/pr-hnsw-rabitq

Conversation

@CLiqing

@CLiqing CLiqing commented Aug 27, 2026

Copy link
Copy Markdown
Owner

What this PR does

Adds an end-to-end HNSW_RABITQ index to Knowhere together with the embedded Faiss RaBitQ kernels required for competitive graph-search performance.

  • exposes 1-9 bit database codes for FP32/FP16/BF16 input
  • supports L2 and inner product search
  • builds the graph with exact FP32 storage, then attaches immutable rotated RaBitQ storage
  • supports request-local query quantization for the 1-bit path and optional refinement
  • adds a dense-packed multi-bit row layout for HNSW
  • adds single and batch-4 scalar/AVX2/AVX-512/Sapphire Rapids scorers
  • keeps BMI2 operations behind a runtime capability gate
  • supports search, range search, iterator, distance-by-id, and serialization round trips

Scope and limitations

This PR is intentionally scoped to HNSW. The dense-packed layout is selected by HNSW_RABITQ; DiskANN and IVF integration are not included.

The initial implementation does not support cosine, scalar partition builds, mmap, incremental add after construction, or direct graph construction from RaBitQ code-to-code distances.

Validation

  • Knowhere [hnsw_rabitq]: 6 test cases, 377,568 assertions passed
  • Faiss RaBitQ*: 16 SIMD correctness tests passed
  • Release knowhere_tests target built successfully on the latest main (a337f6a7)

@CLiqing
CLiqing force-pushed the codex/pr-hnsw-rabitq branch 3 times, most recently from e778c24 to aae6c99 Compare August 28, 2026 02:15
…ech#1799)

Bump pinned cardinal tags to the freshly cut releases:
- v2 line (master): v3.0.6 -> v3.0.7 (includes zilliztech#922 zilliztech#928 zilliztech#931 zilliztech#929)
- v1 line: v2.5.111 -> v2.5.112 (includes zilliztech#932)

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@CLiqing
CLiqing force-pushed the codex/pr-hnsw-rabitq branch 4 times, most recently from 89f7b5d to f78e4ac Compare September 2, 2026 09:24
Signed-off-by: ChenLiqing <23721160+CLiqing@users.noreply.github.com>
@CLiqing
CLiqing force-pushed the codex/pr-hnsw-rabitq branch from f78e4ac to c74c993 Compare September 2, 2026 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants