feat(config): add runtime property reindex support (Weaviate v1.39 GA API) - #2098
Open
g-despot wants to merge 12 commits into
Open
feat(config): add runtime property reindex support (Weaviate v1.39 GA API)#2098g-despot wants to merge 12 commits into
g-despot wants to merge 12 commits into
Conversation
Adds collection.config methods for the GA runtime property reindex API:
- update_property_index: declarative create-or-migrate upsert (PUT
/schema/{class}/properties/{prop}/index/{indexType}) with optional
wait_for_completion polling of the index status endpoint
- rebuild_property_index: rebuild an existing index from scratch, with
tenant selection and optional wait_for_completion
- cancel_property_index_task: idempotent cancellation of a live task
- get_property_indexes: parse GET /schema/{class}/indexes into new
CollectionPropertyIndexes/PropertyIndexStatus read-side types
All methods raise WeaviateUnsupportedFeatureError below server 1.39.0.
Mock tests exercise the exact REST routes against a 1.39-advertising mock server (upsert 202/NO_OP, rebuild, cancel CANCELLED/NO_OP, tenants csv encoding, status parsing incl. coupled task entries) and assert WeaviateUnsupportedFeatureError against a 1.36 mock. Integration tests cover the searchable lifecycle, rangeFilters creation, coupled tokenization changes, multi-tenant selection and the async client, skipping below server 1.39.0.
- accept a bare string for the tenants argument of update_property_index and rebuild_property_index, normalizing it to a single-element list so it cannot be exploded into a per-character csv (export API precedent) - override to_dict on _PropertyIndexes/_CollectionPropertyIndexes so the nested dataclass lists serialize to JSON-compatible dicts (_CollectionConfig precedent) - drop the dead list branch for dataType in the index status parser - cover the ReindexFailedError/ReindexCanceledError wait paths of both update and rebuild, pin the empty request body of the rebuild/cancel mocks, and prove json.dumps(...to_dict()) round-trips
![orca-security-eu[bot]](https://avatars.githubusercontent.com/in/276244?s=60&v=4){kind=small}
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
 Passed |
Infrastructure as Code |  0  0  0  0 |
View in Orca |
| Passed |
SAST | 0 0 0 0 |
View in Orca |
| Passed |
Secrets | 0 0 0 0 |
View in Orca |
| Passed |
Vulnerabilities | 0 0 0 0 |
View in Orca |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds client support for Weaviate v1.39’s Runtime Property Reindex GA API by introducing new collection-config methods (sync + async), output models, and errors, plus mock and integration test coverage for the new endpoints and wait/poll behavior.
Changes:
- Add
update_property_index,rebuild_property_index,cancel_property_index_task, andget_property_indexesto the collection config executor, includingwait_for_completionpolling and version gating (>= 1.39.0). - Introduce new output dataclasses/enums for property index tasks/status listings and export them via
weaviate.outputs.config, plus new terminal exceptions for failed/canceled reindex tasks. - Add mock tests and integration tests covering routes/bodies, parsing, tenants encoding, version gating, and wait-path terminal failures.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| weaviate/outputs/config.py | Re-exports new property-index output types for public consumption. |
| weaviate/exceptions.py | Adds new exceptions for terminal reindex outcomes (failed/canceled). |
| weaviate/collections/config/sync.pyi | Adds typed sync stubs for the new property-index config methods (incl. overloads). |
| weaviate/collections/config/async_.pyi | Adds typed async stubs for the new property-index config methods (incl. overloads). |
| weaviate/collections/config/executor.py | Implements the new runtime property index endpoints + polling logic + version gating. |
| weaviate/collections/classes/config.py | Adds enums/dataclasses representing property-index task and status payloads. |
| weaviate/collections/classes/config_methods.py | Adds JSON->dataclass parsing helpers for the new property-index responses. |
| mock_tests/test_property_reindex.py | Adds mock tests for request/response shapes, tenants encoding, version gating, and wait-path errors. |
| integration/test_collection_config.py | Adds integration tests for the searchable/rangeFilters lifecycle and async variants (skipping <1.39). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2098 +/- ##
==========================================
+ Coverage 86.64% 88.49% +1.85%
==========================================
Files 300 303 +3
Lines 23172 23756 +584
==========================================
+ Hits 20077 21024 +947
+ Misses 3095 2732 -363 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Validate the tenants argument of update_property_index and rebuild_property_index as str | List[str] | None before the csv join so invalid input raises WeaviateInvalidInputError instead of a raw TypeError, matching the library's _validate_input idiom already applied to property_name/index_name. Document the WeaviateInvalidInputError in the affected docstrings and cover the validation with mock tests.
The static delete_property_index error prefix asserted a single cause
("Property may not exist") but the DELETE 422 also covers an invalid
index type and the in-flight-reindex mutation guard, whose server
message the prefix contradicted. Name the failed operation instead and
let the appended response body carry the cause, matching the file's
phrasing style. Pin the behavior with a mock test surfacing a
server-style 422 in-flight-reindex message.
Adds a PropertyIndexType str-enum (SEARCHABLE, FILTERABLE, RANGE_FILTERS) accepted alongside the IndexName literals on update_property_index, rebuild_property_index, cancel_property_index_task and delete_property_index (backward- compatible widening of the v1.36 signature). The value is normalized to its wire form at the top of each method so paths, params and error messages never render the enum repr. Route-equality mock tests pin that the enum and literal forms hit identical routes, incl. RANGE_FILTERS -> rangeFilters. Read-side status types keep plain literals/str for forward compatibility.
Documents on update_property_index that a tokenization change via the searchable index also retokenizes an existing filterable index as one coupled task (shared taskId) and thereby changes filter matching, with filterable as the target for bucket-only changes and cancellation applying to the whole task. Explains why PropertyIndexes.data_type is a plain str (primitives match the DataType enum, references carry the target collection name) and pins reference-property parsing with a mock test.
The RANGE_FILTERS docstring said "rangeable", the internal write-path alias the RFC deliberately keeps unsurfaced — say rangeFilters instead. Extends the route-equality parametrization with FILTERABLE and adds enum-vs-literal route cases for rebuild_property_index and cancel_property_index_task.
Renames the unreleased public types PropertyIndexType/State/Status/ Task/TaskStatus to InvertedIndexType/State/Status/Task/TaskStatus and PropertyIndexes/CollectionPropertyIndexes to PropertyInvertedIndexes/ CollectionInvertedIndexes, incl. private counterparts and parser names. The runtime reindex API only ever touches inverted indexes; a future vector reindex must not collide with these names. Method names, the released IndexName alias and the deliberately generic Reindex*Error exceptions are unchanged.
Tightens index_name to InvertedIndexType only on update_property_index, rebuild_property_index and cancel_property_index_task (all overloads and impls); delete_property_index keeps accepting the IndexName literals as released v1.36 API. Runtime leniency is preserved — the value is still normalized and validated as a str, so raw strings keep working and keep hitting the same routes (pinned by the literal legs of the route-equality mock tests, which sit outside the pyright scope).
The CI pin 1.39.0-rc.0-b41225e was a stale mid-review build of core #12252 that emitted an IN_PROGRESS task status which never shipped; pin 1.39.0-rc.1 instead. Parse the task status tolerantly — known values map to InvertedIndexTaskStatus, unknown values pass through as plain strings, since the spec declares the field an open-vocabulary string — and pin that with a mock test. Upgrade the coupled-tokenization test to the final join contract: an identical re-PUT while the task is in flight returns 202 with the existing taskId and STARTED, verified live against 1.39.0-rc.1 (post-completion re-PUT stays 200 NO_OP).
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (8)
weaviate/collections/config/sync.pyi:125
rebuild_property_indexis currently typed to only acceptInvertedIndexType, but the runtime accepts the canonicalIndexNameliterals too. Widenindex_nametoUnion[InvertedIndexType, IndexName]for both overloads to match the supported API surface.
def rebuild_property_index(
self,
property_name: str,
index_name: InvertedIndexType,
*,
weaviate/collections/config/async_.pyi:127
rebuild_property_indexis currently typed to only acceptInvertedIndexType, but the runtime accepts canonicalIndexNameliterals too. Widenindex_nametoUnion[InvertedIndexType, IndexName]for both overloads to match the supported API surface.
async def rebuild_property_index(
self,
property_name: str,
index_name: InvertedIndexType,
*,
weaviate/collections/config/async_.pyi:142
cancel_property_index_taskis currently typed to only acceptInvertedIndexType, but the runtime accepts canonicalIndexNameliterals as well. Widenindex_nametoUnion[InvertedIndexType, IndexName]to match runtime behavior and avoid forcing# type: ignoreat call sites.
async def cancel_property_index_task(
self, property_name: str, index_name: InvertedIndexType
) -> InvertedIndexTask: ...
weaviate/collections/config/sync.pyi:140
cancel_property_index_taskis currently typed to only acceptInvertedIndexType, but the runtime accepts canonicalIndexNameliterals as well. Widenindex_nametoUnion[InvertedIndexType, IndexName]to match the supported API and avoid# type: ignoreat call sites.
def cancel_property_index_task(
self, property_name: str, index_name: InvertedIndexType
) -> InvertedIndexTask: ...
weaviate/collections/config/executor.py:781
update_property_index/rebuild_property_index/cancel_property_index_taskacceptIndexNameliterals at runtime (see theelse index_namecast), but the public type signature here restrictsindex_nametoInvertedIndexType. This forces callers using the canonical string literals (e.g. "searchable") to use# type: ignore, and is inconsistent withdelete_property_index, which already advertisesUnion[InvertedIndexType, IndexName]. Consider widening these signatures (and their overloads + the .pyi stubs) toUnion[InvertedIndexType, IndexName]so the declared API matches the supported runtime behavior.
def update_property_index(
self,
property_name: str,
index_name: InvertedIndexType,
*,
weaviate/collections/config/sync.pyi:119
update_property_indexis currently typed to only acceptInvertedIndexType, but the API is also invoked throughout the repo with canonicalIndexNameliterals (e.g. "searchable"). Align the stub withdelete_property_indexand the runtime behavior by allowingUnion[InvertedIndexType, IndexName]forindex_namein both overloads.
This issue also appears in the following locations of the same file:
- line 121
- line 138
def update_property_index(
self,
property_name: str,
index_name: InvertedIndexType,
*,
weaviate/collections/config/async_.pyi:121
update_property_indexis currently typed to only acceptInvertedIndexType, but the API is also used with canonicalIndexNameliterals (e.g. "searchable"). Align the async stub withdelete_property_indexand the runtime behavior by allowingUnion[InvertedIndexType, IndexName]forindex_namein both overloads.
This issue also appears in the following locations of the same file:
- line 123
- line 140
async def update_property_index(
self,
property_name: str,
index_name: InvertedIndexType,
*,
weaviate/outputs/config.py:10
- The PR description and the implemented/exported naming don’t match: the description refers to
PropertyIndexTask/PropertyIndexStatus/CollectionPropertyIndexes, but this PR exportsInvertedIndexTask/InvertedIndexStatus/CollectionInvertedIndexes(and the executor/stubs/tests use those names). Please update the PR description (and any user-facing docs/release notes) to reflect the actual public API names, or rename consistently if thePropertyIndex*naming is still intended.
CollectionInvertedIndexes,
GenerativeConfig,
GenerativeSearches,
InvertedIndexConfig,
InvertedIndexState,
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2097
Adds client support for the Runtime Property Reindex GA API (Weaviate v1.39), per the RFC linked from #2097. The server side (weaviate/weaviate#12252) merged on 2026-07-22; CI runs against
1.39.0-rc.1.New API (sync + async)
collection.config.update_property_index(property_name, index_name: InvertedIndexType, *, tokenization=None, algorithm=None, tenants=None, wait_for_completion=False): declarative upsert (create the index or migrate its config). ReturnsInvertedIndexTask(202STARTEDwith task id, or200NO_OP); withwait_for_completion=Truepolls the status endpoint until the index reportsreadyand returns the finalInvertedIndexStatus(raisesReindexFailedError/ReindexCanceledErroron terminal failure). An identical re-submit while the task is in flight joins it:202STARTEDwith the existing task id.collection.config.rebuild_property_index(property_name, index_name, *, tenants=None, wait_for_completion=False): same-config rebuild.collection.config.cancel_property_index_task(property_name, index_name): cancels the in-flight reindex task on the index (CANCELLEDorNO_OP, idempotent).collection.config.get_property_indexes(): per-property, per-index status listing (CollectionInvertedIndexes), including migration progress, task ids, and in-flight target tokenization/algorithm.delete_property_indexkeeps its v1.36 behavior; itsindex_nameparameter is widened (backward-compatibly) toUnion[InvertedIndexType, IndexName].index_nameon the new methods takes the newInvertedIndexTypeenum (SEARCHABLE,FILTERABLE,RANGE_FILTERS); onlydelete_property_indexalso accepts the v1.36 string literals. The client always sends canonical index type values (filterable,searchable,rangeFilters).tenantsaccepts a single tenant name or a list. All new methods are gated on server >= 1.39.0 and raiseWeaviateUnsupportedFeatureErrorbelow that.A tokenization change submitted via the
searchableindex also retokenizes an existingfilterableindex (one coupled task, shared task id on both status entries); this contract is stated explicitly in the docstrings.Implementation notes
tools/stubs_regen.sh);wait_for_completionoverloads modeled on the export executor.InvertedIndexTask,InvertedIndexStatus,PropertyInvertedIndexes,CollectionInvertedIndexes) and enums (InvertedIndexType,InvertedIndexState,InvertedIndexTaskStatus) exported viaweaviate.classes.config/weaviate.outputs.config; new exceptionsReindexFailedError/ReindexCanceledError.InvertedIndexTask.statusis open-vocabulary (Union[InvertedIndexTaskStatus, str]): known values parse to the enum, unknown server values pass through as strings, matching the spec (the field has no enum there).data_typeon status entries staysstrbecause the endpoint also lists reference properties, whosedataTypeis the qualified target collection name.Testing
202+ task id and200NO_OPparsing, join semantics, cancel semantics, tenants encoding (including a bare-string regression test), version-gate errors, wait-path failed/cancelled raises, tolerant status parsing, and reference-propertydataTypehandling.NO_OPre-put, rebuild, cancel, delete), rangeFilters creation, coupled tokenization change with shared task id, multi-tenant scoping.semitechnologies/weaviate:1.39.0-rc.1: all reindex integration tests pass. This PR also bumps the CIWEAVIATE_139image from a stale pre-merge build to1.39.0-rc.1.Status
Draft until Weaviate 1.39.0 GA is released.