Use server-side batching in the quickstart import step - #487
Merged
Conversation
Move every quickstart import step that has a one-shot server-side batching
equivalent to `data.ingest`, in Python and TypeScript.
Python: replaces `batch.fixed_size(...)` context managers with a single
`data.ingest(...)` call. Where vectors are supplied, objects are wrapped in
`DataObject` so the vector rides along.
TypeScript: replaces `data.insertMany(...)` with `data.ingest(...)`. The TS
client requires the `{ properties: ... }` object shape, so plain property
lists are mapped before the call.
Error handling: `data.ingest` has no context manager, so the old
`batch.failed_objects` / `batch.number_errors` guidance no longer applies.
The snippets now use the returned batch result. Python checks both
`has_errors` and `errors` because `has_errors` is not currently set on the
ingest path; TypeScript sets it correctly, so it checks `hasErrors`.
Go, Java and C# keep their current correct form: Go has no server-side
batching at all, and the Java and C# quickstarts already use the client-side
convenience method. Per-language asymmetry is acceptable, an invented API is
not.
The self-hosted quickstart gains a minimum server version note. Server-side
batching needs Weaviate 1.36 or later, and `data.ingest` has no client-side
version guard, so an older instance would fail with an opaque transport
error.
Snippets executed against Weaviate 1.38.0 with the CI-pinned clients.
- Import includes: name the API again in the Python tab, and give the TypeScript tab the error-handling prose it was missing. The shared "check the result" sentence now sits above the tabs, so it also covers the Go, Java and C# tabs. - Quickstart snippets: explain why the Python examples check `errors` directly, and label the TypeScript ingest call, matching the how-to. - cloud/quickstart.mdx, tutorials/quick-tour-of-weaviate.mdx: the batch tip claimed batching sends everything in a single request, which is no longer what the code beneath it does. Rewritten language-agnostically, and the doubled parenthesis and number disagreement are fixed. - quickstart/local.md: move the version caveat below the examples, switch it to the house `:::note Requirement: ...` pattern, and lead with the reassurance rather than the caveat.
Gate the Python import examples on result.errors alone. The has_errors flag is not set on the ingest path, so the Python tab prose now names result.errors as the check to read. The TypeScript tab keeps hasErrors, which does fire on that client. Also drop the version-floor note from the local quickstart.
…steps Java and C# were the last quickstart tabs still using client-side batching while the docs recommend server-side batching as the default. Java: `collection.data.insertMany(...)` becomes `collection.batch.start()`, which returns a `BatchContext`. The context is closed with try-with-resources so `close()` flushes the remaining objects and waits for the results, and `batch.numberOfErrors()` is read after the block, once the tally is complete. C#: `collection.Data.InsertMany(...)` becomes `collection.Batch.InsertMany(...)`. Both return a `BatchInsertResponse`, so the error handling keeps its shape and gains a per-entry loop. The `HasErrors` check moves inside the `END Import` marker so the docs tab renders it, matching the Java tab. Go keeps client-side batching because the Go client has no server-side batching API. The README examples are left alone. The shared line above the tabs now says a result reports whether any objects failed rather than which ones, because the Java tally is a count. The Java and C# tabs each gain a sentence naming their own result type.
The shared line above the tabs claimed every import "returns a result". That is false for Java: `batch.start()` hands back the `BatchContext` before any object is added, the try-with-resources block returns nothing, and the tally is read off a handle the reader already holds. It also primes the mental model that causes the premature read, because the only candidate for "the result the import gave me" is the object returned before the import ran. The line now says imports report whether any objects failed, without claiming a return value. The C# sentence said each entry carries "the `Error` that failed it", but `Error` is nullable and null on success, as the sample two lines below already shows with `.Where(o => o.Error is not null)`. It also left `Index` unnamed while the code prints it and the Python and TS tabs explain positional keying. The sentence now names `HasErrors`, `Errors`, `Objects`, `Index`, and `Error`, and says only failed entries carry an `Error`. The Java timing moves out of a trailing adverb into its own sentence, since reading `numberOfErrors()` before the batch closes is the one thing a reader can silently get wrong. In the C# samples, the comment above the failure loop now names `Index` and matches the Python and TS phrasing instead of saying "in the order they were sent", and the success count uses `insertResponse.Count` like the four sibling quickstart files rather than `Objects.Count()`.
Both comments described the same dictionary. Replace them with a single line above the check that says what `errors` holds and how it is keyed, matching the prose under the snippet.
![orca-security-eu[bot]](https://avatars.githubusercontent.com/in/276244?s=60&v=4){kind=small}
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
 Passed |
Infrastructure as Code |  0  0  0  0 |
View in Orca |
| Passed |
SAST | 0 0 0 0 |
View in Orca |
| Passed |
Secrets | 0 0 0 0 |
View in Orca |
| Passed |
Vulnerabilities | 0 0 0 0 |
View in Orca |
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the Weaviate quickstart “import objects” step (and its reused snippets) to use server-side batching APIs across client libraries, aligning examples and error-handling guidance with the newer ingest/batch flows.
Changes:
- Reworks quickstart prose to describe batch imports and server-side batching without client-specific claims.
- Updates Python/TypeScript examples to use
data.ingest(...)(with required TS property wrappers where needed) and adjusts failure-check guidance. - Updates Java and C# examples to use server-side batching (
collection.batch.start()/BatchContextin Java;collection.Batch.InsertMany(...)in C#).
Reviewed changes
Copilot reviewed 30 out of 30 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/weaviate/tutorials/quick-tour-of-weaviate.mdx | Updates quick-tour import-step wording and batch-import tip to reference server-side batching. |
| docs/cloud/quickstart.mdx | Mirrors the quickstart import-step prose updates for Cloud quickstart. |
| _includes/code/typescript/quickstart.short.local.import_vectors.create_collection.ts | Switches vector import to data.ingest(...). |
| _includes/code/typescript/quickstart.short.local.create_collection.ts | Switches import to data.ingest(...) with required { properties } wrappers. |
| _includes/code/typescript/quickstart.short.import_vectors.create_collection.ts | Switches vector import to data.ingest(...). |
| _includes/code/typescript/quickstart.short.create_collection.ts | Switches import to data.ingest(...) with required { properties } wrappers. |
| _includes/code/typescript/quickstart.import_objects.ts | Updates quickstart import to data.ingest(...) and adds per-object error reporting. |
| _includes/code/typescript/local.quickstart.import_objects.ts | Same as above for the local quickstart variant. |
| _includes/code/quickstart/quickstart.import_objects.mdx | Updates shared tabbed snippet prose for ingest/batch error handling across languages. |
| _includes/code/quickstart/local.quickstart.import_objects.mdx | Same as above for the local quickstart snippet wrapper. |
| _includes/code/python/quickstart.short.local.import_vectors.create_collection.py | Switches vector import to data.ingest(...) using DataObject. |
| _includes/code/python/quickstart.short.local.create_collection.py | Switches import to data.ingest(...) for local short quickstart. |
| _includes/code/python/quickstart.short.import_vectors.create_collection.py | Switches vector import to data.ingest(...) using DataObject. |
| _includes/code/python/quickstart.short.create_collection.py | Switches import to data.ingest(...) for hosted short quickstart. |
| _includes/code/python/quickstart.import_objects.py | Replaces manual batching with data.ingest(...) and updates failure checking to result.errors. |
| _includes/code/python/local.quickstart.import_objects.py | Same as above for the local quickstart variant. |
| _includes/code/llms-txt/typescript/quickstart.ts | Updates LLMs.txt TS quickstart import to data.ingest(...). |
| _includes/code/llms-txt/python/quickstart.py | Updates LLMs.txt Python quickstart import to data.ingest(...). |
| _includes/code/java-v6/src/test/java/QuickstartTest.java | Switches to server-side batching via batch.start() / BatchContext and updates error counting. |
| _includes/code/java-v6/src/test/java/QuickstartLocalTest.java | Same as above for the local Java quickstart test. |
| _includes/code/java-v6/src/test/java/quickstart/QuickstartLocalCreateVectors.java | Switches vector import to server-side batching (batch.start()). |
| _includes/code/java-v6/src/test/java/quickstart/QuickstartLocalCreate.java | Switches import to server-side batching (batch.start()). |
| _includes/code/java-v6/src/test/java/quickstart/QuickstartCreateVectors.java | Switches vector import to server-side batching (batch.start()). |
| _includes/code/java-v6/src/test/java/quickstart/QuickstartCreate.java | Switches import to server-side batching (batch.start()). |
| _includes/code/csharp/QuickstartTest.cs | Switches to Batch.InsertMany(...) and updates failure reporting. |
| _includes/code/csharp/QuickstartLocalTest.cs | Same as above for the local C# quickstart test. |
| _includes/code/csharp/quickstart/QuickstartLocalCreateVectors.cs | Switches vector import to Batch.InsertMany(...). |
| _includes/code/csharp/quickstart/QuickstartLocalCreate.cs | Switches import to Batch.InsertMany(...). |
| _includes/code/csharp/quickstart/QuickstartCreateVectors.cs | Switches vector import to Batch.InsertMany(...). |
| _includes/code/csharp/quickstart/QuickstartCreate.cs | Switches import to Batch.InsertMany(...). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| batch.add_object(properties=obj) | ||
| movies.data.ingest(data_objects) | ||
|
|
||
| print(f"Imported & vectorized {len(movies)} objects into the Movie collection") |
| batch.add_object(properties=obj) | ||
| movies.data.ingest(data_objects) | ||
|
|
||
| print(f"Imported & vectorized {len(movies)} objects into the Movie collection") |
g-despot
added a commit
that referenced
this pull request
Aug 1, 2026
llms.txt is hand-maintained in weaviate-io, but its code blocks must match the START/END regions of the snippets in this repo. The only enforcement was the weekly test_llms_txt_snippets_are_covered job, which fetches the live file. So PR #485/#487 moved the quickstart snippets to `data.ingest`, nothing here told the author that weaviate-io/static/llms.txt had to move too, and the break surfaced days later in a scheduled run. It is still red: 2 of 51 blocks. Add tests/check_llms_txt_drift.py, run by llms_txt_snippet_sync.yml on any PR touching a file that matches SNIPPET_GLOBS. It answers one question: once this PR merges, which llms.txt blocks would the weekly coverage test no longer find? It reuses that test's globs, regexes, _normalize and _load_llms_txt, so the two cannot disagree about what "matches" means. The check is advisory and always exits 0. When a snippet PR is opened, weaviate-io has not merged or deployed yet, so the live llms.txt legitimately cannot match; a blocking check would fail every honest PR and would be routinely overridden, which teaches people to ignore it. Findings are GitHub warning annotations on the changed marker lines plus a job summary carrying the exact block to paste into llms.txt. It stays quiet when there is nothing to do: weaviate-io shipped first and llms.txt already carries the new block, only scaffolding outside the markers changed, or a region moved or was renamed with its code intact. It degrades to a generic reminder, never a failure, if the base ref or llms.txt cannot be read. The path filter also covers the Java and C# snippets, which live with their language suites rather than under _includes/code/llms-txt/. llms_txt_tests.yml is untouched; the scheduled job behaves exactly as before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves the quickstart import step to server-side batching. Go is unchanged — the Go client has no server-side batching API.
data.ingest()collection.batch.start()/BatchContextcollection.Batch.InsertMany()Two things that look removable but are not. The TypeScript
.map((properties) => ({ properties }))wrappers are required, or properties are not persisted (weaviate/typescript-client#456). Andbatch.numberOfErrors()in Java must be read after the batch closes, not inside the try block, or it reports zero.Merge after #486.