Bug
The idle-guard condition in the automatic batch flusher uses or instead of and, causing premature flushes in batches that contain only objects (no references) or only references (no objects).
Affected files and lines:
weaviate/collections/batch/sync.py ~line 180
weaviate/collections/batch/async_.py ~line 228
weaviate/collections/batch/base.py ~line 437
The condition reads something like:
if len_o == len(self.__batch_objects) or len_r == len(self.__batch_references):
# flush — nothing changed
In an objects-only batch (no references being added), len_r is always 0 and len(self.__batch_references) is always 0, so the right-hand side of or is always True. This means the idle-guard fires after every sleep tick regardless of whether objects have accumulated — the batch flushes prematurely before reaching batch_size, degrading throughput.
Expected behavior
The guard should use and — a flush is "idle" (nothing changed) only when both the object count and the reference count are unchanged:
if len_o == len(self.__batch_objects) and len_r == len(self.__batch_references):
# truly idle — flush
Impact
- Objects-only batches never accumulate to
batch_size; every sleep tick triggers a flush
- Effective batch size is 1 (or whatever accumulated in a single sleep interval) instead of the configured limit
- Throughput scales with sleep interval, not
batch_size
Bug
The idle-guard condition in the automatic batch flusher uses
orinstead ofand, causing premature flushes in batches that contain only objects (no references) or only references (no objects).Affected files and lines:
weaviate/collections/batch/sync.py~line 180weaviate/collections/batch/async_.py~line 228weaviate/collections/batch/base.py~line 437The condition reads something like:
In an objects-only batch (no references being added),
len_ris always0andlen(self.__batch_references)is always0, so the right-hand side oforis alwaysTrue. This means the idle-guard fires after every sleep tick regardless of whether objects have accumulated — the batch flushes prematurely before reachingbatch_size, degrading throughput.Expected behavior
The guard should use
and— a flush is "idle" (nothing changed) only when both the object count and the reference count are unchanged:Impact
batch_size; every sleep tick triggers a flushbatch_size