Skip to content

Commit abd96ca

Browse files
committed
fix: memory corruption due to accessing out of bounds output vector index
In numpy_scan.cpp, the OBJECT/STRING scan path uses pos = i + offset (the absolute row index in the source array) as the output vector index for setNull() when handling None or NaN values: ```cpp // BEFORE (buggy): outputVector->setNull(pos, true /* isNull */); // pos = i + offset ``` The output vector is sized for DEFAULT_VECTOR_CAPACITY (= 2048) entries and expects 0-based indices within each batch. But pos is the absolute dataframe row index. When the COPY spans multiple batches (i.e., when offset >= DEFAULT_VECTOR_CAPACITY), pos exceeds the null mask buffer, causing a heap buffer overrun → the segfault / free(): invalid next size / munmap_chunk(): invalid pointer crashes seen on Linux.
1 parent 2a28669 commit abd96ca

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

‎src_cpp/numpy/numpy_scan.cpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -172,7 +172,7 @@ void NumpyScan::scan(PandasColumnBindData* bindData, uint64_t count, uint64_t of
172172
!py::isinstance<py::str>(val)) {
173173
if (val == Py_None ||
174174
(py::isinstance<py::float_>(val) && std::isnan(PyFloat_AsDouble(val)))) {
175-
outputVector->setNull(pos, true /* isNull */);
175+
outputVector->setNull(i, true /* isNull */);
176176
continue;
177177
}
178178
if (!py::isinstance<py::str>(val)) {

0 commit comments

Comments
 (0)