Repository navigation
perf: JIRA-17824 stream exports instead of building a spreadsheet - #116
Merged
Merged
Conversation
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Team Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The CSV driver built a complete PhpSpreadsheet workbook in memory to emit a plain CSV. That cost roughly 0.5 ms and 14 KB per row, and time per row grew as the working set did, so a large export needed gigabytes and slowed down as it went. Writing rows straight out is about 100 times faster and holds memory flat: 318,000 rows now takes 1.7 seconds and 8 MB, against a projected 165 seconds and 4.4 GB. Two correctness fixes come with it. The header used to be taken from the first row's keys while every row was written in its own key order, so any row whose keys differed silently put values under the wrong headers. The header is now the set of keys any row has, in the order first seen, and every row is written against it. Values are no longer coerced. A spreadsheet infers a type per cell, so a numeric-looking string lost its formatting and a leading = was read as a formula: 1234.50 was written as 1234.5, and =SUM(A1) as 0. A CSV holds text, so values now reach the file as given. BREAKING CHANGE: phpoffice/phpspreadsheet and guzzlehttp/guzzle are no longer required. toSpreadsheet() and toCsvWriter() are removed, and toStream() now takes the processor data and returns a stream resource rather than taking a writer. saveToStorage() takes a stream rather than a string. Exports containing numeric-looking values change: the file keeps what the processor produced instead of a coerced version. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
webhanspeter
force-pushed
the
JIRA-17824-stream-csv-exports
branch
from
September 2, 2026 15:09
f516517 to
1da8e83
Compare
This adds exceptions when the provided stream is not a resource.
owenvoke
force-pushed
the
JIRA-17824-stream-csv-exports
branch
from
September 3, 2026 08:59
7835ce6 to
b4553c7
Compare
owenvoke
approved these changes
Sep 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CsvDriverbuilt a complete PhpSpreadsheet workbook in memory in order to emit a plain CSV. Rows are now written straight to a stream.About 100x less time per row, and memory is flat — the buffer is capped and spills to a temp file beyond it, so it no longer scales with the export. Note the old figures get worse per row as the set grows (0.338 to 0.523 ms) because the working set runs into gigabytes; the new ones do not.
Along the way the driver stopped needing two dependencies, and two correctness bugs fell out.
The header only described the first row
toSpreadsheet()took the header fromarray_keys($data->first())butfromArray()wrote each row in that row's own key order. Any row whose keys differed in order or membership silently put values under the wrong headers — wrong data, no error.The header is now the set of keys any row has, in the order first seen, and every row is written against that list. A missing key writes an empty field instead of shifting everything left.
Values were being coerced
A spreadsheet infers a type per cell.
DefaultValueBinderdecided a numeric-looking string was a number andCell::setValueExplicit()did$this->value = 0 + $value, at which point the text was gone. A leading=was parsed as a formula and the calculated result written instead.8.0088.001234.501234.51234.500.100.10.10=SUM(A1)0=SUM(A1)+45124512+45121e310001e3A CSV holds text, so nothing is inferred any more. Leading zeros and long integers were already safe — the binder has explicit guards for those — but trailing decimal zeros were not.
There is a second coercion site in the same library worth knowing about: the CSV writer separately strips trailing zeros from decimals beyond 15 digits of precision. Also gone.
Quoting is unchanged, deliberately
fputcsvquotes any field containing a space, which the old writer withsetEnclosureRequired(false)did not — that would have put quotes around most text columns in every row. So rows are assembled directly, enclosing only on the delimiter, the enclosure, or a newline, which is whatWriter\Csvdid:A bare carriage return is also enclosed now, since rows are separated by one. Any value containing one previously produced a broken file.
Verified by byte comparison
Ran a realistic export through both writers — 21 columns, values including a comma, embedded quotes, a padded identifier and decimals — and diffed the files. Exactly two fields differ, both the fidelity fix:
Quoting, column order, line endings and every other value are byte-identical.
Test plan
Four new tests: value fidelity, header alignment across rows with differing keys, quoting only where required, and empty input.
Existing driver tests unchanged in expectation, including the exact
"name\r\nJohn Doe\r\nJane Doe\r\n"assertion.Full suite 24 passed, 73 assertions.
ECS and PHPStan clean.
Suite re-run with both dependencies uninstalled, to prove neither is still needed.
Rector clean. (It used to fail on a clean checkout under coding-style 4.x, which registered the deprecated
ArraySpreadInsteadOfArrayMergeRector; the 5.x bump now onmainfixed that, so it runs properly again.)Breaking changes
Needs a major release.
phpoffice/phpspreadsheetandguzzlehttp/guzzleare no longer required. Neither is referenced anywhere insrc/any more — guzzle existed only fortoStream()'s return type.toSpreadsheet()andtoCsvWriter()are removed. Both took or returned PhpSpreadsheet types, and the package has never produced a spreadsheet:GeneratorTypehas one case,CSV, andGeneratorManagerregisters one driver.toStream()now takes the processor data and returns a stream resource, rather than taking a writer and returning a PSR stream. It was not called anywhere inside the package before.saveToStorage()takes a stream rather than a string, and usesStorage::writeStream().🤖 Generated with Claude Code