Skip to content

perf: JIRA-17824 stream exports instead of building a spreadsheet - #116

Merged
webhanspeter merged 2 commits into
mainfrom
JIRA-17824-stream-csv-exports
Sep 3, 2026
Merged

webhanspeter merged 2 commits into
mainfrom
JIRA-17824-stream-csv-exports

Conversation

@webhanspeter

@webhanspeter webhanspeter commented Sep 1, 2026 •

Copy link
Copy Markdown
Member

Summary

CsvDriver built a complete PhpSpreadsheet workbook in memory in order to emit a plain CSV. Rows are now written straight to a stream.

rows before after
10,000 0.338 ms/row, 135 MB 0.0030 ms/row, 4 MB
25,000 0.341 ms/row, 358 MB 0.0033 ms/row, 8 MB
50,000 0.358 ms/row, 685 MB 0.0041 ms/row, 8 MB
100,000 0.523 ms/row, 1,364 MB 0.0047 ms/row, 8 MB
318,000 ~165 s, ~4.4 GB (projected) 1.66 s, 8 MB

About 100x less time per row, and memory is flat — the buffer is capped and spills to a temp file beyond it, so it no longer scales with the export. Note the old figures get worse per row as the set grows (0.338 to 0.523 ms) because the working set runs into gigabytes; the new ones do not.

Along the way the driver stopped needing two dependencies, and two correctness bugs fell out.

The header only described the first row

toSpreadsheet() took the header from array_keys($data->first()) but fromArray() wrote each row in that row's own key order. Any row whose keys differed in order or membership silently put values under the wrong headers — wrong data, no error.

The header is now the set of keys any row has, in the order first seen, and every row is written against that list. A missing key writes an empty field instead of shifting everything left.

Values were being coerced

A spreadsheet infers a type per cell. DefaultValueBinder decided a numeric-looking string was a number and Cell::setValueExplicit() did $this->value = 0 + $value, at which point the text was gone. A leading = was parsed as a formula and the calculated result written instead.

value produced old file new file
8.00 8 8.00
1234.50 1234.5 1234.50
0.10 0.1 0.10
=SUM(A1) 0 =SUM(A1)
+4512 4512 +4512
1e3 1000 1e3

A CSV holds text, so nothing is inferred any more. Leading zeros and long integers were already safe — the binder has explicit guards for those — but trailing decimal zeros were not.

There is a second coercion site in the same library worth knowing about: the CSV writer separately strips trailing zeros from decimals beyond 15 digits of precision. Also gone.

Quoting is unchanged, deliberately

fputcsv quotes any field containing a space, which the old writer with setEnclosureRequired(false) did not — that would have put quotes around most text columns in every row. So rows are assembled directly, enclosing only on the delimiter, the enclosure, or a newline, which is what Writer\Csv did:

if (!$this->enclosureRequired && strpbrk($element, "$delimiter$enclosure\n") === false) {

A bare carriage return is also enclosed now, since rows are separated by one. Any value containing one previously produced a broken file.

Verified by byte comparison

Ran a realistic export through both writers — 21 columns, values including a comma, embedded quotes, a padded identifier and decimals — and diffed the files. Exactly two fields differ, both the fidelity fix:

duration   old='7.5'   new='7.50'
duration   old='0.1'   new='0.10'

Quoting, column order, line endings and every other value are byte-identical.

Test plan

  • Four new tests: value fidelity, header alignment across rows with differing keys, quoting only where required, and empty input.

  • Existing driver tests unchanged in expectation, including the exact "name\r\nJohn Doe\r\nJane Doe\r\n" assertion.

  • Full suite 24 passed, 73 assertions.

  • ECS and PHPStan clean.

  • Suite re-run with both dependencies uninstalled, to prove neither is still needed.

  • Rector clean. (It used to fail on a clean checkout under coding-style 4.x, which registered the deprecated ArraySpreadInsteadOfArrayMergeRector; the 5.x bump now on main fixed that, so it runs properly again.)

Breaking changes

Needs a major release.

  • phpoffice/phpspreadsheet and guzzlehttp/guzzle are no longer required. Neither is referenced anywhere in src/ any more — guzzle existed only for toStream()'s return type.
  • toSpreadsheet() and toCsvWriter() are removed. Both took or returned PhpSpreadsheet types, and the package has never produced a spreadsheet: GeneratorType has one case, CSV, and GeneratorManager registers one driver.
  • toStream() now takes the processor data and returns a stream resource, rather than taking a writer and returning a PSR stream. It was not called anywhere inside the package before.
  • saveToStorage() takes a stream rather than a string, and uses Storage::writeStream().
  • Exports containing numeric-looking values change, as above. Anything reading them numerically is unaffected; anything comparing the file as text will see restored decimals.

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Sep 1, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: 76a6dcf9-1006-4fec-bf21-bb5b8db0d43d


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

The CSV driver built a complete PhpSpreadsheet workbook in memory to emit
a plain CSV. That cost roughly 0.5 ms and 14 KB per row, and time per row
grew as the working set did, so a large export needed gigabytes and slowed
down as it went. Writing rows straight out is about 100 times faster and
holds memory flat: 318,000 rows now takes 1.7 seconds and 8 MB, against a
projected 165 seconds and 4.4 GB.

Two correctness fixes come with it.

The header used to be taken from the first row's keys while every row was
written in its own key order, so any row whose keys differed silently put
values under the wrong headers. The header is now the set of keys any row
has, in the order first seen, and every row is written against it.

Values are no longer coerced. A spreadsheet infers a type per cell, so a
numeric-looking string lost its formatting and a leading = was read as a
formula: 1234.50 was written as 1234.5, and =SUM(A1) as 0. A CSV holds
text, so values now reach the file as given.

BREAKING CHANGE: phpoffice/phpspreadsheet and guzzlehttp/guzzle are no
longer required. toSpreadsheet() and toCsvWriter() are removed, and
toStream() now takes the processor data and returns a stream resource
rather than taking a writer. saveToStorage() takes a stream rather than a
string. Exports containing numeric-looking values change: the file keeps
what the processor produced instead of a coerced version.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@webhanspeter
webhanspeter force-pushed the JIRA-17824-stream-csv-exports branch from f516517 to 1da8e83 Compare September 2, 2026 15:09
This adds exceptions when the provided stream is not a resource.
@owenvoke
owenvoke force-pushed the JIRA-17824-stream-csv-exports branch from 7835ce6 to b4553c7 Compare September 3, 2026 08:59
@webhanspeter
webhanspeter marked this pull request as ready for review September 3, 2026 09:40
@webhanspeter
webhanspeter merged commit e2bed01 into main Sep 3, 2026
15 checks passed
@webhanspeter
webhanspeter deleted the JIRA-17824-stream-csv-exports branch September 3, 2026 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants