Skip to content

fix(parser): avoid Math.max spread on large files - #25

Merged
daggerok merged 1 commit into
masterfrom
fix/parser-large-files
Oct 25, 2026
Merged

daggerok merged 1 commit into
masterfrom
fix/parser-large-files

Conversation

@daggerok

Copy link
Copy Markdown
Owner

What

  • New maxRowLength(rows, min = 0) loop helper
  • extractValidTableData: both Math.max(...rows.map(r => r.length)) calls use it
  • Merged export: maxCols is a reduce over blocks with maxRowLength(block.rows) instead of nested spreads
  • applyFilters uses the same helper instead of its own reduce (same result)

Why

CSV-H1: spreading one argument per row into Math.max throws RangeError: Maximum call stack size exceeded in V8 (Chrome, Edge, Node). A big export failed to load at all, and the merged export failed for a large block

Checks

  • bun run build
  • tsc --noEmit --strict --skipLibCheck --jsx react-jsx on src/main.tsx (no errors)
  • Old and new parseCSVText + extractValidTableData transpiled with Bun.Transpiler and run in Node (V8):
    • old: 100k rows ok; 200k rows, 1M rows and a 300k single-column file throw Maximum call stack size exceeded
    • new: 100k, 200k, 1M rows and 300k single-column all parse
  • Same output from old and new on a Fidelity file (trailing commas plus footer), a preamble, ragged rows, a single column and quoted fields
  • Merged export width: old and new formula equal on empty, header-only, mixed-width and single-row inputs; old throws on a 300k-row block, new returns 2
  • git diff --check
  • No test file: the repo has no test setup (README "No tests")

Math.max(...rows.map(r => r.length)) passes one argument per row and throws Maximum call stack size exceeded in V8 somewhere between 100k and 200k rows, so large files failed to load and the merged export failed. A maxRowLength loop helper replaces the spreads in extractValidTableData and the merged export width, and the existing reduce in applyFilters.
@daggerok
daggerok merged commit bc778df into master Oct 25, 2026
3 checks passed
@daggerok
daggerok deleted the fix/parser-large-files branch October 25, 2026 03:34

This branch was successfully deployed

2 active deployments
ci — 22656696 Deployed Oct 25, 2026 by daggerok via npm-check-updates #93
github-pages — 22656696 Deployed Oct 25, 2026 by daggerok via deploy #89
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant