Repository navigation
Conversation
Every quoted cell's content is found by scanning it for the close quote. The common field ends with its close quote and holds no other one; for up to 63 content bytes, one 64-byte compare (the scanner's byte mask) now proves that, and every other field keeps the byte loop. Results are the same by construction (2M random fields matched main's loop). findcontent: 20-28-byte clean fields 11-13 -> 6.2 ns; escaped fields about 1 ns slower. quoted.csv (1.5M rows, all fields quoted): about 10% less CPU single-threaded, 15-20% less wall at 8 tasks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1223 +/- ##
=======================================
Coverage 95.79% 95.79%
=======================================
Files 9 9
Lines 5566 5570 +4
=======================================
+ Hits 5332 5336 +4
Misses 234 234 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every quoted cell's content is found by
findcontent, which scans the field for its close quote; on an all-quoted file that was about 30% of a single-threaded read. The common field ends with its close quote and holds no other one. For up to 63 content bytes, one 64-byte compare (the structural scanner'sbyte_mask_vec) now proves that; any other field (longer, near the end of the buffer, or with an inner quote) keeps the byte loop, so results are the same by construction. Only the default doubled-quote rule takes the check.findcontentper field (Apple M3 Max, Julia 1.12.6, interleaved runs):"alpha, bravo; charlie delta""1234 echo St, Apt 56""1234""8123.45""say ""hi"""(escaped)On NEON the mask extraction costs about 6 ns, so very short and escaped fields lose up to 1 ns; x86 extracts the mask in one instruction.
quoted.csv(113 MB, 1.5M rows, 5 quoted columns with embedded delimiters, newlines, and doubled quotes), interleaved process pairs: 1 task CPU 0.40-0.44 s vs 0.46-0.49 s on main (about 10% less); 8 tasks wall 0.073 / 0.079 / 0.094 s vs 0.082 / 0.096 / 0.121 s. An unquoted text file was unchanged within noise.Verification:
quoted field content around the 64-byte checkcomparesfindcontentwith a plain reference loop for content lengths 0-130 (around the 63/64 limit), 0-65 bytes of buffer after the field, and lone or doubled inner quotes at the first and last content byte (270 cases).--check-bounds=yes -t4: 332,048 passed.🤖 Generated with Claude Code