Skip to content

Add opt-in support for long character fields - #95

Open
omggga wants to merge 2 commits into
yortus:masterfrom
omggga:agent/support-long-character-fields
Open

Add opt-in support for long character fields#95
omggga wants to merge 2 commits into
yortus:masterfrom
omggga:agent/support-long-character-fields

Conversation

@omggga

@omggga omggga commented Aug 13, 2026

Copy link
Copy Markdown

Summary

  • add opt-in support for FoxPro/Clipper character fields whose length is stored as an unsigned 16-bit value across field descriptor bytes 16 and 17
  • introduce longCharacterFields: 'off' | 'auto', defaulting to 'off' for backward compatibility
  • in 'auto' mode, select the extended interpretation only when it exactly matches the record length declared in the DBF header
  • read the header record length as unsigned while keeping creation of character fields larger than 255 bytes disabled
  • document the option, behavior, and limitations in the README and API reference

Root cause

The reader currently always treats byte 16 as the complete character field size and byte 17 as the decimal count. For DBF variants that use byte 17 as the high byte of a character field length, this produces a smaller computed record size and fails with Invalid DBF: Incorrect record length.

The new behavior is disabled by default. When a caller explicitly passes {longCharacterFields: 'auto'}, detection first calculates the existing one-byte layout. It only uses 16-bit character sizes when the existing layout does not match and the extended layout exactly reconciles with the declared record length. Files whose one-byte layout already matches retain the current behavior.

This is a follow-up to #80 and the earlier approach in #81.

Validation

  • added runtime-generated synthetic DBF cases for a 300-byte character field with multiple records
  • verified that the default 'off' mode preserves the previous failure behavior
  • covered strict and loose read modes
  • covered fallback to the existing one-byte interpretation
  • covered rejection when neither layout matches
  • covered unsigned record lengths above 32767
  • verified that writing character fields larger than 255 bytes remains rejected
  • covered validation of unsupported longCharacterFields values
  • npm test passes on Node.js 12 and Node.js 18: 60 passing

All new fixtures are generated from synthetic values during the test run; no external DBF files or data are included.

@omggga
omggga marked this pull request as ready for review August 13, 2026 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant