Support POSIX character classes in gitignore bracket expressions - #128
Conversation
Git's wildmatch supports POSIX bracket character classes such as [[:alpha:]] and [[:digit:]], but pathspec passed the bracket body straight to Python's `re`, which does not understand them. This built a broken regex (Python warns "Possible nested set", slated to become a hard error) and silently returned the wrong match for every pattern using a POSIX class. Translate each of the 12 classes to the explicit ASCII range that git's wildmatch uses. These are deliberately ASCII, not Python's Unicode-aware \w/\d/\s, so matching stays consistent with git (for example [[:digit:]] does not match non-ASCII digits). The bracket scan now also skips over [:name:] tokens so the class's internal `]` is not mistaken for the end of the expression, and an unknown or negated class name ([[:^alpha:]]) is treated as a malformed pattern, matching git.
|
Thanks for the pull request. Can you show that git/gitignore actually supports POSIX character classes? I see no reference in the gitignore docs. |
|
The docs reference it indirectly: gitignore(5) says bracket ranges follow fnmatch(3) with the FNM_PATHNAME flag, and POSIX fnmatch defines the [:alpha:] style classes. Empirically git honors them — with |
|
The implementation and tests all look good. I've confirmed git does indeed use POSIX classes which I didn't expect. Thanks! |
Summary
Git's gitignore matching (wildmatch) supports POSIX bracket character classes —
[[:alpha:]],[[:digit:]],[[:space:]]and the other nine — but pathspec supports none of them. The bracket handler inpathspec/patterns/gitignore/base.pypasses the bracket body straight to Python'sre, which does not understand POSIX classes, so it builds a broken regex (Python emitsFutureWarning: Possible nested set at position 1, which is slated to become a hardre.error) and silently returns the wrong match.The existing code comment already states the design goal — "Maintain consistency with Git because that is the expected behavior." This closes that gap for POSIX classes.
Reproduction
Oracle is
git check-ignore(ground truth wildmatch):Fix
Translate each of the 12 POSIX classes to the explicit ASCII range that git's wildmatch uses:
alnum0-9A-Za-zlowera-zalphaA-Za-zprint\x20-\x7eblank\tspacepunct!-/:-@\[-`{-~cntrl\x00-\x1f\x7fspace\t\n\rspacedigit0-9upperA-Zgraph\x21-\x7exdigit0-9A-Fa-fThese are deliberately ASCII — git's wildmatch uses locale-independent
is*checks on bytes — and not Python's Unicode-aware\w/\d/\s. For example[[:digit:]]matches0–9but not the Arabic-Indic digit٠(U+0660), matching git.Two supporting changes:
[:name:]tokens, so the class's internal]is no longer mistaken for the end of the whole bracket expression (that truncation is what produced the broken regex before).[[:bogus:]],[[:^alpha:]]) is treated as a malformed pattern that matches nothing, which is what git does (verified againstgit check-ignore).Classes compose with other bracket members, ranges and bracket negation as git allows:
[[:digit:]a-f_],[![:digit:]],[[:alpha:]0-9],[[:upper:][:digit:]].Conformance
I built a differential harness against real
git check-ignore(git 2.50.1) covering, for each of the 12 classes, every testable byte0x01–0xFF, plus negated/unknown names, composition, and the ASCII boundary. Before this change: 466 byte-level divergences from git; after: 0.Tests
New tests in
test_04_gitignore_spec.pycover the exact regex translation for all 12 classes, per-class match behaviour, composition, the ASCII-only boundary (non-ASCII digits/letters must not match), and discarding of unknown/negated class names. Full suite passes on thereandre2backends (156 passed, 46 skipped for the uninstalled hyperscan backend).Scope
This is limited to POSIX character classes. Backslash escaping inside a bracket (e.g.
[\]]to match a literal]) is a separate pre-existing gap and is intentionally not touched here.