Repository navigation
Conversation
EXPORT DATABASE wrote each FTS index to index.cypher with only stemmer and stopWords, so IMPORT DATABASE created an index with the default ignore_pattern and tokenizer: queries and rows inserted after the import were normalized differently from the rows built from the exported macro. toCypher now writes the options that differ from their defaults, so exports of default indexes do not change, and escapes every option value. The escaping helper moves to FTSUtils and now also covers the stopwords path in CREATE_FTS_INDEX, which broke on a path containing a quote. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This work was produced with the help of language models.
Fixes #120.
EXPORT DATABASEwrote each FTS index toindex.cypherwith onlystemmerandstopWords, soIMPORT DATABASEcreated it with the defaultignore_patternand tokenizer: a jieba index came back with the simple tokenizer, a custom pattern as the default one, and queries and rows inserted after the import were normalized differently from the rows built from the exported macro.Fix.
toCypherwritesignore_pattern,tokenizerandjieba_dict_dirwhen they differ from their defaults, soindex.cypherand theSHOW_INDEXESdefinition of a default index do not change. Every option value is escaped for a single-quoted Cypher literal; the helper moves toFTSUtils::escapeCypherString, shared withCREATE_FTS_INDEX, where it now also covers the stopwords path, which broke on a path containing a quote. This overlaps #124 increate_fts_index.cpp; whichever lands second gets rebased.Tests.
export_options.testexports an index withignore_pattern := '[!]+'and a jieba index, imports them into a fresh database, finds12345and北京清华, then inserts a row and finds it with a jieba query, which checks the imported index entry as well as the macro; a second case uses a stopwords file whose path contains a quote. Both fail on main and pass here. FTS extension tests: no new failures.