Repository navigation
fix(stats): match a book once when several of its hashes share a fingerprint - #60
Merged
Merged
Conversation
…erprint Writing library metadata into EPUBs changes a file's SHA but often not KOReader's partial MD5, which samples only a few chunks. A book then has two book_hashes rows with the same fingerprint, and the lookups that joined book_hashes returned it twice: the KOReader statistics import failed with "Multiple rows were found" and imported no sessions. Match on the book (books.id IN the hashes' book_ids) so each book counts once; two different books sharing a fingerprint are still ambiguous.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Since #58 was deployed, every library scan has logged
stats DB import failed: Multiple rows were found when one or none was required. No KOReader reading sessions have been imported since. They're still in KOReader's statistics database, so nothing is lost.When Shelfloom writes metadata into an EPUB, the file's SHA changes. KOReader's partial MD5 often doesn't, because it samples only a few chunks of the file. 22 live books now have two
book_hashesrows with the same KOReader fingerprint. Lookups that joinedbook_hashesreturned those books twice, andoneBooktreated that as ambiguous.What
The three lookups that joined
book_hashes(stats match,.sdrmatch,findByHash) now match on the book:books.id IN (SELECT book_id FROM book_hashes WHERE …). Each book counts once. Two different books with the same fingerprint are still reported as ambiguous.Testing
TestStatsMatchSharedFingerprintfails without the fix and passes with it. Fullgo vetandgo test -racepass.statistics.sqlite3. Without the fix it fails with the same error; with the fix it succeeds.