Repository navigation
fix: *arr log file ingestion emits no entries (timestamp regex + context parser contract) - #57
Merged
itz4blitz merged 5 commits intoOct 13, 2026
Conversation
…amps *arr apps write log files with the NLog layout yyyy-MM-dd HH:mm:ss.f (a single fractional digit), so real Sonarr/Radarr/Prowlarr/Whisparr lines did not match the parser regex, which required exactly three digits, and were dropped or treated as continuations.
LogFileProcessor finalizes the pending entry before asking the provider to parse a new header line, as the parseLogLineWithContext contract in core expects (Emby and others return the current entry). The *arr context parser instead returned the previous entry and stashed the current one in the caller's context, which the processor discards, so file-tailed Sonarr, Radarr, Prowlarr and Whisparr logs produced no entries at all. - parseArrLogLineWithContext now returns the current entry for header lines, flags continuations, and ignores blank lines (NLog writes one between a message and its exception) - Exception extraction moves to finalizeArrLogEntry, invoked through a new optional finalizeLogEntry hook in LogFileProcessor; it strips the "[vX.Y.Z.W]" prefix so the exception type and message are populated - End-to-end LogFileProcessor tests with real Sonarr and Emby providers
This was referenced Oct 10, 2026
…re finalizeLogEntry on core provider interface
Now that *arr file ingestion works, the shared *.txt pattern also picked up the apps' rolling debug and trace logs (Radarr keeps 50+ radarr.debug.N.txt files), duplicating the main log at higher verbosity. Backfilling them exhausted a 384 MB backend's Node heap and caused Postgres deadlocks from dozens of parallel tailers. - LogFileConfig gains optional excludePatterns (core) - every *arr log config excludes *.debug.* and *.trace.* - FileDiscoveryService skips excluded basenames; servers with custom logFilePatterns get no excludes, so debug logs can still be opted into
vbrzezina
marked this pull request as ready for review
October 12, 2026 16:43
itz4blitz
approved these changes
Oct 13, 2026
itz4blitz
left a comment
Owner
There was a problem hiding this comment.
Approving. The two ingestion bugs are fixed.
NLOG_LINE_REGEX accepts 1–7 fractional digits, and new Date treats them as a decimal fraction of a second (.2 is 200ms). parseArrLogLineWithContext now returns the entry for the current line, which matches LogFileProcessor: the processor finalizes the pending entry before the next header, skips blank lines, and calls finalizeLogEntry only when the provider implements it. Emby, Jellyfin, and Plex are unchanged. *.debug.* and *.trace.* are excluded for the default *arr configs, and a custom logFilePatterns list is left alone.
Non-blocking notes, fine as follow-ups:
- The new comment on
FileDiscoveryService.isIncludedsays includes are matched against the base name. Only the exclude check uses the base name. Directory scans already pass the base name, so the behavior is right; the comment is not. finalizeArrLogEntrymisses a bareSystem.Exception(\w+requires a character beforeException) and types outside the System/Microsoft/NLog/NzbDrone allowlist. The stack trace is still stored.- The
parseArrLogLineWithContextdocblock restates the processor contract. The blank-line comment is the one worth keeping. - The processor spec title says an 8-digit fraction is not attached, but the assertion checks that it is kept as a continuation.
This was referenced Oct 13, 2026
itz4blitz
pushed a commit
that referenced
this pull request
Oct 16, 2026
itz4blitz
added a commit
that referenced
this pull request
Oct 16, 2026
…owups chore(provider-arr): address #57 review follow-ups
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #63. Overview of all related PRs: #64
Summary
File-tailed logs from the *arr apps (Sonarr, Radarr, Prowlarr, Whisparr) currently produce no log entries at all. Two independent bugs combine to cause it:
The timestamp regex rejects real lines (
fix(provider-arr): …)yyyy-MM-dd HH:mm:ss.f, which has one fractional digit (e.g.2026-10-10 12:00:01.2|Info|…).NLOG_LINE_REGEXrequired exactly three digits, so real lines failed to match..2is parsed as 200 ms.*The arr context parser breaks the
parseLogLineWithContextcontract (fix(file-ingestion): …)LogFileProcessorfinalizes its pending entry before asking the provider to parse a new header line. That is the contract documented inpackages/core/src/types.ts, and the Emby parser follows it.parseArrLogLineWithContextinstead returned the previous entry and stashed the current one in the context object, which the processor throws away. So no entry was ever emitted.finalizeArrLogEntry. It runs through a new optionalfinalizeLogEntryhook inLogFileProcessor, and strips the[vX.Y.Z.W]prefix soexceptionand the exception message are filled in. Providers that don't implement the hook (Emby, Jellyfin, Plex) behave as before.*Shared building blocks for the arr providers (
refactor(provider-arr): …)finalizeLogEntry?is now declared on the coreMediaServerProviderinterface, next toparseLogLineWithContext?andisLogContinuation?. The processor checks for it with a type guard.provider-arr, used by the Lidarr and Readarr PRs:resolveArrEventNameturns a numeric, numeric-string or wrong-case event type into its canonical name.describeArrEventlooks up an event's label and severity in a table.toArrItemIdgives a positive-integer item id or nothing.Testing
log-file-processor.spec.tsruns realistic *arr logs end to end through the real processor, using realSonarrProviderandEmbyProviderinstances. It covers:arr.parser.spec.tscases cover 1–7 digit fractions, a missing fraction, blank lines, consecutive errors, and thefinalizeArrLogEntryhook.provider-arr76 tests and backend 856 tests pass.build,lintandtypecheckare clean.pnpm format:checkalready fails onmaster(pre-existing files); the files changed here are prettier-clean.The Lidarr and Readarr integration PRs are stacked on this branch.