Skip to content

fix: *arr log file ingestion emits no entries (timestamp regex + context parser contract) - #57

Merged
itz4blitz merged 5 commits into
itz4blitz:masterfrom
vbrzezina:fix/arr-log-timestamp-precision
Oct 13, 2026
Merged

itz4blitz merged 5 commits into
itz4blitz:masterfrom
vbrzezina:fix/arr-log-timestamp-precision

Conversation

@vbrzezina

@vbrzezina vbrzezina commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #63. Overview of all related PRs: #64

Summary

File-tailed logs from the *arr apps (Sonarr, Radarr, Prowlarr, Whisparr) currently produce no log entries at all. Two independent bugs combine to cause it:

  1. The timestamp regex rejects real lines (fix(provider-arr): …)

    • *arr apps write log files with the NLog layout yyyy-MM-dd HH:mm:ss.f, which has one fractional digit (e.g. 2026-10-10 12:00:01.2|Info|…). NLOG_LINE_REGEX required exactly three digits, so real lines failed to match.
    • The fractional part now accepts 1–7 digits, and .2 is parsed as 200 ms.
  2. *The arr context parser breaks the parseLogLineWithContext contract (fix(file-ingestion): …)

    • LogFileProcessor finalizes its pending entry before asking the provider to parse a new header line. That is the contract documented in packages/core/src/types.ts, and the Emby parser follows it.
    • parseArrLogLineWithContext instead returned the previous entry and stashed the current one in the context object, which the processor throws away. So no entry was ever emitted.
    • The parser now returns the current entry for header lines, flags continuation lines, and ignores blank lines. NLog writes a blank line between a message and its exception, and that blank line used to drop the stack trace.
    • Exception extraction moved to finalizeArrLogEntry. It runs through a new optional finalizeLogEntry hook in LogFileProcessor, and strips the [vX.Y.Z.W] prefix so exception and the exception message are filled in. Providers that don't implement the hook (Emby, Jellyfin, Plex) behave as before.
  3. *Shared building blocks for the arr providers (refactor(provider-arr): …)

    • finalizeLogEntry? is now declared on the core MediaServerProvider interface, next to parseLogLineWithContext? and isLogContinuation?. The processor checks for it with a type guard.
    • New helpers in provider-arr, used by the Lidarr and Readarr PRs:
      • resolveArrEventName turns a numeric, numeric-string or wrong-case event type into its canonical name.
      • describeArrEvent looks up an event's label and severity in a table.
      • toArrItemId gives a positive-integer item id or nothing.
    • Sonarr, Radarr, Prowlarr and Whisparr are unchanged. Moving them onto these helpers is a possible follow-up.

Testing

  • New log-file-processor.spec.ts runs realistic *arr logs end to end through the real processor, using real SonarrProvider and EmbyProvider instances. It covers:
    • multi-line exceptions with blank lines;
    • flush at EOF;
    • a stray continuation line at the start of a file;
    • unparseable lines;
    • a regression guard for Emby.
  • New arr.parser.spec.ts cases cover 1–7 digit fractions, a missing fraction, blank lines, consecutive errors, and the finalizeArrLogEntry hook.
  • Results: provider-arr 76 tests and backend 856 tests pass. build, lint and typecheck are clean.

pnpm format:check already fails on master (pre-existing files); the files changed here are prettier-clean.

The Lidarr and Readarr integration PRs are stacked on this branch.

…amps

*arr apps write log files with the NLog layout yyyy-MM-dd HH:mm:ss.f (a single
fractional digit), so real Sonarr/Radarr/Prowlarr/Whisparr lines did not match
the parser regex, which required exactly three digits, and were dropped or
treated as continuations.
LogFileProcessor finalizes the pending entry before asking the provider to
parse a new header line, as the parseLogLineWithContext contract in core
expects (Emby and others return the current entry). The *arr context parser
instead returned the previous entry and stashed the current one in the
caller's context, which the processor discards, so file-tailed Sonarr,
Radarr, Prowlarr and Whisparr logs produced no entries at all.

- parseArrLogLineWithContext now returns the current entry for header
  lines, flags continuations, and ignores blank lines (NLog writes one
  between a message and its exception)
- Exception extraction moves to finalizeArrLogEntry, invoked through a new
  optional finalizeLogEntry hook in LogFileProcessor; it strips the
  "[vX.Y.Z.W]" prefix so the exception type and message are populated
- End-to-end LogFileProcessor tests with real Sonarr and Emby providers
…re finalizeLogEntry on core provider interface
Now that *arr file ingestion works, the shared *.txt pattern also picked up
the apps' rolling debug and trace logs (Radarr keeps 50+ radarr.debug.N.txt
files), duplicating the main log at higher verbosity. Backfilling them
exhausted a 384 MB backend's Node heap and caused Postgres deadlocks from
dozens of parallel tailers.

- LogFileConfig gains optional excludePatterns (core)
- every *arr log config excludes *.debug.* and *.trace.*
- FileDiscoveryService skips excluded basenames; servers with custom
  logFilePatterns get no excludes, so debug logs can still be opted into

@itz4blitz itz4blitz left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. The two ingestion bugs are fixed.

NLOG_LINE_REGEX accepts 1–7 fractional digits, and new Date treats them as a decimal fraction of a second (.2 is 200ms). parseArrLogLineWithContext now returns the entry for the current line, which matches LogFileProcessor: the processor finalizes the pending entry before the next header, skips blank lines, and calls finalizeLogEntry only when the provider implements it. Emby, Jellyfin, and Plex are unchanged. *.debug.* and *.trace.* are excluded for the default *arr configs, and a custom logFilePatterns list is left alone.

Non-blocking notes, fine as follow-ups:

  • The new comment on FileDiscoveryService.isIncluded says includes are matched against the base name. Only the exclude check uses the base name. Directory scans already pass the base name, so the behavior is right; the comment is not.
  • finalizeArrLogEntry misses a bare System.Exception (\w+ requires a character before Exception) and types outside the System/Microsoft/NLog/NzbDrone allowlist. The stack trace is still stored.
  • The parseArrLogLineWithContext docblock restates the processor contract. The blank-line comment is the one worth keeping.
  • The processor spec title says an 8-digit fraction is not attached, but the assertion checks that it is kept as a continuation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

*arr log file ingestion produces no log entries (Sonarr/Radarr/Prowlarr/Whisparr)

2 participants