Skip to content

Remove severity-number exporter patch; map severity at emit time - #119

Merged
jrothrock merged 5 commits into
mainfrom
fix-severity-number-remove-patch
Oct 19, 2026
Merged

jrothrock merged 5 commits into
mainfrom
fix-severity-number-remove-patch

Conversation

@jrothrock

@jrothrock jrothrock commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Problem

CI started failing after opentelemetry-exporter-otlp-logs 0.7.0:

NoMethodError: undefined method 'as_otlp_any_value' for an instance of
OpenTelemetry::Exporter::OTLP::Logs::LogsExporter
# .../opentelemetry/log_record_patch.rb:20

log_record_patch.rb overrode the exporter's entire as_otlp_log_record method to remap Ruby's Logger::Severity integers (DEBUG=0, INFO=1, WARN=2, ERROR=3, FATAL=4) onto OpenTelemetry's SeverityNumber enum (DEBUG=5, INFO=9, WARN=13, …), the scale our UI matches on. Exporter 0.7.0 moved encoding into opentelemetry-exporter-otlp-common (Common.as_encoded_elsr), so as_otlp_log_record and as_otlp_any_value no longer exist and the override is never called. The CI failure was the patch's own unit spec calling the method directly.

This also means that on exporter 0.7.0, main was already sending Ruby's 0–5 integers as severity_number (INFO=1 is SEVERITY_NUMBER_TRACE in OTel).

The patch also carried the #114 guard (dropped_attributes_count vs total_recorded_attributes), so it couldn't just be deleted.

Fix

Severity is mapped at emit time. Formatter#emit_log sends the OTel SeverityNumber directly via SEVERITY_NUMBER_MAP, so every exporter version passes it through unmodified. log_record_patch.rb, its require_relative, and its spec are deleted.

  • logger.unknown is labelled 'ANY' by Ruby's Logger. It now maps to FATAL, since it ranks above FATAL in Ruby; previously Logger::Severity.const_get('ANY') raised NameError. Unrecognized labels fall back to UNSPECIFIED.

#114 guard → LogRecordData shim (log_record_data_compat.rb). Every exporter release only requires opentelemetry-logs-sdk ~> 0.1, so on Ruby ≥ 3.3 Bundler can resolve pairs that disagree on LogRecordData's fields:

Exporter reads Missing from
total_recorded_attributes (≤ 0.5.1) logs-sdk ≥ 0.7.0
dropped_attributes_count (≥ 0.6.0) logs-sdk < 0.7.0
event_name (≥ 0.5.0) logs-sdk < 0.6.0

Any of these raises inside encode, which the exporter rescues, so every log is silently dropped. The shim defines only the missing field(s), derived from the others, and overrides no exporter methods. Checked against exporter 0.5.1 + sdk 0.8.1, 0.2.2 + 0.8.1, 0.6.0 + 0.4.0, and 0.5.1 + 0.4.0: all drop logs without the shim and export correctly with it. Matching pairs are unaffected.

Raising the gemspec floors instead was considered and rejected: exporter ≥ 0.6.0 and logs-sdk ≥ 0.7.0 both require Ruby ≥ 3.3, which would drop 2.x support for Ruby 3.1/3.2. (On 3.1/3.2, Bundler can only resolve exporter ≤ 0.3.0 with logs-sdk ≤ 0.4.0, which agree.)

Also: opentelemetry-logs-api is now a direct dependency, since the formatter references its constants.

Tests

  • spec/unit/loggers/formatter_spec.rb (new): severity enum for every level, plus ANY and an unknown label, through an InMemoryLogRecordExporter.
  • spec/unit/loggers/opentelemetry/log_record_data_compat_spec.rb (new): encodes a record through whichever encoder the installed exporter uses and asserts body, severity, and dropped_attributes_count == 0.
  • spec/integration/rails/lifecycle_spec.rb: the stubbed receiver decodes severity_number / severity_text off the wire and asserts WARN/ERROR/FATAL/INFO. The receiver count assertions now check receiver_messages (they were re-checking local_messages).

All unit specs and the Rails lifecycle spec pass via rake test (Docker, Ruby 3.3, exporter 0.7.0 / logs-sdk 0.8.1). CI resolves the newest gems, so the mismatched pairs were checked separately against locally installed versions.

🤖 Generated with Claude Code

jrothrock and others added 2 commits October 16, 2026 12:36
The log_record_patch overrode the whole OTLP logs exporter's
`as_otlp_log_record` to remap Ruby's `Logger::Severity` integers (0..5)
onto OpenTelemetry's `SeverityNumber` enum, which the UI matches on. As a
whole-method override it drifted from upstream: opentelemetry-exporter-
otlp-logs 0.2.2 changed that method's body, breaking the patch (the
`undefined method 'as_otlp_any_value'` failures in CI).

Fix the mismatch at the source instead. `formatter.rb` now emits the
correct OTel `SeverityNumber` directly in `on_emit`, so the exporter's
native `as_otlp_log_record` passes it straight through unmodified. This
drops the brittle override entirely and keeps us upstream-compatible.

Also add a severity-number drift guard to the Rails lifecycle test: the
receiver now captures `severity_number`/`severity_text` off the wire and
asserts WARN/ERROR/FATAL/INFO arrive as the OTel enum values, catching
future scale drift end-to-end.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The method now extracts severity alongside the body, pushing its ABC size
just over the threshold (17.72/17). Inlining the extraction keeps it
readable, so disable the cop for this test helper.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment thread lib/scout_apm/logging/loggers/opentelemetry/log_record_patch.rb
Comment thread lib/scout_apm/logging/loggers/formatter.rb Outdated
Comment thread spec/unit/loggers/opentelemetry/log_record_patch_spec.rb
Comment thread spec/integration/rails/lifecycle_spec.rb Outdated
jrothrock and others added 2 commits October 16, 2026 15:54
Logger labels `logger.unknown` as 'ANY', so the 'UNKNOWN' map key never
matched and those lines fell through to SEVERITY_NUMBER_UNSPECIFIED, the
lowest value. Key it as 'ANY' and map it to FATAL, since it ranks above
FATAL in Ruby.

Add a Formatter unit spec covering every level (and an unrecognized
label) through an in-memory exporter, so the map is checked without the
Docker-only Rails spec. Fix the receiver assertions in the lifecycle
spec that were re-checking local_messages. Declare opentelemetry-logs-api
directly now that the formatter references its constants.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Removing the exporter patch also removed the #114 guard. Every OTLP logs
exporter release only requires `opentelemetry-logs-sdk ~> 0.1`, so on
Ruby >= 3.3 Bundler can pair versions that disagree on LogRecordData:
exporter <= 0.5.1 reads `total_recorded_attributes` (gone in logs-sdk
>= 0.7.0), exporter >= 0.6.0 reads `dropped_attributes_count` (absent
before logs-sdk 0.7.0), and exporter >= 0.5.0 reads `event_name` (absent
before logs-sdk 0.6.0). Any of these raises inside `encode`, which the
exporter rescues, silently dropping every log.

Define only the missing field(s) on LogRecordData, derived from the
others, rather than overriding exporter methods. Raising the gemspec
floors instead would require Ruby >= 3.3 and drop 2.x support for 3.1/3.2.

Note: the earlier commit d399005 attributes the original breakage to
exporter 0.2.2; it was 0.7.0, which moved encoding into
opentelemetry-exporter-otlp-common so the patched method stopped being
called.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread spec/unit/loggers/formatter_spec.rb

@quinnmil quinnmil left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm approving. Had claude post a nitpic if you want to address.

logger_spec replaced Formatter#emit_log via class_exec at load time, which
lasted for the whole process, so formatter_spec's InMemoryLogRecordExporter
saw nothing when both ran together (`bundle exec rspec spec/unit`). Stub it
with allow_any_instance_of in a before block instead; spec_helper already
resets mocks after each example. Same change as in #120.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jrothrock
jrothrock merged commit 32a0b0f into main Oct 19, 2026
18 checks passed
@jrothrock
jrothrock deleted the fix-severity-number-remove-patch branch October 19, 2026 22:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants