test(otlp-trace-metrics): maintain parity with the OTEP spec - #7363
Merged
Conversation
Resolves the mechanically-actionable open threads from PR #6834: - FR02 mutual-exclusion test now also enables native stats computation explicitly, proving OTLP wins even when both mechanisms are turned on. - FR01 test renamed to test_fr01_5_disabled_when_tracing_is_disabled, plus a new test_fr01_6 covering DD_APM_TRACING_ENABLED=false. - span.kind casing is now tolerant of both "server" and "SERVER" pending spec finalization, via a new SPAN_KIND_SERVER_VALUES tuple. - FR06 resource/span-name test now also asserts datadog.resource.name is absent, matching the module's "in both modes" claim. Two open design questions (service.name repetition on data points, and the OTel-semantics-mode attribute set) are flagged with docstring notes rather than resolved, since they depend on the still-unfinalized OTEL spec (RFC SEMCON-1093). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
|
|
🎉 All green!🧪 All tests passed 🔗 Commit SHA: 8b21a05 | Docs | Datadog PR Page | Give us feedback! |
mabdinur
changed the title
…ty assertions Resolves the two open design questions from PR #6834 review with concrete decisions: service.name is now required on every data point, and OTel- semantics mode is restricted to exactly the Span Metrics Connector's default attribute set (service.name, span.name, span.kind, status.code). Also narrows ERROR_STATUS_VALUES and SPAN_KIND_SERVER_VALUES to the SMC's actual output format (STATUS_CODE_ERROR / SPAN_KIND_SERVER), based on reading the SMC connector source and traceutil.go directly rather than guessing at tolerant casing. Verified live against TEST_LIBRARY=python: 4 tests fail against the current Python implementation and are gated missing_feature in manifests/python.yml (fr01_6, fr06_2, fr06_8, fr09_2), in addition to the 3 already-known gates for the service.name/SMC-attrs requirements.
…_trace_root spec Reverts the SMC-parity restriction on OTel-semantics-mode attributes and the unconditional service.name-on-every-data-point requirement from #7363, since the RFC does not require either. Adds coverage for datadog.is_trace_root and datadog.peer_tags (skipped pending test-agent support for a peer_tags allowlist), and rewrites the process-tags test for the single datadog.process_tags list-attribute shape. Drops the DD_TAGS / OTEL_RESOURCE_ATTRIBUTES tests, which are now out of scope, along with their now-stale manifest gates. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… one list attribute additional_metric_tags does not share peer_tags/process_tags' combined colon-joined-list shape. Each DD_TRACE_STATS_ADDITIONAL_TAGS key surfaces as its own data-point attribute (e.g. customer.tier=gold), with no datadog. prefix and no additional_metric_tags container attribute. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…tional_metric_tags container Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Remove new assertions that only check a value is absent (datadog.resource.name, additional_metric_tags/datadog.<key> containers) -- they test for the absence of alternate shapes rather than validating the spec's actual dimensions. Trim duplicated "not asserted / implementations differ" doc caveats introduced alongside them. No assertion on an existing/expected value changed. Co-authored-by: Cursor <cursoragent@cursor.com>
Verified locally against the merged dd-trace-py#18354 build: datadog.process_tags is never emitted on OTLP trace metrics, and DD_TRACE_STATS_ADDITIONAL_TAGS keys are not forwarded to the native exporter at all. Mark both missing_feature until implemented. Co-authored-by: Cursor <cursoragent@cursor.com>
mabdinur
marked this pull request as ready for review
August 4, 2026 18:09
mabdinur
requested review from
MilanGarnier,
claponcet,
florentinl,
manuel-alvarez-alvarez and
ncybul
and removed request for
a team
August 4, 2026 18:09
…ments ERROR_STATUS_VALUES and SPAN_KIND_SERVER_VALUES each held one literal value; inline them directly in the assertions and drop the surrounding explanatory comments to shrink the diff. Co-authored-by: Cursor <cursoragent@cursor.com>
test_fr08_14_peer_tags is blocked on ddapm-test-agent support, not a per-library gap, but manifests are the preferred activation mechanism. Declare it missing_feature in python.yml and nodejs.yml (the only manifests where the surrounding class isn't already skipped wholesale) instead of an in-line @pytest.mark.skip. Co-authored-by: Cursor <cursoragent@cursor.com>
…ics-address-review-comments Co-authored-by: Cursor <cursoragent@cursor.com> # Conflicts: # manifests/java.yml # manifests/python.yml # tests/parametric/test_otlp_trace_metrics.py
The merge from main changed test_fr08_8_process_tags to check for individual datadog.<key> resource attributes (matching JS/.NET), which python already emits, so drop its now-stale missing_feature entry. Add missing_feature entries for the three tests that now fail against python: test_fr08_13_is_trace_root (emitted as _datadog.is_trace_root, with a leading underscore) and the two new DD_TAGS/OTEL_RESOURCE_ATTRIBUTES tracer_dd_tags tests (not yet reported for OTLP trace metrics). Co-authored-by: Cursor <cursoragent@cursor.com>
This was referenced Aug 4, 2026
Contributor
Author
|
While implementing the dd-trace-js side of this spec (DataDog/dd-trace-js#9685), noticed def test_fr08_8_process_tags(...):
"""Process tags surface as the resource attribute datadog.process_tags: an arrayValue of
colon-joined "key:value" strings. ...
"""
...
assert any(f"datadog.{tag}" in resource_attrs for tag in _PROCESS_TAG_KEYS), (
f"Expected at least one datadog.<process-tag> resource attribute, got: {list(resource_attrs)}"
)The docstring (and the |
This was referenced Aug 4, 2026
mabdinur
marked this pull request as draft
August 4, 2026 21:13
…spec, expand manifest skips - test_fr08_8_process_tags: revert to checking the single combined datadog.process_tags arrayValue (the main merge had regressed it to check per-key datadog.<key> attributes, which no longer matches its own docstring; flagged in PR review comment 5183792859). - test_fr08_10/test_fr08_11 (DD_TAGS / OTEL_RESOURCE_ATTRIBUTES): rename the expected resource attribute from tracer_dd_tags to datadog.tracer_tags, using the same colon-joined arrayValue shape as datadog.process_tags. - Add/refresh missing_feature skips in python/nodejs/java/dotnet manifests based on the cross-repo OTLP trace-metrics findings audit and the latest CI run (span.kind canonicalization, status.code SMC string convention, is_trace_root wiring, process_tags shape, DD_APM_TRACING_ENABLED gating). Co-authored-by: Cursor <cursoragent@cursor.com>
mabdinur
marked this pull request as ready for review
August 5, 2026 04:40
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7e08a9f299
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Fix OTLP env/fixture usage in FR01/FR02/FR08 tests, normalize Java protobuf payloads via _wait_for_otlp_metrics, align default service.name assertions with spec, and register FFE_FLAG_CONFIGURATION_RULES for java >=1.64.2. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
test_fr02_3 asserts Datadog-Client-Computed-Stats on native trace requests, so keep DEFAULT_ENVVARS (not OTLP trace export) while still setting Java's DD_TRACE_TRACER_METRICS_ENABLED alongside DD_TRACE_STATS_COMPUTATION_ENABLED. Co-authored-by: Cursor <cursoragent@cursor.com>
Removed several tests marked as missing features from the golang manifest.
…hon fr06_7 Add golang missing_feature skips for the four parametric failures on 2.11.0-dev.1 and drop fr08_2/fr08_6 skips that XPASS on dev. Enable python test_fr06_7_rpc_status_code at v4.14.0-dev once libdatadog v38 lands. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Updates
tests/parametric/test_otlp_trace_metrics.pyto match the OTLP Trace Metrics RFC (SEMCON-1093) and adds manifestmissing_featureskips so CI stays green while tracer implementations catch up.Spec corrections:
datadog.tracer_tags/datadog.process_tags/datadog.peer_tags: singlearrayValueof colon-joined"key:value"stringsservice.nameon every data point, including the configured default (test_fr06_15)span.kind/status.code: Span Metrics Connector string convention (SPAN_KIND_*,STATUS_CODE_*)additional_metric_tags: per-key unprefixed data-point attributesManifests: targeted skips in python, nodejs, java, and dotnet for gaps documented against current released SDKs. Skips are removed in a follow-up PR after the tracer fix PRs land:
Merge this before the tracer PRs so parametric CI pins the corrected spec.
Test plan
./format.shpasses