Conversation
…ainer The mapproxy and mapproxinator containers do not speak the same OTLP transport. mapproxy uses the gRPC exporter (src/telemetry/__init__.py imports opentelemetry.exporter.otlp.proto.grpc) and needs a bare host:4317, while mapproxinator uses @map-colonies/telemetry, which is built on @opentelemetry/exporter-trace-otlp-proto - OTLP over HTTP, requiring host:4318/v1/traces. Both configmaps rendered from the same merged tracing.url, so no single value could satisfy both: whichever port was configured, one of the two containers silently dropped every span. Add an opt-in mapproxinator.tracing.url that takes precedence over the shared endpoint. Empty by default, so the rendered output is unchanged for anyone not setting it.
Contributor
Author
|
Downstream drafts are now open and waiting on this release:
Both downstream PRs assume this lands as a |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The two containers in this chart's pod do not speak the same OTLP transport, but the chart renders both from a single merged
tracing.url.mapproxy(python)opentelemetry.exporter.otlp.proto.grpc(src/telemetry/__init__.py:20)host:4317, no pathmapproxinator(node)@map-colonies/telemetry->@opentelemetry/exporter-trace-otlp-protohost:4318/v1/traces-protois the easy trap here: it means protobuf over HTTP, not gRPC (that would be-grpc).Both
templates/mapproxy/mapproxy-configmap.yaml:24andtemplates/mapproxinator/mapproxinator-configmap.yaml:27read the samecommon.tracing.mergedvalue, so whichever port is configured, one container drops every span - silently, because OTel's diagnostics are off by default.Observed in a live deployment
With
tracing.url = http://infra-otel.infra-services:4317, probing the collector from inside the running mapproxinator container:HPE_INVALID_CONSTANTis Node's HTTP/1.1 parser hitting the gRPC listener's HTTP/2 response. The image ships onlyexporter-trace-otlp-proto- there is no gRPC exporter present, so this is not a configuration preference. Meanwhile mapproxy logs[otel-probe] collector REACHABLE at ...:4317and traces fine.Upgrading mapproxinator does not resolve it:
mapproxinator@1.3.0moved to@map-colonies/tracing@1.0.0, which still depends on@opentelemetry/exporter-trace-otlp-proto.Change
An opt-in
mapproxinator.tracing.urlthat takes precedence over the shared endpoint, empty by default:The configmap now renders
.Values.mapproxinator.tracing.url | default $tracing.url.Chart.yamlis deliberately untouched - release-please owns$.versionand$.appVersionviarelease-please-config.json. As afeatthis should land as 2.2.0.Verification
helm templateagainst this branch:helm lintpasses (only the pre-existing "icon is recommended" info). Rendered output is byte-identical for any consumer that does not set the new key.Follow-ups (separate repos)
helm-charts- bump themapproxydependency inraster/charts/serving/Chart.yamlonce this releasessite-values- bumpchartsVersions.servingand setmapproxinator.tracing.urlto the collector's HTTP endpointConsidered and rejected
Adding a matching override for the mapproxy container, or restructuring
tracingintogrpcUrl/httpUrl. Both widen the chart's public API for a problem that only needs one escape hatch; the sharedtracing.urlalready serves mapproxy correctly.