From 1a7d2e43e5c0a8920ed2ff8d7f174eca648abcb6 Mon Sep 17 00:00:00 2001 From: Joana Carvalho Date: Sat, 3 Oct 2026 13:42:04 +0000 Subject: [PATCH] test(e2e): run the Rust golden crate end to end, and assert http.route (#118) tests/e2e/run.sh gains a Rust leg: the golden crate (Cargo.lock included) is built with cargo run --locked on rust:1.88, exports OTLP/HTTP to the golden agent Collector config, and products-api is asserted in Jaeger with its service.* resource attributes. assert-traces.sh gains --expect-span k=v. The Rust leg uses it for http.route=/products/{id}, because the tower crate's axum feature is off by default and without it the server span still exports with correct resource attributes but carries no route. Resource-level assertions cannot see that. assert-traces.test.sh covers the new flag, including the route-less case. The e2e job timeout goes 25 -> 35 minutes for the cold cargo build. README, ROADMAP and the e2e README are updated. No plugin version bump: no command, agent, skill or hook changed. Verified locally without Docker (not available in this environment): the seeded directory builds with --locked on 1.88, and run against a local Collector 0.161.0 it exports the expected resource and GET /products/{id} with http.route=/products/{id}. The compose leg itself has not been run; its first run is in CI. --- .github/workflows/ci.yml | 5 +++-- README.md | 6 +++--- ROADMAP.md | 4 ++-- tests/e2e/README.md | 13 +++++++++++++ tests/e2e/assert-traces.sh | 25 ++++++++++++++++++++---- tests/e2e/assert-traces.test.sh | 19 ++++++++++++++++++ tests/e2e/docker-compose.yml | 24 +++++++++++++++++++++++ tests/e2e/run.sh | 34 +++++++++++++++++++++++++++++---- 8 files changed, 115 insertions(+), 15 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 173726e..be0b614 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -171,8 +171,9 @@ jobs: # the Java agent download + a .NET `dotnet restore`/build); a healthy run is ~4-8 min. 25 caps # a genuine hang — run.sh's internal waits bound a healthy run well under this, and without a # job cap a stuck container would run to the 6-hour default (as a run on PR #76 was heading - # toward). Bumped 20→25 when the .NET app was added (a fifth inline restore+build). - timeout-minutes: 25 + # toward). Bumped 20→25 when the .NET app was added (a fifth inline restore+build), and 25→35 + # when the Rust app was added: a cold `cargo run` compiles a few hundred crates. + timeout-minutes: 35 steps: - uses: actions/checkout@v4 diff --git a/README.md b/README.md index 353200b..17fd324 100644 --- a/README.md +++ b/README.md @@ -28,14 +28,14 @@ Terraform for four backends (Grafana, Datadog, New Relic, Dash0). Generated output is validated for **syntax and schema** — every backend module is `terraform validate`d in CI, and the OTel SDK/semconv pins are checked against current -releases weekly (see the drift check). The Node.js, Python, Java, .NET, Go, and Ruby +releases weekly (see the drift check). The Node.js, Python, Java, .NET, Go, Ruby, and Rust instrumentation is additionally **proven end-to-end in CI**: the generated SDK bootstrap (for Java, the OpenTelemetry Java agent; for .NET, the generated `OpenTelemetry.cs` wired into a real ASP.NET Core app; for Go, the generated `tracing.go` wired into a real `net/http` server; for Ruby, the generated `tracing.rb` wired into a real Sinatra app) exports through the generated Collector config into a running trace store, asserted on every push (see -[`tests/e2e/`](tests/e2e/)). Rust is newer: its golden crate is compiled in CI, and the output -was run once against a local Collector by hand, but there is no CI end-to-end run for it yet. +[`tests/e2e/`](tests/e2e/)). For Rust, the generated `telemetry.rs` wired into a real axum app is run the same way, and the +server span is checked for `http.route`. The **Terraform is not yet proven against live vendor backends** — before trusting it, run `terraform plan` / `apply` against your own account. Treat backend modules as a reviewed starting point, not turnkey infrastructure. An opt-in CI job diff --git a/ROADMAP.md b/ROADMAP.md index 8e70f28..8b0c813 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -87,8 +87,8 @@ just a larger surface to be wrong on. So the near-term work is depth. ## Later: toward v1 — [milestone](https://github.com/radra23/otel-as-code-plugin/milestone/3) - **PHP, and the rest of Rust.** Rust phase 1 shipped in 0.8.15: axum 0.8, with the golden crate - compiled in CI, and the weekly drift check watches its crate pins. Still to come for Rust are an - end-to-end CI run, actix-web, and semconv-lint support for Rust attribute setters + run end-to-end in CI, and the weekly drift check watches its crate pins. Still to come for Rust + are actix-web and semconv-lint support for Rust attribute setters ([#118](https://github.com/radra23/otel-as-code-plugin/issues/118)). PHP is at the design stage ([#117](https://github.com/radra23/otel-as-code-plugin/issues/117)). - **Cardinality and cost budgets as generated policy**, not just as guardrail comments — a diff --git a/tests/e2e/README.md b/tests/e2e/README.md index 836f67b..ed4c988 100644 --- a/tests/e2e/README.md +++ b/tests/e2e/README.md @@ -89,6 +89,19 @@ every push/PR. #4 stays local-only. collector and both apps, runs `docker compose down -v --remove-orphans`, and removes `tests/e2e/.work/` — regardless of whether the assertions passed. +## The Rust leg + +`rust-app` (products-api, publishes `8084:8080`) runs the **golden crate** from +`tests/snapshots/instrument/rust/` (Cargo.lock included) with `cargo run --locked` on +`rust:1.88`, exporting OTLP/HTTP to the collector's `4318`. It is the slowest leg: a cold +build compiles a few hundred crates, so its readiness wait is 600 s and the job timeout +allows for it. Its fixture's liveness route is `/healthz`, not `/health`. + +Besides the usual resource attributes, it passes `--expect-span 'http.route=/products/{id}'` +to `assert-traces.sh`. That is deliberate: the tower crate's `axum` feature is off by +default, and without it the server span still exports with the right resource but carries +no route. Only a span-level assertion catches that. + ## Expected collector errors (not a failure) During a `run.sh` run the golden collector config routes **all three** pipelines diff --git a/tests/e2e/assert-traces.sh b/tests/e2e/assert-traces.sh index 01b63c3..620a86e 100755 --- a/tests/e2e/assert-traces.sh +++ b/tests/e2e/assert-traces.sh @@ -5,17 +5,18 @@ # response for offline unit testing. JSON parsed with python3 (repo convention). set -euo pipefail -JAEGER=""; SERVICE=""; EXPECT=""; TRACES_JSON="" +JAEGER=""; SERVICE=""; EXPECT=""; TRACES_JSON=""; SPAN_EXPECT="" while [ $# -gt 0 ]; do case "$1" in --jaeger) JAEGER="$2"; shift 2;; --service) SERVICE="$2"; shift 2;; --expect) EXPECT="$2"; shift 2;; --traces-json) TRACES_JSON="$2"; shift 2;; + --expect-span) SPAN_EXPECT="$2"; shift 2;; *) echo "unknown arg: $1" >&2; exit 2;; esac done -[ -n "$SERVICE" ] && [ -n "$EXPECT" ] || { echo "usage: --service --expect k=v,... (--jaeger | --traces-json )" >&2; exit 2; } +[ -n "$SERVICE" ] && [ -n "$EXPECT" ] || { echo "usage: --service --expect k=v,... (--jaeger | --traces-json ) [--expect-span k=v,...]" >&2; exit 2; } # NOTE: the python source is captured into a variable via `$(cat <<'PY' ... PY)` # rather than run directly as `python3 - <<'PY'` — the latter feeds the heredoc as @@ -48,12 +49,28 @@ if not matched: for k, v in expect.items(): if tags.get(k) != v: print(f"FAIL: {service} expected {k}={v}, got {tags.get(k)!r} (tags present: {sorted(tags)})"); sys.exit(1) -print(f"OK: {service} has {', '.join(f'{k}={v}' for k,v in expect.items())}") +# Optional span-level check (--expect-span): some span of this service, in any returned trace, +# must carry every key=value. Resource attrs alone cannot tell a route-named server span from an +# anonymous one, which is the failure a missing instrumentation feature produces. +span_expect = dict(kv.split("=", 1) for kv in os.environ.get("SPAN_EXPECT", "").split(",") if kv) +if span_expect: + hit = False + for tr in traces: + pids = {pid for pid, proc in (tr.get("processes") or {}).items() if proc.get("serviceName") == service} + for sp in tr.get("spans") or []: + if sp.get("processID") in pids: + st = {t["key"]: str(t.get("value")) for t in sp.get("tags") or []} + if all(st.get(k) == v for k, v in span_expect.items()): + hit = True + if not hit: + print(f"FAIL: {service} has no span with {', '.join(f'{k}={v}' for k,v in span_expect.items())}"); sys.exit(1) +print(f"OK: {service} has {', '.join(f'{k}={v}' for k,v in expect.items())}" + + (f" and a span with {', '.join(f'{k}={v}' for k,v in span_expect.items())}" if span_expect else "")) PY ) check() { # stdin: /api/traces JSON. Args: service, expect. Exit 0 ok / 1 mismatch. - SERVICE="$1" EXPECT="$2" python3 -c "$PY_CHECK_SRC" + SERVICE="$1" EXPECT="$2" SPAN_EXPECT="$SPAN_EXPECT" python3 -c "$PY_CHECK_SRC" } if [ -n "$TRACES_JSON" ]; then diff --git a/tests/e2e/assert-traces.test.sh b/tests/e2e/assert-traces.test.sh index 4e81d99..cc78eac 100755 --- a/tests/e2e/assert-traces.test.sh +++ b/tests/e2e/assert-traces.test.sh @@ -15,5 +15,24 @@ check "missing service.version fails" \ check "absent service fails" \ '! bash assert-traces.sh --traces-json fixtures/jaeger-ok.json --service nope --expect service.name=nope >/dev/null' +# --expect-span: a span-level attribute must be present on some span of the service +tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT +TMP="$tmp" python3 - <<'PY' +import json, os +d = json.load(open("fixtures/jaeger-ok.json")) +d["data"][0]["spans"][0]["tags"] = [{"key": "http.route", "type": "string", "value": "/checkout"}] +json.dump(d, open(os.environ["TMP"] + "/ok.json", "w")) +d["data"][0]["spans"][0]["tags"] = [] +json.dump(d, open(os.environ["TMP"] + "/missing.json", "w")) +PY +check "span attr present passes" \ + 'bash assert-traces.sh --traces-json $tmp/ok.json --service checkout-api --expect service.name=checkout-api --expect-span http.route=/checkout >/dev/null' +check "span attr absent fails (route-less server span)" \ + '! bash assert-traces.sh --traces-json $tmp/missing.json --service checkout-api --expect service.name=checkout-api --expect-span http.route=/checkout >/dev/null' +check "span attr with the wrong value fails" \ + '! bash assert-traces.sh --traces-json $tmp/ok.json --service checkout-api --expect service.name=checkout-api --expect-span http.route=/other >/dev/null' +check "no --expect-span keeps the old behaviour" \ + 'bash assert-traces.sh --traces-json $tmp/missing.json --service checkout-api --expect service.name=checkout-api >/dev/null' + echo "Results: $pass passed, $fail failed" [ "$fail" -eq 0 ] diff --git a/tests/e2e/docker-compose.yml b/tests/e2e/docker-compose.yml index 1d71258..38f60aa 100644 --- a/tests/e2e/docker-compose.yml +++ b/tests/e2e/docker-compose.yml @@ -193,3 +193,27 @@ services: # collector. command: ["sh", "-c", "bundle install && ruby app.rb"] depends_on: [collector] + + rust-app: + # SDK image because the container compiles the source inline, like the Go and Ruby legs. + # 1.88 is the floor the golden crate's locked dependency tree needs (reqwest -> url -> icu); + # the rust-golden CI job checks the same crate on 1.88 and stable. + image: rust:1.88-bookworm + working_dir: /app + environment: + # The golden telemetry.rs reads service.name from OTEL_SERVICE_NAME. Use a name DISTINCT + # from every other service (the golden's own default, search-api, is the Go leg's name). + OTEL_SERVICE_NAME: "products-api" + DEPLOYMENT_ENV: "e2e" + # Rust exports OTLP/HTTP protobuf, so point at the collector's HTTP port (4318), like the + # Ruby and Next.js legs and unlike the gRPC 4317 the rest use. + OTEL_EXPORTER_OTLP_ENDPOINT: "http://collector:4318" + ports: + - "8084:8080" # load is driven from the host (see run.sh) + volumes: + - ./.work/rust:/app + # The golden crate: the pristine axum fixture plus the generated src/telemetry.rs, the marked + # Cargo.toml block and the three printed main.rs lines, with its Cargo.lock. --locked makes + # the run reproduce the dependency tree the rust-golden job compiles. + command: ["sh", "-c", "cargo run --locked"] + depends_on: [collector] diff --git a/tests/e2e/run.sh b/tests/e2e/run.sh index f6a408e..5f8e628 100755 --- a/tests/e2e/run.sh +++ b/tests/e2e/run.sh @@ -16,7 +16,7 @@ cleanup() { # metrics/logs pipelines (Jaeger's OTLP receiver only accepts traces) — see the # "Expected collector errors" section in README.md before treating those as the failure. echo "--- collector + app logs (tail, timestamped) ---" - docker compose logs --timestamps --tail=60 jaeger collector node-app python-app java-app nextjs-app dotnet-app go-app ruby-app 2>/dev/null || true + docker compose logs --timestamps --tail=60 jaeger collector node-app python-app java-app nextjs-app dotnet-app go-app ruby-app rust-app 2>/dev/null || true echo "--- jaeger /api/services (what actually landed) ---" curl -fsS "http://localhost:16686/api/services" 2>/dev/null && echo || echo "(jaeger /api/services unreachable)" docker compose down -v --remove-orphans 2>/dev/null || true @@ -28,7 +28,7 @@ cleanup() { trap cleanup EXIT # 1. seed throwaway app copies (fixtures stay pristine) -rm -rf "$WORK"; mkdir -p "$WORK/nodejs" "$WORK/python" "$WORK/java" "$WORK/nextjs" "$WORK/dotnet" "$WORK/go" "$WORK/ruby" +rm -rf "$WORK"; mkdir -p "$WORK/nodejs" "$WORK/python" "$WORK/java" "$WORK/nextjs" "$WORK/dotnet" "$WORK/go" "$WORK/ruby" "$WORK/rust" # The collector's file exporter (logs-overlay) writes logs.json here. World-writable so the # collector container's non-root user can write regardless of the host UID that created it. mkdir -p "$WORK/collector-out"; chmod 777 "$WORK/collector-out" @@ -90,6 +90,13 @@ cp -R ../../fixtures/ruby-greenfield/. "$WORK/ruby/" cp ../snapshots/instrument/ruby/tracing.rb "$WORK/ruby/tracing.rb" cp ../snapshots/instrument/ruby/Gemfile "$WORK/ruby/Gemfile" # instrumented manifest +# Rust: the golden crate IS the instrumented fixture (pristine axum app + generated telemetry.rs + +# the marked Cargo.toml block + the three printed main.rs lines), so seed the whole golden +# directory, Cargo.lock included, minus any local build output. +mkdir -p "$WORK/rust/src" +cp ../snapshots/instrument/rust/Cargo.toml ../snapshots/instrument/rust/Cargo.lock "$WORK/rust/" +cp ../snapshots/instrument/rust/src/*.rs "$WORK/rust/src/" + # 2. up + wait for jaeger health docker compose up -d echo "waiting for stack..." @@ -101,15 +108,16 @@ done # 3. drive load from the host against both apps (both ports are published — see # docker-compose.yml). Each wait is bounded so CI can't hang. -wait_ready() { # base_url [timeout_s] +wait_ready() { # base_url [timeout_s] [health_path] local base="$1" + local health_path="${3:-/health}" # Default 150s: the python-app runs `pip install` (grpcio + otlp proto exporter) inline # before serving, which on a cold CI runner can approach 90s alone — 90s was too # tight and risked spurious red on the first live run. The Next.js app does npm install + # next build inline, which is heavier, so it passes a larger timeout (2nd arg). local APP_READY_TIMEOUT_S="${2:-150}" local deadline=$(( $(date +%s) + APP_READY_TIMEOUT_S )) - until curl -fsS "$base/health" >/dev/null 2>&1; do + until curl -fsS "$base$health_path" >/dev/null 2>&1; do [ "$(date +%s)" -lt "$deadline" ] || { echo "FAIL: $base did not become ready within timeout" >&2; exit 1; } sleep 2 done @@ -173,6 +181,16 @@ for i in 1 2 3 4 5; do curl -fsS "http://localhost:8083/notify" >/dev/null || true done +# Rust app (cargo run compiles the crate inline: a cold build fetches and compiles ~300 crates, so +# allow the most time of any leg). Hitting /products/{id} makes opentelemetry-instrumentation-tower +# emit a server span named after the route, exported over OTLP/HTTP (:4318). The fixture's liveness +# route is /healthz, not /health. +wait_ready "http://localhost:8084" 600 /healthz +for i in 1 2 3 4 5; do + curl -fsS "http://localhost:8084/healthz" >/dev/null || true + curl -fsS "http://localhost:8084/products/42" >/dev/null || true +done + # 4. assert (Jaeger reachable on host 16686) JA="http://localhost:16686" # Check BOTH services in one run (don't fail-fast on the first) so a single CI run reports @@ -204,6 +222,14 @@ POLL_TIMEOUT=60 bash assert-traces.sh --jaeger "$JA" --service search-api \ POLL_TIMEOUT=60 bash assert-traces.sh --jaeger "$JA" --service notifications-api \ --expect service.name=notifications-api,service.version=1.1.0,service.namespace=storefront,deployment.environment.name=e2e || rc=1 +# Rust SDK wiring (golden telemetry.rs). service.version and service.namespace are the golden's +# defaults; service.name comes from OTEL_SERVICE_NAME and deployment.environment.name from +# DEPLOYMENT_ENV. --expect-span proves the tower layer's `axum` feature is on: without it the +# server span exists but carries no http.route. +POLL_TIMEOUT=90 bash assert-traces.sh --jaeger "$JA" --service products-api \ + --expect service.name=products-api,service.version=0.4.0,service.namespace=storefront,deployment.environment.name=e2e \ + --expect-span 'http.route=/products/{id}' || rc=1 + # Logs (#12, --experimental): the Python app logs `reserved sku=...` on /inventory/reserve, the # --experimental bootstrap bridges it to OTLP, and the collector's file exporter writes it to # collector-out/logs.json. Assert that record actually arrived with the right service.name.