Skip to content

Add OpenTelemetry tracing instrumentation to API layer#13132

Open
sureshanaparti wants to merge 2 commits intoapache:4.22from
shapeblue:otel-tracing-support
Open

Add OpenTelemetry tracing instrumentation to API layer#13132
sureshanaparti wants to merge 2 commits intoapache:4.22from
shapeblue:otel-tracing-support

Conversation

@sureshanaparti
Copy link
Copy Markdown
Contributor

@sureshanaparti sureshanaparti commented May 8, 2026

Description

This PR add OpenTelemetry tracing instrumentation to API layer.

When the OTel Java agent is attached (via -javaagent), every API request is traced with:

  • Per-command span naming: Each trace is tagged with the API command name (e.g. listVirtualMachines, deployVirtualMachine) via Span.current().updateName() and a filterable api.command attribute, making it possible to analyze latency per API command.
  • Automatic JDBC tracing: The OTel agent auto-instruments all database queries — every SELECT, INSERT, UPDATE shows up as a child span with table name and duration. No code changes required for this.
  • Automatic HTTP client tracing: Outbound HTTP calls (e.g. to hypervisor agents) are auto-instrumented by the agent.

All changes are no-ops without the OTel agent deployed. The @WithSpan annotation is ignored, Span.current() returns a no-op, and the opentelemetry-api calls return immediately with zero overhead. The two added dependencies (opentelemetry-instrumentation-annotations and opentelemetry-api) are lightweight JARs (~50KB + ~200KB) with no transitive dependencies.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • Build/CI
  • Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

Tracing output is visible in Grafana/Tempo when the OTel agent is attached.

How did you try to break this feature and the system with this change?

tmckeon and others added 2 commits May 8, 2026 14:17
Add OTel annotations and API dependencies to enable distributed tracing
of CloudStack API requests. The @WithSpan annotation on handleRequest()
tags each trace with the API command name (e.g. listVirtualMachines),
making it possible to filter and analyze per-command latency. The OTel
agent auto-instruments JDBC and HTTP calls, providing full request
waterfalls with no additional code changes.

All instrumentation is inert without the OTel Java agent attached:
@WithSpan is ignored, Span.current() returns a no-op, and the
opentelemetry-api calls return immediately with zero overhead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…gement

Centralizes OTel version definitions to avoid version skew across modules
and removes unused OTel dependency from api module.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@sureshanaparti
Copy link
Copy Markdown
Contributor Author

@blueorangutan package

@blueorangutan
Copy link
Copy Markdown

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 8, 2026

Codecov Report

❌ Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.67%. Comparing base (ddcc0c8) to head (bd73ccf).

Files with missing lines Patch % Lines
server/src/main/java/com/cloud/api/ApiServer.java 0.00% 3 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               4.22   #13132      +/-   ##
============================================
- Coverage     17.68%   17.67%   -0.01%     
+ Complexity    15794    15790       -4     
============================================
  Files          5922     5922              
  Lines        533095   533098       +3     
  Branches      65208    65209       +1     
============================================
- Hits          94281    94237      -44     
- Misses       428171   428223      +52     
+ Partials      10643    10638       -5     
Flag Coverage Δ
uitests 3.69% <ø> (ø)
unittests 18.75% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@blueorangutan
Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17781

@sureshanaparti sureshanaparti requested a review from nvazquez May 8, 2026 12:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants