Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions docs/examples/multi-destination-exporting/README.rst
Comment thread
tammy-baylis-swi marked this conversation as resolved.
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
Multi-Destination Exporting
===========================

This example shows how to export telemetry data to multiple destinations
simultaneously. As described in the `OTLP specification
<https://opentelemetry.io/docs/specs/otlp/#multi-destination-exporting>`_,
each destination should have implemented its own queuing, acknowledgement
handling, and retry logic to prevent one slow or unavailable destination
from blocking the others.

The OpenTelemetry Python SDK achieves this by using a separate processor
or reader per destination:

* **Traces**: Use one ``BatchSpanProcessor`` per destination, each wrapping
its own ``SpanExporter``. Add each processor to the ``TracerProvider``
via ``add_span_processor()``.

* **Metrics**: Pass multiple ``MetricReader`` instances (each wrapping its
own ``MetricExporter``) to the ``MeterProvider`` constructor via the
``metric_readers`` parameter.

* **Logs**: Use one ``BatchLogRecordProcessor`` per destination, each
wrapping its own ``LogExporter``. Add each processor to the
``LoggerProvider`` via ``add_log_record_processor()``.

.. note::

The **Profiles** signal is not yet supported in the Python SDK.
When it becomes available, the same pattern will apply.

The source files of these examples are available :scm_web:`here <docs/examples/multi-destination-exporting/>`.

Installation
------------

.. code-block:: sh

pip install opentelemetry-api
pip install opentelemetry-sdk
pip install opentelemetry-exporter-otlp-proto-grpc
pip install opentelemetry-exporter-otlp-proto-http
pip install opentelemetry-instrumentation-logging # For LoggingHandler

Run the Example
---------------

.. code-block:: sh

python multi_destination_traces.py
python multi_destination_metrics.py
python multi_destination_logs.py

The output will be shown in the console for the ``ConsoleSpanExporter``,
``ConsoleMetricExporter``, and ``ConsoleLogRecordExporter`` destinations.
The OTLP destinations require a running `collector
<https://opentelemetry.io/docs/collector/>`_.

Useful links
------------

- `OTLP multi-destination exporting specification <https://opentelemetry.io/docs/specs/otlp/#multi-destination-exporting>`_
- OpenTelemetry_
- :doc:`../../api/trace`
- :doc:`../../api/metrics`
- :doc:`../../api/_logs`

.. _OpenTelemetry: https://github.com/open-telemetry/opentelemetry-python/
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Copyright The OpenTelemetry Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""
This example shows how to export logs to multiple destinations.
Each BatchLogRecordProcessor has its own queue and retry logic, so
destinations do not block each other.
"""

import logging

from opentelemetry._logs import set_logger_provider
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import (
OTLPLogExporter as GrpcLogExporter,
)
from opentelemetry.exporter.otlp.proto.http._log_exporter import (
OTLPLogExporter as HttpLogExporter,
)

# this is available in the opentelemetry-instrumentation-logging package
from opentelemetry.instrumentation.logging.handler import LoggingHandler
from opentelemetry.sdk._logs import LoggerProvider
from opentelemetry.sdk._logs.export import (
BatchLogRecordProcessor,
ConsoleLogRecordExporter,
)

logger_provider = LoggerProvider()
set_logger_provider(logger_provider)

# Destination 1: OTLP over gRPC
grpc_exporter = GrpcLogExporter(
endpoint="http://localhost:4317", insecure=True
)
logger_provider.add_log_record_processor(
BatchLogRecordProcessor(grpc_exporter)
)

# Destination 2: OTLP over HTTP
http_exporter = HttpLogExporter(endpoint="http://localhost:4318/v1/logs")
logger_provider.add_log_record_processor(
BatchLogRecordProcessor(http_exporter)
)

# Destination 3: Console (for debugging)
logger_provider.add_log_record_processor(
BatchLogRecordProcessor(ConsoleLogRecordExporter())
)

# Bridge Python's logging to OpenTelemetry
handler = LoggingHandler(level=logging.NOTSET, logger_provider=logger_provider)
logging.getLogger().setLevel(logging.NOTSET)
logging.getLogger().addHandler(handler)

logger = logging.getLogger("myapp")
logger.info("Logs are exported to all three destinations.")
logger.warning("This warning also goes everywhere.")

logger_provider.shutdown()
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Copyright The OpenTelemetry Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""
This example shows how to export metrics to multiple destinations.
Each PeriodicExportingMetricReader has its own collection interval
and export queue, so destinations do not block each other.
"""

from opentelemetry import metrics
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import (
OTLPMetricExporter as GrpcMetricExporter,
)
from opentelemetry.exporter.otlp.proto.http.metric_exporter import (
OTLPMetricExporter as HttpMetricExporter,
)
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import (
ConsoleMetricExporter,
PeriodicExportingMetricReader,
)

# Destination 1: OTLP over gRPC
grpc_reader = PeriodicExportingMetricReader(
GrpcMetricExporter(endpoint="http://localhost:4317", insecure=True)
)

# Destination 2: OTLP over HTTP
http_reader = PeriodicExportingMetricReader(
HttpMetricExporter(endpoint="http://localhost:4318/v1/metrics")
)

# Destination 3: Console (for debugging)
console_reader = PeriodicExportingMetricReader(ConsoleMetricExporter())

# Pass all readers to the MeterProvider
provider = MeterProvider(
metric_readers=[grpc_reader, http_reader, console_reader]
)
metrics.set_meter_provider(provider)

meter = metrics.get_meter(__name__)
counter = meter.create_counter(
"request.count", description="Number of requests"
)

counter.add(1, {"endpoint": "/api/users"})
counter.add(1, {"endpoint": "/api/orders"})

print("Metrics are exported to all three destinations.")

provider.shutdown()
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Copyright The OpenTelemetry Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""
This example shows how to export traces to multiple destinations.
Each BatchSpanProcessor has its own queue and retry logic, so
destinations do not block each other.
"""

from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import (
OTLPSpanExporter as GrpcSpanExporter,
)
from opentelemetry.exporter.otlp.proto.http.trace_exporter import (
OTLPSpanExporter as HttpSpanExporter,
)
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import (
BatchSpanProcessor,
ConsoleSpanExporter,
)

provider = TracerProvider()
trace.set_tracer_provider(provider)

# Destination 1: OTLP over gRPC
grpc_exporter = GrpcSpanExporter(
endpoint="http://localhost:4317", insecure=True
)
provider.add_span_processor(BatchSpanProcessor(grpc_exporter))

# Destination 2: OTLP over HTTP
http_exporter = HttpSpanExporter(endpoint="http://localhost:4318/v1/traces")
provider.add_span_processor(BatchSpanProcessor(http_exporter))

# Destination 3: Console (for debugging)
provider.add_span_processor(BatchSpanProcessor(ConsoleSpanExporter()))

tracer = trace.get_tracer(__name__)

with tracer.start_as_current_span("example-request"):
with tracer.start_as_current_span("fetch-data"):
print("Spans are exported to all three destinations.")

provider.shutdown()
Loading