http_server: allow trace http output to allowlisted endpoints

[paulorangeljr]

_Requested by Paulo Junior ·

Before: the trace API (/api/v1/trace and /api/v1/traces) only accepts the stdout and calyptia outputs. Requesting http returns 403, so traced records can only be read from the Fluent Bit log, which is awkward in Kubernetes and mixes trace output with regular logs.

After: operators can opt in to sending trace records to specific HTTP endpoints with a new service option, chunk_trace_http_allowed_endpoints. A trace request with "output": "http" is accepted only when its host, port and (optionally) uri match an entry of that list. With the option unset, which is the default, the http output stays rejected exactly as today.

service:
  http_server: on
  http_listen: 127.0.0.1
  enable_chunk_trace: on
  chunk_trace_http_allowed_endpoints: trace-collector.observability.svc:8080/trace, [::1]:9880

curl -X POST http://127.0.0.1:2020/api/v1/trace/syslog_in \
  -H 'Content-Type: application/json' \
  -d '{"output":"http","params":{"host":"trace-collector.observability.svc","port":"8080","uri":"/trace","format":"json"},"limit":{"count":1}}'

How

• Entries are host:port or [ipv6]:port, optionally followed by a path. With a path, the requested uri must be exactly that path; without one, any uri on that host and port is allowed. Hosts are compared case-insensitively and are not resolved. A missing port is treated as 80 and a missing uri as /, matching out_http defaults; host is required. Malformed entries are skipped with a warning.
• Only parameters that pick the endpoint or encode the payload are accepted for http: host, port, uri, format, json_date_key, json_date_format, header, compress, tls. Everything else (proxy, http_user/http_passwd, tls.* file options, aws_*, oauth2.*, workers, match...) is rejected with 403, since those could redirect records elsewhere or read local files and credentials.
• Parameter values containing ${ are rejected, because output properties expand environment variables and a request could otherwise exfiltrate the environment to the allowed endpoint.
• The allowlist and the existing stdout/calyptia checks run before any input lookup, so the batch endpoint reports returncode: 403 per input as before.

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• Example configuration file for the change
• Debug log output from testing the change
• Attached Valgrind output that shows no leaks or memory corruption was found

New integration tests in tests/integration/scenarios/internal_http_server:

• test_internal_http_server_trace_http_output_allowlist: ten rejected variants (wrong uri, port, host, malformed port, missing host, proxy, tls.ca_file, http_user, ${HOME} in header and uri) return 403 on both endpoints and nothing reaches the receiver; the allowed endpoint then receives the traced records.
• test_internal_http_server_trace_http_output_disabled_by_default: with the option unset, http is still 403 and the log names the option to set.

$ FLUENT_BIT_BINARY=build/bin/fluent-bit pytest -q scenarios/internal_http_server/tests/test_internal_http_server_001.py
12 passed

$ VALGRIND=1 VALGRIND_STRICT=1 pytest -q ... -k "trace_http or rejects_unsafe"
3 passed
==18812== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Log output for rejected requests:

[error] trace output 'http' endpoint '127.0.0.1:18080/other' is not allowed
[error] trace output 'http' endpoint '127.0.0.1:18081/trace' is not allowed
[error] trace output parameter 'proxy' is not allowed
[error] trace output parameter 'header' must not reference environment variables
[error] trace output 'http' requires the 'host' parameter
[error] trace output 'http' has an invalid port '18080x'
[error] trace output 'HTTP' is not allowed

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• [N/A] Run local packaging test showing all targets (including any new ones) build.
• [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

• Documentation required for this feature

A docs PR to fluent/fluent-bit-docs will follow, covering the new service option and the security trade-off (the HTTP server has no authentication, so http_listen 127.0.0.1 remains the recommendation).

Backporting

• Backport to latest stable release.

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features
  • HTTP trace output destinations can now be explicitly allowlisted, with controls for host, port, and optional URI path.
  • HTTP trace output is rejected by default. Requests to unlisted destinations or with unsupported options are denied, and values containing variable placeholders are not accepted.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	⚠️ Failed 2026-10-16T18:15:17.962696Z	70b0430	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: bf156de3-6181-4208-b033-fbc14865251f

📥 Commits

Reviewing files that changed from the base of the PR and between 4dcadc0 and 70b0430.

📒 Files selected for processing: 5

• include/fluent-bit/flb_config.h
• src/flb_config.c
• src/http_server/api/v1/trace.c
• tests/integration/scenarios/internal_http_server/config/internal_http_server_trace_http.yaml
• tests/integration/scenarios/internal_http_server/tests/test_internal_http_server_001.py

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

📝 Walkthrough

The service configuration now accepts an HTTP trace destination allowlist. The internal HTTP API validates HTTP trace output properties and destinations against that list. Integration tests cover rejected and accepted destinations, as well as the default rejection.

Changes: HTTP trace destination allowlist

Layer / File(s)	Summary
Allowlist configuration and lifecycle include/fluent-bit/flb_config.h, src/flb_config.c	The service configuration defines the allowed HTTP trace destination property, maps it to the configuration field, and frees the field during configuration cleanup.
HTTP output validation and integration coverage src/http_server/api/v1/trace.c, tests/integration/scenarios/internal_http_server/config/internal_http_server_trace_http.yaml, tests/integration/scenarios/internal_http_server/tests/test_internal_http_server_001.py	HTTP trace requests are checked against allowed host, port, and URI destinations. The validation rejects unlisted properties and values containing ${. Integration tests cover rejected requests, accepted output, and rejection when no allowlist is configured.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant TraceAPI
  participant trace_output_allowed
  participant HTTPEndpointValidation
  participant flb_config
  participant HTTPReceiver
  TraceAPI->>trace_output_allowed: Pass output properties and server configuration
  trace_output_allowed->>HTTPEndpointValidation: Validate HTTP output
  HTTPEndpointValidation->>flb_config: Read allowed HTTP trace destinations
  alt Destination matches the allowlist
    HTTPEndpointValidation-->>TraceAPI: Accept output
    TraceAPI->>HTTPReceiver: Send trace records
  else Destination is not allowed
    HTTPEndpointValidation-->>TraceAPI: Reject output
    TraceAPI-->>TraceAPI: Return HTTP 403
  end

Loading

Suggested reviewers: edsiper

Merge Risk: ⚪ Minimal · up to 70b04

No actionable merge-blocking issue remains; normal checks can proceed.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 13.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 15 functions across 4 files. (1 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: allowing trace HTTP output only for allowlisted endpoints.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 13.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 15 functions across 4 files. (1 skipped: 1 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}