Skip to content

in_podman_metrics: add exclude_name_regex option - #11970

Open
stondo wants to merge 1 commit into
fluent:masterfrom
stondo:in_podman_metrics-exclude-name-regex
Open

stondo wants to merge 1 commit into
fluent:masterfrom
stondo:in_podman_metrics-exclude-name-regex

Conversation

@stondo

@stondo stondo commented Jun 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add an optional exclude_name_regex configuration option to in_podman_metrics
that skips containers whose name matches a user-supplied regular expression,
before their metrics are collected and emitted.

This mirrors the regex-filtering model already used elsewhere in Fluent Bit,
e.g. netdev.ignore_device_regex in in_node_exporter_metrics. A typical use
is dropping Podman pod infrastructure containers, for example
exclude_name_regex '-(infra|service)$' to skip the pause and catatonit
containers that carry no useful application metrics.

The option defaults to unset, so existing behaviour (collect every container)
is unchanged and the feature is fully opt-in.

Example configuration

pipeline:
  inputs:
    - name: podman_metrics
      scrape_interval: 60
      exclude_name_regex: '-(infra|service)$'

Implementation

  • New FLB_CONFIG_MAP_STR option exclude_name_regex (default NULL).
  • The regex is compiled once in in_metrics_init via flb_regex_create.
  • Init-failure handling: the regex is compiled and a bad pattern fails init
    cleanly before flb_input_set_context / flb_input_set_collector_time, so
    no collector is ever armed against an uninitialised context. The compiled
    regex is also released on the later init error paths (collector setup and
    initial scrape failures) so it never leaks on a failed startup.
  • During collect_container_data, a container is skipped when its name matches
    the compiled regex; otherwise it is added and counted as before.
  • The compiled regex is released in in_metrics_exit.

Testing

  • Built and run against Podman pods on an aarch64 Linux device.
  • With exclude_name_regex '-(infra|service)$': only application containers
    emit metrics; pod -infra/-service (pause/catatonit) containers are
    excluded at the source. Verified via the prometheus exporter output.
  • With the option unset: output identical to current behaviour (every
    container emitted).
  • Invalid regex: init fails with could not compile exclude_name_regex '<value>',
    no crash and no armed collector.

Documentation

  • A companion documentation PR for fluent-bit-docs documents the new
    exclude_name_regex option.

Signed-off-by: Stefano Tondo stondo@gmail.com

Summary by CodeRabbit

Release Notes

  • New Features
    • Added the optional exclude_name_regex configuration option to omit Podman containers with matching names from metrics collection. Containers that do not match the configured pattern continue to be collected.
  • Bug Fixes
    • Container image names now default to "unknown" when metadata is unavailable.

@coderabbitai

coderabbitai Bot commented Jun 20, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b78e9c5b-0173-41da-be4e-16841ea50853
Commits

Reviewing files that changed from the base of the PR and between 5781050 and 6e797ed.

Files selected for processing: 2
  • plugins/in_podman_metrics/podman_metrics.c
  • plugins/in_podman_metrics/podman_metrics_config.h

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

Walkthrough

The Podman metrics input plugin adds an exclude_name_regex option. The plugin compiles the configured regex during initialization and skips containers whose names match it. It destroys the compiled regex if collector setup fails or when the plugin exits.

Changes: Podman metrics container-name exclusion

Layer / File(s) Summary
Config contract: struct fields and config_map entry
plugins/in_podman_metrics/podman_metrics_config.h, plugins/in_podman_metrics/podman_metrics.h
flb_in_metrics gains fields for the regex text and compiled regex. The config map connects exclude_name_regex to the text field.
Regex initialization, filtering, and cleanup
plugins/in_podman_metrics/podman_metrics.c
The plugin compiles the configured regex during initialization and returns an error if compilation fails. Container collection skips names that match the regex. The plugin destroys the compiled regex if collector setup fails or when the plugin exits.

Priority: Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Feature

Merge Risk: Low · up to 6e797

A rare initialization failure can leak plugin context memory; free the context on collector setup failure before merging.

Pre-merge checks: Passed 5
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title "in_podman_metrics: add exclude_name_regex option" names the main change. The PR adds an optional exclude_name_regex setting that skips containers whose names match the regex before their me…
Docstring Coverage Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 2 files.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title "in_podman_metrics: add exclude_name_regex option" names the main change. The PR adds an optional exclude_name_regex setting that skips containers whose names match the regex before their metrics are collected. The title is short, specific, and uses the plugin name and option name the changeset uses. It does not describe the compile, cleanup, or documentation details, which is acceptable.

Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@stondo

stondo commented Jun 20, 2026

Copy link
Copy Markdown
Contributor Author

Companion documentation PR: fluent/fluent-bit-docs#2604

Testing evidence

Built via Yocto (Fluent Bit 5.0.2 base) and run on an aarch64 Linux device
with Podman pods. Config:

pipeline:
  inputs:
    - name: podman_metrics
      scrape_interval: 60
      scrape_on_start: true
      exclude_name_regex: '-(infra|service)$'

With 9 containers running (3 application + 6 pod -infra/-service), only the
3 application containers emit metrics; all 6 infra/service containers are
excluded at the source. Sample from the Prometheus exporter:

container_memory_usage_bytes{name="ti-logcreator-pod-ti-logcreator",...} 790528
container_memory_usage_bytes{name="example-app-pod-example-app",...} 16777216
container_memory_usage_bytes{name="gw-cloud-connector-pod-gw-cloud-connector",...} 29143040

No -infra/-service series present. With the option unset, output is
unchanged (all 9 containers emitted). An invalid regex fails init cleanly with
could not compile exclude_name_regex '<value>' and leaves no collector armed.

I can add Valgrind output from a dedicated local build if that is required for
the review gate.

(This PR supersedes #11968, which was accidentally orphaned by a force-push from
a shallow clone; same change, clean history.)

stondo added a commit to stondo/fluent-bit-docs that referenced this pull request Jun 20, 2026
Document the new exclude_name_regex configuration option for the
in_podman_metrics input plugin, added in fluent/fluent-bit#11970. The
option skips containers whose name matches a user-supplied regular
expression before their metrics are collected (for example pod infra and
service containers).

Signed-off-by: Stefano Tondo <stondo@gmail.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/in_podman_metrics/podman_metrics.c`:
- Around line 463-470: The error handling path for the collector setup failure
(when coll_fd_runtime equals -1) destroys the exclude_name_regex but fails to
free the ctx structure itself before returning. Add a call to free the ctx
structure after destroying the regex to ensure proper cleanup and prevent memory
leaks. Check other error paths in the same function to match the cleanup pattern
used elsewhere when releasing ctx on initialization failures.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9b161edf-6e8e-421c-96d0-17db3d76e49b

📥 Commits

Reviewing files that changed from the base of the PR and between 9bf9c1b and 5781050.

📒 Files selected for processing (3)
  • plugins/in_podman_metrics/podman_metrics.c
  • plugins/in_podman_metrics/podman_metrics.h
  • plugins/in_podman_metrics/podman_metrics_config.h

Comment on lines 463 to 470
flb_input_set_context(in, ctx);
coll_fd_runtime = flb_input_set_collector_time(in, cb_metrics_collect_runtime, ctx->scrape_interval, 0, config);
if (coll_fd_runtime == -1) {
flb_plg_error(ctx->ins, "Could not set collector for podman metrics plugin");
if (ctx->exclude_name_regex) {
flb_regex_destroy(ctx->exclude_name_regex);
}
return -1;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Free plugin context on collector setup failure.

Line 465-470 returns after destroying the compiled regex, but ctx is not released on this error path. That makes init cleanup inconsistent with other failure branches and can leak initialization state.

Suggested fix
-    flb_input_set_context(in, ctx);
     coll_fd_runtime = flb_input_set_collector_time(in, cb_metrics_collect_runtime, ctx->scrape_interval, 0, config);
     if (coll_fd_runtime == -1) {
         flb_plg_error(ctx->ins, "Could not set collector for podman metrics plugin");
         if (ctx->exclude_name_regex) {
             flb_regex_destroy(ctx->exclude_name_regex);
         }
+        flb_free(ctx);
         return -1;
     }
     ctx->coll_fd_runtime = coll_fd_runtime;
+    flb_input_set_context(in, ctx);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/in_podman_metrics/podman_metrics.c` around lines 463 - 470, The error
handling path for the collector setup failure (when coll_fd_runtime equals -1)
destroys the exclude_name_regex but fails to free the ctx structure itself
before returning. Add a call to free the ctx structure after destroying the
regex to ensure proper cleanup and prevent memory leaks. Check other error paths
in the same function to match the cleanup pattern used elsewhere when releasing
ctx on initialization failures.

@stondo

stondo commented Jun 20, 2026

Copy link
Copy Markdown
Contributor Author

CodeRabbit flagged on the previous iteration (#11968) that the compiled exclude_name_regex was leaked on the init error paths after allocation. That's addressed in this PR: flb_regex_destroy(ctx->exclude_name_regex) is now called on both later failure paths in in_metrics_init — the collector-setup failure (flb_input_set_collector_time returning -1) and the initial-scrape failure (scrape_metrics(...) == -1) — in addition to the normal in_metrics_exit cleanup. Combined with compiling the regex before the collector is armed, a bad pattern or a failed startup no longer leaks the regex or leaves a live collector behind.

@stondo

stondo commented Aug 21, 2026

Copy link
Copy Markdown
Contributor Author

Friendly ping. This has been approved by @bachp since early July and CI is green. Is anything else needed from my side before it can be merged?

The companion docs change is fluent/fluent-bit-docs#2604.

@stondo

stondo commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Second gentle ping. This has been approved by @bachp since early July and CI is green; the companion docs change is fluent/fluent-bit-docs#2604, which is waiting on this one to merge.

Is anything still needed from my side?

Related, in case it helps a reviewer batch the plugin work: #11719 is still open, and I filed #12391 today for a separate issue where a failing initial scrape with scrape_on_start aborts the whole Fluent Bit process rather than degrading just this input. All three are independent.

@stondo

stondo commented Sep 21, 2026

Copy link
Copy Markdown
Contributor Author

@cosmo0920 This one adds an exclude_name_regex option to in_podman_metrics, approved by @bachp in July. It depends on #11719 landing first. Docs PR is #2604.

Add an optional 'exclude_name_regex' configuration option to skip
containers whose name matches a user-supplied regular expression before
their metrics are collected and emitted.

This mirrors the regex filtering model already used by the node exporter
(netdev.ignore_device_regex) and the diskstats/filesystem collectors. A
typical use is dropping Podman pod infrastructure containers, e.g.
exclude_name_regex '-(infra|service)$' to skip pause and catatonit
containers that carry no useful application metrics.

The option defaults to unset, so existing behaviour (collect every
container) is unchanged.

Signed-off-by: Stefano Tondo <stondo@gmail.com>

This branch was previously deployed

1 inactive deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants