Skip to content

feat: add Network Operator Grafana dashboard - #3593

Draft
almaslennikov wants to merge 2 commits into
Mellanox:masterfrom
almaslennikov:issue-1435-grafana
Draft

almaslennikov wants to merge 2 commits into
Mellanox:masterfrom
almaslennikov:issue-1435-grafana

Conversation

@almaslennikov

@almaslennikov almaslennikov commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #1435.

Add an importable Grafana dashboard for component readiness, container restarts, controller reconciliation health, and optional host RDMA traffic/errors. Workload discovery supports release-specific and NicNodePolicy-generated names. The setup guide documents scrape prerequisites, single-cluster scope, separate job/namespace selectors, and missing-data semantics.

Validation: all 24 PromQL fixture checks passed with promtool 3.2.1, covering expected values, unrelated-job isolation, and absent metrics. JSON parsing, unique panel IDs, and git diff --check passed. Metric names and labels were checked against kube-state-metrics, the pinned controller-runtime source, and node exporter. No Go or deployment behavior changes.

Grafana runtime import and hardware/cluster validation remain unverified: local Docker containers stayed in Created, preventing the Grafana API check.

Design review requested (draft)

This introduces the dashboard's metric and user-facing interpretation contract. Please review these choices before marking ready:

  • Are the selected readiness/restart/reconciliation panels and label/job/namespace selectors the right default view for Network Operator deployments, including deployment-only operator namespaces and NicNodePolicy-generated workload names?
  • Should the restart chart use a five-minute rolling window, as implemented, or should it show an instant total for the selected dashboard range?
  • Are optional host RDMA panels and their missing-data behavior useful and clear without promising that a zero value means healthy hardware?

Ready after maintainers agree on the dashboard scope and query semantics, the pending Helm/KIND checks finish, and a Grafana import/render check or explicit reviewer acceptance of that remaining validation gap is recorded. The 24 PromQL fixture checks and static checks are recorded above; Grafana runtime import and cluster/hardware behavior remain unverified.

Provide an importable dashboard for component readiness, controller
health, and optional host RDMA telemetry. Document scrape prerequisites,
selector scope, and missing-data behavior.

Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 27, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@greptile-apps

greptile-apps Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Low impact] The PR appears safe to merge; both previous findings are fixed and no new actionable issues were found.

Summary

Adds an importable Grafana dashboard for Network Operator monitoring.

  • Operators can compare component readiness and container restarts in Grafana.
  • Operators can track controller reconciliations and errors over time.
  • Operators can view optional RDMA traffic and port errors.

Grafana import and cluster behavior remain unverified, as noted in the PR description.

Reviews (2) · Last reviewed commit: "fix: correct Grafana namespace and resta..." · Reviewed by Greptile

Comment thread grafana/network-operator-dashboard.json Outdated
Comment thread grafana/network-operator-dashboard.json Outdated
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
@almaslennikov
almaslennikov marked this pull request as draft October 27, 2026 15:18

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Standard Grafana Dashboard Nvidia Network Operator

1 participant