feat: estimate offloaded throughput on the API dashboards

AGENTS.md

@@ -8,7 +8,7 @@ This project should provide an easy-to-install Docker Compose stack for basic Pa
 
 The core use case is helping users understand what a firewall is actually doing with minimal setup. Prioritize CPU, RAM, sessions, CPS, disk where useful, interface status, and interface throughput over deep feature-specific monitoring.
 
-Throughput must be calculated from cumulative interface octet counters. The standard Palo Alto and Fortinet dashboards use `ifHCInOctets` / `ifHCOutOctets`. The Palo Alto API dashboard is intentionally API-only and uses the MAC-level `port/rx-bytes` / `port/tx-bytes` counters of physical ports from `show counter interface all` (falling back to `ibytes` / `obytes` only when the `port` block is absent): `ibytes` / `obytes` are dataplane counters that miss hardware-offloaded flows. Do not use dataplane, NPU, session, or feature throughput summaries, which can be incomplete when traffic is offloaded or bypasses their counter path.
+Throughput must be calculated from cumulative interface octet counters. The standard Palo Alto and Fortinet dashboards use `ifHCInOctets` / `ifHCOutOctets`. The Palo Alto API dashboard is intentionally API-only and uses the MAC-level `port/rx-bytes` / `port/tx-bytes` counters of physical ports from `show counter interface all` (falling back to `ibytes` / `obytes` only when the `port` block is absent): `ibytes` / `obytes` are dataplane counters that miss hardware-offloaded flows. The API collector also stores `ibytes` / `obytes` as `dp_in_octets` / `dp_out_octets` next to the port counters, only for the offloaded-traffic estimate (port minus dataplane) of the API dashboards. Do not use dataplane, NPU, session, or feature throughput summaries, which can be incomplete when traffic is offloaded or bypasses their counter path.
 
 For CPU views, the standard SNMP dashboards should show both the global CPU and every per-processor/dataplane CPU exposed by the vendor MIB (`pan_hr_processors` for Palo Alto and `fortinet_processors` for Fortinet). The Palo Alto API dashboard must remain API-only and show management-plane CPU plus every dataplane/core returned by `resource-monitor`. Its all-core dataplane average matches SNMP but includes cores that PAN-OS lists without using them for packet processing (always 0%); `cpu_active_pct` excludes them and is shown next to it. The overview is useful for quick reading; per-CPU lines reveal imbalance and saturated dataplanes.
 

README.md

@@ -809,6 +809,7 @@ Notes:
 - A current-load strip with 30-minute sparklines: DP CPU average, hottest DP core (including the one-minute peak), MP CPU and RAM, sessions, session table, CPS, and throughput.
 - CPU, RAM, session-utilization, resource-pressure, and per-core panels with dashed 70% and 90% guide lines. The **CPU MP / DP** overview shows the management plane, the average of all dataplanes (multi-dataplane systems only), the active-core average of all dataplanes, and one all-core average per dataplane; hottest-core and active-core lines are also in each dataplane's CPU summary.
 - **DP CPU (avg)** and **DP CPU (active cores)** tiles. PAN-OS lists every core of a dataplane in `resource-monitor`, but only the pan task cores process packets: on a PA-5580 dataplane, 80 of the 128 listed cores. The others stay at exactly 0% and halve the all-core average, which is also the value SNMP `hrProcessorLoad` reports (45% all-core for 72% on the packet-processing cores during a load test). The collector writes `cpu_active_pct` (mean of the cores with any load in the minute) and `active_cores` on each dataplane's `core=average` point; the active-core figure is the one that reaches 100% at saturation. It is also drawn in the Load Test ramp and scatter plot.
+- A **Throughput Received: Dataplane vs Offloaded** panel. The physical ports count every received frame, while the dataplane counter (`ibytes`) misses hardware-offloaded flows, so the difference estimates the offloaded traffic. It is an estimate: it also includes frames dropped before the dataplane and framing overhead, it is averaged over at least 2 minutes, and it reads 0 below 5% of the total because small platforms refresh their port counters in bursts. A firewall without offload, such as a PA-440, shows 0. SNMP has no dataplane byte counter, so the SNMP dashboards have no equivalent.
 - An **Ingress Backlog by Dataplane** overview panel, a link-utilization table (interface, In %, Out %, rates, and speed, readable without scrolling), worst-dataplane resource pressure, and the global drop rate stacked by counter category.
 - Collapsible sections for HA (role timeline, sync state, and an **HA Links and Monitoring** table), interfaces and errors/discards, a **VSYS** section and a **Dataplanes** section whose panels repeat side by side for every VSYS or dataplane, zones and logical interfaces, session protocols, DoS/zone-protection drops, filtered global counters, MP load/storage, and environmental sensors split into temperature, fan, power, and alarm panels.
 - A collapsed **Logging and Management Health** section: log rate, logs discarded, content versions, management processes not running, GlobalProtect users, and RAID state.
@@ -827,7 +828,7 @@ Fixed multi-dataplane models such as PA-5580 get the per-DP, interface, MP, sens
 
 Every API panel carries an (i) icon next to its title. Hovering it shows what the panel means plus the PAN-OS CLI command behind the data (for example `show session info` or `show running resource-monitor minute last 1`) and its default polling interval, so a value can be checked on the firewall itself. The mapping lives in `MEASUREMENT_SOURCES` in `scripts/build_paloalto_api_dashboard.py`.
 
-Per-VSYS sessions come from `show session meter`. Per-zone and per-VSYS throughput come from the logical interface (`ifnet`) counters already returned by `show counter interface all`, tagged with the zone and VSYS learned from `show interface all`. Per-VSYS CPS and packet rate come from `show session info` scoped to each VSYS (the SNMP `panVsysTotalCps` equivalent); VSYS slots that `show session meter` lists but PAN-OS reports as not configured are skipped and probed again every `system_interval`. The XML API has no per-zone CPS equivalent to the SNMP zone CPS objects, so zone CPS remains SNMP-only. Global throughput uses only the MAC-level port counters (`port/rx-bytes` / `port/tx-bytes`) of physical Ethernet ports; the dataplane `ibytes` / `obytes` counters are a fallback only, because they miss hardware-offloaded flows. Only physical ports are summed, so aggregate `internal`, `vlan`, `loopback`, and `tunnel` counters are not double-counted. The repeated per-interface throughput and errors/discards panels cover the same interfaces as the SNMP dashboard: physical ports from the MAC-level port counters, and subinterfaces, tunnels, VLAN, loopback and aggregate interfaces from the logical (`ifnet`) counters. The `ifnet` counters are maintained by the dataplane, so per-zone, per-VSYS and logical-interface throughput can under-report hardware-offloaded flows; PAN-OS exposes no MAC-level equivalent for logical interfaces.
+Per-VSYS sessions come from `show session meter`. Per-zone and per-VSYS throughput come from the logical interface (`ifnet`) counters already returned by `show counter interface all`, tagged with the zone and VSYS learned from `show interface all`. Per-VSYS CPS and packet rate come from `show session info` scoped to each VSYS (the SNMP `panVsysTotalCps` equivalent); VSYS slots that `show session meter` lists but PAN-OS reports as not configured are skipped and probed again every `system_interval`. The XML API has no per-zone CPS equivalent to the SNMP zone CPS objects, so zone CPS remains SNMP-only. Global throughput uses only the MAC-level port counters (`port/rx-bytes` / `port/tx-bytes`) of physical Ethernet ports; the dataplane `ibytes` / `obytes` counters are a fallback only, because they miss hardware-offloaded flows. Only physical ports are summed, so aggregate `internal`, `vlan`, `loopback`, and `tunnel` counters are not double-counted. The repeated per-interface throughput and errors/discards panels cover the same interfaces as the SNMP dashboard: physical ports from the MAC-level port counters, and subinterfaces, tunnels, VLAN, loopback and aggregate interfaces from the logical (`ifnet`) counters. The dataplane `ibytes` / `obytes` of the same physical ports are stored as `dp_in_octets` / `dp_out_octets` (only when the port counters exist) and are used only for the offloaded-traffic estimate. The `ifnet` counters are maintained by the dataplane, so per-zone, per-VSYS and logical-interface throughput can under-report hardware-offloaded flows; PAN-OS exposes no MAC-level equivalent for logical interfaces.
 
 ### API Coverage and Deliberate Limits
 

docs/adr/0001-paloalto-xml-api-collector.md

@@ -17,7 +17,7 @@ Prefer exact `${VARIABLE}` references in YAML for new installations. Resolve the
 
 Use one general API dashboard for compact and chassis platforms. Per-core CPU carries `dataplane` and `core` tags, and Grafana repeats a collapsible detail row for every dataplane. Collect resource pressure per dataplane, interface operational metadata, HA state, storage, and environmental sensors on slower schedules than session and octet counters.
 
-Keep this dashboard API-only: calculate throughput from deltas of the per-port MAC counters `port/rx-bytes` and `port/tx-bytes` returned by `show counter interface all`, with `ibytes` and `obytes` only as a fallback when the `port` block is missing. Amended 2026-09-24: a load test with half its traffic hardware-offloaded showed `ibytes`/`obytes` (and the session kbps summary) at 47 Gb/s while the port counters and SNMP `ifHCInOctets` read 97 Gb/s; the `ibytes`/`obytes` counters are updated by the dataplane and miss offloaded flows. SNMP `ifHCInOctets` returns the same port counter, cached by the SNMP agent for several seconds. Do not use the session throughput summary. The standard dashboards continue using SNMP `ifHCInOctets` and `ifHCOutOctets`.
+Keep this dashboard API-only: calculate throughput from deltas of the per-port MAC counters `port/rx-bytes` and `port/tx-bytes` returned by `show counter interface all`, with `ibytes` and `obytes` only as a fallback when the `port` block is missing. Amended 2026-09-24: a load test with half its traffic hardware-offloaded showed `ibytes`/`obytes` (and the session kbps summary) at 47 Gb/s while the port counters and SNMP `ifHCInOctets` read 97 Gb/s; the `ibytes`/`obytes` counters are updated by the dataplane and miss offloaded flows. SNMP `ifHCInOctets` returns the same port counter, cached by the SNMP agent for several seconds. Amended 2026-09-26: the dataplane `ibytes`/`obytes` of the physical ports are kept as `dp_in_octets`/`dp_out_octets` when the port counters exist, only to estimate offloaded traffic as port minus dataplane; the estimate is averaged over at least 2 minutes and floored at 5% of the total because a PA-440 without offload refreshes its port counters in bursts (20 s differences swung by several Mb/s both ways while 3-minute totals agreed within 3%). Do not use the session throughput summary. The standard dashboards continue using SNMP `ifHCInOctets` and `ifHCOutOctets`.
 
 Provide a separate high-end API chassis dashboard for PA-5200, PA-5400, PA-5500, PA-7000, and PA-7500 systems. Fixed multi-DP appliances use the same per-DP views. Detect modular PA-5450, PA-7050, PA-7080, and PA-7500 models from `show system info` before polling `show chassis inventory`, `show chassis status`, and `show chassis power`, avoiding unsupported calls on fixed appliances.