feat: added capability to extract metrics from the logs - #101
CptSchnitz wants to merge 3 commits into
Conversation
shimoncohen
left a comment
shimoncohen
left a comment
There was a problem hiding this comment.
A few notes from review — nothing blocking. Cardinality note on the WMS layer label, two label/parsing edge cases, and one nit for awareness.
| regex: ogc_service ^WMS$ | ||
| label_field: | ||
| - ogc_operation | ||
| - ogc_layer |
There was a problem hiding this comment.
Note (not blocking): WMS LAYERS is comma-separated, and custom.lua sets ogc_layer verbatim, so multi-layer combos would each be a distinct series on mapproxy_wms_requests_total — same cardinality concern zoom_group bucketing guards against, but for layer. In our case only one layer is used per request, so this is fine; noting it since it's not enforced anywhere.
| metric_name: requests_total | ||
| metric_description: Total WMTS requests | ||
| regex: ogc_service ^WMTS$ | ||
| add_label: client_name $Attributes['mapcolonies.http.auth.token.client_name'] |
There was a problem hiding this comment.
client_name is added unconditionally, but mapcolonies.http.auth.token.client_name was only populated under and .opalaEnabled .nginx.authorization.enabled (old log_format.conf). With authorization disabled the accessor won't resolve → empty/literal client_name label on every WMTS series. Gate the add_label on the same condition, or accept the noise deliberately.
| local lower_path = string.lower(path) | ||
|
|
||
| if lower_path:find("/wmts/") then | ||
| local pattern = "/wmts/([^/]+)/([^/]+)/(%d+)/(%d+)/(%d+)%.(%w+)" |
There was a problem hiding this comment.
Zoom is captured as (%d+), so if the RESTful TileMatrix segment isn't a bare integer (e.g. gridset-prefixed EPSG:4326:10) the whole tile pattern fails, falls through to return nil, and the request is dropped from all metrics. Fine if our gridsets use numeric TileMatrix ids — worth confirming.
| local function parse_query_string(query) | ||
| local params = {} | ||
| if not query or query == "" then return params end | ||
| for k, v in query:gmatch("([^&]+)=([^&]+)") do |
There was a problem hiding this comment.
Nit (awareness only): ([^&]+)=([^&]+) silently skips valueless params (&flag& or key=). Harmless for the OGC params we parse.
2 participants
Main feature: derive Prometheus metrics from nginx access logs via Fluent Bit, instead of just shipping raw logs.
Net effect: mapproxy nginx now exports structured request/latency metrics (by OGC service, operation, layer, zoom bucket, format, and error status) directly from access logs, rather than relying on log aggregation alone.