starred/nix-config
1
0 Fork 0
mirror of https://github.com/ryan4yin/nix-config.git synced 2026-11-13 12:13:46 +01:00
Code Issues Packages Projects Releases 10 Wiki Activity

fix(monitoring): ignore openapi discovery in apiserver latency alert

Browse Source 
KubernetesApiServerLatency fired continuously on the p99 of
apiserver_request_duration_seconds_bucket. Every firing series was
verb="GET", subresource="openapi/v3" (kubelet, kube-controller-manager,
kube-scheduler, apiserver) at ~8s, while real verbs (POST/LIST/DELETE)
stay around 0.2s.

openapi/v3 is a streaming discovery endpoint, like WATCH, so exclude
subresource=~"^openapi/" from the rule.
This commit is contained in:
Ryan Yin committed 2026-10-07 13:21:43 +08:00
1 parent 76c4566e9b
commit fabccbafda
1 file changed
+1 -1
Download Patch File Download Diff File Expand all files Collapse all files
hosts/12kingdoms-youko/homelab-services/monitoring/alert_rules/kube-state-metrics-exporter.yml
+1 -1
View File
@@ -422,7 +422,7 @@ groups:
- alert: KubernetesApiServerLatency
expr:
'histogram_quantile(0.99,
sum(rate(apiserver_request_duration_seconds_bucket{subresource!="log",verb!~"^(?:CONNECT|WATCHLIST|WATCH|PROXY)$"}
sum(rate(apiserver_request_duration_seconds_bucket{subresource!="log",subresource!~"^openapi/",verb!~"^(?:CONNECT|WATCHLIST|WATCH|PROXY)$"}
[10m])) WITHOUT (instance, resource)) > 1'
for: 2m
labels: