mirror of
https://github.com/ryan4yin/nix-config.git
synced 2026-11-13 12:13:46 +01:00
Ryan Yin
9b68cfdb8a
The backup unit publishes `restic_backup_last_success_timestamp_seconds` for the node_exporter textfile collector, and a rejected alert now fails the notify unit instead of exiting 0. Assisted-by: DeepSeek Harness 0.2.1-rc.2 (deepseek-flash)
1.1 KiB
1.1 KiB
Monitoring & Alerting
Alert Rules & Recording Rules
- awesome-prometheus-alerts
- Collection of Prometheus alerting rules.
- victoria-metrics-k8s-stack/files/rules
- Alert Rules & Recording Rules used by kube-prometheus-stack.
Backup Freshness
modules/nixos/base/restic-backup.nix writes restic_backup_last_success_timestamp_seconds to
node_exporter's textfile collector after a successful run. alert_rules/restic-backup.yml alerts
when it is older than two days (the schedule is daily) or absent entirely.
TODO
- Dead-man switch:
Watchdoginalert_rules/general.ymlalways fires (vector(1)) but is routed to thenullreceiver inalert.nix, so a dead alerting pipeline looks identical to a healthy one. Route it to an external heartbeat service (healthchecks.io, Uptime Kuma, PagerDuty DeadMansSnitch) that notifies when the heartbeat stops, or run an out-of-band probe against Alertmanager.