Commit Graph
14 Commits
Author SHA1 Message Date
ryan4yin | 二花 9c38819517 docs: review and correct comments and documentation (#436) 2026-10-12 10:02:38 +08:00
ryan4yin | 二花 835b7f6df1 feat(shoukei): pin home Wi-Fi to a static IPv4 & adjust networking.nix (#407) 2026-10-07 21:27:34 +08:00
ryan4yin | 二花 6259359945 security(monitoring): bind exporters to static LAN addresses (#373)
node_exporter listened on 0.0.0.0, so access control rested entirely on
the nftables allowlist. Bind node/smartctl exporters to the host static
LAN IPv4 (the same address VictoriaMetrics scrapes), keeping the socket
off tailscale/bridge interfaces even if the firewall regresses.

Hosts without a static LAN address (shoukei, akane) now run no exporters
at all instead of loopback-bound ones nobody scrapes; akane, a UTM VM,
previously ran a loopback smartctl exporter with no physical disks.

LAN-bound exporters also get the boot-race treatment from #372
(network-online ordering plus patient restart); loopback-bound hosts
must not pull in network-online because NM wait-online would stall
their offline boots.
2026-10-03 19:07:21 +08:00
ryan4yin | 二花 1b31730db4 fix(monitoring): order smartctl exporter after network-online.target (#372)
The exporter binds to the host LAN address, which systemd-networkd
assigns during boot. Without ordering, the service starts before the
address exists, `bind: cannot assign requested address` trips the
start limit, and the unit stays failed until restarted manually.

Order the unit after network-online.target, and since wait-online is
best-effort (120s timeout), also retry patiently so the unit self-heals
when the link comes up late.
2026-10-03 18:54:08 +08:00
ryan4yin | 二花 dc3ec8c159 feat(storage): relax ai NVMe workarounds and add drive SMART monitoring (#359)
* fix(ai): relax NVMe power-management workarounds and enable smartd

Logs show no NVMe errors, so allow APST again (capped at 100 ms), restore
the default io_timeout, and drop pcie_aspm=off. Add smartd for drive
health visibility.

* feat(storage): add drive/SSD SMART monitoring on physical hosts

node_exporter's nvme collector only exposes node_nvme_info and namespace
capacity, not SMART health, so SSD wear/lifespan and drive temperature were
not observable anywhere.

- enable services.prometheus.exporters.smartctl on physical hosts, gated on
  modules.hardwareTools (off on QEMU/MicroVM guests), bound to the host's LAN
  address on port 9633
- firewall: drop 9633 from anything but the monitoring host
- youko: scrape smartctl-<host> for the physical hosts
- alert rules (storage.yml): SMART health, NVMe wear, available spare, media
  errors, critical warning, and drive temperature (warn 70 / crit 80 C)
- Grafana dashboard "Drive SMART (smartctl)"
2026-10-03 11:13:46 +08:00
Ryan Yin 662316c18a Nixos 26.05 (#261)
* flake.lock: Update

Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/7aaa00e7cc9be6c316cb5f6617bd740dd435c59d?narHash=sha256-WacE23EbHTsBKvr8cu%2B1DFNbP6Rh1brHUH5SDUI0NQI%3D' (2026-04-30)
  → 'github:NixOS/nixpkgs/e9a7635a57597d9754eccebdfc7045e6c8600e6b?narHash=sha256-u6WU/yd/o8iYQrHX3RAwO1hYa3LkoSL%2BWNQD0rJfJZQ%3D' (2026-05-29)

* feat: upgrade to nixos 26.05
2026-06-01 10:02:49 +08:00
Ryan Yin 3bf1b986cf feat: upgrade nixpkgs stable to 25.11, update nixos-apple-silicon, gh… (#236)
* feat: upgrade nixpkgs stable to 25.11, update nixos-apple-silicon, ghostty, anyrun, etc
* fix: asahi-nixos - revert mesa to 25.2.6
* fix: disable gitui - it's broken on aarch64-darwin currently

---------

Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
2025-11-26 20:54:44 -06:00
Ryan Yin 1b663c757b fix: node_exporter - exclude also run/credentials/.+ & run/user/.+ 2025-09-14 11:28:12 +08:00
Ryan Yin a5b57cd16e fix: alert - exclude also /persistent-prefixed paths 2025-09-14 11:09:48 +08:00
Ryan Yin c8182216ae fix: node_exporter - exclude docker/podman/kubelet mounts and /home/ryan bindmounts 2025-09-14 10:15:20 +08:00
Ryan Yin a2007fd88c feat: bump to nixos 25.05 2025-05-21 12:16:00 +08:00
Ryan Yin 0683ef0a13 chore: bump nixpkgs to 24.11 2024-11-22 13:58:51 +08:00
Ryan Yin 9adf87aaf5 feat: upgrade to 24.05
fix: remove or update some packages
2024-06-02 11:17:32 +08:00
Ryan Yin b6e51e1950 feat: monitoring + containers - grafana + prometheus + node_exporter + other exporters 2024-02-17 23:33:48 +08:00