mirror of
https://github.com/ryan4yin/nix-config.git
synced 2026-11-13 12:13:46 +01:00
security(monitoring): bind exporters to static LAN addresses (#373)
node_exporter listened on 0.0.0.0, so access control rested entirely on the nftables allowlist. Bind node/smartctl exporters to the host static LAN IPv4 (the same address VictoriaMetrics scrapes), keeping the socket off tailscale/bridge interfaces even if the firewall regresses. Hosts without a static LAN address (shoukei, akane) now run no exporters at all instead of loopback-bound ones nobody scrapes; akane, a UTM VM, previously ran a loopback smartctl exporter with no physical disks. LAN-bound exporters also get the boot-race treatment from #372 (network-online ordering plus patient restart); loopback-bound hosts must not pull in network-online because NM wait-online would stall their offline boots.
This commit is contained in:
ryan4yin | 二花 authored and 
GitHub committed
1 parent
1b31730db4
commit
6259359945
2 files changed
+47
-35
No files matched your search
+12
-12
@@ -33,18 +33,18 @@ small, reversible controls and narrow exceptions; benchmark before adopting expe
|
||||
|
||||
## Architecture and trust boundaries
|
||||
|
||||
| Boundary | Current design | Limitation |
|
||||
| --------------- | -------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| Administration | Key-based SSH; root key login retained for Colmena/MicroVM deployment | Deployment credentials are root-equivalent; password login being disabled does not restrict stolen keys |
|
||||
| Host network | Shared nftables firewall trusts the home LAN, tailnet interface and local Podman bridge; IPv6 link-local is accepted | These are broad trust zones, not service-level least privilege; router exposure and tailnet ACLs require separate verification |
|
||||
| Monitoring | Existing firewall rules restrict exporter ports 9100, 9835 and 9633 to the monitoring host over IPv4 | IPv6 denial is planned in the baseline implementation; current running hosts require separate verification |
|
||||
| VM hosts | shoryu, shushou and youko run guests behind `br0`; ruby/kana are libvirt guests | A bridge is not network isolation; host compromise affects its guests |
|
||||
| k3s | Three control-plane and three worker MicroVMs; Cilium, API VIP and NFS CSI | Cluster-admin, privileged/host-mounted pods and writable NFS data are powerful trust boundaries |
|
||||
| MicroVM storage | Host store shared read-only through virtiofs; guest `/etc`, `/var` and `/home` persisted in separate images | Read-only store does not isolate the host-side virtiofs implementation or protect writable guest state |
|
||||
| Desktop apps | Selected apps use nixpak/bubblewrap; Podman is rootless | Rules and shared directories vary per app; user namespaces are needed for these sandboxes |
|
||||
| Secrets | agenix declarations here, encrypted data in the private nix-secrets repository | Recipients and plaintext file permissions must be reviewed; root on a recipient can read its decrypted secrets |
|
||||
| Supply chain | Locked flake inputs; explicit binary-cache trust keys; normal user is not a Nix trusted-user | A trusted cache or imported Nix module can supply code executed as root; a lock file is not a security audit |
|
||||
| Recovery | NixOS generations, btrbk/restic and documented backup procedures | A rollback does not undo compromised data or secrets; backup configuration alone does not prove restore works |
|
||||
| Boundary | Current design | Limitation |
|
||||
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Administration | Key-based SSH; root key login retained for Colmena/MicroVM deployment | Deployment credentials are root-equivalent; password login being disabled does not restrict stolen keys |
|
||||
| Host network | Shared nftables firewall trusts the home LAN, tailnet interface and local Podman bridge; IPv6 link-local is accepted | These are broad trust zones, not service-level least privilege; router exposure and tailnet ACLs require separate verification |
|
||||
| Monitoring | node/smartctl exporters bind the host LAN IPv4 (9100/9633); firewall rules restrict exporter ports 9100, 9835 and 9633 to the monitoring host over IPv4 | nvidia_gpu_exporter (9835) still binds wildcard; IPv6 denial is planned in the baseline implementation; current running hosts require separate verification |
|
||||
| VM hosts | shoryu, shushou and youko run guests behind `br0`; ruby/kana are libvirt guests | A bridge is not network isolation; host compromise affects its guests |
|
||||
| k3s | Three control-plane and three worker MicroVMs; Cilium, API VIP and NFS CSI | Cluster-admin, privileged/host-mounted pods and writable NFS data are powerful trust boundaries |
|
||||
| MicroVM storage | Host store shared read-only through virtiofs; guest `/etc`, `/var` and `/home` persisted in separate images | Read-only store does not isolate the host-side virtiofs implementation or protect writable guest state |
|
||||
| Desktop apps | Selected apps use nixpak/bubblewrap; Podman is rootless | Rules and shared directories vary per app; user namespaces are needed for these sandboxes |
|
||||
| Secrets | agenix declarations here, encrypted data in the private nix-secrets repository | Recipients and plaintext file permissions must be reviewed; root on a recipient can read its decrypted secrets |
|
||||
| Supply chain | Locked flake inputs; explicit binary-cache trust keys; normal user is not a Nix trusted-user | A trusted cache or imported Nix module can supply code executed as root; a lock file is not a security audit |
|
||||
| Recovery | NixOS generations, btrbk/restic and documented backup procedures | A rollback does not undo compromised data or secrets; backup configuration alone does not prove restore works |
|
||||
|
||||
Sources: [host layout](hosts/README.md), [k3s deployment](hosts/k8s/README.md),
|
||||
[firewall](modules/nixos/base/networking/firewall.nix), [SSH](modules/nixos/base/ssh.nix),
|
||||
|
||||
@@ -4,12 +4,19 @@
|
||||
myvars,
|
||||
...
|
||||
}:
|
||||
let
|
||||
# Exporters bind to this host's static LAN IPv4 (the same address
|
||||
# VictoriaMetrics scrapes), so the socket exists only on the LAN interface —
|
||||
# defence in depth behind the firewall allowlist. Hosts without a static LAN
|
||||
# address (laptops like shoukei, DHCP VMs like akane) run no exporters.
|
||||
lanAddr = myvars.networking.hostsAddr.${config.networking.hostName}.ipv4 or null;
|
||||
in
|
||||
{
|
||||
# enable the node exporter on all nixos hosts
|
||||
# https://github.com/NixOS/nixpkgs/blob/nixos-26.05/nixos/modules/services/monitoring/prometheus/exporters/node.nix
|
||||
services.prometheus.exporters.node = {
|
||||
services.prometheus.exporters.node = lib.mkIf (lanAddr != null) {
|
||||
enable = true;
|
||||
listenAddress = "0.0.0.0";
|
||||
listenAddress = lanAddr;
|
||||
port = 9100;
|
||||
# There're already a lot of collectors enabled by default
|
||||
# https://github.com/prometheus/node_exporter?tab=readme-ov-file#enabled-by-default
|
||||
@@ -45,26 +52,31 @@
|
||||
# node_nvme_info and namespace capacity, NOT SMART health, so this is the only
|
||||
# source of SSD lifespan data. `hardwareTools` is off on MicroVM/QEMU guests,
|
||||
# which have no physical disks to inspect.
|
||||
services.prometheus.exporters.smartctl = lib.mkIf config.modules.hardwareTools.enable {
|
||||
enable = true;
|
||||
# Bind to this host's LAN address rather than 0.0.0.0; the firewall also
|
||||
# drops 9633 from anything but the monitoring host. Fall back to loopback
|
||||
# for hosts without a static LAN address (e.g. shoukei).
|
||||
listenAddress = myvars.networking.hostsAddr.${config.networking.hostName}.ipv4 or "127.0.0.1";
|
||||
port = 9633;
|
||||
maxInterval = "60s";
|
||||
};
|
||||
services.prometheus.exporters.smartctl =
|
||||
lib.mkIf (lanAddr != null && config.modules.hardwareTools.enable)
|
||||
{
|
||||
enable = true;
|
||||
listenAddress = lanAddr;
|
||||
port = 9633;
|
||||
maxInterval = "60s";
|
||||
};
|
||||
|
||||
# The exporter binds to the LAN address, which networkd assigns during boot;
|
||||
# without ordering it races the address and trips the start limit (seen on ai).
|
||||
# wait-online is best-effort (120s timeout), so also retry patiently to
|
||||
# self-heal when the link comes up late.
|
||||
systemd.services.prometheus-smartctl-exporter = lib.mkIf config.modules.hardwareTools.enable {
|
||||
after = [ "network-online.target" ];
|
||||
wants = [ "network-online.target" ];
|
||||
serviceConfig = {
|
||||
RestartSec = "10s";
|
||||
StartLimitIntervalSec = 0;
|
||||
};
|
||||
};
|
||||
# LAN-bound exporters race networkd's address assignment at boot; without
|
||||
# ordering they trip the start limit (seen on ai). wait-online is best-effort
|
||||
# (120s timeout), so also retry patiently.
|
||||
systemd.services = lib.mkIf (lanAddr != null) (
|
||||
lib.genAttrs
|
||||
(
|
||||
[ "prometheus-node-exporter" ]
|
||||
++ lib.optionals config.modules.hardwareTools.enable [ "prometheus-smartctl-exporter" ]
|
||||
)
|
||||
(_: {
|
||||
after = [ "network-online.target" ];
|
||||
wants = [ "network-online.target" ];
|
||||
serviceConfig = {
|
||||
RestartSec = "10s";
|
||||
StartLimitIntervalSec = 0;
|
||||
};
|
||||
})
|
||||
);
|
||||
}
|
||||
Reference in new issue
Block a user