* security: make firewall secure-by-default, disable explicitly on servers
Flip the base firewall default to ON so new hosts are protected by default. Servers keep the firewall off (trusted internal LAN, WAN protected at the router) via explicit overrides in modules/nixos/server/{server,server-aarch64}.nix. Behavior-preserving for all 18 current hosts; adds a security-firewall eval test guarding the per-host state.
* test: fix security-firewall eval test to scalar per-host values
* security: enable AppArmor (complain) on all Linux hosts
* security: bind aquamarine metrics exporters to loopback
* security: restrict k3s kubeconfig file mode to 600
* security: disable SSH X11 forwarding on servers, keep on desktops
* security: add conservative systemd hardening to aquamarine services
* docs(apparmor): correct FHS-alias comment (nixpkgs has no broad FHS layer)
nixpkgs only creates /run/current-system, /usr/bin/env and /bin/sh; it does not
provide a broad FHS->store alias tree, so FHS-oriented abstractions are incomplete
on NixOS. They are safe only because the profiles run in complain mode.
* fix(aquamarine): scrape same-host exporters over loopback
The v2ray/postgres/sftpgo exporters were bound to 127.0.0.1 but VictoriaMetrics
scraped them via the host's routable IP, so those scrapes refused connections.
Point the three same-host scrape targets at 127.0.0.1 (VM runs on the same host).
node-exporter keeps the host IP since it binds 0.0.0.0.
* Revert "security: add conservative systemd hardening to aquamarine services"
This reverts commit f9cf99dadb.
* feat(nushell): add trash command using the home trash can
On a tmpfs root with persistent dirs bind-mounted in, `rm --trash`
scatters items into per-mount .Trash-$uid dirs that file managers never
show, and gio refuses to trash on those internal mounts.
Add a `trash` command implementing the freedesktop trash spec
'failsafe' mode: items are moved to ~/.local/share/Trash with
spec-compliant .trashinfo entries, so Thunar can list and restore them.
Also preserve ~/.local/share/Trash on idols-ai so the trash survives
reboots.
* feat(nixos): manage scattered trash dirs with trash-cli retention timer
Replace the hand-written nushell trash command with the established
trash-cli tooling. On a tmpfs root with persistent dirs bind-mounted
in, the trash crate (nushell rm --trash) scatters items into per-mount
.Trash-$uid dirs that file managers never show and nothing cleans up.
- add trash-cli to system packages: trash-list scans/trash-restore
handles every mount point's trash dir
- daily systemd timer runs 'trash-empty 30 -f', purging items older
than 30 days across the home trash and all mount points
- drop the hand-written trash.nu module
- note in preservation.nix: do NOT persist ~/.local/share/Trash; a
bind-mounted trash dir breaks the trash crate's home-topdir match
for files straight under $HOME (it would try /.Trash-$uid, EACCES)
* fix(nixos): order trash-empty after preservation.target
The scattered .Trash-$uid dirs live inside the preservation bind
mounts, which use DefaultDependencies=no and are therefore NOT ordered
after local-fs.target. Add preservation.target to After= so trash-empty
only runs once all those mounts are up.
- Add an offlineHosts list to VictoriaMetrics scrape generation: SBCs
and the whole k3s-prod-1 cluster are powered off, shoukei's exporter
is disabled on the machine; also comment out the dnsmasq-exporter
job (runs on suzi).
- Disable node-exporter on shoukei: a laptop on untrusted networks
should not expose :9100 (the firewall is off repo-wide).
- Route severity none|info alerts to the null receiver; meta alerts
and info noise should never page.
Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
node-exporter always reports device_error=1 for containerd sandbox shm
bind mounts (tmpfs), producing ~50 bogus critical alerts on the
kubevirt hosts. Verified against live metrics that no non-shm device
errors exist, so nothing real is filtered out.
Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
The NixOS alertmanager module pipes the generated config through
envsubst to inject secrets, which silently replaced the template's
$i/$a variables with empty strings, so every telegram notification
failed with a template parse error since Friday's deploy.
Rewrite the template without variables: range + dot, define/template
for the per-alert block, and an if/else split to cap a group at 5
alerts (slice errors when fewer than 5). Verified with amtool template
render against post-envsubst content for both branches.
Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
- Show group-level labels (alertgroup/cluster/env/namespace) once in
the header instead of repeating them per alert.
- Use each rule's summary annotation as the per-alert headline (it
already carries the distinguishing resource name), falling back to
the instance label; show nodename on its own line when present.
- Drop the noisy per-alert label dump and the always-N/A value line.
Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
- Trim the telegram template: drop the full label dump, render at
most 5 alerts per group, and note how many were omitted, so
messages stay within Telegram's 4096-char limit.
- Route severity=none meta alerts (Watchdog, InfoInhibitor) to a new
null receiver so they no longer notify.
Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
ryan4yin | 二花