Ryan Yin
7dc4e7a39f
The homelab hosts accept root SSH with long-lived file keys, run several exposed services with almost no systemd sandboxing, and keep no audit trail of privileged actions. Close those three gaps, weighing each measure against its cost instead of adopting a policy wholesale. SSH - Turn off KbdInteractiveAuthentication: nixpkgs defaults it to true and PAM then still accepts the Unix password despite PasswordAuthentication = false. - authorizedKeysInHomedir = false: only keys declared in Nix take effect; youko keeps Gitea's forced-command file through `Match User gitea`. - Pin aquamarine's passphrase-free remote-builder keys to `restrict,command="nix-daemon --stdio --store daemon"`, so builds run in the system daemon instead of an audited root login session. systemd sandboxing - Add the mylib.systemdHardening preset and apply it to the exposed, weakly sandboxed services: caddy 7.8 -> 1.5, vmalert 8.2 -> 1.4, rustfs 6.6 -> 1.4, whisper-asr 7.7 -> 1.6. An eval test pins the key restrictions. - Rejected: a global all-keys policy (<=0.2 score gains on services nixpkgs already sandboxes) and per-service confinement (bind paths to maintain). auditd on all four hosts, reusing nixpkgs' loader and unit - Audit login sessions only: a first `-a never,task -F auid=unset` rule keeps services out of the per-syscall audit path. A compromised service is therefore unrecorded until it opens a session. - Record root commands, credential and host-key access, writes to SSH, account, PAM, Nix-trust and audit configuration, module loads and session denials. - Rejected: global denial logging, service-UID watches and --loginuid-immutable (the kernel already records loginuid changes). Verified on all four hosts after deployment: zero failed units, k3s 6/6 Ready, and the exposed services healthy. Assisted-by: Claude Code (claude-opus-5-5)
Flake Outputs
Is such a complex and fine-grained structure necessary?
There is no need to do this when you have a small number of machines.
But when you have a large number of machines, it is necessary to manage them in a fine-grained way, otherwise, it will be difficult to manage and maintain them.
I have more than a dozen machines now, and a fine-grained layout keeps them manageable.
Tests
Simple configurations do not need tests, but a dozen machines make a broken setting easy to miss.
There are two types of tests: eval tests and NixOS tests. They catch configuration errors before a change reaches a real machine.
Related projects & docs:
- haumea: Filesystem-based module system for Nix
- Unveiling the Power of the NixOS Integration Test Driver (Part 1)
- NixOS Tests - NixOS Manual
1. Eval Tests
Eval Tests evaluate the expressions and compare the results with the expected results. It runs fast, but it doesn't build a real machine. We use eval tests to ensure that some attributes are correctly set for each NixOS and nix-darwin host.
How to run all the eval tests:
just test
just test exits non-zero unless the suite returns true. The underlying
nix eval .#evalTests --show-trace --print-build-logs --verbose prints the result but exits 0 even
on false, so read its output when you need the failing trace.
Each test is a directory outputs/<system>/tests/<name>/ holding expr.nix and expected.nix; the
suite passes when every expr equals its expected.
Which tests cover a new host
Some tests iterate over every configuration, so a new host is covered automatically and must satisfy
them. Others name specific hosts — or a name prefix, like k3s-api-endpoint, which covers every
k3s-test-1-* configuration — and only cover a new host if you add it.
| Kind | Tests |
|---|---|
| Every configuration | hostname, kernel, nix-system-features, security-apparmor, security-container-groups, security-exporters, security-firewall, security-k3s-kubeconfig, security-k3s-pod-firewall, security-kernel, security-ssh-x11 |
| Named hosts only | btrbk, cpu-power, home-manager, home-manager-xdg, computer-use-headless, k3s-api-endpoint, k3s-node-home-manager, idols-ai-gpu, idols-ai-net, idols-ai-voice-input, idols-aquamarine, security-postgres-auth, youko-metrics, ups-metrics, shoukei-logind, mihomo-dns, colmena-parity, microvm-guest-parity |
hostname expects each configuration's networking.hostName to equal its name. Niri desktop
configurations are the exception (ai-niri → ai, shoukei-niri → shoukei) and are listed in
specialExpected in that platform's hostname/expected.nix.
2. NixOS Tests
WIP: not working yet
NixOS Tests builds and starts virtual machines using our NixOS configuration and run tests on them. Comparing to eval tests, it runs slow, but it builds a real machine, and we can test the whole system actually works as expected.
Problems:
- We need a private cache server, so that our NixOS tests do not need to build some custom packages every time we run the tests.
- Cannot test the whole host, because my host relies on its unique ssh host key to decrypt its
agenix secrets.
- Maybe it's better to test every service separately, not the whole host?
Full-host tests are tracked as WA-013 in WORKAROUNDS.md.
outputs/x86_64-linux/nixos-tests/idols-ruby.nix is exported as
packages.x86_64-linux.ruby-nixos-tests, but it is not a checks entry, so CI never builds it and
nothing else runs it. What runs today is the eval set above with just test, plus one NixOS-driver
check of the shared firewall — checks.x86_64-linux.security-exporters, built in CI.
Overview
All the outputs of this flake are defined here.
› tree
.
├── default.nix # The entry point, all the outputs are composed here.
├── README.md
├── aarch64-darwin # All outputs for macOS Apple Silicon
│ ├── default.nix
│ ├── src # every host has its own file in this directory
│ │ ├── fern.nix
│ │ └── frieren.nix
│ └── tests # eval tests
├── aarch64-linux # All outputs for Linux ARM64
│ ├── default.nix
│ ├── src # every host has its own file in this directory
│ │ ├── 12kingdoms-shoukei.nix
│ │ └── idols-akane.nix
│ └── tests # eval tests
└── x86_64-linux # All outputs for Linux x86_64
├── default.nix
├── nixos-tests
├── src # every host has its own file in this directory
│ ├── 12kingdoms-shoryu.nix
│ ├── 12kingdoms-shushou.nix
│ ├── 12kingdoms-youko.nix
│ ├── idols-ai.nix
│ ├── idols-aquamarine.nix
│ ├── idols-kana.nix
│ ├── idols-ruby.nix
│ ├── k3s-test-1-master-1.nix
│ ├── k3s-test-1-master-2.nix
│ ├── k3s-test-1-master-3.nix
│ ├── k3s-test-1-worker-1.nix
│ ├── k3s-test-1-worker-2.nix
│ └── k3s-test-1-worker-3.nix
└── tests # eval tests (btrbk, computer-use-headless, hostname,
# kernel, security-*, ups-metrics, ...)
The tree lists the per-host sources and tests. The flake also exposes a checks output: every
system has eval-tests and pre-commit-check, and x86_64-linux adds security-exporters, which
imports the top-level tests/security-exporters.nix. See ../SECURITY.md for
when to run it.