Files
nix-config/hosts
ryan4yin | 二花 e37042f61c feat(youko): restic backup of the service state to the HDD (part 1) (#299)
* feat(hosts): per-host restic backups to the SMB remote

Replace the disabled aggregate job (which rsynced every host's /persistent
into /tmp - a tmpfs - and silently produced incomplete backups) with one
restic repo per host, backing up its own data directly:

- shared module lib/genResticBackup.nix: secrets, rclone/SMB target
  (rclone:smb-downloads:/Downloads/homelab-backup/<host>), retention, and
  excludes for the regenerable/bulky paths (podman overlay layers, microvm
  and libvirt disk images, nfs golden images, logs, caches)
- youko additionally backs up the service state under /persistent/var/lib
  and dumps all postgres databases first (a consistent SQL artifact beside
  the crash-consistent data dir)
- shoryu/shushou back up /etc/agenix, /etc/ssh and the k3s token

Needs three agenix secrets before it can run: restic-password,
ssh-key-for-restic-backup and rclone-conf-for-restic-backup.

* refactor(hosts): only restic does off-host; btrbk stays local

btrbk works at the subvolume level, so a  would ship the whole
@persistent - including podman's overlay storage, the microVM/libvirt
images and the nfs golden images - which is exactly the regenerable bulk
we want out of the backups.

So: btrbk keeps making the cheap local CoW snapshots (quick rollbacks),
and restic owns the off-host copies with file-level excludes. Back up the
whole /persistent minus those excludes, so nothing is missed.

Update the btrbk module's comment to say so.

* fix(restic): never back up keys; drop the implicit /etc/agenix

restic's own password lives in /etc/agenix, so backing that up would store
the repository's password inside the repository. Exclude agenix, the ssh
host keys and the usual credential dirs (.ssh, .gnupg, .aws, gcloud).

* refactor(restic): make the backup a proper module

Replace lib/genResticBackup.nix with modules/nixos/base/restic-backup.nix,
which declares modules.restic-backup options (enable, repository, paths,
exclude, requiresMountsFor, postgresDump) like the btrbk module does.
youko just sets the options now.

* refactor(restic): exclude the whole podman storage, trim retention

Exclude /persistent/var/lib/containers wholesale: the image layers are
re-pullable and podman's storage DB is not consistent when copied from a
running podman. uptime-kuma's named volume lives there, but its data is
not wanted anyway.

Retention: daily 3, weekly 2, monthly 2 - no yearly.

* refactor(restic): back up a short-lived btrfs snapshot, drop pg_dump

A live copy of a running postgres data dir is only crash-consistent. Instead
of a daily pg_dumpall (which duplicates the data in the repository and gets
slower as the DB grows), take a short-lived read-only btrfs snapshot in
backupPrepareCommand, back that up, and delete it right after - a
consistency step, not a second backup. The excludes are relative now so they
work for both the snapshot and a direct path.

* feat(secrets): restic-password for modules.restic-backup

With the rekeyed secret in nix-secrets (storage group, alongside
hdd-luks-crypt-key), the module's passwordFile resolves.

* fix(restic): agenix's secret is under /run/agenix, not /etc/agenix

* fix(restic): take the password path from agenix, not a hardcoded one

config.age.secrets."restic-password".path follows agenix's secretsDir, so it
stays correct if that changes.
2026-09-21 19:00:35 +08:00
..
2026-09-21 17:24:40 +08:00

Hosts

This directory contains all host-specific configurations for my NixOS and macOS systems.

Current Host Inventory

Physical Machines

idols - Main Workstations

Named after characters from "Oshi no Ko":

Host Platform Hardware Purpose Status
ai NixOS i5-13600KF + RTX 4090 Gaming & Daily Use ✅ Active
aquamarine VM Virtual Monitoring & Services ✅ Active
kana NixOS Virtual Reserved ⚪ Not Used
ruby NixOS Virtual Reserved ⚪ Not Used

darwin - macOS Systems

Named after characters from "Frieren: Beyond Journey's End":

Host Platform Hardware Purpose Status
fern macOS MacBook Pro M2 13" 16GB Personal Use ✅ Active
frieren macOS MacBook Pro M4Pro 14" 48GB Work Use ✅ Active

12kingdoms - Homelab Servers & Apple Silicon Linux

Named after "Twelve Kingdoms":

Host Platform Hardware Purpose Status
shoukei NixOS MacBook Pro M2 NixOS on Apple Silicon ✅ Active
shoryu NixOS MoreFine S500Plus (AMD Ryzen 9 5900HX) VM Host ✅ Active
shushou NixOS MinisForum UM560 (AMD Ryzen 5 5625U) VM Host ✅ Active
youko NixOS MinisForum HX99G (AMD Ryzen 9 6900HX) VM Host ✅ Active

Virtual Machines & Clusters

k8s - Kubernetes Infrastructure

  • VM Cluster: 3 physical mini PCs (shoryu, shushou, youko) running all VMs
  • K3s Testing: 3 masters + 3 workers for testing and development

External Systems

All my riscv64 hosts:

Naming Conventions

  • idols: Characters from "Oshi no Ko" anime/manga
  • 12kingdoms: Characters from "Twelve Kingdoms" anime/novel series
  • darwin: Characters from "Frieren: Beyond Journey's End" anime/manga
  • k8s: Kubernetes-related systems follow standard naming patterns

How to Add a New Host

The easiest way to add a new host is to copy and adapt an existing similar configuration. All host configurations follow similar patterns but are customized for specific hardware and use cases.

General Process

  1. Identify a similar existing host from the directory structure above
  2. Copy the entire directory and rename it for your new host
  3. Adapt the configuration files for your specific hardware and requirements
  4. Update references in the flake outputs and networking configuration

Essential Steps

  1. Under hosts/
    1. Create a new folder under hosts/ with the name of the new host.
    2. Create & add the new host's hardware-configuration.nix to the new folder, and add the new host's configuration.nix to hosts/<name>/default.nix.
    3. If the new host need to use home-manager, add its custom config into home/hosts/linux/<name>.nix or home/hosts/darwin/<name>.nix.
  2. Under outputs/
    1. Add a new nix file named outputs/<system-architecture>/src/<name>.nix.
    2. Copy the content from one of the existing similar host, and modify it to fit the new host.
      1. Usually, you only need to modify the name and tags fields.
    3. [Optional] Add a new unit test file under outputs/<system-architecture>/tests/<name>.nix to test the new host's nix file.
    4. [Optional] Add a new integration test file under outputs/<system-architecture>/integration-tests/<name>.nix to test whether the new host's nix config can be built and deployed correctly.
  3. Under vars/networking.nix
    1. Add the new host's static IP address.
    2. Skip this step if the new host is not in the local network or is a mobile device.

File Templates

Use existing hosts as templates. The key files typically include:

  • default.nix - Main host configuration
  • hardware-configuration.nix - Auto-generated hardware settings
  • Platform-specific files (e.g., nvidia.nix, apple-silicon.nix, etc.)

Examples to Reference

  • Desktop systems: See idols-ai/ for gaming/workstation setup
  • Server systems: See shoryu/ for K8s/VM hosts
  • macOS systems: See darwin-fern/ for macOS configurations
  • Apple Silicon: See 12kingdoms-shoukei/ for ARM Linux setup

Deploying VM Hosts

The three VM hosts (shoryu, shushou, youko) put their VMs' secondary network on the Linux bridge br0 (attached by the bridge CNI plugin); the pod network is Cilium (flannel disabled).

  • switch is fine for changes that don't touch the network stack (e.g. journald or other service settings). Deploy serially (-p 1) and re-check VM reachability afterwards.
  • Use boot + a serial reboot for anything that can drop the network mid-flight — systemd.network / br0 changes, nixpkgs updates, or any broad rebuild.
  • If a host loses VM networking, check br0 and the VM taps, or reboot the host.

Distributed Building

I usually run the build command on Ai and nix will distribute the build to other NixOS machines, which is convenient and fast.

When building some packages for riscv64 or aarch64, I often have no cache available because of various changes under the hood, so I need to build much more packages than usual, which is one of the reasons why the cluster was originally built, and another reason is distributed building is cool!

References

Oshi no Ko 【推しの子】 - Wikipedia:

The Rolling Girls【ローリング☆ガールズ】 - Wikipedia:

List of Twelve Kingdoms characters

List of Frieren characters