Drop the now-meaningless kubevirt- prefix and move the hosts out of
hosts/k8s/, following the existing 12kingdoms-shoukei layout: the
directory and output carry the theme prefix while the configuration,
hostname and colmena target are the bare name (shoryu, shushou, youko).
- hosts/k8s/kubevirt-* -> hosts/12kingdoms-*; the shared disko config
becomes host-disko-fs.nix; the now-different relative imports are fixed
- outputs, vars/networking.nix keys, the Justfile recipes/tags, the eval
tests and the docs follow
- colmena reaches these hosts by IP (a new targetHost override), since
their DNS name changes with the hostname and is not resolvable until
the router learns it
This is not a Python package, so drop the packaging/build/pyinstaller/
framework/type-checker/pyenv/poetry/pdm/pixi/celery/rabbitmq rules and the
unused Marimo and Streamlit sections. Keep the bits occasional Python
scripts do need: byte-compiled files, virtualenvs, local secrets and the
common tool caches.
The Python section of the .gitignore template has a bare 'lib/' (and
'lib64/'), which also matched the Nix helper modules in lib/. The
'!lib/*.nix' line added earlier could not help, since a file cannot be
re-included while its parent directory is excluded.
The physical hosts no longer need k3s or KubeVirt: aquamarine runs
natively, the k3s-test masters run as microVMs and kana/ruby under
libvirt. So drop the host cluster and split the shared parts out of the
KubeVirt host module:
- lib/genVmHostModule.nix: the host bits the VMs still need (br0 + the
physical NIC as a bridge port, KVM, sysctls, earlyoom, libvirt,
journald, stateVersion)
- the three hosts use it and drop genK3sServerModule; also drop the
longhorn workarounds (openiscsi + the /usr/local/bin tmpfile) and the
virtctl package
- remove lib/genKubeVirtHostModule.nix
The file is not KubeVirt-specific: it is the hardware configuration for
any QEMU guest that boots from a disk (KubeVirt or libvirt). All it does
is pull in qemu-guest.nix and set fileSystems."/" from the disk labeled
nixos.
hosts/k8s/README.md: how the microVMs keep state, the tap/br0 naming, how
to move a KubeVirt VM over in place, and the refreshed TODOs. Also note
the migration in Virtual-Machine.md and flag the volume images in
AGENTS.md.
Replace the KubeVirt/etcd members with microVMs on the three physical
hosts, keeping the existing cluster: the guests inherit their host key
and k3s state from the old disks, so etcd recovers in place instead of
bootstrapping a new cluster.
- convert hosts/k8s/k3s-test-1-master-{1,2,3} to genMicrovmGuestModule
and switch their outputs to the microvm module (keep home-modules)
- declare the vms on shoryu/youko/shushou and bridge their taps to br0
- each guest config is self-contained (its secret group and the
nixpkgs.config override live in the guest, not the host module)
- guest specialArgs carry agenix/mysecrets so secrets/nixos.nix works
lib/genMicrovmGuestModule.nix: qemu guest with a shared read-only store
(virtiofs), persistent /etc, /var and /home volumes, a tap on br0 (its
name derived from the guest IP, as IFNAMSIZ caps it at 15 chars), and a
deterministic MAC. Also keep lib/*.nix tracked.
The service modules moved two levels deeper, so their certs/ paths
resolved relative to the wrong directory. Use mylib.relativeToRoot so
they no longer depend on the module's depth.
Drop the retired host's name from the filenames:
aquamarine-services.nix -> homelab-services.nix and the aquamarine/
module dir -> homelab-services/. The "aqua" name now stays only in the
docs/vars, where it is kept for a possible later test microvm/qemu-vm on
idols-ai.
Aquamarine's services now run directly on kubevirt-youko, so the guest
definition is no longer needed:
- move its service modules to hosts/k8s/kubevirt-youko/aquamarine/ and
drop the explicit import list, leaving aquamarine-services.nix with
just its IP and secret groups
- delete the guest base, journald override, output entry and home config
- repoint the eval tests that read the guest's config to kubevirt-youko
- drop the now-dead 'aqua' colmena recipe and its image upload
The aqua KubeVirt VM itself is deleted separately; the name is kept in
the docs for a possible later test microvm/qemu-vm on idols-ai.
The VMs now declare their root disk as a DataVolume with `source: upload`, so
upload straight into `<vm>-disk` (CDI only accepts an upload into a DataVolume)
instead of a shared golden PVC. Takes the disk size as an argument.
* feat(k3s-test-1): add three worker nodes and taint the control plane
- hosts/k8s/k3s-test-1-worker-{1,2,3} (k3s agents) + flake outputs
- vars/networking: .117/.118/.119 (the .111-.113 range is still held by the
never-deployed k3s-prod-1 workers)
- k3s-test-1-master-{1,2,3}: nodeTaints=node-role.kubernetes.io/control-plane:NoSchedule
- Justfile: upload the worker images too
* chore(k3s-test-1): move the workers to .111-.113
The k3s-prod-1 entries that held those addresses are gone (nix-config#284).
It was never bootstrapped (no flux deploy key, VMs commented out since
2024-10) and only carried a stale duplicate of the test cluster's config.
- hosts/k8s/k3s-prod-1-{master,worker}-{1,2,3} + flake outputs + the home module
- the k3s-prod-1-token agenix entry (the .age file lives in the private secrets repo)
- the k3s-prod-1-* IPs, the offlineHosts entries, the Justfile recipes and the docs
* feat(kubevirt-youko): export a shared golden-image store over NFS
* feat(vm-images): upload golden images into the cluster
Replace the rsync-to-orangepi5 step with a CDI upload into the golden-image
PVCs (vms/golden-images in k8s-gitops), so VM images no longer depend on the
single-board file server.
services.earlyoom.freeMemThresholdPercent does not exist in this nixpkgs; the module exposes freeMemThreshold/freeSwapThreshold. Just enable earlyoom with its defaults (trigger at 10% free mem/swap).
* refactor(kubevirt): drop the static 1G hugetlb pool, use THP on demand
The hosts pre-reserved 48/48/15 GiB of 1G hugepages, but the VMs only used 32/16/11 GiB, so ~52 GiB was stranded (hugetlb is exclusive and cannot be overcommitted or shared with the host / non-hugepage VMs). It also blocked scheduling (a 16 GiB hugepage VM cannot fit on a host with fewer free pages, e.g. youko with 15/24 GiB) and made the full intended VM set (idols 3x8G + k3s-test 3x16G + k3s-prod 3x4G + 3x16G = 132 GiB vs 146 GiB physical) impossible to fit.
Use transparent huge pages on demand (madvise) instead, and let the VMs use ordinary memory (instancetypes updated in k8s-gitops). This frees the stranded memory, removes the scheduling rigidity, and makes overcommit/swap usable again.
* feat(kubevirt): add earlyoom; stop overriding swappiness (zram already enabled)
zram is already provided by modules/nixos/base/zram.nix (enabled by default: 50% RAM, zstd, prio 100 + swappiness/watermark tunings). But genKubeVirtHostModule hard-coded vm.swappiness = 0, which overrode the modules mkDefault 180 and disabled swapping entirely - so the zram device was never used (Swap used: 0B).
- Remove the hard-coded vm.swappiness (let the zram modules tuning apply); the disk swapfile from disko stays as the last resort.
- Add services.earlyoom to kill the greediest process before the host thrashes / OOMs.
vmagent (and other senders) can deliver the same series+timestamp twice (retries/HA), which vmsingle rejected with "duplicate sample for timestamp; overrides not allowed". Set -dedup.minScrapeInterval=30s (>= the senders scrapeInterval) so duplicates collapse instead of being rejected.
- genKubeVirtHostModule: replace the Open vSwitch bridge (ovsbr1) with a Linux
bridge br0 for the VMs secondary network; the bridge CNI plugin attaches the VM
taps to br0 and the host keeps its address on br0. Drop the now-unused
multus-cni host package.
- kubevirt hosts: stop forcing disableFlannel = false, so k3s disables flannel and
Cilium provides the pod network.
shoukei is a laptop using DHCP and never consumes hostsAddr.shoukei.
Its 192.168.5.108 collided with k3s-prod-1-master-1 and produced a wrong
ssh Host shoukei entry.
The Settings UI stored `theme.mode` in ~/.local/state/noctalia/settings.toml, so it was not versioned. Pin it in the config baseline; the [location] sunrise/sunset schedule still drives light/dark.
At background_opacity 0.2 the light-mode surface disappears into dark wallpapers, so the dark light-mode text sits on a dark bar. Raise it to 0.61 to keep contrast while staying slightly translucent.
`programs.noctalia.settings` deploys the TOML as a store symlink, so every edit needs a `home-manager switch`. Deploy the baseline from the config layer with `mkOutOfStoreSymlink` instead: Noctalia hot-reloads config-layer files and never rewrites them, so edits apply live while Settings UI changes stay in ~/.local/state/noctalia/settings.toml (which loads last) and out of the repo.
Update the docs and host-specific comments to match the two-layer model.
The sandbox remaps XDG_DATA_HOME/XDG_CONFIG_HOME, so paths Firefox exposes to
or shares with the host ended up isolated. Bind the MPRIS artwork dir and the
real trash rw, and mimeapps.list, user-dirs.dirs and the shared NSS cert store
ro.
recently-used.xbel stays isolated: GTK writes it atomically, which a
single-file rw bind would break.
Add cpu, ram, disk and network up/down sysmon widgets to the bar start lane,
matching the old v4 SystemMonitor. Set the clock to ISO date, full English
weekday and 24h time (e.g. "2026-09-19 Saturday 16:06").
English stays the default (LANG=en_US.UTF-8). Drop the Chinese overrides for
LC_TIME, LC_NUMERIC, LC_ADDRESS, LC_NAME, LC_TELEPHONE and LC_IDENTIFICATION:
LC_NUMERIC matches en_US, the others are effectively unused, and LC_TIME forced
Chinese date and weekday names.
Keep zh_CN only where the en_US default is wrong for China: metric units, A4
paper and CNY.
Ryan Yin
ryan4yin | 二花