Commit Graph
95 Commits
Author SHA1 Message Date
ryan4yin | 二花 4e78fd964a fix(outputs): keep microvm-guest-parity from forcing guest config (#484)
Reading the host's install script evaluated each guest, which pulls the private secrets input, so
CI could not evaluate the suite at all.

Assisted-by: DeepSeek Harness 0.2.1-rc.2 (deepseek-flash)
2026-10-21 21:49:07 +08:00
ryan4yin | 二花 de6a66ccd4 fix(microvm): run k3s guests from their nixosConfigurations entry (#483)
Assisted-by: DeepSeek Harness 0.2.1-rc.2 (claude-opus-5-5)
2026-10-21 21:32:56 +08:00
Ryan Yin 3b3992468b fix(colmena): deploy the systems nixosConfigurations evaluates
Colmena deployments now ship the same toplevel as nixosConfigurations: nodes gain the nixpkgs
registry, nixPath and dated version, and akane no longer gets the x86_64 dsh.

Assisted-by: DeepSeek Harness 0.2.1-rc.2 (claude-opus-5-5)
2026-10-21 20:59:07 +08:00
ryan4yin | 二花 274590df63 feat(aquamarine): track linuxPackages_latest (#478)
Assisted-by: DeepSeek Harness (deepseek-flash)
2026-10-20 23:12:37 +08:00
Ryan Yin 52573278bd feat(base): install python313 on every NixOS host
Hosts without Home Manager (aquamarine, the VM hosts, k3s guests) had no
interpreter. The system profile is separate from the per-user one, so desktops
keep their Home Manager copy in PATH.

Assisted-by: DeepSeek Harness 0.2.1-rc.2 (deepseek-flash)
2026-10-20 00:20:12 +08:00
ryan4yin | 二花 4258538588 feat(aquamarine): make the iGPU usable for media and OpenVINO (#474)
* feat(aquamarine): make the iGPU usable for media and OpenVINO

The R9700s stay reserved for inference: VA-API/QSV transcoding moves to the iGPU media
engine, and `openvino-gpu` works around ROCm's OpenCL loader shadowing the plugin's
`ocl-icd` (WA-038).

Assisted-by: DeepSeek Harness 0.2.1-rc.2 (deepseek-v4.1-flash)

* docs(aquamarine): document OpenVINO on the iGPU

How to run models on the iGPU: the `openvino-gpu` wrapper, the encoder staticization Whisper needs
on `GPU`, compile/steady timings and memory, and the language and cache caveats.

Assisted-by: DeepSeek Harness 0.2.1-rc.2 (deepseek-v4.1-flash)
2026-10-18 11:15:56 +08:00
ryan4yin | 二花 3d38ff7433 fix(aquamarine): persist the user's cache directory (#466)
On a tmpfs root the flake/fetch cache is empty at every boot, so
`nix run <flake-ref>` re-downloads its input and fails until DNS is up: a reboot
test showed llama-swap's first start failing on that. Other caches write into
RAM meanwhile.

Assisted-by: DeepSeek Harness (deepseek-flash)
2026-10-16 21:36:00 +08:00
ryan4yin | 二花 afa7a4c72e fix(aquamarine): persist the user's unit directories (#464)
Assisted-by: DeepSeek Harness (deepseek-flash)
2026-10-16 21:08:41 +08:00
ryan4yin | 二花 3266aad4a2 fix(power): drop TuneD for native CPU scaling (#461) 2026-10-16 16:45:08 +08:00
ryan4yin | 二花 bb89987640 fix(mihomo): use fake-IP for gateway IPv6 (#442) 2026-10-14 23:50:40 +08:00
Ryan Yin 492dcb13fe fix(ai): match the wired NIC by prefix, not by name
Assisted-by: DeepSeek Harness (deepseek-flash)
2026-10-14 20:15:01 +08:00
Ryan Yin 55b7741108 refactor(ai): return ai to the MSI MAG B760M MORTAR and the RTX 4090
The second NVMe and its LUKS volume leave with the Z890 board, so the data-disk disko module is deleted; `ai` must activate this change before the disk is pulled, or its initrd blocks on the missing volume.

Assisted-by: DeepSeek Harness (deepseek-flash)
2026-10-14 20:15:01 +08:00
Ryan Yin 72a7a2ae88 feat(aquamarine): revive aquamarine as a two-GPU inference host
The Z890 board and its root SSD move here from `ai`, and the retired VM's name and 192.168.5.101 are reused.

Assisted-by: DeepSeek Harness (deepseek-flash)
2026-10-14 20:15:01 +08:00
ryan4yin | 二花 c347bfe6a9 fix(k3s): drop Home Manager from the test workers (#432)
The worker MicroVMs ran `home-manager-ryan.service`, which failed on two of them
(`/nix/var/nix/temproots` permission denied, and no `/home/ryan`), so
`HostSystemdServiceCrashed` fired for days. Mirror the masters and extend their
regression test to all six nodes.

Assisted-by: DeepSeek Harness 0.2.0-rc.2 (deepseek-flash)
2026-10-11 19:48:14 +08:00
ryan4yin | 二花 2f6f2b6926 fix(monitoring): restore youko metrics, alerts and VM-host deploys (#431)
* flake.lock: Update

nixpkgs only. The new revision builds VictoriaMetrics with cgo
(CGO_ENABLED=1); its pure-Go zstd decoder rejected a stored values block, so
every small-part merge panicked on youko.

* fix(colmena): address the VM hosts by name, not by their IP

`~/.ssh/config` pins the passphrase-protected builder key with IdentitiesOnly
for 192.168.*, so IP-addressed ssh cannot authenticate without a terminal. The
name resolves through the aliases in /etc/ssh/ssh_config.

* fix(monitoring): repair two alert rules VictoriaMetrics rejects

`on(uid)` matches on a label kube-state-metrics no longer emits, and
`!= ON(__name__, instance)` treats every pg_settings series as a duplicate.
They now join on (namespace, pod) and compare with `changes(...[5m])`.

* fix(monitoring): keep disk latency alerts off mechanical disks

Both rules averaged per-op latency with no device or sample guard, so the 4 TB
USB HDD volumes paged constantly (>100 ms/write in 26% of the last 30 days).
They now require a non-rotational physical device and >=5 ops/s.

Assisted-by: DeepSeek Harness 0.2.0-rc.2 (deepseek-flash)
2026-10-11 18:16:04 +08:00
ryan4yin | 二花 3e66bbf24f feat(hosts): add WiVRn CLI streaming for Quest 3 #392 2026-10-10 22:49:48 +08:00
ryan4yin | 二花 60bd70bd0c refactor(agents): configure dsh from its profile patch (#415) 2026-10-08 15:29:31 +08:00
ryan4yin | 二花 3694961e74 fix(postgres,darwin): close the loopback superuser path and scope agenix ownership (#401)
* fix(postgres): require a password for superuser over TCP

* test: assert the postgres loopback auth rules

* fix(darwin): keep root-only agenix secrets root-only

* docs(security): record the local auth boundaries and retire WA-015
2026-10-07 15:22:03 +08:00
ryan4yin | 二花 43f39e8e9f fix(k3s): deny credential-bearing host ports to pod sources (#399)
* fix(k3s): deny credential-bearing host ports to pod sources

* test: assert the k3s pod firewall drops sensitive ports

* docs(security): record the pod firewall boundary
2026-10-07 15:03:56 +08:00
ryan4yin | 二花 28e0b156ce refactor(agents): link dsh shared policy as a home-layer patch (#388)
dsh rewrites a profile's cordis.patch.yml at runtime, so linking the web
profile put GUI state in the checkout and let runtime writes clobber manual
edits. Link the stable policy as the $DSH_HOME home layer instead: dsh applies
it after every profile and only reads it.

The host-plane compaction-basic row is disabled on the web surface and each
agent preset owns its own copy, so the top-level override was inert. Drop it;
the model contextWindow/maxTokens are the effective knobs.

- add home/base/tui/agents/dsh/cordis.patch.yml, linked from default.nix
- drop the ~/.dsh/profiles/web link and the tracked web/ profile
- keep runtime state and private/local model routes in the profile patch
- update the eval test and README
2026-10-06 14:04:55 +08:00
ryan4yin | 二花 205f7c622b feat(agents): version the dsh web profile and add a proxied launcher #387 2026-10-06 00:54:30 +08:00
ryan4yin | 二花 6b8c5bfae4 feat(vinput): drop NPU/OpenVINO support (#384)
Vinput's streaming Zipformer models have unbounded dynamic shapes, so
OpenVINO cannot compile them for the GPU or the NPU; the `openvino`
provider and ai's Intel NPU enablement could never run on the NPU (noted
as unverified in WORKAROUNDS WA-017). Remove the backend option, the
daemon's OpenVINO library path, and the intel-npu module enable, and
update the eval test, README and WORKAROUNDS (drop WA-016 and WA-017).

Keep Vinput's stock package and its bundled sherpa-onnx (1.13.4): the
fork-based and nixpkgs-master sherpa overrides existed only to provide the
`openvino` provider, so drop package.nix with them.

Assisted-by: opencode (DeepSeek V4.1 Flash)
2026-10-05 15:42:48 +08:00
ryan4yin | 二花 8f0961d9c2 fix(desktop): tie ai's link DNS takeover to mihomo's lifecycle (#380)
* fix(desktop): tie ai's link DNS takeover to mihomo's lifecycle

sing-tun normally points systemd-resolved at the TUN DNS by running
resolvectl (dns/domain/default-route). Under nixpkgs' hardened
DynamicUser mihomo unit that hand-off fails silently: the unit's
RestrictAddressFamilies blocks resolvectl's D-Bus socket and resolve1
requires privileged authorization, so resolved keeps querying the
physical-link DNS, fake-ip is bypassed, and domain rules degrade to
MATCH.

Do the hand-off as '+' ExecStartPost/ExecStopPost hooks instead: '+'
commands run outside the sandbox (systemd.exec: address-family
restrictions do not apply to them) and ExecStopPost reverts even on
crashes, so a dead mihomo leaves DNS usable.

Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>

* docs(mihomo): record sing-tun's resolvectl hand-off as the leak cause

Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>

* refactor(desktop): keep mihomo only on ai and shoukei

ruby/kana never ran the core (service inactive live), and their
resolved + TUN combo would reintroduce the resolver leak once enabled.
Drop the enable and the explicit module import from computer-use; the
eval test now pins mihomo=false for both guests.

---------

Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
2026-10-05 00:07:32 +08:00
ryan4yin | 二花 bbfe1fa53b feat(desktop): replace Clash Verge with native mihomo on all NixOS hosts (#379) 2026-10-04 21:04:41 +08:00
Ryan Yin a695a3ce3d feat(vinput): add NPU speech input on ai 2026-10-04 19:51:22 +08:00
ryan4yin | 二花 6a1b4c699a chore(docs): prune stale TODOs (#370) 2026-10-03 17:50:28 +08:00
ryan4yin | 二花 37d79fbb08 chore(hardening): consolidate system hardening and fix stale docs (#368) 2026-10-03 17:11:51 +08:00
ryan4yin | 二花 afdaa3b869 refactor(agents): manage agent config in a home-manager module (#367) 2026-10-03 16:57:38 +08:00
ryan4yin | 二花 f03907c6a4 feat(security): add balanced kernel and exporter baseline (#365) 2026-10-03 16:33:50 +08:00
ryan4yin | 二花 3fdeedaa78 docs(security): document homelab architecture and agent-first audits
Add SECURITY.md covering the threat model, trust boundaries,
proportionate kernel/process hardening, the update/reboot routine and
remaining prioritized work. Record the dated read-only audit findings
(listeners, privileged workloads, effective RBAC, secret metadata and
sandbox/SSH boundaries) separately from planned controls and runtime
verification.

Define agent-first diagnosis: scoped root read-only access on homelab
hosts, and normal-user diagnostics on core desktops with no new sudo
grants. Add canonical domain-based k3s API endpoint regression tests,
record the unverified Darwin secret-ownership concern as WA-015, and
correct comments claiming mode 0000 blocks privileged root.
2026-10-03 16:26:32 +08:00
ryan4yin | 二花 18925eac8b fix(colmena): expose colmenaHive for the nix-community evaluator (#361)
* fix(colmena): expose colmenaHive for the new evaluator

nixpkgs now ships the nix-community Colmena rewrite, whose default
evaluator reads an evaluated colmenaHive flake output instead of the raw
colmena hive. Without it every colmena apply recipe failed with
"does not provide attribute colmenaHive".

Add a colmena input pinned to the same v0.5.0 as nixpkgs and expose
colmenaHive = colmena.lib.makeHive <raw hive>, with the raw hive kept as
an internal binding rather than a flake output of its own.

* chore: clean up stale colmena and aquamarine references

- point the Colmena links at nix-community/colmena after the move
- rename the security-aquamarine-metrics test to youko-metrics (it
  already tested youko) and update the outputs README
- drop the commented aquamarine remote-builder example
2026-10-03 12:14:13 +08:00
ryan4yin | 二花 4909f6357b security: scope privileges and stop a world-readable secret copy (#335) 2026-10-01 21:18:34 +08:00
ryan4yin | 二花 d690f45183 feat: add CanoKey hardware security key support (#334) 2026-10-01 20:50:13 +08:00
Ryan Yin 2e5f539a47 fix(hosts): move UPS/nut metrics to youko after the swap 2026-09-29 21:36:47 +08:00
ryan4yin | 二花 e5b95affa6 feat(nixos): default-deny host inbound, trust the LAN/tailnet (#325)
Hosts used to trust the router as the only boundary (servers ran with the
firewall off; service ports were opened globally), so any LAN device could
reach every host service. Move filtering to the hosts as defence in depth
behind the router, which keeps its firewall on and uses a port mapping for
the few ports we actually need.

- base: nftables backend; deny inbound by default; trust the home LAN by
  source and the tailnet by interface (restrict the tailnet with ACLs);
  suppress per-service `openFirewall` global ports; allow node_exporter only
  from the monitoring host. The local podman bridge, the tailnet and the pod
  CIDRs are accepted only after the node_exporter drop.
- servers: stop disabling the firewall; drop redundant global port opens
  (caddy 80/443, NFS 2049, Jellyfin, localsend/test ports); delete the
  redundant desktop firewall module; aarch64/riscv64 import the firewall
  explicitly (akane was missing it).
- k3s: Cilium pods (10.0.0.0/8, fd00::/8) may reach host services; drop the
  flannel-era cni0/flannel.1 trust.
- youko: postgres listens on localhost + the LAN with pg_hba limited to the
  LAN; the NFS export stays LAN-scoped; remove the unauthenticated v2ray
  proxy.
- tests: `security-firewall` asserts every host runs the shared firewall, has
  the LAN CIDR rule and no global ports.
2026-09-26 14:02:03 +08:00
Ryan Yin 9a6923b4e9 fix(k3s): drop Home Manager from test masters
Keep the six node configuration intact while removing the failing user activation from the three control-plane MicroVMs. Update the regression test and current K3s configuration notes.
2026-09-24 13:35:33 +08:00
Ryan Yin a3efb42202 chore(k8s): update secrets 2026-09-23 12:07:28 +08:00
ryan4yin | 二花 3b13291216 feat(computer-use): sync the X clipboard and document the input-level design (#308)
- run two autocutsel instances so the X PRIMARY/CLIPBOARD selections stay in
  sync, making copy/paste work in the session and through VNC
- drop --disable-blink-features=AutomationControlled: nothing drives the
  browser over CDP, so navigator.webdriver is already false and the flag only
  marked the setup as automated
- document why the driver stays on real X11/AT-SPI input and is never granted
  a CDP attach, and note the SSH-tunnel path for manual VNC access
- extend the computer-use eval test
2026-09-22 20:45:09 +08:00
ryan4yin | 二花 b19818ef44 feat(k8s): add the k3s-test-1 worker nodes (#307)
Add worker-1 (4 vCPU / 16 GiB) on shoryu, and worker-2 (4 vCPU / 16 GiB) plus worker-3 (2 vCPU / 8 GiB) on shushou, as microVMs. They come with a genK3sAgentModule and the per-host/output/home-manager wiring, and are labelled node-role.kubernetes.io/worker=true. The masters keep their control-plane:NoSchedule taint, so workloads land on the workers.
2026-09-21 23:09:13 +08:00
Ryan Yin 00aab5f562 feat(computer-use): add a headless X11/i3 environment
Give the idols-ruby and idols-kana VMs a headless computer-use
environment: Xvfb plus i3, driven by cua-driver (and computer-use-linux
as a fallback). Includes the system module (AT-SPI, fonts, Mesa, Clash
Verge proxy, proxy-region timezone), the home i3 module (Xvfb/i3/x11vnc
services, browser flags), host wiring and an eval test.
2026-09-21 21:31:27 +08:00
Ryan Yin 6180072646 chore: drop the KubeVirt guest bits
The k3s-test workers never ran (they were stuck waiting for volume
binding) and will be re-added as microVMs, so remove their KubeVirt
configs and everything that only existed for KubeVirt guests:

- hosts/k8s/k3s-test-1-worker-{1,2,3} and their outputs
- lib/genKubeVirtGuestModule.nix and lib/genK3sAgentModule.nix (the
  latter was only used by the workers)
- the KubeVirt image upload recipes (upload-vm, upload-idols,
  upload-k3s-test) and the kubevirt image package in the idols outputs
- refresh the docs; keep only accurate historical mentions (e.g. the
  restic backup path must stay as it is)
2026-09-21 17:24:40 +08:00
Ryan Yin 2c7629000f refactor: rename the three VM hosts to 12kingdoms-*
Drop the now-meaningless kubevirt- prefix and move the hosts out of
hosts/k8s/, following the existing 12kingdoms-shoukei layout: the
directory and output carry the theme prefix while the configuration,
hostname and colmena target are the bare name (shoryu, shushou, youko).

- hosts/k8s/kubevirt-* -> hosts/12kingdoms-*; the shared disko config
  becomes host-disko-fs.nix; the now-different relative imports are fixed
- outputs, vars/networking.nix keys, the Justfile recipes/tags, the eval
  tests and the docs follow
- colmena reaches these hosts by IP (a new targetHost override), since
  their DNS name changes with the hostname and is not resolvable until
  the router learns it
2026-09-21 17:10:14 +08:00
Ryan Yin 01110e93d2 Merge remote-tracking branch 'origin/main' into chore/rename-kubevirt-hw-config
# Conflicts:
#	outputs/x86_64-linux/src/k3s-test-1-master-1.nix
#	outputs/x86_64-linux/src/k3s-test-1-master-2.nix
#	outputs/x86_64-linux/src/k3s-test-1-master-3.nix
2026-09-21 16:50:25 +08:00
Ryan Yin 60c59543b3 refactor: rename the KubeVirt hardware config to qemu-guest
The file is not KubeVirt-specific: it is the hardware configuration for
any QEMU guest that boots from a disk (KubeVirt or libvirt). All it does
is pull in qemu-guest.nix and set fileSystems."/" from the disk labeled
nixos.
2026-09-21 16:48:47 +08:00
Ryan Yin 4ad359696e feat(microvm): run the k3s-test-1 masters as microVMs
Replace the KubeVirt/etcd members with microVMs on the three physical
hosts, keeping the existing cluster: the guests inherit their host key
and k3s state from the old disks, so etcd recovers in place instead of
bootstrapping a new cluster.

- convert hosts/k8s/k3s-test-1-master-{1,2,3} to genMicrovmGuestModule
  and switch their outputs to the microvm module (keep home-modules)
- declare the vms on shoryu/youko/shushou and bridge their taps to br0
- each guest config is self-contained (its secret group and the
  nixpkgs.config override live in the guest, not the host module)
- guest specialArgs carry agenix/mysecrets so secrets/nixos.nix works
2026-09-21 16:32:23 +08:00
Ryan Yin c292509351 refactor: retire the aquamarine VM host
Aquamarine's services now run directly on kubevirt-youko, so the guest
definition is no longer needed:

- move its service modules to hosts/k8s/kubevirt-youko/aquamarine/ and
  drop the explicit import list, leaving aquamarine-services.nix with
  just its IP and secret groups
- delete the guest base, journald override, output entry and home config
- repoint the eval tests that read the guest's config to kubevirt-youko
- drop the now-dead 'aqua' colmena recipe and its image upload

The aqua KubeVirt VM itself is deleted separately; the name is kept in
the docs for a possible later test microvm/qemu-vm on idols-ai.
2026-09-21 15:45:02 +08:00
ryan4yin | 二花 d40ed42f6c feat(k3s-test-1): add three worker nodes and taint the control plane (#283)
* feat(k3s-test-1): add three worker nodes and taint the control plane

- hosts/k8s/k3s-test-1-worker-{1,2,3} (k3s agents) + flake outputs
- vars/networking: .117/.118/.119 (the .111-.113 range is still held by the
  never-deployed k3s-prod-1 workers)
- k3s-test-1-master-{1,2,3}: nodeTaints=node-role.kubernetes.io/control-plane:NoSchedule
- Justfile: upload the worker images too

* chore(k3s-test-1): move the workers to .111-.113

The k3s-prod-1 entries that held those addresses are gone (nix-config#284).
2026-09-21 12:56:49 +08:00
ryan4yin | 二花 01c6b3290a chore: remove the never-deployed k3s-prod-1 cluster (#284)
It was never bootstrapped (no flux deploy key, VMs commented out since
2024-10) and only carried a stale duplicate of the test cluster's config.

- hosts/k8s/k3s-prod-1-{master,worker}-{1,2,3} + flake outputs + the home module
- the k3s-prod-1-token agenix entry (the .age file lives in the private secrets repo)
- the k3s-prod-1-* IPs, the offlineHosts entries, the Justfile recipes and the docs
2026-09-21 12:53:43 +08:00
ryan4yin | 二花 f8d12ada24 feat(btrbk): make snapshots explicit, guarded, and stored in @snapshots (#271)
* feat(btrbk): make snapshots explicit and guarded

Replace the always-on, hardcoded module with modules.btrbk options (enable/volume/subvolume/snapshotDir/target/onCalendar), and assert the btrfs top-level subvolume is mounted so a missing mount fails evaluation instead of silently failing at runtime.

Store snapshots in the @snapshots subvolume (snapshot_dir=@snapshots) rather than next to the live subvolume, make the off-host target explicit and off by default, and document that local snapshots are not off-host backups.

Enable the feature explicitly on idols-ai, shoukei and the three kubevirt hosts; qemu-guest now disables it through the option. Add eval tests for shoukei and ai.

* feat(monitoring): alert on btrbk snapshot failure

Add HostBtrbkSnapshotFailed on node_systemd_unit_state for btrbk-btrbk.service, so a broken snapshot schedule alerts instead of only surfacing in the journal.
2026-09-19 11:43:11 +08:00
Ryan Yin 1ec29059ad fix(sunshine): stabilize remote Niri sessions 2026-09-01 22:32:19 +08:00