Commit Graph
642 Commits
Author SHA1 Message Date
Ryan Yin 124bcd8490 docs: drop the stale "just test exit code" caveat
`just test` has enforced the suite result since #350: the recipe raises unless
`evalTests` is `true`, so a zero exit already means pass. Update the places that
still described the old raw-`nix eval` behaviour: AGENTS.md, README.md,
SECURITY.md, outputs/README.md, and the nix-config-debug / nix-config-new-host /
nix-config-update skills.

The exit-0 caveat is kept only where the raw `nix eval .#evalTests` command is
documented, because that command genuinely exits 0 even on `false`.
2026-10-07 19:54:32 +08:00
ryan4yin | 二花 6f136b50e4 fix(shoukei): mitigate brcmfmac wifi auth failures, document duplicate-profile bug (#406)
BCM4378/brcmfmac intermittently fails Wi-Fi association (ASSOC-REJECT /
authentication timeout) on this laptop. NetworkManager misreads the failure
as a bad key and re-prompts; whichever client reconnects (Noctalia, nmtui,
nmcli) then saves the answer as a duplicate "<SSID> 1" profile. The problem
predates the Noctalia v5 shell and is driven by the driver, not the shell.

- disable Wi-Fi powersave on shoukei, a common source of brcmfmac auth timeouts
- WORKAROUNDS.md WA-024: the powersave mitigation and its removal condition
- WORKAROUNDS.md WA-025: the pending upstream brcmfmac port_authorized fix
- document how to recover a wedged supplicant without rebooting, and that
  deleting an active NM profile is what turned a reconnect into a dead link

Refs: https://patchew.org/linux/20260915-brcmfmac-4way-handshake-offload-authenticated-event-v2-1-fbb9cc7ac177@jannau.net/
2026-10-07 19:52:57 +08:00
Ryan Yin 3e4e093869 feat(shoukei): run x86/x86_64 apps and games via muvm+FEX
Apple Silicon's 16K pages mean FEX cannot run natively and qemu-user cannot
run complex x86_64 binaries. Wire the maintained rowanG077/nix-x86-on-aarch64
runtime (FEX inside a 4K muvm microVM) into shoukei so games, Steam, Wine and
other x86/x86_64 programs work, and let the module own the x86 ELF handlers
instead of qemu-user.

The input is pinned to our fork (ryan4yin/nix-x86-on-aarch64) at the reviewed
commit. Dropping boot.binfmt.emulatedSystems also removes x86_64 from
nix.settings.extra-platforms, so the qemu-user/uncached-x86-build gap is
recorded as WA-022.

- flake.nix: add nix-x86-on-aarch64 input (WA-021)
- hosts/12kingdoms-shoukei/game/: programs.x86-on-arm + steam/wine, 6 GiB VM
- hardware-configuration.nix: drop emulatedSystems/preferStaticEmulators
- apple-silicon.nix: drop the now-redundant box64
- docs: WA-012 resolved; WA-021 and WA-022 added; host README updated
2026-10-07 15:55:55 +08:00
ryan4yin | 二花 f5a111c638 feat(backup): hourly read-only snapshots and per-backup integrity checks (#402)
* feat(restic): verify the repository after each backup

* feat(youko): snapshot the restic store hourly

* docs(backup): document the snapshot and check layers
2026-10-07 15:26:57 +08:00
ryan4yin | 二花 3694961e74 fix(postgres,darwin): close the loopback superuser path and scope agenix ownership (#401)
* fix(postgres): require a password for superuser over TCP

* test: assert the postgres loopback auth rules

* fix(darwin): keep root-only agenix secrets root-only

* docs(security): record the local auth boundaries and retire WA-015
2026-10-07 15:22:03 +08:00
ryan4yin | 二花 3acb5d4ee5 fix(nfs,libvirt): restrict the golden export and bind VNC to loopback (#400)
* fix(nfs): restrict the golden export to its real clients

* fix(libvirt): bind VNC consoles to loopback

* fix(firewall): drop VNC ports on every host

* docs(security): record the NFS and VNC boundaries
2026-10-07 15:10:52 +08:00
ryan4yin | 二花 5f6a1dadf4 docs: fix stale claims and AI-flavored prose (#397) 2026-10-07 14:48:36 +08:00
Ryan Yin fabccbafda fix(monitoring): ignore openapi discovery in apiserver latency alert
KubernetesApiServerLatency fired continuously on the p99 of
apiserver_request_duration_seconds_bucket. Every firing series was
verb="GET", subresource="openapi/v3" (kubelet, kube-controller-manager,
kube-scheduler, apiserver) at ~8s, while real verbs (POST/LIST/DELETE)
stay around 0.2s.

openapi/v3 is a streaming discovery endpoint, like WATCH, so exclude
subresource=~"^openapi/" from the rule.
2026-10-07 13:21:43 +08:00
ryan4yin | 二花 76c4566e9b fix(k8s): pass inputs to the MicroVM guests (#396)
Host builds have been broken since #375 - function called without argument inputs
2026-10-07 13:20:08 +08:00
ryan4yin | 二花 95b849420e feat(gaming): declarative Proton and dGPU game rendering (#393) 2026-10-07 13:14:42 +08:00
ryan4yin | 二花 90bbe8bc4f fix(monitoring): stop recurring false alerts (#395)
The Telegram channel was dominated by alerts that can never be actionable:

- DriveTemperatureHigh/Critical did not filter temperature_type, so
  max(smartctl_device_temperature) picked the drive's rated
  op_limit_max/critical_limit_max instead of the actual reading. ai reported
  95 C and youko 130 C while the drives actually ran at 40-48 C.
- HostPhysicalComponentTooHot excluded label!="tctl", but the AMD k10temp
  hwmon sensor labels itself Tctl, so the CPU control temperature (up to
  ~100 C under load) tripped the alert on every AMD host.
- MicroVM guests share the host's read-only /nix/store, so their
  nix-gc.service always failed and tripped HostSystemdServiceCrashed on all
  six k3s guests. Disable automatic GC in the guest; the host's nix.gc
  reclaims the shared store.

The remaining k3s alert noise comes from workloads running on the masters
that belong on the workers; that is fixed in k8s-gitops, not by resizing
the control-plane VMs.
2026-10-07 12:51:30 +08:00
Ryan Yin a7cc474a50 docs(hosts): update the idols-ai memory notes for 1.28 V
The bring-up runs at the default 1.25 V; afterwards Gear mode is set to
Gear2 and VDD/VDDQ are raised to 1.28 V, changing nothing else. 1.29 V and
up do not train. Also record that the 1.25 V config passed stressapptest
but still hard-froze under a real llama.cpp load, so a passing test is not
proof of stability.
2026-10-07 07:38:52 +08:00
Ryan Yin a091fe1aca docs(hosts): refresh the ai screenshots and correct its hardware notes
- replace the btop screenshot with the 2026-10-06 annotated capture
  (btop + fastfetch + the strata/llama.cpp serving logs); the old
  2026-10-01 file is gone, so the README link moves to the new name
- drop the Helix/AI-agents screenshot and its README reference
- fix stale hardware descriptions: the hosts/README.md inventory row and
  the idols-ai header comment still said i5-13600KF, and the zswap pool
  comment sized its percentages against the old 96G of RAM (now 128G)
- record the hardware history: the platform was rebuilt on 2026-04-27
  (PR #257/#258) onto the Colorful CVN Z890 ARK FROZEN + Core Ultra 7
  270K Plus, replacing the MSI board and i5-13600KF the host was added
  with in 2023-05; the 2x48G kit predates that rebuild, the 2x16G kit
  was added 2026-10-06
- make the DIMM bring-up order mandatory and record its failure mode:
  with the 48G pair in first, or all four DIMMs from the start, the
  board does not POST
- gloss the Chinese BIOS labels in English, marked as translations of
  the labels rather than the firmware's own English wording

Left alone on purpose: the 96G swapfile in hosts/idols-ai/disko-fs.nix.
It was grown from 20G to 1x-RAM on 2026-10-02 (a26ab3be), when the host
still had 96G; resizing it again means re-running disko on the machine,
so that stays a separate decision.
2026-10-06 22:32:34 +08:00
ryan4yin | 二花 e653ec3306 docs(hosts): add mixed-memory notes for idols-ai (#391)
Document the 4-DIMM kit mix (2×48G JUHOR + 2×16G GLOWY) on the Colorful
CVN Z890 ARK FROZEN V20:

- hardware (CPU, board, and both kits)
- the install order that trains (16G pair first)
- the only stable settings (XMP off, DDR5-4800 @ 1.25 V)
- the stress-test recipe and results
- the XMP @ 5600 failure and how to recover a board that will not POST
- re-enrolling the LUKS TPM unlock after a BIOS change
2026-10-06 21:39:23 +08:00
Ryan Yin 87e87aad7d fix(agents): perserve dsh data 2026-10-06 17:00:01 +08:00
Ryan Yin 975d2668a8 feat(idols-ai): back up the dsh state directory (~/.dsh) with restic
dsh keeps its session transcripts, workspace storages, and profiles in
DSH_HOME=~/.dsh, which was outside the backup allowlist. Exclude its
.credentials.yaml like the module's other credential paths.
2026-10-05 23:46:04 +08:00
ryan4yin | 二花 6b8c5bfae4 feat(vinput): drop NPU/OpenVINO support (#384)
Vinput's streaming Zipformer models have unbounded dynamic shapes, so
OpenVINO cannot compile them for the GPU or the NPU; the `openvino`
provider and ai's Intel NPU enablement could never run on the NPU (noted
as unverified in WORKAROUNDS WA-017). Remove the backend option, the
daemon's OpenVINO library path, and the intel-npu module enable, and
update the eval test, README and WORKAROUNDS (drop WA-016 and WA-017).

Keep Vinput's stock package and its bundled sherpa-onnx (1.13.4): the
fork-based and nixpkgs-master sherpa overrides existed only to provide the
`openvino` provider, so drop package.nix with them.

Assisted-by: opencode (DeepSeek V4.1 Flash)
2026-10-05 15:42:48 +08:00
ryan4yin | 二花 502d0c85c7 fix(preservation): keep vinput config and models across reboots (#381)
Vinput stores its core config in ~/.config/vinput/config.json and its
downloaded sherpa-onnx models in ~/.local/share/vinput/models/. Neither
was bind-mounted from /persistent on the tmpfs-root hosts, so a reboot
wiped both and vinput-daemon started with "Local ASR provider model is
not configured". Persist both directories and document the one-shot CLI
model setup in the vinput README.
2026-10-05 00:18:01 +08:00
ryan4yin | 二花 8f0961d9c2 fix(desktop): tie ai's link DNS takeover to mihomo's lifecycle (#380)
* fix(desktop): tie ai's link DNS takeover to mihomo's lifecycle

sing-tun normally points systemd-resolved at the TUN DNS by running
resolvectl (dns/domain/default-route). Under nixpkgs' hardened
DynamicUser mihomo unit that hand-off fails silently: the unit's
RestrictAddressFamilies blocks resolvectl's D-Bus socket and resolve1
requires privileged authorization, so resolved keeps querying the
physical-link DNS, fake-ip is bypassed, and domain rules degrade to
MATCH.

Do the hand-off as '+' ExecStartPost/ExecStopPost hooks instead: '+'
commands run outside the sandbox (systemd.exec: address-family
restrictions do not apply to them) and ExecStopPost reverts even on
crashes, so a dead mihomo leaves DNS usable.

Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>

* docs(mihomo): record sing-tun's resolvectl hand-off as the leak cause

Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>

* refactor(desktop): keep mihomo only on ai and shoukei

ruby/kana never ran the core (service inactive live), and their
resolved + TUN combo would reintroduce the resolver leak once enabled.
Drop the enable and the explicit module import from computer-use; the
eval test now pins mihomo=false for both guests.

---------

Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
2026-10-05 00:07:32 +08:00
ryan4yin | 二花 bbfe1fa53b feat(desktop): replace Clash Verge with native mihomo on all NixOS hosts (#379) 2026-10-04 21:04:41 +08:00
Ryan Yin a695a3ce3d feat(vinput): add NPU speech input on ai 2026-10-04 19:51:22 +08:00
ryan4yin | 二花 fdafca930b feat(agents): add dsh from nur-packages (#378) 2026-10-04 17:00:17 +08:00
Ryan Yin bf193e4082 chore(agents): drop the kimi-code CLI
Kimi Code moved its desktop client into an internal repository in July 2026 (MoonshotAI/kimi-code#1849), so the polished surface is no longer open source; the CLI that stays behind overlaps opencode2 and pi. Removing it drops one more redundant harness.
2026-10-04 12:52:50 +08:00
Ryan Yin 349fe7f5cf chore(agents): drop the oh-my-pi (omp) CLI
oh-my-pi is a feature-complete superset of pi, so its role is already covered: opencode2 for a batteries-included harness, pi for the minimal/local-model one. Keeping it meant maintaining two nearly identical harnesses.
2026-10-04 12:39:08 +08:00
ryan4yin | 二花 2ee0b9c7eb chore(tui): remove standalone herdr, superseded by tuios (#374) 2026-10-04 11:23:16 +08:00
ryan4yin | 二花 3a5b9ec4c6 docs: README audit fixes and de-duplication (#371) 2026-10-03 18:28:58 +08:00
ryan4yin | 二花 6a1b4c699a chore(docs): prune stale TODOs (#370) 2026-10-03 17:50:28 +08:00
Ryan Yin 524770b351 security(shoukei): disable prometheus exporters 2026-10-03 17:21:37 +08:00
ryan4yin | 二花 d056aece28 chore(workarounds): re-evaluate stale hardware/kernel fixes (#360)
* chore(workarounds): re-evaluate stale hardware/kernel fixes

Prune workarounds whose root cause is gone, fix one vendor typo, and
document the ones we keep:

- drop `transparent_hugepage=madvise` on shoryu/shushou/youko: it was tied to
  the old fixed 1G hugetlb pool, and the VMs now use ordinary memory
- shoryu: `hardware.cpu.intel.updateMicrocode` -> `amd` (it is an AMD APU)
- ai: drop `i915.enable_guc=3` (GuC/HuC submission is the default on modern
  i915/Xe)
- kernel-hardening: name the Dirty Frag CVEs (CVE-2026-43284 / CVE-2026-43500)
  and the re-evaluation condition
- document re-eval conditions for the Insta360 resume fix, the darwin
  auto-optimise-store workaround, and the youko USB-SATA bridge mitigations

* chore(workarounds): keep the Dirty Frag module blacklist as defence in depth

* chore(ai): correct the i915.enable_guc rationale (verified against the kernel)

* chore(darwin): drop the auto-optimise-store workaround (nix#7273 fixed)
2026-10-03 11:25:34 +08:00
ryan4yin | 二花 dc3ec8c159 feat(storage): relax ai NVMe workarounds and add drive SMART monitoring (#359)
* fix(ai): relax NVMe power-management workarounds and enable smartd

Logs show no NVMe errors, so allow APST again (capped at 100 ms), restore
the default io_timeout, and drop pcie_aspm=off. Add smartd for drive
health visibility.

* feat(storage): add drive/SSD SMART monitoring on physical hosts

node_exporter's nvme collector only exposes node_nvme_info and namespace
capacity, not SMART health, so SSD wear/lifespan and drive temperature were
not observable anywhere.

- enable services.prometheus.exporters.smartctl on physical hosts, gated on
  modules.hardwareTools (off on QEMU/MicroVM guests), bound to the host's LAN
  address on port 9633
- firewall: drop 9633 from anything but the monitoring host
- youko: scrape smartctl-<host> for the physical hosts
- alert rules (storage.yml): SMART health, NVMe wear, available spare, media
  errors, critical warning, and drive temperature (warn 70 / crit 80 C)
- Grafana dashboard "Drive SMART (smartctl)"
2026-10-03 11:13:46 +08:00
ryan4yin | 二花 2bacb5c138 refactor: drop redundant package declarations (#353)
Remove no-op overrides and packages already provided elsewhere in the
same profile.
2026-10-03 10:11:30 +08:00
Ryan Yin d9208f492b fix(preservation): keep logrotate state in a persisted directory 2026-10-03 09:42:14 +08:00
Ryan Yin efdf153f0a fix(shoukei): stop latched nscd failures on resolvconf restarts 2026-10-03 09:42:13 +08:00
ryan4yin | 二花 9254b63a18 chore(idols-ai): move tmp state off RAM onto btrfs and harden tmp mounts (#348)
* feat(idols-ai): persist /var/tmp and small state dirs, wipe /var/tmp at boot

Move the strata model staging area (~3G) off the root tmpfs onto the
persistent btrfs volume to free RAM, preserve btrbk/cups/logrotate
state, and wipe /var/tmp at boot so it only accumulates between
reboots.

* chore(idols-ai): add nosuid,nodev to root tmpfs and tmp btrfs mounts

Hardens /, /tmp and /snapshots; safe because /dev, /run, /nix and
/run/setuid-programs are separate mounts, so nothing there needs suid
bits or device nodes.

* chore(idols-ai): add noatime to the @tmp btrfs mount

Temp files do not need access times; matches the @nix subvolume style
and saves btrfs metadata writes.

* chore(idols-ai): harden the /var/tmp bind mount with nosuid,nodev

World-writable staging data should not honor suid bits or device
nodes, same as /tmp and /snapshots.

* fix(idols-ai): preserve logrotate state as a file, fix persisted dir ownership

- /var/lib/logrotate.status is a regular file, not a directory: move it from
  preservation directories to files. A missing state entry makes logrotate
  treat every log as freshly rotated, so it must survive reboots.
- keep /var/lib/btrbk owned by btrbk:btrbk 0750 so preservation does not
  override the btrbk module's rule with root:root.
- set /var/tmp mode 1777 explicitly instead of relying on systemd's tmp.conf
  ordering.
- rewrite comments to describe the paths generically.

* chore(idols-ai): cap journal, wipe coredumps at boot, persist upower/noctalia

- services.journald.settings.Journal: SystemMaxUse=10G, RuntimeMaxUse=256M,
  matching the cap already set on idols-kana/idols-ruby.
- wipe /var/lib/systemd/coredump at boot: it lives in the persisted
  /var/lib/systemd and crash dumps can contain secrets copied from memory.
- persist /var/lib/upower (device battery history) and
  ~/.local/state/noctalia (shell settings, notification/clipboard history).
2026-10-02 21:58:21 +08:00
Ryan Yin a26ab3be63 chore(disko): grow swapfile from 20G to 96G
Running strata with qwen3.8-flash-next keeps ~53G of model weights
resident, leaving little RAM for the desktop side and pushing ~13G of
anonymous pages into the 20G swapfile. With swap that full, a burst
risks OOM-killing processes. Disk space is not a constraint (1.9T
btrfs, ~1.4T free), so grow the swapfile to the classic 1x-RAM size.

Keep zram off and zswap at 10%: zram's compressed pool would steal page
cache that the mmap'd weights rely on.
2026-10-02 19:45:08 +08:00
Ryan Yin ef61c458e6 chore(go-musicfox): declare config.toml and reset it on rebuild
Manage ~/.config/go-musicfox/config.toml declaratively: the upstream default plus the mpv engine, ~/Music download dir, 2 GiB cache and cover art.

The app rewrites the file with a temp file plus rename, which would replace a plain symlink, so use force = true like fcitx5's profile: the repo copy is restored on every rebuild and runtime edits are ephemeral. Runtime data and state stay on the persistent volume.
2026-10-02 14:39:19 +08:00
Ryan Yin 25f059a49b chore: add go-musicfox 2026-10-02 14:12:05 +08:00
ryan4yin | 二花 969a20ac7b docs: refresh repository images (#343) 2026-10-02 12:30:53 +08:00
ryan4yin | 二花 3a0afe5499 chore(youko): drop stale KubeVirt references (#341)
KubeVirt is gone: remove its Grafana dashboard and the commented
Homepage service entry.
2026-10-02 11:16:45 +08:00
ryan4yin | 二花 2b0f01d99c chore(agents): workspace layout, related repositories, and ~/src checkouts (#340)
* docs(skills): point nix-config-secrets at the private repo under ~/codes

Record where the private nix-secrets checkout lives and that agents should
do the mechanical secret work themselves, handing back only the
interactive/sudo agenix operations and pushes.

* docs(agents): add related repositories to nix-config

* chore(agents): adopt ~/src for source checkouts

Move third-party/source checkouts from ~/codes/src to ~/src so the home
layout reads as three siblings: ~/codes (personal), ~/work (work),
~/src (checkouts). Preserve and back up the new path, and update the
global rule and the two nixpkgs skill references.

* docs(agents): tighten the code layout rule

* docs(agents): fold the work-confidentiality note into the layout line

* docs(agents): base the confidentiality rule on repo visibility

* docs(agents): drop the stale KubeVirt mention
2026-10-02 11:16:24 +08:00
ryan4yin | 二花 68ea04f173 fix(docs): correct stale and mismatched documentation (#338) 2026-10-02 10:21:55 +08:00
ryan4yin | 二花 b2ec56db66 docs(skills): add repo-scoped task skills and give each fact one home (#337) 2026-10-02 10:13:03 +08:00
ryan4yin | 二花 d690f45183 feat: add CanoKey hardware security key support (#334) 2026-10-01 20:50:13 +08:00
Ryan Yin 14db49660b chore(preservation): drop persist entries for removed apps 2026-10-01 20:10:08 +08:00
ryan4yin | 二花 ab2fd0fee9 feat: add plocate filesystem search and cap podman log size (#333) 2026-10-01 20:00:53 +08:00
ryan4yin | 二花 8f9fdda2bb refactor: trim unused virtualisation config and packages (#332) 2026-10-01 19:42:20 +08:00
Ryan Yin 01ed0aa52f chore(agents): upgrade to opencode v2 2026-10-01 09:39:05 +08:00
ryan4yin | 二花 9ab5eca60c feat(monitoring): GPU-aware btop variants and GPU metrics (#330) 2026-09-30 00:16:50 +08:00
Ryan Yin c7d8db2bf3 fix(k8s): rebalance k3s-test-1 VMs after the youko/shushou swap 2026-09-29 21:46:09 +08:00
Ryan Yin 2e5f539a47 fix(hosts): move UPS/nut metrics to youko after the swap 2026-09-29 21:36:47 +08:00