Commit Graph
681 Commits
Author SHA1 Message Date
ryan4yin | 二花 d6450d48a2 feat(remote-builders): build ai and shoukei on aquamarine (#467)
Aquamarine builds for ai and shoukei across the three platforms, with both
daemons logging in as root. They authenticate with their passphrase-free SSH
host keys, and ssh_known_hosts now carries each builder's address.

Assisted-by: DeepSeek Harness 0.2.0-rc.2 (deepseek-flash)
2026-10-17 00:49:22 +08:00
ryan4yin | 二花 3d38ff7433 fix(aquamarine): persist the user's cache directory (#466)
On a tmpfs root the flake/fetch cache is empty at every boot, so
`nix run <flake-ref>` re-downloads its input and fails until DNS is up: a reboot
test showed llama-swap's first start failing on that. Other caches write into
RAM meanwhile.

Assisted-by: DeepSeek Harness (deepseek-flash)
2026-10-16 21:36:00 +08:00
Ryan Yin e42d7adc88 chore(ai,aquamarine): shrink the swapfile to 32G
disko creates the swapfile only when it formats the pool, so the live 96G files
keep their size until they are recreated with btrfs filesystem mkswapfile; that
frees 64G per host.

Assisted-by: DeepSeek Harness 0.2.0-rc.2 (deepseek-flash)
2026-10-16 21:18:33 +08:00
Ryan Yin 789b230780 chore: drop conda from the Linux and macOS profiles
Assisted-by: DeepSeek Harness 0.2.0-rc.2 (deepseek-flash)
2026-10-16 21:18:33 +08:00
ryan4yin | 二花 afa7a4c72e fix(aquamarine): persist the user's unit directories (#464)
Assisted-by: DeepSeek Harness (deepseek-flash)
2026-10-16 21:08:41 +08:00
ryan4yin | 二花 a0ff9f7fd7 fix(ssh,ai): keep connections to idle LAN peers fast (#463)
* fix(ssh): retry and multiplex LAN connections
* fix(ai): shrink the neighbour-probe fallback to ~1s

Assisted-by: DeepSeek Harness 0.2.0-rc.2 (deepseek-flash)
2026-10-16 20:07:19 +08:00
ryan4yin | 二花 3266aad4a2 fix(power): drop TuneD for native CPU scaling (#461) 2026-10-16 16:45:08 +08:00
ryan4yin | 二花 b4b293ebfa fix(power): pin the tuned profile per host, cap the VM hosts (#458)
Assisted-by: DeepSeek Harness (deepseek-flash)
2026-10-16 00:42:48 +08:00
ryan4yin | 二花 65f1c8e306 fix(aquamarine): load the power-cap unit's PATH at the service level (#459)
systemd ignores `path` inside [Service], so the unit ran with the default PATH
instead of the coreutils one it asks for.

Assisted-by: DeepSeek Harness (deepseek-flash)
2026-10-16 00:41:59 +08:00
ryan4yin | 二花 e6deb4b564 feat(shoukei): cap battery charging at 90% (#457)
Assisted-by: DeepSeek Harness (deepseek-flash)
2026-10-16 00:37:06 +08:00
ryan4yin | 二花 bebe5bcf42 feat(ai,aquamarine): expose board fans and temperatures via SuperIO (#456)
* feat(ai): load nct6683 for the board's NCT6687D SuperIO
* feat(aquamarine): load nct6775 for the board's NCT6798D SuperIO

Assisted-by: DeepSeek Harness (deepseek-flash)
2026-10-15 23:58:23 +08:00
ryan4yin | 二花 32b4605ab6 feat(aquamarine): cap each R9700 power limit at 250 W #453 2026-10-15 23:27:08 +08:00
Ryan Yin 47dabbf55e docs(ai): drop the tracker links from the monitor and ARGB comments
Assisted-by: DeepSeek Harness (deepseek-flash)
2026-10-15 22:29:16 +08:00
Ryan Yin 7eb5ac5017 feat(ai): add OpenRGB control for the board ARGB
Enabling the module runs `openrgb --server` at boot and installs its udev rules,
so the GUI and the CLI reach the MSI controller without root.

Assisted-by: DeepSeek Harness (deepseek-flash)
2026-10-15 22:25:24 +08:00
Ryan Yin b3700b1d9f feat(ai): control external monitors over DDC/CI
The module loads i2c-dev and grants /dev/i2c-* to the i2c group and locally
logged-in users, so ddcutil no longer needs root.

Assisted-by: DeepSeek Harness (deepseek-flash)
2026-10-15 22:25:24 +08:00
ryan4yin | 二花 e8819de40c feat(vinput): own the config here and read keys from the environment (#445)
* feat(vinput): read provider keys from an EnvironmentFile
* feat(vinput): manage the config from this repository
2026-10-15 13:54:20 +08:00
ryan4yin | 二花 0034b5f131 feat(aquamarine): enable user lingering (#444)
Assisted-by: DeepSeek Harness (gpt-6-sol)
2026-10-15 13:14:17 +08:00
ryan4yin | 二花 bb89987640 fix(mihomo): use fake-IP for gateway IPv6 (#442) 2026-10-14 23:50:40 +08:00
Ryan Yin 55b7741108 refactor(ai): return ai to the MSI MAG B760M MORTAR and the RTX 4090
The second NVMe and its LUKS volume leave with the Z890 board, so the data-disk disko module is deleted; `ai` must activate this change before the disk is pulled, or its initrd blocks on the missing volume.

Assisted-by: DeepSeek Harness (deepseek-flash)
2026-10-14 20:15:01 +08:00
Ryan Yin 72a7a2ae88 feat(aquamarine): revive aquamarine as a two-GPU inference host
The Z890 board and its root SSD move here from `ai`, and the retired VM's name and 192.168.5.101 are reused.

Assisted-by: DeepSeek Harness (deepseek-flash)
2026-10-14 20:15:01 +08:00
Ryan Yin b24f30c38c fix(shoukei): route DNS through the mihomo TUN
Assisted-by: DeepSeek Harness (gpt-6-astra)
2026-10-14 18:34:48 +08:00
ryan4yin | 二花 9c38819517 docs: review and correct comments and documentation (#436) 2026-10-12 10:02:38 +08:00
ryan4yin | 二花 5c5c11d646 docs(idols-ai): record the mixed-memory freeze and current state (#433)
Assisted-by: DeepSeek Harness (gpt-6.1-sol, deepseek-flash)
2026-10-11 20:13:06 +08:00
ryan4yin | 二花 2f6f2b6926 fix(monitoring): restore youko metrics, alerts and VM-host deploys (#431)
* flake.lock: Update

nixpkgs only. The new revision builds VictoriaMetrics with cgo
(CGO_ENABLED=1); its pure-Go zstd decoder rejected a stored values block, so
every small-part merge panicked on youko.

* fix(colmena): address the VM hosts by name, not by their IP

`~/.ssh/config` pins the passphrase-protected builder key with IdentitiesOnly
for 192.168.*, so IP-addressed ssh cannot authenticate without a terminal. The
name resolves through the aliases in /etc/ssh/ssh_config.

* fix(monitoring): repair two alert rules VictoriaMetrics rejects

`on(uid)` matches on a label kube-state-metrics no longer emits, and
`!= ON(__name__, instance)` treats every pg_settings series as a duplicate.
They now join on (namespace, pod) and compare with `changes(...[5m])`.

* fix(monitoring): keep disk latency alerts off mechanical disks

Both rules averaged per-op latency with no device or sample guard, so the 4 TB
USB HDD volumes paged constantly (>100 ms/write in 26% of the last 30 days).
They now require a non-rotational physical device and >=5 ops/s.

Assisted-by: DeepSeek Harness 0.2.0-rc.2 (deepseek-flash)
2026-10-11 18:16:04 +08:00
ryan4yin | 二花 3e66bbf24f feat(hosts): add WiVRn CLI streaming for Quest 3 #392 2026-10-10 22:49:48 +08:00
Ryan Yin 0041afc2e7 fix(hosts): drop the legacy ctx7 preservation entry
The ctx7 login still lives in the old directory; activation without moving it loses the login.

Assisted-by: DeepSeek Harness (qwen3.8-flash-iq3s-text)
2026-10-10 19:58:33 +08:00
Ryan Yin 7b21ea5adb feat(agents): rewrite find-docs onto the nixpkgs ctx7 CLI
Assisted-by: DeepSeek Harness (claude-opus-5-5, deepseek-flash, qwen3.8-flash-next iq3)
2026-10-10 19:49:44 +08:00
ryan4yin | 二花 b52e242bb1 Merge pull request #425 from ryan4yin/docs/agents-rule-gaps
fix: close rule gaps behind repeated corrections and sync stale docs and recipes
2026-10-10 08:50:13 +08:00
Ryan Yin 8843df1508 docs: correct stale claims found while reviewing the skills
No host defines a *-nixos-tests output; the pod firewall drops 5900-5903
from pod sources; the mihomo core keeps geodata and cache in its
StateDirectory; WA-020 was used twice and the cited WA-028 had no row.
2026-10-10 08:42:38 +08:00
Ryan Yin 60b80e64c8 fix(microvm): restart the guest unit on the physical host to deploy
microvm.nix's sshSwitch step runs nix-store --load-db against the guest's
read-only virtiofs /nix/store and dies there, so the recipe installs the
runner on the physical host and restarts microvm@<guest> instead.
2026-10-10 08:42:38 +08:00
Ryan Yin a2cfa5549c fix(skills): correct stale guidance and widen task triggers 2026-10-10 08:42:38 +08:00
Ryan Yin 3b2c328fd2 docs(idols-ai): mark the mixed-memory settings as daily-use verified 2026-10-10 07:19:45 +08:00
ryan4yin | 二花 0e12440148 refactor(secrets): move nushell-secrets.nu out of agenix (#423) 2026-10-09 22:40:50 +08:00
ryan4yin | 二花 63b7183381 feat(mihomo): keep unproxiable ranges out of TUN, fix the startup DNS race (#419)
* feat(mihomo): keep ranges that can never be proxied out of TUN

The list lives in policy.yaml as tun_exclude_address; sources.yaml may override
it. 198.18.0.0/15 stays in TUN: that is the fake-ip range itself.

* feat(mihomo): match Valve's content servers by ASN

The 20 hand-copied prefixes had drifted from what AS32590 announces. The rule
type needs GeoLite2-ASN.mmdb, so geox-url.asn points at the same mirror.

* feat(mihomo): render the gateway config from the same sources

The hand-kept copy had drifted: a manual sync carried the desktop's secret onto
the gateway and dropped external-ui-url. --redact yields a shareable template.

* fix(mihomo): open the link DNS takeover only once the core serves it

The core opens :1053 after parsing rules, and rule parsing is where missing
geodata gets fetched. Flipping at fork time made that fetch SERVFAIL, the config
fatal, and the unit unstartable.

* fix(justfile): keep local endpoints out of the agent proxy

With only HTTP_PROXY set, a request to this host's own llama-swap endpoint went
to mihomo. Node matches exact hosts only, so the literals carry it; the CIDRs
serve curl and Go.

* feat(justfile): seed mihomo geodata into the service state dir

So a cold start never needs the download that the DNS takeover made unstartable.
2026-10-09 12:31:59 +08:00
Ryan Yin fe302fa950 docs(idols-ai): record overnight mixed-memory voltage tests 2026-10-09 07:44:51 +08:00
Ryan Yin 841214c325 docs(idols-ai): summarize mixed-memory voltage tests
Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
2026-10-09 00:30:15 +08:00
ryan4yin | 二花 48ac4f76bf fix(tuios): keep sessions across reboots and split them by purpose (#416) 2026-10-08 20:29:52 +08:00
ryan4yin | 二花 9e2633f496 feat(gaming): generic umu game installer, skill, and desktop entries (#411) 2026-10-08 14:59:11 +08:00
ryan4yin | 二花 835b7f6df1 feat(shoukei): pin home Wi-Fi to a static IPv4 & adjust networking.nix (#407) 2026-10-07 21:27:34 +08:00
Ryan Yin 124bcd8490 docs: drop the stale "just test exit code" caveat
`just test` has enforced the suite result since #350: the recipe raises unless
`evalTests` is `true`, so a zero exit already means pass. Update the places that
still described the old raw-`nix eval` behaviour: AGENTS.md, README.md,
SECURITY.md, outputs/README.md, and the nix-config-debug / nix-config-new-host /
nix-config-update skills.

The exit-0 caveat is kept only where the raw `nix eval .#evalTests` command is
documented, because that command genuinely exits 0 even on `false`.
2026-10-07 19:54:32 +08:00
ryan4yin | 二花 6f136b50e4 fix(shoukei): mitigate brcmfmac wifi auth failures, document duplicate-profile bug (#406)
BCM4378/brcmfmac intermittently fails Wi-Fi association (ASSOC-REJECT /
authentication timeout) on this laptop. NetworkManager misreads the failure
as a bad key and re-prompts; whichever client reconnects (Noctalia, nmtui,
nmcli) then saves the answer as a duplicate "<SSID> 1" profile. The problem
predates the Noctalia v5 shell and is driven by the driver, not the shell.

- disable Wi-Fi powersave on shoukei, a common source of brcmfmac auth timeouts
- WORKAROUNDS.md WA-024: the powersave mitigation and its removal condition
- WORKAROUNDS.md WA-025: the pending upstream brcmfmac port_authorized fix
- document how to recover a wedged supplicant without rebooting, and that
  deleting an active NM profile is what turned a reconnect into a dead link

Refs: https://patchew.org/linux/20260915-brcmfmac-4way-handshake-offload-authenticated-event-v2-1-fbb9cc7ac177@jannau.net/
2026-10-07 19:52:57 +08:00
Ryan Yin 3e4e093869 feat(shoukei): run x86/x86_64 apps and games via muvm+FEX
Apple Silicon's 16K pages mean FEX cannot run natively and qemu-user cannot
run complex x86_64 binaries. Wire the maintained rowanG077/nix-x86-on-aarch64
runtime (FEX inside a 4K muvm microVM) into shoukei so games, Steam, Wine and
other x86/x86_64 programs work, and let the module own the x86 ELF handlers
instead of qemu-user.

The input is pinned to our fork (ryan4yin/nix-x86-on-aarch64) at the reviewed
commit. Dropping boot.binfmt.emulatedSystems also removes x86_64 from
nix.settings.extra-platforms, so the qemu-user/uncached-x86-build gap is
recorded as WA-022.

- flake.nix: add nix-x86-on-aarch64 input (WA-021)
- hosts/12kingdoms-shoukei/game/: programs.x86-on-arm + steam/wine, 6 GiB VM
- hardware-configuration.nix: drop emulatedSystems/preferStaticEmulators
- apple-silicon.nix: drop the now-redundant box64
- docs: WA-012 resolved; WA-021 and WA-022 added; host README updated
2026-10-07 15:55:55 +08:00
ryan4yin | 二花 f5a111c638 feat(backup): hourly read-only snapshots and per-backup integrity checks (#402)
* feat(restic): verify the repository after each backup

* feat(youko): snapshot the restic store hourly

* docs(backup): document the snapshot and check layers
2026-10-07 15:26:57 +08:00
ryan4yin | 二花 3694961e74 fix(postgres,darwin): close the loopback superuser path and scope agenix ownership (#401)
* fix(postgres): require a password for superuser over TCP

* test: assert the postgres loopback auth rules

* fix(darwin): keep root-only agenix secrets root-only

* docs(security): record the local auth boundaries and retire WA-015
2026-10-07 15:22:03 +08:00
ryan4yin | 二花 3acb5d4ee5 fix(nfs,libvirt): restrict the golden export and bind VNC to loopback (#400)
* fix(nfs): restrict the golden export to its real clients

* fix(libvirt): bind VNC consoles to loopback

* fix(firewall): drop VNC ports on every host

* docs(security): record the NFS and VNC boundaries
2026-10-07 15:10:52 +08:00
ryan4yin | 二花 5f6a1dadf4 docs: fix stale claims and AI-flavored prose (#397) 2026-10-07 14:48:36 +08:00
Ryan Yin fabccbafda fix(monitoring): ignore openapi discovery in apiserver latency alert
KubernetesApiServerLatency fired continuously on the p99 of
apiserver_request_duration_seconds_bucket. Every firing series was
verb="GET", subresource="openapi/v3" (kubelet, kube-controller-manager,
kube-scheduler, apiserver) at ~8s, while real verbs (POST/LIST/DELETE)
stay around 0.2s.

openapi/v3 is a streaming discovery endpoint, like WATCH, so exclude
subresource=~"^openapi/" from the rule.
2026-10-07 13:21:43 +08:00
ryan4yin | 二花 76c4566e9b fix(k8s): pass inputs to the MicroVM guests (#396)
Host builds have been broken since #375 - function called without argument inputs
2026-10-07 13:20:08 +08:00
ryan4yin | 二花 95b849420e feat(gaming): declarative Proton and dGPU game rendering (#393) 2026-10-07 13:14:42 +08:00
ryan4yin | 二花 90bbe8bc4f fix(monitoring): stop recurring false alerts (#395)
The Telegram channel was dominated by alerts that can never be actionable:

- DriveTemperatureHigh/Critical did not filter temperature_type, so
  max(smartctl_device_temperature) picked the drive's rated
  op_limit_max/critical_limit_max instead of the actual reading. ai reported
  95 C and youko 130 C while the drives actually ran at 40-48 C.
- HostPhysicalComponentTooHot excluded label!="tctl", but the AMD k10temp
  hwmon sensor labels itself Tctl, so the CPU control temperature (up to
  ~100 C under load) tripped the alert on every AMD host.
- MicroVM guests share the host's read-only /nix/store, so their
  nix-gc.service always failed and tripped HostSystemdServiceCrashed on all
  six k3s guests. Disable automatic GC in the guest; the host's nix.gc
  reclaims the shared store.

The remaining k3s alert noise comes from workloads running on the masters
that belong on the workers; that is fixed in k8s-gitops, not by resizing
the control-plane VMs.
2026-10-07 12:51:30 +08:00