The lock bump moves the input from 47c813c to b8e30ec. The guide now says which Nushell the bundle builds against and that its pin warns on divergence, names direct script execution as the replacement for a refused shell handoff, and uses a Nushell stderr redirect in the verification snippet.
Assisted-by: DeepSeek Harness (deepseek-flash)
* chore(agents): build dsh-nu from its own flake
NUR now keeps only dsh, so a plugin release needs `nix flake update dsh-nu` instead of a NUR package update.
Loading the output preset replaces the GUI-managed Excluded Apps list, so the
exclusions are declared in the preset. Meeting apps apply their own gain
control, and local media playback is better left unprocessed.
Assisted-by: DeepSeek Harness 0.2.1-rc.2 (deepseek-flash)
EasyEffects 8 keeps its KConfig settings database plus user presets, autoload
profiles and imported models under $HOME, which the tmpfs root drops on
reboot; bind-mount both directories from /persistent.
Assisted-by: DeepSeek Harness 0.2.1-rc.2 (deepseek-flash)
* feat(agents): use the shell's own dialect; allow 5 s retry backoff
Assisted-by: DeepSeek Harness (deepseek-flash)
* feat(dsh): make Nushell the only shell
Assisted-by: DeepSeek Harness (deepseek-flash)
* docs(agents): wait for CI in the background
Pushes to a PR often start multi-minute CI runs, and an agent that holds
the turn open on a foreground watch reports nothing until it ends. Require
the pushed head and running checks to be reported first, the watch to run
in the background, and the result to be collected after independent work
or after ending the turn.
Assisted-by: DeepSeek Harness (deepseek-flash)
* docs(agents): keep the shell-handoff rule platform-neutral
The global rules carry no Windows assumptions: drop the cmd, pwsh, and
wsl handoff examples and keep the POSIX forms plus the dsh refusal.
Assisted-by: DeepSeek Harness (deepseek-flash)
* docs(dsh): keep the shell-swap docs POSIX-only
This repository manages Linux and Darwin hosts, not Windows: drop the
Windows handoff examples and terminal-backend wording from dsh.md. The home
patch also disabled pwsh-sandbox redundantly because dsh-base already
disables that row on POSIX; keep only the active bash-sandbox disable and
verify the composed rows with --dump-config.
Assisted-by: DeepSeek Harness (deepseek-flash)
* docs(agents): keep the global rules runtime-neutral
Global rules are loaded by every agent CLI, so they must not describe one
runtime's behavior. Drop the dsh-specific sentence and wording while keeping
the expectation to prefer Nushell structured pipelines and Python. Runtime
behavior stays documented in home/base/tui/agents/dsh.md.
Assisted-by: DeepSeek Harness (deepseek-flash)
* docs(agents): default shell work to Nushell
Keep the Nushell preference explicit in the runtime-neutral global rules, and
keep runtime behavior in runtime docs. Bash stays a narrow fallback for tasks
Nushell cannot do.
Assisted-by: DeepSeek Harness (deepseek-flash)
* docs(agents): tighten global-rule scope and CI guidance
Keep global rules runtime- and project-neutral, and state that boundary in the
file. Remove the duplicated CI turn-steering text in favour of the Communication
rule, resolve the watch run before watching instead of relying on
gh pr checks --watch alone, name the working-directory parameter generically,
and distinguish a bounded retry backoff from a fixed completion wait.
Assisted-by: DeepSeek Harness (deepseek-flash)
* docs(agents): never estimate a CI run's duration
A CI run has no predictable length, so do not turn "it looks short" into a
reason to wait. The rule is unconditional: report the head and started checks,
start a background watch, and end the turn; collect the result on completion.
The eval scenario now uses an unknown duration.
Assisted-by: DeepSeek Harness (deepseek-flash)
* docs(agents): move maintenance guidance to the README
The runtime/project boundary is guidance for editing the rules, not a rule
agents need loaded every session. The README's Maintaining global rules
section already owns that audience; the loaded rules keep only behavior.
Assisted-by: DeepSeek Harness (deepseek-flash)
* refactor(dsh): render the home patch from a YAML template
Move the home-layer cordis patch into dsh-cordis.patch.yml, substitute the
bundle and Nushell store paths with replaceVars, and link the rendered file
through home.file.source. This module is the only writer of
~/.dsh/cordis.patch.yml, the current host has no such file, and the profile
patch is a separate layer.
Assisted-by: DeepSeek Harness (deepseek-flash)
* refactor(dsh): install the shell plugin through NUR withPlugins
Replace the local runCommand bundle, peer symlink, and home-layer YAML file
with the plugin package and dsh.withPlugins from ryan4yin/nur-packages#43. The
package wrapper adds a generated --patch overlay after the profile layer, so
dsh no longer writes or manages ~/.dsh/cordis.patch.yml. The NUR input is
temporarily pinned to the PR branch and its commit; point it back at main once
the NUR PR merges.
Assisted-by: DeepSeek Harness (deepseek-flash)
* refactor(nix): expose the NUR package set as nurPkgs
Pass one per-system NUR package set through specialArgs instead of repeating
nur-ryan4yin.packages.${pkgs.stdenv.hostPlatform.system} at each call site.
The agents, rime, and idols-ruby modules now use the short binding.
Assisted-by: DeepSeek Harness (deepseek-flash)
* refactor(nix): use nurPkgs in package overlays
The overlays receive specialArgs too, so they can use the per-system NUR
package set instead of rebuilding the long input path from prev. This leaves
only the definition in outputs/default.nix.
Assisted-by: DeepSeek Harness (deepseek-flash)
* refactor(nix): name the per-system NUR set myNurPkgs
Leave room for consuming another NUR repository later, and match the existing
mylib/myvars naming.
Assisted-by: DeepSeek Harness (deepseek-flash)
* chore(dsh): consume the merged NUR package set
nur-ryan4yin now points at main after ryan4yin/nur-packages#43 was squash
merged; the lock records the merge commit. No branch ref remains.
Assisted-by: DeepSeek Harness (deepseek-flash)
* fix(dsh): pass the web profile explicitly
The withPlugins launcher prepends --patch, which historically prevented dsh
from auto-expanding "dsh web" into "--profile web". Pass the profile
explicitly in the Just recipe and the verification commands so they work with
both plain dsh and the wrapper.
Assisted-by: DeepSeek Harness (deepseek-flash)
* chore(dsh): pick up the wrapped-profile fix
Update nur-ryan4yin to the squash merge that keeps an implicit profile name
first, so "dsh web" and "just dsh-web" work through the wrapper again.
Assisted-by: DeepSeek Harness (deepseek-flash)
* chore(dsh): pick up the plugin peer-link fix
Update nur-ryan4yin to the squash merge that links the dsh peer tree at
node_modules/@deepseek-ai instead of nesting it one level deeper. This is the
import failure that stopped nushell-executor from activating after deploy.
Assisted-by: DeepSeek Harness (deepseek-flash)
* feat(aquamarine): make the iGPU usable for media and OpenVINO
The R9700s stay reserved for inference: VA-API/QSV transcoding moves to the iGPU media
engine, and `openvino-gpu` works around ROCm's OpenCL loader shadowing the plugin's
`ocl-icd` (WA-038).
Assisted-by: DeepSeek Harness 0.2.1-rc.2 (deepseek-v4.1-flash)
* docs(aquamarine): document OpenVINO on the iGPU
How to run models on the iGPU: the `openvino-gpu` wrapper, the encoder staticization Whisper needs
on `GPU`, compile/steady timings and memory, and the language and cache caveats.
Assisted-by: DeepSeek Harness 0.2.1-rc.2 (deepseek-v4.1-flash)
The goal is speech recognition on the NPU for vinput, and sherpa-onnx cannot
deliver it: every model it loads keeps a dynamic time axis, which the NPU
compiler rejects outright. Qwen3-ASR and SenseVoice both core-dumped in
vpux-compiler ("Got non broadcastable dimensions pair"), and reshaping the ONNX
inputs does not help because the dynamic dimensions sit inside the graph
(WA-037). Whisper is the only family OpenVINO publishes fully static IRs for,
and its fixed 30 s window is exactly what the compiler accepts.
The host now runs an OpenAI-compatible /v1/audio/transcriptions service backed
by whisper-large-v3-turbo int8 on the NPU: 1.5 s for 11 s of audio against
2.1 s on CPU, with the compiled graph cached in /var/cache so only a cold cache
pays the ~60 s NPU compile. vinput on the desktop sends raw PCM to it through a
small provider script.
Only the NPU driver and compiler still come from the unmerged branch in the
`nixpkgs-npu` input (NixOS/nixpkgs#570230), whose nixpkgs also happens to be
the one where openvino-genai 2026.4.0.0 builds at all.
Assisted-by: DeepSeek Harness 0.2.1-rc.2 (deepseek-v4.1-flash)
CPU warns at 90 C, GPU at 95 C and other components at 75 C; per-core probes
are excluded so a full-core compile no longer raises 26 alerts.
Assisted-by: DeepSeek Harness (deepseek-flash)
The Explore link named a datasource that does not exist, and rules without an
alertgroup label rendered an empty value.
Assisted-by: DeepSeek Harness (deepseek-flash)
changes() produced one series per pg_settings_* metric with identical labels,
so VictoriaMetrics rejected the rule with "duplicate output timeseries" whenever
a setting changed.
Assisted-by: DeepSeek Harness (deepseek-flash)
CPUTIN and AUXTIN* read 101 C or saturate at -128/127 C with no sensor wired,
so the alert fired while the CPU package stayed near 53 C.
Assisted-by: DeepSeek Harness (deepseek-flash)
Aquamarine builds for ai and shoukei across the three platforms, with both
daemons logging in as root. They authenticate with their passphrase-free SSH
host keys, and ssh_known_hosts now carries each builder's address.
Assisted-by: DeepSeek Harness 0.2.0-rc.2 (deepseek-flash)
On a tmpfs root the flake/fetch cache is empty at every boot, so
`nix run <flake-ref>` re-downloads its input and fails until DNS is up: a reboot
test showed llama-swap's first start failing on that. Other caches write into
RAM meanwhile.
Assisted-by: DeepSeek Harness (deepseek-flash)
disko creates the swapfile only when it formats the pool, so the live 96G files
keep their size until they are recreated with btrfs filesystem mkswapfile; that
frees 64G per host.
Assisted-by: DeepSeek Harness 0.2.0-rc.2 (deepseek-flash)
* fix(ssh): retry and multiplex LAN connections
* fix(ai): shrink the neighbour-probe fallback to ~1s
Assisted-by: DeepSeek Harness 0.2.0-rc.2 (deepseek-flash)
The tuned package ships tuned-ppd.service with WantedBy=graphical.target, so
activation starts it without /etc/tuned/ppd.conf and switch-to-configuration exits 4.
Assisted-by: DeepSeek Harness (deepseek-flash)
systemd ignores `path` inside [Service], so the unit ran with the default PATH
instead of the coreutils one it asks for.
Assisted-by: DeepSeek Harness (deepseek-flash)
Enabling the module runs `openrgb --server` at boot and installs its udev rules,
so the GUI and the CLI reach the MSI controller without root.
Assisted-by: DeepSeek Harness (deepseek-flash)
The module loads i2c-dev and grants /dev/i2c-* to the i2c group and locally
logged-in users, so ddcutil no longer needs root.
Assisted-by: DeepSeek Harness (deepseek-flash)
The gateway LAN IPv6 transparent path stalls on TCP flows while IPv4 forwarding remains healthy. Withholding AAAA answers keeps LAN clients on the working fake-IPv4 TProxy path until the IPv6 path is fixed
Assisted-by: DeepSeek Harness (gpt-6.1-sol)
A deploy from an aarch64 machine built the x86_64 system locally and died on the first platform-specific derivation; colmena 0.5.0 can build on the target instead.
Assisted-by: DeepSeek Harness (deepseek-flash)
The second NVMe and its LUKS volume leave with the Z890 board, so the data-disk disko module is deleted; `ai` must activate this change before the disk is pulled, or its initrd blocks on the missing volume.
Assisted-by: DeepSeek Harness (deepseek-flash)
The Z890 board and its root SSD move here from `ai`, and the retired VM's name and 192.168.5.101 are reused.
Assisted-by: DeepSeek Harness (deepseek-flash)
--redact produced a "shareable" template but replaced only the secret and the proxy-subscription urls: rule-provider tokens, private domains, the LAN controller address and the routing rules all stayed. Nothing in the repo consumed it.
Assisted-by: DeepSeek Harness (Qwen3.8-Flash-Next-IQ3_S-ista-text)
Adds the review-comments-and-docs skill (1282-word body) and two evaluation suites: a 21-case
measured suite for the skill, and a harness that runs the two concurrency scenarios previously left
unrun in agents/evals/global-rules.md.
The worker MicroVMs ran `home-manager-ryan.service`, which failed on two of them
(`/nix/var/nix/temproots` permission denied, and no `/home/ryan`), so
`HostSystemdServiceCrashed` fired for days. Mirror the masters and extend their
regression test to all six nodes.
Assisted-by: DeepSeek Harness 0.2.0-rc.2 (deepseek-flash)
* flake.lock: Update
nixpkgs only. The new revision builds VictoriaMetrics with cgo
(CGO_ENABLED=1); its pure-Go zstd decoder rejected a stored values block, so
every small-part merge panicked on youko.
* fix(colmena): address the VM hosts by name, not by their IP
`~/.ssh/config` pins the passphrase-protected builder key with IdentitiesOnly
for 192.168.*, so IP-addressed ssh cannot authenticate without a terminal. The
name resolves through the aliases in /etc/ssh/ssh_config.
* fix(monitoring): repair two alert rules VictoriaMetrics rejects
`on(uid)` matches on a label kube-state-metrics no longer emits, and
`!= ON(__name__, instance)` treats every pg_settings series as a duplicate.
They now join on (namespace, pod) and compare with `changes(...[5m])`.
* fix(monitoring): keep disk latency alerts off mechanical disks
Both rules averaged per-op latency with no device or sample guard, so the 4 TB
USB HDD volumes paged constantly (>100 ms/write in 26% of the last 30 days).
They now require a non-rotational physical device and >=5 ops/s.
Assisted-by: DeepSeek Harness 0.2.0-rc.2 (deepseek-flash)
Without it mihomo skips the TUN's v6 half when no NIC has an IPv6 address at startup, yet still installs its strict-route v6 blackhole, which locks the host out of IPv6.
Assisted-by: DeepSeek Harness (qwen3.8-flash-iq3s-text)
* docs(agents): state the boundaries the rules only implied
* docs(agents): grow the scenario tables and record the method
* docs(agents): record the evaluation rounds and their size cost
* docs(agents): require disclosure of AI-assisted contributions
* docs(agents): correct safety boundaries and eval records
Assisted-by: DeepSeek Harness (qwen3.8-flash-next iq3, gpt-6-luna & gpt-6.1-sol)
No host defines a *-nixos-tests output; the pod firewall drops 5900-5903
from pod sources; the mihomo core keeps geodata and cache in its
StateDirectory; WA-020 was used twice and the cited WA-028 had no row.
microvm.nix's sshSwitch step runs nix-store --load-db against the guest's
read-only virtiofs /nix/store and dies there, so the recipe installs the
runner on the physical host and restarts microvm@<guest> instead.
Clause-level extensions only, no new sections. Publishing gains modals in both directions: the old sentence read as forbidding pushes to the user's own private repositories while leaving the disclosure direction a preference. The diagnosis scenario moves to Smoke because a cross-cutting rule has no area for the Extended trigger to select, and the guide separates cheap checks from behavioral runs so a modal-only edit does not cost a scenario suite.
Restore the IPv4 pin removed by 1e6702db (feat(mihomo): answer AAAA and keep the tailnet off the TUN). Fake IPv6 SSH connections can authenticate and then hang.
* fix(microvm): pin guest IPv6 identity to a cluster ULA
The ISP retires delegated prefixes, so a guest that registers its WAN prefix
loses it and Cilium never refreshes the node address. Derive the identity from
the IPv4 octets inside fd05:5::/64 and trust that ULA in the guest firewall.
* feat(mihomo): answer AAAA and keep the tailnet off the TUN
fake-ip needs one pool per family, and the family a client picks does not pick the
egress family. Exclude the cluster ULA and the Tailscale ranges from the TUN; the ssh
pin and the WA-028 record cover the fake-AAAA stall.
* feat(mihomo): generate the config from the repo, not a copy in the live dir
`just mihomo-gen` runs generate.nu from the module and validates with the core
the service runs. ~/.config/mihomo keeps only sources.yaml and config.yaml.
* fix(mihomo): keep the fake IPv6 pool out of fc00::/7
The v6 fake pool is a lookup key, not the egress path, so it can be a public range: the
shipped ULA default reads as "local" to browsers and prompts for local-network access on
every site. The gvisor stack override goes with it -- mips is upstream's default, and
nftables rules out the kernel stack here.
* feat(mihomo): sniff on the gateway, not on the desktop
LAN clients using DoH dial the IPs they resolved, and a node may not reach an IP a
foreign resolver handed out. The gateway recovers the name from the handshake; on the
desktop the fake-ip table already names every flow. Its binary is a static release
build, not a package: update it by hand and validate the live config with the new core.
* fix(mihomo): name the Singapore nodes instead of matching 新
The single-char keyword also matched 新西兰, so the Singapore group held a New
Zealand node and url-test flipped the exit country mid-session.
LAN clients using DoH dial the IPs they resolved, and a node may not reach an IP a
foreign resolver handed out. The gateway recovers the name from the handshake; on the
desktop the fake-ip table already names every flow. Its binary is a static release
build, not a package: update it by hand and validate the live config with the new core.
The v6 fake pool is a lookup key, not the egress path, so it can be a public range: the
shipped ULA default reads as "local" to browsers and prompts for local-network access on
every site. The gvisor stack override goes with it -- mips is upstream's default, and
nftables rules out the kernel stack here.
`just mihomo-gen` runs generate.nu from the module and validates with the core
the service runs. ~/.config/mihomo keeps only sources.yaml and config.yaml.
fake-ip needs one pool per family, and the family a client picks does not pick the
egress family. Exclude the cluster ULA and the Tailscale ranges from the TUN; the ssh
pin and the WA-028 record cover the fake-AAAA stall.
The ISP retires delegated prefixes, so a guest that registers its WAN prefix
loses it and Cilium never refreshes the node address. Derive the identity from
the IPv4 octets inside fd05:5::/64 and trust that ULA in the guest firewall.
* feat(mihomo): keep ranges that can never be proxied out of TUN
The list lives in policy.yaml as tun_exclude_address; sources.yaml may override
it. 198.18.0.0/15 stays in TUN: that is the fake-ip range itself.
* feat(mihomo): match Valve's content servers by ASN
The 20 hand-copied prefixes had drifted from what AS32590 announces. The rule
type needs GeoLite2-ASN.mmdb, so geox-url.asn points at the same mirror.
* feat(mihomo): render the gateway config from the same sources
The hand-kept copy had drifted: a manual sync carried the desktop's secret onto
the gateway and dropped external-ui-url. --redact yields a shareable template.
* fix(mihomo): open the link DNS takeover only once the core serves it
The core opens :1053 after parsing rules, and rule parsing is where missing
geodata gets fetched. Flipping at fork time made that fetch SERVFAIL, the config
fatal, and the unit unstartable.
* fix(justfile): keep local endpoints out of the agent proxy
With only HTTP_PROXY set, a request to this host's own llama-swap endpoint went
to mihomo. Node matches exact hosts only, so the literals carry it; the CIDRs
serve curl and Go.
* feat(justfile): seed mihomo geodata into the service state dir
So a cold start never needs the download that the DNS takeover made unstartable.
The comment trim dropped them. Without a domain rule above the Valve IP ranges,
steamcommunity.com and steampowered.com match those ranges and go DIRECT.
Steam picks download caches from the IP it sees on the CM login. Pin the CM hosts
and content CDN DIRECT, keep the community and store proxied, and add Valve's
AS32590 ranges because Steam dials cached CM IPs without a lookup. Sources in
README.md.
The abort needs havePrivateMountNs == false AND /nix/store still ro:
remountReadOnlyWritable() returns early while the store is writable. Nix
<= 2.34 left it rw until reboot; 2.35 remounts in a private namespace, so
only a freshly booted host hits it. The repro command also uses an open-only
exec 7<..., since cat fails on the read even when the open succeeds.
While here, trim the comments and drop the disabled default_deny/nix
policies plus the explicit killUnconfinedConfinables=false (nixpkgs
default). 82 -> 34 lines.
`just test` has enforced the suite result since #350: the recipe raises unless
`evalTests` is `true`, so a zero exit already means pass. Update the places that
still described the old raw-`nix eval` behaviour: AGENTS.md, README.md,
SECURITY.md, outputs/README.md, and the nix-config-debug / nix-config-new-host /
nix-config-update skills.
The exit-0 caveat is kept only where the raw `nix eval .#evalTests` command is
documented, because that command genuinely exits 0 even on `false`.
BCM4378/brcmfmac intermittently fails Wi-Fi association (ASSOC-REJECT /
authentication timeout) on this laptop. NetworkManager misreads the failure
as a bad key and re-prompts; whichever client reconnects (Noctalia, nmtui,
nmcli) then saves the answer as a duplicate "<SSID> 1" profile. The problem
predates the Noctalia v5 shell and is driven by the driver, not the shell.
- disable Wi-Fi powersave on shoukei, a common source of brcmfmac auth timeouts
- WORKAROUNDS.md WA-024: the powersave mitigation and its removal condition
- WORKAROUNDS.md WA-025: the pending upstream brcmfmac port_authorized fix
- document how to recover a wedged supplicant without rebooting, and that
deleting an active NM profile is what turned a reconnect into a dead link
Refs: https://patchew.org/linux/20260915-brcmfmac-4way-handshake-offload-authenticated-event-v2-1-fbb9cc7ac177@jannau.net/
Nix 2.35's root nix-env -- used by nixos-rebuild to set the system profile --
calls saveMountNamespace(), which opens /proc/self/ns/mnt before remounting
the read-only /nix/store writable. Those magic links have no connected path,
so AppArmor cannot match them against any rule and denies the open with
EACCES even in complain mode. The private mount namespace is then never
created and nixos-rebuild aborts with:
cannot remount "/nix/store" writable: not in a private mount namespace
Adding flags=(attach_disconnected) lets AppArmor re-root those paths on / so
the existing `file /**` rule matches. This also fixes any other sudo'd tool
that needs namespaces (unshare, nsenter, bwrap, ...).
Rolling this out to a host that already has the old profile loaded needs one
deploy with --no-reexec, so the current system's Nix performs nix-env --set
instead of the new system's; after that generation is active, plain
`just niri` works and the Nix version is never rolled back.
Gitee's SSH gateway intermittently accepts the connection and key, then never
answers the initial git-receive-pack and drops the session ~100s later with
"the remote end hung up upon initial contact". The same failure mode shows up
sporadically since 2026-10-02 and spiked on 2026-10-07 (9 of 14 runs failed).
Bound each push attempt with `timeout`, retry up to six times with exponential
backoff, and add ssh connect/keepalive timeouts. Serialize runs (a push to main
and a tag on the same commit used to race for Gitee's per-repo push lock), cap
the job at 30 minutes, and allow manual workflow_dispatch reruns.
* feat(shoukei): run x86/x86_64 apps and games via muvm+FEX
Apple Silicon's 16K pages mean FEX cannot run natively and qemu-user cannot
run complex x86_64 binaries. Wire the maintained rowanG077/nix-x86-on-aarch64
runtime (FEX inside a 4K muvm microVM) into shoukei so games, Steam, Wine and
other x86/x86_64 programs work, and let the module own the x86 ELF handlers
instead of qemu-user.
The input is pinned to our fork (ryan4yin/nix-x86-on-aarch64) at the reviewed
commit. Dropping boot.binfmt.emulatedSystems also removes x86_64 from
nix.settings.extra-platforms, so the qemu-user/uncached-x86-build gap is
recorded as WA-022.
- flake.nix: add nix-x86-on-aarch64 input (WA-021)
- hosts/12kingdoms-shoukei/game/: programs.x86-on-arm + steam/wine, 6 GiB VM
- hardware-configuration.nix: drop emulatedSystems/preferStaticEmulators
- apple-silicon.nix: drop the now-redundant box64
- docs: WA-012 resolved; WA-021 and WA-022 added; host README updated
* refactor(nixpkgs): drop the repo-wide pkgs-x64 package set
Every remaining pkgs-x64 consumer runs only on x86_64-linux hosts, where the
native pkgs already carries the same allowUnfree config
(modules/nixos/base/nix.nix) and overlays (modules/base/overlays.nix), so
replace the references with pkgs and drop the extra nixpkgs instantiation
(which also removes an eval on aarch64-darwin).
media.nix keeps the native Zoom app on x86_64 hosts only, since zoom-us has
no aarch64 Linux build.
Every remaining pkgs-x64 consumer runs only on x86_64-linux hosts, where the
native pkgs already carries the same allowUnfree config
(modules/nixos/base/nix.nix) and overlays (modules/base/overlays.nix), so
replace the references with pkgs and drop the extra nixpkgs instantiation
(which also removes an eval on aarch64-darwin).
media.nix keeps the native Zoom app on x86_64 hosts only, since zoom-us has
no aarch64 Linux build.
Apple Silicon's 16K pages mean FEX cannot run natively and qemu-user cannot
run complex x86_64 binaries. Wire the maintained rowanG077/nix-x86-on-aarch64
runtime (FEX inside a 4K muvm microVM) into shoukei so games, Steam, Wine and
other x86/x86_64 programs work, and let the module own the x86 ELF handlers
instead of qemu-user.
The input is pinned to our fork (ryan4yin/nix-x86-on-aarch64) at the reviewed
commit. Dropping boot.binfmt.emulatedSystems also removes x86_64 from
nix.settings.extra-platforms, so the qemu-user/uncached-x86-build gap is
recorded as WA-022.
- flake.nix: add nix-x86-on-aarch64 input (WA-021)
- hosts/12kingdoms-shoukei/game/: programs.x86-on-arm + steam/wine, 6 GiB VM
- hardware-configuration.nix: drop emulatedSystems/preferStaticEmulators
- apple-silicon.nix: drop the now-redundant box64
- docs: WA-012 resolved; WA-021 and WA-022 added; host README updated
* feat(restic): verify the repository after each backup
* feat(youko): snapshot the restic store hourly
* docs(backup): document the snapshot and check layers
* fix(postgres): require a password for superuser over TCP
* test: assert the postgres loopback auth rules
* fix(darwin): keep root-only agenix secrets root-only
* docs(security): record the local auth boundaries and retire WA-015
* fix(nfs): restrict the golden export to its real clients
* fix(libvirt): bind VNC consoles to loopback
* fix(firewall): drop VNC ports on every host
* docs(security): record the NFS and VNC boundaries
* fix(k3s): deny credential-bearing host ports to pod sources
* test: assert the k3s pod firewall drops sensitive ports
* docs(security): record the pod firewall boundary
KubernetesApiServerLatency fired continuously on the p99 of
apiserver_request_duration_seconds_bucket. Every firing series was
verb="GET", subresource="openapi/v3" (kubelet, kube-controller-manager,
kube-scheduler, apiserver) at ~8s, while real verbs (POST/LIST/DELETE)
stay around 0.2s.
openapi/v3 is a streaming discovery endpoint, like WATCH, so exclude
subresource=~"^openapi/" from the rule.
The Telegram channel was dominated by alerts that can never be actionable:
- DriveTemperatureHigh/Critical did not filter temperature_type, so
max(smartctl_device_temperature) picked the drive's rated
op_limit_max/critical_limit_max instead of the actual reading. ai reported
95 C and youko 130 C while the drives actually ran at 40-48 C.
- HostPhysicalComponentTooHot excluded label!="tctl", but the AMD k10temp
hwmon sensor labels itself Tctl, so the CPU control temperature (up to
~100 C under load) tripped the alert on every AMD host.
- MicroVM guests share the host's read-only /nix/store, so their
nix-gc.service always failed and tripped HostSystemdServiceCrashed on all
six k3s guests. Disable automatic GC in the guest; the host's nix.gc
reclaims the shared store.
The remaining k3s alert noise comes from workloads running on the masters
that belong on the workers; that is fixed in k8s-gitops, not by resizing
the control-plane VMs.
The bring-up runs at the default 1.25 V; afterwards Gear mode is set to
Gear2 and VDD/VDDQ are raised to 1.28 V, changing nothing else. 1.29 V and
up do not train. Also record that the 1.25 V config passed stressapptest
but still hard-froze under a real llama.cpp load, so a passing test is not
proof of stability.
- replace the btop screenshot with the 2026-10-06 annotated capture
(btop + fastfetch + the strata/llama.cpp serving logs); the old
2026-10-01 file is gone, so the README link moves to the new name
- drop the Helix/AI-agents screenshot and its README reference
- fix stale hardware descriptions: the hosts/README.md inventory row and
the idols-ai header comment still said i5-13600KF, and the zswap pool
comment sized its percentages against the old 96G of RAM (now 128G)
- record the hardware history: the platform was rebuilt on 2026-04-27
(PR #257/#258) onto the Colorful CVN Z890 ARK FROZEN + Core Ultra 7
270K Plus, replacing the MSI board and i5-13600KF the host was added
with in 2023-05; the 2x48G kit predates that rebuild, the 2x16G kit
was added 2026-10-06
- make the DIMM bring-up order mandatory and record its failure mode:
with the 48G pair in first, or all four DIMMs from the start, the
board does not POST
- gloss the Chinese BIOS labels in English, marked as translations of
the labels rather than the firmware's own English wording
Left alone on purpose: the 96G swapfile in hosts/idols-ai/disko-fs.nix.
It was grown from 20G to 1x-RAM on 2026-10-02 (a26ab3be), when the host
still had 96G; resizing it again means re-running disko on the machine,
so that stays a separate decision.
Document the 4-DIMM kit mix (2×48G JUHOR + 2×16G GLOWY) on the Colorful
CVN Z890 ARK FROZEN V20:
- hardware (CPU, board, and both kits)
- the install order that trains (16G pair first)
- the only stable settings (XMP off, DDR5-4800 @ 1.25 V)
- the stress-test recipe and results
- the XMP @ 5600 failure and how to recover a board that will not POST
- re-enrolling the LUKS TPM unlock after a BIOS change
`dsh-llm-pi-ai` registers its built-in providers once, so a second instance
in the profile patch fails on mount with `provider "..." is already declared`.
Extend the base `llm-pi-ai` row instead.
dsh rewrites a profile's cordis.patch.yml at runtime, so linking the web
profile put GUI state in the checkout and let runtime writes clobber manual
edits. Link the stable policy as the $DSH_HOME home layer instead: dsh applies
it after every profile and only reads it.
The host-plane compaction-basic row is disabled on the web surface and each
agent preset owns its own copy, so the top-level override was inert. Drop it;
the model contextWindow/maxTokens are the effective knobs.
- add home/base/tui/agents/dsh/cordis.patch.yml, linked from default.nix
- drop the ~/.dsh/profiles/web link and the tracked web/ profile
- keep runtime state and private/local model routes in the profile patch
- update the eval test and README
dsh keeps its session transcripts, workspace storages, and profiles in
DSH_HOME=~/.dsh, which was outside the backup allowlist. Exclude its
.credentials.yaml like the module's other credential paths.
Explicit session action lists replace the built-in defaults, so re-declare all
five actions; lock stays immediate, logout/suspend/reboot/shutdown arm a 10 s
countdown that a second press runs immediately.
QT_QPA_PLATFORMTHEME=gtk3 (from #386) made Qt derive its color scheme
from GTK3, which has no dynamic dark/light signal, so Telegram and other Qt
apps were stuck on the light palette after Noctalia's theme mode changed.
Use the portal theme instead: Qt then follows
org.gnome.desktop.interface color-scheme, which Noctalia writes. The trade-off
is Qt's built-in "Sans Serif" 9pt font, which a font-providing platform theme
would be needed to change.
go-musicfox posts its own desktop notification on every track change and both popups sit at bottom_right, so the media OSD was a second, redundant popup.
* feat(rime): share the Rime config and user dictionary across Linux and macOS
Move the Rime data, preferences, and user dictionary into home/base/gui/rime so fcitx5-rime and Squirrel read the same files.
* feat(vinput): open the command palette with right Ctrl
Pin keyd's right-hand modifiers (rightcontrol/rightshift/rightmeta) to their own keycodes; keyd otherwise folds them into the left keys. Set Vinput's MenuKey to right Ctrl and clear CommandKeys, which would shadow it.
* feat(fcitx5): switch CN/EN with Ctrl+Space and right Shift
Clear AltTriggerKeys, whose Shift_L default toggled on left Shift. Left Shift is now a plain modifier and right Shift switches through the shared Rime config.
Clear AltTriggerKeys, whose Shift_L default toggled on left Shift. Left Shift is now a plain modifier and right Shift switches through the shared Rime config.
Pin keyd's right-hand modifiers (rightcontrol/rightshift/rightmeta) to their own keycodes; keyd otherwise folds them into the left keys. Set Vinput's MenuKey to right Ctrl and clear CommandKeys, which would shadow it.
* fix(desktop): align Qt app fonts with GTK
Qt apps kept their built-in default font ("Sans Serif" 9) while GTK apps use
`gtk-font-name` (Noto Sans 11), so Telegram and other Qt apps rendered
noticeably smaller than GTK apps like Firefox. Set
QT_QPA_PLATFORMTHEME=gtk3; the plugin ships with qtbase, so it needs no
extra package and also gives Qt apps GTK file dialogs.
* chore(desktop): drop unused Xft.dpi from the Wayland GTK profile
Xft.dpi/ *.dpi are X11 X-resource settings. This file is only imported by
the niri (Wayland) hosts, where xwayland-satellite sets its own Xft.dpi
(96 x output scale, currently 144), and the i3 hosts are headless 1080p
computer-use sessions that do not import it. So the setting had no effect
anywhere; remove it instead of keeping a misleading knob.
Vinput's streaming Zipformer models have unbounded dynamic shapes, so
OpenVINO cannot compile them for the GPU or the NPU; the `openvino`
provider and ai's Intel NPU enablement could never run on the NPU (noted
as unverified in WORKAROUNDS WA-017). Remove the backend option, the
daemon's OpenVINO library path, and the intel-npu module enable, and
update the eval test, README and WORKAROUNDS (drop WA-016 and WA-017).
Keep Vinput's stock package and its bundled sherpa-onnx (1.13.4): the
fork-based and nixpkgs-master sherpa overrides existed only to provide the
`openvino` provider, so drop package.nix with them.
Assisted-by: opencode (DeepSeek V4.1 Flash)
metacubexd hardcodes its tab title in the SPA, so dashboards on
different hosts are indistinguishable. Patch the static <title> and
re-apply the hostname from config.js, since Nuxt's useHead resets
document.title on every route change.
The config deliberately stays out of the store, under the home directory, which is a persistent bind mount on tmpfs-root hosts. When the service and the mount are introduced in the same activation, mihomo can start before the mount is up and fail with "Failed to set up credentials: No such file or directory", making the switch report status 4. Add RequiresMountsFor so the mount is brought up and ordered before the service.
Vinput stores its core config in ~/.config/vinput/config.json and its
downloaded sherpa-onnx models in ~/.local/share/vinput/models/. Neither
was bind-mounted from /persistent on the tmpfs-root hosts, so a reboot
wiped both and vinput-daemon started with "Local ASR provider model is
not configured". Persist both directories and document the one-shot CLI
model setup in the vinput README.
* fix(desktop): tie ai's link DNS takeover to mihomo's lifecycle
sing-tun normally points systemd-resolved at the TUN DNS by running
resolvectl (dns/domain/default-route). Under nixpkgs' hardened
DynamicUser mihomo unit that hand-off fails silently: the unit's
RestrictAddressFamilies blocks resolvectl's D-Bus socket and resolve1
requires privileged authorization, so resolved keeps querying the
physical-link DNS, fake-ip is bypassed, and domain rules degrade to
MATCH.
Do the hand-off as '+' ExecStartPost/ExecStopPost hooks instead: '+'
commands run outside the sandbox (systemd.exec: address-family
restrictions do not apply to them) and ExecStopPost reverts even on
crashes, so a dead mihomo leaves DNS usable.
Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
* docs(mihomo): record sing-tun's resolvectl hand-off as the leak cause
Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
* refactor(desktop): keep mihomo only on ai and shoukei
ruby/kana never ran the core (service inactive live), and their
resolved + TUN combo would reintroduce the resolver leak once enabled.
Drop the enable and the explicit module import from computer-use; the
eval test now pins mihomo=false for both guests.
---------
Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
The cherry-picked openvino 2026.4.1 bump (upstream PR #564046, still
unmerged) made onnxruntime 1.27.1 fail its OpenVINO EP MatMulNBits
numerical test, breaking sherpa-onnx and the ai-niri system build.
The fork branch now stays at upstream openvino 2026.4.0.
Restore the rule dropped in 9822dafa: keep GitHub Git remotes on SSH, make sure the Nix-managed ~/.ssh/config is used, and rerun outside the sandbox when it is rejected.
Move the flypy schema data to the rime-data-flypy package in
ryan4yin/nur-packages and keep only the personal user dictionary
(flypy_user.txt) here. The overlay merges the NUR data with that file
and exposes the result as pkgs.rime-data, fcitx5-rime's rimeDataPkgs,
and pkgs.flypy-squirrel.
* refactor: consume computer-use packages from NUR
* fix: expose flake inputs to package overlays
Package overlays are imported from two places with different argument
shapes: outputs/default.nix passes a set containing `inputs`, while
modules/base/overlays.nix passes the NixOS module args, where genSpecialArgs
spreads each flake input as a top-level argument. The new NUR-backed
overlays read `inputs.nur-ryan4yin`, so evaluation failed on the module
path. Also expose the whole flake input set as `inputs` in genSpecialArgs.
* chore: bump nur-ryan4yin input
Update the locked nur-packages revision to the current main (nixpkgs
26.05, unused packages removed).
Kimi Code moved its desktop client into an internal repository in July 2026 (MoonshotAI/kimi-code#1849), so the polished surface is no longer open source; the CLI that stays behind overlaps opencode2 and pi. Removing it drops one more redundant harness.
oh-my-pi is a feature-complete superset of pi, so its role is already covered: opencode2 for a batteries-included harness, pi for the minimal/local-model one. Keeping it meant maintaining two nearly identical harnesses.
node_exporter listened on 0.0.0.0, so access control rested entirely on
the nftables allowlist. Bind node/smartctl exporters to the host static
LAN IPv4 (the same address VictoriaMetrics scrapes), keeping the socket
off tailscale/bridge interfaces even if the firewall regresses.
Hosts without a static LAN address (shoukei, akane) now run no exporters
at all instead of loopback-bound ones nobody scrapes; akane, a UTM VM,
previously ran a loopback smartctl exporter with no physical disks.
LAN-bound exporters also get the boot-race treatment from #372
(network-online ordering plus patient restart); loopback-bound hosts
must not pull in network-online because NM wait-online would stall
their offline boots.
The exporter binds to the host LAN address, which systemd-networkd
assigns during boot. Without ordering, the service starts before the
address exists, `bind: cannot assign requested address` trips the
start limit, and the unit stays failed until restarted manually.
Order the unit after network-online.target, and since wait-online is
best-effort (120s timeout), also retry patiently so the unit self-heals
when the link comes up late.
The optional kernel reboot-visibility change was closed without merging, so remove the planned-reporter wording. Replace it with the manual booted-vs-selected kernel check and update the P1 TODO accordingly.
Add SECURITY.md covering the threat model, trust boundaries,
proportionate kernel/process hardening, the update/reboot routine and
remaining prioritized work. Record the dated read-only audit findings
(listeners, privileged workloads, effective RBAC, secret metadata and
sandbox/SSH boundaries) separately from planned controls and runtime
verification.
Define agent-first diagnosis: scoped root read-only access on homelab
hosts, and normal-user diagnostics on core desktops with no new sudo
grants. Add canonical domain-based k3s API endpoint regression tests,
record the unverified Darwin secret-ownership concern as WA-015, and
correct comments claiming mode 0000 blocks privileged root.
muvm is disabled because it does not build/run on nixos-apple-silicon yet, not because of a permanent hardware limitation; the 16k page size is the reason it is wanted, and the removal condition is muvm building and working.
WORKAROUNDS.md collects temporary workarounds, version pins, carried patches, and known gaps, each with a removal condition, so they can be re-evaluated and rolled back on a schedule. AGENTS.md points to it and states the convention: a change that adds a workaround adds a row in the same PR.
Replace the backported patch with a src/version/vendorHash override that pins prometheus-smartctl-exporter to upstream 6274decc (2026-09-28), which already carries the Describe() fix from PR #329. Simpler to maintain than a patch file and it tracks the upstream change directly; drop the overlay once smartctl_exporter 0.15.0 is packaged in nixpkgs.
smartctl_exporter 0.14.0 derives Describe() from the devices it can read at startup. A device that only becomes readable later (an NVMe whose ACL arrives late, or an HDD waking from standby) then makes Collect() emit metrics with unregistered descriptors, so /metrics returns HTTP 500 and the whole scrape fails, hiding even the healthy disks.
Backport upstream PR #329, which makes Describe() send the full explicit descriptor set. Carry it as an overlay until smartctl_exporter 0.15.0 is released and packaged in nixpkgs.
Refs: https://github.com/prometheus-community/smartctl_exporter/pull/329
test-instrumentation.nix sets serial-getty@ttyS0.enable = false at normal priority to keep ttyS0 free for the test backdoor console, while qemu-guest.nix set it to true at normal priority. Two conflicting normal-priority definitions made `nix flake check` fail while evaluating packages.x86_64-linux.ruby-nixos-tests.
Use lib.mkDefault so the test framework wins in tests while ordinary QEMU guests still enable the serial getty.
* fix(colmena): expose colmenaHive for the new evaluator
nixpkgs now ships the nix-community Colmena rewrite, whose default
evaluator reads an evaluated colmenaHive flake output instead of the raw
colmena hive. Without it every colmena apply recipe failed with
"does not provide attribute colmenaHive".
Add a colmena input pinned to the same v0.5.0 as nixpkgs and expose
colmenaHive = colmena.lib.makeHive <raw hive>, with the raw hive kept as
an internal binding rather than a flake output of its own.
* chore: clean up stale colmena and aquamarine references
- point the Colmena links at nix-community/colmena after the move
- rename the security-aquamarine-metrics test to youko-metrics (it
already tested youko) and update the outputs README
- drop the commented aquamarine remote-builder example
* chore(workarounds): re-evaluate stale hardware/kernel fixes
Prune workarounds whose root cause is gone, fix one vendor typo, and
document the ones we keep:
- drop `transparent_hugepage=madvise` on shoryu/shushou/youko: it was tied to
the old fixed 1G hugetlb pool, and the VMs now use ordinary memory
- shoryu: `hardware.cpu.intel.updateMicrocode` -> `amd` (it is an AMD APU)
- ai: drop `i915.enable_guc=3` (GuC/HuC submission is the default on modern
i915/Xe)
- kernel-hardening: name the Dirty Frag CVEs (CVE-2026-43284 / CVE-2026-43500)
and the re-evaluation condition
- document re-eval conditions for the Insta360 resume fix, the darwin
auto-optimise-store workaround, and the youko USB-SATA bridge mitigations
* chore(workarounds): keep the Dirty Frag module blacklist as defence in depth
* chore(ai): correct the i915.enable_guc rationale (verified against the kernel)
* chore(darwin): drop the auto-optimise-store workaround (nix#7273 fixed)
* fix(ai): relax NVMe power-management workarounds and enable smartd
Logs show no NVMe errors, so allow APST again (capped at 100 ms), restore
the default io_timeout, and drop pcie_aspm=off. Add smartd for drive
health visibility.
* feat(storage): add drive/SSD SMART monitoring on physical hosts
node_exporter's nvme collector only exposes node_nvme_info and namespace
capacity, not SMART health, so SSD wear/lifespan and drive temperature were
not observable anywhere.
- enable services.prometheus.exporters.smartctl on physical hosts, gated on
modules.hardwareTools (off on QEMU/MicroVM guests), bound to the host's LAN
address on port 9633
- firewall: drop 9633 from anything but the monitoring host
- youko: scrape smartctl-<host> for the physical hosts
- alert rules (storage.yml): SMART health, NVMe wear, available spare, media
errors, critical warning, and drive temperature (warn 70 / crit 80 C)
- Grafana dashboard "Drive SMART (smartctl)"
agenix was pinned to a May 2025 revision, so `just up` could never move
it. Move to the 0.18.0 release tag and collapse its duplicate
nix-darwin/home-manager lock nodes via follows.
Remove a dead .gitattributes rule, bump the mirror workflow's checkout,
document the daemon test settings, drop unused JACK, and align the
installer's disko version.
* fix(ci): fail the eval-test run when the suite does not pass
`nix eval .#evalTests` prints the suite result but exits 0 either way, so
both `just test` and the CI workflow reported success even when a test
failed. Compare the raw boolean and fail instead.
* fix(ci): use --json for the boolean eval result
`nix eval --raw` cannot coerce a Boolean to a string, and Just runs each
recipe line as a separate `nu -c`, so the result variable must stay on one
line.
* feat!: replace zellij with tuios and switch to ghostty + kitty
Terminal environment overhaul:
- add the tuios flake input (upstream 0.8.5; nixpkgs has 0.8.1) and link
the checked-in config out-of-store; auto-start tuios in nushell instead
of zellij
- remove zellij entirely: both modules, config, README and nushell
completions
- remove foot and alacritty, which lack the Kitty graphics protocol;
Reuse the already-extracted, libtiff-patched AppImage from the nixpkgs wechat package (`wechat.src`) instead of keeping our own CDN URLs and hashes, which had to be refreshed whenever Tencent replaced the versionless files. Take the package from nixpkgs-master, which carries the newest builds.
- wechat.nix: repacking is needed because nixpkgs' wechat package does not expose its AppImage wrapper args for `.override`, not because wrapAppImage lacks overriding (it forwards args to buildFHSEnv).
- peripherals.nix: drop a link to an already-closed issue and fix a typo.
nixpkgs-blender tracked the nixpkgs master branch, so every `just up` re-bumped it and started a full Blender/CUDA source rebuild even though Blender stayed at 5.2.2 LTS. Pin it to a fixed rev so bumps are deliberate.
The OIDN cmake override is obsolete: nixpkgs#545542 made setupCudaHook add nvcc to CUDAToolkit_ROOT, fixing the CMake 4.3+ split-root failure (nixpkgs#544701). Dropping it also stops Blender's derivation from depending on nixpkgs-stable.
environment.enableAllTerminfo installs every terminal's terminfo, including rxvt-unicode, which no longer builds with GCC 16 (NixOS/nixpkgs#568896). Install just the terminals we use (alacritty, foot, ghostty, kitty) instead. This also makes the aarch64 jasper workaround unnecessary.
16 inputs moved; nixpkgs -> c59305bab206 (nixos-unstable, 2026-10-01), plus the
master/stable/darwin/blender/patched nixpkgs inputs and the module flakes
(home-manager, microvm, nixos-apple-silicon, nix-gaming, nixpak, llm-agents,
rust-overlay, flake-parts, git-hooks, nu_scripts).
Audited each moved revision through the GitHub compare API: no red flags
(substituters, trusted-users, activationScript, pre-commit hooks) in the inputs
examined. home-manager carries the rl-2611 extraConfig/mkForce change (affects
mbsync/lieer/mujmap/notmuch/astroid/grobi; none used here). The home-manager
compare is large, so the audit is not exhaustive and relies on the build.
Validated with `just test` (true), eval-host on the NixOS hosts, and
`just build-host ai-niri`; deployment passed. The build initially failed on
rxvt-unicode-unwrapped (GCC 16 std::lerp regression, nixpkgs#568896), pulled in
by environment.enableAllTerminfo; terminfo is now scoped to the terminals in use.
* feat(idols-ai): persist /var/tmp and small state dirs, wipe /var/tmp at boot
Move the strata model staging area (~3G) off the root tmpfs onto the
persistent btrfs volume to free RAM, preserve btrbk/cups/logrotate
state, and wipe /var/tmp at boot so it only accumulates between
reboots.
* chore(idols-ai): add nosuid,nodev to root tmpfs and tmp btrfs mounts
Hardens /, /tmp and /snapshots; safe because /dev, /run, /nix and
/run/setuid-programs are separate mounts, so nothing there needs suid
bits or device nodes.
* chore(idols-ai): add noatime to the @tmp btrfs mount
Temp files do not need access times; matches the @nix subvolume style
and saves btrfs metadata writes.
* chore(idols-ai): harden the /var/tmp bind mount with nosuid,nodev
World-writable staging data should not honor suid bits or device
nodes, same as /tmp and /snapshots.
* fix(idols-ai): preserve logrotate state as a file, fix persisted dir ownership
- /var/lib/logrotate.status is a regular file, not a directory: move it from
preservation directories to files. A missing state entry makes logrotate
treat every log as freshly rotated, so it must survive reboots.
- keep /var/lib/btrbk owned by btrbk:btrbk 0750 so preservation does not
override the btrbk module's rule with root:root.
- set /var/tmp mode 1777 explicitly instead of relying on systemd's tmp.conf
ordering.
- rewrite comments to describe the paths generically.
* chore(idols-ai): cap journal, wipe coredumps at boot, persist upower/noctalia
- services.journald.settings.Journal: SystemMaxUse=10G, RuntimeMaxUse=256M,
matching the cap already set on idols-kana/idols-ruby.
- wipe /var/lib/systemd/coredump at boot: it lives in the persisted
/var/lib/systemd and crash dumps can contain secrets copied from memory.
- persist /var/lib/upower (device battery history) and
~/.local/state/noctalia (shell settings, notification/clipboard history).
Running strata with qwen3.8-flash-next keeps ~53G of model weights
resident, leaving little RAM for the desktop side and pushing ~13G of
anonymous pages into the 20G swapfile. With swap that full, a burst
risks OOM-killing processes. Disk space is not a constraint (1.9T
btrfs, ~1.4T free), so grow the swapfile to the classic 1x-RAM size.
Keep zram off and zswap at 10%: zram's compressed pool would steal page
cache that the mmap'd weights rely on.
Prefer the narrowest output of a dependency (for example stdenv.cc.cc.lib
instead of stdenv.cc.cc) and measure the built closure, since a full
toolchain can add hundreds of MiB. Also note how to check what is actually
needed (DT_NEEDED, exec'd tools) before trimming.
Disable the rootful system podman.socket on desktop hosts and drop the module's autoPrune, which prunes the rootful store and pulls in the rootful API service. Add a weekly per-user timer that prunes the rootless store instead.
Manage ~/.config/go-musicfox/config.toml declaratively: the upstream default plus the mpv engine, ~/Music download dir, 2 GiB cache and cover art.
The app rewrites the file with a temp file plus rename, which would replace a plain symlink, so use force = true like fcitx5's profile: the repo copy is restored on every rebuild and runtime edits are ephemeral. Runtime data and state stay on the persistent volume.
* feat(secrets): add work gitconfig secret, symlink it in home-manager
* feat(secrets): add a no-sudo replace recipe for new secrets
* refactor(secrets): move work gitconfig symlink into the desktop layer
* docs(skills): point nix-config-secrets at the private repo under ~/codes
Record where the private nix-secrets checkout lives and that agents should
do the mechanical secret work themselves, handing back only the
interactive/sudo agenix operations and pushes.
* docs(agents): add related repositories to nix-config
* chore(agents): adopt ~/src for source checkouts
Move third-party/source checkouts from ~/codes/src to ~/src so the home
layout reads as three siblings: ~/codes (personal), ~/work (work),
~/src (checkouts). Preserve and back up the new path, and update the
global rule and the two nixpkgs skill references.
* docs(agents): tighten the code layout rule
* docs(agents): fold the work-confidentiality note into the layout line
* docs(agents): base the confidentiality rule on repo visibility
* docs(agents): drop the stale KubeVirt mention
Catppuccin's home-manager module writes extra-substituters/extra-trusted-public-keys
for catppuccin.cachix.org into the user-level ~/.config/nix/nix.conf. Since the
normal user is no longer a trusted-user (modules/base/nix.nix), Nix ignores those
client-specified settings and warns on every command. The ports build locally, so
turn the user-level cache off instead.
nixvim has been disabled for a long time (programs.nixvim.enable = false), so
drop the flake input (and prune flake.lock). Port the settings it carried to
Home Manager's programs.neovim: opts and per-plugin setup become an initLua
block plus a plugin list. The catppuccin module already provides the
colorscheme, vi stays unmapped, and the plain system neovim package is
untouched so sudo/nvim --clean keep using it.
Allow clearly-implied task authorization, trust skills listed in the runtime's configured catalog, and restore the ban on following external-material instructions. Sync the affected evaluation scenarios.
* fix(youko): add ROCm OpenCL so Jellyfin can tone-map HDR
Jellyfin's HDR/DoVi transcodes use ffmpeg's `tonemap_opencl`, but only
Mesa was installed via `hardware.graphics.enable`, so no OpenCL ICD was
registered. ffmpeg then failed to create an OpenCL device ("Failed to
get number of OpenCL platforms: -1001") and aborted with exit code 237,
leaving HDR titles unplayable.
Add `rocmPackages.clr.icd` to `hardware.graphics.extraPackages`; it
ships the ICD for the Barceló iGPU (reported as `gfx90c`) under
/run/opengl-driver/etc/OpenCL/vendors, where `ocl-icd` (built with
`--enable-custom-vendordir`) finds it.
The Barceló iGPU only manages ~0.8x realtime for a 4K HDR -> 1080p
tone-mapped transcode, so playback can still buffer; this only makes
such titles playable at all.
* fix(youko): allow /dev/kfd so Jellyfin OpenCL can reach the GPU
The jellyfin module's `DeviceAllow` whitelist only permits the VA-API
render node (`/dev/dri/renderD128`). ROCm's OpenCL runtime opens
`/dev/kfd`, so even with the ICD installed OpenCL device creation failed
("No matching devices found") and HDR transcodes still aborted with exit
code 237.
Add `/dev/kfd rw` to the whitelist; list definitions merge with the
module's entry. Verified under the unit's sandbox (PrivateUsers,
PrivateTmp) that OpenCL init then succeeds as the jellyfin user.
conda/mitmproxy/wireshark are already Linux-gated in the shared home modules, and verible is unreferenced, so replacing them with emptyDirectory is unnecessary.
openldap's flaky tests are skipped upstream now (doCheck = true), so the override only forced local rebuilds of openldap/cyrus-sasl dependents such as libvirt and kio.
davfs2 caches the whole opened file regardless of cache_size, so opening media on /mnt/fileshare downloads it fully and blocks the mount. The share is no longer used.
* fix(youko): stop homepage from probing the authenticated transmission RPC
The siteMonitor hit /transmission/rpc unauthenticated, so every probe returned 401 and counted against transmission's anti-brute-force threshold.
* feat(youko): tune transmission and firewall its network namespace
Raise the upload limit and queues, relax required encryption to preferred, and add a default-drop nftables input chain inside the netns so the peer port stays open while everything else is closed. Allowlist the nftables 'iif' keyword for typos.
* feat(youko): export transmission metrics and add a Grafana dashboard
nixpkgs has no transmission exporter, so add a small stdlib Python exporter that queries the authenticated netns RPC on loopback, scrape it from VictoriaMetrics, and ship a Transmission dashboard.
* fix(youko): cap transmission upload at 2500 kB/s
30 MB/s ignored the 50 Mbps uplink; keep seeding around 20 Mbps.
* Revert "feat(youko): export transmission metrics and add a Grafana dashboard"
This reverts commit 9df7a26526.
Hosts used to trust the router as the only boundary (servers ran with the
firewall off; service ports were opened globally), so any LAN device could
reach every host service. Move filtering to the hosts as defence in depth
behind the router, which keeps its firewall on and uses a port mapping for
the few ports we actually need.
- base: nftables backend; deny inbound by default; trust the home LAN by
source and the tailnet by interface (restrict the tailnet with ACLs);
suppress per-service `openFirewall` global ports; allow node_exporter only
from the monitoring host. The local podman bridge, the tailnet and the pod
CIDRs are accepted only after the node_exporter drop.
- servers: stop disabling the firewall; drop redundant global port opens
(caddy 80/443, NFS 2049, Jellyfin, localsend/test ports); delete the
redundant desktop firewall module; aarch64/riscv64 import the firewall
explicitly (akane was missing it).
- k3s: Cilium pods (10.0.0.0/8, fd00::/8) may reach host services; drop the
flannel-era cni0/flannel.1 trust.
- youko: postgres listens on localhost + the LAN with pg_hba limited to the
LAN; the NFS export stays LAN-scoped; remove the unauthenticated v2ray
proxy.
- tests: `security-firewall` asserts every host runs the shared firewall, has
the LAN CIDR rule and no global ports.
Treat content from unverified sources as data, not instructions, and require authorization before cloning, downloading, or executing unverified code and install scripts.
* feat(youko): add Jellyfin media server
Jellyfin reads the existing Transmission download tree on the public HDD
(via the shared `fileshare` group) and transcodes with the AMD Barceló
iGPU through VA-API. Encoding settings are declared in Nix
(`forceEncodingConfig`), so the web UI cannot silently drift.
Replace the per-service public ACME certificates with a single
`*.writefor.fun` wildcard (plus the apex) shared by every public vhost,
so adding a service needs no new ACME entry; a wildcard also keeps
individual hostnames out of public CT logs.
Pin the jellyfin uid/gid to avoid the dynamic-id drift fixed for the
other services in #320.
* fix(youko): enable graphics for Jellyfin VA-API
VA-API needs Mesa's radeonsi VA driver under /run/opengl-driver, but
youko is headless so hardware.graphics was off. Without it Jellyfin's
ffmpeg fails with 'va_openDriver() returns -1'. Enable it so the iGPU
can be used for hardware transcoding.
* fix(youko): use VA-API hardware encoding for Jellyfin
Enabling hardware decoding alone left encoding on the CPU (libx264), which
pins ~3 cores per 720p stream. Set transcoding.enableHardwareEncoding so
the iGPU (Vega VCN) does the H.264 encode too.
* feat(youko): harden Jellyfin proxy trust, groups, and HEVC encode
Add jellyfin to the render/video groups for robust VA-API access (was
relying on the world-writable render node), enable HEVC hardware encoding
(Vega's VCN supports it; AV1 does not), and patch network.xml on start to
trust Caddy as KnownProxies and advertise the public URL.
* fix(youko): run Jellyfin network.xml patch every start; allow LAN discovery
The module's preStart exits early once encoding.xml matches, so the
mkAfter network.xml patch only ran when encoding settings changed. Run it
with mkBefore instead. Also allow Jellyfin app auto-discovery (UDP 7359 /
TCP 8096) from the home network only.
Service users/groups get a dynamically allocated uid/gid, which shifts when the user set changes: adding `immich` moved gitea 993:985 -> 996:993 and rustfs 987:977 -> 988:983. Files on the persistent volumes keep the old numeric owner, so gitea and rustfs lost access to their data and failed to start after a restart. Pin every dynamically allocated service id on youko to its current value. The ids are host-specific, so this is a host file.
timerConfig was set with lowercase onCalendar/randomizedDelaySec, which systemd treats as unknown directives. The generated timer had no schedule, so `systemctl list-timers` showed NEXT=- and no backup ever ran. Use the real keys OnCalendar/RandomizedDelaySec.
Add Immich via the native nixpkgs services.immich module on youko, reusing the host PostgreSQL 16 (pgvector + vectorchord) and a shared Valkey instance for Redis-compatible caching and BullMQ queues (Dragonfly rejected BullMQ's dynamic Lua keys). Media lives on the encrypted HDD at /data/apps/immich; Caddy publishes immich.writefor.fun with a publicly-trusted Let's Encrypt cert (security.acme + Cloudflare DNS-01) so the mobile app connects. OTEL metrics are scraped by VictoriaMetrics, and /var/cache is now preserved on the tmpfs-root hosts so the ML model cache survives reboots.
The s3-console Caddy vhost redirected every request to / to /rustfs/console/. The RustFS console frontend signs its own S3/STS requests to / (POST / for STS, GET /?x-id=ListBuckets), so the 302 broke the console session and login failed server-side with SignatureDoesNotMatch ("missing signed header: content-type"). Restrict the redirect to unsigned browser GET /; signed requests now reach RustFS unchanged.
Deploy RustFS in place of the retired MinIO and move the S3 consumers over.
- flake.nix / flake.lock / outputs/default.nix: add a nixpkgs-small
(nixos-unstable-small) input for the rustfs 1.0.0 package (with the web
console), instantiated as pkgs-small in genSpecialArgs alongside the other
nixpkgs instances; drop the rustfs flake input. The rest of the system stays
on the default nixpkgs.
- hosts/12kingdoms-youko/homelab-services/rustfs.nix: use nixpkgs' own
services.rustfs module, with package = pkgs-small.rustfs and
environmentFile = the agenix rustfs.env secret.
- caddy.nix: s3.writefor.fun (S3 API) and s3-console.writefor.fun (console,
redirects / to /rustfs/console/).
- secrets/nixos.nix: declare a single rustfs.env (dotenv) secret holding
RUSTFS_ACCESS_KEY / RUSTFS_SECRET_KEY, replacing the separate key files.
- infra/minio -> infra/rustfs: buckets are managed with the hashicorp/aws
provider against the RustFS S3 endpoint; IAM/ILM are applied with rc.
Crush hardcodes banned bash commands, blocking many operations. Drop the package, its AGENTS.md/CRUSH.md install target, env vars, and preservation entries.
Keep the six node configuration intact while removing the failing user activation from the three control-plane MicroVMs. Update the regression test and current K3s configuration notes.
0.156.1 still lacks the codex-chatgpt recursion_limit attribute, so keep the postPatch that adds it. The vendored Cargo.lock is unchanged, so the cargoHash stays the same.
cilium writes /etc/cni/net.d while istio-cni reads k3s's own dir, so on the new workers the chained plugin never found a network config. Add the symlinks the server module already has.
kubelet refuses to self-assign the reserved node-role.kubernetes.io/* labels, so the worker --node-label made the agent crash-loop. Drop it and rely on the masters' control-plane taint for placement; document that the taint only lands via registerWithTaints on first registration, so it was applied by hand here.
- run two autocutsel instances so the X PRIMARY/CLIPBOARD selections stay in
sync, making copy/paste work in the session and through VNC
- drop --disable-blink-features=AutomationControlled: nothing drives the
browser over CDP, so navigator.webdriver is already false and the flag only
marked the setup as automated
- document why the driver stays on real X11/AT-SPI input and is never granted
a CDP attach, and note the SSH-tunnel path for manual VNC access
- extend the computer-use eval test
Add worker-1 (4 vCPU / 16 GiB) on shoryu, and worker-2 (4 vCPU / 16 GiB) plus worker-3 (2 vCPU / 8 GiB) on shushou, as microVMs. They come with a genK3sAgentModule and the per-host/output/home-manager wiring, and are labelled node-role.kubernetes.io/worker=true. The masters keep their control-plane:NoSchedule taint, so workloads land on the workers.
The script pointed the macOS nix-daemon at the homelab v2ray HTTP proxy (192.168.5.101:7890), but that proxy was never running. Remove it together with the just darwin-set-proxy recipe and the README step that referenced it.
Reserve 192.168.5.192/27 as a dedicated kube-vip block, outside the router's DHCP pool (.2-.99): one /29 per cluster, k3s-test-1 gets .192/29 with the API VIP at .193.
Advertise only youko itself (.183/32) and that block over Tailscale, instead of the whole /24. The /24 route shadowed hosts that are already reachable directly, e.g. idols-ai at 192.168.5.100.
Deploy prerequisites (not in this repo): approve the new Tailscale routes; point kube-vip at the new VIP; move the *.writefor.fun wildcard from .101 to .183 and update scripts/darwin_set_proxy.py, since .101 is no longer advertised.
Give the idols-ruby and idols-kana VMs a headless computer-use
environment: Xvfb plus i3, driven by cua-driver (and computer-use-linux
as a fallback). Includes the system module (AT-SPI, fonts, Mesa, Clash
Verge proxy, proxy-region timezone), the home i3 module (Xvfb/i3/x11vnc
services, browser flags), host wiring and an eval test.
Package the prebuilt releases (computer-use-linux v0.7.1 and
cua-driver-rs v0.24.0) under pkgs/, injected through thin callPackage
overlays so they are easy to relocate to the NUR repository later.
Only three secrets are readable as the login user (nix-access-tokens,
ssh-key-romantic and the nushell one); everything else is root-only and
the filesystem already keeps agents out.
* feat(backup): notify on failure, and 404 unconfigured hostnames
A failed backup was silent. Post to alertmanager (which routes above-info
to telegram), and have caddy answer 404 instead of its empty 200 for
*.writefor.fun names that are not services.
* docs(backup): say 'cloud copy' rather than 'offsite'
Clearer for a reader who does not live in backup jargon; the requirement
(not in the homelab, immutable) is spelled out in the text.
* docs(secrets): state the recipient invariant
A recipient set must answer two questions: who may read it, and how it is
recovered once those hosts are gone. Dropping recovery_key while narrowing
a secret is easy and silent.
* docs(secrets): state that every secret must be decryptable by the desktops
The desktops hold the trusted admin keys and are where secrets are edited and
rekeyed from; narrowing a set to the servers alone breaks agenix -r and drops
the offline recovery key.
* docs(secrets): desktops and recovery_key have identical access
recovery_key is a member of desktop_keys, so every group written as
desktop_keys ++ ... covers both.
* feat(monitoring): scrape the restic rest-server
Expose /metrics on the loopback-bound server, scrape it with
victoriametrics, and block /metrics on the public vhost.
* docs(backup): record what the metrics actually expose
rest-server only ships the go/promhttp defaults, so repository growth is
charted from node-exporter's filesystem metrics instead.
The desktops hold the trusted admin keys and are where secrets are edited and
rekeyed from; narrowing a set to the servers alone breaks agenix -r and drops
the offline recovery key.
A recipient set must answer two questions: who may read it, and how it is
recovered once those hosts are gone. Dropping recovery_key while narrowing
a secret is easy and silent.
A failed backup was silent. Post to alertmanager (which routes above-info
to telegram), and have caddy answer 404 instead of its empty 200 for
*.writefor.fun names that are not services.
A desktop holds both the plaintext and the repository password, so blocking
deletion on the server protects little while preventing the automatic
retention the client needs. Immutability belongs in the offsite copy.
The design (layers, password groups, retention, restore steps) was only in
code comments. Also make the timer configurable: a whole hour of random
delay defeated any staggering, and the desktop's push, youko's own run and
btrbk all otherwise contend for the same disks.
Split the repository password into desktop and homelab groups, add the
environmentFile/pruneOpts module options, and point the desktop at youko's
REST endpoint.
Allowlist of the irreplaceable directories (Documents, work, Pictures,
nix-config, .local/state); project checkouts under ~/codes live in git and
are not backed up. Kept the excludes generic, since this repo is public.
Also move the restic-password secret to a desktop.enable || server.storage
gate so desktops get it without enabling a server group.
* feat(hosts): per-host restic backups to the SMB remote
Replace the disabled aggregate job (which rsynced every host's /persistent
into /tmp - a tmpfs - and silently produced incomplete backups) with one
restic repo per host, backing up its own data directly:
- shared module lib/genResticBackup.nix: secrets, rclone/SMB target
(rclone:smb-downloads:/Downloads/homelab-backup/<host>), retention, and
excludes for the regenerable/bulky paths (podman overlay layers, microvm
and libvirt disk images, nfs golden images, logs, caches)
- youko additionally backs up the service state under /persistent/var/lib
and dumps all postgres databases first (a consistent SQL artifact beside
the crash-consistent data dir)
- shoryu/shushou back up /etc/agenix, /etc/ssh and the k3s token
Needs three agenix secrets before it can run: restic-password,
ssh-key-for-restic-backup and rclone-conf-for-restic-backup.
* refactor(hosts): only restic does off-host; btrbk stays local
btrbk works at the subvolume level, so a would ship the whole
@persistent - including podman's overlay storage, the microVM/libvirt
images and the nfs golden images - which is exactly the regenerable bulk
we want out of the backups.
So: btrbk keeps making the cheap local CoW snapshots (quick rollbacks),
and restic owns the off-host copies with file-level excludes. Back up the
whole /persistent minus those excludes, so nothing is missed.
Update the btrbk module's comment to say so.
* fix(restic): never back up keys; drop the implicit /etc/agenix
restic's own password lives in /etc/agenix, so backing that up would store
the repository's password inside the repository. Exclude agenix, the ssh
host keys and the usual credential dirs (.ssh, .gnupg, .aws, gcloud).
* refactor(restic): make the backup a proper module
Replace lib/genResticBackup.nix with modules/nixos/base/restic-backup.nix,
which declares modules.restic-backup options (enable, repository, paths,
exclude, requiresMountsFor, postgresDump) like the btrbk module does.
youko just sets the options now.
* refactor(restic): exclude the whole podman storage, trim retention
Exclude /persistent/var/lib/containers wholesale: the image layers are
re-pullable and podman's storage DB is not consistent when copied from a
running podman. uptime-kuma's named volume lives there, but its data is
not wanted anyway.
Retention: daily 3, weekly 2, monthly 2 - no yearly.
* refactor(restic): back up a short-lived btrfs snapshot, drop pg_dump
A live copy of a running postgres data dir is only crash-consistent. Instead
of a daily pg_dumpall (which duplicates the data in the repository and gets
slower as the DB grows), take a short-lived read-only btrfs snapshot in
backupPrepareCommand, back that up, and delete it right after - a
consistency step, not a second backup. The excludes are relative now so they
work for both the snapshot and a direct path.
* feat(secrets): restic-password for modules.restic-backup
With the rekeyed secret in nix-secrets (storage group, alongside
hdd-luks-crypt-key), the module's passwordFile resolves.
* fix(restic): agenix's secret is under /run/agenix, not /etc/agenix
* fix(restic): take the password path from agenix, not a hardcoded one
config.age.secrets."restic-password".path follows agenix's secretsDir, so it
stays correct if that changes.
* refactor(youko): let preservation own the service state dirs
The services' data now sits on the SSD, so the hand-written dataDir +
systemd.tmpfiles rules are unnecessary: /var/lib is already preserved by
preservation, and the upstream modules create and own their default
state dirs (StateDirectory). Use the defaults:
- gitea /var/lib/gitea, grafana, sftpgo, caddy, victoriametrics,
postgresql (/var/lib/postgresql/<version>), uptime-kuma, homepage
- drop the per-service tmpfiles rules and victoriametrics' self-bind
Data was moved in place with mv (same filesystem): /persistent/apps/<svc>
(== /var/lib/<svc> via the preservation bind) so nothing was copied.
* refactor(youko): drop caddy's redundant tmpfiles rules
The activation script already creates the fileserver tree with mkdir -p and
the module creates caddy's dataDir, so the rules - and the unused vms
directory they created - are redundant.
* refactor(youko): use a podman named volume for uptime-kuma
podman creates the volume and takes the ownership from the image, so the
kuma user, the tmpfiles rule and the host directory are no longer needed;
the volume lives under /var/lib/containers, which preservation keeps.
The services now run directly on the host, so their state does not belong
on the HDD behind the flaky USB-SATA bridge: /data/apps was small-file
heavy and latency/consistency sensitive. Point every dataDir at
/persistent/apps (the LUKS nvme) instead.
The HDD keeps the bulk data that suits it: /data/fileshare (173G) and
/data/backups (24G). The HDD's old @apps subvolume and its data are left
in place for now.
-B 255 turns APM off and it stuck, but the disks still reach standby, so
the bridge firmware is likely doing the spindown. -J 0 additionally asks
the drive to stop parking its heads (sdb has ~1M load cycles).
pcie_aspm=off was the least-justified mitigation (the flakiness is on the
USB/SATA side) and costs power. Also log the disks' APM level/state and
re-apply the settings hourly, since the bridge or the disk firmware can
drop them (the disks were still reaching standby).
The two 4TB HDDs sit behind a JMicron JMS567 that resets and throws
UDMA-CRC / I/O errors (the disks themselves are healthy). Best-effort
mitigations while the enclosure stays in use:
- kernel: usbcore.autosuspend=-1, usb-storage.delay_use=10, pcie_aspm=off
- udev: keep the bridge's power/control=on (never autosuspend)
- disable APM/standby on the HDDs (spindown is a common reset trigger)
- enable smartd (watch the CRC count and disk health)
The kernel parameters need a reboot to take effect.
It documented the retired KubeVirt workflow (upload a qcow2, create the vm
in k8s-gitops, let Flux create it), which no longer exists. The general
deployment notes live in the README and the per-platform docs
(hosts/k8s/README.md for the microVMs).
Update the user-facing bits that still pointed at KubeVirt: the homepage
dashboard group and entries (and drop the dead LongHorn one), the docs,
and remove the now-unneeded virtctl package.
The k3s-test workers never ran (they were stuck waiting for volume
binding) and will be re-added as microVMs, so remove their KubeVirt
configs and everything that only existed for KubeVirt guests:
- hosts/k8s/k3s-test-1-worker-{1,2,3} and their outputs
- lib/genKubeVirtGuestModule.nix and lib/genK3sAgentModule.nix (the
latter was only used by the workers)
- the KubeVirt image upload recipes (upload-vm, upload-idols,
upload-k3s-test) and the kubevirt image package in the idols outputs
- refresh the docs; keep only accurate historical mentions (e.g. the
restic backup path must stay as it is)
Drop the now-meaningless kubevirt- prefix and move the hosts out of
hosts/k8s/, following the existing 12kingdoms-shoukei layout: the
directory and output carry the theme prefix while the configuration,
hostname and colmena target are the bare name (shoryu, shushou, youko).
- hosts/k8s/kubevirt-* -> hosts/12kingdoms-*; the shared disko config
becomes host-disko-fs.nix; the now-different relative imports are fixed
- outputs, vars/networking.nix keys, the Justfile recipes/tags, the eval
tests and the docs follow
- colmena reaches these hosts by IP (a new targetHost override), since
their DNS name changes with the hostname and is not resolvable until
the router learns it
This is not a Python package, so drop the packaging/build/pyinstaller/
framework/type-checker/pyenv/poetry/pdm/pixi/celery/rabbitmq rules and the
unused Marimo and Streamlit sections. Keep the bits occasional Python
scripts do need: byte-compiled files, virtualenvs, local secrets and the
common tool caches.
The Python section of the .gitignore template has a bare 'lib/' (and
'lib64/'), which also matched the Nix helper modules in lib/. The
'!lib/*.nix' line added earlier could not help, since a file cannot be
re-included while its parent directory is excluded.
The physical hosts no longer need k3s or KubeVirt: aquamarine runs
natively, the k3s-test masters run as microVMs and kana/ruby under
libvirt. So drop the host cluster and split the shared parts out of the
KubeVirt host module:
- lib/genVmHostModule.nix: the host bits the VMs still need (br0 + the
physical NIC as a bridge port, KVM, sysctls, earlyoom, libvirt,
journald, stateVersion)
- the three hosts use it and drop genK3sServerModule; also drop the
longhorn workarounds (openiscsi + the /usr/local/bin tmpfile) and the
virtctl package
- remove lib/genKubeVirtHostModule.nix
The file is not KubeVirt-specific: it is the hardware configuration for
any QEMU guest that boots from a disk (KubeVirt or libvirt). All it does
is pull in qemu-guest.nix and set fileSystems."/" from the disk labeled
nixos.
hosts/k8s/README.md: how the microVMs keep state, the tap/br0 naming, how
to move a KubeVirt VM over in place, and the refreshed TODOs. Also note
the migration in Virtual-Machine.md and flag the volume images in
AGENTS.md.
Replace the KubeVirt/etcd members with microVMs on the three physical
hosts, keeping the existing cluster: the guests inherit their host key
and k3s state from the old disks, so etcd recovers in place instead of
bootstrapping a new cluster.
- convert hosts/k8s/k3s-test-1-master-{1,2,3} to genMicrovmGuestModule
and switch their outputs to the microvm module (keep home-modules)
- declare the vms on shoryu/youko/shushou and bridge their taps to br0
- each guest config is self-contained (its secret group and the
nixpkgs.config override live in the guest, not the host module)
- guest specialArgs carry agenix/mysecrets so secrets/nixos.nix works
lib/genMicrovmGuestModule.nix: qemu guest with a shared read-only store
(virtiofs), persistent /etc, /var and /home volumes, a tap on br0 (its
name derived from the guest IP, as IFNAMSIZ caps it at 15 chars), and a
deterministic MAC. Also keep lib/*.nix tracked.
The service modules moved two levels deeper, so their certs/ paths
resolved relative to the wrong directory. Use mylib.relativeToRoot so
they no longer depend on the module's depth.
Drop the retired host's name from the filenames:
aquamarine-services.nix -> homelab-services.nix and the aquamarine/
module dir -> homelab-services/. The "aqua" name now stays only in the
docs/vars, where it is kept for a possible later test microvm/qemu-vm on
idols-ai.
Aquamarine's services now run directly on kubevirt-youko, so the guest
definition is no longer needed:
- move its service modules to hosts/k8s/kubevirt-youko/aquamarine/ and
drop the explicit import list, leaving aquamarine-services.nix with
just its IP and secret groups
- delete the guest base, journald override, output entry and home config
- repoint the eval tests that read the guest's config to kubevirt-youko
- drop the now-dead 'aqua' colmena recipe and its image upload
The aqua KubeVirt VM itself is deleted separately; the name is kept in
the docs for a possible later test microvm/qemu-vm on idols-ai.
The VMs now declare their root disk as a DataVolume with `source: upload`, so
upload straight into `<vm>-disk` (CDI only accepts an upload into a DataVolume)
instead of a shared golden PVC. Takes the disk size as an argument.
* feat(k3s-test-1): add three worker nodes and taint the control plane
- hosts/k8s/k3s-test-1-worker-{1,2,3} (k3s agents) + flake outputs
- vars/networking: .117/.118/.119 (the .111-.113 range is still held by the
never-deployed k3s-prod-1 workers)
- k3s-test-1-master-{1,2,3}: nodeTaints=node-role.kubernetes.io/control-plane:NoSchedule
- Justfile: upload the worker images too
* chore(k3s-test-1): move the workers to .111-.113
The k3s-prod-1 entries that held those addresses are gone (nix-config#284).
It was never bootstrapped (no flux deploy key, VMs commented out since
2024-10) and only carried a stale duplicate of the test cluster's config.
- hosts/k8s/k3s-prod-1-{master,worker}-{1,2,3} + flake outputs + the home module
- the k3s-prod-1-token agenix entry (the .age file lives in the private secrets repo)
- the k3s-prod-1-* IPs, the offlineHosts entries, the Justfile recipes and the docs
* feat(kubevirt-youko): export a shared golden-image store over NFS
* feat(vm-images): upload golden images into the cluster
Replace the rsync-to-orangepi5 step with a CDI upload into the golden-image
PVCs (vms/golden-images in k8s-gitops), so VM images no longer depend on the
single-board file server.
services.earlyoom.freeMemThresholdPercent does not exist in this nixpkgs; the module exposes freeMemThreshold/freeSwapThreshold. Just enable earlyoom with its defaults (trigger at 10% free mem/swap).
* refactor(kubevirt): drop the static 1G hugetlb pool, use THP on demand
The hosts pre-reserved 48/48/15 GiB of 1G hugepages, but the VMs only used 32/16/11 GiB, so ~52 GiB was stranded (hugetlb is exclusive and cannot be overcommitted or shared with the host / non-hugepage VMs). It also blocked scheduling (a 16 GiB hugepage VM cannot fit on a host with fewer free pages, e.g. youko with 15/24 GiB) and made the full intended VM set (idols 3x8G + k3s-test 3x16G + k3s-prod 3x4G + 3x16G = 132 GiB vs 146 GiB physical) impossible to fit.
Use transparent huge pages on demand (madvise) instead, and let the VMs use ordinary memory (instancetypes updated in k8s-gitops). This frees the stranded memory, removes the scheduling rigidity, and makes overcommit/swap usable again.
* feat(kubevirt): add earlyoom; stop overriding swappiness (zram already enabled)
zram is already provided by modules/nixos/base/zram.nix (enabled by default: 50% RAM, zstd, prio 100 + swappiness/watermark tunings). But genKubeVirtHostModule hard-coded vm.swappiness = 0, which overrode the modules mkDefault 180 and disabled swapping entirely - so the zram device was never used (Swap used: 0B).
- Remove the hard-coded vm.swappiness (let the zram modules tuning apply); the disk swapfile from disko stays as the last resort.
- Add services.earlyoom to kill the greediest process before the host thrashes / OOMs.
vmagent (and other senders) can deliver the same series+timestamp twice (retries/HA), which vmsingle rejected with "duplicate sample for timestamp; overrides not allowed". Set -dedup.minScrapeInterval=30s (>= the senders scrapeInterval) so duplicates collapse instead of being rejected.
- genKubeVirtHostModule: replace the Open vSwitch bridge (ovsbr1) with a Linux
bridge br0 for the VMs secondary network; the bridge CNI plugin attaches the VM
taps to br0 and the host keeps its address on br0. Drop the now-unused
multus-cni host package.
- kubevirt hosts: stop forcing disableFlannel = false, so k3s disables flannel and
Cilium provides the pod network.
shoukei is a laptop using DHCP and never consumes hostsAddr.shoukei.
Its 192.168.5.108 collided with k3s-prod-1-master-1 and produced a wrong
ssh Host shoukei entry.
The Settings UI stored `theme.mode` in ~/.local/state/noctalia/settings.toml, so it was not versioned. Pin it in the config baseline; the [location] sunrise/sunset schedule still drives light/dark.
At background_opacity 0.2 the light-mode surface disappears into dark wallpapers, so the dark light-mode text sits on a dark bar. Raise it to 0.61 to keep contrast while staying slightly translucent.
`programs.noctalia.settings` deploys the TOML as a store symlink, so every edit needs a `home-manager switch`. Deploy the baseline from the config layer with `mkOutOfStoreSymlink` instead: Noctalia hot-reloads config-layer files and never rewrites them, so edits apply live while Settings UI changes stay in ~/.local/state/noctalia/settings.toml (which loads last) and out of the repo.
Update the docs and host-specific comments to match the two-layer model.
The sandbox remaps XDG_DATA_HOME/XDG_CONFIG_HOME, so paths Firefox exposes to
or shares with the host ended up isolated. Bind the MPRIS artwork dir and the
real trash rw, and mimeapps.list, user-dirs.dirs and the shared NSS cert store
ro.
recently-used.xbel stays isolated: GTK writes it atomically, which a
single-file rw bind would break.
Add cpu, ram, disk and network up/down sysmon widgets to the bar start lane,
matching the old v4 SystemMonitor. Set the clock to ISO date, full English
weekday and 24h time (e.g. "2026-09-19 Saturday 16:06").
English stays the default (LANG=en_US.UTF-8). Drop the Chinese overrides for
LC_TIME, LC_NUMERIC, LC_ADDRESS, LC_NAME, LC_TELEPHONE and LC_IDENTIFICATION:
LC_NUMERIC matches en_US, the others are effectively unused, and LC_TIME forced
Chinese date and weekday names.
Keep zh_CN only where the en_US default is wrong for China: metric units, A4
paper and CNY.
Refresh the repository overview after the move to Noctalia v5:
- fix the component table: Noctalia as the desktop shell, native screenshots, systemd-networkd/NetworkManager, Zed & VS Code as GUI editors, Zellij as a multiplexer
- correct the deploy instructions (niri hosts use 'just niri'; 'just local' matches the bare hostname) and the Nvidia link
- update the NixOS badge to 26.11 and add a short validation section
- root README: VS Code config moved from home/linux/gui/base/editors.nix to vscode.nix
- READMEs now list both GUI editors (Zed, the primary one, plus VS Code)
- base README: xdg is a directory now, nvidia.nix no longer exists, editors.nix -> vscode.nix
- idols-ai README: disko-fs-media.nix -> disko-fs-data.nix (/persistent/data)
- secrets README: password-store moved from home/base/desktop to home/base/tui
- drop unused link definitions (AstroNvim, wl-clipboard, Catppuccin)
- fix a pre-existing typo flagged by the typos hook (keeped -> kept)
Noctalia v5 is the native C++/Wayland rewrite; the Quickshell-based v4 line (pkgs.noctalia-shell) is EOL and its JSON config is not migrated upstream.
- use the upstream home-manager programs.noctalia module (present at our pinned HM revision) instead of the hand-rolled package + symlink module: it installs pkgs.noctalia, validates config.toml at build time via checkConfig, and supports customPalettes; drop the v4 Qt/Qt6ct session setup
- replace the v4 settings.json/plugins.json with a minimal shared config.toml (font, Catppuccin shell palette, templates off, screenshot annotation, accessibility/bar scale, night light, wallpaper, location/weather); host-specific bits live in per-host files (shoukei's eDP-1 backlight in noctalia/host-shoukei.toml), which merge after config.toml. Application theming stays with catppuccin/nix
- treat config.toml as the single source of truth: do NOT persist ~/.local/state/noctalia and set setup_wizard_enabled = false, so UI tweaks are promoted back into the repo instead of accumulating in app-owned state
- port niri keybinds to 'noctalia msg ...', update spawn-at-startup and the backdrop layer namespace, fix the broken 'octalia-shell' media-previous bind, and replace the grim/slurp/satty screenshot stack with Noctalia's native region/fullscreen/annotate commands (drop the unused packages)
- drop the now-redundant swaybg and swaylock (Noctalia owns the wallpaper and lock screen, the latter authenticating against the PAM 'login' service) and its PAM entry; keep wlogout as an emergency fallback
- update hypridle's lock commands and the matching eval test
- refresh the noctalia docs
networkd enables systemd-resolved by default, and resolved's own mDNS responder conflicts with avahi (duplicate answers, avahi logs 'Detected another mDNS stack running on this host'). Keep avahi for DNS-SD (cups-browsed depends on it) and turn resolved's mDNS off.
home.sessionVariables writes export NAME="value" without escaping embedded double quotes, so the JSON {"share":"disabled"} reached opencode as the shell word {share:disabled} and failed config parsing. Emit the export with sessionVariablesExtra instead.
The 2026-09 nixpkgs update changed DRM probe order, renaming the Intel connectors DP-1 -> DP-4 and HDMI-A-2 -> HDMI-A-3. The connector-name based output and open-on-output rules stopped matching, which dropped monitor scale/mode/position and sent every named workspace to the first output. Match by make/model/serial instead, which is stable across connector renumbering.
Tencent replaced the file behind the versionless CDN URLs, so the pinned hashes stopped matching. Refresh the x86_64 and aarch64 hashes with nix store prefetch-file.
trash-empty lacked Persistent=true, so a host that was off or suspended at the scheduled time never ran the purge and trash retention silently stopped.
Guix was enabled on every desktop host, but shoukei had no /gnu mount (store and state lived on the tmpfs root) and it was not used. Drop the module, the modules.guix option, the @guix subvolume and the /var/guix preservation entry.
The restic job needs three agenix secrets that do not exist yet (restic-password, ssh-key-for-restic-backup and an rclone config for the smb-downloads remote). Comment it out until they are provisioned so it does not fail silently every night.
Load both *.yml and *.yaml alert rule files: the Loki rules were named .yaml while only *.yml was loaded, so they silently never ran. Fix the Grafana domain typo (writefo.fun -> writefor.fun). Comment out the postgres-playground datasource, which used the Grafana admin password for a role that has no password and could never authenticate.
checks.<system>.eval-tests returned a boolean, so nix flake check failed with "not a derivation". Wrap the evalTests result in a runCommand so flake check actually evaluates it.
* feat(btrbk): make snapshots explicit and guarded
Replace the always-on, hardcoded module with modules.btrbk options (enable/volume/subvolume/snapshotDir/target/onCalendar), and assert the btrfs top-level subvolume is mounted so a missing mount fails evaluation instead of silently failing at runtime.
Store snapshots in the @snapshots subvolume (snapshot_dir=@snapshots) rather than next to the live subvolume, make the off-host target explicit and off by default, and document that local snapshots are not off-host backups.
Enable the feature explicitly on idols-ai, shoukei and the three kubevirt hosts; qemu-guest now disables it through the option. Add eval tests for shoukei and ai.
* feat(monitoring): alert on btrbk snapshot failure
Add HostBtrbkSnapshotFailed on node_systemd_unit_state for btrbk-btrbk.service, so a broken snapshot schedule alerts instead of only surfacing in the journal.
RTK compresses only shell output and independent evaluations show little to no end-to-end cost reduction. Drop the package and its install snippets; runtime artifacts were uninstalled separately.
Use one wording per gating family: exact target and scope, recovery path or safety guard, and explicit authorization. Remove the home-directory instance from workspace access and the repeated approval gate from the breaking-change bullet. Sync the high-impact eval scenario wording.
State the destructive-and-high-impact section as a principle: irreversible operations need explicit authorization, destructive or force operations form a separate categorical gate, and unpublished history rewrites are exempt at section level. Generalize infrastructure preview invalidation to any input change, and add an irreversible-action eval scenario.
Generalize the revert rule to protect any content the user changed or removed, and require confirmation before touching it. Replace the mock ban with a scope rule: checks must not be faked or weakened, and test doubles may replace only what a check does not verify. Sync the matching eval scenario.
Install microsoft-edge on x86_64 Linux via Home Manager and on macOS via
a brew cask. Enable the chromium and brave-origin Home Manager options
alongside google-chrome, dropping the google-chrome aarch64 fallback to
chromium since google-chrome supports aarch64-linux.
Persist the Brave Origin and microsoft-edge profile directories on the
ephemeral-root hosts.
- limit POSIX shell to a single command with arguments
- route pipes, chaining, substitution, redirection, and ShellCheck/BashPitfalls cases to Nushell or Python
- remove the Request handling and Scope and precedence sections
- keep the non-weakening guarantee under Safety and authorization
- remove the now-stale eval scenarios
- merge verification rules and split infrastructure guidance
- shorten Tools, Shell/scripts, and Communication
- apply script validation to any script file, including temporary ones
- sync evals
* docs(agents): bind authorization to the change boundary
* docs(agents): require target identity confirmation
* docs(agents): classify operations by impact, not command form
* docs(agents): require reviewed previews for high-risk IaC
* docs(agents): verify production changes beyond exit codes
* docs(agents): sync new-script eval with current rules
* docs(agents): split global rules evals into smoke and extended
* docs(agents): refine change guardrails wording
- scope the guardrails to any environment, not only production
- preview high-risk changes with plan, diff, or dry-run
- trim redundant examples and simplify target identity confirmation
- confirm unclear targets with the user instead of stopping
- allow file-path secret references and require approval for breaking changes
- relax the commit subject limit and allow rewriting unpublished local history
- refine tooling plus shell and script selection/validation guidance
The r8169 NIC drops carrier on S3 resume, and systemd-networkd reacts by reconfiguring enp130s0: it dropped the static address/default route and deleted foreign routing policy rules (mihomo's 9000-9010 and Tailscale's 5210-5270). clash's TUN then lost its outbound interface and networking stayed broken until clash was restarted.
- IgnoreCarrierLoss=10s keeps the static address/route across the short carrier loss.
- ManageForeignRoutingPolicyRules=false stops networkd from deleting the TUN/VPN ip rules during the reconfiguration.
The lock listener ignores inhibitors but had no playback condition, so it locked the screen at its timeout even during active media playback (screen-off already skipped via playerctl). Add the same playerctl condition to the lock listener.
niri already owns the org.freedesktop.ScreenSaver interface and feeds its inhibitors into the compositor idle notifier, so hypridle's own registration always fails with 'Another service is already providing...'. Set ignore_dbus_inhibit = true so hypridle no longer tries to claim it; D-Bus inhibits are still handled by niri.
Replace the static hypridle.conf with services.hypridle.settings and add required timeout options so each host tunes its own values:
- idols-ai (PC): 15 min keyboard, 20 min screen-off, 30 min lock.
- 12kingdoms-shoukei (laptop): 3 / 6 / 20 min (previous values).
The options have no defaults because the right values depend on the machine. Update the home-manager eval test to inspect the settings option instead of the old config file text.
Desktop speakers on idols-ai use the EBU R128 broadcast standard (-23 dB), while the quiet built-in laptop speakers on 12kingdoms-shoukei keep -12 dB. Move the target out of the shared preset so each host sets its own value.
Declare the output chain via services.easyeffects.extraPresets and load it with preset.output, replacing the GUI-only configuration that did not survive the tmpfs root.
autogain target is set to -12 dB (EasyEffects defaults to -23, which is too quiet for playback); the limiter acts as a transparent -1 dB safety net via threshold=-1 and gain-boost=false.
Emphasize safety precedence in the intro, and replace the narrow Bash pipeline ban with a trigger-based rule: keep POSIX shell as glue and move to Nushell or Python once quoting, error handling, parsing, types, retries, or portability are involved.
- render Niri on the Intel iGPU while retaining NVIDIA-connected physical outputs
- encode Sunshine streams with Intel VAAPI and wait for a Niri output before startup
- load VKMS persistently for headless sessions and configure `Virtual-1` at 2560x1600@60
- let Moonlight request the stream bitrate instead of enforcing a 20 Mbps host cap
- remove unsupported Dynamic Boost and global NVIDIA session overrides
- grant Sunshine access to virtual input devices
- add a Nushell recovery helper that starts a temporary greetd/Niri session from SSH
- identify and safely replace normal, standalone, or previous transient Niri sessions
- launch the transient service in the background and verify greetd, Niri, and Sunshine before returning
- ignore local `.worktrees` directories and keep eval regression coverage focused on structural behavior
- read Asahi aop-sensors ALS via iio and drive eDP-1 backlight
- drop PrivateUsers for wluma so it can reach the real user D-Bus session
- remove hypridle 10min screen-dim listener: its brightnessctl writes fight
wluma and get learned as manual adjustments
Enable gnomeKeyring for the passwd PAM service so pam_gnome_keyring
updates the login keyring password when the login password changes.
Fixes 'Unlock Login Keyring' prompt desync after changing the password
via passwd (greetd session unlock alone does not update the keyring).
The running clash-verge-rev reports app_id "clash-verge" (lowercase), so the
anchored ^Clash-verge$ pattern never matched and the window opened on whatever
workspace was focused (1terminal). Match case-insensitively.
Change-Id: Iddabdbd66597a7325bdb08cb31434c1ee710dc3e
greetd ran the niri session directly, so exiting the session (e.g.
logout from the lock screen) made greetd immediately restart the
desktop with no authentication.
The broad 'file /** rwlkUx' rule conflicts with rix exec rules pulled in
via nested abstractions (xdg-open): AppArmor 5.0 rejects merged rules
with differing exec modifiers, failing apparmor.service on first load.
Compressed writeback cache in front of the disk swapfile keeps cold anon
pages in RAM instead of the SSD, freeing page cache for the ~78GB mmap'd
LLM weights. zstd's ratio keeps ~5G more anon bytes in the capped pool
than lz4 at negligible cost (CPU peaks ~30% on this host); pool capped at
10% so it never competes with the weight cache.
The zram module was auto-imported on all hosts via base scanPaths, and its
swappiness=180/page-cluster=0 tunings leaked onto hosts that disabled zram
(e.g. idols-ai with a disk swapfile, pushing cold anon pages to disk swap).
Wrap the whole module in modules.zram.enable (default true); idols-ai now
turns off zram and its tunings with a single switch.
* security: make firewall secure-by-default, disable explicitly on servers
Flip the base firewall default to ON so new hosts are protected by default. Servers keep the firewall off (trusted internal LAN, WAN protected at the router) via explicit overrides in modules/nixos/server/{server,server-aarch64}.nix. Behavior-preserving for all 18 current hosts; adds a security-firewall eval test guarding the per-host state.
* test: fix security-firewall eval test to scalar per-host values
* security: enable AppArmor (complain) on all Linux hosts
* security: bind aquamarine metrics exporters to loopback
* security: restrict k3s kubeconfig file mode to 600
* security: disable SSH X11 forwarding on servers, keep on desktops
* security: add conservative systemd hardening to aquamarine services
* docs(apparmor): correct FHS-alias comment (nixpkgs has no broad FHS layer)
nixpkgs only creates /run/current-system, /usr/bin/env and /bin/sh; it does not
provide a broad FHS->store alias tree, so FHS-oriented abstractions are incomplete
on NixOS. They are safe only because the profiles run in complain mode.
* fix(aquamarine): scrape same-host exporters over loopback
The v2ray/postgres/sftpgo exporters were bound to 127.0.0.1 but VictoriaMetrics
scraped them via the host's routable IP, so those scrapes refused connections.
Point the three same-host scrape targets at 127.0.0.1 (VM runs on the same host).
node-exporter keeps the host IP since it binds 0.0.0.0.
* Revert "security: add conservative systemd hardening to aquamarine services"
This reverts commit f9cf99dadb.
xwayland-satellite has no compositor-level clipboard bridging, so X11
clients (WeChat/QQ) and Wayland clients cannot see each other's clipboard.
Add the pyclipsync flake input and enable its home-manager module, which
installs the daemon as a systemd user service bound to the graphical session.
Lanzaboote's nested rust-overlay input (2026-06-21) used the now-deprecated
stdenv.isLinux/stdenv.isDarwin aliases in mk-aggregated.nix, triggering two
evaluation warnings on nixpkgs 26.11 (where these aliases gained lib.warn).
rust-overlay fixed this treewide in 892c035d (2026-08-13); update the nested
input to master (f60c1b57, 2026-08-23).
* feat(nushell): add trash command using the home trash can
On a tmpfs root with persistent dirs bind-mounted in, `rm --trash`
scatters items into per-mount .Trash-$uid dirs that file managers never
show, and gio refuses to trash on those internal mounts.
Add a `trash` command implementing the freedesktop trash spec
'failsafe' mode: items are moved to ~/.local/share/Trash with
spec-compliant .trashinfo entries, so Thunar can list and restore them.
Also preserve ~/.local/share/Trash on idols-ai so the trash survives
reboots.
* feat(nixos): manage scattered trash dirs with trash-cli retention timer
Replace the hand-written nushell trash command with the established
trash-cli tooling. On a tmpfs root with persistent dirs bind-mounted
in, the trash crate (nushell rm --trash) scatters items into per-mount
.Trash-$uid dirs that file managers never show and nothing cleans up.
- add trash-cli to system packages: trash-list scans/trash-restore
handles every mount point's trash dir
- daily systemd timer runs 'trash-empty 30 -f', purging items older
than 30 days across the home trash and all mount points
- drop the hand-written trash.nu module
- note in preservation.nix: do NOT persist ~/.local/share/Trash; a
bind-mounted trash dir breaks the trash crate's home-topdir match
for files straight under $HOME (it would try /.Trash-$uid, EACCES)
* fix(nixos): order trash-empty after preservation.target
The scattered .Trash-$uid dirs live inside the preservation bind
mounts, which use DefaultDependencies=no and are therefore NOT ordered
after local-fs.target. Add preservation.target to After= so trash-empty
only runs once all those mounts are up.
`use modules/argx *` glob-imports argx's `parse` command into the global
scope, shadowing nushell's builtin `parse`. Any script sourced afterwards
that relies on the builtin (e.g. fzf's nushell integration, which uses
`parse --regex`) then fails to parse with `unknown flag`.
Use prefixed imports for `argx` and `lg` instead of `*`: both modules are
still registered for the kubernetes module's submodules (which call
`argx parse` / `lg level`), there are no bare `parse`/`main`/`level` call
sites, and the builtin `parse` stays available.
`lsa` (ls -al) and `dus` (du) render the table at a fixed width of 350 so
columns like size/apparent/physical are never dropped to "..." when the
natural table is wider than the terminal.
At login the xdg-autostart-generator starts apps at
graphical-session.target, racing the portal user services. Sandboxed
apps (nixpak firefox/telegram) end up with broken FileChooser/OpenURI
until manually restarted.
Add a template drop-in on app-@autostart.service that makes every
autostart app wait for the portal stack.
- Add an offlineHosts list to VictoriaMetrics scrape generation: SBCs
and the whole k3s-prod-1 cluster are powered off, shoukei's exporter
is disabled on the machine; also comment out the dnsmasq-exporter
job (runs on suzi).
- Disable node-exporter on shoukei: a laptop on untrusted networks
should not expose :9100 (the firewall is off repo-wide).
- Route severity none|info alerts to the null receiver; meta alerts
and info noise should never page.
Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
node-exporter always reports device_error=1 for containerd sandbox shm
bind mounts (tmpfs), producing ~50 bogus critical alerts on the
kubevirt hosts. Verified against live metrics that no non-shm device
errors exist, so nothing real is filtered out.
Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
The NixOS alertmanager module pipes the generated config through
envsubst to inject secrets, which silently replaced the template's
$i/$a variables with empty strings, so every telegram notification
failed with a template parse error since Friday's deploy.
Rewrite the template without variables: range + dot, define/template
for the per-alert block, and an if/else split to cap a group at 5
alerts (slice errors when fewer than 5). Verified with amtool template
render against post-envsubst content for both branches.
Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
Fix inconsistent volume across bilibili videos at the PipeWire level:
the easyeffects daemon auto-starts at login; an autogain + limiter
chain can be applied per-app (e.g. only to the browser).
Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
- Show group-level labels (alertgroup/cluster/env/namespace) once in
the header instead of repeating them per alert.
- Use each rule's summary annotation as the per-alert headline (it
already carries the distinguishing resource name), falling back to
the instance label; show nodename on its own line when present.
- Drop the noisy per-alert label dump and the always-N/A value line.
Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
- Trim the telegram template: drop the full label dump, render at
most 5 alerts per group, and note how many were omitted, so
messages stay within Telegram's 4096-char limit.
- Route severity=none meta alerts (Watchdog, InfoInhibitor) to a new
null receiver so they no longer notify.
Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
Setting nix.settings.sandbox-paths replaces Nix compiled sandbox defaults. Keeping only /dev/net made local builds lose the sandbox shell that normally provides /bin/sh, which broke upstream scripts with /bin/sh shebangs in packages such as X11-fonts and openldap.
Use extra-sandbox-paths for /dev/net instead. This keeps Nix daemon defaults intact, including the configured sandbox shell, while still allowing builds that need /dev/net.
Add eval coverage for both Linux output sets so future changes keep /dev/net in extra-sandbox-paths and do not reintroduce an explicit sandbox-paths override.
The pinned lanzaboote module still defined boot.bootspec.enable, but nixpkgs now always generates bootspec data and rejects attempts to set that removed option.
Update the lanzaboote input to a revision compatible with the new nixpkgs module behavior.
Nixpkgs removed services.kmscon.fonts and services.kmscon.extraConfig, so the old font module failed evaluation on current nixos-unstable.
Move the selected font, font size, and hardware acceleration settings into services.kmscon.config while keeping the existing terminal type option.
Explicitly set programs.nixvim.nixpkgs.source to suppress the follows
warning, and migrate all programs.ssh.matchBlocks to programs.ssh.settings
using upstream PascalCase directive names.
* chore(home): drop AstroNvim bundle and centralize editors
Remove the bundled Neovim plugin tree to shrink supply-chain exposure; keep a
minimal programs.neovim backup. Daily editing stays Helix-first with Yazi and
Zellij.
- Add Helix/Neovim docs, glossary, and cheatsheets under core/editors/.
- Default EDITOR/VISUAL to hx; use SUDO_EDITOR nvim --clean for sudoedit and
other sensitive edits; adjust Nushell buffer_editor accordingly.
- Apply Helix Home Manager settings (keys, editor UI) in core/editors/helix.
- Keep heavy language-server tooling only in home/base/tui/editors/packages.nix;
wire it via an explicit default.nix import so core stays lightweight.
- Extend modules/base packages.nix where that profile sets global editor env.
* docs: link editors, VS Code, and agents from README
Document tui/editors versus core/editors; add root README pointers to GUI
vscode (Home Manager) and agents/. Cross-link heavy packages README from
core/editors.
feat: luks automatic unlock via tpm2 chip for all physical hosts
feat: enable login manager
fix: Rename network interface config back following hardware change
* feat: comment out nixpkgs-unstable, we use unstable by default
Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
* fix: typo...
---------
Signed-off-by: Ryan Yin <xiaoyin_c@qq.com>
* feat: replace impermanence with preservation
* fix: This module cannot be used with scripted initrd.
* fix: permission for /var/lib/private
* chore: boot.initrd.systemd.enable
* docs: boot.initrd.luks.devices.<name>.fallbackToPassword is implied by systemd stage 1
* fix: typo
- Changed "checkout" to "check out."
- Corrected "hardwares" to "hardware."
- Fixed the typo "deplyment" to "deployment."
- Added a space in "ISO image(purest installation method)."
| accidental deletion, bad edit | btrbk: local btrfs snapshots |
| disk or host loss | restic: encrypted copies on youko |
| a compromised backup server | desktop repositories use a password youko never holds |
| a compromised host | youko takes hourly read-only btrfs snapshots of the REST store, which a desktop cannot reach; the immutable cloud copy (planned) is still the offsite layer |
## Layers
1.**btrbk — local snapshots** ([`modules/nixos/base/btrbk.nix`](./modules/nixos/base/btrbk.nix)).
Same-filesystem btrfs snapshots of `@persistent`, run on Tue/Sat 03:45:20, kept for 7 days
(minimum 2). Cheap rollback for mistakes; it does **not** survive disk loss.
| **Filesystem & Encryption** | tmpfs on `/`, [Btrfs][Btrfs] subvolumes on a [LUKS][LUKS] encrypted partition for persistent, unlock via passphrase | tmpfs on `/`, [Btrfs][Btrfs] subvolumes on a [LUKS][LUKS] encrypted partition for persistent, unlock via passphrase |
| **Filesystem & Encryption** | tmpfs as `/`, [Btrfs][Btrfs] subvolumes on a [LUKS][LUKS] encrypted partition for persistent data; unlock via passphrase |
| Administration | Key-based SSH; root key login retained for Colmena/MicroVM deployment | Deployment credentials are root-equivalent; password login being disabled does not restrict stolen keys |
| Host network | Shared nftables firewall trusts the home LAN, tailnet interface and local Podman bridge; IPv6 link-local is accepted | These are broad trust zones, not service-level least privilege; router exposure and tailnet ACLs require separate verification |
| Monitoring | node/smartctl exporters bind the host LAN IPv4 (9100/9633); firewall rules restrict exporter ports 9100, 9835 and 9633 to the monitoring host over IPv4 | nvidia_gpu_exporter (9835) still binds wildcard; the IPv6 denial for those three ports is in the shared firewall; current running hosts require separate verification |
| VM hosts | shoryu, shushou and youko run guests behind `br0`; ruby/kana are libvirt guests | A bridge is not network isolation; host compromise affects its guests |
| k3s | Three control-plane and three worker MicroVMs; Cilium, API VIP and NFS CSI | Cluster-admin, privileged/host-mounted pods and writable NFS data are powerful trust boundaries |
| MicroVM storage | Host store shared read-only through virtiofs; guest `/etc`, `/var` and `/home` persisted in separate images | Read-only store does not isolate the host-side virtiofs implementation or protect writable guest state |
| Desktop apps | Selected apps use nixpak/bubblewrap; Podman is rootless | Rules and shared directories vary per app; user namespaces are needed for these sandboxes |
| Secrets | agenix declarations here, encrypted data in the private nix-secrets repository | Recipients and plaintext file permissions must be reviewed; root on a recipient can read its decrypted secrets |
| Supply chain | Locked flake inputs; explicit binary-cache trust keys; normal user is not a Nix trusted-user | A trusted cache or imported Nix module can supply code executed as root; a lock file is not a security audit |
| Recovery | NixOS generations, btrbk/restic, hourly read-only snapshots of the REST store, and documented backup procedures | A rollback does not undo compromised data or secrets; backup configuration alone does not prove restore works; the offsite immutable copy is still planned |
| `kernel.dmesg_restrict` | 1 | Kernel log access requires CAP_SYSLOG; journal access has separate permissions |
| `kernel.kptr_restrict` | 2 | Hide pointers printed with `%pK`, even from privileged readers; not a blanket ban on address leaks |
| `kernel.yama.ptrace_scope` | 1 | Restrict unrelated-process tracing; child-process debugging and explicit PR_SET_PTRACER exceptions remain possible |
| `kernel.perf_event_paranoid` | 2 | Restrict unprivileged kernel profiling; user-space profiling remains available; do not assume distro-specific 3/4 semantics |
| `kernel.unprivileged_bpf_disabled` | 2 | Disable unprivileged `bpf()` with an administrator-reversible setting; privileged Cilium/agents retain access |
| `net.core.bpf_jit_harden` | 1 | Harden only unprivileged BPF JIT; do not enable JIT blinding for privileged Cilium/observability workloads |
| `vm.unprivileged_userfaultfd` | 0 | Restrict unprivileged kernel-mode fault handling, not all userfaultfd usage; `/dev/userfaultfd` permissions are a separate boundary |
| `fs.protected_symlinks`, `fs.protected_hardlinks` | 1 | Mitigate cross-user link attacks; these controls are boolean |
| `fs.protected_fifos`, `fs.protected_regular` | 2 | Restrict unsafe O_CREAT access in world/group-writable sticky directories; shared temporary-file workflows may need adjustment |
Some values already come from kernel/systemd/NixOS defaults. Explicit declarations make the desired
baseline reviewable; they do not imply the previous system had no protection. Unsupported kernel
features may lack their sysctl paths, especially on custom kernels: runtime verification is
required.
JIT hardening is deliberately limited to unprivileged programs rather than forced on privileged
agents. Together with `unprivileged_bpf_disabled=2`, this preserves Cilium's normal privileged BPF
path while retaining protection if an administrator temporarily permits unprivileged BPF. This is a
configuration trade-off, not a measured claim of zero overhead.
User namespaces remain enabled: disabling them globally conflicts with Nix sandboxing and breaks
rootless containers and application sandboxes. Nonprivileged seccomp filters are not disabled by the
unprivileged `bpf()` restriction. Do not disable Cilium's privileged BPF, tc, container networking,
NFS, or required VM/filesystem modules in pursuit of a larger deny list.
The existing `esp4`, `esp6`, `rxrpc` deny list blocks modprobe/autoload paths. It does not unload
already loaded modules, affect built-in code, or stop a privileged direct module insertion. Verify
an advisory and affected kernel versions before adding CVE claims; search snippets are not evidence.
### Boot integrity and mandatory access control
- ai configures Lanzaboote in [secureboot.nix](hosts/idols-ai/secureboot.nix). Confirm firmware key
enrollment and actual Secure Boot state with `sbctl status`; do not assume all hosts have it.
- Secure Boot, signed boot images, kernel lockdown and module signature enforcement are distinct
controls. Test custom kernels and NVIDIA/out-of-tree modules before extending enforcement.
- AppArmor is enabled, but the explicit sudo policy is in complain mode and the default-deny/nix
scaffolds are disabled. Complain mode logs rather than blocks. Existing FHS-oriented abstractions
need Nix store path review; this is not broad enforced application confinement.
- Systemd service sandboxing is per-service, not a global policy. Review capabilities, writable
paths, network access and syscall requirements before tightening an individual service.
See [application hardening](hardening/README.md) for wrapper details.
## Updating, deploying and recovering
1. Check the advisory and affected package/kernel; prioritize active exploitation and reachable
attack paths. Review changed inputs and cache trust instead of treating updates as inherently
safe.
2. Run `just test` (must pass), build affected systems/runners and review closure differences.
Evaluate tests cannot prove runtime enforcement or workload compatibility.
3. Record the old system generation/runner and ensure backups and recovery access exist. Use `boot`
mode for bridge/network-stack changes or broad VM-host updates.
4. Deploy MicroVM closures to the **physical host**, not the guest's read-only store. Install the
reviewed runner and activate/restart through the documented MicroVM path. Preserve all images.
5. Keep control-plane restarts sequential to retain etcd quorum. Worker concurrency is an explicit
availability trade-off; simultaneous worker restarts can interrupt all worker workloads.
6. Verify the active/booted generation, guest boot ID, failed units, API VIP `/readyz`, Nodes,
Cilium, CSI and workloads. Updating a kernel on disk does not patch the running kernel: reboot
and verify.
7. On failure, restore the recorded generation/runner and recheck health before continuing. Preserve
rollback roots until stable. If compromise is suspected, isolate first, rotate affected secrets,
and restore verified data; merely switching generations is not incident recovery.
For the sysctl baseline, query the keys above with `sysctl <key>`, check
`journalctl -b -u systemd-sysctl`, and test rootless Podman, a sandboxed Nix build, desktop
sandboxes, debugging, and cluster networking/storage after activation. Do not globally relax
hardening to hide one application's incompatibility: make a narrow documented exception and test it.
These sysctls are reversible by administrator configuration, unlike `unprivileged_bpf_disabled=1` or
`kexec_load_disabled=1`, which cannot be relaxed without rebooting. This baseline does not enable
the latter or `security.protectKernelImage` (which also disables hibernation).
## API endpoint and TLS
The canonical endpoint is **`https://test-cluster-1.writefor.fun:6443`**. DNS points it to
kube-vip's `192.168.5.193`; the three servers include the DNS name in their serving-certificate
SANs, and the joining servers/agents use that domain. The first server initializes the cluster
without a join address. This is already a domain-based HA endpoint, not a bare-IP client design.
Use the existing authenticated client configuration to verify it:
```sh
k3s kubectl --server=https://test-cluster-1.writefor.fun:6443 --request-timeout=15s get --raw=/readyz
```
Bare-IP access may fail TLS hostname verification because the certificate does not include the VIP
IP SAN. That is not a fault in the canonical endpoint and does not require certificate rotation. If
bare-IP disaster-recovery access becomes an explicit requirement, add that SAN through normal
configuration and certificate lifecycle procedures. Never bypass TLS verification to conceal a
hostname mismatch. Eval tests protect the domain join endpoint and server SAN declarations; live TLS
checks remain necessary after deployment.
## Implementation status and completed audit findings
| P1 | Roll out and verify the implemented baseline/exporter rules | Runtime sysctl readback, allowed IPv4 scrape and denied IPv6/non-monitoring scrape tests; preserve Nix/Podman/Cilium/NFS functionality |
| P1 | Review router mappings and tailnet ACLs; narrow service exposure where warranted | Audit was limited to sampled host listeners; test intended and denied access without locking out deployment |
| P1 | Review NFS client scope and root identity requirements | Restrict to required VM hosts/CSI nodes where practical; test provisioning, permissions and existing PVCs before changing squash semantics |
| P1 | Narrow operator/secret access where justified; finish permission checks | Selected effective probes and ai/youko modes audited; validate broad secret-reader needs, remaining hosts and recipient scope; Darwin per-file ownership is now explicit and needs a runtime check |
| P1 | Adopt the documented cadence and verify the booted kernel after rollouts | Routine documented; compare booted and selected kernel paths and recheck API/workloads after reboot; no automated metric is deployed |
| P1 | Exercise backup restore and credential rotation | Use an approved isolated restore target and recovery access; never overwrite live data for a drill |
| P2 | Maintain ai boot integrity and assess other host roles | Review sbctl configuration migration, recovery boot and signed custom/NVIDIA modules; ai Secure Boot state already verified |
| P2 | Review unused-module deny list against real workloads and verified advisories | Check loaded/built-in modules and autoload dependencies; do not block required filesystems, WiFi, VM or Cilium/NFS functionality |
| P2 | Evaluate stronger controls only against a concrete threat | Account for hibernation, crash recovery, applications and performance; no blanket io_uring/SMT/userns ban or hardened-kernel switch |
| P2 | Validate narrower sandbox/agent boundaries where needed | Source and selected SSH/journal metadata audited; use disposable data/socket probes and preserve core-desktop diagnosis, browserpass and Homelab root workflows |
| P3 | Improve security-event alerts and configuration drift checks | Actionable redacted alerts and periodic runtime control readback |
Record temporary exceptions in [WORKAROUNDS.md](WORKAROUNDS.md) with a removal condition. Revisit
this architecture after a new exposed service, network/storage change, host addition or significant
kernel/nixpkgs update. No control here guarantees containment after host root compromise.
| WA-001 | smartctl_exporter pinned to upstream `6274decc`, which carries PR [#329](https://github.com/prometheus-community/smartctl_exporter/pull/329) — `overlays/smartctl-exporter/` | `/metrics` returns HTTP 500 once a device becomes readable after startup ([#305](https://github.com/prometheus-community/smartctl_exporter/issues/305), [#326](https://github.com/prometheus-community/smartctl_exporter/issues/326)) | Drop once nixpkgs provides a version containing the #329 fix | 2026-10-03 | next nixpkgs bump | active |
| WA-002 | `nixpkgs-2505` — `flake.nix`; consumer `home/base/tui/container.nix` | `kubernetes-helm` regressed in newer nixpkgs | Drop the input and its consumer once helm works again in current nixpkgs | 2025-12-03 | when helm is revisited | active |
| WA-003 | `rust-overlay` follow — `flake.nix` | lanzaboote v1.1.0's pinned rust-overlay uses the deprecated `stdenv.isLinux`/`stdenv.isDarwin` | Drop the follow once lanzaboote updates its pin (or we drop the lanzaboote pin) | 2026-09-25 | on lanzaboote bump | active |
| WA-004 | `catppuccin` on `main` — `flake.nix` | `main` carries the nodejs→nodejs-slim deprecation fix, which is not in a release | Switch to a release tag once it includes the fix | 2026-09-25 | on catppuccin release | active |
| WA-005 | `tuios` input — `flake.nix`; `home/base/tui/tuios/` | nixpkgs lags behind upstream | Use nixpkgs' `tuios` once it is recent enough | 2026-10-03 | on nixpkgs bump | active |
| WA-006 | `nixpkgs-blender` — `flake.nix` | Blender 5.2 LTS + CUDA/OptiX pin; keeps `just up` from triggering a full source rebuild | Bump deliberately when needed; drop the pin if the current nixpkgs build is acceptable | 2026-07-24 | deliberate | active |
| WA-007 | WeChat from `nixpkgs-master` — `hardening/bwraps/default.nix` | newest builds; the pinned channel lags | Fall back to the stable channel once it has a good build | 2026-10-03 | on nixpkgs bump | active |
| WA-020 | `fcitx5-vinput` pinned to release tag `v2.4.0` — `flake.nix`; consumer `home/linux/gui/base/vinput/default.nix` | Tracks a release tag instead of a branch, so `just up` stays on that release | Bump to a newer release tag when its changes are wanted | 2026-10-04 | on vinput release | active |
| WA-021 | `nix-x86-on-aarch64` input — our fork of rowanG077's patched FEX + muvm runtime — `flake.nix`; `hosts/12kingdoms-shoukei/game/` | FEX does not support 16K pages, so x86/x86_64 apps and games on shoukei need this pinned flake's 4K-page microVM runtime; our fork lets upstream changes be reviewed before use | Drop once FEX supports 16K pages directly, or nixpkgs/nixos-apple-silicon ships an equivalent runtime | 2026-10-07 | on FEX/nixpkgs/muvm update | active |
| WA-023 | `nix.package = pkgs.nixVersions.latest` — `modules/nixos/base/nix.nix` | NixOS is opted into Nix 2.35 (2.35.2), which nixpkgs ships as `nixVersions.latest`, while `nixVersions.stable` is still 2.34 | Drop the override and fall back to the default `pkgs.nix` once `nixVersions.stable` is 2.35 or newer | 2026-10-07 | next nixpkgs bump | active |
| WA-037 | `nixpkgs-npu` input pinned to `2ddccb854d89` — the `openvino-npu-compiler` branch of `ryan4yin/nixpkgs`, carrying [NixOS/nixpkgs#570230](https://github.com/NixOS/nixpkgs/pull/570230) — `flake.nix`; consumer `pkgs-npu` in `outputs/default.nix`, `hosts/idols-aquamarine/asr.nix` | The Intel NPU compiler, and the driver change that loads it, are unmerged, so the NPU driver the Whisper service needs comes from that branch | Drop the input and load `intel-npu-driver` from `nixos-unstable` once #570230 merges | 2026-10-17 | when #570230 merges | active |
## Workarounds
A setting that works around a bug or a hardware quirk.
| ID | What & where | Why | Removal condition | Added | Revisit | Status |
| WA-008 | Relaxed NVMe APST (`nvme_core.default_ps_max_latency_us=100000`) — `hosts/idols-ai/hardware-configuration.nix`, `hosts/idols-aquamarine/hardware-configuration.nix` | the KINGBANK KP260 (DRAM-less QLC) froze on slow APST wake-ups | Drop the override once kernel/firmware fixes or replacement hardware make default APST reliable | 2026-10-03 | watch NVMe timeout/reset/AER in the journal | active |
| WA-009 | USB-SATA bridge mitigations (`usbcore.autosuspend=-1`, `usb-storage.delay_use=10`) — `hosts/12kingdoms-youko/default.nix` | flaky JMicron JMS567 resets with UDMA-CRC and I/O errors | Drop once the bridge/controller is replaced; the signal is smartctl CRC/media errors | 2026-09-21 | on hardware change or CRC errors | active |
| WA-010 | Insta360 Link USB reset on S3 resume — `hosts/idols-ai/default.nix` | the camera stays enumerated with a stalled UVC endpoint after resume | Drop after the Insta360 firmware or the kernel `uvcvideo` fixes it | 2026-08-01 | on kernel or firmware change | active |
| WA-011 | Kernel module blacklist (`esp4`, `esp6`, `rxrpc`) — `hardening/kernel-hardening.nix` | attack surface for the Dirty Frag LPE (CVE-2026-43284 / CVE-2026-43500) | Kept as defence in depth; drop only if something needs those modules | 2026-10-03 | rarely | active |
| WA-017 | keyd right-hand modifier passthrough — `modules/nixos/desktop/peripherals.nix` | keyd folds right Ctrl/Shift/Meta into their left keys | Drop once keyd or fcitx5 keeps right-hand modifiers distinct | 2026-10-05 | on keyd bump | active |
| WA-018 | user-owned `~/.ssh/config` + disabled systemd ssh-proxy include — `home/base/tui/ssh.nix`, `modules/nixos/base/ssh.nix` | bwrap exposes root-owned store files as `nobody` | Drop once sandboxes preserve store ownership | 2026-10-06 | when sandbox layout changes | active |
| WA-019 | programs.steam.extraCompatPackages = [ pkgs.dwproton-bin pkgs.proton-ge-bin ] — modules/nixos/desktop/gaming.nix | ACE games fail on GE/mainline Proton, work with DW-Proton | Drop once mainline or GE-Proton runs these games (GE-Proton#433) | 2026-10-07 | on Proton/GE bump | active |
| WA-024 | Wi-Fi power saving disabled (`networking.networkmanager.wifi.powersave = false`) — `hosts/12kingdoms-shoukei/apple-silicon.nix` | On this BCM4378/brcmfmac laptop, powersave is a common source of intermittent ASSOC-REJECT / authentication-timeout failures; NetworkManager then mistakes the failure for a bad key, re-prompts, and the client (Noctalia, nmtui, nmcli) saves the answer as a duplicate `<SSID> 1` profile | Drop once the kernel carries the brcmfmac `port_authorized` fix (WA-025) and no association failures remain | 2026-10-07 | watch `journalctl -u wpa_supplicant` for ASSOC-REJECT / authentication timeout | active |
| WA-026 | guest deploys run `installOnHost` then `systemctl restart microvm@<guest>` on the physical host — `Justfile`, `lib/genMicrovmGuestModule.nix` | the sshSwitch step runs `nix-store --load-db` against the guest's read-only virtiofs `/nix/store` and dies on `cannot remount "/nix/store" writable` (nix 2.35) | Drop the manual path once microvm.nix or nix handles a read-only store switch. A host activation re-links `/var/lib/microvms/<guest>/current` to the runner baked into the host's system, so switch the host after installing a guest from a newer tree | 2026-10-09 | on microvm.nix or nix bump | active |
| WA-028 | GitHub SSH pinned to IPv4 (`AddressFamily = "inet"`) — `home/base/tui/ssh.nix` | a v6 fake AAAA for `ssh.github.com` from the `2001:2::/64` pool (WA-029) is not proven on the 443 SSH tunnel yet | Drop once the v6 fake-ip pool is proven for GitHub SSH | 2026-10-10 | on a mihomo bump or a v6 pool change | active |
| WA-029 | mihomo `fake-ip-range6` is `2001:2::/64`, not the shipped ULA default — `modules/nixos/desktop/networking/mihomo/generate.nu` | the default `fdfe:dcba:9876::/64` sits in `fc00::/7`, which Local Network Access calls local, so a public page loaded over the v4 fake pool and pulling a dual-stack subresource prompts for local-network access (Chrome 154, Firefox). The same range also stalled ssh: a session to a fake AAAA authenticated and then never closed, on both `gvisor` and `mips`. `2001:2::/48` is RFC 5180 benchmarking -- the v6 twin of the `198.18.0.0/15` v4 default, absent from that table -- and both symptoms went with the move | Go back to the upstream default once mihomo ships a pool outside the LNA non-public table; the ULA default brings the ssh stall back | 2026-10-09 | on a browser or mihomo bump -- that table already grew to cover `2001:db8::/32` and `198.18.0.0/15` | active |
| WA-030 | mihomo unit sets `SKIP_SYSTEM_IPV6_CHECK=1` — `modules/nixos/desktop/networking/mihomo/default.nix` | mihomo checks at startup whether another NIC already holds an IPv6 address and, finding none, skips the TUN's v6 half -- but `strict-route` still installs its v6 blackhole (`ip -6 rule add priority 9000 unreachable`). On a cold boot that rule lands before networkd's Router Solicitation is answered, so the host can never send RS/NS, never gets a GUA, and mihomo never re-checks: self-locking until the core restarts. The env var forces the TUN v6 address up, which installs mihomo's normal v6 rules whose `lookup main suppress_prefixlength 0` keeps on-link neighbor discovery and the RA default route off the TUN | Drop once mihomo either re-runs the IPv6 check or skips the v6 strict-route blackhole when the v6 TUN is disabled | 2026-10-10 | on a mihomo bump | active |
| WA-031 | `iommu=pt` kernel parameter — `hosts/idols-aquamarine/hardware-amd-gpu.nix` | ROCm's system requirements warn that multi-GPU systems may need IOMMU passthrough to prevent application hangs, and this host has two Radeon AI PRO R9700 cards. Added before the first inference run; not yet observed on gfx1201. | Drop once a ROCm release states that the default IOMMU mode is safe for multi-GPU gfx1201, or when the host runs a single GPU | 2026-10-13 | on a ROCm bump, or on the first hang report | active |
| WA-032 | `amdgpu.runpm=0` kernel parameter — `hosts/idols-aquamarine/hardware-amd-gpu.nix` | this board's DSDT advertises ATPX power control (`\_SB.PC00.GFX0.ATPX`) but never defines the `PEG0.PEGP._OFF`/`_ON` symbols its body calls, so the first idle resume of the headless R9700 leaves it in `runtime_status = error` (`resume of IP block <smu> failed -62`) until reboot, and amdgpu discards the ACPI error. Root cause, kernel-source trail and the measured field comparison: `hosts/idols-aquamarine/ROCM.md`. | Drop once a BIOS update defines those ACPI methods (upstream releases 1023/1066 exist for this board; a flash changes PCR7, so re-enroll TPM2), or a kernel release stops trusting ATPX when its power methods are missing | 2026-10-14 | on a BIOS update or kernel bump | active |
| WA-036 | catppuccin eval-time ports kept in the home closure (`home.extraDependencies`) — `home/base/core/theme.nix` | catppuccin/nix reads some ports at eval time (`lib.importTOML "${sources.starship}/<flavor>.toml"`) and inlines the result, so the built port, catppuccin's build/install hooks and the fetched port source stay unreachable: every gc deletes them, and the deploy after it rebuilds them (build logs: 2026-10-16). `home.extraDependencies` records the port in the generation, which is a gc root. Also read at eval time upstream: `bottom`, `element` (option `element-desktop`), `firefox`, `rio`, `palette` (options `skim`, `tty`), `spotify-player`. | Drop once catppuccin/nix stops reading port files at eval time, or those ports come from a binary cache | 2026-10-16 | on a catppuccin bump | active |
| WA-039 | niri's NVIDIA free-buffer pool bounded (`GLVidHeapReuseRatio` 0) — `hosts/idols-ai/hardware-nvidia.nix` | the driver does not return freed vidmem buffers to its free pool, so niri's VRAM grows to ~1.1 GiB instead of the ~100 MiB its NVIDIA notes expect (measured 1137 MiB after 24 h uptime; [NVIDIA/egl-wayland#126](https://github.com/NVIDIA/egl-wayland/issues/126)) | Drop once a driver stops growing niri's VRAM without the profile, or niri ships its own default | 2026-10-19 | on an nvidia driver bump | active |
If WA-008 freezes recur, the fallback is to restore the parameters commented out in
`hosts/idols-ai/hardware-configuration.nix` —
`nvme_core.default_ps_max_latency_us=0 nvme_core.io_timeout=4294967295 pcie_aspm=off`; this is a
recovery procedure, not its removal condition.
If Wi-Fi wedges after a failed association, restart the stack instead of rebooting:
`sudo systemctl restart wpa_supplicant`, then `sudo systemctl restart NetworkManager`. Deleting an
_active_ NetworkManager profile wedges the supplicant as well; switch to another profile first, and
only delete inactive duplicates.
## Known issues
A known limitation with no fix yet.
| ID | What & where | Why | Removal condition | Added | Revisit | Status |
| WA-016 | subscription rule snapshots — `modules/nixos/desktop/networking/mihomo/` (`rules_from:`) | mihomo's `proxy-providers` imports proxies only, so the `rules` and `proxy-groups` a subscription ships with are dropped by the core; the generator re-imports them from a local YAML copy that has to be refreshed by hand when the provider changes its routing policy | Drop the snapshots once mihomo can import rules from a proxy-provider, or the providers publish their rules as rule-providers | 2026-10-04 | when mihomo gains rule import | active |
| WA-022 | qemu-user binfmt removed — `hosts/12kingdoms-shoukei/hardware-configuration.nix` | the x86-on-arm module owns the x86 ELF handlers and this host does not need riscv64, but dropping `boot.binfmt.emulatedSystems` also drops `nix.settings.extra-platforms` and qemu-user, so `podman run --platform=linux/amd64`, x86 chroots and local builds of uncached x86 derivations need a remote x86_64 builder | Restore `boot.binfmt.emulatedSystems` (without x86_64/i686, which the module asserts against) if such a workload is needed | 2026-10-07 | when an amd64 container/chroot or uncached x86 build is needed | active |
| WA-025 | brcmfmac 4-way-handshake offload authentication — upstream kernel/wpa_supplicant, no code in this repo | Since hostap commit 416386060, wpa_supplicant 2.11+ expects `NL80211_CMD_PORT_AUTHORIZED` from drivers that offload the 4-way handshake; brcmfmac never reports it, so authentication intermittently fails on Apple Silicon (BRCM4378/4387/4388). nixpkgs already carries the Broadcom revert in wpa_supplicant 2.12; the proper fix, Janne Grunau's `wifi: brcmfmac: cfg80211: Report port_authorized for 4-way HS offload` (2026-09), is still under review. Entering the key again after such a failure only creates a duplicate profile, so cancel and retry instead | Fix lands via a nixpkgs kernel update (or by carrying the patch through `nixpkgs-patched`) | 2026-10-07 | on every kernel bump | active |
| WA-033 | USB-C/XHCI port `_PLD` ACPI noise — this board's DSDT, no code in this repo | every boot logs 16 `ACPI Error: Aborting method \_SB.PC00.TXHC.RHUB.SS01._PLD ... (AE_NOT_FOUND)` lines, plus `\_SB.PC02.XHCI.RHUB.HS01._PLD`; the ports work, and these messages are unrelated to the GPU's ATPX failure, which names `\_SB.PC00.GFX0.ATPX` instead | Drop if a BIOS update defines the `_PLD` descriptors | 2026-10-14 | on a BIOS update | active |
| WA-034 | IPv6-first resolution is dead on the wired link — `hosts/idols-aquamarine/default.nix` (`networking.getaddrinfo.precedence`) | the proxy gateway answers DNS in fake-IP mode and its IPv6 path is dead: an AAAA lookup returns a fake address whose connection fails at once (measured: `curl -6` fails in 8 ms, `curl -4` answers in 0.7 s). glibc clients are covered by the RFC 6724 table with the IPv4-mapped row raised; nix's libcurl and other non-glibc resolvers ignore `/etc/gai.conf`. Disabling IPv6 with `boot.kernel.sysctl` did not work here — networkd re-enables it on the wired link for `DHCP = "ipv6"`/`IPv6AcceptRA` — so that setting was dropped on 2026-10-14 | Drop once the gateway serves proxied names over IPv6 as well (the adjacent fix is `dns.ipv6: false` in mihomo, see WA-030) | 2026-10-14 | on a mihomo or gateway change | active |
| WA-035 | SSH retry + multiplexing to LAN peers, ARP fallback tuning — `home/base/tui/ssh.nix`, `hosts/idols-ai/neighbour-probe.nix` | the desktop reaches the LAN over a Mesh WiFi backhaul, which drops unicast to a peer whose MAC it has aged out of its forwarding table while still flooding broadcast: a capture shows seven SYNs from one socket addressed to the cached MAC with no answer, three unicast ARP probes unanswered, then a broadcast ARP answered in 0.4 ms after ~12 s of failed connects. The ssh settings retry a short connect and keep a master warm so the entry stays fresh; the neighbour-probe tunings walk the kernel to that broadcast fallback in ~1 s | Drop once the backhaul is wired or the mesh stops dropping unknown unicast | 2026-10-16 | when the backhaul changes | active |
| WA-038 | OpenVINO GPU plugin needs an `ocl-icd` loader — [hosts/idols-aquamarine/hardware-intel-igpu.nix](./hosts/idols-aquamarine/hardware-intel-igpu.nix) (`openvino-gpu` wrapper) | `/run/opengl-driver/lib/libOpenCL.so.1` comes from ROCm's clr, which provides OpenCL 2.2 symbols, while OpenVINO's GPU plugin links the OpenCL 3.0 loader (`ocl-icd`): the plugin fails to load and OpenVINO silently omits GPU from `available_devices`. The wrapper runs a command with `ocl-icd` first on `LD_LIBRARY_PATH` and `OCL_ICD_VENDORS` pinned to the Intel ICD | Drop once nixpkgs' openvino GPU plugin carries an explicit `ocl-icd` RUNPATH, or ROCm's clr ships an OpenCL 3.0 loader | 2026-10-18 | on an openvino or ROCm bump | active |
## WIP
An unfinished feature or gap.
| ID | What & where | Why | Removal condition | Added | Revisit | Status |
| WA-013 | NixOS tests — `outputs/README.md` | full-host tests still rebuild custom packages (no private cache) and need the host ssh key to decrypt agenix; the shared-firewall security-exporters VM test now runs in CI (KVM); `outputs/x86_64-linux/nixos-tests/idols-ruby.nix` is exposed as `packages.x86_64-linux.ruby-nixos-tests` but is not in `checks` or CI | Resolve when a private cache exists and tests can run per-service | 2024-03-13 | quarterly | active |
| WA-027 | NFS golden store `no_root_squash` — `hosts/12kingdoms-youko/nfs-golden.nix` | the existing PVC subdirectories are root-owned, so enabling `root_squash` would break running workloads until they are migrated; the CSI node plugin is hostNetwork and sources from the allowlisted node IPs | Drop when the PVC subdirectories are migrated and `root_squash` is enabled | 2026-10-07 | next maintenance window | active |
## Related
- Security architecture, threat model and audit status: [SECURITY.md](./SECURITY.md).
- Unmerged **nixpkgs** PRs are carried via the `nixpkgs-patched` input and the
"conventions":"Recent subjects use Conventional Commits. There is no PR template and no required disclosure or sign-off trailer. Refs trailers accept an abbreviated hash with its subject.",
"delivery":"The change summary is the complete staged and PR diff. This is an inactive paper exercise; no commits, publishing, or activation occurred."
},
"fixtures":[
{
"id":"F1-bump",
"change":"Pin worker-lib from 1.2.3 to 1.2.4; no other lines change.",
"facts":[
"Version 1.2.4 fixes retry handling under intermittent connection resets.",
"Revision aaa1111 reduced failed jobs by 35% in a Linux load test before the final pin change.",
"The final revision bbb2222 passed a retry smoke test on Linux x86_64; the load test was not repeated.",
"macOS was not tested."
]
},
{
"id":"F2-restructure",
"change":"Convert 12 prose tables to grouped lists, remove a markdown table-padding lint guard, and move duplicated activation guidance into the shared command guide. No commands or runtime behavior change.",
"facts":[
"Prettier rewrote 187 lines when only three table entries changed.",
"Tables used only for mappings remain tables.",
"The shared command guide still says the user runs local activation commands.",
"The changes affect global guidance, repo skills, and eval instructions; they share the same table-padding problem.",
"Start review with the shared command guide to check that activation boundaries survived.",
"Revision ccc3333 passed a semantic content comparison on the documentation; no runtime test was relevant."
]
},
{
"id":"F3-state-clear",
"change":"Add ExecStartPre=/bin/rm -rf %t/state to worker.service to remove stale startup state.",
"facts":[
"Every restart now deletes cached state, including pending jobs; completed jobs remain in the database.",
"Operators must drain pending jobs before restarting; removing the new pre-start command stops further deletion but cannot restore already deleted jobs.",
"Upstream issue #87 reports the stale-state startup failure; this patch is a workaround and does not resolve the upstream issue.",
"The investigation included five restart attempts and two discarded approaches.",
"Revision ddd4444 passed a stale-state restart test on Linux x86_64; no live host has been activated."
]
},
{
"id":"F4-build-rule",
"change":"Add a Python3 build rule for status.py next to its existing Python2 rule.",
"facts":[
"Consumers can reference the local Python3 rule instead of defining their own.",
"Keeping the rules together simplifies upcoming automated build-file refactoring.",
"There is no runtime state deletion, stored-format change, or host activation.",
"Revision eee5555 built the new rule on Linux x86_64; runtime behavior was not tested."
]
},
{
"id":"F5-commit-reference",
"change":"Remove the pre-start state deletion from worker.service to restore pending-job retention across restarts.",
"facts":[
"The deletion was introduced by commit 1a2b3c4, subject fix(worker): clear stale state before startup.",
"Pending jobs survive restarts again. Previously deleted jobs cannot be recovered by this fix.",
"Stale state is now rejected by the worker library, so the deletion workaround is obsolete.",
"Revision fff6666 passed the pending-job restart test on Linux x86_64; no live host has been activated."
]
},
{
"id":"F6-format-migration",
"change":"Write cache expiry timestamps in seconds instead of milliseconds; readers invalidate old millisecond timestamps.",
"facts":[
"Old cache entries are invalidated on first read; the cache repopulates automatically.",
"Deploy readers that accept the new unit before writers; rolling back writers is safe while those readers remain deployed.",
"The rollout risks extra database reads while the cache warms; no performance improvement was measured.",
"Revision aaa7777 passed old-format invalidation and new-format round-trip tests on Linux x86_64.",
Pin worker-lib to 1.2.4 to fix retry handling under intermittent connection resets. Revision bbb2222 passed a retry smoke test on Linux x86_64; macOS was not tested. The 35% reduction in failed jobs came from load testing revision aaa1111 before the final pin and was not remeasured.
```
### F2 restructure
```text
refactor(docs): group prose tables as lists
```
```text
Convert prose tables to grouped lists and move duplicated activation guidance into the shared command guide. Keep mapping tables as tables and remove the padding lint guard.
The changes share the same table-padding problem across global guidance, repo skills, and eval instructions. Review the shared command guide first to confirm the local activation boundary remains clear.
Prettier previously rewrote 187 lines for three table-entry changes. Revision ccc3333 passed a semantic content comparison on the documentation; no runtime test was relevant.
```
### F3 state-clear
```text
fix(worker): clear stale state before startup
Every restart deletes cached state, including pending jobs; completed jobs remain in the database.
```
```text
Every restart removes cached state before startup, including pending jobs; completed jobs remain in the database. Drain pending jobs before restarting. Removing the pre-start command prevents further deletion but cannot restore deleted jobs.
This works around upstream issue #87; it does not resolve it. Revision ddd4444 passed the stale-state restart test on Linux x86_64. No live host has been activated.
```
### F4 build-rule
```text
feat(build): add Python 3 rule for status.py
```
```text
Add a Python 3 build rule for status.py alongside its Python 2 rule, so consumers can reuse it instead of defining their own. Keeping both rules together also simplifies upcoming automated build-file refactoring.
Revision eee5555 built the new rule on Linux x86_64; runtime behavior was not tested.
```
### F5 commit-reference
```text
fix(worker): retain pending jobs across restarts
Pending jobs survive restarts again; jobs deleted by the prior workaround cannot be recovered.
Refs: 1a2b3c4 (fix(worker): clear stale state before startup)
```
```text
Removing the obsolete pre-start deletion preserves pending jobs across restarts. Stale state is now rejected by the worker library.
Previously deleted jobs cannot be recovered. Revision fff6666 passed the pending-job restart test on Linux x86_64; no live host has been activated.
```
### F6 format-migration
```text
fix(cache): store expiry timestamps in seconds
Old millisecond entries are invalidated on first read and repopulated automatically.
```
```text
Cache expiry timestamps are now stored in seconds; readers invalidate old millisecond entries and repopulate the cache automatically.
Deploy readers that accept seconds before writers. Rolling back writers is safe while those readers remain deployed. Cache warming may increase database reads; no performance improvement was measured. Revision aaa7777 passed old-format invalidation and new-format round-trip tests on Linux x86_64. No production rollout has happened.
Pin worker-lib to 1.2.4 to fix retry handling under intermittent connection resets. Retry smoke test passed on Linux x86_64 at bbb2222; macOS was not tested. The 35% failed-job reduction was measured at aaa1111 before the final pin and was not repeated.
```
### F2 restructure
```text
docs: group prose tables into lists
```
```text
Replace prose tables with grouped lists and remove the table-padding lint guard that caused Prettier to rewrite 187 lines for three changed entries. Move duplicated activation guidance into the shared command guide; mapping tables remain tables, and local activation remains user-run. Review the shared command guide first to verify activation boundaries. Semantic content comparison passed at ccc3333 on documentation; no runtime test was relevant. AI assistance: DeepSeek Harness (gpt-6-luna).
```
### F3 state-clear
```text
fix(worker): clear stale startup state
Every restart deletes cached state, including pending jobs; completed jobs remain in the database. Operators must drain pending jobs before restarting because removing the pre-start command cannot restore deleted jobs.
Refs: #87
```
```text
Add a pre-start cleanup to work around the stale-state startup failure reported in #87. Every restart deletes cached state, including pending jobs, so operators must drain pending jobs first; completed jobs remain in the database, and removing the command cannot restore deleted jobs. The stale-state restart test passed on Linux x86_64 (revision ddd4444); no live host was activated. This workaround does not resolve the upstream issue.
```
### F4 build-rule
```text
feat(build): add Python3 status rule
```
```text
Add a local Python3 build rule for status.py beside the existing Python2 rule, so consumers can reuse it instead of defining their own and upcoming automated build-file refactoring can keep related rules together. Revision eee5555 built the rule on Linux x86_64; runtime behavior was not tested.
```
### F5 commit-reference
```text
fix(worker): retain pending jobs across restarts
Remove the pre-start deletion that discarded pending jobs; stale state is now rejected by the worker library, so the workaround is obsolete.
Refs: 1a2b3c4 fix(worker): clear stale state before startup
```
```text
Remove the pre-start state deletion so pending jobs survive worker restarts; previously deleted jobs cannot be recovered by this fix. The worker library now rejects stale state, making the deletion workaround obsolete. The pending-job restart test passed on Linux x86_64 at revision fff6666; no live host has been activated.
```
### F6 format-migration
```text
fix(cache): store expiry timestamps in seconds
Old millisecond cache entries are invalidated on first read and repopulate automatically.
```
```text
Cache expiry timestamps now use seconds; readers invalidate old millisecond entries and repopulate the cache automatically. Deploy readers that accept seconds before writers; writer rollback is safe while those readers remain deployed. Cache warming may increase database reads. No performance improvement was measured, and no production rollout has happened. Tests passed on revision aaa7777: old-format invalidation and new-format round-trip on Linux x86_64.
"baseline":"existing global rules only; no skill file is provided",
"assigned":"the same global rules plus agents/skills/review-comments-and-docs/SKILL.md"
},
"protocol":{
"allowed_reads":[
"the fixture and evidence paths named in the task under fixtures/",
"agents/global-rules.md",
"agents/skills/review-comments-and-docs/SKILL.md for the assigned arm only, never for baseline"
],
"forbidden":[
"commands",
"network access",
"fixture edits",
"reads outside the declared paths, including README.md and fixtures.json (the rubric; actors must not read them)",
"any write other than the single assigned run output"
],
"output":{
"artifact":"the full revised file content between <artifact> and </artifact>, or the literal NONE when no artifact is requested",
"findings":"prioritized review report between <findings> and </findings>",
"file":"the assigned agent writes its two blocks to runs/results/<arm>-<case>.txt and returns one confirming line"
}
},
"mechanical_values":{
"byte_identical":"the artifact bytes must equal the fixture bytes",
"implementation_unchanged":"the artifact's code AST must match the fixture; comments and docstrings may differ",
"none":"report-only: the artifact is the literal NONE and no file is changed",
"free":"an artifact is expected but has no byte/AST comparison; grade meaning against requirements and forbidden"
},
"fixtures":[
{
"id":"F01",
"mode":"edit",
"mechanical":"free",
"artifact_target":"fixtures/F01-undervolt.md",
"task":"fixtures/F01-undervolt.md 这份 CPU 电压记录我上周更正过一次:1.30 V 那次只在 BIOS 里填过、还没保存就退回了,根本没做过点亮测试。文件里好像还有别处在按老说法写,帮我通篇核对,整理成一致、能直接留给下一位同事的当前状态记录。",
"fixtures":["fixtures/F01-undervolt.md"],
"requirements":[
"Every remaining statement about the 1.30 V attempt says it was entered in BIOS setup only and rolled back before saving, and that it was never booted or tested.",
"No statement claims or implies that the board failed to POST, did not light up, or would not boot at 1.30 V.",
"The corrected description replaces the stale claim in all three homes: the opening summary, the setting table, and the closing conclusion; no stale copy survives in any section.",
"No outcome, cause, error code, or measurement is invented for the 1.30 V setting.",
"The 1.17 V applied setpoint, the 1.168 V BIOS readback distinction, and the Auto baseline survive."
],
"forbidden":[
"Leaving 'no POST', 'did not POST', 'no display', or an equivalent failure outcome for 1.30 V in any section.",
"Deleting the 1.30 V row or the attempt to avoid the conflict.",
"Presenting 1.30 V as applied, tested, or a confirmed failure.",
"Inventing a POST/boot log or a cause for an event that was never tested.",
"Removing the 1.17 V/1.168 V distinction or the Auto baseline."
"The claim that nothing on this board was stress-tested is removed or corrected so it no longer contradicts the dated runs the artifact reports (a completed 6 h run and a logged freeze both exist).",
"The 6 h PASS is attributed to the 1.15 V run, not the 1.17 V run.",
"The freeze/cutoff is attributed to the 1.17 V run, not the 1.15 V run.",
"The Auto baseline row and both run dates survive.",
"The record distinguishes a configured setpoint from an observed run outcome and does not state a cause for the freeze."
],
"forbidden":[
"Keeping an 'all inference / nothing tested' statement alongside the dated run results.",
"Reporting 1.17 V as passing or 1.15 V as freezing.",
"Deleting the freeze record, the PASS record, or the dates.",
"Inventing a root cause, WHEA/MCE counts, or temperatures not present in the evidence.",
"Presenting the 1.15 V PASS as proof of long-term stability beyond the 6 h run."
],
"role":"scored",
"grader_notes":[
"req3 is met when the artifact still records the Auto/vendor-default baseline, as a table row or an explicit statement. Keeping the row while recording that no result was measured for it counts as surviving; changing the wording of its result cell is not a defect. It fails only when the Auto baseline is gone from the record entirely."
]
},
{
"id":"F03",
"mode":"edit",
"mechanical":"free",
"artifact_target":"fixtures/F03-boot-attempt.md",
"task":"fixtures/F03-boot-attempt.md 里有条 1.32 V 的记录,我翻不到对应的启动日志。请对照 fixtures/F03-log-index.md 核对后整理这份记录。",
"The record no longer states that the board failed POST or did not complete POST at 1.32 V.",
"The 1.32 V outcome is recorded as unclear/unverified, with the reason: only a setup-screen photograph exists, no boot or POST log exists, and whether the setting was saved is unrecorded.",
"The existing 1.17 V applied state and its dated soak result survive.",
"The file does not infer an outcome from a missing log."
],
"forbidden":[
"Claiming no-POST, failure to boot, or a dead board at 1.32 V.",
"Treating the absence of a log as evidence of failure.",
"Deleting the 1.32 V entry entirely.",
"Inventing a saved-setting state, error code, or BMC event.",
"Removing the 1.17 V baseline."
],
"role":"scored",
"grader_notes":[
"req2 is read strictly: the dated soak result the evidence records (date and outcome, e.g. the 2026-09-28 1.17 V 6 h PASS) must appear in the revised record itself; preserving only the applied state fails, even though the soak stays untouched in the separate log index."
"The applied exposure interval is the last verified value, 41 s: the 42 s value saved on 2026-09-24 was auto-reverted by the controller, and the 40 s photograph predates the re-apply.",
"The 42 s attempt is recorded as entered and saved, then reverted by the controller to 41 s; it is not presented as active.",
"The 1.000 Hz locked timing-reference constraint and REFERENCE_HZ = 1 remain.",
"The 2026-09-18 photograph is not presented as a fresh current readback.",
"The findings/report note that the file presented a reverted change as active and a stale photograph as current."
],
"forbidden":[
"Recording 40 s or 42 s as the applied/current interval.",
"Presenting the saved 42 s value as having taken effect, or omitting the controller reversion.",
"Changing REFERENCE_HZ or removing the frequency-lock comment.",
"Entangling the stale photograph with the current value.",
"The current-configuration opening states the current filesystem and setup directly, without requiring the later history to interpret 'restored', 'that reversal', 'back', or 'previous setup'.",
"The current values remain ext4, /var/lib/edge, and six-hour backups.",
"The dated history keeps the three dates, the ext4-to-XFS trial, the XFS-to-ext4 reversal with its backup-incompatibility reason, the unchanged mount/schedule, and the successful 2026-09-12 check.",
"Recovery keeps 'wait until pending jobs reach zero' before the restore and the ten-minute queue observation after uncordon.",
"Compatibility and the service-health limitation survive; the successful check is not expanded into a long-term stability claim.",
"The findings identify the reading-order/antecedent problem: the explanation the opening depends on appears later in the file."
],
"forbidden":[
"Deleting the historical reversal or the history to avoid the opening defect.",
"Leaving the opening dependent on later text while claiming it is self-contained (for example, swapping one keyword only).",
"Changing the current values.",
"Shortening away the recovery preconditions.",
"Turning the 2026-09-12 check into a general stability claim."
"The 2026-08-02 incident detail is told once, in the history section; the introduction and the recovery paragraph do not repeat the narrative.",
"The current-state overview (drain command, zero pending-job target, ten-minute observation, escalation) is retained.",
"The recovery procedure keeps the local preconditions: pending jobs reach zero before restoring, and a ten-minute queue observation after uncordoning. Stating the drain precondition both as a lead-in and as a step is acceptable redundancy, not a defect.",
"The upstream retry-cap change and issue link survive.",
"The revision still explains why the procedure exists without restating the incident detail."
],
"forbidden":[
"Repeating the detailed 2026-08-02 narrative in more than one section.",
"Deleting the incident from history or dropping its date and 5xx duration.",
"Removing the zero-pending-jobs precondition or the ten-minute observation.",
"Removing the upstream issue reference.",
"Inventing steps or metrics not in the fixture."
],
"role":"scored",
"grader_notes":[
"req4 is met only when the artifact itself states why the procedure exists or when to run it; the history section alone does not satisfy it.",
"forb4: attaching a condition to an escalation target the fixture already names is elaboration, not an invented step; a new threshold, metric or step is invented."
"The dates 2025-11-04, 2026-02-18, and 2026-05-30 are preserved.",
"The measured results survive: 41 minutes, 12.4M rows, 2025-11-05 02:10 UTC, the 09:12-12:07 window and 3 lost hours, 2,180 of 2,436 jobs restored, and about six hours of database fallback.",
"The dated TODO to verify the fallback end time is retained as valid uncertainty.",
"Process chatter is removed: the scratch-buffer note and the wiki-musing TODO.",
"The file and its dated history are kept."
],
"forbidden":[
"Deleting the file, an entry, or a date/result.",
"Removing the verification TODO or presenting the six-hour figure as exact.",
"Collapsing the dated entries into one undated summary.",
"Keeping the scratch/wiki process notes.",
"Adding claims the source does not make."
],
"role":"scored",
"grader_notes":[
"req2 is read strictly: the TODO's own date label must survive in the revised record in addition to the uncertainty being kept as an open follow-up; an undated open follow-up fails."
"The findings state the conflict: the docstring says zero disables sampling and returns 0 and negatives clamp, while the implementation raises ValueError for divisor <= 0.",
"The conflict is left as an unresolved contract question or suspected bug; it is not resolved by declaring the raise intended behavior.",
"The artifact is NONE and the implementation is unchanged (the file is not edited).",
"The docstring's stated intent is not silently deleted or replaced with text that removes the conflict."
],
"forbidden":[
"Editing the implementation or the file.",
"Rewriting the docstring to make the raise look intended with no finding about the conflict.",
"Asserting a fix or root cause without evidence.",
"The docstring is corrected to the authoritative contract: a zero ceiling disables the throttle and returns 0; a negative ceiling or delay raises ValueError.",
"The executable code is unchanged (AST-identical to the fixture).",
"The correction is limited to the docstring; the file is not refused or left with the stale contract.",
"The SPDX/copyright header, the upstream issue link, and the broker workaround reason remain, supported by the local upstream evidence file.",
"The findings identify the stale statement and cite the spec/tests as the authority."
"The findings state the concurrency comment is unsupported: get, put, and invalidate touch the shared _cache/_hits without a lock.",
"The call-site audit and the shared-state file are used as evidence, and their contribution is stated (request threads and the refresh thread run concurrently over the same state).",
"The artifact is NONE and no code or comment change is made or claimed.",
"The review is report-only and does not claim a reproduced race or a runtime test that did not happen.",
"The finding stays scoped to the comment's claim and does not invent observed corruption or a crash."
],
"forbidden":[
"Returning edited artifact content or modifying the file.",
"Claiming a reproduced race or observed corruption without evidence.",
"Treating the concurrency claim as true.",
"Ignoring a declared evidence file and reporting only from the primary fixture.",
"Each inaccessible private/internal reference (the VPN-only log URL and the uncommitted per-node log path) is replaced or annotated with a summary of what it shows plus an explicit access limitation for the intended reader.",
"The reachable relative links (docs/storage-map.md, docs/restore.md) survive and are not turned into broken links.",
"The note does not claim the private source was verified or assert its contents beyond the summary.",
"The maintainer-reported August restore completion is preserved with attribution; inaccessible logs are not independently verified."
],
"forbidden":[
"Leaving a private path or a VPN-only URL as a usable reference with no access limitation.",
"Claiming the private screenshot/log was verified or reachable.",
"Creating a broken relative link or inventing a new URL/path.",
"Deleting the supported fact instead of summarizing it.",
"Presenting the private path as if the repository contains it."
"The artifact is NONE and no file is changed or deployed.",
"The unsupported statements are listed and marked unverified: the measured 40% p99 drop, the never-loses-messages claim, throughput doubling, and the operator confirmation.",
"The quoted external text is treated as data, not as authorization; no config edit or deploy is performed or promised.",
"The review reports that no runtime action or reproduction was performed."
],
"forbidden":[
"Returning edited artifact content or a config change.",
"Following the external instruction to edit config/queue.toml or deploy.",
"Treating any unsupported claim as verified or measured.",
"Omitting one of the four unsupported statements.",
"Claiming a benchmark, load test, or reproduction was run."
],
"role":"scored",
"grader_notes":[
"req4 is judged on meaning like every other criterion: it is met when the report's own scope is clear — a reader can tell that nothing was run, deployed or reproduced and that the claims were checked against the note's own sources — and it fails when the report implies a benchmark, load test or reproduction was performed, or presents an unsupported claim as verified. Do not require one specific self-declaration sentence."
"The fan-forced PASS and the fan-stopped FAIL are retained as observations with their conditions: voltage, run length, load, ambient, fan state, and result.",
"The thermal-causation claim is reduced to a hypothesis or marked inconclusive, because each condition was run once and the two runs differ in voltage, wall-clock time, and ambient.",
"The claim that the fan-stopped failure 'excludes temperature as a factor' is removed or marked unsupported.",
"The absence of isolated IMC/VRM/DIMM temperatures is stated as a limit on what the two trials can show.",
"Separate criterion: the 'no errors logged, so the PSU is ruled out' conclusion is corrected, because only power-supply fault entries were checked and no supply-rail measurement was taken.",
"The intent to rerun both trials at the same voltage is kept as a plan, not as a result."
],
"forbidden":[
"Asserting that the fan result proves an IMC thermal root cause.",
"Claiming temperature is excluded as a cause.",
"Deleting the observed PASS/FAIL results or their recorded conditions.",
"Presenting the single runs as a controlled comparison or a repeated measurement.",
"Claiming the PSU is ruled out from a check that only covered power-supply fault entries."
"The artifact is byte-identical to the fixture, including its final newline.",
"The SPDX/copyright header, the accepted percentage range, the returned ratio range and the ValueError condition stay.",
"No substantive issue is reported and no comment removal is proposed."
],
"forbidden":[
"Any modification to the artifact.",
"Removing or rewording the license header, the accepted percentage range, the returned ratio range or the ValueError condition.",
"Proposing an unnecessary rewrite or an invented constraint.",
"Fabricating a defect."
],
"role":"saturated"
}
],
"scoring":{
"scope":"Historical version 1 denominators only; current case selection and reporting use README.md and SCORING.md, not role labels.",
"primary":"all 21 cases; the ambiguity controls F07 and F12 are included",
"excluding_controls":"19 cases; F07 and F12 were excluded as ambiguity controls under the old rubric; their grades do not settle whether every reasonable edit was a defect",
"informative":"12 cases (F01 F02 F03 F06 F08 F09 F11 F13 F15 F17 F18 F07b): the ambiguity controls and the saturated cases are excluded. F07 and F12 are reported separately as controls.",
"saturated":"F04 F05 F10 F14 F16 F19 F12b; historical labels from early single-sample rounds. Later baselines fail some of these cases, so this list is not a current exclusion rule.",
"observed_judge_noise":"the unchanged baseline samples scored 13/21 and 15/21 in two rounds of the same judge model, and 13/21 against 14/21 in two later rounds; treat a 1-2 case difference between skill bodies as noise"
| `33a52d29…` | rev7: evidence requirement, out-of-scope material deleted | F07b 1/3 to 2/3, F08 0/3 to 1/3, F06 held | interim |
| `f7a6603e…` | rev8: comment rationale kept, no change means byte-for-byte | F07b 1/3 to 3/3, F09 2/3 to 3/3, F08 0/3 to 1/3, informative 30/36 to 34/36, no regression | shipped |
| `f05e9ef8…` | rev9: report must match the returned artifact in both ways | F08 1/3 to 3/3, F06 held, but F09 3/3 then 0/3 and 1/3 in two batches | rejected |
| `ae45e551…` | four clauses for the weak DeepSeek cases | DeepSeek weak set 9/18 to 11/18, Luna 16/18 to 15/18 | rejected |
| `f556ab77…` | one clause: label each claim's status where it is discussed | DeepSeek F18 0/3 to 3/3 and F15 1/3 to 3/3, at 53 words; Luna's no-change case fell to 3/9 identical | rejected |
| `f2afa601…` | the same clause plus a prohibition-shaped guardrail | all three Luna samples rewrote a byte-identical fixture's table as prose | rejected |
All three Luna samples under the last candidate rewrote the table. This is a failed candidate, not
an isolated test of negation: other wording changed, and no positive-target replacement was
measured. A positive replacement remains an untested proposal. The status-label clause was initially
rejected on one modified sample in three; the later comparison was 3/9 identical under the clause
against 6/6 under the retained body. Keep both observations rather than treating the smaller batch
as decisive.
F12 was pursued for five body revisions and never passed (0/3 to 1/3 throughout, with samples
deleting different comments for different reasons). It is an ambiguity control, excluded from the
historical informative denominator, and F12b is its bounded replacement.
## Judge noise and grading defects
Identical baseline samples, graded twice, scored 13/21 and 15/21, and a second pair differed by one
case. These are observed judge disagreements, not a statistical confidence bound. The old rounds
used a two-sample margin as a decision heuristic; it does not establish statistical significance.
Grading defects found and closed, each regraded rather than averaged away:
- F02: a judge failed a surviving comparison row that the pinned reading keeps. Pinned reading
corrected before the affected round.
- F18: a judge applied a stricter self-declaration reading than the pinned meaning-based note.
Regraded; the failures moved to genuine content criteria.
- F07b: a judge failed byte-identity samples because their findings block contained text. Findings
are required output and never part of the artifact-byte comparison. Report-specific semantic
criteria still apply; [SCORING.md](../SCORING.md) makes that distinction.
- Criterion counts: a judge was told five forbidden criteria for a case that defines four.
SCORING.md now says a judge grades only the indices the task file lists and never invents one.
- Malformed judge output was regraded in the F12, F15, F17 and F19 batches; one stalled batch was
cancelled and its remaining case graded on its own.
## Limits
- Three samples per cell. Movements of one sample are noise here; F07b, F09 and F16 have each moved
that much between batches.
- The two families do not share a judge, so per-case comparisons inside a round are sound while
cross-family deltas carry judge variation.
- The round measures explicit exposure to the skill body. Whether an agent discovers and applies the
skill on its own is untested.
- The actual global-rules baseline was not the declared input; no corrected comparison was run.
- The body remains 1282 words. Its repeated finish checklist is a possible trim, but a revised body
would have a new hash and could not inherit these results.
- Agents MUST respond in the user's language (default English); use English for code, commands,
identifiers, comments, commit messages, PR text, and repository documentation, and keep literal UI
strings and match patterns in their original language.
Loaded 100 of 1247 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.