diff --git a/hosts/idols-ai/MIXED-MEMORY.md b/hosts/idols-ai/MIXED-MEMORY.md deleted file mode 100644 index 27b6147c..00000000 --- a/hosts/idols-ai/MIXED-MEMORY.md +++ /dev/null @@ -1,98 +0,0 @@ -# Mixed memory on idols-ai - -128G from JUHOR 2×48G + GLOWY 2×16G. All four DIMMs are needed because 96G is insufficient. - -## Current configuration - -CPU VDD2 is set to **1.17 V** (BIOS reads **1.168 V**). This is the best configuration tested so -far, but it still froze during daily use on 2026-10-11. Keep it for normal use for now; higher -tested VDD2 settings have not provided a stable alternative. - -| Field | Setting | -| ----------------------- | -------------------------------------------- | -| XMP | Disabled | -| Gear mode | Gear2 | -| Memory speed | 4800 MT/s | -| CPU VDD2 | **1.17 V fixed (BIOS reads 1.168 V)** | -| VCCSA | Auto (BIOS reads 1.288 V) | -| VDD2_MV (BIOS label) | Auto (BIOS reads 1.280 V) | -| DRAM VDD | **1.28 V fixed** | -| DRAM VDDQ | **1.28 V fixed** | -| DRAM VPP | Auto (1.8 V) | -| tCL / tRCD / tRP / tRAS | 40 / 40 / 40 / 77 (last photographed values) | - -Voltage settings and readings come from BIOS and user reports, not OS measurements. - -## Hardware - -- CPU: Intel Core Ultra 7 270K Plus. -- Board: Colorful CVN Z890 ARK FROZEN V20, BIOS 1013. -- Each channel: 48G (2R) in DIMM0, 16G (1R) in DIMM1. -- The 16G pair was added on 2026-10-06. - -| Kit | Capacity | XMP rating | Timings | Voltage | Rank | Part number | -| ----- | -------- | ---------- | ------------- | ------- | ---- | --------------------- | -| JUHOR | 2×48G | DDR5-5600 | CL46-45-45-90 | 1.25 V | 2R | `JHE5600U4648JG` | -| GLOWY | 2×16G | DDR5-6400 | CL32-38-38-90 | 1.35 V | 1R | `VGM5UH64C32AG-DTACW` | - -## Bring-up and recovery - -Earlier attempts starting with the 48G pair or all four DIMMs did not reach BIOS. Working order: - -1. Install only the 2×16G in `ChannelA-DIMM1` / `ChannelB-DIMM1`. -2. Enter BIOS and disable XMP. -3. Power off and add the 2×48G in DIMM0. -4. Allow 1 to 3 minutes for retraining, possibly with several reboots. -5. Apply the current settings above. - -If failed training prevents BIOS entry, repeat this order; clear CMOS first if necessary. Failed -training can silently restore defaults; re-check settings in BIOS afterwards. - -## Test results - -CPU VDD2 entries below are BIOS settings: Auto read 1.104 V in the earlier DRAM tests; the 1.17 V -setting reads 1.168 V. These BIOS readings are not measurements during the OS tests. - -| CPU VDD2 | DRAM VDD | DRAM VDDQ | Result | Highest sampled DIMM temperature | -| ---------- | -------------------- | -------------------- | ------------------------------------------------------------------------------------------------------ | ---------------------------------------------- | -| Auto | 1.10 V | 1.10 V | Immediate freezes during memory tests, even 4G. | Not recorded | -| Auto | 1.25 V | 1.25 V | 8G / 32G / 100G short tests passed; llama.cpp load froze. | Not recorded | -| Auto | 1.28 V | 1.28 V | 100G short test passed; daily use froze on 2026-10-08. | Not recorded | -| Auto | 1.282 V | 1.282 V | Display corruption in BIOS. | Not recorded | -| Auto | 1.29 / 1.30 / 1.32 V | 1.29 / 1.30 / 1.32 V | Each setting failed training. | Not recorded | -| 1.13 V | 1.28 V | 1.28 V | BIOS normal; no long test. | Not recorded | -| 1.15 V | 1.28 V | 1.28 V | 100G, 15 minutes: PASS. | 45.5°C* | -| 1.15 V | 1.28 V | 1.28 V | Long test froze; last saved duration 21 min 17 s. | 56.5°C | -| **1.17 V** | **1.28 V** | **1.28 V** | **100G, 6 hours: PASS, 0 errors or hardware incidents.** | **56.8°C** | -| 1.17 V | 1.28 V | 1.28 V | About 56 hours across three boots ending cleanly; next boot froze during LLM serving (2026-10-11). | 47.2°C in the last available monitoring sample | -| 1.20 V | 1.28 V | 1.28 V | Earlier attempt: BIOS display corruption and freeze within 10 to 20 s. | Not recorded | -| 1.20 V | 1.28 V | 1.28 V | 100G test froze with screen corruption; last saved duration 7 min 23 s (2026-10-11). | 55.25°C | -| 1.18 V | 1.28 V | 1.28 V | 100G test froze with screen corruption; last saved duration 14 min 36 s (2026-10-11). | 59.5°C | -| 1.28 V | 1.28 V | 1.28 V | Reached the desktop, then froze; journal spans 16:30:16 to 16:31:20 (64 s). No recorded pressure test. | Not recorded | -| 1.30 V | 1.28 V | 1.28 V | Set in BIOS only; rolled back without booting or testing. | Not tested | -| **1.17 V** | **1.28 V** | **1.28 V** | **100G, 15 minutes: PASS, 0 errors or hardware incidents (2026-10-11 17:29–17:45).** | **59.25°C** | - -*The 1.15 V short test used sparse temperature samples; its true peak is unknown. The monitored long -tests and the 2026-10-11 100G tests sampled about every 5 seconds with a 60°C stop threshold. Saved -durations are lower bounds on time before interruption, not exact freeze times. - -The passing 1.17 V test ran on 2026-10-09 from 01:29 to 07:30 without swap use. Subsequent daily use -included displays on and off with LLM serving. Sleep/wake is untested. - -The 1.17 V freeze on 2026-10-11 ended logging at 13:55:46, about 4 h 47 min after boot. Logs stopped -during generation without a shutdown sequence or a recorded machine check, BUG or Oops. - -## BIOS limitations and conclusion - -Attempts to apply tRFC 1180 and primary timings 42 / 42 / 42 / 82 reverted automatically after -saving, according to the user's BIOS checks. Those relaxed settings never became a tested -configuration. The memory frequency control is also locked at 4800 MT/s in the current BIOS setup. -The table above retains the last photographed timings; a later readback has not been recorded. - -Keep CPU VDD2 at 1.17 V and DRAM VDD/VDDQ at 1.28 V for now. The passing stress test and daily-use -record make this the best tested option, but the later freeze means stability remains unresolved. - -The user reports a possible ambient temperature rise of about 10°C. Temperature may affect the -remaining margin, but neither temperature nor an IMC, DIMM or motherboard fault has been isolated. -Manual timing relaxation and frequency reduction are currently unavailable; further tuning is -paused. Revisit only if freezes continue or those BIOS controls become usable. diff --git a/hosts/idols-ai/README.md b/hosts/idols-ai/README.md index 9700eb7d..84c48717 100644 --- a/hosts/idols-ai/README.md +++ b/hosts/idols-ai/README.md @@ -3,17 +3,75 @@ Desktop (NixOS + preservation, LUKS + btrfs on nvme). Disk layout is declarative via [disko](./disko-fs.nix) (target device: the KINGBANK NVMe by `/dev/disk/by-id/`, set in that file). +Desktop on the MSI **MAG B760M MORTAR** with the i5-13600KF, 32G DDR5 and the RTX 4090, on the +KINGBANK NVMe alone. The platform split is recorded in [hosts/README.md](../README.md); the wired +NIC and its `en*` match are in [default.nix](./default.nix). + +TPM2 comes from the CPU's Intel PTT unless a module is plugged into the board's TPM header; check +_Settings → Security → Trusted Computing_ if `systemd-cryptenroll --tpm2-device=auto` finds no +device. + Related: - [nixos-installer README](../../nixos-installer/README.md) – install from ISO using disko - [disko-fs.nix](./disko-fs.nix) – main disk layout (ESP + LUKS + btrfs). From `nix-config/nixos-installer`: `nix run github:nix-community/disko -- --mode destroy,format,mount ../hosts/idols-ai/disko-fs.nix` -- [disko-fs-data.nix](./disko-fs-data.nix) – data disk layout (LUKS + btrfs at /data) -- [MIXED-MEMORY.md](./MIXED-MEMORY.md) – mixed 4-DIMM RAM bring-up, stress testing, and recovery +- [VR.md](./VR.md) – WiVRn headset setup ## Info -The disk layout is fully declarative — see [disko-fs.nix](./disko-fs.nix) for the root disk and -[disko-fs-data.nix](./disko-fs-data.nix) for the data disk. Run `df -Th` / `lsblk` on the host if an -up-to-date snapshot of the current mounts is needed. +The disk layout is fully declarative — see [disko-fs.nix](./disko-fs.nix) for the root disk. Run +`df -Th` / `lsblk` on the host if an up-to-date snapshot of the current mounts is needed. + +Steam libraries live under the persisted home directory, so they survive the tmpfs root: +`~/Games/SteamLibrary` is the library that used to sit on the `/data` disk. Steam finds a moved +library when it is added once through _Settings → Storage → Add Drive_, or by starting Steam with +the `libraryfolders.vdf` already listing it. + +## 4090 water block + +After the 2026-10-13 platform swap the card's water block stopped moving heat: the pump ran but made +abnormal noise, and the card still overheated. It had been RMA'd for the same symptom before, came +back with no fault found, and worked again after the return shipment. Repacking the card in its +original box and dropping the box five or six times freed it, so a stuck impeller, with transit +shock as the accidental cure, is the best guess. Treat that as a stopgap before an RMA: the jolt can +damage the card, and a pump that sticks again should be replaced. + +## Booting after the platform swap + +The KINGBANK disk was last enrolled with the Z890 board's TPM2 and its UKI is signed with a key that +only that board's Secure Boot database trusts. On the B760M board: + +1. Boot with the LUKS **passphrase** — the old TPM2 enrollment is bound to the previous board's PCR + values and will not unlock here. +2. Secure Boot: either leave it off, or enroll a fresh PKI — `sbctl create-keys` + + `sbctl enroll-keys` in setup mode. lanzaboote signs with `pkiBundle` (`/etc/secureboot` in + [secureboot.nix](./secureboot.nix)), so the new keys must end up there; `sbctl create-keys` + writes to its own keydir. Re-enrolling changes PCR7, so re-run the enrollment in step 3 + afterwards. +3. Re-enroll this board's TPM2 once the Secure Boot state is final. Enroll and wipe in **one** call: + `--wipe-slot=tpm2` matches every TPM2 slot, so wiping in a separate call would also delete the + slot just created. Combined, systemd-cryptenroll enrolls first and then wipes the old slots, + always sparing the new one. + + ```bash + disk=/dev/disk/by-id/nvme-KINGBANK_KP260_CN092BH3203917-part2 + sudo systemd-cryptenroll --tpm2-device=auto --tpm2-pcrs=0+7 --wipe-slot=tpm2 "$disk" + sudo systemd-cryptenroll "$disk" # read the slot list back: tpm2 + passphrase + ``` + + PCR0 covers the firmware code, so a BIOS update needs this enrollment redone; the passphrase slot + keeps booting either way. The `0+7` pair is the repo's existing shape — see + [nixos-installer/README.md](../../nixos-installer/README.md). + +4. The B760M's NVRAM has no boot entry for this disk. If it does not boot on its own, pick the disk + once from the one-time boot menu (the ESP also carries `\EFI\BOOT\BOOTX64.EFI`); the first + activation should then write the entry. If it does not, add one by hand — `ls /boot/EFI/Linux` + shows the UKI name lanzaboote signed: + + ```bash + sudo ls /boot/EFI/Linux + sudo nix run nixpkgs#efibootmgr -- -c -d /dev/disk/by-id/nvme-KINGBANK_KP260_CN092BH3203917 \ + -p 1 -L NixOS -l '\EFI\Linux\nixos-generation-1.efi' + ``` diff --git a/hosts/idols-ai/VR.md b/hosts/idols-ai/VR.md index 7c330ff7..03bd6199 100644 --- a/hosts/idols-ai/VR.md +++ b/hosts/idols-ai/VR.md @@ -67,7 +67,7 @@ Manage paired headsets with `wivrnctl list-paired`, `wivrnctl rename`, or `wivrn ## Service, settings and logs -Manage the server through systemd so starts retain NVIDIA offload and the CAP_SYS_NICE wrapper: +Manage the server through systemd so starts retain the CAP_SYS_NICE wrapper: ```console systemctl --user restart wivrn @@ -83,26 +83,26 @@ configuration schema, then restart the service. The NixOS module also supports d configuration; an enabled declarative config takes precedence over the user file. Headset-side settings remain available in the headset application. -## Launch games on the RTX 4090 +## Games and the RTX 4090 -Set this in each VR game's Steam Properties -> Launch Options: +The RTX 4090 is the only GPU on this host, so the WiVRn server, the compositor and the game land on +the same card by themselves. + +That is not how it used to be. While `ai` ran the Z890 board, its Arrow Lake iGPU did the desktop +rendering under PRIME offload, and a game left on the iGPU while the server ran on the dGPU failed +with `vkAllocateMemory: VK_ERROR_OUT_OF_DEVICE_MEMORY` and `xrt_comp_create_swapchain failed` in +`journalctl --user -u wivrn`, followed by an xrizer panic that killed the game. The fix then was the +variables below, written out as Steam launch options because Steam runs inside a bubblewrap sandbox +that does not mount `/run`, so it cannot see `/run/current-system/sw/bin/nvidia-offload`: ```text __NV_PRIME_RENDER_OFFLOAD=1 __NV_PRIME_RENDER_OFFLOAD_PROVIDER=NVIDIA-G0 __GLX_VENDOR_LIBRARY_NAME=nvidia __VK_LAYER_NV_optimus=NVIDIA_only %command% ``` -These are the variables the system's `nvidia-offload` wrapper sets, written out because Steam runs -inside a bubblewrap sandbox that does not mount `/run`, so it cannot see -`/run/current-system/sw/bin/nvidia-offload`. - -The game has to land on the same GPU as the server. A game left on the Intel iGPU makes the server's -swapchain allocation fail, which shows up as `vkAllocateMemory: VK_ERROR_OUT_OF_DEVICE_MEMORY` and -`xrt_comp_create_swapchain failed` in `journalctl --user -u wivrn`, followed by an xrizer panic that -kills the game. This applies when launching from the headset too: WiVRn starts applications in -separate systemd units, and an already running Steam client keeps its own environment, so neither -path inherits the server's GPU environment. For non-Steam applications, start them under -`nvidia-offload` from a normal shell. Select a Proton version according to the game's compatibility -reports; GE-Proton or DW-Proton alone does not guarantee VR compatibility. +It mattered for headset-launched games too: WiVRn starts applications in separate systemd units, and +an already running Steam client keeps its own environment. Re-add those variables only if a second +GPU comes back. Select a Proton version according to the game's compatibility reports; GE-Proton or +DW-Proton alone does not guarantee VR compatibility. ## USB connection @@ -131,9 +131,8 @@ mapping after disconnecting USB. The package enables Vulkan Video encoding but does not enable NVENC. WiVRn 26.9 probes Vulkan encoding before falling back to VAAPI or x264 when NVENC is absent. Check the journal's encoder -configuration after connecting: offload variables alone do not prove hardware encoding. Vulkan -encoding in this release supports H.264/H.265, not AV1. NVENC/AV1 requires a CUDA-enabled WiVRn -build. +configuration after connecting: GPU activity alone does not prove hardware encoding. Vulkan encoding +in this release supports H.264/H.265, not AV1. NVENC/AV1 requires a CUDA-enabled WiVRn build. - Pair, connect, launch a game, and check both compositor and game GPU usage: `nvidia-smi --query-compute-apps=pid,process_name,used_memory --format=csv` has to list both diff --git a/hosts/idols-ai/ai/vr.nix b/hosts/idols-ai/ai/vr.nix index f49db4f4..b6757123 100644 --- a/hosts/idols-ai/ai/vr.nix +++ b/hosts/idols-ai/ai/vr.nix @@ -1,7 +1,7 @@ { pkgs, ... }: { # WiVRn supplies the OpenXR runtime; xrizer translates OpenVR games. - # The server uses the host GPU environment from hardware-nvidia.nix. + # The RTX 4090 is the only GPU, so the server needs no GPU environment of its own. services.wivrn = { enable = true; autoStart = false; # Start only when requested with just vr. @@ -24,8 +24,8 @@ ''; }; - # CAP_SYS_NICE for the server. Games need their own PRIME offload: WiVRn - # starts them in separate units, which do not inherit this environment. + # CAP_SYS_NICE for the server. WiVRn starts games in separate units, and an + # already running Steam client keeps its own environment. highPriority = true; steam.importOXRRuntimes = true; }; diff --git a/hosts/idols-ai/default.nix b/hosts/idols-ai/default.nix index 7f498dad..fe72e527 100644 --- a/hosts/idols-ai/default.nix +++ b/hosts/idols-ai/default.nix @@ -7,8 +7,8 @@ }: ############################################################# # -# Ai - my main computer, with NixOS + Core Ultra 7 270K Plus + RTX 4090 GPU, -# 128G DDR5-4800 (mixed 4-DIMM, see MIXED-MEMORY.md), for gaming & daily use. +# Ai - my gaming and daily-use desktop. Hardware and the platform history: +# ./README.md. # ############################################################# let @@ -24,7 +24,6 @@ in disko.nixosModules.default # disks ./disko-fs.nix - ./disko-fs-data.nix # Include the results of the hardware scan. ./hardware-configuration.nix @@ -45,15 +44,13 @@ in # disk swapfile instead, created by disko at /swap (see ./disko-fs.nix). The two must move together. modules.zram.enable = false; - # zswap: compressed writeback cache in front of the disk swapfile. - # Keeps swapped cold anon pages compressed in RAM instead of the SSD, which frees more - # page cache for the ~78GB mmap'd LLM weights (mmap file pages never go through zswap). - # Safe here: disk swapfile remains the real backing store, no zram deadlock structure. - # - zstd: CPU is not the bottleneck here (llama.cpp peaks ~30% util); its ~2.5x ratio - # keeps ~5G more anon bytes in the capped pool than lz4, and ~5us decompress is noise - # vs the ~100us SSD fault it avoids. - # - 10% pool (~13G on 128G): the module default 25% (~32G) would compete with the weight - # cache. + # zswap: compressed writeback cache in front of the disk swapfile, so cold + # anonymous pages stay in RAM instead of hitting the SSD. Safe here: the disk + # swapfile remains the real backing store, no zram deadlock structure. + # - zstd: ~2.5x the ratio of lz4 for ~5us decompress, which is noise against + # the ~100us SSD fault it avoids; the CPU is not the bottleneck. + # - 10% pool (~3G on 32G): the module default 25% would compete with the page + # cache that games and VMs rely on. boot.zswap = { enable = true; compressor = "zstd"; @@ -65,7 +62,9 @@ in modules.btrbk.enable = true; - # Permit the desktop user to access Intel accelerator devices for local ASR. + # Give the desktop user the render group outright instead of relying only on + # the logind seat ACL, so local ASR keeps its /dev/dri access in sessions + # that have no seat (the Sunshine headless session, for example). users.users.${myvars.username}.extraGroups = [ "render" ]; # systemd-journal - cap disk usage @@ -105,13 +104,15 @@ in # networkd removes routing policy rules not defined in .network files # (ManageForeignRoutingPolicyRules defaults to yes) each time it reconfigures - # a link. On S3 resume it reconfigures enp130s0, which wiped mihomo's + # a link. On S3 resume it reconfigures the NIC, which wiped mihomo's # (9000-9010) and Tailscale's rules and briefly broke routing. Keep foreign # rules so the VPN/TUN rules survive. systemd.network.config.networkConfig.ManageForeignRoutingPolicyRules = false; - systemd.network.networks."10-${iface}" = { - matchConfig.Name = [ iface ]; + # Predictable names follow the PCIe topology, so a board or slot change renames the + # wired NIC. Match the `en*` prefix instead of a name; one wired port, so one link. + systemd.network.networks."10-lan" = { + matchConfig.Name = [ "en*" ]; networkConfig = { Address = [ ipv4WithMask @@ -153,6 +154,8 @@ in # parsing is where missing geodata gets fetched. Flipping at fork time pointed # that fetch at a port nobody was listening on -- SERVFAIL, a fatal config # error, a unit that could never start. + # The hook names the link with `hostsAddr.ai.iface`: networkd matches an `en*` + # prefix, but `resolvectl` takes one name or index, not a pattern. systemd.services.mihomo.serviceConfig = { ExecStartPost = lib.mkDefault "+${pkgs.writeShellScript "mihomo-dns-takeover" '' probe() { (exec 3<>/dev/tcp/127.0.0.1/1053) 2>/dev/null; } diff --git a/hosts/idols-ai/disko-fs-data.nix b/hosts/idols-ai/disko-fs-data.nix deleted file mode 100644 index dabf97cf..00000000 --- a/hosts/idols-ai/disko-fs-data.nix +++ /dev/null @@ -1,62 +0,0 @@ -# Disko layout for idols-ai data disk (LUKS + btrfs, mount at /data). -# -# Destroy, format & mount (wipes disk; from nixos-installer: cd nix-config/nixos-installer): -# nix run github:nix-community/disko -- --mode destroy,format,mount ../hosts/idols-ai/disko-fs-data.nix -# Mount only (after first format): -# nix run github:nix-community/disko -- --mode mount ../hosts/idols-ai/disko-fs-data.nix -# -{ - disko.devices = { - disk.data = { - type = "disk"; - device = "/dev/disk/by-id/nvme-Fanxiang_S790_2TB_FXS790254050582"; - content = { - type = "gpt"; - partitions = { - datapart = { - size = "100%"; - content = { - type = "luks"; - name = "data-luks"; # Mapper name; match boot.initrd.luks - settings = { - allowDiscards = true; # TRIM for SSDs; slightly less secure, better performance - }; - # Add boot.initrd.luks.devices so initrd prompts for passphrase at boot - initrdUnlock = true; - # cryptsetup luksFormat options - extraFormatArgs = [ - "--type luks2" - "--cipher aes-xts-plain64" - "--hash sha512" - "--iter-time 5000" - "--key-size 256" - "--pbkdf argon2id" - "--use-random" # Block until enough entropy from /dev/random - ]; - extraOpenArgs = [ - "--timeout 10" - ]; - content = { - type = "btrfs"; - extraArgs = [ "-f" ]; # Force overwrite if filesystem already exists - subvolumes = { - "@data" = { - mountpoint = "/data"; - mountOptions = [ - "compress-force=zstd:1" - ]; - }; - }; - postMountHook = '' - chown ryan:users /mnt/data - # Set SGID + rwx for owner/group, read-only for others; new files inherit group - chmod 2755 /mnt/data - ''; - }; - }; - }; - }; - }; - }; - }; -} diff --git a/hosts/idols-ai/hardware-intel.nix b/hosts/idols-ai/hardware-intel.nix index 27497cc6..a79543f5 100644 --- a/hosts/idols-ai/hardware-intel.nix +++ b/hosts/idols-ai/hardware-intel.nix @@ -1,5 +1,4 @@ { - pkgs, ... }: { @@ -18,34 +17,8 @@ ''; boot.extraModulePackages = [ ]; - # Intel Graphics - # https://wiki.nixos.org/wiki/Intel_Graphics - - services.xserver.videoDrivers = [ "modesetting" ]; - - hardware.graphics = { - enable = true; - extraPackages = with pkgs; [ - # Required for modern Intel GPUs (Xe iGPU and ARC) - intel-media-driver # VA-API (iHD) userspace - vpl-gpu-rt # oneVPL (QSV) runtime - - # Optional (compute / tooling): - intel-compute-runtime # OpenCL (NEO) + Level Zero for Arc/Xe - ]; - }; - - environment.sessionVariables = { - LIBVA_DRIVER_NAME = "iHD"; # Prefer the modern iHD backend - }; - - # May help if FFmpeg/VAAPI/QSV init fails (esp. on Arc with i915): + # The i5-13600KF has no iGPU; the RTX 4090 and vkms are the DRM devices. hardware.enableRedistributableFirmware = true; - # (dropped) `i915.enable_guc=3`: i915's `enable_guc` is a "dangerous" option - # (it taints the kernel), and since Alder Lake the driver already auto-enables - # HuC + GuC submission on non-TGL/RKL Gen12+ platforms -- including this Arrow - # Lake-S iGPU, where auto = `HuC | GuC submission` = 3. Re-add only if - # VAAPI/QSV init regresses. # May help services that have trouble accessing /dev/dri (e.g., jellyfin/plex): # users.users..extraGroups = [ "video" "render" ]; diff --git a/hosts/idols-ai/hardware-nvidia.nix b/hosts/idols-ai/hardware-nvidia.nix index ed5c8b21..adc1ed04 100644 --- a/hosts/idols-ai/hardware-nvidia.nix +++ b/hosts/idols-ai/hardware-nvidia.nix @@ -3,47 +3,17 @@ pkgs, ... }: -let - # prime.offload keeps the Intel iGPU as the default renderer so the dGPU stays - # free for LLM workloads, so each consumer that should use the RTX 4090 opts - # into PRIME render offload explicitly. Games need these variables in their - # Steam launch options: Steam's bubblewrap sandbox does not expose /run, so - # the nvidia-offload wrapper on PATH is unusable there. WiVRn's server is a - # headless Vulkan app and needs the same environment; headset-launched games - # start in separate units or an already running Steam client, so they need - # the same per-game launch option (see hosts/idols-ai/VR.md). - dgpuEnv = { - __NV_PRIME_RENDER_OFFLOAD = "1"; - __NV_PRIME_RENDER_OFFLOAD_PROVIDER = "NVIDIA-G0"; - __GLX_VENDOR_LIBRARY_NAME = "nvidia"; - __VK_LAYER_NV_optimus = "NVIDIA_only"; - }; -in { # =============================================================================================== # for Nvidia GPU # https://wiki.nixos.org/wiki/NVIDIA # https://wiki.hyprland.org/Nvidia/ + # + # The RTX 4090 is the only GPU on this host: the i5-13600KF has no iGPU, so + # there is no PRIME offload to configure and nothing to keep the dGPU "free" + # for. # =============================================================================================== - # Hybrid graphics with PRIME[integrated GPU (iGPU) + dedicated GPU (dGPU)] - hardware.nvidia.prime = { - # puts dGPU(Nvidia) to sleep and lets the iGPU handle all tasks by default. - offload = { - enable = true; - enableOffloadCmd = true; # generate a nvidia-offload command - }; - - intelBusId = "PCI:0@0:2:0"; - nvidiaBusId = "PCI:2@0:0:0"; - }; - - programs.gamescope.env = dgpuEnv; - - # Apply offload to the WiVRn server (see ai/vr.nix). Games need their own - # offload configuration; this service environment does not propagate to them. - services.wivrn.monadoEnvironment = dgpuEnv; - boot.kernelParams = [ # Since NVIDIA does not load kernel mode setting by default, # enabling it is required to make Wayland compositors function properly. @@ -72,23 +42,20 @@ in enable32Bit = true; }; - # GPU monitoring: CUDA btop (CAP_PERFMON also lets it read the Intel iGPU), - # nvtop, and the community nvidia_gpu_exporter. - modules.btop = { - package = pkgs.btop-cuda; - perfmon = true; - }; + # GPU monitoring: CUDA btop, nvtop, and the community nvidia_gpu_exporter. + modules.btop.package = pkgs.btop-cuda; environment.systemPackages = [ pkgs.nvtopPackages.full ]; services.prometheus.exporters.nvidia-gpu.enable = true; services.sunshine.settings = { - adapter_name = "/dev/dri/by-path/pci-0000:00:02.0-render"; # Intel iGPU - encoder = "vaapi"; + max_bitrate = 20000; # in Kbps + # No adapter_name: Sunshine picks the only adapter, the 4090. + nvenc_preset = 3; + nvenc_twopass = "full_res"; }; systemd.user.services.sunshine = { after = [ "niri.service" ]; - environment.LIBVA_DRIVER_NAME = "iHD"; preStart = '' for _ in {1..20}; do if ${config.programs.niri.package}/bin/niri msg outputs | ${pkgs.gnugrep}/bin/grep -q '^Output '; then diff --git a/hosts/idols-ai/niri-hardware.kdl b/hosts/idols-ai/niri-hardware.kdl index dbfe4a50..01b8accb 100644 --- a/hosts/idols-ai/niri-hardware.kdl +++ b/hosts/idols-ai/niri-hardware.kdl @@ -1,13 +1,10 @@ // Outputs are matched by monitor manufacturer/model/serial instead of the // connector name: the kernel/NVIDIA driver decides DRM connector numbering by -// probe order, so Intel's connectors were renamed DP-1 -> DP-4 and -// HDMI-A-2 -> HDMI-A-3 on the 2026-09 nixpkgs update. Run `niri msg outputs` -// to read the exact "make model serial" string. -debug { - // Keep desktop rendering and ordinary applications on the Intel iGPU. - // Physical outputs remain on the NVIDIA GPU and use cross-GPU scanout. - render-drm-device "/dev/dri/by-path/pci-0000:00:02.0-render" -} +// probe order. Run `niri msg outputs` to read the exact "make model serial" +// string. +// +// No renderer pinning: the RTX 4090 is the only GPU here, so niri's own device +// choice and the outputs' scanout are the same card. output "Beihai Century Joint Innovation Technology Co.,Ltd P32A2V 0000000000000" { // Uncomment this line to disable this output. diff --git a/outputs/x86_64-linux/tests/idols-ai-gpu/expected.nix b/outputs/x86_64-linux/tests/idols-ai-gpu/expected.nix index a9fb9732..bbfdccae 100644 --- a/outputs/x86_64-linux/tests/idols-ai-gpu/expected.nix +++ b/outputs/x86_64-linux/tests/idols-ai-gpu/expected.nix @@ -1,20 +1,22 @@ { loadsVirtualDisplay = true; createsVirtualDisplay = true; - niriUsesIntelRenderer = true; + niriPinsNoRenderDevice = true; + noIntelGraphicsStack = true; + noIntelLibvaOverride = true; + noPrimeOffload = true; sunshineUserHasInputAccess = false; sunshineHasSysAdmin = false; sunshineUsesWlrCapture = "wlr"; sunshineStartsAfterNiri = true; sunshineWaitsForNiriOutput = true; + sunshinePinsNoAdapter = true; wivrnEnabled = true; wivrnAutoStarts = false; wivrnHighPriority = true; wivrnWaitsForBus = true; wivrnImportsOpenXRToSteam = true; - wivrnUsesNvidiaVulkan = true; - wivrnSharesDgpuEnvWithGamescope = true; wivrnStatePreserved = true; adbStatePreserved = true; } diff --git a/outputs/x86_64-linux/tests/idols-ai-gpu/expr.nix b/outputs/x86_64-linux/tests/idols-ai-gpu/expr.nix index 1104ca06..7f7395a1 100644 --- a/outputs/x86_64-linux/tests/idols-ai-gpu/expr.nix +++ b/outputs/x86_64-linux/tests/idols-ai-gpu/expr.nix @@ -12,16 +12,36 @@ let lib.any ( d: d.directory == cfg.users.users.ryan.home + "/" + path && d.how == "bindmount" && d.mode == "0700" ) preserved; + # Compare package names: `builtins.elem "intel-media-driver"` compares a string + # against derivations and passes even when the driver is installed. + graphicsPackageNames = map (package: package.pname or "") ( + cfg.hardware.graphics.extraPackages or [ ] + ); in { loadsVirtualDisplay = builtins.elem "vkms" cfg.boot.kernelModules; createsVirtualDisplay = lib.hasInfix "options vkms create_default_dev=1" cfg.boot.extraModprobeConfig; - niriUsesIntelRenderer = lib.hasInfix "/dev/dri/by-path/pci-0000:00:02.0-render" niriHardware; + # The RTX 4090 is the only GPU here, so niri must not be pinned to a render node. + niriPinsNoRenderDevice = !lib.hasInfix "render-drm-device" niriHardware; + noIntelGraphicsStack = + !(builtins.any ( + name: + builtins.elem name [ + "intel-media-driver" + "vpl-gpu-rt" + "intel-compute-runtime" + ] + ) graphicsPackageNames); + # The same change dropped the VA-API override, which only an Intel GPU could serve. + noIntelLibvaOverride = !(cfg.environment.sessionVariables ? LIBVA_DRIVER_NAME); + noPrimeOffload = !cfg.hardware.nvidia.prime.offload.enable; sunshineUserHasInputAccess = builtins.elem "input" cfg.users.users.ryan.extraGroups; sunshineHasSysAdmin = cfg.services.sunshine.capSysAdmin; sunshineUsesWlrCapture = cfg.services.sunshine.settings.capture; sunshineStartsAfterNiri = builtins.elem "niri.service" cfg.systemd.user.services.sunshine.after; sunshineWaitsForNiriOutput = lib.hasInfix "niri msg outputs" cfg.systemd.user.services.sunshine.preStart; + # Sunshine encodes with NVENC on the 4090, so no VA-API adapter is pinned. + sunshinePinsNoAdapter = cfg.services.sunshine.settings.adapter_name or null == null; wivrnEnabled = cfg.services.wivrn.enable; wivrnAutoStarts = builtins.elem "default.target" cfg.systemd.user.services.wivrn.wantedBy; @@ -32,9 +52,6 @@ in && cfg.systemd.user.services.wivrn.serviceConfig.TimeoutStartSec == "30s"; wivrnImportsOpenXRToSteam = cfg.environment.sessionVariables.PRESSURE_VESSEL_IMPORT_OPENXR_1_RUNTIMES == "1"; - wivrnUsesNvidiaVulkan = cfg.services.wivrn.monadoEnvironment.__VK_LAYER_NV_optimus == "NVIDIA_only"; - wivrnSharesDgpuEnvWithGamescope = - cfg.services.wivrn.monadoEnvironment == cfg.programs.gamescope.env; wivrnStatePreserved = privatelyPreserved ".config/wivrn"; adbStatePreserved = privatelyPreserved ".android"; }