mirror of
https://github.com/ryan4yin/nix-config.git
synced 2026-11-13 12:13:46 +01:00
refactor(ai): return ai to the MSI MAG B760M MORTAR and the RTX 4090
The second NVMe and its LUKS volume leave with the Z890 board, so the data-disk disko module is deleted; `ai` must activate this change before the disk is pulled, or its initrd blocks on the missing volume. Assisted-by: DeepSeek Harness (deepseek-flash)
This commit is contained in:
Ryan Yin authored and ryan4yin | 二花 committed
1 parent
72a7a2ae88
commit
55b7741108
11 files changed
+144
-288
No files matched your search
@@ -1,98 +0,0 @@
|
||||
# Mixed memory on idols-ai
|
||||
|
||||
128G from JUHOR 2×48G + GLOWY 2×16G. All four DIMMs are needed because 96G is insufficient.
|
||||
|
||||
## Current configuration
|
||||
|
||||
CPU VDD2 is set to **1.17 V** (BIOS reads **1.168 V**). This is the best configuration tested so
|
||||
far, but it still froze during daily use on 2026-10-11. Keep it for normal use for now; higher
|
||||
tested VDD2 settings have not provided a stable alternative.
|
||||
|
||||
| Field | Setting |
|
||||
| ----------------------- | -------------------------------------------- |
|
||||
| XMP | Disabled |
|
||||
| Gear mode | Gear2 |
|
||||
| Memory speed | 4800 MT/s |
|
||||
| CPU VDD2 | **1.17 V fixed (BIOS reads 1.168 V)** |
|
||||
| VCCSA | Auto (BIOS reads 1.288 V) |
|
||||
| VDD2_MV (BIOS label) | Auto (BIOS reads 1.280 V) |
|
||||
| DRAM VDD | **1.28 V fixed** |
|
||||
| DRAM VDDQ | **1.28 V fixed** |
|
||||
| DRAM VPP | Auto (1.8 V) |
|
||||
| tCL / tRCD / tRP / tRAS | 40 / 40 / 40 / 77 (last photographed values) |
|
||||
|
||||
Voltage settings and readings come from BIOS and user reports, not OS measurements.
|
||||
|
||||
## Hardware
|
||||
|
||||
- CPU: Intel Core Ultra 7 270K Plus.
|
||||
- Board: Colorful CVN Z890 ARK FROZEN V20, BIOS 1013.
|
||||
- Each channel: 48G (2R) in DIMM0, 16G (1R) in DIMM1.
|
||||
- The 16G pair was added on 2026-10-06.
|
||||
|
||||
| Kit | Capacity | XMP rating | Timings | Voltage | Rank | Part number |
|
||||
| ----- | -------- | ---------- | ------------- | ------- | ---- | --------------------- |
|
||||
| JUHOR | 2×48G | DDR5-5600 | CL46-45-45-90 | 1.25 V | 2R | `JHE5600U4648JG` |
|
||||
| GLOWY | 2×16G | DDR5-6400 | CL32-38-38-90 | 1.35 V | 1R | `VGM5UH64C32AG-DTACW` |
|
||||
|
||||
## Bring-up and recovery
|
||||
|
||||
Earlier attempts starting with the 48G pair or all four DIMMs did not reach BIOS. Working order:
|
||||
|
||||
1. Install only the 2×16G in `ChannelA-DIMM1` / `ChannelB-DIMM1`.
|
||||
2. Enter BIOS and disable XMP.
|
||||
3. Power off and add the 2×48G in DIMM0.
|
||||
4. Allow 1 to 3 minutes for retraining, possibly with several reboots.
|
||||
5. Apply the current settings above.
|
||||
|
||||
If failed training prevents BIOS entry, repeat this order; clear CMOS first if necessary. Failed
|
||||
training can silently restore defaults; re-check settings in BIOS afterwards.
|
||||
|
||||
## Test results
|
||||
|
||||
CPU VDD2 entries below are BIOS settings: Auto read 1.104 V in the earlier DRAM tests; the 1.17 V
|
||||
setting reads 1.168 V. These BIOS readings are not measurements during the OS tests.
|
||||
|
||||
| CPU VDD2 | DRAM VDD | DRAM VDDQ | Result | Highest sampled DIMM temperature |
|
||||
| ---------- | -------------------- | -------------------- | ------------------------------------------------------------------------------------------------------ | ---------------------------------------------- |
|
||||
| Auto | 1.10 V | 1.10 V | Immediate freezes during memory tests, even 4G. | Not recorded |
|
||||
| Auto | 1.25 V | 1.25 V | 8G / 32G / 100G short tests passed; llama.cpp load froze. | Not recorded |
|
||||
| Auto | 1.28 V | 1.28 V | 100G short test passed; daily use froze on 2026-10-08. | Not recorded |
|
||||
| Auto | 1.282 V | 1.282 V | Display corruption in BIOS. | Not recorded |
|
||||
| Auto | 1.29 / 1.30 / 1.32 V | 1.29 / 1.30 / 1.32 V | Each setting failed training. | Not recorded |
|
||||
| 1.13 V | 1.28 V | 1.28 V | BIOS normal; no long test. | Not recorded |
|
||||
| 1.15 V | 1.28 V | 1.28 V | 100G, 15 minutes: PASS. | 45.5°C* |
|
||||
| 1.15 V | 1.28 V | 1.28 V | Long test froze; last saved duration 21 min 17 s. | 56.5°C |
|
||||
| **1.17 V** | **1.28 V** | **1.28 V** | **100G, 6 hours: PASS, 0 errors or hardware incidents.** | **56.8°C** |
|
||||
| 1.17 V | 1.28 V | 1.28 V | About 56 hours across three boots ending cleanly; next boot froze during LLM serving (2026-10-11). | 47.2°C in the last available monitoring sample |
|
||||
| 1.20 V | 1.28 V | 1.28 V | Earlier attempt: BIOS display corruption and freeze within 10 to 20 s. | Not recorded |
|
||||
| 1.20 V | 1.28 V | 1.28 V | 100G test froze with screen corruption; last saved duration 7 min 23 s (2026-10-11). | 55.25°C |
|
||||
| 1.18 V | 1.28 V | 1.28 V | 100G test froze with screen corruption; last saved duration 14 min 36 s (2026-10-11). | 59.5°C |
|
||||
| 1.28 V | 1.28 V | 1.28 V | Reached the desktop, then froze; journal spans 16:30:16 to 16:31:20 (64 s). No recorded pressure test. | Not recorded |
|
||||
| 1.30 V | 1.28 V | 1.28 V | Set in BIOS only; rolled back without booting or testing. | Not tested |
|
||||
| **1.17 V** | **1.28 V** | **1.28 V** | **100G, 15 minutes: PASS, 0 errors or hardware incidents (2026-10-11 17:29–17:45).** | **59.25°C** |
|
||||
|
||||
*The 1.15 V short test used sparse temperature samples; its true peak is unknown. The monitored long
|
||||
tests and the 2026-10-11 100G tests sampled about every 5 seconds with a 60°C stop threshold. Saved
|
||||
durations are lower bounds on time before interruption, not exact freeze times.
|
||||
|
||||
The passing 1.17 V test ran on 2026-10-09 from 01:29 to 07:30 without swap use. Subsequent daily use
|
||||
included displays on and off with LLM serving. Sleep/wake is untested.
|
||||
|
||||
The 1.17 V freeze on 2026-10-11 ended logging at 13:55:46, about 4 h 47 min after boot. Logs stopped
|
||||
during generation without a shutdown sequence or a recorded machine check, BUG or Oops.
|
||||
|
||||
## BIOS limitations and conclusion
|
||||
|
||||
Attempts to apply tRFC 1180 and primary timings 42 / 42 / 42 / 82 reverted automatically after
|
||||
saving, according to the user's BIOS checks. Those relaxed settings never became a tested
|
||||
configuration. The memory frequency control is also locked at 4800 MT/s in the current BIOS setup.
|
||||
The table above retains the last photographed timings; a later readback has not been recorded.
|
||||
|
||||
Keep CPU VDD2 at 1.17 V and DRAM VDD/VDDQ at 1.28 V for now. The passing stress test and daily-use
|
||||
record make this the best tested option, but the later freeze means stability remains unresolved.
|
||||
|
||||
The user reports a possible ambient temperature rise of about 10°C. Temperature may affect the
|
||||
remaining margin, but neither temperature nor an IMC, DIMM or motherboard fault has been isolated.
|
||||
Manual timing relaxation and frequency reduction are currently unavailable; further tuning is
|
||||
paused. Revisit only if freezes continue or those BIOS controls become usable.
|
||||
@@ -3,17 +3,75 @@
|
||||
Desktop (NixOS + preservation, LUKS + btrfs on nvme). Disk layout is declarative via
|
||||
[disko](./disko-fs.nix) (target device: the KINGBANK NVMe by `/dev/disk/by-id/`, set in that file).
|
||||
|
||||
Desktop on the MSI **MAG B760M MORTAR** with the i5-13600KF, 32G DDR5 and the RTX 4090, on the
|
||||
KINGBANK NVMe alone. The platform split is recorded in [hosts/README.md](../README.md); the wired
|
||||
NIC and its `en*` match are in [default.nix](./default.nix).
|
||||
|
||||
TPM2 comes from the CPU's Intel PTT unless a module is plugged into the board's TPM header; check
|
||||
_Settings → Security → Trusted Computing_ if `systemd-cryptenroll --tpm2-device=auto` finds no
|
||||
device.
|
||||
|
||||
Related:
|
||||
|
||||
- [nixos-installer README](../../nixos-installer/README.md) – install from ISO using disko
|
||||
- [disko-fs.nix](./disko-fs.nix) – main disk layout (ESP + LUKS + btrfs). From
|
||||
`nix-config/nixos-installer`:
|
||||
`nix run github:nix-community/disko -- --mode destroy,format,mount ../hosts/idols-ai/disko-fs.nix`
|
||||
- [disko-fs-data.nix](./disko-fs-data.nix) – data disk layout (LUKS + btrfs at /data)
|
||||
- [MIXED-MEMORY.md](./MIXED-MEMORY.md) – mixed 4-DIMM RAM bring-up, stress testing, and recovery
|
||||
- [VR.md](./VR.md) – WiVRn headset setup
|
||||
|
||||
## Info
|
||||
|
||||
The disk layout is fully declarative — see [disko-fs.nix](./disko-fs.nix) for the root disk and
|
||||
[disko-fs-data.nix](./disko-fs-data.nix) for the data disk. Run `df -Th` / `lsblk` on the host if an
|
||||
up-to-date snapshot of the current mounts is needed.
|
||||
The disk layout is fully declarative — see [disko-fs.nix](./disko-fs.nix) for the root disk. Run
|
||||
`df -Th` / `lsblk` on the host if an up-to-date snapshot of the current mounts is needed.
|
||||
|
||||
Steam libraries live under the persisted home directory, so they survive the tmpfs root:
|
||||
`~/Games/SteamLibrary` is the library that used to sit on the `/data` disk. Steam finds a moved
|
||||
library when it is added once through _Settings → Storage → Add Drive_, or by starting Steam with
|
||||
the `libraryfolders.vdf` already listing it.
|
||||
|
||||
## 4090 water block
|
||||
|
||||
After the 2026-10-13 platform swap the card's water block stopped moving heat: the pump ran but made
|
||||
abnormal noise, and the card still overheated. It had been RMA'd for the same symptom before, came
|
||||
back with no fault found, and worked again after the return shipment. Repacking the card in its
|
||||
original box and dropping the box five or six times freed it, so a stuck impeller, with transit
|
||||
shock as the accidental cure, is the best guess. Treat that as a stopgap before an RMA: the jolt can
|
||||
damage the card, and a pump that sticks again should be replaced.
|
||||
|
||||
## Booting after the platform swap
|
||||
|
||||
The KINGBANK disk was last enrolled with the Z890 board's TPM2 and its UKI is signed with a key that
|
||||
only that board's Secure Boot database trusts. On the B760M board:
|
||||
|
||||
1. Boot with the LUKS **passphrase** — the old TPM2 enrollment is bound to the previous board's PCR
|
||||
values and will not unlock here.
|
||||
2. Secure Boot: either leave it off, or enroll a fresh PKI — `sbctl create-keys` +
|
||||
`sbctl enroll-keys` in setup mode. lanzaboote signs with `pkiBundle` (`/etc/secureboot` in
|
||||
[secureboot.nix](./secureboot.nix)), so the new keys must end up there; `sbctl create-keys`
|
||||
writes to its own keydir. Re-enrolling changes PCR7, so re-run the enrollment in step 3
|
||||
afterwards.
|
||||
3. Re-enroll this board's TPM2 once the Secure Boot state is final. Enroll and wipe in **one** call:
|
||||
`--wipe-slot=tpm2` matches every TPM2 slot, so wiping in a separate call would also delete the
|
||||
slot just created. Combined, systemd-cryptenroll enrolls first and then wipes the old slots,
|
||||
always sparing the new one.
|
||||
|
||||
```bash
|
||||
disk=/dev/disk/by-id/nvme-KINGBANK_KP260_CN092BH3203917-part2
|
||||
sudo systemd-cryptenroll --tpm2-device=auto --tpm2-pcrs=0+7 --wipe-slot=tpm2 "$disk"
|
||||
sudo systemd-cryptenroll "$disk" # read the slot list back: tpm2 + passphrase
|
||||
```
|
||||
|
||||
PCR0 covers the firmware code, so a BIOS update needs this enrollment redone; the passphrase slot
|
||||
keeps booting either way. The `0+7` pair is the repo's existing shape — see
|
||||
[nixos-installer/README.md](../../nixos-installer/README.md).
|
||||
|
||||
4. The B760M's NVRAM has no boot entry for this disk. If it does not boot on its own, pick the disk
|
||||
once from the one-time boot menu (the ESP also carries `\EFI\BOOT\BOOTX64.EFI`); the first
|
||||
activation should then write the entry. If it does not, add one by hand — `ls /boot/EFI/Linux`
|
||||
shows the UKI name lanzaboote signed:
|
||||
|
||||
```bash
|
||||
sudo ls /boot/EFI/Linux
|
||||
sudo nix run nixpkgs#efibootmgr -- -c -d /dev/disk/by-id/nvme-KINGBANK_KP260_CN092BH3203917 \
|
||||
-p 1 -L NixOS -l '\EFI\Linux\nixos-generation-1.efi'
|
||||
```
|
||||
+17
-18
@@ -67,7 +67,7 @@ Manage paired headsets with `wivrnctl list-paired`, `wivrnctl rename`, or `wivrn
|
||||
|
||||
## Service, settings and logs
|
||||
|
||||
Manage the server through systemd so starts retain NVIDIA offload and the CAP_SYS_NICE wrapper:
|
||||
Manage the server through systemd so starts retain the CAP_SYS_NICE wrapper:
|
||||
|
||||
```console
|
||||
systemctl --user restart wivrn
|
||||
@@ -83,26 +83,26 @@ configuration schema, then restart the service. The NixOS module also supports d
|
||||
configuration; an enabled declarative config takes precedence over the user file. Headset-side
|
||||
settings remain available in the headset application.
|
||||
|
||||
## Launch games on the RTX 4090
|
||||
## Games and the RTX 4090
|
||||
|
||||
Set this in each VR game's Steam Properties -> Launch Options:
|
||||
The RTX 4090 is the only GPU on this host, so the WiVRn server, the compositor and the game land on
|
||||
the same card by themselves.
|
||||
|
||||
That is not how it used to be. While `ai` ran the Z890 board, its Arrow Lake iGPU did the desktop
|
||||
rendering under PRIME offload, and a game left on the iGPU while the server ran on the dGPU failed
|
||||
with `vkAllocateMemory: VK_ERROR_OUT_OF_DEVICE_MEMORY` and `xrt_comp_create_swapchain failed` in
|
||||
`journalctl --user -u wivrn`, followed by an xrizer panic that killed the game. The fix then was the
|
||||
variables below, written out as Steam launch options because Steam runs inside a bubblewrap sandbox
|
||||
that does not mount `/run`, so it cannot see `/run/current-system/sw/bin/nvidia-offload`:
|
||||
|
||||
```text
|
||||
__NV_PRIME_RENDER_OFFLOAD=1 __NV_PRIME_RENDER_OFFLOAD_PROVIDER=NVIDIA-G0 __GLX_VENDOR_LIBRARY_NAME=nvidia __VK_LAYER_NV_optimus=NVIDIA_only %command%
|
||||
```
|
||||
|
||||
These are the variables the system's `nvidia-offload` wrapper sets, written out because Steam runs
|
||||
inside a bubblewrap sandbox that does not mount `/run`, so it cannot see
|
||||
`/run/current-system/sw/bin/nvidia-offload`.
|
||||
|
||||
The game has to land on the same GPU as the server. A game left on the Intel iGPU makes the server's
|
||||
swapchain allocation fail, which shows up as `vkAllocateMemory: VK_ERROR_OUT_OF_DEVICE_MEMORY` and
|
||||
`xrt_comp_create_swapchain failed` in `journalctl --user -u wivrn`, followed by an xrizer panic that
|
||||
kills the game. This applies when launching from the headset too: WiVRn starts applications in
|
||||
separate systemd units, and an already running Steam client keeps its own environment, so neither
|
||||
path inherits the server's GPU environment. For non-Steam applications, start them under
|
||||
`nvidia-offload` from a normal shell. Select a Proton version according to the game's compatibility
|
||||
reports; GE-Proton or DW-Proton alone does not guarantee VR compatibility.
|
||||
It mattered for headset-launched games too: WiVRn starts applications in separate systemd units, and
|
||||
an already running Steam client keeps its own environment. Re-add those variables only if a second
|
||||
GPU comes back. Select a Proton version according to the game's compatibility reports; GE-Proton or
|
||||
DW-Proton alone does not guarantee VR compatibility.
|
||||
|
||||
## USB connection
|
||||
|
||||
@@ -131,9 +131,8 @@ mapping after disconnecting USB.
|
||||
|
||||
The package enables Vulkan Video encoding but does not enable NVENC. WiVRn 26.9 probes Vulkan
|
||||
encoding before falling back to VAAPI or x264 when NVENC is absent. Check the journal's encoder
|
||||
configuration after connecting: offload variables alone do not prove hardware encoding. Vulkan
|
||||
encoding in this release supports H.264/H.265, not AV1. NVENC/AV1 requires a CUDA-enabled WiVRn
|
||||
build.
|
||||
configuration after connecting: GPU activity alone does not prove hardware encoding. Vulkan encoding
|
||||
in this release supports H.264/H.265, not AV1. NVENC/AV1 requires a CUDA-enabled WiVRn build.
|
||||
|
||||
- Pair, connect, launch a game, and check both compositor and game GPU usage:
|
||||
`nvidia-smi --query-compute-apps=pid,process_name,used_memory --format=csv` has to list both
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{ pkgs, ... }:
|
||||
{
|
||||
# WiVRn supplies the OpenXR runtime; xrizer translates OpenVR games.
|
||||
# The server uses the host GPU environment from hardware-nvidia.nix.
|
||||
# The RTX 4090 is the only GPU, so the server needs no GPU environment of its own.
|
||||
services.wivrn = {
|
||||
enable = true;
|
||||
autoStart = false; # Start only when requested with just vr.
|
||||
@@ -24,8 +24,8 @@
|
||||
'';
|
||||
};
|
||||
|
||||
# CAP_SYS_NICE for the server. Games need their own PRIME offload: WiVRn
|
||||
# starts them in separate units, which do not inherit this environment.
|
||||
# CAP_SYS_NICE for the server. WiVRn starts games in separate units, and an
|
||||
# already running Steam client keeps its own environment.
|
||||
highPriority = true;
|
||||
steam.importOXRRuntimes = true;
|
||||
};
|
||||
|
||||
+19
-16
@@ -7,8 +7,8 @@
|
||||
}:
|
||||
#############################################################
|
||||
#
|
||||
# Ai - my main computer, with NixOS + Core Ultra 7 270K Plus + RTX 4090 GPU,
|
||||
# 128G DDR5-4800 (mixed 4-DIMM, see MIXED-MEMORY.md), for gaming & daily use.
|
||||
# Ai - my gaming and daily-use desktop. Hardware and the platform history:
|
||||
# ./README.md.
|
||||
#
|
||||
#############################################################
|
||||
let
|
||||
@@ -24,7 +24,6 @@ in
|
||||
disko.nixosModules.default
|
||||
# disks
|
||||
./disko-fs.nix
|
||||
./disko-fs-data.nix
|
||||
|
||||
# Include the results of the hardware scan.
|
||||
./hardware-configuration.nix
|
||||
@@ -45,15 +44,13 @@ in
|
||||
# disk swapfile instead, created by disko at /swap (see ./disko-fs.nix). The two must move together.
|
||||
modules.zram.enable = false;
|
||||
|
||||
# zswap: compressed writeback cache in front of the disk swapfile.
|
||||
# Keeps swapped cold anon pages compressed in RAM instead of the SSD, which frees more
|
||||
# page cache for the ~78GB mmap'd LLM weights (mmap file pages never go through zswap).
|
||||
# Safe here: disk swapfile remains the real backing store, no zram deadlock structure.
|
||||
# - zstd: CPU is not the bottleneck here (llama.cpp peaks ~30% util); its ~2.5x ratio
|
||||
# keeps ~5G more anon bytes in the capped pool than lz4, and ~5us decompress is noise
|
||||
# vs the ~100us SSD fault it avoids.
|
||||
# - 10% pool (~13G on 128G): the module default 25% (~32G) would compete with the weight
|
||||
# cache.
|
||||
# zswap: compressed writeback cache in front of the disk swapfile, so cold
|
||||
# anonymous pages stay in RAM instead of hitting the SSD. Safe here: the disk
|
||||
# swapfile remains the real backing store, no zram deadlock structure.
|
||||
# - zstd: ~2.5x the ratio of lz4 for ~5us decompress, which is noise against
|
||||
# the ~100us SSD fault it avoids; the CPU is not the bottleneck.
|
||||
# - 10% pool (~3G on 32G): the module default 25% would compete with the page
|
||||
# cache that games and VMs rely on.
|
||||
boot.zswap = {
|
||||
enable = true;
|
||||
compressor = "zstd";
|
||||
@@ -65,7 +62,9 @@ in
|
||||
|
||||
modules.btrbk.enable = true;
|
||||
|
||||
# Permit the desktop user to access Intel accelerator devices for local ASR.
|
||||
# Give the desktop user the render group outright instead of relying only on
|
||||
# the logind seat ACL, so local ASR keeps its /dev/dri access in sessions
|
||||
# that have no seat (the Sunshine headless session, for example).
|
||||
users.users.${myvars.username}.extraGroups = [ "render" ];
|
||||
|
||||
# systemd-journal - cap disk usage
|
||||
@@ -105,13 +104,15 @@ in
|
||||
|
||||
# networkd removes routing policy rules not defined in .network files
|
||||
# (ManageForeignRoutingPolicyRules defaults to yes) each time it reconfigures
|
||||
# a link. On S3 resume it reconfigures enp130s0, which wiped mihomo's
|
||||
# a link. On S3 resume it reconfigures the NIC, which wiped mihomo's
|
||||
# (9000-9010) and Tailscale's rules and briefly broke routing. Keep foreign
|
||||
# rules so the VPN/TUN rules survive.
|
||||
systemd.network.config.networkConfig.ManageForeignRoutingPolicyRules = false;
|
||||
|
||||
systemd.network.networks."10-${iface}" = {
|
||||
matchConfig.Name = [ iface ];
|
||||
# Predictable names follow the PCIe topology, so a board or slot change renames the
|
||||
# wired NIC. Match the `en*` prefix instead of a name; one wired port, so one link.
|
||||
systemd.network.networks."10-lan" = {
|
||||
matchConfig.Name = [ "en*" ];
|
||||
networkConfig = {
|
||||
Address = [
|
||||
ipv4WithMask
|
||||
@@ -153,6 +154,8 @@ in
|
||||
# parsing is where missing geodata gets fetched. Flipping at fork time pointed
|
||||
# that fetch at a port nobody was listening on -- SERVFAIL, a fatal config
|
||||
# error, a unit that could never start.
|
||||
# The hook names the link with `hostsAddr.ai.iface`: networkd matches an `en*`
|
||||
# prefix, but `resolvectl` takes one name or index, not a pattern.
|
||||
systemd.services.mihomo.serviceConfig = {
|
||||
ExecStartPost = lib.mkDefault "+${pkgs.writeShellScript "mihomo-dns-takeover" ''
|
||||
probe() { (exec 3<>/dev/tcp/127.0.0.1/1053) 2>/dev/null; }
|
||||
|
||||
@@ -1,62 +0,0 @@
|
||||
# Disko layout for idols-ai data disk (LUKS + btrfs, mount at /data).
|
||||
#
|
||||
# Destroy, format & mount (wipes disk; from nixos-installer: cd nix-config/nixos-installer):
|
||||
# nix run github:nix-community/disko -- --mode destroy,format,mount ../hosts/idols-ai/disko-fs-data.nix
|
||||
# Mount only (after first format):
|
||||
# nix run github:nix-community/disko -- --mode mount ../hosts/idols-ai/disko-fs-data.nix
|
||||
#
|
||||
{
|
||||
disko.devices = {
|
||||
disk.data = {
|
||||
type = "disk";
|
||||
device = "/dev/disk/by-id/nvme-Fanxiang_S790_2TB_FXS790254050582";
|
||||
content = {
|
||||
type = "gpt";
|
||||
partitions = {
|
||||
datapart = {
|
||||
size = "100%";
|
||||
content = {
|
||||
type = "luks";
|
||||
name = "data-luks"; # Mapper name; match boot.initrd.luks
|
||||
settings = {
|
||||
allowDiscards = true; # TRIM for SSDs; slightly less secure, better performance
|
||||
};
|
||||
# Add boot.initrd.luks.devices so initrd prompts for passphrase at boot
|
||||
initrdUnlock = true;
|
||||
# cryptsetup luksFormat options
|
||||
extraFormatArgs = [
|
||||
"--type luks2"
|
||||
"--cipher aes-xts-plain64"
|
||||
"--hash sha512"
|
||||
"--iter-time 5000"
|
||||
"--key-size 256"
|
||||
"--pbkdf argon2id"
|
||||
"--use-random" # Block until enough entropy from /dev/random
|
||||
];
|
||||
extraOpenArgs = [
|
||||
"--timeout 10"
|
||||
];
|
||||
content = {
|
||||
type = "btrfs";
|
||||
extraArgs = [ "-f" ]; # Force overwrite if filesystem already exists
|
||||
subvolumes = {
|
||||
"@data" = {
|
||||
mountpoint = "/data";
|
||||
mountOptions = [
|
||||
"compress-force=zstd:1"
|
||||
];
|
||||
};
|
||||
};
|
||||
postMountHook = ''
|
||||
chown ryan:users /mnt/data
|
||||
# Set SGID + rwx for owner/group, read-only for others; new files inherit group
|
||||
chmod 2755 /mnt/data
|
||||
'';
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
}
|
||||
@@ -1,5 +1,4 @@
|
||||
{
|
||||
pkgs,
|
||||
...
|
||||
}:
|
||||
{
|
||||
@@ -18,34 +17,8 @@
|
||||
'';
|
||||
boot.extraModulePackages = [ ];
|
||||
|
||||
# Intel Graphics
|
||||
# https://wiki.nixos.org/wiki/Intel_Graphics
|
||||
|
||||
services.xserver.videoDrivers = [ "modesetting" ];
|
||||
|
||||
hardware.graphics = {
|
||||
enable = true;
|
||||
extraPackages = with pkgs; [
|
||||
# Required for modern Intel GPUs (Xe iGPU and ARC)
|
||||
intel-media-driver # VA-API (iHD) userspace
|
||||
vpl-gpu-rt # oneVPL (QSV) runtime
|
||||
|
||||
# Optional (compute / tooling):
|
||||
intel-compute-runtime # OpenCL (NEO) + Level Zero for Arc/Xe
|
||||
];
|
||||
};
|
||||
|
||||
environment.sessionVariables = {
|
||||
LIBVA_DRIVER_NAME = "iHD"; # Prefer the modern iHD backend
|
||||
};
|
||||
|
||||
# May help if FFmpeg/VAAPI/QSV init fails (esp. on Arc with i915):
|
||||
# The i5-13600KF has no iGPU; the RTX 4090 and vkms are the DRM devices.
|
||||
hardware.enableRedistributableFirmware = true;
|
||||
# (dropped) `i915.enable_guc=3`: i915's `enable_guc` is a "dangerous" option
|
||||
# (it taints the kernel), and since Alder Lake the driver already auto-enables
|
||||
# HuC + GuC submission on non-TGL/RKL Gen12+ platforms -- including this Arrow
|
||||
# Lake-S iGPU, where auto = `HuC | GuC submission` = 3. Re-add only if
|
||||
# VAAPI/QSV init regresses.
|
||||
|
||||
# May help services that have trouble accessing /dev/dri (e.g., jellyfin/plex):
|
||||
# users.users.<service>.extraGroups = [ "video" "render" ];
|
||||
|
||||
@@ -3,47 +3,17 @@
|
||||
pkgs,
|
||||
...
|
||||
}:
|
||||
let
|
||||
# prime.offload keeps the Intel iGPU as the default renderer so the dGPU stays
|
||||
# free for LLM workloads, so each consumer that should use the RTX 4090 opts
|
||||
# into PRIME render offload explicitly. Games need these variables in their
|
||||
# Steam launch options: Steam's bubblewrap sandbox does not expose /run, so
|
||||
# the nvidia-offload wrapper on PATH is unusable there. WiVRn's server is a
|
||||
# headless Vulkan app and needs the same environment; headset-launched games
|
||||
# start in separate units or an already running Steam client, so they need
|
||||
# the same per-game launch option (see hosts/idols-ai/VR.md).
|
||||
dgpuEnv = {
|
||||
__NV_PRIME_RENDER_OFFLOAD = "1";
|
||||
__NV_PRIME_RENDER_OFFLOAD_PROVIDER = "NVIDIA-G0";
|
||||
__GLX_VENDOR_LIBRARY_NAME = "nvidia";
|
||||
__VK_LAYER_NV_optimus = "NVIDIA_only";
|
||||
};
|
||||
in
|
||||
{
|
||||
# ===============================================================================================
|
||||
# for Nvidia GPU
|
||||
# https://wiki.nixos.org/wiki/NVIDIA
|
||||
# https://wiki.hyprland.org/Nvidia/
|
||||
#
|
||||
# The RTX 4090 is the only GPU on this host: the i5-13600KF has no iGPU, so
|
||||
# there is no PRIME offload to configure and nothing to keep the dGPU "free"
|
||||
# for.
|
||||
# ===============================================================================================
|
||||
|
||||
# Hybrid graphics with PRIME[integrated GPU (iGPU) + dedicated GPU (dGPU)]
|
||||
hardware.nvidia.prime = {
|
||||
# puts dGPU(Nvidia) to sleep and lets the iGPU handle all tasks by default.
|
||||
offload = {
|
||||
enable = true;
|
||||
enableOffloadCmd = true; # generate a nvidia-offload command
|
||||
};
|
||||
|
||||
intelBusId = "PCI:0@0:2:0";
|
||||
nvidiaBusId = "PCI:2@0:0:0";
|
||||
};
|
||||
|
||||
programs.gamescope.env = dgpuEnv;
|
||||
|
||||
# Apply offload to the WiVRn server (see ai/vr.nix). Games need their own
|
||||
# offload configuration; this service environment does not propagate to them.
|
||||
services.wivrn.monadoEnvironment = dgpuEnv;
|
||||
|
||||
boot.kernelParams = [
|
||||
# Since NVIDIA does not load kernel mode setting by default,
|
||||
# enabling it is required to make Wayland compositors function properly.
|
||||
@@ -72,23 +42,20 @@ in
|
||||
enable32Bit = true;
|
||||
};
|
||||
|
||||
# GPU monitoring: CUDA btop (CAP_PERFMON also lets it read the Intel iGPU),
|
||||
# nvtop, and the community nvidia_gpu_exporter.
|
||||
modules.btop = {
|
||||
package = pkgs.btop-cuda;
|
||||
perfmon = true;
|
||||
};
|
||||
# GPU monitoring: CUDA btop, nvtop, and the community nvidia_gpu_exporter.
|
||||
modules.btop.package = pkgs.btop-cuda;
|
||||
environment.systemPackages = [ pkgs.nvtopPackages.full ];
|
||||
services.prometheus.exporters.nvidia-gpu.enable = true;
|
||||
|
||||
services.sunshine.settings = {
|
||||
adapter_name = "/dev/dri/by-path/pci-0000:00:02.0-render"; # Intel iGPU
|
||||
encoder = "vaapi";
|
||||
max_bitrate = 20000; # in Kbps
|
||||
# No adapter_name: Sunshine picks the only adapter, the 4090.
|
||||
nvenc_preset = 3;
|
||||
nvenc_twopass = "full_res";
|
||||
};
|
||||
|
||||
systemd.user.services.sunshine = {
|
||||
after = [ "niri.service" ];
|
||||
environment.LIBVA_DRIVER_NAME = "iHD";
|
||||
preStart = ''
|
||||
for _ in {1..20}; do
|
||||
if ${config.programs.niri.package}/bin/niri msg outputs | ${pkgs.gnugrep}/bin/grep -q '^Output '; then
|
||||
|
||||
@@ -1,13 +1,10 @@
|
||||
// Outputs are matched by monitor manufacturer/model/serial instead of the
|
||||
// connector name: the kernel/NVIDIA driver decides DRM connector numbering by
|
||||
// probe order, so Intel's connectors were renamed DP-1 -> DP-4 and
|
||||
// HDMI-A-2 -> HDMI-A-3 on the 2026-09 nixpkgs update. Run `niri msg outputs`
|
||||
// to read the exact "make model serial" string.
|
||||
debug {
|
||||
// Keep desktop rendering and ordinary applications on the Intel iGPU.
|
||||
// Physical outputs remain on the NVIDIA GPU and use cross-GPU scanout.
|
||||
render-drm-device "/dev/dri/by-path/pci-0000:00:02.0-render"
|
||||
}
|
||||
// probe order. Run `niri msg outputs` to read the exact "make model serial"
|
||||
// string.
|
||||
//
|
||||
// No renderer pinning: the RTX 4090 is the only GPU here, so niri's own device
|
||||
// choice and the outputs' scanout are the same card.
|
||||
|
||||
output "Beihai Century Joint Innovation Technology Co.,Ltd P32A2V 0000000000000" {
|
||||
// Uncomment this line to disable this output.
|
||||
|
||||
@@ -1,20 +1,22 @@
|
||||
{
|
||||
loadsVirtualDisplay = true;
|
||||
createsVirtualDisplay = true;
|
||||
niriUsesIntelRenderer = true;
|
||||
niriPinsNoRenderDevice = true;
|
||||
noIntelGraphicsStack = true;
|
||||
noIntelLibvaOverride = true;
|
||||
noPrimeOffload = true;
|
||||
sunshineUserHasInputAccess = false;
|
||||
sunshineHasSysAdmin = false;
|
||||
sunshineUsesWlrCapture = "wlr";
|
||||
sunshineStartsAfterNiri = true;
|
||||
sunshineWaitsForNiriOutput = true;
|
||||
sunshinePinsNoAdapter = true;
|
||||
|
||||
wivrnEnabled = true;
|
||||
wivrnAutoStarts = false;
|
||||
wivrnHighPriority = true;
|
||||
wivrnWaitsForBus = true;
|
||||
wivrnImportsOpenXRToSteam = true;
|
||||
wivrnUsesNvidiaVulkan = true;
|
||||
wivrnSharesDgpuEnvWithGamescope = true;
|
||||
wivrnStatePreserved = true;
|
||||
adbStatePreserved = true;
|
||||
}
|
||||
@@ -12,16 +12,36 @@ let
|
||||
lib.any (
|
||||
d: d.directory == cfg.users.users.ryan.home + "/" + path && d.how == "bindmount" && d.mode == "0700"
|
||||
) preserved;
|
||||
# Compare package names: `builtins.elem "intel-media-driver"` compares a string
|
||||
# against derivations and passes even when the driver is installed.
|
||||
graphicsPackageNames = map (package: package.pname or "") (
|
||||
cfg.hardware.graphics.extraPackages or [ ]
|
||||
);
|
||||
in
|
||||
{
|
||||
loadsVirtualDisplay = builtins.elem "vkms" cfg.boot.kernelModules;
|
||||
createsVirtualDisplay = lib.hasInfix "options vkms create_default_dev=1" cfg.boot.extraModprobeConfig;
|
||||
niriUsesIntelRenderer = lib.hasInfix "/dev/dri/by-path/pci-0000:00:02.0-render" niriHardware;
|
||||
# The RTX 4090 is the only GPU here, so niri must not be pinned to a render node.
|
||||
niriPinsNoRenderDevice = !lib.hasInfix "render-drm-device" niriHardware;
|
||||
noIntelGraphicsStack =
|
||||
!(builtins.any (
|
||||
name:
|
||||
builtins.elem name [
|
||||
"intel-media-driver"
|
||||
"vpl-gpu-rt"
|
||||
"intel-compute-runtime"
|
||||
]
|
||||
) graphicsPackageNames);
|
||||
# The same change dropped the VA-API override, which only an Intel GPU could serve.
|
||||
noIntelLibvaOverride = !(cfg.environment.sessionVariables ? LIBVA_DRIVER_NAME);
|
||||
noPrimeOffload = !cfg.hardware.nvidia.prime.offload.enable;
|
||||
sunshineUserHasInputAccess = builtins.elem "input" cfg.users.users.ryan.extraGroups;
|
||||
sunshineHasSysAdmin = cfg.services.sunshine.capSysAdmin;
|
||||
sunshineUsesWlrCapture = cfg.services.sunshine.settings.capture;
|
||||
sunshineStartsAfterNiri = builtins.elem "niri.service" cfg.systemd.user.services.sunshine.after;
|
||||
sunshineWaitsForNiriOutput = lib.hasInfix "niri msg outputs" cfg.systemd.user.services.sunshine.preStart;
|
||||
# Sunshine encodes with NVENC on the 4090, so no VA-API adapter is pinned.
|
||||
sunshinePinsNoAdapter = cfg.services.sunshine.settings.adapter_name or null == null;
|
||||
|
||||
wivrnEnabled = cfg.services.wivrn.enable;
|
||||
wivrnAutoStarts = builtins.elem "default.target" cfg.systemd.user.services.wivrn.wantedBy;
|
||||
@@ -32,9 +52,6 @@ in
|
||||
&& cfg.systemd.user.services.wivrn.serviceConfig.TimeoutStartSec == "30s";
|
||||
wivrnImportsOpenXRToSteam =
|
||||
cfg.environment.sessionVariables.PRESSURE_VESSEL_IMPORT_OPENXR_1_RUNTIMES == "1";
|
||||
wivrnUsesNvidiaVulkan = cfg.services.wivrn.monadoEnvironment.__VK_LAYER_NV_optimus == "NVIDIA_only";
|
||||
wivrnSharesDgpuEnvWithGamescope =
|
||||
cfg.services.wivrn.monadoEnvironment == cfg.programs.gamescope.env;
|
||||
wivrnStatePreserved = privatelyPreserved ".config/wivrn";
|
||||
adbStatePreserved = privatelyPreserved ".android";
|
||||
}
|
||||
Reference in new issue
Block a user