mirror of
https://github.com/ryan4yin/nix-config.git
synced 2026-11-19 14:18:17 +01:00
docs(hosts): add mixed-memory notes for idols-ai (#391)
Document the 4-DIMM kit mix (2×48G JUHOR + 2×16G GLOWY) on the Colorful CVN Z890 ARK FROZEN V20: - hardware (CPU, board, and both kits) - the install order that trains (16G pair first) - the only stable settings (XMP off, DDR5-4800 @ 1.25 V) - the stress-test recipe and results - the XMP @ 5600 failure and how to recover a board that will not POST - re-enrolling the LUKS TPM unlock after a BIOS change
This commit is contained in:
1 parent
87e87aad7d
commit
e653ec3306
2 files changed
+128
No files matched your search
@@ -0,0 +1,127 @@
|
||||
# Mixed memory on idols-ai
|
||||
|
||||
Notes from adding a 2×16G kit to the existing 2×48G kit on this desktop.
|
||||
|
||||
## Hardware
|
||||
|
||||
### CPU
|
||||
|
||||
- Intel Core Ultra 7 270K Plus (Arrow Lake-S Refresh, socket LGA1851)
|
||||
- 24 cores / 24 threads: 8 P-cores up to 5.5 GHz + 16 E-cores up to 4.7 GHz
|
||||
- 36 MB L3, 40 MB L2, 125 W PL1 / 250 W PL2, 4-core Xe iGPU
|
||||
|
||||
### Motherboard
|
||||
|
||||
- Colorful CVN Z890 ARK FROZEN V20 (Intel Z890, LGA1851, ATX), BIOS version 1013
|
||||
- 4× DDR5 DIMM slots, officially up to 192 GB (48 GB max per slot), DDR5-8400(OC) and slower
|
||||
|
||||
### Memory
|
||||
|
||||
| Kit | Capacity | XMP rating | Timings | Voltage | Rank | Part number |
|
||||
| ----- | -------- | --------------------- | ------------- | ------- | ---- | --------------------- |
|
||||
| JUHOR | 2×48G | DDR5-5600 (PC5-44800) | CL46-45-45-90 | 1.25 V | 2R | `JHE5600U4648JG` |
|
||||
| GLOWY | 2×16G | DDR5-6400 (PC5-51200) | CL32-38-38-90 | 1.35 V | 1R | `VGM5UH64C32AG-DTACW` |
|
||||
|
||||
- Channel A/B, DIMM0 = 48G (2R), DIMM1 = 16G (1R) → 64G per channel, 128G total.
|
||||
- With XMP off both kits run at JEDEC DDR5-4800; the board exposes one XMP profile per kit
|
||||
(`XMP 模块一` / `XMP 模块二`).
|
||||
|
||||
## Bring-up order (install the 16G pair first)
|
||||
|
||||
The install order matters — starting from all four DIMMs often fails to train:
|
||||
|
||||
1. Insert **only the 2×16G**, in the board's primary slots (the first-populate pair the manual
|
||||
marks; the working layout has them in `ChannelA-DIMM1` / `ChannelB-DIMM1`).
|
||||
2. Boot to BIOS; confirm `内存模块资源 = 默认模块` (XMP off) — the DRAM voltage comes back at 1.25
|
||||
V.
|
||||
3. Power off, then insert the **2×48G** in the remaining slots.
|
||||
4. Boot; the board retrains once (1–3 min, possibly a few reboots), then works.
|
||||
|
||||
Resulting layout: `ChannelA/B-DIMM0` = 48G (2R), `ChannelA/B-DIMM1` = 16G (1R).
|
||||
|
||||
## Working configuration
|
||||
|
||||
- Verified: `stressapptest` 8G / 32G / 100G all PASS, 0 hardware incidents.
|
||||
|
||||
Current BIOS settings (`超频OC → 内存设置`):
|
||||
|
||||
| Field | Value |
|
||||
| --------------------------------------- | --------------------- |
|
||||
| `内存模块资源` (module source) | `默认模块` (XMP off) |
|
||||
| `Gear选择` (gear) | `Gear2` |
|
||||
| `内存频率(MHz)` | `4800` |
|
||||
| `CPU VDD2电压` | `1.104 V` (Auto) |
|
||||
| `VCCSA电压` | `1.288 V` (Auto) |
|
||||
| `Vdd2Mv` | `1.250 V` (Auto) |
|
||||
| `Memory Voltage VDD` | **`1.250 V`** (fixed) |
|
||||
| `内存电压VDDQ` | **`1.250 V`** (fixed) |
|
||||
| `内存电压VPP` | `1.800 V` (Auto) |
|
||||
| `Primary Timing` `tCL` / `tRCD` / `tRP` | `40` / `40` |
|
||||
| `Primary Timing` `tRAS` | `77` |
|
||||
|
||||
The only hand-set voltages are the two DRAM rails, `Memory Voltage VDD` and `内存电压VDDQ`, both
|
||||
**1.250 V** (the fields read `1.250`, not `Auto`) — this is what `dmidecode` reports as
|
||||
`Configured Voltage: 1.25 V`.
|
||||
|
||||
Confirm a BIOS change actually applied:
|
||||
|
||||
```bash
|
||||
sudo dmidecode -t memory | grep -E 'Locator|Size|Rank|Speed|Configured Voltage'
|
||||
```
|
||||
|
||||
## Stress testing
|
||||
|
||||
`script` block-buffers its log (use `-f`), and `/home/ryan` is tmpfs, so log to `~/tmp`:
|
||||
|
||||
```bash
|
||||
SP=$(nix build --no-link --print-out-paths nixpkgs#stressapptest)
|
||||
stdbuf -oL -eL "$SP/bin/stressapptest" -s 300 -M 32000 -v 4 2>&1 | tee ~/tmp/memtest-32G.log
|
||||
```
|
||||
|
||||
Pass = exit 0 + `Found 0 hardware incidents` + `Status: PASS`. Escalate 8G → 32G → 100G → overnight.
|
||||
|
||||
## What does not work
|
||||
|
||||
- **XMP @ 5600 with 4 DIMMs**: the board never reaches BIOS. It can be a failed training, or
|
||||
training passing (the debug LED settles on **white** or **green**) with the hang on the
|
||||
display/iGPU side. Two stuck modes were seen: (1) the logo screen (`Del`/`F11`) with a white LED,
|
||||
then a hang; or (2) a black screen on the green LED. In every case there is no BIOS entry and the
|
||||
recovery is the same.
|
||||
- Intel/JEDEC validate two DIMMs per channel well below 4800 (2DPC dual-rank ≈ 4400 or lower), so
|
||||
4800 is already above spec. 5600 needs two DIMMs (1DPC).
|
||||
|
||||
## Recover a non-POSTing board
|
||||
|
||||
All of the above — a failed training, the logo screen with a white LED, or a black screen on the
|
||||
green LED — can leave no BIOS entry. The recovery is the same in every case: redo the **bring-up
|
||||
order** above (pull the 48G pair, boot on the 16G pair with XMP off, then re-add the 48G pair). If
|
||||
BIOS is still unreachable, clear CMOS and repeat.
|
||||
|
||||
## Firmware-change checklist
|
||||
|
||||
LUKS TPM auto-unlock (PCR 0+7) breaks on every BIOS change. Re-enroll both disks **by partlabel**,
|
||||
after all BIOS changes are final:
|
||||
|
||||
```bash
|
||||
for d in /dev/disk/by-partlabel/disk-nixos-ai-root /dev/disk/by-partlabel/disk-data-datapart; do
|
||||
sudo systemd-cryptenroll --wipe-slot=tpm2 "$d"
|
||||
sudo systemd-cryptenroll --tpm2-device=auto --tpm2-pcrs=0+7 "$d"
|
||||
done
|
||||
```
|
||||
|
||||
Keep the passphrase slot (never `--wipe-slot=all`). Secure Boot check: `sudo sbctl status`.
|
||||
|
||||
## Notes
|
||||
|
||||
- A hang that forces a reboot can corrupt the Nix store; it is a risk whenever memory is unstable or
|
||||
not thoroughly tested, which is why a full-RAM stress test matters. After any forced reboot run
|
||||
`nix-store --verify --check-contents --repair` and `btrfs scrub`.
|
||||
- The board silently reverts to defaults on failed training, so re-check with `dmidecode`.
|
||||
- "Apps run fine" proves nothing; only a full-RAM stress test does.
|
||||
- Decision: keep the current 128G 4-DIMM config at 4800. The board supports at most 48 GB per slot,
|
||||
so larger modules are not an option.
|
||||
|
||||
## Open items
|
||||
|
||||
- [ ] Overnight stress test → `~/tmp/memtest-overnight.log`
|
||||
- [ ] `nix-store --verify --check-contents --repair` + `btrfs scrub`
|
||||
@@ -10,6 +10,7 @@ Related:
|
||||
`nix-config/nixos-installer`:
|
||||
`nix run github:nix-community/disko -- --mode destroy,format,mount ../hosts/idols-ai/disko-fs.nix`
|
||||
- [disko-fs-data.nix](./disko-fs-data.nix) – data disk layout (LUKS + btrfs at /data)
|
||||
- [MIXED-MEMORY.md](./MIXED-MEMORY.md) – mixed 4-DIMM RAM bring-up, stress testing, and recovery
|
||||
|
||||
## Info
|
||||
|
||||
|
||||
Reference in new issue
Block a user