diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0980d073..a9e74c65 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -142,12 +142,13 @@ nix build .#checks.x86_64-linux.kwin-effect-geometry-change-komorebi-install ## Plasma guests to develop in [`nix/dev`](nix/dev) has NixOS guests with Plasma for trying a change by hand, -on an Apple silicon Mac: two that QEMU runs and a disk image for UTM. -[`docs/dev/utm.md`](docs/dev/utm.md) says how to build and run them, and what a -builder has to be. They install komorebi with the NixOS module, and carry -nothing of the person who uses them: a key, a password or a configuration of -yours goes into the guest when it runs, never into these files. The -`plasma-guests-eval` check evaluates them, and the VM suite `rdp` runs their +on an Apple silicon Mac (two that QEMU runs and a disk image for UTM) and on a +Linux machine with KVM (the same two QEMU guests, for `x86_64` and for +`aarch64`). [`docs/dev/utm.md`](docs/dev/utm.md) says how to build and run them, +and what a builder has to be for a Mac. They install komorebi with the NixOS +module, and carry nothing of the person who uses them: a key, a password or a +configuration of yours goes into the guest when it runs, never into these files. +The `plasma-guests-eval` check evaluates them, and the VM suite `rdp` runs their RDP server, [`nix/dev/krdp.nix`](nix/dev/krdp.nix). ## Continuous integration diff --git a/README.md b/README.md index 9c9ba4f3..7863231a 100644 --- a/README.md +++ b/README.md @@ -179,10 +179,11 @@ it. `just vm NAME` runs one; [`docs/dev/vm-tests.md`](docs/dev/vm-tests.md) has the details, including how to run them on a Mac. To try komorebi by hand in a real Plasma session, the flake has Plasma guests -to develop in, for an Apple silicon Mac: two that QEMU runs, one with a session -that can be reached over RDP, and a disk image for UTM, which renders on the -Mac's GPU. [`docs/dev/utm.md`](docs/dev/utm.md) says how to build and run them; -`plasma-guests-eval` evaluates them on every system. +to develop in. For an Apple silicon Mac there are two that QEMU runs, one with a +session that can be reached over RDP, and a disk image for UTM, which renders on +the Mac's GPU. For a Linux machine with KVM there are the same two QEMU guests, +for `x86_64` and for `aarch64`. [`docs/dev/utm.md`](docs/dev/utm.md) says how to +build and run them; `plasma-guests-eval` evaluates them on every system. ## How it fits together diff --git a/docs/dev/utm.md b/docs/dev/utm.md index 67d4d108..6c4d0ae2 100644 --- a/docs/dev/utm.md +++ b/docs/dev/utm.md @@ -1,29 +1,40 @@ # Plasma guests to develop in The VM tests boot a machine, run a scenario and power it off. To try komorebi by hand against a -real KWin, the flake has three NixOS guests with Plasma 6 that stay up, all `aarch64-linux`, for -an Apple silicon Mac: +real KWin, the flake has NixOS guests with Plasma 6 that stay up. Each guest is made for one kind +of host, because the accelerator of a host (HVF on a Mac, KVM on Linux) runs guests of the host's +own architecture: -| Guest | Runs under | What it is | -|---|---|---| -| `plasma-vm` | QEMU from nixpkgs, with HVF | Plasma installed, no login: a console and ssh, to run `kwin_wayland --virtual` and build komorebi from the repository | -| `plasma-vm-session` | QEMU from nixpkgs, with HVF | the same, plus a Plasma Wayland session that logs in by itself, komorebi running in it, and an RDP server | -| `plasma-utm` | [UTM](https://mac.getutm.app), with a GPU | the session guest as a disk image that boots by UEFI | +| Guest | Host | Runs under | What it is | +|---|---|---|---| +| `plasma-vm` | Apple silicon Mac | QEMU from nixpkgs, with HVF | Plasma installed, no login: a console and ssh, to run `kwin_wayland --virtual` and build komorebi from the repository | +| `plasma-vm-session` | Apple silicon Mac | QEMU from nixpkgs, with HVF | the same, plus a Plasma Wayland session that logs in by itself, komorebi running in it, and an RDP server | +| `plasma-utm` | Apple silicon Mac | [UTM](https://mac.getutm.app), with a GPU | the session guest as a disk image that boots by UEFI | +| `plasma-vm-x86_64`, `plasma-vm-session-x86_64` | Linux on x86_64 | QEMU from nixpkgs, with KVM | the first two, as `x86_64-linux` guests | +| `plasma-vm-aarch64`, `plasma-vm-session-aarch64` | Linux on aarch64 | QEMU from nixpkgs, with KVM | the first two, as `aarch64-linux` guests | + +The guests of a Mac are `aarch64-linux`. A guest of a Linux host that has no KVM does not start: +the runner refuses, because Plasma under software emulation is too slow to try anything. They are `nixosConfigurations` of the flake, made from [`nix/dev/plasma-vm.nix`](../../nix/dev/plasma-vm.nix), [`nix/dev/plasma-utm.nix`](../../nix/dev/plasma-utm.nix) and -[`nix/dev/krdp.nix`](../../nix/dev/krdp.nix). Each one installs komorebi with the flake's NixOS +[`nix/dev/krdp.nix`](../../nix/dev/krdp.nix), by `mkPlasmaGuest` in +[`flake.nix`](../../flake.nix), which takes the system of the guest and the packages of the machine +that runs QEMU. Each one installs komorebi with the flake's NixOS module, `services.komorebi`, so the komorebi in a guest is the one of the commit the guest was built from, and in the session guests the module's unit starts it at login. They carry no key, no password and no configuration of anyone's. -The QEMU in nixpkgs has no 3D, so in the QEMU guests KWin composites on the CPU (llvmpipe) and -every frame then has to be encoded to reach an RDP client. That is slow. UTM's QEMU has -`virtio-gpu-gl`, which gives the guest OpenGL through Mesa's VirGL, rendered on the Mac's GPU, and -UTM shows the guest in a window of its own, so there is no RDP in the path at all. +In the QEMU guests the guest has a plain `virtio-gpu`, which has no 3D, so KWin composites on the +CPU (llvmpipe) and every frame then has to be encoded to reach an RDP client. That is slow. On a +Mac the QEMU in nixpkgs cannot do better: it has no 3D at all. UTM's QEMU has `virtio-gpu-gl`, +which gives the guest OpenGL through Mesa's VirGL, rendered on the Mac's GPU, and UTM shows the +guest in a window of its own, so there is no RDP in the path at all. The QEMU of an x86_64 Linux +host has and a window (GTK), at time of writing (QEMU 11.1.2 from nixpkgs, 2026-10-28), which these guests +do not use. -## What a builder has to be +## What a builder has to be, on a Mac A Mac cannot build a Linux system, so the guests are built on a builder: a machine with Nix that builds for `aarch64-linux` and that the Mac can reach as a remote store, such as @@ -41,7 +52,7 @@ export KOMOREBI_BUILDER=ssh-ng://builder@localhost # the store URL of your bui export NIX_SSHOPTS="-p 31022 -i $HOME/.ssh/builder_ed25519 -o IdentitiesOnly=yes" # if it needs them ``` -## The QEMU guests +## The QEMU guests on a Mac The system is built on the builder and copied back, and only the QEMU runner, which is a program for the Mac, is built on the Mac. A QEMU guest runs the system of its `vmVariant` (which adds the @@ -73,6 +84,33 @@ In the guest, `/repo` is the repository and `CARGO_TARGET_DIR` is in the guest's target directory on the shared directory is slow. The session guest shows nothing on the Mac; it is reached over RDP ([below](#rdp)). +## The QEMU guests on a Linux machine + +A Linux host builds a guest itself, so there is no builder to set up. It needs Nix with flakes, KVM +(a `/dev/kvm` that the user can open, which on most distributions means the group `kvm`), and room +for the closure (about 10 GB) and for the guest's disk (20 GB, sparse). The runner is built with +its system, and started the same way as on a Mac: + +```sh +guest=plasma-vm-session-x86_64 # or plasma-vm-x86_64, or the -aarch64 pair on an aarch64 host +nix build ".#nixosConfigurations.$guest.config.system.build.vm" -o .vm/result-$guest +cd .vm && KOMOREBI_REPO=$PWD/.. ./result-$guest/bin/run-*-vm +``` + +The console, the user, the ssh ports (2222 and 2223), the RDP port, the disk and `/repo` are as +described for a Mac. What differs is what nixpkgs' runner does on a Linux host: + +- It starts one `virtiofsd` for each shared directory, as the user who runs the guest, and the + guest mounts them by virtiofs where a Mac's guest mounts them by 9p. One of them is the host's + `/nix/store`, so the guest has the closure without a copy of it. +- It refuses to start when `/dev/kvm` is missing or cannot be opened by the user, since the guests + set `virtualisation.qemu.forceAccel`. + +A guest has one display device, a virtio-gpu, so that KWin has a DRM device to draw on. The machine +of an x86_64 guest has a VGA of its own, which would be a second display next to the virtio-gpu, +so the guest has `virtio-vga` (a virtio-gpu that is also a VGA) in its place and `-vga none`. The +session guest shows nothing on the host, as on a Mac: it is reached over RDP ([below](#rdp)). + ## The UTM disk UTM boots `plasma-utm` from a disk image. The image is made by installing the system onto a blank @@ -172,9 +210,13 @@ is known about RDP into the session, and what is not. ## What is not known -At time of writing (2026-10-27), the guests made from these files have been built on an +At time of writing (2026-10-28), the guests of a Mac, made from these files, have been built on an `aarch64-linux` machine, and not booted: no test covers them, and `plasma-guests-eval` only -evaluates them. The RDP server is the exception: the VM suite `rdp` runs the same unit in the test -session. That KWin 6.8 runs well on VirGL under UTM is not known either: the UTM and Mesa notes say -VirGL is the way to render a Linux desktop, and Plasma works on it for other people, but it has not -been measured here. +evaluates them. The `x86_64` pair for a Linux host was built and started under KVM on an `x86_64` +machine. Both guests reached the login on their serial console and had one DRM device with a render +node, and the session guest drew the Plasma desktop on its display (a screenshot taken through +QEMU's monitor) with komorebi running in the session. The `aarch64` pair for a Linux host was +evaluated and not built. The RDP server is the exception to all of this: the VM suite `rdp` runs the +same unit in the test session. That KWin 6.8 runs well on VirGL under UTM is not known either: the +UTM and Mesa notes say VirGL is the way to render a Linux desktop, and Plasma works on it for other +people, but it has not been measured here. diff --git a/flake.nix b/flake.nix index aa556b50..60dd3552 100644 --- a/flake.nix +++ b/flake.nix @@ -285,20 +285,69 @@ # The Plasma guests to develop against, evaluated and never built, so that every system # sees an option that nixpkgs renamed or an assertion that fails. The QEMU guests are - # systems in a VM (`vmVariant`), with a runner for the Mac; UTM's is a system of its own. + # systems in a VM (`vmVariant`), with a runner for the machine that runs QEMU; UTM's is + # a system of its own. Each guest is for one kind of machine, which is also checked: the + # architecture of the guest, the machine that runs its QEMU, and that a guest of a Linux + # machine has KVM and one display device only. plasma-guests-eval = let guests = self.nixosConfigurations; - qemuGuest = name: { - ${name} = guests.${name}.config.virtualisation.vmVariant.system.build.toplevel; - "${name}, its runner" = guests.${name}.config.system.build.vm; - }; - builds = - qemuGuest "plasma-vm" - // qemuGuest "plasma-vm-session" - // { - plasma-utm = guests.plasma-utm.config.system.build.toplevel; + machines = { + plasma-vm = { + guest = "aarch64-linux"; + host = "aarch64-darwin"; }; + plasma-vm-session = { + guest = "aarch64-linux"; + host = "aarch64-darwin"; + }; + plasma-vm-x86_64 = { + guest = "x86_64-linux"; + host = "x86_64-linux"; + }; + plasma-vm-session-x86_64 = { + guest = "x86_64-linux"; + host = "x86_64-linux"; + }; + plasma-vm-aarch64 = { + guest = "aarch64-linux"; + host = "aarch64-linux"; + }; + plasma-vm-session-aarch64 = { + guest = "aarch64-linux"; + host = "aarch64-linux"; + }; + }; + isDisplay = + option: lib.hasPrefix "-device virtio-vga" option || lib.hasPrefix "-device virtio-gpu" option; + qemuGuest = + name: + let + machine = machines.${name}; + vm = guests.${name}.config.virtualisation.vmVariant.virtualisation; + in + assert lib.assertMsg ( + guests.${name}.pkgs.stdenv.hostPlatform.system == machine.guest + ) "${name} is not a ${machine.guest} guest"; + assert lib.assertMsg ( + vm.host.pkgs.stdenv.hostPlatform.system == machine.host + ) "${name} does not run on a ${machine.host} host"; + assert lib.assertMsg (vm.qemu.forceAccel + ) "${name} would run without KVM or HVF, which is too slow to use"; + assert lib.assertMsg ( + builtins.length (builtins.filter isDisplay vm.qemu.options) == 1 + ) "${name} does not have exactly one display device"; + assert lib.assertMsg + # an x86_64 machine has a VGA of its own, which would be a second display + ((machine.guest == "x86_64-linux") == builtins.elem "-vga none" vm.qemu.options) + "${name} has `-vga none` if and only if it is an x86_64 guest"; + { + ${name} = guests.${name}.config.virtualisation.vmVariant.system.build.toplevel; + "${name}, its runner" = guests.${name}.config.system.build.vm; + }; + builds = lib.foldl' (builds: name: builds // qemuGuest name) { } (lib.attrNames machines) // { + plasma-utm = guests.plasma-utm.config.system.build.toplevel; + }; lines = lib.mapAttrsToList ( name: drv: "${name}: ${builtins.unsafeDiscardStringContext drv.drvPath}" ) builds; @@ -456,28 +505,48 @@ }; nixosModules.komorebi = self.nixosModules.default; - # Plasma guests to develop against, for an Apple silicon Mac (docs/dev/utm.md): QEMU runs - # the first two under HVF, and UTM boots the third from a disk image + # Plasma guests to develop against (docs/dev/utm.md). For an Apple silicon Mac, QEMU runs + # the first two under HVF, and UTM boots the third from a disk image. For a Linux machine + # with KVM, QEMU runs the other four: the same two guests for an x86_64 host and for an + # aarch64 one, each of which runs the guest of its own architecture. nixosConfigurations = let + # `guest` is the system of the guest, and `host` the pkgs of the machine that runs QEMU mkPlasmaGuest = + { guest, host }: modules: nixpkgs.lib.nixosSystem { - specialArgs.hostPkgs = nixpkgs.legacyPackages.aarch64-darwin; + specialArgs.hostPkgs = host; modules = [ self.nixosModules.default - { nixpkgs.hostPlatform = "aarch64-linux"; } + { nixpkgs.hostPlatform = guest; } ] ++ modules; }; - in - { - plasma-vm = mkPlasmaGuest [ ./nix/dev/plasma-vm.nix ]; - plasma-vm-session = mkPlasmaGuest [ + onMac = mkPlasmaGuest { + guest = "aarch64-linux"; + host = nixpkgs.legacyPackages.aarch64-darwin; + }; + onLinux = + system: + mkPlasmaGuest { + guest = system; + host = nixpkgs.legacyPackages.${system}; + }; + session = [ ./nix/dev/plasma-vm.nix { komorebi.plasmaVm.session = true; } ]; - plasma-utm = mkPlasmaGuest [ ./nix/dev/plasma-utm.nix ]; + in + { + plasma-vm = onMac [ ./nix/dev/plasma-vm.nix ]; + plasma-vm-session = onMac session; + plasma-utm = onMac [ ./nix/dev/plasma-utm.nix ]; + + plasma-vm-x86_64 = onLinux "x86_64-linux" [ ./nix/dev/plasma-vm.nix ]; + plasma-vm-session-x86_64 = onLinux "x86_64-linux" session; + plasma-vm-aarch64 = onLinux "aarch64-linux" [ ./nix/dev/plasma-vm.nix ]; + plasma-vm-session-aarch64 = onLinux "aarch64-linux" session; }; }; }; diff --git a/nix/dev/plasma-vm.nix b/nix/dev/plasma-vm.nix index 6f9e6c27..d9b8add7 100644 --- a/nix/dev/plasma-vm.nix +++ b/nix/dev/plasma-vm.nix @@ -8,7 +8,9 @@ # A NixOS Plasma 6 guest to develop and test against a real KWin, with komorebi installed by the # flake's module, `services.komorebi`, from the flake's packages. The flake builds it for an Apple -# silicon Mac, which runs it under QEMU with HVF; docs/dev/utm.md says how to build and run it. +# silicon Mac, which runs it under QEMU with HVF, and for a Linux machine, which runs it under QEMU +# with KVM; docs/dev/utm.md says how to build and run it. The system of the guest and the packages +# of the machine that runs QEMU (`hostPkgs`) come from the flake, so that one file serves them all. # # The guest mounts the repository at /repo, and keeps its disk in the directory it is started # from, which is best .vm (gitignored): @@ -24,6 +26,20 @@ # komorebi in it, and it can be reached over RDP (krdp.nix). let cfg = config.komorebi.plasmaVm; + + # The guest's one display, a virtio-gpu, so that KWin has a DRM device to draw on. The machine of + # an aarch64 guest has no display of its own. The machine of an x86_64 guest has a VGA, which QEMU + # adds unless it is told not to, and KWin would draw on it as well as on the virtio-gpu: a second + # head. An x86_64 guest gets `virtio-vga`, a virtio-gpu that is also the VGA, in its place. + gpu = + args: + if pkgs.stdenv.hostPlatform.isx86_64 then + [ + "-vga none" + "-device virtio-vga${args}" + ] + else + [ "-device virtio-gpu-pci${args}" ]; in { imports = [ ./krdp.nix ]; @@ -130,7 +146,7 @@ in diskSize = 20 * 1024; graphics = false; # no display, but a gpu so there is a drm render node - qemu.options = lib.mkIf (!cfg.session) [ "-device virtio-gpu-pci" ]; + qemu.options = lib.mkIf (!cfg.session) (gpu ""); forwardPorts = [ { from = "host"; @@ -166,9 +182,7 @@ in # kwin drives virtio-gpu through drm on the primary node; that is the only place mesa's software # rasterizer can allocate buffers (see docs/dev/PLATFORM.md: the headless --virtual backend # cannot) - virtualisation.vmVariant.virtualisation.qemu.options = [ - "-device virtio-gpu-pci,xres=1920,yres=1080" - ]; + virtualisation.vmVariant.virtualisation.qemu.options = gpu ",xres=1920,yres=1080"; # an idle lock screen or dimmed display changes how kwin animates and what screenshots show environment.etc."xdg/kscreenlockerrc".text = '' [Daemon]