mirror of
https://github.com/ryan4yin/nix-config.git
synced 2026-11-13 12:13:46 +01:00
chore: drop the KubeVirt guest bits
The k3s-test workers never ran (they were stuck waiting for volume
binding) and will be re-added as microVMs, so remove their KubeVirt
configs and everything that only existed for KubeVirt guests:
- hosts/k8s/k3s-test-1-worker-{1,2,3} and their outputs
- lib/genKubeVirtGuestModule.nix and lib/genK3sAgentModule.nix (the
latter was only used by the workers)
- the KubeVirt image upload recipes (upload-vm, upload-idols,
upload-k3s-test) and the kubevirt image package in the idols outputs
- refresh the docs; keep only accurate historical mentions (e.g. the
restic backup path must stay as it is)
This commit is contained in:
1 parent
621da40081
commit
6180072646
33 files changed
+39
-408
No files matched your search
@@ -61,10 +61,10 @@ deployments. Keep repository guidance here; reusable global rules live in `agent
|
||||
- Deployment and upload recipes change systems; use eval/build commands for validation. Remote
|
||||
deployment requires an explicit request. When deployment behavior changes, report the exact `just`
|
||||
command to run.
|
||||
- KubeVirt hosts (`shoryu`/`shushou`/`youko`): avoid mid-flight restarts of the network
|
||||
- VM hosts (`shoryu`/`shushou`/`youko`): avoid mid-flight restarts of the network
|
||||
stack (`systemd-networkd` / the `br0` bridge) — that can drop the VM network. Use `boot` + serial
|
||||
reboot for networking or nixpkgs changes. See
|
||||
[hosts/README.md](./hosts/README.md#deploying-kubevirt-hosts).
|
||||
[hosts/README.md](./hosts/README.md#deploying-vm-hosts).
|
||||
- `just clean`, `just gc`, `just ggc`, and `just game` remove history or amend commits; they are not
|
||||
validation steps and require explicit authorization for their target and scope.
|
||||
- Do not use `just penvof` for process inspection: it can expose secret values.
|
||||
|
||||
@@ -179,7 +179,7 @@ reset-launchpad:
|
||||
|
||||
############################################################################
|
||||
#
|
||||
# Homelab - Kubevirt Cluster related commands
|
||||
# Homelab - VM host related commands
|
||||
#
|
||||
############################################################################
|
||||
|
||||
@@ -189,14 +189,6 @@ reset-launchpad:
|
||||
col tag mode="switch":
|
||||
colmena apply {{mode}} --on '@{{tag}}' --verbose --show-trace
|
||||
|
||||
# Build and upload a vm image
|
||||
[linux]
|
||||
[group('homelab')]
|
||||
upload-vm name size verbosity="normal":
|
||||
#!/usr/bin/env nu
|
||||
use {{utils_nu}} *;
|
||||
upload-vm {{name}} {{size}} {{verbosity}}
|
||||
|
||||
# Deploy all the VM hosts (physical machines running the VMs)
|
||||
[linux]
|
||||
[group('homelab')]
|
||||
@@ -224,15 +216,6 @@ youko mode="switch":
|
||||
#
|
||||
############################################################################
|
||||
|
||||
# Build and upload a vm image
|
||||
[linux]
|
||||
[group('homelab')]
|
||||
upload-idols verbosity="normal":
|
||||
#!/usr/bin/env nu
|
||||
use {{utils_nu}} *;
|
||||
upload-vm ruby 100Gi {{verbosity}}
|
||||
upload-vm kana 100Gi {{verbosity}}
|
||||
|
||||
[linux]
|
||||
[group('homelab')]
|
||||
ruby mode="switch":
|
||||
@@ -249,19 +232,6 @@ kana mode="switch":
|
||||
#
|
||||
############################################################################
|
||||
|
||||
# Build and upload a vm image
|
||||
[linux]
|
||||
[group('homelab')]
|
||||
upload-k3s-test verbosity="normal":
|
||||
#!/usr/bin/env nu
|
||||
use {{utils_nu}} *;
|
||||
upload-vm k3s-test-1-master-1 50Gi {{verbosity}};
|
||||
upload-vm k3s-test-1-master-2 50Gi {{verbosity}};
|
||||
upload-vm k3s-test-1-master-3 50Gi {{verbosity}};
|
||||
upload-vm k3s-test-1-worker-1 50Gi {{verbosity}};
|
||||
upload-vm k3s-test-1-worker-2 50Gi {{verbosity}};
|
||||
upload-vm k3s-test-1-worker-3 50Gi {{verbosity}};
|
||||
|
||||
[linux]
|
||||
[group('homelab')]
|
||||
k3s-test mode="switch":
|
||||
|
||||
@@ -27,11 +27,11 @@ This repository is home to the Nix code that builds all of my systems:
|
||||
agenix.
|
||||
2. **macOS desktops** — nix-darwin + Home Manager, sharing the same `home/` configuration with the
|
||||
NixOS desktops.
|
||||
3. **NixOS servers** — KubeVirt VMs running on three physical mini PCs, hosting K3s clusters,
|
||||
3. **NixOS servers** — VMs running on three physical mini PCs, hosting K3s clusters,
|
||||
monitoring, and other self-hosted services.
|
||||
|
||||
See [./hosts](./hosts) for the host inventory, [./outputs](./outputs) for how the flake outputs are
|
||||
composed, [./Virtual-Machine.md](./Virtual-Machine.md) for creating & managing KubeVirt VMs, and
|
||||
composed, [./Virtual-Machine.md](./Virtual-Machine.md) for creating & managing VMs, and
|
||||
[./AGENTS.md](./AGENTS.md) for the repository conventions.
|
||||
|
||||
## Why NixOS & Flakes?
|
||||
|
||||
+3
-3
@@ -1,7 +1,7 @@
|
||||
## How to create & managage KubeVirt's Virtual Machine from this flake?
|
||||
## How to create & manage VMs (the retired KubeVirt path)
|
||||
|
||||
> [!NOTE] KubeVirt is being retired. aquamarine runs natively on `youko` and the k3s-test
|
||||
> nodes run as [microVMs](./hosts/k8s/README.md); this document only covers the KubeVirt VMs that
|
||||
> nodes run as [microVMs](./hosts/k8s/README.md); this document only covers the VMs that
|
||||
> are left to move (currently the Windows VM).
|
||||
|
||||
Use `aquamarine` as an example, first build and upload the virtual machine's qcow2 image to the file
|
||||
@@ -15,7 +15,7 @@ Then create the virtual machine by creating a yaml file at
|
||||
[ryan4yin/k8s-gitops](https://github.com/ryan4yin/k8s-gitops/tree/main/vms), set the
|
||||
`spec.dataVolumeTemplates[0].source.http.url` to the uploaded file's URL, and fluxcd will
|
||||
automatically apply the changes, then a virtual machine named `aquamarine` will be created in the
|
||||
KubeVirt cluster.
|
||||
VM cluster.
|
||||
|
||||
Once the virtual machine `aquamarine` is created, we can deploy updates to it with the following
|
||||
commands:
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 116 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 981 KiB |
@@ -50,7 +50,7 @@
|
||||
# get some latest packages from the master branch
|
||||
nixpkgs-master.url = "github:nixos/nixpkgs/master";
|
||||
|
||||
# NixOS MicroVMs -- run the VMs directly on the hosts, replacing KubeVirt
|
||||
# NixOS MicroVMs -- run the VMs directly on the hosts, replacing the former KubeVirt
|
||||
microvm = {
|
||||
url = "github:microvm-nix/microvm.nix";
|
||||
inputs.nixpkgs.follows = "nixpkgs";
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
...
|
||||
}:
|
||||
let
|
||||
# Shared golden-image store for the KubeVirt cluster. It is exported over
|
||||
# Shared golden-image store for the VM cluster. It is exported over
|
||||
# NFSv4 so the cluster's NFS CSI driver can hand out RWX volumes to every
|
||||
# node, and VM disks are cloned from it into local-path.
|
||||
#
|
||||
@@ -10,7 +10,7 @@ let
|
||||
# free space on this host.
|
||||
goldenDir = "/persistent/nfs/golden";
|
||||
|
||||
# Only the homelab LAN may mount the export. The KubeVirt nodes are the
|
||||
# Only the homelab LAN may mount the export. The VM hosts are the
|
||||
# clients (the CSI node plugin mounts from the node, not from the pod).
|
||||
clientCidr = "192.168.5.0/24";
|
||||
in
|
||||
|
||||
+8
-8
@@ -13,7 +13,7 @@ Named after characters from "Oshi no Ko":
|
||||
| Host | Platform | Hardware | Purpose | Status |
|
||||
| ------------ | ----------- | --------------------- | --------------------- | ----------- |
|
||||
| `ai` | NixOS | i5-13600KF + RTX 4090 | Gaming & Daily Use | ✅ Active |
|
||||
| `aquamarine` | KubeVirt VM | Virtual | Monitoring & Services | ✅ Active |
|
||||
| `aquamarine` | VM | Virtual | Monitoring & Services | ✅ Active |
|
||||
| `kana` | NixOS | Virtual | Reserved | ⚪ Not Used |
|
||||
| `ruby` | NixOS | Virtual | Reserved | ⚪ Not Used |
|
||||
|
||||
@@ -33,15 +33,15 @@ Named after "Twelve Kingdoms":
|
||||
| Host | Platform | Hardware | Purpose | Status |
|
||||
| --------- | -------- | -------------------------------------- | -------------------------- | --------- |
|
||||
| `shoukei` | NixOS | MacBook Pro M2 | NixOS on Apple Silicon | ✅ Active |
|
||||
| `shoryu` | NixOS | MoreFine S500Plus (AMD Ryzen 9 5900HX) | KubeVirt Host & K3s Master | ✅ Active |
|
||||
| `shushou` | NixOS | MinisForum UM560 (AMD Ryzen 5 5625U) | KubeVirt Host & K3s Master | ✅ Active |
|
||||
| `youko` | NixOS | MinisForum HX99G (AMD Ryzen 9 6900HX) | KubeVirt Host & K3s Master | ✅ Active |
|
||||
| `shoryu` | NixOS | MoreFine S500Plus (AMD Ryzen 9 5900HX) | VM Host | ✅ Active |
|
||||
| `shushou` | NixOS | MinisForum UM560 (AMD Ryzen 5 5625U) | VM Host | ✅ Active |
|
||||
| `youko` | NixOS | MinisForum HX99G (AMD Ryzen 9 6900HX) | VM Host | ✅ Active |
|
||||
|
||||
### Virtual Machines & Clusters
|
||||
|
||||
#### `k8s` - Kubernetes Infrastructure
|
||||
|
||||
- **KubeVirt Cluster**: 3 physical mini PCs (shoryu, shushou, youko) running all VMs
|
||||
- **VM Cluster**: 3 physical mini PCs (shoryu, shushou, youko) running all VMs
|
||||
- **K3s Testing**: 3 masters + 3 workers for testing and development
|
||||
|
||||
### External Systems
|
||||
@@ -104,13 +104,13 @@ Use existing hosts as templates. The key files typically include:
|
||||
### Examples to Reference
|
||||
|
||||
- **Desktop systems**: See `idols-ai/` for gaming/workstation setup
|
||||
- **Server systems**: See `shoryu/` for K8s/KubeVirt hosts
|
||||
- **Server systems**: See `shoryu/` for K8s/VM hosts
|
||||
- **macOS systems**: See `darwin-fern/` for macOS configurations
|
||||
- **Apple Silicon**: See `12kingdoms-shoukei/` for ARM Linux setup
|
||||
|
||||
## Deploying KubeVirt Hosts
|
||||
## Deploying VM Hosts
|
||||
|
||||
The three KubeVirt hosts (`shoryu`, `shushou`, `youko`) put their VMs'
|
||||
The three VM hosts (`shoryu`, `shushou`, `youko`) put their VMs'
|
||||
secondary network on the Linux bridge `br0` (attached by the `bridge` CNI plugin); the pod network
|
||||
is Cilium (flannel disabled).
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
}:
|
||||
#############################################################
|
||||
#
|
||||
# Kana - a NixOS VM running on Proxmox/KubeVirt
|
||||
# Kana - a NixOS VM running under libvirt
|
||||
#
|
||||
#############################################################
|
||||
let
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
}:
|
||||
#############################################################
|
||||
#
|
||||
# Ruby - a NixOS VM running on Proxmox/KubeVirt
|
||||
# Ruby - a NixOS VM running under libvirt
|
||||
#
|
||||
#############################################################
|
||||
let
|
||||
|
||||
@@ -64,7 +64,6 @@ in
|
||||
# cloud-native
|
||||
kubectl
|
||||
istioctl
|
||||
kubevirt # virtctl
|
||||
kubernetes-helm
|
||||
fluxcd
|
||||
terraform
|
||||
|
||||
+4
-6
@@ -7,13 +7,11 @@ I'm running a Kubernetes cluster for testing and development.
|
||||
I prefer to use [k3s] as the Kubernetes distribution, because it's lightweight, easy to install, and
|
||||
full featured(see [what-have-k3s-removed-from-upstream-kubernetes] for details).
|
||||
|
||||
## KubeVirt Cluster
|
||||
## VM Cluster
|
||||
|
||||
The KubeVirt cluster is running on physical machines, all my virtual machines are running on this
|
||||
The VM cluster is running on physical machines, all my virtual machines are running on this
|
||||
cluster, including other Kubernetes clusters.
|
||||
|
||||

|
||||

|
||||
|
||||
## K3s Clusters
|
||||
|
||||
@@ -30,7 +28,7 @@ image has to be built or uploaded; only the volumes above persist. The guest's t
|
||||
bridged onto `br0`, and its name is derived from the guest IP (e.g. `192.168.5.114` -> `vm114`, as
|
||||
`IFNAMSIZ` caps interface names at 15 characters).
|
||||
|
||||
To move an existing KubeVirt VM over in place, copy the host key and `machine-id` from its disk into
|
||||
To move an existing VM over in place, copy the host key and `machine-id` from its disk into
|
||||
`etc.img`, and `/var/lib/rancher/k3s/server` into `var.img`, **before the first boot** — then it
|
||||
keeps its identity and etcd membership instead of bootstraping a new cluster.
|
||||
|
||||
@@ -41,7 +39,7 @@ keeps its identity and etcd membership instead of bootstraping a new cluster.
|
||||
|
||||
- **Retire KubeVirt.** aquamarine now runs natively on `youko`, and the k3s-test masters
|
||||
run as microVMs. Remove KubeVirt/CDI/multus from the cluster (k8s-gitops) once nothing else needs
|
||||
them, and the Windows VM is the only KubeVirt VM left to move.
|
||||
them, and the Windows VM is the only VM left to move.
|
||||
- **The USB HDD bridge is flaky.** aquamarine's two HDDs sit behind a JMicron JMS567 USB-SATA bridge
|
||||
that keeps resetting (`dmesg` on `youko`). Find out how often it resets and how much it
|
||||
matters before putting anything critical (e.g. an NFS export) on it.
|
||||
|
||||
@@ -1,30 +0,0 @@
|
||||
{
|
||||
config,
|
||||
pkgs,
|
||||
myvars,
|
||||
mylib,
|
||||
...
|
||||
}:
|
||||
let
|
||||
hostName = "k3s-test-1-worker-1"; # define your hostname.
|
||||
|
||||
coreModule = mylib.genKubeVirtGuestModule {
|
||||
inherit pkgs hostName;
|
||||
inherit (myvars) networking;
|
||||
};
|
||||
k3sModule = mylib.genK3sAgentModule {
|
||||
inherit pkgs;
|
||||
tokenFile = config.age.secrets."k3s-test-1-token".path;
|
||||
# use my own domain & kube-vip's virtual IP for the API server
|
||||
# so that the API server can always be accessed even if some nodes are down
|
||||
masterHost = "test-cluster-1.writefor.fun";
|
||||
# workloads run on the workers; the masters are tainted (see k3s-test-1-master-*)
|
||||
nodeLabels = [ "node-role.kubernetes.io/worker=true" ];
|
||||
};
|
||||
in
|
||||
{
|
||||
imports = (mylib.scanPaths ./.) ++ [
|
||||
coreModule
|
||||
k3sModule
|
||||
];
|
||||
}
|
||||
@@ -1,30 +0,0 @@
|
||||
{
|
||||
config,
|
||||
pkgs,
|
||||
myvars,
|
||||
mylib,
|
||||
...
|
||||
}:
|
||||
let
|
||||
hostName = "k3s-test-1-worker-2"; # define your hostname.
|
||||
|
||||
coreModule = mylib.genKubeVirtGuestModule {
|
||||
inherit pkgs hostName;
|
||||
inherit (myvars) networking;
|
||||
};
|
||||
k3sModule = mylib.genK3sAgentModule {
|
||||
inherit pkgs;
|
||||
tokenFile = config.age.secrets."k3s-test-1-token".path;
|
||||
# use my own domain & kube-vip's virtual IP for the API server
|
||||
# so that the API server can always be accessed even if some nodes are down
|
||||
masterHost = "test-cluster-1.writefor.fun";
|
||||
# workloads run on the workers; the masters are tainted (see k3s-test-1-master-*)
|
||||
nodeLabels = [ "node-role.kubernetes.io/worker=true" ];
|
||||
};
|
||||
in
|
||||
{
|
||||
imports = (mylib.scanPaths ./.) ++ [
|
||||
coreModule
|
||||
k3sModule
|
||||
];
|
||||
}
|
||||
@@ -1,30 +0,0 @@
|
||||
{
|
||||
config,
|
||||
pkgs,
|
||||
myvars,
|
||||
mylib,
|
||||
...
|
||||
}:
|
||||
let
|
||||
hostName = "k3s-test-1-worker-3"; # define your hostname.
|
||||
|
||||
coreModule = mylib.genKubeVirtGuestModule {
|
||||
inherit pkgs hostName;
|
||||
inherit (myvars) networking;
|
||||
};
|
||||
k3sModule = mylib.genK3sAgentModule {
|
||||
inherit pkgs;
|
||||
tokenFile = config.age.secrets."k3s-test-1-token".path;
|
||||
# use my own domain & kube-vip's virtual IP for the API server
|
||||
# so that the API server can always be accessed even if some nodes are down
|
||||
masterHost = "test-cluster-1.writefor.fun";
|
||||
# workloads run on the workers; the masters are tainted (see k3s-test-1-master-*)
|
||||
nodeLabels = [ "node-role.kubernetes.io/worker=true" ];
|
||||
};
|
||||
in
|
||||
{
|
||||
imports = (mylib.scanPaths ./.) ++ [
|
||||
coreModule
|
||||
k3sModule
|
||||
];
|
||||
}
|
||||
@@ -18,8 +18,6 @@ easier to add new machines.
|
||||
|
||||
5. **`genK3sAgentModule.nix`** - K3s agent node configuration generator
|
||||
6. **`genK3sServerModule.nix`** - K3s server node configuration generator
|
||||
7. **`genKubeVirtGuestModule.nix`** - KubeVirt guest VM configuration generator
|
||||
8. **`genKubeVirtHostModule.nix`** - KubeVirt host configuration generator
|
||||
|
||||
### Entry Point
|
||||
|
||||
|
||||
@@ -7,9 +7,7 @@
|
||||
attrs = import ./attrs.nix { inherit lib; };
|
||||
|
||||
genK3sServerModule = import ./genK3sServerModule.nix;
|
||||
genK3sAgentModule = import ./genK3sAgentModule.nix;
|
||||
genVmHostModule = import ./genVmHostModule.nix;
|
||||
genKubeVirtGuestModule = import ./genKubeVirtGuestModule.nix;
|
||||
genMicrovmGuestModule = import ./genMicrovmGuestModule.nix;
|
||||
|
||||
# use path relative to the root of the project
|
||||
|
||||
@@ -1,45 +0,0 @@
|
||||
{
|
||||
pkgs,
|
||||
masterHost,
|
||||
tokenFile,
|
||||
nodeLabels ? [ ],
|
||||
k3sExtraArgs ? [ ],
|
||||
...
|
||||
}:
|
||||
let
|
||||
package = pkgs.k3s;
|
||||
in
|
||||
{
|
||||
environment.systemPackages = [ package ];
|
||||
|
||||
# Kernel modules required by cilium
|
||||
boot.kernelModules = [
|
||||
"ip6_tables"
|
||||
"ip6table_mangle"
|
||||
"ip6table_raw"
|
||||
"ip6table_filter"
|
||||
];
|
||||
networking.enableIPv6 = true;
|
||||
networking.nat = {
|
||||
enable = true;
|
||||
enableIPv6 = true;
|
||||
};
|
||||
|
||||
services.k3s = {
|
||||
enable = true;
|
||||
inherit package tokenFile;
|
||||
|
||||
role = "agent";
|
||||
serverAddr = "https://${masterHost}:6443";
|
||||
# https://docs.k3s.io/cli/agent
|
||||
extraFlags =
|
||||
let
|
||||
flagList = [
|
||||
"--data-dir /var/lib/rancher/k3s"
|
||||
]
|
||||
++ (map (label: "--node-label=${label}") nodeLabels)
|
||||
++ k3sExtraArgs;
|
||||
in
|
||||
pkgs.lib.concatStringsSep " " flagList;
|
||||
};
|
||||
}
|
||||
@@ -62,7 +62,7 @@ in
|
||||
"--write-kubeconfig=${kubeconfigFile}"
|
||||
"--write-kubeconfig-mode=600"
|
||||
"--service-node-port-range=80-32767"
|
||||
"--kube-apiserver-arg='--allow-privileged=true'" # required by kubevirt
|
||||
"--kube-apiserver-arg='--allow-privileged=true'" # required by the former KubeVirt VMs
|
||||
"--data-dir /var/lib/rancher/k3s"
|
||||
"--etcd-expose-metrics=true"
|
||||
"--etcd-snapshot-schedule-cron='0 */12 * * *'"
|
||||
|
||||
@@ -1,71 +0,0 @@
|
||||
{
|
||||
pkgs,
|
||||
hostName,
|
||||
networking,
|
||||
...
|
||||
}:
|
||||
let
|
||||
inherit (networking) proxyGateway proxyGateway6 nameservers;
|
||||
inherit (networking.hostsAddr.${hostName}) iface ipv4;
|
||||
ipv4WithMask = "${ipv4}/24";
|
||||
in
|
||||
{
|
||||
# supported file systems, so we can mount any removable disks with these filesystems
|
||||
boot.supportedFilesystems = [
|
||||
"ext4"
|
||||
"btrfs"
|
||||
"xfs"
|
||||
"fat"
|
||||
"vfat"
|
||||
"exfat"
|
||||
];
|
||||
|
||||
networking = {
|
||||
inherit hostName;
|
||||
|
||||
# we use networkd instead
|
||||
networkmanager.enable = false;
|
||||
useDHCP = false;
|
||||
};
|
||||
networking.useNetworkd = true;
|
||||
systemd.network.enable = true;
|
||||
|
||||
systemd.network.networks."10-${iface}" = {
|
||||
matchConfig.Name = [ iface ];
|
||||
networkConfig = {
|
||||
Address = [ ipv4WithMask ];
|
||||
# DNS = nameservers;
|
||||
DNS = [ proxyGateway ];
|
||||
DHCP = "ipv6"; # enable DHCPv6 only, so we can get a GUA.
|
||||
IPv6AcceptRA = true; # for Stateless IPv6 Autoconfiguraton (SLAAC)
|
||||
LinkLocalAddressing = "ipv6";
|
||||
};
|
||||
routes = [
|
||||
{
|
||||
Destination = "0.0.0.0/0";
|
||||
Gateway = proxyGateway;
|
||||
}
|
||||
{
|
||||
Destination = "::/0";
|
||||
Gateway = proxyGateway6;
|
||||
GatewayOnLink = true; # it's a gateway on local link.
|
||||
}
|
||||
];
|
||||
linkConfig.RequiredForOnline = "routable";
|
||||
};
|
||||
|
||||
# systemd-journal - reduce disk usage
|
||||
# https://www.freedesktop.org/software/systemd/man/latest/journald.conf.html
|
||||
services.journald.settings.Journal = {
|
||||
SystemMaxUse = "2G";
|
||||
RuntimeMaxUse = "256M";
|
||||
};
|
||||
|
||||
# This value determines the NixOS release from which the default
|
||||
# settings for stateful data, like file locations and database versions
|
||||
# on your system were taken. It‘s perfectly fine and recommended to leave
|
||||
# this value at the release version of the first install of this system.
|
||||
# Before changing this value read the documentation for this option
|
||||
# (e.g. man configuration.nix or on https://nixos.org/nixos/options.html).
|
||||
system.stateVersion = "24.11"; # Did you read the comment?
|
||||
}
|
||||
@@ -2,7 +2,7 @@
|
||||
# (BIOS/q35, virtio disk, bridged NIC, VNC display).
|
||||
#
|
||||
# The NIC is pinned to q35 PCI bus 0x02 and the disk to bus 0x04 so the guest
|
||||
# sees the same interface name it did under KubeVirt (enp2s0) and the same
|
||||
# sees the same interface name it did under the former KubeVirt (enp2s0) and the same
|
||||
# device ordering.
|
||||
{ lib }:
|
||||
{
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
pkgs,
|
||||
hostName,
|
||||
networking,
|
||||
# vCPU count and RAM (MiB) -- mirrors the KubeVirt instancetype
|
||||
# vCPU count and RAM (MiB) -- mirrors the former KubeVirt instancetype
|
||||
vcpu,
|
||||
mem,
|
||||
# persistent volume sizes in MiB
|
||||
|
||||
+1
-1
@@ -57,7 +57,7 @@ modules/
|
||||
│ ├── virtualisation.nix # Virtualization support
|
||||
│ └── xdg.nix # XDG base directory settings
|
||||
└── server/ # Server-specific modules
|
||||
├── kubevirt-hardware-configuration.nix
|
||||
├── qemu-guest-hardware-configuration.nix
|
||||
├── server-aarch64.nix
|
||||
├── server-riscv64.nix
|
||||
└── server.nix
|
||||
|
||||
@@ -5,12 +5,12 @@
|
||||
}:
|
||||
##############################################################################
|
||||
#
|
||||
# Template for KubeVirt's VM, mainly based on:
|
||||
# Template for the VM guests (formerly KubeVirt), mainly based on:
|
||||
# https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/kubevirt.nix
|
||||
#
|
||||
# We write our hardware-configuration.nix, so that we can do some customization more easily.
|
||||
#
|
||||
# the url above is used by `nixos-generator` to generate the KubeVirt's qcow2 image file.
|
||||
# the url above is used by `nixos-generator` to generate the guest's qcow2 image file.
|
||||
#
|
||||
##############################################################################
|
||||
{
|
||||
|
||||
@@ -44,5 +44,4 @@ in
|
||||
|
||||
colmena.${name} = mylib.colmenaSystem (systemArgs // { inherit tags ssh-user; });
|
||||
|
||||
packages.${name} = inputs.self.nixosConfigurations.${name}.config.system.build.images.kubevirt;
|
||||
}
|
||||
@@ -44,8 +44,6 @@ in
|
||||
|
||||
colmena.${name} = mylib.colmenaSystem (systemArgs // { inherit tags ssh-user; });
|
||||
|
||||
packages.${name} = inputs.self.nixosConfigurations.${name}.config.system.build.images.kubevirt;
|
||||
|
||||
# nixos tests
|
||||
packages."${name}-nixos-tests" = import ../nixos-tests/idols-ruby.nix systemArgs;
|
||||
}
|
||||
@@ -1,41 +0,0 @@
|
||||
{
|
||||
# NOTE: the args not used in this file CAN NOT be removed!
|
||||
# because haumea pass argument lazily,
|
||||
# and these arguments are used in the functions like `mylib.nixosSystem`, `mylib.colmenaSystem`, etc.
|
||||
inputs,
|
||||
lib,
|
||||
mylib,
|
||||
myvars,
|
||||
system,
|
||||
genSpecialArgs,
|
||||
...
|
||||
}@args:
|
||||
let
|
||||
name = "k3s-test-1-worker-1";
|
||||
tags = [ name ];
|
||||
ssh-user = "root";
|
||||
|
||||
modules = {
|
||||
nixos-modules =
|
||||
(map mylib.relativeToRoot [
|
||||
# common
|
||||
"secrets/nixos.nix"
|
||||
"modules/nixos/server/server.nix"
|
||||
"modules/nixos/server/qemu-guest-hardware-configuration.nix"
|
||||
# host specific
|
||||
"hosts/k8s/${name}"
|
||||
])
|
||||
++ [
|
||||
{ modules.secrets.server.kubernetes.enable = true; }
|
||||
];
|
||||
};
|
||||
|
||||
systemArgs = modules // args;
|
||||
in
|
||||
{
|
||||
nixosConfigurations.${name} = mylib.nixosSystem systemArgs;
|
||||
|
||||
colmena.${name} = mylib.colmenaSystem (systemArgs // { inherit tags ssh-user; });
|
||||
|
||||
packages.${name} = inputs.self.nixosConfigurations.${name}.config.system.build.images.kubevirt;
|
||||
}
|
||||
@@ -1,41 +0,0 @@
|
||||
{
|
||||
# NOTE: the args not used in this file CAN NOT be removed!
|
||||
# because haumea pass argument lazily,
|
||||
# and these arguments are used in the functions like `mylib.nixosSystem`, `mylib.colmenaSystem`, etc.
|
||||
inputs,
|
||||
lib,
|
||||
mylib,
|
||||
myvars,
|
||||
system,
|
||||
genSpecialArgs,
|
||||
...
|
||||
}@args:
|
||||
let
|
||||
name = "k3s-test-1-worker-2";
|
||||
tags = [ name ];
|
||||
ssh-user = "root";
|
||||
|
||||
modules = {
|
||||
nixos-modules =
|
||||
(map mylib.relativeToRoot [
|
||||
# common
|
||||
"secrets/nixos.nix"
|
||||
"modules/nixos/server/server.nix"
|
||||
"modules/nixos/server/qemu-guest-hardware-configuration.nix"
|
||||
# host specific
|
||||
"hosts/k8s/${name}"
|
||||
])
|
||||
++ [
|
||||
{ modules.secrets.server.kubernetes.enable = true; }
|
||||
];
|
||||
};
|
||||
|
||||
systemArgs = modules // args;
|
||||
in
|
||||
{
|
||||
nixosConfigurations.${name} = mylib.nixosSystem systemArgs;
|
||||
|
||||
colmena.${name} = mylib.colmenaSystem (systemArgs // { inherit tags ssh-user; });
|
||||
|
||||
packages.${name} = inputs.self.nixosConfigurations.${name}.config.system.build.images.kubevirt;
|
||||
}
|
||||
@@ -1,41 +0,0 @@
|
||||
{
|
||||
# NOTE: the args not used in this file CAN NOT be removed!
|
||||
# because haumea pass argument lazily,
|
||||
# and these arguments are used in the functions like `mylib.nixosSystem`, `mylib.colmenaSystem`, etc.
|
||||
inputs,
|
||||
lib,
|
||||
mylib,
|
||||
myvars,
|
||||
system,
|
||||
genSpecialArgs,
|
||||
...
|
||||
}@args:
|
||||
let
|
||||
name = "k3s-test-1-worker-3";
|
||||
tags = [ name ];
|
||||
ssh-user = "root";
|
||||
|
||||
modules = {
|
||||
nixos-modules =
|
||||
(map mylib.relativeToRoot [
|
||||
# common
|
||||
"secrets/nixos.nix"
|
||||
"modules/nixos/server/server.nix"
|
||||
"modules/nixos/server/qemu-guest-hardware-configuration.nix"
|
||||
# host specific
|
||||
"hosts/k8s/${name}"
|
||||
])
|
||||
++ [
|
||||
{ modules.secrets.server.kubernetes.enable = true; }
|
||||
];
|
||||
};
|
||||
|
||||
systemArgs = modules // args;
|
||||
in
|
||||
{
|
||||
nixosConfigurations.${name} = mylib.nixosSystem systemArgs;
|
||||
|
||||
colmena.${name} = mylib.colmenaSystem (systemArgs // { inherit tags ssh-user; });
|
||||
|
||||
packages.${name} = inputs.self.nixosConfigurations.${name}.config.system.build.images.kubevirt;
|
||||
}
|
||||
@@ -96,7 +96,7 @@ export def upload-vm [
|
||||
# node the VM is pinned to.
|
||||
#
|
||||
# Requires the CDI upload proxy to be reachable from here; it is exposed
|
||||
# as a NodePort by the cluster (see infra/configs/base/kubevirt).
|
||||
# as a NodePort by the cluster.
|
||||
let uploadProxyUrl = "https://192.168.5.181:30443"
|
||||
let image = (glob result/nixos-image-*.qcow2 | first)
|
||||
virtctl image-upload dv $"($name)-disk" -n vms --size $size --image-path $image --uploadproxy-url $uploadProxyUrl --insecure
|
||||
|
||||
+1
-1
@@ -37,7 +37,7 @@ Comprehensive network configuration including:
|
||||
The networking configuration covers:
|
||||
|
||||
- **Physical machines**: Desktop PCs, Apple Silicon systems, SBCs
|
||||
- **Virtual machines**: KubeVirt guests, K3s nodes
|
||||
- **Virtual machines**: VM guests, K3s nodes
|
||||
- **Kubernetes clusters**: Production and testing environments
|
||||
- **Network infrastructure**: Routers, gateways, and DNS configuration
|
||||
|
||||
|
||||
+6
-6
@@ -105,32 +105,32 @@ rec {
|
||||
# Kubernetes Clusters
|
||||
# ============================================
|
||||
k3s-test-1-master-1 = {
|
||||
# KubeVirt VM
|
||||
# VM
|
||||
iface = "enp2s0";
|
||||
ipv4 = "192.168.5.114";
|
||||
};
|
||||
k3s-test-1-master-2 = {
|
||||
# KubeVirt VM
|
||||
# VM
|
||||
iface = "enp2s0";
|
||||
ipv4 = "192.168.5.115";
|
||||
};
|
||||
k3s-test-1-master-3 = {
|
||||
# KubeVirt VM
|
||||
# VM
|
||||
iface = "enp2s0";
|
||||
ipv4 = "192.168.5.116";
|
||||
};
|
||||
k3s-test-1-worker-1 = {
|
||||
# KubeVirt VM
|
||||
# VM
|
||||
iface = "enp2s0";
|
||||
ipv4 = "192.168.5.111";
|
||||
};
|
||||
k3s-test-1-worker-2 = {
|
||||
# KubeVirt VM
|
||||
# VM
|
||||
iface = "enp2s0";
|
||||
ipv4 = "192.168.5.112";
|
||||
};
|
||||
k3s-test-1-worker-3 = {
|
||||
# KubeVirt VM
|
||||
# VM
|
||||
iface = "enp2s0";
|
||||
ipv4 = "192.168.5.113";
|
||||
};
|
||||
|
||||
Reference in new issue
Block a user