starred/nix-config-ryan4yin
1
0 Fork 0
mirror of https://github.com/ryan4yin/nix-config.git synced 2026-11-13 12:13:36 +01:00
Code Issues Packages Projects Releases 10 Wiki Activity

docs(workarounds): drop WA-043 now that the bootstrap fix is live

Browse Source 
k8s-gitops#54 merged and reconciled, a full cold start of all six guests
brought the VIP back without manual action, and a probe from a pod shows
2379 denied while 2381 stays open. SECURITY.md records the fix and the
probe; the workaround row and its TODO entry are gone.

Assisted-by: DeepSeek Harness (gpt-6-astra)
This commit is contained in:
Ryan Yin committed 2026-10-24 14:26:53 +08:00
1 parent 8affae69d6
commit c562f47049
2 files changed
+11 -12
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

SECURITY.md
+7 -7
View File
@@ -276,15 +276,16 @@ export, credential, DNS record or firmware configuration was changed during the
cluster-wide and cert-manager can delete them. Narrowing proposals target k8s-gitops. cluster-wide and cert-manager can delete them. Narrowing proposals target k8s-gitops.
- k3s-test-1 pod networking has been down since the 2026-10-14 full guest cold-restart: Cilium - k3s-test-1 pod networking has been down since the 2026-10-14 full guest cold-restart: Cilium
bootstraps through the VIP while the VIP requires kube-vip's lease over ClusterIP — a cycle the bootstraps through the VIP while the VIP requires kube-vip's lease over ClusterIP — a cycle the
kube-proxy-replacement switch created (WA-043). The host nftables drops stay loaded, but the kube-proxy-replacement switch created. The host nftables drops stay loaded, but the CLiCNP layer
CLiCNP layer enforces nothing while Cilium is down and the nft rule's pod-source-IP precondition enforces nothing while Cilium is down and the nft rule's pod-source-IP precondition is not met, so
is not met, so the pod-to-node boundary is degraded until the cluster recovers. the pod-to-node boundary is degraded until the cluster recovers.
- The cluster recovered manually (VIP attached on one master, then handed back to kube-vip), and the - The cluster recovered manually (VIP attached on one master, then handed back to kube-vip), and the
cycle itself is fixed in k8s-gitops: kube-vip's pod now overrides the kubelet-injected cycle itself is fixed: kube-vip's pod now overrides the kubelet-injected
`KUBERNETES_SERVICE_HOST/PORT` to reach its own node's apiserver (port 6443) instead of the `KUBERNETES_SERVICE_HOST/PORT` to reach its own node's apiserver (port 6443) instead of the
ClusterIP, so the VIP no longer needs Cilium on a cold start. A full cold start of all six guests ClusterIP, so the VIP no longer needs Cilium on a cold start. A full cold start of all six guests
on 2026-10-24 brought the VIP back on master-1 with no manual action (6/6 nodes, Cilium 6/6, on 2026-10-24 brought the VIP back with no manual action (6/6 nodes, Cilium 6/6, kube-vip 3/3, no
kube-vip 3/3, no connection to the ClusterIP); WA-043 is dropped once that change merges. connection to the ClusterIP), and k8s-gitops#54 is merged and reconciled. A probe from a pod
confirms 2379 is denied while 2381 stays open for in-cluster metric scraping.
### Sandbox and agent boundaries ### Sandbox and agent boundaries
@@ -354,7 +355,6 @@ separate verification.
| Priority | Work | Acceptance / constraints | | Priority | Work | Acceptance / constraints |
| -------- | -------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | -------- | -------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| P1 | Re-verify the pod-to-node drops and the CLiCNP on k3s-test-1 | The bootstrap fix passed a full cold start on 2026-10-24; probe the host nftables rules and the CLiCNP from a pod now that networking is back, and confirm the 2379 deny once #54 merges |
| P1 | Roll out and verify the implemented baseline/exporter rules | Runtime sysctl readback, allowed IPv4 scrape and denied IPv6/non-monitoring scrape tests; preserve Nix/Podman/Cilium/NFS functionality | | P1 | Roll out and verify the implemented baseline/exporter rules | Runtime sysctl readback, allowed IPv4 scrape and denied IPv6/non-monitoring scrape tests; preserve Nix/Podman/Cilium/NFS functionality |
| P1 | Review router mappings and tailnet ACLs; narrow service exposure where warranted | Audit was limited to sampled host listeners; test intended and denied access without locking out deployment | | P1 | Review router mappings and tailnet ACLs; narrow service exposure where warranted | Audit was limited to sampled host listeners; test intended and denied access without locking out deployment |
| P1 | Review NFS client scope and root identity requirements | Restrict to required VM hosts/CSI nodes where practical; test provisioning, permissions and existing PVCs before changing squash semantics | | P1 | Review NFS client scope and root identity requirements | Restrict to required VM hosts/CSI nodes where practical; test provisioning, permissions and existing PVCs before changing squash semantics |
WORKAROUNDS.md
+4 -5
View File
@@ -84,11 +84,10 @@ A known limitation with no fix yet.
An unfinished feature or gap. An unfinished feature or gap.
| ID | What & where | Why | Removal condition | Added | Revisit | Status | | ID | What & where | Why | Removal condition | Added | Revisit | Status |
| ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | ---------- | ----------------------- | ------ | | ------ | --------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------- | ---------- | ----------------------- | ------ |
| WA-013 | NixOS tests — `outputs/README.md` | full-host tests still rebuild custom packages (no private cache) and need the host ssh key to decrypt agenix; the shared-firewall security-exporters VM test now runs in CI (KVM); `outputs/x86_64-linux/nixos-tests/idols-ruby.nix` is exposed as `packages.x86_64-linux.ruby-nixos-tests` but is not in `checks` or CI | Resolve when a private cache exists and tests can run per-service | 2024-03-13 | quarterly | active | | WA-013 | NixOS tests — `outputs/README.md` | full-host tests still rebuild custom packages (no private cache) and need the host ssh key to decrypt agenix; the shared-firewall security-exporters VM test now runs in CI (KVM); `outputs/x86_64-linux/nixos-tests/idols-ruby.nix` is exposed as `packages.x86_64-linux.ruby-nixos-tests` but is not in `checks` or CI | Resolve when a private cache exists and tests can run per-service | 2024-03-13 | quarterly | active |
| WA-027 | NFS golden store `no_root_squash` — `hosts/12kingdoms-youko/nfs-golden.nix` | the existing PVC subdirectories are root-owned, so enabling `root_squash` would break running workloads until they are migrated; the CSI node plugin is hostNetwork and sources from the allowlisted node IPs | Drop when the PVC subdirectories are migrated and `root_squash` is enabled | 2026-10-07 | next maintenance window | active | | WA-027 | NFS golden store `no_root_squash` — `hosts/12kingdoms-youko/nfs-golden.nix` | the existing PVC subdirectories are root-owned, so enabling `root_squash` would break running workloads until they are migrated; the CSI node plugin is hostNetwork and sources from the allowlisted node IPs | Drop when the PVC subdirectories are migrated and `root_squash` is enabled | 2026-10-07 | next maintenance window | active |
| WA-043 | k3s-test-1 bootstrap cycle: kube-vip's API client pinned to the local apiserver — k8s-gitops `infra/pre-controllers/base/kube-vip/helm-release.yaml` | a full cold start of all six microVM guests (2026-10-14) left no node holding the VIP: the in-cluster kube-vip could only win its lease through ClusterIP, ClusterIP is Cilium's, and Cilium's config init crash-loops until the VIP exists (`no route to host 192.168.5.193:6443`); all pod networking stays down until broken by hand. The pod now overrides the kubelet-injected service env (`KUBERNETES_SERVICE_HOST` = `status.hostIP`, `KUBERNETES_SERVICE_PORT` = 6443), so kube-vip reaches the local apiserver before any CNI exists | Drop once k8s-gitops#54 is merged and reconciled; the 2026-10-24 full cold start already returned the VIP without manual action | 2026-10-24 | when #54 merges | active |
## Related ## Related