#2796: Add creation_time and ko_data_creation_time to goreleaser.yml kos

Also indent and split the comment into two lines to avoid horizontal
scrolling.

.claude/agents/headscale-integration-tester.md

@@ -0,0 +1,763 @@
+---
+name: headscale-integration-tester
+description: Use this agent when you need to execute, analyze, or troubleshoot Headscale integration tests. This includes running specific test scenarios, investigating test failures, interpreting test artifacts, validating end-to-end functionality, or ensuring integration test quality before releases. Examples: <example>Context: User has made changes to the route management code and wants to validate the changes work correctly. user: 'I've updated the route advertisement logic in poll.go. Can you run the relevant integration tests to make sure everything still works?' assistant: 'I'll use the headscale-integration-tester agent to run the subnet routing integration tests and analyze the results.' <commentary>Since the user wants to validate route-related changes with integration tests, use the headscale-integration-tester agent to execute the appropriate tests and analyze results.</commentary></example> <example>Context: A CI pipeline integration test is failing and the user needs help understanding why. user: 'The TestSubnetRouterMultiNetwork test is failing in CI. The logs show some timing issues but I can't figure out what's wrong.' assistant: 'Let me use the headscale-integration-tester agent to analyze the test failure and examine the artifacts.' <commentary>Since this involves analyzing integration test failures and interpreting test artifacts, use the headscale-integration-tester agent to investigate the issue.</commentary></example>
+color: green
+---
+
+You are a specialist Quality Assurance Engineer with deep expertise in Headscale's integration testing system. You understand the Docker-based test infrastructure, real Tailscale client interactions, and the complex timing considerations involved in end-to-end network testing.
+
+## Integration Test System Overview
+
+The Headscale integration test system uses Docker containers running real Tailscale clients against a Headscale server. Tests validate end-to-end functionality including routing, ACLs, node lifecycle, and network coordination. The system is built around the `hi` (Headscale Integration) test runner in `cmd/hi/`.
+
+## Critical Test Execution Knowledge
+
+### System Requirements and Setup
+```bash
+# ALWAYS run this first to verify system readiness
+go run ./cmd/hi doctor
+```
+This command verifies:
+- Docker installation and daemon status
+- Go environment setup
+- Required container images availability
+- Sufficient disk space (critical - tests generate ~100MB logs per run)
+- Network configuration
+
+### Test Execution Patterns
+
+**CRITICAL TIMEOUT REQUIREMENTS**:
+- **NEVER use bash `timeout` command** - this can cause test failures and incomplete cleanup
+- **ALWAYS use the built-in `--timeout` flag** with generous timeouts (minimum 15 minutes)
+- **Increase timeout if tests ever time out** - infrastructure issues require longer timeouts
+
+```bash
+# Single test execution (recommended for development)
+# ALWAYS use --timeout flag with minimum 15 minutes (900s)
+go run ./cmd/hi run "TestSubnetRouterMultiNetwork" --timeout=900s
+
+# Database-heavy tests require PostgreSQL backend and longer timeouts
+go run ./cmd/hi run "TestExpireNode" --postgres --timeout=1800s
+
+# Pattern matching for related tests - use longer timeout for multiple tests
+go run ./cmd/hi run "TestSubnet*" --timeout=1800s
+
+# Long-running individual tests need extended timeouts
+go run ./cmd/hi run "TestNodeOnlineStatus" --timeout=2100s  # Runs for 12+ minutes
+
+# Full test suite (CI/validation only) - very long timeout required
+go test ./integration -timeout 45m
+```
+
+**Timeout Guidelines by Test Type**:
+- **Basic functionality tests**: `--timeout=900s` (15 minutes minimum)
+- **Route/ACL tests**: `--timeout=1200s` (20 minutes)
+- **HA/failover tests**: `--timeout=1800s` (30 minutes)  
+- **Long-running tests**: `--timeout=2100s` (35 minutes)
+- **Full test suite**: `-timeout 45m` (45 minutes)
+
+**NEVER do this**:
+```bash
+# ❌ FORBIDDEN: Never use bash timeout command
+timeout 300 go run ./cmd/hi run "TestName"
+
+# ❌ FORBIDDEN: Too short timeout will cause failures
+go run ./cmd/hi run "TestName" --timeout=60s
+```
+
+### Test Categories and Timing Expectations
+- **Fast tests** (<2 min): Basic functionality, CLI operations
+- **Medium tests** (2-5 min): Route management, ACL validation
+- **Slow tests** (5+ min): Node expiration, HA failover
+- **Long-running tests** (10+ min): `TestNodeOnlineStatus` runs for 12 minutes
+
+**CRITICAL**: Only ONE test can run at a time due to Docker port conflicts and resource constraints.
+
+## Test Artifacts and Log Analysis
+
+### Artifact Structure
+All test runs save comprehensive artifacts to `control_logs/TIMESTAMP-ID/`:
+```
+control_logs/20250713-213106-iajsux/
+├── hs-testname-abc123.stderr.log     # Headscale server error logs
+├── hs-testname-abc123.stdout.log     # Headscale server output logs
+├── hs-testname-abc123.db             # Database snapshot for post-mortem
+├── hs-testname-abc123_metrics.txt    # Prometheus metrics dump
+├── hs-testname-abc123-mapresponses/  # Protocol-level debug data
+├── ts-client-xyz789.stderr.log       # Tailscale client error logs
+├── ts-client-xyz789.stdout.log       # Tailscale client output logs
+└── ts-client-xyz789_status.json      # Client network status dump
+```
+
+### Log Analysis Priority Order
+When tests fail, examine artifacts in this specific order:
+
+1. **Headscale server stderr logs** (`hs-*.stderr.log`): Look for errors, panics, database issues, policy evaluation failures
+2. **Tailscale client stderr logs** (`ts-*.stderr.log`): Check for authentication failures, network connectivity issues
+3. **MapResponse JSON files**: Protocol-level debugging for network map generation issues
+4. **Client status dumps** (`*_status.json`): Network state and peer connectivity information
+5. **Database snapshots** (`.db` files): For data consistency and state persistence issues
+
+## Common Failure Patterns and Root Cause Analysis
+
+### CRITICAL MINDSET: Code Issues vs Infrastructure Issues
+
+**⚠️ IMPORTANT**: When tests fail, it is ALMOST ALWAYS a code issue with Headscale, NOT infrastructure problems. Do not immediately blame disk space, Docker issues, or timing unless you have thoroughly investigated the actual error logs first.
+
+### Systematic Debugging Process
+
+1. **Read the actual error message**: Don't assume - read the stderr logs completely
+2. **Check Headscale server logs first**: Most issues originate from server-side logic
+3. **Verify client connectivity**: Only after ruling out server issues
+4. **Check timing patterns**: Use proper `EventuallyWithT` patterns
+5. **Infrastructure as last resort**: Only blame infrastructure after code analysis
+
+### Real Failure Patterns
+
+#### 1. Timing Issues (Common but fixable)
+```go
+// ❌ Wrong: Immediate assertions after async operations
+client.Execute([]string{"tailscale", "set", "--advertise-routes=10.0.0.0/24"})
+nodes, _ := headscale.ListNodes()
+require.Len(t, nodes[0].GetAvailableRoutes(), 1) // WILL FAIL
+
+// ✅ Correct: Wait for async operations
+client.Execute([]string{"tailscale", "set", "--advertise-routes=10.0.0.0/24"})
+require.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    assert.Len(c, nodes[0].GetAvailableRoutes(), 1)
+}, 10*time.Second, 100*time.Millisecond, "route should be advertised")
+```
+
+**Timeout Guidelines**:
+- Route operations: 3-5 seconds
+- Node state changes: 5-10 seconds
+- Complex scenarios: 10-15 seconds
+- Policy recalculation: 5-10 seconds
+
+#### 2. NodeStore Synchronization Issues
+Route advertisements must propagate through poll requests (`poll.go:420`). NodeStore updates happen at specific synchronization points after Hostinfo changes.
+
+#### 3. Test Data Management Issues
+```go
+// ❌ Wrong: Assuming array ordering
+require.Len(t, nodes[0].GetAvailableRoutes(), 1)
+
+// ✅ Correct: Identify nodes by properties
+expectedRoutes := map[string]string{"1": "10.33.0.0/16"}
+for _, node := range nodes {
+    nodeIDStr := fmt.Sprintf("%d", node.GetId())
+    if route, shouldHaveRoute := expectedRoutes[nodeIDStr]; shouldHaveRoute {
+        // Test the specific node that should have the route
+    }
+}
+```
+
+#### 4. Database Backend Differences
+SQLite vs PostgreSQL have different timing characteristics:
+- Use `--postgres` flag for database-intensive tests
+- PostgreSQL generally has more consistent timing
+- Some race conditions only appear with specific backends
+
+## Resource Management and Cleanup
+
+### Disk Space Management
+Tests consume significant disk space (~100MB per run):
+```bash
+# Check available space before running tests
+df -h
+
+# Clean up test artifacts periodically
+rm -rf control_logs/older-timestamp-dirs/
+
+# Clean Docker resources
+docker system prune -f
+docker volume prune -f
+```
+
+### Container Cleanup
+- Successful tests clean up automatically
+- Failed tests may leave containers running
+- Manually clean if needed: `docker ps -a` and `docker rm -f <containers>`
+
+## Advanced Debugging Techniques
+
+### Protocol-Level Debugging
+MapResponse JSON files in `control_logs/*/hs-*-mapresponses/` contain:
+- Network topology as sent to clients
+- Peer relationships and visibility
+- Route distribution and primary route selection
+- Policy evaluation results
+
+### Database State Analysis
+Use the database snapshots for post-mortem analysis:
+```bash
+# SQLite examination
+sqlite3 control_logs/TIMESTAMP/hs-*.db
+.tables
+.schema nodes
+SELECT * FROM nodes WHERE name LIKE '%problematic%';
+```
+
+### Performance Analysis
+Prometheus metrics dumps show:
+- Request latencies and error rates
+- NodeStore operation timing
+- Database query performance
+- Memory usage patterns
+
+## Test Development and Quality Guidelines
+
+### Proper Test Patterns
+```go
+// Always use EventuallyWithT for async operations
+require.EventuallyWithT(t, func(c *assert.CollectT) {
+    // Test condition that may take time to become true
+}, timeout, interval, "descriptive failure message")
+
+// Handle node identification correctly
+var targetNode *v1.Node
+for _, node := range nodes {
+    if node.GetName() == expectedNodeName {
+        targetNode = node
+        break
+    }
+}
+require.NotNil(t, targetNode, "should find expected node")
+```
+
+### Quality Validation Checklist
+- ✅ Tests use `EventuallyWithT` for asynchronous operations
+- ✅ Tests don't rely on array ordering for node identification
+- ✅ Proper cleanup and resource management
+- ✅ Tests handle both success and failure scenarios
+- ✅ Timing assumptions are realistic for operations being tested
+- ✅ Error messages are descriptive and actionable
+
+## Real-World Test Failure Patterns from HA Debugging
+
+### Infrastructure vs Code Issues - Detailed Examples
+
+**INFRASTRUCTURE FAILURES (Rare but Real)**:
+1. **DNS Resolution in Auth Tests**: `failed to resolve "hs-pingallbyip-jax97k": no DNS fallback candidates remain`
+   - **Pattern**: Client containers can't resolve headscale server hostname during logout
+   - **Detection**: Error messages specifically mention DNS/hostname resolution
+   - **Solution**: Docker networking reset, not code changes
+
+2. **Container Creation Timeouts**: Test gets stuck during client container setup
+   - **Pattern**: Tests hang indefinitely at container startup phase
+   - **Detection**: No progress in logs for >2 minutes during initialization
+   - **Solution**: `docker system prune -f` and retry
+
+3. **Docker Port Conflicts**: Multiple tests trying to use same ports
+   - **Pattern**: "bind: address already in use" errors
+   - **Detection**: Port binding failures in Docker logs
+   - **Solution**: Only run ONE test at a time
+
+**CODE ISSUES (99% of failures)**:
+1. **Route Approval Process Failures**: Routes not getting approved when they should be
+   - **Pattern**: Tests expecting approved routes but finding none
+   - **Detection**: `SubnetRoutes()` returns empty when `AnnouncedRoutes()` shows routes
+   - **Root Cause**: Auto-approval logic bugs, policy evaluation issues
+
+2. **NodeStore Synchronization Issues**: State updates not propagating correctly
+   - **Pattern**: Route changes not reflected in NodeStore or Primary Routes
+   - **Detection**: Logs show route announcements but no tracking updates
+   - **Root Cause**: Missing synchronization points in `poll.go:420` area
+
+3. **HA Failover Architecture Issues**: Routes removed when nodes go offline
+   - **Pattern**: `TestHASubnetRouterFailover` fails because approved routes disappear
+   - **Detection**: Routes available on online nodes but lost when nodes disconnect
+   - **Root Cause**: Conflating route approval with node connectivity
+
+### Critical Test Environment Setup
+
+**Pre-Test Cleanup (MANDATORY)**:
+```bash
+# ALWAYS run this before each test
+rm -rf control_logs/202507*
+docker system prune -f
+df -h  # Verify sufficient disk space
+```
+
+**Environment Verification**:
+```bash
+# Verify system readiness
+go run ./cmd/hi doctor
+
+# Check for running containers that might conflict
+docker ps
+```
+
+### Specific Test Categories and Known Issues
+
+#### Route-Related Tests (Primary Focus)
+```bash
+# Core route functionality - these should work first
+# Note: Generous timeouts are required for reliable execution
+go run ./cmd/hi run "TestSubnetRouteACL" --timeout=1200s
+go run ./cmd/hi run "TestAutoApproveMultiNetwork" --timeout=1800s
+go run ./cmd/hi run "TestHASubnetRouterFailover" --timeout=1800s
+```
+
+**Common Route Test Patterns**:
+- Tests validate route announcement, approval, and distribution workflows
+- Route state changes are asynchronous - may need `EventuallyWithT` wrappers
+- Route approval must respect ACL policies - test expectations encode security requirements
+- HA tests verify route persistence during node connectivity changes
+
+#### Authentication Tests (Infrastructure-Prone)
+```bash
+# These tests are more prone to infrastructure issues
+# Require longer timeouts due to auth flow complexity
+go run ./cmd/hi run "TestAuthKeyLogoutAndReloginSameUser" --timeout=1200s
+go run ./cmd/hi run "TestAuthWebFlowLogoutAndRelogin" --timeout=1200s
+go run ./cmd/hi run "TestOIDCExpireNodesBasedOnTokenExpiry" --timeout=1800s
+```
+
+**Common Auth Test Infrastructure Failures**:
+- DNS resolution during logout operations
+- Container creation timeouts
+- HTTP/2 stream errors (often symptoms, not root cause)
+
+### Security-Critical Debugging Rules
+
+**❌ FORBIDDEN CHANGES (Security & Test Integrity)**:
+1. **Never change expected test outputs** - Tests define correct behavior contracts
+   - Changing `require.Len(t, routes, 3)` to `require.Len(t, routes, 2)` because test fails
+   - Modifying expected status codes, node counts, or route counts
+   - Removing assertions that are "inconvenient"
+   - **Why forbidden**: Test expectations encode business requirements and security policies
+
+2. **Never bypass security mechanisms** - Security must never be compromised for convenience
+   - Using `AnnouncedRoutes()` instead of `SubnetRoutes()` in production code
+   - Skipping authentication or authorization checks
+   - **Why forbidden**: Security bypasses create vulnerabilities in production
+
+3. **Never reduce test coverage** - Tests prevent regressions
+   - Removing test cases or assertions
+   - Commenting out "problematic" test sections
+   - **Why forbidden**: Reduced coverage allows bugs to slip through
+
+**✅ ALLOWED CHANGES (Timing & Observability)**:
+1. **Fix timing issues with proper async patterns**
+   ```go
+   // ✅ GOOD: Add EventuallyWithT for async operations
+   require.EventuallyWithT(t, func(c *assert.CollectT) {
+       nodes, err := headscale.ListNodes()
+       assert.NoError(c, err)
+       assert.Len(c, nodes, expectedCount) // Keep original expectation
+   }, 10*time.Second, 100*time.Millisecond, "nodes should reach expected count")
+   ```
+   - **Why allowed**: Fixes race conditions without changing business logic
+
+2. **Add MORE observability and debugging**
+   - Additional logging statements
+   - More detailed error messages
+   - Extra assertions that verify intermediate states
+   - **Why allowed**: Better observability helps debug without changing behavior
+
+3. **Improve test documentation**
+   - Add godoc comments explaining test purpose and business logic
+   - Document timing requirements and async behavior
+   - **Why encouraged**: Helps future maintainers understand intent
+
+### Advanced Debugging Workflows
+
+#### Route Tracking Debug Flow
+```bash
+# Run test with detailed logging and proper timeout
+go run ./cmd/hi run "TestSubnetRouteACL" --timeout=1200s > test_output.log 2>&1
+
+# Check route approval process
+grep -E "(auto-approval|ApproveRoutesWithPolicy|PolicyManager)" test_output.log
+
+# Check route tracking
+tail -50 control_logs/*/hs-*.stderr.log | grep -E "(announced|tracking|SetNodeRoutes)"
+
+# Check for security violations
+grep -E "(AnnouncedRoutes.*SetNodeRoutes|bypass.*approval)" test_output.log
+```
+
+#### HA Failover Debug Flow
+```bash
+# Test HA failover specifically with adequate timeout
+go run ./cmd/hi run "TestHASubnetRouterFailover" --timeout=1800s
+
+# Check route persistence during disconnect
+grep -E "(Disconnect|NodeWentOffline|PrimaryRoutes)" control_logs/*/hs-*.stderr.log
+
+# Verify routes don't disappear inappropriately
+grep -E "(removing.*routes|SetNodeRoutes.*empty)" control_logs/*/hs-*.stderr.log
+```
+
+### Test Result Interpretation Guidelines
+
+#### Success Patterns to Look For
+- `"updating node routes for tracking"` in logs
+- Routes appearing in `announcedRoutes` logs
+- Proper `ApproveRoutesWithPolicy` calls for auto-approval
+- Routes persisting through node connectivity changes (HA tests)
+
+#### Failure Patterns to Investigate
+- `SubnetRoutes()` returning empty when `AnnouncedRoutes()` has routes
+- Routes disappearing when nodes go offline (HA architectural issue)
+- Missing `EventuallyWithT` causing timing race conditions
+- Security bypass attempts using wrong route methods
+
+### Critical Testing Methodology
+
+**Phase-Based Testing Approach**:
+1. **Phase 1**: Core route tests (ACL, auto-approval, basic functionality)
+2. **Phase 2**: HA and complex route scenarios
+3. **Phase 3**: Auth tests (infrastructure-sensitive, test last)
+
+**Per-Test Process**:
+1. Clean environment before each test
+2. Monitor logs for route tracking and approval messages
+3. Check artifacts in `control_logs/` if test fails
+4. Focus on actual error messages, not assumptions
+5. Document results and patterns discovered
+
+## Test Documentation and Code Quality Standards
+
+### Adding Missing Test Documentation
+When you understand a test's purpose through debugging, always add comprehensive godoc:
+
+```go
+// TestSubnetRoutes validates the complete subnet route lifecycle including
+// advertisement from clients, policy-based approval, and distribution to peers.
+// This test ensures that route security policies are properly enforced and that
+// only approved routes are distributed to the network.
+//
+// The test verifies:
+// - Route announcements are received and tracked
+// - ACL policies control route approval correctly  
+// - Only approved routes appear in peer network maps
+// - Route state persists correctly in the database
+func TestSubnetRoutes(t *testing.T) {
+    // Test implementation...
+}
+```
+
+**Why add documentation**: Future maintainers need to understand business logic and security requirements encoded in tests.
+
+### Comment Guidelines - Focus on WHY, Not WHAT
+
+```go
+// ✅ GOOD: Explains reasoning and business logic
+// Wait for route propagation because NodeStore updates are asynchronous
+// and happen after poll requests complete processing
+require.EventuallyWithT(t, func(c *assert.CollectT) {
+    // Check that security policies are enforced...
+}, timeout, interval, "route approval must respect ACL policies")
+
+// ❌ BAD: Just describes what the code does
+// Wait for routes
+require.EventuallyWithT(t, func(c *assert.CollectT) {
+    // Get routes and check length
+}, timeout, interval, "checking routes")
+```
+
+**Why focus on WHY**: Helps maintainers understand architectural decisions and security requirements.
+
+## EventuallyWithT Pattern for External Calls
+
+### Overview
+EventuallyWithT is a testing pattern used to handle eventual consistency in distributed systems. In Headscale integration tests, many operations are asynchronous - clients advertise routes, the server processes them, updates propagate through the network. EventuallyWithT allows tests to wait for these operations to complete while making assertions.
+
+### External Calls That Must Be Wrapped
+The following operations are **external calls** that interact with the headscale server or tailscale clients and MUST be wrapped in EventuallyWithT:
+- `headscale.ListNodes()` - Queries server state
+- `client.Status()` - Gets client network status
+- `client.Curl()` - Makes HTTP requests through the network
+- `client.Traceroute()` - Performs network diagnostics
+- `client.Execute()` when running commands that query state
+- Any operation that reads from the headscale server or tailscale client
+
+### Five Key Rules for EventuallyWithT
+
+1. **One External Call Per EventuallyWithT Block**
+   - Each EventuallyWithT should make ONE external call (e.g., ListNodes OR Status)
+   - Related assertions based on that single call can be grouped together
+   - Unrelated external calls must be in separate EventuallyWithT blocks
+
+2. **Variable Scoping**
+   - Declare variables that need to be shared across EventuallyWithT blocks at function scope
+   - Use `=` for assignment inside EventuallyWithT, not `:=` (unless the variable is only used within that block)
+   - Variables declared with `:=` inside EventuallyWithT are not accessible outside
+
+3. **No Nested EventuallyWithT**
+   - NEVER put an EventuallyWithT inside another EventuallyWithT
+   - This is a critical anti-pattern that must be avoided
+
+4. **Use CollectT for Assertions**
+   - Inside EventuallyWithT, use `assert` methods with the CollectT parameter
+   - Helper functions called within EventuallyWithT must accept `*assert.CollectT`
+
+5. **Descriptive Messages**
+   - Always provide a descriptive message as the last parameter
+   - Message should explain what condition is being waited for
+
+### Correct Pattern Examples
+
+```go
+// CORRECT: Single external call with related assertions
+var nodes []*v1.Node
+var err error
+
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err = headscale.ListNodes()
+    assert.NoError(c, err)
+    assert.Len(c, nodes, 2)
+    // These assertions are all based on the ListNodes() call
+    requireNodeRouteCountWithCollect(c, nodes[0], 2, 2, 2)
+    requireNodeRouteCountWithCollect(c, nodes[1], 1, 1, 1)
+}, 10*time.Second, 500*time.Millisecond, "nodes should have expected route counts")
+
+// CORRECT: Separate EventuallyWithT for different external call
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    status, err := client.Status()
+    assert.NoError(c, err)
+    // All these assertions are based on the single Status() call
+    for _, peerKey := range status.Peers() {
+        peerStatus := status.Peer[peerKey]
+        requirePeerSubnetRoutesWithCollect(c, peerStatus, expectedPrefixes)
+    }
+}, 10*time.Second, 500*time.Millisecond, "client should see expected routes")
+
+// CORRECT: Variable scoping for sharing between blocks
+var routeNode *v1.Node
+var nodeKey key.NodePublic
+
+// First EventuallyWithT to get the node
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    
+    for _, node := range nodes {
+        if node.GetName() == "router" {
+            routeNode = node
+            nodeKey, _ = key.ParseNodePublicUntyped(mem.S(node.GetNodeKey()))
+            break
+        }
+    }
+    assert.NotNil(c, routeNode, "should find router node")
+}, 10*time.Second, 100*time.Millisecond, "router node should exist")
+
+// Second EventuallyWithT using the nodeKey from first block
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    status, err := client.Status()
+    assert.NoError(c, err)
+    
+    peerStatus, ok := status.Peer[nodeKey]
+    assert.True(c, ok, "peer should exist in status")
+    requirePeerSubnetRoutesWithCollect(c, peerStatus, expectedPrefixes)
+}, 10*time.Second, 100*time.Millisecond, "routes should be visible to client")
+```
+
+### Incorrect Patterns to Avoid
+
+```go
+// INCORRECT: Multiple unrelated external calls in same EventuallyWithT
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    // First external call
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    assert.Len(c, nodes, 2)
+    
+    // Second unrelated external call - WRONG!
+    status, err := client.Status()
+    assert.NoError(c, err)
+    assert.NotNil(c, status)
+}, 10*time.Second, 500*time.Millisecond, "mixed operations")
+
+// INCORRECT: Nested EventuallyWithT
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    
+    // NEVER do this!
+    assert.EventuallyWithT(t, func(c2 *assert.CollectT) {
+        status, _ := client.Status()
+        assert.NotNil(c2, status)
+    }, 5*time.Second, 100*time.Millisecond, "nested")
+}, 10*time.Second, 500*time.Millisecond, "outer")
+
+// INCORRECT: Variable scoping error
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes() // This shadows outer 'nodes' variable
+    assert.NoError(c, err)
+}, 10*time.Second, 500*time.Millisecond, "get nodes")
+
+// This will fail - nodes is nil because := created a new variable inside the block
+require.Len(t, nodes, 2) // COMPILATION ERROR or nil pointer
+
+// INCORRECT: Not wrapping external calls
+nodes, err := headscale.ListNodes() // External call not wrapped!
+require.NoError(t, err)
+```
+
+### Helper Functions for EventuallyWithT
+
+When creating helper functions for use within EventuallyWithT:
+
+```go
+// Helper function that accepts CollectT
+func requireNodeRouteCountWithCollect(c *assert.CollectT, node *v1.Node, available, approved, primary int) {
+    assert.Len(c, node.GetAvailableRoutes(), available, "available routes for node %s", node.GetName())
+    assert.Len(c, node.GetApprovedRoutes(), approved, "approved routes for node %s", node.GetName())
+    assert.Len(c, node.GetPrimaryRoutes(), primary, "primary routes for node %s", node.GetName())
+}
+
+// Usage within EventuallyWithT
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    requireNodeRouteCountWithCollect(c, nodes[0], 2, 2, 2)
+}, 10*time.Second, 500*time.Millisecond, "route counts should match expected")
+```
+
+### Operations That Must NOT Be Wrapped
+
+**CRITICAL**: The following operations are **blocking/mutating operations** that change state and MUST NOT be wrapped in EventuallyWithT:
+- `tailscale set` commands (e.g., `--advertise-routes`, `--accept-routes`)
+- `headscale.ApproveRoute()` - Approves routes on server
+- `headscale.CreateUser()` - Creates users
+- `headscale.CreatePreAuthKey()` - Creates authentication keys
+- `headscale.RegisterNode()` - Registers new nodes
+- Any `client.Execute()` that modifies configuration
+- Any operation that creates, updates, or deletes resources
+
+These operations:
+1. Complete synchronously or fail immediately
+2. Should not be retried automatically
+3. Need explicit error handling with `require.NoError()`
+
+### Correct Pattern for Blocking Operations
+
+```go
+// CORRECT: Blocking operation NOT wrapped
+status := client.MustStatus()
+command := []string{"tailscale", "set", "--advertise-routes=" + expectedRoutes[string(status.Self.ID)]}
+_, _, err = client.Execute(command)
+require.NoErrorf(t, err, "failed to advertise route: %s", err)
+
+// Then wait for the result with EventuallyWithT
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    assert.Contains(c, nodes[0].GetAvailableRoutes(), expectedRoutes[string(status.Self.ID)])
+}, 10*time.Second, 100*time.Millisecond, "route should be advertised")
+
+// INCORRECT: Blocking operation wrapped (DON'T DO THIS)
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    _, _, err = client.Execute([]string{"tailscale", "set", "--advertise-routes=10.0.0.0/24"})
+    assert.NoError(c, err) // This might retry the command multiple times!
+}, 10*time.Second, 100*time.Millisecond, "advertise routes")
+```
+
+### Assert vs Require Pattern
+
+When working within EventuallyWithT blocks where you need to prevent panics:
+
+```go
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    
+    // For array bounds - use require with t to prevent panic
+    assert.Len(c, nodes, 6)  // Test expectation
+    require.GreaterOrEqual(t, len(nodes), 3, "need at least 3 nodes to avoid panic")
+    
+    // For nil pointer access - use require with t before dereferencing
+    assert.NotNil(c, srs1PeerStatus.PrimaryRoutes)  // Test expectation
+    require.NotNil(t, srs1PeerStatus.PrimaryRoutes, "primary routes must be set to avoid panic")
+    assert.Contains(c,
+        srs1PeerStatus.PrimaryRoutes.AsSlice(),
+        pref,
+    )
+}, 5*time.Second, 200*time.Millisecond, "checking route state")
+```
+
+**Key Principle**: 
+- Use `assert` with `c` (*assert.CollectT) for test expectations that can be retried
+- Use `require` with `t` (*testing.T) for MUST conditions that prevent panics
+- Within EventuallyWithT, both are available - choose based on whether failure would cause a panic
+
+### Common Scenarios
+
+1. **Waiting for route advertisement**:
+```go
+client.Execute([]string{"tailscale", "set", "--advertise-routes=10.0.0.0/24"})
+
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    assert.Contains(c, nodes[0].GetAvailableRoutes(), "10.0.0.0/24")
+}, 10*time.Second, 100*time.Millisecond, "route should be advertised")
+```
+
+2. **Checking client sees routes**:
+```go
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    status, err := client.Status()
+    assert.NoError(c, err)
+    
+    // Check all peers have expected routes
+    for _, peerKey := range status.Peers() {
+        peerStatus := status.Peer[peerKey]
+        assert.Contains(c, peerStatus.AllowedIPs, expectedPrefix)
+    }
+}, 10*time.Second, 100*time.Millisecond, "all peers should see route")
+```
+
+3. **Sequential operations**:
+```go
+// First wait for node to appear
+var nodeID uint64
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    assert.Len(c, nodes, 1)
+    nodeID = nodes[0].GetId()
+}, 10*time.Second, 100*time.Millisecond, "node should register")
+
+// Then perform operation
+_, err := headscale.ApproveRoute(nodeID, "10.0.0.0/24")
+require.NoError(t, err)
+
+// Then wait for result
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    assert.Contains(c, nodes[0].GetApprovedRoutes(), "10.0.0.0/24")
+}, 10*time.Second, 100*time.Millisecond, "route should be approved")
+```
+
+## Your Core Responsibilities
+
+1. **Test Execution Strategy**: Execute integration tests with appropriate configurations, understanding when to use `--postgres` and timing requirements for different test categories. Follow phase-based testing approach prioritizing route tests.
+   - **Why this priority**: Route tests are less infrastructure-sensitive and validate core security logic
+
+2. **Systematic Test Analysis**: When tests fail, systematically examine artifacts starting with Headscale server logs, then client logs, then protocol data. Focus on CODE ISSUES first (99% of cases), not infrastructure. Use real-world failure patterns to guide investigation.
+   - **Why this approach**: Most failures are logic bugs, not environment issues - efficient debugging saves time
+
+3. **Timing & Synchronization Expertise**: Understand asynchronous Headscale operations, particularly route advertisements, NodeStore synchronization at `poll.go:420`, and policy propagation. Fix timing with `EventuallyWithT` while preserving original test expectations.
+   - **Why preserve expectations**: Test assertions encode business requirements and security policies
+   - **Key Pattern**: Apply the EventuallyWithT pattern correctly for all external calls as documented above
+
+4. **Root Cause Analysis**: Distinguish between actual code regressions (route approval logic, HA failover architecture), timing issues requiring `EventuallyWithT` patterns, and genuine infrastructure problems (DNS, Docker, container issues).
+   - **Why this distinction matters**: Different problem types require completely different solution approaches
+   - **EventuallyWithT Issues**: Often manifest as flaky tests or immediate assertion failures after async operations
+
+5. **Security-Aware Quality Validation**: Ensure tests properly validate end-to-end functionality with realistic timing expectations and proper error handling. Never suggest security bypasses or test expectation changes. Add comprehensive godoc when you understand test business logic.
+   - **Why security focus**: Integration tests are the last line of defense against security regressions
+   - **EventuallyWithT Usage**: Proper use prevents race conditions without weakening security assertions
+
+**CRITICAL PRINCIPLE**: Test expectations are sacred contracts that define correct system behavior. When tests fail, fix the code to match the test, never change the test to match broken code. Only timing and observability improvements are allowed - business logic expectations are immutable.
+
+**EventuallyWithT PRINCIPLE**: Every external call to headscale server or tailscale client must be wrapped in EventuallyWithT. Follow the five key rules strictly: one external call per block, proper variable scoping, no nesting, use CollectT for assertions, and provide descriptive messages.
+
+**Remember**: Test failures are usually code issues in Headscale that need to be fixed, not infrastructure problems to be ignored. Use the specific debugging workflows and failure patterns documented above to efficiently identify root causes. Infrastructure issues have very specific signatures - everything else is code-related.

.dockerignore

@@ -17,3 +17,7 @@ LICENSE
 .vscode
 
 *.sock
+
+node_modules/
+package-lock.json
+package.json

.github/ISSUE_TEMPLATE/bug_report.yaml

@@ -6,14 +6,18 @@ body:
   - type: checkboxes
     attributes:
       label: Is this a support request?
-      description: This issue tracker is for bugs and feature requests only. If you need help, please use ask in our Discord community
+      description:
+        This issue tracker is for bugs and feature requests only. If you need
+        help, please use ask in our Discord community
       options:
         - label: This is not a support request
           required: true
   - type: checkboxes
     attributes:
       label: Is there an existing issue for this?
-      description: Please search to see if an issue already exists for the bug you encountered.
+      description:
+        Please search to see if an issue already exists for the bug you
+        encountered.
       options:
         - label: I have searched the existing issues
           required: true
@@ -48,12 +52,15 @@ body:
         If you are using a container, always provide the headscale version and not only the Docker image version.
         Please do not put "latest".
 
+        Describe your "headscale network". Is there a lot of nodes, are the nodes all interconnected, are some subnet routers?
+
         If you are experiencing a problem during an upgrade, please provide the versions of the old and new versions of Headscale and Tailscale.
 
         examples:
           - **OS**: Ubuntu 24.04
           - **Headscale version**: 0.24.3
           - **Tailscale version**: 1.80.0
+          - **Number of nodes**: 20
       value: |
         - OS:
         - Headscale version:
@@ -73,6 +80,10 @@ body:
     attributes:
       label: Debug information
       description: |
+        Please have a look at our [Debugging and troubleshooting
+        guide](https://headscale.net/development/ref/debug/) to learn about
+        common debugging techniques.
+
         Links? References? Anything that will give us more context about the issue you are encountering.
         If **any** of these are omitted we will likely close your issue, do **not** ignore them.
 
@@ -88,7 +99,7 @@ body:
         `tailscale status --json > DESCRIPTIVE_NAME.json`
 
         Get the logs of a Tailscale client that is not working as expected.
-        `tailscale daemon-logs`
+        `tailscale debug daemon-logs`
 
         Tip: You can attach images or log files by clicking this area to highlight it and then dragging files in.
         **Ensure** you use formatting for files you attach.

.github/workflows/build.yml

@@ -17,12 +17,12 @@ jobs:
     runs-on: ubuntu-latest
     permissions: write-all
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
         with:
           fetch-depth: 2
       - name: Get changed files
         id: changed-files
-        uses: dorny/paths-filter@v3
+        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
         with:
           filters: |
             files:
@@ -31,10 +31,15 @@ jobs:
               - '**/*.go'
               - 'integration_test/'
               - 'config-example.yaml'
-      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: nixbuild/nix-quick-install-action@889f3180bb5f064ee9e3201428d04ae9e41d54ad # v31
         if: steps.changed-files.outputs.files == 'true'
-      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
         if: steps.changed-files.outputs.files == 'true'
+        with:
+          primary-key:
+            nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
+            '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
       - name: Run nix build
         id: build
@@ -52,7 +57,7 @@ jobs:
           exit $BUILD_STATUS
 
       - name: Nix gosum diverging
-        uses: actions/github-script@v6
+        uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
         if: failure() && steps.build.outcome == 'failure'
         with:
           github-token: ${{secrets.GITHUB_TOKEN}}
@@ -64,7 +69,7 @@ jobs:
               body: 'Nix build failed with wrong gosum, please update "vendorSha256" (${{ steps.build.outputs.OLD_HASH }}) for the "headscale" package in flake.nix with the new SHA: ${{ steps.build.outputs.NEW_HASH }}'
             })
 
-      - uses: actions/upload-artifact@v4
+      - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
         if: steps.changed-files.outputs.files == 'true'
         with:
           name: headscale-linux
@@ -74,22 +79,27 @@ jobs:
     strategy:
       matrix:
         env:
-          - "GOARCH=arm   GOOS=linux GOARM=5"
-          - "GOARCH=arm   GOOS=linux GOARM=6"
-          - "GOARCH=arm   GOOS=linux GOARM=7"
           - "GOARCH=arm64 GOOS=linux"
-          - "GOARCH=386   GOOS=linux"
           - "GOARCH=amd64 GOOS=linux"
           - "GOARCH=arm64 GOOS=darwin"
           - "GOARCH=amd64 GOOS=darwin"
     steps:
-      - uses: actions/checkout@v4
-      - uses: DeterminateSystems/nix-installer-action@main
-      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: nixbuild/nix-quick-install-action@889f3180bb5f064ee9e3201428d04ae9e41d54ad # v31
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
+        with:
+          primary-key:
+            nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
+            '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
       - name: Run go cross compile
-        run: env ${{ matrix.env }} nix develop --command -- go build -o "headscale" ./cmd/headscale
-      - uses: actions/upload-artifact@v4
+        env:
+          CGO_ENABLED: 0
+        run:
+          env ${{ matrix.env }} nix develop --command -- go build -o "headscale"
+          ./cmd/headscale
+      - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
         with:
           name: "headscale-${{ matrix.env }}"
           path: "headscale"

.github/workflows/check-generated.yml

@@ -0,0 +1,55 @@
+name: Check Generated Files
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - main
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  check-generated:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        with:
+          fetch-depth: 2
+      - name: Get changed files
+        id: changed-files
+        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
+        with:
+          filters: |
+            files:
+              - '*.nix'
+              - 'go.*'
+              - '**/*.go'
+              - '**/*.proto'
+              - 'buf.gen.yaml'
+              - 'tools/**'
+      - uses: nixbuild/nix-quick-install-action@889f3180bb5f064ee9e3201428d04ae9e41d54ad # v31
+        if: steps.changed-files.outputs.files == 'true'
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
+        if: steps.changed-files.outputs.files == 'true'
+        with:
+          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix', '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
+
+      - name: Run make generate
+        if: steps.changed-files.outputs.files == 'true'
+        run: nix develop --command -- make generate
+
+      - name: Check for uncommitted changes
+        if: steps.changed-files.outputs.files == 'true'
+        run: |
+          if ! git diff --exit-code; then
+            echo "❌ Generated files are not up to date!"
+            echo "Please run 'make generate' and commit the changes."
+            exit 1
+          else
+            echo "✅ All generated files are up to date."
+          fi

.github/workflows/check-tests.yaml

@@ -10,12 +10,12 @@ jobs:
   check-tests:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
         with:
           fetch-depth: 2
       - name: Get changed files
         id: changed-files
-        uses: dorny/paths-filter@v3
+        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
         with:
           filters: |
             files:
@@ -24,10 +24,15 @@ jobs:
               - '**/*.go'
               - 'integration_test/'
               - 'config-example.yaml'
-      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: nixbuild/nix-quick-install-action@889f3180bb5f064ee9e3201428d04ae9e41d54ad # v31
         if: steps.changed-files.outputs.files == 'true'
-      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
         if: steps.changed-files.outputs.files == 'true'
+        with:
+          primary-key:
+            nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
+            '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
       - name: Generate and check integration tests
         if: steps.changed-files.outputs.files == 'true'

.github/workflows/docs-deploy.yml

@@ -21,15 +21,15 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
         with:
           fetch-depth: 0
       - name: Install python
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
         with:
           python-version: 3.x
       - name: Setup cache
-        uses: actions/cache@v4
+        uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3
         with:
           key: ${{ github.ref }}
           path: .cache

.github/workflows/docs-test.yml

@@ -11,13 +11,13 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
       - name: Install python
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
         with:
           python-version: 3.x
       - name: Setup cache
-        uses: actions/cache@v4
+        uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3
         with:
           key: ${{ github.ref }}
           path: .cache

.github/workflows/gh-action-integration-generator.go

@@ -38,11 +38,12 @@ func findTests() []string {
 	return tests
 }
 
-func updateYAML(tests []string, testPath string) {
+func updateYAML(tests []string, jobName string, testPath string) {
 	testsForYq := fmt.Sprintf("[%s]", strings.Join(tests, ", "))
 
 	yqCommand := fmt.Sprintf(
-		"yq eval '.jobs.integration-test.strategy.matrix.test = %s' %s -i",
+		"yq eval '.jobs.%s.strategy.matrix.test = %s' %s -i",
+		jobName,
 		testsForYq,
 		testPath,
 	)
@@ -59,7 +60,7 @@ func updateYAML(tests []string, testPath string) {
 		log.Fatalf("failed to run yq command: %s", err)
 	}
 
-	fmt.Printf("YAML file (%s) updated successfully\n", testPath)
+	fmt.Printf("YAML file (%s) job %s updated successfully\n", testPath, jobName)
 }
 
 func main() {
@@ -70,5 +71,21 @@ func main() {
 		quotedTests[i] = fmt.Sprintf("\"%s\"", test)
 	}
 
-	updateYAML(quotedTests, "./test-integration.yaml")
+	// Define selected tests for PostgreSQL
+	postgresTestNames := []string{
+		"TestACLAllowUserDst",
+		"TestPingAllByIP",
+		"TestEphemeral2006DeletedTooQuickly",
+		"TestPingAllByIPManyUpDown",
+		"TestSubnetRouterMultiNetwork",
+	}
+
+	quotedPostgresTests := make([]string, len(postgresTestNames))
+	for i, test := range postgresTestNames {
+		quotedPostgresTests[i] = fmt.Sprintf("\"%s\"", test)
+	}
+
+	// Update both SQLite and PostgreSQL job matrices
+	updateYAML(quotedTests, "sqlite", "./test-integration.yaml")
+	updateYAML(quotedPostgresTests, "postgres", "./test-integration.yaml")
 }

.github/workflows/gh-actions-updater.yaml

@@ -11,13 +11,13 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
         with:
           # [Required] Access token with `workflow` scope.
           token: ${{ secrets.WORKFLOW_SECRET }}
 
       - name: Run GitHub Actions Version Updater
-        uses: saadmk11/github-actions-version-updater@v0.8.1
+        uses: saadmk11/github-actions-version-updater@64be81ba69383f81f2be476703ea6570c4c8686e # v0.8.1
         with:
           # [Required] Access token with `workflow` scope.
           token: ${{ secrets.WORKFLOW_SECRET }}

.github/workflows/integration-test-template.yml

@@ -0,0 +1,82 @@
+name: Integration Test Template
+
+on:
+  workflow_call:
+    inputs:
+      test:
+        required: true
+        type: string
+      postgres_flag:
+        required: false
+        type: string
+        default: ""
+      database_name:
+        required: true
+        type: string
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    env:
+      # Github does not allow us to access secrets in pull requests,
+      # so this env var is used to check if we have the secret or not.
+      # If we have the secrets, meaning we are running on push in a fork,
+      # there might be secrets available for more debugging.
+      # If TS_OAUTH_CLIENT_ID and TS_OAUTH_SECRET is set, then the job
+      # will join a debug tailscale network, set up SSH and a tmux session.
+      # The SSH will be configured to use the SSH key of the Github user
+      # that triggered the build.
+      HAS_TAILSCALE_SECRET: ${{ secrets.TS_OAUTH_CLIENT_ID }}
+    steps:
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        with:
+          fetch-depth: 2
+      - name: Get changed files
+        id: changed-files
+        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
+        with:
+          filters: |
+            files:
+              - '*.nix'
+              - 'go.*'
+              - '**/*.go'
+              - 'integration_test/'
+              - 'config-example.yaml'
+      - name: Tailscale
+        if: ${{ env.HAS_TAILSCALE_SECRET }}
+        uses: tailscale/github-action@6986d2c82a91fbac2949fe01f5bab95cf21b5102 # v3.2.2
+        with:
+          oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
+          oauth-secret: ${{ secrets.TS_OAUTH_SECRET }}
+          tags: tag:gh
+      - name: Setup SSH server for Actor
+        if: ${{ env.HAS_TAILSCALE_SECRET }}
+        uses: alexellis/setup-sshd-actor@master
+      - uses: nixbuild/nix-quick-install-action@889f3180bb5f064ee9e3201428d04ae9e41d54ad # v31
+        if: steps.changed-files.outputs.files == 'true'
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
+        if: steps.changed-files.outputs.files == 'true'
+        with:
+          primary-key:
+            nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
+            '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
+      - name: Run Integration Test
+        if: always() && steps.changed-files.outputs.files == 'true'
+        run:
+          nix develop --command -- hi run --stats --ts-memory-limit=300 --hs-memory-limit=1500 "^${{ inputs.test }}$" \
+          --timeout=120m \
+          ${{ inputs.postgres_flag }}
+      - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
+        if: always() && steps.changed-files.outputs.files == 'true'
+        with:
+          name: ${{ inputs.database_name }}-${{ inputs.test }}-logs
+          path: "control_logs/*/*.log"
+      - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
+        if: always() && steps.changed-files.outputs.files == 'true'
+        with:
+          name: ${{ inputs.database_name }}-${{ inputs.test }}-archives
+          path: "control_logs/*/*.tar"
+      - name: Setup a blocking tmux session
+        if: ${{ env.HAS_TAILSCALE_SECRET }}
+        uses: alexellis/block-with-tmux-action@master

.github/workflows/lint.yml

@@ -10,12 +10,12 @@ jobs:
   golangci-lint:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
         with:
           fetch-depth: 2
       - name: Get changed files
         id: changed-files
-        uses: dorny/paths-filter@v3
+        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
         with:
           filters: |
             files:
@@ -24,24 +24,34 @@ jobs:
               - '**/*.go'
               - 'integration_test/'
               - 'config-example.yaml'
-      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: nixbuild/nix-quick-install-action@889f3180bb5f064ee9e3201428d04ae9e41d54ad # v31
         if: steps.changed-files.outputs.files == 'true'
-      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
         if: steps.changed-files.outputs.files == 'true'
+        with:
+          primary-key:
+            nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
+            '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
       - name: golangci-lint
         if: steps.changed-files.outputs.files == 'true'
-        run: nix develop --command -- golangci-lint run --new-from-rev=${{github.event.pull_request.base.sha}} --out-format=colored-line-number
+        run: nix develop --command -- golangci-lint run
+          --new-from-rev=${{github.event.pull_request.base.sha}}
+          --output.text.path=stdout
+          --output.text.print-linter-name
+          --output.text.print-issued-lines
+          --output.text.colors
 
   prettier-lint:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
         with:
           fetch-depth: 2
       - name: Get changed files
         id: changed-files
-        uses: dorny/paths-filter@v3
+        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
         with:
           filters: |
             files:
@@ -55,21 +65,32 @@ jobs:
               - '**/*.css'
               - '**/*.scss'
               - '**/*.html'
-      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: nixbuild/nix-quick-install-action@889f3180bb5f064ee9e3201428d04ae9e41d54ad # v31
         if: steps.changed-files.outputs.files == 'true'
-      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
         if: steps.changed-files.outputs.files == 'true'
+        with:
+          primary-key:
+            nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
+            '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
       - name: Prettify code
         if: steps.changed-files.outputs.files == 'true'
-        run: nix develop --command -- prettier --no-error-on-unmatched-pattern --ignore-unknown --check **/*.{ts,js,md,yaml,yml,sass,css,scss,html}
+        run: nix develop --command -- prettier --no-error-on-unmatched-pattern
+          --ignore-unknown --check **/*.{ts,js,md,yaml,yml,sass,css,scss,html}
 
   proto-lint:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v4
-      - uses: DeterminateSystems/nix-installer-action@main
-      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: nixbuild/nix-quick-install-action@889f3180bb5f064ee9e3201428d04ae9e41d54ad # v31
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
+        with:
+          primary-key:
+            nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
+            '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
       - name: Buf lint
         run: nix develop --command -- buf lint proto

.github/workflows/release.yml

@@ -13,25 +13,30 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout
-        uses: actions/checkout@v4
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
         with:
           fetch-depth: 0
 
       - name: Login to DockerHub
-        uses: docker/login-action@v3
+        uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
         with:
           username: ${{ secrets.DOCKERHUB_USERNAME }}
           password: ${{ secrets.DOCKERHUB_TOKEN }}
 
       - name: Login to GHCR
-        uses: docker/login-action@v3
+        uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
         with:
           registry: ghcr.io
           username: ${{ github.repository_owner }}
           password: ${{ secrets.GITHUB_TOKEN }}
 
-      - uses: DeterminateSystems/nix-installer-action@main
-      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: nixbuild/nix-quick-install-action@889f3180bb5f064ee9e3201428d04ae9e41d54ad # v31
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
+        with:
+          primary-key:
+            nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
+            '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
       - name: Run goreleaser
         run: nix develop --command -- goreleaser release --clean

.github/workflows/stale.yml

@@ -12,13 +12,17 @@ jobs:
       issues: write
       pull-requests: write
     steps:
-      - uses: actions/stale@v9
+      - uses: actions/stale@5bef64f19d7facfb25b37b414482c7164d639639 # v9.1.0
         with:
           days-before-issue-stale: 90
           days-before-issue-close: 7
           stale-issue-label: "stale"
-          stale-issue-message: "This issue is stale because it has been open for 90 days with no activity."
-          close-issue-message: "This issue was closed because it has been inactive for 14 days since being marked as stale."
+          stale-issue-message:
+            "This issue is stale because it has been open for 90 days with no
+            activity."
+          close-issue-message:
+            "This issue was closed because it has been inactive for 14 days
+            since being marked as stale."
           days-before-pr-stale: -1
           days-before-pr-close: -1
           exempt-issue-labels: "no-stale-bot"

.github/workflows/test-integration.yaml

@@ -1,4 +1,4 @@
-name: Integration Tests
+name: integration
 # To debug locally on a branch, and when needing secrets
 # change this to include `push` so the build is ran on
 # the main repository.
@@ -7,8 +7,7 @@ concurrency:
   group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
   cancel-in-progress: true
 jobs:
-  integration-test:
-    runs-on: ubuntu-latest
+  sqlite:
     strategy:
       fail-fast: false
       matrix:
@@ -22,6 +21,14 @@ jobs:
           - TestACLNamedHostsCanReach
           - TestACLDevice1CanAccessDevice2
           - TestPolicyUpdateWhileRunningWithCLIInDatabase
+          - TestACLAutogroupMember
+          - TestACLAutogroupTagged
+          - TestACLAutogroupSelf
+          - TestACLPolicyPropagationOverTime
+          - TestAPIAuthenticationBypass
+          - TestAPIAuthenticationBypassCurl
+          - TestGRPCAuthenticationBypass
+          - TestCLIWithConfigAuthenticationBypass
           - TestAuthKeyLogoutAndReloginSameUser
           - TestAuthKeyLogoutAndReloginNewUser
           - TestAuthKeyLogoutAndReloginSameUserExpiredKey
@@ -30,8 +37,11 @@ jobs:
           - TestOIDC024UserCreation
           - TestOIDCAuthenticationWithPKCE
           - TestOIDCReloginSameNodeNewUser
+          - TestOIDCFollowUpUrl
+          - TestOIDCReloginSameNodeSameUser
           - TestAuthWebFlowAuthenticationPingAll
-          - TestAuthWebFlowLogoutAndRelogin
+          - TestAuthWebFlowLogoutAndReloginSameUser
+          - TestAuthWebFlowLogoutAndReloginNewUser
           - TestUserCommand
           - TestPreAuthKeyCommand
           - TestPreAuthKeyCommandWithoutExpiry
@@ -78,91 +88,24 @@ jobs:
           - TestSSHNoSSHConfigured
           - TestSSHIsBlockedInACL
           - TestSSHUserOnlyIsolation
-        database: [postgres, sqlite]
-    env:
-      # Github does not allow us to access secrets in pull requests,
-      # so this env var is used to check if we have the secret or not.
-      # If we have the secrets, meaning we are running on push in a fork,
-      # there might be secrets available for more debugging.
-      # If TS_OAUTH_CLIENT_ID and TS_OAUTH_SECRET is set, then the job
-      # will join a debug tailscale network, set up SSH and a tmux session.
-      # The SSH will be configured to use the SSH key of the Github user
-      # that triggered the build.
-      HAS_TAILSCALE_SECRET: ${{ secrets.TS_OAUTH_CLIENT_ID }}
-    steps:
-      - uses: actions/checkout@v4
-        with:
-          fetch-depth: 2
-      - name: Get changed files
-        id: changed-files
-        uses: dorny/paths-filter@v3
-        with:
-          filters: |
-            files:
-              - '*.nix'
-              - 'go.*'
-              - '**/*.go'
-              - 'integration_test/'
-              - 'config-example.yaml'
-      - name: Tailscale
-        if: ${{ env.HAS_TAILSCALE_SECRET }}
-        uses: tailscale/github-action@v2
-        with:
-          oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
-          oauth-secret: ${{ secrets.TS_OAUTH_SECRET }}
-          tags: tag:gh
-      - name: Setup SSH server for Actor
-        if: ${{ env.HAS_TAILSCALE_SECRET }}
-        uses: alexellis/setup-sshd-actor@master
-      - uses: DeterminateSystems/nix-installer-action@main
-        if: steps.changed-files.outputs.files == 'true'
-      - uses: DeterminateSystems/magic-nix-cache-action@main
-        if: steps.changed-files.outputs.files == 'true'
-      - uses: satackey/action-docker-layer-caching@main
-        if: steps.changed-files.outputs.files == 'true'
-        continue-on-error: true
-      - name: Run Integration Test
-        uses: Wandalen/wretry.action@master
-        if: steps.changed-files.outputs.files == 'true'
-        env:
-          USE_POSTGRES: ${{ matrix.database == 'postgres' && '1' || '0' }}
-        with:
-          # Our integration tests are started like a thundering herd, often
-          # hitting limits of the various external repositories we depend on
-          # like docker hub. This will retry jobs every 5 min, 10 times,
-          # hopefully letting us avoid manual intervention and restarting jobs.
-          # One could of course argue that we should invest in trying to avoid
-          # this, but currently it seems like a larger investment to be cleverer
-          # about this.
-          # Some of the jobs might still require manual restart as they are really
-          # slow and this will cause them to eventually be killed by Github actions.
-          attempt_delay: 300000 # 5 min
-          attempt_limit: 10
-          command: |
-            nix develop --command -- docker run \
-              --tty --rm \
-              --volume ~/.cache/hs-integration-go:/go \
-              --name headscale-test-suite \
-              --volume $PWD:$PWD -w $PWD/integration \
-              --volume /var/run/docker.sock:/var/run/docker.sock \
-              --volume $PWD/control_logs:/tmp/control \
-              --env HEADSCALE_INTEGRATION_POSTGRES=${{env.USE_POSTGRES}} \
-              golang:1 \
-                go run gotest.tools/gotestsum@latest -- ./... \
-                  -failfast \
-                  -timeout 120m \
-                  -parallel 1 \
-                  -run "^${{ matrix.test }}$"
-      - uses: actions/upload-artifact@v4
-        if: always() && steps.changed-files.outputs.files == 'true'
-        with:
-          name: ${{ matrix.test }}-${{matrix.database}}-logs
-          path: "control_logs/*.log"
-      - uses: actions/upload-artifact@v4
-        if: always() && steps.changed-files.outputs.files == 'true'
-        with:
-          name: ${{ matrix.test }}-${{matrix.database}}-pprof
-          path: "control_logs/*.pprof.tar"
-      - name: Setup a blocking tmux session
-        if: ${{ env.HAS_TAILSCALE_SECRET }}
-        uses: alexellis/block-with-tmux-action@master
+          - TestSSHAutogroupSelf
+    uses: ./.github/workflows/integration-test-template.yml
+    with:
+      test: ${{ matrix.test }}
+      postgres_flag: "--postgres=0"
+      database_name: "sqlite"
+  postgres:
+    strategy:
+      fail-fast: false
+      matrix:
+        test:
+          - TestACLAllowUserDst
+          - TestPingAllByIP
+          - TestEphemeral2006DeletedTooQuickly
+          - TestPingAllByIPManyUpDown
+          - TestSubnetRouterMultiNetwork
+    uses: ./.github/workflows/integration-test-template.yml
+    with:
+      test: ${{ matrix.test }}
+      postgres_flag: "--postgres=1"
+      database_name: "postgres"

.github/workflows/test.yml

@@ -11,13 +11,13 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
         with:
           fetch-depth: 2
 
       - name: Get changed files
         id: changed-files
-        uses: dorny/paths-filter@v3
+        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
         with:
           filters: |
             files:
@@ -27,10 +27,15 @@ jobs:
               - 'integration_test/'
               - 'config-example.yaml'
 
-      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: nixbuild/nix-quick-install-action@889f3180bb5f064ee9e3201428d04ae9e41d54ad # v31
         if: steps.changed-files.outputs.files == 'true'
-      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
         if: steps.changed-files.outputs.files == 'true'
+        with:
+          primary-key:
+            nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
+            '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
       - name: Run tests
         if: steps.changed-files.outputs.files == 'true'

.github/workflows/update-flake.yml

@@ -10,10 +10,10 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
       - name: Install Nix
-        uses: DeterminateSystems/nix-installer-action@main
+        uses: DeterminateSystems/nix-installer-action@21a544727d0c62386e78b4befe52d19ad12692e3 # v17
       - name: Update flake.lock
-        uses: DeterminateSystems/update-flake-lock@main
+        uses: DeterminateSystems/update-flake-lock@428c2b58a4b7414dabd372acb6a03dba1084d3ab # v25
         with:
           pr-title: "Update flake.lock"

.gitignore

@@ -1,6 +1,9 @@
 ignored/
 tailscale/
 .vscode/
+.claude/
+
+*.prof
 
 # Binaries for programs and plugins
 *.exe
@@ -20,9 +23,9 @@ vendor/
 
 dist/
 /headscale
-config.json
 config.yaml
 config*.yaml
+!config-example.yaml
 derp.yaml
 *.hujson
 *.key
@@ -46,3 +49,7 @@ integration_test/etc/config.dump.yaml
 /site
 
 __debug_bin
+
+node_modules/
+package-lock.json
+package.json

.golangci.yaml

@@ -24,6 +24,7 @@ linters:
     - revive
     - tagliatelle
     - testpackage
+    - varnamelen
     - wrapcheck
     - wsl
   settings:

.goreleaser.yml

@@ -2,11 +2,39 @@
 version: 2
 before:
   hooks:
-    - go mod tidy -compat=1.24
+    - go mod tidy -compat=1.25
     - go mod vendor
 
 release:
   prerelease: auto
+  draft: true
+  header: |
+    ## Upgrade
+
+    Please follow the steps outlined in the [upgrade guide](https://headscale.net/stable/setup/upgrade/) to update your existing Headscale installation.
+
+    **It's best to update from one stable version to the next** (e.g., 0.24.0 → 0.25.1 → 0.26.1) in case you are multiple releases behind. You should always pick the latest available patch release.
+
+    Be sure to check the changelog above for version-specific upgrade instructions and breaking changes.
+
+    ### Backup Your Database
+
+    **Always backup your database before upgrading.** Here's how to backup a SQLite database:
+
+    ```bash
+    # Stop headscale
+    systemctl stop headscale
+
+    # Backup sqlite database
+    cp /var/lib/headscale/db.sqlite /var/lib/headscale/db.sqlite.backup
+
+    # Backup sqlite WAL/SHM files (if they exist)
+    cp /var/lib/headscale/db.sqlite-wal /var/lib/headscale/db.sqlite-wal.backup
+    cp /var/lib/headscale/db.sqlite-shm /var/lib/headscale/db.sqlite-shm.backup
+
+    # Start headscale (migration will run automatically)
+    systemctl start headscale
+    ```
 
 builds:
   - id: headscale
@@ -18,18 +46,10 @@ builds:
       - darwin_amd64
       - darwin_arm64
       - freebsd_amd64
-      - linux_386
       - linux_amd64
       - linux_arm64
-      - linux_arm_5
-      - linux_arm_6
-      - linux_arm_7
     flags:
       - -mod=readonly
-    ldflags:
-      - -s -w
-      - -X github.com/juanfont/headscale/hscontrol/types.Version={{ .Version }}
-      - -X github.com/juanfont/headscale/hscontrol/types.GitCommitHash={{ .Commit }}
     tags:
       - ts2019
 
@@ -63,8 +83,15 @@ nfpms:
     vendor: headscale
     maintainer: Kristoffer Dalby <kristoffer@dalby.cc>
     homepage: https://github.com/juanfont/headscale
-    license: BSD
+    description: |-
+      Open source implementation of the Tailscale control server.
+      Headscale aims to implement a self-hosted, open source alternative to the
+      Tailscale control server. Headscale's goal is to provide self-hosters and
+      hobbyists with an open-source server they can use for their projects and
+      labs. It implements a narrow scope, a single Tailscale network (tailnet),
+      suitable for a personal use, or a small open-source organisation.
     bindir: /usr/bin
+    section: net
     formats:
       - deb
     contents:
@@ -73,15 +100,21 @@ nfpms:
         type: config|noreplace
         file_info:
           mode: 0644
-      - src: ./docs/packaging/headscale.systemd.service
+      - src: ./packaging/systemd/headscale.service
         dst: /usr/lib/systemd/system/headscale.service
       - dst: /var/lib/headscale
         type: dir
-      - dst: /var/run/headscale
-        type: dir
+      - src: LICENSE
+        dst: /usr/share/doc/headscale/copyright
     scripts:
-      postinstall: ./docs/packaging/postinstall.sh
-      postremove: ./docs/packaging/postremove.sh
+      postinstall: ./packaging/deb/postinst
+      postremove: ./packaging/deb/postrm
+      preremove: ./packaging/deb/prerm
+    deb:
+      lintian_overrides:
+        - no-changelog # Our CHANGELOG.md uses a different formatting
+        - no-manual-page
+        - statically-linked-binary
 
 kos:
   - id: ghcr
@@ -99,9 +132,7 @@ kos:
       - CGO_ENABLED=0
     platforms:
       - linux/amd64
-      - linux/386
       - linux/arm64
-      - linux/arm/v7
     tags:
       - "{{ if not .Prerelease }}latest{{ end }}"
       - "{{ if not .Prerelease }}{{ .Major }}.{{ .Minor }}.{{ .Patch }}{{ end }}"
@@ -114,6 +145,8 @@ kos:
       - "{{ .Tag }}"
       - '{{ trimprefix .Tag "v" }}'
       - "sha-{{ .ShortCommit }}"
+    creation_time: "{{.CommitTimestamp}}"
+    ko_data_creation_time: "{{.CommitTimestamp}}"
 
   - id: ghcr-debug
     repositories:
@@ -128,9 +161,7 @@ kos:
       - CGO_ENABLED=0
     platforms:
       - linux/amd64
-      - linux/386
       - linux/arm64
-      - linux/arm/v7
     tags:
       - "{{ if not .Prerelease }}latest-debug{{ end }}"
       - "{{ if not .Prerelease }}{{ .Major }}.{{ .Minor }}.{{ .Patch }}-debug{{ end }}"

.mcp.json

@@ -0,0 +1,48 @@
+{
+  "mcpServers": {
+    "claude-code-mcp": {
+      "type": "stdio",
+      "command": "npx",
+      "args": [
+        "-y",
+        "@steipete/claude-code-mcp@latest"
+      ],
+      "env": {}
+    },
+    "sequential-thinking": {
+      "type": "stdio",
+      "command": "npx",
+      "args": [
+        "-y",
+        "@modelcontextprotocol/server-sequential-thinking"
+      ],
+      "env": {}
+    },
+    "nixos": {
+      "type": "stdio",
+      "command": "uvx",
+      "args": [
+        "mcp-nixos"
+      ],
+      "env": {}
+    },
+    "context7": {
+      "type": "stdio",
+      "command": "npx",
+      "args": [
+        "-y",
+        "@upstash/context7-mcp"
+      ],
+      "env": {}
+    },
+    "git": {
+      "type": "stdio",
+      "command": "npx",
+      "args": [
+        "-y",
+        "@cyanheads/git-mcp-server"
+      ],
+      "env": {}
+    }
+  }
+}

.prettierignore

@@ -1,4 +1,5 @@
 .github/workflows/test-integration-v2*
 docs/about/features.md
 docs/ref/configuration.md
+docs/ref/oidc.md
 docs/ref/remote-cli.md

CHANGELOG.md

@@ -2,6 +2,152 @@
 
 ## Next
 
+## 0.27.0 (2025-xx-xx)
+
+**Minimum supported Tailscale client version: v1.64.0**
+
+### Database integrity improvements
+
+This release includes a significant database migration that addresses
+longstanding issues with the database schema and data integrity that has
+accumulated over the years. The migration introduces a `schema.sql` file as the
+source of truth for the expected database schema to ensure new migrations that
+will cause divergence does not occur again.
+
+These issues arose from a combination of factors discovered over time: SQLite
+foreign keys not being enforced for many early versions, all migrations being
+run in one large function until version 0.23.0, and inconsistent use of GORM's
+AutoMigrate feature. Moving forward, all new migrations will be explicit SQL
+operations rather than relying on GORM AutoMigrate, and foreign keys will be
+enforced throughout the migration process.
+
+We are only improving SQLite databases with this change - PostgreSQL databases
+are not affected.
+
+Please read the
+[PR description](https://github.com/juanfont/headscale/pull/2617) for more
+technical details about the issues and solutions.
+
+**SQLite Database Backup Example:**
+
+```bash
+# Stop headscale
+systemctl stop headscale
+
+# Backup sqlite database
+cp /var/lib/headscale/db.sqlite /var/lib/headscale/db.sqlite.backup
+
+# Backup sqlite WAL/SHM files (if they exist)
+cp /var/lib/headscale/db.sqlite-wal /var/lib/headscale/db.sqlite-wal.backup
+cp /var/lib/headscale/db.sqlite-shm /var/lib/headscale/db.sqlite-shm.backup
+
+# Start headscale (migration will run automatically)
+systemctl start headscale
+```
+
+### DERPMap update frequency
+
+The default DERPMap update frequency has been changed from 24 hours to 3 hours.
+If you set the `derp.update_frequency` configuration option, it is recommended
+to change it to `3h` to ensure that the headscale instance gets the latest
+DERPMap updates when upstream is changed.
+
+### Autogroups
+
+This release adds support for the three missing autogroups: `self`
+(experimental), `member`, and `tagged`. Please refer to the
+[documentation](https://tailscale.com/kb/1018/autogroups/) for a detailed
+explanation.
+
+`autogroup:self` is marked as experimental and should be used with caution, but
+we need help testing it. Experimental here means two things; first, generating
+the packet filter from policies that use `autogroup:self` is very expensive, and
+it might perform, or straight up not work on Headscale installations with a
+large number of nodes. Second, the implementation might have bugs or edge cases
+we are not aware of, meaning that nodes or users might gain _more_ access than
+expected. Please report bugs.
+
+### Node store (in memory database)
+
+Under the hood, we have added a new datastructure to store nodes in memory. This
+datastructure is called `NodeStore` and aims to reduce the reading and writing
+of nodes to the database layer. We have not benchmarked it, but expect it to
+improve performance for read heavy workloads. We think of it as, "worst case" we
+have moved the bottle neck somewhere else, and "best case" we should see a good
+improvement in compute resource usage at the expense of memory usage. We are
+quite excited for this change and think it will make it easier for us to improve
+the code base over time and make it more correct and efficient.
+
+### BREAKING
+
+- Remove support for 32-bit binaries
+  [#2692](https://github.com/juanfont/headscale/pull/2692)
+- Policy: Zero or empty destination port is no longer allowed
+  [#2606](https://github.com/juanfont/headscale/pull/2606)
+- Stricter hostname validation [#2383](https://github.com/juanfont/headscale/pull/2383)
+  - Hostnames must be valid DNS labels (2-63 characters, alphanumeric and
+    hyphens only, cannot start/end with hyphen)
+  - **Client Registration (New Nodes)**: Invalid hostnames are automatically
+    renamed to `invalid-XXXXXX` format
+    - `my-laptop` → accepted as-is
+    - `My-Laptop` → `my-laptop` (lowercased)
+    - `my_laptop` → `invalid-a1b2c3` (underscore not allowed)
+    - `test@host` → `invalid-d4e5f6` (@ not allowed)
+    - `laptop-🚀` → `invalid-j1k2l3` (emoji not allowed)
+  - **Hostinfo Updates / CLI**: Invalid hostnames are rejected with an error
+    - Valid names are accepted or lowercased
+    - Names with invalid characters, too short (<2), too long (>63), or
+      starting/ending with hyphen are rejected
+
+### Changes
+
+- **Database schema migration improvements for SQLite**
+  [#2617](https://github.com/juanfont/headscale/pull/2617)
+  - **IMPORTANT: Backup your SQLite database before upgrading**
+  - Introduces safer table renaming migration strategy
+  - Addresses longstanding database integrity issues
+- Add flag to directly manipulate the policy in the database
+  [#2765](https://github.com/juanfont/headscale/pull/2765)
+- DERPmap update frequency default changed from 24h to 3h
+  [#2741](https://github.com/juanfont/headscale/pull/2741)
+- DERPmap update mechanism has been improved with retry, and is now failing
+  conservatively, preserving the old map upon failure.
+  [#2741](https://github.com/juanfont/headscale/pull/2741)
+- Add support for `autogroup:member`, `autogroup:tagged`
+  [#2572](https://github.com/juanfont/headscale/pull/2572)
+- Fix bug where return routes were being removed by policy
+  [#2767](https://github.com/juanfont/headscale/pull/2767)
+- Remove policy v1 code [#2600](https://github.com/juanfont/headscale/pull/2600)
+- Refactor Debian/Ubuntu packaging and drop support for Ubuntu 20.04.
+  [#2614](https://github.com/juanfont/headscale/pull/2614)
+- Remove redundant check regarding `noise` config
+  [#2658](https://github.com/juanfont/headscale/pull/2658)
+- Refactor OpenID Connect documentation
+  [#2625](https://github.com/juanfont/headscale/pull/2625)
+- Don't crash if config file is missing
+  [#2656](https://github.com/juanfont/headscale/pull/2656)
+- Adds `/robots.txt` endpoint to avoid crawlers
+  [#2643](https://github.com/juanfont/headscale/pull/2643)
+- OIDC: Use group claim from UserInfo
+  [#2663](https://github.com/juanfont/headscale/pull/2663)
+- OIDC: Update user with claims from UserInfo _before_ comparing with allowed
+  groups, email and domain
+  [#2663](https://github.com/juanfont/headscale/pull/2663)
+- Policy will now reject invalid fields, making it easier to spot spelling
+  errors [#2764](https://github.com/juanfont/headscale/pull/2764)
+- Add FAQ entry on how to recover from an invalid policy in the database
+  [#2776](https://github.com/juanfont/headscale/pull/2776)
+- EXPERIMENTAL: Add support for `autogroup:self`
+  [#2789](https://github.com/juanfont/headscale/pull/2789)
+- Add healthcheck command [#2659](https://github.com/juanfont/headscale/pull/2659)
+
+## 0.26.1 (2025-06-06)
+
+### Changes
+
+- Ensure nodes are matching both node key and machine key when connecting.
+  [#2642](https://github.com/juanfont/headscale/pull/2642)
+
 ## 0.26.0 (2025-05-14)
 
 ### BREAKING
@@ -60,7 +206,7 @@ new policy code passes all of our tests.
 - Error messages should be more descriptive and informative.
   - There is still work to be here, but it is already improved with "typing"
     (e.g. only Users can be put in Groups)
-- All users must contain an `@` character.
+- All users in the policy must contain an `@` character.
   - If your user naturally contains and `@`, like an email, this will just work.
   - If its based on usernames, or other identifiers not containing an `@`, an
     `@` should be appended at the end. For example, if your user is `john`, it
@@ -117,6 +263,11 @@ working in v1 and not tested might be broken in v2 (and vice versa).
   [#2542](https://github.com/juanfont/headscale/pull/2542)
 - Pre auth key API/CLI now uses ID over username
   [#2542](https://github.com/juanfont/headscale/pull/2542)
+- A non-empty list of global nameservers needs to be specified via
+  `dns.nameservers.global` if the configuration option `dns.override_local_dns`
+  is enabled or is not specified in the configuration file. This aligns with
+  behaviour of tailscale.com.
+  [#2438](https://github.com/juanfont/headscale/pull/2438)
 
 ### Changes
 

CLAUDE.md

@@ -0,0 +1,531 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Overview
+
+Headscale is an open-source implementation of the Tailscale control server written in Go. It provides self-hosted coordination for Tailscale networks (tailnets), managing node registration, IP allocation, policy enforcement, and DERP routing.
+
+## Development Commands
+
+### Quick Setup
+```bash
+# Recommended: Use Nix for dependency management
+nix develop
+
+# Full development workflow
+make dev  # runs fmt + lint + test + build
+```
+
+### Essential Commands
+```bash
+# Build headscale binary
+make build
+
+# Run tests
+make test
+go test ./...                    # All unit tests
+go test -race ./...              # With race detection
+
+# Run specific integration test
+go run ./cmd/hi run "TestName" --postgres
+
+# Code formatting and linting
+make fmt         # Format all code (Go, docs, proto)
+make lint        # Lint all code (Go, proto)
+make fmt-go      # Format Go code only
+make lint-go     # Lint Go code only
+
+# Protocol buffer generation (after modifying proto/)
+make generate
+
+# Clean build artifacts  
+make clean
+```
+
+### Integration Testing
+```bash
+# Use the hi (Headscale Integration) test runner
+go run ./cmd/hi doctor                    # Check system requirements
+go run ./cmd/hi run "TestPattern"         # Run specific test
+go run ./cmd/hi run "TestPattern" --postgres  # With PostgreSQL backend
+
+# Test artifacts are saved to control_logs/ with logs and debug data
+```
+
+## Project Structure & Architecture
+
+### Top-Level Organization
+
+```
+headscale/
+├── cmd/                    # Command-line applications
+│   ├── headscale/         # Main headscale server binary
+│   └── hi/               # Headscale Integration test runner
+├── hscontrol/            # Core control plane logic
+├── integration/          # End-to-end Docker-based tests
+├── proto/               # Protocol buffer definitions
+├── gen/                 # Generated code (protobuf)
+├── docs/                # Documentation
+└── packaging/           # Distribution packaging
+```
+
+### Core Packages (`hscontrol/`)
+
+**Main Server (`hscontrol/`)**
+- `app.go`: Application setup, dependency injection, server lifecycle
+- `handlers.go`: HTTP/gRPC API endpoints for management operations
+- `grpcv1.go`: gRPC service implementation for headscale API
+- `poll.go`: **Critical** - Handles Tailscale MapRequest/MapResponse protocol
+- `noise.go`: Noise protocol implementation for secure client communication
+- `auth.go`: Authentication flows (web, OIDC, command-line)
+- `oidc.go`: OpenID Connect integration for user authentication
+
+**State Management (`hscontrol/state/`)**
+- `state.go`: Central coordinator for all subsystems (database, policy, IP allocation, DERP)
+- `node_store.go`: **Performance-critical** - In-memory cache with copy-on-write semantics
+- Thread-safe operations with deadlock detection
+- Coordinates between database persistence and real-time operations
+
+**Database Layer (`hscontrol/db/`)**
+- `db.go`: Database abstraction, GORM setup, migration management
+- `node.go`: Node lifecycle, registration, expiration, IP assignment
+- `users.go`: User management, namespace isolation
+- `api_key.go`: API authentication tokens
+- `preauth_keys.go`: Pre-authentication keys for automated node registration
+- `ip.go`: IP address allocation and management
+- `policy.go`: Policy storage and retrieval
+- Schema migrations in `schema.sql` with extensive test data coverage
+
+**Policy Engine (`hscontrol/policy/`)**
+- `policy.go`: Core ACL evaluation logic, HuJSON parsing
+- `v2/`: Next-generation policy system with improved filtering
+- `matcher/`: ACL rule matching and evaluation engine
+- Determines peer visibility, route approval, and network access rules
+- Supports both file-based and database-stored policies
+
+**Network Management (`hscontrol/`)**
+- `derp/`: DERP (Designated Encrypted Relay for Packets) server implementation
+  - NAT traversal when direct connections fail
+  - Fallback relay for firewall-restricted environments
+- `mapper/`: Converts internal Headscale state to Tailscale's wire protocol format
+  - `tail.go`: Tailscale-specific data structure generation
+- `routes/`: Subnet route management and primary route selection
+- `dns/`: DNS record management and MagicDNS implementation
+
+**Utilities & Support (`hscontrol/`)**
+- `types/`: Core data structures, configuration, validation
+- `util/`: Helper functions for networking, DNS, key management
+- `templates/`: Client configuration templates (Apple, Windows, etc.)
+- `notifier/`: Event notification system for real-time updates
+- `metrics.go`: Prometheus metrics collection
+- `capver/`: Tailscale capability version management
+
+### Key Subsystem Interactions
+
+**Node Registration Flow**
+1. **Client Connection**: `noise.go` handles secure protocol handshake
+2. **Authentication**: `auth.go` validates credentials (web/OIDC/preauth)
+3. **State Creation**: `state.go` coordinates IP allocation via `db/ip.go`
+4. **Storage**: `db/node.go` persists node, `NodeStore` caches in memory
+5. **Network Setup**: `mapper/` generates initial Tailscale network map
+
+**Ongoing Operations**
+1. **Poll Requests**: `poll.go` receives periodic client updates
+2. **State Updates**: `NodeStore` maintains real-time node information
+3. **Policy Application**: `policy/` evaluates ACL rules for peer relationships
+4. **Map Distribution**: `mapper/` sends network topology to all affected clients
+
+**Route Management**
+1. **Advertisement**: Clients announce routes via `poll.go` Hostinfo updates
+2. **Storage**: `db/` persists routes, `NodeStore` caches for performance
+3. **Approval**: `policy/` auto-approves routes based on ACL rules
+4. **Distribution**: `routes/` selects primary routes, `mapper/` distributes to peers
+
+### Command-Line Tools (`cmd/`)
+
+**Main Server (`cmd/headscale/`)**
+- `headscale.go`: CLI parsing, configuration loading, server startup
+- Supports daemon mode, CLI operations (user/node management), database operations
+
+**Integration Test Runner (`cmd/hi/`)**
+- `main.go`: Test execution framework with Docker orchestration
+- `run.go`: Individual test execution with artifact collection
+- `doctor.go`: System requirements validation
+- `docker.go`: Container lifecycle management
+- Essential for validating changes against real Tailscale clients
+
+### Generated & External Code
+
+**Protocol Buffers (`proto/` → `gen/`)**
+- Defines gRPC API for headscale management operations
+- Client libraries can generate from these definitions
+- Run `make generate` after modifying `.proto` files
+
+**Integration Testing (`integration/`)**
+- `scenario.go`: Docker test environment setup
+- `tailscale.go`: Tailscale client container management
+- Individual test files for specific functionality areas
+- Real end-to-end validation with network isolation
+
+### Critical Performance Paths
+
+**High-Frequency Operations**
+1. **MapRequest Processing** (`poll.go`): Every 15-60 seconds per client
+2. **NodeStore Reads** (`node_store.go`): Every operation requiring node data
+3. **Policy Evaluation** (`policy/`): On every peer relationship calculation
+4. **Route Lookups** (`routes/`): During network map generation
+
+**Database Write Patterns**
+- **Frequent**: Node heartbeats, endpoint updates, route changes
+- **Moderate**: User operations, policy updates, API key management
+- **Rare**: Schema migrations, bulk operations
+
+### Configuration & Deployment
+
+**Configuration** (`hscontrol/types/config.go`)**
+- Database connection settings (SQLite/PostgreSQL)
+- Network configuration (IP ranges, DNS settings)
+- Policy mode (file vs database)
+- DERP relay configuration
+- OIDC provider settings
+
+**Key Dependencies**
+- **GORM**: Database ORM with migration support
+- **Tailscale Libraries**: Core networking and protocol code
+- **Zerolog**: Structured logging throughout the application
+- **Buf**: Protocol buffer toolchain for code generation
+
+### Development Workflow Integration
+
+The architecture supports incremental development:
+- **Unit Tests**: Focus on individual packages (`*_test.go` files)
+- **Integration Tests**: Validate cross-component interactions
+- **Database Tests**: Extensive migration and data integrity validation
+- **Policy Tests**: ACL rule evaluation and edge cases
+- **Performance Tests**: NodeStore and high-frequency operation validation
+
+## Integration Testing System
+
+### Overview
+Headscale uses Docker-based integration tests with real Tailscale clients to validate end-to-end functionality. The integration test system is complex and requires specialized knowledge for effective execution and debugging.
+
+### **MANDATORY: Use the headscale-integration-tester Agent**
+
+**CRITICAL REQUIREMENT**: For ANY integration test execution, analysis, troubleshooting, or validation, you MUST use the `headscale-integration-tester` agent. This agent contains specialized knowledge about:
+
+- Test execution strategies and timing requirements  
+- Infrastructure vs code issue distinction (99% vs 1% failure patterns)
+- Security-critical debugging rules and forbidden practices
+- Comprehensive artifact analysis workflows
+- Real-world failure patterns from HA debugging experiences
+
+### Quick Reference Commands
+
+```bash
+# Check system requirements (always run first)
+go run ./cmd/hi doctor
+
+# Run single test (recommended for development)  
+go run ./cmd/hi run "TestName"
+
+# Use PostgreSQL for database-heavy tests
+go run ./cmd/hi run "TestName" --postgres
+
+# Pattern matching for related tests
+go run ./cmd/hi run "TestPattern*"
+```
+
+**Critical Notes**:
+- Only ONE test can run at a time (Docker port conflicts)
+- Tests generate ~100MB of logs per run in `control_logs/`
+- Clean environment before each test: `rm -rf control_logs/202507* && docker system prune -f`
+
+### Test Artifacts Location
+All test runs save comprehensive debugging artifacts to `control_logs/TIMESTAMP-ID/` including server logs, client logs, database dumps, MapResponse protocol data, and Prometheus metrics.
+
+**For all integration test work, use the headscale-integration-tester agent - it contains the complete knowledge needed for effective testing and debugging.**
+
+## NodeStore Implementation Details
+
+**Key Insight from Recent Work**: The NodeStore is a critical performance optimization that caches node data in memory while ensuring consistency with the database. When working with route advertisements or node state changes:
+
+1. **Timing Considerations**: Route advertisements need time to propagate from clients to server. Use `require.EventuallyWithT()` patterns in tests instead of immediate assertions.
+
+2. **Synchronization Points**: NodeStore updates happen at specific points like `poll.go:420` after Hostinfo changes. Ensure these are maintained when modifying the polling logic.
+
+3. **Peer Visibility**: The NodeStore's `peersFunc` determines which nodes are visible to each other. Policy-based filtering is separate from monitoring visibility - expired nodes should remain visible for debugging but marked as expired.
+
+## Testing Guidelines
+
+### Integration Test Patterns
+
+#### **CRITICAL: EventuallyWithT Pattern for External Calls**
+
+**All external calls in integration tests MUST be wrapped in EventuallyWithT blocks** to handle eventual consistency in distributed systems. External calls include:
+- `client.Status()` - Getting Tailscale client status
+- `client.Curl()` - Making HTTP requests through clients
+- `client.Traceroute()` - Running network diagnostics
+- `headscale.ListNodes()` - Querying headscale server state
+- Any other calls that interact with external systems or network operations
+
+**Key Rules**:
+1. **Never use bare `require.NoError(t, err)` with external calls** - Always wrap in EventuallyWithT
+2. **Keep related assertions together** - If multiple assertions depend on the same external call, keep them in the same EventuallyWithT block
+3. **Split unrelated external calls** - Different external calls should be in separate EventuallyWithT blocks
+4. **Never nest EventuallyWithT calls** - Each EventuallyWithT should be at the same level
+5. **Declare shared variables at function scope** - Variables used across multiple EventuallyWithT blocks must be declared before first use
+
+**Examples**:
+
+```go
+// CORRECT: External call wrapped in EventuallyWithT
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    status, err := client.Status()
+    assert.NoError(c, err)
+    
+    // Related assertions using the same status call
+    for _, peerKey := range status.Peers() {
+        peerStatus := status.Peer[peerKey]
+        assert.NotNil(c, peerStatus.PrimaryRoutes)
+        requirePeerSubnetRoutesWithCollect(c, peerStatus, expectedRoutes)
+    }
+}, 5*time.Second, 200*time.Millisecond, "Verifying client status and routes")
+
+// INCORRECT: Bare external call without EventuallyWithT
+status, err := client.Status()  // ❌ Will fail intermittently
+require.NoError(t, err)
+
+// CORRECT: Separate EventuallyWithT for different external calls
+// First external call - headscale.ListNodes()
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    assert.Len(c, nodes, 2)
+    requireNodeRouteCountWithCollect(c, nodes[0], 2, 2, 2)
+}, 10*time.Second, 500*time.Millisecond, "route state changes should propagate to nodes")
+
+// Second external call - client.Status() 
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    status, err := client.Status()
+    assert.NoError(c, err)
+    
+    for _, peerKey := range status.Peers() {
+        peerStatus := status.Peer[peerKey]
+        requirePeerSubnetRoutesWithCollect(c, peerStatus, []netip.Prefix{tsaddr.AllIPv4(), tsaddr.AllIPv6()})
+    }
+}, 10*time.Second, 500*time.Millisecond, "routes should be visible to client")
+
+// INCORRECT: Multiple unrelated external calls in same EventuallyWithT
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()  // ❌ First external call
+    assert.NoError(c, err)
+    
+    status, err := client.Status()  // ❌ Different external call - should be separate
+    assert.NoError(c, err)
+}, 10*time.Second, 500*time.Millisecond, "mixed calls")
+
+// CORRECT: Variable scoping for shared data
+var (
+    srs1, srs2, srs3       *ipnstate.Status
+    clientStatus           *ipnstate.Status
+    srs1PeerStatus         *ipnstate.PeerStatus
+)
+
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    srs1 = subRouter1.MustStatus()  // = not :=
+    srs2 = subRouter2.MustStatus()
+    clientStatus = client.MustStatus()
+    
+    srs1PeerStatus = clientStatus.Peer[srs1.Self.PublicKey]
+    // assertions...
+}, 5*time.Second, 200*time.Millisecond, "checking router status")
+
+// CORRECT: Wrapping client operations
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    result, err := client.Curl(weburl)
+    assert.NoError(c, err)
+    assert.Len(c, result, 13)
+}, 5*time.Second, 200*time.Millisecond, "Verifying HTTP connectivity")
+
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    tr, err := client.Traceroute(webip)
+    assert.NoError(c, err)
+    assertTracerouteViaIPWithCollect(c, tr, expectedRouter.MustIPv4())
+}, 5*time.Second, 200*time.Millisecond, "Verifying network path")
+```
+
+**Helper Functions**:
+- Use `requirePeerSubnetRoutesWithCollect` instead of `requirePeerSubnetRoutes` inside EventuallyWithT
+- Use `requireNodeRouteCountWithCollect` instead of `requireNodeRouteCount` inside EventuallyWithT
+- Use `assertTracerouteViaIPWithCollect` instead of `assertTracerouteViaIP` inside EventuallyWithT
+
+```go
+// Node route checking by actual node properties, not array position
+var routeNode *v1.Node
+for _, node := range nodes {
+    if nodeIDStr := fmt.Sprintf("%d", node.GetId()); expectedRoutes[nodeIDStr] != "" {
+        routeNode = node
+        break
+    }
+}
+```
+
+### Running Problematic Tests
+- Some tests require significant time (e.g., `TestNodeOnlineStatus` runs for 12 minutes)
+- Infrastructure issues like disk space can cause test failures unrelated to code changes  
+- Use `--postgres` flag when testing database-heavy scenarios
+
+## Quality Assurance and Testing Requirements
+
+### **MANDATORY: Always Use Specialized Testing Agents**
+
+**CRITICAL REQUIREMENT**: For ANY task involving testing, quality assurance, review, or validation, you MUST use the appropriate specialized agent at the END of your task list. This ensures comprehensive quality validation and prevents regressions.
+
+**Required Agents for Different Task Types**:
+
+1. **Integration Testing**: Use `headscale-integration-tester` agent for:
+   - Running integration tests with `cmd/hi`
+   - Analyzing test failures and artifacts
+   - Troubleshooting Docker-based test infrastructure
+   - Validating end-to-end functionality changes
+
+2. **Quality Control**: Use `quality-control-enforcer` agent for:
+   - Code review and validation
+   - Ensuring best practices compliance  
+   - Preventing common pitfalls and anti-patterns
+   - Validating architectural decisions
+
+**Agent Usage Pattern**: Always add the appropriate agent as the FINAL step in any task list to ensure quality validation occurs after all work is complete.
+
+### Integration Test Debugging Reference
+
+Test artifacts are preserved in `control_logs/TIMESTAMP-ID/` including:
+- Headscale server logs (stderr/stdout)
+- Tailscale client logs and status  
+- Database dumps and network captures
+- MapResponse JSON files for protocol debugging
+
+**For integration test issues, ALWAYS use the headscale-integration-tester agent - do not attempt manual debugging.**
+
+## EventuallyWithT Pattern for Integration Tests
+
+### Overview
+EventuallyWithT is a testing pattern used to handle eventual consistency in distributed systems. In Headscale integration tests, many operations are asynchronous - clients advertise routes, the server processes them, updates propagate through the network. EventuallyWithT allows tests to wait for these operations to complete while making assertions.
+
+### External Calls That Must Be Wrapped
+The following operations are **external calls** that interact with the headscale server or tailscale clients and MUST be wrapped in EventuallyWithT:
+- `headscale.ListNodes()` - Queries server state
+- `client.Status()` - Gets client network status
+- `client.Curl()` - Makes HTTP requests through the network
+- `client.Traceroute()` - Performs network diagnostics
+- `client.Execute()` when running commands that query state
+- Any operation that reads from the headscale server or tailscale client
+
+### Operations That Must NOT Be Wrapped
+The following are **blocking operations** that modify state and should NOT be wrapped in EventuallyWithT:
+- `tailscale set` commands (e.g., `--advertise-routes`, `--exit-node`)
+- Any command that changes configuration or state
+- Use `client.MustStatus()` instead of `client.Status()` when you just need the ID for a blocking operation
+
+### Five Key Rules for EventuallyWithT
+
+1. **One External Call Per EventuallyWithT Block**
+   - Each EventuallyWithT should make ONE external call (e.g., ListNodes OR Status)
+   - Related assertions based on that single call can be grouped together
+   - Unrelated external calls must be in separate EventuallyWithT blocks
+
+2. **Variable Scoping**
+   - Declare variables that need to be shared across EventuallyWithT blocks at function scope
+   - Use `=` for assignment inside EventuallyWithT, not `:=` (unless the variable is only used within that block)
+   - Variables declared with `:=` inside EventuallyWithT are not accessible outside
+
+3. **No Nested EventuallyWithT**
+   - NEVER put an EventuallyWithT inside another EventuallyWithT
+   - This is a critical anti-pattern that must be avoided
+
+4. **Use CollectT for Assertions**
+   - Inside EventuallyWithT, use `assert` methods with the CollectT parameter
+   - Helper functions called within EventuallyWithT must accept `*assert.CollectT`
+
+5. **Descriptive Messages**
+   - Always provide a descriptive message as the last parameter
+   - Message should explain what condition is being waited for
+
+### Correct Pattern Examples
+
+```go
+// CORRECT: Blocking operation NOT wrapped
+for _, client := range allClients {
+    status := client.MustStatus()
+    command := []string{
+        "tailscale",
+        "set",
+        "--advertise-routes=" + expectedRoutes[string(status.Self.ID)],
+    }
+    _, _, err = client.Execute(command)
+    require.NoErrorf(t, err, "failed to advertise route: %s", err)
+}
+
+// CORRECT: Single external call with related assertions
+var nodes []*v1.Node
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err = headscale.ListNodes()
+    assert.NoError(c, err)
+    assert.Len(c, nodes, 2)
+    requireNodeRouteCountWithCollect(c, nodes[0], 2, 2, 2)
+}, 10*time.Second, 500*time.Millisecond, "nodes should have expected route counts")
+
+// CORRECT: Separate EventuallyWithT for different external call
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    status, err := client.Status()
+    assert.NoError(c, err)
+    for _, peerKey := range status.Peers() {
+        peerStatus := status.Peer[peerKey]
+        requirePeerSubnetRoutesWithCollect(c, peerStatus, expectedPrefixes)
+    }
+}, 10*time.Second, 500*time.Millisecond, "client should see expected routes")
+```
+
+### Incorrect Patterns to Avoid
+
+```go
+// INCORRECT: Blocking operation wrapped in EventuallyWithT
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    status, err := client.Status()
+    assert.NoError(c, err)
+    
+    // This is a blocking operation - should NOT be in EventuallyWithT!
+    command := []string{
+        "tailscale",
+        "set",
+        "--advertise-routes=" + expectedRoutes[string(status.Self.ID)],
+    }
+    _, _, err = client.Execute(command)
+    assert.NoError(c, err)
+}, 5*time.Second, 200*time.Millisecond, "wrong pattern")
+
+// INCORRECT: Multiple unrelated external calls in same EventuallyWithT
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    // First external call
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    assert.Len(c, nodes, 2)
+    
+    // Second unrelated external call - WRONG!
+    status, err := client.Status()
+    assert.NoError(c, err)
+    assert.NotNil(c, status)
+}, 10*time.Second, 500*time.Millisecond, "mixed operations")
+```
+
+## Important Notes
+
+- **Dependencies**: Use `nix develop` for consistent toolchain (Go, buf, protobuf tools, linting)
+- **Protocol Buffers**: Changes to `proto/` require `make generate` and should be committed separately
+- **Code Style**: Enforced via golangci-lint with golines (width 88) and gofumpt formatting
+- **Database**: Supports both SQLite (development) and PostgreSQL (production/testing)
+- **Integration Tests**: Require Docker and can consume significant disk space - use headscale-integration-tester agent
+- **Performance**: NodeStore optimizations are critical for scale - be careful with changes to state management
+- **Quality Assurance**: Always use appropriate specialized agents for testing and validation tasks
+- **NEVER create gists in the user's name**: Do not use the `create_gist` tool - present information directly in the response instead

Dockerfile.integration

@@ -2,7 +2,7 @@
 # and are in no way endorsed by Headscale's maintainers as an
 # official nor supported release or distribution.
 
-FROM docker.io/golang:1.24-bookworm
+FROM docker.io/golang:1.25-bookworm
 ARG VERSION=dev
 ENV GOPATH /go
 WORKDIR /go/src/headscale
@@ -13,14 +13,18 @@ RUN apt-get update \
   && apt-get clean
 RUN mkdir -p /var/run/headscale
 
+# Install delve debugger
+RUN go install github.com/go-delve/delve/cmd/dlv@latest
+
 COPY go.mod go.sum /go/src/headscale/
 RUN go mod download
 
 COPY . .
 
-RUN CGO_ENABLED=0 GOOS=linux go install -a ./cmd/headscale && test -e /go/bin/headscale
+# Build debug binary with debug symbols for delve
+RUN CGO_ENABLED=0 GOOS=linux go build -gcflags="all=-N -l" -o /go/bin/headscale ./cmd/headscale
 
 # Need to reset the entrypoint or everything will run as a busybox script
 ENTRYPOINT []
-EXPOSE 8080/tcp
-CMD ["headscale"]
+EXPOSE 8080/tcp 40000/tcp
+CMD ["/go/bin/dlv", "--listen=0.0.0.0:40000", "--headless=true", "--api-version=2", "--accept-multiclient", "exec", "/go/bin/headscale", "--"]

Dockerfile.tailscale-HEAD

@@ -4,7 +4,7 @@
 # This Dockerfile is more or less lifted from tailscale/tailscale
 # to ensure a similar build process when testing the HEAD of tailscale.
 
-FROM golang:1.24-alpine AS build-env
+FROM golang:1.25-alpine AS build-env
 
 WORKDIR /go/src
 

Makefile

@@ -1,64 +1,129 @@
-# Calculate version
-version ?= $(shell git describe --always --tags --dirty)
+# Headscale Makefile
+# Modern Makefile following best practices
 
-rwildcard=$(foreach d,$(wildcard $1*),$(call rwildcard,$d/,$2) $(filter $(subst *,%,$2),$d))
+# Version calculation
+VERSION ?= $(shell git describe --always --tags --dirty)
 
-# Determine if OS supports pie
+# Build configuration
 GOOS ?= $(shell uname | tr '[:upper:]' '[:lower:]')
-ifeq ($(filter $(GOOS), openbsd netbsd soloaris plan9), )
-	pieflags = -buildmode=pie
-else
+ifeq ($(filter $(GOOS), openbsd netbsd solaris plan9), )
+	PIE_FLAGS = -buildmode=pie
 endif
 
-# GO_SOURCES = $(wildcard *.go)
-# PROTO_SOURCES = $(wildcard **/*.proto)
-GO_SOURCES = $(call rwildcard,,*.go)
-PROTO_SOURCES = $(call rwildcard,,*.proto)
+# Tool availability check with nix warning
+define check_tool
+	@command -v $(1) >/dev/null 2>&1 || { \
+		echo "Warning: $(1) not found. Run 'nix develop' to ensure all dependencies are available."; \
+		exit 1; \
+	}
+endef
+
+# Source file collections using shell find for better performance
+GO_SOURCES := $(shell find . -name '*.go' -not -path './gen/*' -not -path './vendor/*')
+PROTO_SOURCES := $(shell find . -name '*.proto' -not -path './gen/*' -not -path './vendor/*')
+DOC_SOURCES := $(shell find . \( -name '*.md' -o -name '*.yaml' -o -name '*.yml' -o -name '*.ts' -o -name '*.js' -o -name '*.html' -o -name '*.css' -o -name '*.scss' -o -name '*.sass' \) -not -path './gen/*' -not -path './vendor/*' -not -path './node_modules/*')
+
+# Default target
+.PHONY: all
+all: lint test build
+
+# Dependency checking
+.PHONY: check-deps
+check-deps:
+	$(call check_tool,go)
+	$(call check_tool,golangci-lint)
+	$(call check_tool,gofumpt)
+	$(call check_tool,prettier)
+	$(call check_tool,clang-format)
+	$(call check_tool,buf)
+
+# Build targets
+.PHONY: build
+build: check-deps $(GO_SOURCES) go.mod go.sum
+	@echo "Building headscale..."
+	go build $(PIE_FLAGS) -ldflags "-X main.version=$(VERSION)" -o headscale ./cmd/headscale
+
+# Test targets
+.PHONY: test
+test: check-deps $(GO_SOURCES) go.mod go.sum
+	@echo "Running Go tests..."
+	go test -race ./...
 
 
-build:
-	nix build
-
-dev: lint test build
-
-test:
-	gotestsum -- -short -race -coverprofile=coverage.out ./...
-
-test_integration:
-	docker run \
-		-t --rm \
-		-v ~/.cache/hs-integration-go:/go \
-		--name headscale-test-suite \
-		-v $$PWD:$$PWD -w $$PWD/integration \
-		-v /var/run/docker.sock:/var/run/docker.sock \
-		-v $$PWD/control_logs:/tmp/control \
-		golang:1 \
-		go run gotest.tools/gotestsum@latest -- -race -failfast ./... -timeout 120m -parallel 8
-
-lint:
-	golangci-lint run --fix --timeout 10m
-
+# Formatting targets
+.PHONY: fmt
 fmt: fmt-go fmt-prettier fmt-proto
 
-fmt-prettier:
-	prettier --write '**/**.{ts,js,md,yaml,yml,sass,css,scss,html}'
-	prettier --write --print-width 80 --prose-wrap always CHANGELOG.md
-
-fmt-go:
-	# TODO(kradalby): Reeval if we want to use 88 in the future.
-	# golines --max-len=88 --base-formatter=gofumpt -w $(GO_SOURCES)
+.PHONY: fmt-go
+fmt-go: check-deps $(GO_SOURCES)
+	@echo "Formatting Go code..."
 	gofumpt -l -w .
 	golangci-lint run --fix
 
-fmt-proto:
+.PHONY: fmt-prettier
+fmt-prettier: check-deps $(DOC_SOURCES)
+	@echo "Formatting documentation and config files..."
+	prettier --write '**/*.{ts,js,md,yaml,yml,sass,css,scss,html}'
+	prettier --write --print-width 80 --prose-wrap always CHANGELOG.md
+
+.PHONY: fmt-proto
+fmt-proto: check-deps $(PROTO_SOURCES)
+	@echo "Formatting Protocol Buffer files..."
 	clang-format -i $(PROTO_SOURCES)
 
-proto-lint:
-	cd proto/ && go run github.com/bufbuild/buf/cmd/buf lint
+# Linting targets
+.PHONY: lint
+lint: lint-go lint-proto
 
-compress: build
-	upx --brute headscale
+.PHONY: lint-go
+lint-go: check-deps $(GO_SOURCES) go.mod go.sum
+	@echo "Linting Go code..."
+	golangci-lint run --timeout 10m
 
-generate:
-	rm -rf gen
-	buf generate proto
+.PHONY: lint-proto
+lint-proto: check-deps $(PROTO_SOURCES)
+	@echo "Linting Protocol Buffer files..."
+	cd proto/ && buf lint
+
+# Code generation
+.PHONY: generate
+generate: check-deps
+	@echo "Generating code..."
+	go generate ./...
+
+# Clean targets
+.PHONY: clean
+clean:
+	rm -rf headscale gen
+
+# Development workflow
+.PHONY: dev
+dev: fmt lint test build
+
+# Help target
+.PHONY: help
+help:
+	@echo "Headscale Development Makefile"
+	@echo ""
+	@echo "Main targets:"
+	@echo "  all          - Run lint, test, and build (default)"
+	@echo "  build        - Build headscale binary"
+	@echo "  test         - Run Go tests"
+	@echo "  fmt          - Format all code (Go, docs, proto)"
+	@echo "  lint         - Lint all code (Go, proto)"
+	@echo "  generate     - Generate code from Protocol Buffers"
+	@echo "  dev          - Full development workflow (fmt + lint + test + build)"
+	@echo "  clean        - Clean build artifacts"
+	@echo ""
+	@echo "Specific targets:"
+	@echo "  fmt-go       - Format Go code only"
+	@echo "  fmt-prettier - Format documentation only" 
+	@echo "  fmt-proto    - Format Protocol Buffer files only"
+	@echo "  lint-go      - Lint Go code only"
+	@echo "  lint-proto   - Lint Protocol Buffer files only"
+	@echo ""
+	@echo "Dependencies:"
+	@echo "  check-deps   - Verify required tools are available"
+	@echo ""
+	@echo "Note: If not running in a nix shell, ensure dependencies are available:"
+	@echo "  nix develop"

README.md

@@ -11,8 +11,8 @@ to ensure you have the correct example configuration. The `main` branch might
 contain unreleased changes. The documentation is available for stable and
 development versions:
 
-* [Documentation for the stable version](https://headscale.net/stable/)
-* [Documentation for the development version](https://headscale.net/development/)
+- [Documentation for the stable version](https://headscale.net/stable/)
+- [Documentation for the development version](https://headscale.net/development/)
 
 ## What is Tailscale
 
@@ -138,16 +138,29 @@ make test
 
 To build the program:
 
-```shell
-nix build
-```
-
-or
-
 ```shell
 make build
 ```
 
+### Development workflow
+
+We recommend using Nix for dependency management to ensure you have all required tools. If you prefer to manage dependencies yourself, you can use Make directly:
+
+**With Nix (recommended):**
+```shell
+nix develop
+make test
+make build
+```
+
+**With your own dependencies:**
+```shell
+make test
+make build
+```
+
+The Makefile will warn you if any required tools are missing and suggest running `nix develop`. Run `make help` to see all available targets.
+
 ## Contributors
 
 <a href="https://github.com/juanfont/headscale/graphs/contributors">

cmd/headscale/cli/debug.go

@@ -117,7 +117,7 @@ var createNodeCmd = &cobra.Command{
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf("Cannot create node: %s", status.Convert(err).Message()),
+				"Cannot create node: "+status.Convert(err).Message(),
 				output,
 			)
 		}

cmd/headscale/cli/health.go

@@ -0,0 +1,29 @@
+package cli
+
+import (
+	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
+	"github.com/spf13/cobra"
+)
+
+func init() {
+	rootCmd.AddCommand(healthCmd)
+}
+
+var healthCmd = &cobra.Command{
+	Use:   "health",
+	Short: "Check the health of the Headscale server",
+	Long:  "Check the health of the Headscale server. This command will return an exit code of 0 if the server is healthy, or 1 if it is not.",
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
+
+		response, err := client.Health(ctx, &v1.HealthRequest{})
+		if err != nil {
+			ErrorOutput(err, "Error checking health", output)
+		}
+
+		SuccessOutput(response, "", output)
+	},
+}

cmd/headscale/cli/mockoidc.go

@@ -2,6 +2,7 @@ package cli
 
 import (
 	"encoding/json"
+	"errors"
 	"fmt"
 	"net"
 	"net/http"
@@ -68,7 +69,7 @@ func mockOIDC() error {
 
 	userStr := os.Getenv("MOCKOIDC_USERS")
 	if userStr == "" {
-		return fmt.Errorf("MOCKOIDC_USERS not defined")
+		return errors.New("MOCKOIDC_USERS not defined")
 	}
 
 	var users []mockoidc.MockUser

cmd/headscale/cli/nodes.go

@@ -9,7 +9,6 @@ import (
 	"strings"
 	"time"
 
-	survey "github.com/AlecAivazis/survey/v2"
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
 	"github.com/juanfont/headscale/hscontrol/util"
 	"github.com/pterm/pterm"
@@ -184,7 +183,7 @@ var listNodesCmd = &cobra.Command{
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf("Cannot get nodes: %s", status.Convert(err).Message()),
+				"Cannot get nodes: "+status.Convert(err).Message(),
 				output,
 			)
 		}
@@ -222,8 +221,6 @@ var listNodeRoutesCmd = &cobra.Command{
 				fmt.Sprintf("Error converting ID to integer: %s", err),
 				output,
 			)
-
-			return
 		}
 
 		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
@@ -290,8 +287,6 @@ var expireNodeCmd = &cobra.Command{
 				fmt.Sprintf("Error converting ID to integer: %s", err),
 				output,
 			)
-
-			return
 		}
 
 		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
@@ -312,8 +307,6 @@ var expireNodeCmd = &cobra.Command{
 				),
 				output,
 			)
-
-			return
 		}
 
 		SuccessOutput(response.GetNode(), "Node expired", output)
@@ -333,8 +326,6 @@ var renameNodeCmd = &cobra.Command{
 				fmt.Sprintf("Error converting ID to integer: %s", err),
 				output,
 			)
-
-			return
 		}
 
 		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
@@ -360,8 +351,6 @@ var renameNodeCmd = &cobra.Command{
 				),
 				output,
 			)
-
-			return
 		}
 
 		SuccessOutput(response.GetNode(), "Node renamed", output)
@@ -382,8 +371,6 @@ var deleteNodeCmd = &cobra.Command{
 				fmt.Sprintf("Error converting ID to integer: %s", err),
 				output,
 			)
-
-			return
 		}
 
 		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
@@ -398,14 +385,9 @@ var deleteNodeCmd = &cobra.Command{
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf(
-					"Error getting node node: %s",
-					status.Convert(err).Message(),
-				),
+				"Error getting node node: "+status.Convert(err).Message(),
 				output,
 			)
-
-			return
 		}
 
 		deleteRequest := &v1.DeleteNodeRequest{
@@ -415,16 +397,10 @@ var deleteNodeCmd = &cobra.Command{
 		confirm := false
 		force, _ := cmd.Flags().GetBool("force")
 		if !force {
-			prompt := &survey.Confirm{
-				Message: fmt.Sprintf(
-					"Do you want to remove the node %s?",
-					getResponse.GetNode().GetName(),
-				),
-			}
-			err = survey.AskOne(prompt, &confirm)
-			if err != nil {
-				return
-			}
+			confirm = util.YesNo(fmt.Sprintf(
+				"Do you want to remove the node %s?",
+				getResponse.GetNode().GetName(),
+			))
 		}
 
 		if confirm || force {
@@ -437,14 +413,9 @@ var deleteNodeCmd = &cobra.Command{
 			if err != nil {
 				ErrorOutput(
 					err,
-					fmt.Sprintf(
-						"Error deleting node: %s",
-						status.Convert(err).Message(),
-					),
+					"Error deleting node: "+status.Convert(err).Message(),
 					output,
 				)
-
-				return
 			}
 			SuccessOutput(
 				map[string]string{"Result": "Node deleted"},
@@ -471,8 +442,6 @@ var moveNodeCmd = &cobra.Command{
 				fmt.Sprintf("Error converting ID to integer: %s", err),
 				output,
 			)
-
-			return
 		}
 
 		user, err := cmd.Flags().GetUint64("user")
@@ -482,8 +451,6 @@ var moveNodeCmd = &cobra.Command{
 				fmt.Sprintf("Error getting user: %s", err),
 				output,
 			)
-
-			return
 		}
 
 		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
@@ -498,14 +465,9 @@ var moveNodeCmd = &cobra.Command{
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf(
-					"Error getting node: %s",
-					status.Convert(err).Message(),
-				),
+				"Error getting node: "+status.Convert(err).Message(),
 				output,
 			)
-
-			return
 		}
 
 		moveRequest := &v1.MoveNodeRequest{
@@ -517,14 +479,9 @@ var moveNodeCmd = &cobra.Command{
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf(
-					"Error moving node: %s",
-					status.Convert(err).Message(),
-				),
+				"Error moving node: "+status.Convert(err).Message(),
 				output,
 			)
-
-			return
 		}
 
 		SuccessOutput(moveResponse.GetNode(), "Node moved to another user", output)
@@ -547,34 +504,27 @@ If you remove IPv4 or IPv6 prefixes from the config,
 it can be run to remove the IPs that should no longer
 be assigned to nodes.`,
 	Run: func(cmd *cobra.Command, args []string) {
-		var err error
 		output, _ := cmd.Flags().GetString("output")
 
 		confirm := false
-		prompt := &survey.Confirm{
-			Message: "Are you sure that you want to assign/remove IPs to/from nodes?",
+
+		force, _ := cmd.Flags().GetBool("force")
+		if !force {
+			confirm = util.YesNo("Are you sure that you want to assign/remove IPs to/from nodes?")
 		}
-		err = survey.AskOne(prompt, &confirm)
-		if err != nil {
-			return
-		}
-		if confirm {
+
+		if confirm || force {
 			ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 			defer cancel()
 			defer conn.Close()
 
-			changes, err := client.BackfillNodeIPs(ctx, &v1.BackfillNodeIPsRequest{Confirmed: confirm})
+			changes, err := client.BackfillNodeIPs(ctx, &v1.BackfillNodeIPsRequest{Confirmed: confirm || force})
 			if err != nil {
 				ErrorOutput(
 					err,
-					fmt.Sprintf(
-						"Error backfilling IPs: %s",
-						status.Convert(err).Message(),
-					),
+					"Error backfilling IPs: "+status.Convert(err).Message(),
 					output,
 				)
-
-				return
 			}
 
 			SuccessOutput(changes, "Node IPs backfilled successfully", output)
@@ -773,8 +723,6 @@ var tagCmd = &cobra.Command{
 				fmt.Sprintf("Error converting ID to integer: %s", err),
 				output,
 			)
-
-			return
 		}
 		tagsToSet, err := cmd.Flags().GetStringSlice("tags")
 		if err != nil {
@@ -783,8 +731,6 @@ var tagCmd = &cobra.Command{
 				fmt.Sprintf("Error retrieving list of tags to add to node, %v", err),
 				output,
 			)
-
-			return
 		}
 
 		// Sending tags to node
@@ -799,8 +745,6 @@ var tagCmd = &cobra.Command{
 				fmt.Sprintf("Error while sending tags to headscale: %s", err),
 				output,
 			)
-
-			return
 		}
 
 		if resp != nil {
@@ -830,8 +774,6 @@ var approveRoutesCmd = &cobra.Command{
 				fmt.Sprintf("Error converting ID to integer: %s", err),
 				output,
 			)
-
-			return
 		}
 		routes, err := cmd.Flags().GetStringSlice("routes")
 		if err != nil {
@@ -840,8 +782,6 @@ var approveRoutesCmd = &cobra.Command{
 				fmt.Sprintf("Error retrieving list of routes to add to node, %v", err),
 				output,
 			)
-
-			return
 		}
 
 		// Sending routes to node
@@ -856,8 +796,6 @@ var approveRoutesCmd = &cobra.Command{
 				fmt.Sprintf("Error while sending routes to headscale: %s", err),
 				output,
 			)
-
-			return
 		}
 
 		if resp != nil {

cmd/headscale/cli/policy.go

@@ -6,19 +6,30 @@ import (
 	"os"
 
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
+	"github.com/juanfont/headscale/hscontrol/db"
 	"github.com/juanfont/headscale/hscontrol/policy"
+	"github.com/juanfont/headscale/hscontrol/types"
+	"github.com/juanfont/headscale/hscontrol/util"
 	"github.com/rs/zerolog/log"
 	"github.com/spf13/cobra"
+	"tailscale.com/types/views"
+)
+
+const (
+	bypassFlag = "bypass-grpc-and-access-database-directly"
 )
 
 func init() {
 	rootCmd.AddCommand(policyCmd)
+
+	getPolicy.Flags().BoolP(bypassFlag, "", false, "Uses the headscale config to directly access the database, bypassing gRPC and does not require the server to be running")
 	policyCmd.AddCommand(getPolicy)
 
 	setPolicy.Flags().StringP("file", "f", "", "Path to a policy file in HuJSON format")
 	if err := setPolicy.MarkFlagRequired("file"); err != nil {
 		log.Fatal().Err(err).Msg("")
 	}
+	setPolicy.Flags().BoolP(bypassFlag, "", false, "Uses the headscale config to directly access the database, bypassing gRPC and does not require the server to be running")
 	policyCmd.AddCommand(setPolicy)
 
 	checkPolicy.Flags().StringP("file", "f", "", "Path to a policy file in HuJSON format")
@@ -39,21 +50,58 @@ var getPolicy = &cobra.Command{
 	Aliases: []string{"show", "view", "fetch"},
 	Run: func(cmd *cobra.Command, args []string) {
 		output, _ := cmd.Flags().GetString("output")
-		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
-		defer cancel()
-		defer conn.Close()
+		var policy string
+		if bypass, _ := cmd.Flags().GetBool(bypassFlag); bypass {
+			confirm := false
+			force, _ := cmd.Flags().GetBool("force")
+			if !force {
+				confirm = util.YesNo("DO NOT run this command if an instance of headscale is running, are you sure headscale is not running?")
+			}
 
-		request := &v1.GetPolicyRequest{}
+			if !confirm && !force {
+				ErrorOutput(nil, "Aborting command", output)
+				return
+			}
 
-		response, err := client.GetPolicy(ctx, request)
-		if err != nil {
-			ErrorOutput(err, fmt.Sprintf("Failed loading ACL Policy: %s", err), output)
+			cfg, err := types.LoadServerConfig()
+			if err != nil {
+				ErrorOutput(err, fmt.Sprintf("Failed loading config: %s", err), output)
+			}
+
+			d, err := db.NewHeadscaleDatabase(
+				cfg.Database,
+				cfg.BaseDomain,
+				nil,
+			)
+			if err != nil {
+				ErrorOutput(err, fmt.Sprintf("Failed to open database: %s", err), output)
+			}
+
+			pol, err := d.GetPolicy()
+			if err != nil {
+				ErrorOutput(err, fmt.Sprintf("Failed loading Policy from database: %s", err), output)
+			}
+
+			policy = pol.Data
+		} else {
+			ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+			defer cancel()
+			defer conn.Close()
+
+			request := &v1.GetPolicyRequest{}
+
+			response, err := client.GetPolicy(ctx, request)
+			if err != nil {
+				ErrorOutput(err, fmt.Sprintf("Failed loading ACL Policy: %s", err), output)
+			}
+
+			policy = response.GetPolicy()
 		}
 
 		// TODO(pallabpain): Maybe print this better?
 		// This does not pass output as we dont support yaml, json or json-line
 		// output for this command. It is HuJSON already.
-		SuccessOutput("", response.GetPolicy(), "")
+		SuccessOutput("", policy, "")
 	},
 }
 
@@ -79,14 +127,52 @@ var setPolicy = &cobra.Command{
 			ErrorOutput(err, fmt.Sprintf("Error reading the policy file: %s", err), output)
 		}
 
-		request := &v1.SetPolicyRequest{Policy: string(policyBytes)}
+		_, err = policy.NewPolicyManager(policyBytes, nil, views.Slice[types.NodeView]{})
+		if err != nil {
+			ErrorOutput(err, fmt.Sprintf("Error parsing the policy file: %s", err), output)
+			return
+		}
 
-		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
-		defer cancel()
-		defer conn.Close()
+		if bypass, _ := cmd.Flags().GetBool(bypassFlag); bypass {
+			confirm := false
+			force, _ := cmd.Flags().GetBool("force")
+			if !force {
+				confirm = util.YesNo("DO NOT run this command if an instance of headscale is running, are you sure headscale is not running?")
+			}
 
-		if _, err := client.SetPolicy(ctx, request); err != nil {
-			ErrorOutput(err, fmt.Sprintf("Failed to set ACL Policy: %s", err), output)
+			if !confirm && !force {
+				ErrorOutput(nil, "Aborting command", output)
+				return
+			}
+
+			cfg, err := types.LoadServerConfig()
+			if err != nil {
+				ErrorOutput(err, fmt.Sprintf("Failed loading config: %s", err), output)
+			}
+
+			d, err := db.NewHeadscaleDatabase(
+				cfg.Database,
+				cfg.BaseDomain,
+				nil,
+			)
+			if err != nil {
+				ErrorOutput(err, fmt.Sprintf("Failed to open database: %s", err), output)
+			}
+
+			_, err = d.SetPolicy(string(policyBytes))
+			if err != nil {
+				ErrorOutput(err, fmt.Sprintf("Failed to set ACL Policy: %s", err), output)
+			}
+		} else {
+			request := &v1.SetPolicyRequest{Policy: string(policyBytes)}
+
+			ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+			defer cancel()
+			defer conn.Close()
+
+			if _, err := client.SetPolicy(ctx, request); err != nil {
+				ErrorOutput(err, fmt.Sprintf("Failed to set ACL Policy: %s", err), output)
+			}
 		}
 
 		SuccessOutput(nil, "Policy updated.", "")
@@ -111,7 +197,7 @@ var checkPolicy = &cobra.Command{
 			ErrorOutput(err, fmt.Sprintf("Error reading the policy file: %s", err), output)
 		}
 
-		_, err = policy.NewPolicyManager(policyBytes, nil, nil)
+		_, err = policy.NewPolicyManager(policyBytes, nil, views.Slice[types.NodeView]{})
 		if err != nil {
 			ErrorOutput(err, fmt.Sprintf("Error parsing the policy file: %s", err), output)
 		}

cmd/headscale/cli/root.go

@@ -5,6 +5,7 @@ import (
 	"os"
 	"runtime"
 	"slices"
+	"strings"
 
 	"github.com/juanfont/headscale/hscontrol/types"
 	"github.com/rs/zerolog"
@@ -71,25 +72,64 @@ func initConfig() {
 
 	disableUpdateCheck := viper.GetBool("disable_check_updates")
 	if !disableUpdateCheck && !machineOutput {
+		versionInfo := types.GetVersionInfo()
 		if (runtime.GOOS == "linux" || runtime.GOOS == "darwin") &&
-			types.Version != "dev" {
+			!versionInfo.Dirty {
 			githubTag := &latest.GithubTag{
-				Owner:      "juanfont",
-				Repository: "headscale",
+				Owner:         "juanfont",
+				Repository:    "headscale",
+				TagFilterFunc: filterPreReleasesIfStable(func() string { return versionInfo.Version }),
 			}
-			res, err := latest.Check(githubTag, types.Version)
+			res, err := latest.Check(githubTag, versionInfo.Version)
 			if err == nil && res.Outdated {
 				//nolint
 				log.Warn().Msgf(
 					"An updated version of Headscale has been found (%s vs. your current %s). Check it out https://github.com/juanfont/headscale/releases\n",
 					res.Current,
-					types.Version,
+					versionInfo.Version,
 				)
 			}
 		}
 	}
 }
 
+var prereleases = []string{"alpha", "beta", "rc", "dev"}
+
+func isPreReleaseVersion(version string) bool {
+	for _, unstable := range prereleases {
+		if strings.Contains(version, unstable) {
+			return true
+		}
+	}
+	return false
+}
+
+// filterPreReleasesIfStable returns a function that filters out
+// pre-release tags if the current version is stable.
+// If the current version is a pre-release, it does not filter anything.
+// versionFunc is a function that returns the current version string, it is
+// a func for testability.
+func filterPreReleasesIfStable(versionFunc func() string) func(string) bool {
+	return func(tag string) bool {
+		version := versionFunc()
+
+		// If we are on a pre-release version, then we do not filter anything
+		// as we want to recommend the user the latest pre-release.
+		if isPreReleaseVersion(version) {
+			return false
+		}
+
+		// If we are on a stable release, filter out pre-releases.
+		for _, ignore := range prereleases {
+			if strings.Contains(tag, ignore) {
+				return true
+			}
+		}
+
+		return false
+	}
+}
+
 var rootCmd = &cobra.Command{
 	Use:   "headscale",
 	Short: "headscale - a Tailscale control server",

cmd/headscale/cli/root_test.go

@@ -0,0 +1,293 @@
+package cli
+
+import (
+	"testing"
+)
+
+func TestFilterPreReleasesIfStable(t *testing.T) {
+	tests := []struct {
+		name           string
+		currentVersion string
+		tag            string
+		expectedFilter bool
+		description    string
+	}{
+		{
+			name:           "stable version filters alpha tag",
+			currentVersion: "0.23.0",
+			tag:            "v0.24.0-alpha.1",
+			expectedFilter: true,
+			description:    "When on stable release, alpha tags should be filtered",
+		},
+		{
+			name:           "stable version filters beta tag",
+			currentVersion: "0.23.0",
+			tag:            "v0.24.0-beta.2",
+			expectedFilter: true,
+			description:    "When on stable release, beta tags should be filtered",
+		},
+		{
+			name:           "stable version filters rc tag",
+			currentVersion: "0.23.0",
+			tag:            "v0.24.0-rc.1",
+			expectedFilter: true,
+			description:    "When on stable release, rc tags should be filtered",
+		},
+		{
+			name:           "stable version allows stable tag",
+			currentVersion: "0.23.0",
+			tag:            "v0.24.0",
+			expectedFilter: false,
+			description:    "When on stable release, stable tags should not be filtered",
+		},
+		{
+			name:           "alpha version allows alpha tag",
+			currentVersion: "0.23.0-alpha.1",
+			tag:            "v0.24.0-alpha.2",
+			expectedFilter: false,
+			description:    "When on alpha release, alpha tags should not be filtered",
+		},
+		{
+			name:           "alpha version allows beta tag",
+			currentVersion: "0.23.0-alpha.1",
+			tag:            "v0.24.0-beta.1",
+			expectedFilter: false,
+			description:    "When on alpha release, beta tags should not be filtered",
+		},
+		{
+			name:           "alpha version allows rc tag",
+			currentVersion: "0.23.0-alpha.1",
+			tag:            "v0.24.0-rc.1",
+			expectedFilter: false,
+			description:    "When on alpha release, rc tags should not be filtered",
+		},
+		{
+			name:           "alpha version allows stable tag",
+			currentVersion: "0.23.0-alpha.1",
+			tag:            "v0.24.0",
+			expectedFilter: false,
+			description:    "When on alpha release, stable tags should not be filtered",
+		},
+		{
+			name:           "beta version allows alpha tag",
+			currentVersion: "0.23.0-beta.1",
+			tag:            "v0.24.0-alpha.1",
+			expectedFilter: false,
+			description:    "When on beta release, alpha tags should not be filtered",
+		},
+		{
+			name:           "beta version allows beta tag",
+			currentVersion: "0.23.0-beta.2",
+			tag:            "v0.24.0-beta.3",
+			expectedFilter: false,
+			description:    "When on beta release, beta tags should not be filtered",
+		},
+		{
+			name:           "beta version allows rc tag",
+			currentVersion: "0.23.0-beta.1",
+			tag:            "v0.24.0-rc.1",
+			expectedFilter: false,
+			description:    "When on beta release, rc tags should not be filtered",
+		},
+		{
+			name:           "beta version allows stable tag",
+			currentVersion: "0.23.0-beta.1",
+			tag:            "v0.24.0",
+			expectedFilter: false,
+			description:    "When on beta release, stable tags should not be filtered",
+		},
+		{
+			name:           "rc version allows alpha tag",
+			currentVersion: "0.23.0-rc.1",
+			tag:            "v0.24.0-alpha.1",
+			expectedFilter: false,
+			description:    "When on rc release, alpha tags should not be filtered",
+		},
+		{
+			name:           "rc version allows beta tag",
+			currentVersion: "0.23.0-rc.1",
+			tag:            "v0.24.0-beta.1",
+			expectedFilter: false,
+			description:    "When on rc release, beta tags should not be filtered",
+		},
+		{
+			name:           "rc version allows rc tag",
+			currentVersion: "0.23.0-rc.2",
+			tag:            "v0.24.0-rc.3",
+			expectedFilter: false,
+			description:    "When on rc release, rc tags should not be filtered",
+		},
+		{
+			name:           "rc version allows stable tag",
+			currentVersion: "0.23.0-rc.1",
+			tag:            "v0.24.0",
+			expectedFilter: false,
+			description:    "When on rc release, stable tags should not be filtered",
+		},
+		{
+			name:           "stable version with patch filters alpha",
+			currentVersion: "0.23.1",
+			tag:            "v0.24.0-alpha.1",
+			expectedFilter: true,
+			description:    "Stable version with patch number should filter alpha tags",
+		},
+		{
+			name:           "stable version with patch allows stable",
+			currentVersion: "0.23.1",
+			tag:            "v0.24.0",
+			expectedFilter: false,
+			description:    "Stable version with patch number should allow stable tags",
+		},
+		{
+			name:           "tag with alpha substring in version number",
+			currentVersion: "0.23.0",
+			tag:            "v1.0.0-alpha.1",
+			expectedFilter: true,
+			description:    "Tags with alpha in version string should be filtered on stable",
+		},
+		{
+			name:           "tag with beta substring in version number",
+			currentVersion: "0.23.0",
+			tag:            "v1.0.0-beta.1",
+			expectedFilter: true,
+			description:    "Tags with beta in version string should be filtered on stable",
+		},
+		{
+			name:           "tag with rc substring in version number",
+			currentVersion: "0.23.0",
+			tag:            "v1.0.0-rc.1",
+			expectedFilter: true,
+			description:    "Tags with rc in version string should be filtered on stable",
+		},
+		{
+			name:           "empty tag on stable version",
+			currentVersion: "0.23.0",
+			tag:            "",
+			expectedFilter: false,
+			description:    "Empty tags should not be filtered",
+		},
+		{
+			name:           "dev version allows all tags",
+			currentVersion: "0.23.0-dev",
+			tag:            "v0.24.0-alpha.1",
+			expectedFilter: false,
+			description:    "Dev versions should not filter any tags (pre-release allows all)",
+		},
+		{
+			name:           "stable version filters dev tag",
+			currentVersion: "0.23.0",
+			tag:            "v0.24.0-dev",
+			expectedFilter: true,
+			description:    "When on stable release, dev tags should be filtered",
+		},
+		{
+			name:           "dev version allows dev tag",
+			currentVersion: "0.23.0-dev",
+			tag:            "v0.24.0-dev.1",
+			expectedFilter: false,
+			description:    "When on dev release, dev tags should not be filtered",
+		},
+		{
+			name:           "dev version allows stable tag",
+			currentVersion: "0.23.0-dev",
+			tag:            "v0.24.0",
+			expectedFilter: false,
+			description:    "When on dev release, stable tags should not be filtered",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			result := filterPreReleasesIfStable(func() string { return tt.currentVersion })(tt.tag)
+			if result != tt.expectedFilter {
+				t.Errorf("%s: got %v, want %v\nDescription: %s\nCurrent version: %s, Tag: %s",
+					tt.name,
+					result,
+					tt.expectedFilter,
+					tt.description,
+					tt.currentVersion,
+					tt.tag,
+				)
+			}
+		})
+	}
+}
+
+func TestIsPreReleaseVersion(t *testing.T) {
+	tests := []struct {
+		name        string
+		version     string
+		expected    bool
+		description string
+	}{
+		{
+			name:        "stable version",
+			version:     "0.23.0",
+			expected:    false,
+			description: "Stable version should not be pre-release",
+		},
+		{
+			name:        "alpha version",
+			version:     "0.23.0-alpha.1",
+			expected:    true,
+			description: "Alpha version should be pre-release",
+		},
+		{
+			name:        "beta version",
+			version:     "0.23.0-beta.1",
+			expected:    true,
+			description: "Beta version should be pre-release",
+		},
+		{
+			name:        "rc version",
+			version:     "0.23.0-rc.1",
+			expected:    true,
+			description: "RC version should be pre-release",
+		},
+		{
+			name:        "version with alpha substring",
+			version:     "0.23.0-alphabetical",
+			expected:    true,
+			description: "Version containing 'alpha' should be pre-release",
+		},
+		{
+			name:        "version with beta substring",
+			version:     "0.23.0-betamax",
+			expected:    true,
+			description: "Version containing 'beta' should be pre-release",
+		},
+		{
+			name:        "dev version",
+			version:     "0.23.0-dev",
+			expected:    true,
+			description: "Dev version should be pre-release",
+		},
+		{
+			name:        "empty version",
+			version:     "",
+			expected:    false,
+			description: "Empty version should not be pre-release",
+		},
+		{
+			name:        "version with patch number",
+			version:     "0.23.1",
+			expected:    false,
+			description: "Stable version with patch should not be pre-release",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			result := isPreReleaseVersion(tt.version)
+			if result != tt.expected {
+				t.Errorf("%s: got %v, want %v\nDescription: %s\nVersion: %s",
+					tt.name,
+					result,
+					tt.expected,
+					tt.description,
+					tt.version,
+				)
+			}
+		})
+	}
+}

cmd/headscale/cli/serve.go

@@ -2,10 +2,12 @@ package cli
 
 import (
 	"errors"
+	"fmt"
 	"net/http"
 
 	"github.com/rs/zerolog/log"
 	"github.com/spf13/cobra"
+	"github.com/tailscale/squibble"
 )
 
 func init() {
@@ -21,6 +23,12 @@ var serveCmd = &cobra.Command{
 	Run: func(cmd *cobra.Command, args []string) {
 		app, err := newHeadscaleServerWithConfig()
 		if err != nil {
+			var squibbleErr squibble.ValidationError
+			if errors.As(err, &squibbleErr) {
+				fmt.Printf("SQLite schema failed to validate:\n")
+				fmt.Println(squibbleErr.Diff)
+			}
+
 			log.Fatal().Caller().Err(err).Msg("Error initializing")
 		}
 

cmd/headscale/cli/users.go

@@ -4,9 +4,10 @@ import (
 	"errors"
 	"fmt"
 	"net/url"
+	"strconv"
 
-	survey "github.com/AlecAivazis/survey/v2"
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
+	"github.com/juanfont/headscale/hscontrol/util"
 	"github.com/pterm/pterm"
 	"github.com/rs/zerolog/log"
 	"github.com/spf13/cobra"
@@ -27,10 +28,7 @@ func usernameAndIDFromFlag(cmd *cobra.Command) (uint64, string) {
 		err := errors.New("--name or --identifier flag is required")
 		ErrorOutput(
 			err,
-			fmt.Sprintf(
-				"Cannot rename user: %s",
-				status.Convert(err).Message(),
-			),
+			"Cannot rename user: "+status.Convert(err).Message(),
 			"",
 		)
 	}
@@ -114,10 +112,7 @@ var createUserCmd = &cobra.Command{
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf(
-					"Cannot create user: %s",
-					status.Convert(err).Message(),
-				),
+				"Cannot create user: "+status.Convert(err).Message(),
 				output,
 			)
 		}
@@ -147,16 +142,16 @@ var destroyUserCmd = &cobra.Command{
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf("Error: %s", status.Convert(err).Message()),
+				"Error: "+status.Convert(err).Message(),
 				output,
 			)
 		}
 
 		if len(users.GetUsers()) != 1 {
-			err := fmt.Errorf("Unable to determine user to delete, query returned multiple users, use ID")
+			err := errors.New("Unable to determine user to delete, query returned multiple users, use ID")
 			ErrorOutput(
 				err,
-				fmt.Sprintf("Error: %s", status.Convert(err).Message()),
+				"Error: "+status.Convert(err).Message(),
 				output,
 			)
 		}
@@ -166,16 +161,10 @@ var destroyUserCmd = &cobra.Command{
 		confirm := false
 		force, _ := cmd.Flags().GetBool("force")
 		if !force {
-			prompt := &survey.Confirm{
-				Message: fmt.Sprintf(
-					"Do you want to remove the user %q (%d) and any associated preauthkeys?",
-					user.GetName(), user.GetId(),
-				),
-			}
-			err := survey.AskOne(prompt, &confirm)
-			if err != nil {
-				return
-			}
+			confirm = util.YesNo(fmt.Sprintf(
+				"Do you want to remove the user %q (%d) and any associated preauthkeys?",
+				user.GetName(), user.GetId(),
+			))
 		}
 
 		if confirm || force {
@@ -185,10 +174,7 @@ var destroyUserCmd = &cobra.Command{
 			if err != nil {
 				ErrorOutput(
 					err,
-					fmt.Sprintf(
-						"Cannot destroy user: %s",
-						status.Convert(err).Message(),
-					),
+					"Cannot destroy user: "+status.Convert(err).Message(),
 					output,
 				)
 			}
@@ -220,20 +206,17 @@ var listUsersCmd = &cobra.Command{
 		switch {
 		case id > 0:
 			request.Id = uint64(id)
-			break
 		case username != "":
 			request.Name = username
-			break
 		case email != "":
 			request.Email = email
-			break
 		}
 
 		response, err := client.ListUsers(ctx, request)
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf("Cannot get users: %s", status.Convert(err).Message()),
+				"Cannot get users: "+status.Convert(err).Message(),
 				output,
 			)
 		}
@@ -247,7 +230,7 @@ var listUsersCmd = &cobra.Command{
 			tableData = append(
 				tableData,
 				[]string{
-					fmt.Sprintf("%d", user.GetId()),
+					strconv.FormatUint(user.GetId(), 10),
 					user.GetDisplayName(),
 					user.GetName(),
 					user.GetEmail(),
@@ -287,16 +270,16 @@ var renameUserCmd = &cobra.Command{
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf("Error: %s", status.Convert(err).Message()),
+				"Error: "+status.Convert(err).Message(),
 				output,
 			)
 		}
 
 		if len(users.GetUsers()) != 1 {
-			err := fmt.Errorf("Unable to determine user to delete, query returned multiple users, use ID")
+			err := errors.New("Unable to determine user to delete, query returned multiple users, use ID")
 			ErrorOutput(
 				err,
-				fmt.Sprintf("Error: %s", status.Convert(err).Message()),
+				"Error: "+status.Convert(err).Message(),
 				output,
 			)
 		}
@@ -312,10 +295,7 @@ var renameUserCmd = &cobra.Command{
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf(
-					"Cannot rename user: %s",
-					status.Convert(err).Message(),
-				),
+				"Cannot rename user: "+status.Convert(err).Message(),
 				output,
 			)
 		}

cmd/headscale/cli/utils.go

@@ -169,7 +169,14 @@ func ErrorOutput(errResult error, override string, outputFormat string) {
 		Error string `json:"error"`
 	}
 
-	fmt.Fprintf(os.Stderr, "%s\n", output(errOutput{errResult.Error()}, override, outputFormat))
+	var errorMessage string
+	if errResult != nil {
+		errorMessage = errResult.Error()
+	} else {
+		errorMessage = override
+	}
+
+	fmt.Fprintf(os.Stderr, "%s\n", output(errOutput{errorMessage}, override, outputFormat))
 	os.Exit(1)
 }
 

cmd/headscale/cli/version.go

@@ -7,6 +7,7 @@ import (
 
 func init() {
 	rootCmd.AddCommand(versionCmd)
+	versionCmd.Flags().StringP("output", "o", "", "Output format. Empty for human-readable, 'json', 'json-line' or 'yaml'")
 }
 
 var versionCmd = &cobra.Command{
@@ -15,9 +16,9 @@ var versionCmd = &cobra.Command{
 	Long:  "The version of headscale.",
 	Run: func(cmd *cobra.Command, args []string) {
 		output, _ := cmd.Flags().GetString("output")
-		SuccessOutput(map[string]string{
-			"version": types.Version,
-			"commit":  types.GitCommitHash,
-		}, types.Version, output)
+
+		info := types.GetVersionInfo()
+
+		SuccessOutput(info, info.String(), output)
 	},
 }

cmd/hi/cleanup.go

@@ -0,0 +1,207 @@
+package main
+
+import (
+	"context"
+	"fmt"
+	"strings"
+	"time"
+
+	"github.com/docker/docker/api/types/container"
+	"github.com/docker/docker/api/types/filters"
+	"github.com/docker/docker/api/types/image"
+	"github.com/docker/docker/client"
+	"github.com/docker/docker/errdefs"
+)
+
+// cleanupBeforeTest performs cleanup operations before running tests.
+func cleanupBeforeTest(ctx context.Context) error {
+	if err := killTestContainers(ctx); err != nil {
+		return fmt.Errorf("failed to kill test containers: %w", err)
+	}
+
+	if err := pruneDockerNetworks(ctx); err != nil {
+		return fmt.Errorf("failed to prune networks: %w", err)
+	}
+
+	return nil
+}
+
+// cleanupAfterTest removes the test container after completion.
+func cleanupAfterTest(ctx context.Context, cli *client.Client, containerID string) error {
+	return cli.ContainerRemove(ctx, containerID, container.RemoveOptions{
+		Force: true,
+	})
+}
+
+// killTestContainers terminates and removes all test containers.
+func killTestContainers(ctx context.Context) error {
+	cli, err := createDockerClient()
+	if err != nil {
+		return fmt.Errorf("failed to create Docker client: %w", err)
+	}
+	defer cli.Close()
+
+	containers, err := cli.ContainerList(ctx, container.ListOptions{
+		All: true,
+	})
+	if err != nil {
+		return fmt.Errorf("failed to list containers: %w", err)
+	}
+
+	removed := 0
+	for _, cont := range containers {
+		shouldRemove := false
+		for _, name := range cont.Names {
+			if strings.Contains(name, "headscale-test-suite") ||
+				strings.Contains(name, "hs-") ||
+				strings.Contains(name, "ts-") ||
+				strings.Contains(name, "derp-") {
+				shouldRemove = true
+				break
+			}
+		}
+
+		if shouldRemove {
+			// First kill the container if it's running
+			if cont.State == "running" {
+				_ = cli.ContainerKill(ctx, cont.ID, "KILL")
+			}
+
+			// Then remove the container with retry logic
+			if removeContainerWithRetry(ctx, cli, cont.ID) {
+				removed++
+			}
+		}
+	}
+
+	if removed > 0 {
+		fmt.Printf("Removed %d test containers\n", removed)
+	} else {
+		fmt.Println("No test containers found to remove")
+	}
+
+	return nil
+}
+
+// removeContainerWithRetry attempts to remove a container with exponential backoff retry logic.
+func removeContainerWithRetry(ctx context.Context, cli *client.Client, containerID string) bool {
+	maxRetries := 3
+	baseDelay := 100 * time.Millisecond
+
+	for attempt := range maxRetries {
+		err := cli.ContainerRemove(ctx, containerID, container.RemoveOptions{
+			Force: true,
+		})
+		if err == nil {
+			return true
+		}
+
+		// If this is the last attempt, don't wait
+		if attempt == maxRetries-1 {
+			break
+		}
+
+		// Wait with exponential backoff
+		delay := baseDelay * time.Duration(1<<attempt)
+		time.Sleep(delay)
+	}
+
+	return false
+}
+
+// pruneDockerNetworks removes unused Docker networks.
+func pruneDockerNetworks(ctx context.Context) error {
+	cli, err := createDockerClient()
+	if err != nil {
+		return fmt.Errorf("failed to create Docker client: %w", err)
+	}
+	defer cli.Close()
+
+	report, err := cli.NetworksPrune(ctx, filters.Args{})
+	if err != nil {
+		return fmt.Errorf("failed to prune networks: %w", err)
+	}
+
+	if len(report.NetworksDeleted) > 0 {
+		fmt.Printf("Removed %d unused networks\n", len(report.NetworksDeleted))
+	} else {
+		fmt.Println("No unused networks found to remove")
+	}
+
+	return nil
+}
+
+// cleanOldImages removes test-related and old dangling Docker images.
+func cleanOldImages(ctx context.Context) error {
+	cli, err := createDockerClient()
+	if err != nil {
+		return fmt.Errorf("failed to create Docker client: %w", err)
+	}
+	defer cli.Close()
+
+	images, err := cli.ImageList(ctx, image.ListOptions{
+		All: true,
+	})
+	if err != nil {
+		return fmt.Errorf("failed to list images: %w", err)
+	}
+
+	removed := 0
+	for _, img := range images {
+		shouldRemove := false
+		for _, tag := range img.RepoTags {
+			if strings.Contains(tag, "hs-") ||
+				strings.Contains(tag, "headscale-integration") ||
+				strings.Contains(tag, "tailscale") {
+				shouldRemove = true
+				break
+			}
+		}
+
+		if len(img.RepoTags) == 0 && time.Unix(img.Created, 0).Before(time.Now().Add(-7*24*time.Hour)) {
+			shouldRemove = true
+		}
+
+		if shouldRemove {
+			_, err := cli.ImageRemove(ctx, img.ID, image.RemoveOptions{
+				Force: true,
+			})
+			if err == nil {
+				removed++
+			}
+		}
+	}
+
+	if removed > 0 {
+		fmt.Printf("Removed %d test images\n", removed)
+	} else {
+		fmt.Println("No test images found to remove")
+	}
+
+	return nil
+}
+
+// cleanCacheVolume removes the Docker volume used for Go module cache.
+func cleanCacheVolume(ctx context.Context) error {
+	cli, err := createDockerClient()
+	if err != nil {
+		return fmt.Errorf("failed to create Docker client: %w", err)
+	}
+	defer cli.Close()
+
+	volumeName := "hs-integration-go-cache"
+	err = cli.VolumeRemove(ctx, volumeName, true)
+	if err != nil {
+		if errdefs.IsNotFound(err) {
+			fmt.Printf("Go module cache volume not found: %s\n", volumeName)
+		} else if errdefs.IsConflict(err) {
+			fmt.Printf("Go module cache volume is in use and cannot be removed: %s\n", volumeName)
+		} else {
+			fmt.Printf("Failed to remove Go module cache volume %s: %v\n", volumeName, err)
+		}
+	} else {
+		fmt.Printf("Removed Go module cache volume: %s\n", volumeName)
+	}
+
+	return nil
+}

cmd/hi/docker.go

@@ -0,0 +1,763 @@
+package main
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"io"
+	"log"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"strings"
+	"time"
+
+	"github.com/docker/docker/api/types/container"
+	"github.com/docker/docker/api/types/image"
+	"github.com/docker/docker/api/types/mount"
+	"github.com/docker/docker/client"
+	"github.com/docker/docker/pkg/stdcopy"
+	"github.com/juanfont/headscale/integration/dockertestutil"
+)
+
+var (
+	ErrTestFailed              = errors.New("test failed")
+	ErrUnexpectedContainerWait = errors.New("unexpected end of container wait")
+	ErrNoDockerContext         = errors.New("no docker context found")
+)
+
+// runTestContainer executes integration tests in a Docker container.
+func runTestContainer(ctx context.Context, config *RunConfig) error {
+	cli, err := createDockerClient()
+	if err != nil {
+		return fmt.Errorf("failed to create Docker client: %w", err)
+	}
+	defer cli.Close()
+
+	runID := dockertestutil.GenerateRunID()
+	containerName := "headscale-test-suite-" + runID
+	logsDir := filepath.Join(config.LogsDir, runID)
+
+	if config.Verbose {
+		log.Printf("Run ID: %s", runID)
+		log.Printf("Container name: %s", containerName)
+		log.Printf("Logs directory: %s", logsDir)
+	}
+
+	absLogsDir, err := filepath.Abs(logsDir)
+	if err != nil {
+		return fmt.Errorf("failed to get absolute path for logs directory: %w", err)
+	}
+
+	const dirPerm = 0o755
+	if err := os.MkdirAll(absLogsDir, dirPerm); err != nil {
+		return fmt.Errorf("failed to create logs directory: %w", err)
+	}
+
+	if config.CleanBefore {
+		if config.Verbose {
+			log.Printf("Running pre-test cleanup...")
+		}
+		if err := cleanupBeforeTest(ctx); err != nil && config.Verbose {
+			log.Printf("Warning: pre-test cleanup failed: %v", err)
+		}
+	}
+
+	goTestCmd := buildGoTestCommand(config)
+	if config.Verbose {
+		log.Printf("Command: %s", strings.Join(goTestCmd, " "))
+	}
+
+	imageName := "golang:" + config.GoVersion
+	if err := ensureImageAvailable(ctx, cli, imageName, config.Verbose); err != nil {
+		return fmt.Errorf("failed to ensure image availability: %w", err)
+	}
+
+	resp, err := createGoTestContainer(ctx, cli, config, containerName, absLogsDir, goTestCmd)
+	if err != nil {
+		return fmt.Errorf("failed to create container: %w", err)
+	}
+
+	if config.Verbose {
+		log.Printf("Created container: %s", resp.ID)
+	}
+
+	if err := cli.ContainerStart(ctx, resp.ID, container.StartOptions{}); err != nil {
+		return fmt.Errorf("failed to start container: %w", err)
+	}
+
+	log.Printf("Starting test: %s", config.TestPattern)
+
+	// Start stats collection for container resource monitoring (if enabled)
+	var statsCollector *StatsCollector
+	if config.Stats {
+		var err error
+		statsCollector, err = NewStatsCollector()
+		if err != nil {
+			if config.Verbose {
+				log.Printf("Warning: failed to create stats collector: %v", err)
+			}
+			statsCollector = nil
+		}
+
+		if statsCollector != nil {
+			defer statsCollector.Close()
+
+			// Start stats collection immediately - no need for complex retry logic
+			// The new implementation monitors Docker events and will catch containers as they start
+			if err := statsCollector.StartCollection(ctx, runID, config.Verbose); err != nil {
+				if config.Verbose {
+					log.Printf("Warning: failed to start stats collection: %v", err)
+				}
+			}
+			defer statsCollector.StopCollection()
+		}
+	}
+
+	exitCode, err := streamAndWait(ctx, cli, resp.ID)
+
+	// Ensure all containers have finished and logs are flushed before extracting artifacts
+	if waitErr := waitForContainerFinalization(ctx, cli, resp.ID, config.Verbose); waitErr != nil && config.Verbose {
+		log.Printf("Warning: failed to wait for container finalization: %v", waitErr)
+	}
+
+	// Extract artifacts from test containers before cleanup
+	if err := extractArtifactsFromContainers(ctx, resp.ID, logsDir, config.Verbose); err != nil && config.Verbose {
+		log.Printf("Warning: failed to extract artifacts from containers: %v", err)
+	}
+
+	// Always list control files regardless of test outcome
+	listControlFiles(logsDir)
+
+	// Print stats summary and check memory limits if enabled
+	if config.Stats && statsCollector != nil {
+		violations := statsCollector.PrintSummaryAndCheckLimits(config.HSMemoryLimit, config.TSMemoryLimit)
+		if len(violations) > 0 {
+			log.Printf("MEMORY LIMIT VIOLATIONS DETECTED:")
+			log.Printf("=================================")
+			for _, violation := range violations {
+				log.Printf("Container %s exceeded memory limit: %.1f MB > %.1f MB",
+					violation.ContainerName, violation.MaxMemoryMB, violation.LimitMB)
+			}
+
+			return fmt.Errorf("test failed: %d container(s) exceeded memory limits", len(violations))
+		}
+	}
+
+	shouldCleanup := config.CleanAfter && (!config.KeepOnFailure || exitCode == 0)
+	if shouldCleanup {
+		if config.Verbose {
+			log.Printf("Running post-test cleanup...")
+		}
+		if cleanErr := cleanupAfterTest(ctx, cli, resp.ID); cleanErr != nil && config.Verbose {
+			log.Printf("Warning: post-test cleanup failed: %v", cleanErr)
+		}
+	}
+
+	if err != nil {
+		return fmt.Errorf("test execution failed: %w", err)
+	}
+
+	if exitCode != 0 {
+		return fmt.Errorf("%w: exit code %d", ErrTestFailed, exitCode)
+	}
+
+	log.Printf("Test completed successfully!")
+
+	return nil
+}
+
+// buildGoTestCommand constructs the go test command arguments.
+func buildGoTestCommand(config *RunConfig) []string {
+	cmd := []string{"go", "test", "./..."}
+
+	if config.TestPattern != "" {
+		cmd = append(cmd, "-run", config.TestPattern)
+	}
+
+	if config.FailFast {
+		cmd = append(cmd, "-failfast")
+	}
+
+	cmd = append(cmd, "-timeout", config.Timeout.String())
+	cmd = append(cmd, "-v")
+
+	return cmd
+}
+
+// createGoTestContainer creates a Docker container configured for running integration tests.
+func createGoTestContainer(ctx context.Context, cli *client.Client, config *RunConfig, containerName, logsDir string, goTestCmd []string) (container.CreateResponse, error) {
+	pwd, err := os.Getwd()
+	if err != nil {
+		return container.CreateResponse{}, fmt.Errorf("failed to get working directory: %w", err)
+	}
+
+	projectRoot := findProjectRoot(pwd)
+
+	runID := dockertestutil.ExtractRunIDFromContainerName(containerName)
+
+	env := []string{
+		fmt.Sprintf("HEADSCALE_INTEGRATION_POSTGRES=%d", boolToInt(config.UsePostgres)),
+		"HEADSCALE_INTEGRATION_RUN_ID=" + runID,
+	}
+	containerConfig := &container.Config{
+		Image:      "golang:" + config.GoVersion,
+		Cmd:        goTestCmd,
+		Env:        env,
+		WorkingDir: projectRoot + "/integration",
+		Tty:        true,
+		Labels: map[string]string{
+			"hi.run-id":    runID,
+			"hi.test-type": "test-runner",
+		},
+	}
+
+	// Get the correct Docker socket path from the current context
+	dockerSocketPath := getDockerSocketPath()
+
+	if config.Verbose {
+		log.Printf("Using Docker socket: %s", dockerSocketPath)
+	}
+
+	hostConfig := &container.HostConfig{
+		AutoRemove: false, // We'll remove manually for better control
+		Binds: []string{
+			fmt.Sprintf("%s:%s", projectRoot, projectRoot),
+			dockerSocketPath + ":/var/run/docker.sock",
+			logsDir + ":/tmp/control",
+		},
+		Mounts: []mount.Mount{
+			{
+				Type:   mount.TypeVolume,
+				Source: "hs-integration-go-cache",
+				Target: "/go",
+			},
+		},
+	}
+
+	return cli.ContainerCreate(ctx, containerConfig, hostConfig, nil, nil, containerName)
+}
+
+// streamAndWait streams container output and waits for completion.
+func streamAndWait(ctx context.Context, cli *client.Client, containerID string) (int, error) {
+	out, err := cli.ContainerLogs(ctx, containerID, container.LogsOptions{
+		ShowStdout: true,
+		ShowStderr: true,
+		Follow:     true,
+	})
+	if err != nil {
+		return -1, fmt.Errorf("failed to get container logs: %w", err)
+	}
+	defer out.Close()
+
+	go func() {
+		_, _ = io.Copy(os.Stdout, out)
+	}()
+
+	statusCh, errCh := cli.ContainerWait(ctx, containerID, container.WaitConditionNotRunning)
+	select {
+	case err := <-errCh:
+		if err != nil {
+			return -1, fmt.Errorf("error waiting for container: %w", err)
+		}
+	case status := <-statusCh:
+		return int(status.StatusCode), nil
+	}
+
+	return -1, ErrUnexpectedContainerWait
+}
+
+// waitForContainerFinalization ensures all test containers have properly finished and flushed their output.
+func waitForContainerFinalization(ctx context.Context, cli *client.Client, testContainerID string, verbose bool) error {
+	// First, get all related test containers
+	containers, err := cli.ContainerList(ctx, container.ListOptions{All: true})
+	if err != nil {
+		return fmt.Errorf("failed to list containers: %w", err)
+	}
+
+	testContainers := getCurrentTestContainers(containers, testContainerID, verbose)
+
+	// Wait for all test containers to reach a final state
+	maxWaitTime := 10 * time.Second
+	checkInterval := 500 * time.Millisecond
+	timeout := time.After(maxWaitTime)
+	ticker := time.NewTicker(checkInterval)
+	defer ticker.Stop()
+
+	for {
+		select {
+		case <-timeout:
+			if verbose {
+				log.Printf("Timeout waiting for container finalization, proceeding with artifact extraction")
+			}
+			return nil
+		case <-ticker.C:
+			allFinalized := true
+
+			for _, testCont := range testContainers {
+				inspect, err := cli.ContainerInspect(ctx, testCont.ID)
+				if err != nil {
+					if verbose {
+						log.Printf("Warning: failed to inspect container %s: %v", testCont.name, err)
+					}
+					continue
+				}
+
+				// Check if container is in a final state
+				if !isContainerFinalized(inspect.State) {
+					allFinalized = false
+					if verbose {
+						log.Printf("Container %s still finalizing (state: %s)", testCont.name, inspect.State.Status)
+					}
+
+					break
+				}
+			}
+
+			if allFinalized {
+				if verbose {
+					log.Printf("All test containers finalized, ready for artifact extraction")
+				}
+				return nil
+			}
+		}
+	}
+}
+
+// isContainerFinalized checks if a container has reached a final state where logs are flushed.
+func isContainerFinalized(state *container.State) bool {
+	// Container is finalized if it's not running and has a finish time
+	return !state.Running && state.FinishedAt != ""
+}
+
+// findProjectRoot locates the project root by finding the directory containing go.mod.
+func findProjectRoot(startPath string) string {
+	current := startPath
+	for {
+		if _, err := os.Stat(filepath.Join(current, "go.mod")); err == nil {
+			return current
+		}
+		parent := filepath.Dir(current)
+		if parent == current {
+			return startPath
+		}
+		current = parent
+	}
+}
+
+// boolToInt converts a boolean to an integer for environment variables.
+func boolToInt(b bool) int {
+	if b {
+		return 1
+	}
+	return 0
+}
+
+// DockerContext represents Docker context information.
+type DockerContext struct {
+	Name      string                 `json:"Name"`
+	Metadata  map[string]interface{} `json:"Metadata"`
+	Endpoints map[string]interface{} `json:"Endpoints"`
+	Current   bool                   `json:"Current"`
+}
+
+// createDockerClient creates a Docker client with context detection.
+func createDockerClient() (*client.Client, error) {
+	contextInfo, err := getCurrentDockerContext()
+	if err != nil {
+		return client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
+	}
+
+	var clientOpts []client.Opt
+	clientOpts = append(clientOpts, client.WithAPIVersionNegotiation())
+
+	if contextInfo != nil {
+		if endpoints, ok := contextInfo.Endpoints["docker"]; ok {
+			if endpointMap, ok := endpoints.(map[string]interface{}); ok {
+				if host, ok := endpointMap["Host"].(string); ok {
+					if runConfig.Verbose {
+						log.Printf("Using Docker host from context '%s': %s", contextInfo.Name, host)
+					}
+					clientOpts = append(clientOpts, client.WithHost(host))
+				}
+			}
+		}
+	}
+
+	if len(clientOpts) == 1 {
+		clientOpts = append(clientOpts, client.FromEnv)
+	}
+
+	return client.NewClientWithOpts(clientOpts...)
+}
+
+// getCurrentDockerContext retrieves the current Docker context information.
+func getCurrentDockerContext() (*DockerContext, error) {
+	cmd := exec.Command("docker", "context", "inspect")
+	output, err := cmd.Output()
+	if err != nil {
+		return nil, fmt.Errorf("failed to get docker context: %w", err)
+	}
+
+	var contexts []DockerContext
+	if err := json.Unmarshal(output, &contexts); err != nil {
+		return nil, fmt.Errorf("failed to parse docker context: %w", err)
+	}
+
+	if len(contexts) > 0 {
+		return &contexts[0], nil
+	}
+
+	return nil, ErrNoDockerContext
+}
+
+// getDockerSocketPath returns the correct Docker socket path for the current context.
+func getDockerSocketPath() string {
+	// Always use the default socket path for mounting since Docker handles
+	// the translation to the actual socket (e.g., colima socket) internally
+	return "/var/run/docker.sock"
+}
+
+// checkImageAvailableLocally checks if the specified Docker image is available locally.
+func checkImageAvailableLocally(ctx context.Context, cli *client.Client, imageName string) (bool, error) {
+	_, _, err := cli.ImageInspectWithRaw(ctx, imageName)
+	if err != nil {
+		if client.IsErrNotFound(err) {
+			return false, nil
+		}
+		return false, fmt.Errorf("failed to inspect image %s: %w", imageName, err)
+	}
+
+	return true, nil
+}
+
+// ensureImageAvailable checks if the image is available locally first, then pulls if needed.
+func ensureImageAvailable(ctx context.Context, cli *client.Client, imageName string, verbose bool) error {
+	// First check if image is available locally
+	available, err := checkImageAvailableLocally(ctx, cli, imageName)
+	if err != nil {
+		return fmt.Errorf("failed to check local image availability: %w", err)
+	}
+
+	if available {
+		if verbose {
+			log.Printf("Image %s is available locally", imageName)
+		}
+		return nil
+	}
+
+	// Image not available locally, try to pull it
+	if verbose {
+		log.Printf("Image %s not found locally, pulling...", imageName)
+	}
+
+	reader, err := cli.ImagePull(ctx, imageName, image.PullOptions{})
+	if err != nil {
+		return fmt.Errorf("failed to pull image %s: %w", imageName, err)
+	}
+	defer reader.Close()
+
+	if verbose {
+		_, err = io.Copy(os.Stdout, reader)
+		if err != nil {
+			return fmt.Errorf("failed to read pull output: %w", err)
+		}
+	} else {
+		_, err = io.Copy(io.Discard, reader)
+		if err != nil {
+			return fmt.Errorf("failed to read pull output: %w", err)
+		}
+		log.Printf("Image %s pulled successfully", imageName)
+	}
+
+	return nil
+}
+
+// listControlFiles displays the headscale test artifacts created in the control logs directory.
+func listControlFiles(logsDir string) {
+	entries, err := os.ReadDir(logsDir)
+	if err != nil {
+		log.Printf("Logs directory: %s", logsDir)
+		return
+	}
+
+	var logFiles []string
+	var dataFiles []string
+	var dataDirs []string
+
+	for _, entry := range entries {
+		name := entry.Name()
+		// Only show headscale (hs-*) files and directories
+		if !strings.HasPrefix(name, "hs-") {
+			continue
+		}
+
+		if entry.IsDir() {
+			// Include directories (pprof, mapresponses)
+			if strings.Contains(name, "-pprof") || strings.Contains(name, "-mapresponses") {
+				dataDirs = append(dataDirs, name)
+			}
+		} else {
+			// Include files
+			switch {
+			case strings.HasSuffix(name, ".stderr.log") || strings.HasSuffix(name, ".stdout.log"):
+				logFiles = append(logFiles, name)
+			case strings.HasSuffix(name, ".db"):
+				dataFiles = append(dataFiles, name)
+			}
+		}
+	}
+
+	log.Printf("Test artifacts saved to: %s", logsDir)
+
+	if len(logFiles) > 0 {
+		log.Printf("Headscale logs:")
+		for _, file := range logFiles {
+			log.Printf("  %s", file)
+		}
+	}
+
+	if len(dataFiles) > 0 || len(dataDirs) > 0 {
+		log.Printf("Headscale data:")
+		for _, file := range dataFiles {
+			log.Printf("  %s", file)
+		}
+		for _, dir := range dataDirs {
+			log.Printf("  %s/", dir)
+		}
+	}
+}
+
+// extractArtifactsFromContainers collects container logs and files from the specific test run.
+func extractArtifactsFromContainers(ctx context.Context, testContainerID, logsDir string, verbose bool) error {
+	cli, err := createDockerClient()
+	if err != nil {
+		return fmt.Errorf("failed to create Docker client: %w", err)
+	}
+	defer cli.Close()
+
+	// List all containers
+	containers, err := cli.ContainerList(ctx, container.ListOptions{All: true})
+	if err != nil {
+		return fmt.Errorf("failed to list containers: %w", err)
+	}
+
+	// Get containers from the specific test run
+	currentTestContainers := getCurrentTestContainers(containers, testContainerID, verbose)
+
+	extractedCount := 0
+	for _, cont := range currentTestContainers {
+		// Extract container logs and tar files
+		if err := extractContainerArtifacts(ctx, cli, cont.ID, cont.name, logsDir, verbose); err != nil {
+			if verbose {
+				log.Printf("Warning: failed to extract artifacts from container %s (%s): %v", cont.name, cont.ID[:12], err)
+			}
+		} else {
+			if verbose {
+				log.Printf("Extracted artifacts from container %s (%s)", cont.name, cont.ID[:12])
+			}
+			extractedCount++
+		}
+	}
+
+	if verbose && extractedCount > 0 {
+		log.Printf("Extracted artifacts from %d containers", extractedCount)
+	}
+
+	return nil
+}
+
+// testContainer represents a container from the current test run.
+type testContainer struct {
+	ID   string
+	name string
+}
+
+// getCurrentTestContainers filters containers to only include those from the current test run.
+func getCurrentTestContainers(containers []container.Summary, testContainerID string, verbose bool) []testContainer {
+	var testRunContainers []testContainer
+
+	// Find the test container to get its run ID label
+	var runID string
+	for _, cont := range containers {
+		if cont.ID == testContainerID {
+			if cont.Labels != nil {
+				runID = cont.Labels["hi.run-id"]
+			}
+			break
+		}
+	}
+
+	if runID == "" {
+		log.Printf("Error: test container %s missing required hi.run-id label", testContainerID[:12])
+		return testRunContainers
+	}
+
+	if verbose {
+		log.Printf("Looking for containers with run ID: %s", runID)
+	}
+
+	// Find all containers with the same run ID
+	for _, cont := range containers {
+		for _, name := range cont.Names {
+			containerName := strings.TrimPrefix(name, "/")
+			if strings.HasPrefix(containerName, "hs-") || strings.HasPrefix(containerName, "ts-") {
+				// Check if container has matching run ID label
+				if cont.Labels != nil && cont.Labels["hi.run-id"] == runID {
+					testRunContainers = append(testRunContainers, testContainer{
+						ID:   cont.ID,
+						name: containerName,
+					})
+					if verbose {
+						log.Printf("Including container %s (run ID: %s)", containerName, runID)
+					}
+				}
+
+				break
+			}
+		}
+	}
+
+	return testRunContainers
+}
+
+// extractContainerArtifacts saves logs and tar files from a container.
+func extractContainerArtifacts(ctx context.Context, cli *client.Client, containerID, containerName, logsDir string, verbose bool) error {
+	// Ensure the logs directory exists
+	if err := os.MkdirAll(logsDir, 0o755); err != nil {
+		return fmt.Errorf("failed to create logs directory: %w", err)
+	}
+
+	// Extract container logs
+	if err := extractContainerLogs(ctx, cli, containerID, containerName, logsDir, verbose); err != nil {
+		return fmt.Errorf("failed to extract logs: %w", err)
+	}
+
+	// Extract tar files for headscale containers only
+	if strings.HasPrefix(containerName, "hs-") {
+		if err := extractContainerFiles(ctx, cli, containerID, containerName, logsDir, verbose); err != nil {
+			if verbose {
+				log.Printf("Warning: failed to extract files from %s: %v", containerName, err)
+			}
+			// Don't fail the whole extraction if files are missing
+		}
+	}
+
+	return nil
+}
+
+// extractContainerLogs saves the stdout and stderr logs from a container to files.
+func extractContainerLogs(ctx context.Context, cli *client.Client, containerID, containerName, logsDir string, verbose bool) error {
+	// Get container logs
+	logReader, err := cli.ContainerLogs(ctx, containerID, container.LogsOptions{
+		ShowStdout: true,
+		ShowStderr: true,
+		Timestamps: false,
+		Follow:     false,
+		Tail:       "all",
+	})
+	if err != nil {
+		return fmt.Errorf("failed to get container logs: %w", err)
+	}
+	defer logReader.Close()
+
+	// Create log files following the headscale naming convention
+	stdoutPath := filepath.Join(logsDir, containerName+".stdout.log")
+	stderrPath := filepath.Join(logsDir, containerName+".stderr.log")
+
+	// Create buffers to capture stdout and stderr separately
+	var stdoutBuf, stderrBuf bytes.Buffer
+
+	// Demultiplex the Docker logs stream to separate stdout and stderr
+	_, err = stdcopy.StdCopy(&stdoutBuf, &stderrBuf, logReader)
+	if err != nil {
+		return fmt.Errorf("failed to demultiplex container logs: %w", err)
+	}
+
+	// Write stdout logs
+	if err := os.WriteFile(stdoutPath, stdoutBuf.Bytes(), 0o644); err != nil {
+		return fmt.Errorf("failed to write stdout log: %w", err)
+	}
+
+	// Write stderr logs
+	if err := os.WriteFile(stderrPath, stderrBuf.Bytes(), 0o644); err != nil {
+		return fmt.Errorf("failed to write stderr log: %w", err)
+	}
+
+	if verbose {
+		log.Printf("Saved logs for %s: %s, %s", containerName, stdoutPath, stderrPath)
+	}
+
+	return nil
+}
+
+// extractContainerFiles extracts database file and directories from headscale containers.
+// Note: The actual file extraction is now handled by the integration tests themselves
+// via SaveProfile, SaveMapResponses, and SaveDatabase functions in hsic.go.
+func extractContainerFiles(ctx context.Context, cli *client.Client, containerID, containerName, logsDir string, verbose bool) error {
+	// Files are now extracted directly by the integration tests
+	// This function is kept for potential future use or other file types
+	return nil
+}
+
+// logExtractionError logs extraction errors with appropriate level based on error type.
+func logExtractionError(artifactType, containerName string, err error, verbose bool) {
+	if errors.Is(err, ErrFileNotFoundInTar) {
+		// File not found is expected and only logged in verbose mode
+		if verbose {
+			log.Printf("No %s found in container %s", artifactType, containerName)
+		}
+	} else {
+		// Other errors are actual failures and should be logged as warnings
+		log.Printf("Warning: failed to extract %s from %s: %v", artifactType, containerName, err)
+	}
+}
+
+// extractSingleFile copies a single file from a container.
+func extractSingleFile(ctx context.Context, cli *client.Client, containerID, sourcePath, fileName, logsDir string, verbose bool) error {
+	tarReader, _, err := cli.CopyFromContainer(ctx, containerID, sourcePath)
+	if err != nil {
+		return fmt.Errorf("failed to copy %s from container: %w", sourcePath, err)
+	}
+	defer tarReader.Close()
+
+	// Extract the single file from the tar
+	filePath := filepath.Join(logsDir, fileName)
+	if err := extractFileFromTar(tarReader, filepath.Base(sourcePath), filePath); err != nil {
+		return fmt.Errorf("failed to extract file from tar: %w", err)
+	}
+
+	if verbose {
+		log.Printf("Extracted %s from %s", fileName, containerID[:12])
+	}
+
+	return nil
+}
+
+// extractDirectory copies a directory from a container and extracts its contents.
+func extractDirectory(ctx context.Context, cli *client.Client, containerID, sourcePath, dirName, logsDir string, verbose bool) error {
+	tarReader, _, err := cli.CopyFromContainer(ctx, containerID, sourcePath)
+	if err != nil {
+		return fmt.Errorf("failed to copy %s from container: %w", sourcePath, err)
+	}
+	defer tarReader.Close()
+
+	// Create target directory
+	targetDir := filepath.Join(logsDir, dirName)
+	if err := os.MkdirAll(targetDir, 0o755); err != nil {
+		return fmt.Errorf("failed to create directory %s: %w", targetDir, err)
+	}
+
+	// Extract the directory from the tar
+	if err := extractDirectoryFromTar(tarReader, targetDir); err != nil {
+		return fmt.Errorf("failed to extract directory from tar: %w", err)
+	}
+
+	if verbose {
+		log.Printf("Extracted %s/ from %s", dirName, containerID[:12])
+	}
+
+	return nil
+}

cmd/hi/doctor.go

@@ -0,0 +1,374 @@
+package main
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"log"
+	"os/exec"
+	"strings"
+)
+
+var ErrSystemChecksFailed = errors.New("system checks failed")
+
+// DoctorResult represents the result of a single health check.
+type DoctorResult struct {
+	Name        string
+	Status      string // "PASS", "FAIL", "WARN"
+	Message     string
+	Suggestions []string
+}
+
+// runDoctorCheck performs comprehensive pre-flight checks for integration testing.
+func runDoctorCheck(ctx context.Context) error {
+	results := []DoctorResult{}
+
+	// Check 1: Docker binary availability
+	results = append(results, checkDockerBinary())
+
+	// Check 2: Docker daemon connectivity
+	dockerResult := checkDockerDaemon(ctx)
+	results = append(results, dockerResult)
+
+	// If Docker is available, run additional checks
+	if dockerResult.Status == "PASS" {
+		results = append(results, checkDockerContext(ctx))
+		results = append(results, checkDockerSocket(ctx))
+		results = append(results, checkGolangImage(ctx))
+	}
+
+	// Check 3: Go installation
+	results = append(results, checkGoInstallation())
+
+	// Check 4: Git repository
+	results = append(results, checkGitRepository())
+
+	// Check 5: Required files
+	results = append(results, checkRequiredFiles())
+
+	// Display results
+	displayDoctorResults(results)
+
+	// Return error if any critical checks failed
+	for _, result := range results {
+		if result.Status == "FAIL" {
+			return fmt.Errorf("%w - see details above", ErrSystemChecksFailed)
+		}
+	}
+
+	log.Printf("✅ All system checks passed - ready to run integration tests!")
+
+	return nil
+}
+
+// checkDockerBinary verifies Docker binary is available.
+func checkDockerBinary() DoctorResult {
+	_, err := exec.LookPath("docker")
+	if err != nil {
+		return DoctorResult{
+			Name:    "Docker Binary",
+			Status:  "FAIL",
+			Message: "Docker binary not found in PATH",
+			Suggestions: []string{
+				"Install Docker: https://docs.docker.com/get-docker/",
+				"For macOS: consider using colima or Docker Desktop",
+				"Ensure docker is in your PATH",
+			},
+		}
+	}
+
+	return DoctorResult{
+		Name:    "Docker Binary",
+		Status:  "PASS",
+		Message: "Docker binary found",
+	}
+}
+
+// checkDockerDaemon verifies Docker daemon is running and accessible.
+func checkDockerDaemon(ctx context.Context) DoctorResult {
+	cli, err := createDockerClient()
+	if err != nil {
+		return DoctorResult{
+			Name:    "Docker Daemon",
+			Status:  "FAIL",
+			Message: fmt.Sprintf("Cannot create Docker client: %v", err),
+			Suggestions: []string{
+				"Start Docker daemon/service",
+				"Check Docker Desktop is running (if using Docker Desktop)",
+				"For colima: run 'colima start'",
+				"Verify DOCKER_HOST environment variable if set",
+			},
+		}
+	}
+	defer cli.Close()
+
+	_, err = cli.Ping(ctx)
+	if err != nil {
+		return DoctorResult{
+			Name:    "Docker Daemon",
+			Status:  "FAIL",
+			Message: fmt.Sprintf("Cannot ping Docker daemon: %v", err),
+			Suggestions: []string{
+				"Ensure Docker daemon is running",
+				"Check Docker socket permissions",
+				"Try: docker info",
+			},
+		}
+	}
+
+	return DoctorResult{
+		Name:    "Docker Daemon",
+		Status:  "PASS",
+		Message: "Docker daemon is running and accessible",
+	}
+}
+
+// checkDockerContext verifies Docker context configuration.
+func checkDockerContext(_ context.Context) DoctorResult {
+	contextInfo, err := getCurrentDockerContext()
+	if err != nil {
+		return DoctorResult{
+			Name:    "Docker Context",
+			Status:  "WARN",
+			Message: "Could not detect Docker context, using default settings",
+			Suggestions: []string{
+				"Check: docker context ls",
+				"Consider setting up a specific context if needed",
+			},
+		}
+	}
+
+	if contextInfo == nil {
+		return DoctorResult{
+			Name:    "Docker Context",
+			Status:  "PASS",
+			Message: "Using default Docker context",
+		}
+	}
+
+	return DoctorResult{
+		Name:    "Docker Context",
+		Status:  "PASS",
+		Message: "Using Docker context: " + contextInfo.Name,
+	}
+}
+
+// checkDockerSocket verifies Docker socket accessibility.
+func checkDockerSocket(ctx context.Context) DoctorResult {
+	cli, err := createDockerClient()
+	if err != nil {
+		return DoctorResult{
+			Name:    "Docker Socket",
+			Status:  "FAIL",
+			Message: fmt.Sprintf("Cannot access Docker socket: %v", err),
+			Suggestions: []string{
+				"Check Docker socket permissions",
+				"Add user to docker group: sudo usermod -aG docker $USER",
+				"For colima: ensure socket is accessible",
+			},
+		}
+	}
+	defer cli.Close()
+
+	info, err := cli.Info(ctx)
+	if err != nil {
+		return DoctorResult{
+			Name:    "Docker Socket",
+			Status:  "FAIL",
+			Message: fmt.Sprintf("Cannot get Docker info: %v", err),
+			Suggestions: []string{
+				"Check Docker daemon status",
+				"Verify socket permissions",
+			},
+		}
+	}
+
+	return DoctorResult{
+		Name:    "Docker Socket",
+		Status:  "PASS",
+		Message: fmt.Sprintf("Docker socket accessible (Server: %s)", info.ServerVersion),
+	}
+}
+
+// checkGolangImage verifies the golang Docker image is available locally or can be pulled.
+func checkGolangImage(ctx context.Context) DoctorResult {
+	cli, err := createDockerClient()
+	if err != nil {
+		return DoctorResult{
+			Name:    "Golang Image",
+			Status:  "FAIL",
+			Message: "Cannot create Docker client for image check",
+		}
+	}
+	defer cli.Close()
+
+	goVersion := detectGoVersion()
+	imageName := "golang:" + goVersion
+
+	// First check if image is available locally
+	available, err := checkImageAvailableLocally(ctx, cli, imageName)
+	if err != nil {
+		return DoctorResult{
+			Name:    "Golang Image",
+			Status:  "FAIL",
+			Message: fmt.Sprintf("Cannot check golang image %s: %v", imageName, err),
+			Suggestions: []string{
+				"Check Docker daemon status",
+				"Try: docker images | grep golang",
+			},
+		}
+	}
+
+	if available {
+		return DoctorResult{
+			Name:    "Golang Image",
+			Status:  "PASS",
+			Message: fmt.Sprintf("Golang image %s is available locally", imageName),
+		}
+	}
+
+	// Image not available locally, try to pull it
+	err = ensureImageAvailable(ctx, cli, imageName, false)
+	if err != nil {
+		return DoctorResult{
+			Name:    "Golang Image",
+			Status:  "FAIL",
+			Message: fmt.Sprintf("Golang image %s not available locally and cannot pull: %v", imageName, err),
+			Suggestions: []string{
+				"Check internet connectivity",
+				"Verify Docker Hub access",
+				"Try: docker pull " + imageName,
+				"Or run tests offline if image was pulled previously",
+			},
+		}
+	}
+
+	return DoctorResult{
+		Name:    "Golang Image",
+		Status:  "PASS",
+		Message: fmt.Sprintf("Golang image %s is now available", imageName),
+	}
+}
+
+// checkGoInstallation verifies Go is installed and working.
+func checkGoInstallation() DoctorResult {
+	_, err := exec.LookPath("go")
+	if err != nil {
+		return DoctorResult{
+			Name:    "Go Installation",
+			Status:  "FAIL",
+			Message: "Go binary not found in PATH",
+			Suggestions: []string{
+				"Install Go: https://golang.org/dl/",
+				"Ensure go is in your PATH",
+			},
+		}
+	}
+
+	cmd := exec.Command("go", "version")
+	output, err := cmd.Output()
+	if err != nil {
+		return DoctorResult{
+			Name:    "Go Installation",
+			Status:  "FAIL",
+			Message: fmt.Sprintf("Cannot get Go version: %v", err),
+		}
+	}
+
+	version := strings.TrimSpace(string(output))
+
+	return DoctorResult{
+		Name:    "Go Installation",
+		Status:  "PASS",
+		Message: version,
+	}
+}
+
+// checkGitRepository verifies we're in a git repository.
+func checkGitRepository() DoctorResult {
+	cmd := exec.Command("git", "rev-parse", "--git-dir")
+	err := cmd.Run()
+	if err != nil {
+		return DoctorResult{
+			Name:    "Git Repository",
+			Status:  "FAIL",
+			Message: "Not in a Git repository",
+			Suggestions: []string{
+				"Run from within the headscale git repository",
+				"Clone the repository: git clone https://github.com/juanfont/headscale.git",
+			},
+		}
+	}
+
+	return DoctorResult{
+		Name:    "Git Repository",
+		Status:  "PASS",
+		Message: "Running in Git repository",
+	}
+}
+
+// checkRequiredFiles verifies required files exist.
+func checkRequiredFiles() DoctorResult {
+	requiredFiles := []string{
+		"go.mod",
+		"integration/",
+		"cmd/hi/",
+	}
+
+	var missingFiles []string
+	for _, file := range requiredFiles {
+		cmd := exec.Command("test", "-e", file)
+		if err := cmd.Run(); err != nil {
+			missingFiles = append(missingFiles, file)
+		}
+	}
+
+	if len(missingFiles) > 0 {
+		return DoctorResult{
+			Name:    "Required Files",
+			Status:  "FAIL",
+			Message: "Missing required files: " + strings.Join(missingFiles, ", "),
+			Suggestions: []string{
+				"Ensure you're in the headscale project root directory",
+				"Check that integration/ directory exists",
+				"Verify this is a complete headscale repository",
+			},
+		}
+	}
+
+	return DoctorResult{
+		Name:    "Required Files",
+		Status:  "PASS",
+		Message: "All required files found",
+	}
+}
+
+// displayDoctorResults shows the results in a formatted way.
+func displayDoctorResults(results []DoctorResult) {
+	log.Printf("🔍 System Health Check Results")
+	log.Printf("================================")
+
+	for _, result := range results {
+		var icon string
+		switch result.Status {
+		case "PASS":
+			icon = "✅"
+		case "WARN":
+			icon = "⚠️"
+		case "FAIL":
+			icon = "❌"
+		default:
+			icon = "❓"
+		}
+
+		log.Printf("%s %s: %s", icon, result.Name, result.Message)
+
+		if len(result.Suggestions) > 0 {
+			for _, suggestion := range result.Suggestions {
+				log.Printf("   💡 %s", suggestion)
+			}
+		}
+	}
+
+	log.Printf("================================")
+}

cmd/hi/main.go

@@ -0,0 +1,93 @@
+package main
+
+import (
+	"context"
+	"os"
+
+	"github.com/creachadair/command"
+	"github.com/creachadair/flax"
+)
+
+var runConfig RunConfig
+
+func main() {
+	root := command.C{
+		Name: "hi",
+		Help: "Headscale Integration test runner",
+		Commands: []*command.C{
+			{
+				Name:     "run",
+				Help:     "Run integration tests",
+				Usage:    "run [test-pattern] [flags]",
+				SetFlags: command.Flags(flax.MustBind, &runConfig),
+				Run:      runIntegrationTest,
+			},
+			{
+				Name: "doctor",
+				Help: "Check system requirements for running integration tests",
+				Run: func(env *command.Env) error {
+					return runDoctorCheck(env.Context())
+				},
+			},
+			{
+				Name: "clean",
+				Help: "Clean Docker resources",
+				Commands: []*command.C{
+					{
+						Name: "networks",
+						Help: "Prune unused Docker networks",
+						Run: func(env *command.Env) error {
+							return pruneDockerNetworks(env.Context())
+						},
+					},
+					{
+						Name: "images",
+						Help: "Clean old test images",
+						Run: func(env *command.Env) error {
+							return cleanOldImages(env.Context())
+						},
+					},
+					{
+						Name: "containers",
+						Help: "Kill all test containers",
+						Run: func(env *command.Env) error {
+							return killTestContainers(env.Context())
+						},
+					},
+					{
+						Name: "cache",
+						Help: "Clean Go module cache volume",
+						Run: func(env *command.Env) error {
+							return cleanCacheVolume(env.Context())
+						},
+					},
+					{
+						Name: "all",
+						Help: "Run all cleanup operations",
+						Run: func(env *command.Env) error {
+							return cleanAll(env.Context())
+						},
+					},
+				},
+			},
+			command.HelpCommand(nil),
+		},
+	}
+
+	env := root.NewEnv(nil).MergeFlags(true)
+	command.RunOrFail(env, os.Args[1:])
+}
+
+func cleanAll(ctx context.Context) error {
+	if err := killTestContainers(ctx); err != nil {
+		return err
+	}
+	if err := pruneDockerNetworks(ctx); err != nil {
+		return err
+	}
+	if err := cleanOldImages(ctx); err != nil {
+		return err
+	}
+
+	return cleanCacheVolume(ctx)
+}

cmd/hi/run.go

@@ -0,0 +1,125 @@
+package main
+
+import (
+	"errors"
+	"fmt"
+	"log"
+	"os"
+	"path/filepath"
+	"time"
+
+	"github.com/creachadair/command"
+)
+
+var ErrTestPatternRequired = errors.New("test pattern is required as first argument or use --test flag")
+
+type RunConfig struct {
+	TestPattern   string        `flag:"test,Test pattern to run"`
+	Timeout       time.Duration `flag:"timeout,default=120m,Test timeout"`
+	FailFast      bool          `flag:"failfast,default=true,Stop on first test failure"`
+	UsePostgres   bool          `flag:"postgres,default=false,Use PostgreSQL instead of SQLite"`
+	GoVersion     string        `flag:"go-version,Go version to use (auto-detected from go.mod)"`
+	CleanBefore   bool          `flag:"clean-before,default=true,Clean resources before test"`
+	CleanAfter    bool          `flag:"clean-after,default=true,Clean resources after test"`
+	KeepOnFailure bool          `flag:"keep-on-failure,default=false,Keep containers on test failure"`
+	LogsDir       string        `flag:"logs-dir,default=control_logs,Control logs directory"`
+	Verbose       bool          `flag:"verbose,default=false,Verbose output"`
+	Stats         bool          `flag:"stats,default=false,Collect and display container resource usage statistics"`
+	HSMemoryLimit float64       `flag:"hs-memory-limit,default=0,Fail test if any Headscale container exceeds this memory limit in MB (0 = disabled)"`
+	TSMemoryLimit float64       `flag:"ts-memory-limit,default=0,Fail test if any Tailscale container exceeds this memory limit in MB (0 = disabled)"`
+}
+
+// runIntegrationTest executes the integration test workflow.
+func runIntegrationTest(env *command.Env) error {
+	args := env.Args
+	if len(args) > 0 && runConfig.TestPattern == "" {
+		runConfig.TestPattern = args[0]
+	}
+
+	if runConfig.TestPattern == "" {
+		return ErrTestPatternRequired
+	}
+
+	if runConfig.GoVersion == "" {
+		runConfig.GoVersion = detectGoVersion()
+	}
+
+	// Run pre-flight checks
+	if runConfig.Verbose {
+		log.Printf("Running pre-flight system checks...")
+	}
+	if err := runDoctorCheck(env.Context()); err != nil {
+		return fmt.Errorf("pre-flight checks failed: %w", err)
+	}
+
+	if runConfig.Verbose {
+		log.Printf("Running test: %s", runConfig.TestPattern)
+		log.Printf("Go version: %s", runConfig.GoVersion)
+		log.Printf("Timeout: %s", runConfig.Timeout)
+		log.Printf("Use PostgreSQL: %t", runConfig.UsePostgres)
+	}
+
+	return runTestContainer(env.Context(), &runConfig)
+}
+
+// detectGoVersion reads the Go version from go.mod file.
+func detectGoVersion() string {
+	goModPath := filepath.Join("..", "..", "go.mod")
+
+	if _, err := os.Stat("go.mod"); err == nil {
+		goModPath = "go.mod"
+	} else if _, err := os.Stat("../../go.mod"); err == nil {
+		goModPath = "../../go.mod"
+	}
+
+	content, err := os.ReadFile(goModPath)
+	if err != nil {
+		return "1.25"
+	}
+
+	lines := splitLines(string(content))
+	for _, line := range lines {
+		if len(line) > 3 && line[:3] == "go " {
+			version := line[3:]
+			if idx := indexOf(version, " "); idx != -1 {
+				version = version[:idx]
+			}
+
+			return version
+		}
+	}
+
+	return "1.25"
+}
+
+// splitLines splits a string into lines without using strings.Split.
+func splitLines(s string) []string {
+	var lines []string
+	var current string
+
+	for _, char := range s {
+		if char == '\n' {
+			lines = append(lines, current)
+			current = ""
+		} else {
+			current += string(char)
+		}
+	}
+
+	if current != "" {
+		lines = append(lines, current)
+	}
+
+	return lines
+}
+
+// indexOf finds the first occurrence of substr in s.
+func indexOf(s, substr string) int {
+	for i := 0; i <= len(s)-len(substr); i++ {
+		if s[i:i+len(substr)] == substr {
+			return i
+		}
+	}
+
+	return -1
+}

cmd/hi/stats.go

@@ -0,0 +1,471 @@
+package main
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"log"
+	"sort"
+	"strings"
+	"sync"
+	"time"
+
+	"github.com/docker/docker/api/types"
+	"github.com/docker/docker/api/types/container"
+	"github.com/docker/docker/api/types/events"
+	"github.com/docker/docker/api/types/filters"
+	"github.com/docker/docker/client"
+)
+
+// ContainerStats represents statistics for a single container.
+type ContainerStats struct {
+	ContainerID   string
+	ContainerName string
+	Stats         []StatsSample
+	mutex         sync.RWMutex
+}
+
+// StatsSample represents a single stats measurement.
+type StatsSample struct {
+	Timestamp time.Time
+	CPUUsage  float64 // CPU usage percentage
+	MemoryMB  float64 // Memory usage in MB
+}
+
+// StatsCollector manages collection of container statistics.
+type StatsCollector struct {
+	client            *client.Client
+	containers        map[string]*ContainerStats
+	stopChan          chan struct{}
+	wg                sync.WaitGroup
+	mutex             sync.RWMutex
+	collectionStarted bool
+}
+
+// NewStatsCollector creates a new stats collector instance.
+func NewStatsCollector() (*StatsCollector, error) {
+	cli, err := createDockerClient()
+	if err != nil {
+		return nil, fmt.Errorf("failed to create Docker client: %w", err)
+	}
+
+	return &StatsCollector{
+		client:     cli,
+		containers: make(map[string]*ContainerStats),
+		stopChan:   make(chan struct{}),
+	}, nil
+}
+
+// StartCollection begins monitoring all containers and collecting stats for hs- and ts- containers with matching run ID.
+func (sc *StatsCollector) StartCollection(ctx context.Context, runID string, verbose bool) error {
+	sc.mutex.Lock()
+	defer sc.mutex.Unlock()
+
+	if sc.collectionStarted {
+		return errors.New("stats collection already started")
+	}
+
+	sc.collectionStarted = true
+
+	// Start monitoring existing containers
+	sc.wg.Add(1)
+	go sc.monitorExistingContainers(ctx, runID, verbose)
+
+	// Start Docker events monitoring for new containers
+	sc.wg.Add(1)
+	go sc.monitorDockerEvents(ctx, runID, verbose)
+
+	if verbose {
+		log.Printf("Started container monitoring for run ID %s", runID)
+	}
+
+	return nil
+}
+
+// StopCollection stops all stats collection.
+func (sc *StatsCollector) StopCollection() {
+	// Check if already stopped without holding lock
+	sc.mutex.RLock()
+	if !sc.collectionStarted {
+		sc.mutex.RUnlock()
+		return
+	}
+	sc.mutex.RUnlock()
+
+	// Signal stop to all goroutines
+	close(sc.stopChan)
+
+	// Wait for all goroutines to finish
+	sc.wg.Wait()
+
+	// Mark as stopped
+	sc.mutex.Lock()
+	sc.collectionStarted = false
+	sc.mutex.Unlock()
+}
+
+// monitorExistingContainers checks for existing containers that match our criteria.
+func (sc *StatsCollector) monitorExistingContainers(ctx context.Context, runID string, verbose bool) {
+	defer sc.wg.Done()
+
+	containers, err := sc.client.ContainerList(ctx, container.ListOptions{})
+	if err != nil {
+		if verbose {
+			log.Printf("Failed to list existing containers: %v", err)
+		}
+		return
+	}
+
+	for _, cont := range containers {
+		if sc.shouldMonitorContainer(cont, runID) {
+			sc.startStatsForContainer(ctx, cont.ID, cont.Names[0], verbose)
+		}
+	}
+}
+
+// monitorDockerEvents listens for container start events and begins monitoring relevant containers.
+func (sc *StatsCollector) monitorDockerEvents(ctx context.Context, runID string, verbose bool) {
+	defer sc.wg.Done()
+
+	filter := filters.NewArgs()
+	filter.Add("type", "container")
+	filter.Add("event", "start")
+
+	eventOptions := events.ListOptions{
+		Filters: filter,
+	}
+
+	events, errs := sc.client.Events(ctx, eventOptions)
+
+	for {
+		select {
+		case <-sc.stopChan:
+			return
+		case <-ctx.Done():
+			return
+		case event := <-events:
+			if event.Type == "container" && event.Action == "start" {
+				// Get container details
+				containerInfo, err := sc.client.ContainerInspect(ctx, event.ID)
+				if err != nil {
+					continue
+				}
+
+				// Convert to types.Container format for consistency
+				cont := types.Container{
+					ID:     containerInfo.ID,
+					Names:  []string{containerInfo.Name},
+					Labels: containerInfo.Config.Labels,
+				}
+
+				if sc.shouldMonitorContainer(cont, runID) {
+					sc.startStatsForContainer(ctx, cont.ID, cont.Names[0], verbose)
+				}
+			}
+		case err := <-errs:
+			if verbose {
+				log.Printf("Error in Docker events stream: %v", err)
+			}
+			return
+		}
+	}
+}
+
+// shouldMonitorContainer determines if a container should be monitored.
+func (sc *StatsCollector) shouldMonitorContainer(cont types.Container, runID string) bool {
+	// Check if it has the correct run ID label
+	if cont.Labels == nil || cont.Labels["hi.run-id"] != runID {
+		return false
+	}
+
+	// Check if it's an hs- or ts- container
+	for _, name := range cont.Names {
+		containerName := strings.TrimPrefix(name, "/")
+		if strings.HasPrefix(containerName, "hs-") || strings.HasPrefix(containerName, "ts-") {
+			return true
+		}
+	}
+
+	return false
+}
+
+// startStatsForContainer begins stats collection for a specific container.
+func (sc *StatsCollector) startStatsForContainer(ctx context.Context, containerID, containerName string, verbose bool) {
+	containerName = strings.TrimPrefix(containerName, "/")
+
+	sc.mutex.Lock()
+	// Check if we're already monitoring this container
+	if _, exists := sc.containers[containerID]; exists {
+		sc.mutex.Unlock()
+		return
+	}
+
+	sc.containers[containerID] = &ContainerStats{
+		ContainerID:   containerID,
+		ContainerName: containerName,
+		Stats:         make([]StatsSample, 0),
+	}
+	sc.mutex.Unlock()
+
+	if verbose {
+		log.Printf("Starting stats collection for container %s (%s)", containerName, containerID[:12])
+	}
+
+	sc.wg.Add(1)
+	go sc.collectStatsForContainer(ctx, containerID, verbose)
+}
+
+// collectStatsForContainer collects stats for a specific container using Docker API streaming.
+func (sc *StatsCollector) collectStatsForContainer(ctx context.Context, containerID string, verbose bool) {
+	defer sc.wg.Done()
+
+	// Use Docker API streaming stats - much more efficient than CLI
+	statsResponse, err := sc.client.ContainerStats(ctx, containerID, true)
+	if err != nil {
+		if verbose {
+			log.Printf("Failed to get stats stream for container %s: %v", containerID[:12], err)
+		}
+		return
+	}
+	defer statsResponse.Body.Close()
+
+	decoder := json.NewDecoder(statsResponse.Body)
+	var prevStats *container.Stats
+
+	for {
+		select {
+		case <-sc.stopChan:
+			return
+		case <-ctx.Done():
+			return
+		default:
+			var stats container.Stats
+			if err := decoder.Decode(&stats); err != nil {
+				// EOF is expected when container stops or stream ends
+				if err.Error() != "EOF" && verbose {
+					log.Printf("Failed to decode stats for container %s: %v", containerID[:12], err)
+				}
+				return
+			}
+
+			// Calculate CPU percentage (only if we have previous stats)
+			var cpuPercent float64
+			if prevStats != nil {
+				cpuPercent = calculateCPUPercent(prevStats, &stats)
+			}
+
+			// Calculate memory usage in MB
+			memoryMB := float64(stats.MemoryStats.Usage) / (1024 * 1024)
+
+			// Store the sample (skip first sample since CPU calculation needs previous stats)
+			if prevStats != nil {
+				// Get container stats reference without holding the main mutex
+				var containerStats *ContainerStats
+				var exists bool
+
+				sc.mutex.RLock()
+				containerStats, exists = sc.containers[containerID]
+				sc.mutex.RUnlock()
+
+				if exists && containerStats != nil {
+					containerStats.mutex.Lock()
+					containerStats.Stats = append(containerStats.Stats, StatsSample{
+						Timestamp: time.Now(),
+						CPUUsage:  cpuPercent,
+						MemoryMB:  memoryMB,
+					})
+					containerStats.mutex.Unlock()
+				}
+			}
+
+			// Save current stats for next iteration
+			prevStats = &stats
+		}
+	}
+}
+
+// calculateCPUPercent calculates CPU usage percentage from Docker stats.
+func calculateCPUPercent(prevStats, stats *container.Stats) float64 {
+	// CPU calculation based on Docker's implementation
+	cpuDelta := float64(stats.CPUStats.CPUUsage.TotalUsage) - float64(prevStats.CPUStats.CPUUsage.TotalUsage)
+	systemDelta := float64(stats.CPUStats.SystemUsage) - float64(prevStats.CPUStats.SystemUsage)
+
+	if systemDelta > 0 && cpuDelta >= 0 {
+		// Calculate CPU percentage: (container CPU delta / system CPU delta) * number of CPUs * 100
+		numCPUs := float64(len(stats.CPUStats.CPUUsage.PercpuUsage))
+		if numCPUs == 0 {
+			// Fallback: if PercpuUsage is not available, assume 1 CPU
+			numCPUs = 1.0
+		}
+
+		return (cpuDelta / systemDelta) * numCPUs * 100.0
+	}
+
+	return 0.0
+}
+
+// ContainerStatsSummary represents summary statistics for a container.
+type ContainerStatsSummary struct {
+	ContainerName string
+	SampleCount   int
+	CPU           StatsSummary
+	Memory        StatsSummary
+}
+
+// MemoryViolation represents a container that exceeded the memory limit.
+type MemoryViolation struct {
+	ContainerName string
+	MaxMemoryMB   float64
+	LimitMB       float64
+}
+
+// StatsSummary represents min, max, and average for a metric.
+type StatsSummary struct {
+	Min     float64
+	Max     float64
+	Average float64
+}
+
+// GetSummary returns a summary of collected statistics.
+func (sc *StatsCollector) GetSummary() []ContainerStatsSummary {
+	// Take snapshot of container references without holding main lock long
+	sc.mutex.RLock()
+	containerRefs := make([]*ContainerStats, 0, len(sc.containers))
+	for _, containerStats := range sc.containers {
+		containerRefs = append(containerRefs, containerStats)
+	}
+	sc.mutex.RUnlock()
+
+	summaries := make([]ContainerStatsSummary, 0, len(containerRefs))
+
+	for _, containerStats := range containerRefs {
+		containerStats.mutex.RLock()
+		stats := make([]StatsSample, len(containerStats.Stats))
+		copy(stats, containerStats.Stats)
+		containerName := containerStats.ContainerName
+		containerStats.mutex.RUnlock()
+
+		if len(stats) == 0 {
+			continue
+		}
+
+		summary := ContainerStatsSummary{
+			ContainerName: containerName,
+			SampleCount:   len(stats),
+		}
+
+		// Calculate CPU stats
+		cpuValues := make([]float64, len(stats))
+		memoryValues := make([]float64, len(stats))
+
+		for i, sample := range stats {
+			cpuValues[i] = sample.CPUUsage
+			memoryValues[i] = sample.MemoryMB
+		}
+
+		summary.CPU = calculateStatsSummary(cpuValues)
+		summary.Memory = calculateStatsSummary(memoryValues)
+
+		summaries = append(summaries, summary)
+	}
+
+	// Sort by container name for consistent output
+	sort.Slice(summaries, func(i, j int) bool {
+		return summaries[i].ContainerName < summaries[j].ContainerName
+	})
+
+	return summaries
+}
+
+// calculateStatsSummary calculates min, max, and average for a slice of values.
+func calculateStatsSummary(values []float64) StatsSummary {
+	if len(values) == 0 {
+		return StatsSummary{}
+	}
+
+	min := values[0]
+	max := values[0]
+	sum := 0.0
+
+	for _, value := range values {
+		if value < min {
+			min = value
+		}
+		if value > max {
+			max = value
+		}
+		sum += value
+	}
+
+	return StatsSummary{
+		Min:     min,
+		Max:     max,
+		Average: sum / float64(len(values)),
+	}
+}
+
+// PrintSummary prints the statistics summary to the console.
+func (sc *StatsCollector) PrintSummary() {
+	summaries := sc.GetSummary()
+
+	if len(summaries) == 0 {
+		log.Printf("No container statistics collected")
+		return
+	}
+
+	log.Printf("Container Resource Usage Summary:")
+	log.Printf("================================")
+
+	for _, summary := range summaries {
+		log.Printf("Container: %s (%d samples)", summary.ContainerName, summary.SampleCount)
+		log.Printf("  CPU Usage:    Min: %6.2f%%  Max: %6.2f%%  Avg: %6.2f%%",
+			summary.CPU.Min, summary.CPU.Max, summary.CPU.Average)
+		log.Printf("  Memory Usage: Min: %6.1f MB Max: %6.1f MB Avg: %6.1f MB",
+			summary.Memory.Min, summary.Memory.Max, summary.Memory.Average)
+		log.Printf("")
+	}
+}
+
+// CheckMemoryLimits checks if any containers exceeded their memory limits.
+func (sc *StatsCollector) CheckMemoryLimits(hsLimitMB, tsLimitMB float64) []MemoryViolation {
+	if hsLimitMB <= 0 && tsLimitMB <= 0 {
+		return nil
+	}
+
+	summaries := sc.GetSummary()
+	var violations []MemoryViolation
+
+	for _, summary := range summaries {
+		var limitMB float64
+		if strings.HasPrefix(summary.ContainerName, "hs-") {
+			limitMB = hsLimitMB
+		} else if strings.HasPrefix(summary.ContainerName, "ts-") {
+			limitMB = tsLimitMB
+		} else {
+			continue // Skip containers that don't match our patterns
+		}
+
+		if limitMB > 0 && summary.Memory.Max > limitMB {
+			violations = append(violations, MemoryViolation{
+				ContainerName: summary.ContainerName,
+				MaxMemoryMB:   summary.Memory.Max,
+				LimitMB:       limitMB,
+			})
+		}
+	}
+
+	return violations
+}
+
+// PrintSummaryAndCheckLimits prints the statistics summary and returns memory violations if any.
+func (sc *StatsCollector) PrintSummaryAndCheckLimits(hsLimitMB, tsLimitMB float64) []MemoryViolation {
+	sc.PrintSummary()
+	return sc.CheckMemoryLimits(hsLimitMB, tsLimitMB)
+}
+
+// Close closes the stats collector and cleans up resources.
+func (sc *StatsCollector) Close() error {
+	sc.StopCollection()
+	return sc.client.Close()
+}

cmd/hi/tar_utils.go

@@ -0,0 +1,105 @@
+package main
+
+import (
+	"archive/tar"
+	"errors"
+	"fmt"
+	"io"
+	"os"
+	"path/filepath"
+	"strings"
+)
+
+// ErrFileNotFoundInTar indicates a file was not found in the tar archive.
+var ErrFileNotFoundInTar = errors.New("file not found in tar")
+
+// extractFileFromTar extracts a single file from a tar reader.
+func extractFileFromTar(tarReader io.Reader, fileName, outputPath string) error {
+	tr := tar.NewReader(tarReader)
+
+	for {
+		header, err := tr.Next()
+		if err == io.EOF {
+			break
+		}
+		if err != nil {
+			return fmt.Errorf("failed to read tar header: %w", err)
+		}
+
+		// Check if this is the file we're looking for
+		if filepath.Base(header.Name) == fileName {
+			if header.Typeflag == tar.TypeReg {
+				// Create the output file
+				outFile, err := os.Create(outputPath)
+				if err != nil {
+					return fmt.Errorf("failed to create output file: %w", err)
+				}
+				defer outFile.Close()
+
+				// Copy file contents
+				if _, err := io.Copy(outFile, tr); err != nil {
+					return fmt.Errorf("failed to copy file contents: %w", err)
+				}
+
+				return nil
+			}
+		}
+	}
+
+	return fmt.Errorf("%w: %s", ErrFileNotFoundInTar, fileName)
+}
+
+// extractDirectoryFromTar extracts all files from a tar reader to a target directory.
+func extractDirectoryFromTar(tarReader io.Reader, targetDir string) error {
+	tr := tar.NewReader(tarReader)
+
+	for {
+		header, err := tr.Next()
+		if err == io.EOF {
+			break
+		}
+		if err != nil {
+			return fmt.Errorf("failed to read tar header: %w", err)
+		}
+
+		// Clean the path to prevent directory traversal
+		cleanName := filepath.Clean(header.Name)
+		if strings.Contains(cleanName, "..") {
+			continue // Skip potentially dangerous paths
+		}
+
+		targetPath := filepath.Join(targetDir, cleanName)
+
+		switch header.Typeflag {
+		case tar.TypeDir:
+			// Create directory
+			if err := os.MkdirAll(targetPath, os.FileMode(header.Mode)); err != nil {
+				return fmt.Errorf("failed to create directory %s: %w", targetPath, err)
+			}
+		case tar.TypeReg:
+			// Ensure parent directories exist
+			if err := os.MkdirAll(filepath.Dir(targetPath), 0o755); err != nil {
+				return fmt.Errorf("failed to create parent directories for %s: %w", targetPath, err)
+			}
+
+			// Create file
+			outFile, err := os.Create(targetPath)
+			if err != nil {
+				return fmt.Errorf("failed to create file %s: %w", targetPath, err)
+			}
+
+			if _, err := io.Copy(outFile, tr); err != nil {
+				outFile.Close()
+				return fmt.Errorf("failed to copy file contents: %w", err)
+			}
+			outFile.Close()
+
+			// Set file permissions
+			if err := os.Chmod(targetPath, os.FileMode(header.Mode)); err != nil {
+				return fmt.Errorf("failed to set file permissions: %w", err)
+			}
+		}
+	}
+
+	return nil
+}

cmd/mapresponses/main.go

@@ -0,0 +1,61 @@
+package main
+
+import (
+	"encoding/json"
+	"fmt"
+	"os"
+
+	"github.com/creachadair/command"
+	"github.com/creachadair/flax"
+	"github.com/juanfont/headscale/hscontrol/mapper"
+	"github.com/juanfont/headscale/integration/integrationutil"
+)
+
+type MapConfig struct {
+	Directory string `flag:"directory,Directory to read map responses from"`
+}
+
+var mapConfig MapConfig
+
+func main() {
+	root := command.C{
+		Name: "mapresponses",
+		Help: "MapResponses is a tool to map and compare map responses from a directory",
+		Commands: []*command.C{
+			{
+				Name:     "online",
+				Help:     "",
+				Usage:    "run [test-pattern] [flags]",
+				SetFlags: command.Flags(flax.MustBind, &mapConfig),
+				Run:      runOnline,
+			},
+			command.HelpCommand(nil),
+		},
+	}
+
+	env := root.NewEnv(nil).MergeFlags(true)
+	command.RunOrFail(env, os.Args[1:])
+}
+
+// runIntegrationTest executes the integration test workflow.
+func runOnline(env *command.Env) error {
+	if mapConfig.Directory == "" {
+		return fmt.Errorf("directory is required")
+	}
+
+	resps, err := mapper.ReadMapResponsesFromDirectory(mapConfig.Directory)
+	if err != nil {
+		return fmt.Errorf("reading map responses from directory: %w", err)
+	}
+
+	expected := integrationutil.BuildExpectedOnlineMap(resps)
+
+	out, err := json.MarshalIndent(expected, "", "  ")
+	if err != nil {
+		return fmt.Errorf("marshaling expected online map: %w", err)
+	}
+
+	os.Stderr.Write(out)
+	os.Stderr.Write([]byte("\n"))
+	return nil
+}

config-example.yaml

@@ -60,7 +60,9 @@ prefixes:
   v6: fd7a:115c:a1e0::/48
 
   # Strategy used for allocation of IPs to nodes, available options:
-  # - sequential (default): assigns the next free IP from the previous given IP.
+  # - sequential (default): assigns the next free IP from the previous given
+  #   IP. A best-effort approach is used and Headscale might leave holes in the
+  #   IP range or fill up existing holes in the IP range.
   # - random: assigns the next free IP from a pseudo-random IP generator (crypto/rand).
   allocation: sequential
 
@@ -85,6 +87,9 @@ derp:
     region_code: "headscale"
     region_name: "Headscale Embedded DERP"
 
+    # Only allow clients associated with this server access
+    verify_clients: true
+
     # Listens over UDP at the configured address for STUN connections - to help with NAT traversal.
     # When the embedded DERP server is enabled stun_listen_addr MUST be defined.
     #
@@ -102,7 +107,7 @@ derp:
 
     # For better connection stability (especially when using an Exit-Node and DNS is not working),
     # it is possible to optionally add the public IPv4 and IPv6 address to the Derp-Map using:
-    ipv4: 1.2.3.4
+    ipv4: 198.51.100.1
     ipv6: 2001:db8::1
 
   # List of externally available DERP maps encoded in JSON
@@ -125,7 +130,7 @@ derp:
   auto_update_enabled: true
 
   # How often should we check for DERP updates?
-  update_frequency: 24h
+  update_frequency: 3h
 
 # Disables the automatic check for headscale updates on startup
 disable_check_updates: false
@@ -222,9 +227,11 @@ tls_cert_path: ""
 tls_key_path: ""
 
 log:
+  # Valid log levels: panic, fatal, error, warn, info, debug, trace
+  level: info
+
   # Output formatting for logs: text or json
   format: text
-  level: info
 
 ## Policy
 # headscale supports Tailscale's ACL policies.
@@ -270,9 +277,9 @@ dns:
   # `hostname.base_domain` (e.g., _myhost.example.com_).
   base_domain: example.com
 
-  # Whether to use the local DNS settings of a node (default) or override the
-  # local DNS settings and force the use of Headscale's DNS configuration.
-  override_local_dns: false
+  # Whether to use the local DNS settings of a node or override the local DNS
+  # settings (default) and force the use of Headscale's DNS configuration.
+  override_local_dns: true
 
   # List of DNS servers to expose to clients.
   nameservers:
@@ -288,8 +295,7 @@ dns:
 
     # Split DNS (see https://tailscale.com/kb/1054/dns/),
     # a map of domains and which DNS server to use for each.
-    split:
-      {}
+    split: {}
       # foo.bar.com:
       #   - 1.1.1.1
       # darp.headscale.net:
@@ -319,51 +325,60 @@ dns:
 # Note: for production you will want to set this to something like:
 unix_socket: /var/run/headscale/headscale.sock
 unix_socket_permission: "0770"
-#
-# headscale supports experimental OpenID connect support,
-# it is still being tested and might have some bugs, please
-# help us test it.
+
 # OpenID Connect
 # oidc:
+#   # Block startup until the identity provider is available and healthy.
 #   only_start_if_oidc_is_available: true
+#
+#   # OpenID Connect Issuer URL from the identity provider
 #   issuer: "https://your-oidc.issuer.com/path"
+#
+#   # Client ID from the identity provider
 #   client_id: "your-oidc-client-id"
+#
+#   # Client secret generated by the identity provider
+#   # Note: client_secret and client_secret_path are mutually exclusive.
 #   client_secret: "your-oidc-client-secret"
 #   # Alternatively, set `client_secret_path` to read the secret from the file.
 #   # It resolves environment variables, making integration to systemd's
 #   # `LoadCredential` straightforward:
 #   client_secret_path: "${CREDENTIALS_DIRECTORY}/oidc_client_secret"
-#   # client_secret and client_secret_path are mutually exclusive.
 #
-#   # The amount of time from a node is authenticated with OpenID until it
-#   # expires and needs to reauthenticate.
+#   # The amount of time a node is authenticated with OpenID until it expires
+#   # and needs to reauthenticate.
 #   # Setting the value to "0" will mean no expiry.
 #   expiry: 180d
 #
 #   # Use the expiry from the token received from OpenID when the user logged
-#   # in, this will typically lead to frequent need to reauthenticate and should
-#   # only been enabled if you know what you are doing.
+#   # in. This will typically lead to frequent need to reauthenticate and should
+#   # only be enabled if you know what you are doing.
 #   # Note: enabling this will cause `oidc.expiry` to be ignored.
 #   use_expiry_from_token: false
 #
-#   # Customize the scopes used in the OIDC flow, defaults to "openid", "profile" and "email" and add custom query
-#   # parameters to the Authorize Endpoint request. Scopes default to "openid", "profile" and "email".
+#   # The OIDC scopes to use, defaults to "openid", "profile" and "email".
+#   # Custom scopes can be configured as needed, be sure to always include the
+#   # required "openid" scope.
+#   scope: ["openid", "profile", "email"]
 #
-#   scope: ["openid", "profile", "email", "custom"]
+#   # Provide custom key/value pairs which get sent to the identity provider's
+#   # authorization endpoint.
 #   extra_params:
 #     domain_hint: example.com
 #
-#   # List allowed principal domains and/or users. If an authenticated user's domain is not in this list, the
-#   # authentication request will be rejected.
-#
+#   # Only accept users whose email domain is part of the allowed_domains list.
 #   allowed_domains:
 #     - example.com
-#   # Note: Groups from keycloak have a leading '/'
-#   allowed_groups:
-#     - /headscale
+#
+#   # Only accept users whose email address is part of the allowed_users list.
 #   allowed_users:
 #     - alice@example.com
 #
+#   # Only accept users which are members of at least one group in the
+#   # allowed_groups list.
+#   allowed_groups:
+#     - /headscale
+#
 #   # Optional: PKCE (Proof Key for Code Exchange) configuration
 #   # PKCE adds an additional layer of security to the OAuth 2.0 authorization code flow
 #   # by preventing authorization code interception attacks
@@ -371,17 +386,20 @@ unix_socket_permission: "0770"
 #   pkce:
 #     # Enable or disable PKCE support (default: false)
 #     enabled: false
+#
 #     # PKCE method to use:
 #     # - plain: Use plain code verifier
 #     # - S256: Use SHA256 hashed code verifier (default, recommended)
 #     method: S256
 
 # Logtail configuration
-# Logtail is Tailscales logging and auditing infrastructure, it allows the control panel
-# to instruct tailscale nodes to log their activity to a remote server.
+# Logtail is Tailscales logging and auditing infrastructure, it allows the
+# control panel to instruct tailscale nodes to log their activity to a remote
+# server. To disable logging on the client side, please refer to:
+# https://tailscale.com/kb/1011/log-mesh-traffic#opting-out-of-client-logging
 logtail:
-  # Enable logtail for this headscales clients.
-  # As there is currently no support for overriding the log server in headscale, this is
+  # Enable logtail for tailscale nodes of this Headscale instance.
+  # As there is currently no support for overriding the log server in Headscale, this is
   # disabled by default. Enabling this will make your clients send logs to Tailscale Inc.
   enabled: false
 

derp-example.yaml

@@ -1,5 +1,6 @@
 # If you plan to somehow use headscale, please deploy your own DERP infra: https://tailscale.com/kb/1118/custom-derp-servers/
 regions:
+  1: null   # Disable DERP region with ID 1
   900:
     regionid: 900
     regioncode: custom
@@ -7,9 +8,9 @@ regions:
     nodes:
       - name: 900a
         regionid: 900
-        hostname: myderp.mydomain.no
-        ipv4: 123.123.123.123
-        ipv6: "2604:a880:400:d1::828:b001"
+        hostname: myderp.example.com
+        ipv4: 198.51.100.1
+        ipv6: 2001:db8::1
         stunport: 0
         stunonly: false
         derpport: 0

docs/about/faq.md

@@ -44,6 +44,15 @@ For convenience, we also [build container images with headscale](../setup/instal
 we don't officially support deploying headscale using Docker**. On our [Discord server](https://discord.gg/c84AZQhmpx)
 we have a "docker-issues" channel where you can ask for Docker-specific help to the community.
 
+## What is the recommended update path? Can I skip multiple versions while updating?
+
+Please follow the steps outlined in the [upgrade guide](../setup/upgrade.md) to update your existing Headscale
+installation. Its best to update from one stable version to the next (e.g. 0.24.0 → 0.25.1 → 0.26.1) in case
+you are multiple releases behind. You should always pick the latest available patch release.
+
+Be sure to check the [changelog](https://github.com/juanfont/headscale/blob/main/CHANGELOG.md) for version specific
+upgrade instructions and breaking changes.
+
 ## Scaling / How many clients does Headscale support?
 
 It depends. As often stated, Headscale is not enterprise software and our focus
@@ -51,22 +60,22 @@ is homelabbers and self-hosters. Of course, we do not prevent people from using
 it in a commercial/professional setting and often get questions about scaling.
 
 Please note that when Headscale is developed, performance is not part of the
-consideration as the main audience is considered to be users with a moddest
+consideration as the main audience is considered to be users with a modest
 amount of devices. We focus on correctness and feature parity with Tailscale
 SaaS over time.
 
-To understand if you might be able to use Headscale for your usecase, I will
+To understand if you might be able to use Headscale for your use case, I will
 describe two scenarios in an effort to explain what is the central bottleneck
 of Headscale:
 
 1. An environment with 1000 servers
 
-    - they rarely  "move" (change their endpoints)
-    - new nodes are added rarely
+   - they rarely "move" (change their endpoints)
+   - new nodes are added rarely
 
 2. An environment with 80 laptops/phones (end user devices)
 
-    - nodes move often, e.g. switching from home to office
+   - nodes move often, e.g. switching from home to office
 
 Headscale calculates a map of all nodes that need to talk to each other,
 creating this "world map" requires a lot of CPU time. When an event that
@@ -76,7 +85,7 @@ new "world map" is created for every node in the network.
 This means that under certain conditions, Headscale can likely handle 100s
 of devices (maybe more), if there is _little to no change_ happening in the
 network. For example, in Scenario 1, the process of computing the world map is
-extremly demanding due to the size of the network, but when the map has been
+extremely demanding due to the size of the network, but when the map has been
 created and the nodes are not changing, the Headscale instance will likely
 return to a very low resource usage until the next time there is an event
 requiring the new map.
@@ -94,14 +103,14 @@ learn about the current state of the world.
 We expect that the performance will improve over time as we improve the code
 base, but it is not a focus. In general, we will never make the tradeoff to make
 things faster on the cost of less maintainable or readable code. We are a small
-team and have to optimise for maintainabillity.
+team and have to optimise for maintainability.
 
 ## Which database should I use?
 
 We recommend the use of SQLite as database for headscale:
 
 - SQLite is simple to setup and easy to use
-- It scales well for all of headscale's usecases
+- It scales well for all of headscale's use cases
 - Development and testing happens primarily on SQLite
 - PostgreSQL is still supported, but is considered to be in "maintenance mode"
 
@@ -122,7 +131,6 @@ help to the community.
 
 Running headscale on a machine that is also in the tailnet can cause problems with subnet routers, traffic relay nodes, and MagicDNS. It might work, but it is not supported.
 
-
 ## Why do two nodes see each other in their status, even if an ACL allows traffic only in one direction?
 
 A frequent use case is to allow traffic only from one node to another, but not the other way around. For example, the
@@ -135,3 +143,35 @@ in their output of `tailscale status`. Traffic is still filtered according to th
 ping` which is always allowed in either direction.
 
 See also <https://tailscale.com/kb/1087/device-visibility>.
+
+## My policy is stored in the database and Headscale refuses to start due to an invalid policy. How can I recover?
+
+Headscale checks if the policy is valid during startup and refuses to start if it detects an error. The error message
+indicates which part of the policy is invalid. Follow these steps to fix your policy:
+
+- Dump the policy to a file: `headscale policy get --bypass-grpc-and-access-database-directly > policy.json`
+- Edit and fixup `policy.json`. Use the command `headscale policy check --file policy.json` to validate the policy.
+- Load the modified policy: `headscale policy set --bypass-grpc-and-access-database-directly --file policy.json`
+- Start Headscale as usual.
+
+!!! warning "Full server configuration required"
+
+    The above commands to get/set the policy require a complete server configuration file including database settings. A
+    minimal config to [control Headscale via remote CLI](../ref/remote-cli.md) is not sufficient. You may use `headscale
+    -c /path/to/config.yaml` to specify the path to an alternative configuration file.
+
+## How can I avoid to send logs to Tailscale Inc?
+
+A Tailscale client [collects logs about its operation and connection attempts with other
+clients](https://tailscale.com/kb/1011/log-mesh-traffic#client-logs) and sends them to a central log service operated by
+Tailscale Inc.
+
+Headscale, by default, instructs clients to disable log submission to the central log service. This configuration is
+applied by a client once it successfully connected with Headscale. See the configuration option `logtail.enabled` in the
+[configuration file](../ref/configuration.md) for details.
+
+Alternatively, logging can also be disabled on the client side. This is independent of Headscale and opting out of
+client logging disables log submission early during client startup. The configuration is operating system specific and
+is usually achieved by setting the environment variable `TS_NO_LOGS_NO_SUPPORT=true` or by passing the flag
+`--no-logs-no-support` to `tailscaled`. See
+<https://tailscale.com/kb/1011/log-mesh-traffic#opting-out-of-client-logging> for details.

docs/about/features.md

@@ -19,19 +19,18 @@ provides on overview of Headscale's feature and compatibility with the Tailscale
     - [x] [Exit nodes](../ref/routes.md#exit-node)
 - [x] Dual stack (IPv4 and IPv6)
 - [x] Ephemeral nodes
-- [x] Embedded [DERP server](https://tailscale.com/kb/1232/derp-servers)
+- [x] Embedded [DERP server](../ref/derp.md)
 - [x] Access control lists ([GitHub label "policy"](https://github.com/juanfont/headscale/labels/policy%20%F0%9F%93%9D))
     - [x] ACL management via API
     - [x] Some [Autogroups](https://tailscale.com/kb/1396/targets#autogroups), currently: `autogroup:internet`,
-      `autogroup:nonroot`
+      `autogroup:nonroot`, `autogroup:member`, `autogroup:tagged`, `autogroup:self`
     - [x] [Auto approvers](https://tailscale.com/kb/1337/acl-syntax#auto-approvers) for [subnet
       routers](../ref/routes.md#automatically-approve-routes-of-a-subnet-router) and [exit
       nodes](../ref/routes.md#automatically-approve-an-exit-node-with-auto-approvers)
     - [x] [Tailscale SSH](https://tailscale.com/kb/1193/tailscale-ssh)
-* [ ] Node registration using Single-Sign-On (OpenID Connect) ([GitHub label "OIDC"](https://github.com/juanfont/headscale/labels/OIDC))
+* [x] [Node registration using Single-Sign-On (OpenID Connect)](../ref/oidc.md) ([GitHub label "OIDC"](https://github.com/juanfont/headscale/labels/OIDC))
     - [x] Basic registration
     - [x] Update user profile from identity provider
-    - [ ] Dynamic ACL support
     - [ ] OIDC groups cannot be used in ACLs
 - [ ] [Funnel](https://tailscale.com/kb/1223/funnel) ([#1040](https://github.com/juanfont/headscale/issues/1040))
 - [ ] [Serve](https://tailscale.com/kb/1312/serve) ([#1234](https://github.com/juanfont/headscale/issues/1921))

docs/packaging/README.md

@@ -1,5 +0,0 @@
-# Packaging
-
-We use [nFPM](https://nfpm.goreleaser.com/) for making `.deb`, `.rpm` and `.apk`.
-
-This folder contains files we need to package with these releases.

docs/packaging/postinstall.sh

@@ -1,88 +0,0 @@
-#!/bin/sh
-# Determine OS platform
-# shellcheck source=/dev/null
-. /etc/os-release
-
-HEADSCALE_EXE="/usr/bin/headscale"
-BSD_HIER=""
-HEADSCALE_RUN_DIR="/var/run/headscale"
-HEADSCALE_HOME_DIR="/var/lib/headscale"
-HEADSCALE_USER="headscale"
-HEADSCALE_GROUP="headscale"
-HEADSCALE_SHELL="/usr/sbin/nologin"
-
-ensure_sudo() {
-	if [ "$(id -u)" = "0" ]; then
-		echo "Sudo permissions detected"
-	else
-		echo "No sudo permission detected, please run as sudo"
-		exit 1
-	fi
-}
-
-ensure_headscale_path() {
-	if [ ! -f "$HEADSCALE_EXE" ]; then
-		echo "headscale not in default path, exiting..."
-		exit 1
-	fi
-
-	printf "Found headscale %s\n" "$HEADSCALE_EXE"
-}
-
-create_headscale_user() {
-	printf "PostInstall: Adding headscale user %s\n" "$HEADSCALE_USER"
-	useradd -r -s "$HEADSCALE_SHELL" -d "$HEADSCALE_HOME_DIR" -c "headscale default user" "$HEADSCALE_USER"
-}
-
-create_headscale_group() {
-	if command -V systemctl >/dev/null 2>&1; then
-		printf "PostInstall: Adding headscale group %s\n" "$HEADSCALE_GROUP"
-		groupadd -r "$HEADSCALE_GROUP"
-
-		printf "PostInstall: Adding headscale user %s to group %s\n" "$HEADSCALE_USER" "$HEADSCALE_GROUP"
-		usermod -a -G "$HEADSCALE_GROUP" "$HEADSCALE_USER"
-	fi
-
-	if [ "$ID" = "alpine" ]; then
-		printf "PostInstall: Adding headscale group %s\n" "$HEADSCALE_GROUP"
-		addgroup -S "$HEADSCALE_GROUP"
-
-		printf "PostInstall: Adding headscale user %s to group %s\n" "$HEADSCALE_USER" "$HEADSCALE_GROUP"
-		addgroup "$HEADSCALE_USER" "$HEADSCALE_GROUP"
-	fi
-}
-
-create_run_dir() {
-	printf "PostInstall: Creating headscale run directory \n"
-	mkdir -p "$HEADSCALE_RUN_DIR"
-
-	printf "PostInstall: Modifying group ownership of headscale run directory \n"
-	chown "$HEADSCALE_USER":"$HEADSCALE_GROUP" "$HEADSCALE_RUN_DIR"
-}
-
-summary() {
-	echo "----------------------------------------------------------------------"
-	echo " headscale package has been successfully installed."
-	echo ""
-	echo " Please follow the next steps to start the software:"
-	echo ""
-	echo "    sudo systemctl enable headscale"
-	echo "    sudo systemctl start headscale"
-	echo ""
-	echo " Configuration settings can be adjusted here:"
-	echo "    ${BSD_HIER}/etc/headscale/config.yaml"
-	echo ""
-	echo "----------------------------------------------------------------------"
-}
-
-#
-# Main body of the script
-#
-{
-	ensure_sudo
-	ensure_headscale_path
-	create_headscale_user
-	create_headscale_group
-	create_run_dir
-	summary
-}

docs/packaging/postremove.sh

@@ -1,15 +0,0 @@
-#!/bin/sh
-# Determine OS platform
-# shellcheck source=/dev/null
-. /etc/os-release
-
-if command -V systemctl >/dev/null 2>&1; then
-	echo "Stop and disable headscale service"
-	systemctl stop headscale >/dev/null 2>&1 || true
-	systemctl disable headscale >/dev/null 2>&1 || true
-	echo "Running daemon-reload"
-	systemctl daemon-reload || true
-fi
-
-echo "Removing run directory"
-rm -rf "/var/run/headscale.sock"

docs/ref/acls.md

@@ -9,9 +9,38 @@ When using ACL's the User borders are no longer applied. All machines
 whichever the User have the ability to communicate with other hosts as
 long as the ACL's permits this exchange.
 
-## ACLs use case example
+## ACL Setup
 
-Let's build an example use case for a small business (It may be the place where
+To enable and configure ACLs in Headscale, you need to specify the path to your ACL policy file in the `policy.path` key in `config.yaml`.
+
+Your ACL policy file must be formatted using [huJSON](https://github.com/tailscale/hujson).
+
+Info on how these policies are written can be found
+[here](https://tailscale.com/kb/1018/acls/).
+
+Please reload or restart Headscale after updating the ACL file. Headscale may be reloaded either via its systemd service
+(`sudo systemctl reload headscale`) or by sending a SIGHUP signal (`sudo kill -HUP $(pidof headscale)`) to the main
+process. Headscale logs the result of ACL policy processing after each reload.
+
+## Simple Examples
+
+- [**Allow All**](https://tailscale.com/kb/1192/acl-samples#allow-all-default-acl): If you define an ACL file but completely omit the `"acls"` field from its content, Headscale will default to an "allow all" policy. This means all devices connected to your tailnet will be able to communicate freely with each other.
+
+    ```json
+    {}
+    ```
+
+- [**Deny All**](https://tailscale.com/kb/1192/acl-samples#deny-all): To prevent all communication within your tailnet, you can include an empty array for the `"acls"` field in your policy file.
+
+    ```json
+    {
+      "acls": []
+    }
+    ```
+
+## Complex Example
+
+Let's build a more complex example use case for a small business (It may be the place where
 ACL's are the most useful).
 
 We have a small company with a boss, an admin, two developers and an intern.
@@ -38,10 +67,6 @@ servers.
 
 ![ACL implementation example](../images/headscale-acl-network.png)
 
-## ACL setup
-
-ACLs have to be written in [huJSON](https://github.com/tailscale/hujson).
-
 When [registering the servers](../usage/getting-started.md#register-a-node) we
 will need to add the flag `--advertise-tags=tag:<tag1>,tag:<tag2>`, and the user
 that is registering the server should be allowed to do it. Since anyone can add
@@ -49,14 +74,6 @@ tags to a server they can register, the check of the tags is done on headscale
 server and only valid tags are applied. A tag is valid if the user that is
 registering it is allowed to do it.
 
-To use ACLs in headscale, you must edit your `config.yaml` file. In there you will find a `policy.path` parameter. This
-will need to point to your ACL file. More info on how these policies are written can be found
-[here](https://tailscale.com/kb/1018/acls/).
-
-Please reload or restart Headscale after updating the ACL file. Headscale may be reloaded either via its systemd service
-(`sudo systemctl reload headscale`) or by sending a SIGHUP signal (`sudo kill -HUP $(pidof headscale)`) to the main
-process. Headscale logs the result of ACL policy processing after each reload.
-
 Here are the ACL's to implement the same permissions as above:
 
 ```json title="acl.json"
@@ -177,13 +194,94 @@ Here are the ACL's to implement the same permissions as above:
       "dst": ["tag:dev-app-servers:80,443"]
     },
 
-    // We still have to allow internal users communications since nothing guarantees that each user have
-    // their own users.
-    { "action": "accept", "src": ["boss@"], "dst": ["boss@:*"] },
-    { "action": "accept", "src": ["dev1@"], "dst": ["dev1@:*"] },
-    { "action": "accept", "src": ["dev2@"], "dst": ["dev2@:*"] },
-    { "action": "accept", "src": ["admin1@"], "dst": ["admin1@:*"] },
-    { "action": "accept", "src": ["intern1@"], "dst": ["intern1@:*"] }
+    // Allow users to access their own devices using autogroup:self (see below for more details about performance impact)
+    {
+      "action": "accept",
+      "src": ["autogroup:member"],
+      "dst": ["autogroup:self:*"]
+    }
   ]
 }
 ```
+
+## Autogroups
+
+Headscale supports several autogroups that automatically include users, destinations, or devices with specific properties. Autogroups provide a convenient way to write ACL rules without manually listing individual users or devices.
+
+### `autogroup:internet`
+
+Allows access to the internet through [exit nodes](routes.md#exit-node). Can only be used in ACL destinations.
+
+```json
+{
+  "action": "accept",
+  "src": ["group:users"],
+  "dst": ["autogroup:internet:*"]
+}
+```
+
+### `autogroup:member`
+
+Includes all users who are direct members of the tailnet. Does not include users from shared devices.
+
+```json
+{
+  "action": "accept",
+  "src": ["autogroup:member"],
+  "dst": ["tag:prod-app-servers:80,443"]
+}
+```
+
+### `autogroup:tagged`
+
+Includes all devices that have at least one tag.
+
+```json
+{
+  "action": "accept",
+  "src": ["autogroup:tagged"],
+  "dst": ["tag:monitoring:9090"]
+}
+```
+
+### `autogroup:self`
+**(EXPERIMENTAL)**
+
+!!! warning "The current implementation of `autogroup:self` is inefficient"
+
+Includes devices where the same user is authenticated on both the source and destination. Does not include tagged devices. Can only be used in ACL destinations.
+
+```json
+{
+  "action": "accept",
+  "src": ["autogroup:member"],
+  "dst": ["autogroup:self:*"]
+}
+```
+*Using `autogroup:self` may cause performance degradation on the Headscale coordinator server in large deployments, as filter rules must be compiled per-node rather than globally and the current implementation is not very efficient.*
+
+If you experience performance issues, consider using more specific ACL rules or limiting the use of `autogroup:self`.
+```json
+{
+  // The following rules allow internal users to communicate with their
+  // own nodes in case autogroup:self is causing performance issues.
+  { "action": "accept", "src": ["boss@"], "dst": ["boss@:*"] },
+  { "action": "accept", "src": ["dev1@"], "dst": ["dev1@:*"] },
+  { "action": "accept", "src": ["dev2@"], "dst": ["dev2@:*"] },
+  { "action": "accept", "src": ["admin1@"], "dst": ["admin1@:*"] },
+  { "action": "accept", "src": ["intern1@"], "dst": ["intern1@:*"] }
+}
+```
+
+### `autogroup:nonroot`
+
+Used in Tailscale SSH rules to allow access to any user except root. Can only be used in the `users` field of SSH rules.
+
+```json
+{
+  "action": "accept",
+  "src": ["autogroup:member"],
+  "dst": ["autogroup:self"],
+  "users": ["autogroup:nonroot"]
+}
+```

docs/ref/configuration.md

@@ -5,7 +5,9 @@
     - `/etc/headscale`
     - `$HOME/.headscale`
     - the current working directory
-- Use the command line flag `-c`, `--config` to load the configuration from a different path
+- To load the configuration from a different path, use:
+    - the command line flag `-c`, `--config`
+    - the environment variable `HEADSCALE_CONFIG`
 - Validate the configuration file with: `headscale configtest`
 
 !!! example "Get the [example configuration from the GitHub repository](https://github.com/juanfont/headscale/blob/main/config-example.yaml)"

docs/ref/debug.md

@@ -0,0 +1,115 @@
+# Debugging and troubleshooting
+
+Headscale and Tailscale provide debug and introspection capabilities that can be helpful when things don't work as
+expected. This page explains some debugging techniques to help pinpoint problems.
+
+Please also have a look at [Tailscale's Troubleshooting guide](https://tailscale.com/kb/1023/troubleshooting). It offers
+a many tips and suggestions to troubleshoot common issues.
+
+## Tailscale
+
+The Tailscale client itself offers many commands to introspect its state as well as the state of the network:
+
+- [Check local network conditions](https://tailscale.com/kb/1080/cli#netcheck): `tailscale netcheck`
+- [Get the client status](https://tailscale.com/kb/1080/cli#status): `tailscale status --json`
+- [Get DNS status](https://tailscale.com/kb/1080/cli#dns): `tailscale dns status --all`
+- Client logs: `tailscale debug daemon-logs`
+- Client netmap: `tailscale debug netmap`
+- Test DERP connection: `tailscale debug derp headscale`
+- And many more, see: `tailscale debug --help`
+
+Many of the commands are helpful when trying to understand differences between Headscale and Tailscale SaaS.
+
+## Headscale
+
+### Application logging
+
+The log levels `debug` and `trace` can be useful to get more information from Headscale.
+
+```yaml hl_lines="3"
+log:
+  # Valid log levels: panic, fatal, error, warn, info, debug, trace
+  level: debug
+```
+
+### Database logging
+
+The database debug mode logs all database queries. Enable it to see how Headscale interacts with its database. This also
+requires the application log level to be set to either `debug` or `trace`.
+
+```yaml hl_lines="3 7"
+database:
+  # Enable debug mode. This setting requires the log.level to be set to "debug" or "trace".
+  debug: false
+
+log:
+  # Valid log levels: panic, fatal, error, warn, info, debug, trace
+  level: debug
+```
+
+### Metrics and debug endpoint
+
+Headscale provides a metrics and debug endpoint. It allows to introspect different aspects such as:
+
+- Information about the Go runtime, memory usage and statistics
+- Connected nodes and pending registrations
+- Active ACLs, filters and SSH policy
+- Current DERPMap
+- Prometheus metrics
+
+!!! warning "Keep the metrics and debug endpoint private"
+
+    The listen address and port can be configured with the `metrics_listen_addr` variable in the [configuration
+    file](./configuration.md). By default it listens on localhost, port 9090.
+
+    Keep the metrics and debug endpoint private to your internal network and don't expose it to the Internet.
+
+Query metrics via <http://localhost:9090/metrics> and get an overview of available debug information via
+<http://localhost:9090/debug/>. Metrics may be queried from outside localhost but the debug interface is subject to
+additional protection despite listening on all interfaces.
+
+=== "Direct access"
+
+    Access the debug interface directly on the server where Headscale is installed.
+
+    ```console
+    curl http://localhost:9090/debug/
+    ```
+
+=== "SSH port forwarding"
+
+    Use SSH port forwarding to forward Headscale's metrics and debug port to your device.
+
+    ```console
+    ssh <HEADSCALE_SERVER> -L 9090:localhost:9090
+    ```
+
+    Access the debug interface on your device by opening <http://localhost:9090/debug/> in your web browser.
+
+=== "Via debug key"
+
+    The access control of the debug interface supports the use of a debug key. Traffic is accepted if the path to a
+    debug key is set via the environment variable `TS_DEBUG_KEY_PATH` and the debug key sent as value for `debugkey`
+    parameter with each request.
+
+    ```console
+    openssl rand -hex 32 | tee debugkey.txt
+    export TS_DEBUG_KEY_PATH=debugkey.txt
+    headscale serve
+    ```
+
+    Access the debug interface on your device by opening `http://<IP_OF_HEADSCALE>:9090/debug/?debugkey=<DEBUG_KEY>` in
+    your web browser. The `debugkey` parameter must be sent with every request.
+
+=== "Via debug IP address"
+
+    The debug endpoint expects traffic from localhost. A different debug IP address may be configured by setting the
+    `TS_ALLOW_DEBUG_IP` environment variable before starting Headscale. The debug IP address is ignored when the HTTP
+    header `X-Forwarded-For` is present.
+
+    ```console
+    export TS_ALLOW_DEBUG_IP=192.168.0.10       # IP address of your device
+    headscale serve
+    ```
+
+    Access the debug interface on your device by opening `http://<IP_OF_HEADSCALE>:9090/debug/` in your web browser.

docs/ref/derp.md

@@ -0,0 +1,175 @@
+# DERP
+
+A [DERP (Designated Encrypted Relay for Packets) server](https://tailscale.com/kb/1232/derp-servers) is mainly used to
+relay traffic between two nodes in case a direct connection can't be established. Headscale provides an embedded DERP
+server to ensure seamless connectivity between nodes.
+
+## Configuration
+
+DERP related settings are configured within the `derp` section of the [configuration file](./configuration.md). The
+following sections only use a few of the available settings, check the [example configuration](./configuration.md) for
+all available configuration options.
+
+### Enable embedded DERP
+
+Headscale ships with an embedded DERP server which allows to run your own self-hosted DERP server easily. The embedded
+DERP server is disabled by default and needs to be enabled. In addition, you should configure the public IPv4 and public
+IPv6 address of your Headscale server for improved connection stability:
+
+```yaml title="config.yaml" hl_lines="3-5"
+derp:
+  server:
+    enabled: true
+    ipv4: 198.51.100.1
+    ipv6: 2001:db8::1
+```
+
+Keep in mind that [additional ports are needed to run a DERP server](../setup/requirements.md#ports-in-use). Besides
+relaying traffic, it also uses STUN (udp/3478) to help clients discover their public IP addresses and perform NAT
+traversal. [Check DERP server connectivity](#check-derp-server-connectivity) to see if everything works.
+
+### Remove Tailscale's DERP servers
+
+Once enabled, Headscale's embedded DERP is added to the list of free-to-use [DERP
+servers](https://tailscale.com/kb/1232/derp-servers) offered by Tailscale Inc. To only use Headscale's embedded DERP
+server, disable the loading of the default DERP map:
+
+```yaml title="config.yaml" hl_lines="6"
+derp:
+  server:
+    enabled: true
+    ipv4: 198.51.100.1
+    ipv6: 2001:db8::1
+  urls: []
+```
+
+!!! warning "Single point of failure"
+
+    Removing Tailscale's DERP servers means that there is now just a single DERP server available for clients. This is a
+    single point of failure and could hamper connectivity.
+
+    [Check DERP server connectivity](#check-derp-server-connectivity) with your embedded DERP server before removing
+    Tailscale's DERP servers.
+
+### Customize DERP map
+
+The DERP map offered to clients can be customized with a [dedicated YAML-configuration
+file](https://github.com/juanfont/headscale/blob/main/derp-example.yaml). This allows to modify previously loaded DERP
+maps fetched via URL or to offer your own, custom DERP servers to nodes.
+
+=== "Remove specific DERP regions"
+
+    The free-to-use [DERP servers](https://tailscale.com/kb/1232/derp-servers) are organized into regions via a region
+    ID. You can explicitly disable a specific region by setting its region ID to `null`. The following sample
+    `derp.yaml` disables the New York DERP region (which has the region ID 1):
+
+     ```yaml title="derp.yaml"
+     regions:
+       1: null
+     ```
+
+    Use the following configuration to serve the default DERP map (excluding New York) to nodes:
+
+    ```yaml title="config.yaml" hl_lines="6 7"
+    derp:
+      server:
+        enabled: false
+      urls:
+        - https://controlplane.tailscale.com/derpmap/default
+      paths:
+        - /etc/headscale/derp.yaml
+    ```
+
+=== "Provide custom DERP servers"
+
+    The following sample `derp.yaml` references two custom regions (`custom-east` with ID 900 and `custom-west` with ID 901)
+    with one custom DERP server in each region. Each DERP server offers DERP relay via HTTPS on tcp/443, support for captive
+    portal checks via HTTP on tcp/80 and STUN on udp/3478. See the definitions of
+    [DERPMap](https://pkg.go.dev/tailscale.com/tailcfg#DERPMap),
+    [DERPRegion](https://pkg.go.dev/tailscale.com/tailcfg#DERPRegion) and
+    [DERPNode](https://pkg.go.dev/tailscale.com/tailcfg#DERPNode) for all available options.
+
+    ```yaml title="derp.yaml"
+    regions:
+      900:
+        regionid: 900
+        regioncode: custom-east
+        regionname: My region (east)
+        nodes:
+          - name: 900a
+            regionid: 900
+            hostname: derp900a.example.com
+            ipv4: 198.51.100.1
+            ipv6: 2001:db8::1
+            canport80: true
+      901:
+        regionid: 901
+        regioncode: custom-west
+        regionname: My Region (west)
+        nodes:
+          - name: 901a
+            regionid: 901
+            hostname: derp901a.example.com
+            ipv4: 198.51.100.2
+            ipv6: 2001:db8::2
+            canport80: true
+    ```
+
+    Use the following configuration to only serve the two DERP servers from the above `derp.yaml`:
+
+    ```yaml title="config.yaml" hl_lines="5 6"
+    derp:
+      server:
+        enabled: false
+      urls: []
+      paths:
+        - /etc/headscale/derp.yaml
+    ```
+
+Independent of the custom DERP map, you may choose to [enable the embedded DERP server and have it automatically added
+to the custom DERP map](#enable-embedded-derp).
+
+### Verify clients
+
+Access to DERP serves can be restricted to nodes that are members of your Tailnet. Relay access is denied for unknown
+clients.
+
+=== "Embedded DERP"
+
+    Client verification is enabled by default.
+
+    ```yaml title="config.yaml" hl_lines="3"
+    derp:
+      server:
+        verify_clients: true
+    ```
+
+=== "3rd-party DERP"
+
+    Tailscale's `derper` provides two parameters to configure client verification:
+
+    - Use the `-verify-client-url` parameter of the `derper` and point it towards the `/verify` endpoint of your
+      Headscale server (e.g `https://headscale.example.com/verify`). The DERP server will query your Headscale instance
+      as soon as a client connects with it to ask whether access should be allowed or denied. Access is allowed if
+      Headscale knows about the connecting client and denied otherwise.
+    - The parameter `-verify-client-url-fail-open` controls what should happen when the DERP server can't reach the
+      Headscale instance. By default, it will allow access if Headscale is unreachable.
+
+## Check DERP server connectivity
+
+Any Tailscale client may be used to introspect the DERP map and to check for connectivity issues with DERP servers.
+
+- Display DERP map: `tailscale debug derp-map`
+- Check connectivity with the embedded DERP[^1]:`tailscale debug derp headscale`
+
+Additional DERP related metrics and information is available via the [metrics and debug
+endpoint](./debug.md#metrics-and-debug-endpoint).
+
+[^1]:
+    This assumes that the default region code of the [configuration file](./configuration.md) is used.
+
+## Limitations
+
+- The embedded DERP server can't be used for Tailscale's captive portal checks as it doesn't support the `/generate_204`
+  endpoint via HTTP on port tcp/80.
+- There are no speed or throughput optimisations, the main purpose is to assist in node connectivity.

docs/ref/dns.md

@@ -1,7 +1,7 @@
 # DNS
 
 Headscale supports [most DNS features](../about/features.md) from Tailscale. DNS related settings can be configured
-within `dns` section of the [configuration file](./configuration.md).
+within the `dns` section of the [configuration file](./configuration.md).
 
 ## Setting extra DNS records
 
@@ -9,10 +9,10 @@ Headscale allows to set extra DNS records which are made available via
 [MagicDNS](https://tailscale.com/kb/1081/magicdns). Extra DNS records can be configured either via static entries in the
 [configuration file](./configuration.md) or from a JSON file that Headscale continuously watches for changes:
 
-* Use the `dns.extra_records` option in the [configuration file](./configuration.md) for entries that are static and
+- Use the `dns.extra_records` option in the [configuration file](./configuration.md) for entries that are static and
   don't change while Headscale is running. Those entries are processed when Headscale is starting up and changes to the
   configuration require a restart of Headscale.
-* For dynamic DNS records that may be added, updated or removed while Headscale is running or DNS records that are
+- For dynamic DNS records that may be added, updated or removed while Headscale is running or DNS records that are
   generated by scripts the option `dns.extra_records_path` in the [configuration file](./configuration.md) is useful.
   Set it to the absolute path of the JSON file containing DNS records and Headscale processes this file as it detects
   changes.
@@ -23,8 +23,7 @@ hostname and port combination "http://hostname-in-magic-dns.myvpn.example.com:30
 
 !!! warning "Limitations"
 
-    Currently, [only A and AAAA records are processed by Tailscale](https://github.com/tailscale/tailscale/blob/v1.78.3/ipn/ipnlocal/local.go#L4461-L4479).
-
+    Currently, [only A and AAAA records are processed by Tailscale](https://github.com/tailscale/tailscale/blob/v1.86.5/ipn/ipnlocal/node_backend.go#L662).
 
 1.  Configure extra DNS records using one of the available configuration options:
 

docs/ref/integration/reverse-proxy.md

@@ -13,7 +13,7 @@ Running headscale behind a reverse proxy is useful when running multiple applica
 
 The reverse proxy MUST be configured to support WebSockets to communicate with Tailscale clients.
 
-WebSockets support is also required when using the headscale embedded DERP server. In this case, you will also need to expose the UDP port used for STUN (by default, udp/3478). Please check our [config-example.yaml](https://github.com/juanfont/headscale/blob/main/config-example.yaml).
+WebSockets support is also required when using the Headscale [embedded DERP server](../derp.md). In this case, you will also need to expose the UDP port used for STUN (by default, udp/3478). Please check our [config-example.yaml](https://github.com/juanfont/headscale/blob/main/config-example.yaml).
 
 ### Cloudflare
 

docs/ref/integration/tools.md

@@ -5,10 +5,12 @@
     This page contains community contributions. The projects listed here are not
     maintained by the headscale authors and are written by community members.
 
-This page collects third-party tools and scripts related to headscale.
+This page collects third-party tools, client libraries, and scripts related to headscale.
 
 | Name                  | Repository Link                                                 | Description                                                          |
 | --------------------- | --------------------------------------------------------------- | -------------------------------------------------------------------- |
 | tailscale-manager     | [Github](https://github.com/singlestore-labs/tailscale-manager) | Dynamically manage Tailscale route advertisements                    |
 | headscalebacktosqlite | [Github](https://github.com/bigbozza/headscalebacktosqlite)     | Migrate headscale from PostgreSQL back to SQLite                     |
 | headscale-pf          | [Github](https://github.com/YouSysAdmin/headscale-pf)           | Populates user groups based on user groups in Jumpcloud or Authentik |
+| headscale-client-go   | [Github](https://github.com/hibare/headscale-client-go)         | A Go client implementation for the Headscale HTTP API.               |
+| headscale-zabbix      | [Github](https://github.com/dblanque/headscale-zabbix)          | A Zabbix Monitoring Template for the Headscale Service.              |

docs/ref/integration/web-ui.md

@@ -7,13 +7,14 @@
 
 Headscale doesn't provide a built-in web interface but users may pick one from the available options.
 
-| Name                   | Repository Link                                            | Description                                                                          |
-| ---------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------ |
-| headscale-ui           | [Github](https://github.com/gurucomputing/headscale-ui)    | A web frontend for the headscale Tailscale-compatible coordination server            |
-| HeadscaleUi            | [GitHub](https://github.com/simcu/headscale-ui)            | A static headscale admin ui, no backend environment required                         |
-| Headplane              | [GitHub](https://github.com/tale/headplane)                | An advanced Tailscale inspired frontend for headscale                                |
-| headscale-admin        | [Github](https://github.com/GoodiesHQ/headscale-admin)     | Headscale-Admin is meant to be a simple, modern web interface for headscale          |
-| ouroboros              | [Github](https://github.com/yellowsink/ouroboros)          | Ouroboros is designed for users to manage their own devices, rather than for admins  |
-| unraid-headscale-admin | [Github](https://github.com/ich777/unraid-headscale-admin) | A simple headscale admin UI for Unraid, it offers Local (`docker exec`) and API Mode |
+| Name                   | Repository Link                                             | Description                                                                                  |
+| ---------------------- | ----------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
+| headscale-ui           | [Github](https://github.com/gurucomputing/headscale-ui)     | A web frontend for the headscale Tailscale-compatible coordination server                    |
+| HeadscaleUi            | [GitHub](https://github.com/simcu/headscale-ui)             | A static headscale admin ui, no backend environment required                                 |
+| Headplane              | [GitHub](https://github.com/tale/headplane)                 | An advanced Tailscale inspired frontend for headscale                                        |
+| headscale-admin        | [Github](https://github.com/GoodiesHQ/headscale-admin)      | Headscale-Admin is meant to be a simple, modern web interface for headscale                  |
+| ouroboros              | [Github](https://github.com/yellowsink/ouroboros)           | Ouroboros is designed for users to manage their own devices, rather than for admins          |
+| unraid-headscale-admin | [Github](https://github.com/ich777/unraid-headscale-admin)  | A simple headscale admin UI for Unraid, it offers Local (`docker exec`) and API Mode         |
+| headscale-console      | [Github](https://github.com/rickli-cloud/headscale-console) | WebAssembly-based client supporting SSH, VNC and RDP with optional self-service capabilities |
 
 You can ask for support on our [Discord server](https://discord.gg/c84AZQhmpx) in the "web-interfaces" channel.

docs/ref/oidc.md

@@ -1,162 +1,259 @@
-# Configuring headscale to use OIDC authentication
+# OpenID Connect
 
-In order to authenticate users through a centralized solution one must enable the OIDC integration.
+Headscale supports authentication via external identity providers using OpenID Connect (OIDC). It features:
 
-Known limitations:
+- Auto configuration via OpenID Connect Discovery Protocol
+- [Proof Key for Code Exchange (PKCE) code verification](#enable-pkce-recommended)
+- [Authorization based on a user's domain, email address or group membership](#authorize-users-with-filters)
+- Synchronization of [standard OIDC claims](#supported-oidc-claims)
 
-- No dynamic ACL support
-- OIDC groups cannot be used in ACLs
+Please see [limitations](#limitations) for known issues and limitations.
 
-## Basic configuration
+## Configuration
 
-In your `config.yaml`, customize this to your liking:
+OpenID requires configuration in Headscale and your identity provider:
 
-```yaml title="config.yaml"
-oidc:
-  # Block further startup until the OIDC provider is healthy and available
-  only_start_if_oidc_is_available: true
-  # Specified by your OIDC provider
-  issuer: "https://your-oidc.issuer.com/path"
-  # Specified/generated by your OIDC provider
-  client_id: "your-oidc-client-id"
-  client_secret: "your-oidc-client-secret"
-  # alternatively, set `client_secret_path` to read the secret from the file.
-  # It resolves environment variables, making integration to systemd's
-  # `LoadCredential` straightforward:
-  #client_secret_path: "${CREDENTIALS_DIRECTORY}/oidc_client_secret"
-  # as third option, it's also possible to load the oidc secret from environment variables
-  # set HEADSCALE_OIDC_CLIENT_SECRET to the required value
+- Headscale: The `oidc` section of the Headscale [configuration](configuration.md) contains all available configuration
+  options along with a description and their default values.
+- Identity provider: Please refer to the official documentation of your identity provider for specific instructions.
+  Additionally, there might be some useful hints in the [Identity provider specific
+  configuration](#identity-provider-specific-configuration) section below.
 
-  # Customize the scopes used in the OIDC flow, defaults to "openid", "profile" and "email" and add custom query
-  # parameters to the Authorize Endpoint request. Scopes default to "openid", "profile" and "email".
-  scope: ["openid", "profile", "email", "custom"]
-  # Optional: Passed on to the browser login request – used to tweak behaviour for the OIDC provider
-  extra_params:
-    domain_hint: example.com
+### Basic configuration
 
-  # Optional: List allowed principal domains and/or users. If an authenticated user's domain is not in this list,
-  # the authentication request will be rejected.
-  allowed_domains:
-    - example.com
-  # Optional. Note that groups from Keycloak have a leading '/'.
-  allowed_groups:
-    - /headscale
-  # Optional.
-  allowed_users:
-    - alice@example.com
+A basic configuration connects Headscale to an identity provider and typically requires:
 
-  # Optional: PKCE (Proof Key for Code Exchange) configuration
-  # PKCE adds an additional layer of security to the OAuth 2.0 authorization code flow
-  # by preventing authorization code interception attacks
-  # See https://datatracker.ietf.org/doc/html/rfc7636
-  pkce:
-    # Enable or disable PKCE support (default: false)
-    enabled: false
-    # PKCE method to use:
-    # - plain: Use plain code verifier
-    # - S256: Use SHA256 hashed code verifier (default, recommended)
-    method: S256
-```
+- OpenID Connect Issuer URL from the identity provider. Headscale uses the OpenID Connect Discovery Protocol 1.0 to
+  automatically obtain OpenID configuration parameters (example: `https://sso.example.com`).
+- Client ID from the identity provider (example: `headscale`).
+- Client secret generated by the identity provider (example: `generated-secret`).
+- Redirect URI for your identity provider (example: `https://headscale.example.com/oidc/callback`).
 
-## Azure AD example
+=== "Headscale"
 
-In order to integrate headscale with Azure Active Directory, we'll need to provision an App Registration with the correct scopes and redirect URI. Here with Terraform:
+    ```yaml
+    oidc:
+      issuer: "https://sso.example.com"
+      client_id: "headscale"
+      client_secret: "generated-secret"
+    ```
 
-```hcl title="terraform.hcl"
-resource "azuread_application" "headscale" {
-  display_name = "Headscale"
+=== "Identity provider"
 
-  sign_in_audience = "AzureADMyOrg"
-  fallback_public_client_enabled = false
+    * Create a new confidential client (`Client ID`, `Client secret`)
+    * Add Headscale's OIDC callback URL as valid redirect URL: `https://headscale.example.com/oidc/callback`
+    * Configure additional parameters to improve user experience such as: name, description, logo, …
 
-  required_resource_access {
-    // Microsoft Graph
-    resource_app_id = "00000003-0000-0000-c000-000000000000"
+### Enable PKCE (recommended)
 
-    resource_access {
-      // scope: profile
-      id   = "14dad69e-099b-42c9-810b-d002981feec1"
-      type = "Scope"
-    }
-    resource_access {
-      // scope: openid
-      id   = "37f7f235-527c-4136-accd-4a02d197296e"
-      type = "Scope"
-    }
-    resource_access {
-      // scope: email
-      id   = "64a6cdd6-aab1-4aaf-94b8-3cc8405e90d0"
-      type = "Scope"
-    }
-  }
-  web {
-    # Points at your running headscale instance
-    redirect_uris = ["https://headscale.example.com/oidc/callback"]
+Proof Key for Code Exchange (PKCE) adds an additional layer of security to the OAuth 2.0 authorization code flow by
+preventing authorization code interception attacks, see: <https://datatracker.ietf.org/doc/html/rfc7636>. PKCE is
+recommended and needs to be configured for Headscale and the identity provider alike:
 
-    implicit_grant {
-      access_token_issuance_enabled = false
-      id_token_issuance_enabled = true
-    }
-  }
+=== "Headscale"
 
-  group_membership_claims = ["SecurityGroup"]
-  optional_claims {
-    # Expose group memberships
-    id_token {
-      name = "groups"
-    }
-  }
-}
+    ```yaml hl_lines="5-6"
+    oidc:
+      issuer: "https://sso.example.com"
+      client_id: "headscale"
+      client_secret: "generated-secret"
+      pkce:
+        enabled: true
+    ```
 
-resource "azuread_application_password" "headscale-application-secret" {
-  display_name          = "Headscale Server"
-  application_object_id = azuread_application.headscale.object_id
-}
+=== "Identity provider"
 
-resource "azuread_service_principal" "headscale" {
-  application_id = azuread_application.headscale.application_id
-}
+    * Enable PKCE for the headscale client
+    * Set the PKCE challenge method to "S256"
 
-resource "azuread_service_principal_password" "headscale" {
-  service_principal_id = azuread_service_principal.headscale.id
-  end_date_relative    = "44640h"
-}
+### Authorize users with filters
 
-output "headscale_client_id" {
-  value = azuread_application.headscale.application_id
-}
+Headscale allows to filter for allowed users based on their domain, email address or group membership. These filters can
+be helpful to apply additional restrictions and control which users are allowed to join. Filters are disabled by
+default, users are allowed to join once the authentication with the identity provider succeeds. In case multiple filters
+are configured, a user needs to pass all of them.
 
-output "headscale_client_secret" {
-  value = azuread_application_password.headscale-application-secret.value
-}
-```
+=== "Allowed domains"
 
-And in your headscale `config.yaml`:
+    * Check the email domain of each authenticating user against the list of allowed domains and only authorize users
+      whose email domain matches `example.com`.
+    * Access allowed: `alice@example.com`
+    * Access denied: `bob@example.net`
 
-```yaml title="config.yaml"
-oidc:
-  issuer: "https://login.microsoftonline.com/<tenant-UUID>/v2.0"
-  client_id: "<client-id-from-terraform>"
-  client_secret: "<client-secret-from-terraform>"
+    ```yaml hl_lines="5-6"
+    oidc:
+      issuer: "https://sso.example.com"
+      client_id: "headscale"
+      client_secret: "generated-secret"
+      allowed_domains:
+        - "example.com"
+    ```
 
-  # Optional: add "groups"
-  scope: ["openid", "profile", "email"]
-  extra_params:
-    # Use your own domain, associated with Azure AD
-    domain_hint: example.com
-    # Optional: Force the Azure AD account picker
-    prompt: select_account
-```
+=== "Allowed users/emails"
 
-## Google OAuth Example
+    * Check the email address of each authenticating user against the list of allowed email addresses and only authorize
+      users whose email is part of the `allowed_users` list.
+    * Access allowed: `alice@example.com`, `bob@example.net`
+    * Access denied: `mallory@example.net`
 
-In order to integrate headscale with Google, you'll need to have a [Google Cloud Console](https://console.cloud.google.com) account.
+    ```yaml hl_lines="5-7"
+    oidc:
+      issuer: "https://sso.example.com"
+      client_id: "headscale"
+      client_secret: "generated-secret"
+      allowed_users:
+        - "alice@example.com"
+        - "bob@example.net"
+    ```
 
-Google OAuth has a [verification process](https://support.google.com/cloud/answer/9110914?hl=en) if you need to have users authenticate who are outside of your domain. If you only need to authenticate users from your domain name (ie `@example.com`), you don't need to go through the verification process.
+=== "Allowed groups"
 
-However if you don't have a domain, or need to add users outside of your domain, you can manually add emails via Google Console.
+    * Use the OIDC `groups` claim of each authenticating user to get their group membership and only authorize users
+      which are members in at least one of the referenced groups.
+    * Access allowed: users in the `headscale_users` group
+    * Access denied: users without groups, users with other groups
 
-### Steps
+    ```yaml hl_lines="5-7"
+    oidc:
+      issuer: "https://sso.example.com"
+      client_id: "headscale"
+      client_secret: "generated-secret"
+      scope: ["openid", "profile", "email", "groups"]
+      allowed_groups:
+        - "headscale_users"
+    ```
+
+### Customize node expiration
+
+The node expiration is the amount of time a node is authenticated with OpenID Connect until it expires and needs to
+reauthenticate. The default node expiration is 180 days. This can either be customized or set to the expiration from the
+Access Token.
+
+=== "Customize node expiration"
+
+    ```yaml hl_lines="5"
+    oidc:
+      issuer: "https://sso.example.com"
+      client_id: "headscale"
+      client_secret: "generated-secret"
+      expiry: 30d   # Use 0 to disable node expiration
+    ```
+
+=== "Use expiration from Access Token"
+
+    Please keep in mind that the Access Token is typically a short-lived token that expires within a few minutes. You
+    will have to configure token expiration in your identity provider to avoid frequent re-authentication.
+
+
+    ```yaml hl_lines="5"
+    oidc:
+      issuer: "https://sso.example.com"
+      client_id: "headscale"
+      client_secret: "generated-secret"
+      use_expiry_from_token: true
+    ```
+
+!!! tip "Expire a node and force re-authentication"
+
+    A node can be expired immediately via:
+    ```console
+    headscale node expire -i <NODE_ID>
+    ```
+
+### Reference a user in the policy
+
+You may refer to users in the Headscale policy via:
+
+- Email address
+- Username
+- Provider identifier (only available in the database or from your identity provider)
+
+!!! note "A user identifier in the policy must contain a single `@`"
+
+    The Headscale policy requires a single `@` to reference a user. If the username or provider identifier doesn't
+    already contain a single `@`, it needs to be appended at the end. For example: the username `ssmith` has to be
+    written as `ssmith@` to be correctly identified as user within the policy.
+
+!!! warning "Email address or username might be updated by users"
+
+    Many identity providers allow users to update their own profile. Depending on the identity provider and its
+    configuration, the values for username or email address might change over time. This might have unexpected
+    consequences for Headscale where a policy might no longer work or a user might obtain more access by hijacking an
+    existing username or email address.
+
+## Supported OIDC claims
+
+Headscale uses [the standard OIDC claims](https://openid.net/specs/openid-connect-core-1_0.html#StandardClaims) to
+populate and update its local user profile on each login. OIDC claims are read from the ID Token and from the UserInfo
+endpoint.
+
+| Headscale profile   | OIDC claim           | Notes / examples                                                                                  |
+| ------------------- | -------------------- | ------------------------------------------------------------------------------------------------- |
+| email address       | `email`              | Only used when `email_verified: true`                                                             |
+| display name        | `name`               | eg: `Sam Smith`                                                                                   |
+| username            | `preferred_username` | Depends on identity provider, eg: `ssmith`, `ssmith@idp.example.com`, `\\example.com\ssmith`      |
+| profile picture     | `picture`            | URL to a profile picture or avatar                                                                |
+| provider identifier | `iss`, `sub`         | A stable and unique identifier for a user, typically a combination of `iss` and `sub` OIDC claims |
+|                     | `groups`             | [Only used to filter for allowed groups](#authorize-users-with-filters)                           |
+
+## Limitations
+
+- Support for OpenID Connect aims to be generic and vendor independent. It offers only limited support for quirks of
+  specific identity providers.
+- OIDC groups cannot be used in ACLs.
+- The username provided by the identity provider needs to adhere to this pattern:
+    - The username must be at least two characters long.
+    - It must only contain letters, digits, hyphens, dots, underscores, and up to a single `@`.
+    - The username must start with a letter.
+- A user's email address is only synchronized to the local user profile when the identity provider marks the email
+  address as verified (`email_verified: true`).
+
+Please see the [GitHub label "OIDC"](https://github.com/juanfont/headscale/labels/OIDC) for OIDC related issues.
+
+## Identity provider specific configuration
+
+!!! warning "Third-party software and services"
+
+    This section of the documentation is specific for third-party software and services. We recommend users read the
+    third-party documentation on how to configure and integrate an OIDC client. Please see the [Configuration
+    section](#configuration) for a description of Headscale's OIDC related configuration settings.
+
+Any identity provider with OpenID Connect support should "just work" with Headscale. The following identity providers
+are known to work:
+
+- [Authelia](#authelia)
+- [Authentik](#authentik)
+- [Kanidm](#kanidm)
+- [Keycloak](#keycloak)
+
+### Authelia
+
+Authelia is fully supported by Headscale.
+
+### Authentik
+
+- Authentik is fully supported by Headscale.
+- [Headscale does not JSON Web Encryption](https://github.com/juanfont/headscale/issues/2446). Leave the field
+  `Encryption Key` in the providers section unset.
+
+### Google OAuth
+
+!!! warning "No username due to missing preferred_username"
+
+    Google OAuth does not send the `preferred_username` claim when the scope `profile` is requested. The username in
+    Headscale will be blank/not set.
+
+In order to integrate Headscale with Google, you'll need to have a [Google Cloud
+Console](https://console.cloud.google.com) account.
+
+Google OAuth has a [verification process](https://support.google.com/cloud/answer/9110914?hl=en) if you need to have
+users authenticate who are outside of your domain. If you only need to authenticate users from your domain name (ie
+`@example.com`), you don't need to go through the verification process.
+
+However if you don't have a domain, or need to add users outside of your domain, you can manually add emails via Google
+Console.
+
+#### Steps
 
 1. Go to [Google Console](https://console.cloud.google.com) and login or create an account if you don't have one.
 2. Create a project (if you don't already have one).
@@ -164,50 +261,49 @@ However if you don't have a domain, or need to add users outside of your domain,
 4. Click `Create Credentials` -> `OAuth client ID`
 5. Under `Application Type`, choose `Web Application`
 6. For `Name`, enter whatever you like
-7. Under `Authorised redirect URIs`, use `https://example.com/oidc/callback`, replacing example.com with your headscale URL.
+7. Under `Authorised redirect URIs`, add Headscale's OIDC callback URL: `https://headscale.example.com/oidc/callback`
 8. Click `Save` at the bottom of the form
 9. Take note of the `Client ID` and `Client secret`, you can also download it for reference if you need it.
-10. Edit your headscale config, under `oidc`, filling in your `client_id` and `client_secret`:
-    ```yaml title="config.yaml"
-    oidc:
-      issuer: "https://accounts.google.com"
-      client_id: ""
-      client_secret: ""
-      scope: ["openid", "profile", "email"]
-    ```
+10. [Configure Headscale following the "Basic configuration" steps](#basic-configuration). The issuer URL for Google
+    OAuth is: `https://accounts.google.com`.
 
-You can also use `allowed_domains` and `allowed_users` to restrict the users who can authenticate.
+### Kanidm
 
-## Authelia
-Authelia since v4.39.0, has removed most claims from the `ID Token`, they are still available when application queries [UserInfo Endpoint](https://openid.net/specs/openid-connect-core-1_0.html#UserInfo). 
+- Kanidm is fully supported by Headscale.
+- Groups for the [allowed groups filter](#authorize-users-with-filters) need to be specified with their full SPN, for
+  example: `headscale_users@sso.example.com`.
 
-Following config restores sending 'default' claims in the `ID Token`
+### Keycloak
 
-For more information please read: [Authelia restore functionality prior to claims parameter](https://www.authelia.com/integration/openid-connect/openid-connect-1.0-claims/#restore-functionality-prior-to-claims-parameter)
+Keycloak is fully supported by Headscale.
 
+#### Additional configuration to use the allowed groups filter
 
-```yaml
-identity_providers:
-  oidc:
-    claims_policies:
-      default:
-        id_token: ['groups', 'email', 'email_verified', 'alt_emails', 'preferred_username', 'name']
-    clients:
-      - client_id: 'headscale'
-        client_name: 'headscale'
-        client_secret: ''
-        public: false
-        claims_policy: 'default'
-        authorization_policy: 'two_factor'
-        require_pkce: true
-        pkce_challenge_method: 'S256'
-        redirect_uris:
-          - 'https://headscale.example.com/oidc/callback'
-        scopes:
-          - 'openid'
-          - 'profile'
-          - 'groups'
-          - 'email'
-        userinfo_signed_response_alg: 'none'
-        token_endpoint_auth_method: 'client_secret_basic'
-```
+Keycloak has no built-in client scope for the OIDC `groups` claim. This extra configuration step is **only** needed if
+you need to [authorize access based on group membership](#authorize-users-with-filters).
+
+- Create a new client scope `groups` for OpenID Connect:
+    - Configure a `Group Membership` mapper with name `groups` and the token claim name `groups`.
+    - Add the mapper to at least the UserInfo endpoint.
+- Configure the new client scope for your Headscale client:
+    - Edit the Headscale client.
+    - Search for the client scope `group`.
+    - Add it with assigned type `Default`.
+- [Configure the allowed groups in Headscale](#authorize-users-with-filters). How groups need to be specified depends on
+  Keycloak's `Full group path` option:
+    - `Full group path` is enabled: groups contain their full path, e.g. `/top/group1`
+    - `Full group path` is disabled: only the name of the group is used, e.g. `group1`
+
+### Microsoft Entra ID
+
+In order to integrate Headscale with Microsoft Entra ID, you'll need to provision an App Registration with the correct
+scopes and redirect URI.
+
+[Configure Headscale following the "Basic configuration" steps](#basic-configuration). The issuer URL for Microsoft
+Entra ID is: `https://login.microsoftonline.com/<tenant-UUID>/v2.0`. The following `extra_params` might be useful:
+
+- `domain_hint: example.com` to use your own domain
+- `prompt: select_account` to force an account picker during login
+
+Groups for the [allowed groups filter](#authorize-users-with-filters) need to be specified with their group ID instead
+of the group name.

docs/ref/remote-cli.md

@@ -67,12 +67,6 @@ headscale apikeys expire --prefix "<PREFIX>"
         export HEADSCALE_CLI_API_KEY="<API_KEY_FROM_PREVIOUS_STEP>"
         ```
 
-        !!! bug
-
-            Headscale currently requires at least an empty configuration file when environment variables are used to
-            specify connection details. See [issue 2193](https://github.com/juanfont/headscale/issues/2193) for more
-            information.
-
     This instructs the `headscale` binary to connect to a remote instance at `<HEADSCALE_ADDRESS>:<PORT>`, instead of
     connecting to the local instance.
 

docs/ref/routes.md

@@ -1,4 +1,5 @@
 # Routes
+
 Headscale supports route advertising and can be used to manage [subnet routers](https://tailscale.com/kb/1019/subnets)
 and [exit nodes](https://tailscale.com/kb/1103/exit-nodes) for a tailnet.
 
@@ -10,11 +11,13 @@ and [exit nodes](https://tailscale.com/kb/1103/exit-nodes) for a tailnet.
   from a specific IP address.
 
 ## Subnet router
+
 The setup of a subnet router requires double opt-in, once from a subnet router and once on the control server to allow
 its use within the tailnet. Optionally, use [`autoApprovers` to automatically approve routes from a subnet
 router](#automatically-approve-routes-of-a-subnet-router).
 
 ### Setup a subnet router
+
 #### Configure a node as subnet router
 
 Register a node and advertise the routes it should handle as comma separated list:
@@ -31,7 +34,6 @@ $ sudo tailscale set --advertise-routes=10.0.0.0/8,192.168.0.0/24
 
 Finally, [enable IP forwarding](#enable-ip-forwarding) to route traffic.
 
-
 #### Enable the subnet router on the control server
 
 The routes of a tailnet can be displayed with the `headscale nodes list-routes` command. A subnet router with the
@@ -72,6 +74,7 @@ documentation](https://tailscale.com/kb/1019/subnets#use-your-subnet-routes-from
 router on different operating systems.
 
 ### Restrict the use of a subnet router with ACL
+
 The routes announced by subnet routers are available to the nodes in a tailnet. By default, without an ACL enabled, all
 nodes can accept and use such routes. Configure an ACL to explicitly manage who can use routes.
 
@@ -91,18 +94,15 @@ denied.
   "acls": [
     {
       "action": "accept",
-      "src": [
-        "node"
-      ],
-      "dst": [
-        "service.example.net:80,443"
-      ]
+      "src": ["node"],
+      "dst": ["service.example.net:80,443"]
     }
   ]
 }
 ```
 
 ### Automatically approve routes of a subnet router
+
 The initial setup of a subnet router usually requires manual approval of their announced routes on the control server
 before they can be used by a node in a tailnet. Headscale supports the `autoApprovers` section of an ACL to automate the
 approval of routes served with a subnet router.
@@ -114,15 +114,11 @@ owned by the user `alice` and that also advertises the tag `tag:router`.
 ```json title="Subnet routers owned by alice and tagged with tag:router are automatically approved"
 {
   "tagOwners": {
-    "tag:router": [
-      "alice@"
-    ]
+    "tag:router": ["alice@"]
   },
   "autoApprovers": {
     "routes": {
-      "192.168.0.0/24": [
-        "tag:router"
-      ]
+      "192.168.0.0/24": ["tag:router"]
     }
   },
   "acls": [
@@ -141,11 +137,13 @@ Please see the [official Tailscale documentation](https://tailscale.com/kb/1337/
 information on auto approvers.
 
 ## Exit node
+
 The setup of an exit node requires double opt-in, once from an exit node and once on the control server to allow its use
 within the tailnet. Optionally, use [`autoApprovers` to automatically approve an exit
 node](#automatically-approve-an-exit-node-with-auto-approvers).
 
 ### Setup an exit node
+
 #### Configure a node as exit node
 
 Register a node and make it advertise itself as an exit node:
@@ -162,7 +160,6 @@ $ sudo tailscale set --advertise-exit-node
 
 Finally, [enable IP forwarding](#enable-ip-forwarding) to route traffic.
 
-
 #### Enable the exit node on the control server
 
 The routes of a tailnet can be displayed with the `headscale nodes list-routes` command. An exit node can be recognized
@@ -202,8 +199,9 @@ Please refer to the official [Tailscale documentation](https://tailscale.com/kb/
 how to use an exit node on different operating systems.
 
 ### Restrict the use of an exit node with ACL
+
 An exit node is offered to all nodes in a tailnet. By default, without an ACL enabled, all nodes in a tailnet can select
-and use an exit node. Configure `autogroup:internet` in an ACL rule to restrict who can use *any* of the available exit
+and use an exit node. Configure `autogroup:internet` in an ACL rule to restrict who can use _any_ of the available exit
 nodes.
 
 ```json title="Example use of autogroup:internet"
@@ -211,18 +209,15 @@ nodes.
   "acls": [
     {
       "action": "accept",
-      "src": [
-        "..."
-      ],
-      "dst": [
-        "autogroup:internet:*"
-      ]
+      "src": ["..."],
+      "dst": ["autogroup:internet:*"]
     }
   ]
 }
 ```
 
 ### Automatically approve an exit node with auto approvers
+
 The initial setup of an exit node usually requires manual approval on the control server before it can be used by a node
 in a tailnet. Headscale supports the `autoApprovers` section of an ACL to automate the approval of a new exit node as
 soon as it joins the tailnet.
@@ -234,14 +229,10 @@ is automatically approved:
 ```json title="Exit nodes owned by alice and tagged with tag:exit are automatically approved"
 {
   "tagOwners": {
-    "tag:exit": [
-      "alice@"
-    ]
+    "tag:exit": ["alice@"]
   },
   "autoApprovers": {
-    "exitNode": [
-      "tag:exit"
-    ]
+    "exitNode": ["tag:exit"]
   },
   "acls": [
     // more rules
@@ -272,6 +263,7 @@ availability](https://tailscale.com/kb/1115/high-availability#subnet-router-high
     interruptions for clients. See [issue 2129](https://github.com/juanfont/headscale/issues/2129) for more information.
 
 ## Troubleshooting
+
 ### Enable IP forwarding
 
 A subnet router or exit node is routing traffic on behalf of other nodes and thus requires IP forwarding. Check the

docs/ref/tls.md

@@ -2,7 +2,7 @@
 
 ## Bring your own certificate
 
-Headscale can be configured to expose its web service via TLS. To configure the certificate and key file manually, set the `tls_cert_path` and `tls_cert_path` configuration parameters. If the path is relative, it will be interpreted as relative to the directory the configuration file was read from.
+Headscale can be configured to expose its web service via TLS. To configure the certificate and key file manually, set the `tls_cert_path` and `tls_key_path` configuration parameters. If the path is relative, it will be interpreted as relative to the directory the configuration file was read from.
 
 ```yaml title="config.yaml"
 tls_cert_path: ""

docs/setup/install/container.md

@@ -39,6 +39,7 @@ Registry](https://github.com/juanfont/headscale/pkgs/container/headscale). The c
       --volume "$(pwd)/run:/var/run/headscale" \
       --publish 127.0.0.1:8080:8080 \
       --publish 127.0.0.1:9090:9090 \
+      --health-cmd "CMD headscale health" \
       docker.io/headscale/headscale:<VERSION> \
       serve
     ```
@@ -66,6 +67,8 @@ Registry](https://github.com/juanfont/headscale/pkgs/container/headscale). The c
           - <HEADSCALE_PATH>/lib:/var/lib/headscale
           - <HEADSCALE_PATH>/run:/var/run/headscale
         command: serve
+        healthcheck:
+            test: ["CMD", "headscale", "health"]
     ```
 
 1.  Verify headscale is running:
@@ -112,11 +115,11 @@ docker exec -it headscale \
 
 ### Register a machine using a pre authenticated key
 
-Generate a key using the command line:
+Generate a key using the command line for the user with ID 1:
 
 ```shell
 docker exec -it headscale \
-  headscale preauthkeys create --user myfirstuser --reusable --expiration 24h
+  headscale preauthkeys create --user 1 --reusable --expiration 24h
 ```
 
 This will return a pre-authenticated key that can be used to connect a node to headscale with the `tailscale up` command:

docs/setup/install/official.md

@@ -7,7 +7,7 @@ Both are available on the [GitHub releases page](https://github.com/juanfont/hea
 
 It is recommended to use our DEB packages to install headscale on a Debian based system as those packages configure a
 local user to run headscale, provide a default configuration and ship with a systemd service file. Supported
-distributions are Ubuntu 20.04 or newer, Debian 11 or newer.
+distributions are Ubuntu 22.04 or newer, Debian 11 or newer.
 
 1.  Download the [latest headscale package](https://github.com/juanfont/headscale/releases/latest) for your platform (`.deb` for Ubuntu and Debian).
 
@@ -57,14 +57,14 @@ managed by systemd.
 1.  Download the latest [`headscale` binary from GitHub's release page](https://github.com/juanfont/headscale/releases):
 
     ```shell
-    sudo wget --output-document=/usr/local/bin/headscale \
+    sudo wget --output-document=/usr/bin/headscale \
     https://github.com/juanfont/headscale/releases/download/v<HEADSCALE VERSION>/headscale_<HEADSCALE VERSION>_linux_<ARCH>
     ```
 
 1.  Make `headscale` executable:
 
     ```shell
-    sudo chmod +x /usr/local/bin/headscale
+    sudo chmod +x /usr/bin/headscale
     ```
 
 1.  Add a dedicated local user to run headscale:
@@ -87,8 +87,8 @@ managed by systemd.
     sudo nano /etc/headscale/config.yaml
     ```
 
-1.  Copy [headscale's systemd service file](../../packaging/headscale.systemd.service) to
-    `/etc/systemd/system/headscale.service` and adjust it to suit your local setup. The following parameters likely need
+1.  Copy [headscale's systemd service file](https://github.com/juanfont/headscale/blob/main/packaging/systemd/headscale.service)
+    to `/etc/systemd/system/headscale.service` and adjust it to suit your local setup. The following parameters likely need
     to be modified: `ExecStart`, `WorkingDirectory`, `ReadWritePaths`.
 
 1.  In `/etc/headscale/config.yaml`, override the default `headscale` unix socket with a path that is writable by the

docs/setup/requirements.md

@@ -4,11 +4,35 @@ Headscale should just work as long as the following requirements are met:
 
 - A server with a public IP address for headscale. A dual-stack setup with a public IPv4 and a public IPv6 address is
   recommended.
-- Headscale is served via HTTPS on port 443[^1].
+- Headscale is served via HTTPS on port 443[^1] and [may use additional ports](#ports-in-use).
 - A reasonably modern Linux or BSD based operating system.
 - A dedicated local user account to run headscale.
 - A little bit of command line knowledge to configure and operate headscale.
 
+## Ports in use
+
+The ports in use vary with the intended scenario and enabled features. Some of the listed ports may be changed via the
+[configuration file](../ref/configuration.md) but we recommend to stick with the default values.
+
+- tcp/80
+    - Expose publicly: yes
+    - HTTP, used by Let's Encrypt to verify ownership via the HTTP-01 challenge.
+    - Only required if the built-in Let's Enrypt client with the HTTP-01 challenge is used. See [TLS](../ref/tls.md) for
+      details.
+- tcp/443
+    - Expose publicly: yes
+    - HTTPS, required to make Headscale available to Tailscale clients[^1]
+    - Required if the [embedded DERP server](../ref/derp.md) is enabled
+- udp/3478
+    - Expose publicly: yes
+    - STUN, required if the [embedded DERP server](../ref/derp.md) is enabled
+- tcp/50443
+    - Expose publicly: yes
+    - Only required if the gRPC interface is used to [remote-control Headscale](../ref/remote-cli.md).
+- tcp/9090
+    - Expose publicly: no
+    - [Metrics and debug endpoint](../ref/debug.md#metrics-and-debug-endpoint)
+
 ## Assumptions
 
 The headscale documentation and the provided examples are written with a few assumptions in mind:

docs/usage/connect/android.md

@@ -6,9 +6,23 @@ This documentation has the goal of showing how a user can use the official Andro
 
 Install the official Tailscale Android client from the [Google Play Store](https://play.google.com/store/apps/details?id=com.tailscale.ipn) or [F-Droid](https://f-droid.org/packages/com.tailscale.ipn/).
 
-## Configuring the headscale URL
+## Connect via normal, interactive login
 
 - Open the app and select the settings menu in the upper-right corner
 - Tap on `Accounts`
 - In the kebab menu icon (three dots) in the upper-right corner select `Use an alternate server`
 - Enter your server URL (e.g `https://headscale.example.com`) and follow the instructions
+- The client connects automatically as soon as the node registration is complete on headscale. Until then, nothing is
+  visible in the server logs.
+
+## Connect using a preauthkey
+
+- Open the app and select the settings menu in the upper-right corner
+- Tap on `Accounts`
+- In the kebab menu icon (three dots) in the upper-right corner select `Use an alternate server`
+- Enter your server URL (e.g `https://headscale.example.com`). If login prompts open, close it and continue
+- Open the settings menu in the upper-right corner
+- Tap on `Accounts`
+- In the kebab menu icon (three dots) in the upper-right corner select `Use an auth key`
+- Enter your [preauthkey generated from headscale](../getting-started.md#using-a-preauthkey)
+- If needed, tap `Log in` on the main screen. You should now be connected to your headscale.

docs/usage/getting-started.md

@@ -117,14 +117,14 @@ headscale instance. By default, the key is valid for one hour and can only be us
 === "Native"
 
     ```shell
-    headscale preauthkeys create --user <USER>
+    headscale preauthkeys create --user <USER_ID>
     ```
 
 === "Container"
 
     ```shell
     docker exec -it headscale \
-      headscale preauthkeys create --user <USER>
+      headscale preauthkeys create --user <USER_ID>
     ```
 
 The command returns the preauthkey on success which is used to connect a node to the headscale instance via the

flake.lock

@@ -20,11 +20,11 @@
     },
     "nixpkgs": {
       "locked": {
-        "lastModified": 1746300365,
-        "narHash": "sha256-thYTdWqCRipwPRxWiTiH1vusLuAy0okjOyzRx4hLWh4=",
+        "lastModified": 1760533177,
+        "narHash": "sha256-OwM1sFustLHx+xmTymhucZuNhtq98fHIbfO8Swm5L8A=",
         "owner": "NixOS",
         "repo": "nixpkgs",
-        "rev": "f21e4546e3ede7ae34d12a84602a22246b31f7e0",
+        "rev": "35f590344ff791e6b1d6d6b8f3523467c9217caf",
         "type": "github"
       },
       "original": {

flake.nix

@@ -18,7 +18,8 @@
     {
       overlay = _: prev: let
         pkgs = nixpkgs.legacyPackages.${prev.system};
-        buildGo = pkgs.buildGo124Module;
+        buildGo = pkgs.buildGo125Module;
+        vendorHash = "sha256-GUIzlPRsyEq1uSTzRNds9p1uVu4pTeH5PAxrJ5Njhis=";
       in {
         headscale = buildGo {
           pname = "headscale";
@@ -30,7 +31,7 @@
 
           # When updating go.mod or go.sum, a new sha will need to be calculated,
           # update this if you have a mismatch after doing a change to those files.
-          vendorHash = "sha256-dR8xmUIDMIy08lhm7r95GNNMAbXv4qSH3v9HR40HlNk=";
+          inherit vendorHash;
 
           subPackages = ["cmd/headscale"];
 
@@ -42,6 +43,17 @@
           ];
         };
 
+        hi = buildGo {
+          pname = "hi";
+          version = headscaleVersion;
+          src = pkgs.lib.cleanSource self;
+
+          checkFlags = ["-short"];
+          inherit vendorHash;
+
+          subPackages = ["cmd/hi"];
+        };
+
         protoc-gen-grpc-gateway = buildGo rec {
           pname = "grpc-gateway";
           version = "2.24.0";
@@ -85,9 +97,10 @@
         #   buildGoModule = buildGo;
         # };
 
-        goreleaser = prev.goreleaser.override {
-          buildGoModule = buildGo;
-        };
+        # The package uses buildGo125Module, not the convention.
+        # goreleaser = prev.goreleaser.override {
+        #   buildGoModule = buildGo;
+        # };
 
         gotestsum = prev.gotestsum.override {
           buildGoModule = buildGo;
@@ -101,9 +114,9 @@
           buildGoModule = buildGo;
         };
 
-        gopls = prev.gopls.override {
-          buildGoModule = buildGo;
-        };
+        # gopls = prev.gopls.override {
+        #   buildGoModule = buildGo;
+        # };
       };
     }
     // flake-utils.lib.eachDefaultSystem
@@ -112,7 +125,7 @@
         overlays = [self.overlay];
         inherit system;
       };
-      buildDeps = with pkgs; [git go_1_24 gnumake];
+      buildDeps = with pkgs; [git go_1_25 gnumake];
       devDeps = with pkgs;
         buildDeps
         ++ [
@@ -144,7 +157,11 @@
           buf
           clang-tools # clang-format
           protobuf-language-server
-        ];
+
+          # Add hi to make it even easier to use ci runner.
+          hi
+        ]
+        ++ lib.optional pkgs.stdenv.isLinux [traceroute];
 
       # Add entry to build a docker image with headscale
       # caveat: only works on Linux

gen/go/headscale/v1/apikey.pb.go

@@ -1,6 +1,6 @@
 // Code generated by protoc-gen-go. DO NOT EDIT.
 // versions:
-// 	protoc-gen-go v1.36.6
+// 	protoc-gen-go v1.36.8
 // 	protoc        (unknown)
 // source: headscale/v1/apikey.proto
 

gen/go/headscale/v1/device.pb.go

@@ -1,6 +1,6 @@
 // Code generated by protoc-gen-go. DO NOT EDIT.
 // versions:
-// 	protoc-gen-go v1.36.6
+// 	protoc-gen-go v1.36.8
 // 	protoc        (unknown)
 // source: headscale/v1/device.proto
 

gen/go/headscale/v1/headscale.pb.go

@@ -1,6 +1,6 @@
 // Code generated by protoc-gen-go. DO NOT EDIT.
 // versions:
-// 	protoc-gen-go v1.36.6
+// 	protoc-gen-go v1.36.8
 // 	protoc        (unknown)
 // source: headscale/v1/headscale.proto
 
@@ -11,6 +11,7 @@ import (
 	protoreflect "google.golang.org/protobuf/reflect/protoreflect"
 	protoimpl "google.golang.org/protobuf/runtime/protoimpl"
 	reflect "reflect"
+	sync "sync"
 	unsafe "unsafe"
 )
 
@@ -21,11 +22,94 @@ const (
 	_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
 )
 
+type HealthRequest struct {
+	state         protoimpl.MessageState `protogen:"open.v1"`
+	unknownFields protoimpl.UnknownFields
+	sizeCache     protoimpl.SizeCache
+}
+
+func (x *HealthRequest) Reset() {
+	*x = HealthRequest{}
+	mi := &file_headscale_v1_headscale_proto_msgTypes[0]
+	ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
+	ms.StoreMessageInfo(mi)
+}
+
+func (x *HealthRequest) String() string {
+	return protoimpl.X.MessageStringOf(x)
+}
+
+func (*HealthRequest) ProtoMessage() {}
+
+func (x *HealthRequest) ProtoReflect() protoreflect.Message {
+	mi := &file_headscale_v1_headscale_proto_msgTypes[0]
+	if x != nil {
+		ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
+		if ms.LoadMessageInfo() == nil {
+			ms.StoreMessageInfo(mi)
+		}
+		return ms
+	}
+	return mi.MessageOf(x)
+}
+
+// Deprecated: Use HealthRequest.ProtoReflect.Descriptor instead.
+func (*HealthRequest) Descriptor() ([]byte, []int) {
+	return file_headscale_v1_headscale_proto_rawDescGZIP(), []int{0}
+}
+
+type HealthResponse struct {
+	state                protoimpl.MessageState `protogen:"open.v1"`
+	DatabaseConnectivity bool                   `protobuf:"varint,1,opt,name=database_connectivity,json=databaseConnectivity,proto3" json:"database_connectivity,omitempty"`
+	unknownFields        protoimpl.UnknownFields
+	sizeCache            protoimpl.SizeCache
+}
+
+func (x *HealthResponse) Reset() {
+	*x = HealthResponse{}
+	mi := &file_headscale_v1_headscale_proto_msgTypes[1]
+	ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
+	ms.StoreMessageInfo(mi)
+}
+
+func (x *HealthResponse) String() string {
+	return protoimpl.X.MessageStringOf(x)
+}
+
+func (*HealthResponse) ProtoMessage() {}
+
+func (x *HealthResponse) ProtoReflect() protoreflect.Message {
+	mi := &file_headscale_v1_headscale_proto_msgTypes[1]
+	if x != nil {
+		ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
+		if ms.LoadMessageInfo() == nil {
+			ms.StoreMessageInfo(mi)
+		}
+		return ms
+	}
+	return mi.MessageOf(x)
+}
+
+// Deprecated: Use HealthResponse.ProtoReflect.Descriptor instead.
+func (*HealthResponse) Descriptor() ([]byte, []int) {
+	return file_headscale_v1_headscale_proto_rawDescGZIP(), []int{1}
+}
+
+func (x *HealthResponse) GetDatabaseConnectivity() bool {
+	if x != nil {
+		return x.DatabaseConnectivity
+	}
+	return false
+}
+
 var File_headscale_v1_headscale_proto protoreflect.FileDescriptor
 
 const file_headscale_v1_headscale_proto_rawDesc = "" +
 	"\n" +
-	"\x1cheadscale/v1/headscale.proto\x12\fheadscale.v1\x1a\x1cgoogle/api/annotations.proto\x1a\x17headscale/v1/user.proto\x1a\x1dheadscale/v1/preauthkey.proto\x1a\x17headscale/v1/node.proto\x1a\x19headscale/v1/apikey.proto\x1a\x19headscale/v1/policy.proto2\xa3\x16\n" +
+	"\x1cheadscale/v1/headscale.proto\x12\fheadscale.v1\x1a\x1cgoogle/api/annotations.proto\x1a\x17headscale/v1/user.proto\x1a\x1dheadscale/v1/preauthkey.proto\x1a\x17headscale/v1/node.proto\x1a\x19headscale/v1/apikey.proto\x1a\x19headscale/v1/policy.proto\"\x0f\n" +
+	"\rHealthRequest\"E\n" +
+	"\x0eHealthResponse\x123\n" +
+	"\x15database_connectivity\x18\x01 \x01(\bR\x14databaseConnectivity2\x80\x17\n" +
 	"\x10HeadscaleService\x12h\n" +
 	"\n" +
 	"CreateUser\x12\x1f.headscale.v1.CreateUserRequest\x1a .headscale.v1.CreateUserResponse\"\x17\x82\xd3\xe4\x93\x02\x11:\x01*\"\f/api/v1/user\x12\x80\x01\n" +
@@ -56,109 +140,127 @@ const file_headscale_v1_headscale_proto_rawDesc = "" +
 	"\vListApiKeys\x12 .headscale.v1.ListApiKeysRequest\x1a!.headscale.v1.ListApiKeysResponse\"\x16\x82\xd3\xe4\x93\x02\x10\x12\x0e/api/v1/apikey\x12v\n" +
 	"\fDeleteApiKey\x12!.headscale.v1.DeleteApiKeyRequest\x1a\".headscale.v1.DeleteApiKeyResponse\"\x1f\x82\xd3\xe4\x93\x02\x19*\x17/api/v1/apikey/{prefix}\x12d\n" +
 	"\tGetPolicy\x12\x1e.headscale.v1.GetPolicyRequest\x1a\x1f.headscale.v1.GetPolicyResponse\"\x16\x82\xd3\xe4\x93\x02\x10\x12\x0e/api/v1/policy\x12g\n" +
-	"\tSetPolicy\x12\x1e.headscale.v1.SetPolicyRequest\x1a\x1f.headscale.v1.SetPolicyResponse\"\x19\x82\xd3\xe4\x93\x02\x13:\x01*\x1a\x0e/api/v1/policyB)Z'github.com/juanfont/headscale/gen/go/v1b\x06proto3"
+	"\tSetPolicy\x12\x1e.headscale.v1.SetPolicyRequest\x1a\x1f.headscale.v1.SetPolicyResponse\"\x19\x82\xd3\xe4\x93\x02\x13:\x01*\x1a\x0e/api/v1/policy\x12[\n" +
+	"\x06Health\x12\x1b.headscale.v1.HealthRequest\x1a\x1c.headscale.v1.HealthResponse\"\x16\x82\xd3\xe4\x93\x02\x10\x12\x0e/api/v1/healthB)Z'github.com/juanfont/headscale/gen/go/v1b\x06proto3"
 
+var (
+	file_headscale_v1_headscale_proto_rawDescOnce sync.Once
+	file_headscale_v1_headscale_proto_rawDescData []byte
+)
+
+func file_headscale_v1_headscale_proto_rawDescGZIP() []byte {
+	file_headscale_v1_headscale_proto_rawDescOnce.Do(func() {
+		file_headscale_v1_headscale_proto_rawDescData = protoimpl.X.CompressGZIP(unsafe.Slice(unsafe.StringData(file_headscale_v1_headscale_proto_rawDesc), len(file_headscale_v1_headscale_proto_rawDesc)))
+	})
+	return file_headscale_v1_headscale_proto_rawDescData
+}
+
+var file_headscale_v1_headscale_proto_msgTypes = make([]protoimpl.MessageInfo, 2)
 var file_headscale_v1_headscale_proto_goTypes = []any{
-	(*CreateUserRequest)(nil),         // 0: headscale.v1.CreateUserRequest
-	(*RenameUserRequest)(nil),         // 1: headscale.v1.RenameUserRequest
-	(*DeleteUserRequest)(nil),         // 2: headscale.v1.DeleteUserRequest
-	(*ListUsersRequest)(nil),          // 3: headscale.v1.ListUsersRequest
-	(*CreatePreAuthKeyRequest)(nil),   // 4: headscale.v1.CreatePreAuthKeyRequest
-	(*ExpirePreAuthKeyRequest)(nil),   // 5: headscale.v1.ExpirePreAuthKeyRequest
-	(*ListPreAuthKeysRequest)(nil),    // 6: headscale.v1.ListPreAuthKeysRequest
-	(*DebugCreateNodeRequest)(nil),    // 7: headscale.v1.DebugCreateNodeRequest
-	(*GetNodeRequest)(nil),            // 8: headscale.v1.GetNodeRequest
-	(*SetTagsRequest)(nil),            // 9: headscale.v1.SetTagsRequest
-	(*SetApprovedRoutesRequest)(nil),  // 10: headscale.v1.SetApprovedRoutesRequest
-	(*RegisterNodeRequest)(nil),       // 11: headscale.v1.RegisterNodeRequest
-	(*DeleteNodeRequest)(nil),         // 12: headscale.v1.DeleteNodeRequest
-	(*ExpireNodeRequest)(nil),         // 13: headscale.v1.ExpireNodeRequest
-	(*RenameNodeRequest)(nil),         // 14: headscale.v1.RenameNodeRequest
-	(*ListNodesRequest)(nil),          // 15: headscale.v1.ListNodesRequest
-	(*MoveNodeRequest)(nil),           // 16: headscale.v1.MoveNodeRequest
-	(*BackfillNodeIPsRequest)(nil),    // 17: headscale.v1.BackfillNodeIPsRequest
-	(*CreateApiKeyRequest)(nil),       // 18: headscale.v1.CreateApiKeyRequest
-	(*ExpireApiKeyRequest)(nil),       // 19: headscale.v1.ExpireApiKeyRequest
-	(*ListApiKeysRequest)(nil),        // 20: headscale.v1.ListApiKeysRequest
-	(*DeleteApiKeyRequest)(nil),       // 21: headscale.v1.DeleteApiKeyRequest
-	(*GetPolicyRequest)(nil),          // 22: headscale.v1.GetPolicyRequest
-	(*SetPolicyRequest)(nil),          // 23: headscale.v1.SetPolicyRequest
-	(*CreateUserResponse)(nil),        // 24: headscale.v1.CreateUserResponse
-	(*RenameUserResponse)(nil),        // 25: headscale.v1.RenameUserResponse
-	(*DeleteUserResponse)(nil),        // 26: headscale.v1.DeleteUserResponse
-	(*ListUsersResponse)(nil),         // 27: headscale.v1.ListUsersResponse
-	(*CreatePreAuthKeyResponse)(nil),  // 28: headscale.v1.CreatePreAuthKeyResponse
-	(*ExpirePreAuthKeyResponse)(nil),  // 29: headscale.v1.ExpirePreAuthKeyResponse
-	(*ListPreAuthKeysResponse)(nil),   // 30: headscale.v1.ListPreAuthKeysResponse
-	(*DebugCreateNodeResponse)(nil),   // 31: headscale.v1.DebugCreateNodeResponse
-	(*GetNodeResponse)(nil),           // 32: headscale.v1.GetNodeResponse
-	(*SetTagsResponse)(nil),           // 33: headscale.v1.SetTagsResponse
-	(*SetApprovedRoutesResponse)(nil), // 34: headscale.v1.SetApprovedRoutesResponse
-	(*RegisterNodeResponse)(nil),      // 35: headscale.v1.RegisterNodeResponse
-	(*DeleteNodeResponse)(nil),        // 36: headscale.v1.DeleteNodeResponse
-	(*ExpireNodeResponse)(nil),        // 37: headscale.v1.ExpireNodeResponse
-	(*RenameNodeResponse)(nil),        // 38: headscale.v1.RenameNodeResponse
-	(*ListNodesResponse)(nil),         // 39: headscale.v1.ListNodesResponse
-	(*MoveNodeResponse)(nil),          // 40: headscale.v1.MoveNodeResponse
-	(*BackfillNodeIPsResponse)(nil),   // 41: headscale.v1.BackfillNodeIPsResponse
-	(*CreateApiKeyResponse)(nil),      // 42: headscale.v1.CreateApiKeyResponse
-	(*ExpireApiKeyResponse)(nil),      // 43: headscale.v1.ExpireApiKeyResponse
-	(*ListApiKeysResponse)(nil),       // 44: headscale.v1.ListApiKeysResponse
-	(*DeleteApiKeyResponse)(nil),      // 45: headscale.v1.DeleteApiKeyResponse
-	(*GetPolicyResponse)(nil),         // 46: headscale.v1.GetPolicyResponse
-	(*SetPolicyResponse)(nil),         // 47: headscale.v1.SetPolicyResponse
+	(*HealthRequest)(nil),             // 0: headscale.v1.HealthRequest
+	(*HealthResponse)(nil),            // 1: headscale.v1.HealthResponse
+	(*CreateUserRequest)(nil),         // 2: headscale.v1.CreateUserRequest
+	(*RenameUserRequest)(nil),         // 3: headscale.v1.RenameUserRequest
+	(*DeleteUserRequest)(nil),         // 4: headscale.v1.DeleteUserRequest
+	(*ListUsersRequest)(nil),          // 5: headscale.v1.ListUsersRequest
+	(*CreatePreAuthKeyRequest)(nil),   // 6: headscale.v1.CreatePreAuthKeyRequest
+	(*ExpirePreAuthKeyRequest)(nil),   // 7: headscale.v1.ExpirePreAuthKeyRequest
+	(*ListPreAuthKeysRequest)(nil),    // 8: headscale.v1.ListPreAuthKeysRequest
+	(*DebugCreateNodeRequest)(nil),    // 9: headscale.v1.DebugCreateNodeRequest
+	(*GetNodeRequest)(nil),            // 10: headscale.v1.GetNodeRequest
+	(*SetTagsRequest)(nil),            // 11: headscale.v1.SetTagsRequest
+	(*SetApprovedRoutesRequest)(nil),  // 12: headscale.v1.SetApprovedRoutesRequest
+	(*RegisterNodeRequest)(nil),       // 13: headscale.v1.RegisterNodeRequest
+	(*DeleteNodeRequest)(nil),         // 14: headscale.v1.DeleteNodeRequest
+	(*ExpireNodeRequest)(nil),         // 15: headscale.v1.ExpireNodeRequest
+	(*RenameNodeRequest)(nil),         // 16: headscale.v1.RenameNodeRequest
+	(*ListNodesRequest)(nil),          // 17: headscale.v1.ListNodesRequest
+	(*MoveNodeRequest)(nil),           // 18: headscale.v1.MoveNodeRequest
+	(*BackfillNodeIPsRequest)(nil),    // 19: headscale.v1.BackfillNodeIPsRequest
+	(*CreateApiKeyRequest)(nil),       // 20: headscale.v1.CreateApiKeyRequest
+	(*ExpireApiKeyRequest)(nil),       // 21: headscale.v1.ExpireApiKeyRequest
+	(*ListApiKeysRequest)(nil),        // 22: headscale.v1.ListApiKeysRequest
+	(*DeleteApiKeyRequest)(nil),       // 23: headscale.v1.DeleteApiKeyRequest
+	(*GetPolicyRequest)(nil),          // 24: headscale.v1.GetPolicyRequest
+	(*SetPolicyRequest)(nil),          // 25: headscale.v1.SetPolicyRequest
+	(*CreateUserResponse)(nil),        // 26: headscale.v1.CreateUserResponse
+	(*RenameUserResponse)(nil),        // 27: headscale.v1.RenameUserResponse
+	(*DeleteUserResponse)(nil),        // 28: headscale.v1.DeleteUserResponse
+	(*ListUsersResponse)(nil),         // 29: headscale.v1.ListUsersResponse
+	(*CreatePreAuthKeyResponse)(nil),  // 30: headscale.v1.CreatePreAuthKeyResponse
+	(*ExpirePreAuthKeyResponse)(nil),  // 31: headscale.v1.ExpirePreAuthKeyResponse
+	(*ListPreAuthKeysResponse)(nil),   // 32: headscale.v1.ListPreAuthKeysResponse
+	(*DebugCreateNodeResponse)(nil),   // 33: headscale.v1.DebugCreateNodeResponse
+	(*GetNodeResponse)(nil),           // 34: headscale.v1.GetNodeResponse
+	(*SetTagsResponse)(nil),           // 35: headscale.v1.SetTagsResponse
+	(*SetApprovedRoutesResponse)(nil), // 36: headscale.v1.SetApprovedRoutesResponse
+	(*RegisterNodeResponse)(nil),      // 37: headscale.v1.RegisterNodeResponse
+	(*DeleteNodeResponse)(nil),        // 38: headscale.v1.DeleteNodeResponse
+	(*ExpireNodeResponse)(nil),        // 39: headscale.v1.ExpireNodeResponse
+	(*RenameNodeResponse)(nil),        // 40: headscale.v1.RenameNodeResponse
+	(*ListNodesResponse)(nil),         // 41: headscale.v1.ListNodesResponse
+	(*MoveNodeResponse)(nil),          // 42: headscale.v1.MoveNodeResponse
+	(*BackfillNodeIPsResponse)(nil),   // 43: headscale.v1.BackfillNodeIPsResponse
+	(*CreateApiKeyResponse)(nil),      // 44: headscale.v1.CreateApiKeyResponse
+	(*ExpireApiKeyResponse)(nil),      // 45: headscale.v1.ExpireApiKeyResponse
+	(*ListApiKeysResponse)(nil),       // 46: headscale.v1.ListApiKeysResponse
+	(*DeleteApiKeyResponse)(nil),      // 47: headscale.v1.DeleteApiKeyResponse
+	(*GetPolicyResponse)(nil),         // 48: headscale.v1.GetPolicyResponse
+	(*SetPolicyResponse)(nil),         // 49: headscale.v1.SetPolicyResponse
 }
 var file_headscale_v1_headscale_proto_depIdxs = []int32{
-	0,  // 0: headscale.v1.HeadscaleService.CreateUser:input_type -> headscale.v1.CreateUserRequest
-	1,  // 1: headscale.v1.HeadscaleService.RenameUser:input_type -> headscale.v1.RenameUserRequest
-	2,  // 2: headscale.v1.HeadscaleService.DeleteUser:input_type -> headscale.v1.DeleteUserRequest
-	3,  // 3: headscale.v1.HeadscaleService.ListUsers:input_type -> headscale.v1.ListUsersRequest
-	4,  // 4: headscale.v1.HeadscaleService.CreatePreAuthKey:input_type -> headscale.v1.CreatePreAuthKeyRequest
-	5,  // 5: headscale.v1.HeadscaleService.ExpirePreAuthKey:input_type -> headscale.v1.ExpirePreAuthKeyRequest
-	6,  // 6: headscale.v1.HeadscaleService.ListPreAuthKeys:input_type -> headscale.v1.ListPreAuthKeysRequest
-	7,  // 7: headscale.v1.HeadscaleService.DebugCreateNode:input_type -> headscale.v1.DebugCreateNodeRequest
-	8,  // 8: headscale.v1.HeadscaleService.GetNode:input_type -> headscale.v1.GetNodeRequest
-	9,  // 9: headscale.v1.HeadscaleService.SetTags:input_type -> headscale.v1.SetTagsRequest
-	10, // 10: headscale.v1.HeadscaleService.SetApprovedRoutes:input_type -> headscale.v1.SetApprovedRoutesRequest
-	11, // 11: headscale.v1.HeadscaleService.RegisterNode:input_type -> headscale.v1.RegisterNodeRequest
-	12, // 12: headscale.v1.HeadscaleService.DeleteNode:input_type -> headscale.v1.DeleteNodeRequest
-	13, // 13: headscale.v1.HeadscaleService.ExpireNode:input_type -> headscale.v1.ExpireNodeRequest
-	14, // 14: headscale.v1.HeadscaleService.RenameNode:input_type -> headscale.v1.RenameNodeRequest
-	15, // 15: headscale.v1.HeadscaleService.ListNodes:input_type -> headscale.v1.ListNodesRequest
-	16, // 16: headscale.v1.HeadscaleService.MoveNode:input_type -> headscale.v1.MoveNodeRequest
-	17, // 17: headscale.v1.HeadscaleService.BackfillNodeIPs:input_type -> headscale.v1.BackfillNodeIPsRequest
-	18, // 18: headscale.v1.HeadscaleService.CreateApiKey:input_type -> headscale.v1.CreateApiKeyRequest
-	19, // 19: headscale.v1.HeadscaleService.ExpireApiKey:input_type -> headscale.v1.ExpireApiKeyRequest
-	20, // 20: headscale.v1.HeadscaleService.ListApiKeys:input_type -> headscale.v1.ListApiKeysRequest
-	21, // 21: headscale.v1.HeadscaleService.DeleteApiKey:input_type -> headscale.v1.DeleteApiKeyRequest
-	22, // 22: headscale.v1.HeadscaleService.GetPolicy:input_type -> headscale.v1.GetPolicyRequest
-	23, // 23: headscale.v1.HeadscaleService.SetPolicy:input_type -> headscale.v1.SetPolicyRequest
-	24, // 24: headscale.v1.HeadscaleService.CreateUser:output_type -> headscale.v1.CreateUserResponse
-	25, // 25: headscale.v1.HeadscaleService.RenameUser:output_type -> headscale.v1.RenameUserResponse
-	26, // 26: headscale.v1.HeadscaleService.DeleteUser:output_type -> headscale.v1.DeleteUserResponse
-	27, // 27: headscale.v1.HeadscaleService.ListUsers:output_type -> headscale.v1.ListUsersResponse
-	28, // 28: headscale.v1.HeadscaleService.CreatePreAuthKey:output_type -> headscale.v1.CreatePreAuthKeyResponse
-	29, // 29: headscale.v1.HeadscaleService.ExpirePreAuthKey:output_type -> headscale.v1.ExpirePreAuthKeyResponse
-	30, // 30: headscale.v1.HeadscaleService.ListPreAuthKeys:output_type -> headscale.v1.ListPreAuthKeysResponse
-	31, // 31: headscale.v1.HeadscaleService.DebugCreateNode:output_type -> headscale.v1.DebugCreateNodeResponse
-	32, // 32: headscale.v1.HeadscaleService.GetNode:output_type -> headscale.v1.GetNodeResponse
-	33, // 33: headscale.v1.HeadscaleService.SetTags:output_type -> headscale.v1.SetTagsResponse
-	34, // 34: headscale.v1.HeadscaleService.SetApprovedRoutes:output_type -> headscale.v1.SetApprovedRoutesResponse
-	35, // 35: headscale.v1.HeadscaleService.RegisterNode:output_type -> headscale.v1.RegisterNodeResponse
-	36, // 36: headscale.v1.HeadscaleService.DeleteNode:output_type -> headscale.v1.DeleteNodeResponse
-	37, // 37: headscale.v1.HeadscaleService.ExpireNode:output_type -> headscale.v1.ExpireNodeResponse
-	38, // 38: headscale.v1.HeadscaleService.RenameNode:output_type -> headscale.v1.RenameNodeResponse
-	39, // 39: headscale.v1.HeadscaleService.ListNodes:output_type -> headscale.v1.ListNodesResponse
-	40, // 40: headscale.v1.HeadscaleService.MoveNode:output_type -> headscale.v1.MoveNodeResponse
-	41, // 41: headscale.v1.HeadscaleService.BackfillNodeIPs:output_type -> headscale.v1.BackfillNodeIPsResponse
-	42, // 42: headscale.v1.HeadscaleService.CreateApiKey:output_type -> headscale.v1.CreateApiKeyResponse
-	43, // 43: headscale.v1.HeadscaleService.ExpireApiKey:output_type -> headscale.v1.ExpireApiKeyResponse
-	44, // 44: headscale.v1.HeadscaleService.ListApiKeys:output_type -> headscale.v1.ListApiKeysResponse
-	45, // 45: headscale.v1.HeadscaleService.DeleteApiKey:output_type -> headscale.v1.DeleteApiKeyResponse
-	46, // 46: headscale.v1.HeadscaleService.GetPolicy:output_type -> headscale.v1.GetPolicyResponse
-	47, // 47: headscale.v1.HeadscaleService.SetPolicy:output_type -> headscale.v1.SetPolicyResponse
-	24, // [24:48] is the sub-list for method output_type
-	0,  // [0:24] is the sub-list for method input_type
+	2,  // 0: headscale.v1.HeadscaleService.CreateUser:input_type -> headscale.v1.CreateUserRequest
+	3,  // 1: headscale.v1.HeadscaleService.RenameUser:input_type -> headscale.v1.RenameUserRequest
+	4,  // 2: headscale.v1.HeadscaleService.DeleteUser:input_type -> headscale.v1.DeleteUserRequest
+	5,  // 3: headscale.v1.HeadscaleService.ListUsers:input_type -> headscale.v1.ListUsersRequest
+	6,  // 4: headscale.v1.HeadscaleService.CreatePreAuthKey:input_type -> headscale.v1.CreatePreAuthKeyRequest
+	7,  // 5: headscale.v1.HeadscaleService.ExpirePreAuthKey:input_type -> headscale.v1.ExpirePreAuthKeyRequest
+	8,  // 6: headscale.v1.HeadscaleService.ListPreAuthKeys:input_type -> headscale.v1.ListPreAuthKeysRequest
+	9,  // 7: headscale.v1.HeadscaleService.DebugCreateNode:input_type -> headscale.v1.DebugCreateNodeRequest
+	10, // 8: headscale.v1.HeadscaleService.GetNode:input_type -> headscale.v1.GetNodeRequest
+	11, // 9: headscale.v1.HeadscaleService.SetTags:input_type -> headscale.v1.SetTagsRequest
+	12, // 10: headscale.v1.HeadscaleService.SetApprovedRoutes:input_type -> headscale.v1.SetApprovedRoutesRequest
+	13, // 11: headscale.v1.HeadscaleService.RegisterNode:input_type -> headscale.v1.RegisterNodeRequest
+	14, // 12: headscale.v1.HeadscaleService.DeleteNode:input_type -> headscale.v1.DeleteNodeRequest
+	15, // 13: headscale.v1.HeadscaleService.ExpireNode:input_type -> headscale.v1.ExpireNodeRequest
+	16, // 14: headscale.v1.HeadscaleService.RenameNode:input_type -> headscale.v1.RenameNodeRequest
+	17, // 15: headscale.v1.HeadscaleService.ListNodes:input_type -> headscale.v1.ListNodesRequest
+	18, // 16: headscale.v1.HeadscaleService.MoveNode:input_type -> headscale.v1.MoveNodeRequest
+	19, // 17: headscale.v1.HeadscaleService.BackfillNodeIPs:input_type -> headscale.v1.BackfillNodeIPsRequest
+	20, // 18: headscale.v1.HeadscaleService.CreateApiKey:input_type -> headscale.v1.CreateApiKeyRequest
+	21, // 19: headscale.v1.HeadscaleService.ExpireApiKey:input_type -> headscale.v1.ExpireApiKeyRequest
+	22, // 20: headscale.v1.HeadscaleService.ListApiKeys:input_type -> headscale.v1.ListApiKeysRequest
+	23, // 21: headscale.v1.HeadscaleService.DeleteApiKey:input_type -> headscale.v1.DeleteApiKeyRequest
+	24, // 22: headscale.v1.HeadscaleService.GetPolicy:input_type -> headscale.v1.GetPolicyRequest
+	25, // 23: headscale.v1.HeadscaleService.SetPolicy:input_type -> headscale.v1.SetPolicyRequest
+	0,  // 24: headscale.v1.HeadscaleService.Health:input_type -> headscale.v1.HealthRequest
+	26, // 25: headscale.v1.HeadscaleService.CreateUser:output_type -> headscale.v1.CreateUserResponse
+	27, // 26: headscale.v1.HeadscaleService.RenameUser:output_type -> headscale.v1.RenameUserResponse
+	28, // 27: headscale.v1.HeadscaleService.DeleteUser:output_type -> headscale.v1.DeleteUserResponse
+	29, // 28: headscale.v1.HeadscaleService.ListUsers:output_type -> headscale.v1.ListUsersResponse
+	30, // 29: headscale.v1.HeadscaleService.CreatePreAuthKey:output_type -> headscale.v1.CreatePreAuthKeyResponse
+	31, // 30: headscale.v1.HeadscaleService.ExpirePreAuthKey:output_type -> headscale.v1.ExpirePreAuthKeyResponse
+	32, // 31: headscale.v1.HeadscaleService.ListPreAuthKeys:output_type -> headscale.v1.ListPreAuthKeysResponse
+	33, // 32: headscale.v1.HeadscaleService.DebugCreateNode:output_type -> headscale.v1.DebugCreateNodeResponse
+	34, // 33: headscale.v1.HeadscaleService.GetNode:output_type -> headscale.v1.GetNodeResponse
+	35, // 34: headscale.v1.HeadscaleService.SetTags:output_type -> headscale.v1.SetTagsResponse
+	36, // 35: headscale.v1.HeadscaleService.SetApprovedRoutes:output_type -> headscale.v1.SetApprovedRoutesResponse
+	37, // 36: headscale.v1.HeadscaleService.RegisterNode:output_type -> headscale.v1.RegisterNodeResponse
+	38, // 37: headscale.v1.HeadscaleService.DeleteNode:output_type -> headscale.v1.DeleteNodeResponse
+	39, // 38: headscale.v1.HeadscaleService.ExpireNode:output_type -> headscale.v1.ExpireNodeResponse
+	40, // 39: headscale.v1.HeadscaleService.RenameNode:output_type -> headscale.v1.RenameNodeResponse
+	41, // 40: headscale.v1.HeadscaleService.ListNodes:output_type -> headscale.v1.ListNodesResponse
+	42, // 41: headscale.v1.HeadscaleService.MoveNode:output_type -> headscale.v1.MoveNodeResponse
+	43, // 42: headscale.v1.HeadscaleService.BackfillNodeIPs:output_type -> headscale.v1.BackfillNodeIPsResponse
+	44, // 43: headscale.v1.HeadscaleService.CreateApiKey:output_type -> headscale.v1.CreateApiKeyResponse
+	45, // 44: headscale.v1.HeadscaleService.ExpireApiKey:output_type -> headscale.v1.ExpireApiKeyResponse
+	46, // 45: headscale.v1.HeadscaleService.ListApiKeys:output_type -> headscale.v1.ListApiKeysResponse
+	47, // 46: headscale.v1.HeadscaleService.DeleteApiKey:output_type -> headscale.v1.DeleteApiKeyResponse
+	48, // 47: headscale.v1.HeadscaleService.GetPolicy:output_type -> headscale.v1.GetPolicyResponse
+	49, // 48: headscale.v1.HeadscaleService.SetPolicy:output_type -> headscale.v1.SetPolicyResponse
+	1,  // 49: headscale.v1.HeadscaleService.Health:output_type -> headscale.v1.HealthResponse
+	25, // [25:50] is the sub-list for method output_type
+	0,  // [0:25] is the sub-list for method input_type
 	0,  // [0:0] is the sub-list for extension type_name
 	0,  // [0:0] is the sub-list for extension extendee
 	0,  // [0:0] is the sub-list for field type_name
@@ -180,12 +282,13 @@ func file_headscale_v1_headscale_proto_init() {
 			GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
 			RawDescriptor: unsafe.Slice(unsafe.StringData(file_headscale_v1_headscale_proto_rawDesc), len(file_headscale_v1_headscale_proto_rawDesc)),
 			NumEnums:      0,
-			NumMessages:   0,
+			NumMessages:   2,
 			NumExtensions: 0,
 			NumServices:   1,
 		},
 		GoTypes:           file_headscale_v1_headscale_proto_goTypes,
 		DependencyIndexes: file_headscale_v1_headscale_proto_depIdxs,
+		MessageInfos:      file_headscale_v1_headscale_proto_msgTypes,
 	}.Build()
 	File_headscale_v1_headscale_proto = out.File
 	file_headscale_v1_headscale_proto_goTypes = nil

gen/go/headscale/v1/headscale.pb.gw.go

@@ -809,6 +809,24 @@ func local_request_HeadscaleService_SetPolicy_0(ctx context.Context, marshaler r
 	return msg, metadata, err
 }
 
+func request_HeadscaleService_Health_0(ctx context.Context, marshaler runtime.Marshaler, client HeadscaleServiceClient, req *http.Request, pathParams map[string]string) (proto.Message, runtime.ServerMetadata, error) {
+	var (
+		protoReq HealthRequest
+		metadata runtime.ServerMetadata
+	)
+	msg, err := client.Health(ctx, &protoReq, grpc.Header(&metadata.HeaderMD), grpc.Trailer(&metadata.TrailerMD))
+	return msg, metadata, err
+}
+
+func local_request_HeadscaleService_Health_0(ctx context.Context, marshaler runtime.Marshaler, server HeadscaleServiceServer, req *http.Request, pathParams map[string]string) (proto.Message, runtime.ServerMetadata, error) {
+	var (
+		protoReq HealthRequest
+		metadata runtime.ServerMetadata
+	)
+	msg, err := server.Health(ctx, &protoReq)
+	return msg, metadata, err
+}
+
 // RegisterHeadscaleServiceHandlerServer registers the http handlers for service HeadscaleService to "mux".
 // UnaryRPC     :call HeadscaleServiceServer directly.
 // StreamingRPC :currently unsupported pending https://github.com/grpc/grpc-go/issues/906.
@@ -1295,6 +1313,26 @@ func RegisterHeadscaleServiceHandlerServer(ctx context.Context, mux *runtime.Ser
 		}
 		forward_HeadscaleService_SetPolicy_0(annotatedContext, mux, outboundMarshaler, w, req, resp, mux.GetForwardResponseOptions()...)
 	})
+	mux.Handle(http.MethodGet, pattern_HeadscaleService_Health_0, func(w http.ResponseWriter, req *http.Request, pathParams map[string]string) {
+		ctx, cancel := context.WithCancel(req.Context())
+		defer cancel()
+		var stream runtime.ServerTransportStream
+		ctx = grpc.NewContextWithServerTransportStream(ctx, &stream)
+		inboundMarshaler, outboundMarshaler := runtime.MarshalerForRequest(mux, req)
+		annotatedContext, err := runtime.AnnotateIncomingContext(ctx, mux, req, "/headscale.v1.HeadscaleService/Health", runtime.WithHTTPPathPattern("/api/v1/health"))
+		if err != nil {
+			runtime.HTTPError(ctx, mux, outboundMarshaler, w, req, err)
+			return
+		}
+		resp, md, err := local_request_HeadscaleService_Health_0(annotatedContext, inboundMarshaler, server, req, pathParams)
+		md.HeaderMD, md.TrailerMD = metadata.Join(md.HeaderMD, stream.Header()), metadata.Join(md.TrailerMD, stream.Trailer())
+		annotatedContext = runtime.NewServerMetadataContext(annotatedContext, md)
+		if err != nil {
+			runtime.HTTPError(annotatedContext, mux, outboundMarshaler, w, req, err)
+			return
+		}
+		forward_HeadscaleService_Health_0(annotatedContext, mux, outboundMarshaler, w, req, resp, mux.GetForwardResponseOptions()...)
+	})
 
 	return nil
 }
@@ -1743,6 +1781,23 @@ func RegisterHeadscaleServiceHandlerClient(ctx context.Context, mux *runtime.Ser
 		}
 		forward_HeadscaleService_SetPolicy_0(annotatedContext, mux, outboundMarshaler, w, req, resp, mux.GetForwardResponseOptions()...)
 	})
+	mux.Handle(http.MethodGet, pattern_HeadscaleService_Health_0, func(w http.ResponseWriter, req *http.Request, pathParams map[string]string) {
+		ctx, cancel := context.WithCancel(req.Context())
+		defer cancel()
+		inboundMarshaler, outboundMarshaler := runtime.MarshalerForRequest(mux, req)
+		annotatedContext, err := runtime.AnnotateContext(ctx, mux, req, "/headscale.v1.HeadscaleService/Health", runtime.WithHTTPPathPattern("/api/v1/health"))
+		if err != nil {
+			runtime.HTTPError(ctx, mux, outboundMarshaler, w, req, err)
+			return
+		}
+		resp, md, err := request_HeadscaleService_Health_0(annotatedContext, inboundMarshaler, client, req, pathParams)
+		annotatedContext = runtime.NewServerMetadataContext(annotatedContext, md)
+		if err != nil {
+			runtime.HTTPError(annotatedContext, mux, outboundMarshaler, w, req, err)
+			return
+		}
+		forward_HeadscaleService_Health_0(annotatedContext, mux, outboundMarshaler, w, req, resp, mux.GetForwardResponseOptions()...)
+	})
 	return nil
 }
 
@@ -1771,6 +1826,7 @@ var (
 	pattern_HeadscaleService_DeleteApiKey_0      = runtime.MustPattern(runtime.NewPattern(1, []int{2, 0, 2, 1, 2, 2, 1, 0, 4, 1, 5, 3}, []string{"api", "v1", "apikey", "prefix"}, ""))
 	pattern_HeadscaleService_GetPolicy_0         = runtime.MustPattern(runtime.NewPattern(1, []int{2, 0, 2, 1, 2, 2}, []string{"api", "v1", "policy"}, ""))
 	pattern_HeadscaleService_SetPolicy_0         = runtime.MustPattern(runtime.NewPattern(1, []int{2, 0, 2, 1, 2, 2}, []string{"api", "v1", "policy"}, ""))
+	pattern_HeadscaleService_Health_0            = runtime.MustPattern(runtime.NewPattern(1, []int{2, 0, 2, 1, 2, 2}, []string{"api", "v1", "health"}, ""))
 )
 
 var (
@@ -1798,4 +1854,5 @@ var (
 	forward_HeadscaleService_DeleteApiKey_0      = runtime.ForwardResponseMessage
 	forward_HeadscaleService_GetPolicy_0         = runtime.ForwardResponseMessage
 	forward_HeadscaleService_SetPolicy_0         = runtime.ForwardResponseMessage
+	forward_HeadscaleService_Health_0            = runtime.ForwardResponseMessage
 )

gen/go/headscale/v1/headscale_grpc.pb.go

@@ -43,6 +43,7 @@ const (
 	HeadscaleService_DeleteApiKey_FullMethodName      = "/headscale.v1.HeadscaleService/DeleteApiKey"
 	HeadscaleService_GetPolicy_FullMethodName         = "/headscale.v1.HeadscaleService/GetPolicy"
 	HeadscaleService_SetPolicy_FullMethodName         = "/headscale.v1.HeadscaleService/SetPolicy"
+	HeadscaleService_Health_FullMethodName            = "/headscale.v1.HeadscaleService/Health"
 )
 
 // HeadscaleServiceClient is the client API for HeadscaleService service.
@@ -78,6 +79,8 @@ type HeadscaleServiceClient interface {
 	// --- Policy start ---
 	GetPolicy(ctx context.Context, in *GetPolicyRequest, opts ...grpc.CallOption) (*GetPolicyResponse, error)
 	SetPolicy(ctx context.Context, in *SetPolicyRequest, opts ...grpc.CallOption) (*SetPolicyResponse, error)
+	// --- Health start ---
+	Health(ctx context.Context, in *HealthRequest, opts ...grpc.CallOption) (*HealthResponse, error)
 }
 
 type headscaleServiceClient struct {
@@ -328,6 +331,16 @@ func (c *headscaleServiceClient) SetPolicy(ctx context.Context, in *SetPolicyReq
 	return out, nil
 }
 
+func (c *headscaleServiceClient) Health(ctx context.Context, in *HealthRequest, opts ...grpc.CallOption) (*HealthResponse, error) {
+	cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
+	out := new(HealthResponse)
+	err := c.cc.Invoke(ctx, HeadscaleService_Health_FullMethodName, in, out, cOpts...)
+	if err != nil {
+		return nil, err
+	}
+	return out, nil
+}
+
 // HeadscaleServiceServer is the server API for HeadscaleService service.
 // All implementations must embed UnimplementedHeadscaleServiceServer
 // for forward compatibility.
@@ -361,6 +374,8 @@ type HeadscaleServiceServer interface {
 	// --- Policy start ---
 	GetPolicy(context.Context, *GetPolicyRequest) (*GetPolicyResponse, error)
 	SetPolicy(context.Context, *SetPolicyRequest) (*SetPolicyResponse, error)
+	// --- Health start ---
+	Health(context.Context, *HealthRequest) (*HealthResponse, error)
 	mustEmbedUnimplementedHeadscaleServiceServer()
 }
 
@@ -443,6 +458,9 @@ func (UnimplementedHeadscaleServiceServer) GetPolicy(context.Context, *GetPolicy
 func (UnimplementedHeadscaleServiceServer) SetPolicy(context.Context, *SetPolicyRequest) (*SetPolicyResponse, error) {
 	return nil, status.Errorf(codes.Unimplemented, "method SetPolicy not implemented")
 }
+func (UnimplementedHeadscaleServiceServer) Health(context.Context, *HealthRequest) (*HealthResponse, error) {
+	return nil, status.Errorf(codes.Unimplemented, "method Health not implemented")
+}
 func (UnimplementedHeadscaleServiceServer) mustEmbedUnimplementedHeadscaleServiceServer() {}
 func (UnimplementedHeadscaleServiceServer) testEmbeddedByValue()                          {}
 
@@ -896,6 +914,24 @@ func _HeadscaleService_SetPolicy_Handler(srv interface{}, ctx context.Context, d
 	return interceptor(ctx, in, info, handler)
 }
 
+func _HeadscaleService_Health_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
+	in := new(HealthRequest)
+	if err := dec(in); err != nil {
+		return nil, err
+	}
+	if interceptor == nil {
+		return srv.(HeadscaleServiceServer).Health(ctx, in)
+	}
+	info := &grpc.UnaryServerInfo{
+		Server:     srv,
+		FullMethod: HeadscaleService_Health_FullMethodName,
+	}
+	handler := func(ctx context.Context, req interface{}) (interface{}, error) {
+		return srv.(HeadscaleServiceServer).Health(ctx, req.(*HealthRequest))
+	}
+	return interceptor(ctx, in, info, handler)
+}
+
 // HeadscaleService_ServiceDesc is the grpc.ServiceDesc for HeadscaleService service.
 // It's only intended for direct use with grpc.RegisterService,
 // and not to be introspected or modified (even as a copy)
@@ -999,6 +1035,10 @@ var HeadscaleService_ServiceDesc = grpc.ServiceDesc{
 			MethodName: "SetPolicy",
 			Handler:    _HeadscaleService_SetPolicy_Handler,
 		},
+		{
+			MethodName: "Health",
+			Handler:    _HeadscaleService_Health_Handler,
+		},
 	},
 	Streams:  []grpc.StreamDesc{},
 	Metadata: "headscale/v1/headscale.proto",

gen/go/headscale/v1/node.pb.go

@@ -1,6 +1,6 @@
 // Code generated by protoc-gen-go. DO NOT EDIT.
 // versions:
-// 	protoc-gen-go v1.36.6
+// 	protoc-gen-go v1.36.8
 // 	protoc        (unknown)
 // source: headscale/v1/node.proto
 

gen/go/headscale/v1/policy.pb.go

@@ -1,6 +1,6 @@
 // Code generated by protoc-gen-go. DO NOT EDIT.
 // versions:
-// 	protoc-gen-go v1.36.6
+// 	protoc-gen-go v1.36.8
 // 	protoc        (unknown)
 // source: headscale/v1/policy.proto
 

gen/go/headscale/v1/preauthkey.pb.go

@@ -1,6 +1,6 @@
 // Code generated by protoc-gen-go. DO NOT EDIT.
 // versions:
-// 	protoc-gen-go v1.36.6
+// 	protoc-gen-go v1.36.8
 // 	protoc        (unknown)
 // source: headscale/v1/preauthkey.proto
 

gen/go/headscale/v1/user.pb.go

@@ -1,6 +1,6 @@
 // Code generated by protoc-gen-go. DO NOT EDIT.
 // versions:
-// 	protoc-gen-go v1.36.6
+// 	protoc-gen-go v1.36.8
 // 	protoc        (unknown)
 // source: headscale/v1/user.proto
 

gen/openapiv2/headscale/v1/headscale.swagger.json

@@ -164,6 +164,29 @@
         ]
       }
     },
+    "/api/v1/health": {
+      "get": {
+        "summary": "--- Health start ---",
+        "operationId": "HeadscaleService_Health",
+        "responses": {
+          "200": {
+            "description": "A successful response.",
+            "schema": {
+              "$ref": "#/definitions/v1HealthResponse"
+            }
+          },
+          "default": {
+            "description": "An unexpected error response.",
+            "schema": {
+              "$ref": "#/definitions/rpcStatus"
+            }
+          }
+        },
+        "tags": [
+          "HeadscaleService"
+        ]
+      }
+    },
     "/api/v1/node": {
       "get": {
         "operationId": "HeadscaleService_ListNodes",
@@ -1056,6 +1079,14 @@
         }
       }
     },
+    "v1HealthResponse": {
+      "type": "object",
+      "properties": {
+        "databaseConnectivity": {
+          "type": "boolean"
+        }
+      }
+    },
     "v1ListApiKeysResponse": {
       "type": "object",
       "properties": {

go.mod

@@ -1,59 +1,59 @@
 module github.com/juanfont/headscale
 
-go 1.24.0
-
-toolchain go1.24.2
+go 1.25
 
 require (
-	github.com/AlecAivazis/survey/v2 v2.3.7
-	github.com/arl/statsviz v0.6.0
-	github.com/cenkalti/backoff/v4 v4.3.0
-	github.com/chasefleming/elem-go v0.30.0
-	github.com/coder/websocket v1.8.13
-	github.com/coreos/go-oidc/v3 v3.14.1
+	github.com/arl/statsviz v0.7.2
+	github.com/cenkalti/backoff/v5 v5.0.3
+	github.com/chasefleming/elem-go v0.31.0
+	github.com/coder/websocket v1.8.14
+	github.com/coreos/go-oidc/v3 v3.16.0
+	github.com/creachadair/command v0.2.0
+	github.com/creachadair/flax v0.0.5
 	github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc
+	github.com/docker/docker v28.5.1+incompatible
 	github.com/fsnotify/fsnotify v1.9.0
 	github.com/glebarez/sqlite v1.11.0
-	github.com/go-gormigrate/gormigrate/v2 v2.1.4
+	github.com/go-gormigrate/gormigrate/v2 v2.1.5
+	github.com/go-json-experiment/json v0.0.0-20250813024750-ebf49471dced
 	github.com/gofrs/uuid/v5 v5.3.2
 	github.com/google/go-cmp v0.7.0
 	github.com/gorilla/mux v1.8.1
-	github.com/grpc-ecosystem/go-grpc-middleware v1.4.0
-	github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3
+	github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3
 	github.com/jagottsicher/termcolor v1.0.2
-	github.com/klauspost/compress v1.18.0
 	github.com/oauth2-proxy/mockoidc v0.0.0-20240214162133-caebfff84d25
 	github.com/ory/dockertest/v3 v3.12.0
 	github.com/philip-bui/grpc-zerolog v1.0.1
 	github.com/pkg/profile v1.7.0
-	github.com/prometheus/client_golang v1.22.0
-	github.com/prometheus/common v0.63.0
-	github.com/pterm/pterm v0.12.80
-	github.com/puzpuzpuz/xsync/v3 v3.5.1
+	github.com/prometheus/client_golang v1.23.2
+	github.com/prometheus/common v0.66.1
+	github.com/pterm/pterm v0.12.82
+	github.com/puzpuzpuz/xsync/v4 v4.2.0
 	github.com/rs/zerolog v1.34.0
-	github.com/samber/lo v1.50.0
-	github.com/sasha-s/go-deadlock v0.3.5
-	github.com/spf13/cobra v1.9.1
-	github.com/spf13/viper v1.20.1
-	github.com/stretchr/testify v1.10.0
+	github.com/samber/lo v1.52.0
+	github.com/sasha-s/go-deadlock v0.3.6
+	github.com/spf13/cobra v1.10.1
+	github.com/spf13/viper v1.21.0
+	github.com/stretchr/testify v1.11.1
 	github.com/tailscale/hujson v0.0.0-20250226034555-ec1d1c113d33
+	github.com/tailscale/squibble v0.0.0-20250108170732-a4ca58afa694
 	github.com/tailscale/tailsql v0.0.0-20250421235516-02f85f087b97
 	github.com/tcnksm/go-latest v0.0.0-20170313132115-e3007ae9052e
 	go4.org/netipx v0.0.0-20231129151722-fdeea329fbba
-	golang.org/x/crypto v0.37.0
-	golang.org/x/exp v0.0.0-20250408133849-7e4ce0ab07d0
-	golang.org/x/net v0.39.0
-	golang.org/x/oauth2 v0.29.0
-	golang.org/x/sync v0.13.0
-	google.golang.org/genproto/googleapis/api v0.0.0-20250428153025-10db94c68c34
-	google.golang.org/grpc v1.72.0
-	google.golang.org/protobuf v1.36.6
+	golang.org/x/crypto v0.43.0
+	golang.org/x/exp v0.0.0-20251009144603-d2f985daa21b
+	golang.org/x/net v0.46.0
+	golang.org/x/oauth2 v0.32.0
+	golang.org/x/sync v0.17.0
+	google.golang.org/genproto/googleapis/api v0.0.0-20250929231259-57b25ae835d4
+	google.golang.org/grpc v1.75.1
+	google.golang.org/protobuf v1.36.10
 	gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c
 	gopkg.in/yaml.v3 v3.0.1
-	gorm.io/driver/postgres v1.5.11
-	gorm.io/gorm v1.25.12
-	tailscale.com v1.83.0-pre.0.20250331211809-96fe8a6db6c9
-	zgo.at/zcache/v2 v2.1.0
+	gorm.io/driver/postgres v1.6.0
+	gorm.io/gorm v1.31.0
+	tailscale.com v1.86.5
+	zgo.at/zcache/v2 v2.4.1
 	zombiezen.com/go/postgrestest v1.0.1
 )
 
@@ -75,17 +75,17 @@ require (
 // together, e.g:
 // go get modernc.org/libc@v1.55.3 modernc.org/sqlite@v1.33.1
 require (
-	modernc.org/libc v1.62.1 // indirect
+	modernc.org/libc v1.66.10 // indirect
 	modernc.org/mathutil v1.7.1 // indirect
-	modernc.org/memory v1.10.0 // indirect
-	modernc.org/sqlite v1.37.0 // indirect
+	modernc.org/memory v1.11.0 // indirect
+	modernc.org/sqlite v1.39.1
 )
 
 require (
 	atomicgo.dev/cursor v0.2.0 // indirect
 	atomicgo.dev/keyboard v0.2.9 // indirect
 	atomicgo.dev/schedule v0.1.0 // indirect
-	dario.cat/mergo v1.0.1 // indirect
+	dario.cat/mergo v1.0.2 // indirect
 	filippo.io/edwards25519 v1.1.0 // indirect
 	github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
 	github.com/Microsoft/go-winio v0.6.2 // indirect
@@ -107,29 +107,34 @@ require (
 	github.com/aws/aws-sdk-go-v2/service/sts v1.33.13 // indirect
 	github.com/aws/smithy-go v1.22.2 // indirect
 	github.com/beorn7/perks v1.0.1 // indirect
+	github.com/cenkalti/backoff/v4 v4.3.0 // indirect
 	github.com/cespare/xxhash/v2 v2.3.0 // indirect
-	github.com/containerd/console v1.0.4 // indirect
+	github.com/clipperhouse/uax29/v2 v2.2.0 // indirect
+	github.com/containerd/console v1.0.5 // indirect
 	github.com/containerd/continuity v0.4.5 // indirect
+	github.com/containerd/errdefs v0.3.0 // indirect
+	github.com/containerd/errdefs/pkg v0.3.0 // indirect
 	github.com/coreos/go-iptables v0.7.1-0.20240112124308-65c67c9f46e6 // indirect
-	github.com/creachadair/mds v0.24.1 // indirect
+	github.com/creachadair/mds v0.25.2 // indirect
 	github.com/dblohm7/wingoes v0.0.0-20240123200102-b75a8a7d7eb0 // indirect
 	github.com/digitalocean/go-smbios v0.0.0-20180907143718-390a4f403a8e // indirect
-	github.com/docker/cli v28.1.1+incompatible // indirect
-	github.com/docker/docker v28.1.1+incompatible // indirect
-	github.com/docker/go-connections v0.5.0 // indirect
+	github.com/distribution/reference v0.6.0 // indirect
+	github.com/docker/cli v28.5.1+incompatible // indirect
+	github.com/docker/go-connections v0.6.0 // indirect
 	github.com/docker/go-units v0.5.0 // indirect
 	github.com/dustin/go-humanize v1.0.1 // indirect
 	github.com/felixge/fgprof v0.9.5 // indirect
+	github.com/felixge/httpsnoop v1.0.4 // indirect
 	github.com/fxamacker/cbor/v2 v2.7.0 // indirect
 	github.com/gaissmai/bart v0.18.0 // indirect
 	github.com/glebarez/go-sqlite v1.22.0 // indirect
 	github.com/go-jose/go-jose/v3 v3.0.4 // indirect
-	github.com/go-jose/go-jose/v4 v4.1.0 // indirect
-	github.com/go-json-experiment/json v0.0.0-20250223041408-d3c622f1b874 // indirect
+	github.com/go-jose/go-jose/v4 v4.1.3 // indirect
+	github.com/go-logr/logr v1.4.3 // indirect
+	github.com/go-logr/stdr v1.2.2 // indirect
 	github.com/go-ole/go-ole v1.3.0 // indirect
-	github.com/go-viper/mapstructure/v2 v2.2.1 // indirect
+	github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
 	github.com/godbus/dbus/v5 v5.1.1-0.20230522191255-76236955d466 // indirect
-	github.com/gogo/protobuf v1.3.2 // indirect
 	github.com/golang-jwt/jwt/v5 v5.2.2 // indirect
 	github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
 	github.com/golang/protobuf v1.5.4 // indirect
@@ -137,68 +142,62 @@ require (
 	github.com/google/go-github v17.0.0+incompatible // indirect
 	github.com/google/go-querystring v1.1.0 // indirect
 	github.com/google/nftables v0.2.1-0.20240414091927-5e242ec57806 // indirect
-	github.com/google/pprof v0.0.0-20250501235452-c0086092b71a // indirect
+	github.com/google/pprof v0.0.0-20251007162407-5df77e3f7d1d // indirect
 	github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
 	github.com/google/uuid v1.6.0 // indirect
-	github.com/gookit/color v1.5.4 // indirect
-	github.com/gorilla/csrf v1.7.3 // indirect
-	github.com/gorilla/securecookie v1.1.2 // indirect
+	github.com/gookit/color v1.6.0 // indirect
 	github.com/gorilla/websocket v1.5.3 // indirect
 	github.com/hashicorp/go-version v1.7.0 // indirect
 	github.com/hdevalence/ed25519consensus v0.2.0 // indirect
 	github.com/illarion/gonotify/v3 v3.0.2 // indirect
 	github.com/inconshreveable/mousetrap v1.1.0 // indirect
-	github.com/insomniacslk/dhcp v0.0.0-20240129002554-15c9b8791914 // indirect
 	github.com/jackc/pgpassfile v1.0.0 // indirect
 	github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
-	github.com/jackc/pgx/v5 v5.7.4 // indirect
+	github.com/jackc/pgx/v5 v5.7.6 // indirect
 	github.com/jackc/puddle/v2 v2.2.2 // indirect
 	github.com/jinzhu/inflection v1.0.0 // indirect
 	github.com/jinzhu/now v1.1.5 // indirect
 	github.com/jmespath/go-jmespath v0.4.0 // indirect
 	github.com/jsimonetti/rtnetlink v1.4.1 // indirect
-	github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
-	github.com/kortschak/wol v0.0.0-20200729010619-da482cc4850a // indirect
+	github.com/klauspost/compress v1.18.0 // indirect
 	github.com/kr/pretty v0.3.1 // indirect
 	github.com/kr/text v0.2.0 // indirect
 	github.com/lib/pq v1.10.9 // indirect
 	github.com/lithammer/fuzzysearch v1.1.8 // indirect
 	github.com/mattn/go-colorable v0.1.14 // indirect
 	github.com/mattn/go-isatty v0.0.20 // indirect
-	github.com/mattn/go-runewidth v0.0.16 // indirect
+	github.com/mattn/go-runewidth v0.0.19 // indirect
 	github.com/mdlayher/genetlink v1.3.2 // indirect
 	github.com/mdlayher/netlink v1.7.3-0.20250113171957-fbb4dce95f42 // indirect
 	github.com/mdlayher/sdnotify v1.0.0 // indirect
 	github.com/mdlayher/socket v0.5.0 // indirect
-	github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect
 	github.com/miekg/dns v1.1.58 // indirect
 	github.com/mitchellh/go-ps v1.0.0 // indirect
 	github.com/moby/docker-image-spec v1.3.1 // indirect
+	github.com/moby/sys/atomicwriter v0.1.0 // indirect
 	github.com/moby/sys/user v0.4.0 // indirect
 	github.com/moby/term v0.5.2 // indirect
+	github.com/morikuni/aec v1.0.0 // indirect
 	github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
-	github.com/ncruces/go-strftime v0.1.9 // indirect
+	github.com/ncruces/go-strftime v1.0.0 // indirect
 	github.com/opencontainers/go-digest v1.0.0 // indirect
 	github.com/opencontainers/image-spec v1.1.1 // indirect
-	github.com/opencontainers/runc v1.3.0 // indirect
+	github.com/opencontainers/runc v1.3.2 // indirect
 	github.com/pelletier/go-toml/v2 v2.2.4 // indirect
-	github.com/petermattis/goid v0.0.0-20250319124200-ccd6737f222a // indirect
-	github.com/pierrec/lz4/v4 v4.1.21 // indirect
+	github.com/petermattis/goid v0.0.0-20250904145737-900bdf8bb490 // indirect
 	github.com/pkg/errors v0.9.1 // indirect
 	github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
 	github.com/prometheus-community/pro-bing v0.4.0 // indirect
-	github.com/prometheus/client_model v0.6.1 // indirect
-	github.com/prometheus/procfs v0.15.1 // indirect
+	github.com/prometheus/client_model v0.6.2 // indirect
+	github.com/prometheus/procfs v0.16.1 // indirect
 	github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
-	github.com/rivo/uniseg v0.4.7 // indirect
 	github.com/rogpeppe/go-internal v1.14.1 // indirect
 	github.com/safchain/ethtool v0.3.0 // indirect
-	github.com/sagikazarmark/locafero v0.9.0 // indirect
+	github.com/sagikazarmark/locafero v0.12.0 // indirect
 	github.com/sirupsen/logrus v1.9.3 // indirect
-	github.com/sourcegraph/conc v0.3.0 // indirect
-	github.com/spf13/afero v1.14.0 // indirect
-	github.com/spf13/cast v1.8.0 // indirect
-	github.com/spf13/pflag v1.0.6 // indirect
+	github.com/spf13/afero v1.15.0 // indirect
+	github.com/spf13/cast v1.10.0 // indirect
+	github.com/spf13/pflag v1.0.10 // indirect
 	github.com/subosito/gotenv v1.6.0 // indirect
 	github.com/tailscale/certstore v0.1.1-0.20231202035212-d3fa0460f47e // indirect
 	github.com/tailscale/go-winio v0.0.0-20231025203758-c4f33415bf55 // indirect
@@ -206,26 +205,36 @@ require (
 	github.com/tailscale/netlink v1.1.1-0.20240822203006-4d49adab4de7 // indirect
 	github.com/tailscale/peercred v0.0.0-20250107143737-35a0c7bd7edc // indirect
 	github.com/tailscale/setec v0.0.0-20250305161714-445cadbbca3d // indirect
-	github.com/tailscale/squibble v0.0.0-20250108170732-a4ca58afa694 // indirect
 	github.com/tailscale/web-client-prebuilt v0.0.0-20250124233751-d4cd19a26976 // indirect
-	github.com/tailscale/wireguard-go v0.0.0-20250107165329-0b8b35511f19 // indirect
-	github.com/u-root/uio v0.0.0-20240224005618-d2acac8f3701 // indirect
-	github.com/vishvananda/netns v0.0.4 // indirect
+	github.com/tailscale/wireguard-go v0.0.0-20250716170648-1d0488a3d7da // indirect
+	github.com/vishvananda/netns v0.0.5 // indirect
 	github.com/x448/float16 v0.8.4 // indirect
 	github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
 	github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
 	github.com/xeipuuv/gojsonschema v1.2.0 // indirect
 	github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
-	go.uber.org/multierr v1.11.0 // indirect
+	go.opentelemetry.io/auto/sdk v1.1.0 // indirect
+	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.58.0 // indirect
+	go.opentelemetry.io/otel v1.37.0 // indirect
+	go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.36.0 // indirect
+	go.opentelemetry.io/otel/metric v1.37.0 // indirect
+	go.opentelemetry.io/otel/trace v1.37.0 // indirect
+	go.yaml.in/yaml/v2 v2.4.2 // indirect
+	go.yaml.in/yaml/v3 v3.0.4 // indirect
 	go4.org/mem v0.0.0-20240501181205-ae6ca9944745 // indirect
-	golang.org/x/mod v0.24.0 // indirect
-	golang.org/x/sys v0.32.0 // indirect
-	golang.org/x/term v0.31.0 // indirect
-	golang.org/x/text v0.24.0 // indirect
-	golang.org/x/time v0.10.0 // indirect
-	golang.org/x/tools v0.32.0 // indirect
+	golang.org/x/mod v0.29.0 // indirect
+	golang.org/x/sys v0.37.0 // indirect
+	golang.org/x/term v0.36.0 // indirect
+	golang.org/x/text v0.30.0 // indirect
+	golang.org/x/time v0.11.0 // indirect
+	golang.org/x/tools v0.38.0 // indirect
 	golang.zx2c4.com/wintun v0.0.0-20230126152724-0fa3db229ce2 // indirect
 	golang.zx2c4.com/wireguard/windows v0.5.3 // indirect
-	google.golang.org/genproto/googleapis/rpc v0.0.0-20250428153025-10db94c68c34 // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20250929231259-57b25ae835d4 // indirect
 	gvisor.dev/gvisor v0.0.0-20250205023644-9414b50a5633 // indirect
 )
+
+tool (
+	golang.org/x/tools/cmd/stringer
+	tailscale.com/cmd/viewer
+)

go.sum

@@ -1,3 +1,5 @@
+9fans.net/go v0.0.8-0.20250307142834-96bdba94b63f h1:1C7nZuxUMNz7eiQALRfiqNOm04+m3edWlRff/BYHf0Q=
+9fans.net/go v0.0.8-0.20250307142834-96bdba94b63f/go.mod h1:hHyrZRryGqVdqrknjq5OWDLGCTJ2NeEvtrpR96mjraM=
 atomicgo.dev/assert v0.0.2 h1:FiKeMiZSgRrZsPo9qn/7vmr7mCsh5SZyXY4YGYiYwrg=
 atomicgo.dev/assert v0.0.2/go.mod h1:ut4NcI3QDdJtlmAxQULOmA13Gz6e2DWbSAS8RUOmNYQ=
 atomicgo.dev/cursor v0.2.0 h1:H6XN5alUJ52FZZUkI7AlJbUc1aW38GWZalpYRPpoPOw=
@@ -6,18 +8,14 @@ atomicgo.dev/keyboard v0.2.9 h1:tOsIid3nlPLZ3lwgG8KZMp/SFmr7P0ssEN5JUsm78K8=
 atomicgo.dev/keyboard v0.2.9/go.mod h1:BC4w9g00XkxH/f1HXhW2sXmJFOCWbKn9xrOunSFtExQ=
 atomicgo.dev/schedule v0.1.0 h1:nTthAbhZS5YZmgYbb2+DH8uQIZcTlIrd4eYr3UQxEjs=
 atomicgo.dev/schedule v0.1.0/go.mod h1:xeUa3oAkiuHYh8bKiQBRojqAMq3PXXbJujjb0hw8pEU=
-cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
-dario.cat/mergo v1.0.1 h1:Ra4+bf83h2ztPIQYNP99R6m+Y7KfnARDfID+a+vLl4s=
-dario.cat/mergo v1.0.1/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
+dario.cat/mergo v1.0.2 h1:85+piFYR1tMbRrLcDwR18y4UKJ3aH1Tbzi24VRW1TK8=
+dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA=
 filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA=
 filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
 filippo.io/mkcert v1.4.4 h1:8eVbbwfVlaqUM7OwuftKc2nuYOoTDQWqsoXmzoXZdbc=
 filippo.io/mkcert v1.4.4/go.mod h1:VyvOchVuAye3BoUsPUOOofKygVwLV2KQMVFJNRq+1dA=
-github.com/AlecAivazis/survey/v2 v2.3.7 h1:6I/u8FvytdGsgonrYsVn2t8t4QiRnh6QSTqkkhIiSjQ=
-github.com/AlecAivazis/survey/v2 v2.3.7/go.mod h1:xUTIdE4KCOIjsBAE1JYsUPoCqYdZ1reCfTwbto0Fduo=
 github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c h1:udKWzYgxTojEKWjV8V+WSxDXJ4NFATAsZjh8iIbsQIg=
 github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E=
-github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
 github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c h1:pxW6RcqyfI9/kWtOwnv/G+AzdKuy2ZrqINhenH4HyNs=
 github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
 github.com/MarvinJWendt/testza v0.1.0/go.mod h1:7AxNvlfeHP7Z/hDQ5JtE3OKYT3XFUeLCDE2DQninSqs=
@@ -31,8 +29,6 @@ github.com/MarvinJWendt/testza v0.5.2 h1:53KDo64C1z/h/d/stCYCPY69bt/OSwjq5KpFNwi
 github.com/MarvinJWendt/testza v0.5.2/go.mod h1:xu53QFE5sCdjtMCKk8YMQ2MnymimEctc4n3EjyIYvEY=
 github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
 github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
-github.com/Netflix/go-expect v0.0.0-20220104043353-73e0943537d2 h1:+vx7roKuyA63nhn5WAunQHLTznkw5W8b1Xc0dNjp83s=
-github.com/Netflix/go-expect v0.0.0-20220104043353-73e0943537d2/go.mod h1:HBCaDeC1lPdgDeDbhX8XFpy1jqjK0IBG8W5K+xYqA0w=
 github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5 h1:TngWCqHvy9oXAN6lEVMRuU21PR1EtLVZJmdB18Gu3Rw=
 github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5/go.mod h1:lmUJ/7eu/Q8D7ML55dXQrVaamCz2vxCfdQBasLZfHKk=
 github.com/akutz/memconn v0.1.0 h1:NawI0TORU4hcOMsMr11g7vwlCdkYeLKXBcxWu2W/P8A=
@@ -41,8 +37,8 @@ github.com/alexbrainman/sspi v0.0.0-20231016080023-1a75b4708caa h1:LHTHcTQiSGT7V
 github.com/alexbrainman/sspi v0.0.0-20231016080023-1a75b4708caa/go.mod h1:cEWa1LVoE5KvSD9ONXsZrj0z6KqySlCCNKHlLzbqAt4=
 github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be h1:9AeTilPcZAjCFIImctFaOjnTIavg87rW78vTPkQqLI8=
 github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be/go.mod h1:ySMOLuWl6zY27l47sB3qLNK6tF2fkHG55UZxx8oIVo4=
-github.com/arl/statsviz v0.6.0 h1:jbW1QJkEYQkufd//4NDYRSNBpwJNrdzPahF7ZmoGdyE=
-github.com/arl/statsviz v0.6.0/go.mod h1:0toboo+YGSUXDaS4g1D5TVS4dXs7S7YYT5J/qnW2h8s=
+github.com/arl/statsviz v0.7.2 h1:xnuIfRiXE4kvxEcfGL+IE3mKH1BXNHuE+eJELIh7oOA=
+github.com/arl/statsviz v0.7.2/go.mod h1:XlrbiT7xYT03xaW9JMMfD8KFUhBOESJwfyNJu83PbB0=
 github.com/atomicgo/cursor v0.0.1/go.mod h1:cBON2QmmrysudxNBFthvMtN32r3jxVRIvzkUiF/RuIk=
 github.com/aws/aws-sdk-go-v2 v1.36.0 h1:b1wM5CcE65Ujwn565qcwgtOTT1aT4ADOHHgglKjG7fk=
 github.com/aws/aws-sdk-go-v2 v1.36.0/go.mod h1:5PMILGVKiW32oDzjj6RU52yrNrDPUHcbZQYr1sM7qmM=
@@ -82,16 +78,16 @@ github.com/aws/aws-sdk-go-v2/service/sts v1.33.13 h1:3LXNnmtH3TURctC23hnC0p/39Q5
 github.com/aws/aws-sdk-go-v2/service/sts v1.33.13/go.mod h1:7Yn+p66q/jt38qMoVfNvjbm3D89mGBnkwDcijgtih8w=
 github.com/aws/smithy-go v1.22.2 h1:6D9hW43xKFrRx/tXXfAlIZc4JI+yQe6snnWcQyxSyLQ=
 github.com/aws/smithy-go v1.22.2/go.mod h1:irrKGvNn1InZwb2d7fkIRNucdfwR8R+Ts3wxYa/cJHg=
-github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=
 github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
 github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
 github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8=
 github.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE=
-github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
+github.com/cenkalti/backoff/v5 v5.0.3 h1:ZN+IMa753KfX5hd8vVaMixjnqRZ3y8CuJKRKj1xcsSM=
+github.com/cenkalti/backoff/v5 v5.0.3/go.mod h1:rkhZdG3JZukswDf7f0cwqPNk4K0sa+F97BxZthm/crw=
 github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
 github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
-github.com/chasefleming/elem-go v0.30.0 h1:BlhV1ekv1RbFiM8XZUQeln1Ikb4D+bu2eDO4agREvok=
-github.com/chasefleming/elem-go v0.30.0/go.mod h1:hz73qILBIKnTgOujnSMtEj20/epI+f6vg71RUilJAA4=
+github.com/chasefleming/elem-go v0.31.0 h1:vZsuKmKdv6idnUbu3awMruxTiFqZ/ertFJFAyBCkVhI=
+github.com/chasefleming/elem-go v0.31.0/go.mod h1:UBmmZfso2LkXA0HZInbcwsmhE/LXFClEcBPNCGeARtA=
 github.com/chromedp/cdproto v0.0.0-20230802225258-3cf4e6d46a89/go.mod h1:GKljq0VrfU4D5yc+2qA6OVr8pmO/MBbPEWqWQ/oqGEs=
 github.com/chromedp/chromedp v0.9.2/go.mod h1:LkSXJKONWTCHAfQasKFUZI+mxqS4tZqhmtGzzhLsnLs=
 github.com/chromedp/sysutil v1.0.0/go.mod h1:kgWmDdq8fTzXYcKIBqIYvRRTnYb9aNS9moAV0xufSww=
@@ -103,25 +99,36 @@ github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMn
 github.com/chzyer/test v1.0.0/go.mod h1:2JlltgoNkt4TW/z9V/IzDdFaMTM2JPIi26O1pF38GC8=
 github.com/cilium/ebpf v0.17.3 h1:FnP4r16PWYSE4ux6zN+//jMcW4nMVRvuTLVTvCjyyjg=
 github.com/cilium/ebpf v0.17.3/go.mod h1:G5EDHij8yiLzaqn0WjyfJHvRa+3aDlReIaLVRMvOyJk=
-github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
-github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
-github.com/coder/websocket v1.8.13 h1:f3QZdXy7uGVz+4uCJy2nTZyM0yTBj8yANEHhqlXZ9FE=
-github.com/coder/websocket v1.8.13/go.mod h1:LNVeNrXQZfe5qhS9ALED3uA+l5pPqvwXg3CKoDBB2gs=
+github.com/clipperhouse/uax29/v2 v2.2.0 h1:ChwIKnQN3kcZteTXMgb1wztSgaU+ZemkgWdohwgs8tY=
+github.com/clipperhouse/uax29/v2 v2.2.0/go.mod h1:EFJ2TJMRUaplDxHKj1qAEhCtQPW2tJSwu5BF98AuoVM=
+github.com/coder/websocket v1.8.14 h1:9L0p0iKiNOibykf283eHkKUHHrpG7f65OE3BhhO7v9g=
+github.com/coder/websocket v1.8.14/go.mod h1:NX3SzP+inril6yawo5CQXx8+fk145lPDC6pumgx0mVg=
 github.com/containerd/console v1.0.3/go.mod h1:7LqA/THxQ86k76b8c/EMSiaJ3h1eZkMkXar0TQ1gf3U=
-github.com/containerd/console v1.0.4 h1:F2g4+oChYvBTsASRTz8NP6iIAi97J3TtSAsLbIFn4ro=
-github.com/containerd/console v1.0.4/go.mod h1:YynlIjWYF8myEu6sdkwKIvGQq+cOckRm6So2avqoYAk=
+github.com/containerd/console v1.0.5 h1:R0ymNeydRqH2DmakFNdmjR2k0t7UPuiOV/N/27/qqsc=
+github.com/containerd/console v1.0.5/go.mod h1:YynlIjWYF8myEu6sdkwKIvGQq+cOckRm6So2avqoYAk=
 github.com/containerd/continuity v0.4.5 h1:ZRoN1sXq9u7V6QoHMcVWGhOwDFqZ4B9i5H6un1Wh0x4=
 github.com/containerd/continuity v0.4.5/go.mod h1:/lNJvtJKUQStBzpVQ1+rasXO1LAWtUQssk28EZvJ3nE=
+github.com/containerd/errdefs v0.3.0 h1:FSZgGOeK4yuT/+DnF07/Olde/q4KBoMsaamhXxIMDp4=
+github.com/containerd/errdefs v0.3.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M=
+github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE=
+github.com/containerd/errdefs/pkg v0.3.0/go.mod h1:NJw6s9HwNuRhnjJhM7pylWwMyAkmCQvQ4GpJHEqRLVk=
+github.com/containerd/log v0.1.0 h1:TCJt7ioM2cr/tfR8GPbGf9/VRAX8D2B4PjzCpfX540I=
+github.com/containerd/log v0.1.0/go.mod h1:VRRf09a7mHDIRezVKTRCrOq78v577GXq3bSa3EhrzVo=
 github.com/coreos/go-iptables v0.7.1-0.20240112124308-65c67c9f46e6 h1:8h5+bWd7R6AYUslN6c6iuZWTKsKxUFDlpnmilO6R2n0=
 github.com/coreos/go-iptables v0.7.1-0.20240112124308-65c67c9f46e6/go.mod h1:Qe8Bv2Xik5FyTXwgIbLAnv2sWSBmvWdFETJConOQ//Q=
-github.com/coreos/go-oidc/v3 v3.14.1 h1:9ePWwfdwC4QKRlCXsJGou56adA/owXczOzwKdOumLqk=
-github.com/coreos/go-oidc/v3 v3.14.1/go.mod h1:HaZ3szPaZ0e4r6ebqvsLWlk2Tn+aejfmrfah6hnSYEU=
+github.com/coreos/go-oidc/v3 v3.16.0 h1:qRQUCFstKpXwmEjDQTIbyY/5jF00+asXzSkmkoa/mow=
+github.com/coreos/go-oidc/v3 v3.16.0/go.mod h1:wqPbKFrVnE90vty060SB40FCJ8fTHTxSwyXJqZH+sI8=
 github.com/coreos/go-systemd/v22 v22.5.0/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
 github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
-github.com/creachadair/mds v0.24.1 h1:bzL4ItCtAUxxO9KkotP0PVzlw4tnJicAcjPu82v2mGs=
-github.com/creachadair/mds v0.24.1/go.mod h1:ArfS0vPHoLV/SzuIzoqTEZfoYmac7n9Cj8XPANHocvw=
+github.com/creachadair/command v0.2.0 h1:qTA9cMMhZePAxFoNdnk6F6nn94s1qPndIg9hJbqI9cA=
+github.com/creachadair/command v0.2.0/go.mod h1:j+Ar+uYnFsHpkMeV9kGj6lJ45y9u2xqtg8FYy6cm+0o=
+github.com/creachadair/flax v0.0.5 h1:zt+CRuXQASxwQ68e9GHAOnEgAU29nF0zYMHOCrL5wzE=
+github.com/creachadair/flax v0.0.5/go.mod h1:F1PML0JZLXSNDMNiRGK2yjm5f+L9QCHchyHBldFymj8=
+github.com/creachadair/mds v0.25.2 h1:xc0S0AfDq5GX9KUR5sLvi5XjA61/P6S5e0xFs1vA18Q=
+github.com/creachadair/mds v0.25.2/go.mod h1:+s4CFteFRj4eq2KcGHW8Wei3u9NyzSPzNV32EvjyK/Q=
+github.com/creachadair/taskgroup v0.13.2 h1:3KyqakBuFsm3KkXi/9XIb0QcA8tEzLHLgaoidf0MdVc=
+github.com/creachadair/taskgroup v0.13.2/go.mod h1:i3V1Zx7H8RjwljUEeUWYT30Lmb9poewSb2XI1yTwD0g=
 github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
-github.com/creack/pty v1.1.17/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr4O4=
 github.com/creack/pty v1.1.23 h1:4M6+isWdcStXEf15G/RbrMPOQj1dZ7HPZCGwE4kOeP0=
 github.com/creack/pty v1.1.23/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE=
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -132,27 +139,25 @@ github.com/dblohm7/wingoes v0.0.0-20240123200102-b75a8a7d7eb0 h1:vrC07UZcgPzu/Oj
 github.com/dblohm7/wingoes v0.0.0-20240123200102-b75a8a7d7eb0/go.mod h1:Nx87SkVqTKd8UtT+xu7sM/l+LgXs6c0aHrlKusR+2EQ=
 github.com/digitalocean/go-smbios v0.0.0-20180907143718-390a4f403a8e h1:vUmf0yezR0y7jJ5pceLHthLaYf4bA5T14B6q39S4q2Q=
 github.com/digitalocean/go-smbios v0.0.0-20180907143718-390a4f403a8e/go.mod h1:YTIHhz/QFSYnu/EhlF2SpU2Uk+32abacUYA5ZPljz1A=
+github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
+github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
 github.com/djherbis/times v1.6.0 h1:w2ctJ92J8fBvWPxugmXIv7Nz7Q3iDMKNx9v5ocVH20c=
 github.com/djherbis/times v1.6.0/go.mod h1:gOHeRAz2h+VJNZ5Gmc/o7iD9k4wW7NMVqieYCY99oc0=
-github.com/docker/cli v28.1.1+incompatible h1:eyUemzeI45DY7eDPuwUcmDyDj1pM98oD5MdSpiItp8k=
-github.com/docker/cli v28.1.1+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
-github.com/docker/docker v28.1.1+incompatible h1:49M11BFLsVO1gxY9UX9p/zwkE/rswggs8AdFmXQw51I=
-github.com/docker/docker v28.1.1+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
-github.com/docker/go-connections v0.5.0 h1:USnMq7hx7gwdVZq1L49hLXaFtUdTADjXGp+uj1Br63c=
-github.com/docker/go-connections v0.5.0/go.mod h1:ov60Kzw0kKElRwhNs9UlUHAE/F9Fe6GLaXnqyDdmEXc=
+github.com/docker/cli v28.5.1+incompatible h1:ESutzBALAD6qyCLqbQSEf1a/U8Ybms5agw59yGVc+yY=
+github.com/docker/cli v28.5.1+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
+github.com/docker/docker v28.5.1+incompatible h1:Bm8DchhSD2J6PsFzxC35TZo4TLGR2PdW/E69rU45NhM=
+github.com/docker/docker v28.5.1+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
+github.com/docker/go-connections v0.6.0 h1:LlMG9azAe1TqfR7sO+NJttz1gy6KO7VJBh+pMmjSD94=
+github.com/docker/go-connections v0.6.0/go.mod h1:AahvXYshr6JgfUJGdDCs2b5EZG/vmaMAntpSFH5BFKE=
 github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
 github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
-github.com/dsnet/try v0.0.3 h1:ptR59SsrcFUYbT/FhAbKTV6iLkeD6O18qfIWRml2fqI=
-github.com/dsnet/try v0.0.3/go.mod h1:WBM8tRpUmnXXhY1U6/S8dt6UWdHTQ7y8A5YSkRCkq40=
 github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
 github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
-github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
-github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
-github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
-github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
 github.com/felixge/fgprof v0.9.3/go.mod h1:RdbpDgzqYVh/T9fPELJyV7EYJuHB55UTEULNun8eiPw=
 github.com/felixge/fgprof v0.9.5 h1:8+vR6yu2vvSKn08urWyEuxx75NWPEvybbkBirEpsbVY=
 github.com/felixge/fgprof v0.9.5/go.mod h1:yKl+ERSa++RYOs32d8K6WEXCB4uXdLls4ZaZPpayhMM=
+github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
+github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
 github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
 github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
 github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
@@ -167,27 +172,27 @@ github.com/glebarez/go-sqlite v1.22.0 h1:uAcMJhaA6r3LHMTFgP0SifzgXg46yJkgxqyuyec
 github.com/glebarez/go-sqlite v1.22.0/go.mod h1:PlBIdHe0+aUEFn+r2/uthrWq4FxbzugL0L8Li6yQJbc=
 github.com/glebarez/sqlite v1.11.0 h1:wSG0irqzP6VurnMEpFGer5Li19RpIRi2qvQz++w0GMw=
 github.com/glebarez/sqlite v1.11.0/go.mod h1:h8/o8j5wiAsqSPoWELDUdJXhjAhsVliSn7bWZjOhrgQ=
-github.com/go-gormigrate/gormigrate/v2 v2.1.4 h1:KOPEt27qy1cNzHfMZbp9YTmEuzkY4F4wrdsJW9WFk1U=
-github.com/go-gormigrate/gormigrate/v2 v2.1.4/go.mod h1:y/6gPAH6QGAgP1UfHMiXcqGeJ88/GRQbfCReE1JJD5Y=
+github.com/go-gormigrate/gormigrate/v2 v2.1.5 h1:1OyorA5LtdQw12cyJDEHuTrEV3GiXiIhS4/QTTa/SM8=
+github.com/go-gormigrate/gormigrate/v2 v2.1.5/go.mod h1:mj9ekk/7CPF3VjopaFvWKN2v7fN3D9d3eEOAXRhi/+M=
 github.com/go-jose/go-jose/v3 v3.0.4 h1:Wp5HA7bLQcKnf6YYao/4kpRpVMp/yf6+pJKV8WFSaNY=
 github.com/go-jose/go-jose/v3 v3.0.4/go.mod h1:5b+7YgP7ZICgJDBdfjZaIt+H/9L9T/YQrVfLAMboGkQ=
-github.com/go-jose/go-jose/v4 v4.1.0 h1:cYSYxd3pw5zd2FSXk2vGdn9igQU2PS8MuxrCOCl0FdY=
-github.com/go-jose/go-jose/v4 v4.1.0/go.mod h1:GG/vqmYm3Von2nYiB2vGTXzdoNKE5tix5tuc6iAd+sw=
-github.com/go-json-experiment/json v0.0.0-20250223041408-d3c622f1b874 h1:F8d1AJ6M9UQCavhwmO6ZsrYLfG8zVFWfEfMS2MXPkSY=
-github.com/go-json-experiment/json v0.0.0-20250223041408-d3c622f1b874/go.mod h1:TiCD2a1pcmjd7YnhGH0f/zKNcCD06B029pHhzV23c2M=
-github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY=
-github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A=
-github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
-github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
+github.com/go-jose/go-jose/v4 v4.1.3 h1:CVLmWDhDVRa6Mi/IgCgaopNosCaHz7zrMeF9MlZRkrs=
+github.com/go-jose/go-jose/v4 v4.1.3/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08=
+github.com/go-json-experiment/json v0.0.0-20250813024750-ebf49471dced h1:Q311OHjMh/u5E2TITc++WlTP5We0xNseRMkHDyvhW7I=
+github.com/go-json-experiment/json v0.0.0-20250813024750-ebf49471dced/go.mod h1:TiCD2a1pcmjd7YnhGH0f/zKNcCD06B029pHhzV23c2M=
+github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
+github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
+github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
 github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
 github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
 github.com/go-ole/go-ole v1.3.0 h1:Dt6ye7+vXGIKZ7Xtk4s6/xVdGDQynvom7xCFEdWr6uE=
 github.com/go-ole/go-ole v1.3.0/go.mod h1:5LS6F96DhAwUc7C+1HLexzMXY1xGRSryjyPPKW6zv78=
 github.com/go-sql-driver/mysql v1.8.1 h1:LedoTUt/eveggdHS9qUFC1EFSa8bU2+1pZjSRpvNJ1Y=
 github.com/go-sql-driver/mysql v1.8.1/go.mod h1:wEBSXgmK//2ZFJyE+qWnIsVGmvmEKlqwuVSjsCm7DZg=
-github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
-github.com/go-viper/mapstructure/v2 v2.2.1 h1:ZAaOCxANMuZx5RCeg0mBdEZk7DZasvvZIxtHqx8aGss=
-github.com/go-viper/mapstructure/v2 v2.2.1/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
+github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
+github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
+github.com/go4org/plan9netshell v0.0.0-20250324183649-788daa080737 h1:cf60tHxREO3g1nroKr2osU3JWZsJzkfi7rEg+oAB0Lo=
+github.com/go4org/plan9netshell v0.0.0-20250324183649-788daa080737/go.mod h1:MIS0jDzbU/vuM9MC4YnBITCv+RYuTRq8dJzmCrFsK9g=
 github.com/gobwas/httphead v0.1.0/go.mod h1:O/RXo79gxV8G+RqlR/otEwx4Q36zl9rqC5u12GKvMCM=
 github.com/gobwas/pool v0.2.1/go.mod h1:q8bcK0KcYlCgd9e7WYLm9LpyS+YeLd8JVDW6WezmKEw=
 github.com/gobwas/ws v1.2.1/go.mod h1:hRKAFb8wOxFROYNsT1bqfWnhX+b5MFeJM9r2ZSwg/KY=
@@ -196,22 +201,14 @@ github.com/godbus/dbus/v5 v5.1.1-0.20230522191255-76236955d466 h1:sQspH8M4niEijh
 github.com/godbus/dbus/v5 v5.1.1-0.20230522191255-76236955d466/go.mod h1:ZiQxhyQ+bbbfxUKVvjfO498oPYvtYhZzycal3G/NHmU=
 github.com/gofrs/uuid/v5 v5.3.2 h1:2jfO8j3XgSwlz/wHqemAEugfnTlikAYHhnqQ8Xh4fE0=
 github.com/gofrs/uuid/v5 v5.3.2/go.mod h1:CDOjlDMVAtN56jqyRUZh58JT31Tiw7/oQyEXZV+9bD8=
-github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
-github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
 github.com/golang-jwt/jwt/v5 v5.2.2 h1:Rl4B7itRWVtYIHFrSNd7vhTiz9UpLdi6gZhZ3wEeDy8=
 github.com/golang-jwt/jwt/v5 v5.2.2/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
-github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
 github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
 github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
-github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
-github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
-github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
-github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
 github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
 github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
 github.com/google/btree v1.1.2 h1:xf4v41cLI2Z6FxbKm+8Bu+m8ifhj15JuZ9sa0jZCMUU=
 github.com/google/btree v1.1.2/go.mod h1:qOPhT0dTNdNzV6Z/lhRX0YXUafgPLFUh+gZMl761Gm4=
-github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
 github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
 github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
@@ -220,40 +217,34 @@ github.com/google/go-github v17.0.0+incompatible h1:N0LgJ1j65A7kfXrZnUDaYCs/Sf4r
 github.com/google/go-github v17.0.0+incompatible/go.mod h1:zLgOLi98H3fifZn+44m+umXrS52loVEgC2AApnigrVQ=
 github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8=
 github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU=
-github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0=
-github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
+github.com/google/go-tpm v0.9.4 h1:awZRf9FwOeTunQmHoDYSHJps3ie6f1UlhS1fOdPEt1I=
+github.com/google/go-tpm v0.9.4/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
 github.com/google/nftables v0.2.1-0.20240414091927-5e242ec57806 h1:wG8RYIyctLhdFk6Vl1yPGtSRtwGpVkWyZww1OCil2MI=
 github.com/google/nftables v0.2.1-0.20240414091927-5e242ec57806/go.mod h1:Beg6V6zZ3oEn0JuiUQ4wqwuyqqzasOltcoXPtgLbFp4=
 github.com/google/pprof v0.0.0-20211214055906-6f57359322fd/go.mod h1:KgnwoLYCZ8IQu3XUZ8Nc/bM9CCZFOyjUNOSygVozoDg=
 github.com/google/pprof v0.0.0-20240227163752-401108e1b7e7/go.mod h1:czg5+yv1E0ZGTi6S6vVK1mke0fV+FaUhNGcd6VRS9Ik=
-github.com/google/pprof v0.0.0-20250501235452-c0086092b71a h1:rDA3FfmxwXR+BVKKdz55WwMJ1pD2hJQNW31d+l3mPk4=
-github.com/google/pprof v0.0.0-20250501235452-c0086092b71a/go.mod h1:5hDyRhoBCxViHszMt12TnOpEI4VVi+U8Gm9iphldiMA=
+github.com/google/pprof v0.0.0-20251007162407-5df77e3f7d1d h1:KJIErDwbSHjnp/SGzE5ed8Aol7JsKiI5X7yWKAtzhM0=
+github.com/google/pprof v0.0.0-20251007162407-5df77e3f7d1d/go.mod h1:I6V7YzU0XDpsHqbsyrghnFZLO1gwK6NPTNvmetQIk9U=
 github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 h1:El6M4kTTCOh6aBiKaUGG7oYTSPP8MxqL4YI3kZKwcP4=
 github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ=
 github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
 github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
+github.com/gookit/assert v0.1.1 h1:lh3GcawXe/p+cU7ESTZ5Ui3Sm/x8JWpIis4/1aF0mY0=
+github.com/gookit/assert v0.1.1/go.mod h1:jS5bmIVQZTIwk42uXl4lyj4iaaxx32tqH16CFj0VX2E=
 github.com/gookit/color v1.4.2/go.mod h1:fqRyamkC1W8uxl+lxCQxOT09l/vYfZ+QeiX3rKQHCoQ=
 github.com/gookit/color v1.5.0/go.mod h1:43aQb+Zerm/BWh2GnrgOQm7ffz7tvQXEKV6BFMl7wAo=
-github.com/gookit/color v1.5.4 h1:FZmqs7XOyGgCAxmWyPslpiok1k05wmY3SJTytgvYFs0=
-github.com/gookit/color v1.5.4/go.mod h1:pZJOeOS8DM43rXbp4AZo1n9zCU2qjpcRko0b6/QJi9w=
-github.com/gorilla/csrf v1.7.3 h1:BHWt6FTLZAb2HtWT5KDBf6qgpZzvtbp9QWDRKZMXJC0=
-github.com/gorilla/csrf v1.7.3/go.mod h1:F1Fj3KG23WYHE6gozCmBAezKookxbIvUJT+121wTuLk=
+github.com/gookit/color v1.6.0 h1:JjJXBTk1ETNyqyilJhkTXJYYigHG24TM9Xa2M1xAhRA=
+github.com/gookit/color v1.6.0/go.mod h1:9ACFc7/1IpHGBW8RwuDm/0YEnhg3dwwXpoMsmtyHfjs=
 github.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY=
 github.com/gorilla/mux v1.8.1/go.mod h1:AKf9I4AEqPTmMytcMc0KkNouC66V3BtZ4qD5fmWSiMQ=
-github.com/gorilla/securecookie v1.1.2 h1:YCIWL56dvtr73r6715mJs5ZvhtnY73hBvEF8kXD8ePA=
-github.com/gorilla/securecookie v1.1.2/go.mod h1:NfCASbcHqRSY+3a8tlWJwsQap2VX5pwzwo4h3eOamfo=
 github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
 github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
-github.com/grpc-ecosystem/go-grpc-middleware v1.4.0 h1:UH//fgunKIs4JdUbpDl1VZCDaL56wXCB/5+wF6uHfaI=
-github.com/grpc-ecosystem/go-grpc-middleware v1.4.0/go.mod h1:g5qyo/la0ALbONm6Vbp88Yd8NsDy6rZz+RcrMPxvld8=
-github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3 h1:5ZPtiqj0JL5oKWmcsq4VMaAW5ukBEgSGXEN89zeH1Jo=
-github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3/go.mod h1:ndYquD05frm2vACXE1nsccT4oJzjhw2arTS2cpUD1PI=
+github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3 h1:NmZ1PKzSTQbuGHw9DGPFomqkkLWMC+vZCkfs+FHv1Vg=
+github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3/go.mod h1:zQrxl1YP88HQlA6i9c63DSVPFklWpGX4OWAc9bFuaH4=
 github.com/hashicorp/go-version v1.7.0 h1:5tqGy27NaOTB8yJKUZELlFAS/LTKJkrmONwQKeRZfjY=
 github.com/hashicorp/go-version v1.7.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
 github.com/hdevalence/ed25519consensus v0.2.0 h1:37ICyZqdyj0lAZ8P4D1d1id3HqbbG1N3iBb1Tb4rdcU=
 github.com/hdevalence/ed25519consensus v0.2.0/go.mod h1:w3BHWjwJbFU29IRHL1Iqkw3sus+7FctEyM4RqDxYNzo=
-github.com/hinshun/vt10x v0.0.0-20220119200601-820417d04eec h1:qv2VnGeEQHchGaZ/u7lxST/RaJw+cv273q79D81Xbog=
-github.com/hinshun/vt10x v0.0.0-20220119200601-820417d04eec/go.mod h1:Q48J4R4DvxnHolD5P8pOtXigYlRuPLGl6moFx3ulM68=
 github.com/ianlancetaylor/demangle v0.0.0-20210905161508-09a460cdf81d/go.mod h1:aYm2/VgdVmcIU8iMfdMvDMsRAQjcfZSKFby6HOFvi/w=
 github.com/ianlancetaylor/demangle v0.0.0-20230524184225-eabc099b10ab/go.mod h1:gx7rwoVhcfuVKG5uya9Hs3Sxj7EIvldVofAWIUtGouw=
 github.com/illarion/gonotify/v3 v3.0.2 h1:O7S6vcopHexutmpObkeWsnzMJt/r1hONIEogeVNmJMk=
@@ -266,8 +257,8 @@ github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsI
 github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
 github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
 github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
-github.com/jackc/pgx/v5 v5.7.4 h1:9wKznZrhWa2QiHL+NjTSPP6yjl3451BX3imWDnokYlg=
-github.com/jackc/pgx/v5 v5.7.4/go.mod h1:ncY89UGWxg82EykZUwSpUKEfccBGGYq1xjrOpsbsfGQ=
+github.com/jackc/pgx/v5 v5.7.6 h1:rWQc5FwZSPX58r1OQmkuaNicxdmExaEz5A2DO2hUuTk=
+github.com/jackc/pgx/v5 v5.7.6/go.mod h1:aruU7o91Tc2q2cFp5h4uP3f6ztExVpyVv88Xl/8Vl8M=
 github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
 github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
 github.com/jagottsicher/termcolor v1.0.2 h1:fo0c51pQSuLBN1+yVX2ZE+hE+P7ULb/TY8eRowJnrsM=
@@ -285,10 +276,6 @@ github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfC
 github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
 github.com/jsimonetti/rtnetlink v1.4.1 h1:JfD4jthWBqZMEffc5RjgmlzpYttAVw1sdnmiNaPO3hE=
 github.com/jsimonetti/rtnetlink v1.4.1/go.mod h1:xJjT7t59UIZ62GLZbv6PLLo8VFrostJMPBAheR6OM8w=
-github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs=
-github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
-github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
-github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
 github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
 github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
 github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
@@ -296,7 +283,6 @@ github.com/klauspost/cpuid/v2 v2.0.10/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuOb
 github.com/klauspost/cpuid/v2 v2.0.12/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuObKfj5c0PQa7c=
 github.com/klauspost/cpuid/v2 v2.2.3 h1:sxCkb+qR91z4vsqw4vGGZlDgPz3G7gjaLyK3V8y70BU=
 github.com/klauspost/cpuid/v2 v2.2.3/go.mod h1:RVVoqg1df56z8g3pUjL/3lE5UfnlrJX8tyFgg4nqhuY=
-github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
 github.com/kortschak/wol v0.0.0-20200729010619-da482cc4850a h1:+RR6SqnTkDLWyICxS1xpjCi/3dhyV+TgZwA6Ww3KncQ=
 github.com/kortschak/wol v0.0.0-20200729010619-da482cc4850a/go.mod h1:YTtCCM3ryyfiu4F7t8HQ1mxvp1UBdWM2r6Xa+nGWvDk=
 github.com/kr/fs v0.1.0 h1:Jskdu9ieNAYnjxsi0LbQp1ulIKZV1LAFgK1tWhpZgl8=
@@ -318,18 +304,16 @@ github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
 github.com/lithammer/fuzzysearch v1.1.8 h1:/HIuJnjHuXS8bKaiTMeeDlW2/AyIWk2brx1V8LFgLN4=
 github.com/lithammer/fuzzysearch v1.1.8/go.mod h1:IdqeyBClc3FFqSzYq/MXESsS4S0FsZ5ajtkr5xPLts4=
 github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
-github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
 github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
 github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
 github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
-github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
 github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
 github.com/mattn/go-isatty v0.0.19/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
 github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
 github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
 github.com/mattn/go-runewidth v0.0.13/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
-github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc=
-github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
+github.com/mattn/go-runewidth v0.0.19 h1:v++JhqYnZuu5jSKrk9RbgF5v4CGUjqRfBm05byFGLdw=
+github.com/mattn/go-runewidth v0.0.19/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
 github.com/mdlayher/genetlink v1.3.2 h1:KdrNKe+CTu+IbZnm/GVUMXSqBBLqcGpRDa0xkQy56gw=
 github.com/mdlayher/genetlink v1.3.2/go.mod h1:tcC3pkCrPUGIKKsCsp0B3AdaaKuHtaxoJRz3cc+528o=
 github.com/mdlayher/netlink v1.7.3-0.20250113171957-fbb4dce95f42 h1:A1Cq6Ysb0GM0tpKMbdCXCIfBclan4oHk1Jb+Hrejirg=
@@ -338,23 +322,26 @@ github.com/mdlayher/sdnotify v1.0.0 h1:Ma9XeLVN/l0qpyx1tNeMSeTjCPH6NtuD6/N9XdTlQ
 github.com/mdlayher/sdnotify v1.0.0/go.mod h1:HQUmpM4XgYkhDLtd+Uad8ZFK1T9D5+pNxnXQjCeJlGE=
 github.com/mdlayher/socket v0.5.0 h1:ilICZmJcQz70vrWVes1MFera4jGiWNocSkykwwoy3XI=
 github.com/mdlayher/socket v0.5.0/go.mod h1:WkcBFfvyG8QENs5+hfQPl1X6Jpd2yeLIYgrGFmJiJxI=
-github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=
-github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d h1:5PJl274Y63IEHC+7izoQE9x6ikvDFZS2mDVS3drnohI=
-github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=
 github.com/miekg/dns v1.1.58 h1:ca2Hdkz+cDg/7eNF6V56jjzuZ4aCAE+DbVkILdQWG/4=
 github.com/miekg/dns v1.1.58/go.mod h1:Ypv+3b/KadlvW9vJfXOTf300O4UqaHFzFCuHz+rPkBY=
 github.com/mitchellh/go-ps v1.0.0 h1:i6ampVEEF4wQFF+bkYfwYgY+F/uYJDktmvLPf7qIgjc=
 github.com/mitchellh/go-ps v1.0.0/go.mod h1:J4lOc8z8yJs6vUwklHw2XEIiT4z4C40KtWVN3nvg8Pg=
 github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
 github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
+github.com/moby/sys/atomicwriter v0.1.0 h1:kw5D/EqkBwsBFi0ss9v1VG3wIkVhzGvLklJ+w3A14Sw=
+github.com/moby/sys/atomicwriter v0.1.0/go.mod h1:Ul8oqv2ZMNHOceF643P6FKPXeCmYtlQMvpizfsSoaWs=
+github.com/moby/sys/sequential v0.6.0 h1:qrx7XFUd/5DxtqcoH1h438hF5TmOvzC/lspjy7zgvCU=
+github.com/moby/sys/sequential v0.6.0/go.mod h1:uyv8EUTrca5PnDsdMGXhZe6CCe8U/UiTWd+lL+7b/Ko=
 github.com/moby/sys/user v0.4.0 h1:jhcMKit7SA80hivmFJcbB1vqmw//wU61Zdui2eQXuMs=
 github.com/moby/sys/user v0.4.0/go.mod h1:bG+tYYYJgaMtRKgEmuueC0hJEAZWwtIbZTB+85uoHjs=
 github.com/moby/term v0.5.2 h1:6qk3FJAFDs6i/q3W/pQ97SX192qKfZgGjCQqfCJkgzQ=
 github.com/moby/term v0.5.2/go.mod h1:d3djjFCrjnB+fl8NJux+EJzu0msscUP+f8it8hPkFLc=
+github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
+github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
 github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
 github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
-github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
-github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
+github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
+github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
 github.com/nfnt/resize v0.0.0-20180221191011-83c6a9932646 h1:zYyBkD/k9seD2A7fsi6Oo2LfFZAehjjQMERAvZLEDnQ=
 github.com/nfnt/resize v0.0.0-20180221191011-83c6a9932646/go.mod h1:jpp1/29i3P1S/RLdc7JQKbRpFeM1dOBd8T9ki5s+AY8=
 github.com/oauth2-proxy/mockoidc v0.0.0-20240214162133-caebfff84d25 h1:9bCMuD3TcnjeqjPT2gSlha4asp8NvgcFRYExCaikCxk=
@@ -363,23 +350,21 @@ github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8
 github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
 github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=
 github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M=
-github.com/opencontainers/runc v1.3.0 h1:cvP7xbEvD0QQAs0nZKLzkVog2OPZhI/V2w3WmTmUSXI=
-github.com/opencontainers/runc v1.3.0/go.mod h1:9wbWt42gV+KRxKRVVugNP6D5+PQciRbenB4fLVsqGPs=
-github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=
+github.com/opencontainers/runc v1.3.2 h1:GUwgo0Fx9M/pl2utaSYlJfdBcXAB/CZXDxe322lvJ3Y=
+github.com/opencontainers/runc v1.3.2/go.mod h1:F7UQQEsxcjUNnFpT1qPLHZBKYP7yWwk6hq8suLy9cl0=
 github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde/go.mod h1:nZgzbfBr3hhjoZnS66nKrHmduYNpc34ny7RK4z5/HM0=
 github.com/ory/dockertest/v3 v3.12.0 h1:3oV9d0sDzlSQfHtIaB5k6ghUCVMVLpAY8hwrqoCyRCw=
 github.com/ory/dockertest/v3 v3.12.0/go.mod h1:aKNDTva3cp8dwOWwb9cWuX84aH5akkxXRvO7KCwWVjE=
 github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
 github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
-github.com/petermattis/goid v0.0.0-20240813172612-4fcff4a6cae7/go.mod h1:pxMtw7cyUw6B2bRH0ZBANSPg+AoSud1I1iyJHI69jH4=
-github.com/petermattis/goid v0.0.0-20250319124200-ccd6737f222a h1:S+AGcmAESQ0pXCUNnRH7V+bOUIgkSX5qVt2cNKCrm0Q=
-github.com/petermattis/goid v0.0.0-20250319124200-ccd6737f222a/go.mod h1:pxMtw7cyUw6B2bRH0ZBANSPg+AoSud1I1iyJHI69jH4=
+github.com/petermattis/goid v0.0.0-20250813065127-a731cc31b4fe/go.mod h1:pxMtw7cyUw6B2bRH0ZBANSPg+AoSud1I1iyJHI69jH4=
+github.com/petermattis/goid v0.0.0-20250904145737-900bdf8bb490 h1:QTvNkZ5ylY0PGgA+Lih+GdboMLY/G9SEGLMEGVjTVA4=
+github.com/petermattis/goid v0.0.0-20250904145737-900bdf8bb490/go.mod h1:pxMtw7cyUw6B2bRH0ZBANSPg+AoSud1I1iyJHI69jH4=
 github.com/philip-bui/grpc-zerolog v1.0.1 h1:EMacvLRUd2O1K0eWod27ZP5CY1iTNkhBDLSN+Q4JEvA=
 github.com/philip-bui/grpc-zerolog v1.0.1/go.mod h1:qXbiq/2X4ZUMMshsqlWyTHOcw7ns+GZmlqZZN05ZHcQ=
 github.com/pierrec/lz4/v4 v4.1.21 h1:yOVMLb6qSIDP67pl/5F7RepeKYu/VmTyEXvuMI5d9mQ=
 github.com/pierrec/lz4/v4 v4.1.21/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
 github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
-github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
 github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pkg/profile v1.7.0 h1:hnbDkaNWPCLMO9wGLdBFTIZvzDrDfBM2072E1S9gJkA=
@@ -391,15 +376,14 @@ github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRI
 github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/prometheus-community/pro-bing v0.4.0 h1:YMbv+i08gQz97OZZBwLyvmmQEEzyfyrrjEaAchdy3R4=
 github.com/prometheus-community/pro-bing v0.4.0/go.mod h1:b7wRYZtCcPmt4Sz319BykUU241rWLe1VFXyiyWK/dH4=
-github.com/prometheus/client_golang v1.22.0 h1:rb93p9lokFEsctTys46VnV1kLCDpVZ0a/Y92Vm0Zc6Q=
-github.com/prometheus/client_golang v1.22.0/go.mod h1:R7ljNsLXhuQXYZYtw6GAE9AZg8Y7vEW5scdCXrWRXC0=
-github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
-github.com/prometheus/client_model v0.6.1 h1:ZKSh/rekM+n3CeS952MLRAdFwIKqeY8b62p8ais2e9E=
-github.com/prometheus/client_model v0.6.1/go.mod h1:OrxVMOVHjw3lKMa8+x6HeMGkHMQyHDk9E3jmP2AmGiY=
-github.com/prometheus/common v0.63.0 h1:YR/EIY1o3mEFP/kZCD7iDMnLPlGyuU2Gb3HIcXnA98k=
-github.com/prometheus/common v0.63.0/go.mod h1:VVFF/fBIoToEnWRVkYoXEkq3R3paCoxG9PXP74SnV18=
-github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc=
-github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk=
+github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o=
+github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg=
+github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
+github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
+github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs=
+github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA=
+github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg=
+github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is=
 github.com/pterm/pterm v0.12.27/go.mod h1:PhQ89w4i95rhgE+xedAoqous6K9X+r6aSOI2eFF7DZI=
 github.com/pterm/pterm v0.12.29/go.mod h1:WI3qxgvoQFFGKGjGnJR849gU0TsEOvKn5Q8LlY1U7lg=
 github.com/pterm/pterm v0.12.30/go.mod h1:MOqLIyMOgmTDz9yorcYbcw+HsgoZo3BQfg2wtl3HEFE=
@@ -407,15 +391,13 @@ github.com/pterm/pterm v0.12.31/go.mod h1:32ZAWZVXD7ZfG0s8qqHXePte42kdz8ECtRyEej
 github.com/pterm/pterm v0.12.33/go.mod h1:x+h2uL+n7CP/rel9+bImHD5lF3nM9vJj80k9ybiiTTE=
 github.com/pterm/pterm v0.12.36/go.mod h1:NjiL09hFhT/vWjQHSj1athJpx6H8cjpHXNAK5bUw8T8=
 github.com/pterm/pterm v0.12.40/go.mod h1:ffwPLwlbXxP+rxT0GsgDTzS3y3rmpAO1NMjUkGTYf8s=
-github.com/pterm/pterm v0.12.80 h1:mM55B+GnKUnLMUSqhdINe4s6tOuVQIetQ3my8JGyAIg=
-github.com/pterm/pterm v0.12.80/go.mod h1:c6DeF9bSnOSeFPZlfs4ZRAFcf5SCoTwvwQ5xaKGQlHo=
-github.com/puzpuzpuz/xsync/v3 v3.5.1 h1:GJYJZwO6IdxN/IKbneznS6yPkVC+c3zyY/j19c++5Fg=
-github.com/puzpuzpuz/xsync/v3 v3.5.1/go.mod h1:VjzYrABPabuM4KyBh1Ftq6u8nhwY5tBPKP9jpmh0nnA=
+github.com/pterm/pterm v0.12.82 h1:+D9wYhCaeaK0FIQoZtqbNQuNpe2lB2tajKKsTd5paVQ=
+github.com/pterm/pterm v0.12.82/go.mod h1:TyuyrPjnxfwP+ccJdBTeWHtd/e0ybQHkOS/TakajZCw=
+github.com/puzpuzpuz/xsync/v4 v4.2.0 h1:dlxm77dZj2c3rxq0/XNvvUKISAmovoXF4a4qM6Wvkr0=
+github.com/puzpuzpuz/xsync/v4 v4.2.0/go.mod h1:VJDmTCJMBt8igNxnkQd86r+8KUeN1quSfNKu5bLYFQo=
 github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
 github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
 github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
-github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
-github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
 github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
 github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
 github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
@@ -425,52 +407,48 @@ github.com/rs/zerolog v1.34.0/go.mod h1:bJsvje4Z08ROH4Nhs5iH600c3IkWhwp44iRc54W6
 github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
 github.com/safchain/ethtool v0.3.0 h1:gimQJpsI6sc1yIqP/y8GYgiXn/NjgvpM0RNoWLVVmP0=
 github.com/safchain/ethtool v0.3.0/go.mod h1:SA9BwrgyAqNo7M+uaL6IYbxpm5wk3L7Mm6ocLW+CJUs=
-github.com/sagikazarmark/locafero v0.9.0 h1:GbgQGNtTrEmddYDSAH9QLRyfAHY12md+8YFTqyMTC9k=
-github.com/sagikazarmark/locafero v0.9.0/go.mod h1:UBUyz37V+EdMS3hDF3QWIiVr/2dPrx49OMO0Bn0hJqk=
-github.com/samber/lo v1.50.0 h1:XrG0xOeHs+4FQ8gJR97zDz5uOFMW7OwFWiFVzqopKgY=
-github.com/samber/lo v1.50.0/go.mod h1:RjZyNk6WSnUFRKK6EyOhsRJMqft3G+pg7dCWHQCWvsc=
-github.com/sasha-s/go-deadlock v0.3.5 h1:tNCOEEDG6tBqrNDOX35j/7hL5FcFViG6awUGROb2NsU=
-github.com/sasha-s/go-deadlock v0.3.5/go.mod h1:bugP6EGbdGYObIlx7pUZtWqlvo8k9H6vCBBsiChJQ5U=
+github.com/sagikazarmark/locafero v0.12.0 h1:/NQhBAkUb4+fH1jivKHWusDYFjMOOKU88eegjfxfHb4=
+github.com/sagikazarmark/locafero v0.12.0/go.mod h1:sZh36u/YSZ918v0Io+U9ogLYQJ9tLLBmM4eneO6WwsI=
+github.com/samber/lo v1.52.0 h1:Rvi+3BFHES3A8meP33VPAxiBZX/Aws5RxrschYGjomw=
+github.com/samber/lo v1.52.0/go.mod h1:4+MXEGsJzbKGaUEQFKBq2xtfuznW9oz/WrgyzMzRoM0=
+github.com/sasha-s/go-deadlock v0.3.6 h1:TR7sfOnZ7x00tWPfD397Peodt57KzMDo+9Ae9rMiUmw=
+github.com/sasha-s/go-deadlock v0.3.6/go.mod h1:CUqNyyvMxTyjFqDT7MRg9mb4Dv/btmGTqSR+rky/UXo=
 github.com/sergi/go-diff v1.2.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
 github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 h1:n661drycOFuPLCN3Uc8sB6B/s6Z4t2xvBgU1htSHuq8=
 github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3/go.mod h1:A0bzQcvG0E7Rwjx0REVgAGH58e96+X0MeOfepqsbeW4=
-github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
 github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
 github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
-github.com/sourcegraph/conc v0.3.0 h1:OQTbbt6P72L20UqAkXXuLOj79LfEanQ+YQFNpLA9ySo=
-github.com/sourcegraph/conc v0.3.0/go.mod h1:Sdozi7LEKbFPqYX2/J+iBAM6HpqSLTASQIKqDmF7Mt0=
-github.com/spf13/afero v1.14.0 h1:9tH6MapGnn/j0eb0yIXiLjERO8RB6xIVZRDCX7PtqWA=
-github.com/spf13/afero v1.14.0/go.mod h1:acJQ8t0ohCGuMN3O+Pv0V0hgMxNYDlvdk+VTfyZmbYo=
-github.com/spf13/cast v1.8.0 h1:gEN9K4b8Xws4EX0+a0reLmhq8moKn7ntRlQYgjPeCDk=
-github.com/spf13/cast v1.8.0/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo=
-github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo=
-github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0=
-github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o=
-github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
-github.com/spf13/viper v1.20.1 h1:ZMi+z/lvLyPSCoNtFCpqjy0S4kPbirhpTMwl8BkW9X4=
-github.com/spf13/viper v1.20.1/go.mod h1:P9Mdzt1zoHIG8m2eZQinpiBjo6kCmZSKBClNNqjJvu4=
+github.com/spf13/afero v1.15.0 h1:b/YBCLWAJdFWJTN9cLhiXXcD7mzKn9Dm86dNnfyQw1I=
+github.com/spf13/afero v1.15.0/go.mod h1:NC2ByUVxtQs4b3sIUphxK0NioZnmxgyCrfzeuq8lxMg=
+github.com/spf13/cast v1.10.0 h1:h2x0u2shc1QuLHfxi+cTJvs30+ZAHOGRic8uyGTDWxY=
+github.com/spf13/cast v1.10.0/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
+github.com/spf13/cobra v1.10.1 h1:lJeBwCfmrnXthfAupyUTzJ/J4Nc1RsHC/mSRU2dll/s=
+github.com/spf13/cobra v1.10.1/go.mod h1:7SmJGaTHFVBY0jW4NXGluQoLvhqFQM+6XSKD+P4XaB0=
+github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
+github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
+github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
+github.com/spf13/viper v1.21.0 h1:x5S+0EU27Lbphp4UKm1C+1oQO+rKx36vfCoaVebLFSU=
+github.com/spf13/viper v1.21.0/go.mod h1:P0lhsswPGWD/1lZJ9ny3fYnVqxiegrlNrEmgLjbTCAY=
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
-github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
 github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
 github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
-github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
 github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
 github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
 github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
-github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
-github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
+github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
+github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
 github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
 github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
 github.com/tailscale/certstore v0.1.1-0.20231202035212-d3fa0460f47e h1:PtWT87weP5LWHEY//SWsYkSO3RWRZo4OSWagh3YD2vQ=
 github.com/tailscale/certstore v0.1.1-0.20231202035212-d3fa0460f47e/go.mod h1:XrBNfAFN+pwoWuksbFS9Ccxnopa15zJGgXRFN90l3K4=
 github.com/tailscale/go-winio v0.0.0-20231025203758-c4f33415bf55 h1:Gzfnfk2TWrk8Jj4P4c1a3CtQyMaTVCznlkLZI++hok4=
 github.com/tailscale/go-winio v0.0.0-20231025203758-c4f33415bf55/go.mod h1:4k4QO+dQ3R5FofL+SanAUZe+/QfeK0+OIuwDIRu2vSg=
-github.com/tailscale/golang-x-crypto v0.0.0-20250218230618-9a281fd8faca h1:ecjHwH73Yvqf/oIdQ2vxAX+zc6caQsYdPzsxNW1J3G8=
-github.com/tailscale/golang-x-crypto v0.0.0-20250218230618-9a281fd8faca/go.mod h1:ikbF+YT089eInTp9f2vmvy4+ZVnW5hzX1q2WknxSprQ=
+github.com/tailscale/golang-x-crypto v0.0.0-20250404221719-a5573b049869 h1:SRL6irQkKGQKKLzvQP/ke/2ZuB7Py5+XuqtOgSj+iMM=
+github.com/tailscale/golang-x-crypto v0.0.0-20250404221719-a5573b049869/go.mod h1:ikbF+YT089eInTp9f2vmvy4+ZVnW5hzX1q2WknxSprQ=
 github.com/tailscale/goupnp v1.0.1-0.20210804011211-c64d0f06ea05 h1:4chzWmimtJPxRs2O36yuGRW3f9SYV+bMTTvMBI0EKio=
 github.com/tailscale/goupnp v1.0.1-0.20210804011211-c64d0f06ea05/go.mod h1:PdCqy9JzfWMJf1H5UJW2ip33/d4YkoKN0r67yKH1mG8=
 github.com/tailscale/hujson v0.0.0-20250226034555-ec1d1c113d33 h1:idh63uw+gsG05HwjZsAENCG4KZfyvjK03bpjxa5qRRk=
@@ -489,8 +467,8 @@ github.com/tailscale/web-client-prebuilt v0.0.0-20250124233751-d4cd19a26976 h1:U
 github.com/tailscale/web-client-prebuilt v0.0.0-20250124233751-d4cd19a26976/go.mod h1:agQPE6y6ldqCOui2gkIh7ZMztTkIQKH049tv8siLuNQ=
 github.com/tailscale/wf v0.0.0-20240214030419-6fbb0a674ee6 h1:l10Gi6w9jxvinoiq15g8OToDdASBni4CyJOdHY1Hr8M=
 github.com/tailscale/wf v0.0.0-20240214030419-6fbb0a674ee6/go.mod h1:ZXRML051h7o4OcI0d3AaILDIad/Xw0IkXaHM17dic1Y=
-github.com/tailscale/wireguard-go v0.0.0-20250107165329-0b8b35511f19 h1:BcEJP2ewTIK2ZCsqgl6YGpuO6+oKqqag5HHb7ehljKw=
-github.com/tailscale/wireguard-go v0.0.0-20250107165329-0b8b35511f19/go.mod h1:BOm5fXUBFM+m9woLNBoxI9TaBXXhGNP50LX/TGIvGb4=
+github.com/tailscale/wireguard-go v0.0.0-20250716170648-1d0488a3d7da h1:jVRUZPRs9sqyKlYHHzHjAqKN+6e/Vog6NpHYeNPJqOw=
+github.com/tailscale/wireguard-go v0.0.0-20250716170648-1d0488a3d7da/go.mod h1:BOm5fXUBFM+m9woLNBoxI9TaBXXhGNP50LX/TGIvGb4=
 github.com/tailscale/xnet v0.0.0-20240729143630-8497ac4dab2e h1:zOGKqN5D5hHhiYUp091JqK7DPCqSARyUfduhGUY8Bek=
 github.com/tailscale/xnet v0.0.0-20240729143630-8497ac4dab2e/go.mod h1:orPd6JZXXRyuDusYilywte7k094d7dycXXU5YnWsrwg=
 github.com/tc-hib/winres v0.2.1 h1:YDE0FiP0VmtRaDn7+aaChp1KiF4owBiJa5l964l5ujA=
@@ -499,13 +477,13 @@ github.com/tcnksm/go-latest v0.0.0-20170313132115-e3007ae9052e h1:IWllFTiDjjLIf2
 github.com/tcnksm/go-latest v0.0.0-20170313132115-e3007ae9052e/go.mod h1:d7u6HkTYKSv5m6MCKkOQlHwaShTMl3HjqSGW3XtVhXM=
 github.com/tink-crypto/tink-go/v2 v2.1.0 h1:QXFBguwMwTIaU17EgZpEJWsUSc60b1BAGTzBIoMdmok=
 github.com/tink-crypto/tink-go/v2 v2.1.0/go.mod h1:y1TnYFt1i2eZVfx4OGc+C+EMp4CoKWAw2VSEuoicHHI=
-github.com/u-root/u-root v0.12.0 h1:K0AuBFriwr0w/PGS3HawiAw89e3+MU7ks80GpghAsNs=
-github.com/u-root/u-root v0.12.0/go.mod h1:FYjTOh4IkIZHhjsd17lb8nYW6udgXdJhG1c0r6u0arI=
+github.com/u-root/u-root v0.14.0 h1:Ka4T10EEML7dQ5XDvO9c3MBN8z4nuSnGjcd1jmU2ivg=
+github.com/u-root/u-root v0.14.0/go.mod h1:hAyZorapJe4qzbLWlAkmSVCJGbfoU9Pu4jpJ1WMluqE=
 github.com/u-root/uio v0.0.0-20240224005618-d2acac8f3701 h1:pyC9PaHYZFgEKFdlp3G8RaCKgVpHZnecvArXvPXcFkM=
 github.com/u-root/uio v0.0.0-20240224005618-d2acac8f3701/go.mod h1:P3a5rG4X7tI17Nn3aOIAYr5HbIMukwXG0urG0WuL8OA=
 github.com/vishvananda/netns v0.0.0-20200728191858-db3c7e526aae/go.mod h1:DD4vA1DwXk04H54A1oHXtwZmA0grkVMdPxx/VGLCah0=
-github.com/vishvananda/netns v0.0.4 h1:Oeaw1EM2JMxD51g9uhtC0D7erkIjgmj8+JZc26m1YX8=
-github.com/vishvananda/netns v0.0.4/go.mod h1:SpkAiCQRtJ6TvvxPnOSyH3BMl6unz3xZlaprSwhNNJM=
+github.com/vishvananda/netns v0.0.5 h1:DfiHV+j8bA32MFM7bfEunvT8IAqQ/NzSJHtcmW5zdEY=
+github.com/vishvananda/netns v0.0.5/go.mod h1:SpkAiCQRtJ6TvvxPnOSyH3BMl6unz3xZlaprSwhNNJM=
 github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM=
 github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg=
 github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU=
@@ -518,90 +496,70 @@ github.com/xeipuuv/gojsonschema v1.2.0/go.mod h1:anYRn/JVcOK2ZgGU+IjEV4nwlhoK5sQ
 github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778/go.mod h1:2MuV+tbUrU1zIOPMxZ5EncGwgmMJsa+9ucAQZXxsObs=
 github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no=
 github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
-github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
-github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
 go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
 go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
-go.opentelemetry.io/otel v1.34.0 h1:zRLXxLCgL1WyKsPVrgbSdMN4c0FMkDAskSTQP+0hdUY=
-go.opentelemetry.io/otel v1.34.0/go.mod h1:OWFPOQ+h4G8xpyjgqo4SxJYdDQ/qmRH+wivy7zzx9oI=
-go.opentelemetry.io/otel/metric v1.34.0 h1:+eTR3U0MyfWjRDhmFMxe2SsW64QrZ84AOhvqS7Y+PoQ=
-go.opentelemetry.io/otel/metric v1.34.0/go.mod h1:CEDrp0fy2D0MvkXE+dPV7cMi8tWZwX3dmaIhwPOaqHE=
-go.opentelemetry.io/otel/sdk v1.34.0 h1:95zS4k/2GOy069d321O8jWgYsW3MzVV+KuSPKp7Wr1A=
-go.opentelemetry.io/otel/sdk v1.34.0/go.mod h1:0e/pNiaMAqaykJGKbi+tSjWfNNHMTxoC9qANsCzbyxU=
-go.opentelemetry.io/otel/sdk/metric v1.34.0 h1:5CeK9ujjbFVL5c1PhLuStg1wxA7vQv7ce1EK0Gyvahk=
-go.opentelemetry.io/otel/sdk/metric v1.34.0/go.mod h1:jQ/r8Ze28zRKoNRdkjCZxfs6YvBTG1+YIqyFVFYec5w=
-go.opentelemetry.io/otel/trace v1.34.0 h1:+ouXS2V8Rd4hp4580a8q23bg0azF2nI8cqLYnC8mh/k=
-go.opentelemetry.io/otel/trace v1.34.0/go.mod h1:Svm7lSjQD7kG7KJ/MUHPVXSDGz2OX4h0M2jHBhmSfRE=
-go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
-go.uber.org/goleak v1.1.10/go.mod h1:8a7PlsEVH3e/a/GLqe5IIrQx6GzcnRmZEufDUTk4A7A=
-go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU=
-go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
-go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
-go.uber.org/zap v1.18.1/go.mod h1:xg/QME4nWcxGxrpdeYfq7UvYrLh66cuVKdrbD1XF/NI=
+go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.58.0 h1:yd02MEjBdJkG3uabWP9apV+OuWRIXGDuJEUJbOHmCFU=
+go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.58.0/go.mod h1:umTcuxiv1n/s/S6/c2AT/g2CQ7u5C59sHDNmfSwgz7Q=
+go.opentelemetry.io/otel v1.37.0 h1:9zhNfelUvx0KBfu/gb+ZgeAfAgtWrfHJZcAqFC228wQ=
+go.opentelemetry.io/otel v1.37.0/go.mod h1:ehE/umFRLnuLa/vSccNq9oS1ErUlkkK71gMcN34UG8I=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.36.0 h1:dNzwXjZKpMpE2JhmO+9HsPl42NIXFIFSUSSs0fiqra0=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.36.0/go.mod h1:90PoxvaEB5n6AOdZvi+yWJQoE95U8Dhhw2bSyRqnTD0=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.36.0 h1:nRVXXvf78e00EwY6Wp0YII8ww2JVWshZ20HfTlE11AM=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.36.0/go.mod h1:r49hO7CgrxY9Voaj3Xe8pANWtr0Oq916d0XAmOoCZAQ=
+go.opentelemetry.io/otel/metric v1.37.0 h1:mvwbQS5m0tbmqML4NqK+e3aDiO02vsf/WgbsdpcPoZE=
+go.opentelemetry.io/otel/metric v1.37.0/go.mod h1:04wGrZurHYKOc+RKeye86GwKiTb9FKm1WHtO+4EVr2E=
+go.opentelemetry.io/otel/sdk v1.37.0 h1:ItB0QUqnjesGRvNcmAcU0LyvkVyGJ2xftD29bWdDvKI=
+go.opentelemetry.io/otel/sdk v1.37.0/go.mod h1:VredYzxUvuo2q3WRcDnKDjbdvmO0sCzOvVAiY+yUkAg=
+go.opentelemetry.io/otel/sdk/metric v1.37.0 h1:90lI228XrB9jCMuSdA0673aubgRobVZFhbjxHHspCPc=
+go.opentelemetry.io/otel/sdk/metric v1.37.0/go.mod h1:cNen4ZWfiD37l5NhS+Keb5RXVWZWpRE+9WyVCpbo5ps=
+go.opentelemetry.io/otel/trace v1.37.0 h1:HLdcFNbRQBE2imdSEgm/kwqmQj1Or1l/7bW6mxVK7z4=
+go.opentelemetry.io/otel/trace v1.37.0/go.mod h1:TlgrlQ+PtQO5XFerSPUYG0JSgGyryXewPGyayAWSBS0=
+go.opentelemetry.io/proto/otlp v1.6.0 h1:jQjP+AQyTf+Fe7OKj/MfkDrmK4MNVtw2NpXsf9fefDI=
+go.opentelemetry.io/proto/otlp v1.6.0/go.mod h1:cicgGehlFuNdgZkcALOCh3VE6K/u2tAjzlRhDwmVpZc=
+go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
+go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
+go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI=
+go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU=
+go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
+go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
 go4.org/mem v0.0.0-20240501181205-ae6ca9944745 h1:Tl++JLUCe4sxGu8cTpDzRLd3tN7US4hOxG5YpKCzkek=
 go4.org/mem v0.0.0-20240501181205-ae6ca9944745/go.mod h1:reUoABIJ9ikfM5sgtSF3Wushcza7+WeD01VB9Lirh3g=
 go4.org/netipx v0.0.0-20231129151722-fdeea329fbba h1:0b9z3AuHCjxk0x/opv64kcgZLBseWJUpBw5I82+2U4M=
 go4.org/netipx v0.0.0-20231129151722-fdeea329fbba/go.mod h1:PLyyIXexvUFg3Owu6p/WfdlivPbZJsZdgWZlrGope/Y=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
-golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
-golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
 golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
 golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
-golang.org/x/crypto v0.37.0 h1:kJNSjF/Xp7kU0iB2Z+9viTPMW4EqqsrywMXLJOOsXSE=
-golang.org/x/crypto v0.37.0/go.mod h1:vg+k43peMZ0pUMhYmVAWysMK35e6ioLh3wB8ZCAfbVc=
-golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
-golang.org/x/exp v0.0.0-20250408133849-7e4ce0ab07d0 h1:R84qjqJb5nVJMxqWYb3np9L5ZsaDtB+a39EqjV0JSUM=
-golang.org/x/exp v0.0.0-20250408133849-7e4ce0ab07d0/go.mod h1:S9Xr4PYopiDyqSyp5NjCrhFrqg6A5zA2E/iPHPhqnS8=
+golang.org/x/crypto v0.43.0 h1:dduJYIi3A3KOfdGOHX8AVZ/jGiyPa3IbBozJ5kNuE04=
+golang.org/x/crypto v0.43.0/go.mod h1:BFbav4mRNlXJL4wNeejLpWxB7wMbc79PdRGhWKncxR0=
+golang.org/x/exp v0.0.0-20251009144603-d2f985daa21b h1:18qgiDvlvH7kk8Ioa8Ov+K6xCi0GMvmGfGW0sgd/SYA=
+golang.org/x/exp v0.0.0-20251009144603-d2f985daa21b/go.mod h1:j/pmGrbnkbPtQfxEe5D0VQhZC6qKbfKifgD0oM7sR70=
 golang.org/x/exp/typeparams v0.0.0-20240314144324-c7f7c6466f7f h1:phY1HzDcf18Aq9A8KkmRtY9WvOFIxN8wgfvy6Zm1DV8=
 golang.org/x/exp/typeparams v0.0.0-20240314144324-c7f7c6466f7f/go.mod h1:AbB0pIl9nAr9wVwH+Z2ZpaocVmF5I4GyWCDIsVjR0bk=
-golang.org/x/image v0.24.0 h1:AN7zRgVsbvmTfNyqIbbOraYL8mSwcKncEj8ofjgzcMQ=
-golang.org/x/image v0.24.0/go.mod h1:4b/ITuLfqYq1hqZcjofwctIhi7sZh2WaCjvsBNjjya8=
-golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
-golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
-golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
-golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
-golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
-golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
+golang.org/x/image v0.27.0 h1:C8gA4oWU/tKkdCfYT6T2u4faJu3MeNS5O8UPWlPF61w=
+golang.org/x/image v0.27.0/go.mod h1:xbdrClrAUway1MUTEZDq9mz/UpRwYAkFFNUslZtcB+g=
 golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
 golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
-golang.org/x/mod v0.24.0 h1:ZfthKaKaT4NrhGVZHO1/WDTwGES4De8KtWO0SIbNJMU=
-golang.org/x/mod v0.24.0/go.mod h1:IXM97Txy2VM4PJ3gI61r1YEk/gAj6zAHN3AdZt6S9Ww=
-golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
-golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
-golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
-golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
-golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/mod v0.29.0 h1:HV8lRxZC4l2cr3Zq1LvtOsi/ThTgWnUk/y64QSs8GwA=
+golang.org/x/mod v0.29.0/go.mod h1:NyhrlYXJ2H4eJiRy/WDBO6HMqZQ6q9nk4JzS3NuCK+w=
 golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
 golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
 golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
 golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
 golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
-golang.org/x/net v0.39.0 h1:ZCu7HMWDxpXpaiKdhzIfaltL9Lp31x/3fCP11bc6/fY=
-golang.org/x/net v0.39.0/go.mod h1:X7NRbYVEA+ewNkCNyJ513WmMdQ3BineSwVtN2zD/d+E=
-golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
-golang.org/x/oauth2 v0.29.0 h1:WdYw2tdTK1S8olAzWHdgeqfy+Mtm9XNhv/xJsY65d98=
-golang.org/x/oauth2 v0.29.0/go.mod h1:onh5ek6nERTohokkhCD/y2cV4Do3fxFHFuAejCkRWT8=
-golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/net v0.46.0 h1:giFlY12I07fugqwPuWJi68oOnpfqFnJIJzaIIm2JVV4=
+golang.org/x/net v0.46.0/go.mod h1:Q9BGdFy1y4nkUwiLvT5qtyhAnEHgnQ/zd8PfU6nc210=
+golang.org/x/oauth2 v0.32.0 h1:jsCblLleRMDrxMN29H3z/k1KliIvpLgCkE6R8FXXNgY=
+golang.org/x/oauth2 v0.32.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA=
 golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.13.0 h1:AauUjRAJ9OSnvULf/ARrrVywoJDy0YS2AwQ98I37610=
-golang.org/x/sync v0.13.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
-golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug=
+golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
-golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
-golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200217220822-9197077df867/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200728102440-3e129f6d46b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -609,7 +567,6 @@ golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBc
 golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20211007075335-d3039528d8ac/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20211013075003-97ac67df715c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20211025201205-69cdffdb9359/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20220310020820-b874c991c1a5/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20220319134239-a9b59b0215f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
@@ -623,8 +580,8 @@ golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
-golang.org/x/sys v0.32.0 h1:s77OFDvIQeibCmezSnk/q6iAfkdiQaJi4VzroCFrN20=
-golang.org/x/sys v0.32.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
+golang.org/x/sys v0.37.0 h1:fdNQudmxPjkdUTPnLn5mdQv7Zwvbvpaxqs831goi9kQ=
+golang.org/x/sys v0.37.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
 golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
 golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
@@ -632,60 +589,41 @@ golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuX
 golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
 golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo=
 golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
-golang.org/x/term v0.31.0 h1:erwDkOK1Msy6offm1mOgvspSkslFnIGsFnxOKoufg3o=
-golang.org/x/term v0.31.0/go.mod h1:R4BeIy7D95HzImkxGkTW1UQTtP54tio2RyHz7PwK0aw=
+golang.org/x/term v0.36.0 h1:zMPR+aF8gfksFprF/Nc/rd1wRS1EI6nDBGyWAvDzx2Q=
+golang.org/x/term v0.36.0/go.mod h1:Qu394IJq6V6dCBRgwqshf3mPF85AqzYEzofzRdZkWss=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
-golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
 golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
 golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
 golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
-golang.org/x/text v0.24.0 h1:dd5Bzh4yt5KYA8f9CJHCP4FB4D51c2c6JvN37xJJkJ0=
-golang.org/x/text v0.24.0/go.mod h1:L8rBsPeo2pSS+xqN0d5u2ikmjtmoJbDBT1b7nHvFCdU=
-golang.org/x/time v0.10.0 h1:3usCWA8tQn0L8+hFJQNgzpWbd89begxN66o1Ojdn5L4=
-golang.org/x/time v0.10.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
+golang.org/x/text v0.30.0 h1:yznKA/E9zq54KzlzBEAWn1NXSQ8DIp/NYMy88xJjl4k=
+golang.org/x/text v0.30.0/go.mod h1:yDdHFIX9t+tORqspjENWgzaCVXgk0yYnYuSZ8UzzBVM=
+golang.org/x/time v0.11.0 h1:/bpjEDfN9tkoN/ryeYHnv5hcMlc8ncjMcM4XBk5NWV0=
+golang.org/x/time v0.11.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=
 golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
-golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
-golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
-golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
-golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
-golang.org/x/tools v0.0.0-20191108193012-7d206e10da11/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
 golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
-golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
 golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
-golang.org/x/tools v0.32.0 h1:Q7N1vhpkQv7ybVzLFtTjvQya2ewbwNDZzUgfXGqtMWU=
-golang.org/x/tools v0.32.0/go.mod h1:ZxrU41P/wAbZD8EDa6dDCa6XfpkhJ7HFMjHJXfBDu8s=
+golang.org/x/tools v0.38.0 h1:Hx2Xv8hISq8Lm16jvBZ2VQf+RLmbd7wVUsALibYI/IQ=
+golang.org/x/tools v0.38.0/go.mod h1:yEsQ/d/YK8cjh0L6rZlY8tgtlKiBNTL14pGDJPJpYQs=
 golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
-golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
-golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.zx2c4.com/wintun v0.0.0-20230126152724-0fa3db229ce2 h1:B82qJJgjvYKsXS9jeunTOisW56dUokqW/FOteYJJ/yg=
 golang.zx2c4.com/wintun v0.0.0-20230126152724-0fa3db229ce2/go.mod h1:deeaetjYA+DHMHg+sMSMI58GrEteJUUzzw7en6TJQcI=
 golang.zx2c4.com/wireguard/windows v0.5.3 h1:On6j2Rpn3OEMXqBq00QEDC7bWSZrPIHKIus8eIuExIE=
 golang.zx2c4.com/wireguard/windows v0.5.3/go.mod h1:9TEe8TJmtwyQebdFwAkEWOPr3prrtqm+REGFifP60hI=
-google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
-google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
-google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
-google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
-google.golang.org/genproto v0.0.0-20200423170343-7949de9c1215/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/genproto/googleapis/api v0.0.0-20250428153025-10db94c68c34 h1:0PeQib/pH3nB/5pEmFeVQJotzGohV0dq4Vcp09H5yhE=
-google.golang.org/genproto/googleapis/api v0.0.0-20250428153025-10db94c68c34/go.mod h1:0awUlEkap+Pb1UMeJwJQQAdJQrt3moU7J2moTy69irI=
-google.golang.org/genproto/googleapis/rpc v0.0.0-20250428153025-10db94c68c34 h1:h6p3mQqrmT1XkHVTfzLdNz1u7IhINeZkz67/xTbOuWs=
-google.golang.org/genproto/googleapis/rpc v0.0.0-20250428153025-10db94c68c34/go.mod h1:qQ0YXyHHx3XkvlzUtpXDkS29lDSafHMZBAZDc03LQ3A=
-google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
-google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
-google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
-google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
-google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk=
-google.golang.org/grpc v1.72.0 h1:S7UkcVa60b5AAQTaO6ZKamFp1zMZSU0fGDK2WZLbBnM=
-google.golang.org/grpc v1.72.0/go.mod h1:wH5Aktxcg25y1I3w7H69nHfXdOG3UiadoBtjh3izSDM=
-google.golang.org/protobuf v1.36.6 h1:z1NpPI8ku2WgiWnf+t9wTPsn6eP1L7ksHUlkfLvd9xY=
-google.golang.org/protobuf v1.36.6/go.mod h1:jduwjTPXsFjZGTmRluh+L6NjiWu7pchiJ2/5YcXBHnY=
+gonum.org/v1/gonum v0.16.0 h1:5+ul4Swaf3ESvrOnidPp4GZbzf0mxVQpDCYUQE7OJfk=
+gonum.org/v1/gonum v0.16.0/go.mod h1:fef3am4MQ93R2HHpKnLk4/Tbh/s0+wqD5nfa6Pnwy4E=
+google.golang.org/genproto/googleapis/api v0.0.0-20250929231259-57b25ae835d4 h1:8XJ4pajGwOlasW+L13MnEGA8W4115jJySQtVfS2/IBU=
+google.golang.org/genproto/googleapis/api v0.0.0-20250929231259-57b25ae835d4/go.mod h1:NnuHhy+bxcg30o7FnVAZbXsPHUDQ9qKWAQKCD7VxFtk=
+google.golang.org/genproto/googleapis/rpc v0.0.0-20250929231259-57b25ae835d4 h1:i8QOKZfYg6AbGVZzUAY3LrNWCKF8O6zFisU9Wl9RER4=
+google.golang.org/genproto/googleapis/rpc v0.0.0-20250929231259-57b25ae835d4/go.mod h1:HSkG/KdJWusxU1F6CNrwNDjBMgisKxGnc5dAZfT0mjQ=
+google.golang.org/grpc v1.75.1 h1:/ODCNEuf9VghjgO3rqLcfg8fiOP0nSluljWFlDxELLI=
+google.golang.org/grpc v1.75.1/go.mod h1:JtPAzKiq4v1xcAB2hydNlWI2RnF85XXcV0mhKXr2ecQ=
+google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE=
+google.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
-gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
@@ -698,55 +636,49 @@ gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C
 gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
-gorm.io/driver/postgres v1.5.11 h1:ubBVAfbKEUld/twyKZ0IYn9rSQh448EdelLYk9Mv314=
-gorm.io/driver/postgres v1.5.11/go.mod h1:DX3GReXH+3FPWGrrgffdvCk3DQ1dwDPdmbenSkweRGI=
-gorm.io/gorm v1.25.12 h1:I0u8i2hWQItBq1WfE0o2+WuL9+8L21K9e2HHSTE/0f8=
-gorm.io/gorm v1.25.12/go.mod h1:xh7N7RHfYlNc5EmcI/El95gXusucDrQnHXe0+CgWcLQ=
-gorm.io/gorm v1.26.0 h1:9lqQVPG5aNNS6AyHdRiwScAVnXHg/L/Srzx55G5fOgs=
-gorm.io/gorm v1.26.0/go.mod h1:8Z33v652h4//uMA76KjeDH8mJXPm1QNCYrMeatR0DOE=
+gorm.io/driver/postgres v1.6.0 h1:2dxzU8xJ+ivvqTRph34QX+WrRaJlmfyPqXmoGVjMBa4=
+gorm.io/driver/postgres v1.6.0/go.mod h1:vUw0mrGgrTK+uPHEhAdV4sfFELrByKVGnaVRkXDhtWo=
+gorm.io/gorm v1.31.0 h1:0VlycGreVhK7RF/Bwt51Fk8v0xLiiiFdbGDPIZQ7mJY=
+gorm.io/gorm v1.31.0/go.mod h1:XyQVbO2k6YkOis7C2437jSit3SsDK72s7n7rsSHd+Gs=
 gotest.tools/v3 v3.5.1 h1:EENdUnS3pdur5nybKYIh2Vfgc8IUNBjxDPSjtiJcOzU=
 gotest.tools/v3 v3.5.1/go.mod h1:isy3WKz7GK6uNw/sbHzfKBLvlvXwUyV06n6brMxxopU=
 gvisor.dev/gvisor v0.0.0-20250205023644-9414b50a5633 h1:2gap+Kh/3F47cO6hAu3idFvsJ0ue6TRcEi2IUkv/F8k=
 gvisor.dev/gvisor v0.0.0-20250205023644-9414b50a5633/go.mod h1:5DMfjtclAbTIjbXqO1qCe2K5GKKxWz2JHvCChuTcJEM=
-honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
-honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
 honnef.co/go/tools v0.6.1 h1:R094WgE8K4JirYjBaOpz/AvTyUu/3wbmAoskKN/pxTI=
 honnef.co/go/tools v0.6.1/go.mod h1:3puzxxljPCe8RGJX7BIy1plGbxEOZni5mR2aXe3/uk4=
 howett.net/plist v1.0.0 h1:7CrbWYbPPO/PyNy38b2EB/+gYbjCe2DXBxgtOOZbSQM=
 howett.net/plist v1.0.0/go.mod h1:lqaXoTrLY4hg8tnEzNru53gicrbv7rrk+2xJA/7hw9g=
-modernc.org/cc/v4 v4.25.2 h1:T2oH7sZdGvTaie0BRNFbIYsabzCxUQg8nLqCdQ2i0ic=
-modernc.org/cc/v4 v4.26.0 h1:QMYvbVduUGH0rrO+5mqF/PSPPRZNpRtg2CLELy7vUpA=
-modernc.org/cc/v4 v4.26.0/go.mod h1:uVtb5OGqUKpoLWhqwNQo/8LwvoiEBLvZXIQ/SmO6mL0=
-modernc.org/ccgo/v4 v4.25.1 h1:TFSzPrAGmDsdnhT9X2UrcPMI3N/mJ9/X9ykKXwLhDsU=
-modernc.org/ccgo/v4 v4.26.0 h1:gVzXaDzGeBYJ2uXTOpR8FR7OlksDOe9jxnjhIKCsiTc=
-modernc.org/ccgo/v4 v4.26.0/go.mod h1:Sem8f7TFUtVXkG2fiaChQtyyfkqhJBg/zjEJBkmuAVY=
-modernc.org/fileutil v1.3.1 h1:8vq5fe7jdtEvoCf3Zf9Nm0Q05sH6kGx0Op2CPx1wTC8=
-modernc.org/fileutil v1.3.1/go.mod h1:HxmghZSZVAz/LXcMNwZPA/DRrQZEVP9VX0V4LQGQFOc=
+modernc.org/cc/v4 v4.26.5 h1:xM3bX7Mve6G8K8b+T11ReenJOT+BmVqQj0FY5T4+5Y4=
+modernc.org/cc/v4 v4.26.5/go.mod h1:uVtb5OGqUKpoLWhqwNQo/8LwvoiEBLvZXIQ/SmO6mL0=
+modernc.org/ccgo/v4 v4.28.1 h1:wPKYn5EC/mYTqBO373jKjvX2n+3+aK7+sICCv4Fjy1A=
+modernc.org/ccgo/v4 v4.28.1/go.mod h1:uD+4RnfrVgE6ec9NGguUNdhqzNIeeomeXf6CL0GTE5Q=
+modernc.org/fileutil v1.3.40 h1:ZGMswMNc9JOCrcrakF1HrvmergNLAmxOPjizirpfqBA=
+modernc.org/fileutil v1.3.40/go.mod h1:HxmghZSZVAz/LXcMNwZPA/DRrQZEVP9VX0V4LQGQFOc=
 modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
 modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
-modernc.org/libc v1.62.1 h1:s0+fv5E3FymN8eJVmnk0llBe6rOxCu/DEU+XygRbS8s=
-modernc.org/libc v1.62.1/go.mod h1:iXhATfJQLjG3NWy56a6WVU73lWOcdYVxsvwCgoPljuo=
-modernc.org/libc v1.65.0 h1:e183gLDnAp9VJh6gWKdTy0CThL9Pt7MfcR/0bgb7Y1Y=
-modernc.org/libc v1.65.0/go.mod h1:7m9VzGq7APssBTydds2zBcxGREwvIGpuUBaKTXdm2Qs=
+modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
+modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
+modernc.org/libc v1.66.10 h1:yZkb3YeLx4oynyR+iUsXsybsX4Ubx7MQlSYEw4yj59A=
+modernc.org/libc v1.66.10/go.mod h1:8vGSEwvoUoltr4dlywvHqjtAqHBaw0j1jI7iFBTAr2I=
 modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
 modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
-modernc.org/memory v1.10.0 h1:fzumd51yQ1DxcOxSO+S6X7+QTuVU+n8/Aj7swYjFfC4=
-modernc.org/memory v1.10.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
+modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
+modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
 modernc.org/opt v0.1.4 h1:2kNGMRiUjrp4LcaPuLY2PzUfqM/w9N23quVwhKt5Qm8=
 modernc.org/opt v0.1.4/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
 modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
 modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
-modernc.org/sqlite v1.37.0 h1:s1TMe7T3Q3ovQiK2Ouz4Jwh7dw4ZDqbebSDTlSJdfjI=
-modernc.org/sqlite v1.37.0/go.mod h1:5YiWv+YviqGMuGw4V+PNplcyaJ5v+vQd7TQOgkACoJM=
+modernc.org/sqlite v1.39.1 h1:H+/wGFzuSCIEVCvXYVHX5RQglwhMOvtHSv+VtidL2r4=
+modernc.org/sqlite v1.39.1/go.mod h1:9fjQZ0mB1LLP0GYrp39oOJXx/I2sxEnZtzCmEQIKvGE=
 modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
 modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
 modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
 modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
 software.sslmate.com/src/go-pkcs12 v0.4.0 h1:H2g08FrTvSFKUj+D309j1DPfk5APnIdAQAB8aEykJ5k=
 software.sslmate.com/src/go-pkcs12 v0.4.0/go.mod h1:Qiz0EyvDRJjjxGyUQa2cCNZn/wMyzrRJ/qcDXOQazLI=
-tailscale.com v1.83.0-pre.0.20250331211809-96fe8a6db6c9 h1:mPTb8dGYSqzJhrrYNrLVP717Nh8DME85DWnhBATB/94=
-tailscale.com v1.83.0-pre.0.20250331211809-96fe8a6db6c9/go.mod h1:iU6kohVzG+bP0/5XjqBAnW8/6nSG/Du++bO+x7VJZD0=
-zgo.at/zcache/v2 v2.1.0 h1:USo+ubK+R4vtjw4viGzTe/zjXyPw6R7SK/RL3epBBxs=
-zgo.at/zcache/v2 v2.1.0/go.mod h1:gyCeoLVo01QjDZynjime8xUGHHMbsLiPyUTBpDGd4Gk=
+tailscale.com v1.86.5 h1:yBtWFjuLYDmxVnfnvPbZNZcKADCYgNfMd0rUAOA9XCs=
+tailscale.com v1.86.5/go.mod h1:Lm8dnzU2i/Emw15r6sl3FRNp/liSQ/nYw6ZSQvIdZ1M=
+zgo.at/zcache/v2 v2.4.1 h1:Dfjoi8yI0Uq7NCc4lo2kaQJJmp9Mijo21gef+oJstbY=
+zgo.at/zcache/v2 v2.4.1/go.mod h1:gyCeoLVo01QjDZynjime8xUGHHMbsLiPyUTBpDGd4Gk=
 zombiezen.com/go/postgrestest v1.0.1 h1:aXoADQAJmZDU3+xilYVut0pHhgc0sF8ZspPW9gFNwP4=
 zombiezen.com/go/postgrestest v1.0.1/go.mod h1:marlZezr+k2oSJrvXHnZUs1olHqpE9czlz8ZYkVxliQ=

hscontrol/app.go

@@ -5,7 +5,6 @@ import (
 	"crypto/tls"
 	"errors"
 	"fmt"
-	"io"
 	"net"
 	"net/http"
 	_ "net/http/pprof" // nolint
@@ -18,9 +17,9 @@ import (
 	"syscall"
 	"time"
 
+	"github.com/cenkalti/backoff/v5"
 	"github.com/davecgh/go-spew/spew"
 	"github.com/gorilla/mux"
-	grpcMiddleware "github.com/grpc-ecosystem/go-grpc-middleware"
 	grpcRuntime "github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
 	"github.com/juanfont/headscale"
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
@@ -30,15 +29,15 @@ import (
 	derpServer "github.com/juanfont/headscale/hscontrol/derp/server"
 	"github.com/juanfont/headscale/hscontrol/dns"
 	"github.com/juanfont/headscale/hscontrol/mapper"
-	"github.com/juanfont/headscale/hscontrol/notifier"
-	"github.com/juanfont/headscale/hscontrol/policy"
-	"github.com/juanfont/headscale/hscontrol/routes"
+	"github.com/juanfont/headscale/hscontrol/state"
 	"github.com/juanfont/headscale/hscontrol/types"
+	"github.com/juanfont/headscale/hscontrol/types/change"
 	"github.com/juanfont/headscale/hscontrol/util"
 	zerolog "github.com/philip-bui/grpc-zerolog"
 	"github.com/pkg/profile"
 	zl "github.com/rs/zerolog"
 	"github.com/rs/zerolog/log"
+	"github.com/sasha-s/go-deadlock"
 	"golang.org/x/crypto/acme"
 	"golang.org/x/crypto/acme/autocert"
 	"golang.org/x/sync/errgroup"
@@ -50,13 +49,11 @@ import (
 	"google.golang.org/grpc/peer"
 	"google.golang.org/grpc/reflection"
 	"google.golang.org/grpc/status"
-	"gorm.io/gorm"
 	"tailscale.com/envknob"
 	"tailscale.com/tailcfg"
 	"tailscale.com/types/dnstype"
 	"tailscale.com/types/key"
 	"tailscale.com/util/dnsname"
-	zcache "zgo.at/zcache/v2"
 )
 
 var (
@@ -69,40 +66,41 @@ var (
 	)
 )
 
+var (
+	debugDeadlock        = envknob.Bool("HEADSCALE_DEBUG_DEADLOCK")
+	debugDeadlockTimeout = envknob.RegisterDuration("HEADSCALE_DEBUG_DEADLOCK_TIMEOUT")
+)
+
+func init() {
+	deadlock.Opts.Disable = !debugDeadlock
+	if debugDeadlock {
+		deadlock.Opts.DeadlockTimeout = debugDeadlockTimeout()
+		deadlock.Opts.PrintAllCurrentGoroutines = true
+	}
+}
+
 const (
 	AuthPrefix         = "Bearer "
 	updateInterval     = 5 * time.Second
 	privateKeyFileMode = 0o600
 	headscaleDirPerm   = 0o700
-
-	registerCacheExpiration = time.Minute * 15
-	registerCacheCleanup    = time.Minute * 20
 )
 
 // Headscale represents the base app of the service.
 type Headscale struct {
 	cfg             *types.Config
-	db              *db.HSDatabase
-	ipAlloc         *db.IPAllocator
+	state           *state.State
 	noisePrivateKey *key.MachinePrivate
 	ephemeralGC     *db.EphemeralGarbageCollector
 
-	DERPMap    *tailcfg.DERPMap
 	DERPServer *derpServer.DERPServer
 
-	polManOnce     sync.Once
-	polMan         policy.PolicyManager
+	// Things that generate changes
 	extraRecordMan *dns.ExtraRecordsMan
-	primaryRoutes  *routes.PrimaryRoutes
+	authProvider   AuthProvider
+	mapBatcher     mapper.Batcher
 
-	mapper       *mapper.Mapper
-	nodeNotifier *notifier.Notifier
-
-	registrationCache *zcache.Cache[types.RegistrationID, types.RegisterNode]
-
-	authProvider AuthProvider
-
-	pollNetMapStreamWG sync.WaitGroup
+	clientStreamsOpen sync.WaitGroup
 }
 
 var (
@@ -125,43 +123,37 @@ func NewHeadscale(cfg *types.Config) (*Headscale, error) {
 		return nil, fmt.Errorf("failed to read or create Noise protocol private key: %w", err)
 	}
 
-	registrationCache := zcache.New[types.RegistrationID, types.RegisterNode](
-		registerCacheExpiration,
-		registerCacheCleanup,
-	)
+	s, err := state.NewState(cfg)
+	if err != nil {
+		return nil, fmt.Errorf("init state: %w", err)
+	}
 
 	app := Headscale{
-		cfg:                cfg,
-		noisePrivateKey:    noisePrivateKey,
-		registrationCache:  registrationCache,
-		pollNetMapStreamWG: sync.WaitGroup{},
-		nodeNotifier:       notifier.NewNotifier(cfg),
-		primaryRoutes:      routes.New(),
+		cfg:               cfg,
+		noisePrivateKey:   noisePrivateKey,
+		clientStreamsOpen: sync.WaitGroup{},
+		state:             s,
 	}
 
-	app.db, err = db.NewHeadscaleDatabase(
-		cfg.Database,
-		cfg.BaseDomain,
-		registrationCache,
-	)
-	if err != nil {
-		return nil, fmt.Errorf("new database: %w", err)
-	}
-
-	app.ipAlloc, err = db.NewIPAllocator(app.db, cfg.PrefixV4, cfg.PrefixV6, cfg.IPAllocation)
-	if err != nil {
-		return nil, err
-	}
-
-	app.ephemeralGC = db.NewEphemeralGarbageCollector(func(ni types.NodeID) {
-		if err := app.db.DeleteEphemeralNode(ni); err != nil {
-			log.Err(err).Uint64("node.id", ni.Uint64()).Msgf("failed to delete ephemeral node")
+	// Initialize ephemeral garbage collector
+	ephemeralGC := db.NewEphemeralGarbageCollector(func(ni types.NodeID) {
+		node, ok := app.state.GetNodeByID(ni)
+		if !ok {
+			log.Error().Uint64("node.id", ni.Uint64()).Msg("Ephemeral node deletion failed")
+			log.Debug().Caller().Uint64("node.id", ni.Uint64()).Msg("Ephemeral node deletion failed because node not found in NodeStore")
+			return
 		}
-	})
 
-	if err = app.loadPolicyManager(); err != nil {
-		return nil, fmt.Errorf("loading ACL policy: %w", err)
-	}
+		policyChanged, err := app.state.DeleteNode(node)
+		if err != nil {
+			log.Error().Err(err).Uint64("node.id", ni.Uint64()).Str("node.name", node.Hostname()).Msg("Ephemeral node deletion failed")
+			return
+		}
+
+		app.Change(policyChanged)
+		log.Debug().Caller().Uint64("node.id", ni.Uint64()).Str("node.name", node.Hostname()).Msg("Ephemeral node deleted because garbage collection timeout reached")
+	})
+	app.ephemeralGC = ephemeralGC
 
 	var authProvider AuthProvider
 	authProvider = NewAuthProviderWeb(cfg.ServerURL)
@@ -170,12 +162,9 @@ func NewHeadscale(cfg *types.Config) (*Headscale, error) {
 		defer cancel()
 		oidcProvider, err := NewAuthProviderOIDC(
 			ctx,
+			&app,
 			cfg.ServerURL,
 			&cfg.OIDC,
-			app.db,
-			app.nodeNotifier,
-			app.ipAlloc,
-			app.polMan,
 		)
 		if err != nil {
 			if cfg.OIDC.OnlyStartIfOIDCIsAvailable {
@@ -226,6 +215,14 @@ func NewHeadscale(cfg *types.Config) (*Headscale, error) {
 			)
 		}
 
+		if cfg.DERP.ServerVerifyClients {
+			t := http.DefaultTransport.(*http.Transport) //nolint:forcetypeassert
+			t.RegisterProtocol(
+				derpServer.DerpVerifyScheme,
+				derpServer.NewDERPVerifyTransport(app.handleVerifyRequest),
+			)
+		}
+
 		embeddedDERPServer, err := derpServer.NewDERPServer(
 			cfg.ServerURL,
 			key.NodePrivate(*derpServerKey),
@@ -273,38 +270,41 @@ func (h *Headscale) scheduledTasks(ctx context.Context) {
 			return
 
 		case <-expireTicker.C:
-			var update types.StateUpdate
+			var expiredNodeChanges []change.ChangeSet
 			var changed bool
 
-			if err := h.db.Write(func(tx *gorm.DB) error {
-				lastExpiryCheck, update, changed = db.ExpireExpiredNodes(tx, lastExpiryCheck)
-
-				return nil
-			}); err != nil {
-				log.Error().Err(err).Msg("database error while expiring nodes")
-				continue
-			}
+			lastExpiryCheck, expiredNodeChanges, changed = h.state.ExpireExpiredNodes(lastExpiryCheck)
 
 			if changed {
-				log.Trace().Interface("nodes", update.ChangePatches).Msgf("expiring nodes")
+				log.Trace().Interface("changes", expiredNodeChanges).Msgf("expiring nodes")
 
-				ctx := types.NotifyCtx(context.Background(), "expire-expired", "na")
-				h.nodeNotifier.NotifyAll(ctx, update)
+				// Send the changes directly since they're already in the new format
+				for _, nodeChange := range expiredNodeChanges {
+					h.Change(nodeChange)
+				}
 			}
 
 		case <-derpTickerChan:
 			log.Info().Msg("Fetching DERPMap updates")
-			h.DERPMap = derp.GetDERPMap(h.cfg.DERP)
-			if h.cfg.DERP.ServerEnabled && h.cfg.DERP.AutomaticallyAddEmbeddedDerpRegion {
-				region, _ := h.DERPServer.GenerateRegion()
-				h.DERPMap.Regions[region.RegionID] = &region
-			}
+			derpMap, err := backoff.Retry(ctx, func() (*tailcfg.DERPMap, error) {
+				derpMap, err := derp.GetDERPMap(h.cfg.DERP)
+				if err != nil {
+					return nil, err
+				}
+				if h.cfg.DERP.ServerEnabled && h.cfg.DERP.AutomaticallyAddEmbeddedDerpRegion {
+					region, _ := h.DERPServer.GenerateRegion()
+					derpMap.Regions[region.RegionID] = &region
+				}
 
-			ctx := types.NotifyCtx(context.Background(), "derpmap-update", "na")
-			h.nodeNotifier.NotifyAll(ctx, types.StateUpdate{
-				Type:    types.StateDERPUpdated,
-				DERPMap: h.DERPMap,
-			})
+				return derpMap, nil
+			}, backoff.WithBackOff(backoff.NewExponentialBackOff()))
+			if err != nil {
+				log.Error().Err(err).Msg("failed to build new DERPMap, retrying later")
+				continue
+			}
+			h.state.SetDERPMap(derpMap)
+
+			h.Change(change.DERPSet)
 
 		case records, ok := <-extraRecordsUpdate:
 			if !ok {
@@ -312,19 +312,16 @@ func (h *Headscale) scheduledTasks(ctx context.Context) {
 			}
 			h.cfg.TailcfgDNSConfig.ExtraRecords = records
 
-			ctx := types.NotifyCtx(context.Background(), "dns-extrarecord", "all")
-			// TODO(kradalby): We can probably do better than sending a full update here,
-			// but for now this will ensure that all of the nodes get the new records.
-			h.nodeNotifier.NotifyAll(ctx, types.UpdateFull())
+			h.Change(change.ExtraRecordsSet)
 		}
 	}
 }
 
 func (h *Headscale) grpcAuthenticationInterceptor(ctx context.Context,
-	req interface{},
+	req any,
 	info *grpc.UnaryServerInfo,
 	handler grpc.UnaryHandler,
-) (interface{}, error) {
+) (any, error) {
 	// Check if the request is coming from the on-server client.
 	// This is not secure, but it is to maintain maintainability
 	// with the "legacy" database-based client
@@ -362,7 +359,7 @@ func (h *Headscale) grpcAuthenticationInterceptor(ctx context.Context,
 		)
 	}
 
-	valid, err := h.db.ValidateAPIKey(strings.TrimPrefix(token, AuthPrefix))
+	valid, err := h.state.ValidateAPIKey(strings.TrimPrefix(token, AuthPrefix))
 	if err != nil {
 		return ctx, status.Error(codes.Internal, "failed to validate token")
 	}
@@ -388,42 +385,32 @@ func (h *Headscale) httpAuthenticationMiddleware(next http.Handler) http.Handler
 			Str("client_address", req.RemoteAddr).
 			Msg("HTTP authentication invoked")
 
-		authHeader := req.Header.Get("authorization")
+		authHeader := req.Header.Get("Authorization")
+
+		writeUnauthorized := func(statusCode int) {
+			writer.WriteHeader(statusCode)
+			if _, err := writer.Write([]byte("Unauthorized")); err != nil {
+				log.Error().Err(err).Msg("writing HTTP response failed")
+			}
+		}
 
 		if !strings.HasPrefix(authHeader, AuthPrefix) {
 			log.Error().
 				Caller().
 				Str("client_address", req.RemoteAddr).
 				Msg(`missing "Bearer " prefix in "Authorization" header`)
-			writer.WriteHeader(http.StatusUnauthorized)
-			_, err := writer.Write([]byte("Unauthorized"))
-			if err != nil {
-				log.Error().
-					Caller().
-					Err(err).
-					Msg("Failed to write response")
-			}
-
+			writeUnauthorized(http.StatusUnauthorized)
 			return
 		}
 
-		valid, err := h.db.ValidateAPIKey(strings.TrimPrefix(authHeader, AuthPrefix))
+		valid, err := h.state.ValidateAPIKey(strings.TrimPrefix(authHeader, AuthPrefix))
 		if err != nil {
-			log.Error().
+			log.Info().
 				Caller().
 				Err(err).
 				Str("client_address", req.RemoteAddr).
 				Msg("failed to validate token")
-
-			writer.WriteHeader(http.StatusInternalServerError)
-			_, err := writer.Write([]byte("Unauthorized"))
-			if err != nil {
-				log.Error().
-					Caller().
-					Err(err).
-					Msg("Failed to write response")
-			}
-
+			writeUnauthorized(http.StatusUnauthorized)
 			return
 		}
 
@@ -431,16 +418,7 @@ func (h *Headscale) httpAuthenticationMiddleware(next http.Handler) http.Handler
 			log.Info().
 				Str("client_address", req.RemoteAddr).
 				Msg("invalid token")
-
-			writer.WriteHeader(http.StatusUnauthorized)
-			_, err := writer.Write([]byte("Unauthorized"))
-			if err != nil {
-				log.Error().
-					Caller().
-					Err(err).
-					Msg("Failed to write response")
-			}
-
+			writeUnauthorized(http.StatusUnauthorized)
 			return
 		}
 
@@ -466,7 +444,9 @@ func (h *Headscale) createRouter(grpcMux *grpcRuntime.ServeMux) *mux.Router {
 	router.HandleFunc(ts2021UpgradePath, h.NoiseUpgradeHandler).
 		Methods(http.MethodPost, http.MethodGet)
 
+	router.HandleFunc("/robots.txt", h.RobotsHandler).Methods(http.MethodGet)
 	router.HandleFunc("/health", h.HealthHandler).Methods(http.MethodGet)
+	router.HandleFunc("/version", h.VersionHandler).Methods(http.MethodGet)
 	router.HandleFunc("/key", h.KeyHandler).Methods(http.MethodGet)
 	router.HandleFunc("/register/{registration_id}", h.authProvider.RegisterHandler).
 		Methods(http.MethodGet)
@@ -490,7 +470,7 @@ func (h *Headscale) createRouter(grpcMux *grpcRuntime.ServeMux) *mux.Router {
 		router.HandleFunc("/derp", h.DERPServer.DERPHandler)
 		router.HandleFunc("/derp/probe", derpServer.DERPProbeHandler)
 		router.HandleFunc("/derp/latency-check", derpServer.DERPProbeHandler)
-		router.HandleFunc("/bootstrap-dns", derpServer.DERPBootstrapDNSHandler(h.DERPMap))
+		router.HandleFunc("/bootstrap-dns", derpServer.DERPBootstrapDNSHandler(h.state.DERPMap()))
 	}
 
 	apiRouter := router.PathPrefix("/api").Subrouter()
@@ -502,65 +482,14 @@ func (h *Headscale) createRouter(grpcMux *grpcRuntime.ServeMux) *mux.Router {
 	return router
 }
 
-// TODO(kradalby): Do a variant of this, and polman which only updates the node that has changed.
-// Maybe we should attempt a new in memory state and not go via the DB?
-// Maybe this should be implemented as an event bus?
-// A bool is returned indicating if a full update was sent to all nodes
-func usersChangedHook(db *db.HSDatabase, polMan policy.PolicyManager, notif *notifier.Notifier) error {
-	users, err := db.ListUsers()
-	if err != nil {
-		return err
-	}
-
-	changed, err := polMan.SetUsers(users)
-	if err != nil {
-		return err
-	}
-
-	if changed {
-		ctx := types.NotifyCtx(context.Background(), "acl-users-change", "all")
-		notif.NotifyAll(ctx, types.UpdateFull())
-	}
-
-	return nil
-}
-
-// TODO(kradalby): Do a variant of this, and polman which only updates the node that has changed.
-// Maybe we should attempt a new in memory state and not go via the DB?
-// Maybe this should be implemented as an event bus?
-// A bool is returned indicating if a full update was sent to all nodes
-func nodesChangedHook(
-	db *db.HSDatabase,
-	polMan policy.PolicyManager,
-	notif *notifier.Notifier,
-) (bool, error) {
-	nodes, err := db.ListNodes()
-	if err != nil {
-		return false, err
-	}
-
-	filterChanged, err := polMan.SetNodes(nodes)
-	if err != nil {
-		return false, err
-	}
-
-	if filterChanged {
-		ctx := types.NotifyCtx(context.Background(), "acl-nodes-change", "all")
-		notif.NotifyAll(ctx, types.UpdateFull())
-
-		return true, nil
-	}
-
-	return false, nil
-}
-
 // Serve launches the HTTP and gRPC server service Headscale and the API.
 func (h *Headscale) Serve() error {
+	var err error
 	capver.CanOldCodeBeCleanedUp()
 
 	if profilingEnabled {
 		if profilingPath != "" {
-			err := os.MkdirAll(profilingPath, os.ModePerm)
+			err = os.MkdirAll(profilingPath, os.ModePerm)
 			if err != nil {
 				log.Fatal().Err(err).Msg("failed to create profiling directory")
 			}
@@ -575,14 +504,15 @@ func (h *Headscale) Serve() error {
 		spew.Dump(h.cfg)
 	}
 
-	log.Info().Str("version", types.Version).Str("commit", types.GitCommitHash).Msg("Starting Headscale")
+	versionInfo := types.GetVersionInfo()
+	log.Info().Str("version", versionInfo.Version).Str("commit", versionInfo.Commit).Msg("Starting Headscale")
 	log.Info().
 		Str("minimum_version", capver.TailscaleVersion(capver.MinSupportedCapabilityVersion)).
 		Msg("Clients with a lower minimum version will be rejected")
 
-	// Fetch an initial DERP Map before we start serving
-	h.DERPMap = derp.GetDERPMap(h.cfg.DERP)
-	h.mapper = mapper.NewMapper(h.db, h.cfg, h.DERPMap, h.nodeNotifier, h.polMan, h.primaryRoutes)
+	h.mapBatcher = mapper.NewBatcherAndMapper(h.cfg, h.state)
+	h.mapBatcher.Start()
+	defer h.mapBatcher.Close()
 
 	if h.cfg.DERP.ServerEnabled {
 		// When embedded DERP is enabled we always need a STUN server
@@ -590,33 +520,33 @@ func (h *Headscale) Serve() error {
 			return errSTUNAddressNotSet
 		}
 
-		region, err := h.DERPServer.GenerateRegion()
-		if err != nil {
-			return fmt.Errorf("generating DERP region for embedded server: %w", err)
-		}
-
-		if h.cfg.DERP.AutomaticallyAddEmbeddedDerpRegion {
-			h.DERPMap.Regions[region.RegionID] = &region
-		}
-
 		go h.DERPServer.ServeSTUN()
 	}
 
-	if len(h.DERPMap.Regions) == 0 {
+	derpMap, err := derp.GetDERPMap(h.cfg.DERP)
+	if err != nil {
+		return fmt.Errorf("failed to get DERPMap: %w", err)
+	}
+
+	if h.cfg.DERP.ServerEnabled && h.cfg.DERP.AutomaticallyAddEmbeddedDerpRegion {
+		region, _ := h.DERPServer.GenerateRegion()
+		derpMap.Regions[region.RegionID] = &region
+	}
+
+	if len(derpMap.Regions) == 0 {
 		return errEmptyInitialDERPMap
 	}
 
+	h.state.SetDERPMap(derpMap)
+
 	// Start ephemeral node garbage collector and schedule all nodes
 	// that are already in the database and ephemeral. If they are still
 	// around between restarts, they will reconnect and the GC will
 	// be cancelled.
 	go h.ephemeralGC.Start()
-	ephmNodes, err := h.db.ListEphemeralNodes()
-	if err != nil {
-		return fmt.Errorf("failed to list ephemeral nodes: %w", err)
-	}
-	for _, node := range ephmNodes {
-		h.ephemeralGC.Schedule(node.ID, h.cfg.EphemeralNodeInactivityTimeout)
+	ephmNodes := h.state.ListEphemeralNodes()
+	for _, node := range ephmNodes.All() {
+		h.ephemeralGC.Schedule(node.ID(), h.cfg.EphemeralNodeInactivityTimeout)
 	}
 
 	if h.cfg.DNSConfig.ExtraRecordsPath != "" {
@@ -734,12 +664,10 @@ func (h *Headscale) Serve() error {
 		log.Info().Msgf("Enabling remote gRPC at %s", h.cfg.GRPCAddr)
 
 		grpcOptions := []grpc.ServerOption{
-			grpc.UnaryInterceptor(
-				grpcMiddleware.ChainUnaryServer(
-					h.grpcAuthenticationInterceptor,
-					// Uncomment to debug grpc communication.
-					// zerolog.NewUnaryServerInterceptor(),
-				),
+			grpc.ChainUnaryInterceptor(
+				h.grpcAuthenticationInterceptor,
+				// Uncomment to debug grpc communication.
+				// zerolog.NewUnaryServerInterceptor(),
 			),
 		}
 
@@ -842,38 +770,20 @@ func (h *Headscale) Serve() error {
 			case syscall.SIGHUP:
 				log.Info().
 					Str("signal", sig.String()).
-					Msg("Received SIGHUP, reloading ACL and Config")
+					Msg("Received SIGHUP, reloading ACL policy")
 
 				if h.cfg.Policy.IsEmpty() {
 					continue
 				}
 
-				if err := h.loadPolicyManager(); err != nil {
-					log.Error().Err(err).Msg("failed to reload Policy")
-				}
-
-				pol, err := h.policyBytes()
+				changes, err := h.state.ReloadPolicy()
 				if err != nil {
-					log.Error().Err(err).Msg("failed to get policy blob")
+					log.Error().Err(err).Msgf("reloading policy")
+					continue
 				}
 
-				changed, err := h.polMan.SetPolicy(pol)
-				if err != nil {
-					log.Error().Err(err).Msg("failed to set new policy")
-				}
+				h.Change(changes...)
 
-				if changed {
-					log.Info().
-						Msg("ACL policy successfully reloaded, notifying nodes of change")
-
-					err = h.autoApproveNodes()
-					if err != nil {
-						log.Error().Err(err).Msg("failed to approve routes after new policy")
-					}
-
-					ctx := types.NotifyCtx(context.Background(), "acl-sighup", "na")
-					h.nodeNotifier.NotifyAll(ctx, types.UpdateFull())
-				}
 			default:
 				info := func(msg string) { log.Info().Msg(msg) }
 				log.Info().
@@ -897,11 +807,11 @@ func (h *Headscale) Serve() error {
 					log.Error().Err(err).Msg("failed to shutdown http")
 				}
 
-				info("closing node notifier")
-				h.nodeNotifier.Close()
+				info("closing batcher")
+				h.mapBatcher.Close()
 
 				info("waiting for netmap stream to close")
-				h.pollNetMapStreamWG.Wait()
+				h.clientStreamsOpen.Wait()
 
 				info("shutting down grpc server (socket)")
 				grpcSocket.GracefulStop()
@@ -927,11 +837,11 @@ func (h *Headscale) Serve() error {
 				info("closing socket listener")
 				socketListener.Close()
 
-				// Close db connections
-				info("closing database connection")
-				err = h.db.Close()
+				// Close state connections
+				info("closing state and database")
+				err = h.state.Close()
 				if err != nil {
-					log.Error().Err(err).Msg("failed to close db")
+					log.Error().Err(err).Msg("failed to close state")
 				}
 
 				log.Info().
@@ -1081,123 +991,9 @@ func readOrCreatePrivateKey(path string) (*key.MachinePrivate, error) {
 	return &machineKey, nil
 }
 
-// policyBytes returns the appropriate policy for the
-// current configuration as a []byte array.
-func (h *Headscale) policyBytes() ([]byte, error) {
-	switch h.cfg.Policy.Mode {
-	case types.PolicyModeFile:
-		path := h.cfg.Policy.Path
-
-		// It is fine to start headscale without a policy file.
-		if len(path) == 0 {
-			return nil, nil
-		}
-
-		absPath := util.AbsolutePathFromConfigPath(path)
-		policyFile, err := os.Open(absPath)
-		if err != nil {
-			return nil, err
-		}
-		defer policyFile.Close()
-
-		return io.ReadAll(policyFile)
-
-	case types.PolicyModeDB:
-		p, err := h.db.GetPolicy()
-		if err != nil {
-			if errors.Is(err, types.ErrPolicyNotFound) {
-				return nil, nil
-			}
-
-			return nil, err
-		}
-
-		if p.Data == "" {
-			return nil, nil
-		}
-
-		return []byte(p.Data), err
-	}
-
-	return nil, fmt.Errorf("unsupported policy mode: %s", h.cfg.Policy.Mode)
-}
-
-func (h *Headscale) loadPolicyManager() error {
-	var errOut error
-	h.polManOnce.Do(func() {
-		// Validate and reject configuration that would error when applied
-		// when creating a map response. This requires nodes, so there is still
-		// a scenario where they might be allowed if the server has no nodes
-		// yet, but it should help for the general case and for hot reloading
-		// configurations.
-		// Note that this check is only done for file-based policies in this function
-		// as the database-based policies are checked in the gRPC API where it is not
-		// allowed to be written to the database.
-		nodes, err := h.db.ListNodes()
-		if err != nil {
-			errOut = fmt.Errorf("loading nodes from database to validate policy: %w", err)
-			return
-		}
-		users, err := h.db.ListUsers()
-		if err != nil {
-			errOut = fmt.Errorf("loading users from database to validate policy: %w", err)
-			return
-		}
-
-		pol, err := h.policyBytes()
-		if err != nil {
-			errOut = fmt.Errorf("loading policy bytes: %w", err)
-			return
-		}
-
-		h.polMan, err = policy.NewPolicyManager(pol, users, nodes)
-		if err != nil {
-			errOut = fmt.Errorf("creating policy manager: %w", err)
-			return
-		}
-		log.Info().Msgf("Using policy manager version: %d", h.polMan.Version())
-
-		if len(nodes) > 0 {
-			_, err = h.polMan.SSHPolicy(nodes[0])
-			if err != nil {
-				errOut = fmt.Errorf("verifying SSH rules: %w", err)
-				return
-			}
-		}
-	})
-
-	return errOut
-}
-
-// autoApproveNodes mass approves routes on all nodes. It is _only_ intended for
-// use when the policy is replaced. It is not sending or reporting any changes
-// or updates as we send full updates after replacing the policy.
-// TODO(kradalby): This is kind of messy, maybe this is another +1
-// for an event bus. See example comments here.
-func (h *Headscale) autoApproveNodes() error {
-	err := h.db.Write(func(tx *gorm.DB) error {
-		nodes, err := db.ListNodes(tx)
-		if err != nil {
-			return err
-		}
-
-		for _, node := range nodes {
-			changed := policy.AutoApproveRoutes(h.polMan, node)
-			if changed {
-				err = tx.Save(node).Error
-				if err != nil {
-					return err
-				}
-
-				h.primaryRoutes.SetRoutes(node.ID, node.SubnetRoutes()...)
-			}
-		}
-
-		return nil
-	})
-	if err != nil {
-		return fmt.Errorf("auto approving routes for nodes: %w", err)
-	}
-
-	return nil
+// Change is used to send changes to nodes.
+// All change should be enqueued here and empty will be automatically
+// ignored.
+func (h *Headscale) Change(cs ...change.ChangeSet) {
+	h.mapBatcher.AddWork(cs...)
 }

hscontrol/auth.go

@@ -1,6 +1,7 @@
 package hscontrol
 
 import (
+	"cmp"
 	"context"
 	"errors"
 	"fmt"
@@ -9,10 +10,10 @@ import (
 	"strings"
 	"time"
 
-	"github.com/juanfont/headscale/hscontrol/db"
-	"github.com/juanfont/headscale/hscontrol/policy"
 	"github.com/juanfont/headscale/hscontrol/types"
+	"github.com/juanfont/headscale/hscontrol/types/change"
 	"github.com/juanfont/headscale/hscontrol/util"
+	"github.com/rs/zerolog/log"
 	"gorm.io/gorm"
 	"tailscale.com/tailcfg"
 	"tailscale.com/types/key"
@@ -26,37 +27,98 @@ type AuthProvider interface {
 
 func (h *Headscale) handleRegister(
 	ctx context.Context,
-	regReq tailcfg.RegisterRequest,
+	req tailcfg.RegisterRequest,
 	machineKey key.MachinePublic,
 ) (*tailcfg.RegisterResponse, error) {
-	node, err := h.db.GetNodeByNodeKey(regReq.NodeKey)
-	if err != nil && !errors.Is(err, gorm.ErrRecordNotFound) {
-		return nil, fmt.Errorf("looking up node in database: %w", err)
-	}
+	// Check for logout/expiry FIRST, before checking auth key.
+	// Tailscale clients may send logout requests with BOTH a past expiry AND an auth key.
+	// A past expiry takes precedence - it's a logout regardless of other fields.
+	if !req.Expiry.IsZero() && req.Expiry.Before(time.Now()) {
+		log.Debug().
+			Str("node.key", req.NodeKey.ShortString()).
+			Time("expiry", req.Expiry).
+			Bool("has_auth", req.Auth != nil).
+			Msg("Detected logout attempt with past expiry")
 
-	if node != nil {
-		resp, err := h.handleExistingNode(node, regReq, machineKey)
-		if err != nil {
-			return nil, fmt.Errorf("handling existing node: %w", err)
+		// This is a logout attempt (expiry in the past)
+		if node, ok := h.state.GetNodeByNodeKey(req.NodeKey); ok {
+			log.Debug().
+				Uint64("node.id", node.ID().Uint64()).
+				Str("node.name", node.Hostname()).
+				Bool("is_ephemeral", node.IsEphemeral()).
+				Bool("has_authkey", node.AuthKey().Valid()).
+				Msg("Found existing node for logout, calling handleLogout")
+
+			resp, err := h.handleLogout(node, req, machineKey)
+			if err != nil {
+				return nil, fmt.Errorf("handling logout: %w", err)
+			}
+			if resp != nil {
+				return resp, nil
+			}
+		} else {
+			log.Warn().
+				Str("node.key", req.NodeKey.ShortString()).
+				Msg("Logout attempt but node not found in NodeStore")
 		}
-
-		return resp, nil
 	}
 
-	if regReq.Followup != "" {
-		return h.waitForFollowup(ctx, regReq)
+	// If the register request does not contain a Auth struct, it means we are logging
+	// out an existing node (legacy logout path for clients that send Auth=nil).
+	if req.Auth == nil {
+		// If the register request present a NodeKey that is currently in use, we will
+		// check if the node needs to be sent to re-auth, or if the node is logging out.
+		// We do not look up nodes by [key.MachinePublic] as it might belong to multiple
+		// nodes, separated by users and this path is handling expiring/logout paths.
+		if node, ok := h.state.GetNodeByNodeKey(req.NodeKey); ok {
+			resp, err := h.handleLogout(node, req, machineKey)
+			if err != nil {
+				return nil, fmt.Errorf("handling existing node: %w", err)
+			}
+
+			// If resp is not nil, we have a response to return to the node.
+			// If resp is nil, we should proceed and see if the node is trying to re-auth.
+			if resp != nil {
+				return resp, nil
+			}
+		} else {
+			// If the register request is not attempting to register a node, and
+			// we cannot match it with an existing node, we consider that unexpected
+			// as only register nodes should attempt to log out.
+			log.Debug().
+				Str("node.key", req.NodeKey.ShortString()).
+				Str("machine.key", machineKey.ShortString()).
+				Bool("unexpected", true).
+				Msg("received register request with no auth, and no existing node")
+		}
 	}
 
-	if regReq.Auth != nil && regReq.Auth.AuthKey != "" {
-		resp, err := h.handleRegisterWithAuthKey(regReq, machineKey)
+	// If the [tailcfg.RegisterRequest] has a Followup URL, it means that the
+	// node has already started the registration process and we should wait for
+	// it to finish the original registration.
+	if req.Followup != "" {
+		return h.waitForFollowup(ctx, req, machineKey)
+	}
+
+	// Pre authenticated keys are handled slightly different than interactive
+	// logins as they can be done fully sync and we can respond to the node with
+	// the result as it is waiting.
+	if isAuthKey(req) {
+		resp, err := h.handleRegisterWithAuthKey(req, machineKey)
 		if err != nil {
+			// Preserve HTTPError types so they can be handled properly by the HTTP layer
+			var httpErr HTTPError
+			if errors.As(err, &httpErr) {
+				return nil, httpErr
+			}
+
 			return nil, fmt.Errorf("handling register with auth key: %w", err)
 		}
 
 		return resp, nil
 	}
 
-	resp, err := h.handleRegisterInteractive(regReq, machineKey)
+	resp, err := h.handleRegisterInteractive(req, machineKey)
 	if err != nil {
 		return nil, fmt.Errorf("handling register interactive: %w", err)
 	}
@@ -64,57 +126,111 @@ func (h *Headscale) handleRegister(
 	return resp, nil
 }
 
-func (h *Headscale) handleExistingNode(
-	node *types.Node,
-	regReq tailcfg.RegisterRequest,
+// handleLogout checks if the [tailcfg.RegisterRequest] is a
+// logout attempt from a node. If the node is not attempting to
+func (h *Headscale) handleLogout(
+	node types.NodeView,
+	req tailcfg.RegisterRequest,
 	machineKey key.MachinePublic,
 ) (*tailcfg.RegisterResponse, error) {
-	if node.MachineKey != machineKey {
+	// Fail closed if it looks like this is an attempt to modify a node where
+	// the node key and the machine key the noise session was started with does
+	// not align.
+	if node.MachineKey() != machineKey {
 		return nil, NewHTTPError(http.StatusUnauthorized, "node exist with different machine key", nil)
 	}
 
-	expired := node.IsExpired()
-	if !expired && !regReq.Expiry.IsZero() {
-		requestExpiry := regReq.Expiry
+	// Note: We do NOT return early if req.Auth is set, because Tailscale clients
+	// may send logout requests with BOTH a past expiry AND an auth key.
+	// A past expiry indicates logout, regardless of whether Auth is present.
+	// The expiry check below will handle the logout logic.
 
-		// The client is trying to extend their key, this is not allowed.
-		if requestExpiry.After(time.Now()) {
-			return nil, NewHTTPError(http.StatusBadRequest, "extending key is not allowed", nil)
-		}
-
-		// If the request expiry is in the past, we consider it a logout.
-		if requestExpiry.Before(time.Now()) {
-			if node.IsEphemeral() {
-				err := h.db.DeleteNode(node)
-				if err != nil {
-					return nil, fmt.Errorf("deleting ephemeral node: %w", err)
-				}
-
-				ctx := types.NotifyCtx(context.Background(), "logout-ephemeral", "na")
-				h.nodeNotifier.NotifyAll(ctx, types.UpdatePeerRemoved(node.ID))
-			}
-
-			expired = true
-		}
-
-		err := h.db.NodeSetExpiry(node.ID, requestExpiry)
-		if err != nil {
-			return nil, fmt.Errorf("setting node expiry: %w", err)
-		}
-
-		ctx := types.NotifyCtx(context.Background(), "logout-expiry", "na")
-		h.nodeNotifier.NotifyWithIgnore(ctx, types.UpdateExpire(node.ID, requestExpiry), node.ID)
+	// If the node is expired and this is not a re-authentication attempt,
+	// force the client to re-authenticate.
+	// TODO(kradalby): I wonder if this is a path we ever hit?
+	if node.IsExpired() {
+		log.Trace().Str("node.name", node.Hostname()).
+			Uint64("node.id", node.ID().Uint64()).
+			Interface("reg.req", req).
+			Bool("unexpected", true).
+			Msg("Node key expired, forcing re-authentication")
+		return &tailcfg.RegisterResponse{
+			NodeKeyExpired:    true,
+			MachineAuthorized: false,
+			AuthURL:           "", // Client will need to re-authenticate
+		}, nil
 	}
 
-	return nodeToRegisterResponse(node), nil
+	// If we get here, the node is not currently expired, and not trying to
+	// do an auth.
+	// The node is likely logging out, but before we run that logic, we will validate
+	// that the node is not attempting to tamper/extend their expiry.
+	// If it is not, we will expire the node or in the case of an ephemeral node, delete it.
+
+	// The client is trying to extend their key, this is not allowed.
+	if req.Expiry.After(time.Now()) {
+		return nil, NewHTTPError(http.StatusBadRequest, "extending key is not allowed", nil)
+	}
+
+	// If the request expiry is in the past, we consider it a logout.
+	if req.Expiry.Before(time.Now()) {
+		log.Debug().
+			Uint64("node.id", node.ID().Uint64()).
+			Str("node.name", node.Hostname()).
+			Bool("is_ephemeral", node.IsEphemeral()).
+			Bool("has_authkey", node.AuthKey().Valid()).
+			Time("req.expiry", req.Expiry).
+			Msg("Processing logout request with past expiry")
+
+		if node.IsEphemeral() {
+			log.Info().
+				Uint64("node.id", node.ID().Uint64()).
+				Str("node.name", node.Hostname()).
+				Msg("Deleting ephemeral node during logout")
+
+			c, err := h.state.DeleteNode(node)
+			if err != nil {
+				return nil, fmt.Errorf("deleting ephemeral node: %w", err)
+			}
+
+			h.Change(c)
+
+			return &tailcfg.RegisterResponse{
+				NodeKeyExpired:    true,
+				MachineAuthorized: false,
+			}, nil
+		}
+
+		log.Debug().
+			Uint64("node.id", node.ID().Uint64()).
+			Str("node.name", node.Hostname()).
+			Msg("Node is not ephemeral, setting expiry instead of deleting")
+	}
+
+	// Update the internal state with the nodes new expiry, meaning it is
+	// logged out.
+	updatedNode, c, err := h.state.SetNodeExpiry(node.ID(), req.Expiry)
+	if err != nil {
+		return nil, fmt.Errorf("setting node expiry: %w", err)
+	}
+
+	h.Change(c)
+
+	return nodeToRegisterResponse(updatedNode), nil
 }
 
-func nodeToRegisterResponse(node *types.Node) *tailcfg.RegisterResponse {
+// isAuthKey reports if the register request is a registration request
+// using an pre auth key.
+func isAuthKey(req tailcfg.RegisterRequest) bool {
+	return req.Auth != nil && req.Auth.AuthKey != ""
+}
+
+func nodeToRegisterResponse(node types.NodeView) *tailcfg.RegisterResponse {
 	return &tailcfg.RegisterResponse{
 		// TODO(kradalby): Only send for user-owned nodes
 		// and not tagged nodes when tags is working.
-		User:           *node.User.TailscaleUser(),
-		Login:          *node.User.TailscaleLogin(),
+		User:           node.UserView().TailscaleUser(),
+		Login:          node.UserView().TailscaleLogin(),
 		NodeKeyExpired: node.IsExpired(),
 
 		// Headscale does not implement the concept of machine authorization
@@ -126,9 +242,10 @@ func nodeToRegisterResponse(node *types.Node) *tailcfg.RegisterResponse {
 
 func (h *Headscale) waitForFollowup(
 	ctx context.Context,
-	regReq tailcfg.RegisterRequest,
+	req tailcfg.RegisterRequest,
+	machineKey key.MachinePublic,
 ) (*tailcfg.RegisterResponse, error) {
-	fu, err := url.Parse(regReq.Followup)
+	fu, err := url.Parse(req.Followup)
 	if err != nil {
 		return nil, NewHTTPError(http.StatusUnauthorized, "invalid followup URL", err)
 	}
@@ -138,111 +255,92 @@ func (h *Headscale) waitForFollowup(
 		return nil, NewHTTPError(http.StatusUnauthorized, "invalid registration ID", err)
 	}
 
-	if reg, ok := h.registrationCache.Get(followupReg); ok {
+	if reg, ok := h.state.GetRegistrationCacheEntry(followupReg); ok {
 		select {
 		case <-ctx.Done():
 			return nil, NewHTTPError(http.StatusUnauthorized, "registration timed out", err)
 		case node := <-reg.Registered:
 			if node == nil {
-				return nil, NewHTTPError(http.StatusUnauthorized, "node not found", nil)
+				// registration is expired in the cache, instruct the client to try a new registration
+				return h.reqToNewRegisterResponse(req, machineKey)
 			}
-			return nodeToRegisterResponse(node), nil
+			return nodeToRegisterResponse(node.View()), nil
 		}
 	}
 
-	return nil, NewHTTPError(http.StatusNotFound, "followup registration not found", nil)
+	// if the follow-up registration isn't found anymore, instruct the client to try a new registration
+	return h.reqToNewRegisterResponse(req, machineKey)
 }
 
-// canUsePreAuthKey checks if a pre auth key can be used.
-func canUsePreAuthKey(pak *types.PreAuthKey) error {
-	if pak == nil {
-		return NewHTTPError(http.StatusUnauthorized, "invalid authkey", nil)
-	}
-	if pak.Expiration != nil && pak.Expiration.Before(time.Now()) {
-		return NewHTTPError(http.StatusUnauthorized, "authkey expired", nil)
+// reqToNewRegisterResponse refreshes the registration flow by creating a new
+// registration ID and returning the corresponding AuthURL so the client can
+// restart the authentication process.
+func (h *Headscale) reqToNewRegisterResponse(
+	req tailcfg.RegisterRequest,
+	machineKey key.MachinePublic,
+) (*tailcfg.RegisterResponse, error) {
+	newRegID, err := types.NewRegistrationID()
+	if err != nil {
+		return nil, NewHTTPError(http.StatusInternalServerError, "failed to generate registration ID", err)
 	}
 
-	// we don't need to check if has been used before
-	if pak.Reusable {
-		return nil
+	// Ensure we have a valid hostname
+	hostname := util.EnsureHostname(
+		req.Hostinfo,
+		machineKey.String(),
+		req.NodeKey.String(),
+	)
+
+	// Ensure we have valid hostinfo
+	hostinfo := cmp.Or(req.Hostinfo, &tailcfg.Hostinfo{})
+	hostinfo.Hostname = hostname
+
+	nodeToRegister := types.NewRegisterNode(
+		types.Node{
+			Hostname:   hostname,
+			MachineKey: machineKey,
+			NodeKey:    req.NodeKey,
+			Hostinfo:   hostinfo,
+			LastSeen:   ptr.To(time.Now()),
+		},
+	)
+
+	if !req.Expiry.IsZero() {
+		nodeToRegister.Node.Expiry = &req.Expiry
 	}
 
-	if pak.Used {
-		return NewHTTPError(http.StatusUnauthorized, "authkey already used", nil)
-	}
+	log.Info().Msgf("New followup node registration using key: %s", newRegID)
+	h.state.SetRegistrationCacheEntry(newRegID, nodeToRegister)
 
-	return nil
+	return &tailcfg.RegisterResponse{
+		AuthURL: h.authProvider.AuthURL(newRegID),
+	}, nil
 }
 
 func (h *Headscale) handleRegisterWithAuthKey(
-	regReq tailcfg.RegisterRequest,
+	req tailcfg.RegisterRequest,
 	machineKey key.MachinePublic,
 ) (*tailcfg.RegisterResponse, error) {
-	pak, err := h.db.GetPreAuthKey(regReq.Auth.AuthKey)
+	node, changed, err := h.state.HandleNodeFromPreAuthKey(
+		req,
+		machineKey,
+	)
 	if err != nil {
 		if errors.Is(err, gorm.ErrRecordNotFound) {
 			return nil, NewHTTPError(http.StatusUnauthorized, "invalid pre auth key", nil)
 		}
-		return nil, err
-	}
-
-	err = canUsePreAuthKey(pak)
-	if err != nil {
-		return nil, err
-	}
-
-	nodeToRegister := types.Node{
-		Hostname:       regReq.Hostinfo.Hostname,
-		UserID:         pak.User.ID,
-		User:           pak.User,
-		MachineKey:     machineKey,
-		NodeKey:        regReq.NodeKey,
-		Hostinfo:       regReq.Hostinfo,
-		LastSeen:       ptr.To(time.Now()),
-		RegisterMethod: util.RegisterMethodAuthKey,
-
-		// TODO(kradalby): This should not be set on the node,
-		// they should be looked up through the key, which is
-		// attached to the node.
-		ForcedTags: pak.Proto().GetAclTags(),
-		AuthKey:    pak,
-		AuthKeyID:  &pak.ID,
-	}
-
-	if !regReq.Expiry.IsZero() {
-		nodeToRegister.Expiry = &regReq.Expiry
-	}
-
-	ipv4, ipv6, err := h.ipAlloc.Next()
-	if err != nil {
-		return nil, fmt.Errorf("allocating IPs: %w", err)
-	}
-
-	node, err := db.Write(h.db.DB, func(tx *gorm.DB) (*types.Node, error) {
-		node, err := db.RegisterNode(tx,
-			nodeToRegister,
-			ipv4, ipv6,
-		)
-		if err != nil {
-			return nil, fmt.Errorf("registering node: %w", err)
+		var perr types.PAKError
+		if errors.As(err, &perr) {
+			return nil, NewHTTPError(http.StatusUnauthorized, perr.Error(), nil)
 		}
 
-		if !pak.Reusable {
-			err = db.UsePreAuthKey(tx, pak)
-			if err != nil {
-				return nil, fmt.Errorf("using pre auth key: %w", err)
-			}
-		}
-
-		return node, nil
-	})
-	if err != nil {
 		return nil, err
 	}
 
-	updateSent, err := nodesChangedHook(h.db, h.polMan, h.nodeNotifier)
-	if err != nil {
-		return nil, fmt.Errorf("nodes changed hook: %w", err)
+	// If node is not valid, it means an ephemeral node was deleted during logout
+	if !node.Valid() {
+		h.Change(changed)
+		return nil, nil
 	}
 
 	// This is a bit of a back and forth, but we have a bit of a chicken and egg
@@ -256,26 +354,46 @@ func (h *Headscale) handleRegisterWithAuthKey(
 	// ensure we send an update.
 	// This works, but might be another good candidate for doing some sort of
 	// eventbus.
-	routesChanged := policy.AutoApproveRoutes(h.polMan, node)
-	if err := h.db.DB.Save(node).Error; err != nil {
+	// TODO(kradalby): This needs to be ran as part of the batcher maybe?
+	// now since we dont update the node/pol here anymore
+	routeChange := h.state.AutoApproveRoutes(node)
+
+	if _, _, err := h.state.SaveNode(node); err != nil {
 		return nil, fmt.Errorf("saving auto approved routes to node: %w", err)
 	}
 
-	if !updateSent || routesChanged {
-		ctx := types.NotifyCtx(context.Background(), "node updated", node.Hostname)
-		h.nodeNotifier.NotifyAll(ctx, types.UpdatePeerChanged(node.ID))
+	if routeChange && changed.Empty() {
+		changed = change.NodeAdded(node.ID())
 	}
+	h.Change(changed)
 
-	return &tailcfg.RegisterResponse{
+	// TODO(kradalby): I think this is covered above, but we need to validate that.
+	// // If policy changed due to node registration, send a separate policy change
+	// if policyChanged {
+	// 	policyChange := change.PolicyChange()
+	// 	h.Change(policyChange)
+	// }
+
+	resp := &tailcfg.RegisterResponse{
 		MachineAuthorized: true,
 		NodeKeyExpired:    node.IsExpired(),
-		User:              *pak.User.TailscaleUser(),
-		Login:             *pak.User.TailscaleLogin(),
-	}, nil
+		User:              node.UserView().TailscaleUser(),
+		Login:             node.UserView().TailscaleLogin(),
+	}
+
+	log.Trace().
+		Caller().
+		Interface("reg.resp", resp).
+		Interface("reg.req", req).
+		Str("node.name", node.Hostname()).
+		Uint64("node.id", node.ID().Uint64()).
+		Msg("RegisterResponse")
+
+	return resp, nil
 }
 
 func (h *Headscale) handleRegisterInteractive(
-	regReq tailcfg.RegisterRequest,
+	req tailcfg.RegisterRequest,
 	machineKey key.MachinePublic,
 ) (*tailcfg.RegisterResponse, error) {
 	registrationId, err := types.NewRegistrationID()
@@ -283,26 +401,51 @@ func (h *Headscale) handleRegisterInteractive(
 		return nil, fmt.Errorf("generating registration ID: %w", err)
 	}
 
-	nodeToRegister := types.RegisterNode{
-		Node: types.Node{
-			Hostname:   regReq.Hostinfo.Hostname,
+	// Ensure we have a valid hostname
+	hostname := util.EnsureHostname(
+		req.Hostinfo,
+		machineKey.String(),
+		req.NodeKey.String(),
+	)
+
+	// Ensure we have valid hostinfo
+	hostinfo := cmp.Or(req.Hostinfo, &tailcfg.Hostinfo{})
+	if req.Hostinfo == nil {
+		log.Warn().
+			Str("machine.key", machineKey.ShortString()).
+			Str("node.key", req.NodeKey.ShortString()).
+			Str("generated.hostname", hostname).
+			Msg("Received registration request with nil hostinfo, generated default hostname")
+	} else if req.Hostinfo.Hostname == "" {
+		log.Warn().
+			Str("machine.key", machineKey.ShortString()).
+			Str("node.key", req.NodeKey.ShortString()).
+			Str("generated.hostname", hostname).
+			Msg("Received registration request with empty hostname, generated default")
+	}
+	hostinfo.Hostname = hostname
+
+	nodeToRegister := types.NewRegisterNode(
+		types.Node{
+			Hostname:   hostname,
 			MachineKey: machineKey,
-			NodeKey:    regReq.NodeKey,
-			Hostinfo:   regReq.Hostinfo,
+			NodeKey:    req.NodeKey,
+			Hostinfo:   hostinfo,
 			LastSeen:   ptr.To(time.Now()),
 		},
-		Registered: make(chan *types.Node),
+	)
+
+	if !req.Expiry.IsZero() {
+		nodeToRegister.Node.Expiry = &req.Expiry
 	}
 
-	if !regReq.Expiry.IsZero() {
-		nodeToRegister.Node.Expiry = &regReq.Expiry
-	}
-
-	h.registrationCache.Set(
+	h.state.SetRegistrationCacheEntry(
 		registrationId,
 		nodeToRegister,
 	)
 
+	log.Info().Msgf("Starting node registration using key: %s", registrationId)
+
 	return &tailcfg.RegisterResponse{
 		AuthURL: h.authProvider.AuthURL(registrationId),
 	}, nil

hscontrol/capver/capver.go

@@ -1,17 +1,18 @@
 package capver
 
+//go:generate go run ../../tools/capver/main.go
+
 import (
+	"slices"
 	"sort"
 	"strings"
 
-	"slices"
-
 	xmaps "golang.org/x/exp/maps"
 	"tailscale.com/tailcfg"
 	"tailscale.com/util/set"
 )
 
-const MinSupportedCapabilityVersion tailcfg.CapabilityVersion = 88
+const MinSupportedCapabilityVersion tailcfg.CapabilityVersion = 90
 
 // CanOldCodeBeCleanedUp is intended to be called on startup to see if
 // there are old code that can ble cleaned up, entries should contain

hscontrol/capver/capver_generated.go

@@ -1,14 +1,10 @@
 package capver
 
-//Generated DO NOT EDIT
+// Generated DO NOT EDIT
 
 import "tailscale.com/tailcfg"
 
 var tailscaleToCapVer = map[string]tailcfg.CapabilityVersion{
-	"v1.60.0": 87,
-	"v1.60.1": 87,
-	"v1.62.0": 88,
-	"v1.62.1": 88,
 	"v1.64.0": 90,
 	"v1.64.1": 90,
 	"v1.64.2": 90,
@@ -36,19 +32,20 @@ var tailscaleToCapVer = map[string]tailcfg.CapabilityVersion{
 	"v1.80.3": 113,
 	"v1.82.0": 115,
 	"v1.82.5": 115,
+	"v1.84.0": 116,
+	"v1.84.1": 116,
+	"v1.84.2": 116,
 }
 
-
 var capVerToTailscaleVer = map[tailcfg.CapabilityVersion]string{
-	87:		"v1.60.0",
-	88:		"v1.62.0",
-	90:		"v1.64.0",
-	95:		"v1.66.0",
-	97:		"v1.68.0",
-	102:		"v1.70.0",
-	104:		"v1.72.0",
-	106:		"v1.74.0",
-	109:		"v1.78.0",
-	113:		"v1.80.0",
-	115:		"v1.82.0",
+	90:  "v1.64.0",
+	95:  "v1.66.0",
+	97:  "v1.68.0",
+	102: "v1.70.0",
+	104: "v1.72.0",
+	106: "v1.74.0",
+	109: "v1.78.0",
+	113: "v1.80.0",
+	115: "v1.82.0",
+	116: "v1.84.0",
 }

hscontrol/capver/capver_test.go

@@ -13,11 +13,10 @@ func TestTailscaleLatestMajorMinor(t *testing.T) {
 		stripV   bool
 		expected []string
 	}{
-		{3, false, []string{"v1.78", "v1.80", "v1.82"}},
-		{2, true, []string{"1.80", "1.82"}},
+		{3, false, []string{"v1.80", "v1.82", "v1.84"}},
+		{2, true, []string{"1.82", "1.84"}},
 		// Lazy way to see all supported versions
 		{10, true, []string{
-			"1.64",
 			"1.66",
 			"1.68",
 			"1.70",
@@ -27,6 +26,7 @@ func TestTailscaleLatestMajorMinor(t *testing.T) {
 			"1.78",
 			"1.80",
 			"1.82",
+			"1.84",
 		}},
 		{0, false, nil},
 	}
@@ -46,7 +46,6 @@ func TestCapVerMinimumTailscaleVersion(t *testing.T) {
 		input    tailcfg.CapabilityVersion
 		expected string
 	}{
-		{88, "v1.62.0"},
 		{90, "v1.64.0"},
 		{95, "v1.66.0"},
 		{106, "v1.74.0"},

hscontrol/db/db.go

@@ -3,6 +3,7 @@ package db
 import (
 	"context"
 	"database/sql"
+	_ "embed"
 	"encoding/json"
 	"errors"
 	"fmt"
@@ -15,9 +16,11 @@ import (
 
 	"github.com/glebarez/sqlite"
 	"github.com/go-gormigrate/gormigrate/v2"
+	"github.com/juanfont/headscale/hscontrol/db/sqliteconfig"
 	"github.com/juanfont/headscale/hscontrol/types"
 	"github.com/juanfont/headscale/hscontrol/util"
 	"github.com/rs/zerolog/log"
+	"github.com/tailscale/squibble"
 	"gorm.io/driver/postgres"
 	"gorm.io/gorm"
 	"gorm.io/gorm/logger"
@@ -27,12 +30,23 @@ import (
 	"zgo.at/zcache/v2"
 )
 
+//go:embed schema.sql
+var dbSchema string
+
 func init() {
 	schema.RegisterSerializer("text", TextSerialiser{})
 }
 
 var errDatabaseNotSupported = errors.New("database type not supported")
 
+var errForeignKeyConstraintsViolated = errors.New("foreign key constraints violated")
+
+const (
+	maxIdleConns       = 100
+	maxOpenConns       = 100
+	contextTimeoutSecs = 10
+)
+
 // KV is a key-value store in a psql table. For future use...
 // TODO(kradalby): Is this used for anything?
 type KV struct {
@@ -246,7 +260,7 @@ func NewHeadscaleDatabase(
 									log.Error().Err(err).Msg("Error creating route")
 								} else {
 									log.Info().
-										Uint64("node_id", route.NodeID).
+										Uint64("node.id", route.NodeID).
 										Str("prefix", prefix.String()).
 										Msg("Route migrated")
 								}
@@ -471,6 +485,7 @@ func NewHeadscaleDatabase(
 
 					// Drop the old table.
 					_ = tx.Migrator().DropTable(&preAuthKeyACLTag{})
+
 					return nil
 				},
 				Rollback: func(db *gorm.DB) error { return nil },
@@ -481,7 +496,7 @@ func NewHeadscaleDatabase(
 				ID: "202407191627",
 				Migrate: func(tx *gorm.DB) error {
 					// Fix an issue where the automigration in GORM expected a constraint to
-					// exists that didnt, and add the one it wanted.
+					// exists that didn't, and add the one it wanted.
 					// Fixes https://github.com/juanfont/headscale/issues/2351
 					if cfg.Type == types.DatabasePostgres {
 						err := tx.Exec(`
@@ -602,7 +617,7 @@ COMMIT;
 				},
 				Rollback: func(db *gorm.DB) error { return nil },
 			},
-			// Ensure there are no nodes refering to a deleted preauthkey.
+			// Ensure there are no nodes referring to a deleted preauthkey.
 			{
 				ID: "202502070949",
 				Migrate: func(tx *gorm.DB) error {
@@ -718,6 +733,209 @@ AND auth_key_id NOT IN (
 				},
 				Rollback: func(db *gorm.DB) error { return nil },
 			},
+			// Schema migration to ensure all tables match the expected schema.
+			// This migration recreates all tables to match the exact structure in schema.sql,
+			// preserving all data during the process.
+			// Only SQLite will be migrated for consistency.
+			{
+				ID: "202507021200",
+				Migrate: func(tx *gorm.DB) error {
+					// Only run on SQLite
+					if cfg.Type != types.DatabaseSqlite {
+						log.Info().Msg("Skipping schema migration on non-SQLite database")
+						return nil
+					}
+
+					log.Info().Msg("Starting schema recreation with table renaming")
+
+					// Rename existing tables to _old versions
+					tablesToRename := []string{"users", "pre_auth_keys", "api_keys", "nodes", "policies"}
+
+					// Check if routes table exists and drop it (should have been migrated already)
+					var routesExists bool
+					err := tx.Raw("SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name='routes'").Row().Scan(&routesExists)
+					if err == nil && routesExists {
+						log.Info().Msg("Dropping leftover routes table")
+						if err := tx.Exec("DROP TABLE routes").Error; err != nil {
+							return fmt.Errorf("dropping routes table: %w", err)
+						}
+					}
+
+					// Drop all indexes first to avoid conflicts
+					indexesToDrop := []string{
+						"idx_users_deleted_at",
+						"idx_provider_identifier",
+						"idx_name_provider_identifier",
+						"idx_name_no_provider_identifier",
+						"idx_api_keys_prefix",
+						"idx_policies_deleted_at",
+					}
+
+					for _, index := range indexesToDrop {
+						_ = tx.Exec("DROP INDEX IF EXISTS " + index).Error
+					}
+
+					for _, table := range tablesToRename {
+						// Check if table exists before renaming
+						var exists bool
+						err := tx.Raw("SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name=?", table).Row().Scan(&exists)
+						if err != nil {
+							return fmt.Errorf("checking if table %s exists: %w", table, err)
+						}
+
+						if exists {
+							// Drop old table if it exists from previous failed migration
+							_ = tx.Exec("DROP TABLE IF EXISTS " + table + "_old").Error
+
+							// Rename current table to _old
+							if err := tx.Exec("ALTER TABLE " + table + " RENAME TO " + table + "_old").Error; err != nil {
+								return fmt.Errorf("renaming table %s to %s_old: %w", table, table, err)
+							}
+						}
+					}
+
+					// Create new tables with correct schema
+					tableCreationSQL := []string{
+						`CREATE TABLE users(
+  id integer PRIMARY KEY AUTOINCREMENT,
+  name text,
+  display_name text,
+  email text,
+  provider_identifier text,
+  provider text,
+  profile_pic_url text,
+  created_at datetime,
+  updated_at datetime,
+  deleted_at datetime
+)`,
+						`CREATE TABLE pre_auth_keys(
+  id integer PRIMARY KEY AUTOINCREMENT,
+  key text,
+  user_id integer,
+  reusable numeric,
+  ephemeral numeric DEFAULT false,
+  used numeric DEFAULT false,
+  tags text,
+  expiration datetime,
+  created_at datetime,
+  CONSTRAINT fk_pre_auth_keys_user FOREIGN KEY(user_id) REFERENCES users(id) ON DELETE SET NULL
+)`,
+						`CREATE TABLE api_keys(
+  id integer PRIMARY KEY AUTOINCREMENT,
+  prefix text,
+  hash blob,
+  expiration datetime,
+  last_seen datetime,
+  created_at datetime
+)`,
+						`CREATE TABLE nodes(
+  id integer PRIMARY KEY AUTOINCREMENT,
+  machine_key text,
+  node_key text,
+  disco_key text,
+  endpoints text,
+  host_info text,
+  ipv4 text,
+  ipv6 text,
+  hostname text,
+  given_name varchar(63),
+  user_id integer,
+  register_method text,
+  forced_tags text,
+  auth_key_id integer,
+  last_seen datetime,
+  expiry datetime,
+  approved_routes text,
+  created_at datetime,
+  updated_at datetime,
+  deleted_at datetime,
+  CONSTRAINT fk_nodes_user FOREIGN KEY(user_id) REFERENCES users(id) ON DELETE CASCADE,
+  CONSTRAINT fk_nodes_auth_key FOREIGN KEY(auth_key_id) REFERENCES pre_auth_keys(id)
+)`,
+						`CREATE TABLE policies(
+  id integer PRIMARY KEY AUTOINCREMENT,
+  data text,
+  created_at datetime,
+  updated_at datetime,
+  deleted_at datetime
+)`,
+					}
+
+					for _, createSQL := range tableCreationSQL {
+						if err := tx.Exec(createSQL).Error; err != nil {
+							return fmt.Errorf("creating new table: %w", err)
+						}
+					}
+
+					// Copy data directly using SQL
+					dataCopySQL := []string{
+						`INSERT INTO users (id, name, display_name, email, provider_identifier, provider, profile_pic_url, created_at, updated_at, deleted_at)
+             SELECT id, name, display_name, email, provider_identifier, provider, profile_pic_url, created_at, updated_at, deleted_at
+             FROM users_old`,
+
+						`INSERT INTO pre_auth_keys (id, key, user_id, reusable, ephemeral, used, tags, expiration, created_at)
+             SELECT id, key, user_id, reusable, ephemeral, used, tags, expiration, created_at
+             FROM pre_auth_keys_old`,
+
+						`INSERT INTO api_keys (id, prefix, hash, expiration, last_seen, created_at)
+             SELECT id, prefix, hash, expiration, last_seen, created_at
+             FROM api_keys_old`,
+
+						`INSERT INTO nodes (id, machine_key, node_key, disco_key, endpoints, host_info, ipv4, ipv6, hostname, given_name, user_id, register_method, forced_tags, auth_key_id, last_seen, expiry, approved_routes, created_at, updated_at, deleted_at)
+             SELECT id, machine_key, node_key, disco_key, endpoints, host_info, ipv4, ipv6, hostname, given_name, user_id, register_method, forced_tags, auth_key_id, last_seen, expiry, approved_routes, created_at, updated_at, deleted_at
+             FROM nodes_old`,
+
+						`INSERT INTO policies (id, data, created_at, updated_at, deleted_at)
+             SELECT id, data, created_at, updated_at, deleted_at
+             FROM policies_old`,
+					}
+
+					for _, copySQL := range dataCopySQL {
+						if err := tx.Exec(copySQL).Error; err != nil {
+							return fmt.Errorf("copying data: %w", err)
+						}
+					}
+
+					// Create indexes
+					indexes := []string{
+						"CREATE INDEX idx_users_deleted_at ON users(deleted_at)",
+						`CREATE UNIQUE INDEX idx_provider_identifier ON users(
+  provider_identifier
+) WHERE provider_identifier IS NOT NULL`,
+						`CREATE UNIQUE INDEX idx_name_provider_identifier ON users(
+  name,
+  provider_identifier
+)`,
+						`CREATE UNIQUE INDEX idx_name_no_provider_identifier ON users(
+  name
+) WHERE provider_identifier IS NULL`,
+						"CREATE UNIQUE INDEX idx_api_keys_prefix ON api_keys(prefix)",
+						"CREATE INDEX idx_policies_deleted_at ON policies(deleted_at)",
+					}
+
+					for _, indexSQL := range indexes {
+						if err := tx.Exec(indexSQL).Error; err != nil {
+							return fmt.Errorf("creating index: %w", err)
+						}
+					}
+
+					// Drop old tables only after everything succeeds
+					for _, table := range tablesToRename {
+						if err := tx.Exec("DROP TABLE IF EXISTS " + table + "_old").Error; err != nil {
+							log.Warn().Str("table", table+"_old").Err(err).Msg("Failed to drop old table, but migration succeeded")
+						}
+					}
+
+					log.Info().Msg("Schema recreation completed successfully")
+
+					return nil
+				},
+				Rollback: func(db *gorm.DB) error { return nil },
+			},
+			// From this point, the following rules must be followed:
+			// - NEVER use gorm.AutoMigrate, write the exact migration steps needed
+			// - AutoMigrate depends on the struct staying exactly the same, which it won't over time.
+			// - Never write migrations that requires foreign keys to be disabled.
 		},
 	)
 
@@ -725,6 +943,30 @@ AND auth_key_id NOT IN (
 		log.Fatal().Err(err).Msgf("Migration failed: %v", err)
 	}
 
+	// Validate that the schema ends up in the expected state.
+	// This is currently only done on sqlite as squibble does not
+	// support Postgres and we use our sqlite schema as our source of
+	// truth.
+	if cfg.Type == types.DatabaseSqlite {
+		sqlConn, err := dbConn.DB()
+		if err != nil {
+			return nil, fmt.Errorf("getting DB from gorm: %w", err)
+		}
+
+		// or else it blocks...
+		sqlConn.SetMaxIdleConns(maxIdleConns)
+		sqlConn.SetMaxOpenConns(maxOpenConns)
+		defer sqlConn.SetMaxIdleConns(1)
+		defer sqlConn.SetMaxOpenConns(1)
+
+		ctx, cancel := context.WithTimeout(context.Background(), contextTimeoutSecs*time.Second)
+		defer cancel()
+
+		if err := squibble.Validate(ctx, sqlConn, dbSchema); err != nil {
+			return nil, fmt.Errorf("validating schema: %w", err)
+		}
+	}
+
 	db := HSDatabase{
 		DB:       dbConn,
 		cfg:      &cfg,
@@ -758,32 +1000,26 @@ func openDB(cfg types.DatabaseConfig) (*gorm.DB, error) {
 			Str("path", cfg.Sqlite.Path).
 			Msg("Opening database")
 
+		// Build SQLite configuration with pragmas set at connection time
+		sqliteConfig := sqliteconfig.Default(cfg.Sqlite.Path)
+		if cfg.Sqlite.WriteAheadLog {
+			sqliteConfig.JournalMode = sqliteconfig.JournalModeWAL
+			sqliteConfig.WALAutocheckpoint = cfg.Sqlite.WALAutoCheckPoint
+		}
+
+		connectionURL, err := sqliteConfig.ToURL()
+		if err != nil {
+			return nil, fmt.Errorf("building sqlite connection URL: %w", err)
+		}
+
 		db, err := gorm.Open(
-			sqlite.Open(cfg.Sqlite.Path),
+			sqlite.Open(connectionURL),
 			&gorm.Config{
 				PrepareStmt: cfg.Gorm.PrepareStmt,
 				Logger:      dbLogger,
 			},
 		)
 
-		if err := db.Exec(`
-			PRAGMA foreign_keys=ON;
-			PRAGMA busy_timeout=10000;
-			PRAGMA auto_vacuum=INCREMENTAL;
-			PRAGMA synchronous=NORMAL;
-			`).Error; err != nil {
-			return nil, fmt.Errorf("enabling foreign keys: %w", err)
-		}
-
-		if cfg.Sqlite.WriteAheadLog {
-			if err := db.Exec(fmt.Sprintf(`
-				PRAGMA journal_mode=WAL;
-				PRAGMA wal_autocheckpoint=%d;
-				`, cfg.Sqlite.WALAutoCheckPoint)).Error; err != nil {
-				return nil, fmt.Errorf("setting WAL mode: %w", err)
-			}
-		}
-
 		// The pure Go SQLite library does not handle locking in
 		// the same way as the C based one and we can't use the gorm
 		// connection pool as of 2022/02/23.
@@ -812,7 +1048,7 @@ func openDB(cfg types.DatabaseConfig) (*gorm.DB, error) {
 				dbString += " sslmode=disable"
 			}
 		} else {
-			dbString += fmt.Sprintf(" sslmode=%s", cfg.Postgres.Ssl)
+			dbString += " sslmode=" + cfg.Postgres.Ssl
 		}
 
 		if cfg.Postgres.Port != 0 {
@@ -820,7 +1056,7 @@ func openDB(cfg types.DatabaseConfig) (*gorm.DB, error) {
 		}
 
 		if cfg.Postgres.Pass != "" {
-			dbString += fmt.Sprintf(" password=%s", cfg.Postgres.Pass)
+			dbString += " password=" + cfg.Postgres.Pass
 		}
 
 		db, err := gorm.Open(postgres.Open(dbString), &gorm.Config{
@@ -848,29 +1084,84 @@ func openDB(cfg types.DatabaseConfig) (*gorm.DB, error) {
 }
 
 func runMigrations(cfg types.DatabaseConfig, dbConn *gorm.DB, migrations *gormigrate.Gormigrate) error {
-	// Turn off foreign keys for the duration of the migration if using sqlite to
-	// prevent data loss due to the way the GORM migrator handles certain schema
-	// changes.
 	if cfg.Type == types.DatabaseSqlite {
-		var fkEnabled int
-		if err := dbConn.Raw("PRAGMA foreign_keys").Scan(&fkEnabled).Error; err != nil {
+		// SQLite: Run migrations step-by-step, only disabling foreign keys when necessary
+
+		// List of migration IDs that require foreign keys to be disabled
+		// These are migrations that perform complex schema changes that GORM cannot handle safely with FK enabled
+		// NO NEW MIGRATIONS SHOULD BE ADDED HERE. ALL NEW MIGRATIONS MUST RUN WITH FOREIGN KEYS ENABLED.
+		migrationsRequiringFKDisabled := map[string]bool{
+			"202312101416":  true, // Initial migration with complex table/column renames
+			"202402151347":  true, // Migration that removes last_successful_update column
+			"2024041121742": true, // Migration that changes IP address storage format
+			"202407191627":  true, // User table automigration with FK constraint issues
+			"202408181235":  true, // User table automigration with FK constraint issues
+			"202501221827":  true, // Route table automigration with FK constraint issues
+			"202501311657":  true, // PreAuthKey table automigration with FK constraint issues
+			// Add other migration IDs here as they are identified to need FK disabled
+		}
+
+		// Get the current foreign key status
+		var fkOriginallyEnabled int
+		if err := dbConn.Raw("PRAGMA foreign_keys").Scan(&fkOriginallyEnabled).Error; err != nil {
 			return fmt.Errorf("checking foreign key status: %w", err)
 		}
-		if fkEnabled == 1 {
-			if err := dbConn.Exec("PRAGMA foreign_keys = OFF").Error; err != nil {
-				return fmt.Errorf("disabling foreign keys: %w", err)
-			}
-			defer dbConn.Exec("PRAGMA foreign_keys = ON")
+
+		// Get all migration IDs in order from the actual migration definitions
+		// Only IDs that are in the migrationsRequiringFKDisabled map will be processed with FK disabled
+		// any other new migrations are ran after.
+		migrationIDs := []string{
+			"202312101416",
+			"202312101430",
+			"202402151347",
+			"2024041121742",
+			"202406021630",
+			"202407191627",
+			"202408181235",
+			"202409271400",
+			"202501221827",
+			"202501311657",
+			"202502070949",
+			"202502131714",
+			"202502171819",
+			"202505091439",
+			"202505141324",
+			// As of 2025-07-02, no new IDs should be added here.
+			// They will be ran by the migrations.Migrate() call below.
 		}
-	}
 
-	if err := migrations.Migrate(); err != nil {
-		return err
-	}
+		for _, migrationID := range migrationIDs {
+			log.Trace().Caller().Str("migration_id", migrationID).Msg("Running migration")
+			needsFKDisabled := migrationsRequiringFKDisabled[migrationID]
 
-	// Since we disabled foreign keys for the migration, we need to check for
-	// constraint violations manually at the end of the migration.
-	if cfg.Type == types.DatabaseSqlite {
+			if needsFKDisabled {
+				// Disable foreign keys for this migration
+				if err := dbConn.Exec("PRAGMA foreign_keys = OFF").Error; err != nil {
+					return fmt.Errorf("disabling foreign keys for migration %s: %w", migrationID, err)
+				}
+			} else {
+				// Ensure foreign keys are enabled for this migration
+				if err := dbConn.Exec("PRAGMA foreign_keys = ON").Error; err != nil {
+					return fmt.Errorf("enabling foreign keys for migration %s: %w", migrationID, err)
+				}
+			}
+
+			// Run up to this specific migration (will only run the next pending migration)
+			if err := migrations.MigrateTo(migrationID); err != nil {
+				return fmt.Errorf("running migration %s: %w", migrationID, err)
+			}
+		}
+
+		if err := dbConn.Exec("PRAGMA foreign_keys = ON").Error; err != nil {
+			return fmt.Errorf("restoring foreign keys: %w", err)
+		}
+
+		// Run the rest of the migrations
+		if err := migrations.Migrate(); err != nil {
+			return err
+		}
+
+		// Check for constraint violations at the end
 		type constraintViolation struct {
 			Table           string
 			RowID           int
@@ -904,7 +1195,12 @@ func runMigrations(cfg types.DatabaseConfig, dbConn *gorm.DB, migrations *gormig
 					Msg("Foreign key constraint violated")
 			}
 
-			return fmt.Errorf("foreign key constraints violated")
+			return errForeignKeyConstraintsViolated
+		}
+	} else {
+		// PostgreSQL can run all migrations in one block - no foreign key issues
+		if err := migrations.Migrate(); err != nil {
+			return err
 		}
 	}
 
@@ -949,6 +1245,7 @@ func Read[T any](db *gorm.DB, fn func(rx *gorm.DB) (T, error)) (T, error) {
 		var no T
 		return no, err
 	}
+
 	return ret, nil
 }
 
@@ -970,5 +1267,6 @@ func Write[T any](db *gorm.DB, fn func(tx *gorm.DB) (T, error)) (T, error) {
 		var no T
 		return no, err
 	}
+
 	return ret, tx.Commit().Error
 }

hscontrol/db/db_test.go

@@ -2,14 +2,11 @@ package db
 
 import (
 	"database/sql"
-	"fmt"
-	"io"
 	"net/netip"
 	"os"
 	"os/exec"
 	"path/filepath"
 	"slices"
-	"sort"
 	"strings"
 	"testing"
 	"time"
@@ -24,10 +21,10 @@ import (
 	"zgo.at/zcache/v2"
 )
 
-// TestMigrationsSQLite is the main function for testing migrations,
-// we focus on SQLite correctness as it is the main database used in headscale.
-// All migrations that are worth testing should be added here.
-func TestMigrationsSQLite(t *testing.T) {
+// TestSQLiteMigrationAndDataValidation tests specific SQLite migration scenarios
+// and validates data integrity after migration. All migrations that require data validation
+// should be added here.
+func TestSQLiteMigrationAndDataValidation(t *testing.T) {
 	ipp := func(p string) netip.Prefix {
 		return netip.MustParsePrefix(p)
 	}
@@ -43,12 +40,39 @@ func TestMigrationsSQLite(t *testing.T) {
 	tests := []struct {
 		dbPath   string
 		wantFunc func(*testing.T, *HSDatabase)
-		wantErr  string
 	}{
 		{
-			dbPath: "testdata/0-22-3-to-0-23-0-routes-are-dropped-2063.sqlite",
-			wantFunc: func(t *testing.T, h *HSDatabase) {
-				nodes, err := Read(h.DB, func(rx *gorm.DB) (types.Nodes, error) {
+			dbPath: "testdata/sqlite/0-22-3-to-0-23-0-routes-are-dropped-2063_dump.sql",
+			wantFunc: func(t *testing.T, hsdb *HSDatabase) {
+				t.Helper()
+				// Comprehensive data preservation validation for 0.22.3->0.23.0 migration
+				// Expected data from dump: 4 users, 17 pre_auth_keys, 14 machines/nodes, 12 routes
+
+				// Verify users data preservation - should have 4 users
+				users, err := Read(hsdb.DB, func(rx *gorm.DB) ([]types.User, error) {
+					return ListUsers(rx)
+				})
+				require.NoError(t, err)
+				assert.Len(t, users, 4, "should preserve all 4 users from original schema")
+
+				// Verify pre_auth_keys data preservation - should have 17 keys
+				preAuthKeys, err := Read(hsdb.DB, func(rx *gorm.DB) ([]types.PreAuthKey, error) {
+					var keys []types.PreAuthKey
+					err := rx.Find(&keys).Error
+					return keys, err
+				})
+				require.NoError(t, err)
+				assert.Len(t, preAuthKeys, 17, "should preserve all 17 pre_auth_keys from original schema")
+
+				// Verify all nodes data preservation - should have 14 nodes
+				allNodes, err := Read(hsdb.DB, func(rx *gorm.DB) (types.Nodes, error) {
+					return ListNodes(rx)
+				})
+				require.NoError(t, err)
+				assert.Len(t, allNodes, 14, "should preserve all 14 machines/nodes from original schema")
+
+				// Verify specific nodes and their route migration with detailed validation
+				nodes, err := Read(hsdb.DB, func(rx *gorm.DB) (types.Nodes, error) {
 					n1, err := GetNodeByID(rx, 1)
 					n26, err := GetNodeByID(rx, 26)
 					n31, err := GetNodeByID(rx, 31)
@@ -60,24 +84,66 @@ func TestMigrationsSQLite(t *testing.T) {
 					return types.Nodes{n1, n26, n31, n32}, nil
 				})
 				require.NoError(t, err)
+				assert.Len(t, nodes, 4, "should have retrieved 4 specific nodes")
 
-				// want := types.Routes{
-				// 	r(1, "0.0.0.0/0", true, false),
-				// 	r(1, "::/0", true, false),
-				// 	r(1, "10.9.110.0/24", true, true),
-				// 	r(26, "172.100.100.0/24", true, true),
-				// 	r(26, "172.100.100.0/24", true, false, false),
-				// 	r(31, "0.0.0.0/0", true, false),
-				// 	r(31, "0.0.0.0/0", true, false, false),
-				// 	r(31, "::/0", true, false),
-				// 	r(31, "::/0", true, false, false),
-				// 	r(32, "192.168.0.24/32", true, true),
-				// }
+				// Validate specific node data from dump file
+				nodesByID := make(map[uint64]*types.Node)
+				for i := range nodes {
+					nodesByID[nodes[i].ID.Uint64()] = nodes[i]
+				}
+
+				node1 := nodesByID[1]
+				node26 := nodesByID[26]
+				node31 := nodesByID[31]
+				node32 := nodesByID[32]
+
+				require.NotNil(t, node1, "node 1 should exist")
+				require.NotNil(t, node26, "node 26 should exist")
+				require.NotNil(t, node31, "node 31 should exist")
+				require.NotNil(t, node32, "node 32 should exist")
+
+				// Validate node data using cmp.Diff
+				expectedNodes := map[uint64]struct {
+					Hostname  string
+					GivenName string
+					IPv4      string
+				}{
+					1:  {Hostname: "test_hostname", GivenName: "test_given_name", IPv4: "100.64.0.1"},
+					26: {Hostname: "test_hostname", GivenName: "test_given_name", IPv4: "100.64.0.19"},
+					31: {Hostname: "test_hostname", GivenName: "test_given_name", IPv4: "100.64.0.7"},
+					32: {Hostname: "test_hostname", GivenName: "test_given_name", IPv4: "100.64.0.11"},
+				}
+
+				for nodeID, expected := range expectedNodes {
+					node := nodesByID[nodeID]
+					require.NotNil(t, node, "node %d should exist", nodeID)
+
+					actual := struct {
+						Hostname  string
+						GivenName string
+						IPv4      string
+					}{
+						Hostname:  node.Hostname,
+						GivenName: node.GivenName,
+						IPv4:      node.IPv4.String(),
+					}
+
+					if diff := cmp.Diff(expected, actual); diff != "" {
+						t.Errorf("TestSQLiteMigrationAndDataValidation() node %d mismatch (-want +got):\n%s", nodeID, diff)
+					}
+				}
+
+				// Validate that routes were properly migrated from routes table to approved_routes
+				// Based on the dump file routes data:
+				// Node 1 (machine_id 1): routes 1,2,3 (0.0.0.0/0 enabled, ::/0 enabled, 10.9.110.0/24 enabled+primary)
+				// Node 26 (machine_id 26): route 6 (172.100.100.0/24 enabled+primary), route 7 (172.100.100.0/24 disabled)
+				// Node 31 (machine_id 31): routes 8,10 (0.0.0.0/0 enabled, ::/0 enabled), routes 9,11 (duplicates disabled)
+				// Node 32 (machine_id 32): route 12 (192.168.0.24/32 enabled+primary)
 				want := [][]netip.Prefix{
-					{ipp("0.0.0.0/0"), ipp("10.9.110.0/24"), ipp("::/0")},
-					{ipp("172.100.100.0/24")},
-					{ipp("0.0.0.0/0"), ipp("::/0")},
-					{ipp("192.168.0.24/32")},
+					{ipp("0.0.0.0/0"), ipp("10.9.110.0/24"), ipp("::/0")}, // node 1: 3 enabled routes
+					{ipp("172.100.100.0/24")},                             // node 26: 1 enabled route
+					{ipp("0.0.0.0/0"), ipp("::/0")},                       // node 31: 2 enabled routes
+					{ipp("192.168.0.24/32")},                              // node 32: 1 enabled route
 				}
 				var got [][]netip.Prefix
 				for _, node := range nodes {
@@ -85,14 +151,48 @@ func TestMigrationsSQLite(t *testing.T) {
 				}
 
 				if diff := cmp.Diff(want, got, util.PrefixComparer); diff != "" {
-					t.Errorf("TestMigrations() mismatch (-want +got):\n%s", diff)
+					t.Errorf("TestSQLiteMigrationAndDataValidation() route migration mismatch (-want +got):\n%s", diff)
 				}
+
+				// Verify routes table was dropped after migration
+				var routesTableExists bool
+				err = hsdb.DB.Raw("SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name='routes'").Row().Scan(&routesTableExists)
+				require.NoError(t, err)
+				assert.False(t, routesTableExists, "routes table should have been dropped after migration")
 			},
 		},
 		{
-			dbPath: "testdata/0-22-3-to-0-23-0-routes-fail-foreign-key-2076.sqlite",
-			wantFunc: func(t *testing.T, h *HSDatabase) {
-				node, err := Read(h.DB, func(rx *gorm.DB) (*types.Node, error) {
+			dbPath: "testdata/sqlite/0-22-3-to-0-23-0-routes-fail-foreign-key-2076_dump.sql",
+			wantFunc: func(t *testing.T, hsdb *HSDatabase) {
+				t.Helper()
+				// Comprehensive data preservation validation for foreign key constraint issue case
+				// Expected data from dump: 4 users, 2 pre_auth_keys, 8 nodes
+
+				// Verify users data preservation
+				users, err := Read(hsdb.DB, func(rx *gorm.DB) ([]types.User, error) {
+					return ListUsers(rx)
+				})
+				require.NoError(t, err)
+				assert.Len(t, users, 4, "should preserve all 4 users from original schema")
+
+				// Verify pre_auth_keys data preservation
+				preAuthKeys, err := Read(hsdb.DB, func(rx *gorm.DB) ([]types.PreAuthKey, error) {
+					var keys []types.PreAuthKey
+					err := rx.Find(&keys).Error
+					return keys, err
+				})
+				require.NoError(t, err)
+				assert.Len(t, preAuthKeys, 2, "should preserve all 2 pre_auth_keys from original schema")
+
+				// Verify all nodes data preservation
+				allNodes, err := Read(hsdb.DB, func(rx *gorm.DB) (types.Nodes, error) {
+					return ListNodes(rx)
+				})
+				require.NoError(t, err)
+				assert.Len(t, allNodes, 8, "should preserve all 8 nodes from original schema")
+
+				// Verify specific node route migration
+				node, err := Read(hsdb.DB, func(rx *gorm.DB) (*types.Node, error) {
 					return GetNodeByID(rx, 13)
 				})
 				require.NoError(t, err)
@@ -101,26 +201,26 @@ func TestMigrationsSQLite(t *testing.T) {
 				_ = types.Routes{
 					// These routes exists, but have no nodes associated with them
 					// when the migration starts.
-					// r(1, "0.0.0.0/0", true, true, false),
-					// r(1, "::/0", true, true, false),
-					// r(3, "0.0.0.0/0", true, true, false),
-					// r(3, "::/0", true, true, false),
-					// r(5, "0.0.0.0/0", true, true, false),
-					// r(5, "::/0", true, true, false),
-					// r(6, "0.0.0.0/0", true, true, false),
-					// r(6, "::/0", true, true, false),
+					// r(1, "0.0.0.0/0", true, false),
+					// r(1, "::/0", true, false),
+					// r(3, "0.0.0.0/0", true, false),
+					// r(3, "::/0", true, false),
+					// r(5, "0.0.0.0/0", true, false),
+					// r(5, "::/0", true, false),
+					// r(6, "0.0.0.0/0", true, false),
+					// r(6, "::/0", true, false),
 					// r(6, "10.0.0.0/8", true, false, false),
-					// r(7, "0.0.0.0/0", true, true, false),
-					// r(7, "::/0", true, true, false),
+					// r(7, "0.0.0.0/0", true, false),
+					// r(7, "::/0", true, false),
 					// r(7, "10.0.0.0/8", true, false, false),
-					// r(9, "0.0.0.0/0", true, true, false),
-					// r(9, "::/0", true, true, false),
-					// r(9, "10.0.0.0/8", true, true, false),
-					// r(11, "0.0.0.0/0", true, true, false),
-					// r(11, "::/0", true, true, false),
-					// r(11, "10.0.0.0/8", true, true, true),
-					// r(12, "0.0.0.0/0", true, true, false),
-					// r(12, "::/0", true, true, false),
+					// r(9, "0.0.0.0/0", true, false),
+					// r(9, "::/0", true, false),
+					// r(9, "10.0.0.0/8", true, false),
+					// r(11, "0.0.0.0/0", true, false),
+					// r(11, "::/0", true, false),
+					// r(11, "10.0.0.0/8", true, true),
+					// r(12, "0.0.0.0/0", true, false),
+					// r(12, "::/0", true, false),
 					// r(12, "10.0.0.0/8", true, false, false),
 					//
 					// These nodes exists, so routes should be kept.
@@ -131,8 +231,14 @@ func TestMigrationsSQLite(t *testing.T) {
 				}
 				want := []netip.Prefix{ipp("0.0.0.0/0"), ipp("10.18.80.2/32"), ipp("::/0")}
 				if diff := cmp.Diff(want, node.ApprovedRoutes, util.PrefixComparer); diff != "" {
-					t.Errorf("TestMigrations() mismatch (-want +got):\n%s", diff)
+					t.Errorf("TestSQLiteMigrationAndDataValidation() route migration mismatch (-want +got):\n%s", diff)
 				}
+
+				// Verify routes table was dropped after migration
+				var routesTableExists bool
+				err = hsdb.DB.Raw("SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name='routes'").Row().Scan(&routesTableExists)
+				require.NoError(t, err)
+				assert.False(t, routesTableExists, "routes table should have been dropped after migration")
 			},
 		},
 		// at 14:15:06 ❯ go run ./cmd/headscale preauthkeys list
@@ -143,9 +249,49 @@ func TestMigrationsSQLite(t *testing.T) {
 		// 4  | f20155.. | false    | false     | false | 2024-09-27 | 2024-09-27 | tag:test
 		// 5  | b212b9.. | false    | false     | false | 2024-09-27 | 2024-09-27 | tag:test,tag:woop,tag:dedu
 		{
-			dbPath: "testdata/0-23-0-to-0-24-0-preauthkey-tags-table.sqlite",
-			wantFunc: func(t *testing.T, h *HSDatabase) {
-				keys, err := Read(h.DB, func(rx *gorm.DB) ([]types.PreAuthKey, error) {
+			dbPath: "testdata/sqlite/0-23-0-to-0-24-0-preauthkey-tags-table_dump.sql",
+			wantFunc: func(t *testing.T, hsdb *HSDatabase) {
+				t.Helper()
+				// Comprehensive data preservation validation for pre-auth key tags migration
+				// Expected data from dump: 2 users (kratest, testkra), 5 pre_auth_keys with specific tags
+
+				// Verify users data preservation with specific user data
+				users, err := Read(hsdb.DB, func(rx *gorm.DB) ([]types.User, error) {
+					return ListUsers(rx)
+				})
+				require.NoError(t, err)
+				assert.Len(t, users, 2, "should preserve all 2 users from original schema")
+
+				// Validate specific user data from dump file using cmp.Diff
+				expectedUsers := []types.User{
+					{Model: gorm.Model{ID: 1}, Name: "kratest"},
+					{Model: gorm.Model{ID: 2}, Name: "testkra"},
+				}
+
+				if diff := cmp.Diff(expectedUsers, users,
+					cmpopts.IgnoreFields(types.User{}, "CreatedAt", "UpdatedAt", "DeletedAt", "DisplayName", "Email", "ProviderIdentifier", "Provider", "ProfilePicURL")); diff != "" {
+					t.Errorf("TestSQLiteMigrationAndDataValidation() users mismatch (-want +got):\n%s", diff)
+				}
+
+				// Create maps for easier access in later validations
+				usersByName := make(map[string]*types.User)
+				for i := range users {
+					usersByName[users[i].Name] = &users[i]
+				}
+				kratest := usersByName["kratest"]
+				testkra := usersByName["testkra"]
+
+				// Verify all pre_auth_keys data preservation
+				allKeys, err := Read(hsdb.DB, func(rx *gorm.DB) ([]types.PreAuthKey, error) {
+					var keys []types.PreAuthKey
+					err := rx.Find(&keys).Error
+					return keys, err
+				})
+				require.NoError(t, err)
+				assert.Len(t, allKeys, 5, "should preserve all 5 pre_auth_keys from original schema")
+
+				// Verify specific pre-auth keys and their tag migration with exact data validation
+				keys, err := Read(hsdb.DB, func(rx *gorm.DB) ([]types.PreAuthKey, error) {
 					kratest, err := ListPreAuthKeysByUser(rx, 1) // kratest
 					if err != nil {
 						return nil, err
@@ -159,51 +305,104 @@ func TestMigrationsSQLite(t *testing.T) {
 					return append(kratest, testkra...), nil
 				})
 				require.NoError(t, err)
-
 				assert.Len(t, keys, 5)
-				want := []types.PreAuthKey{
+
+				// Create map for easier validation by ID
+				keysByID := make(map[uint64]*types.PreAuthKey)
+				for i := range keys {
+					keysByID[keys[i].ID] = &keys[i]
+				}
+
+				// Validate specific pre-auth key data and tag migration from pre_auth_key_acl_tags table
+				key1 := keysByID[1]
+				key2 := keysByID[2]
+				key3 := keysByID[3]
+				key4 := keysByID[4]
+				key5 := keysByID[5]
+
+				require.NotNil(t, key1, "pre_auth_key 1 should exist")
+				require.NotNil(t, key2, "pre_auth_key 2 should exist")
+				require.NotNil(t, key3, "pre_auth_key 3 should exist")
+				require.NotNil(t, key4, "pre_auth_key 4 should exist")
+				require.NotNil(t, key5, "pre_auth_key 5 should exist")
+
+				// Validate specific pre-auth key data and tag migration using cmp.Diff
+				expectedKeys := []types.PreAuthKey{
 					{
-						ID:   1,
-						Tags: []string{"tag:derp"},
+						ID:     1,
+						Key:    "09b28f8c3351984874d46dace0a70177a8721933a950b663",
+						UserID: kratest.ID,
+						Tags:   []string{"tag:derp"},
 					},
 					{
-						ID:   2,
-						Tags: []string{"tag:derp"},
+						ID:     2,
+						Key:    "3112b953cb344191b2d5aec1b891250125bf7b437eac5d26",
+						UserID: kratest.ID,
+						Tags:   []string{"tag:derp"},
 					},
 					{
-						ID:   3,
-						Tags: []string{"tag:derp", "tag:merp"},
+						ID:     3,
+						Key:    "7c23b9f215961e7609527aef78bf82fb19064b002d78c36f",
+						UserID: kratest.ID,
+						Tags:   []string{"tag:derp", "tag:merp"},
 					},
 					{
-						ID:   4,
-						Tags: []string{"tag:test"},
+						ID:     4,
+						Key:    "f2015583852b725220cc4b107fb288a4cf7ac259bd458a32",
+						UserID: testkra.ID,
+						Tags:   []string{"tag:test"},
 					},
 					{
-						ID:   5,
-						Tags: []string{"tag:test", "tag:woop", "tag:dedu"},
+						ID:     5,
+						Key:    "b212b990165e897944dd3772786544402729fb349da50f57",
+						UserID: testkra.ID,
+						Tags:   []string{"tag:test", "tag:woop", "tag:dedu"},
 					},
 				}
 
-				if diff := cmp.Diff(want, keys, cmp.Comparer(func(a, b []string) bool {
-					sort.Sort(sort.StringSlice(a))
-					sort.Sort(sort.StringSlice(b))
+				if diff := cmp.Diff(expectedKeys, keys, cmp.Comparer(func(a, b []string) bool {
+					slices.Sort(a)
+					slices.Sort(b)
 					return slices.Equal(a, b)
-				}), cmpopts.IgnoreFields(types.PreAuthKey{}, "Key", "UserID", "User", "CreatedAt", "Expiration")); diff != "" {
-					t.Errorf("TestMigrations() mismatch (-want +got):\n%s", diff)
+				}), cmpopts.IgnoreFields(types.PreAuthKey{}, "User", "CreatedAt", "Reusable", "Ephemeral", "Used", "Expiration")); diff != "" {
+					t.Errorf("TestSQLiteMigrationAndDataValidation() pre-auth key tags migration mismatch (-want +got):\n%s", diff)
 				}
 
-				if h.DB.Migrator().HasTable("pre_auth_key_acl_tags") {
-					t.Errorf("TestMigrations() table pre_auth_key_acl_tags should not exist")
+				// Verify pre_auth_key_acl_tags table was dropped after migration
+				if hsdb.DB.Migrator().HasTable("pre_auth_key_acl_tags") {
+					t.Errorf("TestSQLiteMigrationAndDataValidation() table pre_auth_key_acl_tags should not exist after migration")
 				}
 			},
 		},
 		{
-			dbPath: "testdata/0-23-0-to-0-24-0-no-more-special-types.sqlite",
-			wantFunc: func(t *testing.T, h *HSDatabase) {
-				nodes, err := Read(h.DB, func(rx *gorm.DB) (types.Nodes, error) {
+			dbPath: "testdata/sqlite/0-23-0-to-0-24-0-no-more-special-types_dump.sql",
+			wantFunc: func(t *testing.T, hsdb *HSDatabase) {
+				t.Helper()
+				// Comprehensive data preservation validation for special types removal migration
+				// Expected data from dump: 2 users, 2 pre_auth_keys, 12 nodes
+
+				// Verify users data preservation
+				users, err := Read(hsdb.DB, func(rx *gorm.DB) ([]types.User, error) {
+					return ListUsers(rx)
+				})
+				require.NoError(t, err)
+				assert.Len(t, users, 2, "should preserve all 2 users from original schema")
+
+				// Verify pre_auth_keys data preservation
+				preAuthKeys, err := Read(hsdb.DB, func(rx *gorm.DB) ([]types.PreAuthKey, error) {
+					var keys []types.PreAuthKey
+					err := rx.Find(&keys).Error
+					return keys, err
+				})
+				require.NoError(t, err)
+				assert.Len(t, preAuthKeys, 2, "should preserve all 2 pre_auth_keys from original schema")
+
+				// Verify nodes data preservation and field validation
+				nodes, err := Read(hsdb.DB, func(rx *gorm.DB) (types.Nodes, error) {
 					return ListNodes(rx)
 				})
 				require.NoError(t, err)
+				assert.Len(t, nodes, 12, "should preserve all 12 nodes from original schema")
 
 				for _, node := range nodes {
 					assert.Falsef(t, node.MachineKey.IsZero(), "expected non zero machinekey")
@@ -213,7 +412,7 @@ func TestMigrationsSQLite(t *testing.T) {
 					assert.Falsef(t, node.DiscoKey.IsZero(), "expected non zero discokey")
 					assert.Contains(t, node.DiscoKey.String(), "discokey:")
 					assert.NotNil(t, node.IPv4)
-					assert.NotNil(t, node.IPv4)
+					assert.NotNil(t, node.IPv6)
 					assert.Len(t, node.Endpoints, 1)
 					assert.NotNil(t, node.Hostinfo)
 					assert.NotNil(t, node.MachineKey)
@@ -221,12 +420,31 @@ func TestMigrationsSQLite(t *testing.T) {
 			},
 		},
 		{
-			dbPath: "testdata/failing-node-preauth-constraint.sqlite",
-			wantFunc: func(t *testing.T, h *HSDatabase) {
-				nodes, err := Read(h.DB, func(rx *gorm.DB) (types.Nodes, error) {
+			dbPath: "testdata/sqlite/failing-node-preauth-constraint_dump.sql",
+			wantFunc: func(t *testing.T, hsdb *HSDatabase) {
+				t.Helper()
+				// Comprehensive data preservation validation for node-preauth constraint issue
+				// Expected data from dump: 1 user, 2 api_keys, 6 nodes
+
+				// Verify users data preservation
+				users, err := Read(hsdb.DB, func(rx *gorm.DB) ([]types.User, error) {
+					return ListUsers(rx)
+				})
+				require.NoError(t, err)
+				assert.Len(t, users, 1, "should preserve all 1 user from original schema")
+
+				// Verify api_keys data preservation
+				var apiKeyCount int
+				err = hsdb.DB.Raw("SELECT COUNT(*) FROM api_keys").Scan(&apiKeyCount).Error
+				require.NoError(t, err)
+				assert.Equal(t, 2, apiKeyCount, "should preserve all 2 api_keys from original schema")
+
+				// Verify nodes data preservation and field validation
+				nodes, err := Read(hsdb.DB, func(rx *gorm.DB) (types.Nodes, error) {
 					return ListNodes(rx)
 				})
 				require.NoError(t, err)
+				assert.Len(t, nodes, 6, "should preserve all 6 nodes from original schema")
 
 				for _, node := range nodes {
 					assert.Falsef(t, node.MachineKey.IsZero(), "expected non zero machinekey")
@@ -240,25 +458,262 @@ func TestMigrationsSQLite(t *testing.T) {
 				}
 			},
 		},
+		{
+			dbPath: "testdata/sqlite/wrongly-migrated-schema-0.25.1_dump.sql",
+			wantFunc: func(t *testing.T, hsdb *HSDatabase) {
+				t.Helper()
+				// Test migration of a database that was wrongly migrated in 0.25.1
+				// This database has several issues:
+				// 1. Missing proper user unique constraints (idx_provider_identifier, idx_name_provider_identifier, idx_name_no_provider_identifier)
+				// 2. Still has routes table that should have been migrated to node.approved_routes
+				// 3. Wrong FOREIGN KEY constraint on pre_auth_keys (CASCADE instead of SET NULL)
+				// 4. Missing some required indexes
+
+				// Verify users table data is preserved with specific user data
+				users, err := Read(hsdb.DB, func(rx *gorm.DB) ([]types.User, error) {
+					return ListUsers(rx)
+				})
+				require.NoError(t, err)
+				assert.Len(t, users, 2, "should preserve existing users")
+
+				// Validate specific user data from dump file using cmp.Diff
+				expectedUsers := []types.User{
+					{Model: gorm.Model{ID: 1}, Name: "user2"},
+					{Model: gorm.Model{ID: 2}, Name: "user1"},
+				}
+
+				if diff := cmp.Diff(expectedUsers, users,
+					cmpopts.IgnoreFields(types.User{}, "CreatedAt", "UpdatedAt", "DeletedAt", "DisplayName", "Email", "ProviderIdentifier", "Provider", "ProfilePicURL")); diff != "" {
+					t.Errorf("TestSQLiteMigrationAndDataValidation() users mismatch (-want +got):\n%s", diff)
+				}
+
+				// Create maps for easier access in later validations
+				usersByName := make(map[string]*types.User)
+				for i := range users {
+					usersByName[users[i].Name] = &users[i]
+				}
+				user1 := usersByName["user1"]
+				user2 := usersByName["user2"]
+
+				// Verify nodes table data is preserved and routes migrated to approved_routes
+				nodes, err := Read(hsdb.DB, func(rx *gorm.DB) (types.Nodes, error) {
+					return ListNodes(rx)
+				})
+				require.NoError(t, err)
+				assert.Len(t, nodes, 3, "should preserve existing nodes")
+
+				// Validate specific node data from dump file
+				nodesByID := make(map[uint64]*types.Node)
+				for i := range nodes {
+					nodesByID[nodes[i].ID.Uint64()] = nodes[i]
+				}
+
+				node1 := nodesByID[1]
+				node2 := nodesByID[2]
+				node3 := nodesByID[3]
+				require.NotNil(t, node1, "node 1 should exist")
+				require.NotNil(t, node2, "node 2 should exist")
+				require.NotNil(t, node3, "node 3 should exist")
+
+				// Validate specific node field data using cmp.Diff
+				expectedNodes := map[uint64]struct {
+					Hostname  string
+					GivenName string
+					IPv4      string
+					IPv6      string
+					UserID    uint
+				}{
+					1: {Hostname: "node1", GivenName: "node1", IPv4: "100.64.0.1", IPv6: "fd7a:115c:a1e0::1", UserID: user2.ID},
+					2: {Hostname: "node2", GivenName: "node2", IPv4: "100.64.0.2", IPv6: "fd7a:115c:a1e0::2", UserID: user2.ID},
+					3: {Hostname: "node3", GivenName: "node3", IPv4: "100.64.0.3", IPv6: "fd7a:115c:a1e0::3", UserID: user1.ID},
+				}
+
+				for nodeID, expected := range expectedNodes {
+					node := nodesByID[nodeID]
+					require.NotNil(t, node, "node %d should exist", nodeID)
+
+					actual := struct {
+						Hostname  string
+						GivenName string
+						IPv4      string
+						IPv6      string
+						UserID    uint
+					}{
+						Hostname:  node.Hostname,
+						GivenName: node.GivenName,
+						IPv4:      node.IPv4.String(),
+						IPv6: func() string {
+							if node.IPv6 != nil {
+								return node.IPv6.String()
+							} else {
+								return ""
+							}
+						}(),
+						UserID: node.UserID,
+					}
+
+					if diff := cmp.Diff(expected, actual); diff != "" {
+						t.Errorf("TestSQLiteMigrationAndDataValidation() node %d basic fields mismatch (-want +got):\n%s", nodeID, diff)
+					}
+
+					// Special validation for MachineKey content for node 1 only
+					if nodeID == 1 {
+						assert.Contains(t, node.MachineKey.String(), "mkey:1efe4388236c1c83fe0a19d3ce7c321ab81e138a4da57917c231ce4c01944409")
+					}
+				}
+
+				// Check that routes were migrated from routes table to node.approved_routes using cmp.Diff
+				// Original routes table had 4 routes for nodes 1, 2, 3:
+				// Node 1: 0.0.0.0/0 (enabled), ::/0 (enabled) -> should have 2 approved routes
+				// Node 2: 192.168.100.0/24 (enabled) -> should have 1 approved route
+				// Node 3: 10.0.0.0/8 (disabled) -> should have 0 approved routes
+				expectedRoutes := map[uint64][]netip.Prefix{
+					1: {netip.MustParsePrefix("0.0.0.0/0"), netip.MustParsePrefix("::/0")},
+					2: {netip.MustParsePrefix("192.168.100.0/24")},
+					3: nil,
+				}
+
+				actualRoutes := map[uint64][]netip.Prefix{
+					1: node1.ApprovedRoutes,
+					2: node2.ApprovedRoutes,
+					3: node3.ApprovedRoutes,
+				}
+
+				if diff := cmp.Diff(expectedRoutes, actualRoutes, util.PrefixComparer); diff != "" {
+					t.Errorf("TestSQLiteMigrationAndDataValidation() routes migration mismatch (-want +got):\n%s", diff)
+				}
+
+				// Verify pre_auth_keys data is preserved with specific key data
+				preAuthKeys, err := Read(hsdb.DB, func(rx *gorm.DB) ([]types.PreAuthKey, error) {
+					var keys []types.PreAuthKey
+					err := rx.Find(&keys).Error
+					return keys, err
+				})
+				require.NoError(t, err)
+				assert.Len(t, preAuthKeys, 2, "should preserve existing pre_auth_keys")
+
+				// Validate specific pre_auth_key data from dump file using cmp.Diff
+				expectedKeys := []types.PreAuthKey{
+					{
+						ID:       1,
+						Key:      "3d133ec953e31fd41edbd935371234f762b4bae300cea618",
+						UserID:   user2.ID,
+						Reusable: true,
+						Used:     true,
+					},
+					{
+						ID:       2,
+						Key:      "9813cc1df1832259fb6322dad788bb9bec89d8a01eef683a",
+						UserID:   user1.ID,
+						Reusable: true,
+						Used:     true,
+					},
+				}
+
+				if diff := cmp.Diff(expectedKeys, preAuthKeys,
+					cmpopts.IgnoreFields(types.PreAuthKey{}, "User", "CreatedAt", "Expiration", "Ephemeral", "Tags")); diff != "" {
+					t.Errorf("TestSQLiteMigrationAndDataValidation() pre_auth_keys mismatch (-want +got):\n%s", diff)
+				}
+
+				// Verify api_keys data is preserved with specific key data
+				var apiKeys []struct {
+					ID         uint64
+					Prefix     string
+					Hash       []byte
+					CreatedAt  string
+					Expiration string
+					LastSeen   string
+				}
+				err = hsdb.DB.Raw("SELECT id, prefix, hash, created_at, expiration, last_seen FROM api_keys").Scan(&apiKeys).Error
+				require.NoError(t, err)
+				assert.Len(t, apiKeys, 1, "should preserve existing api_keys")
+
+				// Validate specific api_key data from dump file using cmp.Diff
+				expectedAPIKey := struct {
+					ID     uint64
+					Prefix string
+					Hash   []byte
+				}{
+					ID:     1,
+					Prefix: "ak_test",
+					Hash:   []byte{0xde, 0xad, 0xbe, 0xef},
+				}
+
+				actualAPIKey := struct {
+					ID     uint64
+					Prefix string
+					Hash   []byte
+				}{
+					ID:     apiKeys[0].ID,
+					Prefix: apiKeys[0].Prefix,
+					Hash:   apiKeys[0].Hash,
+				}
+
+				if diff := cmp.Diff(expectedAPIKey, actualAPIKey); diff != "" {
+					t.Errorf("TestSQLiteMigrationAndDataValidation() api_key mismatch (-want +got):\n%s", diff)
+				}
+
+				// Validate date fields separately since they need Contains check
+				assert.Contains(t, apiKeys[0].CreatedAt, "2025-12-31", "created_at should be preserved")
+				assert.Contains(t, apiKeys[0].Expiration, "2025-06-18", "expiration should be preserved")
+
+				// Verify that routes table no longer exists (should have been dropped)
+				var routesTableExists bool
+				err = hsdb.DB.Raw("SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name='routes'").Row().Scan(&routesTableExists)
+				require.NoError(t, err)
+				assert.False(t, routesTableExists, "routes table should have been dropped")
+
+				// Verify all required indexes exist with correct structure using cmp.Diff
+				expectedIndexes := []string{
+					"idx_users_deleted_at",
+					"idx_provider_identifier",
+					"idx_name_provider_identifier",
+					"idx_name_no_provider_identifier",
+					"idx_api_keys_prefix",
+					"idx_policies_deleted_at",
+				}
+
+				expectedIndexMap := make(map[string]bool)
+				for _, index := range expectedIndexes {
+					expectedIndexMap[index] = true
+				}
+
+				actualIndexMap := make(map[string]bool)
+				for _, indexName := range expectedIndexes {
+					var indexExists bool
+					err = hsdb.DB.Raw("SELECT COUNT(*) FROM sqlite_master WHERE type='index' AND name=?", indexName).Row().Scan(&indexExists)
+					require.NoError(t, err)
+					actualIndexMap[indexName] = indexExists
+				}
+
+				if diff := cmp.Diff(expectedIndexMap, actualIndexMap); diff != "" {
+					t.Errorf("TestSQLiteMigrationAndDataValidation() indexes existence mismatch (-want +got):\n%s", diff)
+				}
+
+				// Verify proper foreign key constraints are set
+				// Check that pre_auth_keys has correct FK constraint (SET NULL, not CASCADE)
+				var preAuthKeyConstraint string
+				err = hsdb.DB.Raw("SELECT sql FROM sqlite_master WHERE type='table' AND name='pre_auth_keys'").Row().Scan(&preAuthKeyConstraint)
+				require.NoError(t, err)
+				assert.Contains(t, preAuthKeyConstraint, "ON DELETE SET NULL", "pre_auth_keys should have SET NULL constraint")
+				assert.NotContains(t, preAuthKeyConstraint, "ON DELETE CASCADE", "pre_auth_keys should not have CASCADE constraint")
+
+				// Verify that user unique constraints work properly
+				// Try to create duplicate local user (should fail)
+				err = hsdb.DB.Create(&types.User{Name: users[0].Name}).Error
+				require.Error(t, err, "should not allow duplicate local usernames")
+				assert.Contains(t, err.Error(), "UNIQUE constraint", "should fail with unique constraint error")
+			},
+		},
 	}
 
 	for _, tt := range tests {
 		t.Run(tt.dbPath, func(t *testing.T) {
-			dbPath, err := testCopyOfDatabase(t, tt.dbPath)
-			if err != nil {
-				t.Fatalf("copying db for test: %s", err)
-			}
-
-			hsdb, err := NewHeadscaleDatabase(types.DatabaseConfig{
-				Type: "sqlite3",
-				Sqlite: types.SqliteConfig{
-					Path: dbPath,
-				},
-			}, "", emptyCache())
-			if err != nil && tt.wantErr != err.Error() {
-				t.Errorf("TestMigrations() unexpected error = %v, wantErr %v", err, tt.wantErr)
+			if !strings.HasSuffix(tt.dbPath, ".sql") {
+				t.Fatalf("TestSQLiteMigrationAndDataValidation only supports .sql files, got: %s", tt.dbPath)
 			}
 
+			hsdb := dbForTestWithPath(t, tt.dbPath)
 			if tt.wantFunc != nil {
 				tt.wantFunc(t, hsdb)
 			}
@@ -266,39 +721,27 @@ func TestMigrationsSQLite(t *testing.T) {
 	}
 }
 
-func testCopyOfDatabase(t *testing.T, src string) (string, error) {
-	sourceFileStat, err := os.Stat(src)
-	if err != nil {
-		return "", err
-	}
-
-	if !sourceFileStat.Mode().IsRegular() {
-		return "", fmt.Errorf("%s is not a regular file", src)
-	}
-
-	source, err := os.Open(src)
-	if err != nil {
-		return "", err
-	}
-	defer source.Close()
-
-	tmpDir := t.TempDir()
-	fn := filepath.Base(src)
-	dst := filepath.Join(tmpDir, fn)
-
-	destination, err := os.Create(dst)
-	if err != nil {
-		return "", err
-	}
-	defer destination.Close()
-	_, err = io.Copy(destination, source)
-	return dst, err
-}
-
 func emptyCache() *zcache.Cache[types.RegistrationID, types.RegisterNode] {
 	return zcache.New[types.RegistrationID, types.RegisterNode](time.Minute, time.Hour)
 }
 
+func createSQLiteFromSQLFile(sqlFilePath, dbPath string) error {
+	db, err := sql.Open("sqlite", dbPath)
+	if err != nil {
+		return err
+	}
+	defer db.Close()
+
+	schemaContent, err := os.ReadFile(sqlFilePath)
+	if err != nil {
+		return err
+	}
+
+	_, err = db.Exec(string(schemaContent))
+
+	return err
+}
+
 // requireConstraintFailed checks if the error is a constraint failure with
 // either SQLite and PostgreSQL error messages.
 func requireConstraintFailed(t *testing.T, err error) {
@@ -415,7 +858,13 @@ func TestConstraints(t *testing.T) {
 	}
 }
 
-func TestMigrationsPostgres(t *testing.T) {
+// TestPostgresMigrationAndDataValidation tests specific PostgreSQL migration scenarios
+// and validates data integrity after migration. All migrations that require data validation
+// should be added here.
+//
+// TODO(kradalby): Convert to use plain text SQL dumps instead of binary .pssql dumps for consistency
+// with SQLite tests and easier version control.
+func TestPostgresMigrationAndDataValidation(t *testing.T) {
 	tests := []struct {
 		name     string
 		dbPath   string
@@ -423,9 +872,10 @@ func TestMigrationsPostgres(t *testing.T) {
 	}{
 		{
 			name:   "user-idx-breaking",
-			dbPath: "testdata/pre-24-postgresdb.pssql.dump",
-			wantFunc: func(t *testing.T, h *HSDatabase) {
-				users, err := Read(h.DB, func(rx *gorm.DB) ([]types.User, error) {
+			dbPath: "testdata/postgres/pre-24-postgresdb.pssql.dump",
+			wantFunc: func(t *testing.T, hsdb *HSDatabase) {
+				t.Helper()
+				users, err := Read(hsdb.DB, func(rx *gorm.DB) ([]types.User, error) {
 					return ListUsers(rx)
 				})
 				require.NoError(t, err)
@@ -472,9 +922,27 @@ func TestMigrationsPostgres(t *testing.T) {
 
 func dbForTest(t *testing.T) *HSDatabase {
 	t.Helper()
+	return dbForTestWithPath(t, "")
+}
+
+func dbForTestWithPath(t *testing.T, sqlFilePath string) *HSDatabase {
+	t.Helper()
 
 	dbPath := t.TempDir() + "/headscale_test.db"
 
+	// If SQL file path provided, validate and create database from it
+	if sqlFilePath != "" {
+		// Validate that the file is a SQL text file
+		if !strings.HasSuffix(sqlFilePath, ".sql") {
+			t.Fatalf("dbForTestWithPath only accepts .sql files, got: %s", sqlFilePath)
+		}
+
+		err := createSQLiteFromSQLFile(sqlFilePath, dbPath)
+		if err != nil {
+			t.Fatalf("setting up database from SQL file %s: %s", sqlFilePath, err)
+		}
+	}
+
 	db, err := NewHeadscaleDatabase(
 		types.DatabaseConfig{
 			Type: "sqlite3",
@@ -489,7 +957,59 @@ func dbForTest(t *testing.T) *HSDatabase {
 		t.Fatalf("setting up database: %s", err)
 	}
 
-	t.Logf("database set up at: %s", dbPath)
+	if sqlFilePath != "" {
+		t.Logf("database set up from %s at: %s", sqlFilePath, dbPath)
+	} else {
+		t.Logf("database set up at: %s", dbPath)
+	}
 
 	return db
 }
+
+// TestSQLiteAllTestdataMigrations tests migration compatibility across all SQLite schemas
+// in the testdata directory. It verifies they can be successfully migrated to the current
+// schema version. This test only validates migration success, not data integrity.
+//
+// A lot of the schemas have been automatically generated with old Headscale binaries on empty databases
+// (no user/node data):
+// - `headscale_<VERSION>_schema.sql` (created with `sqlite3 headscale.db .schema`)
+// - `headscale_<VERSION>_dump.sql` (created with `sqlite3 headscale.db .dump`)
+// where `_dump.sql` contains the migration steps that have been applied to the database.
+func TestSQLiteAllTestdataMigrations(t *testing.T) {
+	t.Parallel()
+	schemas, err := os.ReadDir("testdata/sqlite")
+	require.NoError(t, err)
+
+	t.Logf("loaded %d schemas", len(schemas))
+
+	for _, schema := range schemas {
+		if schema.IsDir() {
+			continue
+		}
+
+		t.Logf("validating: %s", schema.Name())
+
+		t.Run(schema.Name(), func(t *testing.T) {
+			t.Parallel()
+
+			dbPath := t.TempDir() + "/headscale_test.db"
+
+			// Setup a database with the old schema
+			schemaPath := filepath.Join("testdata/sqlite", schema.Name())
+			err := createSQLiteFromSQLFile(schemaPath, dbPath)
+			require.NoError(t, err)
+
+			_, err = NewHeadscaleDatabase(
+				types.DatabaseConfig{
+					Type: "sqlite3",
+					Sqlite: types.SqliteConfig{
+						Path: dbPath,
+					},
+				},
+				"",
+				emptyCache(),
+			)
+			require.NoError(t, err)
+		})
+	}
+}

hscontrol/db/ephemeral_garbage_collector_test.go

@@ -11,9 +11,11 @@ import (
 	"github.com/stretchr/testify/assert"
 )
 
-const fiveHundredMillis = 500 * time.Millisecond
-const oneHundredMillis = 100 * time.Millisecond
-const fiftyMillis = 50 * time.Millisecond
+const (
+	fiveHundred = 500 * time.Millisecond
+	oneHundred  = 100 * time.Millisecond
+	fifty       = 50 * time.Millisecond
+)
 
 // TestEphemeralGarbageCollectorGoRoutineLeak is a test for a goroutine leak in EphemeralGarbageCollector().
 // It creates a new EphemeralGarbageCollector, schedules several nodes for deletion with a short expiry,
@@ -41,7 +43,7 @@ func TestEphemeralGarbageCollectorGoRoutineLeak(t *testing.T) {
 	go gc.Start()
 
 	// Schedule several nodes for deletion with short expiry
-	const expiry = fiftyMillis
+	const expiry = fifty
 	const numNodes = 100
 
 	// Set up wait group for expected deletions
@@ -56,7 +58,7 @@ func TestEphemeralGarbageCollectorGoRoutineLeak(t *testing.T) {
 
 	// Check nodes are deleted
 	deleteMutex.Lock()
-	assert.Equal(t, numNodes, len(deletedIDs), "Not all nodes were deleted")
+	assert.Len(t, deletedIDs, numNodes, "Not all nodes were deleted")
 	deleteMutex.Unlock()
 
 	// Schedule and immediately cancel to test that part of the code
@@ -76,7 +78,7 @@ func TestEphemeralGarbageCollectorGoRoutineLeak(t *testing.T) {
 
 		// Give any potential leaked goroutines a chance to exit
 		// Still need a small sleep here as we're checking for absence of goroutines
-		time.Sleep(oneHundredMillis)
+		time.Sleep(oneHundred)
 
 		// Check for leaked goroutines
 		finalGoroutines := runtime.NumGoroutine()
@@ -112,7 +114,7 @@ func TestEphemeralGarbageCollectorReschedule(t *testing.T) {
 	go gc.Start()
 	defer gc.Close()
 
-	const shortExpiry = fiftyMillis
+	const shortExpiry = fifty
 	const longExpiry = 1 * time.Hour
 
 	nodeID := types.NodeID(1)
@@ -128,7 +130,7 @@ func TestEphemeralGarbageCollectorReschedule(t *testing.T) {
 
 	// Verify that the node was deleted once
 	deleteMutex.Lock()
-	assert.Equal(t, 1, len(deletedIDs), "Node should be deleted exactly once")
+	assert.Len(t, deletedIDs, 1, "Node should be deleted exactly once")
 	assert.Equal(t, nodeID, deletedIDs[0], "The correct node should be deleted")
 	deleteMutex.Unlock()
 }
@@ -155,7 +157,7 @@ func TestEphemeralGarbageCollectorCancelAndReschedule(t *testing.T) {
 	defer gc.Close()
 
 	nodeID := types.NodeID(1)
-	const expiry = fiftyMillis
+	const expiry = fifty
 
 	// Schedule node for deletion
 	gc.Schedule(nodeID, expiry)
@@ -172,7 +174,7 @@ func TestEphemeralGarbageCollectorCancelAndReschedule(t *testing.T) {
 	}
 
 	deleteMutex.Lock()
-	assert.Equal(t, 0, len(deletedIDs), "Node should not be deleted after cancellation")
+	assert.Empty(t, deletedIDs, "Node should not be deleted after cancellation")
 	deleteMutex.Unlock()
 
 	// Reschedule the node
@@ -189,7 +191,7 @@ func TestEphemeralGarbageCollectorCancelAndReschedule(t *testing.T) {
 
 	// Verify final state
 	deleteMutex.Lock()
-	assert.Equal(t, 1, len(deletedIDs), "Node should be deleted after rescheduling")
+	assert.Len(t, deletedIDs, 1, "Node should be deleted after rescheduling")
 	assert.Equal(t, nodeID, deletedIDs[0], "The correct node should be deleted")
 	deleteMutex.Unlock()
 }
@@ -212,7 +214,7 @@ func TestEphemeralGarbageCollectorCloseBeforeTimerFires(t *testing.T) {
 	go gc.Start()
 
 	const longExpiry = 1 * time.Hour
-	const shortExpiry = fiftyMillis
+	const shortExpiry = fifty
 
 	// Schedule node deletion with a long expiry
 	gc.Schedule(types.NodeID(1), longExpiry)
@@ -225,7 +227,7 @@ func TestEphemeralGarbageCollectorCloseBeforeTimerFires(t *testing.T) {
 
 	// Verify that no deletion occurred
 	deleteMutex.Lock()
-	assert.Equal(t, 0, len(deletedIDs), "No node should be deleted when GC is closed before timer fires")
+	assert.Empty(t, deletedIDs, "No node should be deleted when GC is closed before timer fires")
 	deleteMutex.Unlock()
 }
 
@@ -269,7 +271,7 @@ func TestEphemeralGarbageCollectorScheduleAfterClose(t *testing.T) {
 		// Give the GC time to fully close and clean up resources
 		// This is still time-based but only affects when we check the goroutine count,
 		// not the actual test logic
-		time.Sleep(oneHundredMillis)
+		time.Sleep(oneHundred)
 		close(gcClosedCheck)
 	}()
 
@@ -279,7 +281,7 @@ func TestEphemeralGarbageCollectorScheduleAfterClose(t *testing.T) {
 	gc.Schedule(nodeID, 1*time.Millisecond)
 
 	// Set up a timeout channel for our test
-	timeout := time.After(fiveHundredMillis)
+	timeout := time.After(fiveHundred)
 
 	// Check if any node was deleted (which shouldn't happen)
 	select {
@@ -329,7 +331,7 @@ func TestEphemeralGarbageCollectorConcurrentScheduleAndClose(t *testing.T) {
 	// Number of concurrent scheduling goroutines
 	const numSchedulers = 10
 	const nodesPerScheduler = 50
-	const schedulingDuration = fiveHundredMillis
+	const schedulingDuration = fiveHundred
 
 	// Use WaitGroup to wait for all scheduling goroutines to finish
 	var wg sync.WaitGroup
@@ -339,14 +341,14 @@ func TestEphemeralGarbageCollectorConcurrentScheduleAndClose(t *testing.T) {
 	stopScheduling := make(chan struct{})
 
 	// Launch goroutines that continuously schedule nodes
-	for i := 0; i < numSchedulers; i++ {
+	for schedulerIndex := range numSchedulers {
 		go func(schedulerID int) {
 			defer wg.Done()
 
 			baseNodeID := schedulerID * nodesPerScheduler
 
 			// Keep scheduling nodes until signaled to stop
-			for j := 0; j < nodesPerScheduler; j++ {
+			for j := range nodesPerScheduler {
 				select {
 				case <-stopScheduling:
 					return
@@ -358,7 +360,7 @@ func TestEphemeralGarbageCollectorConcurrentScheduleAndClose(t *testing.T) {
 					time.Sleep(time.Duration(rand.Intn(5)) * time.Millisecond)
 				}
 			}
-		}(i)
+		}(schedulerIndex)
 	}
 
 	// After a short delay, close the garbage collector while schedulers are still running
@@ -377,7 +379,7 @@ func TestEphemeralGarbageCollectorConcurrentScheduleAndClose(t *testing.T) {
 	wg.Wait()
 
 	// Wait a bit longer to allow any leaked goroutines to do their work
-	time.Sleep(oneHundredMillis)
+	time.Sleep(oneHundred)
 
 	// Check for leaks
 	finalGoroutines := runtime.NumGoroutine()

hscontrol/db/ip.go

@@ -17,6 +17,8 @@ import (
 	"tailscale.com/net/tsaddr"
 )
 
+var errGeneratedIPBytesInvalid = errors.New("generated ip bytes are invalid ip")
+
 // IPAllocator is a singleton responsible for allocating
 // IP addresses for nodes and making sure the same
 // address is not handed out twice. There can only be one
@@ -236,7 +238,7 @@ func randomNext(pfx netip.Prefix) (netip.Addr, error) {
 
 	ip, ok := netip.AddrFromSlice(valInRange.Bytes())
 	if !ok {
-		return netip.Addr{}, fmt.Errorf("generated ip bytes are invalid ip")
+		return netip.Addr{}, errGeneratedIPBytesInvalid
 	}
 
 	if !pfx.Contains(ip) {
@@ -273,7 +275,7 @@ func (db *HSDatabase) BackfillNodeIPs(i *IPAllocator) ([]string, error) {
 			return errors.New("backfilling IPs: ip allocator was nil")
 		}
 
-		log.Trace().Msgf("starting to backfill IPs")
+		log.Trace().Caller().Msgf("starting to backfill IPs")
 
 		nodes, err := ListNodes(tx)
 		if err != nil {
@@ -281,7 +283,7 @@ func (db *HSDatabase) BackfillNodeIPs(i *IPAllocator) ([]string, error) {
 		}
 
 		for _, node := range nodes {
-			log.Trace().Uint64("node.id", node.ID.Uint64()).Msg("checking if need backfill")
+			log.Trace().Caller().Uint64("node.id", node.ID.Uint64()).Str("node.name", node.Hostname).Msg("IP backfill check started because node found in database")
 
 			changed := false
 			// IPv4 prefix is set, but node ip is missing, alloc

hscontrol/db/node.go

@@ -5,17 +5,21 @@ import (
 	"errors"
 	"fmt"
 	"net/netip"
+	"regexp"
 	"slices"
 	"sort"
+	"strconv"
+	"strings"
 	"sync"
+	"testing"
 	"time"
 
 	"github.com/juanfont/headscale/hscontrol/types"
 	"github.com/juanfont/headscale/hscontrol/util"
 	"github.com/rs/zerolog/log"
 	"gorm.io/gorm"
-	"tailscale.com/tailcfg"
 	"tailscale.com/types/key"
+	"tailscale.com/types/ptr"
 )
 
 const (
@@ -23,6 +27,10 @@ const (
 	NodeGivenNameTrimSize   = 2
 )
 
+var (
+	invalidDNSRegex = regexp.MustCompile("[^a-z0-9-.]+")
+)
+
 var (
 	ErrNodeNotFound                  = errors.New("node not found")
 	ErrNodeRouteIsNotAvailable       = errors.New("route is not available on node")
@@ -30,18 +38,13 @@ var (
 		"node not found in registration cache",
 	)
 	ErrCouldNotConvertNodeInterface = errors.New("failed to convert node interface")
-	ErrDifferentRegisteredUser      = errors.New(
-		"node was previously registered with a different user",
-	)
 )
 
 // ListPeers returns peers of node, regardless of any Policy or if the node is expired.
 // If no peer IDs are given, all peers are returned.
 // If at least one peer ID is given, only these peer nodes will be returned.
 func (hsdb *HSDatabase) ListPeers(nodeID types.NodeID, peerIDs ...types.NodeID) (types.Nodes, error) {
-	return Read(hsdb.DB, func(rx *gorm.DB) (types.Nodes, error) {
-		return ListPeers(rx, nodeID, peerIDs...)
-	})
+	return ListPeers(hsdb.DB, nodeID, peerIDs...)
 }
 
 // ListPeers returns peers of node, regardless of any Policy or if the node is expired.
@@ -64,15 +67,13 @@ func ListPeers(tx *gorm.DB, nodeID types.NodeID, peerIDs ...types.NodeID) (types
 }
 
 // ListNodes queries the database for either all nodes if no parameters are given
-// or for the given nodes if at least one node ID is given as parameter
+// or for the given nodes if at least one node ID is given as parameter.
 func (hsdb *HSDatabase) ListNodes(nodeIDs ...types.NodeID) (types.Nodes, error) {
-	return Read(hsdb.DB, func(rx *gorm.DB) (types.Nodes, error) {
-		return ListNodes(rx, nodeIDs...)
-	})
+	return ListNodes(hsdb.DB, nodeIDs...)
 }
 
 // ListNodes queries the database for either all nodes if no parameters are given
-// or for the given nodes if at least one node ID is given as parameter
+// or for the given nodes if at least one node ID is given as parameter.
 func ListNodes(tx *gorm.DB, nodeIDs ...types.NodeID) (types.Nodes, error) {
 	nodes := types.Nodes{}
 	if err := tx.
@@ -120,9 +121,7 @@ func getNode(tx *gorm.DB, uid types.UserID, name string) (*types.Node, error) {
 }
 
 func (hsdb *HSDatabase) GetNodeByID(id types.NodeID) (*types.Node, error) {
-	return Read(hsdb.DB, func(rx *gorm.DB) (*types.Node, error) {
-		return GetNodeByID(rx, id)
-	})
+	return GetNodeByID(hsdb.DB, id)
 }
 
 // GetNodeByID finds a Node by ID and returns the Node struct.
@@ -140,9 +139,7 @@ func GetNodeByID(tx *gorm.DB, id types.NodeID) (*types.Node, error) {
 }
 
 func (hsdb *HSDatabase) GetNodeByMachineKey(machineKey key.MachinePublic) (*types.Node, error) {
-	return Read(hsdb.DB, func(rx *gorm.DB) (*types.Node, error) {
-		return GetNodeByMachineKey(rx, machineKey)
-	})
+	return GetNodeByMachineKey(hsdb.DB, machineKey)
 }
 
 // GetNodeByMachineKey finds a Node by its MachineKey and returns the Node struct.
@@ -163,9 +160,7 @@ func GetNodeByMachineKey(
 }
 
 func (hsdb *HSDatabase) GetNodeByNodeKey(nodeKey key.NodePublic) (*types.Node, error) {
-	return Read(hsdb.DB, func(rx *gorm.DB) (*types.Node, error) {
-		return GetNodeByNodeKey(rx, nodeKey)
-	})
+	return GetNodeByNodeKey(hsdb.DB, nodeKey)
 }
 
 // GetNodeByNodeKey finds a Node by its NodeKey and returns the Node struct.
@@ -266,24 +261,22 @@ func SetLastSeen(tx *gorm.DB, nodeID types.NodeID, lastSeen time.Time) error {
 }
 
 // RenameNode takes a Node struct and a new GivenName for the nodes
-// and renames it. If the name is not unique, it will return an error.
+// and renames it. Validation should be done in the state layer before calling this function.
 func RenameNode(tx *gorm.DB,
 	nodeID types.NodeID, newName string,
 ) error {
-	err := util.CheckForFQDNRules(
-		newName,
-	)
-	if err != nil {
+	if err := util.ValidateHostname(newName); err != nil {
 		return fmt.Errorf("renaming node: %w", err)
 	}
 
-	uniq, err := isUniqueName(tx, newName)
-	if err != nil {
-		return fmt.Errorf("checking if name is unique: %w", err)
+	// Check if the new name is unique
+	var count int64
+	if err := tx.Model(&types.Node{}).Where("given_name = ? AND id != ?", newName, nodeID).Count(&count).Error; err != nil {
+		return fmt.Errorf("failed to check name uniqueness: %w", err)
 	}
 
-	if !uniq {
-		return fmt.Errorf("name is not unique: %s", newName)
+	if count > 0 {
+		return errors.New("name is not unique")
 	}
 
 	if err := tx.Model(&types.Node{}).Where("id = ?", nodeID).Update("given_name", newName).Error; err != nil {
@@ -339,104 +332,19 @@ func (hsdb *HSDatabase) DeleteEphemeralNode(
 	})
 }
 
-// HandleNodeFromAuthPath is called from the OIDC or CLI auth path
-// with a registrationID to register or reauthenticate a node.
-// If the node found in the registration cache is not already registered,
-// it will be registered with the user and the node will be removed from the cache.
-// If the node is already registered, the expiry will be updated.
-// The node, and a boolean indicating if it was a new node or not, will be returned.
-func (hsdb *HSDatabase) HandleNodeFromAuthPath(
-	registrationID types.RegistrationID,
-	userID types.UserID,
-	nodeExpiry *time.Time,
-	registrationMethod string,
-	ipv4 *netip.Addr,
-	ipv6 *netip.Addr,
-) (*types.Node, bool, error) {
-	var newNode bool
-	node, err := Write(hsdb.DB, func(tx *gorm.DB) (*types.Node, error) {
-		if reg, ok := hsdb.regCache.Get(registrationID); ok {
-			if node, _ := GetNodeByNodeKey(tx, reg.Node.NodeKey); node == nil {
-				user, err := GetUserByID(tx, userID)
-				if err != nil {
-					return nil, fmt.Errorf(
-						"failed to find user in register node from auth callback, %w",
-						err,
-					)
-				}
+// RegisterNodeForTest is used only for testing purposes to register a node directly in the database.
+// Production code should use state.HandleNodeFromAuthPath or state.HandleNodeFromPreAuthKey.
+func RegisterNodeForTest(tx *gorm.DB, node types.Node, ipv4 *netip.Addr, ipv6 *netip.Addr) (*types.Node, error) {
+	if !testing.Testing() {
+		panic("RegisterNodeForTest can only be called during tests")
+	}
 
-				log.Debug().
-					Str("registration_id", registrationID.String()).
-					Str("username", user.Username()).
-					Str("registrationMethod", registrationMethod).
-					Str("expiresAt", fmt.Sprintf("%v", nodeExpiry)).
-					Msg("Registering node from API/CLI or auth callback")
-
-				// TODO(kradalby): This looks quite wrong? why ID 0?
-				// Why not always?
-				// Registration of expired node with different user
-				if reg.Node.ID != 0 &&
-					reg.Node.UserID != user.ID {
-					return nil, ErrDifferentRegisteredUser
-				}
-
-				reg.Node.UserID = user.ID
-				reg.Node.User = *user
-				reg.Node.RegisterMethod = registrationMethod
-
-				if nodeExpiry != nil {
-					reg.Node.Expiry = nodeExpiry
-				}
-
-				node, err := RegisterNode(
-					tx,
-					reg.Node,
-					ipv4, ipv6,
-				)
-
-				if err == nil {
-					hsdb.regCache.Delete(registrationID)
-				}
-
-				// Signal to waiting clients that the machine has been registered.
-				select {
-				case reg.Registered <- node:
-				default:
-				}
-				close(reg.Registered)
-
-				newNode = true
-				return node, err
-			} else {
-				// If the node is already registered, this is a refresh.
-				err := NodeSetExpiry(tx, node.ID, *nodeExpiry)
-				if err != nil {
-					return nil, err
-				}
-				return node, nil
-			}
-		}
-
-		return nil, ErrNodeNotFoundRegistrationCache
-	})
-
-	return node, newNode, err
-}
-
-func (hsdb *HSDatabase) RegisterNode(node types.Node, ipv4 *netip.Addr, ipv6 *netip.Addr) (*types.Node, error) {
-	return Write(hsdb.DB, func(tx *gorm.DB) (*types.Node, error) {
-		return RegisterNode(tx, node, ipv4, ipv6)
-	})
-}
-
-// RegisterNode is executed from the CLI to register a new Node using its MachineKey.
-func RegisterNode(tx *gorm.DB, node types.Node, ipv4 *netip.Addr, ipv6 *netip.Addr) (*types.Node, error) {
 	log.Debug().
 		Str("node", node.Hostname).
 		Str("machine_key", node.MachineKey.ShortString()).
 		Str("node_key", node.NodeKey.ShortString()).
 		Str("user", node.User.Username()).
-		Msg("Registering node")
+		Msg("Registering test node")
 
 	// If the a new node is registered with the same machine key, to the same user,
 	// update the existing node.
@@ -446,8 +354,14 @@ func RegisterNode(tx *gorm.DB, node types.Node, ipv4 *netip.Addr, ipv6 *netip.Ad
 	if oldNode != nil && oldNode.UserID == node.UserID {
 		node.ID = oldNode.ID
 		node.GivenName = oldNode.GivenName
-		ipv4 = oldNode.IPv4
-		ipv6 = oldNode.IPv6
+		node.ApprovedRoutes = oldNode.ApprovedRoutes
+		// Don't overwrite the provided IPs with old ones when they exist
+		if ipv4 == nil {
+			ipv4 = oldNode.IPv4
+		}
+		if ipv6 == nil {
+			ipv6 = oldNode.IPv6
+		}
 	}
 
 	// If the node exists and it already has IP(s), we just save it
@@ -464,7 +378,7 @@ func RegisterNode(tx *gorm.DB, node types.Node, ipv4 *netip.Addr, ipv6 *netip.Ad
 			Str("machine_key", node.MachineKey.ShortString()).
 			Str("node_key", node.NodeKey.ShortString()).
 			Str("user", node.User.Username()).
-			Msg("Node authorized again")
+			Msg("Test node authorized again")
 
 		return &node, nil
 	}
@@ -472,8 +386,16 @@ func RegisterNode(tx *gorm.DB, node types.Node, ipv4 *netip.Addr, ipv6 *netip.Ad
 	node.IPv4 = ipv4
 	node.IPv6 = ipv6
 
+	var err error
+	node.Hostname, err = util.NormaliseHostname(node.Hostname)
+	if err != nil {
+		newHostname := util.InvalidString()
+		log.Info().Err(err).Str("invalid-hostname", node.Hostname).Str("new-hostname", newHostname).Msgf("Invalid hostname, replacing")
+		node.Hostname = newHostname
+	}
+
 	if node.GivenName == "" {
-		givenName, err := ensureUniqueGivenName(tx, node.Hostname)
+		givenName, err := EnsureUniqueGivenName(tx, node.Hostname)
 		if err != nil {
 			return nil, fmt.Errorf("failed to ensure unique given name: %w", err)
 		}
@@ -488,7 +410,7 @@ func RegisterNode(tx *gorm.DB, node types.Node, ipv4 *netip.Addr, ipv6 *netip.Ad
 	log.Trace().
 		Caller().
 		Str("node", node.Hostname).
-		Msg("Node registered with the database")
+		Msg("Test node registered with the database")
 
 	return &node, nil
 }
@@ -528,7 +450,10 @@ func NodeSave(tx *gorm.DB, node *types.Node) error {
 }
 
 func generateGivenName(suppliedName string, randomSuffix bool) (string, error) {
-	suppliedName = util.ConvertWithFQDNRules(suppliedName)
+	// Strip invalid DNS characters for givenName
+	suppliedName = strings.ToLower(suppliedName)
+	suppliedName = invalidDNSRegex.ReplaceAllString(suppliedName, "")
+
 	if len(suppliedName) > util.LabelHostnameLength {
 		return "", types.ErrHostnameTooLong
 	}
@@ -561,7 +486,8 @@ func isUniqueName(tx *gorm.DB, name string) (bool, error) {
 	return len(nodes) == 0, nil
 }
 
-func ensureUniqueGivenName(
+// EnsureUniqueGivenName generates a unique given name for a node based on its hostname.
+func EnsureUniqueGivenName(
 	tx *gorm.DB,
 	name string,
 ) (string, error) {
@@ -587,36 +513,6 @@ func ensureUniqueGivenName(
 	return givenName, nil
 }
 
-func ExpireExpiredNodes(tx *gorm.DB,
-	lastCheck time.Time,
-) (time.Time, types.StateUpdate, bool) {
-	// use the time of the start of the function to ensure we
-	// dont miss some nodes by returning it _after_ we have
-	// checked everything.
-	started := time.Now()
-
-	expired := make([]*tailcfg.PeerChange, 0)
-
-	nodes, err := ListNodes(tx)
-	if err != nil {
-		return time.Unix(0, 0), types.StateUpdate{}, false
-	}
-	for _, node := range nodes {
-		if node.IsExpired() && node.Expiry.After(lastCheck) {
-			expired = append(expired, &tailcfg.PeerChange{
-				NodeID:    tailcfg.NodeID(node.ID),
-				KeyExpiry: node.Expiry,
-			})
-		}
-	}
-
-	if len(expired) > 0 {
-		return started, types.UpdatePeerPatch(expired...), true
-	}
-
-	return started, types.StateUpdate{}, false
-}
-
 // EphemeralGarbageCollector is a garbage collector that will delete nodes after
 // a certain amount of time.
 // It is used to delete ephemeral nodes that have disconnected and should be
@@ -727,3 +623,138 @@ func (e *EphemeralGarbageCollector) Start() {
 		}
 	}
 }
+
+func (hsdb *HSDatabase) CreateNodeForTest(user *types.User, hostname ...string) *types.Node {
+	if !testing.Testing() {
+		panic("CreateNodeForTest can only be called during tests")
+	}
+
+	if user == nil {
+		panic("CreateNodeForTest requires a valid user")
+	}
+
+	nodeName := "testnode"
+	if len(hostname) > 0 && hostname[0] != "" {
+		nodeName = hostname[0]
+	}
+
+	// Create a preauth key for the node
+	pak, err := hsdb.CreatePreAuthKey(types.UserID(user.ID), false, false, nil, nil)
+	if err != nil {
+		panic(fmt.Sprintf("failed to create preauth key for test node: %v", err))
+	}
+
+	nodeKey := key.NewNode()
+	machineKey := key.NewMachine()
+	discoKey := key.NewDisco()
+
+	node := &types.Node{
+		MachineKey:     machineKey.Public(),
+		NodeKey:        nodeKey.Public(),
+		DiscoKey:       discoKey.Public(),
+		Hostname:       nodeName,
+		UserID:         user.ID,
+		RegisterMethod: util.RegisterMethodAuthKey,
+		AuthKeyID:      ptr.To(pak.ID),
+	}
+
+	err = hsdb.DB.Save(node).Error
+	if err != nil {
+		panic(fmt.Sprintf("failed to create test node: %v", err))
+	}
+
+	return node
+}
+
+func (hsdb *HSDatabase) CreateRegisteredNodeForTest(user *types.User, hostname ...string) *types.Node {
+	if !testing.Testing() {
+		panic("CreateRegisteredNodeForTest can only be called during tests")
+	}
+
+	node := hsdb.CreateNodeForTest(user, hostname...)
+
+	// Allocate IPs for the test node using the database's IP allocator
+	// This is a simplified allocation for testing - in production this would use State.ipAlloc
+	ipv4, ipv6, err := hsdb.allocateTestIPs(node.ID)
+	if err != nil {
+		panic(fmt.Sprintf("failed to allocate IPs for test node: %v", err))
+	}
+
+	var registeredNode *types.Node
+	err = hsdb.DB.Transaction(func(tx *gorm.DB) error {
+		var err error
+		registeredNode, err = RegisterNodeForTest(tx, *node, ipv4, ipv6)
+		return err
+	})
+	if err != nil {
+		panic(fmt.Sprintf("failed to register test node: %v", err))
+	}
+
+	return registeredNode
+}
+
+func (hsdb *HSDatabase) CreateNodesForTest(user *types.User, count int, hostnamePrefix ...string) []*types.Node {
+	if !testing.Testing() {
+		panic("CreateNodesForTest can only be called during tests")
+	}
+
+	if user == nil {
+		panic("CreateNodesForTest requires a valid user")
+	}
+
+	prefix := "testnode"
+	if len(hostnamePrefix) > 0 && hostnamePrefix[0] != "" {
+		prefix = hostnamePrefix[0]
+	}
+
+	nodes := make([]*types.Node, count)
+	for i := range count {
+		hostname := prefix + "-" + strconv.Itoa(i)
+		nodes[i] = hsdb.CreateNodeForTest(user, hostname)
+	}
+
+	return nodes
+}
+
+func (hsdb *HSDatabase) CreateRegisteredNodesForTest(user *types.User, count int, hostnamePrefix ...string) []*types.Node {
+	if !testing.Testing() {
+		panic("CreateRegisteredNodesForTest can only be called during tests")
+	}
+
+	if user == nil {
+		panic("CreateRegisteredNodesForTest requires a valid user")
+	}
+
+	prefix := "testnode"
+	if len(hostnamePrefix) > 0 && hostnamePrefix[0] != "" {
+		prefix = hostnamePrefix[0]
+	}
+
+	nodes := make([]*types.Node, count)
+	for i := range count {
+		hostname := prefix + "-" + strconv.Itoa(i)
+		nodes[i] = hsdb.CreateRegisteredNodeForTest(user, hostname)
+	}
+
+	return nodes
+}
+
+// allocateTestIPs allocates sequential test IPs for nodes during testing.
+func (hsdb *HSDatabase) allocateTestIPs(nodeID types.NodeID) (*netip.Addr, *netip.Addr, error) {
+	if !testing.Testing() {
+		panic("allocateTestIPs can only be called during tests")
+	}
+
+	// Use simple sequential allocation for tests
+	// IPv4: 100.64.0.x (where x is nodeID)
+	// IPv6: fd7a:115c:a1e0::x (where x is nodeID)
+
+	if nodeID > 254 {
+		return nil, nil, fmt.Errorf("test node ID %d too large for simple IP allocation", nodeID)
+	}
+
+	ipv4 := netip.AddrFrom4([4]byte{100, 64, 0, byte(nodeID)})
+	ipv6 := netip.AddrFrom16([16]byte{0xfd, 0x7a, 0x11, 0x5c, 0xa1, 0xe0, 0, 0, 0, 0, 0, 0, 0, 0, 0, byte(nodeID)})
+
+	return &ipv4, &ipv6, nil
+}

hscontrol/db/node_test.go

@@ -6,7 +6,6 @@ import (
 	"math/big"
 	"net/netip"
 	"regexp"
-	"strconv"
 	"sync"
 	"testing"
 	"time"
@@ -26,82 +25,36 @@ import (
 )
 
 func (s *Suite) TestGetNode(c *check.C) {
-	user, err := db.CreateUser(types.User{Name: "test"})
-	c.Assert(err, check.IsNil)
+	user := db.CreateUserForTest("test")
 
-	pak, err := db.CreatePreAuthKey(types.UserID(user.ID), false, false, nil, nil)
-	c.Assert(err, check.IsNil)
-
-	_, err = db.getNode(types.UserID(user.ID), "testnode")
+	_, err := db.getNode(types.UserID(user.ID), "testnode")
 	c.Assert(err, check.NotNil)
 
-	nodeKey := key.NewNode()
-	machineKey := key.NewMachine()
-
-	node := &types.Node{
-		ID:             0,
-		MachineKey:     machineKey.Public(),
-		NodeKey:        nodeKey.Public(),
-		Hostname:       "testnode",
-		UserID:         user.ID,
-		RegisterMethod: util.RegisterMethodAuthKey,
-		AuthKeyID:      ptr.To(pak.ID),
-	}
-	trx := db.DB.Save(node)
-	c.Assert(trx.Error, check.IsNil)
+	node := db.CreateNodeForTest(user, "testnode")
 
 	_, err = db.getNode(types.UserID(user.ID), "testnode")
 	c.Assert(err, check.IsNil)
+	c.Assert(node.Hostname, check.Equals, "testnode")
 }
 
 func (s *Suite) TestGetNodeByID(c *check.C) {
-	user, err := db.CreateUser(types.User{Name: "test"})
-	c.Assert(err, check.IsNil)
+	user := db.CreateUserForTest("test")
 
-	pak, err := db.CreatePreAuthKey(types.UserID(user.ID), false, false, nil, nil)
-	c.Assert(err, check.IsNil)
-
-	_, err = db.GetNodeByID(0)
+	_, err := db.GetNodeByID(0)
 	c.Assert(err, check.NotNil)
 
-	nodeKey := key.NewNode()
-	machineKey := key.NewMachine()
+	node := db.CreateNodeForTest(user, "testnode")
 
-	node := types.Node{
-		ID:             0,
-		MachineKey:     machineKey.Public(),
-		NodeKey:        nodeKey.Public(),
-		Hostname:       "testnode",
-		UserID:         user.ID,
-		RegisterMethod: util.RegisterMethodAuthKey,
-		AuthKeyID:      ptr.To(pak.ID),
-	}
-	trx := db.DB.Save(&node)
-	c.Assert(trx.Error, check.IsNil)
-
-	_, err = db.GetNodeByID(0)
+	retrievedNode, err := db.GetNodeByID(node.ID)
 	c.Assert(err, check.IsNil)
+	c.Assert(retrievedNode.Hostname, check.Equals, "testnode")
 }
 
 func (s *Suite) TestHardDeleteNode(c *check.C) {
-	user, err := db.CreateUser(types.User{Name: "test"})
-	c.Assert(err, check.IsNil)
+	user := db.CreateUserForTest("test")
+	node := db.CreateNodeForTest(user, "testnode3")
 
-	nodeKey := key.NewNode()
-	machineKey := key.NewMachine()
-
-	node := types.Node{
-		ID:             0,
-		MachineKey:     machineKey.Public(),
-		NodeKey:        nodeKey.Public(),
-		Hostname:       "testnode3",
-		UserID:         user.ID,
-		RegisterMethod: util.RegisterMethodAuthKey,
-	}
-	trx := db.DB.Save(&node)
-	c.Assert(trx.Error, check.IsNil)
-
-	err = db.DeleteNode(&node)
+	err := db.DeleteNode(node)
 	c.Assert(err, check.IsNil)
 
 	_, err = db.getNode(types.UserID(user.ID), "testnode3")
@@ -109,42 +62,21 @@ func (s *Suite) TestHardDeleteNode(c *check.C) {
 }
 
 func (s *Suite) TestListPeers(c *check.C) {
-	user, err := db.CreateUser(types.User{Name: "test"})
-	c.Assert(err, check.IsNil)
+	user := db.CreateUserForTest("test")
 
-	pak, err := db.CreatePreAuthKey(types.UserID(user.ID), false, false, nil, nil)
-	c.Assert(err, check.IsNil)
-
-	_, err = db.GetNodeByID(0)
+	_, err := db.GetNodeByID(0)
 	c.Assert(err, check.NotNil)
 
-	for index := 0; index <= 10; index++ {
-		nodeKey := key.NewNode()
-		machineKey := key.NewMachine()
+	nodes := db.CreateNodesForTest(user, 11, "testnode")
 
-		node := types.Node{
-			ID:             types.NodeID(index),
-			MachineKey:     machineKey.Public(),
-			NodeKey:        nodeKey.Public(),
-			Hostname:       "testnode" + strconv.Itoa(index),
-			UserID:         user.ID,
-			RegisterMethod: util.RegisterMethodAuthKey,
-			AuthKeyID:      ptr.To(pak.ID),
-		}
-		trx := db.DB.Save(&node)
-		c.Assert(trx.Error, check.IsNil)
-	}
-
-	node0ByID, err := db.GetNodeByID(0)
+	firstNode := nodes[0]
+	peersOfFirstNode, err := db.ListPeers(firstNode.ID)
 	c.Assert(err, check.IsNil)
 
-	peersOfNode0, err := db.ListPeers(node0ByID.ID)
-	c.Assert(err, check.IsNil)
-
-	c.Assert(len(peersOfNode0), check.Equals, 9)
-	c.Assert(peersOfNode0[0].Hostname, check.Equals, "testnode2")
-	c.Assert(peersOfNode0[5].Hostname, check.Equals, "testnode7")
-	c.Assert(peersOfNode0[8].Hostname, check.Equals, "testnode10")
+	c.Assert(len(peersOfFirstNode), check.Equals, 10)
+	c.Assert(peersOfFirstNode[0].Hostname, check.Equals, "testnode-1")
+	c.Assert(peersOfFirstNode[5].Hostname, check.Equals, "testnode-6")
+	c.Assert(peersOfFirstNode[9].Hostname, check.Equals, "testnode-10")
 }
 
 func (s *Suite) TestExpireNode(c *check.C) {
@@ -360,12 +292,57 @@ func TestHeadscale_generateGivenName(t *testing.T) {
 
 func TestAutoApproveRoutes(t *testing.T) {
 	tests := []struct {
-		name   string
-		acl    string
-		routes []netip.Prefix
-		want   []netip.Prefix
-		want2  []netip.Prefix
+		name         string
+		acl          string
+		routes       []netip.Prefix
+		want         []netip.Prefix
+		want2        []netip.Prefix
+		expectChange bool // whether to expect route changes
 	}{
+		{
+			name: "no-auto-approvers-empty-policy",
+			acl: `
+{
+	"groups": {
+		"group:admins": ["test@"]
+	},
+	"acls": [
+		{
+			"action": "accept",
+			"src": ["group:admins"],
+			"dst": ["group:admins:*"]
+		}
+	]
+}`,
+			routes:       []netip.Prefix{netip.MustParsePrefix("10.33.0.0/16")},
+			want:         []netip.Prefix{}, // Should be empty - no auto-approvers
+			want2:        []netip.Prefix{}, // Should be empty - no auto-approvers
+			expectChange: false,            // No changes expected
+		},
+		{
+			name: "no-auto-approvers-explicit-empty",
+			acl: `
+{
+	"groups": {
+		"group:admins": ["test@"]
+	},
+	"acls": [
+		{
+			"action": "accept",
+			"src": ["group:admins"],
+			"dst": ["group:admins:*"]
+		}
+	],
+	"autoApprovers": {
+		"routes": {},
+		"exitNode": []
+	}
+}`,
+			routes:       []netip.Prefix{netip.MustParsePrefix("10.33.0.0/16")},
+			want:         []netip.Prefix{}, // Should be empty - explicitly empty auto-approvers
+			want2:        []netip.Prefix{}, // Should be empty - explicitly empty auto-approvers
+			expectChange: false,            // No changes expected
+		},
 		{
 			name: "2068-approve-issue-sub-kube",
 			acl: `
@@ -384,8 +361,9 @@ func TestAutoApproveRoutes(t *testing.T) {
 		}
 	}
 }`,
-			routes: []netip.Prefix{netip.MustParsePrefix("10.42.7.0/24")},
-			want:   []netip.Prefix{netip.MustParsePrefix("10.42.7.0/24")},
+			routes:       []netip.Prefix{netip.MustParsePrefix("10.42.7.0/24")},
+			want:         []netip.Prefix{netip.MustParsePrefix("10.42.7.0/24")},
+			expectChange: true, // Routes should be approved
 		},
 		{
 			name: "2068-approve-issue-sub-exit-tag",
@@ -429,14 +407,14 @@ func TestAutoApproveRoutes(t *testing.T) {
 				tsaddr.AllIPv4(),
 				tsaddr.AllIPv6(),
 			},
+			expectChange: true, // Routes should be approved
 		},
 	}
 
 	for _, tt := range tests {
 		pmfs := policy.PolicyManagerFuncsForTest([]byte(tt.acl))
 		for i, pmf := range pmfs {
-			version := i + 1
-			t.Run(fmt.Sprintf("%s-policyv%d", tt.name, version), func(t *testing.T) {
+			t.Run(fmt.Sprintf("%s-policy-index%d", tt.name, i), func(t *testing.T) {
 				adb, err := newSQLiteTestDB()
 				require.NoError(t, err)
 
@@ -486,32 +464,44 @@ func TestAutoApproveRoutes(t *testing.T) {
 				nodes, err := adb.ListNodes()
 				assert.NoError(t, err)
 
-				pm, err := pmf(users, nodes)
+				pm, err := pmf(users, nodes.ViewSlice())
 				require.NoError(t, err)
 				require.NotNil(t, pm)
 
-				changed1 := policy.AutoApproveRoutes(pm, &node)
-				assert.True(t, changed1)
+				newRoutes1, changed1 := policy.ApproveRoutesWithPolicy(pm, node.View(), node.ApprovedRoutes, tt.routes)
+				assert.Equal(t, tt.expectChange, changed1)
 
-				err = adb.DB.Save(&node).Error
-				require.NoError(t, err)
+				if changed1 {
+					err = SetApprovedRoutes(adb.DB, node.ID, newRoutes1)
+					require.NoError(t, err)
+				}
 
-				_ = policy.AutoApproveRoutes(pm, &nodeTagged)
-
-				err = adb.DB.Save(&nodeTagged).Error
-				require.NoError(t, err)
+				newRoutes2, changed2 := policy.ApproveRoutesWithPolicy(pm, nodeTagged.View(), node.ApprovedRoutes, tt.routes)
+				if changed2 {
+					err = SetApprovedRoutes(adb.DB, nodeTagged.ID, newRoutes2)
+					require.NoError(t, err)
+				}
 
 				node1ByID, err := adb.GetNodeByID(1)
 				require.NoError(t, err)
 
-				if diff := cmp.Diff(tt.want, node1ByID.SubnetRoutes(), util.Comparers...); diff != "" {
+				// For empty auto-approvers tests, handle nil vs empty slice comparison
+				expectedRoutes1 := tt.want
+				if len(expectedRoutes1) == 0 {
+					expectedRoutes1 = nil
+				}
+				if diff := cmp.Diff(expectedRoutes1, node1ByID.SubnetRoutes(), util.Comparers...); diff != "" {
 					t.Errorf("unexpected enabled routes (-want +got):\n%s", diff)
 				}
 
 				node2ByID, err := adb.GetNodeByID(2)
 				require.NoError(t, err)
 
-				if diff := cmp.Diff(tt.want2, node2ByID.SubnetRoutes(), util.Comparers...); diff != "" {
+				expectedRoutes2 := tt.want2
+				if len(expectedRoutes2) == 0 {
+					expectedRoutes2 = nil
+				}
+				if diff := cmp.Diff(expectedRoutes2, node2ByID.SubnetRoutes(), util.Comparers...); diff != "" {
 					t.Errorf("unexpected enabled routes (-want +got):\n%s", diff)
 				}
 			})
@@ -590,6 +580,7 @@ func generateRandomNumber(t *testing.T, max int64) int64 {
 	if err != nil {
 		t.Fatalf("getting random number: %s", err)
 	}
+
 	return n.Int64() + 1
 }
 
@@ -649,7 +640,7 @@ func TestListEphemeralNodes(t *testing.T) {
 	assert.Equal(t, nodeEph.Hostname, ephemeralNodes[0].Hostname)
 }
 
-func TestRenameNode(t *testing.T) {
+func TestNodeNaming(t *testing.T) {
 	db, err := newSQLiteTestDB()
 	if err != nil {
 		t.Fatalf("creating db: %s", err)
@@ -681,6 +672,26 @@ func TestRenameNode(t *testing.T) {
 		Hostinfo:       &tailcfg.Hostinfo{},
 	}
 
+	// Using non-ASCII characters in the hostname can
+	// break your network, so they should be replaced when registering
+	// a node.
+	// https://github.com/juanfont/headscale/issues/2343
+	nodeInvalidHostname := types.Node{
+		MachineKey:     key.NewMachine().Public(),
+		NodeKey:        key.NewNode().Public(),
+		Hostname:       "我的电脑",
+		UserID:         user2.ID,
+		RegisterMethod: util.RegisterMethodAuthKey,
+	}
+
+	nodeShortHostname := types.Node{
+		MachineKey:     key.NewMachine().Public(),
+		NodeKey:        key.NewNode().Public(),
+		Hostname:       "a",
+		UserID:         user2.ID,
+		RegisterMethod: util.RegisterMethodAuthKey,
+	}
+
 	err = db.DB.Save(&node).Error
 	require.NoError(t, err)
 
@@ -688,11 +699,16 @@ func TestRenameNode(t *testing.T) {
 	require.NoError(t, err)
 
 	err = db.DB.Transaction(func(tx *gorm.DB) error {
-		_, err := RegisterNode(tx, node, nil, nil)
+		_, err := RegisterNodeForTest(tx, node, nil, nil)
 		if err != nil {
 			return err
 		}
-		_, err = RegisterNode(tx, node2, nil, nil)
+		_, err = RegisterNodeForTest(tx, node2, nil, nil)
+		if err != nil {
+			return err
+		}
+		_, err = RegisterNodeForTest(tx, nodeInvalidHostname, ptr.To(mpp("100.64.0.66/32").Addr()), nil)
+		_, err = RegisterNodeForTest(tx, nodeShortHostname, ptr.To(mpp("100.64.0.67/32").Addr()), nil)
 		return err
 	})
 	require.NoError(t, err)
@@ -700,10 +716,12 @@ func TestRenameNode(t *testing.T) {
 	nodes, err := db.ListNodes()
 	require.NoError(t, err)
 
-	assert.Len(t, nodes, 2)
+	assert.Len(t, nodes, 4)
 
 	t.Logf("node1 %s %s", nodes[0].Hostname, nodes[0].GivenName)
 	t.Logf("node2 %s %s", nodes[1].Hostname, nodes[1].GivenName)
+	t.Logf("node3 %s %s", nodes[2].Hostname, nodes[2].GivenName)
+	t.Logf("node4 %s %s", nodes[3].Hostname, nodes[3].GivenName)
 
 	assert.Equal(t, nodes[0].Hostname, nodes[0].GivenName)
 	assert.NotEqual(t, nodes[1].Hostname, nodes[1].GivenName)
@@ -715,6 +733,10 @@ func TestRenameNode(t *testing.T) {
 	assert.Len(t, nodes[1].Hostname, 4)
 	assert.Len(t, nodes[0].GivenName, 4)
 	assert.Len(t, nodes[1].GivenName, 13)
+	assert.Contains(t, nodes[2].Hostname, "invalid-") // invalid chars
+	assert.Contains(t, nodes[2].GivenName, "invalid-")
+	assert.Contains(t, nodes[3].Hostname, "invalid-") // too short
+	assert.Contains(t, nodes[3].GivenName, "invalid-")
 
 	// Nodes can be renamed to a unique name
 	err = db.Write(func(tx *gorm.DB) error {
@@ -724,7 +746,7 @@ func TestRenameNode(t *testing.T) {
 
 	nodes, err = db.ListNodes()
 	require.NoError(t, err)
-	assert.Len(t, nodes, 2)
+	assert.Len(t, nodes, 4)
 	assert.Equal(t, "test", nodes[0].Hostname)
 	assert.Equal(t, "newname", nodes[0].GivenName)
 
@@ -736,7 +758,7 @@ func TestRenameNode(t *testing.T) {
 
 	nodes, err = db.ListNodes()
 	require.NoError(t, err)
-	assert.Len(t, nodes, 2)
+	assert.Len(t, nodes, 4)
 	assert.Equal(t, "test", nodes[0].Hostname)
 	assert.Equal(t, "newname", nodes[0].GivenName)
 	assert.Equal(t, "test", nodes[1].GivenName)
@@ -746,6 +768,149 @@ func TestRenameNode(t *testing.T) {
 		return RenameNode(tx, nodes[0].ID, "test")
 	})
 	assert.ErrorContains(t, err, "name is not unique")
+
+	// Rename invalid chars
+	err = db.Write(func(tx *gorm.DB) error {
+		return RenameNode(tx, nodes[2].ID, "我的电脑")
+	})
+	assert.ErrorContains(t, err, "invalid characters")
+
+	// Rename too short
+	err = db.Write(func(tx *gorm.DB) error {
+		return RenameNode(tx, nodes[3].ID, "a")
+	})
+	assert.ErrorContains(t, err, "at least 2 characters")
+
+	// Rename with emoji
+	err = db.Write(func(tx *gorm.DB) error {
+		return RenameNode(tx, nodes[0].ID, "hostname-with-💩")
+	})
+	assert.ErrorContains(t, err, "invalid characters")
+
+	// Rename with only emoji
+	err = db.Write(func(tx *gorm.DB) error {
+		return RenameNode(tx, nodes[0].ID, "🚀")
+	})
+	assert.ErrorContains(t, err, "invalid characters")
+}
+
+func TestRenameNodeComprehensive(t *testing.T) {
+	db, err := newSQLiteTestDB()
+	if err != nil {
+		t.Fatalf("creating db: %s", err)
+	}
+
+	user, err := db.CreateUser(types.User{Name: "test"})
+	require.NoError(t, err)
+
+	node := types.Node{
+		ID:             0,
+		MachineKey:     key.NewMachine().Public(),
+		NodeKey:        key.NewNode().Public(),
+		Hostname:       "testnode",
+		UserID:         user.ID,
+		RegisterMethod: util.RegisterMethodAuthKey,
+		Hostinfo:       &tailcfg.Hostinfo{},
+	}
+
+	err = db.DB.Save(&node).Error
+	require.NoError(t, err)
+
+	err = db.DB.Transaction(func(tx *gorm.DB) error {
+		_, err := RegisterNodeForTest(tx, node, nil, nil)
+		return err
+	})
+	require.NoError(t, err)
+
+	nodes, err := db.ListNodes()
+	require.NoError(t, err)
+	assert.Len(t, nodes, 1)
+
+	tests := []struct {
+		name    string
+		newName string
+		wantErr string
+	}{
+		{
+			name:    "uppercase_rejected",
+			newName: "User2-Host",
+			wantErr: "must be lowercase",
+		},
+		{
+			name:    "underscore_rejected",
+			newName: "test_node",
+			wantErr: "invalid characters",
+		},
+		{
+			name:    "at_sign_uppercase_rejected",
+			newName: "Test@Host",
+			wantErr: "must be lowercase",
+		},
+		{
+			name:    "at_sign_rejected",
+			newName: "test@host",
+			wantErr: "invalid characters",
+		},
+		{
+			name:    "chinese_chars_with_dash_rejected",
+			newName: "server-北京-01",
+			wantErr: "invalid characters",
+		},
+		{
+			name:    "chinese_only_rejected",
+			newName: "我的电脑",
+			wantErr: "invalid characters",
+		},
+		{
+			name:    "emoji_with_text_rejected",
+			newName: "laptop-🚀",
+			wantErr: "invalid characters",
+		},
+		{
+			name:    "mixed_chinese_emoji_rejected",
+			newName: "测试💻机器",
+			wantErr: "invalid characters",
+		},
+		{
+			name:    "only_emojis_rejected",
+			newName: "🎉🎊",
+			wantErr: "invalid characters",
+		},
+		{
+			name:    "only_at_signs_rejected",
+			newName: "@@@",
+			wantErr: "invalid characters",
+		},
+		{
+			name:    "starts_with_dash_rejected",
+			newName: "-test",
+			wantErr: "cannot start or end with a hyphen",
+		},
+		{
+			name:    "ends_with_dash_rejected",
+			newName: "test-",
+			wantErr: "cannot start or end with a hyphen",
+		},
+		{
+			name:    "too_long_hostname_rejected",
+			newName: "this-is-a-very-long-hostname-that-exceeds-sixty-three-characters-limit",
+			wantErr: "must not exceed 63 characters",
+		},
+		{
+			name:    "too_short_hostname_rejected",
+			newName: "a",
+			wantErr: "at least 2 characters",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			err := db.Write(func(tx *gorm.DB) error {
+				return RenameNode(tx, nodes[0].ID, tt.newName)
+			})
+			assert.ErrorContains(t, err, tt.wantErr)
+		})
+	}
 }
 
 func TestListPeers(t *testing.T) {
@@ -788,11 +953,12 @@ func TestListPeers(t *testing.T) {
 	require.NoError(t, err)
 
 	err = db.DB.Transaction(func(tx *gorm.DB) error {
-		_, err := RegisterNode(tx, node1, nil, nil)
+		_, err := RegisterNodeForTest(tx, node1, nil, nil)
 		if err != nil {
 			return err
 		}
-		_, err = RegisterNode(tx, node2, nil, nil)
+		_, err = RegisterNodeForTest(tx, node2, nil, nil)
+
 		return err
 	})
 	require.NoError(t, err)
@@ -805,30 +971,30 @@ func TestListPeers(t *testing.T) {
 	// No parameter means no filter, should return all peers
 	nodes, err = db.ListPeers(1)
 	require.NoError(t, err)
-	assert.Equal(t, len(nodes), 1)
+	assert.Len(t, nodes, 1)
 	assert.Equal(t, "test2", nodes[0].Hostname)
 
 	// Empty node list should return all peers
 	nodes, err = db.ListPeers(1, types.NodeIDs{}...)
 	require.NoError(t, err)
-	assert.Equal(t, len(nodes), 1)
+	assert.Len(t, nodes, 1)
 	assert.Equal(t, "test2", nodes[0].Hostname)
 
 	// No match in IDs should return empty list and no error
 	nodes, err = db.ListPeers(1, types.NodeIDs{3, 4, 5}...)
 	require.NoError(t, err)
-	assert.Equal(t, len(nodes), 0)
+	assert.Empty(t, nodes)
 
 	// Partial match in IDs
 	nodes, err = db.ListPeers(1, types.NodeIDs{2, 3}...)
 	require.NoError(t, err)
-	assert.Equal(t, len(nodes), 1)
+	assert.Len(t, nodes, 1)
 	assert.Equal(t, "test2", nodes[0].Hostname)
 
 	// Several matched IDs, but node ID is still filtered out
 	nodes, err = db.ListPeers(1, types.NodeIDs{1, 2, 3}...)
 	require.NoError(t, err)
-	assert.Equal(t, len(nodes), 1)
+	assert.Len(t, nodes, 1)
 	assert.Equal(t, "test2", nodes[0].Hostname)
 }
 
@@ -872,11 +1038,12 @@ func TestListNodes(t *testing.T) {
 	require.NoError(t, err)
 
 	err = db.DB.Transaction(func(tx *gorm.DB) error {
-		_, err := RegisterNode(tx, node1, nil, nil)
+		_, err := RegisterNodeForTest(tx, node1, nil, nil)
 		if err != nil {
 			return err
 		}
-		_, err = RegisterNode(tx, node2, nil, nil)
+		_, err = RegisterNodeForTest(tx, node2, nil, nil)
+
 		return err
 	})
 	require.NoError(t, err)
@@ -889,32 +1056,32 @@ func TestListNodes(t *testing.T) {
 	// No parameter means no filter, should return all nodes
 	nodes, err = db.ListNodes()
 	require.NoError(t, err)
-	assert.Equal(t, len(nodes), 2)
+	assert.Len(t, nodes, 2)
 	assert.Equal(t, "test1", nodes[0].Hostname)
 	assert.Equal(t, "test2", nodes[1].Hostname)
 
 	// Empty node list should return all nodes
 	nodes, err = db.ListNodes(types.NodeIDs{}...)
 	require.NoError(t, err)
-	assert.Equal(t, len(nodes), 2)
+	assert.Len(t, nodes, 2)
 	assert.Equal(t, "test1", nodes[0].Hostname)
 	assert.Equal(t, "test2", nodes[1].Hostname)
 
 	// No match in IDs should return empty list and no error
 	nodes, err = db.ListNodes(types.NodeIDs{3, 4, 5}...)
 	require.NoError(t, err)
-	assert.Equal(t, len(nodes), 0)
+	assert.Empty(t, nodes)
 
 	// Partial match in IDs
 	nodes, err = db.ListNodes(types.NodeIDs{2, 3}...)
 	require.NoError(t, err)
-	assert.Equal(t, len(nodes), 1)
+	assert.Len(t, nodes, 1)
 	assert.Equal(t, "test2", nodes[0].Hostname)
 
 	// Several matched IDs
 	nodes, err = db.ListNodes(types.NodeIDs{1, 2, 3}...)
 	require.NoError(t, err)
-	assert.Equal(t, len(nodes), 2)
+	assert.Len(t, nodes, 2)
 	assert.Equal(t, "test1", nodes[0].Hostname)
 	assert.Equal(t, "test2", nodes[1].Hostname)
 }

hscontrol/db/preauth_keys.go

@@ -5,6 +5,7 @@ import (
 	"encoding/hex"
 	"errors"
 	"fmt"
+	"slices"
 	"strings"
 	"time"
 
@@ -47,8 +48,9 @@ func CreatePreAuthKey(
 		return nil, err
 	}
 
-	// Remove duplicates
+	// Remove duplicates and sort for consistency
 	aclTags = set.SetOf(aclTags).Slice()
+	slices.Sort(aclTags)
 
 	// TODO(kradalby): factor out and create a reusable tag validation,
 	// check if there is one in Tailscale's lib.
@@ -109,9 +111,7 @@ func ListPreAuthKeysByUser(tx *gorm.DB, uid types.UserID) ([]types.PreAuthKey, e
 }
 
 func (hsdb *HSDatabase) GetPreAuthKey(key string) (*types.PreAuthKey, error) {
-	return Read(hsdb.DB, func(rx *gorm.DB) (*types.PreAuthKey, error) {
-		return GetPreAuthKey(rx, key)
-	})
+	return GetPreAuthKey(hsdb.DB, key)
 }
 
 // GetPreAuthKey returns a PreAuthKey for a given key. The caller is responsible
@@ -155,11 +155,8 @@ func UsePreAuthKey(tx *gorm.DB, k *types.PreAuthKey) error {
 
 // MarkExpirePreAuthKey marks a PreAuthKey as expired.
 func ExpirePreAuthKey(tx *gorm.DB, k *types.PreAuthKey) error {
-	if err := tx.Model(&k).Update("Expiration", time.Now()).Error; err != nil {
-		return err
-	}
-
-	return nil
+	now := time.Now()
+	return tx.Model(&types.PreAuthKey{}).Where("id = ?", k.ID).Update("expiration", now).Error
 }
 
 func generateKey() (string, error) {

hscontrol/db/preauth_keys_test.go

@@ -1,16 +1,15 @@
 package db
 
 import (
-	"sort"
+	"slices"
 	"testing"
 
 	"github.com/juanfont/headscale/hscontrol/types"
 	"github.com/juanfont/headscale/hscontrol/util"
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
-	"tailscale.com/types/ptr"
-
 	"gopkg.in/check.v1"
+	"tailscale.com/types/ptr"
 )
 
 func (*Suite) TestCreatePreAuthKey(c *check.C) {
@@ -58,7 +57,7 @@ func (*Suite) TestPreAuthKeyACLTags(c *check.C) {
 	listedPaks, err := db.ListPreAuthKeys(types.UserID(user.ID))
 	c.Assert(err, check.IsNil)
 	gotTags := listedPaks[0].Proto().GetAclTags()
-	sort.Sort(sort.StringSlice(gotTags))
+	slices.Sort(gotTags)
 	c.Assert(gotTags, check.DeepEquals, tags)
 }
 

hscontrol/db/schema.sql

@@ -0,0 +1,110 @@
+-- This file is the representation of the SQLite schema of Headscale.
+-- It is the "source of truth" and is used to validate any migrations
+-- that are run against the database to ensure it ends in the expected state.
+
+CREATE TABLE migrations(id text,PRIMARY KEY(id));
+
+CREATE TABLE users(
+  id integer PRIMARY KEY AUTOINCREMENT,
+  name text,
+  display_name text,
+  email text,
+  provider_identifier text,
+  provider text,
+  profile_pic_url text,
+
+  created_at datetime,
+  updated_at datetime,
+  deleted_at datetime
+);
+CREATE INDEX idx_users_deleted_at ON users(deleted_at);
+
+
+-- The following three UNIQUE indexes work together to enforce the user identity model:
+--
+-- 1. Users can be either local (provider_identifier is NULL) or from external providers (provider_identifier set)
+-- 2. Each external provider identifier must be unique across the system
+-- 3. Local usernames must be unique among local users
+-- 4. The same username can exist across different providers with different identifiers
+--
+-- Examples:
+-- - Can create local user "alice" (provider_identifier=NULL)
+-- - Can create external user "alice" with GitHub (name="alice", provider_identifier="alice_github")
+-- - Can create external user "alice" with Google (name="alice", provider_identifier="alice_google")
+-- - Cannot create another local user "alice" (blocked by idx_name_no_provider_identifier)
+-- - Cannot create another user with provider_identifier="alice_github" (blocked by idx_provider_identifier)
+-- - Cannot create user "bob" with provider_identifier="alice_github" (blocked by idx_name_provider_identifier)
+CREATE UNIQUE INDEX idx_provider_identifier ON users(
+  provider_identifier
+) WHERE provider_identifier IS NOT NULL;
+CREATE UNIQUE INDEX idx_name_provider_identifier ON users(
+  name,
+  provider_identifier
+);
+CREATE UNIQUE INDEX idx_name_no_provider_identifier ON users(
+  name
+) WHERE provider_identifier IS NULL;
+
+CREATE TABLE pre_auth_keys(
+  id integer PRIMARY KEY AUTOINCREMENT,
+  key text,
+  user_id integer,
+  reusable numeric,
+  ephemeral numeric DEFAULT false,
+  used numeric DEFAULT false,
+  tags text,
+  expiration datetime,
+
+  created_at datetime,
+
+  CONSTRAINT fk_pre_auth_keys_user FOREIGN KEY(user_id) REFERENCES users(id) ON DELETE SET NULL
+);
+
+CREATE TABLE api_keys(
+  id integer PRIMARY KEY AUTOINCREMENT,
+  prefix text,
+  hash blob,
+  expiration datetime,
+  last_seen datetime,
+
+  created_at datetime
+);
+CREATE UNIQUE INDEX idx_api_keys_prefix ON api_keys(prefix);
+
+CREATE TABLE nodes(
+  id integer PRIMARY KEY AUTOINCREMENT,
+  machine_key text,
+  node_key text,
+  disco_key text,
+
+  endpoints text,
+  host_info text,
+  ipv4 text,
+  ipv6 text,
+  hostname text,
+  given_name varchar(63),
+  user_id integer,
+  register_method text,
+  forced_tags text,
+  auth_key_id integer,
+  last_seen datetime,
+  expiry datetime,
+  approved_routes text,
+
+  created_at datetime,
+  updated_at datetime,
+  deleted_at datetime,
+
+  CONSTRAINT fk_nodes_user FOREIGN KEY(user_id) REFERENCES users(id) ON DELETE CASCADE,
+  CONSTRAINT fk_nodes_auth_key FOREIGN KEY(auth_key_id) REFERENCES pre_auth_keys(id)
+);
+
+CREATE TABLE policies(
+  id integer PRIMARY KEY AUTOINCREMENT,
+  data text,
+
+  created_at datetime,
+  updated_at datetime,
+  deleted_at datetime
+);
+CREATE INDEX idx_policies_deleted_at ON policies(deleted_at);