• Pre-Release

    adam released this 2025-12-18 12:42:32 +01:00 | 8 commits to main since this release

    📅 Originally published on GitHub: Thu, 18 Dec 2025 11:55:06 GMT
    🏷️ Git tag created: Thu, 18 Dec 2025 11:42:32 GMT

    Minimum supported Tailscale client version: v1.74.0

    Tags as identity

    Tags are now implemented following the Tailscale model where tags and user ownership are mutually exclusive. Devices can be either
    user-owned (authenticated via web/OIDC) or tagged (authenticated via tagged PreAuthKeys). Tagged devices receive their identity from
    tags rather than users, making them suitable for servers and infrastructure. Applying a tag to a device removes user-based
    ownership. See the Tailscale tags documentation for details on how tags work.

    User-owned nodes can now request tags during registration using --advertise-tags. Tags are validated against the tagOwners policy
    and applied at registration time. Tags can be managed via the CLI or API after registration.

    Smarter map updates

    The map update system has been rewritten to send smaller, partial updates instead of full network maps whenever possible. This reduces bandwidth usage and improves performance, especially for large networks. The system now properly tracks peer
    changes and can send removal notifications when nodes are removed due to policy changes.
    #2856 #2961

    Pre-authentication key security improvements

    Pre-authentication keys now use bcrypt hashing for improved security #2853. Keys
    are stored as a prefix and bcrypt hash instead of plaintext. The full key is only displayed once at creation time. When listing keys,
    only the prefix is shown (e.g., hskey-auth-{prefix}-***). All new keys use the format hskey-auth-{prefix}-{secret}. Legacy plaintext keys in the format {secret} will continue to work for backwards compatibility.

    Web registration templates redesign

    The OIDC callback and device registration web pages have been updated to use the Material for MkDocs design system from the official
    documentation. The templates now use consistent typography, spacing, and colours across all registration flows.

    Database migration support removed for pre-0.25.0 databases

    Headscale no longer supports direct upgrades from databases created before version 0.25.0. Users on older versions must upgrade
    sequentially through each stable release, selecting the latest patch version available for each minor release.

    BREAKING

    • Tags: The gRPC SetTags endpoint now allows converting user-owned nodes to tagged nodes by setting tags. Once a node is tagged, it cannot be converted back to a user-owned node. #2885
    • Tags: Tags are now resolved from the node's stored Tags field only #2931
      • --advertise-tags is processed during registration, not on every policy evaluation
      • PreAuthKey tagged devices ignore --advertise-tags from clients
      • User-owned nodes can use --advertise-tags if authorized by tagOwners policy
      • Tags can be managed via CLI (headscale nodes tag) or the SetTags API after registration
    • Database migration support removed for pre-0.25.0 databases #2883
      • If you are running a version older than 0.25.0, you must upgrade to 0.25.1 first, then upgrade to this release
      • See the upgrade path documentation for detailed guidance
      • In version 0.29, all migrations before 0.28.0 will also be removed
    • Remove ability to move nodes between users #2922
      • The headscale nodes move CLI command has been removed
      • The MoveNode API endpoint has been removed
      • Nodes are permanently associated with their user at registration time

    Changes

    • Smarter change notifications send partial map updates and node removals instead of full maps #2961
      • Send lightweight endpoint and DERP region updates instead of full maps #2856
    • Add oidc.email_verified_required config option to control email verification requirement #2860
      • When true (default), only verified emails can authenticate via OIDC with allowed_domains or allowed_users
      • When false, unverified emails are allowed for OIDC authentication
    • Add NixOS module in repository for faster iteration #2857
    • Add favicon to webpages #2858
    • Redesign OIDC callback and registration web templates #2832
    • Reclaim IPs from the IP allocator when nodes are deleted #2831
    • Add bcrypt hashing for pre-authentication keys #2853
    • Add prefix to API keys (hskey-api-{prefix}-{secret}) #2853
    • Add prefix to registration keys for web authentication tracking (hskey-reg-{random}) #2853
    • Tags can now be tagOwner of other tags #2930
    • Add taildrop.enabled configuration option to enable/disable Taildrop file sharing #2955
    • Allow disabling the metrics server by setting empty metrics_listen_addr #2914
    • Log ACME/autocert errors for easier debugging #2933
    • Improve CLI list output formatting #2951
    • Use Debian 13 distroless base images for containers #2944
    • Fix ACL policy not applied to new OIDC nodes until client restart #2890
    • Fix autogroup:self preventing visibility of nodes matched by other ACL rules #2882
    • Fix nodes being rejected after pre-authentication key expiration #2917
    • Fix list-routes command respecting identifier filter with JSON output #2927

    Upgrade

    Please follow the steps outlined in the upgrade guide to update your existing Headscale installation.

    It's best to update from one stable version to the next (e.g., 0.24.0 → 0.25.1 → 0.26.1) in case you are multiple releases behind. You should always pick the latest available patch release.

    Be sure to check the changelog above for version-specific upgrade instructions and breaking changes.

    Backup Your Database

    Always backup your database before upgrading. Here's how to backup a SQLite database:

    # Stop headscale
    systemctl stop headscale
    
    # Backup sqlite database
    cp /var/lib/headscale/db.sqlite /var/lib/headscale/db.sqlite.backup
    
    # Backup sqlite WAL/SHM files (if they exist)
    cp /var/lib/headscale/db.sqlite-wal /var/lib/headscale/db.sqlite-wal.backup
    cp /var/lib/headscale/db.sqlite-shm /var/lib/headscale/db.sqlite-shm.backup
    
    # Start headscale (migration will run automatically)
    systemctl start headscale
    

    Changelog

    Downloads
  • Pre-Release

    adam released this 2025-11-30 19:10:56 +01:00 | 83 commits to main since this release

    📅 Originally published on GitHub: Sun, 30 Nov 2025 18:41:27 GMT
    🏷️ Git tag created: Sun, 30 Nov 2025 18:10:56 GMT

    Changes

    • Fix ACL policy not applied to new OIDC nodes until client restart
      #2890
    • Fix autogroup:self preventing visibility of nodes matched by other ACL rules
      #2882
    • Fix nodes being rejected after pre-authentication key expiration
      #2917

    Upgrade

    Please follow the steps outlined in the upgrade guide to update your existing Headscale installation.

    It's best to update from one stable version to the next (e.g., 0.24.0 → 0.25.1 → 0.26.1) in case you are multiple releases behind. You should always pick the latest available patch release.

    Be sure to check the changelog above for version-specific upgrade instructions and breaking changes.

    Backup Your Database

    Always backup your database before upgrading. Here's how to backup a SQLite database:

    # Stop headscale
    systemctl stop headscale
    
    # Backup sqlite database
    cp /var/lib/headscale/db.sqlite /var/lib/headscale/db.sqlite.backup
    
    # Backup sqlite WAL/SHM files (if they exist)
    cp /var/lib/headscale/db.sqlite-wal /var/lib/headscale/db.sqlite-wal.backup
    cp /var/lib/headscale/db.sqlite-shm /var/lib/headscale/db.sqlite-shm.backup
    
    # Start headscale (migration will run automatically)
    systemctl start headscale
    

    Changelog

    • 7f1631c4f1 auth: ensure machines are allowed in when pak change (#2917)
    • c6d399a66c changelog: prep for 0.27.2 rc
    • 7e8cee6b10 chore: fix filterHash to work with autogroup:self in the acls (#2882)
    • 4fe5cbe703 hscontrol/oidc: fix ACL policy not applied to new OIDC nodes (#2890)
    Downloads
  • Stable

    adam released this 2025-11-11 20:17:02 +01:00 | 87 commits to main since this release

    📅 Originally published on GitHub: Tue, 11 Nov 2025 19:32:29 GMT
    🏷️ Git tag created: Tue, 11 Nov 2025 19:17:02 GMT

    Minimum supported Tailscale client version: v1.64.0

    Changes

    • Expire nodes with a custom timestamp
      #2828
    • Fix issue where node expiry was reset when tailscaled restarts
      #2875
    • Fix OIDC authentication when multiple login URLs are opened
      #2861
    • Fix node re-registration failing with expired auth keys
      #2859
    • Remove old unused database tables and indices
      #2844
      #2872
    • Ignore litestream tables during database validation
      #2843
    • Fix exit node visibility to respect ACL rules
      #2855
    • Fix SSH policy becoming empty when unknown user is referenced
      #2874
    • Fix policy validation when using bypass-grpc mode
      #2854
    • Fix autogroup:self interaction with other ACL rules
      #2842
    • Fix flaky DERP map shuffle test
      #2848
    • Use current stable base images for Debian and Alpine containers
      #2827

    Upgrade

    Please follow the steps outlined in the upgrade guide to update your existing Headscale installation.

    It's best to update from one stable version to the next (e.g., 0.24.0 → 0.25.1 → 0.26.1) in case you are multiple releases behind. You should always pick the latest available patch release.

    Be sure to check the changelog above for version-specific upgrade instructions and breaking changes.

    Backup Your Database

    Always backup your database before upgrading. Here's how to backup a SQLite database:

    # Stop headscale
    systemctl stop headscale
    
    # Backup sqlite database
    cp /var/lib/headscale/db.sqlite /var/lib/headscale/db.sqlite.backup
    
    # Backup sqlite WAL/SHM files (if they exist)
    cp /var/lib/headscale/db.sqlite-wal /var/lib/headscale/db.sqlite-wal.backup
    cp /var/lib/headscale/db.sqlite-shm /var/lib/headscale/db.sqlite-shm.backup
    
    # Start headscale (migration will run automatically)
    systemctl start headscale
    

    Changelog

    • abed534628 Document how to restrict access to exit nodes per user/group
    • d23fa26395 Fix flaky TestShuffleDERPMapDeterministic by ensuring deterministic map iteration (#2848)
    • 0a43aab8f5 Use Debian 12 as minimum version for the deb package
    • 4bd614a559 Use current stable base images for Debian and Alpine
    • 785168a7b8 changelog: prepare for 0.27.1
    • 19a33394f6 changelog: set 0.27 date (#2823)
    • af2de35b6c chore: fix autogroup:self with other acl rules (#2842)
    • 02c7c1a0e7 cli: only validate bypass-grpc set policy (#2854)
    • 5a2ee0c391 db: add comment about removing migrations
    • 28faf8cd71 db: add defensive removal of old indicies
    • 456a5d5cce db: ignore _litestream tables when validating (#2843)
    • ddbd3e14ba db: remove all old, unused tables (#2844)
    • f9bb88ad24 expire nodes with a custom timestamp (#2828)
    • 5cd15c3656 fix: make state cookies valid when client uses multiple login URLs
    • 3bd4ecd9cd fix: preserve node expiry when tailscaled restarts
    • 3455d1cb59 hscontrol/db: fix RenameUser to use Updates()
    • 4a8dc2d445 hscontrol/state,db: preserve node expiry on MapRequest updates
    • 4728a2ba9e hscontrol/state: allow expired auth keys for node re-registration
    • ddd31ba774 hscontrol: use Updates() instead of Save() for partial updates
    • 773a46a968 integration: add test to replicate #2862
    • 84fe3de251 integration: reduce TestAutoApproveMultiNetwork matrix to 3 tests (#2815)
    • d9c3eaf8c8 matcher: Add func for comparing Dests and TheInternet
    • f658a8eacd mkdocs: 0.27.1
    • c649c89e00 policy: Reproduce exit node visibility issues
    • 21e3f2598d policy: fix issue where non existent user results in empty ssh pol
    • a28d9bed6d policy: reproduce 2863 in test
    • d7a43a7cf1 state: use AllApprovedRoutes instead of SubnetRoutes
    • 2024219bd1 types: Distinguish subnet and exit node access
    • bd9cf42b96 types: NodeView CanAccess uses internal
    • 1c0bb0338d types: split SubnetRoutes and ExitRoutes
    Downloads
  • Stable

    adam released this 2025-10-27 11:18:57 +01:00 | 117 commits to main since this release

    📅 Originally published on GitHub: Mon, 27 Oct 2025 11:16:35 GMT
    🏷️ Git tag created: Mon, 27 Oct 2025 10:18:57 GMT

    Minimum supported Tailscale client version: v1.64.0

    Database integrity improvements

    This release includes a significant database migration that addresses
    longstanding issues with the database schema and data integrity that has
    accumulated over the years. The migration introduces a schema.sql file as the
    source of truth for the expected database schema to ensure new migrations that
    will cause divergence does not occur again.

    These issues arose from a combination of factors discovered over time: SQLite
    foreign keys not being enforced for many early versions, all migrations being
    run in one large function until version 0.23.0, and inconsistent use of GORM's
    AutoMigrate feature. Moving forward, all new migrations will be explicit SQL
    operations rather than relying on GORM AutoMigrate, and foreign keys will be
    enforced throughout the migration process.

    We are only improving SQLite databases with this change - PostgreSQL databases
    are not affected.

    Please read the
    PR description for more
    technical details about the issues and solutions.

    SQLite Database Backup Example:

    # Stop headscale
    systemctl stop headscale
    
    # Backup sqlite database
    cp /var/lib/headscale/db.sqlite /var/lib/headscale/db.sqlite.backup
    
    # Backup sqlite WAL/SHM files (if they exist)
    cp /var/lib/headscale/db.sqlite-wal /var/lib/headscale/db.sqlite-wal.backup
    cp /var/lib/headscale/db.sqlite-shm /var/lib/headscale/db.sqlite-shm.backup
    
    # Start headscale (migration will run automatically)
    systemctl start headscale
    

    DERPMap update frequency

    The default DERPMap update frequency has been changed from 24 hours to 3 hours.
    If you set the derp.update_frequency configuration option, it is recommended
    to change it to 3h to ensure that the headscale instance gets the latest
    DERPMap updates when upstream is changed.

    Autogroups

    This release adds support for the three missing autogroups: self
    (experimental), member, and tagged. Please refer to the
    documentation for a detailed
    explanation.

    autogroup:self is marked as experimental and should be used with caution, but
    we need help testing it. Experimental here means two things; first, generating
    the packet filter from policies that use autogroup:self is very expensive, and
    it might perform, or straight up not work on Headscale installations with a
    large number of nodes. Second, the implementation might have bugs or edge cases
    we are not aware of, meaning that nodes or users might gain more access than
    expected. Please report bugs.

    Node store (in memory database)

    Under the hood, we have added a new datastructure to store nodes in memory. This
    datastructure is called NodeStore and aims to reduce the reading and writing
    of nodes to the database layer. We have not benchmarked it, but expect it to
    improve performance for read heavy workloads. We think of it as, "worst case" we
    have moved the bottle neck somewhere else, and "best case" we should see a good
    improvement in compute resource usage at the expense of memory usage. We are
    quite excited for this change and think it will make it easier for us to improve
    the code base over time and make it more correct and efficient.

    BREAKING

    • Remove support for 32-bit binaries
      #2692
    • Policy: Zero or empty destination port is no longer allowed
      #2606
    • Stricter hostname validation #2383
      • Hostnames must be valid DNS labels (2-63 characters, alphanumeric and
        hyphens only, cannot start/end with hyphen)
      • Client Registration (New Nodes): Invalid hostnames are automatically
        renamed to invalid-XXXXXX format
        • my-laptop → accepted as-is
        • My-Laptopmy-laptop (lowercased)
        • my_laptopinvalid-a1b2c3 (underscore not allowed)
        • test@hostinvalid-d4e5f6 (@ not allowed)
        • laptop-🚀invalid-j1k2l3 (emoji not allowed)
      • Hostinfo Updates / CLI: Invalid hostnames are rejected with an error
        • Valid names are accepted or lowercased
        • Names with invalid characters, too short (<2), too long (>63), or
          starting/ending with hyphen are rejected

    Changes

    • Database schema migration improvements for SQLite
      #2617
      • IMPORTANT: Backup your SQLite database before upgrading
      • Introduces safer table renaming migration strategy
      • Addresses longstanding database integrity issues
    • Add flag to directly manipulate the policy in the database
      #2765
    • DERPmap update frequency default changed from 24h to 3h
      #2741
    • DERPmap update mechanism has been improved with retry, and is now failing
      conservatively, preserving the old map upon failure.
      #2741
    • Add support for autogroup:member, autogroup:tagged
      #2572
    • Fix bug where return routes were being removed by policy
      #2767
    • Remove policy v1 code #2600
    • Refactor Debian/Ubuntu packaging and drop support for Ubuntu 20.04.
      #2614
    • Remove redundant check regarding noise config
      #2658
    • Refactor OpenID Connect documentation
      #2625
    • Don't crash if config file is missing
      #2656
    • Adds /robots.txt endpoint to avoid crawlers
      #2643
    • OIDC: Use group claim from UserInfo
      #2663
    • OIDC: Update user with claims from UserInfo before comparing with allowed
      groups, email and domain
      #2663
    • Policy will now reject invalid fields, making it easier to spot spelling
      errors #2764
    • Add FAQ entry on how to recover from an invalid policy in the database
      #2776
    • EXPERIMENTAL: Add support for autogroup:self
      #2789
    • Add healthcheck command #2659

    Upgrade

    Please follow the steps outlined in the upgrade guide to update your existing Headscale installation.

    It's best to update from one stable version to the next (e.g., 0.24.0 → 0.25.1 → 0.26.1) in case you are multiple releases behind. You should always pick the latest available patch release.

    Be sure to check the changelog above for version-specific upgrade instructions and breaking changes.

    Backup Your Database

    Always backup your database before upgrading. Here's how to backup a SQLite database:

    # Stop headscale
    systemctl stop headscale
    
    # Backup sqlite database
    cp /var/lib/headscale/db.sqlite /var/lib/headscale/db.sqlite.backup
    
    # Backup sqlite WAL/SHM files (if they exist)
    cp /var/lib/headscale/db.sqlite-wal /var/lib/headscale/db.sqlite-wal.backup
    cp /var/lib/headscale/db.sqlite-shm /var/lib/headscale/db.sqlite-shm.backup
    
    # Start headscale (migration will run automatically)
    systemctl start headscale
    

    Changelog

    Downloads
  • Pre-Release

    adam released this 2025-10-23 17:57:41 +02:00 | 122 commits to main since this release

    📅 Originally published on GitHub: Thu, 23 Oct 2025 16:08:52 GMT
    🏷️ Git tag created: Thu, 23 Oct 2025 15:57:41 GMT

    Minimum supported Tailscale client version: v1.64.0

    Database integrity improvements

    This release includes a significant database migration that addresses
    longstanding issues with the database schema and data integrity that has
    accumulated over the years. The migration introduces a schema.sql file as the
    source of truth for the expected database schema to ensure new migrations that
    will cause divergence does not occur again.

    These issues arose from a combination of factors discovered over time: SQLite
    foreign keys not being enforced for many early versions, all migrations being
    run in one large function until version 0.23.0, and inconsistent use of GORM's
    AutoMigrate feature. Moving forward, all new migrations will be explicit SQL
    operations rather than relying on GORM AutoMigrate, and foreign keys will be
    enforced throughout the migration process.

    We are only improving SQLite databases with this change - PostgreSQL databases
    are not affected.

    Please read the
    PR description for more
    technical details about the issues and solutions.

    SQLite Database Backup Example:

    # Stop headscale
    systemctl stop headscale
    
    # Backup sqlite database
    cp /var/lib/headscale/db.sqlite /var/lib/headscale/db.sqlite.backup
    
    # Backup sqlite WAL/SHM files (if they exist)
    cp /var/lib/headscale/db.sqlite-wal /var/lib/headscale/db.sqlite-wal.backup
    cp /var/lib/headscale/db.sqlite-shm /var/lib/headscale/db.sqlite-shm.backup
    
    # Start headscale (migration will run automatically)
    systemctl start headscale
    

    DERPMap update frequency

    The default DERPMap update frequency has been changed from 24 hours to 3 hours.
    If you set the derp.update_frequency configuration option, it is recommended
    to change it to 3h to ensure that the headscale instance gets the latest
    DERPMap updates when upstream is changed.

    Autogroups

    This release adds support for the three missing autogroups: self
    (experimental), member, and tagged. Please refer to the
    documentation for a detailed
    explanation.

    autogroup:self is marked as experimental and should be used with caution, but
    we need help testing it. Experimental here means two things; first, generating
    the packet filter from policies that use autogroup:self is very expensive, and
    it might perform, or straight up not work on Headscale installations with a
    large number of nodes. Second, the implementation might have bugs or edge cases
    we are not aware of, meaning that nodes or users might gain more access than
    expected. Please report bugs.

    Node store (in memory database)

    Under the hood, we have added a new datastructure to store nodes in memory. This
    datastructure is called NodeStore and aims to reduce the reading and writing
    of nodes to the database layer. We have not benchmarked it, but expect it to
    improve performance for read heavy workloads. We think of it as, "worst case" we
    have moved the bottle neck somewhere else, and "best case" we should see a good
    improvement in compute resource usage at the expense of memory usage. We are
    quite excited for this change and think it will make it easier for us to improve
    the code base over time and make it more correct and efficient.

    BREAKING

    • Remove support for 32-bit binaries
      #2692
    • Policy: Zero or empty destination port is no longer allowed
      #2606
    • Stricter hostname validation #2383
      • Hostnames must be valid DNS labels (2-63 characters, alphanumeric and
        hyphens only, cannot start/end with hyphen)
      • Client Registration (New Nodes): Invalid hostnames are automatically
        renamed to invalid-XXXXXX format
        • my-laptop → accepted as-is
        • My-Laptopmy-laptop (lowercased)
        • my_laptopinvalid-a1b2c3 (underscore not allowed)
        • test@hostinvalid-d4e5f6 (@ not allowed)
        • laptop-🚀invalid-j1k2l3 (emoji not allowed)
      • Hostinfo Updates / CLI: Invalid hostnames are rejected with an error
        • Valid names are accepted or lowercased
        • Names with invalid characters, too short (<2), too long (>63), or
          starting/ending with hyphen are rejected

    Changes

    • Database schema migration improvements for SQLite
      #2617
      • IMPORTANT: Backup your SQLite database before upgrading
      • Introduces safer table renaming migration strategy
      • Addresses longstanding database integrity issues
    • Add flag to directly manipulate the policy in the database
      #2765
    • DERPmap update frequency default changed from 24h to 3h
      #2741
    • DERPmap update mechanism has been improved with retry, and is now failing
      conservatively, preserving the old map upon failure.
      #2741
    • Add support for autogroup:member, autogroup:tagged
      #2572
    • Fix bug where return routes were being removed by policy
      #2767
    • Remove policy v1 code #2600
    • Refactor Debian/Ubuntu packaging and drop support for Ubuntu 20.04.
      #2614
    • Remove redundant check regarding noise config
      #2658
    • Refactor OpenID Connect documentation
      #2625
    • Don't crash if config file is missing
      #2656
    • Adds /robots.txt endpoint to avoid crawlers
      #2643
    • OIDC: Use group claim from UserInfo
      #2663
    • OIDC: Update user with claims from UserInfo before comparing with allowed
      groups, email and domain
      #2663
    • Policy will now reject invalid fields, making it easier to spot spelling
      errors #2764
    • Add FAQ entry on how to recover from an invalid policy in the database
      #2776
    • EXPERIMENTAL: Add support for autogroup:self
      #2789
    • Add healthcheck command #2659

    Changelog

    • 047dbda136 Add FAQ on how to disable log submission
    • 2a1392fb5b Add healthcheck to container docs
    • 46477b8021 Downgrade completed broadcast message to debug
    • ed38d00aaa Fix autogroup:self alternative example
    • c97d0ff23d Fix fatal error on missing config file by handling viper.ConfigFileNotFoundError
    • 8becb7e54a Mention explicitly that @ is only required in policy
    • 8010cc574e Remove outdated hint about an empty config file
    • 2c9e98d3f5 fix: guard every error statement with early return (#2810)
    • 66826232ff integration: add tests for api bypass (#2811)
    • 2bf1200483 policy: fix autogroup:self propagation and optimize cache invalidation (#2807)
    • 1cdea7ed9b stricter hostname validation and replace (#2383)
    Downloads
  • Pre-Release

    adam released this 2025-10-17 08:28:30 +02:00 | 133 commits to main since this release

    📅 Originally published on GitHub: Fri, 17 Oct 2025 07:24:08 GMT
    🏷️ Git tag created: Fri, 17 Oct 2025 06:28:30 GMT

    Minimum supported Tailscale client version: v1.64.0

    Database integrity improvements

    This release includes a significant database migration that addresses
    longstanding issues with the database schema and data integrity that has
    accumulated over the years. The migration introduces a schema.sql file as the
    source of truth for the expected database schema to ensure new migrations that
    will cause divergence does not occur again.

    These issues arose from a combination of factors discovered over time: SQLite
    foreign keys not being enforced for many early versions, all migrations being
    run in one large function until version 0.23.0, and inconsistent use of GORM's
    AutoMigrate feature. Moving forward, all new migrations will be explicit SQL
    operations rather than relying on GORM AutoMigrate, and foreign keys will be
    enforced throughout the migration process.

    We are only improving SQLite databases with this change - PostgreSQL databases
    are not affected.

    Please read the PR description for more technical details about the issues and solutions.

    SQLite Database Backup Example:

    # Stop headscale
    systemctl stop headscale
    
    # Backup sqlite database
    cp /var/lib/headscale/db.sqlite /var/lib/headscale/db.sqlite.backup
    
    # Backup sqlite WAL/SHM files (if they exist)
    cp /var/lib/headscale/db.sqlite-wal /var/lib/headscale/db.sqlite-wal.backup
    cp /var/lib/headscale/db.sqlite-shm /var/lib/headscale/db.sqlite-shm.backup
    
    # Start headscale (migration will run automatically)
    systemctl start headscale
    

    DERPMap update frequency

    The default DERPMap update frequency has been changed from 24 hours to 3 hours.
    If you set the derp.update_frequency configuration option, it is recommended
    to change it to 3h to ensure that the headscale instance gets the latest
    DERPMap updates when upstream is changed.

    Autogroups

    This release adds support for the three missing autogroups: self
    (experimental), member, and tagged. Please refer to the
    documentation for a detailed
    explanation.

    autogroup:self is marked as experimental and should be used with caution, but
    we need help testing it. Experimental here means two things; first, generating
    the packet filter from policies that use autogroup:self is very expensive, and
    it might perform, or straight up not work on Headscale installations with a
    large number of nodes. Second, the implementation might have bugs or edge cases
    we are not aware of, meaning that nodes or users might gain more access than
    expected. Please report bugs.

    Node store (in memory database)

    Under the hood, we have added a new datastructure to store nodes in memory. This
    datastructure is called NodeStore and aims to reduce the reading and writing
    of nodes to the database layer. We have not benchmarked it, but expect it to
    improve performance for read heavy workloads. We think of it as, "worst case" we
    have moved the bottle neck somewhere else, and "best case" we should see a good
    improvement in compute resource usage at the expense of memory usage. We are
    quite excited for this change and think it will make it easier for us to improve
    the code base over time and make it more correct and efficient.

    BREAKING

    • Remove support for 32-bit binaries
      #2692
    • Policy: Zero or empty destination port is no longer allowed
      #2606

    Changes

    • Database schema migration improvements for SQLite
      #2617
      • IMPORTANT: Backup your SQLite database before upgrading
      • Introduces safer table renaming migration strategy
      • Addresses longstanding database integrity issues
    • Add flag to directly manipulate the policy in the database
      #2765
    • DERPmap update frequency default changed from 24h to 3h
      #2741
    • DERPmap update mechanism has been improved with retry, and is now failing
      conservatively, preserving the old map upon failure.
      #2741
    • Add support for autogroup:member, autogroup:tagged
      #2572
    • Fix bug where return routes were being removed by policy
      #2767
    • Remove policy v1 code #2600
    • Refactor Debian/Ubuntu packaging and drop support for Ubuntu 20.04.
      #2614
    • Remove redundant check regarding noise config
      #2658
    • Refactor OpenID Connect documentation
      #2625
    • Don't crash if config file is missing
      #2656
    • Adds /robots.txt endpoint to avoid crawlers
      #2643
    • OIDC: Use group claim from UserInfo
      #2663
    • OIDC: Update user with claims from UserInfo before comparing with allowed
      groups, email and domain
      #2663
    • Policy will now reject invalid fields, making it easier to spot spelling
      errors #2764
    • Add FAQ entry on how to recover from an invalid policy in the database
      #2776
    • EXPERIMENTAL: Add support for autogroup:self
      #2789
    • Add healthcheck command #2659

    Changelog

    Downloads
  • Stable

    adam released this 2025-06-06 12:16:37 +02:00 | 257 commits to main since this release

    📅 Originally published on GitHub: Fri, 06 Jun 2025 11:22:02 GMT
    🏷️ Git tag created: Fri, 06 Jun 2025 10:16:37 GMT

    Security fix

    • Ensure nodes are matching both node key and machine key
      when connecting.
      #2642

    Commits

    Downloads
  • Stable

    adam released this 2025-05-14 16:45:14 +02:00 | 258 commits to main since this release

    📅 Originally published on GitHub: Wed, 14 May 2025 15:12:14 GMT
    🏷️ Git tag created: Wed, 14 May 2025 14:45:14 GMT

    BREAKING

    Routes

    Route internals have been rewritten, removing the dedicated route table in the
    database. This was done to simplify the codebase, which had grown unnecessarily
    complex after the routes were split into separate tables. The overhead of having
    to go via the database and keeping the state in sync made the code very hard to
    reason about and prone to errors. The majority of the route state is only
    relevant when headscale is running, and is now only kept in memory. As part of
    this, the CLI and API has been simplified to reflect the changes;

    $ headscale nodes list-routes
    ID | Hostname           | Approved | Available       | Serving (Primary)
    1  | ts-head-ruqsg8     |          | 0.0.0.0/0, ::/0 |
    2  | ts-unstable-fq7ob4 |          | 0.0.0.0/0, ::/0 |
    
    $ headscale nodes approve-routes --identifier 1 --routes 0.0.0.0/0,::/0
    Node updated
    
    $ headscale nodes list-routes
    ID | Hostname           | Approved        | Available       | Serving (Primary)
    1  | ts-head-ruqsg8     | 0.0.0.0/0, ::/0 | 0.0.0.0/0, ::/0 | 0.0.0.0/0, ::/0
    2  | ts-unstable-fq7ob4 |                 | 0.0.0.0/0, ::/0 |
    

    Note that if an exit route is approved (0.0.0.0/0 or ::/0), both IPv4 and IPv6
    will be approved.

    • Route API and CLI has been removed
      #2422
    • Routes are now managed via the Node API
      #2422
    • Only routes accessible to the node will be sent to the node
      #2561

    Policy v2

    This release introduces a new policy implementation. The new policy is a
    complete rewrite, and it introduces some significant quality and consistency
    improvements. In principle, there are not really any new features, but some long
    standing bugs should have been resolved, or be easier to fix in the future. The
    new policy code passes all of our tests.

    Changes

    • The policy is validated and "resolved" when loading, providing errors for
      invalid rules and conditions.
      • Previously this was done as a mix between load and runtime (when it was
        applied to a node).
      • This means that when you convert the first time, what was previously a
        policy that loaded, but failed at runtime, will now fail at load time.
    • Error messages should be more descriptive and informative.
      • There is still work to be here, but it is already improved with "typing"
        (e.g. only Users can be put in Groups)
    • All users must contain an @ character.
      • If your user naturally contains and @, like an email, this will just work.
      • If its based on usernames, or other identifiers not containing an @, an
        @ should be appended at the end. For example, if your user is john, it
        must be written as john@ in the policy.
    Migration notes when the policy is stored in the database.

    This section only applies if the policy is stored in the database and
    Headscale 0.26 doesn't start due to a policy error
    (failed to load ACL policy).

    • Start Headscale 0.26 with the environment variable HEADSCALE_POLICY_V1=1
      set. You can check that Headscale picked up the environment variable by
      observing this message during startup: Using policy manager version: 1
    • Dump the policy to a file: headscale policy get > policy.json
    • Edit policy.json and migrate to policy V2. Use the command
      headscale policy check --file policy.json to check for policy errors.
    • Load the modified policy: headscale policy set --file policy.json
    • Restart Headscale without the environment variable HEADSCALE_POLICY_V1.
      Headscale should now print the message Using policy manager version: 2 and
      startup successfully.

    SSH

    The SSH policy has been reworked to be more consistent with the rest of the
    policy. In addition, several inconsistencies between our implementation and
    Tailscale's upstream has been closed and this might be a breaking change for
    some users. Please refer to the
    upstream documentation
    for more information on which types are allowed in src, dst and users.

    There is one large inconsistency left, we allow * as a destination as we
    currently do not support autogroup:self, autogroup:member and
    autogroup:tagged. The support for * will be removed when we have support for
    the autogroups.

    Current state

    The new policy is passing all tests, both integration and unit tests. This does
    not mean it is perfect, but it is a good start. Corner cases that is currently
    working in v1 and not tested might be broken in v2 (and vice versa).

    We do need help testing this code

    Other breaking changes

    • Disallow server_url and base_domain to be equal
      #2544
    • Return full user in API for pre auth keys instead of string
      #2542
    • Pre auth key API/CLI now uses ID over username
      #2542

    Changes

    • Use Go 1.24 #2427
    • Add headscale policy check command to check policy
      #2553
    • oidc.map_legacy_users and oidc.strip_email_domain has been removed
      #2411
    • Add more information to /debug endpoint
      #2420
      • It is now possible to inspect running goroutines and take profiles
      • View of config, policy, filter, ssh policy per node, connected nodes and
        DERPmap
    • OIDC: Fetch UserInfo to get EmailVerified if necessary
      #2493
      • If a OIDC provider doesn't include the email_verified claim in its ID
        tokens, Headscale will attempt to get it from the UserInfo endpoint.
    • OIDC: Try to populate name, email and username from UserInfo
      #2545
    • Improve performance by only querying relevant nodes from the database for node
      updates #2509
    • node FQDNs in the netmap will now contain a dot (".") at the end. This aligns
      with behaviour of tailscale.com
      #2503
    • Restore support for "Override local DNS"
      #2438
    • Add documentation for routes
      #2496

    Changelog

    Downloads
  • Pre-Release

    adam released this 2025-05-10 09:49:08 +02:00 | 262 commits to main since this release

    📅 Originally published on GitHub: Sat, 10 May 2025 10:17:00 GMT
    🏷️ Git tag created: Sat, 10 May 2025 07:49:08 GMT

    BREAKING

    Routes

    Route internals have been rewritten, removing the dedicated route table in the
    database. This was done to simplify the codebase, which had grown unnecessarily
    complex after the routes were split into separate tables. The overhead of having
    to go via the database and keeping the state in sync made the code very hard to
    reason about and prone to errors. The majority of the route state is only
    relevant when headscale is running, and is now only kept in memory. As part of
    this, the CLI and API has been simplified to reflect the changes;

    $ headscale nodes list-routes
    ID | Hostname           | Approved | Available       | Serving (Primary)
    1  | ts-head-ruqsg8     |          | 0.0.0.0/0, ::/0 |
    2  | ts-unstable-fq7ob4 |          | 0.0.0.0/0, ::/0 |
    
    $ headscale nodes approve-routes --identifier 1 --routes 0.0.0.0/0,::/0
    Node updated
    
    $ headscale nodes list-routes
    ID | Hostname           | Approved        | Available       | Serving (Primary)
    1  | ts-head-ruqsg8     | 0.0.0.0/0, ::/0 | 0.0.0.0/0, ::/0 | 0.0.0.0/0, ::/0
    2  | ts-unstable-fq7ob4 |                 | 0.0.0.0/0, ::/0 |
    

    Note that if an exit route is approved (0.0.0.0/0 or ::/0), both IPv4 and IPv6
    will be approved.

    • Route API and CLI has been removed
      #2422
    • Routes are now managed via the Node API
      #2422
    • Only routes accessible to the node will be sent to the node
      #2561

    Policy v2

    This release introduces a new policy implementation. The new policy is a
    complete rewrite, and it introduces some significant quality and consistency
    improvements. In principle, there are not really any new features, but some long
    standing bugs should have been resolved, or be easier to fix in the future. The
    new policy code passes all of our tests.

    Changes

    • The policy is validated and "resolved" when loading, providing errors for
      invalid rules and conditions.
      • Previously this was done as a mix between load and runtime (when it was
        applied to a node).
      • This means that when you convert the first time, what was previously a
        policy that loaded, but failed at runtime, will now fail at load time.
    • Error messages should be more descriptive and informative.
      • There is still work to be here, but it is already improved with "typing"
        (e.g. only Users can be put in Groups)
    • All users must contain an @ character.
      • If your user naturally contains and @, like an email, this will just work.
      • If its based on usernames, or other identifiers not containing an @, an
        @ should be appended at the end. For example, if your user is john, it
        must be written as john@ in the policy.
    Migration notes when the policy is stored in the database.

    This section only applies if the policy is stored in the database and
    Headscale 0.26 doesn't start due to a policy error (failed to load ACL policy).

    • Start Headscale 0.26 with the environment variable HEADSCALE_POLICY_V1=1
      set. You can check that Headscale picked up the environment variable by
      observing this message during startup: Using policy manager version: 1
    • Dump the policy to a file: headscale policy get > policy.json
    • Edit policy.json and migrate to policy V2. Use the command
      headscale policy check --file policy.json to check for policy errors.
    • Load the modified policy: headscale policy set --file policy.json
    • Restart Headscale without the environment variable HEADSCALE_POLICY_V1.
      Headscale should now print the message Using policy manager version: 2 and
      startup successfully.

    SSH

    The SSH policy has been reworked to be more consistent with the rest of the
    policy. In addition, several inconsistencies between our implementation and
    Tailscale's upstream has been closed and this might be a breaking change for
    some users. Please refer to the
    upstream documentation
    for more information on which types are allowed in src, dst and users.

    There is one large inconsistency left, we allow * as a destination as we
    currently do not support autogroup:self, autogroup:member and
    autogroup:tagged. The support for * will be removed when we have support for
    the autogroups.

    Current state

    The new policy is passing all tests, both integration and unit tests. This does
    not mean it is perfect, but it is a good start. Corner cases that is currently
    working in v1 and not tested might be broken in v2 (and vice versa).

    We do need help testing this code

    Other breaking changes

    • Disallow server_url and base_domain to be equal
      #2544
    • Return full user in API for pre auth keys instead of string
      #2542
    • Pre auth key API/CLI now uses ID over username
      #2542

    Changes

    • Use Go 1.24 #2427
    • Add headscale policy check command to check policy
      #2553
    • oidc.map_legacy_users and oidc.strip_email_domain has been removed
      #2411
    • Add more information to /debug endpoint
      #2420
      • It is now possible to inspect running goroutines and take profiles
      • View of config, policy, filter, ssh policy per node, connected nodes and
        DERPmap
    • OIDC: Fetch UserInfo to get EmailVerified if necessary
      #2493
      • If a OIDC provider doesn't include the email_verified claim in its ID
        tokens, Headscale will attempt to get it from the UserInfo endpoint.
    • OIDC: Try to populate name, email and username from UserInfo
      #2545
    • Improve performance by only querying relevant nodes from the database for node
      updates #2509
    • node FQDNs in the netmap will now contain a dot (".") at the end. This aligns
      with behaviour of tailscale.com
      #2503
    • Restore support for "Override local DNS"
      #2438
    • Add documentation for routes
      #2496

    Changelog

    Downloads
  • Pre-Release

    adam released this 2025-05-04 22:06:44 +02:00 | 269 commits to main since this release

    📅 Originally published on GitHub: Sun, 04 May 2025 20:35:17 GMT
    🏷️ Git tag created: Sun, 04 May 2025 20:06:44 GMT

    BREAKING

    Routes

    Route internals have been rewritten, removing the dedicated route table in the
    database. This was done to simplify the codebase, which had grown unnecessarily
    complex after the routes were split into separate tables. The overhead of having
    to go via the database and keeping the state in sync made the code very hard to
    reason about and prone to errors. The majority of the route state is only
    relevant when headscale is running, and is now only kept in memory. As part of
    this, the CLI and API has been simplified to reflect the changes;

    $ headscale nodes list-routes
    ID | Hostname           | Approved | Available       | Serving (Primary)
    1  | ts-head-ruqsg8     |          | 0.0.0.0/0, ::/0 |
    2  | ts-unstable-fq7ob4 |          | 0.0.0.0/0, ::/0 |
    
    $ headscale nodes approve-routes --identifier 1 --routes 0.0.0.0/0,::/0
    Node updated
    
    $ headscale nodes list-routes
    ID | Hostname           | Approved        | Available       | Serving (Primary)
    1  | ts-head-ruqsg8     | 0.0.0.0/0, ::/0 | 0.0.0.0/0, ::/0 | 0.0.0.0/0, ::/0
    2  | ts-unstable-fq7ob4 |                 | 0.0.0.0/0, ::/0 |
    

    Note that if an exit route is approved (0.0.0.0/0 or ::/0), both IPv4 and IPv6
    will be approved.

    • Route API and CLI has been removed
      #2422
    • Routes are now managed via the Node API
      #2422
    • Only routes accessible to the node will be sent to the node
      #2561

    Policy v2

    This release introduces a new policy implementation. The new policy is a
    complete rewrite, and it introduces some significant quality and consistency
    improvements. In principle, there are not really any new features, but some long
    standing bugs should have been resolved, or be easier to fix in the future. The
    new policy code passes all of our tests.

    Changes

    • The policy is validated and "resolved" when loading, providing errors for
      invalid rules and conditions.
      • Previously this was done as a mix between load and runtime (when it was
        applied to a node).
      • This means that when you convert the first time, what was previously a
        policy that loaded, but failed at runtime, will now fail at load time.
    • Error messages should be more descriptive and informative.
      • There is still work to be here, but it is already improved with "typing"
        (e.g. only Users can be put in Groups)
    • All users must contain an @ character.
      • If your user naturally contains and @, like an email, this will just work.
      • If its based on usernames, or other identifiers not containing an @, an
        @ should be appended at the end. For example, if your user is john, it
        must be written as john@ in the policy.

    SSH

    The SSH policy has been reworked to be more consistent with the rest of the
    policy. In addition, several inconsistencies between our implementation and
    Tailscale's upstream has been closed and this might be a breaking change for
    some users. Please refer to the
    upstream documentation
    for more information on which types are allowed in src, dst and users.

    There is one large inconsistency left, we allow * as a destination as we
    currently do not support autogroup:self, autogroup:member and
    autogroup:tagged. The support for * will be removed when we have support for
    the autogroups.

    Current state

    The new policy is passing all tests, both integration and unit tests. This does
    not mean it is perfect, but it is a good start. Corner cases that is currently
    working in v1 and not tested might be broken in v2 (and vice versa).

    We do need help testing this code

    Other breaking changes

    • Disallow server_url and base_domain to be equal
      #2544
    • Return full user in API for pre auth keys instead of string
      #2542
    • Pre auth key API/CLI now uses ID over username
      #2542

    Changes

    • Use Go 1.24 #2427
    • Add headscale policy check command to check policy
      #2553
    • oidc.map_legacy_users and oidc.strip_email_domain has been removed
      #2411
    • Add more information to /debug endpoint
      #2420
      • It is now possible to inspect running goroutines and take profiles
      • View of config, policy, filter, ssh policy per node, connected nodes and
        DERPmap
    • OIDC: Fetch UserInfo to get EmailVerified if necessary
      #2493
      • If a OIDC provider doesn't include the email_verified claim in its ID
        tokens, Headscale will attempt to get it from the UserInfo endpoint.
    • OIDC: Try to populate name, email and username from UserInfo
      #2545
    • Improve performance by only querying relevant nodes from the database for node
      updates #2509
    • node FQDNs in the netmap will now contain a dot (".") at the end. This aligns
      with behaviour of tailscale.com
      #2503
    • Restore support for "Override local DNS"
      #2438
    • Add documentation for routes
      #2496

    Changelog

    Downloads