starred/headscale
1
0
Fork 0
mirror of https://github.com/juanfont/headscale.git synced 2026-01-13 04:40:29 +01:00
Code Issues 98 Packages Projects Releases 10 Wiki Activity

[Bug] tailscale on ios cannot connect to the postgresql server #750

New Issue
Closed
opened 2025-12-29 02:23:24 +01:00 by adam · 8 comments
adam commented 2025-12-29 02:23:24 +01:00
Owner
Copy Link

Originally created by @wangzhijie0602 on GitHub (Jul 24, 2024).

Is this a support request?

  • This is not a support request

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

After I switched my headscale's database from sqlite to postgresql, I used my iPhone to reconnect, and then he prompted the following problem and could not connect correctly, which did not appear on sqlite's headscale.
IMG_1713

Expected Behavior

Can connect to the server normally

Steps To Reproduce

install headscale and postgresql in ubuntu22, configure the configuration file and start it, then use tailscale of iPhone to connect to the headscale server

Environment

- OS: ubuntu 22.04
- Headscale version: v0.23.0-beta1
- Tailscale version: v1.70.0

Runtime environment

  • Headscale is behind a (reverse) proxy
  • Headscale runs in a container

Anything else?

No response

Originally created by @wangzhijie0602 on GitHub (Jul 24, 2024). ### Is this a support request? - [X] This is not a support request ### Is there an existing issue for this? - [X] I have searched the existing issues ### Current Behavior After I switched my headscale's database from sqlite to postgresql, I used my iPhone to reconnect, and then he prompted the following problem and could not connect correctly, which did not appear on sqlite's headscale. ![IMG_1713](https://github.com/user-attachments/assets/841121fa-c829-4db6-8511-541a3c040556) ### Expected Behavior Can connect to the server normally ### Steps To Reproduce install headscale and postgresql in ubuntu22, configure the configuration file and start it, then use tailscale of iPhone to connect to the headscale server ### Environment ```markdown - OS: ubuntu 22.04 - Headscale version: v0.23.0-beta1 - Tailscale version: v1.70.0 ``` ### Runtime environment - [X] Headscale is behind a (reverse) proxy - [ ] Headscale runs in a container ### Anything else? _No response_
adam added the bug label 2025-12-29 02:23:24 +01:00
adam closed this issue 2025-12-29 02:23:24 +01:00
adam commented 2025-12-29 02:23:24 +01:00
Author
Owner
Copy Link

@panteparak commented on GitHub (Jul 27, 2024):

which Headscale version are you running? I've just migrated SQLite to Postgres with a few hiccups on DB side, but solvable by running a few SQL commands.

Can you outline a few more details as listed below?

  • Outline migration steps/tools used to migrate SQLite to Postgres and what versions are they running on?
  • Show us the current SQL Database Schema. There are a few incompatible fields between SQLite and Postgres
  • Run headscale with DEBUG Log Level and show us the logs
  • Which Reverse Proxy are you running and the configurations are you using
@panteparak commented on GitHub (Jul 27, 2024): which Headscale version are you running? I've just migrated SQLite to Postgres with a few hiccups on DB side, but solvable by running a few SQL commands. Can you outline a few more details as listed below? - Outline migration steps/tools used to migrate SQLite to Postgres and what versions are they running on? - Show us the current SQL Database Schema. There are a few incompatible fields between SQLite and Postgres - Run headscale with `DEBUG` Log Level and show us the logs - Which Reverse Proxy are you running and the configurations are you using
adam commented 2025-12-29 02:23:25 +01:00
Author
Owner
Copy Link

@wangzhijie0602 commented on GitHub (Jul 28, 2024):

I'm glad you can reply to me, I will provide you with more information next

First of all, I didn't migrate from SQLite to Postgres. I directly created an empty Postgres and rejoined it using the tailscale client. I wanted to test Postgres first, so I didn't have time to migrate my SQLite yet.

I created two new servers to reproduce the problem. What follows is a description based on the new environment.

Next I will introduce my environment below

For server side

  • System: ubuntu 22.04 LTS
  • headscale: v0.23.0-beta1
  • Postgres : 14

For client

Client 1:

  • System: ios 17.5.1
  • tailscale: v1.70.0

Client 2:

  • System: ubuntu 22.04 LTS
  • tailscale: v1.70.0

The following is my headscale config file

---
# headscale will look for a configuration file named `config.yaml` (or `config.json`) in the following order:
#
# - `/etc/headscale`
# - `~/.headscale`
# - current working directory

# The url clients will connect to.
# Typically this will be a domain like:
#
# https://myheadscale.example.com:443
#
server_url: http://116.62.188.235:8080

# Address to listen to / bind to on the server
#
# For production:
# listen_addr: 0.0.0.0:8080
listen_addr: 0.0.0.0:8080

# Address to listen to /metrics, you may want
# to keep this endpoint private to your internal
# network
#
metrics_listen_addr: 127.0.0.1:9090

# Address to listen for gRPC.
# gRPC is used for controlling a headscale server
# remotely with the CLI
# Note: Remote access _only_ works if you have
# valid certificates.
#
# For production:
# grpc_listen_addr: 0.0.0.0:50443
grpc_listen_addr: 127.0.0.1:50443

# Allow the gRPC admin interface to run in INSECURE
# mode. This is not recommended as the traffic will
# be unencrypted. Only enable if you know what you
# are doing.
grpc_allow_insecure: false

# The Noise section includes specific configuration for the
# TS2021 Noise protocol
noise:
  # The Noise private key is used to encrypt the
  # traffic between headscale and Tailscale clients when
  # using the new Noise-based protocol.
  private_key_path: /var/lib/headscale/noise_private.key

# List of IP prefixes to allocate tailaddresses from.
# Each prefix consists of either an IPv4 or IPv6 address,
# and the associated prefix length, delimited by a slash.
# It must be within IP ranges supported by the Tailscale
# client - i.e., subnets of 100.64.0.0/10 and fd7a:115c:a1e0::/48.
# See below:
# IPv6: https://github.com/tailscale/tailscale/blob/22ebb25e833264f58d7c3f534a8b166894a89536/net/tsaddr/tsaddr.go#LL81C52-L81C71
# IPv4: https://github.com/tailscale/tailscale/blob/22ebb25e833264f58d7c3f534a8b166894a89536/net/tsaddr/tsaddr.go#L33
# Any other range is NOT supported, and it will cause unexpected issues.
prefixes:
  v6: fd7a:115c:a1e0::/48
  v4: 100.64.0.0/10

  # Strategy used for allocation of IPs to nodes, available options:
  # - sequential (default): assigns the next free IP from the previous given IP.
  # - random: assigns the next free IP from a pseudo-random IP generator (crypto/rand).
  allocation: sequential

# DERP is a relay system that Tailscale uses when a direct
# connection cannot be established.
# https://tailscale.com/blog/how-tailscale-works/#encrypted-tcp-relays-derp
#
# headscale needs a list of DERP servers that can be presented
# to the clients.
derp:
  server:
    # If enabled, runs the embedded DERP server and merges it into the rest of the DERP config
    # The Headscale server_url defined above MUST be using https, DERP requires TLS to be in place
    enabled: false

    # Region ID to use for the embedded DERP server.
    # The local DERP prevails if the region ID collides with other region ID coming from
    # the regular DERP config.
    region_id: 999

    # Region code and name are displayed in the Tailscale UI to identify a DERP region
    region_code: "headscale"
    region_name: "Headscale Embedded DERP"

    # Listens over UDP at the configured address for STUN connections - to help with NAT traversal.
    # When the embedded DERP server is enabled stun_listen_addr MUST be defined.
    #
    # For more details on how this works, check this great article: https://tailscale.com/blog/how-tailscale-works/
    stun_listen_addr: "0.0.0.0:3478"

    # Private key used to encrypt the traffic between headscale DERP
    # and Tailscale clients.
    # The private key file will be autogenerated if it's missing.
    #
    private_key_path: /var/lib/headscale/derp_server_private.key

    # This flag can be used, so the DERP map entry for the embedded DERP server is not written automatically,
    # it enables the creation of your very own DERP map entry using a locally available file with the parameter DERP.paths
    # If you enable the DERP server and set this to false, it is required to add the DERP server to the DERP map using DERP.paths
    automatically_add_embedded_derp_region: true

    # For better connection stability (especially when using an Exit-Node and DNS is not working),
    # it is possible to optionally add the public IPv4 and IPv6 address to the Derp-Map using:
    ipv4: 1.2.3.4
    ipv6: 2001:db8::1

  # List of externally available DERP maps encoded in JSON
  urls:
    - https://controlplane.tailscale.com/derpmap/default

  # Locally available DERP map files encoded in YAML
  #
  # This option is mostly interesting for people hosting
  # their own DERP servers:
  # https://tailscale.com/kb/1118/custom-derp-servers/
  #
  # paths:
  #   - /etc/headscale/derp-example.yaml
  paths: []

  # If enabled, a worker will be set up to periodically
  # refresh the given sources and update the derpmap
  # will be set up.
  auto_update_enabled: true

  # How often should we check for DERP updates?
  update_frequency: 24h

# Disables the automatic check for headscale updates on startup
disable_check_updates: false

# Time before an inactive ephemeral node is deleted?
ephemeral_node_inactivity_timeout: 30m

database:
  type: postgres

  # # SQLite config
  # sqlite:
  #   path: /var/lib/headscale/db.sqlite

  #   # Enable WAL mode for SQLite. This is recommended for production environments.
  #   # https://www.sqlite.org/wal.html
  #   write_ahead_log: true

  # Postgres config
  postgres:
    # If using a Unix socket to connect to Postgres, set the socket path in the 'host' field and leave 'port' blank.
    host: localhost
    port: 5432
    name: headscale
    user: foo
    pass: bar
    max_open_conns: 10
    max_idle_conns: 10
    conn_max_idle_time_secs: 3600

    # If other 'sslmode' is required instead of 'require(true)' and 'disabled(false)', set the 'sslmode' you need
    # in the 'ssl' field. Refers to https://www.postgresql.org/docs/current/libpq-ssl.html Table 34.1.
    ssl: false

### TLS configuration
#
## Let's encrypt / ACME
#
# headscale supports automatically requesting and setting up
# TLS for a domain with Let's Encrypt.
#
# URL to ACME directory
acme_url: https://acme-v02.api.letsencrypt.org/directory

# Email to register with ACME provider
acme_email: ""

# Domain name to request a TLS certificate for:
tls_letsencrypt_hostname: ""

# Path to store certificates and metadata needed by
# letsencrypt
# For production:
tls_letsencrypt_cache_dir: /var/lib/headscale/cache

# Type of ACME challenge to use, currently supported types:
# HTTP-01 or TLS-ALPN-01
# See [docs/tls.md](docs/tls.md) for more information
tls_letsencrypt_challenge_type: HTTP-01
# When HTTP-01 challenge is chosen, letsencrypt must set up a
# verification endpoint, and it will be listening on:
# :http = port 80
tls_letsencrypt_listen: ":http"

## Use already defined certificates:
tls_cert_path: ""
tls_key_path: ""

log:
  # Output formatting for logs: text or json
  format: text
  level: debug

## Policy
# headscale supports Tailscale's ACL policies.
# Please have a look to their KB to better
# understand the concepts: https://tailscale.com/kb/1018/acls/
policy:
  # The mode can be "file" or "database" that defines
  # where the ACL policies are stored and read from.
  mode: file
  # If the mode is set to "file", the
  # path to a file containing ACL policies.
  # The file can be in YAML or HuJSON format.
  path: ""

## DNS
#
# headscale supports Tailscale's DNS configuration and MagicDNS.
# Please have a look to their KB to better understand the concepts:
#
# - https://tailscale.com/kb/1054/dns/
# - https://tailscale.com/kb/1081/magicdns/
# - https://tailscale.com/blog/2021-09-private-dns-with-magicdns/
#
dns_config:
  # Whether to prefer using Headscale provided DNS or use local.
  override_local_dns: true

  # List of DNS servers to expose to clients.
  nameservers:
    - 1.1.1.1

  # NextDNS (see https://tailscale.com/kb/1218/nextdns/).
  # "abc123" is example NextDNS ID, replace with yours.
  #
  # With metadata sharing:
  # nameservers:
  #   - https://dns.nextdns.io/abc123
  #
  # Without metadata sharing:
  # nameservers:
  #   - 2a07:a8c0::ab:c123
  #   - 2a07:a8c1::ab:c123

  # Split DNS (see https://tailscale.com/kb/1054/dns/),
  # list of search domains and the DNS to query for each one.
  #
  # restricted_nameservers:
  #   foo.bar.com:
  #     - 1.1.1.1
  #   darp.headscale.net:
  #     - 1.1.1.1
  #     - 8.8.8.8

  # Search domains to inject.
  domains: []

  # Extra DNS records
  # so far only A-records are supported (on the tailscale side)
  # See https://github.com/juanfont/headscale/blob/main/docs/dns-records.md#Limitations
  # extra_records:
  #   - name: "grafana.myvpn.example.com"
  #     type: "A"
  #     value: "100.64.0.3"
  #
  #   # you can also put it in one line
  #   - { name: "prometheus.myvpn.example.com", type: "A", value: "100.64.0.3" }

  # Whether to use [MagicDNS](https://tailscale.com/kb/1081/magicdns/).
  # Only works if there is at least a nameserver defined.
  magic_dns: true

  # DEPRECATED
  # Use the username as part of the DNS name for nodes, with this option enabled:
  # node1.username.example.com
  # while when this is disabled:
  # node1.example.com
  # This is a legacy option as Headscale has have this wrongly implemented
  # while in upstream Tailscale, the username is not included.
  use_username_in_magic_dns: false

  # Defines the base domain to create the hostnames for MagicDNS.
  # `base_domain` must be a FQDNs, without the trailing dot.
  # The FQDN of the hosts will be
  # `hostname.user.base_domain` (e.g., _myhost.myuser.example.com_).
  base_domain: example.com

# Unix socket used for the CLI to connect without authentication
# Note: for production you will want to set this to something like:
unix_socket: /var/run/headscale/headscale.sock
unix_socket_permission: "0770"
#
# headscale supports experimental OpenID connect support,
# it is still being tested and might have some bugs, please
# help us test it.
# OpenID Connect
# oidc:
#   only_start_if_oidc_is_available: true
#   issuer: "https://your-oidc.issuer.com/path"
#   client_id: "your-oidc-client-id"
#   client_secret: "your-oidc-client-secret"
#   # Alternatively, set `client_secret_path` to read the secret from the file.
#   # It resolves environment variables, making integration to systemd's
#   # `LoadCredential` straightforward:
#   client_secret_path: "${CREDENTIALS_DIRECTORY}/oidc_client_secret"
#   # client_secret and client_secret_path are mutually exclusive.
#
#   # The amount of time from a node is authenticated with OpenID until it
#   # expires and needs to reauthenticate.
#   # Setting the value to "0" will mean no expiry.
#   expiry: 180d
#
#   # Use the expiry from the token received from OpenID when the user logged
#   # in, this will typically lead to frequent need to reauthenticate and should
#   # only been enabled if you know what you are doing.
#   # Note: enabling this will cause `oidc.expiry` to be ignored.
#   use_expiry_from_token: false
#
#   # Customize the scopes used in the OIDC flow, defaults to "openid", "profile" and "email" and add custom query
#   # parameters to the Authorize Endpoint request. Scopes default to "openid", "profile" and "email".
#
#   scope: ["openid", "profile", "email", "custom"]
#   extra_params:
#     domain_hint: example.com
#
#   # List allowed principal domains and/or users. If an authenticated user's domain is not in this list, the
#   # authentication request will be rejected.
#
#   allowed_domains:
#     - example.com
#   # Note: Groups from keycloak have a leading '/'
#   allowed_groups:
#     - /headscale
#   allowed_users:
#     - alice@example.com
#
#   # If `strip_email_domain` is set to `true`, the domain part of the username email address will be removed.
#   # This will transform `first-name.last-name@example.com` to the user `first-name.last-name`
#   # If `strip_email_domain` is set to `false` the domain part will NOT be removed resulting to the following
#   user: `first-name.last-name.example.com`
#
#   strip_email_domain: true

# Logtail configuration
# Logtail is Tailscales logging and auditing infrastructure, it allows the control panel
# to instruct tailscale nodes to log their activity to a remote server.
logtail:
  # Enable logtail for this headscales clients.
  # As there is currently no support for overriding the log server in headscale, this is
  # disabled by default. Enabling this will make your clients send logs to Tailscale Inc.
  enabled: false

# Enabling this option makes devices prefer a random port for WireGuard traffic over the
# default static port 41641. This option is intended as a workaround for some buggy
# firewall devices. See https://tailscale.com/kb/1181/firewalls/ for more information.
randomize_client_port: true

The following is my debug log after starting the headscale server

root@iZbp1db1jr2xfl77stpy0xZ:~# headscale serve
2024-07-28T13:48:45+08:00 INF Opening database database=postgres path="host=localhost dbname=headscale user=foo"
2024-07-28T13:48:46+08:00 DBG Loading DERPMap from path func=GetDERPMap url=https://controlplane.tailscale.com/derpmap/default
2024-07-28T13:48:46+08:00 INF Setting up a DERPMap update worker frequency=86400000
2024-07-28T13:48:46+08:00 INF listening and serving HTTP on: 0.0.0.0:8080
2024-07-28T13:48:46+08:00 INF listening and serving debug and metrics on: 127.0.0.1:9090
2024-07-28T13:55:28+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T13:55:28+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Node not found in database, creating new expiry=-62135596800 followup= machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old=
2024-07-28T13:55:28+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup= machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old=
2024-07-28T13:55:29+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T13:55:34+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old=
2024-07-28T13:55:34+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T13:55:39+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old=
2024-07-28T13:55:39+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T13:55:44+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old=
2024-07-28T13:55:44+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T13:55:49+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old=
2024-07-28T13:55:50+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T13:55:55+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old=
2024-07-28T13:55:55+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T13:56:00+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old=
2024-07-28T13:56:01+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T13:56:06+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old=
2024-07-28T13:56:06+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T13:56:11+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old=
2024-07-28T13:56:12+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T13:56:17+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old=
2024-07-28T13:56:18+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T13:56:22+08:00 DBG Registering node from API/CLI or auth callback expiresAt=<nil> machine_key=[LHXNY] registrationMethod=cli userName=test
2024-07-28T13:56:22+08:00 DBG Registering node machine_key=[LHXNY] node=localhost node_key=[b8IwI] user=test
2024-07-28T13:56:23+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old=
2024-07-28T13:56:25+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T13:56:25+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=localhost
2024-07-28T13:56:25+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=localhost
2024-07-28T13:56:25+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc0005f7380, chan: 0xc00054bbc0 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T13:56:46+08:00 DBG Registering node from API/CLI or auth callback expiresAt=<nil> machine_key=[LHXNY] registrationMethod=cli userName=test
2024-07-28T13:56:52+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc0005f7380, chan: 0xc00054bbc0 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T13:56:53+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T13:56:53+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=localhost
2024-07-28T13:56:53+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=localhost
2024-07-28T13:56:53+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc0005f6480, chan: 0xc00054b0e0 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T13:56:53+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc0005f7680, chan: 0xc000104d80 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T13:56:53+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc0005f6480, chan: 0xc00054b0e0 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T13:56:53+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T13:56:53+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:641 > We have the OldNodeKey in the database. This is a key refresh node=localhost
2024-07-28T13:56:53+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:682 > Node key successfully refreshed node=localhost node_key=[02gJr] old_node_key=[b8IwI]
2024-07-28T13:56:53+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc0005f7680, chan: 0xc000104d80 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T13:56:53+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc0005f7980, chan: 0xc00009bf20 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T14:01:40+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc0005f7980, chan: 0xc00009bf20 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T14:01:43+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:01:43+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=localhost
2024-07-28T14:01:43+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=localhost
2024-07-28T14:01:43+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc0005f6480, chan: 0xc000bd4960 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T14:01:44+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc0005f6480, chan: 0xc000bd4960 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T14:01:44+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000686300, chan: 0xc00057c6c0 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T14:01:44+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:01:44+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:641 > We have the OldNodeKey in the database. This is a key refresh node=localhost
2024-07-28T14:01:44+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:682 > Node key successfully refreshed node=localhost node_key=[J0Y32] old_node_key=[02gJr]
2024-07-28T14:01:44+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc0005f7380, chan: 0xc000776180 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T14:01:44+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000686300, chan: 0xc00057c6c0 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T14:12:34+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc0005f7380, chan: 0xc000776180 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T14:12:40+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:12:40+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=localhost
2024-07-28T14:12:40+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=localhost
2024-07-28T14:12:40+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000686600, chan: 0xc000104720 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T14:12:40+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000686600, chan: 0xc000104720 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T14:12:40+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000002300, chan: 0xc000499260 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T14:12:40+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:12:40+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:641 > We have the OldNodeKey in the database. This is a key refresh node=localhost
2024-07-28T14:12:40+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:682 > Node key successfully refreshed node=localhost node_key=[vVshz] old_node_key=[J0Y32]
2024-07-28T14:12:40+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000002480, chan: 0xc0001304e0 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T14:12:40+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000002300, chan: 0xc000499260 node=localhost node.id=1 omitPeers=false readOnly=false stream=true
2024-07-28T14:25:37+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:25:37+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Node not found in database, creating new expiry=-62135596800 followup= machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old=
2024-07-28T14:25:37+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e expiry=-62135596800 followup= machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old=
2024-07-28T14:25:37+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:25:42+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old=
2024-07-28T14:25:42+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:25:47+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old=
2024-07-28T14:25:47+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:25:52+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old=
2024-07-28T14:25:52+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:25:53+08:00 DBG Registering node from API/CLI or auth callback expiresAt=<nil> machine_key=[4eaGW] registrationMethod=cli userName=test
2024-07-28T14:25:53+08:00 DBG Registering node machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] user=test
2024-07-28T14:25:57+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old=
2024-07-28T14:25:57+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:25:57+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=iZbp128wzc2zwaz21j6x3rZ
2024-07-28T14:25:57+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=iZbp128wzc2zwaz21j6x3rZ
2024-07-28T14:25:57+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000686a80, chan: 0xc000499bc0 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true
2024-07-28T14:26:44+08:00 DBG Registering node from API/CLI or auth callback expiresAt=<nil> machine_key=[4eaGW] registrationMethod=cli userName=test
2024-07-28T14:26:52+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:26:52+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000686a80, chan: 0xc000499bc0 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true
2024-07-28T14:26:52+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=iZbp128wzc2zwaz21j6x3rZ
2024-07-28T14:26:52+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=iZbp128wzc2zwaz21j6x3rZ
2024-07-28T14:26:52+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc0005f7b00, chan: 0xc0004b6840 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true
2024-07-28T14:26:59+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:26:59+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc0005f7b00, chan: 0xc0004b6840 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true
2024-07-28T14:26:59+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=iZbp128wzc2zwaz21j6x3rZ
2024-07-28T14:26:59+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=iZbp128wzc2zwaz21j6x3rZ
2024-07-28T14:26:59+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000686900, chan: 0xc00057d320 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true
2024-07-28T14:27:04+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000686900, chan: 0xc00057d320 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true
2024-07-28T14:27:06+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:27:06+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=iZbp128wzc2zwaz21j6x3rZ
2024-07-28T14:27:06+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=iZbp128wzc2zwaz21j6x3rZ
2024-07-28T14:27:06+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000002300, chan: 0xc000c3d5c0 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true
2024-07-28T14:27:06+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:27:06+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000002300, chan: 0xc000c3d5c0 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true
2024-07-28T14:27:06+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=iZbp128wzc2zwaz21j6x3rZ
2024-07-28T14:27:06+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=iZbp128wzc2zwaz21j6x3rZ
2024-07-28T14:27:06+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000686300, chan: 0xc000776720 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true
2024-07-28T14:27:31+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:27:31+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000686300, chan: 0xc000776720 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true
2024-07-28T14:27:31+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=iZbp128wzc2zwaz21j6x3rZ
2024-07-28T14:27:31+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=iZbp128wzc2zwaz21j6x3rZ
2024-07-28T14:27:31+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000686d80, chan: 0xc00057dbc0 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true
2024-07-28T14:27:37+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000686d80, chan: 0xc00057dbc0 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true
2024-07-28T14:29:03+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:29:03+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=iZbp128wzc2zwaz21j6x3rZ
2024-07-28T14:29:03+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=iZbp128wzc2zwaz21j6x3rZ
2024-07-28T14:29:03+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000686480, chan: 0xc000bd4de0 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true
2024-07-28T14:29:42+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:29:42+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000686480, chan: 0xc000bd4de0 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true
2024-07-28T14:29:42+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=iZbp128wzc2zwaz21j6x3rZ
2024-07-28T14:29:42+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=iZbp128wzc2zwaz21j6x3rZ
2024-07-28T14:29:42+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000686a80, chan: 0xc0004b6ae0 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true
2024-07-28T14:30:02+08:00 ERR ../home/runner/work/headscale/headscale/hscontrol/poll.go:719 > Could not get machine from db error="record not found" node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true
2024/07/28 14:30:02 http2: panic serving 120.26.193.183:52916: runtime error: invalid memory address or nil pointer dereference
goroutine 3669 [running]:
golang.org/x/net/http2.(*serverConn).runHandler.func1()
        /home/runner/go/pkg/mod/golang.org/x/net@v0.25.0/http2/server.go:2363 +0x145
panic({0x1cf2040?, 0x337e5a0?})
        /nix/store/3v17dij8rvg7q99009swxg52995r7s22-go-1.22.5/share/go/src/runtime/panic.go:770 +0x132
github.com/juanfont/headscale/hscontrol.(*mapSession).serveLongPoll.func1()
        /home/runner/work/headscale/headscale/hscontrol/poll.go:207 +0x90
github.com/juanfont/headscale/hscontrol.(*mapSession).serveLongPoll(0xc000686a80)
        /home/runner/work/headscale/headscale/hscontrol/poll.go:291 +0x1254
github.com/juanfont/headscale/hscontrol.(*noiseServer).NoisePollNetMapHandler(0xc0006b3a70, {0x2380150, 0xc000121568}, 0xc0000abd40)
        /home/runner/work/headscale/headscale/hscontrol/noise.go:240 +0x365
net/http.HandlerFunc.ServeHTTP(0x7fc66366f108?, {0x2380150?, 0xc000121568?}, 0x69ec60?)
        /nix/store/3v17dij8rvg7q99009swxg52995r7s22-go-1.22.5/share/go/src/net/http/server.go:2171 +0x29
github.com/juanfont/headscale/hscontrol.prometheusMiddleware.func1({0x2380150, 0xc000121568}, 0xc0000abd40)
        /home/runner/work/headscale/headscale/hscontrol/metrics.go:87 +0x143
net/http.HandlerFunc.ServeHTTP(0xc0000abc20?, {0x2380150?, 0xc000121568?}, 0xc0004cc808?)
        /nix/store/3v17dij8rvg7q99009swxg52995r7s22-go-1.22.5/share/go/src/net/http/server.go:2171 +0x29
github.com/gorilla/mux.(*Router).ServeHTTP(0xc0005263c0, {0x2380150, 0xc000121568}, 0xc00055aa20)
        /home/runner/go/pkg/mod/github.com/gorilla/mux@v1.8.1/mux.go:212 +0x1e2
golang.org/x/net/http2.(*serverConn).runHandler(0x44975d?, 0xc000603fd0?, 0xb9a885?, 0xc000c31200?)
        /home/runner/go/pkg/mod/golang.org/x/net@v0.25.0/http2/server.go:2370 +0xbb
created by golang.org/x/net/http2.(*serverConn).scheduleHandler in goroutine 3655
        /home/runner/go/pkg/mod/golang.org/x/net@v0.25.0/http2/server.go:2305 +0x21d
2024-07-28T14:30:02+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap
2024-07-28T14:30:02+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap
2024-07-28T14:30:02+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap
2024-07-28T14:30:03+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap
2024-07-28T14:30:03+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap
2024-07-28T14:30:03+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap
2024-07-28T14:30:04+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap
2024-07-28T14:30:04+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap
2024-07-28T14:30:05+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap
2024-07-28T14:30:06+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap
2024-07-28T14:30:07+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap
2024-07-28T14:30:08+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap
2024-07-28T14:30:09+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:30:09+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Node not found in database, creating new expiry=-62135596800 followup= machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old=
2024-07-28T14:30:09+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e expiry=-62135596800 followup= machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old=
2024-07-28T14:30:09+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:30:14+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old=
2024-07-28T14:30:14+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:30:19+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old=
2024-07-28T14:30:20+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:30:24+08:00 DBG Registering node from API/CLI or auth callback expiresAt=<nil> machine_key=[4eaGW] registrationMethod=cli userName=test
2024-07-28T14:30:24+08:00 DBG Registering node machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] user=test
2024-07-28T14:30:25+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old=
2024-07-28T14:30:25+08:00 DBG New noise client cap_ver=102 handler=/key
2024-07-28T14:30:25+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=iZbp128wzc2zwaz21j6x3rZ
2024-07-28T14:30:25+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=iZbp128wzc2zwaz21j6x3rZ
2024-07-28T14:30:25+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc0005f7380, chan: 0xc000d50ae0 node=iZbp128wzc2zwaz21j6x3rZ node.id=3 omitPeers=false readOnly=false stream=true
2024-07-28T14:33:16+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc0005f7380, chan: 0xc000d50ae0 node=iZbp128wzc2zwaz21j6x3rZ node.id=3 omitPeers=false readOnly=false stream=true

Here I removed the reverse proxy and used IP to connect directly. The derp server uses the official derp server. It still has the same problem as before, so I infer that this problem has nothing to do with the reverse proxy.

@wangzhijie0602 commented on GitHub (Jul 28, 2024): I'm glad you can reply to me, I will provide you with more information next First of all, I didn't migrate from SQLite to Postgres. I directly created an empty Postgres and rejoined it using the tailscale client. I wanted to test Postgres first, so I didn't have time to migrate my SQLite yet. I created two new servers to reproduce the problem. What follows is a description based on the new environment. **Next I will introduce my environment below** For server side - System: ubuntu 22.04 LTS - headscale: [v0.23.0-beta1](https://github.com/juanfont/headscale/releases/tag/v0.23.0-beta1) - Postgres : 14 For client Client 1: - System: ios 17.5.1 - tailscale: v1.70.0 Client 2: - System: ubuntu 22.04 LTS - tailscale: v1.70.0 **The following is my headscale config file** ```yaml --- # headscale will look for a configuration file named `config.yaml` (or `config.json`) in the following order: # # - `/etc/headscale` # - `~/.headscale` # - current working directory # The url clients will connect to. # Typically this will be a domain like: # # https://myheadscale.example.com:443 # server_url: http://116.62.188.235:8080 # Address to listen to / bind to on the server # # For production: # listen_addr: 0.0.0.0:8080 listen_addr: 0.0.0.0:8080 # Address to listen to /metrics, you may want # to keep this endpoint private to your internal # network # metrics_listen_addr: 127.0.0.1:9090 # Address to listen for gRPC. # gRPC is used for controlling a headscale server # remotely with the CLI # Note: Remote access _only_ works if you have # valid certificates. # # For production: # grpc_listen_addr: 0.0.0.0:50443 grpc_listen_addr: 127.0.0.1:50443 # Allow the gRPC admin interface to run in INSECURE # mode. This is not recommended as the traffic will # be unencrypted. Only enable if you know what you # are doing. grpc_allow_insecure: false # The Noise section includes specific configuration for the # TS2021 Noise protocol noise: # The Noise private key is used to encrypt the # traffic between headscale and Tailscale clients when # using the new Noise-based protocol. private_key_path: /var/lib/headscale/noise_private.key # List of IP prefixes to allocate tailaddresses from. # Each prefix consists of either an IPv4 or IPv6 address, # and the associated prefix length, delimited by a slash. # It must be within IP ranges supported by the Tailscale # client - i.e., subnets of 100.64.0.0/10 and fd7a:115c:a1e0::/48. # See below: # IPv6: https://github.com/tailscale/tailscale/blob/22ebb25e833264f58d7c3f534a8b166894a89536/net/tsaddr/tsaddr.go#LL81C52-L81C71 # IPv4: https://github.com/tailscale/tailscale/blob/22ebb25e833264f58d7c3f534a8b166894a89536/net/tsaddr/tsaddr.go#L33 # Any other range is NOT supported, and it will cause unexpected issues. prefixes: v6: fd7a:115c:a1e0::/48 v4: 100.64.0.0/10 # Strategy used for allocation of IPs to nodes, available options: # - sequential (default): assigns the next free IP from the previous given IP. # - random: assigns the next free IP from a pseudo-random IP generator (crypto/rand). allocation: sequential # DERP is a relay system that Tailscale uses when a direct # connection cannot be established. # https://tailscale.com/blog/how-tailscale-works/#encrypted-tcp-relays-derp # # headscale needs a list of DERP servers that can be presented # to the clients. derp: server: # If enabled, runs the embedded DERP server and merges it into the rest of the DERP config # The Headscale server_url defined above MUST be using https, DERP requires TLS to be in place enabled: false # Region ID to use for the embedded DERP server. # The local DERP prevails if the region ID collides with other region ID coming from # the regular DERP config. region_id: 999 # Region code and name are displayed in the Tailscale UI to identify a DERP region region_code: "headscale" region_name: "Headscale Embedded DERP" # Listens over UDP at the configured address for STUN connections - to help with NAT traversal. # When the embedded DERP server is enabled stun_listen_addr MUST be defined. # # For more details on how this works, check this great article: https://tailscale.com/blog/how-tailscale-works/ stun_listen_addr: "0.0.0.0:3478" # Private key used to encrypt the traffic between headscale DERP # and Tailscale clients. # The private key file will be autogenerated if it's missing. # private_key_path: /var/lib/headscale/derp_server_private.key # This flag can be used, so the DERP map entry for the embedded DERP server is not written automatically, # it enables the creation of your very own DERP map entry using a locally available file with the parameter DERP.paths # If you enable the DERP server and set this to false, it is required to add the DERP server to the DERP map using DERP.paths automatically_add_embedded_derp_region: true # For better connection stability (especially when using an Exit-Node and DNS is not working), # it is possible to optionally add the public IPv4 and IPv6 address to the Derp-Map using: ipv4: 1.2.3.4 ipv6: 2001:db8::1 # List of externally available DERP maps encoded in JSON urls: - https://controlplane.tailscale.com/derpmap/default # Locally available DERP map files encoded in YAML # # This option is mostly interesting for people hosting # their own DERP servers: # https://tailscale.com/kb/1118/custom-derp-servers/ # # paths: # - /etc/headscale/derp-example.yaml paths: [] # If enabled, a worker will be set up to periodically # refresh the given sources and update the derpmap # will be set up. auto_update_enabled: true # How often should we check for DERP updates? update_frequency: 24h # Disables the automatic check for headscale updates on startup disable_check_updates: false # Time before an inactive ephemeral node is deleted? ephemeral_node_inactivity_timeout: 30m database: type: postgres # # SQLite config # sqlite: # path: /var/lib/headscale/db.sqlite # # Enable WAL mode for SQLite. This is recommended for production environments. # # https://www.sqlite.org/wal.html # write_ahead_log: true # Postgres config postgres: # If using a Unix socket to connect to Postgres, set the socket path in the 'host' field and leave 'port' blank. host: localhost port: 5432 name: headscale user: foo pass: bar max_open_conns: 10 max_idle_conns: 10 conn_max_idle_time_secs: 3600 # If other 'sslmode' is required instead of 'require(true)' and 'disabled(false)', set the 'sslmode' you need # in the 'ssl' field. Refers to https://www.postgresql.org/docs/current/libpq-ssl.html Table 34.1. ssl: false ### TLS configuration # ## Let's encrypt / ACME # # headscale supports automatically requesting and setting up # TLS for a domain with Let's Encrypt. # # URL to ACME directory acme_url: https://acme-v02.api.letsencrypt.org/directory # Email to register with ACME provider acme_email: "" # Domain name to request a TLS certificate for: tls_letsencrypt_hostname: "" # Path to store certificates and metadata needed by # letsencrypt # For production: tls_letsencrypt_cache_dir: /var/lib/headscale/cache # Type of ACME challenge to use, currently supported types: # HTTP-01 or TLS-ALPN-01 # See [docs/tls.md](docs/tls.md) for more information tls_letsencrypt_challenge_type: HTTP-01 # When HTTP-01 challenge is chosen, letsencrypt must set up a # verification endpoint, and it will be listening on: # :http = port 80 tls_letsencrypt_listen: ":http" ## Use already defined certificates: tls_cert_path: "" tls_key_path: "" log: # Output formatting for logs: text or json format: text level: debug ## Policy # headscale supports Tailscale's ACL policies. # Please have a look to their KB to better # understand the concepts: https://tailscale.com/kb/1018/acls/ policy: # The mode can be "file" or "database" that defines # where the ACL policies are stored and read from. mode: file # If the mode is set to "file", the # path to a file containing ACL policies. # The file can be in YAML or HuJSON format. path: "" ## DNS # # headscale supports Tailscale's DNS configuration and MagicDNS. # Please have a look to their KB to better understand the concepts: # # - https://tailscale.com/kb/1054/dns/ # - https://tailscale.com/kb/1081/magicdns/ # - https://tailscale.com/blog/2021-09-private-dns-with-magicdns/ # dns_config: # Whether to prefer using Headscale provided DNS or use local. override_local_dns: true # List of DNS servers to expose to clients. nameservers: - 1.1.1.1 # NextDNS (see https://tailscale.com/kb/1218/nextdns/). # "abc123" is example NextDNS ID, replace with yours. # # With metadata sharing: # nameservers: # - https://dns.nextdns.io/abc123 # # Without metadata sharing: # nameservers: # - 2a07:a8c0::ab:c123 # - 2a07:a8c1::ab:c123 # Split DNS (see https://tailscale.com/kb/1054/dns/), # list of search domains and the DNS to query for each one. # # restricted_nameservers: # foo.bar.com: # - 1.1.1.1 # darp.headscale.net: # - 1.1.1.1 # - 8.8.8.8 # Search domains to inject. domains: [] # Extra DNS records # so far only A-records are supported (on the tailscale side) # See https://github.com/juanfont/headscale/blob/main/docs/dns-records.md#Limitations # extra_records: # - name: "grafana.myvpn.example.com" # type: "A" # value: "100.64.0.3" # # # you can also put it in one line # - { name: "prometheus.myvpn.example.com", type: "A", value: "100.64.0.3" } # Whether to use [MagicDNS](https://tailscale.com/kb/1081/magicdns/). # Only works if there is at least a nameserver defined. magic_dns: true # DEPRECATED # Use the username as part of the DNS name for nodes, with this option enabled: # node1.username.example.com # while when this is disabled: # node1.example.com # This is a legacy option as Headscale has have this wrongly implemented # while in upstream Tailscale, the username is not included. use_username_in_magic_dns: false # Defines the base domain to create the hostnames for MagicDNS. # `base_domain` must be a FQDNs, without the trailing dot. # The FQDN of the hosts will be # `hostname.user.base_domain` (e.g., _myhost.myuser.example.com_). base_domain: example.com # Unix socket used for the CLI to connect without authentication # Note: for production you will want to set this to something like: unix_socket: /var/run/headscale/headscale.sock unix_socket_permission: "0770" # # headscale supports experimental OpenID connect support, # it is still being tested and might have some bugs, please # help us test it. # OpenID Connect # oidc: # only_start_if_oidc_is_available: true # issuer: "https://your-oidc.issuer.com/path" # client_id: "your-oidc-client-id" # client_secret: "your-oidc-client-secret" # # Alternatively, set `client_secret_path` to read the secret from the file. # # It resolves environment variables, making integration to systemd's # # `LoadCredential` straightforward: # client_secret_path: "${CREDENTIALS_DIRECTORY}/oidc_client_secret" # # client_secret and client_secret_path are mutually exclusive. # # # The amount of time from a node is authenticated with OpenID until it # # expires and needs to reauthenticate. # # Setting the value to "0" will mean no expiry. # expiry: 180d # # # Use the expiry from the token received from OpenID when the user logged # # in, this will typically lead to frequent need to reauthenticate and should # # only been enabled if you know what you are doing. # # Note: enabling this will cause `oidc.expiry` to be ignored. # use_expiry_from_token: false # # # Customize the scopes used in the OIDC flow, defaults to "openid", "profile" and "email" and add custom query # # parameters to the Authorize Endpoint request. Scopes default to "openid", "profile" and "email". # # scope: ["openid", "profile", "email", "custom"] # extra_params: # domain_hint: example.com # # # List allowed principal domains and/or users. If an authenticated user's domain is not in this list, the # # authentication request will be rejected. # # allowed_domains: # - example.com # # Note: Groups from keycloak have a leading '/' # allowed_groups: # - /headscale # allowed_users: # - alice@example.com # # # If `strip_email_domain` is set to `true`, the domain part of the username email address will be removed. # # This will transform `first-name.last-name@example.com` to the user `first-name.last-name` # # If `strip_email_domain` is set to `false` the domain part will NOT be removed resulting to the following # user: `first-name.last-name.example.com` # # strip_email_domain: true # Logtail configuration # Logtail is Tailscales logging and auditing infrastructure, it allows the control panel # to instruct tailscale nodes to log their activity to a remote server. logtail: # Enable logtail for this headscales clients. # As there is currently no support for overriding the log server in headscale, this is # disabled by default. Enabling this will make your clients send logs to Tailscale Inc. enabled: false # Enabling this option makes devices prefer a random port for WireGuard traffic over the # default static port 41641. This option is intended as a workaround for some buggy # firewall devices. See https://tailscale.com/kb/1181/firewalls/ for more information. randomize_client_port: true ``` **The following is my debug log after starting the headscale server** ```txt root@iZbp1db1jr2xfl77stpy0xZ:~# headscale serve 2024-07-28T13:48:45+08:00 INF Opening database database=postgres path="host=localhost dbname=headscale user=foo" 2024-07-28T13:48:46+08:00 DBG Loading DERPMap from path func=GetDERPMap url=https://controlplane.tailscale.com/derpmap/default 2024-07-28T13:48:46+08:00 INF Setting up a DERPMap update worker frequency=86400000 2024-07-28T13:48:46+08:00 INF listening and serving HTTP on: 0.0.0.0:8080 2024-07-28T13:48:46+08:00 INF listening and serving debug and metrics on: 127.0.0.1:9090 2024-07-28T13:55:28+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T13:55:28+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Node not found in database, creating new expiry=-62135596800 followup= machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old= 2024-07-28T13:55:28+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup= machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old= 2024-07-28T13:55:29+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T13:55:34+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old= 2024-07-28T13:55:34+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T13:55:39+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old= 2024-07-28T13:55:39+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T13:55:44+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old= 2024-07-28T13:55:44+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T13:55:49+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old= 2024-07-28T13:55:50+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T13:55:55+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old= 2024-07-28T13:55:55+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T13:56:00+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old= 2024-07-28T13:56:01+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T13:56:06+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old= 2024-07-28T13:56:06+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T13:56:11+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old= 2024-07-28T13:56:12+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T13:56:17+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old= 2024-07-28T13:56:18+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T13:56:22+08:00 DBG Registering node from API/CLI or auth callback expiresAt=<nil> machine_key=[LHXNY] registrationMethod=cli userName=test 2024-07-28T13:56:22+08:00 DBG Registering node machine_key=[LHXNY] node=localhost node_key=[b8IwI] user=test 2024-07-28T13:56:23+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:2c75cd632751c84596e13328e9315867895df95a7299fe6f486583cd0eb65d25 machine_key=[LHXNY] node=localhost node_key=[b8IwI] node_key_old= 2024-07-28T13:56:25+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T13:56:25+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=localhost 2024-07-28T13:56:25+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=localhost 2024-07-28T13:56:25+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc0005f7380, chan: 0xc00054bbc0 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T13:56:46+08:00 DBG Registering node from API/CLI or auth callback expiresAt=<nil> machine_key=[LHXNY] registrationMethod=cli userName=test 2024-07-28T13:56:52+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc0005f7380, chan: 0xc00054bbc0 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T13:56:53+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T13:56:53+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=localhost 2024-07-28T13:56:53+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=localhost 2024-07-28T13:56:53+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc0005f6480, chan: 0xc00054b0e0 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T13:56:53+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc0005f7680, chan: 0xc000104d80 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T13:56:53+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc0005f6480, chan: 0xc00054b0e0 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T13:56:53+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T13:56:53+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:641 > We have the OldNodeKey in the database. This is a key refresh node=localhost 2024-07-28T13:56:53+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:682 > Node key successfully refreshed node=localhost node_key=[02gJr] old_node_key=[b8IwI] 2024-07-28T13:56:53+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc0005f7680, chan: 0xc000104d80 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T13:56:53+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc0005f7980, chan: 0xc00009bf20 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T14:01:40+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc0005f7980, chan: 0xc00009bf20 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T14:01:43+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:01:43+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=localhost 2024-07-28T14:01:43+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=localhost 2024-07-28T14:01:43+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc0005f6480, chan: 0xc000bd4960 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T14:01:44+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc0005f6480, chan: 0xc000bd4960 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T14:01:44+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000686300, chan: 0xc00057c6c0 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T14:01:44+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:01:44+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:641 > We have the OldNodeKey in the database. This is a key refresh node=localhost 2024-07-28T14:01:44+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:682 > Node key successfully refreshed node=localhost node_key=[J0Y32] old_node_key=[02gJr] 2024-07-28T14:01:44+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc0005f7380, chan: 0xc000776180 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T14:01:44+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000686300, chan: 0xc00057c6c0 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T14:12:34+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc0005f7380, chan: 0xc000776180 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T14:12:40+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:12:40+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=localhost 2024-07-28T14:12:40+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=localhost 2024-07-28T14:12:40+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000686600, chan: 0xc000104720 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T14:12:40+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000686600, chan: 0xc000104720 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T14:12:40+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000002300, chan: 0xc000499260 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T14:12:40+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:12:40+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:641 > We have the OldNodeKey in the database. This is a key refresh node=localhost 2024-07-28T14:12:40+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:682 > Node key successfully refreshed node=localhost node_key=[vVshz] old_node_key=[J0Y32] 2024-07-28T14:12:40+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000002480, chan: 0xc0001304e0 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T14:12:40+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000002300, chan: 0xc000499260 node=localhost node.id=1 omitPeers=false readOnly=false stream=true 2024-07-28T14:25:37+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:25:37+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Node not found in database, creating new expiry=-62135596800 followup= machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old= 2024-07-28T14:25:37+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e expiry=-62135596800 followup= machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old= 2024-07-28T14:25:37+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:25:42+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old= 2024-07-28T14:25:42+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:25:47+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old= 2024-07-28T14:25:47+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:25:52+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old= 2024-07-28T14:25:52+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:25:53+08:00 DBG Registering node from API/CLI or auth callback expiresAt=<nil> machine_key=[4eaGW] registrationMethod=cli userName=test 2024-07-28T14:25:53+08:00 DBG Registering node machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] user=test 2024-07-28T14:25:57+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old= 2024-07-28T14:25:57+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:25:57+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=iZbp128wzc2zwaz21j6x3rZ 2024-07-28T14:25:57+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=iZbp128wzc2zwaz21j6x3rZ 2024-07-28T14:25:57+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000686a80, chan: 0xc000499bc0 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true 2024-07-28T14:26:44+08:00 DBG Registering node from API/CLI or auth callback expiresAt=<nil> machine_key=[4eaGW] registrationMethod=cli userName=test 2024-07-28T14:26:52+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:26:52+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000686a80, chan: 0xc000499bc0 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true 2024-07-28T14:26:52+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=iZbp128wzc2zwaz21j6x3rZ 2024-07-28T14:26:52+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=iZbp128wzc2zwaz21j6x3rZ 2024-07-28T14:26:52+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc0005f7b00, chan: 0xc0004b6840 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true 2024-07-28T14:26:59+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:26:59+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc0005f7b00, chan: 0xc0004b6840 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true 2024-07-28T14:26:59+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=iZbp128wzc2zwaz21j6x3rZ 2024-07-28T14:26:59+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=iZbp128wzc2zwaz21j6x3rZ 2024-07-28T14:26:59+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000686900, chan: 0xc00057d320 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true 2024-07-28T14:27:04+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000686900, chan: 0xc00057d320 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true 2024-07-28T14:27:06+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:27:06+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=iZbp128wzc2zwaz21j6x3rZ 2024-07-28T14:27:06+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=iZbp128wzc2zwaz21j6x3rZ 2024-07-28T14:27:06+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000002300, chan: 0xc000c3d5c0 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true 2024-07-28T14:27:06+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:27:06+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000002300, chan: 0xc000c3d5c0 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true 2024-07-28T14:27:06+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=iZbp128wzc2zwaz21j6x3rZ 2024-07-28T14:27:06+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=iZbp128wzc2zwaz21j6x3rZ 2024-07-28T14:27:06+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000686300, chan: 0xc000776720 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true 2024-07-28T14:27:31+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:27:31+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000686300, chan: 0xc000776720 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true 2024-07-28T14:27:31+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=iZbp128wzc2zwaz21j6x3rZ 2024-07-28T14:27:31+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=iZbp128wzc2zwaz21j6x3rZ 2024-07-28T14:27:31+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000686d80, chan: 0xc00057dbc0 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true 2024-07-28T14:27:37+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000686d80, chan: 0xc00057dbc0 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true 2024-07-28T14:29:03+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:29:03+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=iZbp128wzc2zwaz21j6x3rZ 2024-07-28T14:29:03+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=iZbp128wzc2zwaz21j6x3rZ 2024-07-28T14:29:03+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000686480, chan: 0xc000bd4de0 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true 2024-07-28T14:29:42+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:29:42+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000686480, chan: 0xc000bd4de0 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true 2024-07-28T14:29:42+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=iZbp128wzc2zwaz21j6x3rZ 2024-07-28T14:29:42+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=iZbp128wzc2zwaz21j6x3rZ 2024-07-28T14:29:42+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000686a80, chan: 0xc0004b6ae0 node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true 2024-07-28T14:30:02+08:00 ERR ../home/runner/work/headscale/headscale/hscontrol/poll.go:719 > Could not get machine from db error="record not found" node=iZbp128wzc2zwaz21j6x3rZ node.id=2 omitPeers=false readOnly=false stream=true 2024/07/28 14:30:02 http2: panic serving 120.26.193.183:52916: runtime error: invalid memory address or nil pointer dereference goroutine 3669 [running]: golang.org/x/net/http2.(*serverConn).runHandler.func1() /home/runner/go/pkg/mod/golang.org/x/net@v0.25.0/http2/server.go:2363 +0x145 panic({0x1cf2040?, 0x337e5a0?}) /nix/store/3v17dij8rvg7q99009swxg52995r7s22-go-1.22.5/share/go/src/runtime/panic.go:770 +0x132 github.com/juanfont/headscale/hscontrol.(*mapSession).serveLongPoll.func1() /home/runner/work/headscale/headscale/hscontrol/poll.go:207 +0x90 github.com/juanfont/headscale/hscontrol.(*mapSession).serveLongPoll(0xc000686a80) /home/runner/work/headscale/headscale/hscontrol/poll.go:291 +0x1254 github.com/juanfont/headscale/hscontrol.(*noiseServer).NoisePollNetMapHandler(0xc0006b3a70, {0x2380150, 0xc000121568}, 0xc0000abd40) /home/runner/work/headscale/headscale/hscontrol/noise.go:240 +0x365 net/http.HandlerFunc.ServeHTTP(0x7fc66366f108?, {0x2380150?, 0xc000121568?}, 0x69ec60?) /nix/store/3v17dij8rvg7q99009swxg52995r7s22-go-1.22.5/share/go/src/net/http/server.go:2171 +0x29 github.com/juanfont/headscale/hscontrol.prometheusMiddleware.func1({0x2380150, 0xc000121568}, 0xc0000abd40) /home/runner/work/headscale/headscale/hscontrol/metrics.go:87 +0x143 net/http.HandlerFunc.ServeHTTP(0xc0000abc20?, {0x2380150?, 0xc000121568?}, 0xc0004cc808?) /nix/store/3v17dij8rvg7q99009swxg52995r7s22-go-1.22.5/share/go/src/net/http/server.go:2171 +0x29 github.com/gorilla/mux.(*Router).ServeHTTP(0xc0005263c0, {0x2380150, 0xc000121568}, 0xc00055aa20) /home/runner/go/pkg/mod/github.com/gorilla/mux@v1.8.1/mux.go:212 +0x1e2 golang.org/x/net/http2.(*serverConn).runHandler(0x44975d?, 0xc000603fd0?, 0xb9a885?, 0xc000c31200?) /home/runner/go/pkg/mod/golang.org/x/net@v0.25.0/http2/server.go:2370 +0xbb created by golang.org/x/net/http2.(*serverConn).scheduleHandler in goroutine 3655 /home/runner/go/pkg/mod/golang.org/x/net@v0.25.0/http2/server.go:2305 +0x21d 2024-07-28T14:30:02+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap 2024-07-28T14:30:02+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap 2024-07-28T14:30:02+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap 2024-07-28T14:30:03+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap 2024-07-28T14:30:03+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap 2024-07-28T14:30:03+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap 2024-07-28T14:30:04+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap 2024-07-28T14:30:04+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap 2024-07-28T14:30:05+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap 2024-07-28T14:30:06+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap 2024-07-28T14:30:07+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap 2024-07-28T14:30:08+08:00 ERR Failed to fetch node from the database with node key: nodekey:bc7e2f5c26d9a11eb77260650699cf8e633cd29f9b761498e5d9ae1b52e9af6d handler=NoisePollNetMap 2024-07-28T14:30:09+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:30:09+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Node not found in database, creating new expiry=-62135596800 followup= machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old= 2024-07-28T14:30:09+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e expiry=-62135596800 followup= machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old= 2024-07-28T14:30:09+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:30:14+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old= 2024-07-28T14:30:14+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:30:19+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old= 2024-07-28T14:30:20+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:30:24+08:00 DBG Registering node from API/CLI or auth callback expiresAt=<nil> machine_key=[4eaGW] registrationMethod=cli userName=test 2024-07-28T14:30:24+08:00 DBG Registering node machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] user=test 2024-07-28T14:30:25+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:28 > Successfully sent auth url: http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e expiry=-62135596800 followup=http://116.62.188.235:8080/register/mkey:e1e68658d665ddaca93e161f01115071bc953345d92cdfdd259baf87862b3f6e machine_key=[4eaGW] node=iZbp128wzc2zwaz21j6x3rZ node_key=[vH4vX] node_key_old= 2024-07-28T14:30:25+08:00 DBG New noise client cap_ver=102 handler=/key 2024-07-28T14:30:25+08:00 DBG ../home/runner/work/headscale/headscale/hscontrol/auth.go:596 > Client is registered and we have the current NodeKey. All clear to /map node=iZbp128wzc2zwaz21j6x3rZ 2024-07-28T14:30:25+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/auth.go:627 > Node successfully authorized node=iZbp128wzc2zwaz21j6x3rZ 2024-07-28T14:30:25+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc0005f7380, chan: 0xc000d50ae0 node=iZbp128wzc2zwaz21j6x3rZ node.id=3 omitPeers=false readOnly=false stream=true 2024-07-28T14:33:16+08:00 INF ../home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc0005f7380, chan: 0xc000d50ae0 node=iZbp128wzc2zwaz21j6x3rZ node.id=3 omitPeers=false readOnly=false stream=true ``` Here I removed the reverse proxy and used IP to connect directly. The derp server uses the official derp server. It still has the same problem as before, so I infer that this problem has nothing to do with the reverse proxy.
adam commented 2025-12-29 02:23:25 +01:00
Author
Owner
Copy Link

@wangzhijie0602 commented on GitHub (Jul 28, 2024):

Here I added two new clients. In fact, the two new clients were not fully connected. I exported my Postgres database file.

Here I first list the performance of the two clients
Client 1:
After connecting, the following is displayed
5a4d73c7bb3040a803556e1d94f7d5b

Client 2:
It stays in this interface after connecting.
250844e7bd4ae5425f3449af3d611a4

The following content is the content of the server after I deleted the nodes on the server after the first connection failed on the Linux client.
d5e81ef9cd4b4b8e3355dbc831fbeee

@wangzhijie0602 commented on GitHub (Jul 28, 2024): Here I added two new clients. In fact, the two new clients were not fully connected. I exported my Postgres database file. Here I first list the performance of the two clients Client 1: After connecting, the following is displayed ![5a4d73c7bb3040a803556e1d94f7d5b](https://github.com/user-attachments/assets/f9f3d9db-8a4d-460a-b9c5-82507fe1b3e8) Client 2: It stays in this interface after connecting. ![250844e7bd4ae5425f3449af3d611a4](https://github.com/user-attachments/assets/3f34f708-92ff-4da0-9d2d-c338cac6590b) The following content is the content of the server after I deleted the nodes on the server after the first connection failed on the Linux client. ![d5e81ef9cd4b4b8e3355dbc831fbeee](https://github.com/user-attachments/assets/7a660080-d35d-4c87-9d2a-eedb2edfda1d)
adam commented 2025-12-29 02:23:25 +01:00
Author
Owner
Copy Link

@wangzhijie0602 commented on GitHub (Jul 28, 2024):

Next is the sql dump file I exported containing the information of the two clients.
headscale_backup.zip

@wangzhijie0602 commented on GitHub (Jul 28, 2024): Next is the sql dump file I exported containing the information of the two clients. [headscale_backup.zip](https://github.com/user-attachments/files/16402488/headscale_backup.zip)
adam commented 2025-12-29 02:23:25 +01:00
Author
Owner
Copy Link

@wangzhijie0602 commented on GitHub (Jul 28, 2024):

The server information that appears in the above content is the test server I created. I released it after the test was completed. Now these server IPs no longer belong to me.

@wangzhijie0602 commented on GitHub (Jul 28, 2024): The server information that appears in the above content is the test server I created. I released it after the test was completed. Now these server IPs no longer belong to me.
adam commented 2025-12-29 02:23:25 +01:00
Author
Owner
Copy Link

@wangzhijie0602 commented on GitHub (Jul 28, 2024):

Here are the steps I took when creating the database

sudo -u postgres psql
CREATE USER foo WITH PASSWORD 'bar';
CREATE DATABASE headscale;
GRANT ALL PRIVILEGES ON DATABASE headscale TO foo;
\q

@wangzhijie0602 commented on GitHub (Jul 28, 2024): Here are the steps I took when creating the database sudo -u postgres psql CREATE USER foo WITH PASSWORD 'bar'; CREATE DATABASE headscale; GRANT ALL PRIVILEGES ON DATABASE headscale TO foo; \q
adam commented 2025-12-29 02:23:26 +01:00
Author
Owner
Copy Link

@mpoindexter commented on GitHub (Aug 31, 2024):

I think this is a duplicate of #764

@mpoindexter commented on GitHub (Aug 31, 2024): I think this is a duplicate of #764
adam commented 2025-12-29 02:23:26 +01:00
Author
Owner
Copy Link

@kradalby commented on GitHub (Sep 3, 2024):

Yes, likely, I will close as duplicate, we can open if not.

@kradalby commented on GitHub (Sep 3, 2024): Yes, likely, I will close as duplicate, we can open if not.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
CLI
DERP
DNS
Nix
OIDC
SSH
bug
database
documentation
duplicate
enhancement
faq
good first issue
grants
help wanted
might-come
needs design doc
needs investigation
no-stale-bot
out of scope
performance
policy 📝
pull-request
Mirrored from GitHub Pull Request
 question
regression
routes
stale
tags
tailscale-feature-gap
well described ❤️
wontfix
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/headscale#750
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?