starred/headscale
1
0
Fork 0
mirror of https://github.com/juanfont/headscale.git synced 2026-01-11 20:00:28 +01:00
Code Issues 98 Packages Projects Releases 10 Wiki Activity

[Bug] could not register node: node not found in registration cache #986

New Issue
Closed
opened 2025-12-29 02:27:04 +01:00 by adam · 6 comments
adam commented 2025-12-29 02:27:04 +01:00
Owner
Copy Link

Originally created by @soymgomez on GitHub (Mar 26, 2025).

Is this a support request?

  • This is not a support request

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

Some users with a device already registered and who have previously logged in cannot log in because the server returns a 500 error.

Checking the logs the error is as follows:

ERR http internal server error error="could not register node: node not found in registration cache" code=500

Expected Behavior

The user should be able to log in without problems.

Steps To Reproduce

  1. A user tries to log in
  2. The user receives a 500 error when returning from the OIDC.
  3. The Headscale log shows the specified error.

Environment

- OS: iOS, MacOS, Windows
- Headscale version: 0.25.1
- Tailscale version: N/A

Runtime environment

  • Headscale is behind a (reverse) proxy
  • Headscale runs in a container

Debug information

oidc:
  only_start_if_oidc_is_available: true
  issuer: "https://authentik.domain.com"
  client_id: "<my client id>"
  client_secret: "<my client secret>"
  use_expiry_from_token: false
  scope: ["openid", "profile", "email", "offline_access"]
Originally created by @soymgomez on GitHub (Mar 26, 2025). ### Is this a support request? - [x] This is not a support request ### Is there an existing issue for this? - [x] I have searched the existing issues ### Current Behavior Some users with a device already registered and who have previously logged in cannot log in because the server returns a 500 error. Checking the logs the error is as follows: `ERR http internal server error error="could not register node: node not found in registration cache" code=500` ### Expected Behavior The user should be able to log in without problems. ### Steps To Reproduce 1. A user tries to log in 2. The user receives a 500 error when returning from the OIDC. 3. The Headscale log shows the specified error. ### Environment ```markdown - OS: iOS, MacOS, Windows - Headscale version: 0.25.1 - Tailscale version: N/A ``` ### Runtime environment - [x] Headscale is behind a (reverse) proxy - [ ] Headscale runs in a container ### Debug information ``` oidc: only_start_if_oidc_is_available: true issuer: "https://authentik.domain.com" client_id: "<my client id>" client_secret: "<my client secret>" use_expiry_from_token: false scope: ["openid", "profile", "email", "offline_access"] ```
adam added the bug label 2025-12-29 02:27:04 +01:00
adam closed this issue 2025-12-29 02:27:04 +01:00
adam commented 2025-12-29 02:27:05 +01:00
Author
Owner
Copy Link

@soymgomez commented on GitHub (Mar 26, 2025):

I have seen this MR which may be related but I am not 100% sure: https://github.com/juanfont/headscale/pull/2493

@soymgomez commented on GitHub (Mar 26, 2025): I have seen this MR which may be related but I am not 100% sure: https://github.com/juanfont/headscale/pull/2493
adam commented 2025-12-29 02:27:05 +01:00
Author
Owner
Copy Link

@kradalby commented on GitHub (Mar 26, 2025):

It is very hard for us to reproduce this, you have given almost no information and given that we have not seen any other of the sort, I think this might be a problem with your setup.

Have you asked in discord for help first?

The issue template asks for a lot of information for a reason.

@kradalby commented on GitHub (Mar 26, 2025): It is very hard for us to reproduce this, you have given almost no information and given that we have not seen any other of the sort, I think this might be a problem with your setup. Have you asked in discord for help first? The issue template asks for a lot of information for a reason.
adam commented 2025-12-29 02:27:05 +01:00
Author
Owner
Copy Link

@dmeremyanin commented on GitHub (Apr 30, 2025):

Sorry for pinging a closed issue, but we're experiencing the exact same problem with Headscale and OIDC (Authentik). Here's a sample from the log:

ERR http internal server error error="could not register node: node not found in registration cache" code=500
ERR http internal server error error="could not register node: node not found in registration cache" code=500
ERR http internal server error error="could not register node: node not found in registration cache" code=500
ERR http internal server error error="could not register node: node not found in registration cache" code=500
ERR http internal server error error="could not register node: node not found in registration cache" code=500
ERR http internal server error error="could not register node: node not found in registration cache" code=500
ERR http internal server error error="could not register node: node not found in registration cache" code=500

In the browser, users receive an "internal server" error:
Image

Interestingly, restarting the Tailscale client (just closing and reopening the app) immediately resolves the issue. We've observed this behavior on both macOS and Windows with Tailscale versions 1.80.0 and 1.80.5. Our Headscale version is 0.25.1, running without Docker.

It seems to affect only a small subset of users - about 4 out of 20 in our case. Once it's fixed, it doesn't seem to recur (yet).

Sorry for not providing more detailed information at this point. Any suggestions on how to investigate this would be greatly appreciated.

@dmeremyanin commented on GitHub (Apr 30, 2025): Sorry for pinging a closed issue, but we're experiencing the exact same problem with Headscale and OIDC (Authentik). Here's a sample from the log: ``` ERR http internal server error error="could not register node: node not found in registration cache" code=500 ERR http internal server error error="could not register node: node not found in registration cache" code=500 ERR http internal server error error="could not register node: node not found in registration cache" code=500 ERR http internal server error error="could not register node: node not found in registration cache" code=500 ERR http internal server error error="could not register node: node not found in registration cache" code=500 ERR http internal server error error="could not register node: node not found in registration cache" code=500 ERR http internal server error error="could not register node: node not found in registration cache" code=500 ``` In the browser, users receive an "internal server" error: ![Image](https://github.com/user-attachments/assets/a5dcbef2-7362-41f8-828d-13e9ca7f14ae) Interestingly, restarting the Tailscale client (just closing and reopening the app) immediately resolves the issue. We've observed this behavior on both macOS and Windows with Tailscale versions 1.80.0 and 1.80.5. Our Headscale version is 0.25.1, running without Docker. It seems to affect only a small subset of users - about 4 out of 20 in our case. Once it's fixed, it doesn't seem to recur (yet). Sorry for not providing more detailed information at this point. Any suggestions on how to investigate this would be greatly appreciated.
adam commented 2025-12-29 02:27:05 +01:00
Author
Owner
Copy Link

@asalimonov commented on GitHub (Jul 24, 2025):

Just a comment for history.
Reproduced this bug with config oidc.expiry: 0d. Changed to 360d and the bug is gone.

@asalimonov commented on GitHub (Jul 24, 2025): Just a comment for history. Reproduced this bug with config `oidc.expiry: 0d`. Changed to `360d` and the bug is gone.
adam commented 2025-12-29 02:27:05 +01:00
Author
Owner
Copy Link

@alberand commented on GitHub (Aug 1, 2025):

The @asalimonov workaround doesn't seem to work for me. By default, I didn't have any oidc.expiry, setting it to 360d still leads to:

Aug 01 16:51:46 host headscale-start[27439]: 2025-08-01T16:51:46Z DBG Redirecting to https://id.example.com/authorize?access_type=offline&client_id=24c.... for authentication
Aug 01 16:51:48 host headscale-start[27439]: 2025-08-01T16:51:48Z ERR http internal server error error="could not register node: node not found in registration cache" code=500

For me, it's every time I try to log in.

My setup is:

  • headscale 0.25.1 with caddy as reverse proxy
  • Pocket-ID 1.6.2
  • Client is Tailscale for iOS 1.84.1 on iOS 18.5

Login and headscale works for admin user, although I have an admin user with the same name created with headscale cli (but creating a user doesn't help to solve this issue).

headscale config 
database:
  postgres:
    host: null
    name: null
    password_file: null
    port: null
    user: null
  sqlite:
    path: /var/lib/headscale/db.sqlite
    write_ahead_log: true
  type: sqlite
derp:
  auto_update_enabled: true
  paths: []
  server:
    private_key_path: /var/lib/headscale/derp_server_private.key
  update_frequency: 24h
  urls:
  - https://controlplane.tailscale.com/derpmap/default
disable_check_updates: true
dns:
  base_domain: headscale.example.com
  magic_dns: true
  nameservers:
    global:
    - 194.242.2.9
  override_local_dns: true
  search_domains: []
ephemeral_node_inactivity_timeout: 30m
listen_addr: 0.0.0.0:8080
log:
  format: text
  level: debug
logtail:
  enabled: false
metrics_listen_addr: 0.0.0.0:9090
noise:
  private_key_path: /var/lib/headscale/noise_private.key
oidc:
  allowed_domains: []
  allowed_users:
  - alberand
  client_id: 24c4....
  client_secret_path: /run/agenix/headscale-pocket-id
  extra_params: {}
  issuer: https://id.example.com
  pkce:
    enabled: true
    method: S256
  scope:
  - openid
  - profile
  - email
  - groups
  use_expiry_from_token: false
policy:
  mode: file
  path: null
prefixes:
  allocation: sequential
  v4: 100.64.0.0/24
  v6: fd7a:115c:a1e0::/48
server_url: https://headscale.example.com
tls_cert_path: null
tls_key_path: null
tls_letsencrypt_cache_dir: /var/lib/headscale/.cache
tls_letsencrypt_challenge_type: HTTP-01
tls_letsencrypt_hostname: ''
tls_letsencrypt_listen: :http
unix_socket: /run/headscale/headscale.sock
@alberand commented on GitHub (Aug 1, 2025): The @asalimonov workaround doesn't seem to work for me. By default, I didn't have any `oidc.expiry`, setting it to `360d` still leads to: ``` Aug 01 16:51:46 host headscale-start[27439]: 2025-08-01T16:51:46Z DBG Redirecting to https://id.example.com/authorize?access_type=offline&client_id=24c.... for authentication Aug 01 16:51:48 host headscale-start[27439]: 2025-08-01T16:51:48Z ERR http internal server error error="could not register node: node not found in registration cache" code=500 ``` For me, it's every time I try to log in. My setup is: - `headscale 0.25.1` with `caddy` as reverse proxy - `Pocket-ID 1.6.2` - Client is `Tailscale for iOS 1.84.1` on `iOS 18.5` Login and headscale works for admin user, although I have an admin user with the same name created with headscale cli (but creating a user doesn't help to solve this issue). <details> <summary>headscale config</summary> ``` database: postgres: host: null name: null password_file: null port: null user: null sqlite: path: /var/lib/headscale/db.sqlite write_ahead_log: true type: sqlite derp: auto_update_enabled: true paths: [] server: private_key_path: /var/lib/headscale/derp_server_private.key update_frequency: 24h urls: - https://controlplane.tailscale.com/derpmap/default disable_check_updates: true dns: base_domain: headscale.example.com magic_dns: true nameservers: global: - 194.242.2.9 override_local_dns: true search_domains: [] ephemeral_node_inactivity_timeout: 30m listen_addr: 0.0.0.0:8080 log: format: text level: debug logtail: enabled: false metrics_listen_addr: 0.0.0.0:9090 noise: private_key_path: /var/lib/headscale/noise_private.key oidc: allowed_domains: [] allowed_users: - alberand client_id: 24c4.... client_secret_path: /run/agenix/headscale-pocket-id extra_params: {} issuer: https://id.example.com pkce: enabled: true method: S256 scope: - openid - profile - email - groups use_expiry_from_token: false policy: mode: file path: null prefixes: allocation: sequential v4: 100.64.0.0/24 v6: fd7a:115c:a1e0::/48 server_url: https://headscale.example.com tls_cert_path: null tls_key_path: null tls_letsencrypt_cache_dir: /var/lib/headscale/.cache tls_letsencrypt_challenge_type: HTTP-01 tls_letsencrypt_hostname: '' tls_letsencrypt_listen: :http unix_socket: /run/headscale/headscale.sock ``` </details>
adam commented 2025-12-29 02:27:06 +01:00
Author
Owner
Copy Link

@martijnboers commented on GitHub (Aug 27, 2025):

I'm moving from keycloak to pocket and running into this issue with the server that's running headscale. Works for almost exactly the same NixOS servers. Will update if I find more information.

Edit: simple reboot did the trick for me

@martijnboers commented on GitHub (Aug 27, 2025): I'm moving from keycloak to pocket and running into this issue with the server that's running headscale. Works for almost exactly the same NixOS servers. Will update if I find more information. Edit: simple reboot did the trick for me
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
CLI
DERP
DNS
Nix
OIDC
SSH
bug
database
documentation
duplicate
enhancement
faq
good first issue
grants
help wanted
might-come
needs design doc
needs investigation
no-stale-bot
out of scope
performance
policy 📝
pull-request
Mirrored from GitHub Pull Request
 question
regression
routes
stale
tags
tailscale-feature-gap
well described ❤️
wontfix
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/headscale#986
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?