[Bug] 0.26.0 Custom DERPs not seeing some tailnet nodes after update #1042

Closed
opened 2025-12-29 02:27:53 +01:00 by adam · 2 comments
Owner

Originally created by @alfredomagallon on GitHub (May 31, 2025).

Is this a support request?

  • This is not a support request

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

After updating to 0.26.0, I noticed a decrease in performance.
Investigating custom DERPs, I noticed that they are not seeing much of the tailnet nodes anymore.
The policy has been migrated to include @ after user names, and seems everything else except the DERPs is working fine.

We were using:
https://github.com/tailscale/tailscale/issues/13097#issuecomment-2354349031

Maybe it's not valid anymore?

Expected Behavior

DERPs should have continued seeing all nodes in the tailnet

Steps To Reproduce

Headscale 0.25.1 tailnet with custom DERP servers
Update to headscale 0.26.0 and see if all nodes stay in the tailnet

Environment

- OS: Ubuntu
- Headscale version: 0.26.0
- Tailscale version: 1.84.0

Runtime environment

  • Headscale is behind a (reverse) proxy
  • Headscale runs in a container

Debug information

DERPER LOGS:
journalctl -f -u derper
(Full of)
May 30 23:48:37 derp-europe-west10-a bash[3464]: 2025/05/30 23:48:37 derp: 63.32.235.50:2850: client nodekey:cf5ade5e18760133924736fc641acfc9824400c5938ae0248a2b91acccd9625a rejected: peer nodekey:cf5ade5e18760133924736fc641acfc9824400c5938ae0248a2b91acccd9625a not authorized (not found in local tailscaled)
May 30 23:48:38 derp-europe-west10-a bash[3464]: 2025/05/30 23:48:38 derp: 34.93.48.85:1072: client nodekey:b2fc8042ec07674e4f0ef815bff65091221de793ad0aec27bfb05ab62c1bd368 rejected: peer nodekey:b2fc8042ec07674e4f0ef815bff65091221de793ad0aec27bfb05ab62c1bd368 not authorized (not found in local tailscaled)
May 30 23:48:40 derp-europe-west10-a bash[3464]: 2025/05/30 23:48:40 derp: 20.244.76.6:5122: client nodekey:4f4baa9c6a1064ad4ba9823aceb98d86685218cafb232e0e1521a22575263a70 rejected: peer nodekey:4f4baa9c6a1064ad4ba9823aceb98d86685218cafb232e0e1521a22575263a70 not authorized (not found in local tailscaled)
May 30 23:48:41 derp-europe-west10-a bash[3464]: 2025/05/30 23:48:41 derp: 172.189.93.123:3141: client nodekey:f8628d29b23d92c70d4e19e3f5b1f93fc046435cb49239e8e1d4a04c1e91c95a rejected: peer nodekey:f8628d29b23d92c70d4e19e3f5b1f93fc046435cb49239e8e1d4a04c1e91c95a not authorized (not found in local tailscaled)

Originally created by @alfredomagallon on GitHub (May 31, 2025). ### Is this a support request? - [x] This is not a support request ### Is there an existing issue for this? - [x] I have searched the existing issues ### Current Behavior After updating to 0.26.0, I noticed a decrease in performance. Investigating custom DERPs, I noticed that they are not seeing much of the tailnet nodes anymore. The policy has been migrated to include @ after user names, and seems everything else except the DERPs is working fine. We were using: https://github.com/tailscale/tailscale/issues/13097#issuecomment-2354349031 Maybe it's not valid anymore? ### Expected Behavior DERPs should have continued seeing all nodes in the tailnet ### Steps To Reproduce Headscale 0.25.1 tailnet with custom DERP servers Update to headscale 0.26.0 and see if all nodes stay in the tailnet ### Environment ```markdown - OS: Ubuntu - Headscale version: 0.26.0 - Tailscale version: 1.84.0 ``` ### Runtime environment - [x] Headscale is behind a (reverse) proxy - [x] Headscale runs in a container ### Debug information DERPER LOGS: journalctl -f -u derper (Full of) May 30 23:48:37 derp-europe-west10-a bash[3464]: 2025/05/30 23:48:37 derp: 63.32.235.50:2850: client nodekey:cf5ade5e18760133924736fc641acfc9824400c5938ae0248a2b91acccd9625a rejected: peer nodekey:cf5ade5e18760133924736fc641acfc9824400c5938ae0248a2b91acccd9625a not authorized (not found in local tailscaled) May 30 23:48:38 derp-europe-west10-a bash[3464]: 2025/05/30 23:48:38 derp: 34.93.48.85:1072: client nodekey:b2fc8042ec07674e4f0ef815bff65091221de793ad0aec27bfb05ab62c1bd368 rejected: peer nodekey:b2fc8042ec07674e4f0ef815bff65091221de793ad0aec27bfb05ab62c1bd368 not authorized (not found in local tailscaled) May 30 23:48:40 derp-europe-west10-a bash[3464]: 2025/05/30 23:48:40 derp: 20.244.76.6:5122: client nodekey:4f4baa9c6a1064ad4ba9823aceb98d86685218cafb232e0e1521a22575263a70 rejected: peer nodekey:4f4baa9c6a1064ad4ba9823aceb98d86685218cafb232e0e1521a22575263a70 not authorized (not found in local tailscaled) May 30 23:48:41 derp-europe-west10-a bash[3464]: 2025/05/30 23:48:41 derp: 172.189.93.123:3141: client nodekey:f8628d29b23d92c70d4e19e3f5b1f93fc046435cb49239e8e1d4a04c1e91c95a rejected: peer nodekey:f8628d29b23d92c70d4e19e3f5b1f93fc046435cb49239e8e1d4a04c1e91c95a not authorized (not found in local tailscaled)
adam added the bug label 2025-12-29 02:27:53 +01:00
adam closed this issue 2025-12-29 02:27:53 +01:00
Author
Owner

@alfredomagallon commented on GitHub (May 31, 2025):

Note, just in case is related:

All the nodes that don't appear in the DERP's tailscale status are the ones which users appear without a "@" suffix in my local machine tailscale status (including my other custom DERP)
I appended a @ to all usernames in the policy, following the docs. Not sure why in my tailscale status some of them appear with the "@", some others not.

@alfredomagallon commented on GitHub (May 31, 2025): Note, just in case is related: All the nodes that don't appear in the DERP's `tailscale status` are the ones which users appear without a "@" suffix in my local machine `tailscale status` (including my other custom DERP) I appended a @ to all usernames in the policy, following the docs. Not sure why in my `tailscale status` some of them appear with the "@", some others not.
Author
Owner

@alfredomagallon commented on GitHub (May 31, 2025):

I continued investigating and seems not an issue with headscale update but with tailscale latest version 1.84:
https://github.com/tailscale/tailscale/issues/16099

@alfredomagallon commented on GitHub (May 31, 2025): I continued investigating and seems not an issue with headscale update but with tailscale latest version 1.84: https://github.com/tailscale/tailscale/issues/16099
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/headscale#1042