v0.17.0-alpha2 doesn't serve on 443 #319

Closed
opened 2025-12-29 01:26:52 +01:00 by adam · 8 comments
Owner

Originally created by @papamoose on GitHub (Sep 7, 2022).

headscale serve does not serve anything on port 443 in version v0.17.0-alpha2

ss -tpln output only shows port :80 being used.

LISTEN 0      4096               *:80              *:*    users:(("headscale",pid=2234,fd=10)) 

If you use v0.17.0-alpha1 everything seems to be working as expected. At least I was able to register a new node with out issue.

ss -tpln output when running v0.17.0-apha1

LISTEN 0      4096       127.0.0.1:9090       0.0.0.0:*    users:(("headscale",pid=1830,fd=11))     
LISTEN 0      4096               *:80               *:*    users:(("headscale",pid=1830,fd=14))     
LISTEN 0      4096               *:443              *:*    users:(("headscale",pid=1830,fd=10))     
LISTEN 0      4096               *:50443            *:*    users:(("headscale",pid=1830,fd=9)) 

Both server and client are Ubuntu 22.04.

I'm a new user of headscale so it's possible I don't have something configured properly. This seems unlikely though as v0.17.0-alpha1 works and alpha2 does not.

Originally created by @papamoose on GitHub (Sep 7, 2022). `headscale serve` does not serve anything on port 443 in version [v0.17.0-alpha2](https://github.com/juanfont/headscale/releases/tag/v0.17.0-alpha2) `ss -tpln` output only shows port :80 being used. ``` LISTEN 0 4096 *:80 *:* users:(("headscale",pid=2234,fd=10)) ``` If you use v0.17.0-alpha1 everything seems to be working as expected. At least I was able to register a new node with out issue. `ss -tpln` output when running `v0.17.0-apha1` ``` LISTEN 0 4096 127.0.0.1:9090 0.0.0.0:* users:(("headscale",pid=1830,fd=11)) LISTEN 0 4096 *:80 *:* users:(("headscale",pid=1830,fd=14)) LISTEN 0 4096 *:443 *:* users:(("headscale",pid=1830,fd=10)) LISTEN 0 4096 *:50443 *:* users:(("headscale",pid=1830,fd=9)) ``` Both server and client are Ubuntu 22.04. I'm a new user of headscale so it's possible I don't have something configured properly. This seems unlikely though as v0.17.0-alpha1 works and alpha2 does not.
adam added the bug label 2025-12-29 01:26:52 +01:00
adam closed this issue 2025-12-29 01:26:52 +01:00
Author
Owner

@juanfont commented on GitHub (Sep 7, 2022):

Can you paste here your config?

@juanfont commented on GitHub (Sep 7, 2022): Can you paste here your config?
Author
Owner

@papamoose commented on GitHub (Sep 8, 2022):

  • I don't change the config between switching versions.
  • I download the binary release for amd64 from the releases page.
  • I grab the example config from the git repo and save as config.yaml and modify the following keys: server_url, tls_letsencrypt_hostname, acme_email
---
server_url: https://vpn2.example.com:443
listen_addr: 0.0.0.0:8080
metrics_listen_addr: 127.0.0.1:9090
grpc_listen_addr: 0.0.0.0:50443
grpc_allow_insecure: false
private_key_path: /var/lib/headscale/private.key
noise:
  private_key_path: /var/lib/headscale/noise_private.key
ip_prefixes:
  - fd7a:115c:a1e0::/48
  - 100.64.0.0/10
derp:
  server:
    enabled: false
    region_id: 999
    region_code: "headscale"
    region_name: "Headscale Embedded DERP"
    stun_listen_addr: "0.0.0.0:3478"
  urls:
    - https://controlplane.tailscale.com/derpmap/default
  paths: []
  auto_update_enabled: true
  update_frequency: 24h
disable_check_updates: false
ephemeral_node_inactivity_timeout: 30m
node_update_check_interval: 10s
db_type: sqlite3
db_path: /var/lib/headscale/db.sqlite
acme_url: https://acme-v02.api.letsencrypt.org/directory
acme_email: "me@example.com"
tls_letsencrypt_hostname: "vpn2.example.com"
tls_client_auth_mode: relaxed
tls_letsencrypt_cache_dir: /var/lib/headscale/cache
tls_letsencrypt_challenge_type: HTTP-01
tls_letsencrypt_listen: ":http"
tls_cert_path: ""
tls_key_path: ""
log_level: info
acl_policy_path: ""
dns_config:
  nameservers:
    - 1.1.1.1
  domains: []
  magic_dns: true
  base_domain: example.com
unix_socket: /var/run/headscale.sock
unix_socket_permission: "0770"
logtail:
  enabled: false
randomize_client_port: false
root@localhost:/etc/headscale# headscale serve
An updated version of Headscale has been found (0.17.0-alpha2 vs. your current v0.17.0-alpha1). Check it out https://github.com/juanfont/headscale/releases
2022-09-07T23:54:24Z INF No private key file at path, creating... path=/var/lib/headscale/private.key
2022-09-07T23:54:24Z INF No private key file at path, creating... path=/var/lib/headscale/noise_private.key
2022-09-07T23:54:24Z INF Enabling remote gRPC at 0.0.0.0:50443
2022-09-07T23:54:24Z INF listening and serving gRPC on: 0.0.0.0:50443
2022-09-07T23:54:24Z INF listening and serving HTTP on: 0.0.0.0:8080
2022-09-07T23:54:24Z INF listening and serving metrics on: 127.0.0.1:9090
2022-09-07T23:54:24Z INF Setting up a DERPMap update worker frequency=86400000
root@localhost:/etc/headscale# headscale version
v0.17.0-alpha2
root@localhost:/etc/headscale# rm -r /var/lib/headscale/* /var/run/headscale.sock
root@localhost:/etc/headscale# headscale serve
2022-09-07T23:58:06Z INF No private key file at path, creating... path=/var/lib/headscale/private.key
2022-09-07T23:58:06Z INF No private key file at path, creating... path=/var/lib/headscale/noise_private.key
2022-09-07T23:58:07Z INF Setting up a DERPMap update worker frequency=86400000
@papamoose commented on GitHub (Sep 8, 2022): * I don't change the config between switching versions. * I download the binary release for amd64 from the releases page. * I grab the example config from the git repo and save as config.yaml and modify the following keys: server_url, tls_letsencrypt_hostname, acme_email ``` --- server_url: https://vpn2.example.com:443 listen_addr: 0.0.0.0:8080 metrics_listen_addr: 127.0.0.1:9090 grpc_listen_addr: 0.0.0.0:50443 grpc_allow_insecure: false private_key_path: /var/lib/headscale/private.key noise: private_key_path: /var/lib/headscale/noise_private.key ip_prefixes: - fd7a:115c:a1e0::/48 - 100.64.0.0/10 derp: server: enabled: false region_id: 999 region_code: "headscale" region_name: "Headscale Embedded DERP" stun_listen_addr: "0.0.0.0:3478" urls: - https://controlplane.tailscale.com/derpmap/default paths: [] auto_update_enabled: true update_frequency: 24h disable_check_updates: false ephemeral_node_inactivity_timeout: 30m node_update_check_interval: 10s db_type: sqlite3 db_path: /var/lib/headscale/db.sqlite acme_url: https://acme-v02.api.letsencrypt.org/directory acme_email: "me@example.com" tls_letsencrypt_hostname: "vpn2.example.com" tls_client_auth_mode: relaxed tls_letsencrypt_cache_dir: /var/lib/headscale/cache tls_letsencrypt_challenge_type: HTTP-01 tls_letsencrypt_listen: ":http" tls_cert_path: "" tls_key_path: "" log_level: info acl_policy_path: "" dns_config: nameservers: - 1.1.1.1 domains: [] magic_dns: true base_domain: example.com unix_socket: /var/run/headscale.sock unix_socket_permission: "0770" logtail: enabled: false randomize_client_port: false ``` ``` root@localhost:/etc/headscale# headscale serve An updated version of Headscale has been found (0.17.0-alpha2 vs. your current v0.17.0-alpha1). Check it out https://github.com/juanfont/headscale/releases 2022-09-07T23:54:24Z INF No private key file at path, creating... path=/var/lib/headscale/private.key 2022-09-07T23:54:24Z INF No private key file at path, creating... path=/var/lib/headscale/noise_private.key 2022-09-07T23:54:24Z INF Enabling remote gRPC at 0.0.0.0:50443 2022-09-07T23:54:24Z INF listening and serving gRPC on: 0.0.0.0:50443 2022-09-07T23:54:24Z INF listening and serving HTTP on: 0.0.0.0:8080 2022-09-07T23:54:24Z INF listening and serving metrics on: 127.0.0.1:9090 2022-09-07T23:54:24Z INF Setting up a DERPMap update worker frequency=86400000 ``` ``` root@localhost:/etc/headscale# headscale version v0.17.0-alpha2 root@localhost:/etc/headscale# rm -r /var/lib/headscale/* /var/run/headscale.sock root@localhost:/etc/headscale# headscale serve 2022-09-07T23:58:06Z INF No private key file at path, creating... path=/var/lib/headscale/private.key 2022-09-07T23:58:06Z INF No private key file at path, creating... path=/var/lib/headscale/noise_private.key 2022-09-07T23:58:07Z INF Setting up a DERPMap update worker frequency=86400000 ```
Author
Owner

@statkashaman commented on GitHub (Sep 8, 2022):

Hello, you can change tls_letsencrypt_challenge_type from HTTP-01 to TLS-ALPN-01, or use existing certificate with options tls_cert_path: "" and tls_key_path: "". It's a bug (

@statkashaman commented on GitHub (Sep 8, 2022): Hello, you can change tls_letsencrypt_challenge_type from HTTP-01 to TLS-ALPN-01, or use existing certificate with options tls_cert_path: "" and tls_key_path: "". It's a bug (
Author
Owner

@papamoose commented on GitHub (Sep 8, 2022):

I was able to successfully start headscale using v0.17.0-alpha2 following @statkashaman instructions.

  1. Set tls_letsencrypt_challenge_type: TLS-ALPN-01.
  2. headscale version then complained about listen_addr needing to be set to :443
  3. listen_addr: 0.0.0.0:8080 -> listen_addr: 0.0.0.0:443
root@localhost:/etc/headscale# headscale version
v0.17.0-alpha2
root@localhost:~# headscale serve
2022-09-08T11:28:06Z INF Enabling remote gRPC at 0.0.0.0:50443
2022-09-08T11:28:06Z INF listening and serving gRPC on: 0.0.0.0:50443
2022-09-08T11:28:06Z INF listening and serving HTTP on: 0.0.0.0:443
2022-09-08T11:28:06Z INF listening and serving metrics on: 127.0.0.1:9090
2022-09-08T11:28:06Z INF Setting up a DERPMap update worker frequency=86400000

It looks like there is no service running port 80. I assume that has to do with using TLS-ALPN-01.

root@localhost:~# ss -tpln|grep headscale
LISTEN 0      4096       127.0.0.1:9090       0.0.0.0:*    users:(("headscale",pid=4623,fd=14))     
LISTEN 0      4096               *:443              *:*    users:(("headscale",pid=4623,fd=13))     
LISTEN 0      4096               *:50443            *:*    users:(("headscale",pid=4623,fd=12)) 

I was then able to register two machines using the tailscale client who can ping each other. :)

@papamoose commented on GitHub (Sep 8, 2022): I was able to successfully start headscale using `v0.17.0-alpha2` following @statkashaman instructions. 1. Set `tls_letsencrypt_challenge_type: TLS-ALPN-01`. 1. `headscale version` then complained about `listen_addr` needing to be set to `:443` 1. `listen_addr: 0.0.0.0:8080` -> `listen_addr: 0.0.0.0:443` ``` root@localhost:/etc/headscale# headscale version v0.17.0-alpha2 root@localhost:~# headscale serve 2022-09-08T11:28:06Z INF Enabling remote gRPC at 0.0.0.0:50443 2022-09-08T11:28:06Z INF listening and serving gRPC on: 0.0.0.0:50443 2022-09-08T11:28:06Z INF listening and serving HTTP on: 0.0.0.0:443 2022-09-08T11:28:06Z INF listening and serving metrics on: 127.0.0.1:9090 2022-09-08T11:28:06Z INF Setting up a DERPMap update worker frequency=86400000 ``` It looks like there is no service running port 80. I assume that has to do with using TLS-ALPN-01. ``` root@localhost:~# ss -tpln|grep headscale LISTEN 0 4096 127.0.0.1:9090 0.0.0.0:* users:(("headscale",pid=4623,fd=14)) LISTEN 0 4096 *:443 *:* users:(("headscale",pid=4623,fd=13)) LISTEN 0 4096 *:50443 *:* users:(("headscale",pid=4623,fd=12)) ``` I was then able to register two machines using the tailscale client who can ping each other. :)
Author
Owner

@db48x commented on GitHub (Sep 8, 2022):

Good to know that there is a workaround; too bad I cannot put Headscale on 443 here.

@db48x commented on GitHub (Sep 8, 2022): Good to know that there is a workaround; too bad I cannot put Headscale on 443 here.
Author
Owner

@victorhooi commented on GitHub (Sep 23, 2022):

I possibly seem to be hitting this issue:

I've noticed what appears to be a regression between 0.17-alpha1 and 0.17-alpha2/alpha3.

Specifically - with 0.17-alpha1 - headscale is listening on the specified ports in the configuration file (e.g. 8080, 50443 etc.) as expected:

victorhooi@headscale-test:~$ sudo netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      475280/sshd: /usr/s
tcp        0      0 127.0.0.1:9090          0.0.0.0:*               LISTEN      721302/headscale
tcp6       0      0 :::80                   :::*                    LISTEN      721302/headscale
tcp6       0      0 :::8080                 :::*                    LISTEN      721302/headscale
tcp6       0      0 :::22                   :::*                    LISTEN      475280/sshd: /usr/s
tcp6       0      0 :::50443                :::*                    LISTEN      721302/headscale
udp        0      0 0.0.0.0:68              0.0.0.0:*                           383/dhclient
udp        0      0 127.0.0.1:323           0.0.0.0:*                           22697/chronyd
udp6       0      0 ::1:323                 :::*                                22697/chronyd
victorhooi@headscale-test:~$

Here is the startup console output as well:

An updated version of Headscale has been found (0.17.0-alpha3 vs. your current v0.17.0-alpha1). Check it out https://github.com/juanfont/headscale/releases
2022-09-22T07:40:15Z INF Setting up a DERPMap update worker frequency=86400000
2022-09-22T07:40:15Z INF Enabling remote gRPC at 0.0.0.0:50443
2022-09-22T07:40:15Z INF listening and serving gRPC on: 0.0.0.0:50443
2022-09-22T07:40:15Z INF listening and serving HTTP on: 0.0.0.0:8080
2022-09-22T07:40:15Z INF listening and serving metrics on: 127.0.0.1:9090
2022-09-22T07:40:18Z INF The HTTP2 server was closed error=EOF
2022-09-22T07:40:18Z INF ../runner/work/headscale/headscale/protocol_common.go:677 > Machine successfully refreshed machine=Victors-MacBook-Pro-2 node_key=[bQLA7] noise=true old_node_key=[PgiEl]
2022-09-22T07:40:18Z INF Client is starting up. Probably interested in a DERP map handler=PollNetMap machine=Victors-MacBook-Pro-3 noise=true
2022-09-22T07:40:18Z INF Client is starting up. Probably interested in a DERP map handler=PollNetMap machine=Victors-MacBook-Pro-3 noise=true

However, with 0.17-alpha2 and 0.17-alpha3 and the same configuration file - it does not appear to be listening on those ports anymore:

victorhooi@headscale-test:~$ sudo netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      475280/sshd: /usr/s
tcp6       0      0 :::80                   :::*                    LISTEN      721251/headscale
tcp6       0      0 :::22                   :::*                    LISTEN      475280/sshd: /usr/s
udp        0      0 0.0.0.0:68              0.0.0.0:*                           383/dhclient
udp        0      0 127.0.0.1:323           0.0.0.0:*                           22697/chronyd
udp6       0      0 ::1:323                 :::*                                22697/chronyd

and the startup console lines:

victorhooi@headscale-test:~$ sudo headscale serve
2022-09-22T07:28:51Z DBG Loading DERPMap from path func=GetDERPMap url=https://controlplane.tailscale.com/derpmap/default
2022-09-22T07:28:51Z INF Setting up a DERPMap update worker frequency=86400000

I'm just trying to understand the workaround - does this mean we need to change the configured port from 8080 to 443 as well?

@victorhooi commented on GitHub (Sep 23, 2022): I possibly seem to be hitting this issue: I've noticed what appears to be a regression between 0.17-alpha1 and 0.17-alpha2/alpha3. Specifically - with 0.17-alpha1 - headscale is listening on the specified ports in the configuration file (e.g. 8080, 50443 etc.) as expected: ``` victorhooi@headscale-test:~$ sudo netstat -tunlp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 475280/sshd: /usr/s tcp 0 0 127.0.0.1:9090 0.0.0.0:* LISTEN 721302/headscale tcp6 0 0 :::80 :::* LISTEN 721302/headscale tcp6 0 0 :::8080 :::* LISTEN 721302/headscale tcp6 0 0 :::22 :::* LISTEN 475280/sshd: /usr/s tcp6 0 0 :::50443 :::* LISTEN 721302/headscale udp 0 0 0.0.0.0:68 0.0.0.0:* 383/dhclient udp 0 0 127.0.0.1:323 0.0.0.0:* 22697/chronyd udp6 0 0 ::1:323 :::* 22697/chronyd victorhooi@headscale-test:~$ ``` Here is the startup console output as well: ``` An updated version of Headscale has been found (0.17.0-alpha3 vs. your current v0.17.0-alpha1). Check it out https://github.com/juanfont/headscale/releases 2022-09-22T07:40:15Z INF Setting up a DERPMap update worker frequency=86400000 2022-09-22T07:40:15Z INF Enabling remote gRPC at 0.0.0.0:50443 2022-09-22T07:40:15Z INF listening and serving gRPC on: 0.0.0.0:50443 2022-09-22T07:40:15Z INF listening and serving HTTP on: 0.0.0.0:8080 2022-09-22T07:40:15Z INF listening and serving metrics on: 127.0.0.1:9090 2022-09-22T07:40:18Z INF The HTTP2 server was closed error=EOF 2022-09-22T07:40:18Z INF ../runner/work/headscale/headscale/protocol_common.go:677 > Machine successfully refreshed machine=Victors-MacBook-Pro-2 node_key=[bQLA7] noise=true old_node_key=[PgiEl] 2022-09-22T07:40:18Z INF Client is starting up. Probably interested in a DERP map handler=PollNetMap machine=Victors-MacBook-Pro-3 noise=true 2022-09-22T07:40:18Z INF Client is starting up. Probably interested in a DERP map handler=PollNetMap machine=Victors-MacBook-Pro-3 noise=true ``` However, with 0.17-alpha2 and 0.17-alpha3 and the same configuration file - it does not appear to be listening on those ports anymore: ``` victorhooi@headscale-test:~$ sudo netstat -tunlp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 475280/sshd: /usr/s tcp6 0 0 :::80 :::* LISTEN 721251/headscale tcp6 0 0 :::22 :::* LISTEN 475280/sshd: /usr/s udp 0 0 0.0.0.0:68 0.0.0.0:* 383/dhclient udp 0 0 127.0.0.1:323 0.0.0.0:* 22697/chronyd udp6 0 0 ::1:323 :::* 22697/chronyd ``` and the startup console lines: ``` victorhooi@headscale-test:~$ sudo headscale serve 2022-09-22T07:28:51Z DBG Loading DERPMap from path func=GetDERPMap url=https://controlplane.tailscale.com/derpmap/default 2022-09-22T07:28:51Z INF Setting up a DERPMap update worker frequency=86400000 ``` I'm just trying to understand the workaround - does this mean we need to change the configured port from 8080 to 443 as well?
Author
Owner

@kradalby commented on GitHub (Sep 26, 2022):

I think this was a result of a refactoring, can some of you try https://github.com/juanfont/headscale/pull/831.

@kradalby commented on GitHub (Sep 26, 2022): I think this was a result of a refactoring, can some of you try https://github.com/juanfont/headscale/pull/831.
Author
Owner

@kradalby commented on GitHub (Sep 26, 2022):

This fix is now in 0.17 alpha 4, please give it a go.

@kradalby commented on GitHub (Sep 26, 2022): This fix is now in 0.17 alpha 4, please give it a go.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/headscale#319