Private key not creating #428

Closed
opened 2025-12-29 01:29:02 +01:00 by adam · 5 comments
Owner

Originally created by @magf on GitHub (Feb 9, 2023).

Can't start SystemD daemon with first generating private key

After creating user/group, all needed folders, creating SystemD unit and try to start with systemctl start headscale:

INF No private key file at path, creating... path=/etc/headscale/private.key
FTL home/runner/work/headscale/headscale/cmd/headscale/cli/server.go:21 > Error initializing error="failed to read or create private key: failed to save private key to disk: open /etc/headscale/private.key: read-only file system"

All file permissions are met:

# sudo -u headscale ls -la /etc/headscale
total 20
drwxr-xr-x  2 headscale headscale  4096 Feb  9 08:50 .
drwxr-xr-x 93 root      root       4096 Feb  8 16:28 ..
-rw-r--r--  1 headscale headscale 11514 Feb  8 16:33 config.yaml
-rw-rw-r--  1 headscale headscale     0 Feb  9 08:50 test

# sudo -u headscale ls -la /var/lib/headscale
total 20
drwxr-x---  2 headscale headscale 4096 Feb  9 08:55 .
drwxr-xr-x 39 root      root      4096 Feb  8 16:27 ..
-rw-r--r--  1 headscale headscale  220 Jan  6  2022 .bash_logout
-rw-r--r--  1 headscale headscale 3771 Jan  6  2022 .bashrc
-rw-r--r--  1 headscale headscale  807 Jan  6  2022 .profile
-rw-r--r--  1 headscale headscale    0 Feb  9 08:26 db.sqlite

To Reproduce

My Ansible playbook:

--
- hosts: my-host
  become: yes

  vars:
# Application
    arch: amd64
    headscale_version: '0.20.0'
# Configuration

  tasks:
  - name: Download headscale v{{ headscale_version }} && Make headscale executable
    ansible.builtin.get_url:
      url: 'https://github.com/juanfont/headscale/releases/download/v{{ headscale_version }}/headscale_{{ headscale_version }}_linux_{{ arch }}'
      dest: /usr/local/bin/headscale
      mode: '0755'

  - name: Create headscale group
    group:
      name: headscale
      state: present

  - name: Create headscale service account
    ansible.builtin.user:
      name: headscale
      group: headscale
      home: /var/lib/headscale
      shell: /usr/bin/nologin
      system: true

  - name: Prepare a directory to hold headscale configuration
    ansible.builtin.file:
      path: /etc/headscale
      state: directory
      mode: '0755'
      owner: headscale
      group: headscale

  - name: Create configuration from the template
    ansible.builtin.template:
      src: headscale_config.yaml.j2
      dest: /etc/headscale/config.yaml
      mode: '0644'
      owner: headscale
      group: headscale
    notify:
    - Restart service headscale

  - name: Create an empty SQLite database
    ansible.builtin.file:
      path: /var/lib/headscale/db.sqlite
      state: touch
      mode: '0644'
      owner: headscale
      group: headscale

  - name: Create a SystemD headscale.service configuration
    ansible.builtin.copy:
      src: headscale.service
      dest: /etc/systemd/system/headscale.service
    notify:
    - SystemD reread configs
    - Enable and start service headscale

  handlers:
  - name: SystemD reread configs
    ansible.builtin.systemd:
      daemon_reload: true

  - name: Enable and start service headscale
    ansible.builtin.systemd:
      name: headscale
      state: started
      enabled: true
      masked: no

  - name: Restart service headscale
    ansible.builtin.systemd:
      name: headscale
      state: restarted

Context info

uname -a
Linux my-host 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

ansible-playbook --version
ansible-playbook [core 2.14.2]
  config file = /home/ansible/.ansible.cfg
  configured module search path = ['/home/ansible/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/ansible/.local/lib/python3.10/site-packages/ansible
  ansible collection location = /home/ansible/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/ansible/.local/bin/ansible-playbook
  python version = 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True

Configuration files from playbook archived and attached - headscale_config.tar.gz

Originally created by @magf on GitHub (Feb 9, 2023). **Can't start SystemD daemon with first generating private key** After creating user/group, all needed folders, creating SystemD unit and try to start with `systemctl start headscale`: ```bash INF No private key file at path, creating... path=/etc/headscale/private.key FTL home/runner/work/headscale/headscale/cmd/headscale/cli/server.go:21 > Error initializing error="failed to read or create private key: failed to save private key to disk: open /etc/headscale/private.key: read-only file system" ``` All file permissions are met: ```bash # sudo -u headscale ls -la /etc/headscale total 20 drwxr-xr-x 2 headscale headscale 4096 Feb 9 08:50 . drwxr-xr-x 93 root root 4096 Feb 8 16:28 .. -rw-r--r-- 1 headscale headscale 11514 Feb 8 16:33 config.yaml -rw-rw-r-- 1 headscale headscale 0 Feb 9 08:50 test # sudo -u headscale ls -la /var/lib/headscale total 20 drwxr-x--- 2 headscale headscale 4096 Feb 9 08:55 . drwxr-xr-x 39 root root 4096 Feb 8 16:27 .. -rw-r--r-- 1 headscale headscale 220 Jan 6 2022 .bash_logout -rw-r--r-- 1 headscale headscale 3771 Jan 6 2022 .bashrc -rw-r--r-- 1 headscale headscale 807 Jan 6 2022 .profile -rw-r--r-- 1 headscale headscale 0 Feb 9 08:26 db.sqlite ``` **To Reproduce** My Ansible playbook: ```yaml -- - hosts: my-host become: yes vars: # Application arch: amd64 headscale_version: '0.20.0' # Configuration tasks: - name: Download headscale v{{ headscale_version }} && Make headscale executable ansible.builtin.get_url: url: 'https://github.com/juanfont/headscale/releases/download/v{{ headscale_version }}/headscale_{{ headscale_version }}_linux_{{ arch }}' dest: /usr/local/bin/headscale mode: '0755' - name: Create headscale group group: name: headscale state: present - name: Create headscale service account ansible.builtin.user: name: headscale group: headscale home: /var/lib/headscale shell: /usr/bin/nologin system: true - name: Prepare a directory to hold headscale configuration ansible.builtin.file: path: /etc/headscale state: directory mode: '0755' owner: headscale group: headscale - name: Create configuration from the template ansible.builtin.template: src: headscale_config.yaml.j2 dest: /etc/headscale/config.yaml mode: '0644' owner: headscale group: headscale notify: - Restart service headscale - name: Create an empty SQLite database ansible.builtin.file: path: /var/lib/headscale/db.sqlite state: touch mode: '0644' owner: headscale group: headscale - name: Create a SystemD headscale.service configuration ansible.builtin.copy: src: headscale.service dest: /etc/systemd/system/headscale.service notify: - SystemD reread configs - Enable and start service headscale handlers: - name: SystemD reread configs ansible.builtin.systemd: daemon_reload: true - name: Enable and start service headscale ansible.builtin.systemd: name: headscale state: started enabled: true masked: no - name: Restart service headscale ansible.builtin.systemd: name: headscale state: restarted ``` **Context info** ```bash uname -a Linux my-host 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux ansible-playbook --version ansible-playbook [core 2.14.2] config file = /home/ansible/.ansible.cfg configured module search path = ['/home/ansible/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /home/ansible/.local/lib/python3.10/site-packages/ansible ansible collection location = /home/ansible/.ansible/collections:/usr/share/ansible/collections executable location = /home/ansible/.local/bin/ansible-playbook python version = 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] (/usr/bin/python3) jinja version = 3.1.2 libyaml = True ``` Configuration files from playbook archived and attached - [headscale_config.tar.gz](https://github.com/juanfont/headscale/files/10695509/headscale_config.tar.gz)
adam added the bug label 2025-12-29 01:29:02 +01:00
adam closed this issue 2025-12-29 01:29:02 +01:00
Author
Owner

@magf commented on GitHub (Feb 9, 2023):

As root it's first running normally and creating all needed but with root permissions

root@my-host:~# headscale serve
2023-02-09T09:58:40Z INF No private key file at path, creating... path=/etc/headscale/private.key
2023-02-09T09:58:40Z INF No private key file at path, creating... path=/etc/headscale/noise_private.key
2023-02-09T09:58:41Z INF listening and serving HTTP on: 127.0.0.1:8080
2023-02-09T09:58:41Z INF listening and serving metrics on: 127.0.0.1:9090
2023-02-09T09:58:41Z INF Setting up a DERPMap update worker frequency=86400000
2023-02-09T09:59:08Z INF Received signal to stop, shutting down gracefully signal=interrupt
2023-02-09T09:59:08Z INF Headscale stopped

root@my-host:~# ls -la /etc/headscale/
total 80
drwxr-xr-x  2 headscale headscale  4096 Feb  9 09:58 .
drwxr-xr-x 93 root      root       4096 Feb  8 16:28 ..
-rw-r--r--  1 headscale headscale 11514 Feb  8 16:33 config.yaml
-rw-r--r--  1 root      root      49152 Feb  9 09:58 db.sqlite
-rw-------  1 root      root         72 Feb  9 09:58 noise_private.key
-rw-------  1 root      root         72 Feb  9 09:58 private.key
-rw-rw-r--  1 headscale headscale     0 Feb  9 08:50 test
@magf commented on GitHub (Feb 9, 2023): As root it's first running normally and creating all needed but with root permissions ```bash root@my-host:~# headscale serve 2023-02-09T09:58:40Z INF No private key file at path, creating... path=/etc/headscale/private.key 2023-02-09T09:58:40Z INF No private key file at path, creating... path=/etc/headscale/noise_private.key 2023-02-09T09:58:41Z INF listening and serving HTTP on: 127.0.0.1:8080 2023-02-09T09:58:41Z INF listening and serving metrics on: 127.0.0.1:9090 2023-02-09T09:58:41Z INF Setting up a DERPMap update worker frequency=86400000 2023-02-09T09:59:08Z INF Received signal to stop, shutting down gracefully signal=interrupt 2023-02-09T09:59:08Z INF Headscale stopped root@my-host:~# ls -la /etc/headscale/ total 80 drwxr-xr-x 2 headscale headscale 4096 Feb 9 09:58 . drwxr-xr-x 93 root root 4096 Feb 8 16:28 .. -rw-r--r-- 1 headscale headscale 11514 Feb 8 16:33 config.yaml -rw-r--r-- 1 root root 49152 Feb 9 09:58 db.sqlite -rw------- 1 root root 72 Feb 9 09:58 noise_private.key -rw------- 1 root root 72 Feb 9 09:58 private.key -rw-rw-r-- 1 headscale headscale 0 Feb 9 08:50 test ```
Author
Owner

@magf commented on GitHub (Feb 9, 2023):

First manual run as headscale user is normally too

root@my-host:~# cd /var/lib/headscale/
root@my-host:/var/lib/headscale# sudo -u headscale headscale serve
2023-02-09T10:22:09Z INF No private key file at path, creating... path=/etc/headscale/private.key
2023-02-09T10:22:09Z INF No private key file at path, creating... path=/etc/headscale/noise_private.key
2023-02-09T10:22:10Z INF Setting up a DERPMap update worker frequency=86400000
2023-02-09T10:22:10Z INF listening and serving HTTP on: 127.0.0.1:8080
2023-02-09T10:22:10Z INF listening and serving metrics on: 127.0.0.1:9090
@magf commented on GitHub (Feb 9, 2023): First manual run as `headscale` user is normally too ```bash root@my-host:~# cd /var/lib/headscale/ root@my-host:/var/lib/headscale# sudo -u headscale headscale serve 2023-02-09T10:22:09Z INF No private key file at path, creating... path=/etc/headscale/private.key 2023-02-09T10:22:09Z INF No private key file at path, creating... path=/etc/headscale/noise_private.key 2023-02-09T10:22:10Z INF Setting up a DERPMap update worker frequency=86400000 2023-02-09T10:22:10Z INF listening and serving HTTP on: 127.0.0.1:8080 2023-02-09T10:22:10Z INF listening and serving metrics on: 127.0.0.1:9090 ```
Author
Owner

@magf commented on GitHub (Feb 9, 2023):

SOLVED!

Option WorkingDirectory=/var/lib/headscale in unit-file headscale.service needed.

Fix it here, pls

@magf commented on GitHub (Feb 9, 2023): **SOLVED!** Option `WorkingDirectory=/var/lib/headscale` in unit-file `headscale.service` needed. Fix it [here](https://github.com/juanfont/headscale/blob/main/docs/running-headscale-linux.md#running-headscale-in-the-background-with-systemd), pls
Author
Owner

@kradalby commented on GitHub (Feb 9, 2023):

Would be great if you could submit a pr to update the docs.

@kradalby commented on GitHub (Feb 9, 2023): Would be great if you could submit a pr to update the docs.
Author
Owner

@magf commented on GitHub (Feb 9, 2023):

Would be great if you could submit a pr to update the docs.

done

https://github.com/juanfont/headscale/pull/1210

@magf commented on GitHub (Feb 9, 2023): > Would be great if you could submit a pr to update the docs. done https://github.com/juanfont/headscale/pull/1210
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/headscale#428