starred/headscale
1
0
Fork 0
mirror of https://github.com/juanfont/headscale.git synced 2026-01-12 04:10:32 +01:00
Code Issues 98 Packages Projects Releases 10 Wiki Activity

[Bug] Headscale reads extra DNS records from path too early #1095

New Issue
Open
opened 2025-12-29 02:28:14 +01:00 by adam · 2 comments
adam commented 2025-12-29 02:28:14 +01:00
Owner
Copy Link

Originally created by @nblock on GitHub (Aug 30, 2025).

Is this a support request?

  • This is not a support request

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

I use the dns.extra_records_path configuration option to load DNS records from a JSON file. This file is created by a script that combines multiple files into one. The relevant line is:

jq --sort-keys --slurp 'unique | flatten' dns.d/*.json > /var/lib/headscale/extra-records.json

There are about 150 DNS records written to /var/lib/headscale/extra-records.json with each invocation. From time to time Headscale's filewatcher picks up the file while it is being written. Presumably, the output buffer is flushed by jq and a partial JSON file is persisted for a short while. This causes errors about broken JSON in Headscale's logs:

[snipped]
{\\n    \\\"name\\\": \\\"acgmw.example.com\\\",\\n    \\\"type\\\": \\\"A\\\",\\n    \\\"value\\\": \\\"100.64.0.1\\\"\\n  },\\n  {\\n    \\\"name\\\": \\\"acgmx.example.com\\\",\\n    \\\"type\\\": \\\"A\\\",\\n    \\\"value\\\": \\\"100.64.0.1\\\"\\n  },\\n  {\\n    \\\"name\\\": \\\"acgmy.example.com\\\",\\n    \\\"type\\\": \\\"A\\\",\\n    \\\"v\": 
unexpected end of JSON input"

As a workaround, I use stdbuf to enlarge the output buffer for jq:

stdbuf -o 1M jq --sort-keys --slurp 'unique | flatten' dns.d/*.json > /var/lib/headscale/extra-records.json

Is there a way to configure the filewatcher such that it picks up the file once no other process is writing to it?

Expected Behavior

Unsure what is the best solution to this, some ideas:

  • Read the records in dns.extra_records_path once no other process writes to it
  • Delay the read a bit (probably just a workaround)

Steps To Reproduce

  1. Setup Headscale 0.26.1 with dns.extra_records_path
  2. Create some sample JSON files
  3. Update the JSON file with the DNS records: 
    jq --sort-keys --slurp 'unique | flatten' sample-*.json > .headscale/extra-records.json
  4. Observe the error message in Headscale's log

Environment

- OS: Debian 13
- Headscale version: 0.26.1
- Tailscale version: -

Runtime environment

  • Headscale is behind a (reverse) proxy
  • Headscale runs in a container

Debug information

Originally created by @nblock on GitHub (Aug 30, 2025). ### Is this a support request? - [x] This is not a support request ### Is there an existing issue for this? - [x] I have searched the existing issues ### Current Behavior I use the `dns.extra_records_path` configuration option to load DNS records from a JSON file. This file is created by a script that combines multiple files into one. The relevant line is: ```bash jq --sort-keys --slurp 'unique | flatten' dns.d/*.json > /var/lib/headscale/extra-records.json ``` There are about 150 DNS records written to `/var/lib/headscale/extra-records.json` with each invocation. From time to time Headscale's filewatcher picks up the file while it is being written. Presumably, the output buffer is flushed by `jq` and a partial JSON file is persisted for a short while. This causes errors about broken JSON in Headscale's logs: ``` [snipped] {\\n \\\"name\\\": \\\"acgmw.example.com\\\",\\n \\\"type\\\": \\\"A\\\",\\n \\\"value\\\": \\\"100.64.0.1\\\"\\n },\\n {\\n \\\"name\\\": \\\"acgmx.example.com\\\",\\n \\\"type\\\": \\\"A\\\",\\n \\\"value\\\": \\\"100.64.0.1\\\"\\n },\\n {\\n \\\"name\\\": \\\"acgmy.example.com\\\",\\n \\\"type\\\": \\\"A\\\",\\n \\\"v\": unexpected end of JSON input" ``` As a workaround, I use `stdbuf` to enlarge the output buffer for `jq`: ```bash stdbuf -o 1M jq --sort-keys --slurp 'unique | flatten' dns.d/*.json > /var/lib/headscale/extra-records.json ``` Is there a way to configure the filewatcher such that it picks up the file once no other process is writing to it? ### Expected Behavior Unsure what is the best solution to this, some ideas: - Read the records in `dns.extra_records_path` once no other process writes to it - Delay the read a bit (probably just a workaround) ### Steps To Reproduce 1. Setup Headscale 0.26.1 with `dns.extra_records_path` 1. Create some sample JSON files - [sample-1000.json](https://github.com/user-attachments/files/22057864/sample-1000.json) - [sample-100.json](https://github.com/user-attachments/files/22057863/sample-100.json) - [sample-10.json](https://github.com/user-attachments/files/22057865/sample-10.json) - [gen-extra-records.py](https://github.com/user-attachments/files/22057866/gen-extra-records.py) 1. Update the JSON file with the DNS records: ```bash jq --sort-keys --slurp 'unique | flatten' sample-*.json > .headscale/extra-records.json ``` 1. Observe the error message in Headscale's log ### Environment ```markdown - OS: Debian 13 - Headscale version: 0.26.1 - Tailscale version: - ``` ### Runtime environment - [ ] Headscale is behind a (reverse) proxy - [ ] Headscale runs in a container ### Debug information -
adam added the bugno-stale-bot labels 2025-12-29 02:28:14 +01:00
adam commented 2025-12-29 02:28:15 +01:00
Author
Owner
Copy Link

@reinob commented on GitHub (Sep 18, 2025):

It would probably be safer to first generate a temporary file, and then replace the existing one with the new one, in one go (using mv, which should be atomic).

@reinob commented on GitHub (Sep 18, 2025): It would probably be safer to first generate a temporary file, and then replace the existing one with the new one, in one go (using mv, which should be atomic).
adam commented 2025-12-29 02:28:15 +01:00
Author
Owner
Copy Link

@nblock commented on GitHub (Sep 18, 2025):

(using mv, which should be atomic).

I saw some errors with mv also; but can't remember if they were identical.

@nblock commented on GitHub (Sep 18, 2025): > (using mv, which should be atomic). I saw some errors with `mv` also; but can't remember if they were identical.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
CLI
DERP
DNS
Nix
OIDC
SSH
bug
database
documentation
duplicate
enhancement
faq
good first issue
grants
help wanted
might-come
needs design doc
needs investigation
no-stale-bot
out of scope
performance
policy 📝
pull-request
Mirrored from GitHub Pull Request
 question
regression
routes
stale
tags
tailscale-feature-gap
well described ❤️
wontfix
No Label bug no-stale-bot
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/headscale#1095
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?