[Bug] v0.27.0 Headscale break if I using litestream #1122

Closed
opened 2025-12-29 02:28:22 +01:00 by adam · 2 comments
Owner

Originally created by @gawsoftpl on GitHub (Oct 27, 2025).

Is this a support request?

  • This is not a support request

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

When I run headscale in new version 0.27.0 I received migration table error. I have to remove tables: _litestream_lock and _litestream_seq for start headscale otherwise it wont start.

2025-10-27T14:51:49Z INF Opening database database=sqlite3 path=/var/lib/headscale/db.sqlite3
SQLite schema failed to validate:

>> Remove table "_litestream_lock"

>> Remove table "_litestream_seq"

2025-10-27T14:51:49Z FTL home/runner/work/headscale/headscale/cmd/headscale/cli/serve.go:32 > Error initializing error="creating new headscale: init state: init database: validating schema: invalid schema (-got, +want):\n\n>> Remove table \"_litestream_lock\"\n\n>> Remove table \"_litestream_seq\"\n"

Expected Behavior

Should start normal

Steps To Reproduce

  1. Start headscale with litestream

Environment

- OS:ubuntu 24.0.4
- Headscale version: 0.27.0
- Tailscale version: 1.88.4

Runtime environment

  • Headscale is behind a (reverse) proxy
  • Headscale runs in a container

Debug information

xx

Originally created by @gawsoftpl on GitHub (Oct 27, 2025). ### Is this a support request? - [x] This is not a support request ### Is there an existing issue for this? - [x] I have searched the existing issues ### Current Behavior When I run headscale in new version 0.27.0 I received migration table error. I have to remove tables: `_litestream_lock` and `_litestream_seq` for start headscale otherwise it wont start. ```sh 2025-10-27T14:51:49Z INF Opening database database=sqlite3 path=/var/lib/headscale/db.sqlite3 SQLite schema failed to validate: >> Remove table "_litestream_lock" >> Remove table "_litestream_seq" 2025-10-27T14:51:49Z FTL home/runner/work/headscale/headscale/cmd/headscale/cli/serve.go:32 > Error initializing error="creating new headscale: init state: init database: validating schema: invalid schema (-got, +want):\n\n>> Remove table \"_litestream_lock\"\n\n>> Remove table \"_litestream_seq\"\n" ``` ### Expected Behavior Should start normal ### Steps To Reproduce 1. Start headscale with litestream ### Environment ```markdown - OS:ubuntu 24.0.4 - Headscale version: 0.27.0 - Tailscale version: 1.88.4 ``` ### Runtime environment - [x] Headscale is behind a (reverse) proxy - [x] Headscale runs in a container ### Debug information xx
adam added the bugdatabase labels 2025-12-29 02:28:22 +01:00
adam closed this issue 2025-12-29 02:28:22 +01:00
Author
Owner

@porelli commented on GitHub (Oct 29, 2025):

I've got a similar problem but I'm not using litestream. When starting with v0.27.0 I get this:

2025-10-29T03:42:21Z FTL home/runner/work/headscale/headscale/cmd/headscale/cli/serve.go:32 > Error initializing error="creating new headscale: init state: init database: validating schema: invalid schema (-got, +want):\n\n>> Remove table \"namespaces\"\n"

2025-10-29T03:43:22Z INF Opening database database=sqlite3 path=/var/lib/headscale/db.sqlite

SQLite schema failed to validate:

>> Remove table "namespaces"

Reverting the container to v0.26.1 everything starts properly.

@porelli commented on GitHub (Oct 29, 2025): I've got a similar problem but I'm not using litestream. When starting with v0.27.0 I get this: ``` 2025-10-29T03:42:21Z FTL home/runner/work/headscale/headscale/cmd/headscale/cli/serve.go:32 > Error initializing error="creating new headscale: init state: init database: validating schema: invalid schema (-got, +want):\n\n>> Remove table \"namespaces\"\n" 2025-10-29T03:43:22Z INF Opening database database=sqlite3 path=/var/lib/headscale/db.sqlite SQLite schema failed to validate: >> Remove table "namespaces" ``` Reverting the container to v0.26.1 everything starts properly.
Author
Owner

@kradalby commented on GitHub (Oct 29, 2025):

I have to remove tables: _litestream_lock and _litestream_seq for start headscale otherwise it wont start

Oh yes, this is an issue. I will have to think a bit on how to solve that, as we really do not want to drop down into the previous hole of not-as-strict schema integrity.
https://github.com/tailscale/squibble/issues/1 has been filed, hopefully we find a good solution.

I've got a similar problem but I'm not using litestream. When starting with v0.27.0 I get this:

This must have been a schema we missed in the cleanup, I have to investigate how that happened, but it is good to know as this is the exact condition we now should be able to avoid in the future.

@porelli if you drop that table manually, you should be fine to upgrade.

@kradalby commented on GitHub (Oct 29, 2025): > I have to remove tables: _litestream_lock and _litestream_seq for start headscale otherwise it wont start Oh yes, this is an issue. I will have to think a bit on how to solve that, as we really do not want to drop down into the previous hole of not-as-strict schema integrity. https://github.com/tailscale/squibble/issues/1 has been filed, hopefully we find a good solution. > I've got a similar problem but I'm not using litestream. When starting with v0.27.0 I get this: This must have been a schema we missed in the cleanup, I have to investigate how that happened, but it is good to know as this is the exact condition we now should be able to avoid in the future. @porelli if you drop that table manually, you should be fine to upgrade.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/headscale#1122