[Bug] Latest Docker Release Breaks Foreign Key Constraint in Database #921

@simonccc commented on GitHub (Jan 23, 2025):

I also had this issue ( in sqlite ) but running direct with no docker - I deleted the node using cmd line and it started up ok

@simonccc commented on GitHub (Jan 23, 2025): I also had this issue ( in sqlite ) but running direct with no docker - I deleted the node using cmd line and it started up ok

adam commented

@kradalby commented on GitHub (Jan 23, 2025):

Which release did you upgrade from? This is unfortunate, this part of the code seem to be a minefield as there has been years of no constraint, allowing a lot of stale broken nodes/routes/keys being written to the database.

The biggest challenge for me is figuring out what will blow up or needs to be removed before and what will not.

@kradalby commented on GitHub (Jan 23, 2025): Which release did you upgrade from? This is unfortunate, this part of the code seem to be a minefield as there has been years of no constraint, allowing a lot of stale broken nodes/routes/keys being written to the database. The biggest challenge for me is figuring out what will blow up or needs to be removed before and what will not.

adam commented

@xcjs commented on GitHub (Jan 23, 2025):

@kradalby I was/am, on the :latest Docker tag, so I'm assuming whatever the prior major version was. Watchtower keeps my containers updated for me so I force myself to stay up to date on things. Based on other Docker tags out there, it was probably 0.24.0 or 0.23.0?

Whatever version was assigned to the latest tag earlier today would be the version I was on.

@xcjs commented on GitHub (Jan 23, 2025): @kradalby I was/am, on the `:latest` Docker tag, so I'm assuming whatever the prior major version was. Watchtower keeps my containers updated for me so I force myself to stay up to date on things. Based on other Docker tags out there, it was probably `0.24.0` or `0.23.0`? Whatever version was assigned to the `latest` tag earlier today would be the version I was on.

adam commented

@xcjs commented on GitHub (Jan 23, 2025):

I can also say that pulling a backup and using any other container image updated today results in the same issue, so I was not able to go back to an earlier version unless it was really old.

@xcjs commented on GitHub (Jan 23, 2025): I can also say that pulling a backup and using any other container image updated today results in the same issue, so I was not able to go back to an earlier version unless it was _really_ old.

adam commented

2025-12-29 02:26:02 +01:00

@LucasJanin commented on GitHub (Jan 23, 2025):

The same issue, on my side, I try to upgrade my Headscale from 0.23 to 0.24.1

@LucasJanin commented on GitHub (Jan 23, 2025): The same issue, on my side, I try to upgrade my Headscale from 0.23 to 0.24.1

adam commented

@kradalby commented on GitHub (Jan 23, 2025):

If anyone have a database they could strip of personal info that they can send to me, then that would be helpful for me to include in a test case.

My email is in my profile.

@kradalby commented on GitHub (Jan 23, 2025): If anyone have a database they could strip of personal info that they can send to me, then that would be helpful for me to include in a test case. My email is in my profile.

adam commented

@kradalby commented on GitHub (Jan 23, 2025):

I have not found any of this issue on any of my "long running instances", so I suspect that there is an old node that has been around from before certain constraint was introduced and now they are violating.

@kradalby commented on GitHub (Jan 23, 2025): I have not found any of this issue on any of my "long running instances", so I suspect that there is an old node that has been around from before certain constraint was introduced and now they are violating.

adam commented

@xcjs commented on GitHub (Jan 23, 2025):

@kradalby It depends on what "long running" means in this context, but my server has been up for 2-3 years.

@xcjs commented on GitHub (Jan 23, 2025): @kradalby It depends on what "long running" means in this context, but my server has been up for 2-3 years.

adam commented

@LucasJanin commented on GitHub (Jan 23, 2025):

Thanks @xcjs for your solution!

sudo apt install sqlite3
sudo sqlite3 db.sqlite
sqlite3> UPDATE nodes SET auth_key_id = NULL WHERE id = insert_row_id_here;
sqlite3> .exit

My Headscale v0.24.1 works perfectly!!!

@LucasJanin commented on GitHub (Jan 23, 2025): Thanks @xcjs for your solution! ``` sudo apt install sqlite3 sudo sqlite3 db.sqlite sqlite3> UPDATE nodes SET auth_key_id = NULL WHERE id = insert_row_id_here; sqlite3> .exit ``` My Headscale v0.24.1 works perfectly!!!

adam commented

@xcjs commented on GitHub (Jan 23, 2025):

If anyone have a database they could strip of personal info that they can send to me, then that would be helpful for me to include in a test case.

My email is in my profile.

I can probably do this with my backup copy of my database, but allow me a few days to look into scrubbing it.

@xcjs commented on GitHub (Jan 23, 2025): > If anyone have a database they could strip of personal info that they can send to me, then that would be helpful for me to include in a test case. > > My email is in my profile. > > I can probably do this with my backup copy of my database, but allow me a few days to look into scrubbing it.

adam commented

@xcjs commented on GitHub (Jan 23, 2025):

Thanks @xcjs for your solution!
sudo apt install sqlite3
sudo sqlite3 db.sqlite
sqlite3> UPDATE nodes SET auth_key_id = NULL WHERE id = insert_row_id_here;
sqlite3> .exit
My Headscale v0.24.1 works perfectly!!!

Thanks for publishing this here! I was going to try and mention it in passing at least for other people facing the issue.

@xcjs commented on GitHub (Jan 23, 2025): > Thanks @xcjs for your solution! > > ``` > sudo apt install sqlite3 > sudo sqlite3 db.sqlite > sqlite3> UPDATE nodes SET auth_key_id = NULL WHERE id = insert_row_id_here; > sqlite3> .exit > ``` > > My Headscale v0.24.1 works perfectly!!! Thanks for publishing this here! I was going to try and mention it in passing at least for other people facing the issue.

adam commented

@yqs112358 commented on GitHub (Jan 24, 2025):

Thanks @xcjs for your solution!
sudo apt install sqlite3
sudo sqlite3 db.sqlite
sqlite3> UPDATE nodes SET auth_key_id = NULL WHERE id = insert_row_id_here;
sqlite3> .exit
My Headscale v0.24.1 works perfectly!!!

Same issue encounted. Thanks so much for your solution here 😊

@yqs112358 commented on GitHub (Jan 24, 2025): > Thanks [@xcjs](https://github.com/xcjs) for your solution! > > ``` > sudo apt install sqlite3 > sudo sqlite3 db.sqlite > sqlite3> UPDATE nodes SET auth_key_id = NULL WHERE id = insert_row_id_here; > sqlite3> .exit > ``` > > My Headscale v0.24.1 works perfectly!!! Same issue encounted. Thanks so much for your solution here 😊

adam commented

@kradalby commented on GitHub (Jan 24, 2025):

For you who encountered this, was it typically one node that was bad? Or multiple?

@kradalby commented on GitHub (Jan 24, 2025): For you who encountered this, was it typically one node that was bad? Or multiple?

adam commented

@ghost commented on GitHub (Jan 24, 2025):

@kradalby multiple in my case (7 devices out of 12). Mostly devices that haven't authenticated the last months

@ghost commented on GitHub (Jan 24, 2025): @kradalby multiple in my case (7 devices out of 12). Mostly devices that haven't authenticated the last months

adam commented

@masterwishx commented on GitHub (Jan 24, 2025):

Same issue for me

@masterwishx commented on GitHub (Jan 24, 2025): Same issue for me

adam commented

@masterwishx commented on GitHub (Jan 24, 2025):

For you who encountered this, was it typically one node that was bad? Or multiple?

2025-01-24T11:14:29+02:00 ERR Foreign key constraint violated parent=pre_auth_keys row_id=3 table=nodes
2025-01-24T11:14:29+02:00 ERR Foreign key constraint violated parent=pre_auth_keys row_id=8 table=nodes
2025-01-24T11:14:29+02:00 ERR Foreign key constraint violated parent=pre_auth_keys row_id=9 table=nodes
2025-01-24T11:14:29+02:00 ERR Foreign key constraint violated parent=pre_auth_keys row_id=10 table=nodes
2025-01-24T11:14:29+02:00 ERR Foreign key constraint violated parent=pre_auth_keys row_id=11 table=nodes

@masterwishx commented on GitHub (Jan 24, 2025): > For you who encountered this, was it typically one node that was bad? Or multiple? ``` 2025-01-24T11:14:29+02:00 ERR Foreign key constraint violated parent=pre_auth_keys row_id=3 table=nodes 2025-01-24T11:14:29+02:00 ERR Foreign key constraint violated parent=pre_auth_keys row_id=8 table=nodes 2025-01-24T11:14:29+02:00 ERR Foreign key constraint violated parent=pre_auth_keys row_id=9 table=nodes 2025-01-24T11:14:29+02:00 ERR Foreign key constraint violated parent=pre_auth_keys row_id=10 table=nodes 2025-01-24T11:14:29+02:00 ERR Foreign key constraint violated parent=pre_auth_keys row_id=11 table=nodes ```

adam commented

@kradalby commented on GitHub (Jan 24, 2025):

I'm just trying to understand what was the trigger for this, has the pre_auth_key been deleted for the nodes that violates?

@kradalby commented on GitHub (Jan 24, 2025): I'm just trying to understand what was the trigger for this, has the pre_auth_key been deleted for the nodes that violates?

adam commented

@masterwishx commented on GitHub (Jan 24, 2025):

I'm just trying to understand what was the trigger for this, has the pre_auth_key been deleted for the nodes that violates?

Seems issue coused by have =0 instead of NULL , and i think they all expired

@masterwishx commented on GitHub (Jan 24, 2025): > I'm just trying to understand what was the trigger for this, has the pre_auth_key been deleted for the nodes that violates? Seems issue coused by have =0 instead of NULL , and i think they all expired ![Image](https://github.com/user-attachments/assets/5d52c978-97a4-46e6-b6b2-2fa28b40768e)

adam commented

@masterwishx commented on GitHub (Jan 24, 2025):

I'm just trying to understand what was the trigger for this, has the pre_auth_key been deleted for the nodes that violates?

i had 3,8,9,10,11 id3= expired , id8-id11 = not exist at all (maybe deleted) have only 1-7 preauthkeys in list .

@masterwishx commented on GitHub (Jan 24, 2025): > I'm just trying to understand what was the trigger for this, has the pre_auth_key been deleted for the nodes that violates? i had 3,8,9,10,11 id3= expired , id8-id11 = not exist at all (maybe deleted) have only 1-7 preauthkeys in list .

adam commented

@masterwishx commented on GitHub (Jan 24, 2025):

Thanks @xcjs for your solution!
sudo apt install sqlite3
sudo sqlite3 db.sqlite
sqlite3> UPDATE nodes SET auth_key_id = NULL WHERE id = insert_row_id_here;
sqlite3> .exit
My Headscale v0.24.1 works perfectly!!!
Thanks for publishing this here! I was going to try and mention it in passing at least for other people facing the issue.

Thanks , worked on old db befor update only

@masterwishx commented on GitHub (Jan 24, 2025): > > Thanks [@xcjs](https://github.com/xcjs) for your solution! > > ``` > > sudo apt install sqlite3 > > sudo sqlite3 db.sqlite > > sqlite3> UPDATE nodes SET auth_key_id = NULL WHERE id = insert_row_id_here; > > sqlite3> .exit > > ``` > > > > > > > > > > My Headscale v0.24.1 works perfectly!!! > > Thanks for publishing this here! I was going to try and mention it in passing at least for other people facing the issue. Thanks , worked on old db befor update only

adam commented

@kradalby commented on GitHub (Jan 24, 2025):

Interesting, as part of fixing up Tags, which has to be done and will be done in the not so far future, we need to always keep the pre_auth_key and it cannot be deleted as it contains information about the tags of a node.

Previously we did not appreciate this constraint, and allowed them to be deleted and this is now hunting us.

@kradalby commented on GitHub (Jan 24, 2025): Interesting, as part of fixing up Tags, which has to be done and will be done in the not so far future, we need to _always_ keep the pre_auth_key and it cannot be deleted as it contains information about the tags of a node. Previously we did not appreciate this constraint, and allowed them to be deleted and this is now hunting us.

adam commented

@super-ben commented on GitHub (Jan 25, 2025):

I can also confirm that the sqlite magic in the previous comment did work (in my case, there were 3 problematic rows). As I'm using the latest tag in Docker and upgraded from 0.24.0 (so not a major version upgrade), this was somewhat unexpected, but glad there was a quick solution.

@super-ben commented on GitHub (Jan 25, 2025): I can also confirm that the sqlite magic in the previous comment did work (in my case, there were 3 problematic rows). As I'm using the ```latest``` tag in Docker and upgraded from 0.24.0 (so not a major version upgrade), this was somewhat unexpected, but glad there was a quick solution.

adam commented

@xlemassacre commented on GitHub (Jan 26, 2025):

Also had the same issue upgrading from 0.23.0 to 0.24.1.
The command above updating auth_key_id from '0' to NULL on 5 nodes solved it.

@xlemassacre commented on GitHub (Jan 26, 2025): Also had the same issue upgrading from 0.23.0 to 0.24.1. The command above updating auth_key_id from '0' to NULL on 5 nodes solved it.

adam commented

@NOP4 commented on GitHub (Jan 28, 2025):

This fix don't work for me. I still have the same error after setting auth_key_id to NULL for row 4:

2025-01-28T21:44:24Z ERR Foreign key constraint violated parent=pre_auth_keys row_id=4 table=nodes
2025-01-28T21:44:24Z FTL Migration failed: foreign key constraints violated error="foreign key constraints violated"

I checked in the database, and it was 0, now it's NULL.
I see all devices registered with register_method "cli" have NULL now. Devices registered with authkey have a number. row_id 4 is the first entry with "register_method = cli" in the table.
Any idea?

@NOP4 commented on GitHub (Jan 28, 2025): This fix don't work for me. I still have the same error after setting auth_key_id to NULL for row 4: ``` 2025-01-28T21:44:24Z ERR Foreign key constraint violated parent=pre_auth_keys row_id=4 table=nodes 2025-01-28T21:44:24Z FTL Migration failed: foreign key constraints violated error="foreign key constraints violated" ``` I checked in the database, and it was 0, now it's NULL. I see all devices registered with register_method "cli" have NULL now. Devices registered with authkey have a number. row_id 4 is the first entry with "register_method = cli" in the table. Any idea?

adam commented

@NOP4 commented on GitHub (Jan 28, 2025):

I've done a bit of digging:
Switching back to version 0.23 or 0.24 does not work as the database is now corrupted.
I reloaded some backup, and when I start with version 0.23, everything works fine.
The problem is clearly in the database conversion process between 0.23 and 0.24/latest.

The problem is... it corrupts database. Maybe, while the root cause and a fix is implemented, you should rollback the docker hub version before to many databases are corrupted and require restart from backup like I had to.

@NOP4 commented on GitHub (Jan 28, 2025): I've done a bit of digging: Switching back to version 0.23 or 0.24 does not work as the database is now corrupted. I reloaded some backup, and when I start with version 0.23, everything works fine. The problem is clearly in the database conversion process between 0.23 and 0.24/latest. The problem is... it corrupts database. Maybe, while the root cause and a fix is implemented, you should rollback the docker hub version before to many databases are corrupted and require restart from backup like I had to.

adam commented

@xcjs commented on GitHub (Jan 28, 2025):

This fix don't work for me. I still have the same error after setting auth_key_id to NULL for row 4:
2025-01-28T21:44:24Z ERR Foreign key constraint violated parent=pre_auth_keys row_id=4 table=nodes
2025-01-28T21:44:24Z FTL Migration failed: foreign key constraints violated error="foreign key constraints violated"
I checked in the database, and it was 0, now it's NULL.
I see all devices registered with register_method "cli" have NULL now. Devices registered with authkey have a number. row_id 4 is the first entry with "register_method = cli" in the table.
Any idea?

If you're still getting that error, it doesn't sound like the row was updated properly. Can you check again?

@xcjs commented on GitHub (Jan 28, 2025): > This fix don't work for me. I still have the same error after setting auth_key_id to NULL for row 4: > ``` > 2025-01-28T21:44:24Z ERR Foreign key constraint violated parent=pre_auth_keys row_id=4 table=nodes > 2025-01-28T21:44:24Z FTL Migration failed: foreign key constraints violated error="foreign key constraints violated" > ``` > I checked in the database, and it was 0, now it's NULL. > I see all devices registered with register_method "cli" have NULL now. Devices registered with authkey have a number. row_id 4 is the first entry with "register_method = cli" in the table. > Any idea? If you're still getting that error, it doesn't sound like the row was updated properly. Can you check again?

adam commented

@xcjs commented on GitHub (Jan 28, 2025):

I've done a bit of digging:
Switching back to version 0.23 or 0.24 does not work as the database is now corrupted.
I reloaded some backup, and when I start with version 0.23, everything works fine.
The problem is clearly in the database conversion process between 0.23 and 0.24/latest.

The problem is... it corrupts database. Maybe, while the root cause and a fix is implemented, you should rollback the docker hub version before to many databases are corrupted and require restart from backup like I had to.

I ran into the same issue, but I think it's because all the container versions have the new migrations as they were all updated around the same time.

If a rollback occurs, this could break other environments that have already upgraded without issue or that had the work-around applied.

@xcjs commented on GitHub (Jan 28, 2025): > I've done a bit of digging: > Switching back to version 0.23 or 0.24 does not work as the database is now corrupted. > I reloaded some backup, and when I start with version 0.23, everything works fine. > The problem is clearly in the database conversion process between 0.23 and 0.24/latest. > > The problem is... it corrupts database. Maybe, while the root cause and a fix is implemented, you should rollback the docker hub version before to many databases are corrupted and require restart from backup like I had to. I ran into the same issue, but I think it's because all the container versions have the new migrations as they were all updated around the same time. If a rollback occurs, this could break other environments that have already upgraded without issue or that had the work-around applied.

adam commented

@NOP4 commented on GitHub (Jan 28, 2025):

Yes, you're right, can be dangerous rolling back...

Switching for 0.23 to 0.24 I have:

2025-01-28T22:25:32Z ERR Foreign key constraint violated parent=pre_auth_keys row_id=4 table=nodes
2025-01-28T22:25:32Z FTL Migration failed: foreign key constraints violated error="foreign key constraints violated"

I stop the docker container, then edit the database:

> sqlite3 db.sqlite
SQLite version 3.37.2 2022-01-06 13:25:41
Enter ".help" for usage hints.
sqlite> UPDATE nodes SET auth_key_id = NULL WHERE id=4;
sqlite> .exit

Restart docker container, still in version 0.24 and I have the same error:

2025-01-28T22:27:40Z ERR Foreign key constraint violated parent=pre_auth_keys row_id=4 table=nodes
2025-01-28T22:27:40Z FTL Migration failed: foreign key constraints violated error="foreign key constraints violated"

Content of table is:

I also tried to correct the table before the migration: Start from a working version 0.23. Stop container. Edit table. Start with version 0.24 => same error.

@NOP4 commented on GitHub (Jan 28, 2025): Yes, you're right, can be dangerous rolling back... Switching for 0.23 to 0.24 I have: ``` 2025-01-28T22:25:32Z ERR Foreign key constraint violated parent=pre_auth_keys row_id=4 table=nodes 2025-01-28T22:25:32Z FTL Migration failed: foreign key constraints violated error="foreign key constraints violated" ``` I stop the docker container, then edit the database: ``` > sqlite3 db.sqlite SQLite version 3.37.2 2022-01-06 13:25:41 Enter ".help" for usage hints. sqlite> UPDATE nodes SET auth_key_id = NULL WHERE id=4; sqlite> .exit ``` Restart docker container, still in version 0.24 and I have the same error: ``` 2025-01-28T22:27:40Z ERR Foreign key constraint violated parent=pre_auth_keys row_id=4 table=nodes 2025-01-28T22:27:40Z FTL Migration failed: foreign key constraints violated error="foreign key constraints violated" ``` Content of table is: ![Image](https://github.com/user-attachments/assets/7f3b1923-37c9-4939-8803-a2abbe5d3063) I also tried to correct the table before the migration: Start from a working version 0.23. Stop container. Edit table. Start with version 0.24 => same error.

adam commented

@panteparak commented on GitHub (Feb 1, 2025):

I am seeing this as well, Attempting to Upgrade from 0.22.3 -> 0.24.2.

What I've encountered was auth_key_id is 0 and will cause foreign key issue

What i've notice is, all users that have auth_key_id = 0 are users authenticated via OIDC (Azure Entra)

Will update below on my findings after manual data only import script has been fix with this.

@panteparak commented on GitHub (Feb 1, 2025): I am seeing this as well, Attempting to Upgrade from 0.22.3 -> 0.24.2. What I've encountered was `auth_key_id` is 0 and will cause foreign key issue What i've notice is, all users that have `auth_key_id = 0` are users authenticated via OIDC (Azure Entra) Will update below on my findings after manual data only import script has been fix with this.

adam commented

@danielw97 commented on GitHub (Feb 5, 2025):

Thanks a bunch for the solution here, I also had this issue when upgrading to v0.25.0-beta.1 in my case my machine was a bit behind coming from v0.23.0-alpha12 but everything is working as expected now.

@danielw97 commented on GitHub (Feb 5, 2025): Thanks a bunch for the solution here, I also had this issue when upgrading to v0.25.0-beta.1 in my case my machine was a bit behind coming from v0.23.0-alpha12 but everything is working as expected now.

adam commented

@david0161 commented on GitHub (Feb 6, 2025):

Running in Unraid, updated db and same issue.

@david0161 commented on GitHub (Feb 6, 2025): Running in Unraid, updated db and same issue.

adam commented

@nblock commented on GitHub (Feb 6, 2025):

We'd like to have a look at this. Please send a copy of an old database (0.22, 0.23, …) that fails to upgrade to @kradalby (email in his profile). Please strip sensitive data before sending. Thanks!

@nblock commented on GitHub (Feb 6, 2025): We'd like to have a look at this. Please send a copy of an old database (0.22, 0.23, …) that fails to upgrade to @kradalby (email in his profile). Please strip sensitive data before sending. Thanks!

adam commented

@vbrandl commented on GitHub (Feb 6, 2025):

If anyone have a database they could strip of personal info that they can send to me, then that would be helpful for me to include in a test case.

My email is in my profile.

I made a backup of my SQLite before applying the UPDATE nodes SET auth_key_id = NULL WHERE id IN (...). This fixed the problem for me, headscale started again and everything seems to work.

Do you still need an example of a database that breaks the update? If so, can you help me stripping personal info from the DB, than I can give you a copy.

Edit:

Is this enough to remove personal info or did I miss anything? Are those *_key columns in nodes public or private keys? Should I remove those?

update nodes set hostname = 'hostname_' || id, given_name = 'given_name' || id, host_info = null;
update users set name = 'username_' || id, display_name = 'display_name_' || id, email = 'email_' || id || '@example.com';

@vbrandl commented on GitHub (Feb 6, 2025): > If anyone have a database they could strip of personal info that they can send to me, then that would be helpful for me to include in a test case. > > My email is in my profile. I made a backup of my SQLite before applying the `UPDATE nodes SET auth_key_id = NULL WHERE id IN (...)`. This fixed the problem for me, headscale started again and everything seems to work. Do you still need an example of a database that breaks the update? If so, can you help me stripping personal info from the DB, than I can give you a copy. Edit: Is this enough to remove personal info or did I miss anything? Are those `*_key` columns in `nodes` public or private keys? Should I remove those? ```sql update nodes set hostname = 'hostname_' || id, given_name = 'given_name' || id, host_info = null; update users set name = 'username_' || id, display_name = 'display_name_' || id, email = 'email_' || id || '@example.com'; ```

adam commented

@kradalby commented on GitHub (Feb 6, 2025):

Yes please, the more the better. I am unsure why it is still around, there is too many combinations of versions I suppose.

Please email them to me, and note which version it was on and which version you upgraded to.
Strip all info that is PII and similar as I would like to include them in test, which means checking them into git.

@kradalby commented on GitHub (Feb 6, 2025): Yes please, the more the better. I am unsure why it is still around, there is too many combinations of versions I suppose. Please email them to me, and note which version it was on and which version you upgraded to. Strip all info that is PII and similar as I would like to include them in test, which means checking them into git.

adam commented

@kradalby commented on GitHub (Feb 6, 2025):

Is this enough to remove personal info or did I miss anything? Are those *_key columns in nodes public or private keys? Should I remove those?

Only public, so I would same username, hostname and potentially email is the relevant.
please test a copy so it actually still breaks when I am sent, as in, dont remove something that "fixes" the problem.

@kradalby commented on GitHub (Feb 6, 2025): > Is this enough to remove personal info or did I miss anything? Are those `*_key` columns in `nodes` public or private keys? Should I remove those? Only public, so I would same username, hostname and potentially email is the relevant. please test a copy so it actually still breaks when I am sent, as in, dont remove something that "fixes" the problem.

adam commented

@vbrandl commented on GitHub (Feb 6, 2025):

I sent you a mail with my broken DB and between which versions the bug appeared.

@vbrandl commented on GitHub (Feb 6, 2025): I sent you a mail with my broken DB and between which versions the bug appeared.

adam commented

@xcjs commented on GitHub (Feb 6, 2025):

I sent you a mail with my broken DB and between which versions the bug appeared.

Not the developer, but thanks for doing that! I hadn't gotten around to scrubbing mine yet, and I also wasn't exactly sure what version I was on.

@xcjs commented on GitHub (Feb 6, 2025): > I sent you a mail with my broken DB and between which versions the bug appeared. Not the developer, but thanks for doing that! I hadn't gotten around to scrubbing mine yet, and I also wasn't _exactly_ sure what version I was on.

adam commented