mirror of
https://github.com/netbox-community/netbox.git
synced 2026-01-11 21:10:29 +01:00
ProgrammingError exception when upgrading to v4.0 #9425
Closed
opened 2025-12-29 20:49:40 +01:00 by adam
·
24 comments
No Branch/Tag Specified
main
update-changelog-comments-docs
feature-removal-issue-type
20911-dropdown
20239-plugin-menu-classes-mutable-state
21097-graphql-id-lookups
feature
fix_module_substitution
20923-dcim-templates
20044-elevation-stuck-lightmode
feature-ip-prefix-link
v4.5-beta1-release
20068-import-moduletype-attrs
20766-fix-german-translation-code-literals
20378-del-script
7604-filter-modifiers-v3
circuit-swap
12318-case-insensitive-uniqueness
20637-improve-device-q-filter
20660-script-load
19724-graphql
20614-update-ruff
14884-script
02496-max-page
19720-macaddress-interface-generic-relation
19408-circuit-terminations-export-templates
20203-openapi-check
fix-19669-api-image-download
7604-filter-modifiers
19275-fixes-interface-bulk-edit
fix-17794-get_field_value_return_list
11507-show-aggregate-and-rir-on-api
9583-add_column_specific_search_field_to_tables
v4.5.0
v4.4.10
v4.4.9
v4.5.0-beta1
v4.4.8
v4.4.7
v4.4.6
v4.4.5
v4.4.4
v4.4.3
v4.4.2
v4.4.1
v4.4.0
v4.3.7
v4.4.0-beta1
v4.3.6
v4.3.5
v4.3.4
v4.3.3
v4.3.2
v4.3.1
v4.3.0
v4.2.9
v4.3.0-beta2
v4.2.8
v4.3.0-beta1
v4.2.7
v4.2.6
v4.2.5
v4.2.4
v4.2.3
v4.2.2
v4.2.1
v4.2.0
v4.1.11
v4.1.10
v4.1.9
v4.1.8
v4.2-beta1
v4.1.7
v4.1.6
v4.1.5
v4.1.4
v4.1.3
v4.1.2
v4.1.1
v4.1.0
v4.0.11
v4.0.10
v4.0.9
v4.1-beta1
v4.0.8
v4.0.7
v4.0.6
v4.0.5
v4.0.3
v4.0.2
v4.0.1
v4.0.0
v3.7.8
v3.7.7
v4.0-beta2
v3.7.6
v3.7.5
v4.0-beta1
v3.7.4
v3.7.3
v3.7.2
v3.7.1
v3.7.0
v3.6.9
v3.6.8
v3.6.7
v3.7-beta1
v3.6.6
v3.6.5
v3.6.4
v3.6.3
v3.6.2
v3.6.1
v3.6.0
v3.5.9
v3.6-beta2
v3.5.8
v3.6-beta1
v3.5.7
v3.5.6
v3.5.5
v3.5.4
v3.5.3
v3.5.2
v3.5.1
v3.5.0
v3.4.10
v3.4.9
v3.5-beta2
v3.4.8
v3.5-beta1
v3.4.7
v3.4.6
v3.4.5
v3.4.4
v3.4.3
v3.4.2
v3.4.1
v3.4.0
v3.3.10
v3.3.9
v3.4-beta1
v3.3.8
v3.3.7
v3.3.6
v3.3.5
v3.3.4
v3.3.3
v3.3.2
v3.3.1
v3.3.0
v3.2.9
v3.2.8
v3.3-beta2
v3.2.7
v3.3-beta1
v3.2.6
v3.2.5
v3.2.4
v3.2.3
v3.2.2
v3.2.1
v3.2.0
v3.1.11
v3.1.10
v3.2-beta2
v3.1.9
v3.2-beta1
v3.1.8
v3.1.7
v3.1.6
v3.1.5
v3.1.4
v3.1.3
v3.1.2
v3.1.1
v3.1.0
v3.0.12
v3.0.11
v3.0.10
v3.1-beta1
v3.0.9
v3.0.8
v3.0.7
v3.0.6
v3.0.5
v3.0.4
v3.0.3
v3.0.2
v3.0.1
v3.0.0
v2.11.12
v3.0-beta2
v2.11.11
v2.11.10
v3.0-beta1
v2.11.9
v2.11.8
v2.11.7
v2.11.6
v2.11.5
v2.11.4
v2.11.3
v2.11.2
v2.11.1
v2.11.0
v2.10.10
v2.10.9
v2.11-beta1
v2.10.8
v2.10.7
v2.10.6
v2.10.5
v2.10.4
v2.10.3
v2.10.2
v2.10.1
v2.10.0
v2.9.11
v2.10-beta2
v2.9.10
v2.10-beta1
v2.9.9
v2.9.8
v2.9.7
v2.9.6
v2.9.5
v2.9.4
v2.9.3
v2.9.2
v2.9.1
v2.9.0
v2.9-beta2
v2.8.9
v2.9-beta1
v2.8.8
v2.8.7
v2.8.6
v2.8.5
v2.8.4
v2.8.3
v2.8.2
v2.8.1
v2.8.0
v2.7.12
v2.7.11
v2.7.10
v2.7.9
v2.7.8
v2.7.7
v2.7.6
v2.7.5
v2.7.4
v2.7.3
v2.7.2
v2.7.1
v2.7.0
v2.6.12
v2.6.11
v2.6.10
v2.6.9
v2.7-beta1
Solcon-2020-01-06
v2.6.8
v2.6.7
v2.6.6
v2.6.5
v2.6.4
v2.6.3
v2.6.2
v2.6.1
v2.6.0
v2.5.13
v2.5.12
v2.6-beta1
v2.5.11
v2.5.10
v2.5.9
v2.5.8
v2.5.7
v2.5.6
v2.5.5
v2.5.4
v2.5.3
v2.5.2
v2.5.1
v2.5.0
v2.4.9
v2.5-beta2
v2.4.8
v2.5-beta1
v2.4.7
v2.4.6
v2.4.5
v2.4.4
v2.4.3
v2.4.2
v2.4.1
v2.4.0
v2.3.7
v2.4-beta1
v2.3.6
v2.3.5
v2.3.4
v2.3.3
v2.3.2
v2.3.1
v2.3.0
v2.2.10
v2.3-beta2
v2.2.9
v2.3-beta1
v2.2.8
v2.2.7
v2.2.6
v2.2.5
v2.2.4
v2.2.3
v2.2.2
v2.2.1
v2.2.0
v2.1.6
v2.2-beta2
v2.1.5
v2.2-beta1
v2.1.4
v2.1.3
v2.1.2
v2.1.1
v2.1.0
v2.0.10
v2.1-beta1
v2.0.9
v2.0.8
v2.0.7
v2.0.6
v2.0.5
v2.0.4
v2.0.3
v2.0.2
v2.0.1
v2.0.0
v2.0-beta3
v1.9.6
v1.9.5
v2.0-beta2
v1.9.4-r1
v1.9.3
v2.0-beta1
v1.9.2
v1.9.1
v1.9.0-r1
v1.8.4
v1.8.3
v1.8.2
v1.8.1
v1.8.0
v1.7.3
v1.7.2-r1
v1.7.1
v1.7.0
v1.6.3
v1.6.2-r1
v1.6.1-r1
1.6.1
v1.6.0
v1.5.2
v1.5.1
v1.5.0
v1.4.2
v1.4.1
v1.4.0
v1.3.2
v1.3.1
v1.3.0
v1.2.2
v1.2.1
v1.2.0
v1.1.0
v1.0.7-r1
v1.0.7
v1.0.6
v1.0.5
v1.0.4
v1.0.3-r1
v1.0.3
1.0.0
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
Mirrored from GitHub Pull Request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: starred/netbox#9425
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @opericgithub on GitHub (Apr 3, 2024).
Originally assigned to: @jeremystretch on GitHub.
Deployment Type
Self-hosted
NetBox Version
v3.7.4 and v3.6.9
Python Version
3.10
Steps to Reproduce
I'm not sure whether this is the right place, but please correct me if I should post this issue in some other place.
I tried to do the upgrade from v3.7.4 as well as from v3.6.9, to the new v4.0.x (x = either feature, or beta).
First I tried upgrading to feature branch few days ago, and now I tried upgrading to v4.0-beta1.
I was using the following commands:
Then I get the following error messages:
Am I doing something wrong? Is there anything else I should check for?
I am also afraid that I won't be able to do the upgrade to the final version v4.0 in few weeks.
Expected Behavior
I expect to upgrade to the new (beta) version, as I was able to do this in the past using the same commands.
Observed Behavior
I get the error:
django.db.utils.ProgrammingError: relation "extras_customfield_content_types_id_seq" does not existDetailed log messages are given above.
@jeremystretch commented on GitHub (Apr 3, 2024):
You cannot upgrade from v3.6 directly to v4.0: Per the documentation you'll need to first upgrade to v3.7 to ensure all migrations are applied prior to being squashed.
However, it should work fine when upgrading from v3.7.x. On a clean install of NetBox v3.7.4 (starting with an empty database), I have
extras_customfield_content_types_id_seq:And after switching to the v4.0-beta code base, all migrations apply successfully:
With your original v3.7.4 database in place, do you see
extras_customfield_content_types_id_seqdefined? (You can usemanage.py dbshellto enter the PostgreSQL command prompt and type\dto list all tables.)@opericgithub commented on GitHub (Apr 3, 2024):
With this:
I get the following:
@jeremystretch commented on GitHub (Apr 3, 2024):
You're missing at least one required library, which indicates that the installation/upgrade instructions were not followed. Please go back and ensure that your v3.7 deployment is fully functional before attempting to upgrade to v4.0.
@opericgithub commented on GitHub (Apr 3, 2024):
Thanks, I beleive I now installed all django packages.
Then I tried this:
Here are all the tables (347 rows):
@candlerb commented on GitHub (Apr 3, 2024):
You should not have to install anything.
You should allow Netbox to create a virtualenv and populate it; this is all done using the
./upgrade.shscript. If you have a problem with running upgrade.sh, please show it.If you find you have to install packages manually then there is definitely some other problem: you need to stop and fix that problem, before continuing.
EDIT: if you did this (as you say):
and you got that error, it means you probably hadn't activated the virtualenv before running the command. You definitely should not install django packages globally to workaround that issue.
Actually, there's a safer way to run it:
This will activate the virtualenv for you (regardless of whether it's currently activated or not)
@opericgithub commented on GitHub (Apr 3, 2024):
Yes, after I run this command, I can immediately list all tables with "\d" command, as Jeremy asked.
I can run upgrade.sh without problems:
Even after successful execution of this script on the stable version 3.7.4, I am not able to upgrade to beta.
I mean, first I do this:
sudo ./upgrade.shEverything is OK.
Then this:
sudo git pull origin v4.0-beta1then again this:
sudo ./upgrade.shand I get stucked with the message from the first post (relation "extras_customfield_content_types_id_seq" does not exist).
And really, when one looks at the tables I posted above, there is a missing row with sequence regarding the custom field content type.
As far as I understand there should be sequence for every table (vpntunnel, vpntunnel_id_seq, dcim_site, dcim_site_id_seq etc).
@tomasz-c commented on GitHub (Apr 4, 2024):
I have this problem too.
./upgrade.shon version 3.7.4 goes correctly, but there is noextras_customfield_content_types_id_seqin the databaseTrying to force migration to extras on version 3.7.4 reports no changes:
So I have the same error when trying to update to v4.0-beta1.
I have been updating the netbox for a long time, maybe this table was created only for new installations and the update omitted it?
@candlerb commented on GitHub (Apr 4, 2024):
It's broken for me too; some of the sequences and indexes in my system (and your system, and some others it seems) have different names to the ones that the migration scripts are expecting, and some constraints are missing.
I was able to make the migrations complete with some temporary patches:
https://github.com/netbox-community/netbox/discussions/15601#discussioncomment-8998869
But there's some underlying issue about how machines which have a long history of upgrades having different constraint names than those currently expected.
@opericgithub commented on GitHub (Apr 4, 2024):
Regarding the long history of upgrades, which other users have also discussed, it might be worth mentioning that we initially had version 2.8, then progressed subsequently to 2.11, followed by 3.0, and subsequently to each minor version (3.1, 3.2, etc.).
At one moment we did clean install of 3.x (don't remember the particular version) and did migrate all data from the same version from another server.
@PaulR282 commented on GitHub (Apr 4, 2024):
I'm running into this issue while trying to start a new container with the v4.0-beta1:
@jhammond-git commented on GitHub (Apr 4, 2024):
To add to the data set, I had the same issues as documented in https://github.com/netbox-community/netbox/discussions/15601#discussioncomment-8998869.
The VM I installed onto is a brand new ubuntu 22.04 machine with a fresh install of 3.7.4, but the database that was imported was upgraded over the years from 1.9.5, or at least, that's the first version I know we had on our production box.
@ingvaldlorentzen commented on GitHub (Apr 5, 2024):
I've got this exact issue, and I can't figure out where it is coming from. Wouldn't mind looking into it, but I have no idea where to start.
@v0tti commented on GitHub (Apr 5, 2024):
I am also unable to upgrade to v4.0-beta1 because
./upgrade.shlooks for reports and scripts that are not synced from the data-source yet, thus the files do not exist:@v0tti commented on GitHub (Apr 5, 2024):
After fixing this by manually creating the mentioned files I am also running into the
django.db.utils.ProgrammingError: relation "extras_customfield_content_types_id_seq" does not existproblem.@jeremystretch commented on GitHub (Apr 8, 2024):
I've hidden several comments in the thread which are not relevant to the reported behavior above. Please remember to open a separate bug report for unrelated issues.
@jeremystretch commented on GitHub (Apr 8, 2024):
Could someone who is experiencing this issue please post the list of tables which exist in their database prior to attempting the upgrade to v4.0? This can be done by running
manage.py dbshelland issuing the\dconsole command.)I'm specifically interested in determining how
extras_customfield_content_types_id_seqis named in older databases.@a084ed22 commented on GitHub (Apr 8, 2024):
I have a setup from the 2.11 era, gradually updated, currently on 3.7.5. I don't have a sequence for extras_customfield_content_types_id_seq.
@tobiasge commented on GitHub (Apr 8, 2024):
This is the list from our Netbox 3.7.5 (first installation on this DB was from 2.5 or 2.4):
I think it's named
extras_customfield_obj_type_id_seq:@jeremystretch commented on GitHub (Apr 8, 2024):
Yep, that seems to be the issue. This was from the old
obj_typefield on the CustomField model from before custom fields were completely reimplemented way back in NetBox v2.10. Unfortunately the sequence was never renamed accordingly for upgraded deployments.We can tweak the v4.0 migration to check for either sequence name and rename it accordingly.
@jeremystretch commented on GitHub (Apr 8, 2024):
@opericgithub @a084ed22 @tobiasge (and anyone else experiencing this issue), could you try checking out the
15605-migrate-old-sequencebranch and attempting the migration again? Should be fixed.@tobiasge commented on GitHub (Apr 8, 2024):
The content types are now working for out database.
But now I'm seeing a new error:
@a084ed22 commented on GitHub (Apr 8, 2024):
I can confirm that the fix allows the specific migration to work, however I'm later stuck on this:
@opericgithub commented on GitHub (Apr 8, 2024):
Sure. I tried checking out to that branch, upgrade completes without errors, but I stay on the same version (3.7.5).
Maybe I'm doing something wrong?
@jeremystretch commented on GitHub (Apr 10, 2024):
I'm going to merge PR #15661 and consider this fixed as the above feedback indicates that the
extras.0111_rename_content_typesmigration now completes successfully even for very old databases.If you continue to experience different errors when attempting to upgrade, please submit a new bug report with details.
@opericgithub Try just running the migration command (
manage.py migrate) after checking out the branch, instead of the whole upgrade script.