Reset database migrations for v3.0 release #4939

New Issue

adam · 2025-12-29T19:22:28+01:00

adam commented

2025-12-29 19:22:28 +01:00

Originally created by @jeremystretch on GitHub (May 21, 2021).

Originally assigned to: @DanSheps on GitHub.

Proposed Changes

We should take the upcoming v3.0 release as an opportunity to reset the roughly five years' worth of database migrations we've accumulated since the initial release. I'm not sure yet exactly what the process will entail.

Justification

Simplifies the deployment process for new users
Reduces risk of hidden technical debt in older migrations
Reduces CI testing time

Originally created by @jeremystretch on GitHub (May 21, 2021). Originally assigned to: @DanSheps on GitHub. ### Proposed Changes We should take the upcoming v3.0 release as an opportunity to reset the roughly five years' worth of database migrations we've accumulated since the initial release. I'm not sure yet exactly what the process will entail. ### Justification - Simplifies the deployment process for new users - Reduces risk of hidden technical debt in older migrations - Reduces CI testing time

adam added the status: accepted type: housekeeping labels 2025-12-29 19:22:28 +01:00

adam closed this issue

2025-12-29 19:22:28 +01:00

adam commented

2025-12-29 19:22:28 +01:00

@jeremystretch commented on GitHub (Jul 9, 2021):

@DanSheps and I have been working on this, and I'm confident in the approach we've arrived at. What we've done is remove and rebuild all database migrations created prior to v2.11.0. Individual migrations from v2.11.0 and later releases have been retained. This means that users will be required to upgrade to a v2.11.x release before upgrading to a v3.0 release.

To test the integrity of the squashed migrations, I compared both the resulting table schemas and the end state of the migrations table. When comparing new installations using the original and squashed migrations, the only differences found in the resulting database schema were the names of some sequences, a cosmetic difference. Further, the only differences found in the resulting migration tables were the addition of the squashed migrations.

I have also tested upgrading a v2.11.0 installation using the squashed migrations with similar results.

As a result of squashing the migrations, we have reduced the time needed to run a complete migration on my local machine from ~84 seconds to ~21 seconds. 🎉 This should help significantly reduce the time needed to run our CI tests.

Testing Methodology

Migration timing:

time ./manage.py migrate

Schema export:

sudo -u postgres psql netbox -c "SELECT table_name, column_name, data_type, column_default, is_nullable, character_maximum_length, numeric_precision FROM information_schema.columns WHERE table_schema = 'public' ORDER BY table_name ASC, column_name ASC"

(Note that we're not using pg_dump for this as manner in which it orders table columns makes the output unsuitable for generating a diff.)

Migrations table export:

sudo -u postgres psql netbox -c "select app, name from django_migrations order by app, name"

@jeremystretch commented on GitHub (Jul 9, 2021): @DanSheps and I have been working on this, and I'm confident in the approach we've arrived at. What we've done is remove and rebuild all database migrations created prior to v2.11.0. Individual migrations from v2.11.0 and later releases have been retained. This means that users will be required to upgrade to a v2.11.x release **before** upgrading to a v3.0 release. To test the integrity of the squashed migrations, I compared both the resulting table schemas and the end state of the migrations table. When comparing new installations using the original and squashed migrations, the only differences found in the resulting database schema were the names of some sequences, a cosmetic difference. Further, the only differences found in the resulting migration tables were the addition of the squashed migrations. I have also tested upgrading a v2.11.0 installation using the squashed migrations with similar results. As a result of squashing the migrations, we have reduced the time needed to run a complete migration on my local machine from ~84 seconds to ~21 seconds. :tada: This should help significantly reduce the time needed to run our CI tests. #### Testing Methodology Migration timing: ``` time ./manage.py migrate ``` Schema export: ``` sudo -u postgres psql netbox -c "SELECT table_name, column_name, data_type, column_default, is_nullable, character_maximum_length, numeric_precision FROM information_schema.columns WHERE table_schema = 'public' ORDER BY table_name ASC, column_name ASC" ``` (Note that we're not using `pg_dump` for this as manner in which it orders table columns makes the output unsuitable for generating a diff.) Migrations table export: ``` sudo -u postgres psql netbox -c "select app, name from django_migrations order by app, name" ```

Sign in to join this conversation.

Branches Tags

main

21102-fix-graphiql-explorer

update-changelog-comments-docs

20911-dropdown

20239-plugin-menu-classes-mutable-state

21097-graphql-id-lookups

feature

fix_module_substitution

20923-dcim-templates

20044-elevation-stuck-lightmode

feature-ip-prefix-link

v4.5-beta1-release

20068-import-moduletype-attrs

20766-fix-german-translation-code-literals

20378-del-script

7604-filter-modifiers-v3

circuit-swap

12318-case-insensitive-uniqueness

20637-improve-device-q-filter

20660-script-load

19724-graphql

20614-update-ruff

14884-script

02496-max-page

19720-macaddress-interface-generic-relation

19408-circuit-terminations-export-templates

20203-openapi-check

fix-19669-api-image-download

7604-filter-modifiers

19275-fixes-interface-bulk-edit

fix-17794-get_field_value_return_list

11507-show-aggregate-and-rir-on-api

9583-add_column_specific_search_field_to_tables

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/netbox#4939