Custom scripts are significantly slower after upgrading from 3.3.7 to 3.4.2 #7454

New Issue

adam · 2025-12-29T20:23:38+01:00

adam commented

2025-12-29 20:23:38 +01:00

Originally created by @desnoe on GitHub (Jan 4, 2023).

NetBox version

v3.4.2

Python version

3.10

Steps to Reproduce

Install the below custom script that creates 100 devices on NetBox 3.3.7. On my machine, I get this execution message: Initiated: 2023-01-04 16:36 Duration: 0 minutes, 4.17 seconds Completed
Use the same script on NetBox 3.4.2. On my same machine, I now get: Started: 2023-01-04 16:37 Duration: 0 minutes, 59.31 seconds Completed

class Perf(Script):
    class Meta:
        name = "Device creation performance test"
        description = "Device creation performance test"

    def run(self, data, commit):
        for i in range(100):
            netbox_device = Device(
                name="device" + str(i),
                device_type=DeviceType.objects.first(),
                device_role=DeviceRole.objects.first(),
                site=Site.objects.first(),
            )
            netbox_device.save()

NB: this script is not optimised, but this is not the point. The point is that the same script have inconsistent execution times between versions.

Expected Behavior

The execution time should remain consistent between this 2 versions. Some dozens of percent of increase could still be acceptable.

Observed Behavior

The execution time for the reference script is ~14x times slower on version 3.4.2 compared to 3.3.7.

Originally created by @desnoe on GitHub (Jan 4, 2023). ### NetBox version v3.4.2 ### Python version 3.10 ### Steps to Reproduce 1. Install the below custom script that creates 100 devices on NetBox 3.3.7. On my machine, I get this execution message: Initiated: 2023-01-04 16:36 Duration: 0 minutes, 4.17 seconds Completed 2. Use the same script on NetBox 3.4.2. On my same machine, I now get: Started: 2023-01-04 16:37 Duration: 0 minutes, 59.31 seconds Completed ``` class Perf(Script): class Meta: name = "Device creation performance test" description = "Device creation performance test" def run(self, data, commit): for i in range(100): netbox_device = Device( name="device" + str(i), device_type=DeviceType.objects.first(), device_role=DeviceRole.objects.first(), site=Site.objects.first(), ) netbox_device.save() ``` NB: this script is not optimised, but this is not the point. The point is that the same script have inconsistent execution times between versions. ### Expected Behavior The execution time should remain consistent between this 2 versions. Some dozens of percent of increase could still be acceptable. ### Observed Behavior The execution time for the reference script is ~14x times slower on version 3.4.2 compared to 3.3.7.

adam closed this issue

2025-12-29 20:23:38 +01:00

adam commented

2025-12-29 20:23:39 +01:00

@kkthxbye-code commented on GitHub (Jan 4, 2023):

Hi, this is not a bug and it has nothing to do with scripts. The worse performance specifically relates to creating devices after we fixed an uninteded bug where changelog entries were not created for device components.

https://github.com/netbox-community/netbox/issues/10694
https://github.com/netbox-community/netbox/pull/10900

Before when instantiating a device, components were created with bulk_create which doesn't send any signals, thus skipping both changelog generation and search indexing. The new approach still bulk creates most of the components, but sends the post_save signal manually after creation. As a result device instantiation is vastly slower than it were before. The superior performance was only possible because the old approach was incorrect.

There's an open issue considering moving both changelog creation and search indexing to background tasks: https://github.com/netbox-community/netbox/issues/11202

Do note that this would not eliminate the processing, it would just be moved to another process. The tradeof is a small delay before the changelog for the objects would be created and search indexing would finish.

@kkthxbye-code commented on GitHub (Jan 4, 2023): Hi, this is not a bug and it has nothing to do with scripts. The worse performance specifically relates to creating devices after we fixed an uninteded bug where changelog entries were not created for device components. https://github.com/netbox-community/netbox/issues/10694 https://github.com/netbox-community/netbox/pull/10900 Before when instantiating a device, components were created with bulk_create which doesn't send any signals, thus skipping both changelog generation and search indexing. The new approach still bulk creates most of the components, but sends the `post_save` signal manually after creation. As a result device instantiation is vastly slower than it were before. The superior performance was only possible because the old approach was incorrect. There's an open issue considering moving both changelog creation and search indexing to background tasks: https://github.com/netbox-community/netbox/issues/11202 Do note that this would not eliminate the processing, it would just be moved to another process. The tradeof is a small delay before the changelog for the objects would be created and search indexing would finish.

Sign in to join this conversation.

Branches Tags

main

update-changelog-comments-docs

feature-removal-issue-type

20911-dropdown

20239-plugin-menu-classes-mutable-state

21097-graphql-id-lookups

feature

fix_module_substitution

20923-dcim-templates

20044-elevation-stuck-lightmode

feature-ip-prefix-link

v4.5-beta1-release

20068-import-moduletype-attrs

20766-fix-german-translation-code-literals

20378-del-script

7604-filter-modifiers-v3

circuit-swap

12318-case-insensitive-uniqueness

20637-improve-device-q-filter

20660-script-load

19724-graphql

20614-update-ruff

14884-script

02496-max-page

19720-macaddress-interface-generic-relation

19408-circuit-terminations-export-templates

20203-openapi-check

fix-19669-api-image-download

7604-filter-modifiers

19275-fixes-interface-bulk-edit

fix-17794-get_field_value_return_list

11507-show-aggregate-and-rir-on-api

9583-add_column_specific_search_field_to_tables

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/netbox#7454