starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Repeated ContentType SQL queries in enqueue_webhooks slow down bulk-delete of objects via UI #4245

New Issue
Closed
opened 2025-12-29 18:34:12 +01:00 by adam · 1 comment
adam commented 2025-12-29 18:34:12 +01:00
Owner
Copy Link

Originally created by @glennmatthews on GitHub (Nov 4, 2020).

Originally assigned to: @jeremystretch on GitHub.

Environment

  • Python version: 3.7
  • NetBox version: 2.8.9

Steps to Reproduce

  1. Via the UI, bulk-delete several thousand records (any object type; I observed this with custom models in a plugin but I do not believe that this is a requirement) in a single request
  2. Observe how long the request takes to complete and return a new page.

Expected Behavior

Request should complete within a couple of seconds.

Observed Behavior

Deleting 2000 records takes approximately 28 seconds in my testing.

Analysis

It appears that enqueue_webhooks() is called for each record being deleted, and each time enqueue_webhooks() is called, it makes an SQL query for ContentType objects:

SELECT "django_content_type"."id", "django_content_type"."app_label", "django_content_type"."model"
FROM "django_content_type"
WHERE (
("django_content_type"."app_label" = 'dcim' AND "django_content_type"."model" IN ('consoleport', 'consoleserverport', 'powerport', 'poweroutlet', 'interface', 'frontport', 'rearport', 'devicebay', 'inventoryitem', 'region', 'site', 'rack', 'rackreservation', 'manufacturer', 'devicetype', 'device', 'virtualchassis', 'powerpanel', 'powerfeed', 'cable'))
OR ("django_content_type"."app_label" = 'circuits' AND "django_content_type"."model" IN ('provider', 'circuit'))
OR ("django_content_type"."app_label" = 'virtualization' AND "django_content_type"."model" IN ('cluster', 'virtualmachine')) 
OR ("django_content_type"."app_label" = 'ipam' AND "django_content_type"."model" IN ('vrf', 'aggregate', 'prefix', 'ipaddress', 'vlan', 'service'))
OR ("django_content_type"."app_label" = 'secrets' AND "django_content_type"."model" IN ('secret'))
OR ("django_content_type"."app_label" = 'tenancy' AND "django_content_type"."model" IN ('tenant'))
); args=('dcim', 'consoleport', 'consoleserverport', 'powerport', 'poweroutlet', 'interface', 'frontport', 'rearport', 'devicebay', 'inventoryitem', 'region', 'site', 'rack', 'rackreservation', 'manufacturer', 'devicetype', 'device', 'virtualchassis', 'powerpanel', 'powerfeed', 'cable', 'circuits', 'provider', 'circuit', 'virtualization', 'cluster', 'virtualmachine', 'ipam', 'vrf', 'aggregate', 'prefix', 'ipaddress', 'vlan', 'service', 'secrets', 'secret', 'tenancy', 'tenant')

As a result, deleting 2000 records results in 2000 SQL queries, adding substantially to the request runtime. This appears to be due to the code changes made in #3932 (specifically here). It should be possible to cache the results of the ContentType lookup in some fashion rather than re-executing the same query over and over again, which should resolve this performance issue.

Originally created by @glennmatthews on GitHub (Nov 4, 2020). Originally assigned to: @jeremystretch on GitHub. <!-- NOTE: IF YOUR ISSUE DOES NOT FOLLOW THIS TEMPLATE, IT WILL BE CLOSED. This form is only for reproducible bugs. If you need assistance with NetBox installation, or if you have a general question, DO NOT open an issue. Instead, post to our mailing list: https://groups.google.com/g/netbox-discuss Please describe the environment in which you are running NetBox. Be sure that you are running an unmodified instance of the latest stable release before submitting a bug report, and that any plugins have been disabled. --> ### Environment * Python version: 3.7 * NetBox version: 2.8.9 <!-- Describe in detail the exact steps that someone else can take to reproduce this bug using the current stable release of NetBox. Begin with the creation of any necessary database objects and call out every operation being performed explicitly. If reporting a bug in the REST API, be sure to reconstruct the raw HTTP request(s) being made: Don't rely on a client library such as pynetbox. --> ### Steps to Reproduce 1. Via the UI, bulk-delete several thousand records (any object type; I observed this with custom models in a plugin but I do not believe that this is a requirement) in a single request 2. Observe how long the request takes to complete and return a new page. <!-- What did you expect to happen? --> ### Expected Behavior Request should complete within a couple of seconds. <!-- What happened instead? --> ### Observed Behavior Deleting 2000 records takes approximately 28 seconds in my testing. ### Analysis It appears that `enqueue_webhooks()` is called for each record being deleted, and each time `enqueue_webhooks()` is called, it makes an SQL query for ContentType objects: ``` SELECT "django_content_type"."id", "django_content_type"."app_label", "django_content_type"."model" FROM "django_content_type" WHERE ( ("django_content_type"."app_label" = 'dcim' AND "django_content_type"."model" IN ('consoleport', 'consoleserverport', 'powerport', 'poweroutlet', 'interface', 'frontport', 'rearport', 'devicebay', 'inventoryitem', 'region', 'site', 'rack', 'rackreservation', 'manufacturer', 'devicetype', 'device', 'virtualchassis', 'powerpanel', 'powerfeed', 'cable')) OR ("django_content_type"."app_label" = 'circuits' AND "django_content_type"."model" IN ('provider', 'circuit')) OR ("django_content_type"."app_label" = 'virtualization' AND "django_content_type"."model" IN ('cluster', 'virtualmachine')) OR ("django_content_type"."app_label" = 'ipam' AND "django_content_type"."model" IN ('vrf', 'aggregate', 'prefix', 'ipaddress', 'vlan', 'service')) OR ("django_content_type"."app_label" = 'secrets' AND "django_content_type"."model" IN ('secret')) OR ("django_content_type"."app_label" = 'tenancy' AND "django_content_type"."model" IN ('tenant')) ); args=('dcim', 'consoleport', 'consoleserverport', 'powerport', 'poweroutlet', 'interface', 'frontport', 'rearport', 'devicebay', 'inventoryitem', 'region', 'site', 'rack', 'rackreservation', 'manufacturer', 'devicetype', 'device', 'virtualchassis', 'powerpanel', 'powerfeed', 'cable', 'circuits', 'provider', 'circuit', 'virtualization', 'cluster', 'virtualmachine', 'ipam', 'vrf', 'aggregate', 'prefix', 'ipaddress', 'vlan', 'service', 'secrets', 'secret', 'tenancy', 'tenant') ``` As a result, deleting 2000 records results in 2000 SQL queries, adding substantially to the request runtime. This appears to be due to the code changes made in #3932 (specifically [here](https://github.com/netbox-community/netbox/pull/3932/files#diff-7908dbffe3012a9b0bec4269853a470b009c95ec4c8a2a8f95e22f87810dabb0L14)). It should be possible to cache the results of the ContentType lookup in some fashion rather than re-executing the same query over and over again, which should resolve this performance issue.
adam added the type: bugstatus: accepted labels 2025-12-29 18:34:12 +01:00
adam closed this issue 2025-12-29 18:34:12 +01:00
adam commented 2025-12-29 18:34:13 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Nov 5, 2020):

This came up during work on v2.10. I believe the root cause is the way FeatureQuery retrieves ContentTypes from the database on each call, bypassing the cached copies. We can probably bypass the ContentType lookup entirely and just match on app_label and model for the content type of the current instance.

Request should complete within a couple of seconds.

That's probably not a fair expectation when deleting thousands of objects, considering that at a minimum NetBox is creating a changelog record and executing an individual SQL query for each item as it is deleted. Improving FeatureQuery performance should reduce the time it takes, but I wouldn't expect it to remove a huge portion of the delay, which is attributable to other functions.

@jeremystretch commented on GitHub (Nov 5, 2020): This came up during work on v2.10. I believe the root cause is the way FeatureQuery retrieves ContentTypes from the database on each call, bypassing the cached copies. We can probably bypass the ContentType lookup entirely and just match on `app_label` and `model` for the content type of the current instance. > Request should complete within a couple of seconds. That's probably not a fair expectation when deleting thousands of objects, considering that at a _minimum_ NetBox is creating a changelog record and executing an individual SQL query for each item as it is deleted. Improving FeatureQuery performance should reduce the time it takes, but I wouldn't expect it to remove a huge portion of the delay, which is attributable to other functions.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label status: accepted type: bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#4245
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?