starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Eliminate usage of bulk_create() and bulk_update() methods outside of tests and migrations #7128

New Issue
Closed
opened 2025-12-29 20:19:31 +01:00 by adam · 3 comments
adam commented 2025-12-29 20:19:31 +01:00
Owner
Copy Link

Originally created by @jeremystretch on GitHub (Oct 19, 2022).

Originally assigned to: @jeremystretch on GitHub.

Proposed Changes

While we've generally avoided the use of bulk_create() and bulk_update() for the bulk creation and manipulation of objects due to the caveats associated with their use (namely not triggering the post_save signal), they are still used in a few areas within NetBox. The goal of this issue is to eliminate any usage of these methods, excluding within tests and migrations where specifically needed to ensure optimal performance.

Justification

As calling bulk_create() or bulk_update() does not generate a post_save signal for each of the objects affected, this escapes change logging and (in NetBox v3.4) search indexing. Although calling save() on each instance individual will impose a performance penalty, it ensures that each instance is subject to the appropriate background workflows. (This was initially called out here by @kkthxbye-code.)

Originally created by @jeremystretch on GitHub (Oct 19, 2022). Originally assigned to: @jeremystretch on GitHub. ### Proposed Changes While we've generally avoided the use of `bulk_create()` and `bulk_update()` for the bulk creation and manipulation of objects due to the [caveats associated with their use](https://docs.djangoproject.com/en/4.1/ref/models/querysets/#bulk-create) (namely not triggering the `post_save` signal), they are still used in a few areas within NetBox. The goal of this issue is to eliminate any usage of these methods, excluding within tests and migrations where specifically needed to ensure optimal performance. ### Justification As calling `bulk_create()` or `bulk_update()` does not generate a `post_save` signal for each of the objects affected, this escapes change logging and (in NetBox v3.4) search indexing. Although calling `save()` on each instance individual will impose a performance penalty, it ensures that each instance is subject to the appropriate background workflows. (This was initially called out [here](https://github.com/netbox-community/netbox/pull/10676#issuecomment-1282665864) by @kkthxbye-code.)
adam added the status: acceptedtype: housekeeping labels 2025-12-29 20:19:31 +01:00
adam closed this issue 2025-12-29 20:19:31 +01:00
adam commented 2025-12-29 20:19:31 +01:00
Author
Owner
Copy Link

@arthanson commented on GitHub (Oct 27, 2022):

bulk_create used in:

  • netbox/dcim/models/devices.py
  • netbox/search/backends.py

bulk_update used in:

  • netbox/dcim/models/devices.py
  • netbox/extras/models/customfields.py
  • netbox/ipam/signals.py
  • netbox/ipam/utils.py
@arthanson commented on GitHub (Oct 27, 2022): bulk_create used in: - netbox/dcim/models/devices.py - netbox/search/backends.py bulk_update used in: - netbox/dcim/models/devices.py - netbox/extras/models/customfields.py - netbox/ipam/signals.py - netbox/ipam/utils.py
adam commented 2025-12-29 20:19:31 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Nov 11, 2022):

The goal of this issue is to eliminate any usage of these methods

Rather than eliminating their usage, which would incur significant performance penalties when performing certain operations, we could instead manually trigger the post_save signal for each object after the bulk operation is run. I'm a little hesitant to commit to this approach as it alters Django's default behavior, however it might be reasonable for our purposes. I'm going to put in a PR to test.

@jeremystretch commented on GitHub (Nov 11, 2022): > The goal of this issue is to eliminate any usage of these methods Rather than eliminating their usage, which would incur significant performance penalties when performing certain operations, we could instead manually trigger the `post_save` signal for each object after the bulk operation is run. I'm a little hesitant to commit to this approach as it alters Django's default behavior, however it might be reasonable for our purposes. I'm going to put in a PR to test.
adam commented 2025-12-29 20:19:32 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Nov 14, 2022):

The hybrid approach I mentioned above seems to save a substantial amount of time compared to completely removing the bulk calls (despite still being far slower than the current implementation, which does not support change logging). I've merged PR #10900 using this approach and will test during the v3.4 beta release. Should any serious issues with this approach surface, we'll revert to the approach in PR #10780.

@jeremystretch commented on GitHub (Nov 14, 2022): The hybrid approach I mentioned above seems to save a substantial amount of time compared to completely removing the bulk calls (despite still being far slower than the current implementation, which does not support change logging). I've merged PR #10900 using this approach and will test during the v3.4 beta release. Should any serious issues with this approach surface, we'll revert to the approach in PR #10780.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label status: accepted type: housekeeping
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#7128
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?