starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Batch remove of old jobs #9032

New Issue
Closed
opened 2025-12-29 20:44:25 +01:00 by adam · 4 comments
adam commented 2025-12-29 20:44:25 +01:00
Owner
Copy Link

Originally created by @doc-sheet on GitHub (Jan 3, 2024).

NetBox version

v3.7.0

Feature type

Change to existing functionality

Proposed functionality

Remove old jobs in small batches
Like Job.objects.filter(created__lt=cutoff)[:50] for example

Also I guess it's not very optimal to run this query twice - for count and delete queries.

Use case

When there are tens of millions old jobs count() and all() quries can't finish in some sensible time (even with statement_timeout=5s or so)
c78a792ccc/netbox/extras/management/commands/housekeeping.py (L75)

Database changes

No response

External dependencies

No response

Originally created by @doc-sheet on GitHub (Jan 3, 2024). ### NetBox version v3.7.0 ### Feature type Change to existing functionality ### Proposed functionality Remove old jobs in small batches Like `Job.objects.filter(created__lt=cutoff)[:50]` for example Also I guess it's not very optimal to run this query twice - for count and delete queries. ### Use case When there are tens of millions old jobs count() and all() quries can't finish in some sensible time (even with statement_timeout=5s or so) https://github.com/netbox-community/netbox/blob/c78a792cccfdbe6f373c0d474b1620e56e5f9cf8/netbox/extras/management/commands/housekeeping.py#L75 ### Database changes _No response_ ### External dependencies _No response_
adam added the type: featurestatus: needs ownerpending closurecomplexity: low labels 2025-12-29 20:44:25 +01:00
adam closed this issue 2025-12-29 20:44:25 +01:00
adam commented 2025-12-29 20:44:25 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Mar 9, 2024):

Breaking the delete operation into smaller chunks isn't going to yield any performance gains. However, we could use Django's _raw_delete() method to bypass the signal handlers, which would speed up the bulk deletion considerably and should be safe for this specific use case.

@jeremystretch commented on GitHub (Mar 9, 2024): Breaking the delete operation into smaller chunks isn't going to yield any performance gains. However, we could use Django's `_raw_delete()` method to bypass the signal handlers, which _would_ speed up the bulk deletion considerably and should be safe for this specific use case.
adam commented 2025-12-29 20:44:26 +01:00
Author
Owner
Copy Link

@doc-sheet commented on GitHub (Mar 9, 2024):

In my case it was a trade off between "cleanup don't work" and "cleanup lasts for hours".

Also i kinda don't understand why those millions of job records existed at all (some kind of glitch in scheduler I guess), but it's a different story.

@doc-sheet commented on GitHub (Mar 9, 2024): In my case it was a trade off between "cleanup don't work" and "cleanup lasts for hours". Also i kinda don't understand why those millions of job records existed at all (some kind of glitch in scheduler I guess), but it's a different story.
adam commented 2025-12-29 20:44:26 +01:00
Author
Owner
Copy Link

@github-actions[bot] commented on GitHub (Aug 21, 2024):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Do not attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our contributing guide.

@github-actions[bot] commented on GitHub (Aug 21, 2024): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. **Do not** attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our [contributing guide](https://github.com/netbox-community/netbox/blob/develop/CONTRIBUTING.md).
adam commented 2025-12-29 20:44:26 +01:00
Author
Owner
Copy Link

@github-actions[bot] commented on GitHub (Sep 20, 2024):

This issue has been automatically closed due to lack of activity. In an effort to reduce noise, please do not comment any further. Note that the core maintainers may elect to reopen this issue at a later date if deemed necessary.

@github-actions[bot] commented on GitHub (Sep 20, 2024): This issue has been automatically closed due to lack of activity. In an effort to reduce noise, please do not comment any further. Note that the core maintainers may elect to reopen this issue at a later date if deemed necessary.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label complexity: low pending closure status: needs owner type: feature
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#9032
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?