starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Retain old JobResults #6253

New Issue
Closed
opened 2025-12-29 19:38:33 +01:00 by adam · 11 comments
adam commented 2025-12-29 19:38:33 +01:00
Owner
Copy Link

Originally created by @kkthxbye-code on GitHub (Mar 24, 2022).

Originally assigned to: @kkthxbye-code on GitHub.

NetBox version

v.3.1.9

Feature type

Change to existing functionality

Proposed functionality

Currently all old JobResults are deleted when a new JobResult is in a terminal state. I propose that old JobResults are preserved.

Use case

Deleting old JobResults make it hard to use for automation through the API. Imagine a case where two runs of the same script is triggered via. the API. The API returns an id for the JobResult that can be used to query for the result. The RQ worker will run the scripts one after the other. When the first script finishes, you have the runtime of the second script to get the result. If you do not retrieve the result before the second script finishes, the JobResult is deleted and impossible to retrieve.

Also contributing to the issue is that JobResults are not webhook enabled, which could provide a workaround in some cases. But that's a different feature request.

Database changes

None

External dependencies

No response

Originally created by @kkthxbye-code on GitHub (Mar 24, 2022). Originally assigned to: @kkthxbye-code on GitHub. ### NetBox version v.3.1.9 ### Feature type Change to existing functionality ### Proposed functionality Currently all old JobResults are deleted when a new JobResult is in a terminal state. I propose that old JobResults are preserved. ### Use case Deleting old JobResults make it hard to use for automation through the API. Imagine a case where two runs of the same script is triggered via. the API. The API returns an id for the JobResult that can be used to query for the result. The RQ worker will run the scripts one after the other. When the first script finishes, you have the runtime of the second script to get the result. If you do not retrieve the result before the second script finishes, the JobResult is deleted and impossible to retrieve. Also contributing to the issue is that JobResults are not webhook enabled, which could provide a workaround in some cases. But that's a different feature request. ### Database changes None ### External dependencies _No response_
adam added the status: acceptedtype: feature labels 2025-12-29 19:38:33 +01:00
adam closed this issue 2025-12-29 19:38:33 +01:00
adam commented 2025-12-29 19:38:33 +01:00
Author
Owner
Copy Link

@kkthxbye-code commented on GitHub (Mar 24, 2022):

I have a working implementation shortly as a PR.

@kkthxbye-code commented on GitHub (Mar 24, 2022): I have a working implementation shortly as a PR.
adam commented 2025-12-29 19:38:34 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Mar 24, 2022):

I think this makes sense. The first question that comes to mind is: How will we handle the eventual cleanup of old JobResult instances? Presumably via the housekeeping management command? We'd also need to introduce a configuration parameter to dictate the expiration age similar to CHANGELOG_RETETION.

Also contributing to the issue is that JobResults are not webhook enabled, which could provide a workaround in some cases. But that's a different feature request.

#8958

@jeremystretch commented on GitHub (Mar 24, 2022): I think this makes sense. The first question that comes to mind is: How will we handle the eventual cleanup of old JobResult instances? Presumably via the [housekeeping management command](https://netbox.readthedocs.io/en/stable/administration/housekeeping/)? We'd also need to introduce a configuration parameter to dictate the expiration age similar to [`CHANGELOG_RETETION`](https://netbox.readthedocs.io/en/stable/configuration/dynamic-settings/#changelog_retention). > Also contributing to the issue is that JobResults are not webhook enabled, which could provide a workaround in some cases. But that's a different feature request. #8958
adam commented 2025-12-29 19:38:34 +01:00
Author
Owner
Copy Link

@kkthxbye-code commented on GitHub (Mar 24, 2022):

I actually did consider adding it to the housekeeping command, but figured that as JobResults, opposed to changelog entries, are deletable in the admin panel, it wouldn't be nessecary.

I can add it if you think it would be the best way to handle it. How about default retention? Unlimited would be my preference , but I'm not sure how much data people are actually outputting from their scripts.

@kkthxbye-code commented on GitHub (Mar 24, 2022): I actually did consider adding it to the housekeeping command, but figured that as JobResults, opposed to changelog entries, are deletable in the admin panel, it wouldn't be nessecary. I can add it if you think it would be the best way to handle it. How about default retention? Unlimited would be my preference , but I'm not sure how much data people are actually outputting from their scripts.
adam commented 2025-12-29 19:38:34 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Mar 24, 2022):

IMO unlimited (None) is a reasonable and safe default.

@jeremystretch commented on GitHub (Mar 24, 2022): IMO unlimited (`None`) is a reasonable and safe default.
adam commented 2025-12-29 19:38:34 +01:00
Author
Owner
Copy Link

@kkthxbye-code commented on GitHub (Mar 25, 2022):

I added a JOBRESULT_RETENTION option, added the functionality to the housekeeping script and updated the documentation. Let me know if I missed anything.

@kkthxbye-code commented on GitHub (Mar 25, 2022): I added a JOBRESULT_RETENTION option, added the functionality to the housekeeping script and updated the documentation. Let me know if I missed anything.
adam commented 2025-12-29 19:38:34 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Mar 25, 2022):

Since this is a fairly substantial change to the existing behavior, I'd like to tag this for v3.2 (due to be released the week of April 4th).

@kkthxbye-code would you mind rebasing your PR off the feature branch? I don't think you'll run into much conflict, but please let me know if you need any help.

@jeremystretch commented on GitHub (Mar 25, 2022): Since this is a fairly substantial change to the existing behavior, I'd like to tag this for v3.2 (due to be released the week of April 4th). @kkthxbye-code would you mind rebasing your PR off the `feature` branch? I don't think you'll run into much conflict, but please let me know if you need any help.
adam commented 2025-12-29 19:38:35 +01:00
Author
Owner
Copy Link

@kkthxbye-code commented on GitHub (Mar 25, 2022):

@jeremystretch - I haven't rebased to a different parent before, could you maybe give a little hint. My google-fu comes short here, as there seems to be a bunch of different ways to get the same result.

I've tried git rebase --onto origin/feature origin/develop and then git add netbox/extras/management/commands/runscript.py as that file was shown as "deleted by us:" for some reason. Then force pushing with git push origin HEAD:save-job-results --force. It looked kinda correct, but when I viewed the pull request on github it indicated merge conflicts and it was still trying to merge into develop.

I'm kinda lost I guess :)

@kkthxbye-code commented on GitHub (Mar 25, 2022): @jeremystretch - I haven't rebased to a different parent before, could you maybe give a little hint. My google-fu comes short here, as there seems to be a bunch of different ways to get the same result. I've tried `git rebase --onto origin/feature origin/develop` and then git add netbox/extras/management/commands/runscript.py as that file was shown as "deleted by us:" for some reason. Then force pushing with `git push origin HEAD:save-job-results --force`. It looked kinda correct, but when I viewed the pull request on github it indicated merge conflicts and it was still trying to merge into develop. I'm kinda lost I guess :)
adam commented 2025-12-29 19:38:35 +01:00
Author
Owner
Copy Link

@DanSheps commented on GitHub (Apr 8, 2022):

Should we target this for v3.3 now or release it as a minor in v3.2?

@DanSheps commented on GitHub (Apr 8, 2022): Should we target this for v3.3 now or release it as a minor in v3.2?
adam commented 2025-12-29 19:38:35 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Apr 8, 2022):

I think it's reasonable to implement this in v3.2, given that:

  • There are no changes to the reports or scripts APIs
  • The change will be largely transparent to users
  • I can't think of any way this would disrupt existing workflows

However, perhaps we should alter the default JOBRESULT_RETENTION value from 0 (indefinite) to 90 days (matching the default for CHANGELOG_RETENTION). This will ensure that even an oblivious user doesn't end up with an enormous backlog of job results, provided they are running the housekeeping daemon as advised in the documentation.

@jeremystretch commented on GitHub (Apr 8, 2022): I think it's reasonable to implement this in v3.2, given that: - There are no changes to the reports or scripts APIs - The change will be largely transparent to users - I can't think of any way this would disrupt existing workflows However, perhaps we should alter the default `JOBRESULT_RETENTION` value from 0 (indefinite) to 90 days (matching the default for `CHANGELOG_RETENTION`). This will ensure that even an oblivious user doesn't end up with an enormous backlog of job results, provided they are running the housekeeping daemon as advised in the documentation.
adam commented 2025-12-29 19:38:35 +01:00
Author
Owner
Copy Link

@kkthxbye-code commented on GitHub (Apr 8, 2022):

@jeremystretch - I changed the default to 90. While I prefer unlimited by default, I can see the rationale for 90.

When merged I'll start looking at scheduling jobs. I also think it might make sense to move job results out of the admin ui at some point, espcially if scheduled tasks are on the table.

Also while I tested this pretty extensively, it would ease my mind if one of you guys would just do a small verification to see if everything functions as intended.

@kkthxbye-code commented on GitHub (Apr 8, 2022): @jeremystretch - I changed the default to 90. While I prefer unlimited by default, I can see the rationale for 90. When merged I'll start looking at scheduling jobs. I also think it might make sense to move job results out of the admin ui at some point, espcially if scheduled tasks are on the table. Also while I tested this pretty extensively, it would ease my mind if one of you guys would just do a small verification to see if everything functions as intended.
adam commented 2025-12-29 19:38:35 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Apr 8, 2022):

I changed the default to 90. While I prefer unlimited by default, I can see the rationale for 90.

Thanks! It's all about trying to protect the user ultimately.

I also think it might make sense to move job results out of the admin ui at some point, espcially if scheduled tasks are on the table.

Agreed; we should implement a "proper" UI view as part of the scheduling work.

Also while I tested this pretty extensively, it would ease my mind if one of you guys would just do a small verification to see if everything functions as intended.

Will do! 👍

@jeremystretch commented on GitHub (Apr 8, 2022): > I changed the default to 90. While I prefer unlimited by default, I can see the rationale for 90. Thanks! It's all about trying to protect the user ultimately. > I also think it might make sense to move job results out of the admin ui at some point, espcially if scheduled tasks are on the table. Agreed; we should implement a "proper" UI view as part of the scheduling work. > Also while I tested this pretty extensively, it would ease my mind if one of you guys would just do a small verification to see if everything functions as intended. Will do! :+1:
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label status: accepted type: feature
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#6253
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?