starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

SystemHousekeepingJob runs multiple times in direct succession when executions were missed #11566

New Issue
Closed
opened 2025-12-29 21:46:51 +01:00 by adam · 3 comments
adam commented 2025-12-29 21:46:51 +01:00
Owner
Copy Link

Originally created by @tobiasge on GitHub (Sep 4, 2025).

Originally assigned to: @jeremystretch on GitHub.

Deployment Type

Self-hosted

NetBox Version

v4.4.0

Python Version

3.12

Steps to Reproduce

  1. Don't run the worker for a few days
  2. Start worker

Expected Behavior

Job only runs once and is then re-scheduled in the normal interval

Observed Behavior

netbox-worker-1  | 13:18:07 default: handle(job=<Job: 9790c13b-319f-493a-984c-f72abb8ab29c>) (9790c13b-319f-493a-984c-f72abb8ab29c)
netbox-worker-1  | 13:18:08 Successfully completed handle(job=<Job: 9790c13b-319f-493a-984c-f72abb8ab29c>) job in 0:00:00.842352s on worker e56a179060f94f61a3fa50b4e9653623
netbox-worker-1  | 13:18:08 default: Job OK (9790c13b-319f-493a-984c-f72abb8ab29c)
netbox-worker-1  | 13:18:08 Result is kept for 500 seconds
netbox-worker-1  | 13:18:08 default: handle(job=<Job: System Housekeeping>) (090122fb-5a5d-4df3-93f2-c4080632d3a7)
netbox-worker-1  | 13:18:08 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.505269s on worker e56a179060f94f61a3fa50b4e9653623
netbox-worker-1  | 13:18:08 default: Job OK (090122fb-5a5d-4df3-93f2-c4080632d3a7)
netbox-worker-1  | 13:18:08 Result is kept for 500 seconds
netbox-worker-1  | 13:18:09 default: handle(job=<Job: System Housekeeping>) (6f1c996a-0e81-449a-9633-084d1ee016a7)
netbox-worker-1  | 13:18:09 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.513766s on worker e56a179060f94f61a3fa50b4e9653623
netbox-worker-1  | 13:18:09 default: Job OK (6f1c996a-0e81-449a-9633-084d1ee016a7)
netbox-worker-1  | 13:18:09 Result is kept for 500 seconds
netbox-worker-1  | 13:18:10 default: handle(job=<Job: System Housekeeping>) (c40ea1a9-1c35-4b08-8a53-8a51478f45ca)
netbox-worker-1  | 13:18:10 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.502376s on worker e56a179060f94f61a3fa50b4e9653623
netbox-worker-1  | 13:18:10 default: Job OK (c40ea1a9-1c35-4b08-8a53-8a51478f45ca)
netbox-worker-1  | 13:18:10 Result is kept for 500 seconds
netbox-worker-1  | 13:18:11 default: handle(job=<Job: System Housekeeping>) (b74fbbd7-8ebd-4b1e-8e70-71ab95b1966c)
netbox-worker-1  | 13:18:11 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.549482s on worker e56a179060f94f61a3fa50b4e9653623
netbox-worker-1  | 13:18:11 default: Job OK (b74fbbd7-8ebd-4b1e-8e70-71ab95b1966c)
netbox-worker-1  | 13:18:11 Result is kept for 500 seconds
netbox-worker-1  | 13:18:12 default: handle(job=<Job: System Housekeeping>) (556727ed-c157-497c-aa19-0289075e16b6)
netbox-worker-1  | 13:18:12 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.497449s on worker e56a179060f94f61a3fa50b4e9653623
netbox-worker-1  | 13:18:12 default: Job OK (556727ed-c157-497c-aa19-0289075e16b6)
netbox-worker-1  | 13:18:12 Result is kept for 500 seconds
netbox-worker-1  | 13:18:13 default: handle(job=<Job: System Housekeeping>) (a486417e-6261-46fa-831a-d5657697f150)
netbox-worker-1  | 13:18:13 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.532598s on worker e56a179060f94f61a3fa50b4e9653623
netbox-worker-1  | 13:18:13 default: Job OK (a486417e-6261-46fa-831a-d5657697f150)
netbox-worker-1  | 13:18:13 Result is kept for 500 seconds
netbox-worker-1  | 13:18:14 default: handle(job=<Job: System Housekeeping>) (3da8476f-e39b-4314-b70c-e150b8ebd198)
netbox-worker-1  | 13:18:14 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.508011s on worker e56a179060f94f61a3fa50b4e9653623
netbox-worker-1  | 13:18:14 default: Job OK (3da8476f-e39b-4314-b70c-e150b8ebd198)
netbox-worker-1  | 13:18:14 Result is kept for 500 seconds
Originally created by @tobiasge on GitHub (Sep 4, 2025). Originally assigned to: @jeremystretch on GitHub. ### Deployment Type Self-hosted ### NetBox Version v4.4.0 ### Python Version 3.12 ### Steps to Reproduce 1. Don't run the worker for a few days 2. Start worker ### Expected Behavior Job only runs once and is then re-scheduled in the normal interval ### Observed Behavior ``` netbox-worker-1 | 13:18:07 default: handle(job=<Job: 9790c13b-319f-493a-984c-f72abb8ab29c>) (9790c13b-319f-493a-984c-f72abb8ab29c) netbox-worker-1 | 13:18:08 Successfully completed handle(job=<Job: 9790c13b-319f-493a-984c-f72abb8ab29c>) job in 0:00:00.842352s on worker e56a179060f94f61a3fa50b4e9653623 netbox-worker-1 | 13:18:08 default: Job OK (9790c13b-319f-493a-984c-f72abb8ab29c) netbox-worker-1 | 13:18:08 Result is kept for 500 seconds netbox-worker-1 | 13:18:08 default: handle(job=<Job: System Housekeeping>) (090122fb-5a5d-4df3-93f2-c4080632d3a7) netbox-worker-1 | 13:18:08 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.505269s on worker e56a179060f94f61a3fa50b4e9653623 netbox-worker-1 | 13:18:08 default: Job OK (090122fb-5a5d-4df3-93f2-c4080632d3a7) netbox-worker-1 | 13:18:08 Result is kept for 500 seconds netbox-worker-1 | 13:18:09 default: handle(job=<Job: System Housekeeping>) (6f1c996a-0e81-449a-9633-084d1ee016a7) netbox-worker-1 | 13:18:09 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.513766s on worker e56a179060f94f61a3fa50b4e9653623 netbox-worker-1 | 13:18:09 default: Job OK (6f1c996a-0e81-449a-9633-084d1ee016a7) netbox-worker-1 | 13:18:09 Result is kept for 500 seconds netbox-worker-1 | 13:18:10 default: handle(job=<Job: System Housekeeping>) (c40ea1a9-1c35-4b08-8a53-8a51478f45ca) netbox-worker-1 | 13:18:10 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.502376s on worker e56a179060f94f61a3fa50b4e9653623 netbox-worker-1 | 13:18:10 default: Job OK (c40ea1a9-1c35-4b08-8a53-8a51478f45ca) netbox-worker-1 | 13:18:10 Result is kept for 500 seconds netbox-worker-1 | 13:18:11 default: handle(job=<Job: System Housekeeping>) (b74fbbd7-8ebd-4b1e-8e70-71ab95b1966c) netbox-worker-1 | 13:18:11 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.549482s on worker e56a179060f94f61a3fa50b4e9653623 netbox-worker-1 | 13:18:11 default: Job OK (b74fbbd7-8ebd-4b1e-8e70-71ab95b1966c) netbox-worker-1 | 13:18:11 Result is kept for 500 seconds netbox-worker-1 | 13:18:12 default: handle(job=<Job: System Housekeeping>) (556727ed-c157-497c-aa19-0289075e16b6) netbox-worker-1 | 13:18:12 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.497449s on worker e56a179060f94f61a3fa50b4e9653623 netbox-worker-1 | 13:18:12 default: Job OK (556727ed-c157-497c-aa19-0289075e16b6) netbox-worker-1 | 13:18:12 Result is kept for 500 seconds netbox-worker-1 | 13:18:13 default: handle(job=<Job: System Housekeeping>) (a486417e-6261-46fa-831a-d5657697f150) netbox-worker-1 | 13:18:13 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.532598s on worker e56a179060f94f61a3fa50b4e9653623 netbox-worker-1 | 13:18:13 default: Job OK (a486417e-6261-46fa-831a-d5657697f150) netbox-worker-1 | 13:18:13 Result is kept for 500 seconds netbox-worker-1 | 13:18:14 default: handle(job=<Job: System Housekeeping>) (3da8476f-e39b-4314-b70c-e150b8ebd198) netbox-worker-1 | 13:18:14 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.508011s on worker e56a179060f94f61a3fa50b4e9653623 netbox-worker-1 | 13:18:14 default: Job OK (3da8476f-e39b-4314-b70c-e150b8ebd198) netbox-worker-1 | 13:18:14 Result is kept for 500 seconds ```
adam added the type: bugstatus: acceptedseverity: low labels 2025-12-29 21:46:51 +01:00
adam closed this issue 2025-12-29 21:46:51 +01:00
adam commented 2025-12-29 21:46:52 +01:00
Author
Owner
Copy Link

@tobiasge commented on GitHub (Sep 4, 2025):

I think the problem is, that in netbox/netbox/jobs.py the time in new_scheduled_time (line 96) is not checked if it is already in the past.

@tobiasge commented on GitHub (Sep 4, 2025): I think the problem is, that in `netbox/netbox/jobs.py` the time in `new_scheduled_time` (line 96) is not checked if it is already in the past.
adam commented 2025-12-29 21:46:52 +01:00
Author
Owner
Copy Link

@marcoc610 commented on GitHub (Sep 23, 2025):

On a netbox instance the Housekeeping job did not run some days ago (2025-09-13) because the DB was unavailable and now (2025-09-23) the job seems not to run any more, nor is being rescheduled.

Image

Could this be related to the same issue?

@marcoc610 commented on GitHub (Sep 23, 2025): On a netbox instance the Housekeeping job did not run some days ago (2025-09-13) because the DB was unavailable and now (2025-09-23) the job seems not to run any more, nor is being rescheduled. <img width="1345" height="103" alt="Image" src="https://github.com/user-attachments/assets/20ddb830-ac28-4944-974e-628401003f90" /> Could this be related to the same issue?
adam commented 2025-12-29 21:46:52 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Sep 29, 2025):

  1. Don't run the worker for a few days

To expedite testing, one can reduce the interval for SystemHousekeepingJob from daily to one minute.

@jeremystretch commented on GitHub (Sep 29, 2025): > 1. Don't run the worker for a few days To expedite testing, one can reduce the interval for SystemHousekeepingJob from daily to one minute.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label severity: low status: accepted type: bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#11566
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?