starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Allow plugin authors to register housekeeping tasks #10006

New Issue
Closed
opened 2025-12-29 21:25:32 +01:00 by adam · 9 comments
adam commented 2025-12-29 21:25:32 +01:00
Owner
Copy Link

Originally created by @jsenecal on GitHub (Jul 24, 2024).

Originally assigned to: @alehaa on GitHub.

NetBox version

v4.0.7

Feature type

New functionality

Proposed functionality

NetBox includes a housekeeping management command that usually runs nightly. This command handles:

  • Clearing expired authentication sessions from the database
  • Deleting older changelog records
  • Deleting older job result records
  • Checking for new NetBox releases

It would be great if plugin authors could register additional tasks to run by the housekeeping process without having to write their own management commands and setup mechanisms to run those periodically.

Use case

I often have to write periodic tasks to alter model instances in my plugins, reacting to dates, sending emails etc. and having to setup the whole management commands and related mechanisms to run those periodically is not really convenient, even more so considering that NetBox already has mechanisms built in to do so.

Database changes

None whatsoever

External dependencies

None

Originally created by @jsenecal on GitHub (Jul 24, 2024). Originally assigned to: @alehaa on GitHub. ### NetBox version v4.0.7 ### Feature type New functionality ### Proposed functionality NetBox includes a housekeeping management command that _usually_ runs nightly. This command handles: - Clearing expired authentication sessions from the database - Deleting older changelog records - Deleting older job result records - Checking for new NetBox releases It would be great if plugin authors could register additional tasks to run by the housekeeping process without having to write their own management commands and setup mechanisms to run those periodically. ### Use case I often have to write periodic tasks to alter model instances in my plugins, reacting to dates, sending emails etc. and having to setup the whole management commands and related mechanisms to run those periodically is not really convenient, even more so considering that NetBox already has mechanisms built in to do so. ### Database changes None whatsoever ### External dependencies None
adam added the status: acceptedtype: featurenetboxcomplexity: low labels 2025-12-29 21:25:32 +01:00
adam closed this issue 2025-12-29 21:25:33 +01:00
adam commented 2025-12-29 21:25:34 +01:00
Author
Owner
Copy Link

@alehaa commented on GitHub (Jul 25, 2024):

The background jobs framework of #15692 will include the ability to schedule system jobs, i.e. housekeeping or other tasks that run periodically in the background. While the plugin still requires a special management command for interactive execution (optional), periodic execution is covered by this framework.

@alehaa commented on GitHub (Jul 25, 2024): The background jobs framework of #15692 will include the ability to schedule system jobs, i.e. housekeeping or other tasks that run periodically in the background. While the plugin still requires a special management command for interactive execution (optional), periodic execution is covered by this framework.
adam commented 2025-12-29 21:25:34 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Jul 25, 2024):

I think ultimately we probably want to do away with the separate housekeeping task and integrate everything using the job's framework, particularly in light of @alehaa's recent work.

@jeremystretch commented on GitHub (Jul 25, 2024): I think ultimately we probably want to do away with the separate housekeeping task and integrate everything using the job's framework, particularly in light of @alehaa's recent work.
adam commented 2025-12-29 21:25:34 +01:00
Author
Owner
Copy Link

@alehaa commented on GitHub (Jul 30, 2024):

The JobRunner framework now supports the implementation of any kind of task, including housekeeping. However, the automatic registration of jobs on plugin load did not make it into the PR because the implementation is not stable enough for production. I suggest this FR to focus on this remaining issue, so that even housekeeping tasks can be set up automatically and solve this problem.

Here are the latest findings:

  • Calling JobRunner.enqueue_once() directly from the ready() method of AppConfig logs a warning because the database is accessed during plugin setup. The Django documentation explicitly warns against this use case, as the ready() method is called every time a plugin is loaded, even if it is only for the nbshell management command.
  • Using the connection_created signal solves the console warning. However, this triggers the JobRunner setup even more often, e.g. because the database is connected before each job execution.

So the problem for now is to find a way to automatically trigger the setup method outside of the request-response cycle that is called only once or say every few hours.

@alehaa commented on GitHub (Jul 30, 2024): The `JobRunner` framework now supports the implementation of any kind of task, including housekeeping. However, the automatic registration of jobs on plugin load did not make it into the PR because the implementation is not stable enough for production. I suggest this FR to focus on this remaining issue, so that even housekeeping tasks can be set up automatically and solve this problem. Here are the latest findings: * Calling `JobRunner.enqueue_once()` directly from the `ready()` method of `AppConfig` logs a warning because the database is accessed during plugin setup. The [Django documentation](https://docs.djangoproject.com/en/5.0/ref/applications/#django.apps.AppConfig.ready) explicitly warns against this use case, as the `ready()` method is called every time a plugin is loaded, even if it is only for the `nbshell` management command. * Using the [`connection_created`](https://docs.djangoproject.com/en/5.0/ref/signals/#connection-created) signal solves the console warning. However, this triggers the `JobRunner` setup even more often, e.g. because the database is connected before each job execution. So the problem for now is to find a way to automatically trigger the setup method outside of the request-response cycle that is called only once or say every few hours.
adam commented 2025-12-29 21:25:35 +01:00
Author
Owner
Copy Link

@alehaa commented on GitHub (Jul 30, 2024):

How about adding the auto-registration to the beginning of the rqworker management command? It would only be called once for each worker started - not for the frontend and not for each database connection. After updates, restarting the rqworker would trigger auto-registration again.

I'd implement this using a new method in AppConfig (e.g. enqueue_job_runner()) being called by rqworker.handle() for each plugin. An alternative would be using a signal, which avoids a new method but could obfuscate the logic.

@alehaa commented on GitHub (Jul 30, 2024): How about adding the auto-registration to the beginning of the `rqworker` management command? It would only be called once for each worker started - not for the frontend and not for each database connection. After updates, restarting the `rqworker` would trigger auto-registration again. I'd implement this using a new method in `AppConfig` (e.g. `enqueue_job_runner()`) being called by `rqworker.handle()` for each plugin. An alternative would be using a signal, which avoids a new method but could obfuscate the logic.
adam commented 2025-12-29 21:25:35 +01:00
Author
Owner
Copy Link

@jsenecal commented on GitHub (Jul 31, 2024):

How about adding the auto-registration to the beginning of the rqworker management command? It would only be called once for each worker started - not for the frontend and not for each database connection. After updates, restarting the rqworker would trigger auto-registration again.

I'd implement this using a new method in AppConfig (e.g. enqueue_job_runner()) being called by rqworker.handle() for each plugin. An alternative would be using a signal, which avoids a new method but could obfuscate the logic.

This feels like a decent approach.
Netbox also has existing mechanisms working with "registries"; we could have a "Housekeeping tasks" registry which would be handled in a similar fashion by the worker and would provide a standardized way to register periodic tasks.

@jsenecal commented on GitHub (Jul 31, 2024): > How about adding the auto-registration to the beginning of the `rqworker` management command? It would only be called once for each worker started - not for the frontend and not for each database connection. After updates, restarting the `rqworker` would trigger auto-registration again. > > I'd implement this using a new method in `AppConfig` (e.g. `enqueue_job_runner()`) being called by `rqworker.handle()` for each plugin. An alternative would be using a signal, which avoids a new method but could obfuscate the logic. This feels like a decent approach. Netbox also has existing mechanisms working with "registries"; we could have a "Housekeeping tasks" registry which would be handled in a similar fashion by the worker and would provide a standardized way to register periodic tasks.
adam commented 2025-12-29 21:25:35 +01:00
Author
Owner
Copy Link

@alehaa commented on GitHub (Aug 1, 2024):

I'm not sure if this is possible. Usually the registry handles static variables like a list of middleware to load. To register a JobRunner, its enqueue_once() method needs to be called and its arguments can vary depending on the plugin's implementation. Therefore I think some kind of method needs to be called for each plugin, but of course it can reside in the AppConfig class along with other registry-related variables. In this method, plugins could implement the logic to setup the JobRunner, i.e. by loading configuration or scheduling task depending on other NetBox objects, etc.

@alehaa commented on GitHub (Aug 1, 2024): I'm not sure if this is possible. Usually the registry handles static variables like a list of middleware to load. To register a `JobRunner`, its `enqueue_once()` method needs to be called and its arguments can vary depending on the plugin's implementation. Therefore I think some kind of method needs to be called for each plugin, but of course it can reside in the `AppConfig` class along with other registry-related variables. In this method, plugins could implement the logic to setup the `JobRunner`, i.e. by loading configuration or scheduling task depending on other NetBox objects, etc.
adam commented 2025-12-29 21:25:35 +01:00
Author
Owner
Copy Link

@alehaa commented on GitHub (Aug 16, 2024):

@jeremystretch can you assign this to me so I can work on a PR? Maybe we can still make it into 4.1.

@alehaa commented on GitHub (Aug 16, 2024): @jeremystretch can you assign this to me so I can work on a PR? Maybe we can still make it into 4.1.
adam commented 2025-12-29 21:25:35 +01:00
Author
Owner
Copy Link

@alehaa commented on GitHub (Sep 24, 2024):

@jeremystretch did you have any chance to review this? This would be the last missing piece to add auto-scheduling background tasks, which didn't make it into the JobRunner framework.

@alehaa commented on GitHub (Sep 24, 2024): @jeremystretch did you have any chance to review this? This would be the last missing piece to add auto-scheduling background tasks, which didn't make it into the `JobRunner` framework.
adam commented 2025-12-29 21:25:36 +01:00
Author
Owner
Copy Link

@DanSheps commented on GitHub (Oct 2, 2024):

@alehaa I am not so sure about the approach, I think something more agnostic that can be called after initialization of all apps are done but not specifically tied jobs or RQ might be a better approach.

That said, I don't know if there is a viable solution for this as there is no "apps_ready" signal emitted by django. Your solution might be the most viable right now.

@DanSheps commented on GitHub (Oct 2, 2024): @alehaa I am not so sure about the approach, I think something more agnostic that can be called **after** initialization of all apps are done but not specifically tied jobs or RQ might be a better approach. That said, I don't know if there is a viable solution for this as there is no "apps_ready" signal emitted by django. Your solution might be the most viable right now.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label complexity: low netbox status: accepted type: feature
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#10006
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?