starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Per Script Worker Queue Setting #9830

New Issue
Open
opened 2025-12-29 21:23:20 +01:00 by adam · 7 comments
adam commented 2025-12-29 21:23:20 +01:00
Owner
Copy Link

Originally created by @jchambers2012 on GitHub (Jun 11, 2024).

NetBox version

v3.7.8 and v4.0.5

Feature type

New functionality

Proposed functionality

Looking to enhance the script subsystem to add support to specify what worker queue each script should be run in. This will allow critical scripts to run before background or scheduled scripts. This feature would add a new field in the scripts’ Meta possibly called “rq_queue_name” that can bypass the logic in get_queue_for_model and just use the user defined queue. If “rq_queue_name” is not set then the current function can be called to determine what queue the script should be map too.

Use case

We have a few CI/DI pipeline that hit the API to generate control files in a centralized location versus the logic be distributed across different project code. This simplifies management of the project files but these critical scripts must wait in the queue with other background and user run scripts delaying the running of them.

Some of our non-critical script that update data can take 5 minutes to run and they are not time critical like the CI/DI pipeline.

Database changes

I don’t think any models will need to be updated to support this functionality.

External dependencies

n/a

Originally created by @jchambers2012 on GitHub (Jun 11, 2024). ### NetBox version v3.7.8 and v4.0.5 ### Feature type New functionality ### Proposed functionality Looking to enhance the script subsystem to add support to specify what worker queue each script should be run in. This will allow critical scripts to run before background or scheduled scripts. This feature would add a new field in the scripts’ Meta possibly called “rq_queue_name” that can bypass the logic in get_queue_for_model and just use the user defined queue. If “rq_queue_name” is not set then the current function can be called to determine what queue the script should be map too. ### Use case We have a few CI/DI pipeline that hit the API to generate control files in a centralized location versus the logic be distributed across different project code. This simplifies management of the project files but these critical scripts must wait in the queue with other background and user run scripts delaying the running of them. Some of our non-critical script that update data can take 5 minutes to run and they are not time critical like the CI/DI pipeline. ### Database changes I don’t think any models will need to be updated to support this functionality. ### External dependencies n/a
adam added the type: featurecomplexity: mediumnetboxstatus: backlog labels 2025-12-29 21:23:20 +01:00
adam commented 2025-12-29 21:23:21 +01:00
Author
Owner
Copy Link

@jchambers2012 commented on GitHub (Jun 11, 2024):

I mocked up what i think needs to be changed but have not tested - let me know if you want me to work on the feature

https://github.com/netbox-community/netbox/compare/develop...jchambers2012:netbox:script_rq_queue_name?expand=1

@jchambers2012 commented on GitHub (Jun 11, 2024): I mocked up what i think needs to be changed but have not tested - let me know if you want me to work on the feature https://github.com/netbox-community/netbox/compare/develop...jchambers2012:netbox:script_rq_queue_name?expand=1
adam commented 2025-12-29 21:23:21 +01:00
Author
Owner
Copy Link

@arthanson commented on GitHub (Jun 12, 2024):

@jchambers2012 I've assigned it to you as you said you have some code for it.

@arthanson commented on GitHub (Jun 12, 2024): @jchambers2012 I've assigned it to you as you said you have some code for it.
adam commented 2025-12-29 21:23:21 +01:00
Author
Owner
Copy Link

@jchambers2012 commented on GitHub (Jun 13, 2024):

Stimulation are working will try and button up the PR this afternoon if not tomorrow
image

image

@jchambers2012 commented on GitHub (Jun 13, 2024): Stimulation are working will try and button up the PR this afternoon if not tomorrow ![image](https://github.com/netbox-community/netbox/assets/6090690/764e5610-6be4-4bf7-96b4-cf4d1e7b0bce) ![image](https://github.com/netbox-community/netbox/assets/6090690/09ee0c8a-2d6b-4b64-a594-c15b1214ccc4)
adam commented 2025-12-29 21:23:22 +01:00
Author
Owner
Copy Link

@github-actions[bot] commented on GitHub (Sep 12, 2024):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Do not attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our contributing guide.

@github-actions[bot] commented on GitHub (Sep 12, 2024): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. **Do not** attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our [contributing guide](https://github.com/netbox-community/netbox/blob/develop/CONTRIBUTING.md).
adam commented 2025-12-29 21:23:22 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Sep 23, 2024):

While I agree that it would be valuable to enable mapping custom scripts to specific worker queues, we'll need to address a reasonably flexible mechanism for doing so. Statically mapping each script via an attribute (as in PR #16552) is not sufficiently scalable or easy to manage.

@jeremystretch commented on GitHub (Sep 23, 2024): While I agree that it would be valuable to enable mapping custom scripts to specific worker queues, we'll need to address a reasonably flexible mechanism for doing so. Statically mapping each script via an attribute (as in PR #16552) is not sufficiently scalable or easy to manage.
adam commented 2025-12-29 21:23:22 +01:00
Author
Owner
Copy Link

@alehaa commented on GitHub (Oct 9, 2024):

I propose splitting this into two tasks, maybe solving the issue of scalability:

  1. Add a queue parameter to Job.enqueue() and the JobRunner framework so that background jobs (including scripts) can define a worker queue to use. This can be any string, usually one of high, default or low. Plugins can also use their own custom queues if needed.
  2. Move the queue definition to a ChoiceSet and make it available in the Scripts form. Users would then only be able to select queues defined by NetBox administrators, not those defined by plugins. This should be quite flexible and not overwhelming. If needed, future changes to NetBox could map these defined queue choices to whatever a new scheduler would need.
@alehaa commented on GitHub (Oct 9, 2024): I propose splitting this into two tasks, maybe solving the issue of scalability: 1. Add a `queue` parameter to `Job.enqueue()` and the `JobRunner` framework so that background jobs (including scripts) can define a worker queue to use. This can be any string, usually one of `high`, `default` or `low`. Plugins can also use their own custom queues if needed. 2. Move the queue definition to a `ChoiceSet` and make it available in the `Scripts` form. Users would then only be able to select queues defined by NetBox administrators, not those defined by plugins. This should be quite flexible and not overwhelming. If needed, future changes to NetBox could map these defined queue choices to whatever a new scheduler would need.
adam commented 2025-12-29 21:23:22 +01:00
Author
Owner
Copy Link

@github-actions[bot] commented on GitHub (Apr 25, 2025):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Do not attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our contributing guide.

@github-actions[bot] commented on GitHub (Apr 25, 2025): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. **Do not** attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our [contributing guide](https://github.com/netbox-community/netbox/blob/main/CONTRIBUTING.md).
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label complexity: medium netbox status: backlog type: feature
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#9830
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?