Webhooks: Query RQ queue for job status via API #5670

Closed
opened 2025-12-29 19:31:07 +01:00 by adam · 3 comments
Owner

Originally created by @netgab on GitHub (Nov 16, 2021).

Originally assigned to: @arthanson on GitHub.

NetBox version

v3.0.10

Feature type

New functionality

Proposed functionality

The webhook job status (Admin UI > Background Jobs) should be queried over the netbox API.

Use case

The goal is to make webhooks more reliable in terms of failed jobs. For example if the webhook receiver is not available for any reason (e.g. downtime), the failed jobs are not retried.
To make it more reliable, it should be possible to query netbox for failed jobs and filter based on the date/time and/or webhook name. Basically all the information in Kwargs in the RQ queue should be returned.

A webhook receiver (or any other monitoring system) may query netbox in regular intervals for these failed webhook events to:

  • automatically resolve the missed webhook
  • generate an incident for manual countermeasures (let an admin look and verify)

Database changes

not sure - sorry

External dependencies

No response

Originally created by @netgab on GitHub (Nov 16, 2021). Originally assigned to: @arthanson on GitHub. ### NetBox version v3.0.10 ### Feature type New functionality ### Proposed functionality The webhook job status (Admin UI > Background Jobs) should be queried over the netbox API. ### Use case The goal is to make webhooks more reliable in terms of failed jobs. For example if the webhook receiver is not available for any reason (e.g. downtime), the failed jobs are not retried. To make it more reliable, it should be possible to query netbox for failed jobs and filter based on the date/time and/or webhook name. Basically all the information in `Kwargs` in the RQ queue should be returned. A webhook receiver (or any other monitoring system) may query netbox in regular intervals for these failed webhook events to: - automatically resolve the missed webhook - generate an incident for manual countermeasures (let an admin look and verify) ### Database changes not sure - sorry ### External dependencies _No response_
adam added the status: acceptedtype: featurenetboxcomplexity: low labels 2025-12-29 19:31:07 +01:00
adam closed this issue 2025-12-29 19:31:07 +01:00
Author
Owner

@jeremystretch commented on GitHub (Dec 13, 2021):

@netgab can you provide an example of what you'd expect the API endpoint and response to look like?

@jeremystretch commented on GitHub (Dec 13, 2021): @netgab can you provide an example of what you'd expect the API endpoint and response to look like?
Author
Owner

@netgab commented on GitHub (Dec 13, 2021):

@jeremystretch: Sure - but I'm not a developer ... so it's just a rough / best effort guess.
For a generic approach (not focussing on webhooks in particular, instead focus on queues):
API endpoint: api/extras/background-tasks/<QUEUE-NAME>/{failed|finished|...}/
For my purpose (failed webhook stuff): api/extras/background-tasks/default/failed/

Response: (in YAML, because I'm lazy)

count: <NUMBER-OF-ITEMS-IN-QUEUE>
results:
  - id: "<JOB-ID>"   # example: dbcd6415-af03-4366-8d8e-7f891c0ccb3b
    queue: "<QUEUE-NAME>"    # example: default
    callable: "<CALL-NAME>"   # example: extras.webhooks_worker.process_webhook
    kwargs:    # dictionary of kwargs 

To be more use case centric (failed webhooks):
API endpoint: api/extras/webhook-status/

Response: (in YAML, because I'm lazy)

count: <NUMBER-OF-ITEMS-IN-QUEUE>
results:
  - id: "<JOB-ID>"   # example: dbcd6415-af03-4366-8d8e-7f891c0ccb3b
    webhook: "<WEBHOOK-NAME>"   # Link to the webhook object (/apt/extras/webhook)
    event: "{create|delete| ...}
    data: <DICTIONARY-OF-WEBHOOK-DATA>
    snapshots: <PRE-POST-CHANGE-DICTIONARY>
    timestamp: <TIMESTAMP>

Personally I'm for the use case centric approach.

@netgab commented on GitHub (Dec 13, 2021): @jeremystretch: Sure - but I'm not a developer ... so it's just a rough / best effort guess. For a generic approach (not focussing on webhooks in particular, instead focus on queues): **API endpoint:** `api/extras/background-tasks/<QUEUE-NAME>/{failed|finished|...}/` For my purpose (failed webhook stuff): `api/extras/background-tasks/default/failed/` **Response:** (in YAML, because I'm lazy) ``` count: <NUMBER-OF-ITEMS-IN-QUEUE> results: - id: "<JOB-ID>" # example: dbcd6415-af03-4366-8d8e-7f891c0ccb3b queue: "<QUEUE-NAME>" # example: default callable: "<CALL-NAME>" # example: extras.webhooks_worker.process_webhook kwargs: # dictionary of kwargs ``` To be more use case centric (failed webhooks): **API endpoint:** `api/extras/webhook-status/` **Response:** (in YAML, because I'm lazy) ``` count: <NUMBER-OF-ITEMS-IN-QUEUE> results: - id: "<JOB-ID>" # example: dbcd6415-af03-4366-8d8e-7f891c0ccb3b webhook: "<WEBHOOK-NAME>" # Link to the webhook object (/apt/extras/webhook) event: "{create|delete| ...} data: <DICTIONARY-OF-WEBHOOK-DATA> snapshots: <PRE-POST-CHANGE-DICTIONARY> timestamp: <TIMESTAMP> ``` Personally I'm for the use case centric approach.
Author
Owner

@github-actions[bot] commented on GitHub (Feb 12, 2022):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Please see our contributing guide.

@github-actions[bot] commented on GitHub (Feb 12, 2022): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Please see our [contributing guide](https://github.com/netbox-community/netbox/blob/develop/CONTRIBUTING.md).
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#5670