starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

deficiency in new webhook implementation #1762

New Issue
Closed
opened 2025-12-29 17:18:54 +01:00 by adam · 6 comments
adam commented 2025-12-29 17:18:54 +01:00
Owner
Copy Link

Originally created by @ghost on GitHub (Jun 2, 2018).

Issue type

[ ] Feature request
[X] Bug report
[ ] Documentation

Environment

  • Python version: python 2.7.5
  • NetBox version: develop-2.4

Description

Testing out the webhook implementation and discovered the following bug:
when a model contains a custom field of type date the worker is unable to serialize the data for transmission

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/rq/worker.py", line 793, in perform_job
    rv = job.perform()
  File "/usr/lib/python2.7/site-packages/rq/job.py", line 599, in perform
    self._result = self._execute()
  File "/usr/lib/python2.7/site-packages/rq/job.py", line 605, in _execute
    return self.func(*self.args, **self.kwargs)
  File "/opt/netbox/netbox/extras/webhooks_worker.py", line 44, in process_webhook
    prepared_request = requests.Request(**params).prepare()
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 259, in prepare
    hooks=self.hooks,
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 307, in prepare
    self.prepare_body(data, files, json)
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 427, in prepare_body
    body = json_dumps(json)
  File "/usr/lib64/python2.7/site-packages/simplejson/__init__.py", line 382, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib64/python2.7/site-packages/simplejson/encoder.py", line 291, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib64/python2.7/site-packages/simplejson/encoder.py", line 373, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib64/python2.7/site-packages/simplejson/encoder.py", line 268, in default
    o.__class__.__name__)
TypeError: Object of type date is not JSON serializable
Originally created by @ghost on GitHub (Jun 2, 2018). <!-- Before opening a new issue, please search through the existing issues to see if your topic has already been addressed. Note that you may need to remove the "is:open" filter from the search bar to include closed issues. Check the appropriate type for your issue below by placing an x between the brackets. For assistance with installation issues, or for any other issues other than those listed below, please raise your topic for discussion on our mailing list: https://groups.google.com/forum/#!forum/netbox-discuss Please note that issues which do not fall under any of the below categories will be closed. Due to an excessive backlog of feature requests, we are not currently accepting any proposals which extend NetBox's feature scope. Do not prepend any sort of tag to your issue's title. An administrator will review your issue and assign labels as appropriate. ---> ### Issue type [ ] Feature request <!-- An enhancement of existing functionality --> [X] Bug report <!-- Unexpected or erroneous behavior --> [ ] Documentation <!-- A modification to the documentation --> <!-- Please describe the environment in which you are running NetBox. (Be sure to verify that you are running the latest stable release of NetBox before submitting a bug report.) If you are submitting a bug report and have made any changes to the code base, please first validate that your bug can be recreated while running an official release. --> ### Environment * Python version: python 2.7.5 * NetBox version: develop-2.4 <!-- BUG REPORTS must include: * A list of the steps needed for someone else to reproduce the bug * A description of the expected and observed behavior * Any relevant error messages (screenshots may also help) FEATURE REQUESTS must include: * A detailed description of the proposed functionality * A use case for the new feature * A rough description of any necessary changes to the database schema * Any relevant third-party libraries which would be needed --> ### Description Testing out the webhook implementation and discovered the following bug: when a model contains a custom field of type date the worker is unable to serialize the data for transmission ``` Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/rq/worker.py", line 793, in perform_job rv = job.perform() File "/usr/lib/python2.7/site-packages/rq/job.py", line 599, in perform self._result = self._execute() File "/usr/lib/python2.7/site-packages/rq/job.py", line 605, in _execute return self.func(*self.args, **self.kwargs) File "/opt/netbox/netbox/extras/webhooks_worker.py", line 44, in process_webhook prepared_request = requests.Request(**params).prepare() File "/usr/lib/python2.7/site-packages/requests/models.py", line 259, in prepare hooks=self.hooks, File "/usr/lib/python2.7/site-packages/requests/models.py", line 307, in prepare self.prepare_body(data, files, json) File "/usr/lib/python2.7/site-packages/requests/models.py", line 427, in prepare_body body = json_dumps(json) File "/usr/lib64/python2.7/site-packages/simplejson/__init__.py", line 382, in dumps return _default_encoder.encode(obj) File "/usr/lib64/python2.7/site-packages/simplejson/encoder.py", line 291, in encode chunks = self.iterencode(o, _one_shot=True) File "/usr/lib64/python2.7/site-packages/simplejson/encoder.py", line 373, in iterencode return _iterencode(o, 0) File "/usr/lib64/python2.7/site-packages/simplejson/encoder.py", line 268, in default o.__class__.__name__) TypeError: Object of type date is not JSON serializable ```
adam added the type: bugstatus: accepted labels 2025-12-29 17:18:54 +01:00
adam closed this issue 2025-12-29 17:18:54 +01:00
adam commented 2025-12-29 17:18:55 +01:00
Author
Owner
Copy Link

@lampwins commented on GitHub (Jun 27, 2018):

@ridoline I am unable to reproduce this with a Date custom field on the Device model. Please provide the exact steps necessary to raise this issue.

@lampwins commented on GitHub (Jun 27, 2018): @ridoline I am unable to reproduce this with a Date custom field on the Device model. Please provide the exact steps necessary to raise this issue.
adam commented 2025-12-29 17:18:55 +01:00
Author
Owner
Copy Link

@ghost commented on GitHub (Jun 28, 2018):

@lawpwins, I suppose it might have been an issue with my data or has been resolved, as I recently refreshed my development box with production data and the issue is no longer manifesting itself.

@ghost commented on GitHub (Jun 28, 2018): @lawpwins, I suppose it might have been an issue with my data or has been resolved, as I recently refreshed my development box with production data and the issue is no longer manifesting itself.
adam commented 2025-12-29 17:18:55 +01:00
Author
Owner
Copy Link

@lampwins commented on GitHub (Jun 28, 2018):

I played with this some last night and was able to raise the issue a handful of times, but I have yet to find a root cause or even a reliable way of triggering the issue. I will be digging into this more.

@lampwins commented on GitHub (Jun 28, 2018): I played with this some last night and was able to raise the issue a handful of times, but I have yet to find a root cause or even a reliable way of triggering the issue. I will be digging into this more.
adam commented 2025-12-29 17:18:56 +01:00
Author
Owner
Copy Link

@lampwins commented on GitHub (Jul 21, 2018):

I finally ran this down. There are two separate issues in play. First, date type custom field values are currently actual datetime objects and the default json serializer in the requests library doesn't handle this well. I am still a little unclear why DRF is okay with this, since we are using the same API serializers. Second, the current signaling system for Webhooks suffers from the same problems of related objects (custom fields and tags) not being associated at the time of the post_save signal, as the original Change Logging implementation. This was addresses in Change Logging in df1f339. I think it makes a lot of sense to wrap the Webhook signaling into the ChangeLoggingMiddleware since the mechanics of both features are very similar.

@lampwins commented on GitHub (Jul 21, 2018): I finally ran this down. There are two separate issues in play. First, date type custom field values are currently actual datetime objects and the default json serializer in the requests library doesn't handle this well. I am still a little unclear why DRF is okay with this, since we are using the same API serializers. Second, the current signaling system for Webhooks suffers from the same problems of related objects (custom fields and tags) not being associated at the time of the post_save signal, as the original Change Logging implementation. This was addresses in Change Logging in df1f339. I think it makes a lot of sense to wrap the Webhook signaling into the ChangeLoggingMiddleware since the mechanics of both features are very similar.
adam commented 2025-12-29 17:18:56 +01:00
Author
Owner
Copy Link

@lampwins commented on GitHub (Jul 30, 2018):

Just to be clear, this was broken out from #2282 and thus this issue still needs to be addressed.

@lampwins commented on GitHub (Jul 30, 2018): Just to be clear, this was broken out from #2282 and thus this issue still needs to be addressed.
adam commented 2025-12-29 17:18:56 +01:00
Author
Owner
Copy Link

@lampwins commented on GitHub (Jul 31, 2018):

I finally nailed the root cause down. DRF processes the JSON encoding after the serialization, so by manually using the serialisers we miss the DRF specific encoding which makes use of there own JSON encoder class which correctly handles datatime objects (and many others) 7b1582e00e/rest_framework/utils/encoders.py (L34). Currently we are using the built-in JSON encoder of the requests library, so we simply need to specify an alternate encoder to fix this.

@lampwins commented on GitHub (Jul 31, 2018): I finally nailed the root cause down. DRF processes the JSON encoding after the serialization, so by manually using the serialisers we miss the DRF specific encoding which makes use of there own JSON encoder class which correctly handles datatime objects (and many others) https://github.com/encode/django-rest-framework/blob/7b1582e00e91001ec07cd394520ec5cdd2c2add1/rest_framework/utils/encoders.py#L34. Currently we are using the built-in JSON encoder of the requests library, so we simply need to specify an alternate encoder to fix this.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label status: accepted type: bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#1762
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?