starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

UI corrupts read-only JSON fields #9860

New Issue
Closed
opened 2025-12-29 21:23:39 +01:00 by adam · 8 comments
adam commented 2025-12-29 21:23:39 +01:00
Owner
Copy Link

Originally created by @FliesLikeABrick on GitHub (Jun 18, 2024).

Originally assigned to: @arthanson on GitHub.

Deployment Type

Self-hosted

NetBox Version

v4.0.5

Python Version

3.11

Steps to Reproduce

This issue can be reproduced in a couple different workflows. This was originally reported in #16328 after we saw this behavior, which we believe was introduced between 3.6.3 and 3.7.7. We have now reproduced this in a clean install of 4.0.5 running inside docker using netbox-docker 2.9.1.

Edits through the API are not impacted.

Option 1 - corruption during edit:

  1. Create a JSON custom field - set "UI Editable" to "No". Leave the default value as null.
  2. Create a prefix, it is not necessary to populate the JSON field with a value -- it will default to null
  3. View the prefix, observe that the JSON custom field shows a value of "-" in the UI due to it being null
  4. Edit the prefix and click "save" without making any changes
  5. Observe that the custom value now changes to the quoted string "null"

Option 2 - corruption on creation:

  1. Create a JSON custom field - set "UI Editable" to "No". Set the default to JSON such as {"key1":"value1"}
  2. Create a prefix
  3. View the prefix, observe that the JSON custom field now shows an escaped string value of "{\"key1\": \"value1\"}"
  4. Edit the prefix and click "save" without making any changes
  5. Observe that the custom value is now escaped again to something like "\"{\\\"key1\\\": \\\"value1\\\"}\""
  6. Repeat edit+save (without making changes) and observe that the prefix keeps getting escaped more and more as a JSON string instead of any valid JSON data structure.

Expected Behavior

The JSON value should not be changed due to the UI being set to read-only for this field

Observed Behavior

The server is setting a new and incorrect/corrupt value for this field in the database. The native JSON is being converted to an escaped string

Originally created by @FliesLikeABrick on GitHub (Jun 18, 2024). Originally assigned to: @arthanson on GitHub. ### Deployment Type Self-hosted ### NetBox Version v4.0.5 ### Python Version 3.11 ### Steps to Reproduce This issue can be reproduced in a couple different workflows. This was originally reported in #16328 after we saw this behavior, which we believe was introduced between 3.6.3 and 3.7.7. We have now reproduced this in a clean install of 4.0.5 running inside docker using netbox-docker 2.9.1. Edits through the API are not impacted. Option 1 - corruption during edit: 1) Create a JSON custom field - set "UI Editable" to "No". Leave the default value as null. 2) Create a prefix, it is not necessary to populate the JSON field with a value -- it will default to null 3) View the prefix, observe that the JSON custom field shows a value of "-" in the UI due to it being null 4) Edit the prefix and click "save" without making any changes 5) Observe that the custom value now changes to the quoted string "null" Option 2 - corruption on creation: 1) Create a JSON custom field - set "UI Editable" to "No". Set the default to JSON such as `{"key1":"value1"}` 2) Create a prefix 3) View the prefix, observe that the JSON custom field now shows an escaped string value of `"{\"key1\": \"value1\"}"` 4) Edit the prefix and click "save" without making any changes 5) Observe that the custom value is now escaped again to something like `"\"{\\\"key1\\\": \\\"value1\\\"}\""` 6) Repeat edit+save (without making changes) and observe that the prefix keeps getting escaped more and more as a JSON string instead of any valid JSON data structure. ### Expected Behavior The JSON value should not be changed due to the UI being set to read-only for this field ### Observed Behavior The server is setting a new and incorrect/corrupt value for this field in the database. The native JSON is being converted to an escaped string
adam added the type: bugstatus: acceptedseverity: medium labels 2025-12-29 21:23:39 +01:00
adam closed this issue 2025-12-29 21:23:39 +01:00
adam commented 2025-12-29 21:23:40 +01:00
Author
Owner
Copy Link

@FliesLikeABrick commented on GitHub (Jun 18, 2024):

We have stepped through releases and found that this was introduced in 3.7.7, the issue was not present in 3.7.6. Both of the "steps to reproduce" workflows described above are triggering the bug in 3.7.7.

@FliesLikeABrick commented on GitHub (Jun 18, 2024): We have stepped through releases and found that this was introduced in 3.7.7, the issue was not present in 3.7.6. Both of the "steps to reproduce" workflows described above are triggering the bug in 3.7.7.
adam commented 2025-12-29 21:23:40 +01:00
Author
Owner
Copy Link

@FliesLikeABrick commented on GitHub (Jun 18, 2024):

"editing" and saving a prefix multiple times in the UI (not actually editing anything, let alone the read-only JSON field) results in the field value being escaped multiple times:
image

@FliesLikeABrick commented on GitHub (Jun 18, 2024): "editing" and saving a prefix multiple times in the UI (not actually editing anything, let alone the read-only JSON field) results in the field value being escaped multiple times: <img width="623" alt="image" src="https://github.com/netbox-community/netbox/assets/1180584/7a64db65-cc3c-4a50-9f69-ec35d2a999e2">
adam commented 2025-12-29 21:23:40 +01:00
Author
Owner
Copy Link

@FliesLikeABrick commented on GitHub (Jun 18, 2024):

We have written a script that finds and repairs the affected data, which can be found at https://github.com/FliesLikeABrick/netbox_fix_json

@FliesLikeABrick commented on GitHub (Jun 18, 2024): We have written a script that finds and repairs the affected data, which can be found at https://github.com/FliesLikeABrick/netbox_fix_json
adam commented 2025-12-29 21:23:41 +01:00
Author
Owner
Copy Link

@jeffgdotorg commented on GitHub (Jun 21, 2024):

Thanks for reporting this problem and for providing clear and comprehensive steps to reproduce. I was able to reproduce the option-1 case exactly as described in a fresh 4.0.5 install. The option-2 case reproduces slightly differently from how it's written up in the issue body (the value { "thing1": "value1" } gets escaped and displays as "\"{\\\"thing1\\\": \\\"value1\\\"}\""), but I see your follow-on comment lines up with that. I suggest editing your issue body to match what you (and I) actually see in option 2, but in any case I'm moving your issue out of triage.

Extra thanks for doing the additional archaeology to determine where the problem seems to have been introduced. That's potentially a big help.

@jeffgdotorg commented on GitHub (Jun 21, 2024): Thanks for reporting this problem and for providing clear and comprehensive steps to reproduce. I was able to reproduce the option-1 case exactly as described in a fresh 4.0.5 install. The option-2 case reproduces slightly differently from how it's written up in the issue body (the value `{ "thing1": "value1" }` gets escaped and displays as `"\"{\\\"thing1\\\": \\\"value1\\\"}\""`), but I see your follow-on comment lines up with that. I suggest editing your issue body to match what you (and I) actually see in option 2, but in any case I'm moving your issue out of triage. Extra thanks for doing the additional archaeology to determine where the problem seems to have been introduced. That's potentially a big help.
adam commented 2025-12-29 21:23:41 +01:00
Author
Owner
Copy Link

@FliesLikeABrick commented on GitHub (Jun 21, 2024):

Huh, my option2 text was actually correct but github or markdown was eating the backslashes . I have edited it to be closer to what is being observed but my escaping may not 100% match the values seen in a reproduction (especially depending on whether someone is observing the raw database value vs that returned in the UI vs that returned by the API before or after being decoded as JSON)

@FliesLikeABrick commented on GitHub (Jun 21, 2024): Huh, my option2 text was actually correct but github or markdown was eating the backslashes . I have edited it to be closer to what is being observed but my escaping may not 100% match the values seen in a reproduction (especially depending on whether someone is observing the raw database value vs that returned in the UI vs that returned by the API before or after being decoded as JSON)
adam commented 2025-12-29 21:23:41 +01:00
Author
Owner
Copy Link

@FliesLikeABrick commented on GitHub (Jun 26, 2024):

Hi all, I am the reported of the original issue. This PR appears to fix the issue for null values only. Editing existing values, or creating a prefix with a default value both lead to escaped strings being placed in the field

@FliesLikeABrick commented on GitHub (Jun 26, 2024): Hi all, I am the reported of the original issue. This PR appears to fix the issue for null values only. Editing existing values, or creating a prefix with a default value both lead to escaped strings being placed in the field
adam commented 2025-12-29 21:23:41 +01:00
Author
Owner
Copy Link

@FliesLikeABrick commented on GitHub (Jul 17, 2024):

I have updated Option 2 in the description with better detail, and added code tags around the JSON/field values to prevent github/markdown from hiding the escaping which I think was creating some confusion.

@FliesLikeABrick commented on GitHub (Jul 17, 2024): I have updated Option 2 in the description with better detail, and added code tags around the JSON/field values to prevent github/markdown from hiding the escaping which I think was creating some confusion.
adam commented 2025-12-29 21:23:42 +01:00
Author
Owner
Copy Link

@FliesLikeABrick commented on GitHub (Jul 30, 2024):

The linked PR has been updated and now does prevent this issue from occurring. As for anyone experiencing this issue who needs to fix corrupted data - see https://github.com/FliesLikeABrick/netbox_fix_json

@FliesLikeABrick commented on GitHub (Jul 30, 2024): The linked PR has been updated and now does prevent this issue from occurring. As for anyone experiencing this issue who needs to fix corrupted data - see https://github.com/FliesLikeABrick/netbox_fix_json
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label severity: medium status: accepted type: bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#9860
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?