starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-12 05:20:31 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Serializer sets cached_property custom_fields of CustomFieldMixin #7524

New Issue
Closed
opened 2025-12-29 20:24:43 +01:00 by adam · 1 comment
adam commented 2025-12-29 20:24:43 +01:00
Owner
Copy Link

Originally created by @moonrail on GitHub (Jan 16, 2023).

Originally assigned to: @arthanson on GitHub.

NetBox version

v3.4.2

Python version

3.11

Steps to Reproduce

  1. Create a Custom Field "test" of type "Text" for a model

  2. Create a Custom Validator for it - require the Custom Field "test" to be of a valid value (e.g. IP-Address)

  3. Configure Custom Validator for the model, that fails the Validation, if no value is given for Custom Field "test"

  4. UI: Try to create a instance of the model using this Custom Field, but provide an invalid value

  5. Notice the ValidationError in the UI

  6. API: Try to create a instance of the model using this Custom Field, but provide an invalid value

  7. Notice no ValidationError as Response, but the Custom Field value being applied as is

Expected Behavior

API raises ValidationError as does the UI.

Observed Behavior

API accepts invalid Value.

We've noticed this issue upon running our automatic API tests for our own Custom Fields & Custom Validators.

Upon further investigation we've found that:

  • instance.cf does not yield expected Custom Field data, when used in a Custom Validator, but an empty dict
  • instance.custom_fields is usually a cached_property on CustomFieldsMixin, but is overwritten in API serializer
  • instance.custom_field_data always returns expected Custom Field data (as its the raw model data)

Here an example while using a simple print of each attribute in a CustomValidator that shows the difference in UI & API datatypes as described above:

# this was a simple "Save"-click in UI on the Prefix with an invalid CustomField value
instance.custom_fields type: <class 'utilities.querysets.RestrictedQuerySet'>
instance.custom_fields: <RestrictedQuerySet [<CustomField: Default gateway ip>]>
instance.cf: {'default_gateway_ip': '564'}
instance.custom_field_data: {'default_gateway_ip': '564'}
netbox.views.ObjectEditView: DEBUG    Form validation failed
netbox.config: DEBUG    Cleared configuration
[16/Jan/2023 15:49:59] "POST /ipam/prefixes/1/edit/ HTTP/1.1" 200 98483

# this was a PATCH request via SwaggerUI on the Prefix with an invalid CustomField value
netbox.api.views.ModelViewSet: DEBUG    Using serializer <class 'ipam.api.serializers.PrefixSerializer'>
netbox.config: DEBUG    No previous configuration found in database; proceeding with default values
netbox.config: DEBUG    Initialized configuration
instance.custom_fields type: <class 'dict'>
instance.custom_fields: {'default_gateway_ip': None}
instance.cf: {}
instance.custom_field_data: {'default_gateway_ip': '564'}
netbox.api.views.ModelViewSet: INFO     Updating prefix 127.0.10.0/24 (PK: 1)
netbox.config: DEBUG    Cleared configuration
[16/Jan/2023 15:50:06] "PATCH /api/ipam/prefixes/1/ HTTP/1.1" 200 485

This commit introduced custom_fields as a cached_property: ea6d86e6c4

But here you can see, instance.custom_fields being overwritten in the API serializer:
https://github.com/netbox-community/netbox/blob/v3.4.2/netbox/netbox/api/serializers/features.py#L42

The cached_property of functools provides a Setter.
This is why the error was not noticed earlier, I guess. A regular property would be (without an explicitly defined Setter) readonly and the following error would be noticable as a HTTP 500:

  File "/home/jg/.local/lib/python3.11/site-packages/rest_framework/generics.py", line 110, in get_serializer
    return serializer_class(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/dev/netbox/netbox/netbox/api/serializers/features.py", line 39, in __init__
    self._populate_custom_fields(self.instance, fields)
  File "/var/dev/netbox/netbox/netbox/api/serializers/features.py", line 42, in _populate_custom_fields
    instance.custom_fields = {}
    ^^^^^^^^^^^^^^^^^^^^^^
AttributeError: property 'custom_fields' of 'Prefix' object has no setter
Internal Server Error: /api/ipam/prefixes/1/

As instance.custom_fields is overwritten, it is cached by the property and returned in later usage.
instance.cf uses instance.custom_fields and expects a QuerySet of used CustomFields:

I would guess, that this: https://github.com/netbox-community/netbox/blob/v3.4.2/netbox/netbox/api/serializers/features.py#L42
Should be probably:

    def _populate_custom_field_data(self, instance, custom_fields):
        instance.custom_field_data = {}
        for field in custom_fields:
            instance.custom_field_data[field.name] = instance.cf.get(field.name)
Originally created by @moonrail on GitHub (Jan 16, 2023). Originally assigned to: @arthanson on GitHub. ### NetBox version v3.4.2 ### Python version 3.11 ### Steps to Reproduce 1. Create a Custom Field "test" of type "Text" for a model 2. Create a Custom Validator for it - require the Custom Field "test" to be of a valid value (e.g. IP-Address) 3. Configure Custom Validator for the model, that fails the Validation, if no value is given for Custom Field "test" 1. UI: Try to create a instance of the model using this Custom Field, but provide an invalid value 2. Notice the ValidationError in the UI 1. API: Try to create a instance of the model using this Custom Field, but provide an invalid value 2. Notice no ValidationError as Response, but the Custom Field value being applied as is ### Expected Behavior API raises ValidationError as does the UI. ### Observed Behavior API accepts invalid Value. We've noticed this issue upon running our automatic API tests for our own Custom Fields & Custom Validators. Upon further investigation we've found that: - `instance.cf` does not yield expected Custom Field data, when used in a Custom Validator, but an empty `dict` - `instance.custom_fields` is usually a `cached_property` on `CustomFieldsMixin`, but is overwritten in API serializer - `instance.custom_field_data` always returns expected Custom Field data (as its the raw model data) Here an example while using a simple `print` of each attribute in a CustomValidator that shows the difference in UI & API datatypes as described above: ```bash # this was a simple "Save"-click in UI on the Prefix with an invalid CustomField value instance.custom_fields type: <class 'utilities.querysets.RestrictedQuerySet'> instance.custom_fields: <RestrictedQuerySet [<CustomField: Default gateway ip>]> instance.cf: {'default_gateway_ip': '564'} instance.custom_field_data: {'default_gateway_ip': '564'} netbox.views.ObjectEditView: DEBUG Form validation failed netbox.config: DEBUG Cleared configuration [16/Jan/2023 15:49:59] "POST /ipam/prefixes/1/edit/ HTTP/1.1" 200 98483 # this was a PATCH request via SwaggerUI on the Prefix with an invalid CustomField value netbox.api.views.ModelViewSet: DEBUG Using serializer <class 'ipam.api.serializers.PrefixSerializer'> netbox.config: DEBUG No previous configuration found in database; proceeding with default values netbox.config: DEBUG Initialized configuration instance.custom_fields type: <class 'dict'> instance.custom_fields: {'default_gateway_ip': None} instance.cf: {} instance.custom_field_data: {'default_gateway_ip': '564'} netbox.api.views.ModelViewSet: INFO Updating prefix 127.0.10.0/24 (PK: 1) netbox.config: DEBUG Cleared configuration [16/Jan/2023 15:50:06] "PATCH /api/ipam/prefixes/1/ HTTP/1.1" 200 485 ``` This commit introduced `custom_fields` as a `cached_property`: ea6d86e6c4bb6037465410db6205a7471bc81a6c But here you can see, `instance.custom_fields` being overwritten in the API serializer: https://github.com/netbox-community/netbox/blob/v3.4.2/netbox/netbox/api/serializers/features.py#L42 The `cached_property` of functools provides a Setter. This is why the error was not noticed earlier, I guess. A regular `property` would be (without an explicitly defined Setter) readonly and the following error would be noticable as a HTTP 500: ```bash File "/home/jg/.local/lib/python3.11/site-packages/rest_framework/generics.py", line 110, in get_serializer return serializer_class(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/dev/netbox/netbox/netbox/api/serializers/features.py", line 39, in __init__ self._populate_custom_fields(self.instance, fields) File "/var/dev/netbox/netbox/netbox/api/serializers/features.py", line 42, in _populate_custom_fields instance.custom_fields = {} ^^^^^^^^^^^^^^^^^^^^^^ AttributeError: property 'custom_fields' of 'Prefix' object has no setter Internal Server Error: /api/ipam/prefixes/1/ ``` As `instance.custom_fields` is overwritten, it is cached by the property and returned in later usage. `instance.cf` uses `instance.custom_fields` and expects a QuerySet of used CustomFields: - https://github.com/netbox-community/netbox/blob/v3.4.2/netbox/netbox/models/features.py#L150 - https://github.com/netbox-community/netbox/blob/v3.4.2/netbox/netbox/models/features.py#L161 But instead it gets an empty dictionary, as `instance.custom_fields` was overwritten with it. Therefore `instance.cf` itself also yields no data, as it has nothing to iterate over. I would guess, that this: https://github.com/netbox-community/netbox/blob/v3.4.2/netbox/netbox/api/serializers/features.py#L42 Should be probably: ```python def _populate_custom_field_data(self, instance, custom_fields): instance.custom_field_data = {} for field in custom_fields: instance.custom_field_data[field.name] = instance.cf.get(field.name) ```
adam added the type: bugstatus: accepted labels 2025-12-29 20:24:43 +01:00
adam closed this issue 2025-12-29 20:24:43 +01:00
adam commented 2025-12-29 20:24:43 +01:00
Author
Owner
Copy Link

@arthanson commented on GitHub (Apr 25, 2023):

@moonrail tried this in the latest NetBox 3.4.8 and the validation error was returned correctly, so closing - please update and try again and let me know if you are still seeing an issue. Steps:

  1. created a custom field for site objects, text field need to match regex to match string "testit"
  2. performed API call to create a site with the custom-field to be "ttt"
  3. validation error returned below:

The request failed with code 400 Bad Request: {'all': ["Invalid value for custom field 'cfsitetest': Value must match regex 'testit'"]}

Also tried to update a given site with an invalid custom-field and it also returned the validation error correctly.

@arthanson commented on GitHub (Apr 25, 2023): @moonrail tried this in the latest NetBox 3.4.8 and the validation error was returned correctly, so closing - please update and try again and let me know if you are still seeing an issue. Steps: 1. created a custom field for site objects, text field need to match regex to match string "testit" 2. performed API call to create a site with the custom-field to be "ttt" 3. validation error returned below: The request failed with code 400 Bad Request: {'__all__': ["Invalid value for custom field 'cfsitetest': Value must match regex 'testit'"]} Also tried to update a given site with an invalid custom-field and it also returned the validation error correctly.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label status: accepted type: bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#7524
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?