starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-14 06:13:32 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Custom Field Defaults not respected & different handling in UI & API #5055

New Issue
Closed
opened 2025-12-29 19:23:36 +01:00 by adam · 5 comments
adam commented 2025-12-29 19:23:36 +01:00
Owner
Copy Link

Originally created by @moonrail on GitHub (Jul 8, 2021).

NetBox version

v2.11.8

Python version

3.8

Steps to Reproduce

Let's say we have the following Custom Fields:

  • some_selection: Muliple-Selection, not required, no default
  • some_boolean: Boolean, not required, default: false
  • some_text: Text, not required, no default

On creating via the API with no or empty custom fields passed we receive from the API:

"custom_fields": {
    "some_selection": null,
    "some_boolean": null,
    "some_text": null
},

On the Database the Custom Fields look like this:
{}

On the UI they are rendered as:

  • some_selection: []
  • some_boolean: ---
  • some_text: ---

When saving the Model via PUT/PATCH without any Changes in the API, the response looks like this:

"custom_fields": {
    "some_selection": null,
    "some_boolean": null,
    "some_text": null
},

When saving the Model in UI without any Changes or touching any of the UI-Fields, the Custom Fields in the Database look like this:
{"some_selection": [], "some_boolean": null, "some_text": ""}

The API now provides:

"custom_fields": {
    "some_selection": [],
    "some_boolean": null,
    "some_text": ""
},

Expected Behavior

  1. All Custom Field keys should be present in the database (or at least handled completely the same in UI & API, so that there are no type-defaults applied through the UI)
  2. Default Values should be set on Custom Fields, if they are missing from provided Custom Fields by the User or null
  3. "Required" should only be enforced, when no value is given by either the User or the Administrator via "Default"

Suggestion:
In ec5ed17860/netbox/netbox/models.py (L105)
Move required-handling above validations and ensure all custom fields are present in custom_field_data with their defaults (even if they're None).

    def clean(self):
        super().clean()
        from extras.models import CustomField

        custom_fields = {cf.name: cf for cf in CustomField.objects.get_for_model(self)}

        for cf in custom_fields.values():
            # ensure custom_field_data contains all keys
            # also ensure any given default values
            if self.custom_field_data.get(cf.name) is None:
                self.custom_field_data[cf.name] = cf.default

            # ensure required fields are set
            if cf.required and self.custom_field_data.get(cf.name) is None:
                raise ValidationError(f"Missing value for required custom field '{cf.name}'.")

        # Validate all field values
        for field_name, value in self.custom_field_data.items():
            if field_name not in custom_fields:
                raise ValidationError(f"Unknown field name '{field_name}' in custom field data.")
            try:
                custom_fields[field_name].validate(value)
            except ValidationError as e:
                raise ValidationError(f"Invalid value for custom field '{field_name}': {e.message}")

Additionally any setting of Type-Default-Values should be removed from the UI and be solely handled in the CustomFieldMixin.

Observed Behavior

  • UI sets type-defaults and suggests, Users would have provided anything (e.g. empty string "" for a text field)
  • API does not set anything
  • Database receives only Custom Fields & Values, that are set, so there may always be a gap between model definition and model instance on Custom Fields
    • this makes handling Custom Fields in Views, API and own Plugin unnecessary bloated
Originally created by @moonrail on GitHub (Jul 8, 2021). ### NetBox version v2.11.8 ### Python version 3.8 ### Steps to Reproduce Let's say we have the following Custom Fields: - some_selection: Muliple-Selection, not required, no default - some_boolean: Boolean, not required, default: false - some_text: Text, not required, no default On creating via the API with no or empty custom fields passed we receive from the API: ```json "custom_fields": { "some_selection": null, "some_boolean": null, "some_text": null }, ``` On the Database the Custom Fields look like this: `{}` On the UI they are rendered as: - some_selection: [] - some_boolean: --- - some_text: --- When saving the Model via PUT/PATCH without any Changes in the API, the response looks like this: ```json "custom_fields": { "some_selection": null, "some_boolean": null, "some_text": null }, ``` When saving the Model in UI without any Changes or touching any of the UI-Fields, the Custom Fields in the Database look like this: `{"some_selection": [], "some_boolean": null, "some_text": ""}` The API now provides: ```json "custom_fields": { "some_selection": [], "some_boolean": null, "some_text": "" }, ``` ### Expected Behavior 1. All Custom Field keys should be present in the database (or at least handled completely the same in UI & API, so that there are no type-defaults applied through the UI) 2. Default Values should be set on Custom Fields, if they are missing from provided Custom Fields by the User or `null` 3. "Required" should only be enforced, when no value is given by either the User or the Administrator via "Default" Suggestion: In https://github.com/netbox-community/netbox/blob/ec5ed17860542544ba4205cdcbf651e93c1e6ca5/netbox/netbox/models.py#L105 Move required-handling above validations and ensure all custom fields are present in custom_field_data with their defaults (even if they're None). ```python def clean(self): super().clean() from extras.models import CustomField custom_fields = {cf.name: cf for cf in CustomField.objects.get_for_model(self)} for cf in custom_fields.values(): # ensure custom_field_data contains all keys # also ensure any given default values if self.custom_field_data.get(cf.name) is None: self.custom_field_data[cf.name] = cf.default # ensure required fields are set if cf.required and self.custom_field_data.get(cf.name) is None: raise ValidationError(f"Missing value for required custom field '{cf.name}'.") # Validate all field values for field_name, value in self.custom_field_data.items(): if field_name not in custom_fields: raise ValidationError(f"Unknown field name '{field_name}' in custom field data.") try: custom_fields[field_name].validate(value) except ValidationError as e: raise ValidationError(f"Invalid value for custom field '{field_name}': {e.message}") ``` Additionally any setting of Type-Default-Values should be removed from the UI and be solely handled in the CustomFieldMixin. ### Observed Behavior - UI sets type-defaults and suggests, Users would have provided anything (e.g. empty string "" for a text field) - API does not set anything - Database receives only Custom Fields & Values, that are set, so there may always be a gap between model definition and model instance on Custom Fields - this makes handling Custom Fields in Views, API and own Plugin unnecessary bloated
adam added the type: bug label 2025-12-29 19:23:36 +01:00
adam closed this issue 2025-12-29 19:23:36 +01:00
adam commented 2025-12-29 19:23:36 +01:00
Author
Owner
Copy Link

@moonrail commented on GitHub (Jul 9, 2021):

When creating a Custom Field, or adding a default to an existing Custom Field, it is not added to all Database-Objects, therefore additional logic would be required for ensuring Custom Fields & their default values on any GET either via UI or API.
Most likely here?
ec5ed17860/netbox/netbox/models.py (L101)
Maybe like this, but its not pretty as is:

return OrderedDict([
    (
        field,
        (
            self.custom_field_data.get(field.name) if self.custom_field_data.get(field.name) is not None else field.default
        )
    ) for field in fields
])
@moonrail commented on GitHub (Jul 9, 2021): When creating a Custom Field, or adding a default to an existing Custom Field, it is not added to all Database-Objects, therefore additional logic would be required for ensuring Custom Fields & their default values on any GET either via UI or API. Most likely here? https://github.com/netbox-community/netbox/blob/ec5ed17860542544ba4205cdcbf651e93c1e6ca5/netbox/netbox/models.py#L101 Maybe like this, but its not pretty as is: ```python return OrderedDict([ ( field, ( self.custom_field_data.get(field.name) if self.custom_field_data.get(field.name) is not None else field.default ) ) for field in fields ]) ```
adam commented 2025-12-29 19:23:36 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Aug 16, 2021):

On the Database the Custom Fields look like this:
{}

I'm not able to reproduce this on v2.11.11. The boolean field gets assigned a value of False per the custom field's configured default. The other two fields are set to be none because no default value was specified.

curl -X POST \
-H "Authorization: Token $TOKEN" \
-H "Content-Type: application/json" \
-H "Accept: application/json; indent=4" \
http://localhost:8000/api/ipam/vrfs/ \
--data '{"name": "B", "description": "Created via API"}'

{
    "id": 6,
    "url": "http://localhost:8000/api/ipam/vrfs/6/",
    "display": "B",
    "name": "B",
    ...
    "custom_fields": {
        "some_boolean": false,
        "some_selection": null,
        "some_text": null
    },

This is confirmed by inspecting the instance directly in nbshell:

>>> VRF.objects.get(pk=6).custom_field_data
{'some_text': None, 'some_boolean': False, 'some_selection': None}

I'm going to close this out as NetBox appears to be behaving as expected. If you're able to reproduce behavior which you believe is erroneous on v2.11.11 or later, please feel free to submit a new bug report. (When doing so, it's best to limit the scope of the report as much as possible. E.g. stick with creating a single field for your example, and open separate issues for any related reports.)

@jeremystretch commented on GitHub (Aug 16, 2021): > On the Database the Custom Fields look like this: > ```{}``` I'm not able to reproduce this on v2.11.11. The boolean field gets assigned a value of `False` per the custom field's configured default. The other two fields are set to be none because no default value was specified. ``` curl -X POST \ -H "Authorization: Token $TOKEN" \ -H "Content-Type: application/json" \ -H "Accept: application/json; indent=4" \ http://localhost:8000/api/ipam/vrfs/ \ --data '{"name": "B", "description": "Created via API"}' { "id": 6, "url": "http://localhost:8000/api/ipam/vrfs/6/", "display": "B", "name": "B", ... "custom_fields": { "some_boolean": false, "some_selection": null, "some_text": null }, ``` This is confirmed by inspecting the instance directly in `nbshell`: ```python >>> VRF.objects.get(pk=6).custom_field_data {'some_text': None, 'some_boolean': False, 'some_selection': None} ``` I'm going to close this out as NetBox appears to be behaving as expected. If you're able to reproduce behavior which you believe is erroneous on v2.11.11 or later, please feel free to submit a new bug report. (When doing so, it's best to limit the scope of the report as much as possible. E.g. stick with creating a single field for your example, and open separate issues for any related reports.)
adam commented 2025-12-29 19:23:37 +01:00
Author
Owner
Copy Link

@maximumG commented on GitHub (Aug 20, 2021):

@jeremystretch I believe there was a misunderstanding with this issue. I will try to explain it again because I experienced this as well on netbox 2.11.11.

Observed behavior

When we define custom fields on a model with a default value of null, the behavior when creating new objects in Netbox is different when using the UI (django form) and the API (DRF):

  • With django UI , every custom fields not explicitly filled out in the form are automatically added as empty string ''. This is especially true for CharFields and ChoiceField
  • With DRF every custom fields not explicitly filled out are automatically added with their accurate default value

Potential root cause

This difference in behavior may be due to the way django handles empty value in form for CharFields and ChoiceFields: https://docs.djangoproject.com/en/3.2/ref/forms/fields/#django.forms.CharField.empty_value. As you might see from the django documentation, any CharFields or ChoiceFields that are not filled out gets a default value of empty string. This empty string is then automatically added to the corresponding Customfield - instead of the default value that was defined during CustomField creation.

Proposed Fix

In order to allow Django custom fields to honor the default value when rendered as Form, I propose to:

  1. Explicitly define an empty_value when creating forms.CharField out of a a Text CustomField with the default value set on the Custom field definition: 10847e2956/netbox/extras/models/customfields.py (L273-L281)
        # Text
        else:
            field = forms.CharField(max_length=255, required=required, initial=initial, empty_value=initial)
            if self.validation_regex:
                field.validators = [
                    RegexValidator(
                        regex=self.validation_regex,
                        message=mark_safe(f"Values must match this regex: <code>{self.validation_regex}</code>")
                    )
                ]
  1. Change the Field type for ChoiceField to TypedChoiceField for Select CustomField with the default value set on the Custom field definition: 10847e2956/netbox/extras/models/customfields.py (L257-L261)
            if self.type == CustomFieldTypeChoices.TYPE_SELECT:
                field_class = CSVChoiceField if for_csv_import else forms.ChoiceField
                field = field_class(
                    choices=choices, required=required, initial=initial, empty_value=initial, widget=StaticSelect2()
                )
@maximumG commented on GitHub (Aug 20, 2021): @jeremystretch I believe there was a misunderstanding with this issue. I will try to explain it again because I experienced this as well on netbox 2.11.11. ## Observed behavior When we define custom fields on a model with a default value of null, the behavior when creating new objects in Netbox is different when using the UI (django form) and the API (DRF): * With django UI , every custom fields not explicitly filled out in the form are automatically added as empty string `''`. This is especially true for CharFields and ChoiceField * With DRF every custom fields not explicitly filled out are automatically added with their accurate default value ## Potential root cause This difference in behavior may be due to the way django handles empty value in form for CharFields and ChoiceFields: https://docs.djangoproject.com/en/3.2/ref/forms/fields/#django.forms.CharField.empty_value. As you might see from the django documentation, any CharFields or ChoiceFields that are not filled out gets a default value of empty string. This empty string is then automatically added to the corresponding Customfield - instead of the default value that was defined during CustomField creation. ## Proposed Fix In order to allow Django custom fields to honor the default value when rendered as Form, I propose to: 1. Explicitly define an `empty_value` when creating forms.CharField out of a a Text CustomField with the default value set on the Custom field definition: https://github.com/netbox-community/netbox/blob/10847e2956be50ea62f476fa68f979178fae3260/netbox/extras/models/customfields.py#L273-L281 ```python # Text else: field = forms.CharField(max_length=255, required=required, initial=initial, empty_value=initial) if self.validation_regex: field.validators = [ RegexValidator( regex=self.validation_regex, message=mark_safe(f"Values must match this regex: <code>{self.validation_regex}</code>") ) ] ``` 2. Change the Field type for `ChoiceField `to `TypedChoiceField `for Select CustomField with the default value set on the Custom field definition: https://github.com/netbox-community/netbox/blob/10847e2956be50ea62f476fa68f979178fae3260/netbox/extras/models/customfields.py#L257-L261 ```python if self.type == CustomFieldTypeChoices.TYPE_SELECT: field_class = CSVChoiceField if for_csv_import else forms.ChoiceField field = field_class( choices=choices, required=required, initial=initial, empty_value=initial, widget=StaticSelect2() ) ```
adam commented 2025-12-29 19:23:37 +01:00
Author
Owner
Copy Link

@moonrail commented on GitHub (Aug 23, 2021):

@jeremystretch
I can confirm, that this issue persists and is not percievable by only using the API.
As mentioned, the API has a Wrapper, that sets the default value in its response for custom fields, if the custom field does not exist/is not set yet on the object.
The Frontend does not do this and actively PUTs/POSTs None/Empty String, just as @maximumG described, even if the User did not set anything.
Via using the Django-GET-Methods you may not see what I've described as the state in the database, as the Django-Models do have the above mentioned Wrapper-mechanism.

@moonrail commented on GitHub (Aug 23, 2021): @jeremystretch I can confirm, that this issue persists and is not percievable by only using the API. As mentioned, the API has a Wrapper, that sets the default value in its response for custom fields, if the custom field does not exist/is not set yet on the object. The Frontend does not do this and actively PUTs/POSTs None/Empty String, just as @maximumG described, even if the User did not set anything. Via using the Django-GET-Methods you may not see what I've described as the state in the database, as the Django-Models do have the above mentioned Wrapper-mechanism.
adam commented 2025-12-29 19:23:37 +01:00
Author
Owner
Copy Link

@maximumG commented on GitHub (Aug 25, 2021):

The issue has been actually fixed in Netbox 2.11.12 and addressed by #5968.

@maximumG commented on GitHub (Aug 25, 2021): The issue has been actually fixed in Netbox 2.11.12 and addressed by #5968.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label type: bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#5055
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?