Use of FIELD_CHOICES for Status causes REST to fail with KeyError #7989

Closed
opened 2025-12-29 20:30:54 +01:00 by adam · 9 comments
Owner

Originally created by @moseisleydk on GitHub (May 3, 2023).

NetBox version

3.4.7

Python version

3.10

Steps to Reproduce

I have a running Netbox with a lot of devices, all in the normal statuses that comes with NetBox. I want to add statuses that match our CMDB. After chaning all status'es to match the CMDB, i want to remove the old statuses.

My configuration.py:

root@5411c1680df0:/opt/netbox/netbox/netbox# cat [configuration.py](http://configuration.py/) 

import importlib.util
import sys
from os import scandir
from os.path import abspath, isfile


def _filename(f):
    return [f.name](http://f.name/)


def _import(module_name, path, loaded_configurations):
    spec = importlib.util.spec_from_file_location("", path)
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)
    sys.modules[module_name] = module

    loaded_configurations.insert(0, module)

    print(f"🧬 loaded config '{path}'")


def read_configurations(config_module, config_dir, main_config):
    loaded_configurations = []

    main_config_path = abspath(f"{config_dir}/{main_config}.py")
    if isfile(main_config_path):
        _import(f"{config_module}.{main_config}", main_config_path, loaded_configurations)
    else:
        print(f"⚠️ Main configuration '{main_config_path}' not found.")

    with scandir(config_dir) as it:
        for f in sorted(it, key=_filename):
            if not f.is_file():
                continue

            if f.name.startswith("__"):
                continue

            if not f.name.endswith(".py"):
                continue

            if [f.name](http://f.name/) == f"{main_config}.py":
                continue

            if [f.name](http://f.name/) == f"{config_dir}.py":
                continue

            module_name = f"{config_module}.{f.name[:-len('.py')]}".replace(".", "_")
            _import(module_name, f.path, loaded_configurations)

    if len(loaded_configurations) == 0:
        print(f"‼️ No configuration files found in '{config_dir}'.")
        raise ImportError(f"No configuration files found in '{config_dir}'.")

    return loaded_configurations

_loaded_configurations = read_configurations(
    config_dir="/etc/netbox/config/",
    config_module="netbox.configuration",
    main_config="configuration",
)

def __getattr__(name):
    for config in _loaded_configurations:
        try:
            return getattr(config, name)
        except:
            pass
    raise AttributeError

def __dir__():
    names = []
    for config in _loaded_configurations:
        names.extend(config.__dir__())
    return names
from extras.validators import CustomValidator

class UniqueSerial(CustomValidator):

    def validate(self, instance):
        from dcim.models import Device

        if instance.serial and Device.objects.exclude(pk=instance.pk).filter(serial=instance.serial).exists():
            self.fail("The Serial number already exists in NetBox", field='serial')

class UniqueName(CustomValidator):

    def validate(self, instance):
        from dcim.models import Device

        if [instance.name](http://instance.name/) and Device.objects.exclude(pk=instance.pk).filter(name=instance.name).exists():
            self.fail("The (Host)Name already exists in NetBox", field='name')

CUSTOM_VALIDATORS = {
    'dcim.device': (
        UniqueSerial(),UniqueName(),
    )
}

I am adding following code to the bottom of configuration.py (according to https://github.com/netbox-community/netbox/issues/8054):

FIELD_CHOICES = {
'dcim.Device.status': (
('planned', 'Planned', 'cyan'),
('created', 'Created', 'grey'),
('production', 'Production', 'blue'),
('under decommision', 'Under Decommisioning', 'blue'),
('closed', 'Closed', 'blue'),
('in storage', 'In Storage', 'blue'),
('active','Active'),
('offline','Offline'),
('staged','staged'),
('failed','Failed'),
('inventory','Inventory'),
('decommisioning','Decomissioning')
)
}

And then restarting NetBox - no changes made to all the existing devices.

The UI looks fine, the Status dropdown shows all the old and new statuses and works fine.

Then, lokoking at at script that is calling REST at: https://netboxtest.netic.dk/api/dcim/devices/ - this gives:

Causes Error:

<class 'KeyError'>

'decommisioning'

Python version: 3.10.6
NetBox version: 3.4.7

Expected Behavior

That the REST https://netboxtest.netic.dk/api/dcim/devices/ returns a json list of devices.

Observed Behavior

NetBox throws error:

<class 'KeyError'>

'decommisioning'

Python version: 3.10.6
NetBox version: 3.4.7

Originally created by @moseisleydk on GitHub (May 3, 2023). ### NetBox version 3.4.7 ### Python version 3.10 ### Steps to Reproduce I have a running Netbox with a lot of devices, all in the normal statuses that comes with NetBox. I want to add statuses that match our CMDB. After chaning all status'es to match the CMDB, i want to remove the old statuses. My configuration.py: ``` root@5411c1680df0:/opt/netbox/netbox/netbox# cat [configuration.py](http://configuration.py/) import importlib.util import sys from os import scandir from os.path import abspath, isfile def _filename(f): return [f.name](http://f.name/) def _import(module_name, path, loaded_configurations): spec = importlib.util.spec_from_file_location("", path) module = importlib.util.module_from_spec(spec) spec.loader.exec_module(module) sys.modules[module_name] = module loaded_configurations.insert(0, module) print(f"🧬 loaded config '{path}'") def read_configurations(config_module, config_dir, main_config): loaded_configurations = [] main_config_path = abspath(f"{config_dir}/{main_config}.py") if isfile(main_config_path): _import(f"{config_module}.{main_config}", main_config_path, loaded_configurations) else: print(f"⚠️ Main configuration '{main_config_path}' not found.") with scandir(config_dir) as it: for f in sorted(it, key=_filename): if not f.is_file(): continue if f.name.startswith("__"): continue if not f.name.endswith(".py"): continue if [f.name](http://f.name/) == f"{main_config}.py": continue if [f.name](http://f.name/) == f"{config_dir}.py": continue module_name = f"{config_module}.{f.name[:-len('.py')]}".replace(".", "_") _import(module_name, f.path, loaded_configurations) if len(loaded_configurations) == 0: print(f"‼️ No configuration files found in '{config_dir}'.") raise ImportError(f"No configuration files found in '{config_dir}'.") return loaded_configurations _loaded_configurations = read_configurations( config_dir="/etc/netbox/config/", config_module="netbox.configuration", main_config="configuration", ) def __getattr__(name): for config in _loaded_configurations: try: return getattr(config, name) except: pass raise AttributeError def __dir__(): names = [] for config in _loaded_configurations: names.extend(config.__dir__()) return names from extras.validators import CustomValidator class UniqueSerial(CustomValidator): def validate(self, instance): from dcim.models import Device if instance.serial and Device.objects.exclude(pk=instance.pk).filter(serial=instance.serial).exists(): self.fail("The Serial number already exists in NetBox", field='serial') class UniqueName(CustomValidator): def validate(self, instance): from dcim.models import Device if [instance.name](http://instance.name/) and Device.objects.exclude(pk=instance.pk).filter(name=instance.name).exists(): self.fail("The (Host)Name already exists in NetBox", field='name') CUSTOM_VALIDATORS = { 'dcim.device': ( UniqueSerial(),UniqueName(), ) } ``` I am adding following code to the bottom of [configuration.py](http://configuration.py/) (according to https://github.com/netbox-community/netbox/issues/8054): ``` FIELD_CHOICES = { 'dcim.Device.status': ( ('planned', 'Planned', 'cyan'), ('created', 'Created', 'grey'), ('production', 'Production', 'blue'), ('under decommision', 'Under Decommisioning', 'blue'), ('closed', 'Closed', 'blue'), ('in storage', 'In Storage', 'blue'), ('active','Active'), ('offline','Offline'), ('staged','staged'), ('failed','Failed'), ('inventory','Inventory'), ('decommisioning','Decomissioning') ) } ``` And then restarting NetBox - no changes made to all the existing devices. The UI looks fine, the Status dropdown shows all the old and new statuses and works fine. Then, lokoking at at script that is calling REST at: https://netboxtest.netic.dk/api/dcim/devices/ - this gives: Causes Error: <class 'KeyError'> 'decommisioning' Python version: 3.10.6 NetBox version: 3.4.7 ### Expected Behavior That the REST https://netboxtest.netic.dk/api/dcim/devices/ returns a json list of devices. ### Observed Behavior NetBox throws error: <class 'KeyError'> 'decommisioning' Python version: 3.10.6 NetBox version: 3.4.7
adam added the type: bugstatus: revisions needed labels 2025-12-29 20:30:54 +01:00
adam closed this issue 2025-12-29 20:30:54 +01:00
Author
Owner

@jeremystretch commented on GitHub (May 3, 2023):

Thank you for opening a bug report. Unfortunately, the information you have provided is not sufficient for someone else to attempt to reproduce the reported behavior. Remember, each bug report must include detailed steps that someone else can follow on a clean, empty NetBox installation to reproduce the exact problem you're experiencing. These instructions should include the creation of any involved objects, any configuration changes, and complete accounting of the actions being taken. Also be sure that your report does not reference data on the public NetBox demo, as that is subject to change at any time by an outside party and cannot be relied upon for bug reports.

@jeremystretch commented on GitHub (May 3, 2023): Thank you for opening a bug report. Unfortunately, the information you have provided is not sufficient for someone else to attempt to reproduce the reported behavior. Remember, each bug report must include detailed steps that someone else can follow on a clean, empty NetBox installation to reproduce the exact problem you're experiencing. These instructions should include the creation of any involved objects, any configuration changes, and complete accounting of the actions being taken. Also be sure that your report does not reference data on the public NetBox demo, as that is subject to change at any time by an outside party and cannot be relied upon for bug reports.
Author
Owner

@moseisleydk commented on GitHub (May 3, 2023):

I guess that anyone could add the FIELD_CHOICES to configuration.py on a NetBox Instance, restart it and make the REST .... Its that not sufficient detailed steps?

@moseisleydk commented on GitHub (May 3, 2023): I guess that anyone could add the FIELD_CHOICES to configuration.py on a NetBox Instance, restart it and make the REST .... Its that not sufficient detailed steps?
Author
Owner
@moseisleydk commented on GitHub (May 3, 2023): https://netbox.dev/blog/posts/netbox-v32-custom-status-choices/ https://github.com/netbox-community/netbox/issues/8054
Author
Owner

@jeremystretch commented on GitHub (May 3, 2023):

First, please modify your post above to show the exact, valid Python code from your configuration file. Second, please extend the reproduction steps to indicate whether you have created a device prior to modifying the configuration, and if so, what value you have assigned for its status.

@jeremystretch commented on GitHub (May 3, 2023): First, please modify your post above to show the exact, valid Python code from your configuration file. Second, please extend the reproduction steps to indicate whether you have created a device prior to modifying the configuration, and if so, what value you have assigned for its status.
Author
Owner

@moseisleydk commented on GitHub (May 4, 2023):

Hope the improvement is sufficient :-)

@moseisleydk commented on GitHub (May 4, 2023): Hope the improvement is sufficient :-)
Author
Owner

@abhi1693 commented on GitHub (May 4, 2023):

I tried this with the updated steps and the API loads fine on v3.4.7.

@abhi1693 commented on GitHub (May 4, 2023): I tried this with the updated steps and the API loads fine on v3.4.7.
Author
Owner

@moseisleydk commented on GitHub (May 4, 2023):

Ok, sorry to hear that :-(

I see this in the log:



etbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \| Internal Server Error: /api/dcim/devices/
--
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \| Traceback (most recent call last):
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|   File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 56, in inner
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|     response = get_response(request)
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|   File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|     response = wrapped_callback(request, *callback_args, **callback_kwargs)
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|   File "/opt/netbox/venv/lib/python3.10/site-packages/django/views/decorators/csrf.py", line 55, in wrapped_view
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|     return view_func(*args, **kwargs)
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|   File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/viewsets.py", line 125, in view
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|     return self.dispatch(request, *args, **kwargs)
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|   File "/opt/netbox/netbox/netbox/api/viewsets/__init__.py", line 118, in dispatch
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|     return super().dispatch(request, *args, **kwargs)
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|   File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/views.py", line 509, in dispatch
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|     response = self.handle_exception(exc)
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|   File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/views.py", line 469, in handle_exception
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|     self.raise_uncaught_exception(exc)
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|   File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|     raise exc
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|   File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/views.py", line 506, in dispatch
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|     response = handler(request, *args, **kwargs)
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|   File "/opt/netbox/netbox/netbox/api/viewsets/__init__.py", line 149, in list
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|     return super().list(request, *args, **kwargs)
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|   File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/mixins.py", line 43, in list
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|     return self.get_paginated_response(serializer.data)
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|   File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 768, in data
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|     ret = super().data
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|   File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 253, in data
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|     self._data = self.to_representation(self.instance)
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|   File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 686, in to_representation
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|     return [
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|   File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 687, in <listcomp>
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|     self.child.to_representation(item) for item in iterable
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|   File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 522, in to_representation
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|     ret[field.field_name] = field.to_representation(attribute)
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|   File "/opt/netbox/netbox/netbox/api/fields.py", line 51, in to_representation
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \|     'label': self._choices[obj],
netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk    \| KeyError: 'decommissioning'

@moseisleydk commented on GitHub (May 4, 2023): Ok, sorry to hear that :-( I see this in the log: ``` etbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| Internal Server Error: /api/dcim/devices/ -- netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| Traceback (most recent call last): netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 56, in inner netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| response = get_response(request) netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| response = wrapped_callback(request, *callback_args, **callback_kwargs) netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| File "/opt/netbox/venv/lib/python3.10/site-packages/django/views/decorators/csrf.py", line 55, in wrapped_view netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| return view_func(*args, **kwargs) netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/viewsets.py", line 125, in view netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| return self.dispatch(request, *args, **kwargs) netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| File "/opt/netbox/netbox/netbox/api/viewsets/__init__.py", line 118, in dispatch netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| return super().dispatch(request, *args, **kwargs) netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/views.py", line 509, in dispatch netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| response = self.handle_exception(exc) netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/views.py", line 469, in handle_exception netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| self.raise_uncaught_exception(exc) netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| raise exc netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/views.py", line 506, in dispatch netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| response = handler(request, *args, **kwargs) netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| File "/opt/netbox/netbox/netbox/api/viewsets/__init__.py", line 149, in list netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| return super().list(request, *args, **kwargs) netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/mixins.py", line 43, in list netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| return self.get_paginated_response(serializer.data) netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 768, in data netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| ret = super().data netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 253, in data netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| self._data = self.to_representation(self.instance) netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 686, in to_representation netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| return [ netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 687, in <listcomp> netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| self.child.to_representation(item) for item in iterable netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 522, in to_representation netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| ret[field.field_name] = field.to_representation(attribute) netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| File "/opt/netbox/netbox/netbox/api/fields.py", line 51, in to_representation netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| 'label': self._choices[obj], netbox_test_netbox.1.ia54n8mn6axj@netbox01.netic.dk \| KeyError: 'decommissioning' ```
Author
Owner

@abhi1693 commented on GitHub (May 4, 2023):

If you notice correctly, the stackstrace does not match the FIELD_CHOICES and the error you had provided earlier.

From the stackstrace, it says decommissioning is not in FIELD_CHOICES which is true as you have provided decommisioning.

@abhi1693 commented on GitHub (May 4, 2023): If you notice correctly, the stackstrace does not match the FIELD_CHOICES and the error you had provided earlier. From the stackstrace, it says `decommissioning` is not in FIELD_CHOICES which is true as you have provided `decommisioning`.
Author
Owner

@jeremystretch commented on GitHub (May 4, 2023):

I think it's reasonable to conclude at this point that the error is the result of a misconfiguration. As no reproduction steps have been provided to demonstrate otherwise, I'm going to close this out.

@jeremystretch commented on GitHub (May 4, 2023): I think it's reasonable to conclude at this point that the error is the result of a misconfiguration. As no reproduction steps have been provided to demonstrate otherwise, I'm going to close this out.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#7989