ALLOWED_URL_SCHEMES not honoured for custom fields of type 'URL' #5927

Closed
opened 2025-12-29 19:34:20 +01:00 by adam · 3 comments
Owner

Originally created by @candlerb on GitHub (Jan 11, 2022).

Originally assigned to: @jeremystretch on GitHub.

NetBox version

v3.1.5

Python version

3.8

Steps to Reproduce

  1. Ensure ALLOWED_URL_SCHEMES is not set in configuration.py
  2. Create a custom field
    • name 'blah'
    • type 'URL'
    • content types 'DCIM > Device'
  3. Browse to a device
  4. Enter URL ssh://foo into the blah custom field (also try ssh://foo/, ssh://foo/bar)

Expected Behavior

URL to be accepted, since the default is:

        default=(
            'file', 'ftp', 'ftps', 'http', 'https', 'irc', 'mailto', 'sftp', 'ssh', 'tel', 'telnet', 'tftp', 'vnc',
            'xmpp',
        ),

Observed Behavior

Errors

Blah

  • Enter a valid URL.

Note also that http://foo is accepted but file://foo is not

Originally created by @candlerb on GitHub (Jan 11, 2022). Originally assigned to: @jeremystretch on GitHub. ### NetBox version v3.1.5 ### Python version 3.8 ### Steps to Reproduce 0. Ensure `ALLOWED_URL_SCHEMES` is not set in configuration.py 1. Create a custom field * name 'blah' * type 'URL' * content types 'DCIM > Device' 2. Browse to a device 3. Enter URL `ssh://foo` into the blah custom field (also try `ssh://foo/`, `ssh://foo/bar`) ### Expected Behavior URL to be accepted, since the default is: ``` default=( 'file', 'ftp', 'ftps', 'http', 'https', 'irc', 'mailto', 'sftp', 'ssh', 'tel', 'telnet', 'tftp', 'vnc', 'xmpp', ), ``` ### Observed Behavior Errors ------ **Blah** * Enter a valid URL. ------ Note also that `http://foo` is accepted but `file://foo` is not
adam added the type: bugstatus: accepted labels 2025-12-29 19:34:20 +01:00
adam closed this issue 2025-12-29 19:34:20 +01:00
Author
Owner

@jeremystretch commented on GitHub (Jan 11, 2022):

Thanks @candlerb!

@jeremystretch commented on GitHub (Jan 11, 2022): Thanks @candlerb!
Author
Owner

@candlerb commented on GitHub (Jan 11, 2022):

This looks like a silly bug. But when I make the obvious change (EDIT: first version was wrong):

--- a/netbox/utilities/validators.py
+++ b/netbox/utilities/validators.py
@@ -23,8 +23,7 @@ class EnhancedURLValidator(URLValidator):

     def __init__(self, schemes=None, **kwargs):
         super().__init__(**kwargs)
-        if schemes is not None:
-            self.schemes = get_config().ALLOWED_URL_SCHEMES
+        self.schemes = schemes or get_config().ALLOWED_URL_SCHEMES


 class ExclusionValidator(BaseValidator):

Netbox fails to start due to a circular import:

Jan 11 20:07:59 netbox3 gunicorn[101352]: [2022-01-11 20:07:59 +0000] [101352] [ERROR] Exception in worker process
Jan 11 20:07:59 netbox3 gunicorn[101352]: Traceback (most recent call last):
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
Jan 11 20:07:59 netbox3 gunicorn[101352]:     worker.init_process()
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/venv/lib/python3.8/site-packages/gunicorn/workers/gthread.py", line 92, in init_process
Jan 11 20:07:59 netbox3 gunicorn[101352]:     super().init_process()
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/venv/lib/python3.8/site-packages/gunicorn/workers/base.py", line 134, in init_process
Jan 11 20:07:59 netbox3 gunicorn[101352]:     self.load_wsgi()
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/venv/lib/python3.8/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
Jan 11 20:07:59 netbox3 gunicorn[101352]:     self.wsgi = self.app.wsgi()
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/venv/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi
Jan 11 20:07:59 netbox3 gunicorn[101352]:     self.callable = self.load()
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/venv/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
Jan 11 20:07:59 netbox3 gunicorn[101352]:     return self.load_wsgiapp()
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/venv/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
Jan 11 20:07:59 netbox3 gunicorn[101352]:     return util.import_app(self.app_uri)
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/venv/lib/python3.8/site-packages/gunicorn/util.py", line 359, in import_app
Jan 11 20:07:59 netbox3 gunicorn[101352]:     mod = importlib.import_module(module)
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
Jan 11 20:07:59 netbox3 gunicorn[101352]:     return _bootstrap._gcd_import(name[level:], package, level)
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "<frozen importlib._bootstrap_external>", line 848, in exec_module
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/netbox/netbox/wsgi.py", line 7, in <module>
Jan 11 20:07:59 netbox3 gunicorn[101352]:     application = get_wsgi_application()
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/venv/lib/python3.8/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application
Jan 11 20:07:59 netbox3 gunicorn[101352]:     django.setup(set_prefix=False)
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/venv/lib/python3.8/site-packages/django/__init__.py", line 24, in setup
Jan 11 20:07:59 netbox3 gunicorn[101352]:     apps.populate(settings.INSTALLED_APPS)
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/venv/lib/python3.8/site-packages/django/apps/registry.py", line 114, in populate
Jan 11 20:07:59 netbox3 gunicorn[101352]:     app_config.import_models()
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/venv/lib/python3.8/site-packages/django/apps/config.py", line 301, in import_models
Jan 11 20:07:59 netbox3 gunicorn[101352]:     self.models_module = import_module(models_module_name)
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
Jan 11 20:07:59 netbox3 gunicorn[101352]:     return _bootstrap._gcd_import(name[level:], package, level)
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "<frozen importlib._bootstrap_external>", line 848, in exec_module
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/netbox/circuits/models/__init__.py", line 1, in <module>
Jan 11 20:07:59 netbox3 gunicorn[101352]:     from .circuits import *
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/netbox/circuits/models/circuits.py", line 7, in <module>
Jan 11 20:07:59 netbox3 gunicorn[101352]:     from dcim.models import LinkTermination
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/netbox/dcim/models/__init__.py", line 1, in <module>
Jan 11 20:07:59 netbox3 gunicorn[101352]:     from .cables import *
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/netbox/dcim/models/cables.py", line 16, in <module>
Jan 11 20:07:59 netbox3 gunicorn[101352]:     from utilities.fields import ColorField
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/netbox/utilities/fields.py", line 5, in <module>
Jan 11 20:07:59 netbox3 gunicorn[101352]:     from .forms import ColorSelect
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/netbox/utilities/forms/__init__.py", line 2, in <module>
Jan 11 20:07:59 netbox3 gunicorn[101352]:     from .fields import *
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/netbox/utilities/forms/fields.py", line 107, in <module>
Jan 11 20:07:59 netbox3 gunicorn[101352]:     class LaxURLField(forms.URLField):
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/netbox/utilities/forms/fields.py", line 112, in LaxURLField
Jan 11 20:07:59 netbox3 gunicorn[101352]:     default_validators = [EnhancedURLValidator()]
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/netbox/utilities/validators.py", line 27, in __init__
Jan 11 20:07:59 netbox3 gunicorn[101352]:     self.schemes = get_config().ALLOWED_URL_SCHEMES
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/netbox/netbox/config/__init__.py", line 27, in get_config
Jan 11 20:07:59 netbox3 gunicorn[101352]:     _thread_locals.config = Config()
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/netbox/netbox/config/__init__.py", line 49, in __init__
Jan 11 20:07:59 netbox3 gunicorn[101352]:     self._populate_from_db()
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/netbox/netbox/config/__init__.py", line 77, in _populate_from_db
Jan 11 20:07:59 netbox3 gunicorn[101352]:     from extras.models import ConfigRevision
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/netbox/extras/models/__init__.py", line 2, in <module>
Jan 11 20:07:59 netbox3 gunicorn[101352]:     from .configcontexts import ConfigContext, ConfigContextModel
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/netbox/extras/models/configcontexts.py", line 7, in <module>
Jan 11 20:07:59 netbox3 gunicorn[101352]:     from extras.querysets import ConfigContextQuerySet
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/netbox/extras/querysets.py", line 4, in <module>
Jan 11 20:07:59 netbox3 gunicorn[101352]:     from extras.models.tags import TaggedItem
Jan 11 20:07:59 netbox3 gunicorn[101352]:   File "/opt/netbox/netbox/extras/models/tags.py", line 9, in <module>
Jan 11 20:07:59 netbox3 gunicorn[101352]:     from utilities.fields import ColorField
Jan 11 20:07:59 netbox3 gunicorn[101352]: ImportError: cannot import name 'ColorField' from partially initialized module 'utilities.fields' (most likely due to a circular import) (/opt/netbox/netbox/utilities/fields.py)

@candlerb commented on GitHub (Jan 11, 2022): This looks like a silly bug. But when I make the obvious change (EDIT: first version was wrong): ``` --- a/netbox/utilities/validators.py +++ b/netbox/utilities/validators.py @@ -23,8 +23,7 @@ class EnhancedURLValidator(URLValidator): def __init__(self, schemes=None, **kwargs): super().__init__(**kwargs) - if schemes is not None: - self.schemes = get_config().ALLOWED_URL_SCHEMES + self.schemes = schemes or get_config().ALLOWED_URL_SCHEMES class ExclusionValidator(BaseValidator): ``` Netbox fails to start due to a circular import: ``` Jan 11 20:07:59 netbox3 gunicorn[101352]: [2022-01-11 20:07:59 +0000] [101352] [ERROR] Exception in worker process Jan 11 20:07:59 netbox3 gunicorn[101352]: Traceback (most recent call last): Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker Jan 11 20:07:59 netbox3 gunicorn[101352]: worker.init_process() Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/venv/lib/python3.8/site-packages/gunicorn/workers/gthread.py", line 92, in init_process Jan 11 20:07:59 netbox3 gunicorn[101352]: super().init_process() Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/venv/lib/python3.8/site-packages/gunicorn/workers/base.py", line 134, in init_process Jan 11 20:07:59 netbox3 gunicorn[101352]: self.load_wsgi() Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/venv/lib/python3.8/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi Jan 11 20:07:59 netbox3 gunicorn[101352]: self.wsgi = self.app.wsgi() Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/venv/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi Jan 11 20:07:59 netbox3 gunicorn[101352]: self.callable = self.load() Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/venv/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 58, in load Jan 11 20:07:59 netbox3 gunicorn[101352]: return self.load_wsgiapp() Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/venv/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp Jan 11 20:07:59 netbox3 gunicorn[101352]: return util.import_app(self.app_uri) Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/venv/lib/python3.8/site-packages/gunicorn/util.py", line 359, in import_app Jan 11 20:07:59 netbox3 gunicorn[101352]: mod = importlib.import_module(module) Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module Jan 11 20:07:59 netbox3 gunicorn[101352]: return _bootstrap._gcd_import(name[level:], package, level) Jan 11 20:07:59 netbox3 gunicorn[101352]: File "<frozen importlib._bootstrap>", line 1014, in _gcd_import Jan 11 20:07:59 netbox3 gunicorn[101352]: File "<frozen importlib._bootstrap>", line 991, in _find_and_load Jan 11 20:07:59 netbox3 gunicorn[101352]: File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked Jan 11 20:07:59 netbox3 gunicorn[101352]: File "<frozen importlib._bootstrap>", line 671, in _load_unlocked Jan 11 20:07:59 netbox3 gunicorn[101352]: File "<frozen importlib._bootstrap_external>", line 848, in exec_module Jan 11 20:07:59 netbox3 gunicorn[101352]: File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/netbox/netbox/wsgi.py", line 7, in <module> Jan 11 20:07:59 netbox3 gunicorn[101352]: application = get_wsgi_application() Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/venv/lib/python3.8/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application Jan 11 20:07:59 netbox3 gunicorn[101352]: django.setup(set_prefix=False) Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/venv/lib/python3.8/site-packages/django/__init__.py", line 24, in setup Jan 11 20:07:59 netbox3 gunicorn[101352]: apps.populate(settings.INSTALLED_APPS) Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/venv/lib/python3.8/site-packages/django/apps/registry.py", line 114, in populate Jan 11 20:07:59 netbox3 gunicorn[101352]: app_config.import_models() Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/venv/lib/python3.8/site-packages/django/apps/config.py", line 301, in import_models Jan 11 20:07:59 netbox3 gunicorn[101352]: self.models_module = import_module(models_module_name) Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module Jan 11 20:07:59 netbox3 gunicorn[101352]: return _bootstrap._gcd_import(name[level:], package, level) Jan 11 20:07:59 netbox3 gunicorn[101352]: File "<frozen importlib._bootstrap>", line 1014, in _gcd_import Jan 11 20:07:59 netbox3 gunicorn[101352]: File "<frozen importlib._bootstrap>", line 991, in _find_and_load Jan 11 20:07:59 netbox3 gunicorn[101352]: File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked Jan 11 20:07:59 netbox3 gunicorn[101352]: File "<frozen importlib._bootstrap>", line 671, in _load_unlocked Jan 11 20:07:59 netbox3 gunicorn[101352]: File "<frozen importlib._bootstrap_external>", line 848, in exec_module Jan 11 20:07:59 netbox3 gunicorn[101352]: File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/netbox/circuits/models/__init__.py", line 1, in <module> Jan 11 20:07:59 netbox3 gunicorn[101352]: from .circuits import * Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/netbox/circuits/models/circuits.py", line 7, in <module> Jan 11 20:07:59 netbox3 gunicorn[101352]: from dcim.models import LinkTermination Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/netbox/dcim/models/__init__.py", line 1, in <module> Jan 11 20:07:59 netbox3 gunicorn[101352]: from .cables import * Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/netbox/dcim/models/cables.py", line 16, in <module> Jan 11 20:07:59 netbox3 gunicorn[101352]: from utilities.fields import ColorField Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/netbox/utilities/fields.py", line 5, in <module> Jan 11 20:07:59 netbox3 gunicorn[101352]: from .forms import ColorSelect Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/netbox/utilities/forms/__init__.py", line 2, in <module> Jan 11 20:07:59 netbox3 gunicorn[101352]: from .fields import * Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/netbox/utilities/forms/fields.py", line 107, in <module> Jan 11 20:07:59 netbox3 gunicorn[101352]: class LaxURLField(forms.URLField): Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/netbox/utilities/forms/fields.py", line 112, in LaxURLField Jan 11 20:07:59 netbox3 gunicorn[101352]: default_validators = [EnhancedURLValidator()] Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/netbox/utilities/validators.py", line 27, in __init__ Jan 11 20:07:59 netbox3 gunicorn[101352]: self.schemes = get_config().ALLOWED_URL_SCHEMES Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/netbox/netbox/config/__init__.py", line 27, in get_config Jan 11 20:07:59 netbox3 gunicorn[101352]: _thread_locals.config = Config() Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/netbox/netbox/config/__init__.py", line 49, in __init__ Jan 11 20:07:59 netbox3 gunicorn[101352]: self._populate_from_db() Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/netbox/netbox/config/__init__.py", line 77, in _populate_from_db Jan 11 20:07:59 netbox3 gunicorn[101352]: from extras.models import ConfigRevision Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/netbox/extras/models/__init__.py", line 2, in <module> Jan 11 20:07:59 netbox3 gunicorn[101352]: from .configcontexts import ConfigContext, ConfigContextModel Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/netbox/extras/models/configcontexts.py", line 7, in <module> Jan 11 20:07:59 netbox3 gunicorn[101352]: from extras.querysets import ConfigContextQuerySet Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/netbox/extras/querysets.py", line 4, in <module> Jan 11 20:07:59 netbox3 gunicorn[101352]: from extras.models.tags import TaggedItem Jan 11 20:07:59 netbox3 gunicorn[101352]: File "/opt/netbox/netbox/extras/models/tags.py", line 9, in <module> Jan 11 20:07:59 netbox3 gunicorn[101352]: from utilities.fields import ColorField Jan 11 20:07:59 netbox3 gunicorn[101352]: ImportError: cannot import name 'ColorField' from partially initialized module 'utilities.fields' (most likely due to a circular import) (/opt/netbox/netbox/utilities/fields.py) ```
Author
Owner

@jeremystretch commented on GitHub (Jan 11, 2022):

Yeah, it just needs a bit of refactoring to load the allowed schemes after the config has been initialized.

@jeremystretch commented on GitHub (Jan 11, 2022): Yeah, it just needs a bit of refactoring to load the allowed schemes _after_ the config has been initialized.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#5927