starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Allow customization of queryset get_for_* functions #11534

New Issue
Closed
opened 2025-12-29 21:46:29 +01:00 by adam · 5 comments
adam commented 2025-12-29 21:46:29 +01:00
Owner
Copy Link

Originally created by @Omripresent on GitHub (Aug 26, 2025).

NetBox version

v4.1.11

Feature type

New functionality

Proposed functionality

Netbox has an opinionated way of scoping VLAN to devices/VMs when done by locations, site groups, and regions, forcing the location and its ancestors VLAN groups to be assignable to a given device/VM.

I would be useful for some deployments to allow customization of that functionality by allowing admins to define their desired logic with functions in extras.py that if set can override the builtin get_for_* functions in the relevant querysets.

Our current workaround is to customize the docker image by replacing the querysets.py files where needed, this workaround comes at the cost of comparing the modified files on every upgrade to avoid using old code in other parts of the file.

Use case

Use case 1 - Parent location sees all:

There are times when devices in a given location can be assigned VLANs associated with all descendant locations

if device.location:
    q |= Q(
        scope_type=ContentType.objects.get_by_natural_key('dcim', 'location'),
        scope_id__in=device.location.get_descendants(include_self=True)
    )

Use case 2 - Isolated locations:

In some cases admins might need to make sure locations are fully isolated and would need to match the exclusive location alone and no other VLANs

if device.location:
    q |= Q(
        scope_type=ContentType.objects.get_by_natural_key('dcim', 'location'),
        scope_id=device.location
    )

Database changes

N/A

External dependencies

N/A

Originally created by @Omripresent on GitHub (Aug 26, 2025). ### NetBox version v4.1.11 ### Feature type New functionality ### Proposed functionality Netbox has an opinionated way of scoping VLAN to devices/VMs when done by locations, site groups, and regions, forcing the location and its ancestors VLAN groups to be assignable to a given device/VM. I would be useful for some deployments to allow customization of that functionality by allowing admins to define their desired logic with functions in extras.py that if set can override the builtin get_for_* functions in the relevant querysets. Our current workaround is to customize the docker image by replacing the querysets.py files where needed, this workaround comes at the cost of comparing the modified files on every upgrade to avoid using old code in other parts of the file. ### Use case ### Use case 1 - Parent location sees all: There are times when devices in a given location can be assigned VLANs associated with all descendant locations ```python if device.location: q |= Q( scope_type=ContentType.objects.get_by_natural_key('dcim', 'location'), scope_id__in=device.location.get_descendants(include_self=True) ) ``` ### Use case 2 - Isolated locations: In some cases admins might need to make sure locations are fully isolated and would need to match the exclusive location alone and no other VLANs ```python if device.location: q |= Q( scope_type=ContentType.objects.get_by_natural_key('dcim', 'location'), scope_id=device.location ) ``` ### Database changes N/A ### External dependencies N/A
adam added the type: feature label 2025-12-29 21:46:29 +01:00
adam closed this issue 2025-12-29 21:46:29 +01:00
adam commented 2025-12-29 21:46:30 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Aug 28, 2025):

@Omripresent could you please elaborate on your specific use case? It's not clear based solely on the code snippets you've shared above.

@jeremystretch commented on GitHub (Aug 28, 2025): @Omripresent could you please elaborate on your specific use case? It's not clear based solely on the code snippets you've shared above.
adam commented 2025-12-29 21:46:30 +01:00
Author
Owner
Copy Link

@Omripresent commented on GitHub (Aug 28, 2025):

Sorry for the lack of context. In a fabric (EVPN/VXLAN) deployment and the flexibility of VLAN/MAC mobility there are unique organization options we employ in our deployment.

For example in a fabric with device-1..3 each in their distinct location, that may be a cage or row in the physical datacenter.
loc-1 is the parent for both loc-2 and loc-3. and each VG is scoped to its respective location

┌──────────────────────────────────────┐
│loc-1        ┌───────────┐┌──────────┐│
│ vg-1        │loc-2      ││loc-3     ││
│  vlan-1     │ vg-2      ││ vg-3     ││
│ device-1    │  vlan-2   ││  vlan-3  ││
│             │ device-2  ││ device-3 ││
│             │           ││          ││
│             └───────────┘└──────────┘│
└──────────────────────────────────────┘

The way Netbox is currently working the VLAN assignment options are as follows:
device-1 = [ vlan-1 ]
device-2 = [ vlan-1 vlan-2 ]
device-3 = [ vlan-1 vlan-3 ]

Referring to the 2 use cases I've mentioned originally, both apply to some of our environments, depending on certain conditions be it device role, tag assignment or custom field, we need to change the VLAN availability behavior.

For use case 1 expected VLAN scope availability:
device-1 = [ vlan-1 vlan-2 vlan-3 ]
device-2 = [ vlan-2 ]
device-3 = [ vlan-3 ]

For use case 2 expected VLAN scope availability:
device-1 = [ vlan-1 ]
device-2 = [ vlan-2 ]
device-3 = [ vlan-3 ]

Those use cases are relevant to a subset of our overall deployment and having the flexibility to tailor the querysets to each use case would be extremely helpful as more specialized requirements are being asked of us.

@Omripresent commented on GitHub (Aug 28, 2025): Sorry for the lack of context. In a fabric (EVPN/VXLAN) deployment and the flexibility of VLAN/MAC mobility there are unique organization options we employ in our deployment. For example in a fabric with device-1..3 each in their distinct location, that may be a cage or row in the physical datacenter. loc-1 is the parent for both loc-2 and loc-3. and each VG is scoped to its respective location ``` ┌──────────────────────────────────────┐ │loc-1 ┌───────────┐┌──────────┐│ │ vg-1 │loc-2 ││loc-3 ││ │ vlan-1 │ vg-2 ││ vg-3 ││ │ device-1 │ vlan-2 ││ vlan-3 ││ │ │ device-2 ││ device-3 ││ │ │ ││ ││ │ └───────────┘└──────────┘│ └──────────────────────────────────────┘ ``` The way Netbox is currently working the VLAN assignment options are as follows: device-1 = [ vlan-1 ] device-2 = [ vlan-1 vlan-2 ] device-3 = [ vlan-1 vlan-3 ] Referring to the 2 use cases I've mentioned originally, both apply to some of our environments, depending on certain conditions be it device role, tag assignment or custom field, we need to change the VLAN availability behavior. For use case 1 expected VLAN scope availability: device-1 = [ vlan-1 vlan-2 vlan-3 ] device-2 = [ vlan-2 ] device-3 = [ vlan-3 ] For use case 2 expected VLAN scope availability: device-1 = [ vlan-1 ] device-2 = [ vlan-2 ] device-3 = [ vlan-3 ] Those use cases are relevant to a subset of our overall deployment and having the flexibility to tailor the querysets to each use case would be extremely helpful as more specialized requirements are being asked of us.
adam commented 2025-12-29 21:46:30 +01:00
Author
Owner
Copy Link

@jnovinger commented on GitHub (Sep 15, 2025):

Hi @Omripresent,

Regarding the extensibility approach you described for overriding queryset methods, this isn't something we're planning to implement in NetBox core. We maintain these methods as part of NetBox's stable API surface.

That said, we'd like to better understand your specific use cases to see if there might be other approaches that could work within NetBox's existing architecture. The conditional logic you described seems to depend on installation-specific attributes (roles, tags, custom fields), but understanding the underlying operational patterns might help identify a more generic solution that could benefit the broader NetBox community. Could you clarify a few technical details?

  1. You mentioned being able to "create the functions that we can overload the existing ones" similar to extras.py. Could you clarify what extensibility mechanism you're referring to? Or are you suggesting this as a new mechanism?

  2. How are you currently modifying the queryset methods in your deployment (it's not clear to me from the code snippets above)? To be clear, I mean, what do the modified queryset methods look like?

  3. For the conditional logic you described (based on "device role, tag assignment or custom field"), could you provide examples of what specific attributes you're checking? This would help us understand if there are generic patterns that could work across different NetBox installations.

  4. When you mention "parent sees all descendants" vs "strict isolation", are these behaviors you need simultaneously in the same installation, or alternatives you'd choose between?

The main challenge is that core NetBox can't assume specific device roles, tags, or custom fields exist across different installations.

  1. In your August 28th comment, you mentioned EVPN/VXLAN fabric scenarios. For use cases involving VLANs that span multiple sites, have you considered modeling these using NetBox's L2VPN objects? This might provide a generic way to represent cross-site layer 2 connectivity/fabrics that doesn't rely on installation-specific roles or tags and that may be appropriate for use in figuring out which VLANs to make available via the queryset methods in question.
@jnovinger commented on GitHub (Sep 15, 2025): Hi @Omripresent, Regarding the extensibility approach you described for overriding queryset methods, this isn't something we're planning to implement in NetBox core. We maintain these methods as part of NetBox's stable API surface. That said, we'd like to better understand your specific use cases to see if there might be other approaches that could work within NetBox's existing architecture. The conditional logic you described seems to depend on installation-specific attributes (roles, tags, custom fields), but understanding the underlying operational patterns might help identify a more generic solution that could benefit the broader NetBox community. Could you clarify a few technical details? 1. You mentioned being able to "create the functions that we can overload the existing ones" similar to `extras.py`. Could you clarify what extensibility mechanism you're referring to? Or are you suggesting this as a new mechanism? 2. How are you currently modifying the queryset methods in your deployment (it's not clear to me from the code snippets above)? To be clear, I mean, what do the modified queryset methods look like? 3. For the conditional logic you described (based on "device role, tag assignment or custom field"), could you provide examples of what specific attributes you're checking? This would help us understand if there are generic patterns that could work across different NetBox installations. 4. When you mention "parent sees all descendants" vs "strict isolation", are these behaviors you need simultaneously in the same installation, or alternatives you'd choose between? The main challenge is that core NetBox can't assume specific device roles, tags, or custom fields exist across different installations. 5. In your August 28th comment, you mentioned EVPN/VXLAN fabric scenarios. For use cases involving VLANs that span multiple sites, have you considered modeling these using NetBox's L2VPN objects? This might provide a generic way to represent cross-site layer 2 connectivity/fabrics that doesn't rely on installation-specific roles or tags and that _may_ be appropriate for use in figuring out which VLANs to make available via the queryset methods in question.
adam commented 2025-12-29 21:46:30 +01:00
Author
Owner
Copy Link

@Omripresent commented on GitHub (Sep 15, 2025):

Hi @jnovinger,

I'll try to answer your questions as best as I can.

  1. I was thinking similar to the JINJA2_FILTERS variable in extras.py we could have the option to define queryset get functions that the core Netbox can use where applicable.
    For example, let's say I want to override the VLANQuerySet get_for_device function. I would need to define the function and reference it in a variable. Something along the lines of:

    # extras.py
def my_queryset_function(self, device):
    # some function logic
QUERYSETS = {
    "VLANQuerySet.get_for_device": my_queryset_function
}

    Then in the queryset function itself some functionality along the lines of:

    # ipam/querysets.py
def get_for_device(self, device):
        """
        Return all VLANs available to the specified Device.
        """
        if "VLANQuerySet.get_for_device" in QUERYSETS:
            return QUERYSETS["VLANQuerySet.get_for_device"](self, device)
        ...

  2. I'm not allowed to share the specific code, but the change is to VLANQuerySet.get_for_device and the logic looking at a custom field value of the device's tenant and changes the location scope matching to location.get_descendants instead of get_ancestors.
    We haven't implemented any of our other use cases yet. Those stem from new regulatory requirements that we're working on implementing

  3. See point 2, but there are a few other use cases we're actively working on supporting that will require further modifications.

  4. We're running a single global instance and will require all of the above depending on either device role, custom field, or other possible data point. Since we have multiple network architecture designs depending on the business entity and regulatory requirements, we have to support the different edge cases in a single installation.

  5. I'll look into L2VPN for that implementation, but most of our EVPN/VXLAN are single site to support horizontal growth at scale, all devices participating in a fabric are in the same Netbox location to allow the VLAN assignments correctly.

@Omripresent commented on GitHub (Sep 15, 2025): Hi @jnovinger, I'll try to answer your questions as best as I can. 1. I was thinking similar to the `JINJA2_FILTERS` variable in extras.py we could have the option to define queryset get functions that the core Netbox can use where applicable. For example, let's say I want to override the VLANQuerySet get_for_device function. I would need to define the function and reference it in a variable. Something along the lines of: ```python # extras.py def my_queryset_function(self, device): # some function logic QUERYSETS = { "VLANQuerySet.get_for_device": my_queryset_function } ``` Then in the queryset function itself some functionality along the lines of: ```python # ipam/querysets.py def get_for_device(self, device): """ Return all VLANs available to the specified Device. """ if "VLANQuerySet.get_for_device" in QUERYSETS: return QUERYSETS["VLANQuerySet.get_for_device"](self, device) ... ``` 2. I'm not allowed to share the specific code, but the change is to `VLANQuerySet.get_for_device` and the logic looking at a custom field value of the device's tenant and changes the location scope matching to location.get_descendants instead of get_ancestors. We haven't implemented any of our other use cases yet. Those stem from new regulatory requirements that we're working on implementing 3. See point 2, but there are a few other use cases we're actively working on supporting that will require further modifications. 4. We're running a single global instance and will require all of the above depending on either device role, custom field, or other possible data point. Since we have multiple network architecture designs depending on the business entity and regulatory requirements, we have to support the different edge cases in a single installation. 5. I'll look into L2VPN for that implementation, but most of our EVPN/VXLAN are single site to support horizontal growth at scale, all devices participating in a fabric are in the same Netbox location to allow the VLAN assignments correctly.
adam commented 2025-12-29 21:46:31 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Oct 6, 2025):

@Omripresent thanks for sharing this. It seems that the root of your problem is attempting to treat a VLAN group in different ways depending on the context. Unfortunately this isn't something we're able to support.

VLAN groups exist to denote the geographic scope of an L2 domain (i.e. region-wide, site-wide, etc.). A VLAN group which is scoped to a location is always considered available for all child locations. Where this is not the case, a VLAN group must instead be scoped to the applicable child location.

Admittedly, this can be limiting in certain scenarios, and discussion around improving L2 modeling in general is ongoing. However, the proposed mechanism of introducing conditional logic for queries is almost certain to introduce more challenges than it solves, so I have to decline this specific proposal.

That said, I encourage you to start a discussion and elaborate further on your specific use case(s) and needs. Once they've been fleshed out further, we may be able to build on other recent discussions around improved L2 domain modeling to figure out potential solutions.

Thanks again for the idea!

@jeremystretch commented on GitHub (Oct 6, 2025): @Omripresent thanks for sharing this. It seems that the root of your problem is attempting to treat a VLAN group in different ways depending on the context. Unfortunately this isn't something we're able to support. VLAN groups exist to denote the geographic scope of an L2 domain (i.e. region-wide, site-wide, etc.). A VLAN group which is scoped to a location is always considered available for all child locations. Where this is not the case, a VLAN group must instead be scoped to the applicable child location. Admittedly, this can be limiting in certain scenarios, and discussion around improving L2 modeling in general is ongoing. However, the proposed mechanism of introducing conditional logic for queries is almost certain to introduce more challenges than it solves, so I have to decline this specific proposal. That said, I encourage you to [start a discussion](https://github.com/netbox-community/netbox/discussions/new/choose) and elaborate further on your specific use case(s) and needs. Once they've been fleshed out further, we may be able to build on other recent discussions around improved L2 domain modeling to figure out potential solutions. Thanks again for the idea!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label type: feature
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#11534
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?