VLAN Translation #5406

@jaakub commented on GitHub (Sep 22, 2021):

Please spend some time detailing the proposed implementation and update your post above. Per the FR template:

Describe in detail the new feature or behavior you are proposing. Include any specific changes to work flows, data models, and/or the user interface. The more detail you provide here, the greater chance your proposal has of being discussed. Feature requests which don't include an actionable implementation plan will be rejected.

I've now added more detail in the FR.

@jaakub commented on GitHub (Sep 22, 2021): > Please spend some time detailing the proposed implementation and update your post above. Per the FR template: > > > Describe in detail the new feature or behavior you are proposing. Include any specific changes to work flows, data models, and/or the user interface. The more detail you provide here, the greater chance your proposal has of being discussed. Feature requests which don't include an actionable implementation plan will be rejected. I've now added more detail in the FR.

adam commented

@jeremystretch commented on GitHub (Oct 5, 2021):

I'm not clear on the real-world function being modeled here. How are VLANs being "translated?" From the examples above it looks like you're just correlating VLANs in two different domains.

@jeremystretch commented on GitHub (Oct 5, 2021): I'm not clear on the real-world function being modeled here. How are VLANs being "translated?" From the examples above it looks like you're just correlating VLANs in two different domains.

adam commented

@jaakub commented on GitHub (Oct 5, 2021):

The real-world function would be a single layer 2 service stretching two independent domains, which use a different VLAN tag for that service. In this case, you can rewrite/translate those tags on a border device connecting those two independent domains together. It is known as VLAN translation, although you could also say it is a VLAN rewrite function. Two references - Cisco and Juniper.

In the below example, we have:

VLAN 10 being translated into VLAN 20 and vice versa.
VLAN 30 being translated into VLAN 40 and vice versa.

What I'm trying to model in Netbox is exactly the correlation of two VLAN objects belonging to different VLAN groups providing that end to end service. We could achieve a similar result by using custom fields, however, a custom field wouldn't provide that dynamic link/reference to the other involved VLAN object and would not be as easy to track.

Allowing that link in Netbox would allow easier documentation and tracking of VLAN rewrite/mappings. At the moment, we maintain a separate spreadsheet to document that function and we would love to use Netbox as a single source of truth.

@jaakub commented on GitHub (Oct 5, 2021): The real-world function would be a single layer 2 service stretching two independent domains, which use a different VLAN tag for that service. In this case, you can rewrite/translate those tags on a border device connecting those two independent domains together. It is known as VLAN translation, although you could also say it is a VLAN rewrite function. Two references - [Cisco](https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/93x/vxlan/configuration/guide/b-cisco-nexus-9000-series-nx-os-vxlan-configuration-guide-93x/b-cisco-nexus-9000-series-nx-os-vxlan-configuration-guide-93x_chapter_011110.html) and [Juniper](https://www.juniper.net/documentation/en_US/junos/topics/example/layer-2-bulk-configuration-example-vlan-translation-with-lists-example-mx-solutions.html). In the below example, we have: - VLAN 10 being translated into VLAN 20 and vice versa. - VLAN 30 being translated into VLAN 40 and vice versa. ![image](https://user-images.githubusercontent.com/16577946/136103488-11b7741c-f114-496e-bc11-f8fab93f9fdf.png) What I'm trying to model in Netbox is exactly the correlation of two VLAN objects belonging to different VLAN groups providing that end to end service. We could achieve a similar result by using custom fields, however, a custom field wouldn't provide that dynamic link/reference to the other involved VLAN object and would not be as easy to track. Allowing that link in Netbox would allow easier documentation and tracking of VLAN rewrite/mappings. At the moment, we maintain a separate spreadsheet to document that function and we would love to use Netbox as a single source of truth.

adam commented

@tyler-8 commented on GitHub (Oct 6, 2021):

I like the idea. FWIW - The Aruba CX platform also supports VLAN translation.

Would it not be simpler to just have a single field (like maybe vlan_translation) that points to a single other VLAN object? I don't know that a translation_group would be necessary, but I definitely see the value of properly linking two VLANs together for translation modeling.

@tyler-8 commented on GitHub (Oct 6, 2021): I like the idea. FWIW - The [Aruba CX platform](https://techhub.hpe.com/eginfolib/Aruba/OS-CX_10.04/5200-6699/index.html#GUID-449F12AA-E152-4AED-B18B-C3472E39B603.html) also supports VLAN translation. Would it not be simpler to just have a single field (like maybe `vlan_translation`) that points to a single other VLAN object? I don't know that a translation_group would be necessary, but I definitely see the value of properly linking two VLANs together for translation modeling.

adam commented

@julianze commented on GitHub (Oct 6, 2021):

i like the idea too.
We are using VLAN-Mapping/Translation on our Cisco Catalyst:
https://content.cisco.com/chapter.sjs?uri=/searchable/chapter/content/en/us/td/docs/switches/lan/catalyst9500/software/release/16-9/configuration_guide/lyr2/b_169_lyr2_9500_cg/configuring_vlan_mapping.html.xml&platform=Cisco%20Catalyst%209500%20Series%20Switches

the use case is to "bridge" two vlan together to one l2 broadcast domain.

In example there is a data center switch infrastructure with a separate vlan scope. Then we often bridge a customer vlan from our access network through our backbone to the data center switches.
We don´t have control over the customer vlan id´s and so there could be duplicates over all customer networks.
there´s the point where we have to translate the customer vlan (C-Tag) to our serivce provider vlan (S-Tag) which is then available in the data center for housing or virtualization purposes.

1-to-1 mapping like a fieald with the translated id should be fit most scenarios.

Would it be useful to show interfaces from the translated vlan in the "device-interface"-tab as well?

@julianze commented on GitHub (Oct 6, 2021): i like the idea too. We are using VLAN-Mapping/Translation on our Cisco Catalyst: https://content.cisco.com/chapter.sjs?uri=/searchable/chapter/content/en/us/td/docs/switches/lan/catalyst9500/software/release/16-9/configuration_guide/lyr2/b_169_lyr2_9500_cg/configuring_vlan_mapping.html.xml&platform=Cisco%20Catalyst%209500%20Series%20Switches the use case is to "bridge" two vlan together to one l2 broadcast domain. In example there is a data center switch infrastructure with a separate vlan scope. Then we often bridge a customer vlan from our access network through our backbone to the data center switches. We don´t have control over the customer vlan id´s and so there could be duplicates over all customer networks. there´s the point where we have to translate the customer vlan (C-Tag) to our serivce provider vlan (S-Tag) which is then available in the data center for housing or virtualization purposes. 1-to-1 mapping like a fieald with the translated id should be fit most scenarios. Would it be useful to show interfaces from the translated vlan in the "device-interface"-tab as well?

adam commented

@jaakub commented on GitHub (Oct 7, 2021):

I like the idea. FWIW - The Aruba CX platform also supports VLAN translation.

Would it not be simpler to just have a single field (like maybe vlan_translation) that points to a single other VLAN object? I don't know that a translation_group would be necessary, but I definitely see the value of properly linking two VLANs together for translation modeling.

A single field showing the link between two VLAN objects would be most sufficient, and that's what I'm trying to achieve. I thought displaying a VLAN group of the linked VLAN object might be handy/nice, but it's not necessary - hence the two options. Glad to see others see value in this FR.

@jaakub commented on GitHub (Oct 7, 2021): > I like the idea. FWIW - The [Aruba CX platform](https://techhub.hpe.com/eginfolib/Aruba/OS-CX_10.04/5200-6699/index.html#GUID-449F12AA-E152-4AED-B18B-C3472E39B603.html) also supports VLAN translation. > > > > Would it not be simpler to just have a single field (like maybe `vlan_translation`) that points to a single other VLAN object? I don't know that a translation_group would be necessary, but I definitely see the value of properly linking two VLANs together for translation modeling. A single field showing the link between two VLAN objects would be most sufficient, and that's what I'm trying to achieve. I thought displaying a VLAN group of the linked VLAN object might be handy/nice, but it's not necessary - hence the two options. Glad to see others see value in this FR.

adam commented

@jaakub commented on GitHub (Oct 7, 2021):

Would it be useful to show interfaces from the translated vlan in the "device-interface"-tab as well?

Not sure I follow this idea...

@jaakub commented on GitHub (Oct 7, 2021): > > Would it be useful to show interfaces from the translated vlan in the "device-interface"-tab as well? Not sure I follow this idea...

adam commented

@github-actions[bot] commented on GitHub (Dec 7, 2021):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Please see our contributing guide.

@github-actions[bot] commented on GitHub (Dec 7, 2021): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Please see our [contributing guide](https://github.com/netbox-community/netbox/blob/develop/CONTRIBUTING.md).

adam commented

@jaakub commented on GitHub (Dec 7, 2021):

It was highlighted (in the Netbox Slack channel), that some vendors and platforms, such as Juniper JUNOS, implement VLAN re-write/translation function on a per-interface basis meaning you can translate 1 VLAN into multiple VLANs, whilst others, such as Cisco NXOS implement it on a per-VLAN basis.

The original idea above was to implement it under the VLAN object, meaning it would be a 1:1 mapping.

One idea to support per-interface VLAN translation (messy, as @DanSheps said, which I agree with), would be to implement a table at the bottom of the interface UI, which would have two columns (old and new VLAN), and one empty row by default. Users could expand that table by additional columns.

From data structure point of view, we would need something like dcim/interfaces/{id}/vlan_rewrite to return a list, with rows being presented as either list or tuple, or a dictionary.

Example case:

VLAN 10 being rewritten to VLAN 20
VLAN 100 being rewritten to VLAN 200

tuple = [(10, 20), (100, 200)]
list = [[10, 20], [100, 200]]
dict = [{"ovid": 10, "nvid": 20}, {"ovid": 100, "nvid": 200}]

I would be interested in hearing ideas from others on how this could be implemented both from a UI and API point of view.

@jaakub commented on GitHub (Dec 7, 2021): It was highlighted (in the Netbox Slack channel), that some vendors and platforms, such as Juniper JUNOS, implement VLAN re-write/translation function on a per-interface basis meaning you can translate 1 VLAN into multiple VLANs, whilst others, such as Cisco NXOS implement it on a per-VLAN basis. The original idea above was to implement it under the VLAN object, meaning it would be a 1:1 mapping. One idea to support per-interface VLAN translation (messy, as @DanSheps said, which I agree with), would be to implement a table at the bottom of the interface UI, which would have two columns (old and new VLAN), and one empty row by default. Users could expand that table by additional columns. From data structure point of view, we would need something like `dcim/interfaces/{id}/vlan_rewrite` to return a list, with rows being presented as either list or tuple, or a dictionary. Example case: - VLAN 10 being rewritten to VLAN 20 - VLAN 100 being rewritten to VLAN 200 ``` tuple = [(10, 20), (100, 200)] list = [[10, 20], [100, 200]] dict = [{"ovid": 10, "nvid": 20}, {"ovid": 100, "nvid": 200}] ``` I would be interested in hearing ideas from others on how this could be implemented both from a UI and API point of view.

adam commented

@jcralbino commented on GitHub (Feb 26, 2023):

I would like to see this implemented in a way that we can decouple the concept of the layer 2 end-to-end domain from the vlan object

Currently this is just one use case of several examples where a layer 2 broadcast domain is not tied to the vlan deployed in a switch and associated with the interface of the device

For example:

in a vxlan environment the l2 broadcast domain is extended across several devices without requiring the use of the same vlan id
ultimately in any l2vpn overlay this is also existing

In this case we want to associate the same layer 2 broadcast domain to two vlan ids existing in different vlan groups

This is the struggle I have faced when trying to model a modern SDN environment in the netbox current structure

@jcralbino commented on GitHub (Feb 26, 2023): I would like to see this implemented in a way that we can decouple the concept of the layer 2 end-to-end domain from the vlan object Currently this is just one use case of several examples where a layer 2 broadcast domain is not tied to the vlan deployed in a switch and associated with the interface of the device For example: - in a vxlan environment the l2 broadcast domain is extended across several devices without requiring the use of the same vlan id - ultimately in any l2vpn overlay this is also existing In this case we want to associate the same layer 2 broadcast domain to two vlan ids existing in different vlan groups This is the struggle I have faced when trying to model a modern SDN environment in the netbox current structure

adam commented

https://github.com/netbox-community/netbox/discussions/9418

@jcralbino commented on GitHub (Feb 26, 2023):

This is one discussion i started to improve the l2 forwarding plane that I believe is relevant to my comment above

@jcralbino commented on GitHub (Feb 26, 2023): This is one discussion i started to improve the l2 forwarding plane that I believe is relevant to my comment above https://github.com/netbox-community/netbox/discussions/9418

adam commented

https://github.com/netbox-community/netbox/issues/9373

@jcralbino commented on GitHub (Feb 26, 2023):

Similar issue here about the changes needed for better modelling of layer 2 forwarding planes

@jcralbino commented on GitHub (Feb 26, 2023): Similar issue here about the changes needed for better modelling of layer 2 forwarding planes https://github.com/netbox-community/netbox/issues/9373

adam commented

@DanSheps commented on GitHub (Mar 6, 2023):

It was highlighted (in the Netbox Slack channel), that some vendors and platforms, such as Juniper JUNOS, implement VLAN re-write/translation function on a per-interface basis meaning you can translate 1 VLAN into multiple VLANs, whilst others, such as Cisco NXOS implement it on a per-VLAN basis.

Just want to point out, NX-OS does it per-interface as well. I am not sure where you got the information that it is per-device but it is not. You do need the VLAN to be VXLAN enabled, however it is do-able on a per-interface basis.

#9373 is not relevant to this issue so it would be best to ignore it (You can bring it up in your discussion, it is just not relevant here)

@DanSheps commented on GitHub (Mar 6, 2023): > It was highlighted (in the Netbox Slack channel), that some vendors and platforms, such as Juniper JUNOS, implement VLAN re-write/translation function on a per-interface basis meaning you can translate 1 VLAN into multiple VLANs, whilst others, such as Cisco NXOS implement it on a per-VLAN basis. Just want to point out, NX-OS does it per-interface as well. I am not sure where you got the information that it is per-device but it is not. You do need the VLAN to be VXLAN enabled, however it is do-able on a per-interface basis. #9373 is not relevant to this issue so it would be best to ignore it (You can bring it up in your discussion, it is just not relevant here)

adam commented

@tyler-8 commented on GitHub (Oct 20, 2023):

Just wanted to revisit some ideas on how to model this simply and broadly.

If I'm understanding correctly; Aruba, Cisco, and Juniper vendors all do VLAN translation at the interface level - translation isn't global across all interfaces on the device participating in the "local" VLAN; so perhaps the way forward is to make this an interface-object data structure.

We have tagged_vlans and untagged_vlan - how do we simply model when any given VLAN is translated across the interface? A JSONField that maps "local" VLAN Object IDs to "remote" VLAN Object IDs, perhaps?

For example:

If VLAN 5 (id 1) is used locally on the switch, but I want to translate it to VLAN 10 (id 2) across a trunk interface, the data structure could look like this:

Snippet of interface API output

{
    "tx_power": null,
    "untagged_vlan": null,
    "tagged_vlans": [
        {
          "id": 1,
          "url": "https://demo.netbox.dev/api/ipam/vlans/1/",
          "display": "Data (5)",
          "vid": 5,
          "name": "Data"
        }
      ],
    "tagged_vlan_translation": [{"local": 1, "remote": 2}],
    "untagged_vlan_translation": [],
    "mark_connected": false
}

Alternatively, a new model would have to be created, rough example:

class InterfaceVlanTranslation(models.Model):
    interface = ForeignKey(Interface)
    local_vlan = ForeignKey(VLAN)  # vlan being translated
    remote_vlan = ForeignKey(VLAN)  # vlan that we're translating to

@tyler-8 commented on GitHub (Oct 20, 2023): Just wanted to revisit some ideas on how to model this simply and broadly. If I'm understanding correctly; Aruba, Cisco, and Juniper vendors all do VLAN translation at the interface level - translation isn't global across all interfaces on the device participating in the "local" VLAN; so perhaps the way forward is to make this an interface-object data structure. We have `tagged_vlans` and `untagged_vlan` - how do we _simply_ model when any given VLAN is translated across the interface? A JSONField that maps "local" VLAN Object IDs to "remote" VLAN Object IDs, perhaps? For example: If VLAN 5 (id 1) is used locally on the switch, but I want to translate it to VLAN 10 (id 2) across a trunk interface, the data structure could look like this: Snippet of interface API output ```json { "tx_power": null, "untagged_vlan": null, "tagged_vlans": [ { "id": 1, "url": "https://demo.netbox.dev/api/ipam/vlans/1/", "display": "Data (5)", "vid": 5, "name": "Data" } ], "tagged_vlan_translation": [{"local": 1, "remote": 2}], "untagged_vlan_translation": [], "mark_connected": false } ``` Alternatively, a new model would have to be created, rough example: ```python class InterfaceVlanTranslation(models.Model): interface = ForeignKey(Interface) local_vlan = ForeignKey(VLAN) # vlan being translated remote_vlan = ForeignKey(VLAN) # vlan that we're translating to ```

adam commented

@ITJamie commented on GitHub (Oct 21, 2023):

personally, i think the idea of a separate model InterfaceVlanTranslation is the better approach and would help a lot when it comes to documenting svlans in future. it could also help for virtual circuits / circuit tracing.

though maybe have an option of using id's only instead of having to have two vlan objects...

class InterfaceVlanTranslation(models.Model):
    interface = ForeignKey(Interface)
    local_vlan = ForeignKey(VLAN)  # vlan being translated
    local_vlan_vid = int()
    remote_vlan = ForeignKey(VLAN)  # vlan that we're translating to
    remote_vlan_vid = int()

the reason why I would argue for this is in an isp its normal to handoff a customers vlan on vlan10 for example. it would be great to have an ability to just document it as vlan 10 handoff without having to create 1000's of unique vlan 10's vlan objects.

in a lot of cases we would want to use vlan foreignkeys for both, but in some just a documented integer would be preferred for at least one of the sides of the translation

@ITJamie commented on GitHub (Oct 21, 2023): personally, i think the idea of a separate model `InterfaceVlanTranslation` is the better approach and would help a lot when it comes to documenting svlans in future. it could also help for virtual circuits / circuit tracing. though maybe have an option of using id's only instead of having to have two vlan objects... ``` class InterfaceVlanTranslation(models.Model): interface = ForeignKey(Interface) local_vlan = ForeignKey(VLAN) # vlan being translated local_vlan_vid = int() remote_vlan = ForeignKey(VLAN) # vlan that we're translating to remote_vlan_vid = int() ``` the reason why I would argue for this is in an isp its normal to handoff a customers vlan on vlan10 for example. it would be great to have an ability to just document it as vlan 10 handoff without having to create 1000's of unique vlan 10's vlan objects. in a lot of cases we would want to use vlan foreignkeys for both, but in some just a documented integer would be preferred for at least one of the sides of the translation

adam commented

@jeremystretch commented on GitHub (Sep 25, 2024):

Just reviewing past discussion as this has come up as a candidate FR for NetBox v4.2. Thanks everyone for providing some context.

The biggest sticking for me at this point is whether a translation policy needs to maintain a concrete foreign key relationship to a specific VLAN, or just specify the VLAN ID. It looks like just maintaining the VLAN ID would be sufficient, and would probably allow for a simpler implementation. It would also allow a policy to be reused across multiple devices and domains.

I'm imagining a two-model implementation. Some very hasty pseudo-code:

class VLANTranslationPolicy(models.Model):
    name = models.CharField()
    description = models.CharField()


class VLANTranslationRule(models.Model):
    policy = models.ForeignKey(
        to=VLANTranslationPolicy,
        related_name='rules'
    )
    local_vid = models.IntegerField()
    remote_vid = models.IntegerField()

This would allow for the definition of a policy with multiple one-to-one VID mappings, which can be applied to interfaces arbitrarily.

We would also add a ForeignnKey to the Interface and VMInterface models:

vlan_translation_policy = models.ForeignKey(
    to=VLANTranslationPolicy
)

If this approach will substantially satisfy the FR, I think we can tackle this in v4.2.

@jeremystretch commented on GitHub (Sep 25, 2024): Just reviewing past discussion as this has come up as a candidate FR for NetBox v4.2. Thanks everyone for providing some context. The biggest sticking for me at this point is whether a translation policy needs to maintain a concrete foreign key relationship to a specific VLAN, or just specify the VLAN ID. It _looks_ like just maintaining the VLAN ID would be sufficient, and would probably allow for a simpler implementation. It would also allow a policy to be reused across multiple devices and domains. I'm imagining a two-model implementation. Some very hasty pseudo-code: ```python class VLANTranslationPolicy(models.Model): name = models.CharField() description = models.CharField() class VLANTranslationRule(models.Model): policy = models.ForeignKey( to=VLANTranslationPolicy, related_name='rules' ) local_vid = models.IntegerField() remote_vid = models.IntegerField() ``` This would allow for the definition of a policy with multiple one-to-one VID mappings, which can be applied to interfaces arbitrarily. We would also add a ForeignnKey to the Interface and VMInterface models: ```python vlan_translation_policy = models.ForeignKey( to=VLANTranslationPolicy ) ``` If this approach will substantially satisfy the FR, I think we can tackle this in v4.2.

adam commented

@ITJamie commented on GitHub (Sep 25, 2024):

I would have thought the local_vid would be a foreignkey to a vlan and the remote_vid would be an integer.

Also im assuming those policies could be linked to a specific device or specific interface (some devices translation is unique to the entire device and some the translation is unique to an interface only)

@ITJamie commented on GitHub (Sep 25, 2024): I would have thought the local_vid would be a foreignkey to a vlan and the remote_vid would be an integer. Also im assuming those policies could be linked to a specific device or specific interface (some devices translation is unique to the entire device and some the translation is unique to an interface only)

adam commented

@tyler-8 commented on GitHub (Sep 26, 2024):

I think the challenge with ForeignKey for the VIDs is that the use cases out there are all slightly different. Some people will create the "remote" VLAN because they own or control that VLAN. Others may have no ownership of the remote VLAN and so don't want to create an entire VLAN object in their NetBox for it.

@tyler-8 commented on GitHub (Sep 26, 2024): I think the challenge with ForeignKey for the VIDs is that the use cases out there are all slightly different. Some people will create the "remote" VLAN because they own or control that VLAN. Others may have no ownership of the remote VLAN and so don't want to create an entire VLAN object in their NetBox for it.

adam commented