mirror of
https://github.com/netbox-community/netbox.git
synced 2026-01-12 05:20:31 +01:00
Add support for tracking physical cable plants #8
Closed
opened 2025-12-29 15:28:27 +01:00 by adam
·
100 comments
No Branch/Tag Specified
main
update-changelog-comments-docs
feature-removal-issue-type
20911-dropdown
20239-plugin-menu-classes-mutable-state
21097-graphql-id-lookups
feature
fix_module_substitution
20923-dcim-templates
20044-elevation-stuck-lightmode
feature-ip-prefix-link
v4.5-beta1-release
20068-import-moduletype-attrs
20766-fix-german-translation-code-literals
20378-del-script
7604-filter-modifiers-v3
circuit-swap
12318-case-insensitive-uniqueness
20637-improve-device-q-filter
20660-script-load
19724-graphql
20614-update-ruff
14884-script
02496-max-page
19720-macaddress-interface-generic-relation
19408-circuit-terminations-export-templates
20203-openapi-check
fix-19669-api-image-download
7604-filter-modifiers
19275-fixes-interface-bulk-edit
fix-17794-get_field_value_return_list
11507-show-aggregate-and-rir-on-api
9583-add_column_specific_search_field_to_tables
v4.5.0
v4.4.10
v4.4.9
v4.5.0-beta1
v4.4.8
v4.4.7
v4.4.6
v4.4.5
v4.4.4
v4.4.3
v4.4.2
v4.4.1
v4.4.0
v4.3.7
v4.4.0-beta1
v4.3.6
v4.3.5
v4.3.4
v4.3.3
v4.3.2
v4.3.1
v4.3.0
v4.2.9
v4.3.0-beta2
v4.2.8
v4.3.0-beta1
v4.2.7
v4.2.6
v4.2.5
v4.2.4
v4.2.3
v4.2.2
v4.2.1
v4.2.0
v4.1.11
v4.1.10
v4.1.9
v4.1.8
v4.2-beta1
v4.1.7
v4.1.6
v4.1.5
v4.1.4
v4.1.3
v4.1.2
v4.1.1
v4.1.0
v4.0.11
v4.0.10
v4.0.9
v4.1-beta1
v4.0.8
v4.0.7
v4.0.6
v4.0.5
v4.0.3
v4.0.2
v4.0.1
v4.0.0
v3.7.8
v3.7.7
v4.0-beta2
v3.7.6
v3.7.5
v4.0-beta1
v3.7.4
v3.7.3
v3.7.2
v3.7.1
v3.7.0
v3.6.9
v3.6.8
v3.6.7
v3.7-beta1
v3.6.6
v3.6.5
v3.6.4
v3.6.3
v3.6.2
v3.6.1
v3.6.0
v3.5.9
v3.6-beta2
v3.5.8
v3.6-beta1
v3.5.7
v3.5.6
v3.5.5
v3.5.4
v3.5.3
v3.5.2
v3.5.1
v3.5.0
v3.4.10
v3.4.9
v3.5-beta2
v3.4.8
v3.5-beta1
v3.4.7
v3.4.6
v3.4.5
v3.4.4
v3.4.3
v3.4.2
v3.4.1
v3.4.0
v3.3.10
v3.3.9
v3.4-beta1
v3.3.8
v3.3.7
v3.3.6
v3.3.5
v3.3.4
v3.3.3
v3.3.2
v3.3.1
v3.3.0
v3.2.9
v3.2.8
v3.3-beta2
v3.2.7
v3.3-beta1
v3.2.6
v3.2.5
v3.2.4
v3.2.3
v3.2.2
v3.2.1
v3.2.0
v3.1.11
v3.1.10
v3.2-beta2
v3.1.9
v3.2-beta1
v3.1.8
v3.1.7
v3.1.6
v3.1.5
v3.1.4
v3.1.3
v3.1.2
v3.1.1
v3.1.0
v3.0.12
v3.0.11
v3.0.10
v3.1-beta1
v3.0.9
v3.0.8
v3.0.7
v3.0.6
v3.0.5
v3.0.4
v3.0.3
v3.0.2
v3.0.1
v3.0.0
v2.11.12
v3.0-beta2
v2.11.11
v2.11.10
v3.0-beta1
v2.11.9
v2.11.8
v2.11.7
v2.11.6
v2.11.5
v2.11.4
v2.11.3
v2.11.2
v2.11.1
v2.11.0
v2.10.10
v2.10.9
v2.11-beta1
v2.10.8
v2.10.7
v2.10.6
v2.10.5
v2.10.4
v2.10.3
v2.10.2
v2.10.1
v2.10.0
v2.9.11
v2.10-beta2
v2.9.10
v2.10-beta1
v2.9.9
v2.9.8
v2.9.7
v2.9.6
v2.9.5
v2.9.4
v2.9.3
v2.9.2
v2.9.1
v2.9.0
v2.9-beta2
v2.8.9
v2.9-beta1
v2.8.8
v2.8.7
v2.8.6
v2.8.5
v2.8.4
v2.8.3
v2.8.2
v2.8.1
v2.8.0
v2.7.12
v2.7.11
v2.7.10
v2.7.9
v2.7.8
v2.7.7
v2.7.6
v2.7.5
v2.7.4
v2.7.3
v2.7.2
v2.7.1
v2.7.0
v2.6.12
v2.6.11
v2.6.10
v2.6.9
v2.7-beta1
Solcon-2020-01-06
v2.6.8
v2.6.7
v2.6.6
v2.6.5
v2.6.4
v2.6.3
v2.6.2
v2.6.1
v2.6.0
v2.5.13
v2.5.12
v2.6-beta1
v2.5.11
v2.5.10
v2.5.9
v2.5.8
v2.5.7
v2.5.6
v2.5.5
v2.5.4
v2.5.3
v2.5.2
v2.5.1
v2.5.0
v2.4.9
v2.5-beta2
v2.4.8
v2.5-beta1
v2.4.7
v2.4.6
v2.4.5
v2.4.4
v2.4.3
v2.4.2
v2.4.1
v2.4.0
v2.3.7
v2.4-beta1
v2.3.6
v2.3.5
v2.3.4
v2.3.3
v2.3.2
v2.3.1
v2.3.0
v2.2.10
v2.3-beta2
v2.2.9
v2.3-beta1
v2.2.8
v2.2.7
v2.2.6
v2.2.5
v2.2.4
v2.2.3
v2.2.2
v2.2.1
v2.2.0
v2.1.6
v2.2-beta2
v2.1.5
v2.2-beta1
v2.1.4
v2.1.3
v2.1.2
v2.1.1
v2.1.0
v2.0.10
v2.1-beta1
v2.0.9
v2.0.8
v2.0.7
v2.0.6
v2.0.5
v2.0.4
v2.0.3
v2.0.2
v2.0.1
v2.0.0
v2.0-beta3
v1.9.6
v1.9.5
v2.0-beta2
v1.9.4-r1
v1.9.3
v2.0-beta1
v1.9.2
v1.9.1
v1.9.0-r1
v1.8.4
v1.8.3
v1.8.2
v1.8.1
v1.8.0
v1.7.3
v1.7.2-r1
v1.7.1
v1.7.0
v1.6.3
v1.6.2-r1
v1.6.1-r1
1.6.1
v1.6.0
v1.5.2
v1.5.1
v1.5.0
v1.4.2
v1.4.1
v1.4.0
v1.3.2
v1.3.1
v1.3.0
v1.2.2
v1.2.1
v1.2.0
v1.1.0
v1.0.7-r1
v1.0.7
v1.0.6
v1.0.5
v1.0.4
v1.0.3-r1
v1.0.3
1.0.0
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
Mirrored from GitHub Pull Request
No Label
status: accepted
Milestone
No items
No Milestone
Projects
Clear projects
No project
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: starred/netbox#8
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @jeremystretch on GitHub (Jun 24, 2016).
Currently, NetBox does not support any concept of physical cable plant installations. We can connect one interface to another, but there is no way too associate that connection with information about the physical path taken by the cabling.
Ideally, NetBox should support the construction of physical cable paths to be associated with data (and perhaps console) connections. This would entail the tracking of individual path panels and cables, as well as the ability to support bundles of cabling (e.g. MPO fiber trunks).
The database schema necessary to support this feature will take quite a bit of thought. However, once that schema has been devised and agreed upon, it should be relatively easy to incorporate in NetBox.
@shakalandy commented on GitHub (Jun 27, 2016):
Nice proposal.
Please keep in mind that patch panels could be ordered horizontally or vertically. This would be a great enhancement.
@doon commented on GitHub (Jun 28, 2016):
This is where I've always run into roadblocks in designing the schema, especially when looking at some of the cross connect panels we have here. Rack 1, 2U Fiber Shelf in pos41,42 6 Cassettes (Duplex LC) and each cassette goes to a different Rack / Room, etc. So the schema kept getting hairy. couple that with input / output / the ability to connect to either a interface, another patch panel, or say a circuit / provider it snowballs pretty quickly.
@puck commented on GitHub (Jun 28, 2016):
With @doon's comment about things snowballing quickly. My thought would be to treat anything that a cable can terminate on as a port. Each socket on a patch panel has two ports, one on the back for structured cabling, and one on the front for patching (and yes, I'd treat a punch down block as a port). A cable can have one or more plugs on each end. Those plugs can go into ports. If you need to connect two cables together it might be able easier to have a coupler that has two (or more) ports on it.
This kind of model should also support octopus cables, with mismatching number of plugs (think MTP on one end and many LC on the other).
Cables could have metadata with them about the maximum speed they support, colour etc.
This model also extends to power infrastructure where PDUs can be treated as patch panels. DC bus bars etc.
@mdomore commented on GitHub (Jun 30, 2016):
For a patch panel i create a device type and i insert 1 interface front and one interface rear for each FO patch in the patch panel.
i connect rear from patch 1 to rear from patch 2 and front to active devices.
the only big problem is the form factor.
it's a trick but if it can help others it's free
@isodude commented on GitHub (Jul 1, 2016):
@mdomore Nice trick, I figured I would solve it that way as well, however, I loose the connection between server -> switch in that case since it is just a connection in terminology.
I would like a socket-port, possibly a hub, since that's exactly what it is :)
That would mean instead of add a 10G SFP+ interface I add a Hub instead where you can connect several interfaces. Maybe that will also shorten the timespan for designing the models since we don't need to invent that much around it.
@isodude commented on GitHub (Jul 1, 2016):
I also noted that it's not possible to add new Interfaces, if we could focus on that part, maybe this is just a matter of the SysOp to define them and specify how many connections a interface can have.
Today there's not a XenPak connections i.e., also not 10GE Copper. That would be nice be able to add yourself.
@mdomore commented on GitHub (Jul 1, 2016):
Yes and if we can add connector type like LC, SC, SCA, LCA ... this could be really usefull
@brianaddicks commented on GitHub (Aug 3, 2016):
Something else that I think goes along with this feature is tracking the usage of a patch panel. It'd be great to see that on the Rack list next to the RU Utilization %. That way if I have a rack with lots of free physical space but no patch panel room I won't accidentally tell someone it's available to use.
@swickstrom commented on GitHub (Aug 8, 2016):
My current bodge for this is to create a parent device which acts as a copper/fiber panel, and then create child devices that serve as copper/fiber cassettes. Each cassette has ports and there is a
crosslinkinterface under theOOBinterfaces that connects the cassette pairs. The cassettes are mirrors of eachother as far as port assignments go; if you wish to determine where a device onport-01traces to you follow thecrosslinkport to the cassettes mate and see which device is onport-01.The issue with this idea revolves solely around the binding action of interface types. I specifically ran into issues representing console/serial connections; due to how console interfaces work, I had to have the console ports on the cassette in the rack with the console server be
OOBports, and the console interaces in the rack with the servers act asconsole serverports in order for the ports to be assignable.Enter the
passthroughform-factor. Apassthroughform-factor would be defined as:• Having an
inand anoutinterface assignement• Having no interface type; allowing it to bind with any interface and not require pre-requisites such as being a
console serverport or being anOOBport@amatamalas commented on GitHub (Aug 31, 2016):
We are currently using openDCIM and one of the point to start using in the past was the ability to track cabling (but missing the IPAM feature like netbox has). Basically you have devices with data ports (network, console,...) and power ports (to connect to your PDU). The Patch panels are the only special object which have front and back port in order to be able to represent all the patching path. Then, each connection can have a label and you can represent the full path of a connection by searching by one of the labels of any cable in the path. I think this is really useful when you need to dive into the datacenter. The form factor, cable type (ethernet, fiber, coax,...) or even the color of the cable are very good attributes (optional and configurable) to have into account. I'm not a fan of reinventing the wheel and maybe it could be a good starting point for the cable plan design, as it's working very well for us. Of course there are quite a lot of usage cases and maybe this is too simple to cover all of them.
@iamdadmin commented on GitHub (Sep 7, 2016):
One way you could accommodate single connections would be to allow edit the device types and devices forms to support "back-to-back connection" in the same way that you have "Power Ports" and "Interfaces" only this type of connection has an A side and a B side.
The A side can be connected to a fibre bundle such as Device: "48 CORE BUNDLE 4728" which in turn can have back to back connections. In this way you'd eventually describe the entire path.
@doon commented on GitHub (Sep 7, 2016):
I've been toying with this on my own (in ruby/rails as I not so good with Parseltongue :)).
What I've come up with, and it seems to work so far...
ports belong to a PortGroup, which then belongs to a PortGroupOwner, these are basically abstractions to represent anything that can have a "port". So for example.
PortGroupOwner == ASR9K
PortGroup (index: 0) GigabitEthernet0/0/0
Ports(0) GigabitEthernet0/0/0/0
Ports(1) GigabitEthernet0/0/0/1
or
PortGroupOwner = RandomDellServer
PortGroup (index: 0) Mezzanine Ethernet Card
Ports(0) Eth0
Ports(1) Eth1
or even an abstract entity such as a hand off to another carrier,
PortGroupOwner = RandomCarrier
PortGroup(index:1) 80SS Handoffs
Port(0) TieCircuit XYZ
Then each circuit consists of port connects (cross connects).
this connection has a port_a and a port_b (Which are both FKs into ports) and both are required. the circuit_id is also required unless this connection is marked as permanent.
port_a and port_b have unique constraints on them. So each port can only be used once as port_a and once as a port_b.
Permanent port_connects are used between panels, and are generally created at panel creation time. I have scripts that build perm connections between 2 port_groups with the same number of ports to automate panel creation).
So when assigning to a circuit I just need to create connections between 2 ports.(service tries to create it as a,b, and then b,a, and both violate constraints it fails). if either ports belongs to a perm cross connect, then it updates the circuit_id for the existing port_connection for those ports, else it just creates the new one and assigns the circuit id.
Once we have all the ports mapped out in the circuit you can find the endpoints by taking the set of ports as A ports and the set of ports as B ports and look for which elements are exclusive.
To graph it can just dump it into dot/graphviz as the the series of cross connects.
to build a DLR or the like can just start at one of the ends and follow the port connects til you reach the other end.
So far it seems to solve most of the issues/requirements that I have.
Hope this helps, or feel free to tell me I Am crazy/misguided/etc..
@harsh-kotak commented on GitHub (Sep 20, 2016):
[Suggestion] You can start with adding colums line this in the interface connections menu:
A-END -- Z-END
Row/Rack DeviceA Interface Media -- Row/Rack DeviceB Interface Media
where media is cable/connector type. e.g.: SM LC, MM LC, QSFP twinax, copper, etc.
This does not map the entire Layer 1 but helps to keep track of cabling type used.
@nward commented on GitHub (Oct 1, 2016):
Something that would be useful is to treat fibre pairs as individual fibres, or bundle them together. Some providers track every strand, rather than pairs. This would also enable use of BiDi optics, etc.
MPO/MTP support would be very useful, also. I currently model MTP connected patch panels in Device42 by pretending that there are many panel-to-panel cables, which isn't right obviously!
Perhaps support for inline circulators and taps, too. A company I work with splices these inline, rather than going through a rack mounted device.
@RyanBreaker commented on GitHub (Jan 31, 2017):
I'm glad to see this is on the roadmap to be implemented as this is the most-needed feature for us to switch from a spreadsheet for tracking all our cable paths. Is there an ETA or milestone for when this will be implemented?
@Jluisito commented on GitHub (Feb 15, 2017):
+1000! Great feature to a great software.
It will have a diagram of the connections like Opendcim does?
Thanks
@quentindavid commented on GitHub (Feb 17, 2017):
Hi, this feature is the only reason I cannot use this software at the moment !
May I help you to achieve this ?
@th3goose commented on GitHub (Feb 23, 2017):
I've been spending a lot of time prior to finding netbox thinking about cable and pathway design and documentation. There are really three main parts to this, paneling with ports, cabling and "pathways" which would include not just the cabling but the duct and construction types being aerial, buried, riser, etc, as well as geo information that would be used to create a geo based map from the lat/lon attributes of each part of the pathway. Cable is more of a logical construct in this methodology (including a relationship with the pathway its a part of), but the most important part is the pathway concept that basically lays the groundwork for what your trying to achieve, similar to a circuit.
@jeremystretch can you provide anymore color to what your thinking here with panels, cables, etc? I'd be interested in helping depending on the need for this.
@jeremystretch commented on GitHub (Feb 23, 2017):
I don't think we'll extend beyond representing physical cables and patch panels in NetBox. This by itself is already fairly complex, especially when you consider that cable types can change between the front and back of a panel. Attempting to map the actual physical path is more of facilities management thing, and well outside the scope of NetBox.
@th3goose commented on GitHub (Feb 23, 2017):
@jeremystretch I think thats fine, how would you suggest a custom module be created to interface with netbox all the while limiting the forking of code and continually having to merge branches? Since netbox is its own branch I'm having a hard time wanting to fork it to create a custom module and losing future fixes from the main branch.
@afics commented on GitHub (Mar 29, 2017):
I need support for fanout cables, for example QSFP+ to 4xSFP+.
@VictorJ76 commented on GitHub (Apr 3, 2017):
Any news about patch panels ?
@hyberdk commented on GitHub (Apr 3, 2017):
+1 here, fiber patch panels are a must for me (so I can connect 2 racks together).. else it looks fantastic..
bounty?
@oasys commented on GitHub (Apr 3, 2017):
Registering my support/interest.
This is a critical feature for us.
@RyanBreaker commented on GitHub (Apr 3, 2017):
The same for our site, we're looking for a system for tracking specifically this as well as the other features in Netbox.
@VictorJ76 commented on GitHub (Apr 4, 2017):
Same for me. Critical feature, this is one of the main uses of netbox for us.
@nward commented on GitHub (Apr 4, 2017):
2 ways to make sure stuff gets done in open source - do it yourself, or sponsor someone (the project/a third party) to do it.
I don't have time for the former right now. There's enough people on here that if we all threw in some $ it'd be a reasonable donation. I'd put in a few hundred I reckon. It'd save me paying for device42 or whatever...
@isodude commented on GitHub (Apr 4, 2017):
I love the netbox concept. Not having proper cable plants is the reason we can't use it.
I believe we're also interested in putting in cash for solving this issue. How do we proceed?
Any developer that are interested in looking at this feature? I understand that this could easily be a mess if done incorrectly?
@RyanBreaker commented on GitHub (Apr 4, 2017):
I'd be willing to put the development time in myself, just not sure where to start with this. I'll look around the code later in my free time to see the options on where to best proceed.
@jsenecal commented on GitHub (Apr 5, 2017):
I did some work on this kind of use case on a project I was working on in a
previous company. It happened to be a Django project so if Jeremy feels
like it maybe we could start working on the feature and submit PRs?
Did you already have something in mind?
On Tue, Apr 4, 2017, 12:05 Ryan Breaker, notifications@github.com wrote:
@mkbt commented on GitHub (Jun 19, 2017):
The company I work for will sponsor development time on this also if needed.
I take it the schema design hasn't quite made it yet and the first step would be organising schema/design then having it signed off by @jeremystretch ?
@RyanBreaker commented on GitHub (Jul 12, 2017):
@jsenecal Sorry for late reply, I'm not as active on Github as I used to be. I didn't have any immediate ideas and was hoping to contribute to any development attempts for this mostly because I'm just unsure where to start. But as soon as anyone is able to share a roadmap or more solid ideas on how to get started I'd be much more able to jump in.
@mkbt That would be FANTASTIC. Please do share updates and anything anyone else can do to contribute.
@beggles commented on GitHub (Jul 18, 2017):
Would a starting point be to show basic relationships between equipment racks, patch frames (cross connects), media type, and network / storage switches? For instance;
Cabinet X has copper cabling to patch frame A, and Fibre cabling to cross connect B, and needs to interface a host via copper to Network Segment Y.
Network Segment Y is available on a variety of physical switches, though not all switches are available via patch frame A. In order to "join the dots", we would therefore need to record the particular patch frame where each switch is loomed to.
This method assumes that the equipment rack chosen has cable plant available, and that the outcome sought is to "simply" identify a switch candidate to attach the host to - and that there would be a validation of port availability, against the candidate switch's current run state.
@beggles commented on GitHub (Jul 18, 2017):
So I've done some more thinking on this and have the following to add;
Physical elements to "start" with would be;
A patch panel may be associated to one rack or many (consider consolidation points). A switch loom could be associated to an individual switch or module within a larger switch.
A cable (or datapoint) interconnects a Patch Panel / Switch Loom to a Cross Connect Frame. Similarly, a cable (or datapoint) attaches a device, be that server or network, to another device.
Cable would be further categorised as Fibre (MM/SM) Copper (Cat 5e/6/6a). Similarly Patch Panels would be categorised as Fibre or Copper, you may want to add the presentation interface type for Fibre, ie LC/SC/MPO etc, though this may have limited mileage.
That should mostly take care of the physical layer, and I should add that Patch Panels being cabled to Cross Connect Frames would not be exclusive, Patch Panels can connect to other Patch Panels, just as Cross Connects Frames to other Cross Connect Frames.
As this would go toward some sort of database schema, you should then be able to search for cable capacity between Patch Panels and Switch Looms (Switches) for instance, and in turn recommend the "path" to patch through to get the desired connection.
Finally, further to my original post, a switch (loom) could then be associated to a Network Segment definition (or multiple, such as Corporate/Extranet/Management etc), thus allowing Netbox to "suggest" how to provision the desired "service" based on Equipment Rack where a host is to be installed, and its proximity to a particular Network Segment, as mapped through the above "structure".
@puck commented on GitHub (Jul 18, 2017):
Sounds good.
With the cable interconnects, remember that one cable may go to one or more ports on the patch panel. For a cable which goes to more than one port on a patch panel, it may only go to one port on the other end.
@beggles commented on GitHub (Jul 19, 2017):
@puck, so I think you are referring to something like an MPO trunk into a Fibre cassette? If so, I was more thinking of a "cable" being a discrete mapping to a physical data point on a one to one basis, rather than mapping pairs into a multi-core trunk, which for my requirements would be more than sufficient. To expand upon this, if I was to install a 12 port LC cassette into a patch panel I would add 12 cables to that, rather than 2 MPO trunks - in more concerned with physical capacity / capability, if that makes sense.
@nward commented on GitHub (Jul 19, 2017):
It is reasonably common for a LC->MPO cassette to be installed, where the MPO cable goes to an MPO-only patch panel.
Additionally, it is common for MPO connected equipment (like 100G and 40G multimode) to be split across many single fibres in to a DWDM transponder.
It is common for a 100G or 40G interface with MPO connectors to be split in to pairs of 10G and treated like individual 10G interfaces (look at modern 1RU switches with only QSFP28 holes, for example).
Additionally, MPO cables have multiple different polarities - straight through (A), rolled cable (B), and swapped pairs (C). These have impacts on all of the above items.
I think MPO needs to be modelled, with a lot of flexability.
Single fibre working is common also, so I think fibre needs to be modelled with pairs (and multicore like MPO) being templates for grouping.
I think also that copper should be modelled as individual wires too - what about those environments where someone sticks one of those 2:1 Ethernet cable sharers in place to save a few pennies? Or a cable is rolled for connection to a serial console? RJ45 1:1 should be a template of 8 cores. It should be possible to model other constructs and trace every single electrical connection through between devices.
@nward commented on GitHub (Jul 19, 2017):
Patch panels are also interesting - many patch panels are modular. Some vendors make the mistake of assuming patch panels have only one level of modules (or worse, no levels!)
I have seen patch panels which accept multiple levels.
I would also like to see fibre storage drawers modelled, so it can be documented where a fibre's slack is taken up. This might be a bit tricky though so is a nice to have!
@beggles commented on GitHub (Jul 19, 2017):
@nward at the simplest level if you represent a "data point" as a single point of connect, and then in turn assign it a media type, ie. copper outlet, fibre outlet, MPO trunk etc. Based on the media type you "could" then define a number of connection types (both pre and user defined), where you could then "allocate" the individual conductors/cores, which in turn could be "fanned" out to multiple devices, consider where using all four pairs on a copper data point to deliver multiple modem lines, or an MPO fan out fable to attach multiple 10Gb interfaces across a number of hosts - this would also then require, as an option, the ability to define each core/conductor of a data point.
I think the first step in "solving" the problem is to "map" out a structure for the physical layer, and then define connection types per each media type - if that makes sense?
@nward commented on GitHub (Jul 19, 2017):
I don't really understand what you're saying, sorry.
Let me take a stab at it, it sounds like we might be roughly on the same page here.
I think the best way to model this is to model things at the lowest level, and group them together (i.e. multiple copper cores in to a single cat6 cable, or two fibre cores in to a fibre pair) with some information about polarity (i.e. cores 1/2 in a fibre pair are swapped).
Perhaps most users only look at things at that higher level, and perhaps the initial implementation doesn't even expose the lower level information, but, modelling that up front is in my view the best approach.
@beggles commented on GitHub (Jul 19, 2017):
For the most part, yes, I would expect that users would in the first instance treat a "data point" in a patch panel as nothing more than a place to patch something into, like a server for instance - so by default a given cable is nothing more than a point of connection, irrespective of its core/conductor count, basically a means of getting from point A to point B, and sometimes via point C.
Where there is a need to further "divide" a cable into its cores/conductors, this would be done by means of defining a "connection type", where the default is 1-1, or maybe "host". The connection type characteristic would then be mapped "over" the cable, and in turn inform the active characteristic of the cable, for both A and B ends, if other than the default "host" connection characteristic.
@nward commented on GitHub (Jul 19, 2017):
From what I can tell, you're attempting to model this from an arbitrary level of the stack - i.e. what you consider to be a "normal" cable, and apply characteristics to it if it is not normal (though perhaps "normal" is a characteristic in itself). That is going to drive a lot of complexity for situations we can't foresee right now (though I am fairly certain that with any data model based on this approach, I can foresee problems with by looking at environments I work in which have some fairly complex cabling at times).
Let's model it from fundamental components individual cores/conductors, and group things in to digestible constructs like "cables" for those who don't need that level of detail. I imagine most users will only care about "cables" - be it a 8-core cat6 with straight-through RJ45 termination, or, a 2-core fibre pair, or whatever - however, having that level of detail underneath means:
@beggles commented on GitHub (Jul 19, 2017):
I don't know about you, though in my environment each cable is presented in a "standard" configuration / polarity. An RJ45 outlet for instance is terminated to the 568A standard, and any "alteration" to that is made by what you attach to it - i.e. where we need a serial console, our terminal server is configured to present the "roll-over", or where we need an ethernet crossover, we use a crossover lead.
The cable at its most basic level is a nothing more than a mechanism for connecting endpoints together, irrespective of whether they are serial consoles, ethernet, voice etc, for copper and in a similar manner for fibre.
Maybe I'm over simplifying it, though coming from a cabling background, there is pretty much nothing you can't do with a cable... :)
@nward commented on GitHub (Jul 19, 2017):
In "telco" world we document cores. Copper is typically documented to the "pair" level, but even that can occasionally end up with weird things happening.
It's more and more common for enterprises to buy dark fibre - here in NZ it's very cheap to do so. This is provided as single fibres, not pairs. This is often presented on a patch panel as an LC pair, however.
So, now in a "cable-centric" model, within your cabinet you can run a single cable which is a pair - which is easy. However, you need to have a "cable" which is actually two physical cables out the back, which are two seperate physical circuits which may take different paths through a city. They will almost certainly be spliced at different places.
So, how do you model that? Do you create a special single-use cable that somehow has different circuit identifiers for each fibre? What if they each go to different devices at the far end - perhaps CWDM muxes? I'm sure it's do-able, but, are we just figuring out chains of workarounds?
I think the best approach is to be able to provide the average user with an interface that lets them connect "cables" just like you want - but under the hood model that in much more detail, so that a user can drill down and document individual cores/conductors how they see fit to match their environment - remembering of course that not everyone does things the "right" way, and there are a lot of legacy/hacks around the place.
@nward commented on GitHub (Jul 19, 2017):
Taking things one step further, having indefinite "layers" of encapsulation would be helpful here also.
This would allow us to document, for example the following layers for a cable between two buildings:
In this way, we can trace connectivity between two devices even through complex topologies.
Perhaps this allows modelling of active circuits - i.e. perhaps a telco provides us a single port over which we have multiple services (perhaps differentiated by VLANs). Those services may terminate in different remote locations - or perhaps different sets of remote locations, in the case of a multi-point service.
I guess it depends how far up the stack we want to go - but providing indefinite layers here would no doubt help.
@beggles commented on GitHub (Jul 19, 2017):
I like the idea of "encapsulation", which you could use to as you say layer "characteristics" to a cable. In the "telco" world how would you name / identify internal vs external cable routes?
@shepherdjay commented on GitHub (Aug 31, 2017):
We're also looking for this feature to manage patch connections across our mdf to idfs. Coming from Issue #912 where @jeremystretch specifically discourages using devices to represent patch panels.
@deggler commented on GitHub (Sep 1, 2017):
We too would be very interested in that feature. It would be helpful to know how many fibres are available between to patch panels.
@ghost commented on GitHub (Sep 8, 2017):
Any plugins or something available for this solution?
That's the feature I need for our management and I thought it's build into the DCIM as I installed it. Two days later I run straight into this feature missing.
Do you know when it will be implemented?
@flokli commented on GitHub (Sep 8, 2017):
@alsternerd For me, the problem seems to be that it's currently not clear how the cable tracking part should look like, on the data models and feature side.
There were various people here willing throw coding time on implementing it, but as previously written by @mkbt, first a concecpt/schema needs to be done and signed off by @jeremystretch
@hursjohn commented on GitHub (Sep 9, 2017):
Here is an example model. I think this should cover most cable networks whether it's twisted pair, fiber, or coax. Would love some feedback from those who are also managing plant cables.
Cable routes are made up of many segments. Each cable can be a different type (Coax, Twisted Pair, Fiber). Each medium has a type (e.g. Twisted Pair: #19awg/#22awg, Fiber: OM2/OM3, Coax: 500 Series/750 Series.)
Each cable segment belongs to a cable route. Cable Segments have a GPS Path, pair/fiber count capacity (e.g. 144 fibers in segment, unsigned integer,) and a ground level offset (negative value would be below ground level). A and B sides of a segments are terminated to a PoP (Point of Presence or Interface Point.)
Cable Counts belong to a segment. Each count must be unique within a cable segment. Each count can be assigned to a netbox circuit. Counts can be given a status (Left-In, Working, Bad, Vacant, etc)
Each PoP is of a certain type have the capability of interfacing 1+ cable mediums (Fiber, twisted pair, etc). For example in a Telephone Co network a Pedestal would be a PoP and you can bring in Fiber or Twisted pair into. A Load Point or T1 Repeater would only be able to interface a Twisted Pair medium. Coax plant may have a fiber fed node that can interface fiber and coax mediums.
Within a PoP you may cross connect cable counts to other cable counts (limited to cable segments that terminate in that PoP.) You would probably want a tool that can mass create these cross connects. Example: you have a cable segment with a countCap of 50 going into a pedestal (PoP). Then you have two 25 count cable segments leaving the pedestal. The ability to mass cross connect the 50count cable to the other two segments of 25 count.
@freddy36 commented on GitHub (Sep 11, 2017):
@hursjohn I don't see how I would document complex fiber plants with your model.
Our usecase (FTTH network):
We've various type of ducts, there are various cases where a duct is inside a duct (subducts).
E.g. a worst case nesting scenario:
E.g. we bury a 110m diameter duct, later we insert a duct splitter, splitting that duct into e.g. 2x40mm + 2x32mm ducts. These ducts might contain more subsucts (micro pipes) with e.g. 7mm diameter. The micro pipes finaly contain the actual cable. of course the other duct layers types might contain cables too.
At the beginning e.g. there are e.g. 20 x 7mm micro ducts along the street (They all have the same gpspath/start/end points. Once a new bulding is build/an existing one decides to get connected one of these micro ducts is cut and connected to the ducts leading to the building (start/end/connection point and gps path changes for one of the micro pipes).
A simple redirect looks like this: https://upload.wikimedia.org/wikipedia/commons/3/31/Offenburg-Glasfaserausbau-Telekom.jpg
But there might be more complex scenarios too: http://www.tcss2011.com/images/stories/Products_5.jpg
The (sub)ducts might end in or pass various manholes.
Bigger ducts always end in man holes or (street) cabinets PoPs. Smaller micro ducts or groups of them will end in splicing enclosures (e.g a patch panel in a rack or a dome enclosure in a manhole).
Once there's a continius duct route between the PoP and the building a cable is blown into the duct. On the building side it will typcially end in a small wall slicing enclosure in the basement. From that wall enclosure there's an inhouse cable going to the wall socket where the customer wants to connect his internet devices. Sometimes (especially in bigger multi apparmnet buildings) the cables are protected by micro ducts too (One duct to each appartment).
Either in the basement (multi appartment) or in the street enclosure the customer fibers are connected to splitters. e.g. 1 fiber comming from the central office PoP is splitted to 32 customer fibers.
The higher count fiber cables comming from the central office consist of several mini tubes each containing multiple fibers (12 or 24 typically). E.g. https://d3gx8i893xzz0e.cloudfront.net/fileadmin/telecommunication_systems/products/fiber_optic_cables/fiberconnect.jpg?1479730041
In the splice enclosures at the street the fibers from the central office cable are placed in the corresponding splice trays, the common port fiber from the splitter is then routed and spliced to a splice tray with a free fiber.
The cable towards the central office might be spliced together at various other splice enclosures. At the end it will arrive via various duct segments in the central office PoP in a splice tray inside a patch panel. In the patch panal pig tails are spliced to the fibers from the cables and connected to the patch panel.
For fibers there are various types of connectors available. From simplex connecors where 1 fiber equals 1 port over duplex (2 fibers=1 port) and quad (4 fibers=1 port) to MTP/MPO connectors with 12 or 24 fibers per port. When doing a "cross connect" it should be possible to do it on a port group level (e.g. connect both fibers of a duplex port via duplex cable) but also on a sub port (connect just a single fiber of a duplex port) level.
Especially for data center environments it should be possible to configure fan out cables, e.g. a cable with a 12 fiber MTP connector on one side and 6x2 fiber duplex connectors on the other side.
Are such complex documentation levels targeted by netbox?
@shepherdjay commented on GitHub (Sep 12, 2017):
@freddy36 I think if we try to come up with a solution that could match every edge case we could go another year on this issue with netbox still not being able to handle even the simplest of cable runs and patch panels. It is an odd feature to be missing from a DCIM tool and probably hampers adoption. Instead of chasing the last 5 percent or 10 percent of use cases can we get something in place that will handle the majority of environments which probably have simple runs and carrier x-connects / dmarcs?
@freddy36 commented on GitHub (Sep 12, 2017):
@shepherdjay that's why I'm asking if such a detailed level is within the scope of netbox.
If it is I'm happy to invest some time designing a data model. I fully agree that supporting all these cases from the start would be a lot of work. But just implementing a very basic A-to-B abstract connection will cause migration trouble later on. We should at least support designing cables on the per fiber/wire level with corresponding nesting/grouping to avoid trouble later on. At the beginning it might be only possible to connect them to patch panels, later more details could be implemented (gps path, ducts, ...). But I believe the data model should support the more complex setups from the beginning.
@deggler commented on GitHub (Sep 12, 2017):
@freddy36 i agree. for that level of details you should use a GIS System.
It would be great if we could document what fibers between two sites are in use and if i could track patch cables from panels to interfaces.
If a model could be achieved that inserts connection points in a existing connection of two devices it might be possible to provide a smooth Transition:
switch 01 port 1 -> switch 02 port 1
becomes
switch 01 port 1 -> patch panel 1 port 1 -> patch panel 2 port 1 -> switch 02 port 1
The level of details for all the interfaces and connection would not need to be so high to already be helpful.
@freddy36 commented on GitHub (Sep 12, 2017):
@deggler that's what we're doing today. The problem is that GIS systems are lacking features like IP address management, etc. so you've to maintain the data in >= 2 systems. And keeping them in sync requires human discipline. That's why we would love to see a project bringing these two worlds together and we're considering netbox as a candidate for this (long term).
There's already a request for floorplans (#114), which is a kind of a GIS system too.
@isodude commented on GitHub (Sep 15, 2017):
Well, can't we split it here, to make a distinction. Have in my mind that I
haven't read #114 yet.
If the cable plant does not have a notion of GIS and instead can be related
to GIS data, that make things easier. Let's say I document fibre strands I
can put GIS-data on the whole fiber etc. Which would make it simpler for
those putting in GIS data as well.
So a separate model for GIS data may be of use even for floorplans as well.
And it would make the design of the cable plant easier.
From my point of view we could separate the logic of cables between virtual
and physical connections. That way the virtual model could accomodate for
things like bonding etc, physical models handle how the wires go and the
gis data where stuff are.
On Tue, 12 Sep 2017, 13:54 Frederik Kriewitz notifications@github.com
wrote:
@liotier commented on GitHub (Sep 15, 2017):
Regarding geographical features, from my decade of experience managing telco information systems... The main model must be topological - devices and their connections. The geographical position of a device must be a mere attribute - or it can be recorded in a separate system where the device's identifier is a foreign key.
So, the topological model may be about fibers within a full trench/duct/cable/tube/fiber hierarchy and the fibers may be spliced together within a chamber/enclosure/tray hierarchy or connected to a pigtail plugged into a patch panel... But all that must be purely topological. Then some of these objects may have a geometry (node or polyline). Some of them may even inherit another object's geometry (for example if you decide that a cable does not have a geometry of its own but is only associated to successive ducts & trenches). But whatever the modeling choices, one thing must be clear: a network model must be topological first - geography shall be a distant second.
Damn, now I feel the kick of PTSD from ancient exposure to information systems born as GIS and mutated into logical layers... Please spare yourself that !
@liotier commented on GitHub (Sep 15, 2017):
Yes, this is important... Two visions of the world are locked in an eternal struggle:
In the active devices vision, one port may receive a connector which bears a pair of fiber.
But in the passive infrastructure vision, the same connector is seen as two different fibers.
This is the bane of my existence.
One possible model:
So the tracing of individual optical routes can be performed while recording the fiber pair as a business object.
@gtreliv commented on GitHub (Sep 15, 2017):
Reading through this topic, and bearing in mind Netbox's stated aim of being an 80% solution, it made me think about what I actual need Netbox to do for me, rather than everything that could be done.
So here is my stab at a minimal solution.
Connections & Circuits continue to terminate on devices and do not terminate on patch panels, however they contain an additional path record.
Path Records contain an ordered list of panel/port references, multiple ports may be specified for each hop.
Path
[ [ PanelId1:[PortId1, PortId2],
[ PanelId2:[PortId1, PortId2]
]
Panel
{ PortId1 : PathId1,
PortId2 : PathId1,
..
}
Adding connector / media fields to port record would be useful for reference but I would avoid building any logic around that other than to document & display and leave it to the human.
Beyond that I think that is a bit of a rabbit hole. This would cover what I need, which is to free my organisation from the tyranny of spreadsheets!
I have used tools that model the passive infrastructure, and that gets complex fast.
@liotier commented on GitHub (Sep 15, 2017):
I do not have much experience with the electrical stuff found in LAN & hosting, so I'll focus on the optical world...
One object that we have found critically important is the optical route - the object that represents the service rendered by the passive optical infrastructure layer. In my definition, an optical route is an optical continuity (light gets in at one extremity, goes out at the other) that has been labeled so that it can be handled.
The optical route is the object to which optical path elements (connector, fiber, splice etc.) may be allocated within Netbox but it may also be useful to Netbox to point to passive optical infrastructure resources that another system may manage.
The optical route frees the user from having to fully model the physical topology: some physical connections may be recorded, others may have no record... It doesn't matter: that model supports partial knowledge of the infrastructure layer. Where the physical connections are recorded, Netbox can manage actual topology, where they are not, it merely manages fiber cable capacity and some fiber cables may be missing altogether... That flexibility is useful, especially as few users have full knowledge of the infrastructure layer (yes, even telco inventories have some shameful holes)
An L2 link between two ports may use one or more of optical routes:
Now, I don't know how that sort of model feels to RJ-45 people...
@freddy36 commented on GitHub (Sep 18, 2017):
@gtreliv the problem with your suggested data model is that you could easily end up with inconsistent data (the optional path doesn't match the Connections/Circuits). To avoid it I believe it's much better idea to extract the high level (end-to-end connection) information from the lower level information (path).
@falkowich commented on GitHub (Oct 18, 2017):
Hi,
I started to look at this problem/challenge today, and found this thread :)
I'm interested if there perhaps where any work started out there in one of the 450+ forks?
I was looking at doing something simple with racks -> odf's.
Or even a crud form where you can document the odf and fibers in a rack?
But there seems to be a great deal of knowledge and work already done in this thread..
It seems that there is a few suggestions out there, like the datamodel from @hursjohn.
From what I read of the model that really fits the 80% rule right of the bat :)
What do you guys think?
--
Regards Falk
@RyanBreaker commented on GitHub (Oct 24, 2017):
To add my simple two cents, I think it'll be best to just have a dedicated "Patch Panel" device model, something that we can associate a "backlink" to all interfaces added that can connect to anything else, much like how the LinkManagement RackTables plugin adds the equivalent functionality to its target project.
I unfortunately severely lack enough Django experience to try to bring this into fruition myself though, as much as I've been trying to toy around with it in my own local branch.
Thoughts?
@RyanBreaker commented on GitHub (Dec 14, 2017):
Now that I think of it, I've been considering changing the
InterfaceConnectionmodel fromOneToOnefields toForeignKeyfields, and add validation depending on if the device its going to has "couplers" or not, if that makes sense. I like this approach because it's simple and to the point; devices are already very flexible in how they can be configured in netbox, and using that to add ports/interfaces to a "Patch Panel" device, however it's used, should be able to cover most use cases I think.Thoughts? I've already started trying this out in my own branch.
@callumstrubi commented on GitHub (Mar 5, 2018):
Visualising these things is less of a problem for us, but we do have a very real use case where we have structured cabling in the racks and then a network rack that aggregates the structured cable. What this means is that a link between a server and a switch may actually comprise of 3 separate cables.
[ Server ] > [ Top of Rack Patch Panel ] > [ Network Cabinet Patch ] > [ Switch ]
I'll describe the patch panel for verbosity: A patch panel would consist of, for instance, a bank of RJ45 or LC connections that then link to a corresponding patch panel in another location, joined by copper/ethernet (Cat5e?/Cat6A?) or Fibre (SM/MM OM[3,4] for instance). The cable type is significant, as of course each cable type has different limits on bandwidth and or optical impedance on the route.
In terms of just mapping a raw connection this would be useful even if it was agnostic about the cable type, but if we do know the cable type, and we know what length it is, and the number of "Joins" in a link, we can actually predict the kind of optics one would need to transmit a signal effectively. Although optics are rated for "xxxx meters" we know that vendors include slack in this number, but that each join or connection in a path can have a notable drop in signal intensity. A link that is 300m long, with 5 connections may actually not work on a SR optic rated for 300m for instance. This would be an added value to being able to map this data.
[edit]Another thing that we may need to support that is already mentioned here is the 40G QSFP+ rat tail to 4x 10G SFP+ connection from a switch to 4x nodes, for instance. Mellanox produce such a thing, but I'm sort of hoping we dont buy such an ugly solution...[/edit]
In conclusion, a patch panel can just comprise a series of interfaces that are dual connected, and is no more complex than that. But the options of data analytics using the information of cables that join the patches, their lengths and length of cables between can actually be valuable.
@leoluk commented on GitHub (Mar 5, 2018):
40G QSFP+ rat tails are very common, we have quite a few of them.
@goebelmeier commented on GitHub (Mar 5, 2018):
Visualization should be easy as soon as the data model has been defined. Our use case is pretty much the same. We do have centralized network racks (horizontal distribution area) at each site. These have patch panels which are guiding to racks in the same and the next row. Furthermore there are patch panels guiding to the facilities main distribution area (meet-me room). Mostly there is one rack for copper cabling and another one for fiber cabling. Usually we do have two redundant rack rows, let's say row 1 and row 2. Row 1 Rack 1 and Row 2 Rack 1 do have fiber patch panels installed. Each patch panel is basically an enclosure which holds up to eight optical cassetes. These cassetes do have ports presenting to the front and ports presenting to the back. Let's say 12x LC Duplex at the front and 1x MPO at the back. The fiber type of this cassete is OM4. Four of these cassetes will guide their MPO cables to row 1 rack 3, the other four cassetes are connected to row 2 rack 3 with their MPO cables. So basically front-end ports 1-24 are connected to row 1 rack 3 and 24-48 to row 2 rack 3.
The other way round, row 1 rack 3 has front-end ports 1-24 to row 1 rack 1 and 25-48 to row 2 rack 1.
So for inter-rack fiber cabling we're using multi-fiber cabling and copper cabling is straight 1:1 between two patch panels.
Some devices are connected with MPO fanout cables either directly to a switch or to a patch panel. If they're connected to a patch panel, we would use the first port of cassete 1, first port of cassete 2, first port of cassete 3 and first port of cassete 4 for redundancy reasons.
Each patch cable between a device and a patch panel or between two patch panel front-end ports do have an alphanumeric label, the same at both ends of each patch cable. This label is currently globally unique, but i think it would suffice if the uniqueness is limited to the site.
I think the last thing to think about is the naming convention of the fiber connectors. We usually use LC or SC duplex connectors, so we're always talking about ports. Our providers are usually talking about fiber strands, mostly strand A and strand B. A combination of stand A and strand B would result to our duplex port.
I hope this description will help you to better understand our needs. If not, don't hesitate to ask, i'm doing my best in supporting this issue.
@candlerb commented on GitHub (Jun 29, 2018):
@freddy36:
I have been working with some complex GIS systems and I'm leaning towards the opposite view: the key documentation should be the end-to-end connection, and the lower-level information should be supporting.
The reason is that the end-to-end connection is "truth": you can validate it using LLDP, or using OTDR equipment or whatever, to prove that there is an end-to-end path from A to B. But if you derive that information from a micro-model of the individual connection segments which make up that path, then any small mistake in updating that model can result in the information that A is connected to B to be lost forever.
Also: you may want a connection from A to B to be "planned", but at that point not define the detail of what route it should take.
@candlerb commented on GitHub (Jul 3, 2018):
I will try to gather together some possible requirements.
Canonical use case
The connection comprises multiple segments. Intermediate segments are joined together at ports.
Since in general the patch panel or ODF / fiber tray takes rack space, it should be modelled as a Device (or something which consumes rack space in the same way as a Device).
Additional use cases
Not all of these will be in scope, but are listed for consideration.
Console connections
Netbox has a separate model of console ports linked to console server ports - but it is not uncommon for such connections to be routed via the same RJ45 patching infrastructure as interface connections. Therefore it's desirable to document them in the same way.
There is no need for power port to power outlet connections to modelled in more detail; however, there is an argument that if interfaces and console ports are modelled as explicit connections, maybe power connections should too for consistency.
Planned connections
A connection in Netbox can currently be modelled as "planned".
It may be desirable to retain the ability to have an intended connection from A to B, without actually choosing the physical route. This can easily be modelled as a direct connection from A to B, as if it were a single long cable, which is replaced with the correct routing later.
Conversely, it may be helpful to plan the individual segments, and mark them as active when they have been installed. This would also allow part of the path to exist, and part to be waiting installation.
It may be useful to label a connection as "path unknown" when you know from other sources of information that it exists (e.g. LLDP), but not the actual route taken.
Separate enhancement: it may also be useful to add a "deprecated" or "retiring" state for interface connections - indicating that the link is still active, but should not be used for new services.
Permanent and failed connections
The connection between two patch panels is permanent infrastructure, and it should be very hard to accidentally change or delete it (unlike the connection from a device to a patch panel, which is a flying lead)
Conversely, it can also happen that a connection between two patch panels (or one of the ports at either end) fails. Would like to be able to mark the path/connection as unusable - again this should be "permanent".
Port labelling
A patch panel port may have a label, in addition to its intrinsic port number. The labels at the two ends of a connection may be different.
A special case of patch panel is a wall jack. Patch panel 1 port 3 may be labelled "Jack 101-1"; and the wall jack in room 101 at the other end may be labelled "P1-3"
Add and remove intermediate ports
Given a connection from A to B, be able to edit its physical path by:
whilst ensuring the end-to-end connectivity remains.
For example: if you want to change
A--BtoA--X--Y--B, it would be easier and less error-prone to insert X and Y into the existing connection, than have to deleteA--Bbefore creatingA--X,X--YandY--Bseparately.In the case where
Xis part of a "permanent" connection (see above), then addingYinto the path can be done automatically.Conversely: if there is a path
A--X==Y--B, whereX==Yis permanent, and in the GUI I ask to delete the connection from A to B, it should ask me to confirm whether I want to deleteA--XandY--B, but not the permanent connectionX==Y.Incomplete paths
It may be useful to model partial connections, even when they do not connect end-to-end: e.g. if one patch cord is removed you could get
There is no longer no path from device A port X to device B port Y, but some of the intermediate resources are still in use. This is a situation which can occur in real life, and a report can highlight remaining cables which can be removed.
Related to this: if you have an existing multi-segment path, it could be helpful to be able to disconnect one end from a port and reconnect it to another port, without having to recreate the entire path from scratch.
(Netbox currently does not allow just the A or B end of a connection to be changed: you have to delete the connection and create a new one)
Cable attributes
A cable is itself a physical asset. It may be one of several different types (e.g. Fibre: MM/SM, single or duplex; Copper: CAT5/5e/6/6A). It has a length. It may have other useful distinguishing attributes like colour or ID label. It may have a manufacturer and model (e.g. Netgear AXC763 SFP+ cable). In some cases it may be expensive enough to have its own asset tag. Custom fields may be required.
This would apply to each segment of a connection. The implication is that these would become explicit editable entities, whereas in Netbox an InterfaceConnection can only be created, destroyed, or switched between Planned and Active.
There are a few odd cases where conductors are not mapped 1:1, e.g. RJ45 crossover cables - that can be just another cable type.
Splicing
Optical strands may be spliced directly together, rather than going via a device/port. (At worst, a "splice" pseudo-device could be used for this)
Optical loss
It may be useful to model optical loss, as a connection attribute and/or a cable attribute.
Simplex and duplex
Some cables are intrinsically bundles of a lower-level entity, e.g. copper wires in a CATx or stacking cable, or the 2 fibres in a duplex fibre cable. Netbox currently considers these as a single "connection" into a single "interface".
However it's possible that fibres considered as a pair when entering a device, need to be traced as separate strands in cable plant. The two strands will be numbered and/or coloured individually.
For example: connecting two 12-port LC duplex ODFs back-to-back with a 24-fibre cable. While each ODF may be modelled as a device with 12 duplex "ports", each connection from port X on ODF A to port X on ODF B will be two numbered or coloured fibres.
As long as these fibres don't take different routes, it may be sufficient to have a "fibre pair" as a segment type, with a pair of numbers and colours as its attributes.
Cable bundling
Multiple cables may be combined in a physical bundle with 1:1 endpoints: e.g. a cable linking two multiport ODFs, or a fibre optic "hydra" cable.
Within such a bundle, each component cable will be identifiable, e.g. with a number and/or colour.
Bulk insert
Possibly related to cable bundling: when creating connections between (say) two 24-port patch-panels, it would be helpful either to have a tool to bulk-create them from a pattern - similar to bulk-creation of interfaces - rather than having to enter them one at a time.
Cable splitting
There are more complex asymmetric break-out cables with different ports at each end. Examples:
(*) See "Simplex and duplex"
It's also possible that you can pull some fibres out of a cable and splice onto them, while leaving the rest of the cable to run on.
Internal cross-connects
There are some devices which contain internal connections, e.g. a patch panel which presents 24 x RJ45 to the front and 24 x RJ45 to the rear.
Another example is a fiber cassette which presents 12 x LC to the front and 1 x MPO to the back. This could be considered another instance of the "cable splitting" problem.
Devices with pigtails
Some devices have integrated pigtails which integrated connectors, e.g. certain types of GPON optical splitter. These pigtails can connect directly into another port without a separate cable.
These could be modelled as devices with ports, and separate cables going into those ports.
Physical cable route
It has been suggested that the physical path of the cable run should be recorded (GPS coordinate path). I think at this point you are heavily into GIS territory, and it is sufficient just to record the endpoints of each cable / segment.
Attestation
There may be external data which witnesses the validity (or otherwise) of the data in Netbox. For example:
It may be useful to record this information. A report can then highlight where the witness data conflicts with the model, or the witness data is absent or not sufficiently recent.
GPON
This is an example of another cable plant task.
From a physical point of view, this all consists of point-to-point fiber connections, and so the simple physical model is in principle accurate and sufficient.
On the other hand, it's undoubtedly true that the optical path runs end-to-end from OLT to all the ONTs: this is a multi-access network. It would be helpful to answer questions quickly such as "which ONTs are visible on this OLT port?"
It could be argued that Netbox is primarily for Data Center Inventory Management, and that GPON is outside of its scope. However, it may reasonably fit into the data centre cabling model anyway. For example, even if a splitter has no special behaviour, it should still be possible trace the connection from OLT to the splitter, and separately from the splitter downlink ports to the ONTs.
Tracing
Starting from a given device/interface, determine the chain of devices/ports forming the end-to-end connection. In the API return it as a list; in the GUI, visualise it as a chain of connected devices/ports.
Invariants
Port usage
It should not be possible to connect two or more cables into the same device port - except where the port is an intermediate connection in a path (e.g. patch panel or ODF) when there should be no more than two.
Connectivity
Once a connection from device A port X to device B port Y is established, the system should avoid accidental disconnection (e.g. require user confirmation for any change which would cause an existing connection to be lost)
Port types
It might be desirable to prevent invalid connections, such as connecting between an RJ45 port and an LC port. But in practice, this would be very hard to get right. For example, an SFP port can take either RJ45 or LC, depending on what kind of SFP is plugged into it (but that is not currently modelled). It would also require modelling of various types of adapters, such as RJ45 to DB9 console adapters.
@candlerb commented on GitHub (Jul 3, 2018):
And some thoughts around data models.
Current data model
Features of this model:
For console ports: when it is connected, the Console Port object contains a direct pointer to the linked Console Server Port object. There is no separate ConsoleConnection object. (Ditto for Power Port into Power Outlet)
Possible approaches
Proposal 1: Cable Segment replaces InterfaceConnection
Cable segment entirely replaces the InterfaceConnection model. This approach avoids potential duplication or conflicts in connectivity data.
You can still have an "unknown path" connection between device ports, by installing a single direct cable of type "unknown". This is what existing InterfaceConnection objects would be migrated to. (Or we could just add a whole load of attributes to existing InterfaceConnection, to form the Cable model)
Passive ports can be represented as dumb Interfaces on a Device. There is no need for a separate table. However we still need to distinguish passive ports, because a passive port can accept two connections, whereas a normal port can only accept one. We can also hide the MAC address and MTU settings on a passive port, and not allow addition of IP Addresses.
Interface neighbors
The main downside I see in this proposal is that it doesn't immediately provide you with the far-end device/port that any given device/port is connected to. When browsing a device in the GUI, this is what you are normally interested in seeing - not what the first patch-panel port it's connected to is.
Tracing the connectivity for all ports every time you browse a device could be very expensive.
I suggest that this information is cached within the interface object itself in a "neighbor" attribute: that is, update each endpoint interface with a pointer to the furthest device/port of the chain of connections. Any changes to Cable information would have to update the neighbor information of all affected devices. Intermediate devices (with both upstream and downstream neighbors) would point to Null.
We could add a separate object representing the end-to-end connection, but then it turns into proposal (2).
Console ports
For consistency, the direct pointer from Console Port to Console Server Port should be removed, and these also migrated to Cable objects of type "unknown". This allows console connections to be mapped through patch panels.
UI implications
We want to be able to manage Cables as standalone entities which can be created and delete, listed, and edited (e.g. change attributes like length and color).
The device view would still show connected interfaces as the "far end" device and port (neighbor).
"Disconnecting" an interface connection on a device would provide a list of all the cable segments forming that connection, and allow the user to select which ones to delete (with "permanent" ones excluded by default). Switching a connection between "planned" and "active" would similarly have to ask for which cables are to be switched.
In this model, the only record of end-to-end connectivity is via the individual cables. Therefore I think that any changes which result in changes to end-to-end device connectivity (i.e. updating the cached "neighbors") should ask for extra confirmation in the UI to avoid loss of information. For example, suppose device/port A is connected to device/portB, as
A--X--Y--B. If you remove the cableX--Y, you should be prompted "This will remove the end-to-end connection from A to B, are you sure?"To avoid such problems, I would like to have a view which allows the user to perform edits on an entire path which retains the end-to-end connectivity automatically where possible.
For example, if you edit the path
A--X--Y--Band ask to delete the cable between X and Y, it is automatically reconnected asA--B(with cable type "unknown")Similarly, I want to be able to insert ports in a path. Given
A--X--Y--B, if I select the link between X and Y and add port P, it should becomeA--X--P--Y--B(i.e. there are now cables X--P and P--Y). If the database already knows about a connection P--Q, then when I insert P it should becomeA--X--P--Q--Y--B.The main purpose of this is to allow low-risk editing of point-to-point connections, whose path is unknown, to real paths via patch panels; and for repatching. Otherwise it becomes necessary to delete
A--Band addA--X,X--Y,Y--Band if you get any part of this wrong, the information that A should connect to B is lost forever.Passive loops are silly, and should be rejected. As a side benefit this prevents you making the same connection twice (as
A--BandB--A)A/B matching and flipping
Optionally, we could add a new database invariant: that when two cables connect, the 'A' end of one can only connect to the 'B' end of the other. This makes it easier to trace them linearly. But there is one added complication, which is when you insert a cable which ends up joining two other sequences, it may be necessary to flip around the A/B ends of some of the existing cables to maintain this invariant.
Say you have these connections:
A--B,B--C,F--E,E--D. If someone then adds a cableC--Dthen it's necessary to flip around theF--EandE--Dconnections to makeA--B B--C C--D D--E E--F.This change could allow us to enforce the two-connections-per-port limit at the database level, by adding uniqueness constraints on (device, A-port) and (device, B-port) on the Cable model - or it would, if it weren't for console ports being a completely different type of object :-(
If we can't enforce this, then there is less value in enforcing A-to-B matching.
Proposal 2: Cable Segment augments InterfaceConnection
Features of this model:
A very attractive feature of this option is that these Cable objects can be added to the existing data model without changing InterfaceConnection, i.e. it's a simple bolt-on and does not invalidate any existing Netbox data. I really like this.
Changes will be required to manage the cable objects (create/edit/delete).
There will need to be logic so that you cannot create an InterfaceConnection which conflicts with the cable objects, nor can you change the cable objects so that they conflict with an existing InterfaceConnection. However, it's always permitted to delete a Cable or InterfaceConnection.
Hence at worst, you can delete the InterfaceConnection whilst leaving the Cable objects in place; modify the Cable objects; then recreate the InterfaceConnection. Or if you want to create a new InterfaceConnection which conflicts with existing Cable objects then you can delete one or more of the conflicting Cable objects (either by hand, or have this offered automatically)
InterfaceConnection can have a flag indicating whether it's fully modelled by Cable segments or not.
I can make a more detailed proposal for the logic required separately.
The main benefit of this approach is that it's much harder to accidentally lose the information about which interfaces actually are connected together (or are intended to be connected together) while making changes to the cable model. It also naturally retains the information needed to display device port "neighbors" efficiently in the GUI.
Coupler ports
A special type of interface port ("coupler?" - marked "P") is allowed to receive two connections instead of one. As this is a passive port, it can also disallow setting of IP addresses, MAC address and MTU.
In this model, I propose making such intermediate passive (coupler) ports completely separate objects from Device Interfaces, in their own table. The benefit of doing so is that InterfaceConnection objects can still only be created between Device Interfaces, which is what they are intended to represent: end-to-end connections between active devices. It avoids all ambiguity over whether an InterfaceConnection can be created between two passive ports or not (it can't). In any case, we need to treat the passive port differently, because it can accept two connections.
The downside is that Cable segments may have either a Device Interface or a Passive Port at each end; but on balance I think that making the distinction between active and passive ports is worthwhile in this model.
Console connections
If we include console ports in this, then:
The Console Port object in one device can still retain its direct pointer to the Console Server port in another, representing the logical console connection separately from the physical path.
The consistency logic would therefore need to be extended so that a change to a Cable which conflicts with a Console Port association, or vice versa, is rejected. As before, it's always safe to delete a cable or remove a console port association. We could also add a flag in the Console Port which indicates whether the console cabling for that connection is fully modelled.
A/B flipping
The optional A/B matching and flipping from Proposal (1) could be done here too.
Other options considered and rejected (for now)
3. Connection per segment - internal device connectivity
Each patch panel type of device has explicit "in/out" or "front/back" interfaces, and we model internal connections between those ports. Here I've suggested this is modelled as "master/slave":
Advantages of this model:
Downside: larger data model (e.g. a 24-port patch panel has 48 interfaces), and more work to model the internal interface cross-connects - although these could be incorporated into the interfaces in the Device Type.
4. Connection per segment - inverted (without A/B ends)
If you invert the relationship between interface and connection, so that each interface points to the connection, then you automatically enforce the invariant that a port can only have one connection, and you remove the A port / B port asymmetry.
It permits a cable connected at one end only (dangling), but unfortunately it also permits 3 or more devices to connect to the same cable. It's not useful when cables are asymmetric. But I thought it was worth mentioning.
5. End-to-end connections with annotation
Keep InterfaceConnection as a single end-to-end object, and annotate it with path information.
Conceptually, it records the end-to-end link connectivity - which may be automatically verified (e.g. via LLDP) - separately from the patch-by-patch physical routing, of which we may have less certainty.
Whilst this seemed attractive at first, I don't think it works well: in particular we want to model the permanent connections between patch panels before any devices exist. These would then become InterfaceConnections with null interfaces at both ends, which is ugly. When connecting things up, it may be necessary to merge separate InterfaceConnections into the same object. And it is hard to enforce data consistency, such as no more than two connections going into the same port.
However this approach does have a couple of advantages:
6. Multipoint interface connection linked to cable segments
Like proposal 2, but with the link between InterfaceConnection and Interface inverted, so that each Interface points to Connection.
The advantage here is that InterfaceConnection allows multi-point connections: for example, an InterfaceConnection can represent a GPON network with OLT and multiple ONTs all joined together. It would be expected that the relevant Cable sections, together with splitter(s), would form a tree spanning all those interfaces. It also means that everything points at InterfaceConnection; InterfaceConnection is just a set of endpoint Device Interfaces, and Cable segments. The only thing which points at an interface is a Cable segment.
However, this does fundamentally change the InterfaceConnection model, and might be considered harder to understand.
7. Multipoint interface connection and inverted cable connection
The connection between interface and Cable segment could also similarly be inverted, but this would require master/slave passive ports and is probably not worth it, when a Cable segment is clearly between two endpoints. But it would mean that both Device Interfaces and Passive Ports could both point to a Cable; and it does explicitly model the internal connectivity of a GPON splitter.
Additional notes
Bundling
Bundling can be done by associating multiple cables with a bundle group, just like rack groups.
GPON
A few thoughts about how how these models might be compatible with GPON.
As already said: at the cable level the network consists of point-to-point fiber connections only (some of which go into splitters). However if a splitter is modelled as a "Device" with normal ports, then you need to trace separately from OLT to splitter, and from splitter to ONTs, as if these are separate network segments - when in fact they are one multipoint connection.
In proposal (2) above, some passive ports could be configured to allow more than 2 connections. A passive port which allows up to 9 members could represent an 8:1 GPON splitter. Or in the models where devices have internal passive connections between ports (3/4/7), we could model the splitter as having one uplink (master) port with 8 linked downlink (slave) ports. In either case, we can trace the whole GPON network directly - which then becomes a tree rather than a straight line.
It can be argued that a GPON optical domain passing through one or more splitters is just a special case of other broadcast domains: in particular it's like a VLAN which propagates through multiple switches. Netbox models this as a VLAN object, plus interface-to-VLAN assignments. This is a completely separate logical model: you can associate end device ports with VLANs, without any need to model physical links between switches or how they are trunked through.
Therefore, the same could be done for GPON: have a GPON domain object, and make the OLT port and all the ONTs point to it. This would give you the GPON logical connectivity, completely independent of the underlying splitter links (which may or may not even exist in the data model)
But I don't see value in modelling this separately. It would be a useful side-effect of multi-point InterfaceConnections (option 6/7), or maybe "VLAN" could be re-used for this purpose.
@spencerryan commented on GitHub (Jul 3, 2018):
This looks really great!
Two things that come to mind, first is for multi-conductor/path cables it would be nice to know the pinout/config. Straight/Crossover for RJ45esque cables and Polarity A/B/C for MPO as an example.
Second, in the case of GPON, you have multiple devices all connected to the same port on the headend (OLT). Physically you have a 1:1 mapping between the OLT, splitter, and downstream ONT's, but from a logical perspective you may have dozens of ONT's connected to a single OLT port.
@candlerb commented on GitHub (Jul 4, 2018):
I think that's identified from the cable type. Nobody's going to trace individual conductors through an RJ45 for example. However, MPO break-out cables are a different issue, and I've not yet got a good answer for that.
I've added GPON as a use case.
Apart from GPON, multi-point connections are rarer today than before: thankfully, no more ethernet hubs or 10base2 :-) They are useful when modelling software bridges, but that's not related to physical connectivity.
@sebschrader commented on GitHub (Jul 12, 2018):
What about WDM multiplexers/demultiplexers and OADMs? Can they be modelled properly with the proposed data model?
@candlerb commented on GitHub (Jul 12, 2018):
In this model a WDM mux/demux or an OADM is just an active device, akin to a switch. The passive patching will get you to a port on the device. If you want to trace a wavelength through such a device then it would be up to you to know about the internals - akin to tracing VLANs through switches.
The big question mark I have at the moment is about breakout cabling: in particular, mapping fibre pairs on a duplex connector to individual fibre strands in a cable. I need to think about that some more.
@smiley2billion commented on GitHub (Jul 14, 2018):
As for the fiber pairs question, are you meaning how to match the port number on the panel to the color of the fiber on the back side? Or do you mean do we count a duplex connector as one physical cable?
I would personally prefer to see all single physical fiber strands counted as a physical cable, the duplex part can be tagged somewhere on the individual strands. For example if we have a 12 count fiber and followed then standard (B, O, G, Br, S, W, R, Bl, Y, V, Ro, Aq) then they should match 1 through 12 port numbers to those colors. We could maybe use a pair tag of some kind on the individual strands for a given patch panel to create the interface (B/O = Pair 1, G/Br = Pair 2, etc). That would be very helpful as in the real world the duplex pairs don’t always follow the standard on the back side. The ‘pair’ interface would then connect to the end device.
If I’ve misread what you were getting at I apologize, disregard.
@candlerb commented on GitHub (Jul 14, 2018):
That. Currently, Netbox considers a duplex LC connector as a single "port", and a connection from one LC to another LC as a single "connection". The model is simple, and it works for simple point-to-point duplex fibre connections.
But when you connect two ODFs with a multi-strand cable, you want to think at the level of individual strands, not pairs. Similar issues occur with MPO cables, QSFP+ to 4 x fibre pair cables etc.
One possibility is that instead of having an A end and a B end, a cable may have an A end, a B1 end and a B2 end (or B1-B4 or whatever).
There is a circle to be squared. For example, a switch with a QSFP+ port may make it appear as four separate interfaces. Do we treat this as one physical port with four virtual ports behind it? And yet with a QSFP break-out cable, it may connect to four separate physical ports at the the other end!
@nward commented on GitHub (Jul 15, 2018):
The other side to this is single fibre services. Common uses I have for these are bidirectional light on a single fibre, and the output of fibre taps to monitoring equipment.
@nward commented on GitHub (Jul 15, 2018):
My view is we need a flexible model where strands can be joined with an unknown number of other strands to make a container (cable?) which can be nested inside another such cable indefinitely - allowing pairs to be joined inside a larger cable. Each strand can have its own termination, or be joined with an unknown number of other strands in to a connector of some type.
Device ports should have appropriate connectors - I.e. an LC duplex port should accept either one LC, two discrete LCs, or an LC duplex. That’s up to the user to define though as some environments may have different needs.
If the termination is multi-port, then we also need to define which strand is in which position on the connector, so we can model cables of different polarities.
I then think we need to be able to model devices which are generally passive, so, a patch panel with a multi core connector in the back needs to map each strand to each front panel connector.
Trying to impose any structure on top of this, I’m my view, is going to lead to a lot of headaches in the future - the physical layer is single strands so let’s start there, and if a user wants to build structure (cables, multi core connectors) on top of that they are welcome to do so. Perhaps we ship some basics (LC duplex connector, x over LC duplex cable, etc.), but have the flexibility there for the user to define or redefine these.
@sebschrader commented on GitHub (Jul 15, 2018):
On July 14, 2018 8:23:32 PM UTC, Brian Candler notifications@github.com wrote:
The model also breaks down if you use BiDi transceivers and mix them with regular duplex connections on the same box.
@candlerb commented on GitHub (Jul 18, 2018):
I think I have a decent model designed now - but before I write it up, I'd like to check something about QSFP+ (since I don't have any such equipment).
Does a QSFP+ port typically appear as four separate 10G interfaces in the device configuration?
The reason I ask: I am wondering whether there is a need for separate models for "interface" (logical) and "connector" (physical), with an N:1 relationship between them. At the moment, Netbox combines the two concepts.
Is this separation important? Or would it be acceptable to treat a QSFP+ port as a single "interface/connector", and then create four virtual interfaces representing the actual 10G circuits? If so, I think it would be necessary to be able to create a LAG between these - but Netbox does not currently allow LAGs between virtual interfaces. Also, it would be desirable to associate these interfaces with the physical port. At that point, it becomes much like a separate interface/port model.
Alternatively, a QSFP+ connector might be considered as four separate connectors bound together. But then the model would probably require you to plug in the four "parts" separately.
@spencerryan commented on GitHub (Jul 18, 2018):
It depends on the platform. Typically they can be configured for 4 x 10G mode or 1 x 40G mode (and similar for 25/100G)
As an example, on an Arista switch a QSFP will show up as something like Ethernet52/1, 52/2, 52/3, and 52/4. In 4 x 10G mode all 4 subinterfaces work, and in 40G mode subinterfaces 2-4 are in err-disable and /1 is the up/up interface.
@isodude commented on GitHub (Jul 18, 2018):
Hi,
QSFP+/QSFP28 can be split into 4x10G, 4x25G, 2x50G, allowing 400G
interfaces we will probably see more splits. In the switch it's one
400G/200G/100G/40G optic, and in the other end you have an 10/25/50/100G
optic that eventually decide what splitting there will be. In the end you
will have several interfaces in the switch.
If you don't break out the connector it will just be one interface.
Will have my hand on the equipment in a month, but I assume that they are
treated as separate interfaces in the switch. I will implement 14 ports in
a 32p 100G as 25G ports, having them as 100G ports in one end and 25G in
the other would be helpful. You will terminate them in a split-out cable or
casette that will be an intermediate anyway.
So the physical ports:
MPO 100G SR4 -------- MPO->4x25G(LC-LC) (
https://www.fs.com/products/68402.html) --------- LC 25G SR4
Virtual would be
TwentyFiveGigabitEthernet1/3 ------ eth0
So in this case there will be two physical entries, between the optic and
the casette, the casette and the server. The virtual interface actually
passes several physical ports.
After doing a bit of research on the fiber cables there is a lot of
information that should exist on the cables as well, like Single or Multi
mode? Type A/B/C? Female-Female/Female-Male? OM4/OM3/OS/2? MPO-8 12 or 24?
Copper cable is so much easier.. but still it must be important if it's
Cat5 or 6. Do we have kind of detail on the cables right now?
I could try out your changes to see if I can document my new setup properly?
On Wed, 18 Jul 2018, 20:14 Brian Candler, notifications@github.com wrote:
@ljb2of3 commented on GitHub (Jul 18, 2018):
On my Extreme 7100 series switch it shows up as 4x10g connections in the
config, but that may be due to the optic I have plugged in, as I'm using a
MPO to 4x LC breakout cable for separate 10g connections. I'm not sure what
would happen if I put a single 40g LC optic in the slot.
I just want to say thank you for your work on this. I have a need to map
out our fiber usage on campus and am looking forward to using netbox for
this.
From a UI perspective, it would be nice to group things together, such as
"single mode from building 1 to 2" and see a free vs used chart, perhaps
like the % utilized for IP space in prefixes, or even a list of strands in
the group.
-Landy
Senior Network Engineer
The University of Tulsa
On Wed, Jul 18, 2018, 13:14 Brian Candler notifications@github.com wrote:
@nward commented on GitHub (Jul 18, 2018):
I have a couple hundred optics with >10G in production so can speak pretty authoritatively about this.
As @spencerryan the presentation to the OS can depend on the platform.
The connector can be:
Presentation to the OS is typically as a single 100G or 40G link, however, it can be "channelised" in to the per-fibre or per-wavelength signalling rate. i.e. a 40G optic can be 1x40G or 4x10G. Not all OSes support this channelisation.
I have never seen an option for 100G being divided to 2x50 as @isodude mentions - I don't know how that would work, as that would be combining 2x50G services in the same manner as 4x25G for 100G - I don't think there are standards for that right now but no reason there couldn't be. I think we should allow for modelling whatever combination of these we can think of - even combining channels differently (so maybe a 4x25G running as 1x50G and 2x25G).
Note that when you have a multi-channel optic, running it in 40G or 100G mode does not behave like an 802.1AD "etherchannel"/"bond"/whatever. It is a single 40G or 100G service.
@isodude commented on GitHub (Jul 19, 2018):
Regarding 50G, I've seen it on the switch specs, but searching for it
reveals both NICs and DAC-cables at least. Not the same amount that exists
for 25G though.
Mellanox SN2100 can carry 32 50G ports i.e.
Regarding conmectors I hear that some are using 24 core trunks to cabinets
to just split out to raw LC, and replace LC harnesses that way. Just for
upstream fiber patches or patches betweeen cabinets. Sounds like a smart
thing :)
On Wed, 18 Jul 2018, 23:50 Nathan Ward, notifications@github.com wrote:
@tardoe commented on GitHub (Jul 19, 2018):
I can backup the information provided by Nathan, having also deployed a large amount of 10/40/100G optics.
Using the Extreme Exos platform as an example, the 32 QSFP28 switch interfaces on the X870 model are labelled 1,5,9,13... 125 with the option to carve these into either 1x100G, 2x50G, 4x25G, 1x40G or 4x10G.
So 25G/10G interfaces would be 1,2,3,4 and 40G/100G interfaces would be 1,5,9,13... Leaving 50G interfaces to be 1,3,5,9 etc. This is a fairly stock broadcom platform which is quite popular so this kit might form a good reference.
Cheers,
Tim Raphael
@nward commented on GitHub (Jul 19, 2018):
Hey Tim :-)
Juniper boxes use a sort of sub-interface when channelising:
et-0/0/0 - 40G
et-0/0/0:0 - 10G
et-0/0/0:1 - 10G
et-0/0/0:2 - 10G
et-0/0/0:3 - 10G
I think the key is that this stuff is flexible. The QSFP channelisation is something I hadn't considered though. When tracking individual strands, breakout with same wavelength - i.e. MTP/MPO - is quite easy.
It's also possible to break out multi wavelength - i.e. take a 40G-LR4, put it in to a mux/demux, and then get 4x10G channels on a WDM system. I think this is covered though, as the mux/demux can be modelled as a "device" - maybe? Not sure.
@tardoe commented on GitHub (Jul 19, 2018):
I would agree with this, any sort of mux should be modelled as a device as 1) it performs a unique network function and 2) it changes the physical / logical path.
Regards,
Tim Raphael
@candlerb commented on GitHub (Jul 26, 2018):
This is the more physically-oriented outline design I've been working on.
Connectors and ConnectorTypes
Example: ConnectorTypes
ConnectorType
Normally the connector has one name, but in some cases the male and female are known by different names (e.g. C14 and C13)
Pins are internally numbered from 1, but can have user-defined pin names.
Example: mated RJ45 connectors
Suppose that there are devices 51 and 52. Each has an RJ45 socket. Then there is a lead with id 2000, which has an RJ45 plug on both ends.
Connector
A connector must have either device_id or lead_id set (but not both).
A connector may have either is_female or mated_id set (but not both). That is: when a connector is connected, the mated_id of the male connector points to the id of the female connector. A male connector can have mated_id set to null, if it is unconnected.
Alternative proposal for connector model
This data model shown above allows a connector on one lead to plug directly into a connector on another lead - something that happens in real life - e.g. extension cables, and couplers/adapters like RJ45 to DB9 serial adapters.
I did consider a more compact approach, where the connector on the Device always points to the Lead:
lead_id and lead_end are set to null when the device port has nothing plugged in.
This has the advantage that when you create a Lead you only need to create one object instead of three. A Device connector is 'mated' by means of being associated with a particular end on a lead. A unique constraint on (lead_id, lead_end) avoids the same lead end being used more than once.
However that model does not allow two Leads to be connected to each other, unless we introduce a new model for this case, e.g. Coupling.
This is also a reasonable approach, although it does complicate some logic. When tracing a path from a Lead, you would have to look for Devices which point to this lead, and also Coupling objects where a_lead_id or b_lead_id equals this lead. And the process for connecting/disconnecting two Lead ends to each other is different for connecting/disconnecting a lead into a Device.
How Connectors relate to Interfaces
I see Interfaces as a logical entity. In the existing model an interface on one device can be "connected to" an interface on another device, meaning they are direct neighbors, and I think this should remain, to be able to display neighbors without having to trace physical paths. (Or it can be simplified so that each interface has a "neighbor" attribute pointing directly at another Interface)
Furthermore: multiple interfaces may be presented on the same physical connector: e.g. there may be four 10G interfaces but they are on the same QSFP+ connector.
So I see there being a N:1 relationship between Interface and Connector. However most of the physical attributes of Interface would move on to Connector. An Interface would still have a MAC address and MTU, and be what IP addresses are attached to.
The ConnectorType model allows the user to add new types of connector, without having them hard-coded in Netbox (e.g. special stacking connectors).
If each Interface has a link to Connector, we also gain for free the ability to associate subinterfaces with the same physical port (e.g.
Gi0/1.100,Gi0/1.200etc) - #1519Leads and LeadTypes
A Lead is somewhat like a device, in that it has two or more Connectors - but it is a free-standing entity. You are likely to have many identical or similar Leads in the network. The LeadType defines the set of connectors which a Lead should have, so that when you create a Lead the related connectors can be created automatically.
Example: RJ45 patch lead
LeadType
"Symmetrical" means that all the cable ends are equivalent and interchangable, so "A" and "B" are just arbitrary labels without any physical meaning. (Note to self: maybe the connectors on a lead should just be numbered from 1 anyway)
Lead
In this model I've assumed that you don't want to repeat the definition of LeadType for minor variations of leads (e.g. lengths and colours), so we make them attributes of the Lead itself.
I've shown these fields as JSON, because it's compact and is how I'd do it; but it could be broken into a bunch of separate normalised tables in the way that CustomFields are.
I have included "serial" and "asset_tag" because some types of lead are expensive assets to be tracked in their own right - for example a copper SFP+ Direct Attach cable.
It seems reasonable to me to permit a lead to exist with no connections, i.e. unused lead. It might be worth having an optional location (maybe just site_id) on the Lead object for that case. This situation is comparable to a spare, unracked Device. I would never expect a Lead to span sites anyway.
Example: IEC Y cable
LeadType
Example: duplex LC to 2 x ST
LeadType
Harness
The "harness" indicates how the connector pins are wired together: in the RJ45 example lead end A pins 1-8 are connected to lead end B pins 1-8, and in IEC Y cable example all three pins of connector A are linked to the corresponding three pins of connectors B1 and B2. In other words, there are groups of linked pins. Again, this is shown as JSON for compactness.
A duplex LC to LC fibre patch lead would have the two strands swapped over (A pin 1 to B pin 2, and vice versa).
The harness is optional. If you just want to define a lead with N ends and N different connector types, but no harness, that's fine. The internal harness may simply not be worth modeling, or the lead may have active components (e.g. a USB to RS232 lead).
SFPs as Leads
We can usefully model an SFP as a "Lead" with an SFP (electrical) connector at one end, and another connector (e.g. duplex LC or RJ45) at the other. There would be no direct connection between the pins of the connectors (i.e. harness), because this is an active element. That's fine.
The LeadType can distinguish between different types of SFP, and the serial and asset_tag attributes of Lead helps to track them.
This is another reason why it's very useful to allow Leads to connect into Leads:
Cable plant
The above doesn't add all that much to the existing data model: a lead (or series of leads) connects a device port to one or more device ports.
The other side required is cable plant, which allows fixed infrastructure connections between patch panels and ODF ports to be modelled. This model is in some ways similar to Lead and harness, but represents permanent infrastructure rather than dynamically mated connectors.
CrossLink
A "CrossLink" represents one or more strands of pre-installed copper or fibre between two connectors, normally on two different Devices. These are internal connections, in the sense that they are at the "back" of the connector, not the mating face.
I could have just modelled individual Strands, but I didn't want to have to create 8 separate objects to represent the very common case of a connection between two RJ45 patch panel ports.
CrossLink
I had originally called this StrandSet, but it's useful to be able to model that there is some sort of connection between two device ports without having to model any strand-level detail; that is, leave
a_pinsandb_pinsas null.A CrossLink can also exist between two connectors on the same device: this allows internal cross-connects to be modelled, e.g. a patch panel with RJ45 sockets front and back, and various types of break-out device.
If you want to route different strands on one connector to different remote connectors, then you create separate CrossLinks carrying a subset of strands:
NOTE: a CrossLink represents an end-to-end connection between certain pins of a connector on one device, and certain pins of a connector on another device. At this level there are no splices, nor does it model which cable(s) or specific strands on each cable are used.
However, an electrical test or an optical test can confirm the accuracy of a CrossLink model.
Cables
Cables are the (optional) physical manifestation of the CrossLink model, and may be spliced. One cable may serve multiple CrossLinks, and one CrossLink may be spread over multiple cables.
CableType
Cable strands are internally numbered from 1, but can also have user-visible labels such as tube/core colour.
The cables themselves are simple objects that are just instances of a CableType:
Cable
(But we could have additional attributes, such as length or dB loss)
Cable attributes
I think it would be useful to add attributes like manufacturer, colour, diameter on the CableType. (Alternatively we could have user-settable attributes on the Cable itself, but I don't think that creating extra CableTypes when required is a major problem)
Cable Segments
A Cable Segment links one or more strands in a CrossLink to the corresponding set of strands in a Cable.
Example 1: two ODFs with duplex LC connectors, connected via two four-strand fibre cables which are patched back-to-back 1:1 in the middle.
CableSegment
Example 2: one ODF with two duplex LC connectors, going to two LC ports on two different ODFs. There is a single 4-strand cable to start with, then it splits onto two separate 4-strand cables, with 2 strands in each of the second cables unused.
NOTE: there are no explicit splices as such. If one particular CrossLink strand uses cable A strand 5 and cable B strand 8, then it's implicit that cable A strand 5 must be spliced onto cable B strand 8.
A limitation of this is that it's not possible to record a splice, when the strand doesn't run all the way end-to-end between two device ports (i.e. it's not part of a CrossLink). In practice, I don't think this is a major problem, as you're unlikely to pay for splices which don't get used. I am open to suggestions for better ways to model this, but I think this keeps things simple.
The proposed model also allows you to change the cable strand assignments safely, without touching the end-to-end CrossLink connectivity (which as I said before, may have been verified physically e.g. by OTDR tests)
Splice locations
The above model does not allow you to verify that a physical path has been fully modelled; it just tells you that CrossLink A passes through particular strands on cables W, X and Y, but can't tell you if you forgot to include cable Z.
However, if we include the cable endpoint locations, we can do this. A cable endpoint may be at a Device, or at a JointLocation (which could represent e.g. a splice chamber; many cables can have have their ends in this location)
If the places where cables 1, 2 and 3 come together is called X, then it would look like this:
We can now trace the Cables which make up a SpliceSet. As long as every cable with an end at a particular JointLocation is paired with another cable that also has an end at the same JointLocation, we know that the whole path has been modelled.
This has the additional benefit that you can model the endpoints of a cable, even if they are not connected at all - e.g. are just coiled up in the JointLocation waiting to be connected.
A JointLocation could usefully have some location attributes, e.g. GPS coordinates or just a description of the location, and/or a site_id.
It should be stressed that this model builds up in layers:
You don't have to use all the levels if you don't want to model at that level of detail.
Cable plant alternatives
As described above, the CrossLink can only connect pin(s) on one connector to pin(s) on one other connector; it does not allow 3 or more pins to be electrically or optically connected together. I don't think this is an important limitation; if you want to model power distribution or passive optical networks, you can model PDUs or optical splitters as separate devices. But this makes it different to Lead, which can model such multi-way connections, e.g. the IEC Y connector example.
There is an argument to consider about whether to model CrossLink in a similar way, for consistency. I think this would end up with a join table, which I've called CrossLinkConnector here:
You can see that separate CrossLinks can be coalesced, and hence the granularity of CrossLink is up to the user - it could be at the level of normal cables (e.g. 8 strands for a CAT5e cable) but could be at the level of the entire "wiring loom" if you wanted: this would mean giving each strand in the loom a unique number though.
I don't think this an improvement though. Logically, it's helpful if a CrossLink represents a bundle of pin connections between exactly two connectors: it makes it much easier to trace through the network, in the common case where the path does not bifurcate. That is: if I have an RJ45 port on patch panel A, and another RJ45 port on patch panel B, and those two ports are linked by a single CrossLink (and no other ports share the same CrossLink) this makes the CrossLink a helpful logical model that these two things are connected, without having to trace individual strands. Indeed, you could have a CrossLink with zero strands and still use it to identify that there is some sort of path between those ports on A and B, without having to do any strand-level modelling or tracing at all.
Discussion of the above model
Managing connections
Instead of (or as well as) adding a "Connection" between two device interfaces, you would create a Lead and connect it to two (or possibly more) device connectors. The process for doing this would be similar to interface connections, but the lead itself has attributes (a lead type, and maybe optional attributes like colour, length, serial number etc), and therefore needs to be visible as an object in its own right.
It should be possible to unplug a single Lead connector, and reconnect it to a different connector on the same device or a different device (whereas Netbox currently requires you to destroy and recreate an InterfaceConnection)
The existing Interface Connection model can remain, representing the logical neighbor relationship between two Interfaces, alongside the physical model. When you are entering data, you may find that the Interface Connection model may either not have a corresponding Connector / Lead / CrossLink model, or the latter may be incomplete, or inconsistent with the former. There should be reports for highlighting those cases. Alternatively, whenever changing the Lead model, it can flag whether changes are inconsistent with the Interface Connection model, and whether you want to make the corresponding updates automatically. That is: people who create physical connection models will get the logical model updated automatically as a side effect, when possible. (It might not always be possible, e.g. when one physical connector related to multiple logical interfaces)
Even with explicit patching it may not always be possible to update the logical interface connections automatically. For example: a single QSFP+ port may be linked to four logical interfaces; the transceiver has an MTP/MPO connector with 8 pins; and then a break-out cable (with five connectors) connects to four duplex 10G interfaces on other devices. Extra modelling would be required to be able to determine which interface connects to which: e.g. those four logical interfaces might have to be associated with "channels" of the QSFP+ connector, which in turn pass through the transceiver as four "channels" (fibre pairs) on an MTP/MPO connector, which in turn are associated with four fibre pairs on the MTP/MPO break-out cable. But if this is not done, it would still be possible to create the logical interface connections manually.
Interfaces and Connectors
A Device will now have both Interfaces and Connectors. That is:
When adding an Interface to a Device, you would need to be able to associate it with an existing Connector, or create a new Connector. Or you might want to be able to create a Connector without creating an Interface. (A patch panel has Connectors, but no Interfaces)
DeviceType templates would need to gain connectors too. To be complete, DeviceTypes should also have CrossLinks representing internal cross-connections (e.g. patch panels with internal front-back connections)
Interface and Connector naming
Both Connectors and Interfaces need to be named. In simple cases they might be the same name (e.g. "Gi0/1"). However moving forward, this may not be true. Consider for example what happens when we extend this to a chassis switch model: on each line card the physical connector port names may be 0 to 7, and hence "slot 1 port 0", "slot 2 port 0" etc; but the corresponding logical interface names may be "Gi0/0", "Gi1/0" etc.
Therefore, long term, I think the set of Connectors (and their names) should become a fixed part of the DeviceType (and LineCardType). After all, in real life you do not modify a device by drilling it to add more Connectors! But on a particular Device instance, you can change the set of interfaces, and which connectors they are associated with. With modular Line Cards, a logical interface on the Device can be associated with a physical connector on a Line Card.
Moving the set of connectors up into DeviceType has some other advantages. It makes LeadType and DeviceType much more similar to each other; and the connections from Lead to Device and Lead to Lead could be represented by a single object, representing the mating, rather than the individual connectors. That is, you would not need to create unmated connectors on Devices and Leads.
This idea is interesting enough that I'll sketch it out separately.
Console and power
Netbox currently has completely separate models for Console ports and Power ports and connections into them. I think moving Console ports into the above model would make a huge amount of sense: console connections can be patched via RJ45 patch panels.
Whilst that is not true of power, I still think there is a strong argument for modelling power like this too; see the example of the Y IEC lead (which in Netbox today I have to model as an unracked "device" with one power inlet and two power outlets). I think the general solution here is to create port roles: network, console [in/out], power [in/out]. People may wish to extend this, e.g. to add USB, or if they want to consider Fibre Channel as a role distinct to "network". But having a hard-coded set of roles is probably good enough.
In the UI, ports can be grouped by role to keep the same logical separation that they have today.
Lifecycle
I have not addressed the lifecycle status of any of the entities above: e.g. whether they are "planned" or "in service" or "failed". Marking a CrossLink as Planned before it's build would most likely be useful; marking a Crosslink as Failed is definitely useful, as this does happen in practice.
@candlerb commented on GitHub (Jul 26, 2018):
Updated model - Connectors fixed to DeviceType
Now taking the idea of applying physical connectors to DeviceType instead of Device, whilst Interfaces remain on Device.
Let me sketch out a sort of pseudo-ERD which includes some sample data. This is the connectors and leads part:
You can mate a lead to a device, or a lead to a lead, but not device to device. There are no explicit "connector" objects any more: rather, the pairing of (device_id, index) or (lead_id, index) refers to the Nth connector on the device type of the given device, or the lead type of the given lead.
A DeviceTypePort is flagged "passive" if it is a patch panel or ODF port which can be the target of a CrossLink (such ports cannot be linked to an Interface).
Not shown: InterfaceTemplate should gain a link to DeviceTypeConnector. (InterfaceTemplate gives the default set of interfaces when an instance of a Device are created from a DeviceType, and when those interfaces are non-virtual, we want them linked to the correct connector)
And this is the CrossLink part:
Notes
Comparison to current model
In Netbox today, "Interface" is linked to a hard-coded list of InterfaceTypes. We would instead be linking to a DeviceTypeConnector, which is a particular port (named instance of a ConnectorType) on a given DeviceType.
A "mating" looks at first glance rather similar to the existing Interface Connection. But the key difference is, instead of going Device to Device, it goes Device to Lead or Lead to Lead. That is, one (logical) connection is formed by at least two matings, for the two ends of the cables. Also: it goes between Connectors and not Interfaces (although the Connectors themselves are not explicit objects, but the Nth instance of DeviceTypeConnector or LeadTypeConnector)
Data migration
An Interface normally links to a DeviceTypeConnector, but it doesn't have to (if the link is null). This is how we deal with virtual interfaces, but it's also how we could deal with the data migration from old model to new. Existing DeviceTypes would have no Connectors on them, so existing Interfaces would all not be linked to any Connector. It would be up to users to add DeviceTypeConnectors to the relevant DeviceTypes, and then update their Interfaces to link to them, before making use of the physical model. This is a one-time change.
However it would also be desirable to retain existing interface physical characteristics, rather than lose them; maybe they should be retained for now.
Line cards
Netbox doesn't yet have the concept of line cards, and this model suffers without them, because a given DeviceType will have a fixed number of ports. Therefore, maybe line cards should be added at the same time.
Line Cards would be fairly simple I think:
I don't really like the duplication between DeviceTypeConnector and LineTypeConnector. The simplest way to deal with this is that DeviceTypeConnector just becomes Connector, and it can link to either DeviceType, LineCardType, or LeadType.
Note that this doesn't describe an individual connector, but the Nth connector on a give template (DeviceType, LineCardType or LeadType).
When a LineCard is added to a Device, it goes in a particular Slot. It would be good if default Interfaces could be created with a slot-based name, e.g.
Gi%d/1,Gi%d/2where %d is slot number. That is: LineCard needs a feature comparable to DeviceType's InterfaceTemplate, but with slot number insertion.@nward commented on GitHub (Jul 27, 2018):
I havne't read details of your proposal yet but I noted the line card thing. A "LineCard" should perhaps just be a "module". In this way, it could be a PCI card, or whatever.
Modules should be able to contain other modules - for example:
and on, and on.
In short - they should permit nesting.
@candlerb commented on GitHub (Jul 27, 2018):
I presume by "module" you mean "Inventory Item"?
Turns out they already do in the data model - it's just not exposed in the UI (#1692)
For the benefit of other people reading, Netbox currently has two types of "module-like" things:
What I want is a thing which has a pre-defined fixed set of Connectors; but those connectors extend the device that it's plugged into, rather than being independent like a child device.
ISTM that making InventoryItem become more grown-up, by having InventoryItemType, and adding optional sets of Interfaces and Connectors defined on the InventoryItem, is the way to go. I think this is essentially what #824 is proposing. InventoryItemType and DeviceType would end up duplicating a large amount of functionality and it would be nice to share that somehow.
Now, regarding modules which plug into other modules, like WICs and possibly SFPs: I'm going to do some thinking aloud here.
Consider the following configuration:
In the cable plant model I've outlined, I suggested modelling the SFP transceiver as a "lead" which has two connectors: an SFP copper connector on one side, and an LC duplex connector on the other. This gives the benefit that you can support both direct-attach cables that plug directly into the SFP connector, and SFP transceivers.
So how does a line card differ from an SFP? The way I see it is that a line card adds new connectors to a device, while an SFP plugs into an existing connector on the device, and provides another connector on the other side.
Of course, in the real world, a line card slot has a very special custom connector at the back, so we could also model a device as having special line card connectors, and line cards are just "leads" that have one of these connectors on one side, and on the other side have more normal connectors (e.g. RJ45), or other custom connectors (e.g. WIC connectors).
But if we forget about all these fancy custom connectors, we can just say that a "line card" adds standard connectors to the underlying device - in which case a WIC is an example of a "line card", even though it is physically installed inside another line card. In terms of adding connectors, all "line cards" are associated directly with the device. But physically they may fit into a line card. That is: logically, the WIC adds interfaces to the Device; but physically, it sits inside a Line Card.
As I said before, the existing InventoryItems model already supports this:
The device_id associates each inventory item with the device, but the nullable parent_id lets you say that the item is installed inside another inventory item.
@kaso17 commented on GitHub (Jul 27, 2018):
To throw in another use case for the LineCard/lead discussion.
Some newer devices support "compact SFPs" (two independent BiDi optics/interfaces in a single SFP module) or fanout cables (e.g. 1 x 100G QSFP28 to 4 x 25G SFP28). Depending on which transceiver you insert the switch will add the corresponding interfaces. This should be supported too.
@dowlingw commented on GitHub (Jul 31, 2018):
Netflix folk mentioned they might consider opening up their CircuitDB project, might be worth considering whether integration and/or taking some lessons learnt from their model has value in the discussion: https://twitter.com/dtemkin/status/1024037852622012421
@jeremystretch commented on GitHub (Oct 26, 2018):
Quick update. Work on this feature is underway, and anyone interested can follow along in the 20-physical-cabling branch. Hoping to have a v2.5 beta out in November.
@vink78 commented on GitHub (Oct 29, 2018):
LSH connector is common in my area. Is it possible to add it (check
b22dd4bfaf)?@jeremystretch commented on GitHub (Nov 1, 2018):
20-physical-cablinghas been merged intodevelop-2.5. This completes the initial implementation of the feature. Additional minor development on this and related functionality will occur in thedevelop-2.5branch as we work toward the beta release.This feature request is now closed. 🎉
@Folg0re commented on GitHub (Jun 7, 2019):
hi,
i'm on master branch v2.5.13 and i'm tring to add patch panel. i read all this discussion but i can't understand wich is the best practice.
i create a device with front port and rear port; the rear port are connected to rear port of other patch panel. when i connect front port, to interface of active device, apparently i have the right trace.
is it the best practice for patch panels??
thanks
@candlerb commented on GitHub (Jun 7, 2019):
Please post support requests on the Google group.
Github tickets are for development issues / bug reports.