starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Bulk GET to /api/dcim/interface-connections/ seems to return duplicates, is missing data. #1969

New Issue
Closed
opened 2025-12-29 17:21:01 +01:00 by adam · 1 comment
adam commented 2025-12-29 17:21:01 +01:00
Owner
Copy Link

Originally created by @pm17788 on GitHub (Aug 23, 2018).

Environment

  • Python version: 3.5.2
  • NetBox version: 2.4.3

Steps to Reproduce

HTTP GET to /api/dcim/interface-connections/, iterate over the 'results' list, iterate over list elements and store data in a DICT with the 'id' as the key.
A script I wrote to do this: https://github.com/pm17788/NetBox-fu/blob/master/helpers_and_one-offs/save_netbox_data.py
Full command:

python3 helpers_and_one-offs/save_netbox_data.py \
  --conf ./conf_salt_importer_interface-connections-only.yaml \
  --savefile json/netbox_data_interfaces_only.json \
  --api_endpoint_results_savefile interface-connections_whurr.save \
  --verbose \
  --logfile  interface-connections_whurr.log

Script's runtime log: interface-connections_whurr.log.gz
Script's saved "results" list: (6619 elements) interface-connections_whurr.save.gz
Script's saved "hashed by id DICT: (5801 elements) netbox_data_interfaces_only.json.gz

Expected:

Unique connection IDs, full data

What appears to happen:

Duplicate connection entries (duplicate 'id' and duplicate data), missing data

I have a script which parses a salt-gathered LLDP scrape in our environment, and tries to create the connections in NetBox based on those results. It starts the run by gathering all the interface-connections so that it can handle when an interface is already connected to something. This check (so yeah, I got inventive in a function name, and called it is_nic_in_connection :) ) has been failing, because the connection wasn't being found in the bulk get.

zgrep 6708248 json/netbox_data_interfaces_only.json.gz
zgrep 6708248 interface-connections_whurr.save.gz

The script failing looks like this:

2018-08-21 21:57:37,367 - INFO - MainProcess[MainThread|main] - Salt entry: 'bserver39' (16 of 6063, # of NICs: 3)
2018-08-21 21:57:37,368 - DEBUG - MainProcess[MainThread|main] - |-> NIC: eth1
2018-08-21 21:57:37,368 - INFO - MainProcess[MainThread|main] -   |-> LLDP data for: 'bserver39':'eth1'
2018-08-21 21:57:37,369 - INFO - MainProcess[MainThread|main] -   |-> LLDP: Dev 'bserver39' <=> Dev 'switch1'
2018-08-21 21:57:37,370 - DEBUG - MainProcess[MainThread|main] -     |-> Remote chassis ('switch1') info in NetBox
2018-08-21 21:57:37,371 - DEBUG - MainProcess[MainThread|main] -       |-> Remote port ('Ethernet1/17') in NetBox listing for remote chassis
2018-08-21 21:57:37,371 - DEBUG - MainProcess[MainThread|main] -         |-> Interface A ID: 6708248
2018-08-21 21:57:37,386 - DEBUG - MainProcess[MainThread|is_nic_in_connection] - Interface 6708248 didn't show up as part of any connection
2018-08-21 21:57:37,386 - DEBUG - MainProcess[MainThread|main] - Interface A (6708248) doesn't seem to be involved in any connections
2018-08-21 21:57:37,386 - DEBUG - MainProcess[MainThread|main] -         |-> Interface B ID: 6863690
2018-08-21 21:57:37,394 - DEBUG - MainProcess[MainThread|is_nic_in_connection] - Interface 6863690 didn't show up as part of any connection
2018-08-21 21:57:37,394 - DEBUG - MainProcess[MainThread|main] - Interface B (6863690) doesn't seem to be involved in any connections
2018-08-21 21:57:37,394 - INFO - MainProcess[MainThread|main] -      LLDP: Dev:Port bserver39:eth1(6708248) <=> Dev:Port switch1:Ethernet1/17(6863690) match.
2018-08-21 21:57:37,394 - INFO - MainProcess[MainThread|create_netbox_connection] -   Attempting to create NetBox connection between interface IDs: 6708248 and 6863690, status: True
2018-08-21 21:57:37,394 - DEBUG - MainProcess[MainThread|create_netbox_connection] -   Request URL: 'https://netbox.DOMAIN:443/api/dcim/interface-connections/'
2018-08-21 21:57:37,394 - DEBUG - MainProcess[MainThread|create_netbox_connection] -   Interface connection creation JSON: '{"connection_status": true, "interface_b": 6863690, "interface_a": 6708248}'
2018-08-21 21:57:37,395 - DEBUG - MainProcess[MainThread|create_netbox_connection] -   AutoCreate flag set, trying to create interface connection
2018-08-21 21:57:37,448 - WARNING - MainProcess[MainThread|create_netbox_connection] - Response Code: 500
Reason: Internal Server Error
Text:
<!DOCTYPE html>
<html lang="en">

<head>
    <title>Server Error</title>
    <link rel="stylesheet" href="/static/bootstrap-3.3.7-dist/css/bootstrap.min.css">
    <link rel="stylesheet" href="/static/font-awesome-4.7.0/css/font-awesome.min.css">
    <meta charset="UTF-8">
</head>

<body>
    <div class="container-fluid">
        <div class="row">
            <div class="col-md-6 col-md-offset-3">
                <div class="panel panel-danger" style="margin-top: 200px">
                    <div class="panel-heading">
                        <strong>
                            <i class="fa fa-warning"></i>
                            Server Error
                        </strong>
                    </div>
                    <div class="panel-body">

                            <p>
                                There was a problem with your request. Please contact an administrator.
                            </p>

                        <hr />
                        <p>
                            The complete exception is provided below:
                        </p>
<pre><strong>&lt;class &#39;django.db.utils.IntegrityError&#39;&gt;</strong><br />
duplicate key value violates unique constraint &quot;dcim_interfaceconnection_interface_a_id_key&quot;
DETAIL:  Key (interface_a_id)=(6708248) already exists.
</pre>
                        <p>
                            If further assistance is required, please post to the <a href="https://groups.google.com/forum/#!forum/netbox-discuss">NetBox mailing list</a>.
                        </p>
                        <div class="text-right">
                            <a href="/" class="btn btn-primary">Home Page</a>
                        </div>
                    </div>
                </div>
            </div>
        </div>
    </div>
</body>

</html>

When I look at the WebUI for the system, the NIC is connected.

Querying the API at https://netbox.DOMAIN/api/dcim/interface-connections/?device=bserver39 shows the connection between the NIC eth1 and the switch's Ethernet1/7:

{
  "count": 6,
  "next": null,
  "previous": null,
  "results": [
    {
      "id": 327507,
      "interface_a": {
        "id": 6708248,
        "url": "https://netbox.DOMAIN/api/dcim/interfaces/6708248/",
        "device": {
          "id": 380882,
          "url": "https://netbox.DOMAIN/api/dcim/devices/380882/",
          "name": "bserver39",
          "display_name": "bserver39"
        },
        "name": "eth1"
      },
      "interface_b": {
        "id": 6863690,
        "url": "https://netbox.DOMAIN/api/dcim/interfaces/6863690/",
        "device": {
          "id": 392004,
          "url": "https://netbox.DOMAIN/api/dcim/devices/392004/",
          "name": "switch1",
          "display_name": "switch1"
        },
        "name": "Ethernet1/17"
      },
      "connection_status": {
        "label": "Connected",
        "value": true
      }
    }
[Truncated for brevity]

Querying the API at https://netbox.DOMAIN/api/dcim/interface-connections/327507/ returns the correct data.

Originally created by @pm17788 on GitHub (Aug 23, 2018). ### Environment * Python version: 3.5.2 * NetBox version: 2.4.3 ### Steps to Reproduce HTTP GET to `/api/dcim/interface-connections/`, iterate over the 'results' list, iterate over list elements and store data in a DICT with the 'id' as the key. A script I wrote to do this: https://github.com/pm17788/NetBox-fu/blob/master/helpers_and_one-offs/save_netbox_data.py Full command: ``` python3 helpers_and_one-offs/save_netbox_data.py \ --conf ./conf_salt_importer_interface-connections-only.yaml \ --savefile json/netbox_data_interfaces_only.json \ --api_endpoint_results_savefile interface-connections_whurr.save \ --verbose \ --logfile interface-connections_whurr.log ``` Script's runtime log: [interface-connections_whurr.log.gz](https://github.com/digitalocean/netbox/files/2314851/interface-connections_whurr.log.gz) Script's saved "results" list: (6619 elements) [interface-connections_whurr.save.gz](https://github.com/digitalocean/netbox/files/2314852/interface-connections_whurr.save.gz) Script's saved "hashed by `id` DICT: (5801 elements) [netbox_data_interfaces_only.json.gz](https://github.com/digitalocean/netbox/files/2314854/netbox_data_interfaces_only.json.gz) ### Expected: Unique connection IDs, full data ### What appears to happen: Duplicate connection entries (duplicate 'id' and duplicate data), missing data I have a script which parses a salt-gathered LLDP scrape in our environment, and tries to create the connections in NetBox based on those results. It starts the run by gathering all the `interface-connections` so that it can handle when an interface is already connected to something. This check (so yeah, I got inventive in a function name, and called it *is_nic_in_connection* :) ) has been failing, because the connection wasn't being found in the bulk get. ``` zgrep 6708248 json/netbox_data_interfaces_only.json.gz zgrep 6708248 interface-connections_whurr.save.gz ``` The script failing looks like this: ``` 2018-08-21 21:57:37,367 - INFO - MainProcess[MainThread|main] - Salt entry: 'bserver39' (16 of 6063, # of NICs: 3) 2018-08-21 21:57:37,368 - DEBUG - MainProcess[MainThread|main] - |-> NIC: eth1 2018-08-21 21:57:37,368 - INFO - MainProcess[MainThread|main] - |-> LLDP data for: 'bserver39':'eth1' 2018-08-21 21:57:37,369 - INFO - MainProcess[MainThread|main] - |-> LLDP: Dev 'bserver39' <=> Dev 'switch1' 2018-08-21 21:57:37,370 - DEBUG - MainProcess[MainThread|main] - |-> Remote chassis ('switch1') info in NetBox 2018-08-21 21:57:37,371 - DEBUG - MainProcess[MainThread|main] - |-> Remote port ('Ethernet1/17') in NetBox listing for remote chassis 2018-08-21 21:57:37,371 - DEBUG - MainProcess[MainThread|main] - |-> Interface A ID: 6708248 2018-08-21 21:57:37,386 - DEBUG - MainProcess[MainThread|is_nic_in_connection] - Interface 6708248 didn't show up as part of any connection 2018-08-21 21:57:37,386 - DEBUG - MainProcess[MainThread|main] - Interface A (6708248) doesn't seem to be involved in any connections 2018-08-21 21:57:37,386 - DEBUG - MainProcess[MainThread|main] - |-> Interface B ID: 6863690 2018-08-21 21:57:37,394 - DEBUG - MainProcess[MainThread|is_nic_in_connection] - Interface 6863690 didn't show up as part of any connection 2018-08-21 21:57:37,394 - DEBUG - MainProcess[MainThread|main] - Interface B (6863690) doesn't seem to be involved in any connections 2018-08-21 21:57:37,394 - INFO - MainProcess[MainThread|main] - LLDP: Dev:Port bserver39:eth1(6708248) <=> Dev:Port switch1:Ethernet1/17(6863690) match. 2018-08-21 21:57:37,394 - INFO - MainProcess[MainThread|create_netbox_connection] - Attempting to create NetBox connection between interface IDs: 6708248 and 6863690, status: True 2018-08-21 21:57:37,394 - DEBUG - MainProcess[MainThread|create_netbox_connection] - Request URL: 'https://netbox.DOMAIN:443/api/dcim/interface-connections/' 2018-08-21 21:57:37,394 - DEBUG - MainProcess[MainThread|create_netbox_connection] - Interface connection creation JSON: '{"connection_status": true, "interface_b": 6863690, "interface_a": 6708248}' 2018-08-21 21:57:37,395 - DEBUG - MainProcess[MainThread|create_netbox_connection] - AutoCreate flag set, trying to create interface connection 2018-08-21 21:57:37,448 - WARNING - MainProcess[MainThread|create_netbox_connection] - Response Code: 500 Reason: Internal Server Error Text: <!DOCTYPE html> <html lang="en"> <head> <title>Server Error</title> <link rel="stylesheet" href="/static/bootstrap-3.3.7-dist/css/bootstrap.min.css"> <link rel="stylesheet" href="/static/font-awesome-4.7.0/css/font-awesome.min.css"> <meta charset="UTF-8"> </head> <body> <div class="container-fluid"> <div class="row"> <div class="col-md-6 col-md-offset-3"> <div class="panel panel-danger" style="margin-top: 200px"> <div class="panel-heading"> <strong> <i class="fa fa-warning"></i> Server Error </strong> </div> <div class="panel-body"> <p> There was a problem with your request. Please contact an administrator. </p> <hr /> <p> The complete exception is provided below: </p> <pre><strong>&lt;class &#39;django.db.utils.IntegrityError&#39;&gt;</strong><br /> duplicate key value violates unique constraint &quot;dcim_interfaceconnection_interface_a_id_key&quot; DETAIL: Key (interface_a_id)=(6708248) already exists. </pre> <p> If further assistance is required, please post to the <a href="https://groups.google.com/forum/#!forum/netbox-discuss">NetBox mailing list</a>. </p> <div class="text-right"> <a href="/" class="btn btn-primary">Home Page</a> </div> </div> </div> </div> </div> </div> </body> </html> ``` When I look at the WebUI for the system, the NIC **is** connected. Querying the API at `https://netbox.DOMAIN/api/dcim/interface-connections/?device=bserver39` shows the connection between the NIC eth1 and the switch's `Ethernet1/7`: ``` { "count": 6, "next": null, "previous": null, "results": [ { "id": 327507, "interface_a": { "id": 6708248, "url": "https://netbox.DOMAIN/api/dcim/interfaces/6708248/", "device": { "id": 380882, "url": "https://netbox.DOMAIN/api/dcim/devices/380882/", "name": "bserver39", "display_name": "bserver39" }, "name": "eth1" }, "interface_b": { "id": 6863690, "url": "https://netbox.DOMAIN/api/dcim/interfaces/6863690/", "device": { "id": 392004, "url": "https://netbox.DOMAIN/api/dcim/devices/392004/", "name": "switch1", "display_name": "switch1" }, "name": "Ethernet1/17" }, "connection_status": { "label": "Connected", "value": true } } [Truncated for brevity] ``` Querying the API at `https://netbox.DOMAIN/api/dcim/interface-connections/327507/` returns the correct data.
adam closed this issue 2025-12-29 17:21:01 +01:00
adam commented 2025-12-29 17:21:02 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Sep 5, 2018):

This is far too involved for a bug report. We cannot troubleshoot a custom script. If you believe this is an error in NetBox, please open a new bug including the exact steps someone else can take to reproduce the issue, using only native NetBox UI/API operations.

@jeremystretch commented on GitHub (Sep 5, 2018): This is far too involved for a bug report. We cannot troubleshoot a custom script. If you believe this is an error in NetBox, please open a new bug including the **exact** steps someone else can take to reproduce the issue, using only native NetBox UI/API operations.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#1969
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?