starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Inconsistent /api/virtualization/interfaces/ results #1819

New Issue
Closed
opened 2025-12-29 17:19:25 +01:00 by adam · 12 comments
adam commented 2025-12-29 17:19:25 +01:00
Owner
Copy Link

Originally created by @dsanader on GitHub (Jun 28, 2018).

Issue type

[ ] Feature request
[ x ] Bug report
[ ] Documentation

Environment

  • Python version: 3.6.5
  • NetBox version: 2.3.4

Description

Querying all virtualized interfaces returns inconsistents results : the count is OK, but some interfaces are missing and some are duplicated.

The underlying query is ordering by an empty column ("dcim_device"."name") which seems to fall under the non predictable results case : https://www.postgresql.org/docs/current/static/queries-limit.html

... WHERE "dcim_interface"."virtual_machine_id" IS NOT NULL 
ORDER BY "dcim_device"."name" ASC, "dcim_interface"."name" ASC LIMIT 1000 OFFSET 50
Originally created by @dsanader on GitHub (Jun 28, 2018). <!-- Before opening a new issue, please search through the existing issues to see if your topic has already been addressed. Note that you may need to remove the "is:open" filter from the search bar to include closed issues. Check the appropriate type for your issue below by placing an x between the brackets. For assistance with installation issues, or for any other issues other than those listed below, please raise your topic for discussion on our mailing list: https://groups.google.com/forum/#!forum/netbox-discuss Please note that issues which do not fall under any of the below categories will be closed. Due to an excessive backlog of feature requests, we are not currently accepting any proposals which extend NetBox's feature scope. Do not prepend any sort of tag to your issue's title. An administrator will review your issue and assign labels as appropriate. ---> ### Issue type [ ] Feature request <!-- An enhancement of existing functionality --> [ x ] Bug report <!-- Unexpected or erroneous behavior --> [ ] Documentation <!-- A modification to the documentation --> <!-- Please describe the environment in which you are running NetBox. (Be sure to verify that you are running the latest stable release of NetBox before submitting a bug report.) If you are submitting a bug report and have made any changes to the code base, please first validate that your bug can be recreated while running an official release. --> ### Environment * Python version: 3.6.5 * NetBox version: 2.3.4 <!-- BUG REPORTS must include: * A list of the steps needed for someone else to reproduce the bug * A description of the expected and observed behavior * Any relevant error messages (screenshots may also help) FEATURE REQUESTS must include: * A detailed description of the proposed functionality * A use case for the new feature * A rough description of any necessary changes to the database schema * Any relevant third-party libraries which would be needed --> ### Description Querying all virtualized interfaces returns inconsistents results : the count is OK, but some interfaces are missing and some are duplicated. The underlying query is ordering by an empty column ("dcim_device"."name") which seems to fall under the non predictable results case : https://www.postgresql.org/docs/current/static/queries-limit.html ```sql ... WHERE "dcim_interface"."virtual_machine_id" IS NOT NULL ORDER BY "dcim_device"."name" ASC, "dcim_interface"."name" ASC LIMIT 1000 OFFSET 50 ```
adam added the type: bugstatus: accepted labels 2025-12-29 17:19:25 +01:00
adam closed this issue 2025-12-29 17:19:25 +01:00
adam commented 2025-12-29 17:19:28 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Jun 29, 2018):

Thank you for taking the time to report a bug. We'll need more information to investigate this issue. Per the contributing guidelines please ensure that you have included all of the following:

  • The environment in which NetBox is running
  • The exact steps that can be taken to reproduce the issue
  • Any error messages generated
  • Screenshots (if applicable)

Please update your issue to include all of the information listed above. If no response is received within a week, this issue will be closed. Thanks!

@jeremystretch commented on GitHub (Jun 29, 2018): Thank you for taking the time to report a bug. We'll need more information to investigate this issue. Per the [contributing guidelines](https://github.com/digitalocean/netbox/blob/develop/CONTRIBUTING.md) please ensure that you have included all of the following: * The environment in which NetBox is running * **The exact steps that can be taken to reproduce the issue** * Any error messages generated * Screenshots (if applicable) Please update your issue to include all of the information listed above. If no response is received within a week, this issue will be closed. Thanks!
adam commented 2025-12-29 17:19:30 +01:00
Author
Owner
Copy Link

@dsanader commented on GitHub (Jun 29, 2018):

import pynetbox

for vif in nb.virtualization.interfaces.all():
     print(vif.id)

Duplicate ids are printed.

I guess it requires you to have enough interfaces to get this bug triggered consistently.

I've "fixed" it by changing the ordering in the Interface model (ordering = ['id', 'name'] instead of ordering = ['device', 'name']).

@dsanader commented on GitHub (Jun 29, 2018): ```python import pynetbox for vif in nb.virtualization.interfaces.all(): print(vif.id) ``` Duplicate ids are printed. I guess it requires you to have enough interfaces to get this bug triggered consistently. I've "fixed" it by changing the ordering in the Interface model (```ordering = ['id', 'name']``` instead of ```ordering = ['device', 'name']```).
adam commented 2025-12-29 17:19:30 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Jun 29, 2018):

Still can't reproduce this. The queryset which retrieves the interfaces is:

Interface.objects.filter(virtual_machine__isnull=False).select_related('virtual_machine')

Not sure how that could result in duplicate objects being returned. I'll need sample data which triggers the bug to dig any further into this.

@jeremystretch commented on GitHub (Jun 29, 2018): Still can't reproduce this. The queryset which retrieves the interfaces is: ``` Interface.objects.filter(virtual_machine__isnull=False).select_related('virtual_machine') ``` Not sure how that could result in duplicate objects being returned. I'll need sample data which triggers the bug to dig any further into this.
adam commented 2025-12-29 17:19:30 +01:00
Author
Owner
Copy Link

@dsanader commented on GitHub (Jun 29, 2018):

Can I send you a dump of my database by email ?

@dsanader commented on GitHub (Jun 29, 2018): Can I send you a dump of my database by email ?
adam commented 2025-12-29 17:19:30 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Jul 18, 2018):

Sorry, you'll need to derive a sample set from your data which can be used to replicate the problem and post it here.

@jeremystretch commented on GitHub (Jul 18, 2018): Sorry, you'll need to derive a sample set from your data which can be used to replicate the problem and post it here.
adam commented 2025-12-29 17:19:31 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Jul 27, 2018):

Closing due to lack of activity.

@jeremystretch commented on GitHub (Jul 27, 2018): Closing due to lack of activity.
adam commented 2025-12-29 17:19:31 +01:00
Author
Owner
Copy Link

@dsanader commented on GitHub (Feb 21, 2019):

Hello Jeremy,

Finally got some time to work on this.

The key is to a have a mixed set of device and vm interfaces and to have a count above the paging size (>1000).

On a fresh 2.5.6 install with the initial data and after loading some dummy devices and vms with the attached script, you should witness the inconsistency :

vifs = [iface.id for iface in nb.virtualization.interfaces.all()]

len(vifs)
600

len(set(vifs))
567

reproduce_2207.py.txt

@dsanader commented on GitHub (Feb 21, 2019): Hello Jeremy, Finally got some time to work on this. The key is to a have a mixed set of device and vm interfaces and to have a count above the paging size (>1000). On a fresh 2.5.6 install with the initial data and after loading some dummy devices and vms with the attached script, you should witness the inconsistency : ```python vifs = [iface.id for iface in nb.virtualization.interfaces.all()] len(vifs) 600 len(set(vifs)) 567 ``` [reproduce_2207.py.txt](https://github.com/digitalocean/netbox/files/2886832/reproduce_2207.py.txt)
adam commented 2025-12-29 17:19:31 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Feb 21, 2019):

@dsanader Unless you can recreate this using bare HTTP calls NetBox REST API, you'll need to open this as a bug against the pynetbox library.

@jeremystretch commented on GitHub (Feb 21, 2019): @dsanader Unless you can recreate this using bare HTTP calls NetBox REST API, you'll need to open this as a bug against the pynetbox library.
adam commented 2025-12-29 17:19:31 +01:00
Author
Owner
Copy Link

@dsanader commented on GitHub (Feb 21, 2019):

You're hard on me...

Anyway, same outcome with curl :

curl -H "authorization: Token d34db33f" "https://netbox-test.example.com/api/virtualization/interfaces/" > first.json
curl -H "authorization: Token d34db33f" "https://netbox-test.example.com/api/virtualization/interfaces/?limit=600&offset=50"> second.json
jq '.results[] | .id' first.json second.json | sort | uniq -c | wc -l
567

The issues goes away (600 unique ids) as soon as I :

index 3b0c02b2..f9502071 100644
--- a/netbox/virtualization/api/views.py
+++ b/netbox/virtualization/api/views.py
@@ -65,7 +65,7 @@ class VirtualMachineViewSet(CustomFieldModelViewSet):
 class InterfaceViewSet(ModelViewSet):
     queryset = Interface.objects.filter(
         virtual_machine__isnull=False
-    ).select_related('virtual_machine').prefetch_related('tags')
+    ).select_related('virtual_machine').prefetch_related('tags').order_by('id')
     serializer_class = serializers.InterfaceSerializer
     filterset_class = filters.InterfaceFilter
@dsanader commented on GitHub (Feb 21, 2019): You're hard on me... Anyway, same outcome with curl : ``` curl -H "authorization: Token d34db33f" "https://netbox-test.example.com/api/virtualization/interfaces/" > first.json curl -H "authorization: Token d34db33f" "https://netbox-test.example.com/api/virtualization/interfaces/?limit=600&offset=50"> second.json jq '.results[] | .id' first.json second.json | sort | uniq -c | wc -l 567 ``` The issues goes away (600 unique ids) as soon as I : ```diff --git a/netbox/virtualization/api/views.py b/netbox/virtualization/api/views.py index 3b0c02b2..f9502071 100644 --- a/netbox/virtualization/api/views.py +++ b/netbox/virtualization/api/views.py @@ -65,7 +65,7 @@ class VirtualMachineViewSet(CustomFieldModelViewSet): class InterfaceViewSet(ModelViewSet): queryset = Interface.objects.filter( virtual_machine__isnull=False - ).select_related('virtual_machine').prefetch_related('tags') + ).select_related('virtual_machine').prefetch_related('tags').order_by('id') serializer_class = serializers.InterfaceSerializer filterset_class = filters.InterfaceFilter ```
adam commented 2025-12-29 17:19:31 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Mar 13, 2019):

This is almost identical to #2938, which was fixed in v2.5.8, except that the Interface model uses its own manager.

@jeremystretch commented on GitHub (Mar 13, 2019): This is almost identical to #2938, which was fixed in v2.5.8, except that the Interface model uses its own manager.
adam commented 2025-12-29 17:19:31 +01:00
Author
Owner
Copy Link

@pm17788 commented on GitHub (Apr 1, 2019):

Tested PR 2996 on my NetBox v 2.5.8 instance:

69         sql_col = '{}.name'.format(self.model._meta.db_table)
70         ordering = [
71             '_slot', '_subslot', '_position', '_subposition', '_type', '_id', '_channel', '_vc', 'name', 'pk'
72         ]
73

Still the same issue with interfaces not being returned as a unique list.

@pm17788 commented on GitHub (Apr 1, 2019): Tested PR 2996 on my NetBox v 2.5.8 instance: ```python 69 sql_col = '{}.name'.format(self.model._meta.db_table) 70 ordering = [ 71 '_slot', '_subslot', '_position', '_subposition', '_type', '_id', '_channel', '_vc', 'name', 'pk' 72 ] 73 ``` Still the same issue with interfaces not being returned as a unique list.
adam commented 2025-12-29 17:19:31 +01:00
Author
Owner
Copy Link

@pm17788 commented on GitHub (Apr 3, 2019):

Tested on a full 2.5.9, and that works. Probably something I messed up with my test or some such. Huzzah!

@pm17788 commented on GitHub (Apr 3, 2019): Tested on a full 2.5.9, and that works. Probably something I messed up with my test or some such. Huzzah!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label status: accepted type: bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#1819
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?