starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Slow API calls when set to no limit #1434

New Issue
Closed
opened 2025-12-29 16:32:10 +01:00 by adam · 4 comments
adam commented 2025-12-29 16:32:10 +01:00
Owner
Copy Link

Originally created by @matt852 on GitHub (Nov 30, 2017).

Issue type

[ ] Feature request
[X] Bug report
[ ] Documentation

Environment

  • Python version: 2.7.6
  • NetBox version: 2.2.1

Description

I have the following Python v2 code that I wrote, making API calls against my Netbox server to pull all devices:

response = []
url = "http://netbox.domain.com/api/dcim/devices/?limit=0"
response = urlopen(url)
json_result = load(response)
return json_result['results']

For reference, I currently have 559 total devices in Netbox. When refreshing my webpage displaying the API call results, it takes the following amount of time to load (5x full-refreshes):

 4.58s, 4.61s, 5.18s, 4.05s, 4.44s

When I set the limit to 100, these are the load times (5x full-refreshes):

 1.82s, 821ms, 1.69s, 1.04s, 998ms

However since the API call doesn't pull all of the devices, I can't filter out the devices I need (based on a specific parameter). If there's a better way to call filtered results, or speed up this call, please advise. Thanks.

Originally created by @matt852 on GitHub (Nov 30, 2017). <!-- Before opening a new issue, please search through the existing issues to see if your topic has already been addressed. Note that you may need to remove the "is:open" filter from the search bar to include closed issues. Check the appropriate type for your issue below by placing an x between the brackets. For assistance with installation issues, or for any other issues other than those listed below, please raise your topic for discussion on our mailing list: https://groups.google.com/forum/#!forum/netbox-discuss Please note that issues which do not fall under any of the below categories will be closed. Due to an excessive backlog of feature requests, we are not currently accepting any proposals which extend NetBox's feature scope. Do not prepend any sort of tag to your issue's title. An administrator will review your issue and assign labels as appropriate. ---> ### Issue type [ ] Feature request <!-- An enhancement of existing functionality --> [X] Bug report <!-- Unexpected or erroneous behavior --> [ ] Documentation <!-- A modification to the documentation --> <!-- Please describe the environment in which you are running NetBox. (Be sure to verify that you are running the latest stable release of NetBox before submitting a bug report.) If you are submitting a bug report and have made any changes to the code base, please first validate that your bug can be recreated while running an official release. --> ### Environment * Python version: 2.7.6 <!-- Example: 3.5.4 --> * NetBox version: 2.2.1 <!-- Example: 2.1.3 --> <!-- BUG REPORTS must include: * A list of the steps needed for someone else to reproduce the bug * A description of the expected and observed behavior * Any relevant error messages (screenshots may also help) FEATURE REQUESTS must include: * A detailed description of the proposed functionality * A use case for the new feature * A rough description of any necessary changes to the database schema * Any relevant third-party libraries which would be needed --> ### Description I have the following Python v2 code that I wrote, making API calls against my Netbox server to pull all devices: response = [] url = "http://netbox.domain.com/api/dcim/devices/?limit=0" response = urlopen(url) json_result = load(response) return json_result['results'] For reference, I currently have 559 total devices in Netbox. When refreshing my webpage displaying the API call results, it takes the following amount of time to load (5x full-refreshes): 4.58s, 4.61s, 5.18s, 4.05s, 4.44s When I set the limit to 100, these are the load times (5x full-refreshes): 1.82s, 821ms, 1.69s, 1.04s, 998ms However since the API call doesn't pull all of the devices, I can't filter out the devices I need (based on a specific parameter). If there's a better way to call filtered results, or speed up this call, please advise. Thanks.
adam closed this issue 2025-12-29 16:32:10 +01:00
adam commented 2025-12-29 16:32:11 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Nov 30, 2017):

Using a local development instance, retrieving 500 devices takes around one second:

$ time curl -s -H "Content-Type: application/json" -H "Accept: application/json; indent=4" http://localhost:8000/api/dcim/devices/?limit=500 > /dev/null

real	0m0.992s
user	0m0.008s
sys	0m0.000s

What times are you getting on a local instance using curl? (Avoid using the browser for testing, since rendering the DOM takes additional time.)

However since the API call doesn't pull all of the devices, I can't filter out the devices I need (based on a specific parameter). If there's a better way to call filtered results, or speed up this call, please advise.

You can filter on pretty much any field. What devices do you want to retrieve?

@jeremystretch commented on GitHub (Nov 30, 2017): Using a local development instance, retrieving 500 devices takes around one second: ``` $ time curl -s -H "Content-Type: application/json" -H "Accept: application/json; indent=4" http://localhost:8000/api/dcim/devices/?limit=500 > /dev/null real 0m0.992s user 0m0.008s sys 0m0.000s ``` What times are you getting on a local instance using `curl`? (Avoid using the browser for testing, since rendering the DOM takes additional time.) > However since the API call doesn't pull all of the devices, I can't filter out the devices I need (based on a specific parameter). If there's a better way to call filtered results, or speed up this call, please advise. You can filter on pretty much any field. What devices do you want to retrieve?
adam commented 2025-12-29 16:32:11 +01:00
Author
Owner
Copy Link

@matt852 commented on GitHub (Nov 30, 2017):

Thanks Jeremy. For the filtering on fields, I'm trying to filter on a custom field I've created, just not having luck querying a nested JSON object. I want to return results where the label is set to Yes.

        "custom_fields": {
            "Netconfig": {
                "value": 47,
                "label": "Yes"
            },

Here are my CURL results from running it on the Netbox server, making API calls against itself. Are you retrieving a full 500 devices on your local dev instance with that curl call? Also, do you get the same times for limit=0? If yes to both, it may just be an issue with my own server install.

user@netbox:~$ time curl -s -H "Content-Type: application/json" -H "Accept: application/json; indent=4" http://127.0.0.1/api/dcim/devices/?limit=0 > /dev/null

real 0m3.535s
user 0m0.004s
sys 0m0.004s

user@netbox:~$ time curl -s -H "Content-Type: application/json" -H "Accept: application/json; indent=4" http://127.0.0.1/api/dcim/devices/?limit=100 > /dev/null

real 0m0.636s
user 0m0.012s
sys 0m0.000s

mvitale@netbox:~$ time curl -s -H "Content-Type: application/json" -H "Accept: application/json; indent=4" http://127.0.0.1/api/dcim/devices/?limit=500 > /dev/null

real 0m3.185s
user 0m0.008s
sys 0m0.000s

user@netbox:~$ time curl -s -H "Content-Type: application/json" -H "Accept: application/json; indent=4" http://127.0.0.1/api/dcim/devices/?limit=1000 > /dev/null

real 0m3.609s
user 0m0.004s
sys 0m0.000s

@matt852 commented on GitHub (Nov 30, 2017): Thanks Jeremy. For the filtering on fields, I'm trying to filter on a custom field I've created, just not having luck querying a nested JSON object. I want to return results where the label is set to Yes. "custom_fields": { "Netconfig": { "value": 47, "label": "Yes" }, Here are my CURL results from running it on the Netbox server, making API calls against itself. Are you retrieving a full 500 devices on your local dev instance with that curl call? Also, do you get the same times for limit=0? If yes to both, it may just be an issue with my own server install. user@netbox:~$ time curl -s -H "Content-Type: application/json" -H "Accept: application/json; indent=4" http://127.0.0.1/api/dcim/devices/?limit=0 > /dev/null real 0m3.535s user 0m0.004s sys 0m0.004s user@netbox:~$ time curl -s -H "Content-Type: application/json" -H "Accept: application/json; indent=4" http://127.0.0.1/api/dcim/devices/?limit=100 > /dev/null real 0m0.636s user 0m0.012s sys 0m0.000s mvitale@netbox:~$ time curl -s -H "Content-Type: application/json" -H "Accept: application/json; indent=4" http://127.0.0.1/api/dcim/devices/?limit=500 > /dev/null real 0m3.185s user 0m0.008s sys 0m0.000s user@netbox:~$ time curl -s -H "Content-Type: application/json" -H "Accept: application/json; indent=4" http://127.0.0.1/api/dcim/devices/?limit=1000 > /dev/null real 0m3.609s user 0m0.004s sys 0m0.000s
adam commented 2025-12-29 16:32:11 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Dec 5, 2017):

Are you retrieving a full 500 devices on your local dev instance with that curl call?

Yes

Also, do you get the same times for limit=0?

No; I have many thousands of devices defined in NetBox, so retrieving all of them takes longer, but the increase in delay seems pretty linear.

I suggest playing around a bit with the debug toolbar (you'll need to set DEBUG=True in configuration.py) to see if you can get a feel for where the delay is occurring.

@jeremystretch commented on GitHub (Dec 5, 2017): > Are you retrieving a full 500 devices on your local dev instance with that curl call? Yes > Also, do you get the same times for limit=0? No; I have many thousands of devices defined in NetBox, so retrieving all of them takes longer, but the increase in delay seems pretty linear. I suggest playing around a bit with the debug toolbar (you'll need to set `DEBUG=True` in configuration.py) to see if you can get a feel for where the delay is occurring.
adam commented 2025-12-29 16:32:11 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Dec 13, 2017):

Not sure there's a problem here. Closing due to lack of activity.

@jeremystretch commented on GitHub (Dec 13, 2017): Not sure there's a problem here. Closing due to lack of activity.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#1434
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?