starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Nginx returns 502 Bad Gateway error during fetching all the items or if limit parameter more than 1000 #3285

New Issue
Closed
opened 2025-12-29 18:27:25 +01:00 by adam · 5 comments
adam commented 2025-12-29 18:27:25 +01:00
Owner
Copy Link

Originally created by @heineek on GitHub (Feb 6, 2020).

Environment

  • Python version: 3.6.9
  • NetBox version: 2.7.4

Steps to Reproduce

  1. Open the URL http://netbox_address/api/dcim/devices/?limit=1001 or http://netbox_address/api/dcim/devices/?limit=0
  2. nginx returns 502 Bad Gateway error
  3. There is same behavior when I make any request to any endpoint with more than 1000 items whatever with API or Web-interface. MAX_PAGE_SIZE=0 in configuration.py

Expected Behavior

Full list of items must be returned like in release v2.6

Observed Behavior

502 Bad Gateway Error. Possible it is request timeout.

Originally created by @heineek on GitHub (Feb 6, 2020). ### Environment * Python version: 3.6.9 * NetBox version: 2.7.4 ### Steps to Reproduce 1. Open the URL http://netbox_address/api/dcim/devices/?limit=1001 or http://netbox_address/api/dcim/devices/?limit=0 2. nginx returns 502 Bad Gateway error 3. There is same behavior when I make any request to any endpoint with more than 1000 items whatever with API or Web-interface. MAX_PAGE_SIZE=0 in configuration.py ### Expected Behavior Full list of items must be returned like in release v2.6 ### Observed Behavior 502 Bad Gateway Error. Possible it is request timeout.
adam added the status: revisions needed label 2025-12-29 18:27:25 +01:00
adam closed this issue 2025-12-29 18:27:25 +01:00
adam commented 2025-12-29 18:27:26 +01:00
Author
Owner
Copy Link

@heineek commented on GitHub (Feb 6, 2020):

Also when I use pynetbox and try to run "sites = nb.dcim.sites.all()" I get the same error.

@heineek commented on GitHub (Feb 6, 2020): Also when I use pynetbox and try to run "sites = nb.dcim.sites.all()" I get the same error.
adam commented 2025-12-29 18:27:26 +01:00
Author
Owner
Copy Link

@hSaria commented on GitHub (Feb 6, 2020):

I wasn't able to reproduce this, but this seems to be related to gunicorn or nginx, rather than NetBox.

@hSaria commented on GitHub (Feb 6, 2020): I wasn't able to reproduce this, but this seems to be related to gunicorn or nginx, rather than NetBox.
adam commented 2025-12-29 18:27:26 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Feb 6, 2020):

This needs more detail to be actionable. How many items can you successfully request? What is the nature of the 502 error? What do the nginx logs say? How is nginx configured?

@jeremystretch commented on GitHub (Feb 6, 2020): This needs more detail to be actionable. How many items can you _successfully_ request? What is the nature of the 502 error? What do the nginx logs say? How is nginx configured?
adam commented 2025-12-29 18:27:26 +01:00
Author
Owner
Copy Link

@heineek commented on GitHub (Feb 6, 2020):

The number of items that can be obtained successfully varies, but the quantity must be greater than 1000 in order for an error to be occur. Sometimes I can retrieve 2000 items but usually I cannot get more than 1000 of items. I suppose some timeout is root cause because these requests take more than 30 secs of time. The problem has appeared after updating from version 2.6 to latest release.

/var/log/nginx/error.log:

upstream prematurely closed connection while reading response header from upstream, client: xxx, server: xxx, request: "GET /api/dcim/devices/?limit=999 HTTP/1.1", upstream: "http://127.0.0.1:8001/api/dcim/devices/?limit=999", host: "xxx"

netbox@netbox:~$ cat /etc/nginx/sites-enabled/netbox
server {
listen 80;

server_name xxx;

client_max_body_size 25m;

location /static/ {
    alias /opt/netbox/netbox/static/;
}

location / {
    proxy_pass http://127.0.0.1:8001;
    proxy_set_header X-Forwarded-Host $server_name;
    proxy_set_header X-RealIP $remote_addr;
    proxy_set_header X-Forwarded-Proto $scheme;
    add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
}

}

netbox@netbox:~$ cat /opt/netbox/gunicorn.py

bind = '127.0.0.1:8002'
workers = 10
threads = 5
timeout = 240
max_requests = 5000
max_requests_jitter = 500

@heineek commented on GitHub (Feb 6, 2020): The number of items that can be obtained successfully varies, but the quantity must be greater than 1000 in order for an error to be occur. Sometimes I can retrieve 2000 items but usually I cannot get more than 1000 of items. I suppose some timeout is root cause because these requests take more than 30 secs of time. The problem has appeared after updating from version 2.6 to latest release. /var/log/nginx/error.log: upstream prematurely closed connection while reading response header from upstream, client: xxx, server: xxx, request: "GET /api/dcim/devices/?limit=999 HTTP/1.1", upstream: "http://127.0.0.1:8001/api/dcim/devices/?limit=999", host: "xxx" netbox@netbox:~$ cat /etc/nginx/sites-enabled/netbox server { listen 80; server_name xxx; client_max_body_size 25m; location /static/ { alias /opt/netbox/netbox/static/; } location / { proxy_pass http://127.0.0.1:8001; proxy_set_header X-Forwarded-Host $server_name; proxy_set_header X-RealIP $remote_addr; proxy_set_header X-Forwarded-Proto $scheme; add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"'; } } netbox@netbox:~$ cat /opt/netbox/gunicorn.py bind = '127.0.0.1:8002' workers = 10 threads = 5 timeout = 240 max_requests = 5000 max_requests_jitter = 500
adam commented 2025-12-29 18:27:26 +01:00
Author
Owner
Copy Link

@heineek commented on GitHub (Feb 7, 2020):

I solved the problem. My first mistake was that I did not delete the supervisord's configuration after migrating to systemd. Because of this, gunicorn was run twice on different ports and my setting changes had no visible results and there was a 502 error. After deleting supervisor, the error code became 504 (gateway timeout). Secondly, due to the fact that I have about 16,000 sites and 9,000 devices (yes, this is wi-fi infrastructure :-), requesting all positions takes more than a minute. After changing the following settings in /etc/nginx/conf.d/timeout.conf all is OK.

proxy_connect_timeout 600;
proxy_send_timeout 600;
proxy_read_timeout 600;
send_timeout 600;

Thank for your attention!

@heineek commented on GitHub (Feb 7, 2020): I solved the problem. My first mistake was that I did not delete the supervisord's configuration after migrating to systemd. Because of this, gunicorn was run twice on different ports and my setting changes had no visible results and there was a 502 error. After deleting supervisor, the error code became 504 (gateway timeout). Secondly, due to the fact that I have about 16,000 sites and 9,000 devices (yes, this is wi-fi infrastructure :-), requesting all positions takes more than a minute. After changing the following settings in /etc/nginx/conf.d/timeout.conf all is OK. proxy_connect_timeout 600; proxy_send_timeout 600; proxy_read_timeout 600; send_timeout 600; Thank for your attention!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label status: revisions needed
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#3285
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?