starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Displaying the page of a prefix with many child prefixes (each having a child range) runs into gateway timeout #8952

New Issue
Closed
opened 2025-12-29 20:43:16 +01:00 by adam · 6 comments
adam commented 2025-12-29 20:43:16 +01:00
Owner
Copy Link

Originally created by @reduzent on GitHub (Dec 12, 2023).

Originally assigned to: @Smixi on GitHub.

Deployment Type

Self-hosted

NetBox Version

v3.6.6

Python Version

3.11

Steps to Reproduce

The page /ipam/prefixes/1/ on our Netbox instance takes more than 20mins to display and runs into a timeout when not increasing timeout values.

My test system is a Debian 12 VM (VMware VM) with netbox 3.6.6. The system has two CPU cores and 4 GB Memory. Initial state is an empty database.

I can trigger the 504 Gateway Time-out (nginx error) when I create a prefix with 1000 child prefixes where each child prefix has one child range.

This is how I create the objects in nbshell:

mainpref = Prefix(prefix='10.128.0.0/11', status='active')
mainpref.full_clean()
mainpref.save()

mainpref = Prefix.objects.filter(prefix='10.128.0.0/11')[0]

prefsize = 256
prefcount = 1000

for i in range(0, prefcount*prefsize, prefsize):
    newpref = Prefix(prefix=mainpref.prefix[i].format() + '/24', status='active')
    newpref.full_clean()
    newpref.save()

for cpref in mainpref.get_child_prefixes():
    newrange = IPRange(start_address=cpref.prefix[100].format() + '/24' , end_address=cpref.prefix[250].format() + '/24' , status='active')
    newrange.full_clean()
    newrange.save()

If you apply this to a new installation, this URL triggers the timeout:

/ipam/prefixes/1/

Or click the result of this search:

/ipam/prefixes/?q=10.128.0.0%2F11

There is a related https://github.com/netbox-community/netbox/discussions/13471 where it appears that we can't do much about it but hope that this going to be addressed in Netbox itself.

Expected Behavior

The page should be rendered within reasonable time.

Observed Behavior

We get a 504 when trying to visit the particular page.

Originally created by @reduzent on GitHub (Dec 12, 2023). Originally assigned to: @Smixi on GitHub. ### Deployment Type Self-hosted ### NetBox Version v3.6.6 ### Python Version 3.11 ### Steps to Reproduce The page `/ipam/prefixes/1/` on our Netbox instance takes more than 20mins to display and runs into a timeout when not increasing timeout values. My test system is a Debian 12 VM (VMware VM) with netbox 3.6.6. The system has two CPU cores and 4 GB Memory. Initial state is an empty database. I can trigger the 504 Gateway Time-out (nginx error) when I create a prefix with 1000 child prefixes where each child prefix has one child range. This is how I create the objects in nbshell: ``` mainpref = Prefix(prefix='10.128.0.0/11', status='active') mainpref.full_clean() mainpref.save() mainpref = Prefix.objects.filter(prefix='10.128.0.0/11')[0] prefsize = 256 prefcount = 1000 for i in range(0, prefcount*prefsize, prefsize): newpref = Prefix(prefix=mainpref.prefix[i].format() + '/24', status='active') newpref.full_clean() newpref.save() for cpref in mainpref.get_child_prefixes(): newrange = IPRange(start_address=cpref.prefix[100].format() + '/24' , end_address=cpref.prefix[250].format() + '/24' , status='active') newrange.full_clean() newrange.save() ``` If you apply this to a new installation, this URL triggers the timeout: ``` /ipam/prefixes/1/ ``` Or click the result of this search: ``` /ipam/prefixes/?q=10.128.0.0%2F11 ``` There is a related https://github.com/netbox-community/netbox/discussions/13471 where it appears that we can't do much about it but hope that this going to be addressed in Netbox itself. ### Expected Behavior The page should be rendered within reasonable time. ### Observed Behavior We get a 504 when trying to visit the particular page.
adam added the type: bugstatus: acceptedseverity: medium labels 2025-12-29 20:43:16 +01:00
adam closed this issue 2025-12-29 20:43:16 +01:00
adam commented 2025-12-29 20:43:16 +01:00
Author
Owner
Copy Link

@reduzent commented on GitHub (Dec 12, 2023):

This is a duplicate of #13880 which was closed due missing information. Missing information is now hopefully provided.

@reduzent commented on GitHub (Dec 12, 2023): This is a duplicate of #13880 which was closed due missing information. Missing information is now hopefully provided.
adam commented 2025-12-29 20:43:16 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Dec 12, 2023):

The delay stems from the call to get_available_ips() for the parent prefix. It's actually happening twice: first with the call to return the number of available IPs, and again to return the first available IP. We need to optimize the logic for calculating available IPs when accounting for ranges.

@jeremystretch commented on GitHub (Dec 12, 2023): The delay stems from the call to `get_available_ips()` for the parent prefix. It's actually happening twice: first with the call to return the number of available IPs, and again to return the first available IP. We need to optimize the logic for calculating available IPs when accounting for ranges.
adam commented 2025-12-29 20:43:17 +01:00
Author
Owner
Copy Link

@Smixi commented on GitHub (Feb 5, 2024):

Hello,
I made a small PR for this if you want to look at it. I just investigated specifically on the mentionned function, but I guess it can be used anywhere where netaddr.ipset are used.

This is related to how adding and removing items in the sets works. It's seem its way faster to make a list of network items then transform it as a ipset rather than adding them one by one. I think this is because it make internal loops everytime you add items in it.

The page loads with the given script in ~6s on my computer.

@Smixi commented on GitHub (Feb 5, 2024): Hello, I made a small PR for this if you want to look at it. I just investigated specifically on the mentionned function, but I guess it can be used anywhere where netaddr.ipset are used. This is related to how adding and removing items in the sets works. It's seem its way faster to make a list of network items then transform it as a ipset rather than adding them one by one. I think this is because it make internal loops everytime you add items in it. The page loads with the given script in ~6s on my computer.
adam commented 2025-12-29 20:43:17 +01:00
Author
Owner
Copy Link

@arthanson commented on GitHub (Feb 5, 2024):

Assigning to you @Smixi as I'm working on other bugfixes right now.

@arthanson commented on GitHub (Feb 5, 2024): Assigning to you @Smixi as I'm working on other bugfixes right now.
adam commented 2025-12-29 20:43:17 +01:00
Author
Owner
Copy Link

@Smixi commented on GitHub (Feb 5, 2024):

Note: Maybe we can also add cached property into prefix as it is made inside IPRange ? It will solve the second point mentionned here :

The delay stems from the call to get_available_ips() for the parent prefix. It's actually happening twice: first with the call to return the number of available IPs, and again to return the first available IP.

Or create another issue ?

@Smixi commented on GitHub (Feb 5, 2024): Note: Maybe we can also add cached property into prefix as it is made inside IPRange ? It will solve the second point mentionned here : > The delay stems from the call to `get_available_ips()` for the parent prefix. It's actually happening twice: first with the call to return the number of available IPs, and again to return the first available IP. Or create another issue ?
adam commented 2025-12-29 20:43:17 +01:00
Author
Owner
Copy Link

@reduzent commented on GitHub (Feb 6, 2024):

Nice. Thanks for fixing!

@reduzent commented on GitHub (Feb 6, 2024): Nice. Thanks for fixing!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label severity: medium status: accepted type: bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#8952
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?