High CPU Usage & Crashes During ASP.NET API Calls #10661

Closed
opened 2025-12-29 21:34:24 +01:00 by adam · 3 comments
Owner

Originally created by @PapouMansot on GitHub (Jan 16, 2025).

Deployment Type

Self-hosted

Triage priority

N/A

NetBox Version

v4.2.1

Python Version

3.11

Steps to Reproduce

Set up a NetBox instance with virtual machines data
Create an ASP.NET application with the following service code:

public class NetBoxService : INetBoxService
{
private readonly HttpClient _httpClient;
private readonly ILogger _logger;

public NetBoxService(HttpClient httpClient, ILogger<NetBoxService> logger)
{
    _httpClient = httpClient;
    _logger = logger;
}

public async Task<List<VirtualMachine>> GetAllVirtualMachinesAsync()
{
    var allVMs = new List<VirtualMachine>();
    try
    {
        string url = "virtualization/virtual-machines/?limit=20";
        int maxPages = 10;
        int currentPage = 0;
        while (!string.IsNullOrEmpty(url) && currentPage < maxPages)
        {
            currentPage++;
            _logger.LogInformation($"Fetching page {currentPage}: {url}");
            var response = await _httpClient.GetAsync(url);
            if (!response.IsSuccessStatusCode)
            {
                _logger.LogError($"API Error: {response.StatusCode}");
                throw new Exception($"API Error: {response.StatusCode}");
            }
            var content = await response.Content.ReadAsStringAsync();
            var netBoxResponse = JsonConvert.DeserializeObject<NetBoxResponse<VirtualMachine>>(content);
            if (netBoxResponse?.Results != null)
            {
                allVMs.AddRange(netBoxResponse.Results);
            }
            url = netBoxResponse?.Next;
        }
    }
    catch (Exception ex)
    {
        _logger.LogError($"An error occurred: {ex.Message}");
        throw;
    }
    return allVMs;
}

}

Execute API calls to retrieve virtual machines data

Expected Behavior

API calls should complete successfully
CPU usage should remain stable
Service should continue running normally

Observed Behavior

CPU usage spikes to 100%
NetBox container crashes with signal 11 (core dumped)
Container enters restart loop
Error logs show:
2025/01/16 13:18:08 [alert] 7#7 process 174 exited on signal 11 (core dumped)
2025/01/16 13:18:08 [alert] 7#7 sendmsg(10, -1, -1, 1) failed (32: Broken pipe)
Workaround
Downgrading to NetBox 4.1.11 resolves the issue:
services:
netbox:
image: netboxcommunity/netbox:4.1.11
Additional Context

Issue occurs specifically with ASP.NET API calls
Problem is reproducible across multiple deployments
Affects virtual machines endpoint but may impact other endpoints as well

Originally created by @PapouMansot on GitHub (Jan 16, 2025). ### Deployment Type Self-hosted ### Triage priority N/A ### NetBox Version v4.2.1 ### Python Version 3.11 ### Steps to Reproduce Set up a NetBox instance with virtual machines data Create an ASP.NET application with the following service code: public class NetBoxService : INetBoxService { private readonly HttpClient _httpClient; private readonly ILogger<NetBoxService> _logger; public NetBoxService(HttpClient httpClient, ILogger<NetBoxService> logger) { _httpClient = httpClient; _logger = logger; } public async Task<List<VirtualMachine>> GetAllVirtualMachinesAsync() { var allVMs = new List<VirtualMachine>(); try { string url = "virtualization/virtual-machines/?limit=20"; int maxPages = 10; int currentPage = 0; while (!string.IsNullOrEmpty(url) && currentPage < maxPages) { currentPage++; _logger.LogInformation($"Fetching page {currentPage}: {url}"); var response = await _httpClient.GetAsync(url); if (!response.IsSuccessStatusCode) { _logger.LogError($"API Error: {response.StatusCode}"); throw new Exception($"API Error: {response.StatusCode}"); } var content = await response.Content.ReadAsStringAsync(); var netBoxResponse = JsonConvert.DeserializeObject<NetBoxResponse<VirtualMachine>>(content); if (netBoxResponse?.Results != null) { allVMs.AddRange(netBoxResponse.Results); } url = netBoxResponse?.Next; } } catch (Exception ex) { _logger.LogError($"An error occurred: {ex.Message}"); throw; } return allVMs; } } Execute API calls to retrieve virtual machines data ### Expected Behavior API calls should complete successfully CPU usage should remain stable Service should continue running normally ### Observed Behavior CPU usage spikes to 100% NetBox container crashes with signal 11 (core dumped) Container enters restart loop Error logs show: 2025/01/16 13:18:08 [alert] 7#7 process 174 exited on signal 11 (core dumped) 2025/01/16 13:18:08 [alert] 7#7 sendmsg(10, -1, -1, 1) failed (32: Broken pipe) **Workaround** Downgrading to NetBox 4.1.11 resolves the issue: services: netbox: image: netboxcommunity/netbox:4.1.11 **Additional Context** Issue occurs specifically with ASP.NET API calls Problem is reproducible across multiple deployments Affects virtual machines endpoint but may impact other endpoints as well
adam closed this issue 2025-12-29 21:34:24 +01:00
Author
Owner

@DanSheps commented on GitHub (Jan 16, 2025):

Can you identify what process that is part of NetBox itself that is causing this CPU spike?

@DanSheps commented on GitHub (Jan 16, 2025): Can you identify what process that is part of NetBox itself that is causing this CPU spike?
Author
Owner

@PapouMansot commented on GitHub (Jan 17, 2025):

Even with all logging enabled, I couldn't identify the root cause of the CPU spike within NetBox itself. After the API request, the container crashes without providing any significant error messages or logs. This suggests that the issue might be related to an underlying process in NetBox or an external dependency. Further investigation, such as profiling the application or monitoring the container's resource usage in real-time, might be necessary to pinpoint the problem.

@PapouMansot commented on GitHub (Jan 17, 2025): Even with all logging enabled, I couldn't identify the root cause of the CPU spike within NetBox itself. After the API request, the container crashes without providing any significant error messages or logs. This suggests that the issue might be related to an underlying process in NetBox or an external dependency. Further investigation, such as profiling the application or monitoring the container's resource usage in real-time, might be necessary to pinpoint the problem.
Author
Owner

@jeremystretch commented on GitHub (Jan 17, 2025):

I'm converting this to a discussion as no specific unexpected behavior in NetBox has been identified. Please also note that we cannot accept arbitrary scripts as a means for reproducing issues; you would need to provide specific reproduction steps and explicit API calls.

@jeremystretch commented on GitHub (Jan 17, 2025): I'm converting this to a discussion as no specific unexpected behavior in NetBox has been identified. Please also note that we cannot accept arbitrary scripts as a means for reproducing issues; you would need to provide specific reproduction steps and explicit API calls.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#10661