starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Error running parallel tasks (same output as closed issue #5142) #4584

New Issue
Closed
opened 2025-12-29 18:37:56 +01:00 by adam · 2 comments
adam commented 2025-12-29 18:37:56 +01:00
Owner
Copy Link

Originally created by @chet-space on GitHub (Feb 23, 2021).

Environment

  • Python version: 3.6.9
  • NetBox version: 2.9.9

When running multiple tasks in parallel Netbox generates an error. The error is the same as reported in issue #5142 but I'm able to consistently reproduce this one. I'm using concurrent.futures to spawn multiple threads. I was also seeing the issue when a used a bash loop instead of concurrent.futures - I can share more info on that if needed.

Once data has started to write to Netbox the error will occur. It will happen using as low as 2 or as many as 10 threads. On each run the error will occur on different nodes, at different times during the process.

Set number of threads using argparse

if args.threads: workers = int(args.threads)

Concurrently gathers data from nodes using Napalm/Netmiko and writes data to a dict

with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as fact_executor: futures_to_getFacts = {fact_executor.submit(get_facts, n): n for n in device_inventory} for future in concurrent.futures.as_completed(futures_to_getFacts,timeout=None): nodes.append(future.result())

Takes gathered data from previous code and calls the core function to parse and write to Netbox.

with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as core_executor: futures_to_core = {core_executor.submit(core, n): n for n in nodes} for future in concurrent.futures.as_completed(futures_to_core,timeout=None): if future.exception() is None: pass else: print(future.exception())

{"error": "Cannot assign "<SimpleLazyObject: <django.contrib.auth.models.AnonymousUser object at 0x7fcfa77490b8>>": "ObjectChange.user" must be a "User" instance.", "exception": "V
alueError", "netbox_version": "2.9.9", "python_version": "3.6.9"}

{"error": "Cannot assign "<SimpleLazyObject: <django.contrib.auth.models.AnonymousUser object at 0x7fcfac1c0198>>": "ObjectChange.user" must be a "User" instance.", "exception": "V
alueError", "netbox_version": "2.9.9", "python_version": "3.6.9"}

{"error": "Cannot assign "<SimpleLazyObject: <django.contrib.auth.models.AnonymousUser object at 0x7fcfac511f28>>": "ObjectChange.user" must be a "User" instance.", "exception": "V
alueError", "netbox_version": "2.9.9", "python_version": "3.6.9"}

The request failed with code 500 Internal Server Error: {'error': 'Cannot assign "<SimpleLazyObject: <django.contrib.auth.models.AnonymousUser object at 0x7fcfa679c208>>": "ObjectChange.
user" must be a "User" instance.', 'exception': 'ValueError', 'netbox_version': '2.9.9', 'python_version': '3.6.9'}

{"error": "Cannot assign "<SimpleLazyObject: <django.contrib.auth.models.AnonymousUser object at 0x7fcfac19e7f0>>": "ObjectChange.user" must be a "User" instance.", "exception": "V
alueError", "netbox_version": "2.9.9", "python_version": "3.6.9"}

Originally created by @chet-space on GitHub (Feb 23, 2021). ### Environment * Python version: 3.6.9 * NetBox version: 2.9.9 When running multiple tasks in parallel Netbox generates an error. The error is the same as reported in issue #5142 but I'm able to consistently reproduce this one. I'm using concurrent.futures to spawn multiple threads. I was also seeing the issue when a used a bash loop instead of concurrent.futures - I can share more info on that if needed. Once data has started to write to Netbox the error will occur. It will happen using as low as 2 or as many as 10 threads. On each run the error will occur on different nodes, at different times during the process. ## Set number of threads using argparse ` if args.threads: workers = int(args.threads) ` ## Concurrently gathers data from nodes using Napalm/Netmiko and writes data to a dict ` with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as fact_executor: futures_to_getFacts = {fact_executor.submit(get_facts, n): n for n in device_inventory} for future in concurrent.futures.as_completed(futures_to_getFacts,timeout=None): nodes.append(future.result()) ` ## Takes gathered data from previous code and calls the core function to parse and write to Netbox. ` with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as core_executor: futures_to_core = {core_executor.submit(core, n): n for n in nodes} for future in concurrent.futures.as_completed(futures_to_core,timeout=None): if future.exception() is None: pass else: print(future.exception()) ` {"error": "Cannot assign \"<SimpleLazyObject: <django.contrib.auth.models.AnonymousUser object at 0x7fcfa77490b8>>\": \"ObjectChange.user\" must be a \"User\" instance.", "exception": "V alueError", "netbox_version": "2.9.9", "python_version": "3.6.9"} {"error": "Cannot assign \"<SimpleLazyObject: <django.contrib.auth.models.AnonymousUser object at 0x7fcfac1c0198>>\": \"ObjectChange.user\" must be a \"User\" instance.", "exception": "V alueError", "netbox_version": "2.9.9", "python_version": "3.6.9"} {"error": "Cannot assign \"<SimpleLazyObject: <django.contrib.auth.models.AnonymousUser object at 0x7fcfac511f28>>\": \"ObjectChange.user\" must be a \"User\" instance.", "exception": "V alueError", "netbox_version": "2.9.9", "python_version": "3.6.9"} The request failed with code 500 Internal Server Error: {'error': 'Cannot assign "<SimpleLazyObject: <django.contrib.auth.models.AnonymousUser object at 0x7fcfa679c208>>": "ObjectChange. user" must be a "User" instance.', 'exception': 'ValueError', 'netbox_version': '2.9.9', 'python_version': '3.6.9'} {"error": "Cannot assign \"<SimpleLazyObject: <django.contrib.auth.models.AnonymousUser object at 0x7fcfac19e7f0>>\": \"ObjectChange.user\" must be a \"User\" instance.", "exception": "V alueError", "netbox_version": "2.9.9", "python_version": "3.6.9"}
adam closed this issue 2025-12-29 18:37:56 +01:00
adam commented 2025-12-29 18:37:57 +01:00
Author
Owner
Copy Link

@chet-space commented on GitHub (Feb 23, 2021):

This was resolved by changing threads = 1 in gunicorn.py.

@chet-space commented on GitHub (Feb 23, 2021): This was resolved by changing `threads = 1` in gunicorn.py.
adam commented 2025-12-29 18:37:57 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Feb 23, 2021):

@chet-space thank you for taking the time to note your solution!

@jeremystretch commented on GitHub (Feb 23, 2021): @chet-space thank you for taking the time to note your solution!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#4584
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?