Datasource Sync Stuck in "Queued" State #8293

Closed
opened 2025-12-29 20:34:59 +01:00 by adam · 2 comments
Owner

Originally created by @Moehritz on GitHub (Jul 5, 2023).

NetBox version

v3.5.4 463a388ac674

Python version

3.8

Steps to Reproduce

  1. Create A new Datasource (Git, Filesystem or S3 does not matter)
  2. Sync it
  3. Wait

Our Setup:
Hosted with docker, compose file based on https://github.com/netbox-community/netbox-docker/blob/release/docker-compose.yml
(with minor changes to healthchecks, volumes, but services are defined the same)

I can reproduce it in our setup easily, but in the official netbox demo it works. I am writing this bug report as our setup is fairly standard (eg no plugins) and there may be some edge-case which leads to this status, we can provide extra details as necessary.

Expected Behavior

The datasource syncs and shows its files.

Observed Behavior

Error Message reported in https://netbox-url/admin/background-tasks/queues/0/adb2e5d1-d386-47b6-bf51-b751c11481f8/

Traceback (most recent call last):
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/apps/registry.py", line 158, in get_app_config
    return self.app_configs[app_label]
KeyError: 'core'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/netbox/venv/lib/python3.10/site-packages/rq/job.py", line 417, in _deserialize_data
    self._func_name, self._instance, self._args, self._kwargs = self.serializer.loads(self.data)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 2477, in model_unpickle
    model = apps.get_model(*model_id)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/apps/registry.py", line 208, in get_model
    app_config = self.get_app_config(app_label)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/apps/registry.py", line 165, in get_app_config
    raise LookupError(message)
LookupError: No installed app with label 'core'.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/netbox/venv/lib/python3.10/site-packages/rq/worker.py", line 1353, in perform_job
    self.prepare_job_execution(job)
  File "/opt/netbox/venv/lib/python3.10/site-packages/rq/worker.py", line 1203, in prepare_job_execution
    self.procline(msg.format(job.func_name, job.origin, time.time()))
  File "/opt/netbox/venv/lib/python3.10/site-packages/rq/job.py", line 451, in func_name
    self._deserialize_data()
  File "/opt/netbox/venv/lib/python3.10/site-packages/rq/job.py", line 419, in _deserialize_data
    raise DeserializationError() from e
rq.exceptions.DeserializationError

The same error can be observed with docker-compse logs netbox-worker.

image
image

Originally created by @Moehritz on GitHub (Jul 5, 2023). ### NetBox version v3.5.4 463a388ac674 ### Python version 3.8 ### Steps to Reproduce 1. Create A new Datasource (Git, Filesystem or S3 does not matter) 2. Sync it 3. Wait Our Setup: Hosted with docker, compose file based on https://github.com/netbox-community/netbox-docker/blob/release/docker-compose.yml (with minor changes to healthchecks, volumes, but services are defined the same) I can reproduce it in our setup easily, but in the official netbox demo it works. I am writing this bug report as our setup is fairly standard (eg no plugins) and there may be some edge-case which leads to this status, we can provide extra details as necessary. ### Expected Behavior The datasource syncs and shows its files. ### Observed Behavior Error Message reported in `https://netbox-url/admin/background-tasks/queues/0/adb2e5d1-d386-47b6-bf51-b751c11481f8/` ``` Traceback (most recent call last): File "/opt/netbox/venv/lib/python3.10/site-packages/django/apps/registry.py", line 158, in get_app_config return self.app_configs[app_label] KeyError: 'core' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/netbox/venv/lib/python3.10/site-packages/rq/job.py", line 417, in _deserialize_data self._func_name, self._instance, self._args, self._kwargs = self.serializer.loads(self.data) File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 2477, in model_unpickle model = apps.get_model(*model_id) File "/opt/netbox/venv/lib/python3.10/site-packages/django/apps/registry.py", line 208, in get_model app_config = self.get_app_config(app_label) File "/opt/netbox/venv/lib/python3.10/site-packages/django/apps/registry.py", line 165, in get_app_config raise LookupError(message) LookupError: No installed app with label 'core'. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/opt/netbox/venv/lib/python3.10/site-packages/rq/worker.py", line 1353, in perform_job self.prepare_job_execution(job) File "/opt/netbox/venv/lib/python3.10/site-packages/rq/worker.py", line 1203, in prepare_job_execution self.procline(msg.format(job.func_name, job.origin, time.time())) File "/opt/netbox/venv/lib/python3.10/site-packages/rq/job.py", line 451, in func_name self._deserialize_data() File "/opt/netbox/venv/lib/python3.10/site-packages/rq/job.py", line 419, in _deserialize_data raise DeserializationError() from e rq.exceptions.DeserializationError ``` The same error can be observed with `docker-compse logs netbox-worker`. ![image](https://github.com/netbox-community/netbox/assets/1659422/26896578-d2e5-4d61-b74a-f3f1e00b28a5) ![image](https://github.com/netbox-community/netbox/assets/1659422/b966403c-addc-4df1-ac82-e93092502361)
adam added the type: bug label 2025-12-29 20:34:59 +01:00
adam closed this issue 2025-12-29 20:34:59 +01:00
Author
Owner

@kkthxbye-code commented on GitHub (Jul 5, 2023):

Looks like your netbox-worker might be running an older image than your netbox container.

@kkthxbye-code commented on GitHub (Jul 5, 2023): Looks like your netbox-worker might be running an older image than your netbox container.
Author
Owner

@Moehritz commented on GitHub (Jul 5, 2023):

Well you seem to be a genius. I feared it was something this "trivial" that I just did not see despite looking at everything multiple times. Thank you very much!

It seems the anchors in docker-compose.yml are expanded before the overrides are applied, so the image version only got passed to the netbox image, with the others being stuck.

So basically we had the image version set in docker-compose.override.yml

Thank you :)

@Moehritz commented on GitHub (Jul 5, 2023): Well you seem to be a genius. I feared it was something this "trivial" that I just did not see despite looking at everything multiple times. Thank you very much! It seems the anchors in docker-compose.yml are expanded before the overrides are applied, so the image version only got passed to the netbox image, with the others being stuck. So basically we had the image version set in [docker-compose.override.yml](https://github.com/netbox-community/netbox-docker/blob/release/docker-compose.override.yml.example) Thank you :)
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#8293