Assigning tags to objects triggers an IntegrityError exception #2681

Closed
opened 2025-12-29 18:21:05 +01:00 by adam · 2 comments
Owner

Originally created by @esapozhnikov-wish on GitHub (Jun 21, 2019).

Environment

  • Python version: 3.6.3
  • NetBox version: 2.6.0

Steps to Reproduce

  1. Edit a device and assign a tag to it. Click save to apply
  2. Edit the device again and set a second tag. Click save to apply
    3.Edit the device again and set a third tag. Click save to apply

Expected Behavior

Expected behavior is for new tag to be appended to the device

Observed Behavior

IntegrityError error page:


Request Method: | POST
-- | --
https://10.10.2.76/dcim/devices/4464/edit/
2.2.2
IntegrityError
duplicate key value violates unique constraint "extras_taggeditem_pkey" DETAIL:  Key (id)=(4) already exists.
/usr/local/lib/python3.6/dist-packages/django_prometheus/db/common.py in execute, line 63
/usr/bin/python3
3.6.3
['/opt/netbox/netbox',  '/opt/netbox-2.6.0/netbox',  '/usr/local/bin',  '/usr/lib/python36.zip',  '/usr/lib/python3.6',  '/usr/lib/python3.6/lib-dynload',  '/usr/local/lib/python3.6/dist-packages',  '/usr/lib/python3/dist-packages']
Fri, 21 Jun 2019 18:30:39 +0000

TRACEBACK:

Environment:


Request Method: POST
Request URL: https://10.10.2.76/dcim/devices/4464/edit/

Django Version: 2.2.2
Python Version: 3.6.3
Installed Applications:
['django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.humanize',
 'cacheops',
 'corsheaders',
 'debug_toolbar',
 'django_filters',
 'django_tables2',
 'django_prometheus',
 'mptt',
 'rest_framework',
 'taggit',
 'taggit_serializer',
 'timezone_field',
 'circuits',
 'dcim',
 'ipam',
 'extras',
 'secrets',
 'tenancy',
 'users',
 'utilities',
 'virtualization',
 'drf_yasg']
Installed Middleware:
('debug_toolbar.middleware.DebugToolbarMiddleware',
 'django_prometheus.middleware.PrometheusBeforeMiddleware',
 'corsheaders.middleware.CorsMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'django.middleware.security.SecurityMiddleware',
 'utilities.middleware.ExceptionHandlingMiddleware',
 'utilities.middleware.LoginRequiredMiddleware',
 'utilities.middleware.APIVersionMiddleware',
 'extras.middleware.ObjectChangeMiddleware',
 'django_prometheus.middleware.PrometheusAfterMiddleware')



Traceback:

File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py" in get_or_create
  538.             return self.get(**kwargs), False

File "/usr/local/lib/python3.6/dist-packages/cacheops/query.py" in get
  356.         return qs._no_monkey.get(qs, *args, **kwargs)

File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py" in get
  408.                 self.model._meta.object_name

During handling of the above exception (TaggedItem matching query does not exist.), another exception occurred:

File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py" in _execute
  84.                 return self.cursor.execute(sql, params)

File "/usr/local/lib/python3.6/dist-packages/django_prometheus/db/common.py" in execute
  63.                 return super(CursorWrapper, self).execute(*args, **kwargs)

The above exception (duplicate key value violates unique constraint "extras_taggeditem_pkey"
DETAIL:  Key (id)=(4) already exists.
) was the direct cause of the following exception:

File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/exception.py" in inner
  34.             response = get_response(request)

File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/base.py" in _get_response
  115.                 response = self.process_exception_by_middleware(e, request)

File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/base.py" in _get_response
  113.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/usr/local/lib/python3.6/dist-packages/django/views/generic/base.py" in view
  71.             return self.dispatch(request, *args, **kwargs)

File "/usr/local/lib/python3.6/dist-packages/django/contrib/auth/mixins.py" in dispatch
  85.         return super().dispatch(request, *args, **kwargs)

File "/usr/local/lib/python3.6/dist-packages/django/views/generic/base.py" in dispatch
  97.         return handler(request, *args, **kwargs)

File "/opt/netbox/netbox/utilities/views.py" in post
  224.             obj = form.save()

File "/opt/netbox/netbox/extras/forms.py" in save
  139.         obj = super().save(commit)

File "/usr/local/lib/python3.6/dist-packages/django/forms/models.py" in save
  459.             self._save_m2m()

File "/usr/local/lib/python3.6/dist-packages/django/forms/models.py" in _save_m2m
  441.                 f.save_form_data(self.instance, cleaned_data[f.name])

File "/usr/local/lib/python3.6/dist-packages/taggit/managers.py" in save_form_data
  518.         getattr(instance, self.name).set(*value)

File "/usr/local/lib/python3.6/dist-packages/taggit/utils.py" in inner
  124.         return func(self, *args, **kwargs)

File "/usr/local/lib/python3.6/dist-packages/taggit/managers.py" in set
  266.             self.add(*new_objs)

File "/usr/local/lib/python3.6/dist-packages/taggit/utils.py" in inner
  124.         return func(self, *args, **kwargs)

File "/usr/local/lib/python3.6/dist-packages/taggit/managers.py" in add
  155.                 tag=tag, **self._lookup_kwargs()

File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py" in get_or_create
  541.             return self._create_object_from_params(kwargs, params)

File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py" in _create_object_from_params
  583.             raise e

File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py" in _create_object_from_params
  575.                 obj = self.create(**params)

File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py" in create
  422.         obj.save(force_insert=True, using=self.db)

File "/usr/local/lib/python3.6/dist-packages/django/db/models/base.py" in save
  741.                        force_update=force_update, update_fields=update_fields)

File "/usr/local/lib/python3.6/dist-packages/django/db/models/base.py" in save_base
  779.                 force_update, using, update_fields,

File "/usr/local/lib/python3.6/dist-packages/django/db/models/base.py" in _save_table
  870.             result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)

File "/usr/local/lib/python3.6/dist-packages/django/db/models/base.py" in _do_insert
  908.                                using=using, raw=raw)

File "/usr/local/lib/python3.6/dist-packages/django/db/models/manager.py" in manager_method
  82.                 return getattr(self.get_queryset(), name)(*args, **kwargs)

File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py" in _insert
  1186.         return query.get_compiler(using=using).execute_sql(return_id)

File "/usr/local/lib/python3.6/dist-packages/django/db/models/sql/compiler.py" in execute_sql
  1335.                 cursor.execute(sql, params)

File "/usr/local/lib/python3.6/dist-packages/debug_toolbar/panels/sql/tracking.py" in execute
  192.         return self._record(self.cursor.execute, sql, params)

File "/usr/local/lib/python3.6/dist-packages/debug_toolbar/panels/sql/tracking.py" in _record
  126.             return method(sql, params)

File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py" in execute
  99.             return super().execute(sql, params)

File "/usr/local/lib/python3.6/dist-packages/cacheops/transaction.py" in execute
  99.         result = self._no_monkey.execute(self, sql, params)

File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py" in execute
  67.         return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)

File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py" in _execute_with_wrappers
  76.         return executor(sql, params, many, context)

File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py" in _execute
  84.                 return self.cursor.execute(sql, params)

File "/usr/local/lib/python3.6/dist-packages/django/db/utils.py" in __exit__
  89.                 raise dj_exc_value.with_traceback(traceback) from exc_value

File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py" in _execute
  84.                 return self.cursor.execute(sql, params)

File "/usr/local/lib/python3.6/dist-packages/django_prometheus/db/common.py" in execute
  63.                 return super(CursorWrapper, self).execute(*args, **kwargs)

Exception Type: IntegrityError at /dcim/devices/4464/edit/
Exception Value: duplicate key value violates unique constraint "extras_taggeditem_pkey"
DETAIL:  Key (id)=(4) already exists.

Originally created by @esapozhnikov-wish on GitHub (Jun 21, 2019). <!-- NOTE: This form is only for reproducible bugs. If you need assistance with NetBox installation, or if you have a general question, DO NOT open an issue. Instead, post to our mailing list: https://groups.google.com/forum/#!forum/netbox-discuss Please describe the environment in which you are running NetBox. Be sure that you are running an unmodified instance of the latest stable release before submitting a bug report. --> ### Environment * Python version: 3.6.3 * NetBox version: 2.6.0 <!-- Describe in detail the exact steps that someone else can take to reproduce this bug using the current stable release of NetBox (or the current beta release where applicable). Begin with the creation of any necessary database objects and call out every operation being performed explicitly. If reporting a bug in the REST API, be sure to reconstruct the raw HTTP request(s) being made: Don't rely on a wrapper like pynetbox. --> ### Steps to Reproduce 1. Edit a device and assign a tag to it. Click save to apply 2. Edit the device again and set a second tag. Click save to apply 3.Edit the device again and set a third tag. Click save to apply <!-- What did you expect to happen? --> ### Expected Behavior Expected behavior is for new tag to be appended to the device <!-- What happened instead? --> ### Observed Behavior IntegrityError error page: ``` Request Method: | POST -- | -- https://10.10.2.76/dcim/devices/4464/edit/ 2.2.2 IntegrityError duplicate key value violates unique constraint "extras_taggeditem_pkey" DETAIL: Key (id)=(4) already exists. /usr/local/lib/python3.6/dist-packages/django_prometheus/db/common.py in execute, line 63 /usr/bin/python3 3.6.3 ['/opt/netbox/netbox', '/opt/netbox-2.6.0/netbox', '/usr/local/bin', '/usr/lib/python36.zip', '/usr/lib/python3.6', '/usr/lib/python3.6/lib-dynload', '/usr/local/lib/python3.6/dist-packages', '/usr/lib/python3/dist-packages'] Fri, 21 Jun 2019 18:30:39 +0000 ``` TRACEBACK: ``` Environment: Request Method: POST Request URL: https://10.10.2.76/dcim/devices/4464/edit/ Django Version: 2.2.2 Python Version: 3.6.3 Installed Applications: ['django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'django.contrib.humanize', 'cacheops', 'corsheaders', 'debug_toolbar', 'django_filters', 'django_tables2', 'django_prometheus', 'mptt', 'rest_framework', 'taggit', 'taggit_serializer', 'timezone_field', 'circuits', 'dcim', 'ipam', 'extras', 'secrets', 'tenancy', 'users', 'utilities', 'virtualization', 'drf_yasg'] Installed Middleware: ('debug_toolbar.middleware.DebugToolbarMiddleware', 'django_prometheus.middleware.PrometheusBeforeMiddleware', 'corsheaders.middleware.CorsMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'utilities.middleware.ExceptionHandlingMiddleware', 'utilities.middleware.LoginRequiredMiddleware', 'utilities.middleware.APIVersionMiddleware', 'extras.middleware.ObjectChangeMiddleware', 'django_prometheus.middleware.PrometheusAfterMiddleware') Traceback: File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py" in get_or_create 538. return self.get(**kwargs), False File "/usr/local/lib/python3.6/dist-packages/cacheops/query.py" in get 356. return qs._no_monkey.get(qs, *args, **kwargs) File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py" in get 408. self.model._meta.object_name During handling of the above exception (TaggedItem matching query does not exist.), another exception occurred: File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py" in _execute 84. return self.cursor.execute(sql, params) File "/usr/local/lib/python3.6/dist-packages/django_prometheus/db/common.py" in execute 63. return super(CursorWrapper, self).execute(*args, **kwargs) The above exception (duplicate key value violates unique constraint "extras_taggeditem_pkey" DETAIL: Key (id)=(4) already exists. ) was the direct cause of the following exception: File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/exception.py" in inner 34. response = get_response(request) File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/base.py" in _get_response 115. response = self.process_exception_by_middleware(e, request) File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/base.py" in _get_response 113. response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/local/lib/python3.6/dist-packages/django/views/generic/base.py" in view 71. return self.dispatch(request, *args, **kwargs) File "/usr/local/lib/python3.6/dist-packages/django/contrib/auth/mixins.py" in dispatch 85. return super().dispatch(request, *args, **kwargs) File "/usr/local/lib/python3.6/dist-packages/django/views/generic/base.py" in dispatch 97. return handler(request, *args, **kwargs) File "/opt/netbox/netbox/utilities/views.py" in post 224. obj = form.save() File "/opt/netbox/netbox/extras/forms.py" in save 139. obj = super().save(commit) File "/usr/local/lib/python3.6/dist-packages/django/forms/models.py" in save 459. self._save_m2m() File "/usr/local/lib/python3.6/dist-packages/django/forms/models.py" in _save_m2m 441. f.save_form_data(self.instance, cleaned_data[f.name]) File "/usr/local/lib/python3.6/dist-packages/taggit/managers.py" in save_form_data 518. getattr(instance, self.name).set(*value) File "/usr/local/lib/python3.6/dist-packages/taggit/utils.py" in inner 124. return func(self, *args, **kwargs) File "/usr/local/lib/python3.6/dist-packages/taggit/managers.py" in set 266. self.add(*new_objs) File "/usr/local/lib/python3.6/dist-packages/taggit/utils.py" in inner 124. return func(self, *args, **kwargs) File "/usr/local/lib/python3.6/dist-packages/taggit/managers.py" in add 155. tag=tag, **self._lookup_kwargs() File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py" in get_or_create 541. return self._create_object_from_params(kwargs, params) File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py" in _create_object_from_params 583. raise e File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py" in _create_object_from_params 575. obj = self.create(**params) File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py" in create 422. obj.save(force_insert=True, using=self.db) File "/usr/local/lib/python3.6/dist-packages/django/db/models/base.py" in save 741. force_update=force_update, update_fields=update_fields) File "/usr/local/lib/python3.6/dist-packages/django/db/models/base.py" in save_base 779. force_update, using, update_fields, File "/usr/local/lib/python3.6/dist-packages/django/db/models/base.py" in _save_table 870. result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "/usr/local/lib/python3.6/dist-packages/django/db/models/base.py" in _do_insert 908. using=using, raw=raw) File "/usr/local/lib/python3.6/dist-packages/django/db/models/manager.py" in manager_method 82. return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py" in _insert 1186. return query.get_compiler(using=using).execute_sql(return_id) File "/usr/local/lib/python3.6/dist-packages/django/db/models/sql/compiler.py" in execute_sql 1335. cursor.execute(sql, params) File "/usr/local/lib/python3.6/dist-packages/debug_toolbar/panels/sql/tracking.py" in execute 192. return self._record(self.cursor.execute, sql, params) File "/usr/local/lib/python3.6/dist-packages/debug_toolbar/panels/sql/tracking.py" in _record 126. return method(sql, params) File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py" in execute 99. return super().execute(sql, params) File "/usr/local/lib/python3.6/dist-packages/cacheops/transaction.py" in execute 99. result = self._no_monkey.execute(self, sql, params) File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py" in execute 67. return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py" in _execute_with_wrappers 76. return executor(sql, params, many, context) File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py" in _execute 84. return self.cursor.execute(sql, params) File "/usr/local/lib/python3.6/dist-packages/django/db/utils.py" in __exit__ 89. raise dj_exc_value.with_traceback(traceback) from exc_value File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py" in _execute 84. return self.cursor.execute(sql, params) File "/usr/local/lib/python3.6/dist-packages/django_prometheus/db/common.py" in execute 63. return super(CursorWrapper, self).execute(*args, **kwargs) Exception Type: IntegrityError at /dcim/devices/4464/edit/ Exception Value: duplicate key value violates unique constraint "extras_taggeditem_pkey" DETAIL: Key (id)=(4) already exists. ```
adam added the type: bugstatus: accepted labels 2025-12-29 18:21:05 +01:00
adam closed this issue 2025-12-29 18:21:05 +01:00
Author
Owner

@jeremystretch commented on GitHub (Jun 21, 2019):

It looks like the sequence was missed in the migration from the built-in Tag model (provided by django-taggit) to our own custom Tag. For example, here I have 23 Tags already defined, but after applying the schema migrations for NetBox v2.6, the database will try to use a PK of 1 for the next Tag to be created:

netbox=> SELECT COUNT(*) FROM extras_tag;
 count 
-------
    23
(1 row)

netbox=> SELECT last_value FROM extras_tag_id_seq;
 last_value 
------------
          1
(1 row)

I'll look into fixing the migration, but for now it's possible to work around by manually resetting the sequence. First, enter the database shell and determine the most recent tag ID:

$ ./manage.py dbshell
psql (9.5.17)
SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off)
Type "help" for help.

netbox=> SELECT id FROM extras_tag ORDER BY id DESC LIMIT 1;
 id 
----
 43
(1 row)

Here, the most recent ID is 43. So, we can "fast forward" the sequence using this number plus one:

netbox=> ALTER SEQUENCE extras_tag_id_seq RESTART with 44;
ALTER SEQUENCE

New tags will be created with unique IDs from this point forward.

Edit: We need to make the same adjustment for the extras_taggeditem_id_seq sequence as well:

netbox=> SELECT id FROM extras_taggeditem ORDER BY id DESC LIMIT 1;
  id   
-------
 24330
(1 row)

netbox=> ALTER SEQUENCE extras_taggeditem_id_seq RESTART with 24331;
ALTER SEQUENCE
@jeremystretch commented on GitHub (Jun 21, 2019): It looks like the sequence was missed in the migration from the built-in Tag model (provided by django-taggit) to our own custom Tag. For example, here I have 23 Tags already defined, but after applying the schema migrations for NetBox v2.6, the database will try to use a PK of 1 for the next Tag to be created: ``` netbox=> SELECT COUNT(*) FROM extras_tag; count ------- 23 (1 row) netbox=> SELECT last_value FROM extras_tag_id_seq; last_value ------------ 1 (1 row) ``` I'll look into fixing the migration, but for now it's possible to work around by manually resetting the sequence. First, enter the database shell and determine the most recent tag ID: ``` $ ./manage.py dbshell psql (9.5.17) SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off) Type "help" for help. netbox=> SELECT id FROM extras_tag ORDER BY id DESC LIMIT 1; id ---- 43 (1 row) ``` Here, the most recent ID is `43`. So, we can "fast forward" the sequence using this number **plus one**: ``` netbox=> ALTER SEQUENCE extras_tag_id_seq RESTART with 44; ALTER SEQUENCE ``` New tags will be created with unique IDs from this point forward. **Edit:** We need to make the same adjustment for the `extras_taggeditem_id_seq` sequence as well: ``` netbox=> SELECT id FROM extras_taggeditem ORDER BY id DESC LIMIT 1; id ------- 24330 (1 row) netbox=> ALTER SEQUENCE extras_taggeditem_id_seq RESTART with 24331; ALTER SEQUENCE ```
Author
Owner

@jeremystretch commented on GitHub (Jun 21, 2019):

I've added a new migration to take care of the sequence adjustment automatically. For anyone wanting to work around it before the v2.6.1 release, here's a copy & paste:

./manage.py dbshell
SELECT setval('extras_tag_id_seq', (SELECT id FROM extras_tag ORDER BY id DESC LIMIT 1) + 1);
SELECT setval('extras_taggeditem_id_seq', (SELECT id FROM extras_taggeditem ORDER BY id DESC LIMIT 1) + 1);

(Running the migration after applying the workaround won't cause any issues.)

@jeremystretch commented on GitHub (Jun 21, 2019): I've added a new migration to take care of the sequence adjustment automatically. For anyone wanting to work around it before the v2.6.1 release, here's a copy & paste: ``` ./manage.py dbshell SELECT setval('extras_tag_id_seq', (SELECT id FROM extras_tag ORDER BY id DESC LIMIT 1) + 1); SELECT setval('extras_taggeditem_id_seq', (SELECT id FROM extras_taggeditem ORDER BY id DESC LIMIT 1) + 1); ``` (Running the migration after applying the workaround won't cause any issues.)
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#2681