Bulk editing IP addresses throws error &#34; This field cannot be null.&#34; #7928

@kkthxbye-code commented on GitHub (Apr 20, 2023):

@morten-starvik - I cannot replicate this either, works with no issues here.

@kkthxbye-code commented on GitHub (Apr 20, 2023): @morten-starvik - I cannot replicate this either, works with no issues here.

adam commented

@jeremystretch commented on GitHub (Apr 21, 2023):

I'm wondering if maybe the existing objects are somehow corrupt. @morten-starvik can you try creating a few new IPs via the UI, and seeing if you can replicate this behavior while editing those?

@jeremystretch commented on GitHub (Apr 21, 2023): I'm wondering if maybe the existing objects are somehow corrupt. @morten-starvik can you try creating a few _new_ IPs via the UI, and seeing if you can replicate this behavior while editing those?

adam commented

@kkthxbye-code commented on GitHub (Apr 21, 2023):

@jeremystretch - Managed to replicate it, it's caused by https://github.com/netbox-community/netbox/pull/11550

Replicated it by setting assigned_object_id = null on an ip address, but leaving assigned_object_type_id as is. I've seen a fair amount of people having these cases, which the PR intends to fix, but if the database is already in that state it will prevent editing the object instead.

Not sure what the correct solution is here. Potential solutions:

Instruct the users how to fix it via. API, nbshell or direct DB editing.
Instead of throwing a validation error, set both id and type id to null if one is missing (could be confusing).
Add a job to the housekeeping script that finds and removes invalid gfk definitions from the database.

@kkthxbye-code commented on GitHub (Apr 21, 2023): @jeremystretch - Managed to replicate it, it's caused by https://github.com/netbox-community/netbox/pull/11550 Replicated it by setting `assigned_object_id` = null on an ip address, but leaving `assigned_object_type_id` as is. I've seen a fair amount of people having these cases, which the PR intends to fix, but if the database is already in that state it will prevent editing the object instead. Not sure what the correct solution is here. Potential solutions: * Instruct the users how to fix it via. API, nbshell or direct DB editing. * Instead of throwing a validation error, set both id and type id to null if one is missing (could be confusing). * Add a job to the housekeeping script that finds and removes invalid gfk definitions from the database.

adam commented

2025-12-29 20:30:06 +01:00

@morten-starvik commented on GitHub (Apr 21, 2023):

I'm wondering if maybe the existing objects are somehow corrupt. @morten-starvik can you try creating a few new IPs via the UI, and seeing if you can replicate this behavior while editing those?

Seems like you're on to something. I created some new IP's, and they could be edited without error. We have quite recently upgraded to v3.4.8, and have not experienced this error before the upgrade. I will check with my colleague who did the upgrade, whether he had any issues during the upgrade

@morten-starvik commented on GitHub (Apr 21, 2023): > I'm wondering if maybe the existing objects are somehow corrupt. @morten-starvik can you try creating a few _new_ IPs via the UI, and seeing if you can replicate this behavior while editing those? Seems like you're on to something. I created some new IP's, and they could be edited without error. We have quite recently upgraded to v3.4.8, and have not experienced this error before the upgrade. I will check with my colleague who did the upgrade, whether he had any issues during the upgrade

adam commented

@jeremystretch commented on GitHub (Apr 21, 2023):

I believe @kkthxbye-code has identified the issue above. These IP addresses likely have partial assignment information. You can check this by inspecting their representation in the REST API: Look for IP addresses with only an assigned_object_type or assigned_object_id set.

@jeremystretch commented on GitHub (Apr 21, 2023): I believe @kkthxbye-code has identified the issue above. These IP addresses likely have partial assignment information. You can check this by inspecting their representation in the REST API: Look for IP addresses with only an `assigned_object_type` or `assigned_object_id` set.

adam commented

2025-12-29 20:30:06 +01:00

@candlerb commented on GitHub (Apr 25, 2023):

Replicated it by setting assigned_object_id = null on an ip address, but leaving assigned_object_type_id as is.

This case certainly does happen in real life (unclear how), but seems to be impossible for the user to clear via the UI.

I suggest that if either assigned_object_type_id or assigned_object_id is None, that as part of validation the other is silently set to None as well: see https://github.com/netbox-community/netbox/issues/10221#issuecomment-1450183759.

It might also be possible to add some sort of database row level validation (i.e. both fields are either null or not-null)

As a stop-gap, a Custom Script could be written which looks for these cases:

>>> IPAddress.objects.filter(assigned_object_type_id__isnull=True, assigned_object_id__isnull=False)
<RestrictedQuerySet []>
>>> IPAddress.objects.filter(assigned_object_type_id__isnull=False, assigned_object_id__isnull=True)
<RestrictedQuerySet [<IPAddress: 10.12.11.59/24>, ..... >

EDIT: I have created the script and added a PR for it here: https://github.com/netbox-community/customizations/pull/85

But I guess really it ought to walk all objects with generic foreign keys.

@candlerb commented on GitHub (Apr 25, 2023): > Replicated it by setting `assigned_object_id = null` on an ip address, but leaving `assigned_object_type_id` as is. This case certainly does happen in real life (unclear how), but seems to be impossible for the user to clear via the UI. I suggest that if either assigned_object_type_id or assigned_object_id is None, that as part of validation the other is silently set to None as well: see https://github.com/netbox-community/netbox/issues/10221#issuecomment-1450183759. It *might* also be possible to add some sort of database row level validation (i.e. both fields are either null or not-null) As a stop-gap, a Custom Script could be written which looks for these cases: ``` >>> IPAddress.objects.filter(assigned_object_type_id__isnull=True, assigned_object_id__isnull=False) <RestrictedQuerySet []> >>> IPAddress.objects.filter(assigned_object_type_id__isnull=False, assigned_object_id__isnull=True) <RestrictedQuerySet [<IPAddress: 10.12.11.59/24>, ..... > ``` EDIT: I have created the script and added a PR for it here: https://github.com/netbox-community/customizations/pull/85 But I guess really it ought to walk *all* objects with generic foreign keys.

adam commented

@candlerb commented on GitHub (Apr 25, 2023):

Looking at my own system: the IPs I have in this state are all from cases which were previously attached to an VM or device interface which no longer exists.

Somehow it was possible to delete the interface without it deleting the attached IP; and the generic relation set the assigned_object_id to null, without setting the assigned_object_type_id to null.

@candlerb commented on GitHub (Apr 25, 2023): Looking at my own system: the IPs I have in this state are all from cases which *were* previously attached to an VM or device interface which no longer exists. Somehow it was possible to delete the interface without it deleting the attached IP; and the generic relation set the assigned_object_id to null, without setting the assigned_object_type_id to null.

adam commented

@morten-starvik commented on GitHub (Apr 26, 2023):

@candlerb, That's the situation in our environment as well; IP addresses in this state have previously been assigned to interfaces

@morten-starvik commented on GitHub (Apr 26, 2023): @candlerb, That's the situation in our environment as well; IP addresses in this state have previously been assigned to interfaces

adam commented

@candlerb commented on GitHub (Apr 26, 2023):

I tried replicating this a couple of ways with 3.4.8 (delete VM, bulk delete VM) but in both cases the IP address attached to the interface was deleted. Either this was triggered some other way, or it is intermittent, or there was a bug which has since been fixed.

@candlerb commented on GitHub (Apr 26, 2023): I tried replicating this a couple of ways with 3.4.8 (delete VM, bulk delete VM) but in both cases the IP address attached to the interface was deleted. Either this was triggered some other way, or it is intermittent, or there was a bug which has since been fixed.

adam commented

@bobbwest commented on GitHub (May 3, 2023):

About half the IP address in my DB have assigned_object_id = null and assigned_object_type_id = 5. None of them have ever been assigned to an interface (django_content_type 5).

I think these all were created on or before 2020-09-09. I'm not sure what NetBox version we were running at that time.

@bobbwest commented on GitHub (May 3, 2023): About half the IP address in my DB have `assigned_object_id = null` and `assigned_object_type_id = 5`. None of them have ever been assigned to an interface (django_content_type 5). I think these all were created on or before 2020-09-09. I'm not sure what NetBox version we were running at that time.

adam commented

@DanSheps commented on GitHub (May 3, 2023):

What object type is object_type_id=5?

@DanSheps commented on GitHub (May 3, 2023): What object type is object_type_id=5?

adam commented

@candlerb commented on GitHub (May 3, 2023):

He said "interface". But on a freshly-installed system it's different:

netbox=# select * from django_content_type where id=5;
 id |  app_label   |    model
----+--------------+-------------
  5 | contenttypes | contenttype
(1 row)

netbox=# select * from django_content_type where model like '%interface%';
 id  |   app_label    |       model
-----+----------------+-------------------
  32 | dcim           | interface
  33 | dcim           | interfacetemplate
 111 | virtualization | vminterface
(3 rows)

adam commented

@DanSheps commented on GitHub (May 3, 2023):

Yeah, I have never seen a ctype id of 5 for anything netbox related. And I have some old instances.

@DanSheps commented on GitHub (May 3, 2023): Yeah, I have never seen a ctype id of 5 for anything netbox related. And I have some old instances.

adam commented

@bobbwest commented on GitHub (May 4, 2023):

netbox=# select * from django_content_type where id=5;
 id | app_label |   model   
----+-----------+-----------
  5 | dcim      | interface
(1 row)

@bobbwest commented on GitHub (May 4, 2023): ``` netbox=# select * from django_content_type where id=5; id | app_label | model ----+-----------+----------- 5 | dcim | interface (1 row) ````

adam commented

@DanSheps commented on GitHub (May 4, 2023):

Well, that is... Broken.

Is it possible that these were attached at some point in the far past? Perhaps even in the 2.x days?

Honestly, for this type of thing an invalid data scrubber might be a good idea. Not just for here but for any GFK.

@DanSheps commented on GitHub (May 4, 2023): Well, that is... Broken. Is it possible that these were attached at some point in the far past? Perhaps even in the 2.x days? Honestly, for this type of thing an invalid data scrubber might be a good idea. Not just for here but for any GFK.

adam commented

@candlerb commented on GitHub (May 4, 2023):

About half the IP address in my DB have assigned_object_id = null and assigned_object_type_id = 5

For cases like these, the scrubber at https://github.com/netbox-community/customizations/pull/85/files will do the job. Drop it in /opt/netbox/netbox/scripts/ and then run it via the GUI at Other > Scripts. Or via SQL:

update ipam_ipaddress set assigned_object_type_id=null where assigned_object_id is null and assigned_object_type_id is not null;

Certainly would be nice to have this for all generic foreign keys, but so far almost all the people who have been hit by this (including me) have only seen it on IPAddress and where one field is null and the other is not null.

A proper scrubber would also check for cases where both fields are not null, but the referenced target object does not exist.

@candlerb commented on GitHub (May 4, 2023): > About half the IP address in my DB have `assigned_object_id = null` and `assigned_object_type_id = 5` For cases like these, the scrubber at https://github.com/netbox-community/customizations/pull/85/files will do the job. Drop it in `/opt/netbox/netbox/scripts/` and then run it via the GUI at Other > Scripts. Or via SQL: ``` update ipam_ipaddress set assigned_object_type_id=null where assigned_object_id is null and assigned_object_type_id is not null; ``` Certainly would be nice to have this for all generic foreign keys, but so far almost all the people who have been hit by this (including me) have only seen it on IPAddress and where one field is null and the other is not null. A proper scrubber would also check for cases where both fields are not null, but the referenced target object does not exist.

adam commented

@jeremystretch commented on GitHub (May 5, 2023):

I believe the root issue here (the introduction of incomplete GFK assignments) has been addressed by #11454 in v3.4.8. Is there any outstanding work to be done at this time?

@jeremystretch commented on GitHub (May 5, 2023): I believe the root issue here (the introduction of incomplete GFK assignments) has been addressed by #11454 in v3.4.8. Is there any outstanding work to be done at this time?

adam commented

@candlerb commented on GitHub (May 6, 2023):

#11550 will raise a ValidationError if one of (ct_field, fk_field) is null, which may prevent this happening in future (although we still don't have a good explanation of how only one of the fields became null in the first place)

The remaining problem I see is if the user is editing an object which is in this state, I don't believe this gives them the ability to fix such objects in the UI.

My suggestion is that in model validation, if one field is null but the other isn't, then quietly set the other to null. Or are model changes not permitted within the validation function?

@candlerb commented on GitHub (May 6, 2023): #11550 will raise a ValidationError if one of (ct_field, fk_field) is null, which may prevent this happening in future (although we still don't have a good explanation of how only one of the fields became null in the first place) The remaining problem I see is if the user is editing an object which is in this state, I don't believe this gives them the ability to *fix* such objects in the UI. My suggestion is that in model validation, if one field is null but the other isn't, then quietly set the other to null. Or are model changes not permitted within the validation function?

adam commented

@bobbwest commented on GitHub (May 6, 2023):

I've managed to reproduce the issue from a fresh install.

Using a fresh install of netbox 2.5.2 / netbox-docker 0.7.0, create just one IP address. The database looks like this:

netbox=# select * from ipam_ipaddress;
 id |  created   |         last_updated         | family |  address   | description | interface_id | nat_inside_id | vrf_id | tenant_id | status | role
 ----+------------+------------------------------+--------+------------+-------------+--------------+---------------+--------+-----------+--------+------
  1 | 2023-05-06 | 2023-05-06 08:28:30.30666+00 |      4 | 1.1.1.1/24 |             |              |               |        |           |      1 |
(1 row)

Upgrade directly to netbox 2.11.12 / netbox-docker 1.2.0 by backing up the original database from 2.5.2, removing the docker database volume, bring up new postgres container and restore from the backup SQL file. Then bring up the other containers so the migrations are run. This results in exactly what my database had:

netbox=# select * from ipam_ipaddress;
 id |  created   |         last_updated         |  address   | description | assigned_object_id | nat_inside_id | vrf_id | tenant_id | status | role | dns_name | assigned_object_type_id | custom_field_data
----+------------+------------------------------+------------+-------------+--------------------+---------------+--------+-----------+--------+------+----------+-------------------------+-------------------
  1 | 2023-05-06 | 2023-05-06 08:28:30.30666+00 | 1.1.1.1/24 |             |                    |               |        |           | active |      |          |                       5 | {}
(1 row)

@bobbwest commented on GitHub (May 6, 2023): I've managed to reproduce the issue from a fresh install. Using a fresh install of netbox 2.5.2 / netbox-docker 0.7.0, create just one IP address. The database looks like this: ``` netbox=# select * from ipam_ipaddress; id | created | last_updated | family | address | description | interface_id | nat_inside_id | vrf_id | tenant_id | status | role ----+------------+------------------------------+--------+------------+-------------+--------------+---------------+--------+-----------+--------+------ 1 | 2023-05-06 | 2023-05-06 08:28:30.30666+00 | 4 | 1.1.1.1/24 | | | | | | 1 | (1 row) ``` Upgrade directly to netbox 2.11.12 / netbox-docker 1.2.0 by backing up the original database from 2.5.2, removing the docker database volume, bring up new postgres container and restore from the backup SQL file. Then bring up the other containers so the migrations are run. This results in exactly what my database had: ``` netbox=# select * from ipam_ipaddress; id | created | last_updated | address | description | assigned_object_id | nat_inside_id | vrf_id | tenant_id | status | role | dns_name | assigned_object_type_id | custom_field_data ----+------------+------------------------------+------------+-------------+--------------------+---------------+--------+-----------+--------+------+----------+-------------------------+------------------- 1 | 2023-05-06 | 2023-05-06 08:28:30.30666+00 | 1.1.1.1/24 | | | | | | active | | | 5 | {} (1 row) ```

adam commented

9cc4992fad/netbox/ipam/migrations/0037_ipaddress_assignment.py (L9-L10)

@bobbwest commented on GitHub (May 6, 2023):

This is the culprit:

Given this is caused by past migrations, it would make sense to correct it automatically, either in a migration or as part of the model.

@bobbwest commented on GitHub (May 6, 2023): This is the culprit: https://github.com/netbox-community/netbox/blob/9cc4992fad2fe04ef0211d998c517414e8871d8c/netbox/ipam/migrations/0037_ipaddress_assignment.py#L9-L10 Given this is caused by past migrations, it would make sense to correct it automatically, either in a migration or as part of the model.

adam commented