Django 1.10 prevents by default deletion of large numbers of records #374

Closed
opened 2025-12-29 16:21:26 +01:00 by adam · 3 comments
Owner

Originally created by @ingeba on GitHub (Aug 18, 2016).

After pulling Netbox yesterday, an upgrade was made to Django 1.10. After this it was no longer possible to do a bulk delete of ~2500 prefixes (I got "Internal server error"). The Netbox log error was:

TooManyFieldsSent: The number of GET/POST parameters exceeded settings.DATA_UPLOAD_MAX_NUMBER_FIELDS.
[2016-08-17 13:15:54 +0000] [15415] [ERROR] Error handling request /ipam/prefixes/delete/

and a traceback was generated. According to the Django docs, the number of HTTP fields are on 1.10 limited by default to 1000. I resolved the issue by adding to netbox/netbox/settings.py:

DATA_UPLOAD_MAX_NUMBER_FIELDS = None

Originally created by @ingeba on GitHub (Aug 18, 2016). After pulling Netbox yesterday, an upgrade was made to Django 1.10. After this it was no longer possible to do a bulk delete of ~2500 prefixes (I got "Internal server error"). The Netbox log error was: TooManyFieldsSent: The number of GET/POST parameters exceeded settings.DATA_UPLOAD_MAX_NUMBER_FIELDS. [2016-08-17 13:15:54 +0000] [15415] [ERROR] Error handling request /ipam/prefixes/delete/ and a traceback was generated. According to the Django docs, the number of HTTP fields are on 1.10 limited by default to 1000. I resolved the issue by adding to netbox/netbox/settings.py: DATA_UPLOAD_MAX_NUMBER_FIELDS = None
adam added the type: bug label 2025-12-29 16:21:26 +01:00
adam closed this issue 2025-12-29 16:21:26 +01:00
Author
Owner

@jeremystretch commented on GitHub (Aug 18, 2016):

How were you performing the bulk deletion? Deleting objects using the "select all" box should only send one field with a list of objects IDs. Otherwise, the number of fields sent should be limited to the maximum page size IIRC.

@jeremystretch commented on GitHub (Aug 18, 2016): How were you performing the bulk deletion? Deleting objects using the "select all" box should only send one field with a list of objects IDs. Otherwise, the number of fields sent should be limited to the maximum page size IIRC.
Author
Owner

@ingeba commented on GitHub (Aug 18, 2016):

I used the check box over the list - "Name" in "Devices", "Prefix" in "Prefixes" etc. It then prompts me with a list of objects to delete and asks me to confirm. After the confirmation it gives the error I cited above. Deleting 500-600 items at the time worked well, but not > 1000. After changing the Django config, deleting work regardless of the number of items.

EDIT: Note that I have done deletion of >1000 items at a time several times before (prior to the Django upgrade to 1.10) without problems.

@ingeba commented on GitHub (Aug 18, 2016): I used the check box over the list - "Name" in "Devices", "Prefix" in "Prefixes" etc. It then prompts me with a list of objects to delete and asks me to confirm. After the confirmation it gives the error I cited above. Deleting 500-600 items at the time worked well, but not > 1000. After changing the Django config, deleting work regardless of the number of items. EDIT: Note that I have done deletion of >1000 items at a time several times before (prior to the Django upgrade to 1.10) without problems.
Author
Owner

@jeremystretch commented on GitHub (Aug 18, 2016):

Right, ok. On the second submission (post-confirmation), each object ID is passed independently.

I suppose I don't really see a reason for DATA_UPLOAD_MAX_NUMBER_FIELDS as far as NetBox is concerned. I'm going to disable it in settings.py.

@jeremystretch commented on GitHub (Aug 18, 2016): Right, ok. On the second submission (post-confirmation), each object ID is passed independently. I suppose I don't really see a reason for `DATA_UPLOAD_MAX_NUMBER_FIELDS` as far as NetBox is concerned. I'm going to disable it in `settings.py`.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#374