starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Interface Bulk Import using Data File CSV format fails due to 1st field skipped #7691

New Issue
Closed
opened 2025-12-29 20:27:02 +01:00 by adam · 6 comments
adam commented 2025-12-29 20:27:02 +01:00
Owner
Copy Link

Originally created by @proy-rbbn on GitHub (Feb 28, 2023).

Originally assigned to: @kkthxbye-code on GitHub.

NetBox version

v3.4.5

Python version

3.8

Steps to Reproduce

  1. Create a csv file for use with Device (or Interface or IP) Bulk Import. Set the device name or other required field as the 1st field in the CSV file (e..g for interface
    device,type,name,enabled
    WOODCREEPER,1000base-t,Management 1,TRUE
  2. Under Interface Bulk Import choose the file you just created and click submit.
  3. Interface bulk import will fail due to the 1st field (device) being skipped in the import.
    Note, if you add an extra field at the beginning of the line, then the import will succeed - but the CSV isn't really valid though because the 1st description line isn't valid... eg.. this format below does work is 3.4.5:
    ,device,type,name,enabled
    ,WOODCREEPER,1000base-t,Management 1,TRUE

Note that CSV import worked in 3.4.5
This errant behavior for CSV import affects Device, Interface and IP bulk file import at least (didn't try others). If I directly past the content into the Data Import "Data" area as CSV then import works successfully.

Expected Behavior

New Interface (or device or IP) will be successfully created.

Observed Behavior

An error indicating Field "device" missing is generated (or whatever the 1st field in the file is).

Originally created by @proy-rbbn on GitHub (Feb 28, 2023). Originally assigned to: @kkthxbye-code on GitHub. ### NetBox version v3.4.5 ### Python version 3.8 ### Steps to Reproduce 1. Create a csv file for use with Device (or Interface or IP) Bulk Import. Set the device name or other required field as the 1st field in the CSV file (e..g for interface device,type,name,enabled WOODCREEPER,1000base-t,Management 1,TRUE 4. Under Interface Bulk Import choose the file you just created and click submit. 5. Interface bulk import will fail due to the 1st field (device) being skipped in the import. Note, if you add an extra field at the beginning of the line, then the import will succeed - but the CSV isn't really valid though because the 1st description line isn't valid... eg.. this format below does work is 3.4.5: ,device,type,name,enabled ,WOODCREEPER,1000base-t,Management 1,TRUE Note that CSV import worked in 3.4.5 This errant behavior for CSV import affects Device, Interface and IP bulk file import at least (didn't try others). If I directly past the content into the Data Import "Data" area as CSV then import works successfully. ### Expected Behavior New Interface (or device or IP) will be successfully created. ### Observed Behavior An error indicating Field "device" missing is generated (or whatever the 1st field in the file is).
adam added the type: bugstatus: accepted labels 2025-12-29 20:27:02 +01:00
adam closed this issue 2025-12-29 20:27:04 +01:00
adam commented 2025-12-29 20:27:08 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Feb 28, 2023):

I've tried following your instructions and the CSV import works as expected: The interface is imported successfully, with all the assigned parameters.

What specific error message are you seeing? Why do you believe the first column is being "skipped?"

For reference, I uploaded a file named csv.test with the following content:

device,type,name,enabled
device3,1000base-t,Management 1,TRUE
@jeremystretch commented on GitHub (Feb 28, 2023): I've tried following your instructions and the CSV import works as expected: The interface is imported successfully, with all the assigned parameters. What specific error message are you seeing? Why do you believe the first column is being "skipped?" For reference, I uploaded a file named `csv.test` with the following content: ``` device,type,name,enabled device3,1000base-t,Management 1,TRUE ```
adam commented 2025-12-29 20:27:09 +01:00
Author
Owner
Copy Link

@bschalk commented on GitHub (Mar 6, 2023):

Good afternoon,

I am observing this same issue. When the csv upload fails (in this example using an import file to add Aggregates) I get the " Record 1 prefix: This field is required." error indicating that it cannot read the first column of information:

image

However, if I just prepend each row with a comma to add a field at the start, the import works just fine. This is NOT the case in the Data Import tab, where you do not need to have a blank column for that to work. I have included both import file examples for comparison.

Aggregate_Bulk_Upload_Test_Fail.csv
Aggregate_Bulk_Upload_Test_Success.csv

Note that the content of the file that fails for the file import works fine for the manual data import, so there does appear to be some sort of bug with the file import process.

@bschalk commented on GitHub (Mar 6, 2023): Good afternoon, I am observing this same issue. When the csv upload fails (in this example using an import file to add Aggregates) I get the " Record 1 prefix: This field is required." error indicating that it cannot read the first column of information: ![image](https://user-images.githubusercontent.com/26799703/223224925-fc7802a9-290d-41f5-b2f3-cf92fe64d5a6.png) However, if I just prepend each row with a comma to add a field at the start, the import works just fine. This is NOT the case in the Data Import tab, where you do not need to have a blank column for that to work. I have included both import file examples for comparison. [Aggregate_Bulk_Upload_Test_Fail.csv](https://github.com/netbox-community/netbox/files/10902599/Aggregate_Bulk_Upload_Test_Fail.csv) [Aggregate_Bulk_Upload_Test_Success.csv](https://github.com/netbox-community/netbox/files/10902600/Aggregate_Bulk_Upload_Test_Success.csv) Note that the content of the file that fails for the file import works fine for the manual data import, so there does appear to be some sort of bug with the file import process.
adam commented 2025-12-29 20:27:09 +01:00
Author
Owner
Copy Link

@kkthxbye-code commented on GitHub (Mar 6, 2023):

There's a BOM at the start of your file which causes the issue. I guess excel adds it in some cases, can't be sure as I don't have it.

I guess we could fix it by changing the following line to:

ee5b707e68/netbox/utilities/forms/forms.py (L183)

data = file.read().decode('utf-8-sig')

@kkthxbye-code commented on GitHub (Mar 6, 2023): There's a [BOM](https://en.wikipedia.org/wiki/Byte_order_mark) at the start of your file which causes the issue. I guess excel adds it in some cases, can't be sure as I don't have it. I guess we could fix it by changing the following line to: https://github.com/netbox-community/netbox/blob/ee5b707e688f62e3b08242fe2ea39e26b42d1534/netbox/utilities/forms/forms.py#L183 `data = file.read().decode('utf-8-sig')`
adam commented 2025-12-29 20:27:09 +01:00
Author
Owner
Copy Link

@bschalk commented on GitHub (Mar 6, 2023):

I can confirm that if I build the file in notepad and save as a .csv the problem does not exist, so I think you are absolutely spot on with your analysis. However, I do think that for bulk adds, many folks would expect to be able to use Excel, so the proposed fix is likely warranted. Thank you very much for the quick response, this was incredible!

@bschalk commented on GitHub (Mar 6, 2023): I can confirm that if I build the file in notepad and save as a .csv the problem does not exist, so I think you are absolutely spot on with your analysis. However, I do think that for bulk adds, many folks would expect to be able to use Excel, so the proposed fix is likely warranted. Thank you very much for the quick response, this was incredible!
adam commented 2025-12-29 20:27:09 +01:00
Author
Owner
Copy Link

@proy-rbbn commented on GitHub (Mar 6, 2023):

This makes sense to me as a root cause of my issue as well. Agreed that my team wants to use Excel to generate and bulk change inputs - so I would pick this fix up once available right away!

@proy-rbbn commented on GitHub (Mar 6, 2023): This makes sense to me as a root cause of my issue as well. Agreed that my team wants to use Excel to generate and bulk change inputs - so I would pick this fix up once available right away!
adam commented 2025-12-29 20:27:10 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Mar 6, 2023):

Nice catch @kkthxbye-code! Seems like a reasonable fix.

@jeremystretch commented on GitHub (Mar 6, 2023): Nice catch @kkthxbye-code! Seems like a reasonable fix.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label status: accepted type: bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#7691
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?