starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

[PR #13677] [CLOSED] Add support for TSV and semicolon-seperated CSV #14238

New Issue
Closed
opened 2025-12-29 23:23:20 +01:00 by adam · 0 comments
adam commented 2025-12-29 23:23:20 +01:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/netbox-community/netbox/pull/13677
Author: @pv2b
Created: 9/5/2023
Status: ❌ Closed

Base: develop ← Head: 13239-csv-alternate-seperators-pv2b

📝 Commits (8)

  • 350e79e Extract _test_valid_tags function
  • 8668e5e Add tests for CSV autodetect, as well as cases for CSV with semicolon and TSV
  • 15c3651 Add choices for CSV-semicolon and TSV
  • df3a5cd Add support for CSV-semicolon and TSV
  • c554443 Fix detection of YAML and JSON files with leading whitespace
  • 369fe57 Fix help text to reflect support for TSV imports.
  • 236c178 Fix missing .strip
  • 31d3dcf Fix typo clean_tsv -> clean_csv

📊 Changes

3 files changed (+93 additions, -18 deletions)

View changed files

📝 netbox/netbox/tests/test_import.py (+68 -11)
📝 netbox/utilities/choices.py (+4 -0)
📝 netbox/utilities/forms/bulk_import.py (+21 -7)

📄 Description

Fixes: #13239

This is an alternative PR for issue #13239. A different PR #13563 exists that attempts to solve the same issue, however I think my approach is better.

  1. The other approach creates a new field for CSV seperators. This makes the UI a lot messier and adds more clicks to importing CSV files for the user. In this solution, instead, semicolon-seperated CSV and tab-seperated CSV are added as new formats and the user only has one drop-down to make a selection in.
  2. There was some confusion in the code regarding auto-detection of file formats in general and autodetecting different types of CSV file specifically. @abhi1693 was attempting to use csv.Sniff to detect a specific CSV file format, but was running into some problems that I don't understand with making it work with tab-seperated files. Anyway, this approach completely sidesteps csv.Sniff, just adding a couple more cases into _detect_format.
  3. The other approach wasn't complete, for example, it was not able to automatically detect tab- or semicolon-seperated CSV files when set to automatic mode. This I think is also due there being two ways of detecting CSV files.

There's one area of concern for breakage here. I've had to disable Django's own built-in stripping of whitespace, because with TSV files, whitespace is significant, and removing trailing tabs can remove an empty column, which isn't what you want. I think I've anticipated every case where this could break something (for example, _detect_format expects the file to be stripped, so I had to add some stripping back into that function) but this could probably use a second pair of eyes to make sure.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/netbox-community/netbox/pull/13677 **Author:** [@pv2b](https://github.com/pv2b) **Created:** 9/5/2023 **Status:** ❌ Closed **Base:** `develop` ← **Head:** `13239-csv-alternate-seperators-pv2b` --- ### 📝 Commits (8) - [`350e79e`](https://github.com/netbox-community/netbox/commit/350e79e61465a09b818dde8925fec93eeb647752) Extract _test_valid_tags function - [`8668e5e`](https://github.com/netbox-community/netbox/commit/8668e5eded4af31f3d5e9ed092fad5057f41c11b) Add tests for CSV autodetect, as well as cases for CSV with semicolon and TSV - [`15c3651`](https://github.com/netbox-community/netbox/commit/15c36514e66ee754f08db9d25d26ccd7ebecb077) Add choices for CSV-semicolon and TSV - [`df3a5cd`](https://github.com/netbox-community/netbox/commit/df3a5cd422c0afa957337e7a6330b062d7914e6a) Add support for CSV-semicolon and TSV - [`c554443`](https://github.com/netbox-community/netbox/commit/c554443a6a724dd2d0f445a6c0eab26037d46f4a) Fix detection of YAML and JSON files with leading whitespace - [`369fe57`](https://github.com/netbox-community/netbox/commit/369fe57ba0ada754e80ab71f418191909b6d694e) Fix help text to reflect support for TSV imports. - [`236c178`](https://github.com/netbox-community/netbox/commit/236c178f993c97c819f461ad7cd89e41eb48a350) Fix missing .strip - [`31d3dcf`](https://github.com/netbox-community/netbox/commit/31d3dcffdf1a07f91092bba691df8f738ddb1e24) Fix typo clean_tsv -> clean_csv ### 📊 Changes **3 files changed** (+93 additions, -18 deletions) <details> <summary>View changed files</summary> 📝 `netbox/netbox/tests/test_import.py` (+68 -11) 📝 `netbox/utilities/choices.py` (+4 -0) 📝 `netbox/utilities/forms/bulk_import.py` (+21 -7) </details> ### 📄 Description <!-- Thank you for your interest in contributing to NetBox! Please note that our contribution policy requires that a feature request or bug report be approved and assigned prior to opening a pull request. This helps avoid waste time and effort on a proposed change that we might not be able to accept. IF YOUR PULL REQUEST DOES NOT REFERENCE AN ISSUE WHICH HAS BEEN ASSIGNED TO YOU, IT WILL BE CLOSED AUTOMATICALLY. Please specify your assigned issue number on the line below. --> ### Fixes: #13239 <!-- Please include a summary of the proposed changes below. --> This is an alternative PR for issue #13239. A different PR #13563 exists that attempts to solve the same issue, however I think my approach is better. 1. The other approach creates a new field for CSV seperators. This makes the UI a lot messier and adds more clicks to importing CSV files for the user. In this solution, instead, semicolon-seperated CSV and tab-seperated CSV are added as new formats and the user only has one drop-down to make a selection in. 2. There was some confusion in the code regarding auto-detection of file formats in general and autodetecting different types of CSV file specifically. @abhi1693 was attempting to use csv.Sniff to detect a specific CSV file format, but was running into some problems that I don't understand with making it work with tab-seperated files. Anyway, this approach completely sidesteps csv.Sniff, just adding a couple more cases into _detect_format. 3. The other approach wasn't complete, for example, it was not able to automatically detect tab- or semicolon-seperated CSV files when set to automatic mode. This I think is also due there being two ways of detecting CSV files. There's one area of concern for breakage here. I've had to disable Django's own built-in stripping of whitespace, because with TSV files, whitespace is significant, and removing trailing tabs can remove an empty column, which isn't what you want. I think I've anticipated every case where this could break something (for example, _detect_format expects the file to be stripped, so I had to add some stripping back into that function) but this could probably use a second pair of eyes to make sure. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
adam added the pull-request label 2025-12-29 23:23:20 +01:00
adam closed this issue 2025-12-29 23:23:21 +01:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#14238
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?