feature: Allow 'ignore duplicates' on import #1108

New Issue

adam · 2025-12-29T16:29:04+01:00

adam commented

2025-12-29 16:29:04 +01:00

Originally created by @InsaneSplash on GitHub (Jul 17, 2017).

Issue type: feature

Python version:
NetBox version:2.0.10

I would like to for a feature to be considered when importing bulk data in to Netbox. In some cases, the information may have been manually inputted to Netbox, causing duplicates when trying to perform an import. It would be helpful to add an option to ignore duplicates and import the unique values. This way one does not need to try count through import lines to remove the duplicate entries.

Originally created by @InsaneSplash on GitHub (Jul 17, 2017). ### Issue type: feature **Python version:** **NetBox version:2.0.10** I would like to for a feature to be considered when importing bulk data in to Netbox. In some cases, the information may have been manually inputted to Netbox, causing duplicates when trying to perform an import. It would be helpful to add an option to ignore duplicates and import the unique values. This way one does not need to try count through import lines to remove the duplicate entries.

adam closed this issue

2025-12-29 16:29:04 +01:00

adam commented

2025-12-29 16:29:05 +01:00

@jeremystretch commented on GitHub (Jul 17, 2017):

The onus is on the user to provide correct data to NetBox. If duplicate entries exist in the list of objects to be imported, the user will need to determine where they came from and whether it's symptomatic of a larger issue (e.g. objects exported from another source were not formatted correctly) to be confident that the data they're importing is correct. Simply ignoring duplicate entries will likely result in erroneous and/or missing data.

@jeremystretch commented on GitHub (Jul 17, 2017): The onus is on the user to provide correct data to NetBox. If duplicate entries exist in the list of objects to be imported, the user will need to determine where they came from and whether it's symptomatic of a larger issue (e.g. objects exported from another source were not formatted correctly) to be confident that the data they're importing is correct. Simply ignoring duplicate entries will likely result in erroneous and/or missing data.

adam commented

2025-12-29 16:29:05 +01:00

@InsaneSplash commented on GitHub (Jul 17, 2017):

The issue isn't really duplicate data within the import source, but rather having the ability to merge the data with data that is already existing within Netbox. In many cases one cannot wait for an import to take place from a legacy system and some data is added before hand. It would help to allow a user to select whether to ignore any duplicates between the 2 sources.

Its not an easy task importing a CSV and having to remove the duplicate entries through line counting etc.

Example: I have 2000 VRFs to import, some have already been created so that IP Prefixes could be linked to a VRF/Device.

@InsaneSplash commented on GitHub (Jul 17, 2017): The issue isn't really duplicate data within the import source, but rather having the ability to merge the data with data that is already existing within Netbox. In many cases one cannot wait for an import to take place from a legacy system and some data is added before hand. It would help to allow a user to select whether to ignore any duplicates between the 2 sources. Its not an easy task importing a CSV and having to remove the duplicate entries through line counting etc. Example: I have 2000 VRFs to import, some have already been created so that IP Prefixes could be linked to a VRF/Device.

adam commented

2025-12-29 16:29:05 +01:00

@jeremystretch commented on GitHub (Jul 17, 2017):

Its not an easy task importing a CSV and having to remove the duplicate entries through line counting etc.

Just take what's already defined in NetBox (via the API or CSV export, whichever is easier) and compare it to the set to be imported, then only import the difference between the two.

for vrf in legacy_vrfs:
    if vrf not in netbox_vrfs:
        print vrf

@jeremystretch commented on GitHub (Jul 17, 2017): > Its not an easy task importing a CSV and having to remove the duplicate entries through line counting etc. Just take what's already defined in NetBox (via the API or CSV export, whichever is easier) and compare it to the set to be imported, then only import the difference between the two. ```python for vrf in legacy_vrfs: if vrf not in netbox_vrfs: print vrf ```

Sign in to join this conversation.

Branches Tags

main

update-changelog-comments-docs

feature-removal-issue-type

20911-dropdown

20239-plugin-menu-classes-mutable-state

21097-graphql-id-lookups

feature

fix_module_substitution

20923-dcim-templates

20044-elevation-stuck-lightmode

feature-ip-prefix-link

v4.5-beta1-release

20068-import-moduletype-attrs

20766-fix-german-translation-code-literals

20378-del-script

7604-filter-modifiers-v3

circuit-swap

12318-case-insensitive-uniqueness

20637-improve-device-q-filter

20660-script-load

19724-graphql

20614-update-ruff

14884-script

02496-max-page

19720-macaddress-interface-generic-relation

19408-circuit-terminations-export-templates

20203-openapi-check

fix-19669-api-image-download

7604-filter-modifiers

19275-fixes-interface-bulk-edit

fix-17794-get_field_value_return_list

11507-show-aggregate-and-rir-on-api

9583-add_column_specific_search_field_to_tables

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/netbox#1108