starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Implement a mechanism to synchronize data from external sources #7544

New Issue
Closed
opened 2025-12-29 20:24:59 +01:00 by adam · 2 comments
adam commented 2025-12-29 20:24:59 +01:00
Owner
Copy Link

Originally created by @jeremystretch on GitHub (Jan 21, 2023).

Originally assigned to: @jeremystretch on GitHub.

NetBox version

v3.4.3

Feature type

New functionality

Proposed functionality

Introduce the ability for NetBox to pull and store in the database arbitrary data from external sources. Specifically, we would want to support HTTP and FTP, remote git repositories, and local directories (which should still be considered "external" to the database). Files from within each data source would be replicated to database objects automatically at configured intervals and/or via designated triggers.

Once replicated to NetBox, the data will be directly accessible via both the UI and APIs. However, this functionality should be considered secondary to the employment of the data for other NetBox functions as described below.

Use case

There are cases where allowing a user to specify an external source of data (rather than storing it natively within NetBox) grants an additional degree of flexibility. For example, I opened #9073 to explore this idea specifically for config contexts, but it could be applied to export templates and likely other models as well.

Database changes

This proposal entails two new models, outlined below.

DataSource

  • name - User-configured name
  • type - Local, HTTP, FTP, git, etc.
  • description
  • url - Root path to the remote source
  • Authentication credentials (TBD)

DataFile

  • source - FK to DataSource
  • path - String path to the file relative to the source's root
  • mime_type - e.g. application/json
  • last_updated
  • size - File size
  • checksum - SHA256 checksum

External dependencies

Depending on specific implementation decisions, we might opt to use GitPython for replicating git repositories. However we might also get by just using the standard requests library.

Originally created by @jeremystretch on GitHub (Jan 21, 2023). Originally assigned to: @jeremystretch on GitHub. ### NetBox version v3.4.3 ### Feature type New functionality ### Proposed functionality Introduce the ability for NetBox to pull and store in the database arbitrary data from external sources. Specifically, we would want to support HTTP and FTP, remote git repositories, and local directories (which should still be considered "external" to the database). Files from within each data source would be replicated to database objects automatically at configured intervals and/or via designated triggers. Once replicated to NetBox, the data will be directly accessible via both the UI and APIs. However, this functionality should be considered secondary to the employment of the data for other NetBox functions as described below. ### Use case There are cases where allowing a user to specify an external source of data (rather than storing it natively within NetBox) grants an additional degree of flexibility. For example, I opened #9073 to explore this idea specifically for [config contexts](https://docs.netbox.dev/en/stable/features/context-data/), but it could be applied to export templates and likely other models as well. ### Database changes This proposal entails two new models, outlined below. #### DataSource - `name` - User-configured name - `type` - Local, HTTP, FTP, git, etc. - `description` - `url` - Root path to the remote source - Authentication credentials (TBD) #### DataFile - `source` - FK to `DataSource` - `path` - String path to the file relative to the source's root - `mime_type` - e.g. `application/json` - `last_updated` - `size` - File size - `checksum` - SHA256 checksum ### External dependencies Depending on specific implementation decisions, we might opt to use [GitPython](https://github.com/gitpython-developers/GitPython) for replicating git repositories. However we might also get by just using the standard `requests` library.
adam added the status: acceptedtype: feature labels 2025-12-29 20:24:59 +01:00
adam closed this issue 2025-12-29 20:24:59 +01:00
adam commented 2025-12-29 20:24:59 +01:00
Author
Owner
Copy Link

@rasanentimo commented on GitHub (Jan 28, 2023):

Would it be possible to support S3 as data source as well?

@rasanentimo commented on GitHub (Jan 28, 2023): Would it be possible to support S3 as data source as well?
adam commented 2025-12-29 20:24:59 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Feb 2, 2023):

@rasanentimo it's not in scope for my initial implementation, but it's likely we can add S3 support before the v3.5 release.

@jeremystretch commented on GitHub (Feb 2, 2023): @rasanentimo it's not in scope for my initial implementation, but it's likely we can add S3 support before the v3.5 release.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label status: accepted type: feature
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#7544
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?