starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Add SSH Key field when making git datasources #9340

New Issue
Open
opened 2025-12-29 20:48:44 +01:00 by adam · 17 comments
adam commented 2025-12-29 20:48:44 +01:00
Owner
Copy Link

Originally created by @Mailstorm-ctrl on GitHub (Mar 11, 2024).

NetBox version

3.6.3

Feature type

New functionality

Proposed functionality

This is in regards to https://github.com/netbox-community/netbox/discussions/14941

Add a field that allows a user to supply a path to an SSH key file to use to authenticate to GitHub.

Use case

GitHub enterprise tenants with Managed Users require the use of an external IdP to authenticate to GitHub. Username and password authentication will not work to the API or to login. Allowing the use of SSH keys allows people using an EMU instance (or just preference) to use git over SSH.

Database changes

Most likely need a new field that will allow a user to supply a SSH key path. I can see a use case for requiring multiple different keys

External dependencies

No response

Originally created by @Mailstorm-ctrl on GitHub (Mar 11, 2024). ### NetBox version 3.6.3 ### Feature type New functionality ### Proposed functionality This is in regards to https://github.com/netbox-community/netbox/discussions/14941 Add a field that allows a user to supply a path to an SSH key file to use to authenticate to GitHub. ### Use case GitHub enterprise tenants with Managed Users require the use of an external IdP to authenticate to GitHub. Username and password authentication will not work to the API or to login. Allowing the use of SSH keys allows people using an EMU instance (or just preference) to use git over SSH. ### Database changes Most likely need a new field that will allow a user to supply a SSH key path. I can see a use case for requiring multiple different keys ### External dependencies _No response_
adam added the type: featurecomplexity: mediumnetboxstatus: backlog labels 2025-12-29 20:48:44 +01:00
adam commented 2025-12-29 20:48:45 +01:00
Author
Owner
Copy Link

@ross-cello commented on GitHub (Mar 12, 2024):

Would love to see this implemented

@ross-cello commented on GitHub (Mar 12, 2024): Would love to see this implemented
adam commented 2025-12-29 20:48:45 +01:00
Author
Owner
Copy Link

@cpmills1975 commented on GitHub (Mar 20, 2024):

Please do implement this. My local git repo requires SSH keys to authenticate. Given the complexity of maintaining SSH keys, IMHO it would be worth splitting out the authentication methods from the data sources at this point and creating a 'Credential' model to store the keys in the DB. The Credential model would be one of either username/password combination or SSH key and optional password. The datasource would have a foreign key relationship to the credential making the credential re-usable across several datasources if desired. If Dulwich needs the key in a file (I'll admit to only cursory glances at the docs), then the key would need to be written to a temporary file when any remote file sync activity takes place.

@cpmills1975 commented on GitHub (Mar 20, 2024): Please do implement this. My local git repo requires SSH keys to authenticate. Given the complexity of maintaining SSH keys, IMHO it would be worth splitting out the authentication methods from the data sources at this point and creating a 'Credential' model to store the keys in the DB. The Credential model would be one of either username/password combination or SSH key and optional password. The datasource would have a foreign key relationship to the credential making the credential re-usable across several datasources if desired. If Dulwich needs the key in a file (I'll admit to only cursory glances at the docs), then the key would need to be written to a temporary file when any remote file sync activity takes place.
adam commented 2025-12-29 20:48:46 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Mar 20, 2024):

This is open for volunteers.

@jeremystretch commented on GitHub (Mar 20, 2024): This is open for volunteers.
adam commented 2025-12-29 20:48:46 +01:00
Author
Owner
Copy Link

@markkuleinio commented on GitHub (Mar 22, 2024):

Does someone know how to even provide the keyfile to porcelain.clone()? I mean, I've tried to provide key_filename="path_to_keyfile" in the call, but it doesn't seem to do anything.

SyncError('Fetching remote data failed (HangupException): git@github.com: Permission denied (publickey).\r')

I don't seem to find any example anywhere.

(aaand update: key_filename works just fine, provided that I also restart netbox-rq, not just netbox... to be continued)

@markkuleinio commented on GitHub (Mar 22, 2024): Does someone know how to even provide the keyfile to `porcelain.clone()`? I mean, I've tried to provide `key_filename="path_to_keyfile"` in the call, but it doesn't seem to do anything. SyncError('Fetching remote data failed (HangupException): git@github.com: Permission denied (publickey).\r') I don't seem to find any example anywhere. (aaand update: `key_filename` works just fine, provided that I **also** restart `netbox-rq`, not just `netbox`... to be continued)
adam commented 2025-12-29 20:48:47 +01:00
Author
Owner
Copy Link

@markkuleinio commented on GitHub (Mar 23, 2024):

Workaround to use SSH key:

Add this in /etc/systemd/system/netbox-rq.service, in the [Service] section:

Environment="GIT_SSH_COMMAND=ssh -i /path/to/ssh_private_key"

and restart the service:

sudo systemctl daemon-reload
sudo systemctl restart netbox-rq

Limitation: You can only select one keyfile --> If you have more than one data source, you need to authorize the same private key to access all those data sources.

@markkuleinio commented on GitHub (Mar 23, 2024): Workaround to use SSH key: Add this in `/etc/systemd/system/netbox-rq.service`, in the `[Service]` section: ``` Environment="GIT_SSH_COMMAND=ssh -i /path/to/ssh_private_key" ``` and restart the service: ``` sudo systemctl daemon-reload sudo systemctl restart netbox-rq ``` Limitation: You can only select one keyfile --> If you have more than one data source, you need to authorize the same private key to access all those data sources.
adam commented 2025-12-29 20:48:47 +01:00
Author
Owner
Copy Link

@Mailstorm-ctrl commented on GitHub (May 22, 2024):

Well this is exciting. I got this to work with no code changes...though this is dirty and I'm almost sure it goes against some kind of practice. I have successfully gotten this to work with multiple repos with multiple keys. Here's how I did it:

We are going to be enabling login on the local netbox user on the host server! If you don't want to do this, then sorry you can't do this

  1. Enable local login on the netbox user (or whatever user you used for the system account when initially installing netbox.
    sudo usermod -s /bin/bash netbox
  2. Switch to the user to generate your SSH keys
    sudo su - netbox
  3. Generate your ssh keys and name them whatever you want.
  4. Run eval "$(ssh-agent)" (See here for why)
  5. Run the following for however many keys you made:
    ssh-add <path to sshkey>
  6. In GitHub, create your deploy keys
  7. Back on the netbox host, create or modify the ~/.ssh/config file while still acting as the netbox user. Modify the following to your liking:
Host repo1.github.com
        HostName github.com
        PreferredAuthentications publickey
        IdentityFile ~/.ssh/key1
Host repo2.github.com
        HostName github.com
        PreferredAuthentications publickey
        IdentityFile ~/.ssh/key2
  1. In netbox, go ahead and add a git datasource. However, for the url you will do something similar to this: git@repo1.github.com:orgname/repo1. Leave the username and password blank. I can't verify if the branch works or not.
  2. Enjoy your SSH connected repos
@Mailstorm-ctrl commented on GitHub (May 22, 2024): Well this is exciting. I got this to work with no code changes...though this is dirty and I'm almost sure it goes against some kind of practice. I have successfully gotten this to work with multiple repos with multiple keys. Here's how I did it: **We are going to be enabling login on the local netbox user on the host server! If you don't want to do this, then sorry you can't do this** 1. Enable local login on the netbox user (or whatever user you used for the system account when initially installing netbox. `sudo usermod -s /bin/bash netbox` 2. Switch to the user to generate your SSH keys `sudo su - netbox` 3. Generate your ssh keys and name them whatever you want. 4. Run `eval "$(ssh-agent)"` ([See here for why](https://unix.stackexchange.com/questions/48863/ssh-add-complains-could-not-open-a-connection-to-your-authentication-agent)) 5. Run the following for however many keys you made: `ssh-add <path to sshkey>` 6. In GitHub, create your [deploy keys](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/managing-deploy-keys) 7. Back on the netbox host, create or modify the ~/.ssh/config file while still acting as the netbox user. Modify the following to your liking: ``` Host repo1.github.com HostName github.com PreferredAuthentications publickey IdentityFile ~/.ssh/key1 Host repo2.github.com HostName github.com PreferredAuthentications publickey IdentityFile ~/.ssh/key2 ``` 8. In netbox, go ahead and add a git datasource. However, for the url you will do something similar to this: `git@repo1.github.com:orgname/repo1`. Leave the username and password blank. I can't verify if the branch works or not. 9. Enjoy your SSH connected repos
adam commented 2025-12-29 20:48:48 +01:00
Author
Owner
Copy Link

@llamafilm commented on GitHub (Jun 3, 2024):

You also have to include the server in /home/netbox/.ssh/known_hosts or else the sync will fail.

@llamafilm commented on GitHub (Jun 3, 2024): You also have to include the server in `/home/netbox/.ssh/known_hosts` or else the sync will fail.
adam commented 2025-12-29 20:48:48 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Jul 19, 2024):

Add a field that allows a user to supply a path to an SSH key file to use to authenticate to GitHub.

Reading through this again, I'm not sure this is a suitable solution to the root problem. Asking for a path to a file on disk assumes 1) the user has knowledge of the underlying filesystem and 2) the user has access to install a file.

Given the complexity of maintaining SSH keys, IMHO it would be worth splitting out the authentication methods from the data sources at this point and creating a 'Credential' model to store the keys in the DB.

This gets closer to a solution, but IMO we should stop short of actually storing private keys in the database. That's just not something we can do securely.

Unfortunately I don't have a better proposal at the moment, but I also don't want to burn time and effort delivering an implementation that likely won't suffice for a large portion of the user base.

@jeremystretch commented on GitHub (Jul 19, 2024): > Add a field that allows a user to supply a path to an SSH key file to use to authenticate to GitHub. Reading through this again, I'm not sure this is a suitable solution to the root problem. Asking for a path to a file on disk assumes 1) the user has knowledge of the underlying filesystem and 2) the user has access to install a file. > Given the complexity of maintaining SSH keys, IMHO it would be worth splitting out the authentication methods from the data sources at this point and creating a 'Credential' model to store the keys in the DB. This gets closer to a solution, but IMO we should stop short of actually storing private keys in the database. That's just not something we can do securely. Unfortunately I don't have a better proposal at the moment, but I also don't want to burn time and effort delivering an implementation that likely won't suffice for a large portion of the user base.
adam commented 2025-12-29 20:48:48 +01:00
Author
Owner
Copy Link

@Mailstorm-ctrl commented on GitHub (Jul 19, 2024):

Add a field that allows a user to supply a path to an SSH key file to use to authenticate to GitHub.

Reading through this again, I'm not sure this is a suitable solution to the root problem. Asking for a path to a file on disk assumes 1) the user has knowledge of the underlying filesystem and 2) the user has access to install a file.

To me setting something like this up doesn't need to be "user friendly". It's like making scripts. Making scripts assumes you have good knowledge of netbox models and python in general.

Unfortunately I don't have a better proposal at the moment, but I also don't want to burn time and effort delivering an implementation that likely won't suffice for a large portion of the user base.

The work-around I provided seems to be working as intended. I can write better documentation on how to achieve this if you think it's beneficial overall. From what I read, this is probably the most supported option when you need to interact with multiple keys in git.

@Mailstorm-ctrl commented on GitHub (Jul 19, 2024): > > Add a field that allows a user to supply a path to an SSH key file to use to authenticate to GitHub. > > Reading through this again, I'm not sure this is a suitable solution to the root problem. Asking for a path to a file on disk assumes 1) the user has knowledge of the underlying filesystem and 2) the user has access to install a file. > To me setting something like this up doesn't need to be "user friendly". It's like making scripts. Making scripts assumes you have good knowledge of netbox models and python in general. > Unfortunately I don't have a better proposal at the moment, but I also don't want to burn time and effort delivering an implementation that likely won't suffice for a large portion of the user base. The work-around I provided seems to be working as intended. I can write better documentation on how to achieve this if you think it's beneficial overall. From what I read, this is probably the most supported option when you need to interact with multiple keys in git.
adam commented 2025-12-29 20:48:48 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Jul 19, 2024):

It's like making scripts.

We discovered we needed to add the ability to upload/sync scripts directly into NetBox specifically because the local filesystem was inaccessible to many users, which precluded them from using scripts.

@jeremystretch commented on GitHub (Jul 19, 2024): > It's like making scripts. We discovered we needed to add the ability to upload/sync scripts directly into NetBox specifically because the local filesystem was inaccessible to many users, which precluded them from using scripts.
adam commented 2025-12-29 20:48:48 +01:00
Author
Owner
Copy Link

@Mailstorm-ctrl commented on GitHub (Sep 11, 2024):

Not exactly sure what changed, but when upgrading from 4.0.3 to 4.1.0, this method no longer works. You get "Permission Denied" as the error. Logging in as netbox on the underlying OS and then using a sample python file that runs the same instructions as netbox does completes without errors.

False alarm. The netbox user didn't have permissions to the scripts directory

@Mailstorm-ctrl commented on GitHub (Sep 11, 2024): ~~Not exactly sure what changed, but when upgrading from 4.0.3 to 4.1.0, this method no longer works. You get "Permission Denied" as the error. Logging in as netbox on the underlying OS and then using a sample python file that runs the same instructions as netbox does completes without errors.~~ False alarm. The netbox user didn't have permissions to the scripts directory
adam commented 2025-12-29 20:48:49 +01:00
Author
Owner
Copy Link

@psharp-vs commented on GitHub (Oct 22, 2024):

This gets closer to a solution, but IMO we should stop short of actually storing private keys in the database. That's just not something we can do securely.

A number of popular git providers (GitHub, GitLab, Bitbucket) allow for the creation of PAT tokens which are repo specific and reduce the risk of storing an entire private key, might be a good middle ground.

@psharp-vs commented on GitHub (Oct 22, 2024): > This gets closer to a solution, but IMO we should stop short of actually storing private keys in the database. That's just not something we can do securely. A number of popular git providers (GitHub, GitLab, Bitbucket) allow for the creation of PAT tokens which are repo specific and reduce the risk of storing an entire private key, might be a good middle ground.
adam commented 2025-12-29 20:48:49 +01:00
Author
Owner
Copy Link

@Mailstorm-ctrl commented on GitHub (Oct 22, 2024):

This gets closer to a solution, but IMO we should stop short of actually storing private keys in the database. That's just not something we can do securely.

A number of popular git providers (GitHub, GitLab, Bitbucket) allow for the creation of PAT tokens which are repo specific and reduce the risk of storing an entire private key, might be a good middle ground.

These are personal access tokens which means they are tied to a user...not the organization itself. Not ideal in enterprise environments in my opinion. Would be fine for sandbox environments or similar.

@Mailstorm-ctrl commented on GitHub (Oct 22, 2024): > > This gets closer to a solution, but IMO we should stop short of actually storing private keys in the database. That's just not something we can do securely. > > A number of popular git providers (GitHub, GitLab, Bitbucket) allow for the creation of PAT tokens which are repo specific and reduce the risk of storing an entire private key, might be a good middle ground. These are personal access tokens which means they are tied to a user...not the organization itself. Not ideal in enterprise environments in my opinion. Would be fine for sandbox environments or similar.
adam commented 2025-12-29 20:48:49 +01:00
Author
Owner
Copy Link

@psharp-vs commented on GitHub (Oct 22, 2024):

This gets closer to a solution, but IMO we should stop short of actually storing private keys in the database. That's just not something we can do securely.

A number of popular git providers (GitHub, GitLab, Bitbucket) allow for the creation of PAT tokens which are repo specific and reduce the risk of storing an entire private key, might be a good middle ground.

These are personal access tokens which means they are tied to a user...not the organization itself. Not ideal in enterprise environments in my opinion. Would be fine for sandbox environments or similar.

I had imagined them tied to a service account. That's at least our orgs preferred method to have applications access git resources. In fact, after exploring a bit more we actually disable PAT tokens for individuals but enable them for service accounts for precisely this use case.

@psharp-vs commented on GitHub (Oct 22, 2024): > > > This gets closer to a solution, but IMO we should stop short of actually storing private keys in the database. That's just not something we can do securely. > > > > > > A number of popular git providers (GitHub, GitLab, Bitbucket) allow for the creation of PAT tokens which are repo specific and reduce the risk of storing an entire private key, might be a good middle ground. > > These are personal access tokens which means they are tied to a user...not the organization itself. Not ideal in enterprise environments in my opinion. Would be fine for sandbox environments or similar. I had imagined them tied to a service account. That's at least our orgs preferred method to have applications access git resources. In fact, after exploring a bit more we actually disable PAT tokens for individuals but enable them for service accounts for precisely this use case.
adam commented 2025-12-29 20:48:50 +01:00
Author
Owner
Copy Link

@cpmills1975 commented on GitHub (Oct 22, 2024):

I'd be happy to accept the risk of holding SSH keys in the database. I'm stuck in that my corporate git repo requires keys for authentication. These keys can (should be) be entirely separate to any and all other systems and the key can be granted read only access to just one repo if required. The risks in uploading keys that have full access to multiple systems should be somewhat obvious, but documentation can perhaps help with that.

Failing that, I'd accept SSH keys on the file system and a pointer to them in the DB, but appreciate that's because I can put them there and this won't work for everyone in every environment.

@cpmills1975 commented on GitHub (Oct 22, 2024): I'd be happy to accept the risk of holding SSH keys in the database. I'm stuck in that my corporate git repo requires keys for authentication. These keys can (should be) be entirely separate to any and all other systems and the key can be granted read only access to just one repo if required. The risks in uploading keys that have full access to multiple systems should be somewhat obvious, but documentation can perhaps help with that. Failing that, I'd accept SSH keys on the file system and a pointer to them in the DB, but appreciate that's because I can put them there and this won't work for everyone in every environment.
adam commented 2025-12-29 20:48:50 +01:00
Author
Owner
Copy Link

@psharp-vs commented on GitHub (Oct 23, 2024):

I did a bit of exploring on this to see what kind of code changes might be needed.

The user/password/branch data is stored as a JSON blob in the database. Adding an ssh key would be trivial and not require any database changes. Is it secure? Probably not, but we're already storing plain-text passwords so is it already any worse? Adding a little bit of logic to the fetch method to support an ssh key if it exists should be pretty easy as well.

I had mentioned PAT tokens, which is something many git providers can generate. I have only tested on GitHub, but this already works, the PAT token acts like a password.

@psharp-vs commented on GitHub (Oct 23, 2024): I did a bit of exploring on this to see what kind of code changes might be needed. The user/password/branch data is stored as a JSON blob in the database. Adding an ssh key would be trivial and not require any database changes. Is it secure? Probably not, but we're already storing plain-text passwords so is it already any worse? Adding a little bit of logic to the fetch method to support an ssh key if it exists should be pretty easy as well. I had mentioned PAT tokens, which is something many git providers can generate. I have only tested on GitHub, but this already works, the PAT token acts like a password.
adam commented 2025-12-29 20:48:50 +01:00
Author
Owner
Copy Link

@mskalecki commented on GitHub (Nov 13, 2024):

If using GitHub, the new (still in preview) Fine-Grained Personal Access Tokens are an exceptionally good alternative to storing multiple ssh keys. You can lock a PAT down to have read-only access to only the contents of an individual repo (or multiple individual repos). And, as @psharp-vs mentioned, is is already supported by Netbox. Just use the repo's https url, username = oauth2, and password = <personal access token>

Does it make sense to add a note to the documentation that Personal Access Tokens can be used in place of username / password in this way? Seems really likely that a lot of users won't be Git / GitHub experts and are just used to always using SSH keys (like I was).

@mskalecki commented on GitHub (Nov 13, 2024): If using GitHub, the new (still in preview) Fine-Grained Personal Access Tokens are an exceptionally good alternative to storing multiple ssh keys. You can lock a PAT down to have read-only access to only the contents of an individual repo (or multiple individual repos). And, as @psharp-vs mentioned, is is already supported by Netbox. Just use the repo's https url, username = `oauth2`, and password = `<personal access token>` Does it make sense to add a note to the documentation that Personal Access Tokens can be used in place of username / password in this way? Seems really likely that a lot of users won't be Git / GitHub experts and are just used to always using SSH keys (like I was).
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label complexity: medium netbox status: backlog type: feature
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#9340
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?