Device queries from Netbox GUI via Napalm fail after upgrade from 3.1.2 to 3.1.5 #5932

Closed
opened 2025-12-29 19:34:26 +01:00 by adam · 5 comments
Owner

Originally created by @Eschin on GitHub (Jan 12, 2022).

NetBox version

v3.1.5

Python version

3.8

Steps to Reproduce

  1. Connecting to device to pull Status, LLDP Neighbors, or Configuration from Netbox GUI results in ConnectAuthError (picture attached) on Juniper SRX devices, but not Juniper MX devices.

Expected Behavior

Should connect via SSH and pull device configuration, lldp neighbors, or status and display on Netbox GUI.

Observed Behavior

When attempting to connect to devices via GUI, the following error displays.

image

SSH host/server/device shows this error for failed inbound connections in the logs; userauth_pubkey: unsupported public key algorithm: rsa-sha2-512 [preauth].

Manual SSH via backend CLI OpenSSH_8.2p1 using the same key pair(that fails in Netbox GUI) works fine.

Attempting to switch to different key type(ed25519) unsuccessful.

Really odd, as we're only seeing this behavior on Juniper SRX devices, but Juniper MX devices working normally.

Originally created by @Eschin on GitHub (Jan 12, 2022). ### NetBox version v3.1.5 ### Python version 3.8 ### Steps to Reproduce 1. Connecting to device to pull Status, LLDP Neighbors, or Configuration from Netbox GUI results in `ConnectAuthError` (picture attached) on Juniper SRX devices, but not Juniper MX devices. ### Expected Behavior Should connect via SSH and pull device configuration, lldp neighbors, or status and display on Netbox GUI. ### Observed Behavior When attempting to connect to devices via GUI, the following error displays. ![image](https://user-images.githubusercontent.com/5325383/149167756-257d7168-5056-4bcc-a941-4c7bd2b1c7d5.png) SSH host/server/device shows this error for failed inbound connections in the logs; `userauth_pubkey: unsupported public key algorithm: rsa-sha2-512 [preauth]`. Manual SSH via backend CLI OpenSSH_8.2p1 using the same key pair(that fails in Netbox GUI) works fine. Attempting to switch to different key type(ed25519) unsuccessful. Really odd, as we're only seeing this behavior on Juniper SRX devices, but Juniper MX devices working normally.
adam closed this issue 2025-12-29 19:34:27 +01:00
Author
Owner

@jeremystretch commented on GitHub (Jan 12, 2022):

Was the NAPALM driver upgraded when you upgraded NetBox? That's the only thing I can think of, assuming the configuration has not been modified. NetBox has no control over the actual NAPALM connection.

@jeremystretch commented on GitHub (Jan 12, 2022): Was the NAPALM driver upgraded when you upgraded NetBox? That's the only thing I can think of, assuming the configuration has not been modified. NetBox has no control over the actual NAPALM connection.
Author
Owner

@Eschin commented on GitHub (Jan 12, 2022):

Napalm was not recently upgraded. We're currently running on Napalm 3.3.1, which was released Jun 16, 2021 (I believe?). We stay current with all system packages and Python packages on our Netbox servers, so we've been running on Napalm 3.3.1 since middle of last year roughly(We update once or twice monthly). I will look through our Ansible upgrade logs to pinpoint exactly when we upgraded to 3.3.1.

We have 3 systems running Netbox, all kept on identical versions. We were able to confirm this issue only appeared after upgrading from 3.1.2 to 3.1.5 on the third(final) system we upgraded.

One of these systems is a "dog food/testing" system, so I can easily roll it back to last week and scope out the state the system was in pre-upgrade.

Our configuration.py has remained unchanged for years. We've always kept the NAPALM_USERNAME and NAPALM_PASSWORD blank, so it defaults to the netbox system user the software stack runs under, and authenticates via the netbox user's ssh-key pair.

I was already suspicious of the Napalm package being the culprit as mentioned, but when I saw "#7246 - Don't attempt to URL-decode NAPALM response payloads" on the 3.1.3 release notes, I got suspicious these two issues may be connected somehow... But looking over that issue, I see no immediate way they could be.... Maybe a coincidence?

@Eschin commented on GitHub (Jan 12, 2022): Napalm was not recently upgraded. We're currently running on Napalm 3.3.1, which was released Jun 16, 2021 (I believe?). We stay current with all system packages and Python packages on our Netbox servers, so we've been running on Napalm 3.3.1 since middle of last year roughly(We update once or twice monthly). I will look through our Ansible upgrade logs to pinpoint exactly when we upgraded to 3.3.1. We have 3 systems running Netbox, all kept on identical versions. We were able to confirm this issue only appeared after upgrading from 3.1.2 to 3.1.5 on the third(final) system we upgraded. One of these systems is a "dog food/testing" system, so I can easily roll it back to last week and scope out the state the system was in pre-upgrade. Our configuration.py has remained unchanged for years. We've always kept the NAPALM_USERNAME and NAPALM_PASSWORD blank, so it defaults to the `netbox` system user the software stack runs under, and authenticates via the `netbox` user's ssh-key pair. I was already suspicious of the Napalm package being the culprit as mentioned, but when I saw "#7246 - Don't attempt to URL-decode NAPALM response payloads" on the 3.1.3 release notes, I got suspicious these two issues may be connected somehow... But looking over that issue, I see no immediate way they could be.... Maybe a coincidence?
Author
Owner

@jeremystretch commented on GitHub (Jan 12, 2022):

I don't believe #7246 is related as it deals solely with the processing of a response after a connection has been established and a request has been sent.

The error message certainly seems to suggest an issue with an underlying library. Can you connect to one of these devices using napalm directly (within a Python shell) and the same credentials?

@jeremystretch commented on GitHub (Jan 12, 2022): I don't believe #7246 is related as it deals solely with the processing of a response after a connection has been established and a request has been sent. The error message certainly seems to suggest an issue with an underlying library. Can you connect to one of these devices using napalm directly (within a Python shell) and the same credentials?
Author
Owner

@Eschin commented on GitHub (Jan 12, 2022):

Hey Jeremy,

I agree, starting to think the issue is an underlying library.

Napalm direct connections from Netbox's Python environment to SRX devices also fail.

I did not notice this before, but there was a minor release upgrade to the SSH client during our last upgrade. We were on OpenSSH_8.2p1 Ubuntu-4ubuntu0.3, OpenSSL 1.1.1f 31 Mar 2020, and after the upgrade, ssh client was upgraded to OpenSSH_8.2p1 Ubuntu-4ubuntu0.4, OpenSSL 1.1.1f 31 Mar 2020. Ubuntu-4ubuntu0.3 changed to Ubuntu-4ubuntu0.4. Napalm was still on 3.3.1 pre/post upgrade.

So yeah, starting to think the issue is in the (Ubuntu) SSH package itself.

Appreciate your time, apologies for the false alarm. Going to close this out now :)

@Eschin commented on GitHub (Jan 12, 2022): Hey Jeremy, I agree, starting to think the issue is an underlying library. Napalm direct connections from Netbox's Python environment to SRX devices also fail. I did not notice this before, but there was a minor release upgrade to the SSH client during our last upgrade. We were on `OpenSSH_8.2p1 Ubuntu-4ubuntu0.3, OpenSSL 1.1.1f 31 Mar 2020`, and after the upgrade, ssh client was upgraded to `OpenSSH_8.2p1 Ubuntu-4ubuntu0.4, OpenSSL 1.1.1f 31 Mar 2020`. `Ubuntu-4ubuntu0.3` changed to `Ubuntu-4ubuntu0.4`. Napalm was still on 3.3.1 pre/post upgrade. So yeah, starting to think the issue is in the (Ubuntu) SSH package itself. Appreciate your time, apologies for the false alarm. Going to close this out now :)
Author
Owner

@jeremystretch commented on GitHub (Jan 12, 2022):

No worries. Thanks for following up! I suspect you just need to restrict the set of algorithms offered by the SSH client somehow.

@jeremystretch commented on GitHub (Jan 12, 2022): No worries. Thanks for following up! I suspect you just need to restrict the set of algorithms offered by the SSH client somehow.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#5932