Insecure requirements.txt strategy #4625

Closed
opened 2025-12-29 18:38:24 +01:00 by adam · 8 comments
Owner

Originally created by @job on GitHub (Mar 4, 2021).

Originally assigned to: @jeremystretch on GitHub.

Environment

  • Python version: 3.7.3
  • NetBox version: master and 2.10.5

Steps to Reproduce

  1. follow netbox instructions on 'getting started'
  2. follow 'install dependencies' step at https://netbox.readthedocs.io/en/stable/development/getting-started/

Expected Behavior

Netbox is expected to install libraries and auxiliary utilities that don't have known security issues.

Observed Behavior

Netbox installs libraries and auxiliary utilities with known security issues.

Relevant prior discussion: https://github.com/netbox-community/netbox/issues/5904

Originally created by @job on GitHub (Mar 4, 2021). Originally assigned to: @jeremystretch on GitHub. ### Environment * Python version: 3.7.3 * NetBox version: master and 2.10.5 ### Steps to Reproduce 1. follow netbox instructions on 'getting started' 2. follow 'install dependencies' step at https://netbox.readthedocs.io/en/stable/development/getting-started/ 3. <!-- What did you expect to happen? --> ### Expected Behavior Netbox is expected to install libraries and auxiliary utilities that don't have known security issues. <!-- What happened instead? --> ### Observed Behavior Netbox installs libraries and auxiliary utilities with known security issues. Relevant prior discussion: https://github.com/netbox-community/netbox/issues/5904
adam added the status: acceptedtype: documentation labels 2025-12-29 18:38:24 +01:00
adam closed this issue 2025-12-29 18:38:24 +01:00
Author
Owner

@job commented on GitHub (Mar 4, 2021):

It might be worth investigating whether the https://requires.io/ service can be of any help. At the https://github.com/bgpfu/bgpfu project this service is used to stay aware of security issues and requires.io submits easy to merge, example: https://github.com/bgpfu/bgpfu/pull/14/

Another method might be to use the https://pyup.io/ service which tracks security issues across libraries and submits pull requests to the subscribed project, an example is here https://github.com/irrdnet/irrd/pull/450 where IRRd receives free notifications & easy to act upon information in a timely fashion (weekly).

There are quite a few interesting services out there to help stay on top of things and make life easier! :-)

@job commented on GitHub (Mar 4, 2021): It might be worth investigating whether the https://requires.io/ service can be of any help. At the https://github.com/bgpfu/bgpfu project this service is used to stay aware of security issues and requires.io submits easy to merge, example: https://github.com/bgpfu/bgpfu/pull/14/ Another method might be to use the https://pyup.io/ service which tracks security issues across libraries and submits pull requests to the subscribed project, an example is here https://github.com/irrdnet/irrd/pull/450 where IRRd receives free notifications & easy to act upon information in a timely fashion (weekly). There are quite a few interesting services out there to help stay on top of things and make life easier! :-)
Author
Owner

@job commented on GitHub (Mar 4, 2021):

Another mechanism is Dependabot (https://dependabot.com/) which is part of github nowadays

@job commented on GitHub (Mar 4, 2021): Another mechanism is Dependabot (https://dependabot.com/) which is part of github nowadays
Author
Owner

@xkilian commented on GitHub (Mar 4, 2021):

3 things:

  • Include text in installation notes about the Netbox update schedule of requirements for various types of releases.
  • Include information on the possibility of updating local releases to address security vulnerabilities versus the risk of hitting potential bugs in libraries that haven't been tested yet.
  • update automation as suggested above to inform and ease informing, updating, testing in a timely manner. Great suggestions IMO.
@xkilian commented on GitHub (Mar 4, 2021): 3 things: - Include text in installation notes about the Netbox update schedule of requirements for various types of releases. - Include information on the possibility of updating local releases to address security vulnerabilities versus the risk of hitting potential bugs in libraries that haven't been tested yet. - update automation as suggested above to inform and ease informing, updating, testing in a timely manner. Great suggestions IMO.
Author
Owner

@jeremystretch commented on GitHub (Mar 4, 2021):

Netbox is expected to install libraries and auxiliary utilities that don't have known security issues.

I feel the need to point out that just because a library is known to be vulnerable under certain conditions, this does not mean that it's vulnerable as implemented within NetBox. In the (very) few instances in its history where someone has reported a verifiable security concern in the product, we have fixed the issue and published a new release immediately.

Further, please note that as it is an open source product published at no cost to its users, NetBox is provided as-is and without warranty, as cited in the Apache 2 license.

Include text in installation notes about the Netbox update schedule of requirements for various types of releases.

You can find our documented release checklist here.

Include information on the possibility of updating local releases to address security vulnerabilities versus the risk of hitting potential bugs in libraries that haven't been tested yet.

This concept is not unique to NetBox, and is something anyone responsible for administering such a system should understand. It is beyond the scope of our documentation.

update automation as suggested above to inform and ease informing, updating, testing in a timely manner.

NetBox already provides automated release checks. The onus is on the administrator to acknowledge and act on them.

As far as addressing the root concern, I'm happy to modify our release process to include upgrading dependent packages for every release. (Currently, this is done only for milestone releases.) This will ensure that every release ships with the latest version of each required package. However, it should be noted that this increases the testing needed for each release, and will likely delay new releases somewhat.

Historically, NetBox releases have been fairly frequent, with typically just a few weeks between releases. If an administrator desires more frequent upgrades of dependent packages, he or she continues to have the option of updating these packages manually, with the the understanding that any resulting bugs are his or her own responsibility to address, as mentioned above.

@jeremystretch commented on GitHub (Mar 4, 2021): > Netbox is expected to install libraries and auxiliary utilities that don't have known security issues. I feel the need to point out that just because a library is known to be vulnerable under certain conditions, this does not mean that it's vulnerable _as implemented within NetBox_. In the (very) few instances in its history where someone has reported a **verifiable** security concern in the product, we have fixed the issue and published a new release immediately. Further, please note that as it is an open source product published at no cost to its users, NetBox is provided as-is and without warranty, as cited in the Apache 2 license. > Include text in installation notes about the Netbox update schedule of requirements for various types of releases. You can find our documented release checklist [here](https://netbox.readthedocs.io/en/stable/development/release-checklist/). > Include information on the possibility of updating local releases to address security vulnerabilities versus the risk of hitting potential bugs in libraries that haven't been tested yet. This concept is not unique to NetBox, and is something anyone responsible for administering such a system should understand. It is beyond the scope of our documentation. > update automation as suggested above to inform and ease informing, updating, testing in a timely manner. NetBox already provides automated release checks. The onus is on the administrator to acknowledge and act on them. As far as addressing the root concern, I'm happy to modify our release process to include upgrading dependent packages for every release. (Currently, this is done only for milestone releases.) This will ensure that every release ships with the latest version of each required package. However, it should be noted that this increases the testing needed for each release, and will likely delay new releases somewhat. Historically, NetBox releases have been fairly frequent, with typically just a few weeks between releases. If an administrator desires more frequent upgrades of dependent packages, he or she continues to have the option of updating these packages manually, with the the understanding that any resulting bugs are his or her own responsibility to address, as mentioned above.
Author
Owner

@job commented on GitHub (Mar 4, 2021):

Dear Jeremy,

I agree with you on all points. Netbox is valuable to many network operators and it is incredible it exists at all (for free!)

If a library proves to be difficult (difficult in the sense that minor version bumps often result in major breakage) it might be good to evaluate whether a different library can do a better job.

My suggestion is to use a tool like https://pyup.io/ to help bump the requirements.txt more often when security updates are propagating through the ecosystem.

Kind regards,

Job

@job commented on GitHub (Mar 4, 2021): Dear Jeremy, I agree with you on all points. Netbox is valuable to many network operators and it is incredible it exists at all (for free!) If a library proves to be difficult (difficult in the sense that minor version bumps often result in major breakage) it might be good to evaluate whether a different library can do a better job. My suggestion is to use a tool like https://pyup.io/ to help bump the `requirements.txt` more often when security updates are propagating through the ecosystem. Kind regards, Job
Author
Owner

@xkilian commented on GitHub (Mar 4, 2021):

The release requirements are adequate IMO.

I agree with Jeremy that minor releases have their requirements updated. 2.X.Y should not have all their requirements blindly updated to maintain stability and integrity. That is more important and the automated tests do not catch all problems.

We maintain our own security posture through OS and external validation tools.

Pyup.io is interesting as it can flag "major" security issues and the maintainers can then choose to update something in the requirements for a 2.X.Y release. This is more of a process optimisation, to not have to wait for a user to flag a major security issue so that it can be covered in a point release without waiting for a minor release. Maybe it could be included in the CI chain as an informational addition. Great suggestion @job .

If a library proves to be difficult (difficult in the sense that minor version bumps often result in major breakage) it might be good to evaluate whether a different library can do a better job. I think that is a given in the development cycle.

Otherwise not much to add. Thank you Jeremy for the answers and feedback.

@xkilian commented on GitHub (Mar 4, 2021): The release requirements are adequate IMO. I agree with Jeremy that minor releases have their requirements updated. 2.X.Y should not have all their requirements blindly updated to maintain stability and integrity. That is more important and the automated tests do not catch all problems. We maintain our own security posture through OS and external validation tools. Pyup.io is interesting as it can flag "major" security issues and the maintainers can then _choose_ to update something in the requirements for a 2.X.Y release. This is more of a process optimisation, to not have to wait for a user to flag a major security issue so that it can be covered in a point release without waiting for a minor release. Maybe it could be included in the CI chain as an informational addition. Great suggestion @job . `If a library proves to be difficult (difficult in the sense that minor version bumps often result in major breakage) it might be good to evaluate whether a different library can do a better job.` I think that is a given in the development cycle. Otherwise not much to add. Thank you Jeremy for the answers and feedback.
Author
Owner

@jeremystretch commented on GitHub (Mar 4, 2021):

Mind if I keep this open as a reminder to update the release instructions?

@jeremystretch commented on GitHub (Mar 4, 2021): Mind if I keep this open as a reminder to update the release instructions?
Author
Owner

@synnack commented on GitHub (Mar 8, 2021):

I'd like to apologize for my aggressive language in #5904.

I do still feel that libraries should be kept up to date if security issues exist, even with point releases. Of course after running the test suite.

Adding clarification for local updates is definitely a good thing.

+1 for pyup.io , I use that myself to catch issues (which is how I found the library problems in the first place).

@synnack commented on GitHub (Mar 8, 2021): I'd like to apologize for my aggressive language in #5904. I do still feel that libraries should be kept up to date if security issues exist, even with point releases. Of course after running the test suite. Adding clarification for local updates is definitely a good thing. +1 for pyup.io , I use that myself to catch issues (which is how I found the library problems in the first place).
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#4625