starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Implement Caching Mechanism #2177

New Issue
Closed
opened 2025-12-29 17:23:02 +01:00 by adam · 9 comments
adam commented 2025-12-29 17:23:02 +01:00
Owner
Copy Link

Originally created by @tyler-8 on GitHub (Dec 6, 2018).

Environment

  • Python version: 3.6.7
  • NetBox version: 2.4.8

Proposed Functionality

Django offers a caching mechanism to improve page load performance. The three primary methods are

  • per-site cache
  • per-view cache
  • template fragments

Per-site is invalidated strictly by the cache expiry time - meaning if you update an object, the changes don't appear until the cache for that view/page/object expires. This isn't very useful for Netbox workflows.

Per-view and template fragments could both benefit Netbox but would require changes across the project, mostly in the Django views in each area of Netbox. This would involve some sort of CacheMixin. Django's caching interface supports several methods for cache storage (filesystem, database, memcached, redis w/plugins).

The plugin django-cacheops sets up the cache using redis and automatically hooks in to Django's signal framework for invalidating caches as objects are created/updated/deleted. This option requires very little code changes other than adding new configurations.

Use Case

Ideally a user would only need to make a few changes to the configuration in order to select the caching storage method (in the case of the per-view method) and location. From there Netbox would cache on the fly without further intervention.

Database Changes

The database or Netbox model schema would not require any changes.

External Dependencies

Depends on the path chosen

Potentially:

  • redis
  • memcached
  • django-cacheops
  • Another SQL database/table (for cache storage)
Originally created by @tyler-8 on GitHub (Dec 6, 2018). <!-- NOTE: This form is only for proposing specific new features or enhancements. If you have a general idea or question, please post to our mailing list instead of opening an issue: https://groups.google.com/forum/#!forum/netbox-discuss NOTE: Due to an excessive backlog of feature requests, we are not currently accepting any proposals which significantly extend NetBox's feature scope. Please describe the environment in which you are running NetBox. Be sure that you are running an unmodified instance of the latest stable release before submitting a bug report. --> ### Environment * Python version: 3.6.7 <!-- Example: 3.6.4 --> * NetBox version: 2.4.8 <!-- Example: 2.3.6 --> <!-- Describe in detail the new functionality you are proposing. Include any specific changes to work flows, data models, or the user interface. --> ### Proposed Functionality Django offers a [caching mechanism](https://docs.djangoproject.com/en/2.1/topics/cache/#) to improve page load performance. The three primary methods are - per-site cache - per-view cache - template fragments Per-site is invalidated strictly by the cache expiry time - meaning if you update an object, the changes don't appear until the cache for that view/page/object expires. This isn't very useful for Netbox workflows. Per-view and template fragments could both benefit Netbox but would require changes across the project, mostly in the Django `views` in each area of Netbox. This would involve some sort of [CacheMixin](https://www.gyford.com/phil/writing/2018/05/15/invalidating-django-cache/). Django's caching interface supports several methods for cache storage (filesystem, database, memcached, redis w/plugins). The plugin [django-cacheops](https://github.com/Suor/django-cacheops) sets up the cache using redis and automatically hooks in to Django's signal framework for invalidating caches as objects are created/updated/deleted. This option requires very little code changes other than adding new configurations. <!-- Convey an example use case for your proposed feature. Write from the perspective of a NetBox user who would benefit from the proposed functionality and describe how. ---> ### Use Case Ideally a user would only need to make a few changes to the configuration in order to select the caching storage method (in the case of the per-view method) and location. From there Netbox would cache on the fly without further intervention. <!-- Note any changes to the database schema necessary to support the new feature. For example, does the proposal require adding a new model or field? (Not all new features require database changes.) ---> ### Database Changes The database or Netbox model schema would not require any changes. <!-- List any new dependencies on external libraries or services that this new feature would introduce. For example, does the proposal require the installation of a new Python package? (Not all new features introduce new dependencies.) --> ### External Dependencies Depends on the path chosen Potentially: - redis - memcached - django-cacheops - Another SQL database/table (for cache storage)
adam added the status: accepted label 2025-12-29 17:23:02 +01:00
adam closed this issue 2025-12-29 17:23:02 +01:00
adam commented 2025-12-29 17:23:03 +01:00
Author
Owner
Copy Link

@sdktr commented on GitHub (Dec 10, 2018):

Very curious to the latency enhancements this could bring in both UI and API. Caching on the 'view' level means a 'calculated' value is stored right? Not just a [set of] database call saved, but the actual rendering of the data (like prefix hierarcy)?

@sdktr commented on GitHub (Dec 10, 2018): Very curious to the latency enhancements this could bring in both UI and API. Caching on the 'view' level means a 'calculated' value is stored right? Not just a [set of] database call saved, but the actual rendering of the data (like prefix hierarcy)?
adam commented 2025-12-29 17:23:03 +01:00
Author
Owner
Copy Link

@tyler-8 commented on GitHub (Dec 11, 2018):

@sdktr That's correct. The final rendered output is what is cached so the prefix page would see big improvements for follow-on visits. The first person to visit the page after the cache expiry time would still experience the page load delay due to the IP calculations, but any additional views would be almost instant.

@tyler-8 commented on GitHub (Dec 11, 2018): @sdktr That's correct. The final rendered output is what is cached so the prefix page would see big improvements for _follow-on_ visits. The first person to visit the page after the cache expiry time would still experience the page load delay due to the IP calculations, but any additional views would be almost instant.
adam commented 2025-12-29 17:23:03 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Dec 19, 2018):

Tagging this tentatively for v2.6. We'll need to do a lot of exploration and testing but it seems feasible.

@jeremystretch commented on GitHub (Dec 19, 2018): Tagging this **tentatively** for v2.6. We'll need to do a lot of exploration and testing but it seems feasible.
adam commented 2025-12-29 17:23:03 +01:00
Author
Owner
Copy Link

@paravoid commented on GitHub (Jan 24, 2019):

While implementing this and looking at all the complexities around cacheability (what can be cached, for whom, for how long, etc.) it would be cool if the @cache_control and @vary_on_cookie decorators (or equivalent functionality) were also considered. They'd help with caching both in the browser, and frontend caches (like Varnish etc.)

@paravoid commented on GitHub (Jan 24, 2019): While implementing this and looking at all the complexities around cacheability (what can be cached, for whom, for how long, etc.) it would be cool if the `@cache_control` and `@vary_on_cookie` decorators (or equivalent functionality) were also considered. They'd help with caching both in the browser, and frontend caches (like Varnish etc.)
adam commented 2025-12-29 17:23:03 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Apr 16, 2019):

@lampwins One issue I've noticed with caching is that the cache is not expired on login/logout. We probably want to wrap views with vary_on_headers() to ensure a different cache per user.

@jeremystretch commented on GitHub (Apr 16, 2019): @lampwins One issue I've noticed with caching is that the cache is not expired on login/logout. We probably want to wrap views with [`vary_on_headers()`](https://docs.djangoproject.com/en/2.2/topics/cache/#using-vary-headers) to ensure a different cache per user.
adam commented 2025-12-29 17:23:03 +01:00
Author
Owner
Copy Link

@sdktr commented on GitHub (Apr 16, 2019):

Is the direction of the cache implementation focussed on GUI views only or does it speed up API responses as well?

@sdktr commented on GitHub (Apr 16, 2019): Is the direction of the cache implementation focussed on GUI views only or does it speed up API responses as well?
adam commented 2025-12-29 17:23:03 +01:00
Author
Owner
Copy Link

@lampwins commented on GitHub (Apr 19, 2019):

Implemented in develop-2.6

@lampwins commented on GitHub (Apr 19, 2019): Implemented in `develop-2.6`
adam commented 2025-12-29 17:23:03 +01:00
Author
Owner
Copy Link

@vali-um commented on GitHub (Jun 13, 2019):

I was just looking through those changes as we have a small instances of netbox running and don't really want to install redis for a functionality not needed... and i'd imagine that i'm not the only one in a similar situation.
Would it be feasible to keep redis as an option and have caching optional as well? As far as i understood the changes this wold only require minor changes.

@vali-um commented on GitHub (Jun 13, 2019): I was just looking through those changes as we have a small instances of netbox running and don't really want to install redis for a functionality not needed... and i'd imagine that i'm not the only one in a similar situation. Would it be feasible to keep redis as an option and have caching optional as well? As far as i understood the changes this wold only require minor changes.
adam commented 2025-12-29 17:23:04 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Jun 14, 2019):

Redis is now a requirement of NetBox starting with v2.6, not only for caching and webhooks but for future planned features as well.

@jeremystretch commented on GitHub (Jun 14, 2019): Redis is now a requirement of NetBox starting with v2.6, not only for caching and webhooks but for future planned features as well.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label status: accepted
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#2177
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?