Cache counts of related objects for recursively-nested models #11424

New Issue

adam · 2025-12-29T21:45:04+01:00

adam commented

2025-12-29 21:45:04 +01:00

Originally created by @jeremystretch on GitHub (Jul 29, 2025).

NetBox version

v4.3.4

Feature type

Change to existing functionality

Proposed functionality

There are many instances throughout the application where we attach a count of related objects to a queryset. For example, consider the locations REST API endpoint. It extends the base queryset to attach rack_count and device_count for each location. (This occurs regardless of whether a subset of fields was specified in the API request.)

c736ce3179/netbox/dcim/api/views.py (L137-L150)

We should explore opportunities where these complex queries can be obviated, potentially through the use of CounterCacheField. This is already utilized in a few places; most notably on the Device and DeviceType models to store counts of child components:

c736ce3179/netbox/dcim/models/devices.py (L641-L649)

It remains to be determined whether CounterCacheField can be adapted for this purpose, or whether a different approach is needed to handle the cumulative nature of such counts. (The count of related objects should include both direct and indirect children.)

Use case

Adopting cached counters, if proven viable, would obviate the need for complex queries to count related objects at query time, and instead allow us to return pre-calculated counts directly. This would substantially improve query performance in many scenarios.

Database changes

In the example above concerning the Location model, this would involve introducing two new fields on the model:

class Location:
    rack_count = CounterCacheField( 
        to_model='dcim.Rack', 
        to_field='location' 
    ) 
    device_count = CounterCacheField( 
        to_model='dcim.Device', 
        to_field='location' 
    )

External dependencies

N/A

Originally created by @jeremystretch on GitHub (Jul 29, 2025). ### NetBox version v4.3.4 ### Feature type Change to existing functionality ### Proposed functionality There are many instances throughout the application where we attach a count of related objects to a queryset. For example, consider the locations REST API endpoint. It extends the base queryset to attach `rack_count` and `device_count` for each location. (This occurs regardless of whether a subset of fields was specified in the API request.) https://github.com/netbox-community/netbox/blob/c736ce3179a8bb7bbcbf29324a08a62efa8747b6/netbox/dcim/api/views.py#L137-L150 We should explore opportunities where these complex queries can be obviated, potentially through the use of [`CounterCacheField`](https://github.com/netbox-community/netbox/blob/c736ce3179a8bb7bbcbf29324a08a62efa8747b6/netbox/utilities/fields.py#L156). This is already utilized in a few places; most notably on the Device and DeviceType models to store counts of child components: https://github.com/netbox-community/netbox/blob/c736ce3179a8bb7bbcbf29324a08a62efa8747b6/netbox/dcim/models/devices.py#L641-L649 It remains to be determined whether `CounterCacheField` can be adapted for this purpose, or whether a different approach is needed to handle the cumulative nature of such counts. (The count of related objects should include both direct and indirect children.) ### Use case Adopting cached counters, if proven viable, would obviate the need for complex queries to count related objects at query time, and instead allow us to return pre-calculated counts directly. This would substantially improve query performance in many scenarios. ### Database changes In the example above concerning the Location model, this would involve introducing two new fields on the model: ```python class Location: rack_count = CounterCacheField( to_model='dcim.Rack', to_field='location' ) device_count = CounterCacheField( to_model='dcim.Device', to_field='location' ) ``` ### External dependencies N/A

adam added the type: feature netbox status: backlog complexity: high labels 2025-12-29 21:45:04 +01:00

Sign in to join this conversation.

Branches Tags

main

feature

19034-rackreservation-unit-count

15513-add-bulk-create-for-prefixes

21556-fix-dropdown-clearing

20152-support-for-marking-module-bays-and-device-bays-as-disabled

21157-export-template-public-models

20698-add-a-read-only-vlan-ids-count-to-the-vlanrange-model

21579-script-add-button-permissions

21357-register-model-actions

21025-pre-render-config-contexts

21364-swagger

feature-ip-prefix-link

20911-dropdown-3

fix_module_substitution

21203-q-attr-denorm

21160-filterset

20911-dropdown-2

20044-elevation-stuck-lightmode

v4.5-beta1-release

7604-filter-modifiers-v3

12318-case-insensitive-uniqueness

20660-script-load

19724-graphql

14884-script

19720-macaddress-interface-generic-relation

fix-19669-api-image-download

7604-filter-modifiers

19275-fixes-interface-bulk-edit

fix-17794-get_field_value_return_list

11507-show-aggregate-and-rir-on-api

9583-add_column_specific_search_field_to_tables

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/netbox#11424