starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-13 13:53:31 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Conditionally disable automatic prefetches in BaseTable #7333

New Issue
Closed
opened 2025-12-29 20:22:02 +01:00 by adam · 4 comments
adam commented 2025-12-29 20:22:02 +01:00
Owner
Copy Link

Originally created by @amyasnikov on GitHub (Dec 9, 2022).

NetBox version

v3.3.9

Feature type

Change to existing functionality

Proposed functionality

Currently tables.BaseTable.__init__() method does several different things including automatic calls of the prefetch_related() for relation fields.
https://github.com/netbox-community/netbox/blob/v3.3.9/netbox/netbox/tables/tables.py#L77
This behaviour can cause problems for plugin authors who already done manual (more precise) select_related/prefetch_related on the queryset and want to feed this queryset to NetBoxTable (derived from BaseTable).
Also I got the situation when this automatic prefetch in BaseTable accidentally caused excessive DB queries on the page.

So the exact problem is plugin developer's inability to do not execute these auto-prefetches and at the same time reuse the rest of code stored in the BaseTable.__init__() method.

My proposal:
Add another __init__() bool parameter to control the execution of the prefetches. Like this

def __init__(self, *args, user=None, prefetch_columns=True, **kwargs):
...  # rest of the __init__ code here
if prefetch_columns and isinstance(self.data, TableQuerysetData):
  ...  # existing prefetch code here

Or possible alternative: move auto-prefetch part of code into separate method and call it inside __init__(). In this case plugin developer can override this separate method to disable prefetching.

Use case

This will allow plugin developers:

  1. Conditionally disable automatic prefetches of the model fields which can break manual prefetches
  2. Reuse the rest of the BaseTable.__init__() code

Database changes

No response

External dependencies

No response

Originally created by @amyasnikov on GitHub (Dec 9, 2022). ### NetBox version v3.3.9 ### Feature type Change to existing functionality ### Proposed functionality Currently `tables.BaseTable.__init__()` method does several different things including automatic calls of the **prefetch_related()** for relation fields. https://github.com/netbox-community/netbox/blob/v3.3.9/netbox/netbox/tables/tables.py#L77 This behaviour can cause problems for plugin authors who already done manual (more precise) select_related/prefetch_related on the queryset and want to feed this queryset to NetBoxTable (derived from BaseTable). Also I got the situation when this automatic prefetch in BaseTable accidentally caused excessive DB queries on the page. So the exact problem is plugin developer's inability to do not execute these auto-prefetches and at the same time reuse the rest of code stored in the `BaseTable.__init__()` method. My proposal: Add another `__init__()` bool parameter to control the execution of the prefetches. Like this ``` def __init__(self, *args, user=None, prefetch_columns=True, **kwargs): ... # rest of the __init__ code here if prefetch_columns and isinstance(self.data, TableQuerysetData): ... # existing prefetch code here ``` Or possible alternative: move auto-prefetch part of code into separate method and call it inside `__init__()`. In this case plugin developer can override this separate method to disable prefetching. ### Use case This will allow plugin developers: 1. Conditionally disable automatic prefetches of the model fields which can break manual prefetches 2. Reuse the rest of the `BaseTable.__init__()` code ### Database changes _No response_ ### External dependencies _No response_
adam added the type: featurestatus: revisions needed labels 2025-12-29 20:22:02 +01:00
adam closed this issue 2025-12-29 20:22:02 +01:00
adam commented 2025-12-29 20:22:03 +01:00
Author
Owner
Copy Link

@amyasnikov commented on GitHub (Dec 13, 2022):

I can do it by myself if you approve. It's literally 2 lines of code

@amyasnikov commented on GitHub (Dec 13, 2022): I can do it by myself if you approve. It's literally 2 lines of code
adam commented 2025-12-29 20:22:03 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Jan 5, 2023):

The automatic prefetching is done to ensure efficient database queries are done depending on which table columns are selected.

This behaviour can cause problems for plugin authors who already done manual (more precise) select_related/prefetch_related on the queryset and want to feed this queryset to NetBoxTable (derived from BaseTable).

What problems are you encountering, specifically, what would necessitate the removal of dynamic prefetching?

It's literally 2 lines of code

That's twice the number of lines required to induce a critical bug.

@jeremystretch commented on GitHub (Jan 5, 2023): The automatic prefetching is done to ensure efficient database queries are done depending on which table columns are selected. > This behaviour can cause problems for plugin authors who already done manual (more precise) select_related/prefetch_related on the queryset and want to feed this queryset to NetBoxTable (derived from BaseTable). What problems are you encountering, specifically, what would necessitate the removal of dynamic prefetching? > It's literally 2 lines of code That's twice the number of lines required to induce a critical bug.
adam commented 2025-12-29 20:22:04 +01:00
Author
Owner
Copy Link

@amyasnikov commented on GitHub (Jan 5, 2023):

@jeremystretch I faced excessive duplicate db queries (according to django-debug-toolbar) on a page containing NetBoxTable. All my efforts to optimize the queryset were failed. The only thing that helped is to completely replace init method of the table and throw out auto-prefetches. Maybe I can try to reproduce it on a tiny piece of model-table-view demonstration code

@amyasnikov commented on GitHub (Jan 5, 2023): @jeremystretch I faced excessive duplicate db queries (according to django-debug-toolbar) on a page containing NetBoxTable. All my efforts to optimize the queryset were failed. The only thing that helped is to completely replace __init__ method of the table and throw out auto-prefetches. Maybe I can try to reproduce it on a tiny piece of model-table-view demonstration code
adam commented 2025-12-29 20:22:04 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Feb 2, 2023):

I'm going to close this out as no proof of concept has been provided, but if you do come up with something please share it here and I'll be happy to revisit this.

@jeremystretch commented on GitHub (Feb 2, 2023): I'm going to close this out as no proof of concept has been provided, but if you do come up with something please share it here and I'll be happy to revisit this.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label status: revisions needed type: feature
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#7333
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?