starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Replace django-mptt #8034

New Issue
Open
opened 2025-12-29 20:31:31 +01:00 by adam · 7 comments
adam commented 2025-12-29 20:31:31 +01:00
Owner
Copy Link

Originally created by @jeremystretch on GitHub (May 10, 2023).

Proposed Changes

Replace usage of django-mptt in NetBox with one of the alternatives listed below:

django-tree-queries

django-tree-queries employs recursive common table expressions (CTE). It was originally proposed in #6587.

django-treebeard

django-treebeard offers several tree implementations, one of which (nested sets) is roughly analogous to django-mptt's implementation.

PotsgreSQL ltree

PostgreSQL's ltree module offers another potential solution. It is employed by django-ltree, however that library appears to no longer be maintained.

Justification

django-mptt is no longer maintained. Additionally, this presents an opportunity to adopt a recursive nesting solution that may prove to be more performant and/or scalable than MPTT. This idea was originally raised in #11421.

Originally created by @jeremystretch on GitHub (May 10, 2023). ### Proposed Changes Replace usage of [django-mptt](https://github.com/django-mptt/django-mptt) in NetBox with one of the alternatives listed below: ### django-tree-queries [django-tree-queries](https://github.com/matthiask/django-tree-queries) employs recursive common table expressions (CTE). It was originally proposed in #6587. ### django-treebeard [django-treebeard](https://github.com/django-treebeard/django-treebeard/) offers several tree implementations, one of which (nested sets) is roughly analogous to django-mptt's implementation. ### PotsgreSQL ltree [PostgreSQL's ltree module](https://www.postgresql.org/docs/12/ltree.html) offers another potential solution. It is employed by [django-ltree](https://github.com/mariocesar/django-ltree), however that library appears to no longer be maintained. ### Justification django-mptt is no longer maintained. Additionally, this presents an opportunity to adopt a recursive nesting solution that may prove to be more performant and/or scalable than MPTT. This idea was originally raised in #11421.
adam added the type: featurenetboxstatus: backlogcomplexity: high labels 2025-12-29 20:31:31 +01:00
adam commented 2025-12-29 20:31:32 +01:00
Author
Owner
Copy Link

@arthanson commented on GitHub (May 16, 2023):

Looked more into replacing MPTT. The issue to cache the number of component types (https://github.com/netbox-community/netbox/issues/6347) I think might remove the need for add_related_count from MPTT which none of the other tree queries support and would be a bit of a pain to implement for CTE. If we don't need add_related_count, this makes it much easier to use any of the tree systems.

  1. postgres ltree does need a postgres extension, which can be added via a migration, but it is an extension. If this is not acceptable then we can rule it out. django-ltree also looks like it isn't actively maintained, but is just a thin wrapper around the postgres stuff.
  2. django-tree-queries and django-treebeard are both well maintained, treebeard has more stars but is much older.
  3. leaning towards django-tree-queries as it is much lighter codebase and doesn't have three different tree structures to complicate the codebase. Otherwise the materialized path in treebeard looks like the best option.

Also, The author of django-tree-queries has a very good post at (https://406.ch/writing/django-tree-queries/) outlining the reasoning behind creating it and how it compares and the issues he had with the other tree libraries.

@arthanson commented on GitHub (May 16, 2023): Looked more into replacing MPTT. The issue to cache the number of component types (https://github.com/netbox-community/netbox/issues/6347) I think might remove the need for add_related_count from MPTT which none of the other tree queries support and would be a bit of a pain to implement for CTE. If we don't need add_related_count, this makes it much easier to use any of the tree systems. 1. postgres ltree does need a postgres extension, which can be added via a migration, but it is an extension. If this is not acceptable then we can rule it out. django-ltree also looks like it isn't actively maintained, but is just a thin wrapper around the postgres stuff. 2. django-tree-queries and django-treebeard are both well maintained, treebeard has more stars but is much older. 3. leaning towards django-tree-queries as it is much lighter codebase and doesn't have three different tree structures to complicate the codebase. Otherwise the materialized path in treebeard looks like the best option. Also, The author of django-tree-queries has a very good post at (https://406.ch/writing/django-tree-queries/) outlining the reasoning behind creating it and how it compares and the issues he had with the other tree libraries.
adam commented 2025-12-29 20:31:32 +01:00
Author
Owner
Copy Link

@DanSheps commented on GitHub (May 18, 2023):

IMO, CTE seems to be the way to go. The lighter code being a big draw. It is somewhat newer but like mentioned is still actively maintained.

@DanSheps commented on GitHub (May 18, 2023): IMO, CTE seems to be the way to go. The lighter code being a big draw. It is somewhat newer but like mentioned is still actively maintained.
adam commented 2025-12-29 20:31:32 +01:00
Author
Owner
Copy Link

@arthanson commented on GitHub (May 22, 2023):

Did a test implementation and now moving more towards treebeard. the issue with django-tree-queries came down to replacing add_related_count as everything else was very straight-forward.

add_related_count is basically doing a count on a subquery, the subquery being the tree of child elements. I.E. to get Regions (which is a tree) with a count of sites, you do an annotation of the count of the select of sites.regions filtering for the tree of the region.

The issue is the tree fields to filter on are not in the database as they are dynamically created by the CTE query, but django-tree-queries doesn't have any way to add these to a subquery. It could be done by either patching django-tree-queries to add this support, or by just doing RAW SQL to do the count in the subquery.

After discussion with Jeremy, would want to avoid the RAW sql if at all possible so looking at treebeard, since that has the tree fields in the database the implementation of add_related_count would be much more straightforward.

For treebeard looking at the materialized-path as that seems to have the best tradeoffs between the implementations.

@arthanson commented on GitHub (May 22, 2023): Did a test implementation and now moving more towards treebeard. the issue with django-tree-queries came down to replacing add_related_count as everything else was very straight-forward. add_related_count is basically doing a count on a subquery, the subquery being the tree of child elements. I.E. to get Regions (which is a tree) with a count of sites, you do an annotation of the count of the select of sites.regions filtering for the tree of the region. The issue is the tree fields to filter on are not in the database as they are dynamically created by the CTE query, but django-tree-queries doesn't have any way to add these to a subquery. It could be done by either patching django-tree-queries to add this support, or by just doing RAW SQL to do the count in the subquery. After discussion with Jeremy, would want to avoid the RAW sql if at all possible so looking at treebeard, since that has the tree fields in the database the implementation of add_related_count would be much more straightforward. For treebeard looking at the materialized-path as that seems to have the best tradeoffs between the implementations.
adam commented 2025-12-29 20:31:32 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Jun 22, 2023):

Blocked by #12759

@jeremystretch commented on GitHub (Jun 22, 2023): Blocked by #12759
adam commented 2025-12-29 20:31:33 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Jun 28, 2023):

This implementation of CTE has proven unviable as it doesn't seem feasible to sort by depth and node name. There are some alternative solutions worth exploring further but I'm going to bump this from v3.6.

@jeremystretch commented on GitHub (Jun 28, 2023): This implementation of CTE has proven unviable as it doesn't seem feasible to sort by depth _and_ node name. There are some alternative solutions worth exploring further but I'm going to bump this from v3.6.
adam commented 2025-12-29 20:31:33 +01:00
Author
Owner
Copy Link

@github-actions[bot] commented on GitHub (Sep 27, 2023):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Do not attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our contributing guide.

@github-actions[bot] commented on GitHub (Sep 27, 2023): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. **Do not** attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our [contributing guide](https://github.com/netbox-community/netbox/blob/develop/CONTRIBUTING.md).
adam commented 2025-12-29 20:31:33 +01:00
Author
Owner
Copy Link

@github-actions[bot] commented on GitHub (Oct 28, 2023):

This issue has been automatically closed due to lack of activity. In an effort to reduce noise, please do not comment any further. Note that the core maintainers may elect to reopen this issue at a later date if deemed necessary.

@github-actions[bot] commented on GitHub (Oct 28, 2023): This issue has been automatically closed due to lack of activity. In an effort to reduce noise, please do not comment any further. Note that the core maintainers may elect to reopen this issue at a later date if deemed necessary.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label complexity: high netbox status: backlog type: feature
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#8034
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?