mirror of
https://github.com/netbox-community/netbox.git
synced 2026-01-11 21:10:29 +01:00
No Branch/Tag Specified
main
update-changelog-comments-docs
feature-removal-issue-type
20911-dropdown
20239-plugin-menu-classes-mutable-state
21097-graphql-id-lookups
feature
fix_module_substitution
20923-dcim-templates
20044-elevation-stuck-lightmode
feature-ip-prefix-link
v4.5-beta1-release
20068-import-moduletype-attrs
20766-fix-german-translation-code-literals
20378-del-script
7604-filter-modifiers-v3
circuit-swap
12318-case-insensitive-uniqueness
20637-improve-device-q-filter
20660-script-load
19724-graphql
20614-update-ruff
14884-script
02496-max-page
19720-macaddress-interface-generic-relation
19408-circuit-terminations-export-templates
20203-openapi-check
fix-19669-api-image-download
7604-filter-modifiers
19275-fixes-interface-bulk-edit
fix-17794-get_field_value_return_list
11507-show-aggregate-and-rir-on-api
9583-add_column_specific_search_field_to_tables
v4.5.0
v4.4.10
v4.4.9
v4.5.0-beta1
v4.4.8
v4.4.7
v4.4.6
v4.4.5
v4.4.4
v4.4.3
v4.4.2
v4.4.1
v4.4.0
v4.3.7
v4.4.0-beta1
v4.3.6
v4.3.5
v4.3.4
v4.3.3
v4.3.2
v4.3.1
v4.3.0
v4.2.9
v4.3.0-beta2
v4.2.8
v4.3.0-beta1
v4.2.7
v4.2.6
v4.2.5
v4.2.4
v4.2.3
v4.2.2
v4.2.1
v4.2.0
v4.1.11
v4.1.10
v4.1.9
v4.1.8
v4.2-beta1
v4.1.7
v4.1.6
v4.1.5
v4.1.4
v4.1.3
v4.1.2
v4.1.1
v4.1.0
v4.0.11
v4.0.10
v4.0.9
v4.1-beta1
v4.0.8
v4.0.7
v4.0.6
v4.0.5
v4.0.3
v4.0.2
v4.0.1
v4.0.0
v3.7.8
v3.7.7
v4.0-beta2
v3.7.6
v3.7.5
v4.0-beta1
v3.7.4
v3.7.3
v3.7.2
v3.7.1
v3.7.0
v3.6.9
v3.6.8
v3.6.7
v3.7-beta1
v3.6.6
v3.6.5
v3.6.4
v3.6.3
v3.6.2
v3.6.1
v3.6.0
v3.5.9
v3.6-beta2
v3.5.8
v3.6-beta1
v3.5.7
v3.5.6
v3.5.5
v3.5.4
v3.5.3
v3.5.2
v3.5.1
v3.5.0
v3.4.10
v3.4.9
v3.5-beta2
v3.4.8
v3.5-beta1
v3.4.7
v3.4.6
v3.4.5
v3.4.4
v3.4.3
v3.4.2
v3.4.1
v3.4.0
v3.3.10
v3.3.9
v3.4-beta1
v3.3.8
v3.3.7
v3.3.6
v3.3.5
v3.3.4
v3.3.3
v3.3.2
v3.3.1
v3.3.0
v3.2.9
v3.2.8
v3.3-beta2
v3.2.7
v3.3-beta1
v3.2.6
v3.2.5
v3.2.4
v3.2.3
v3.2.2
v3.2.1
v3.2.0
v3.1.11
v3.1.10
v3.2-beta2
v3.1.9
v3.2-beta1
v3.1.8
v3.1.7
v3.1.6
v3.1.5
v3.1.4
v3.1.3
v3.1.2
v3.1.1
v3.1.0
v3.0.12
v3.0.11
v3.0.10
v3.1-beta1
v3.0.9
v3.0.8
v3.0.7
v3.0.6
v3.0.5
v3.0.4
v3.0.3
v3.0.2
v3.0.1
v3.0.0
v2.11.12
v3.0-beta2
v2.11.11
v2.11.10
v3.0-beta1
v2.11.9
v2.11.8
v2.11.7
v2.11.6
v2.11.5
v2.11.4
v2.11.3
v2.11.2
v2.11.1
v2.11.0
v2.10.10
v2.10.9
v2.11-beta1
v2.10.8
v2.10.7
v2.10.6
v2.10.5
v2.10.4
v2.10.3
v2.10.2
v2.10.1
v2.10.0
v2.9.11
v2.10-beta2
v2.9.10
v2.10-beta1
v2.9.9
v2.9.8
v2.9.7
v2.9.6
v2.9.5
v2.9.4
v2.9.3
v2.9.2
v2.9.1
v2.9.0
v2.9-beta2
v2.8.9
v2.9-beta1
v2.8.8
v2.8.7
v2.8.6
v2.8.5
v2.8.4
v2.8.3
v2.8.2
v2.8.1
v2.8.0
v2.7.12
v2.7.11
v2.7.10
v2.7.9
v2.7.8
v2.7.7
v2.7.6
v2.7.5
v2.7.4
v2.7.3
v2.7.2
v2.7.1
v2.7.0
v2.6.12
v2.6.11
v2.6.10
v2.6.9
v2.7-beta1
Solcon-2020-01-06
v2.6.8
v2.6.7
v2.6.6
v2.6.5
v2.6.4
v2.6.3
v2.6.2
v2.6.1
v2.6.0
v2.5.13
v2.5.12
v2.6-beta1
v2.5.11
v2.5.10
v2.5.9
v2.5.8
v2.5.7
v2.5.6
v2.5.5
v2.5.4
v2.5.3
v2.5.2
v2.5.1
v2.5.0
v2.4.9
v2.5-beta2
v2.4.8
v2.5-beta1
v2.4.7
v2.4.6
v2.4.5
v2.4.4
v2.4.3
v2.4.2
v2.4.1
v2.4.0
v2.3.7
v2.4-beta1
v2.3.6
v2.3.5
v2.3.4
v2.3.3
v2.3.2
v2.3.1
v2.3.0
v2.2.10
v2.3-beta2
v2.2.9
v2.3-beta1
v2.2.8
v2.2.7
v2.2.6
v2.2.5
v2.2.4
v2.2.3
v2.2.2
v2.2.1
v2.2.0
v2.1.6
v2.2-beta2
v2.1.5
v2.2-beta1
v2.1.4
v2.1.3
v2.1.2
v2.1.1
v2.1.0
v2.0.10
v2.1-beta1
v2.0.9
v2.0.8
v2.0.7
v2.0.6
v2.0.5
v2.0.4
v2.0.3
v2.0.2
v2.0.1
v2.0.0
v2.0-beta3
v1.9.6
v1.9.5
v2.0-beta2
v1.9.4-r1
v1.9.3
v2.0-beta1
v1.9.2
v1.9.1
v1.9.0-r1
v1.8.4
v1.8.3
v1.8.2
v1.8.1
v1.8.0
v1.7.3
v1.7.2-r1
v1.7.1
v1.7.0
v1.6.3
v1.6.2-r1
v1.6.1-r1
1.6.1
v1.6.0
v1.5.2
v1.5.1
v1.5.0
v1.4.2
v1.4.1
v1.4.0
v1.3.2
v1.3.1
v1.3.0
v1.2.2
v1.2.1
v1.2.0
v1.1.0
v1.0.7-r1
v1.0.7
v1.0.6
v1.0.5
v1.0.4
v1.0.3-r1
v1.0.3
1.0.0
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
Mirrored from GitHub Pull Request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: starred/netbox#1655
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @cimnine on GitHub (Apr 5, 2018).
Originally assigned to: @jeremystretch on GitHub.
Issue type
[X] Feature request
[ ] Bug report
[ ] Documentation
Environment
Description
I've talked about it a while back, but it came up in a different context in the networktocode#netbox slack channel recently.
We've implemented a Netbox integration for GraphQL a while back, but we're not able to maintain it at the moment. It's called django-netbox-graphql. Right now, it's built as a django module that can be plugged into Netbox quite easily. So it should be straight-forward to get going.
Some features:
Whats the case for GraphQL?
GraphQL is a "new" way to consume from a remote system. In contrast to REST, where there is a route for every resource, GraphQL only knows one route. The request to that route contains a query in a well-defined format. In the query the requesting application specifies which resources it would like to consume and, and this is essential, which objects it would like to have nested.
A short example
I would like to get all IPs of pool subnets and their corresponding MAC addresses to write them into a DHCP config file.
With REST
interfaceobject. But I also get a lot of unrelated information.With GraphQL
I would have to write one query that returns exactly the fields I need. It would look like this:
(Note: This query does not work yet with django-netbox-graphql!)
Conclusion
I understand, that this feature is no priority for Netbox. Also, the Netbox REST API is very powerful and already nests a lot of important information. But I also think that people want Netbox to become their single-source-of-truth for all kinds of information related to devices on their network. A versatile query language, that offers full flexibility and reduces requests to Netbox might be required in such a future.
@jeremystretch commented on GitHub (Apr 12, 2018):
I'd also like to explore this at some point, but I have to close this out as our current policy does not allow for any new major feature requests until the backlog of existing requests has been worked through.
@cimnine commented on GitHub (Oct 5, 2018):
We've started an integration of GraphQL into Netbox a while back. It worked great for all implemented queries. Unfortunately we can't maintain it anymore, because we never got to use it. But it might still serve as reference should someone else ever see the huge benefit over static REST APIs and decide to either maintain the project in a fork or integrate it into Netbox.
https://github.com/ninech/django-netbox-graphql/
@mraerino commented on GitHub (Jun 8, 2019):
@jeremystretch at what point is it possible to reopen this issue?
@jeremystretch commented on GitHub (May 21, 2021):
Re-opening this as a candidate for v3.0.
@ryanmerolle commented on GitHub (May 21, 2021):
I think MVP release for GraphQL should focus on READ / GET only since that is where most people have headaches with REST / post GET data manipulations.
Sure there would be some interesting write/updates using GraphQL, but I am doubtful the ROI is as high. It also allows you to iterate on the integration with a limited scope.
@ziggekatten commented on GitHub (May 21, 2021):
Agree with ryanmerolle. Biggest gains are by far the GET requests.
@candlerb commented on GitHub (May 29, 2021):
There are some important things which are impossible via REST at the moment, and graphQL might be able to offer a solution. In particular: add a device (or VM) + interface + primary IP address in one transaction. Currently there's a minimum of 3 creates plus 1 update to achieve this.
@jeremystretch commented on GitHub (May 29, 2021):
Yes, and it makes perfect sense to keep those as discrete actions, because each is doing something very different. Each object has its own validation logic and implications. Trying to bundle them all together increases the time and memory needed to perform the request. It also greatly complicates the reporting of errors, sending webhooks, etc.
@jeremystretch commented on GitHub (May 29, 2021):
It's also worth pointing out that creating ten devices with a few dozen interfaces each and an IP address on each interface via the REST API also takes four total requests.
@jeremystretch commented on GitHub (Jun 7, 2021):
I've been experimenting with this for the past few days, and I'm happy to say I've made a surprising amount of progress in a fairly short time. The traditional approach to implementing GraphQL in a Django application requires a fair bit of overhead in defining object types, much of which is redundant to what we've already done with Django REST framework for the REST API.
To help reduce developer burden and help ensure long-term maintainability, I've oped to employ graph_wrap by @PaulGilmartin to leverage our DRF serializers to serve GraphQL requests. Paul has been extremely helpful in working through some initial bumps, and I've already got a functional PoC going.
As you're likely aware, NetBox today answers REST API requests in one of two modes: brief and normal. A brief response includes only the bare minimum representation of each obect (typically its ID, name, and a user-friendly display string), whereas the normal response includes a complete representation, exclusive of downstream related objects. For example, a rack will show its assigned site, but not associated devices within the rack.
graph_wrap, however, requires a complete serialization of each object to support the inclusion of downstream related objects. I can think of two approaches to meeting this requirement:
The first approach may or may not be practical, given the intricacies of serializer initialization, however its at least worth exploring briefly. The second approach I have already confirmed as a potential solution; the only downside is needing to declare separate serializer classes. However, this may actually be more manageable in the long term, and easier to test.
There's an added benefit of either approach: it will potentially enable "full" REST API requests. For example, a request to
/api/dcim/intefaces/123/?mode=fullmight include all assigned IP addresses. We would likely need to figure out some way to dynamically optimize the underlying queries to make this approach feasible for real-world needs, but it's a possibility.Just wanted to get my thoughts down for now. Will provide further updates when I have more to share.
@PaulGilmartin commented on GitHub (Jun 7, 2021):
@jeremystretch Happy to help!
If you get time, I'd be interested if you could be a bit more explicit about this statement "graph_wrap, however, requires a complete serialization of each object to support the inclusion of downstream related objects.". Perhaps you could provide an example of the kind of code you're talking about?
@jeremystretch commented on GitHub (Jun 7, 2021):
@PaulGilmartin sure thing! I probably shouldn't have said graph_wrap, specifically, but rather GraphQL in general, requires this. Here's an example.
In NetBox, we model circuits and providers; every circuit has a provider, and a provider may have one or more circuits. Here's what our ProviderSerializer current looks like:
7444110c79/netbox/circuits/api/serializers.py (L19-L28)You'll notice that we include a count of child circuits with the provider, but not the circuits themselves. This precludes GraphQL from querying related circuits. To support GraphQL (via graph_wrap), we need to extend our serializers to incorporate child objects. In this example, we need to add something like
and update
Meta.filedsaccordingly. But we don't normally want to include these related objects in REST responses, because the performance penalty can be severe (e.g. imagine we have a thousand circuits defined per provider). So the crux of the matter for me at the moment is deciding how to control this; my two options cited above.I find myself leaning more toward the second option, if only because working with a single serializer per model is going to make it very difficult (likely impossible) to avoid cyclical imports. For example, this obviously won't work:
But creating a new "full" serializer by subclassing the existing serializer should:
@jeremystretch commented on GitHub (Jun 7, 2021):
Thinking this through a bit further, I think we'll run into an issue in the reverse direction: traversing upstream related objects. For example: Interface -> Device -> Rack -> Site -> Region -> Region -> Region. It may just not be worth pursuing the parallel to DRF any further, owing to the complexity of our data model.
@PaulGilmartin commented on GitHub (Jun 7, 2021):
@jeremystretch Perhaps it's getting a bit late here and I'm not completely following, but I believe the problem you're describing is very similar to the one I discuss here https://github.com/PaulGilmartin/graph_wrap#which-problems-does-graphwrap-address. If so, it shouldn't be the case that you need to embed the
CircuitSerializerinside the ProviderSerializergraph_wrap should be smart enough to get the circuits_type purely from a HyperlinkedIdentityField (as we describe in the link above with the Authors/Posts example). So you could have something like
and that should give you a full representation of the nested circuits object via /graphql but not via the REST endpoint (providing circuit-detail does point to some sort of appropriate CircuitSerializier). Have you tried this at all?
@jeremystretch commented on GitHub (Jun 7, 2021):
I did give that a shot early on, but it seems to just load the related objects as strings:
Even if it was able to generate the serializer dynamically, I'm not sure we'd want to give up the control over the displayed object.
@PaulGilmartin commented on GitHub (Jun 7, 2021):
Not sure why that error came about, but it wouldn't be generating a serializer dynamically. What it does is fetches the serializer associated to the view_name. So for example if we wrote
then the view_name 'circuit-detail' must be associated to a pre-defined
CircuitSerializer. This would mean you don't need to explicitly embed it in theProviderSerializer(which I understood to be your main concern).I'd be happy to look at the code which gave the error you mentioned if you want to explore this further.
@jeremystretch commented on GitHub (Jun 7, 2021):
Ah, I see. For whatever reason that doesn't seem to be working at the moment. Ok, it may be worth coming back to that.
However, I think the bigger issue is figuring out how to navigate upstream relationships. Consider this model:
Our current InterfaceSerializer defines its
deviceas a nested serializer, traversing only one level up in the hierarchy. We want to keep this as-is in the REST API serializer. So to support GraphQL, we would need - as far as I can tell - to introduce a separate serializer which definesdeviceas a full serializer, which in turn defines itsrackas a full serializer, and so on. Is that right, or am I missing a more elegant solution?@PaulGilmartin commented on GitHub (Jun 8, 2021):
@jeremystretch My first point would be that to support GraphQL, we don't necessarily need to have available full representations of every object to the client. GraphQL can still be of great use when only the limited scope of nested objects is ever exposed (as you do with NestedXXXSerializers).
If however your aim of using GraphQL is to be able to expose full representations of nested objects, then I think doing something as I described above wouldn't be too much effort. Let's continue with the interface-device example above.
Let's just suppose that the REST API exposes interfaces at /interface/{id}. Currently that's modelled like this:
So a request to fetch an Interface would give JSON of the form
Now, I see in the codebase that what I think you describe as a "full" serializer already exists for Device:
What we want :
DeviceSerializerdirectly insideInterfaceSerializerso that requests to the REST API are not cluttered with the full device representation.My claim is that this is possible, provided your
DeviceSerializeris already exposed via your REST API. Let's assume it is for now, at the URL /device/{id} and corresponding view name device-detail. Then if you add the following field toInterfaceSerializerthen graph_wrap should (i.e. it's a bug if it can't) be able to fetch the full representation of the nested device just from the fact the device url is exposed. Importantly, graph_wrap would make use of the existing
DeviceSerializerto generate this full representation. Moreover, adding this device_2 field does not expose the full representation to /interface/id - it simply exposes the nested device URL.So what if
DeviceSerializeris not already exposed via your REST API. Then I would claim that it's perhaps inappropriate to expose it on the /graphql API and we should stick with just exposing the limitedNestedDeviceSerializerrepresentation there too.@jeremystretch commented on GitHub (Jun 8, 2021):
Agreed, however I think the expectation from our user base is the ability to fully traverse these relationships.
Just confirming that this is a correct assumption.
This is very helpful to know, thanks! I'm going to continue experimenting with this a bit more as time allows and see what I can work out. Thanks again for all your help!
@jeremystretch commented on GitHub (Jun 17, 2021):
After a lot more exploration, I've decided to push forward just using graphene_django for this implementation. I have a very rough PoC going int the
2007-graphqlbranch, which is currently limited to thecircuitsapp. I still need to work out support for custom fields, tests, documentation, etc. Once that's all in place, we can extend it to the entire application.It would be extremely helpful if some of the folks who have expressed interested in GraphQL would take this opportunity to help test. As with every major release, we'll hold a beta period, but the sooner we can act on proposed modifications to the implementation, the better off we'll be.
@jeremystretch commented on GitHub (Jun 25, 2021):
One consideration that's come up is whether we should namespace the GraphQL entry points with the application name for each model: for example,
dcim_deviceinstead of justdevice. This isn't strictly necessary today as we don't have any models with overlapping names, however this is a future possibility. Take ipam.Role and dcim.DeviceRole, for instance. We've discussed in the past renaming the later to dcim.Role, which would introduce a conflict (though ultimately decided to leave it as-is).I don't have a strong preference either way, though I want to avoid needing to substantially redefine the GraphQL schema in a future release. Curious to hear any opinions on the matter.
@jeremystretch commented on GitHub (Jun 25, 2021):
Another issue I've run into: Initially, I planned to use the singular form of a model's verbose name to fetch a single object, and the plural form to fetch a list (e.g.
site(id:123)andsites(status:"active"). However, this breaks down when we encounter a model with identical singular and plural names, namely dcim.VirtualChassis.We can work around this by perhaps appending
_listto the model name instead, but I'm not sure what convention is here. I don't have a strong preference, however I do want to avoid any corner case hacks. For example, if we opt to use the_listapproach, we do so for all models, not just virtual chassis.@sdktr commented on GitHub (Jun 30, 2021):
Are you referring to how naming is represented in the schema, or internally within netbox?
If external/schema, I wouldn't grasp the need to expose singles. If a single site is requested it can be signaled in the qcl query?
@jeremystretch commented on GitHub (Jun 30, 2021):
@sdktr I think there's a lot of value in providing an interface to grab specific objects, just as with the REST API. It covers the (fairly frequent, I'd expect) case where the client expects to receive exactly one object, and that zero or 2+ objects should result in an error. It saves the client from having to implement that logic locally.
@jeremystretch commented on GitHub (Jun 30, 2021):
I ended up just appending
_listto the query name for list fields. Curious to see the feedback during the v3.0 beta, but it suffices for now at least.