Ability to post >1 item at a time via the API #1125

Closed
opened 2025-12-29 16:29:14 +01:00 by adam · 4 comments
Owner

Originally created by @shelson on GitHub (Jul 22, 2017).

Currently it seems that I can only post a single item at once via the API. When importing IP addresses or interfaces this is hugely inefficient and slows things down a whole lot. Would it be possible for POST (and maybe PUT/PATCH) to accept lists of json data so that a single API call can create or modify multiple items? Right now network latency has a huge impact on import speed. Thanks!

Originally created by @shelson on GitHub (Jul 22, 2017). Currently it seems that I can only post a single item at once via the API. When importing IP addresses or interfaces this is hugely inefficient and slows things down a whole lot. Would it be possible for POST (and maybe PUT/PATCH) to accept lists of json data so that a single API call can create or modify multiple items? Right now network latency has a huge impact on import speed. Thanks!
adam closed this issue 2025-12-29 16:29:14 +01:00
Author
Owner

@jeremystretch commented on GitHub (Jul 26, 2017):

Not sure how I feel about this. REST APIs traditionally operate on one object at a time. This provides a very predictable workflow. When you start creating or updating multiple objects simultaneously, you have to figure out how to handle validation errors.

For example, suppose a consumer posts 50 objects to be created. It takes several seconds for NetBox to process the entire list, and and object toward the end of the batch fails validation. Do we roll back the creation of the other objects, which results in all that processing time being wasted and requires the entire batch to be resubmitted? Or do we keep those objects and indicate to the consumer which object(s) have failed, requiring them to reconstruct their request (to avoid recreating the same objects) and leaving the data model in an unexpected state? Neither seem like appealing options to me.

I'd rather keep the API simple as it is now and focus on how we might be able to make individual requests more efficient. Could you elaborate on what you're doing with the API and the time it's taking?

@jeremystretch commented on GitHub (Jul 26, 2017): Not sure how I feel about this. REST APIs traditionally operate on one object at a time. This provides a very predictable workflow. When you start creating or updating multiple objects simultaneously, you have to figure out how to handle validation errors. For example, suppose a consumer posts 50 objects to be created. It takes several seconds for NetBox to process the entire list, and and object toward the end of the batch fails validation. Do we roll back the creation of the other objects, which results in all that processing time being wasted and requires the entire batch to be resubmitted? Or do we keep those objects and indicate to the consumer which object(s) have failed, requiring them to reconstruct their request (to avoid recreating the same objects) and leaving the data model in an unexpected state? Neither seem like appealing options to me. I'd rather keep the API simple as it is now and focus on how we might be able to make individual requests more efficient. Could you elaborate on what you're doing with the API and the time it's taking?
Author
Owner

@shelson commented on GitHub (Jul 29, 2017):

Good points well made. I agree that it probably creates more questions than answers, and also opens the door to things getting really complex.

For the use case, I have a script that performs discovery on the network, grabbing interfaces and arp table entries via pyez and then importing/updating netbox with it's findings (update also touches a "last seen" custom field, so that discovered items can be purged if/when they disappear off the network). For us currently this means that we're updating around 10k ipaddress objects one at a time, which just felt really inefficient. It takes about 15-20 minutes for the script to run, and since we only run it every 4 hours right now it's not the end of the world - and it got a lot better when I moved the script to be as close to the netbox install as possible to reduce the network latency from the equation.

I'm happy for this to be dropped.

@shelson commented on GitHub (Jul 29, 2017): Good points well made. I agree that it probably creates more questions than answers, and also opens the door to things getting really complex. For the use case, I have a script that performs discovery on the network, grabbing interfaces and arp table entries via pyez and then importing/updating netbox with it's findings (update also touches a "last seen" custom field, so that discovered items can be purged if/when they disappear off the network). For us currently this means that we're updating around 10k ipaddress objects one at a time, which just felt really inefficient. It takes about 15-20 minutes for the script to run, and since we only run it every 4 hours right now it's not the end of the world - and it got a lot better when I moved the script to be as close to the netbox install as possible to reduce the network latency from the equation. I'm happy for this to be dropped.
Author
Owner

@LBegnaud commented on GitHub (Jul 31, 2017):

@shelson that feels counter to the goal of netbox being the golden image / desired state of the network. We currently use netbox for more solid definitions of the network, and use netdisco + rancid to get the real state of the network.

Netdisco has a configurable scan frequency as well as options for discovering neighbors via LLDP/CDP (and some requests for other methods). Might be worth checking out.

@LBegnaud commented on GitHub (Jul 31, 2017): @shelson that feels counter to the goal of netbox being the golden image / desired state of the network. We currently use netbox for more solid definitions of the network, and use [netdisco ](https://github.com/netdisco/netdisco)+ [rancid ](http://www.shrubbery.net/rancid/) to get the real state of the network. Netdisco has a configurable scan frequency as well as options for discovering neighbors via LLDP/CDP (and some requests for other methods). Might be worth checking out.
Author
Owner

@jeremystretch commented on GitHub (Aug 2, 2017):

I'm going to reject this feature request in the interest of simplicity. It's something we may revisit in the future but I'd like to keep the API simple at least until the project matures more.

@jeremystretch commented on GitHub (Aug 2, 2017): I'm going to reject this feature request in the interest of simplicity. It's something we may revisit in the future but I'd like to keep the API simple at least until the project matures more.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#1125