Potential race condition when allocating "next available" resources #2849

Closed
opened 2025-12-29 18:22:45 +01:00 by adam · 5 comments
Owner

Originally created by @juan-vg on GitHub (Sep 3, 2019).

Environment

  • Python version: 3.5.3
  • NetBox version: 2.6.2

Steps to Reproduce

  1. Get a prefix ID (is_pool must be true for the prefix)
  2. Run two simultaneous POST requests to the /ipam/prefixes/{id}/available-ips/ endpoint

Expected Behavior

It was also mentioned on the https://github.com/netbox-community/netbox/issues/1246#issuecomment-306878192. There is a critical section when setting an IP address as Active (in use). It must be an atomic operation that respects the mutual exclusion by setting a lock for the (pool) prefix.

Observed Behavior

Both POST requests got the same IP address as answer.

$  curl -X POST "http://netbox/api/ipam/prefixes/123/available-ips/" -H  "accept: application/json" -H  "Content-Type: application/json" -H  "X-CSRFToken: CSRFToken" -d "{}" -H "Authorization: Token AuthToken"
{"id":19746,"family":{"value":4,"label":"IPv4"},"address":"172.16.16.4/24","vrf":null,"tenant":null,"status":{"value":1,"label":"Active"},"role":null,"interface":null,"nat_inside":null,"nat_outside":null,"dns_name":"","description":"","tags":[],"created":"2019-09-03","last_updated":"2019-09-03T14:01:54.216398Z"}


$  curl -X POST "http://netbox/api/ipam/prefixes/123/available-ips/" -H  "accept: application/json" -H  "Content-Type: application/json" -H  "X-CSRFToken: CSRFToken" -d "{}" -H "Authorization: Token AuthToken"
{"id":19745,"family":{"value":4,"label":"IPv4"},"address":"172.16.16.4/24","vrf":null,"tenant":null,"status":{"value":1,"label":"Active"},"role":null,"interface":null,"nat_inside":null,"nat_outside":null,"dns_name":"","description":"","tags":[],"created":"2019-09-03","last_updated":"2019-09-03T14:01:54.212623Z"}

imagen

Originally created by @juan-vg on GitHub (Sep 3, 2019). ### Environment * Python version: 3.5.3 * NetBox version: 2.6.2 ### Steps to Reproduce 1. Get a prefix ID (`is_pool` must be true for the prefix) 2. Run **two simultaneous** POST requests to the `/ipam/prefixes/{id}/available-ips/` endpoint ### Expected Behavior It was also mentioned on the https://github.com/netbox-community/netbox/issues/1246#issuecomment-306878192. There is a critical section when setting an IP address as `Active` (in use). It must be an atomic operation that respects the mutual exclusion by setting a lock for the (pool) prefix. ### Observed Behavior Both POST requests got the same IP address as answer. ```bash $ curl -X POST "http://netbox/api/ipam/prefixes/123/available-ips/" -H "accept: application/json" -H "Content-Type: application/json" -H "X-CSRFToken: CSRFToken" -d "{}" -H "Authorization: Token AuthToken" {"id":19746,"family":{"value":4,"label":"IPv4"},"address":"172.16.16.4/24","vrf":null,"tenant":null,"status":{"value":1,"label":"Active"},"role":null,"interface":null,"nat_inside":null,"nat_outside":null,"dns_name":"","description":"","tags":[],"created":"2019-09-03","last_updated":"2019-09-03T14:01:54.216398Z"} $ curl -X POST "http://netbox/api/ipam/prefixes/123/available-ips/" -H "accept: application/json" -H "Content-Type: application/json" -H "X-CSRFToken: CSRFToken" -d "{}" -H "Authorization: Token AuthToken" {"id":19745,"family":{"value":4,"label":"IPv4"},"address":"172.16.16.4/24","vrf":null,"tenant":null,"status":{"value":1,"label":"Active"},"role":null,"interface":null,"nat_inside":null,"nat_outside":null,"dns_name":"","description":"","tags":[],"created":"2019-09-03","last_updated":"2019-09-03T14:01:54.212623Z"} ``` ![imagen](https://user-images.githubusercontent.com/17771395/64184986-a619a880-ce6c-11e9-9dab-36f82e286d13.png)
adam added the status: duplicate label 2025-12-29 18:22:45 +01:00
adam closed this issue 2025-12-29 18:22:45 +01:00
Author
Owner

@juan-vg commented on GitHub (Sep 3, 2019):

Redis could be used to manage the locks

@juan-vg commented on GitHub (Sep 3, 2019): Redis could be used to manage the locks
Author
Owner

@DanSheps commented on GitHub (Sep 6, 2019):

Do you have a production use-case where this has happened?

@DanSheps commented on GitHub (Sep 6, 2019): Do you have a production use-case where this has happened?
Author
Owner

@juan-vg commented on GitHub (Sep 9, 2019):

TBH no. Before doing the curl tests I've checked the source code of this function and I realized that no mutual exclusion checks were happening. I'm not going to use it in production if I know it could fail.

Moreover, where I want to use this is a distributed system that allocates IP addresses (similar to a DHCP) using netbox as the source of free IPs. I'm currently doing this using phpIPAM, and now I want to migrate the IP management to netbox (aggregating several sources of truth). This system could be running from several sources at the same time and eventually can happen that it could wrongly get the same IP for two (or even more) clients.

I hope that this explanation is enough, but don't hesitate to come back to me if you have any other question :)

@juan-vg commented on GitHub (Sep 9, 2019): TBH no. Before doing the curl tests I've checked the source code of this function and I realized that no mutual exclusion checks were happening. I'm not going to use it in production if I know it could fail. Moreover, where I want to use this is a distributed system that allocates IP addresses (similar to a DHCP) using netbox as the source of free IPs. I'm currently doing this using phpIPAM, and now I want to migrate the IP management to netbox (aggregating several sources of truth). This system could be running from several sources at the same time and eventually can happen that it could wrongly get the same IP for two (or even more) clients. I hope that this explanation is enough, but don't hesitate to come back to me if you have any other question :)
Author
Owner

@juan-vg commented on GitHub (Sep 19, 2019):

Hello!

I have one now! Terraform deploying a VM with two network interfaces on the same subnet queries Netbox in order to get the proper available IP addresses. I get an error because the two addresses are the same.

Obviously I can run it again and it will eventually work, but considering the purpose of this endpoint (get the next available IP) I believe it should work without having to repeat the operation.

Regards!

@juan-vg commented on GitHub (Sep 19, 2019): Hello! I have one now! Terraform deploying a VM with two network interfaces on the same subnet queries Netbox in order to get the proper available IP addresses. I get an error because the two addresses are the same. Obviously I can run it again and it will eventually work, but considering the purpose of this endpoint (get the next **available** IP) I believe it should work without having to repeat the operation. Regards!
Author
Owner

@jeremystretch commented on GitHub (Nov 13, 2019):

Turns out this is a duplicate of #2519

@jeremystretch commented on GitHub (Nov 13, 2019): Turns out this is a duplicate of #2519
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#2849