Add Virtual Machine Types #4557

@jeremystretch commented on GitHub (Feb 12, 2021):

But your use case

and would like to track the EC2 instance type

specifically entails clusters. I think it's fair to require users who need to track VM types to define clusters for this purpose.

@jeremystretch commented on GitHub (Feb 12, 2021): But your use case > and would like to track the EC2 instance type specifically entails clusters. I think it's fair to require users who need to track VM types to define clusters for this purpose.

adam commented

@sdktr commented on GitHub (Feb 12, 2021):

Besides instance specs (fixed cpu/mem assignments are great examples of pre-filled VMTypes imo) there is the interface definitions as well. I'd like to dictate/prefill what users should have as their interface name(s) for a VM.

So a VM Type would contain the interface with name 'vNIC0', a tag and 2G ram and 1vcpu as an example. This can be applied to cluster types esx, hyperv, aws to administer the underlying hypervisor cluster.0

@sdktr commented on GitHub (Feb 12, 2021): Besides instance specs (fixed cpu/mem assignments are great examples of pre-filled VMTypes imo) there is the interface definitions as well. I'd like to dictate/prefill what users should have as their interface name(s) for a VM. So a VM Type would contain the interface with name 'vNIC0', a tag and 2G ram and 1vcpu as an example. This can be applied to cluster types esx, hyperv, aws to administer the underlying hypervisor cluster.0

adam commented

@jrbeilke commented on GitHub (Feb 15, 2021):

But your use case

and would like to track the EC2 instance type

specifically entails clusters. I think it's fair to require users who need to track VM types to define clusters for this purpose.

Hmm maybe there's a better way to organize things, but I'm not sure tracking instance type with separate clusters makes much sense.

ie. with clusters for app1-frontend, app1-backend, etc. then I would need to fragment existing clusters further to track instance type into say app1-frontend-ec2-t3-small or app1-ec2-t3-small-frontend

I'm also not sure how to easily track this for clusters with mixed instance types (ie. while in the process of migrating a cluster between instance types, or scaled up instance types added to a cluster for load balancing). Seems like this would make it difficult to track instance types used between clusters as well for tracking cumulative instance types used.

The more I think about this it seems like this would be useful for other vm providers too ie. if I'm running a vmware cluster I may have pre-defined templates with certain resources or configuration types for small/medium/large vm's and I wouldn't necessarily have separate clusters for each of those.

@jrbeilke commented on GitHub (Feb 15, 2021): > But your use case > > > and would like to track the EC2 instance type > > specifically entails clusters. I think it's fair to require users who need to track VM types to define clusters for this purpose. Hmm maybe there's a better way to organize things, but I'm not sure tracking instance type with separate clusters makes much sense. ie. with clusters for `app1-frontend`, `app1-backend`, etc. then I would need to fragment existing clusters further to track instance type into say `app1-frontend-ec2-t3-small` or `app1-ec2-t3-small-frontend` I'm also not sure how to easily track this for clusters with mixed instance types (ie. while in the process of migrating a cluster between instance types, or scaled up instance types added to a cluster for load balancing). Seems like this would make it difficult to track instance types used between clusters as well for tracking cumulative instance types used. The more I think about this it seems like this would be useful for other vm providers too ie. if I'm running a vmware cluster I may have pre-defined templates with certain resources or configuration types for small/medium/large vm's and I wouldn't necessarily have separate clusters for each of those.

adam commented

@xorrkaz commented on GitHub (Mar 26, 2021):

In my case, I have a number of virtual appliances (e.g., DNS appliances) that all have the same basic interface structure, vCPU, mem, etc. I'd like to have a VM template (like a device type is today) that I can use to initially drive the object's creation. For now, we use tags and roles for some of this, but it does nothing when it comes to inheriting interfaces and VM properties.

@xorrkaz commented on GitHub (Mar 26, 2021): In my case, I have a number of virtual appliances (e.g., DNS appliances) that all have the same basic interface structure, vCPU, mem, etc. I'd like to have a VM template (like a device type is today) that I can use to initially drive the object's creation. For now, we use tags and roles for some of this, but it does nothing when it comes to inheriting interfaces and VM properties.

adam commented

@jeremystretch commented on GitHub (Apr 13, 2021):

From the comments here, I think we're conflating two different functions of device types:

Indicating the device manufacturer and model.
Automatically populating components on newly-created devices.

Re-purposed for VMs, the first use case would store the VM instance classification (e.g. t2.small or t3.large). The second function would automatically create interfaces on a new VM. I don't think it makes sense to combine these two functions, because you would likely not restrict every VM of a certain classification to have the same exact interfaces.

Re-reading @jrbeilke's stated use case above it sounds like this FR is just focusing on the classification of VMs. Is that accurate?

If so, I think it could make sense to introduce a VirtualMachineType class to define these classifications. A new VM could inherit its CPU, memory, and disk attributes automatically upon creation, and VMs could be filtered by their assigned type. However, I'm not sure that it makes sense to templatize VM interfaces as well.

@jeremystretch commented on GitHub (Apr 13, 2021): From the comments here, I think we're conflating two different functions of device types: 1. Indicating the device manufacturer and model. 2. Automatically populating components on newly-created devices. Re-purposed for VMs, the first use case would store the VM instance classification (e.g. `t2.small` or `t3.large`). The second function would automatically create interfaces on a new VM. I don't think it makes sense to combine these two functions, because you would likely not restrict every VM of a certain classification to have the same exact interfaces. Re-reading @jrbeilke's stated use case above it sounds like this FR is just focusing on the classification of VMs. Is that accurate? If so, I think it could make sense to introduce a VirtualMachineType class to define these classifications. A new VM could inherit its CPU, memory, and disk attributes automatically upon creation, and VMs could be filtered by their assigned type. However, I'm not sure that it makes sense to templatize VM interfaces as well.

adam commented

@xorrkaz commented on GitHub (Apr 13, 2021):

In my case, I have virtual appliances that I spin up. They will always have the same interfaces, and I'd like to have them pre-populated based on the type template. So I do have, what I think, is a valid reason to have VMs of a certain type to have the same interfaces (and other "hardware" parameters like you state here).

@xorrkaz commented on GitHub (Apr 13, 2021): In my case, I have virtual appliances that I spin up. They will always have the same interfaces, and I'd like to have them pre-populated based on the type template. So I do have, what I think, is a valid reason to have VMs of a certain type to have the same interfaces (and other "hardware" parameters like you state here).

adam commented

@jeremystretch commented on GitHub (Apr 13, 2021):

@xorrkaz What I'm getting at is that e.g. the t3.large type is decoupled from the "VM with interfaces X, Y, and Z" definition: You'll very likely have different iterations of t3.large with different interfaces defined on them. Thus, I don't think it makes sense to overload a VirtualMachineType model to inform interface creation in the way we do DeviceType.

@jeremystretch commented on GitHub (Apr 13, 2021): @xorrkaz What I'm getting at is that e.g. the `t3.large` type is decoupled from the "VM with interfaces X, Y, and Z" definition: You'll very likely have different iterations of `t3.large` with different interfaces defined on them. Thus, I don't think it makes sense to overload a VirtualMachineType model to inform interface creation in the way we do DeviceType.

adam commented

@xorrkaz commented on GitHub (Apr 13, 2021):

Yes, in that case, I agree. But the ability to create VMs from a template/type still has some value. I guess you're suggesting another issue for that?

@xorrkaz commented on GitHub (Apr 13, 2021): Yes, in that case, I agree. But the ability to create VMs from a template/type still has some value. I guess you're suggesting another issue for that?

adam commented

@jeremystretch commented on GitHub (Apr 13, 2021):

I'd suggest defining a custom script to provision VMs in that manner. That way, you're free to automatically provision interfaces, IP addresses, VLANs, and whatever else you may need.

@jeremystretch commented on GitHub (Apr 13, 2021): I'd suggest defining a [custom script](https://netbox.readthedocs.io/en/stable/additional-features/custom-scripts/) to provision VMs in that manner. That way, you're free to automatically provision interfaces, IP addresses, VLANs, and whatever else you may need.

adam commented

@xorrkaz commented on GitHub (Apr 13, 2021):

Yeah, we're working on a custom plugin to do device/VM adds. This would just be another layer of convenience.

@xorrkaz commented on GitHub (Apr 13, 2021): Yeah, we're working on a custom plugin to do device/VM adds. This would just be another layer of convenience.

adam commented

@candlerb commented on GitHub (May 15, 2021):

Thought 1: as of Netbox v2.11, custom fields can now be enabled in listing columns. This means that a custom field may now be a much more usable way to store attributes such as instance type (e.g. "t2.small").

Thought 2: I don't like the idea of auto-inheriting initial values for vCPUs and memory from a "VirtualMachineType". It means that if the type is changed (e.g. "t2.small" to "t2.large"), the CPU and memory values reported will be inconsistent with it. And there's no sensible default for disk space anyway.

Thought 3: Templating of VM interfaces would be useful - or at least, creating one initial interface automatically, since a VM with no interfaces is not very useful. Maybe just an initial interface in the /virtualization/virtual-machines/add/ page.

For creating VMs via a custom script, there is a sample here.

@candlerb commented on GitHub (May 15, 2021): Thought 1: as of Netbox v2.11, custom fields can now be enabled in listing columns. This means that a custom field may now be a much more usable way to store attributes such as instance type (e.g. "t2.small"). Thought 2: I don't like the idea of auto-inheriting *initial* values for vCPUs and memory from a "VirtualMachineType". It means that if the type is changed (e.g. "t2.small" to "t2.large"), the CPU and memory values reported will be inconsistent with it. And there's no sensible default for disk space anyway. Thought 3: Templating of VM interfaces would be useful - or at least, creating one initial interface automatically, since a VM with no interfaces is not very useful. Maybe just an initial interface in the `/virtualization/virtual-machines/add/` page. For creating VMs via a custom script, there is a sample [here](https://github.com/netbox-community/reports/blob/master/scripts/create_vm.py).

adam commented

@dkirhlarov commented on GitHub (Feb 8, 2022):

NetBox version 3.1.0

We are using VM's with CEPH store or local store in our own OpenNebula clouds. For now we have to use custom field for separate store types. We would like to use VM types or inventory items for VMs like as Device.

@dkirhlarov commented on GitHub (Feb 8, 2022): NetBox version 3.1.0 We are using VM's with CEPH store or local store in our own OpenNebula clouds. For now we have to use custom field for separate store types. We would like to use VM types or inventory items for VMs like as Device.

adam commented

@emersonfelipesp commented on GitHub (Apr 7, 2022):

Virtual Machine Type that would be pretty handy at least for my environment is:

Qemu
LXC
Docker

About the Docker and LXC types, I think we could have Containers views, like virtualization/containers/.
On my Proxmox Plugin, It was always weird to me having to register LXC and Docker containers as Virtual Machines on Netbox.

@emersonfelipesp commented on GitHub (Apr 7, 2022): Virtual Machine Type that would be pretty handy at least for my environment is: - Qemu - LXC - Docker About the Docker and LXC types, I think we could have Containers views, like **virtualization/containers/**. On my [Proxmox Plugin](https://github.com/netdevopsbr/netbox-proxbox), It was always weird to me having to register LXC and Docker containers as Virtual Machines on Netbox.

adam commented

@ZPrimed commented on GitHub (Jun 3, 2022):

Virtual Machine Type that would be pretty handy at least for my environment is:

Qemu

LXC

Docker

About the Docker and LXC types, I think we could have Containers views, like virtualization/containers/. On my Proxmox Plugin, It was always weird to me having to register LXC and Docker containers as Virtual Machines on Netbox.

I feel like tracking containers is more of a separate FR on its own. I do agree that tracking containers as a separate sort of thing would make sense, because containers don't really have number of vCPUs to care about (I guess they might have a RAM value if you can limit the consumption of a container?), so putting them in as "VMs" doesn't really seem logical.

I can definitely see the value of tracking a "Type" of VM when using Cloud systems like AWS / Azure which have clearly defined names for each "size and flavor" of VM. But, this field needs to be optional, and ideally there'd be a way to hide it, because it serves no real purpose for most people with an "on-prem" cluster like Proxmox/Nutanix/Hyper-V... what would we even put there?

@ZPrimed commented on GitHub (Jun 3, 2022): > Virtual Machine Type that would be pretty handy at least for my environment is: > > * Qemu > * LXC > * Docker > > About the Docker and LXC types, I think we could have Containers views, like **virtualization/containers/**. On my [Proxmox Plugin](https://github.com/netdevopsbr/netbox-proxbox), It was always weird to me having to register LXC and Docker containers as Virtual Machines on Netbox. I feel like tracking containers is more of a separate FR on its own. I do agree that tracking containers as a separate sort of thing would make sense, because containers don't really have number of vCPUs to care about (I guess they *might* have a RAM value if you can limit the consumption of a container?), so putting them in as "VMs" doesn't really seem logical. I can definitely see the value of tracking a "Type" of VM when using Cloud systems like AWS / Azure which have clearly defined names for each "size and flavor" of VM. But, this field needs to be optional, and ideally there'd be a way to hide it, because it serves no real purpose for most people with an "on-prem" cluster like Proxmox/Nutanix/Hyper-V... what would we even put there?

adam commented

@candlerb commented on GitHub (Jun 3, 2022):

containers don't really have number of vCPUs to care about

Sure they do: you can limit the number of vCPUs in a cgroup - see e.g. here (as well as RAM, as you observe).

But I agree that instance types are not relevant non-cloud environments.

@candlerb commented on GitHub (Jun 3, 2022): > containers don't really have number of vCPUs to care about Sure they do: you can limit the number of vCPUs in a cgroup - see e.g. [here](https://discuss.linuxcontainers.org/t/using-limits-cpu-allowance-will-reduce-the-amount-of-cores-to-1/5499) (as well as RAM, as you observe). But I agree that instance types are not relevant non-cloud environments.

adam commented

@ZPrimed commented on GitHub (Jun 3, 2022):

containers don't really have number of vCPUs to care about

Sure they do: you can limit the number of vCPUs in a cgroup - see e.g. here (as well as RAM, as you observe).

🤐 My almost total inexperience with containers is showing... I support one Docker app but I do nothing other than install it and let it run 😛 Much more of a "full-on" VM person thus far. I appreciate the education!

@ZPrimed commented on GitHub (Jun 3, 2022): > > containers don't really have number of vCPUs to care about > > Sure they do: you can limit the number of vCPUs in a cgroup - see e.g. [here](https://discuss.linuxcontainers.org/t/using-limits-cpu-allowance-will-reduce-the-amount-of-cores-to-1/5499) (as well as RAM, as you observe). 🤐 My almost total inexperience with containers is showing... I support one Docker app but I do nothing other than install it and let it run 😛 Much more of a "full-on" VM person thus far. I appreciate the education!

adam commented

@jeremystretch commented on GitHub (Jun 20, 2022):

I'd like to spend some more time digging into the two related but distinct use cases I cited above, to ensure that whatever approach we take here doesn't preclude a natural implementation of the remaining functionality in the future.

@jeremystretch commented on GitHub (Jun 20, 2022): I'd like to spend some more time digging into the two related but distinct use cases I cited [above](https://github.com/netbox-community/netbox/issues/5795#issuecomment-818987390), to ensure that whatever approach we take here doesn't preclude a natural implementation of the remaining functionality in the future.

adam commented

@Slepwin commented on GitHub (Sep 2, 2022):

VM type is also would be useful for virtual appliances like load balancers, firewall, etc All this appliances have multiple interfaces, ip addresses, etc.

@Slepwin commented on GitHub (Sep 2, 2022): VM type is also would be useful for virtual appliances like load balancers, firewall, etc All this appliances have multiple interfaces, ip addresses, etc.

adam commented

@jeremystretch commented on GitHub (Oct 17, 2022):

We're going to punt on this for v3.4 as it doesn't seem quite fully baked yet. It might help to open a complementary FR to separate out the two discrete use cases I call out above.

@jeremystretch commented on GitHub (Oct 17, 2022): We're going to punt on this for v3.4 as it doesn't seem quite fully baked yet. It might help to open a complementary FR to separate out the two discrete use cases I call out [above](https://github.com/netbox-community/netbox/issues/5795#issuecomment-818987390).

adam commented

@tomasz-c commented on GitHub (Mar 21, 2023):

With virtual machines (or rather containers) I have another problem.
Let's say we have a cluster C1 in it we have devices D1 and D2 o now I would like to be able to add V1 (Active) on on C1/D1 and V1 (Offline) C1/D2. Currently Netbox does not allow this because the VM name must be unique for the entire cluster.

This option would be useful, as it allows you to quickly determine where you have what disabled containers if they are migrated between devices.

In fact, for me it would be sufficient to be able to assign a container directly to a device, (unique container name per device) without having to create a cluster in Netbox and add devices to it. Containers like LXC or Docker can actually be run and moved directly between devices without creating clusters, and it seems like a good idea to me to allow this in Netbox.

I don't know if this is the right place to report this, if there is a better place please point it out.

@tomasz-c commented on GitHub (Mar 21, 2023): With virtual machines (or rather containers) I have another problem. Let's say we have a cluster C1 in it we have devices D1 and D2 o now I would like to be able to add V1 (Active) on on C1/D1 and V1 (Offline) C1/D2. Currently Netbox does not allow this because the VM name must be unique for the entire cluster. This option would be useful, as it allows you to quickly determine where you have what disabled containers if they are migrated between devices. In fact, for me it would be sufficient to be able to assign a container directly to a device, (unique container name per device) without having to create a cluster in Netbox and add devices to it. Containers like LXC or Docker can actually be run and moved directly between devices without creating clusters, and it seems like a good idea to me to allow this in Netbox. I don't know if this is the right place to report this, if there is a better place please point it out.

adam commented

@candlerb commented on GitHub (Mar 21, 2023):

[Aside: it would be much more polite to open a new discussion, rather than hijack an existing ticket]

In fact, for me it would be sufficient to be able to assign a container directly to a device, (unique container name per device) without having to create a cluster in Netbox and add devices to it.

That feature already exists in current versions of Netbox: it was added in v3.3.0. You can associate a VM directly with a site and optionally an individual device, without specifying a cluster. What version are you using?

@candlerb commented on GitHub (Mar 21, 2023): [Aside: it would be much more polite to open a new discussion, rather than hijack an existing ticket] > In fact, for me it would be sufficient to be able to assign a container directly to a device, (unique container name per device) without having to create a cluster in Netbox and add devices to it. That feature already exists in current versions of Netbox: it was added in [v3.3.0](https://github.com/netbox-community/netbox/releases/v3.3.0). You can associate a VM directly with a site and optionally an individual device, without specifying a cluster. What version are you using?

adam commented