Merge pull request #6947 from netbox-community/develop

Release v2.11.11

.github/ISSUE_TEMPLATE/bug_report.yaml

@@ -17,7 +17,7 @@ body:
         What version of NetBox are you currently running? (If you don't have access to the most
         recent NetBox release, consider testing on our [demo instance](https://demo.netbox.dev/)
         before opening a bug report to see if your issue has already been addressed.)
-      placeholder: v2.11.4
+      placeholder: v2.11.11
     validations:
       required: true
   - type: dropdown

.github/ISSUE_TEMPLATE/config.yml

@@ -3,7 +3,10 @@ blank_issues_enabled: false
 contact_links:
   - name: 📖 Contributing Policy
     url: https://github.com/netbox-community/netbox/blob/develop/CONTRIBUTING.md
-    about: Please read through our contributing policy before opening an issue or pull request
-  - name: 💬 Discussion Group
-    url: https://groups.google.com/g/netbox-discuss
-    about: Join our discussion group for assistance with installation issues and other problems
+    about: "Please read through our contributing policy before opening an issue or pull request"
+  - name: ❓ Discussion
+    url: https://github.com/netbox-community/netbox/discussions
+    about: "If you're just looking for help, try starting a discussion instead"
+  - name: 💬 Community Slack
+    url: https://netdev.chat/
+    about: "Join #netbox on the NetDev Community Slack for assistance with installation issues and other problems"

.github/ISSUE_TEMPLATE/feature_request.yaml

@@ -14,7 +14,7 @@ body:
     attributes:
       label: NetBox version
       description: What version of NetBox are you currently running?
-      placeholder: v2.11.4
+      placeholder: v2.11.11
     validations:
       required: true
   - type: dropdown

.github/workflows/stale.yml

@@ -8,7 +8,7 @@ jobs:
   stale:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/stale@v3
+      - uses: actions/stale@v4
         with:
           close-issue-message: >
             This issue has been automatically closed due to lack of activity. In an

.gitignore

@@ -2,6 +2,8 @@
 *.swp
 /netbox/netbox/configuration.py
 /netbox/netbox/ldap_config.py
+/netbox/project-static/.cache
+/netbox/project-static/node_modules
 /netbox/reports/*
 !/netbox/reports/__init__.py
 /netbox/scripts/*

CONTRIBUTING.md

@@ -25,7 +25,7 @@ discussions.
 
 ### Slack
 
-For real-time chat, you can join the **#netbox** Slack channel on [NetDev Community](https://slack.netbox.dev/).
+For real-time chat, you can join the **#netbox** Slack channel on [NetDev Community](https://netdev.chat/).
 Unfortunately, the Slack channel does not provide long-term retention of chat
 history, so try to avoid it for any discussions would benefit from being
 preserved for future reference.
@@ -160,17 +160,20 @@ accumulating a large backlog of work.
 The core maintainers group has chosen to make use of GitHub's [Stale bot](https://github.com/apps/stale)
 to aid in issue management.
 
-* Issues will be marked as stale after 45 days of no activity.
-* Then after 15 more days of inactivity, the issue will be closed.
+* Issues will be marked as stale after 60 days of no activity.
+* If the stable label is not removed in the following 30 days, the issue will
+  be closed automatically.
 * Any issue bearing one of the following labels will be exempt from all Stale
   bot actions:
   * `status: accepted`
   * `status: blocked`
   * `status: needs milestone`
 
-It is natural that some new issues get more attention than others. Stale bot
-helps bring renewed attention to potentially valuable issues that may have been
-overlooked.
+It is natural that some new issues get more attention than others. The stale
+bot helps bring renewed attention to potentially valuable issues that may have
+been overlooked. **Do not** comment on an issue that has been marked stale in
+an effort to circumvent the bot: Doing so will not remove the stale label.
+(Stale labels can be removed only by maintainers.)
 
 ## Maintainer Guidance
 

README.md

@@ -2,8 +2,10 @@
   <img src="https://raw.githubusercontent.com/netbox-community/netbox/develop/docs/netbox_logo.svg" width="400" alt="NetBox logo" />
 </div>
 
-NetBox is an IP address management (IPAM) and data center infrastructure
-management (DCIM) tool. Initially conceived by the network engineering team at
+![Master branch build status](https://github.com/netbox-community/netbox/workflows/CI/badge.svg?branch=master)
+
+NetBox is an infrastructure resource modeling (IRM) tool designed to empower
+network automation. Initially conceived by the network engineering team at
 [DigitalOcean](https://www.digitalocean.com/), NetBox was developed specifically
 to address the needs of network and infrastructure engineers. It is intended to
 function as a domain-specific source of truth for network operations.
@@ -14,18 +16,15 @@ complete list of requirements, see `requirements.txt`. The code is available [on
 
 The complete documentation for NetBox can be found at [Read the Docs](https://netbox.readthedocs.io/en/stable/). A public demo instance is available at https://demo.netbox.dev.
 
-|             | status |
-|-------------|------------|
-| **master**  | ![Build status](https://github.com/netbox-community/netbox/workflows/CI/badge.svg?branch=master) |
-| **develop** | ![Build status](https://github.com/netbox-community/netbox/workflows/CI/badge.svg?branch=develop) |
-
 <div align="center">
   <h4>Thank you to our sponsors!</h4>
 
   [![DigitalOcean](https://raw.githubusercontent.com/wiki/netbox-community/netbox/images/sponsors/digitalocean.png)](https://try.digitalocean.com/developer-cloud)
   &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
-  [![NS1](https://raw.githubusercontent.com/wiki/netbox-community/netbox/images/sponsors/ns1.png)](https://ns1.com/)
+  [![Equinix Metal](https://raw.githubusercontent.com/wiki/netbox-community/netbox/images/sponsors/equinix.png)](https://metal.equinix.com/)
   &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
+  [![NS1](https://raw.githubusercontent.com/wiki/netbox-community/netbox/images/sponsors/ns1.png)](https://ns1.com/)
+  <br />
   [![Stellar Technologies](https://raw.githubusercontent.com/wiki/netbox-community/netbox/images/sponsors/stellar.png)](https://stellar.tech/)
 
 </div>
@@ -33,7 +32,7 @@ The complete documentation for NetBox can be found at [Read the Docs](https://ne
 ### Discussion
 
 * [GitHub Discussions](https://github.com/netbox-community/netbox/discussions) - Discussion forum hosted by GitHub; ideal for Q&A and other structured discussions
-* [Slack](https://slack.netbox.dev/) - Real-time chat hosted by the NetDev Community; best for unstructured discussion or just hanging out
+* [Slack](https://netdev.chat/) - Real-time chat hosted by the NetDev Community; best for unstructured discussion or just hanging out
 * [Google Group](https://groups.google.com/g/netbox-discuss) - Legacy mailing list; slowly being replaced by GitHub discussions
 
 ### Installation

contrib/nginx.conf

@@ -1,5 +1,5 @@
 server {
-    listen 443 ssl;
+    listen [::]:443 ssl ipv6only=off;
 
     # CHANGE THIS TO YOUR SERVER'S NAME
     server_name netbox.example.com;
@@ -23,7 +23,7 @@ server {
 
 server {
     # Redirect HTTP traffic to HTTPS
-    listen 80;
+    listen [::]:80 ipv6only=off;
     server_name _;
     return 301 https://$host$request_uri;
 }

docs/additional-features/caching.md

@@ -1,6 +1,9 @@
 # Caching
 
-NetBox supports database query caching using [django-cacheops](https://github.com/Suor/django-cacheops) and Redis. When a query is made, the results are cached in Redis for a short period of time, as defined by the [CACHE_TIMEOUT](../configuration/optional-settings.md#cache_timeout) parameter (15 minutes by default). Within that time, all recurrences of that specific query will return the pre-fetched results from the cache.
+NetBox supports database query caching using [django-cacheops](https://github.com/Suor/django-cacheops) and Redis. When a query is made, the results are cached in Redis for a short period of time, as defined by the [CACHE_TIMEOUT](../configuration/optional-settings.md#cache_timeout) parameter. Within that time, all recurrences of that specific query will return the pre-fetched results from the cache.
+
+!!! warning
+    In NetBox v2.11.10 and later queryset caching is disabled by default, and must be configured.
 
 If a change is made to any of the objects returned by the query within that time, or if the timeout expires, the results are automatically invalidated and the next request for those results will be sent to the database.
 

docs/additional-features/custom-fields.md

@@ -24,7 +24,7 @@ Marking a field as required will force the user to provide a value for the field
 
 The filter logic controls how values are matched when filtering objects by the custom field. Loose filtering (the default) matches on a partial value, whereas exact matching requires a complete match of the given string to a field's value. For example, exact filtering with the string "red" will only match the exact value "red", whereas loose filtering will match on the values "red", "red-orange", or "bored". Setting the filter logic to "disabled" disables filtering by the field entirely.
 
-A custom field must be assigned to one or object types, or models, in NetBox. Once created, custom fields will automatically appear as part of these models in the web UI and REST API. Note that not all models support custom fields.
+A custom field must be assigned to one or more object types, or models, in NetBox. Once created, custom fields will automatically appear as part of these models in the web UI and REST API. Note that not all models support custom fields.
 
 ### Custom Field Validation
 

docs/additional-features/custom-links.md

@@ -17,6 +17,9 @@ When viewing a device named Router4, this link would render as:
 
 Custom links appear as buttons at the top right corner of the page. Numeric weighting can be used to influence the ordering of links.
 
+!!! warning
+    Custom links rely on user-created code to generate arbitrary HTML output, which may be dangerous. Only grant permission to create or modify custom links to trusted users.
+
 ## Context Data
 
 The following context data is available within the template when rendering a custom link's text or URL.

docs/additional-features/custom-scripts.md

@@ -175,7 +175,7 @@ A particular object within NetBox. Each ObjectVar must specify a particular mode
 * `null_option` - A label representing a "null" or empty choice (optional)
 
 !!! warning
-    The `display_field` parameter is now deprecated, and will be removed in NetBox v2.12. All ObjectVar instances will
+    The `display_field` parameter is now deprecated, and will be removed in NetBox v3.0. All ObjectVar instances will
     instead use the new standard `display` field for all serializers (introduced in NetBox v2.11).
 
 To limit the selections available within the list, additional query parameters can be passed as the `query_params` dictionary. For example, to show only devices with an "active" status:

docs/additional-features/export-templates.md

@@ -4,10 +4,13 @@ NetBox allows users to define custom templates that can be used when exporting o
 
 Each export template is associated with a certain type of object. For instance, if you create an export template for VLANs, your custom template will appear under the "Export" button on the VLANs list. Each export template must have a name, and may optionally designate a specific export [MIME type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types) and/or file extension.
 
+Export templates must be written in [Jinja2](https://jinja.palletsprojects.com/).
+
 !!! note
     The name `table` is reserved for internal use.
 
-Export templates must be written in [Jinja2](https://jinja.palletsprojects.com/).
+!!! warning
+    Export templates are rendered using user-submitted code, which may pose security risks under certain conditions. Only grant permission to create or modify export templates to trusted users.
 
 The list of objects returned from the database when rendering an export template is stored in the `queryset` variable, which you'll typically want to iterate through using a `for` loop. Object properties can be access by name. For example:
 

docs/additional-features/reports.md

@@ -80,7 +80,7 @@ class DeviceConnectionsReport(Report):
                 self.log_success(device)
 ```
 
-As you can see, reports are completely customizable. Validation logic can be as simple or as complex as needed.
+As you can see, reports are completely customizable. Validation logic can be as simple or as complex as needed. Also note that the `description` attribute support markdown syntax. It will be rendered in the report list page.
 
 !!! warning
     Reports should never alter data: If you find yourself using the `create()`, `save()`, `update()`, or `delete()` methods on objects within reports, stop and re-evaluate what you're trying to accomplish. Note that there are no safeguards against the accidental alteration or destruction of data.
@@ -93,7 +93,7 @@ The following methods are available to log results within a report:
 * log_warning(object, message)
 * log_failure(object, message)
 
-The recording of one or more failure messages will automatically flag a report as failed. It is advised to log a success for each object that is evaluated so that the results will reflect how many objects are being reported on. (The inclusion of a log message is optional for successes.) Messages recorded with `log()` will appear in a report's results but are not associated with a particular object or status.
+The recording of one or more failure messages will automatically flag a report as failed. It is advised to log a success for each object that is evaluated so that the results will reflect how many objects are being reported on. (The inclusion of a log message is optional for successes.) Messages recorded with `log()` will appear in a report's results but are not associated with a particular object or status. Log messages also support using markdown syntax and will be rendered on the report result page.
 
 To perform additional tasks, such as sending an email or calling a webhook, after a report has been run, extend the `post_run()` method. The status of the report is available as `self.failed` and the results object is `self.result`.
 

docs/additional-features/webhooks.md

@@ -2,6 +2,9 @@
 
 A webhook is a mechanism for conveying to some external system a change that took place in NetBox. For example, you may want to notify a monitoring system whenever the status of a device is updated in NetBox. This can be done by creating a webhook for the device model in NetBox and identifying the webhook receiver. When NetBox detects a change to a device, an HTTP request containing the details of the change and who made it be sent to the specified receiver. Webhooks are configured in the admin UI under Extras > Webhooks.
 
+!!! warning
+    Webhooks support the inclusion of user-submitted code to generate custom headers and payloads, which may pose security risks under certain conditions. Only grant permission to create or modify webhooks to trusted users.
+
 ## Configuration
 
 * **Name** - A unique name for the webhook. The name is not included with outbound messages.

docs/configuration/optional-settings.md

@@ -54,9 +54,9 @@ BASE_PATH = 'netbox/'
 
 ## CACHE_TIMEOUT
 
-Default: 900
+Default: 0 (disabled)
 
-The number of seconds that cache entries will be retained before expiring.
+The number of seconds that cached database queries will be retained before expiring.
 
 ---
 

docs/development/adding-models.md

@@ -0,0 +1,85 @@
+# Adding Models
+
+## 1. Define the model class
+
+Models within each app are stored in either `models.py` or within a submodule under the `models/` directory. When creating a model, be sure to subclass the [appropriate base model](models.md) from `netbox.models`. This will typically be PrimaryModel or OrganizationalModel. Remember to add the model class to the `__all__` listing for the module.
+
+Each model should define, at a minimum:
+
+* A `__str__()` method returning a user-friendly string representation of the instance
+* A `get_absolute_url()` method returning an instance's direct URL (using `reverse()`)
+* A `Meta` class specifying a deterministic ordering (if ordered by fields other than the primary ID)
+
+## 2. Define field choices
+
+If the model has one or more fields with static choices, define those choices in `choices.py` by subclassing `utilities.choices.ChoiceSet`.
+
+## 3. Generate database migrations
+
+Once your model definition is complete, generate database migrations by running `manage.py -n $NAME --no-header`. Always specify a short unique name when generating migrations.
+
+!!! info
+    Set `DEVELOPER = True` in your NetBox configuration to enable the creation of new migrations.
+
+## 4. Add all standard views
+
+Most models will need view classes created in `views.py` to serve the following operations:
+
+* List view
+* Detail view
+* Edit view
+* Delete view
+* Bulk import
+* Bulk edit
+* Bulk delete
+
+## 5. Add URL paths
+
+Add the relevant URL path for each view created in the previous step to `urls.py`.
+
+## 6. Create the FilterSet
+
+Each model should have a corresponding FilterSet class defined. This is used to filter UI and API queries. Subclass the appropriate class from `netbox.filtersets` that matches the model's parent class.
+
+Every model FilterSet should define a `q` filter to support general search queries.
+
+## 7. Create the table
+
+Create a table class for the model in `tables.py` by subclassing `utilities.tables.BaseTable`. Under the table's `Meta` class, be sure to list both the fields and default columns.
+
+## 8. Create the object template
+
+Create the HTML template for the object view. (The other views each typically employ a generic template.) This template should extend `generic/object.html`.
+
+## 9. Add the model to the navigation menu
+
+For NetBox releases prior to v3.0, add the relevant link(s) to the navigation menu template. For later releases, add the relevant items in `netbox/netbox/navigation_menu.py`.
+
+## 10. REST API components
+
+Create the following for each model:
+
+* Detailed (full) model serializer in `api/serializers.py`
+* Nested serializer in `api/nested_serializers.py`
+* API view in `api/views.py`
+* Endpoint route in `api/urls.py`
+
+## 11. GraphQL API components (v3.0+)
+
+Create a Graphene object type for the model in `graphql/types.py` by subclassing the appropriate class from `netbox.graphql.types`.
+
+Also extend the schema class defined in `graphql/schema.py` with the individual object and object list fields per the established convention.
+
+## 12. Add tests
+
+Add tests for the following:
+
+* UI views
+* API views
+* Filter sets
+
+## 13. Documentation
+
+Create a new documentation page for the model in `docs/models/<app_label>/<model_name>.md`. Include this file under the "features" documentation where appropriate.
+
+Also add your model to the index in `docs/development/models.md`.

docs/development/index.md

@@ -8,7 +8,7 @@ There are several official forums for communication among the developers and com
 
 * [GitHub issues](https://github.com/netbox-community/netbox/issues) - All feature requests, bug reports, and other substantial changes to the code base **must** be documented in an issue.
 * [GitHub Discussions](https://github.com/netbox-community/netbox/discussions) - The preferred forum for general discussion and support issues. Ideal for shaping a feature request prior to submitting an issue.
-* [#netbox on NetDev Community Slack](https://slack.netbox.dev/) - Good for quick chats. Avoid any discussion that might need to be referenced later on, as the chat history is not retained long.
+* [#netbox on NetDev Community Slack](https://netdev.chat/) - Good for quick chats. Avoid any discussion that might need to be referenced later on, as the chat history is not retained long.
 * [Google Group](https://groups.google.com/g/netbox-discuss) - Legacy mailing list; slowly being phased out in favor of GitHub discussions.
 
 ## Governance

docs/index.md

@@ -1,8 +1,8 @@
-![NetBox](netbox_logo.svg "NetBox logo")
+![NetBox](netbox_logo.svg "NetBox logo"){style="height: 100px; margin-bottom: 3em"}
 
 # What is NetBox?
 
-NetBox is an open source web application designed to help manage and document computer networks. Initially conceived by the network engineering team at [DigitalOcean](https://www.digitalocean.com/), NetBox was developed specifically to address the needs of network and infrastructure engineers. It encompasses the following aspects of network management:
+NetBox is an infrastructure resource modeling (IRM) application designed to empower network automation. Initially conceived by the network engineering team at [DigitalOcean](https://www.digitalocean.com/), NetBox was developed specifically to address the needs of network and infrastructure engineers. NetBox is made available as open source under the Apache 2 license. It encompasses the following aspects of network management:
 
 * **IP address management (IPAM)** - IP networks and addresses, VRFs, and VLANs
 * **Equipment racks** - Organized by group and site

docs/installation/1-postgresql.md

@@ -11,13 +11,13 @@ This section entails the installation and configuration of a local PostgreSQL da
 
     ```no-highlight
     sudo apt update
-    sudo apt install -y postgresql libpq-dev
+    sudo apt install -y postgresql
     ```
 
 === "CentOS"
 
     ```no-highlight
-    sudo yum install -y postgresql-server libpq-devel
+    sudo yum install -y postgresql-server
     sudo postgresql-setup --initdb
     ```
 

docs/installation/3-netbox.md

@@ -18,7 +18,7 @@ Begin by installing all system packages required by NetBox and its dependencies.
 === "CentOS"
 
     ```no-highlight
-    sudo yum install -y gcc python36 python36-devel python3-pip libxml2-devel libxslt-devel libffi-devel openssl-devel redhat-rpm-config
+    sudo yum install -y gcc python36 python36-devel python3-pip libxml2-devel libxslt-devel libffi-devel libpq-devel openssl-devel redhat-rpm-config
     ```
 
 Before continuing with either platform, update pip (Python's package management tool) to its latest release:
@@ -73,6 +73,11 @@ Next, clone the **master** branch of the NetBox GitHub repository into the curre
 
 ```no-highlight
 $ sudo git clone -b master https://github.com/netbox-community/netbox.git .
+```
+
+The screen below should be the result:
+
+```
 Cloning into '.'...
 remote: Counting objects: 1994, done.
 remote: Compressing objects: 100% (150/150), done.
@@ -262,6 +267,13 @@ Starting development server at http://0.0.0.0:8000/
 Quit the server with CONTROL-C.
 ```
 
+!!! note
+    By default RHEL based distros will likely block your testing attempts with firewalld. The development server port can be opened with `firewall-cmd` (add `--permanent` if you want the rule to survive server restarts):
+
+    ```no-highlight
+    firewall-cmd --zone=public --add-port=8000/tcp
+    ```
+
 Next, connect to the name or IP of the server (as defined in `ALLOWED_HOSTS`) on port 8000; for example, <http://127.0.0.1:8000/>. You should be greeted with the NetBox home page.
 
 !!! danger

docs/installation/6-ldap.md

@@ -74,7 +74,7 @@ STARTTLS can be configured by setting `AUTH_LDAP_START_TLS = True` and using the
 ### User Authentication
 
 !!! info
-    When using Windows Server 2012, `AUTH_LDAP_USER_DN_TEMPLATE` should be set to None.
+    When using Windows Server 2012+, `AUTH_LDAP_USER_DN_TEMPLATE` should be set to None.
 
 ```python
 from django_auth_ldap.config import LDAPSearch

docs/installation/index.md

@@ -24,7 +24,7 @@ The video below demonstrates the installation of NetBox v2.10.3 on Ubuntu 20.04
 | Redis      | 4.0             |
 
 !!! note
-    Python 3.7 or later will be required in NetBox v2.12. Users are strongly encouraged to install NetBox using Python 3.7 or later for new deployments.
+    Python 3.7 or later will be required in NetBox v3.0. Users are strongly encouraged to install NetBox using Python 3.7 or later for new deployments.
 
 Below is a simplified overview of the NetBox application stack for reference:
 

docs/plugins/index.md

@@ -89,3 +89,58 @@ Restart the WSGI service to load the new plugin:
 ```no-highlight
 # sudo systemctl restart netbox
 ```
+
+## Removing Plugins
+
+Follow these steps to completely remove a plugin.
+
+### Update Configuration
+
+Remove the plugin from the `PLUGINS` list in `configuration.py`. Also remove any relevant configuration parameters from `PLUGINS_CONFIG`.
+
+### Remove the Python Package
+
+Use `pip` to remove the installed plugin:
+
+```no-highlight
+$ source /opt/netbox/venv/bin/activate
+(venv) $ pip uninstall <package>
+```
+
+### Restart WSGI Service
+
+Restart the WSGI service:
+
+```no-highlight
+# sudo systemctl restart netbox
+```
+
+### Drop Database Tables
+
+!!! note
+    This step is necessary only for plugin which have created one or more database tables (generally through the introduction of new models). Check your plugin's documentation if unsure.
+
+Enter the PostgreSQL database shell to determine if the plugin has created any SQL tables. Substitute `pluginname` in the example below for the name of the plugin being removed. (You can also run the `\dt` command without a pattern to list _all_ tables.)
+
+```no-highlight
+netbox=> \dt pluginname_*
+                   List of relations
+                   List of relations
+ Schema |       Name     | Type  | Owner
+--------+----------------+-------+--------
+ public | pluginname_foo | table | netbox
+ public | pluginname_bar | table | netbox
+(2 rows)
+```
+
+!!! warning
+    Exercise extreme caution when removing tables. Users are strongly encouraged to perform a backup of their database immediately before taking these actions.
+
+Drop each of the listed tables to remove it from the database:
+
+```no-highlight
+netbox=> DROP TABLE pluginname_foo;
+DROP TABLE
+netbox=> DROP TABLE pluginname_bar;
+DROP TABLE
+```

docs/release-notes/version-2.11.md

@@ -1,5 +1,143 @@
 # NetBox v2.11
 
+## v2.11.11 (2021-08-12)
+
+### Enhancements
+
+* [#6883](https://github.com/netbox-community/netbox/issues/6883) - Add C21 & C22 power types
+* [#6921](https://github.com/netbox-community/netbox/issues/6921) - Employ a sandbox when rendering Jinja2 code for increased security
+
+### Bug Fixes
+
+* [#6740](https://github.com/netbox-community/netbox/issues/6740) - Add import button to VM interfaces list
+* [#6892](https://github.com/netbox-community/netbox/issues/6892) - Fix validation of unit ranges when creating a rack reservation
+* [#6896](https://github.com/netbox-community/netbox/issues/6896) - Fix validation of IP address assigned as device/VM primary via NAT relation
+* [#6902](https://github.com/netbox-community/netbox/issues/6902) - Populate device field when cloning device components
+* [#6908](https://github.com/netbox-community/netbox/issues/6908) - Allow assignment of scope to VLAN groups upon import
+* [#6909](https://github.com/netbox-community/netbox/issues/6909) - Remove extraneous `site` column from VLAN group import form
+* [#6910](https://github.com/netbox-community/netbox/issues/6910) - Fix exception on invalid CSV import column name
+* [#6918](https://github.com/netbox-community/netbox/issues/6918) - Fix return URL persistence when adding multiple objects sequentially
+* [#6935](https://github.com/netbox-community/netbox/issues/6935) - Remove extraneous columns from inventory item and device bay tables
+* [#6936](https://github.com/netbox-community/netbox/issues/6936) - Add missing `parent` column to inventory item import form
+
+---
+
+## v2.11.10 (2021-07-28)
+
+### Enhancements
+
+* [#6560](https://github.com/netbox-community/netbox/issues/6560) - Enable CSV import via uploaded file
+* [#6644](https://github.com/netbox-community/netbox/issues/6644) - Add 6P/4P pass-through port types
+* [#6771](https://github.com/netbox-community/netbox/issues/6771) - Add count of inventory items to manufacturer view
+* [#6785](https://github.com/netbox-community/netbox/issues/6785) - Add "hardwired" type for power port types
+
+### Bug Fixes
+
+* [#5442](https://github.com/netbox-community/netbox/issues/5442) - Fix assignment of permissions based on LDAP groups
+* [#5627](https://github.com/netbox-community/netbox/issues/5627) - Fix filtering of interface connections list
+* [#6759](https://github.com/netbox-community/netbox/issues/6759) - Fix assignment of parent interfaces for bulk import
+* [#6773](https://github.com/netbox-community/netbox/issues/6773) - Add missing `display` field to rack unit serializer
+* [#6774](https://github.com/netbox-community/netbox/issues/6774) - Fix A/Z assignment when swapping circuit terminations
+* [#6777](https://github.com/netbox-community/netbox/issues/6777) - Fix default value validation for custom text fields
+* [#6778](https://github.com/netbox-community/netbox/issues/6778) - Rack reservation should display rack's location
+* [#6780](https://github.com/netbox-community/netbox/issues/6780) - Include rack location in navigation breadcrumbs
+* [#6794](https://github.com/netbox-community/netbox/issues/6794) - Fix device name display on device status view
+* [#6812](https://github.com/netbox-community/netbox/issues/6812) - Limit reported prefix utilization to 100%
+* [#6822](https://github.com/netbox-community/netbox/issues/6822) - Use consistent maximum value for interface MTU
+
+### Other Changes
+
+* [#6781](https://github.com/netbox-community/netbox/issues/6781) - Database query caching is now disabled by default
+
+---
+
+## v2.11.9 (2021-07-08)
+
+### Bug Fixes
+
+* [#6456](https://github.com/netbox-community/netbox/issues/6456) - API schema type should be boolean for `_occupied` on cable termination models
+* [#6710](https://github.com/netbox-community/netbox/issues/6710) - Fix assignment of VM interface parent via REST API
+* [#6714](https://github.com/netbox-community/netbox/issues/6714) - Fix rendering of device type component creation forms
+
+---
+
+## v2.11.8 (2021-07-06)
+
+### Enhancements
+
+* [#5503](https://github.com/netbox-community/netbox/issues/5503) - Annotate short date & time fields with their longer form
+* [#6138](https://github.com/netbox-community/netbox/issues/6138) - Add an `empty` filter modifier for character fields
+* [#6200](https://github.com/netbox-community/netbox/issues/6200) - Add rack reservations to global search
+* [#6368](https://github.com/netbox-community/netbox/issues/6368) - Enable virtual chassis assignment during bulk import of devices
+* [#6620](https://github.com/netbox-community/netbox/issues/6620) - Show assigned VMs count under device role view
+* [#6666](https://github.com/netbox-community/netbox/issues/6666) - Show management-only status under interface detail view
+* [#6667](https://github.com/netbox-community/netbox/issues/6667) - Display VM memory as GB/TB as appropriate
+
+### Bug Fixes
+
+* [#6626](https://github.com/netbox-community/netbox/issues/6626) - Fix site field on VM search form; add site group
+* [#6637](https://github.com/netbox-community/netbox/issues/6637) - Fix group assignment in "available VLANs" link under VLAN group view
+* [#6640](https://github.com/netbox-community/netbox/issues/6640) - Disallow numeric values in custom text fields
+* [#6652](https://github.com/netbox-community/netbox/issues/6652) - Fix exception when adding components in bulk to multiple devices
+* [#6676](https://github.com/netbox-community/netbox/issues/6676) - Fix device/VM counts per cluster under cluster type/group views
+* [#6680](https://github.com/netbox-community/netbox/issues/6680) - Allow setting custom field values for VM interfaces on initial creation
+* [#6695](https://github.com/netbox-community/netbox/issues/6695) - Fix exception when importing device type with invalid front port definition
+
+---
+
+## v2.11.7 (2021-06-16)
+
+### Enhancements
+
+* [#6455](https://github.com/netbox-community/netbox/issues/6455) - Permit /32 IPv4 and /128 IPv6 prefixes
+* [#6493](https://github.com/netbox-community/netbox/issues/6493) - Show change log diff for non-atomic (pre-2.11) changes
+* [#6564](https://github.com/netbox-community/netbox/issues/6564) - Add N connector type for pass-through ports
+* [#6588](https://github.com/netbox-community/netbox/issues/6588) - Add support for webp files as front/rear device type images
+* [#6589](https://github.com/netbox-community/netbox/issues/6589) - Standardize breadcrumb navigation for power panels and feeds
+
+### Bug Fixes
+
+* [#6553](https://github.com/netbox-community/netbox/issues/6553) - ProviderNetwork search should match on name
+* [#6562](https://github.com/netbox-community/netbox/issues/6562) - Disable ordering of secrets by assigned object
+* [#6563](https://github.com/netbox-community/netbox/issues/6563) - Fix filtering by location for cable connection forms
+* [#6584](https://github.com/netbox-community/netbox/issues/6584) - Fix ordering of nested inventory items
+* [#6602](https://github.com/netbox-community/netbox/issues/6602) - Fix deletion of devices with cables attached
+
+---
+
+## v2.11.6 (2021-06-04)
+
+### Bug Fixes
+
+* [#6544](https://github.com/netbox-community/netbox/issues/6544) - Fix migration error when upgrading with VRF(s) defined
+
+---
+
+## v2.11.5 (2021-06-04)
+
+**NOTE:** This release includes a database migration that calculates and annotates prefix depth. It may impose a noticeable delay on the upgrade process: Users should anticipate roughly one minute of delay per 100 thousand prefixes being updated.
+
+### Enhancements
+
+* [#6087](https://github.com/netbox-community/netbox/issues/6087) - Improved prefix hierarchy rendering
+* [#6487](https://github.com/netbox-community/netbox/issues/6487) - Add location filter to cable connection form
+* [#6501](https://github.com/netbox-community/netbox/issues/6501) - Expose prefix depth and children on REST API serializer
+* [#6527](https://github.com/netbox-community/netbox/issues/6527) - Support Markdown for report descriptions
+* [#6540](https://github.com/netbox-community/netbox/issues/6540) - Add a "flat" column to the prefix table
+
+### Bug Fixes
+
+* [#6064](https://github.com/netbox-community/netbox/issues/6064) - Fix object permission assignments for user and group models
+* [#6217](https://github.com/netbox-community/netbox/issues/6217) - Disallow passing of string values for integer custom fields
+* [#6284](https://github.com/netbox-community/netbox/issues/6284) - Avoid sending redundant webhooks when adding/removing tags
+* [#6492](https://github.com/netbox-community/netbox/issues/6492) - Correct tag population in post-change data resulting from REST API changes
+* [#6496](https://github.com/netbox-community/netbox/issues/6496) - Fix upgrade script when Python installed in nonstandard path
+* [#6502](https://github.com/netbox-community/netbox/issues/6502) - Correct permissions evaluation for running a report via the REST API
+* [#6517](https://github.com/netbox-community/netbox/issues/6517) - Fix assignment of user when creating rack reservations via REST API
+* [#6525](https://github.com/netbox-community/netbox/issues/6525) - Paginate related IPs table under IP address view
+
+---
+
 ## v2.11.4 (2021-05-25)
 
 ### Enhancements
@@ -93,7 +231,7 @@
 
 ## v2.11.0 (2021-04-16)
 
-**Note:** NetBox v2.11 is the last major release that will support Python 3.6. Beginning with NetBox v2.12, Python 3.7 or later will be required.
+**Note:** NetBox v2.11 is the last major release that will support Python 3.6. Beginning with NetBox v3.0, Python 3.7 or later will be required.
 
 ### Breaking Changes
 
@@ -151,7 +289,7 @@ Devices can now be assigned to locations (formerly known as rack groups) within
 
 When exporting a list of objects in NetBox, users now have the option of selecting the "current view". This will render CSV output matching the current configuration of the table being viewed. For example, if you modify the sites list to display only the site name, tenant, and status, the rendered CSV will include only these columns, and they will appear in the order chosen.
 
-The legacy static export behavior has been retained to ensure backward compatibility for dependent integrations. However, users are strongly encouraged to adapt custom export templates where needed as this functionality will be removed in v2.12.
+The legacy static export behavior has been retained to ensure backward compatibility for dependent integrations. However, users are strongly encouraged to adapt custom export templates where needed as this functionality will be removed in v3.0.
 
 #### Variable Scope Support for VLAN Groups ([#5284](https://github.com/netbox-community/netbox/issues/5284))
 

docs/rest-api/filtering.md

@@ -61,27 +61,48 @@ These lookup expressions can be applied by adding a suffix to the desired field'
 
 Numeric based fields (ASN, VLAN ID, etc) support these lookup expressions:
 
-- `n` - not equal to (negation)
-- `lt` - less than
-- `lte` - less than or equal
-- `gt` - greater than
-- `gte` - greater than or equal
+| Filter | Description |
+|--------|-------------|
+| `n` | Not equal to |
+| `lt` | Less than |
+| `lte` | Less than or equal to |
+| `gt` | Greater than |
+| `gte` | Greater than or equal to |
+
+Here is an example of a numeric field lookup expression that will return all VLANs with a VLAN ID greater than 900:
+
+```no-highlight
+GET /api/ipam/vlans/?vid__gt=900
+```
 
 ### String Fields
 
 String based (char) fields (Name, Address, etc) support these lookup expressions:
 
-- `n` - not equal to (negation)
-- `ic` - case insensitive contains
-- `nic` - negated case insensitive contains
-- `isw` - case insensitive starts with
-- `nisw` - negated case insensitive starts with
-- `iew` - case insensitive ends with
-- `niew` - negated case insensitive ends with
-- `ie` - case insensitive exact match
-- `nie` - negated case insensitive exact match
+| Filter | Description |
+|--------|-------------|
+| `n` | Not equal to |
+| `ic` | Contains (case-insensitive) |
+| `nic` | Does not contain (case-insensitive) |
+| `isw` | Starts with (case-insensitive) |
+| `nisw` | Does not start with (case-insensitive) |
+| `iew` | Ends with (case-insensitive) |
+| `niew` | Does not end with (case-insensitive) |
+| `ie` | Exact match (case-insensitive) |
+| `nie` | Inverse exact match (case-insensitive) |
+| `empty` | Is empty (boolean) |
+
+Here is an example of a lookup expression on a string field that will return all devices with `switch` in the name:
+
+```no-highlight
+GET /api/dcim/devices/?name__ic=switch
+```
 
 ### Foreign Keys & Other Fields
 
 Certain other fields, namely foreign key relationships support just the negation
-expression: `n`.
+expression: `n`. Here is an example of a lookup expression on a foreign key, it would return all the VLANs that don't have a VLAN Group ID of 3203:
+
+```no-highlight
+GET /api/ipam/vlans/?group_id__n=3203
+```

mkdocs.yml

@@ -1,15 +1,19 @@
 site_name: NetBox Documentation
 site_url: https://netbox.readthedocs.io/
+repo_name: netbox-community/netbox
 repo_url: https://github.com/netbox-community/netbox
 python:
     install:
         - requirements: docs/requirements.txt
 theme:
-  name: material
+    name: material
+    icon:
+        repo: fontawesome/brands/github
 extra_css:
     - extra.css
 markdown_extensions:
     - admonition
+    - attr_list
     - markdown_include.include:
         headingOffset: 1
     - pymdownx.emoji:
@@ -76,6 +80,7 @@ nav:
         - Getting Started: 'development/getting-started.md'
         - Style Guide: 'development/style-guide.md'
         - Models: 'development/models.md'
+        - Adding Models: 'development/adding-models.md'
         - Extending Models: 'development/extending-models.md'
         - Application Registry: 'development/application-registry.md'
         - User Preferences: 'development/user-preferences.md'

netbox/circuits/filtersets.py

@@ -104,6 +104,7 @@ class ProviderNetworkFilterSet(PrimaryModelFilterSet):
         if not value.strip():
             return queryset
         return queryset.filter(
+            Q(name__icontains=value) |
             Q(description__icontains=value) |
             Q(comments__icontains=value)
         ).distinct()

netbox/circuits/views.py

@@ -287,6 +287,10 @@ class CircuitSwapTerminations(generic.ObjectEditView):
                     termination_z.save()
                     termination_a.term_side = 'Z'
                     termination_a.save()
+                    circuit.refresh_from_db()
+                    circuit.termination_a = termination_z
+                    circuit.termination_z = termination_a
+                    circuit.save()
             elif termination_a:
                 termination_a.term_side = 'Z'
                 termination_a.save()
@@ -300,9 +304,6 @@ class CircuitSwapTerminations(generic.ObjectEditView):
                 circuit.termination_z = None
                 circuit.save()
 
-            print(f'term A: {circuit.termination_a}')
-            print(f'term Z: {circuit.termination_z}')
-
             messages.success(request, f"Swapped terminations for circuit {circuit}.")
             return redirect('circuits:circuit', pk=circuit.pk)
 

netbox/dcim/api/serializers.py

@@ -25,6 +25,7 @@ from .nested_serializers import *
 class CableTerminationSerializer(serializers.ModelSerializer):
     cable_peer_type = serializers.SerializerMethodField(read_only=True)
     cable_peer = serializers.SerializerMethodField(read_only=True)
+    _occupied = serializers.SerializerMethodField(read_only=True)
 
     def get_cable_peer_type(self, obj):
         if obj._cable_peer is not None:
@@ -42,6 +43,10 @@ class CableTerminationSerializer(serializers.ModelSerializer):
             return serializer(obj._cable_peer, context=context).data
         return None
 
+    @swagger_serializer_method(serializer_or_field=serializers.BooleanField)
+    def get__occupied(self, obj):
+        return obj._occupied
+
 
 class ConnectedEndpointSerializer(serializers.ModelSerializer):
     connected_endpoint_type = serializers.SerializerMethodField(read_only=True)
@@ -205,6 +210,10 @@ class RackUnitSerializer(serializers.Serializer):
     face = ChoiceField(choices=DeviceFaceChoices, read_only=True)
     device = NestedDeviceSerializer(read_only=True)
     occupied = serializers.BooleanField(read_only=True)
+    display = serializers.SerializerMethodField(read_only=True)
+
+    def get_display(self, obj):
+        return obj['name']
 
 
 class RackReservationSerializer(PrimaryModelSerializer):

netbox/dcim/api/views.py

@@ -246,10 +246,6 @@ class RackReservationViewSet(ModelViewSet):
     serializer_class = serializers.RackReservationSerializer
     filterset_class = filtersets.RackReservationFilterSet
 
-    # Assign user from request
-    def perform_create(self, serializer):
-        serializer.save(user=self.request.user)
-
 
 #
 # Manufacturers
@@ -596,11 +592,9 @@ class PowerConnectionViewSet(ListModelMixin, GenericViewSet):
 
 class InterfaceConnectionViewSet(ListModelMixin, GenericViewSet):
     queryset = Interface.objects.prefetch_related('device', '_path').filter(
-        # Avoid duplicate connections by only selecting the lower PK in a connected pair
         _path__destination_type__app_label='dcim',
         _path__destination_type__model='interface',
-        _path__destination_id__isnull=False,
-        pk__lt=F('_path__destination_id')
+        _path__destination_id__isnull=False
     )
     serializer_class = serializers.InterfaceConnectionSerializer
     filterset_class = filtersets.InterfaceConnectionFilterSet

netbox/dcim/choices.py

@@ -252,6 +252,7 @@ class PowerPortTypeChoices(ChoiceSet):
     TYPE_IEC_C14 = 'iec-60320-c14'
     TYPE_IEC_C16 = 'iec-60320-c16'
     TYPE_IEC_C20 = 'iec-60320-c20'
+    TYPE_IEC_C22 = 'iec-60320-c22'
     # IEC 60309
     TYPE_IEC_PNE4H = 'iec-60309-p-n-e-4h'
     TYPE_IEC_PNE6H = 'iec-60309-p-n-e-6h'
@@ -341,6 +342,8 @@ class PowerPortTypeChoices(ChoiceSet):
     TYPE_DC = 'dc-terminal'
     # Proprietary
     TYPE_SAF_D_GRID = 'saf-d-grid'
+    # Other
+    TYPE_HARDWIRED = 'hardwired'
 
     CHOICES = (
         ('IEC 60320', (
@@ -349,6 +352,7 @@ class PowerPortTypeChoices(ChoiceSet):
             (TYPE_IEC_C14, 'C14'),
             (TYPE_IEC_C16, 'C16'),
             (TYPE_IEC_C20, 'C20'),
+            (TYPE_IEC_C22, 'C22'),
         )),
         ('IEC 60309', (
             (TYPE_IEC_PNE4H, 'P+N+E 4H'),
@@ -447,6 +451,9 @@ class PowerPortTypeChoices(ChoiceSet):
         ('Proprietary', (
             (TYPE_SAF_D_GRID, 'Saf-D-Grid'),
         )),
+        ('Other', (
+            (TYPE_HARDWIRED, 'Hardwired'),
+        )),
     )
 
 
@@ -462,6 +469,7 @@ class PowerOutletTypeChoices(ChoiceSet):
     TYPE_IEC_C13 = 'iec-60320-c13'
     TYPE_IEC_C15 = 'iec-60320-c15'
     TYPE_IEC_C19 = 'iec-60320-c19'
+    TYPE_IEC_C21 = 'iec-60320-c21'
     # IEC 60309
     TYPE_IEC_PNE4H = 'iec-60309-p-n-e-4h'
     TYPE_IEC_PNE6H = 'iec-60309-p-n-e-6h'
@@ -553,6 +561,7 @@ class PowerOutletTypeChoices(ChoiceSet):
             (TYPE_IEC_C13, 'C13'),
             (TYPE_IEC_C15, 'C15'),
             (TYPE_IEC_C19, 'C19'),
+            (TYPE_IEC_C21, 'C21'),
         )),
         ('IEC 60309', (
             (TYPE_IEC_PNE4H, 'P+N+E 4H'),
@@ -917,6 +926,11 @@ class PortTypeChoices(ChoiceSet):
     TYPE_8P6C = '8p6c'
     TYPE_8P4C = '8p4c'
     TYPE_8P2C = '8p2c'
+    TYPE_6P6C = '6p6c'
+    TYPE_6P4C = '6p4c'
+    TYPE_6P2C = '6p2c'
+    TYPE_4P4C = '4p4c'
+    TYPE_4P2C = '4p2c'
     TYPE_GG45 = 'gg45'
     TYPE_TERA4P = 'tera-4p'
     TYPE_TERA2P = 'tera-2p'
@@ -924,6 +938,7 @@ class PortTypeChoices(ChoiceSet):
     TYPE_110_PUNCH = '110-punch'
     TYPE_BNC = 'bnc'
     TYPE_F = 'f'
+    TYPE_N = 'n'
     TYPE_MRJ21 = 'mrj21'
     TYPE_ST = 'st'
     TYPE_SC = 'sc'
@@ -947,6 +962,11 @@ class PortTypeChoices(ChoiceSet):
                 (TYPE_8P6C, '8P6C'),
                 (TYPE_8P4C, '8P4C'),
                 (TYPE_8P2C, '8P2C'),
+                (TYPE_6P6C, '6P6C'),
+                (TYPE_6P4C, '6P4C'),
+                (TYPE_6P2C, '6P2C'),
+                (TYPE_4P4C, '4P4C'),
+                (TYPE_4P2C, '4P2C'),
                 (TYPE_GG45, 'GG45'),
                 (TYPE_TERA4P, 'TERA 4P'),
                 (TYPE_TERA2P, 'TERA 2P'),
@@ -954,6 +974,7 @@ class PortTypeChoices(ChoiceSet):
                 (TYPE_110_PUNCH, '110 Punch'),
                 (TYPE_BNC, 'BNC'),
                 (TYPE_F, 'F Connector'),
+                (TYPE_N, 'N Connector'),
                 (TYPE_MRJ21, 'MRJ21'),
             ),
         ),

netbox/dcim/constants.py

@@ -2,6 +2,9 @@ from django.db.models import Q
 
 from .choices import InterfaceTypeChoices
 
+# Exclude SVG images (unsupported by PIL)
+DEVICETYPE_IMAGE_FORMATS = 'image/bmp,image/gif,image/jpeg,image/png,image/tiff,image/webp'
+
 
 #
 # Racks
@@ -26,7 +29,7 @@ REARPORT_POSITIONS_MAX = 1024
 #
 
 INTERFACE_MTU_MIN = 1
-INTERFACE_MTU_MAX = 32767  # Max value of a signed 16-bit integer
+INTERFACE_MTU_MAX = 65536
 
 VIRTUAL_IFACE_TYPES = [
     InterfaceTypeChoices.TYPE_VIRTUAL,

netbox/dcim/forms.py

@@ -102,6 +102,12 @@ class InterfaceCommonForm(forms.Form):
         required=False,
         label='MAC address'
     )
+    mtu = forms.IntegerField(
+        required=False,
+        min_value=INTERFACE_MTU_MIN,
+        max_value=INTERFACE_MTU_MAX,
+        label='MTU'
+    )
 
     def clean(self):
         super().clean()
@@ -1172,12 +1178,11 @@ class DeviceTypeForm(BootstrapMixin, CustomFieldModelForm):
         )
         widgets = {
             'subdevice_role': StaticSelect2(),
-            # Exclude SVG images (unsupported by PIL)
             'front_image': forms.ClearableFileInput(attrs={
-                'accept': 'image/bmp,image/gif,image/jpeg,image/png,image/tiff'
+                'accept': DEVICETYPE_IMAGE_FORMATS
             }),
             'rear_image': forms.ClearableFileInput(attrs={
-                'accept': 'image/bmp,image/gif,image/jpeg,image/png,image/tiff'
+                'accept': DEVICETYPE_IMAGE_FORMATS
             })
         }
 
@@ -1879,8 +1884,7 @@ class FrontPortTemplateImportForm(ComponentTemplateImportForm):
     )
     rear_port = forms.ModelChoiceField(
         queryset=RearPortTemplate.objects.all(),
-        to_field_name='name',
-        required=False
+        to_field_name='name'
     )
 
     class Meta:
@@ -2237,6 +2241,12 @@ class BaseDeviceCSVForm(CustomFieldModelCSVForm):
         choices=DeviceStatusChoices,
         help_text='Operational status'
     )
+    virtual_chassis = CSVModelChoiceField(
+        queryset=VirtualChassis.objects.all(),
+        to_field_name='name',
+        required=False,
+        help_text='Virtual chassis'
+    )
     cluster = CSVModelChoiceField(
         queryset=Cluster.objects.all(),
         to_field_name='name',
@@ -2247,6 +2257,10 @@ class BaseDeviceCSVForm(CustomFieldModelCSVForm):
     class Meta:
         fields = []
         model = Device
+        help_texts = {
+            'vc_position': 'Virtual chassis position',
+            'vc_priority': 'Virtual chassis priority',
+        }
 
     def __init__(self, data=None, *args, **kwargs):
         super().__init__(data, *args, **kwargs)
@@ -2285,7 +2299,8 @@ class DeviceCSVForm(BaseDeviceCSVForm):
     class Meta(BaseDeviceCSVForm.Meta):
         fields = [
             'name', 'device_role', 'tenant', 'manufacturer', 'device_type', 'platform', 'serial', 'asset_tag', 'status',
-            'site', 'location', 'rack', 'position', 'face', 'cluster', 'comments',
+            'site', 'location', 'rack', 'position', 'face', 'virtual_chassis', 'vc_position', 'vc_priority', 'cluster',
+            'comments',
         ]
 
     def __init__(self, data=None, *args, **kwargs):
@@ -2320,7 +2335,7 @@ class ChildDeviceCSVForm(BaseDeviceCSVForm):
     class Meta(BaseDeviceCSVForm.Meta):
         fields = [
             'name', 'device_role', 'tenant', 'manufacturer', 'device_type', 'platform', 'serial', 'asset_tag', 'status',
-            'parent', 'device_bay', 'cluster', 'comments',
+            'parent', 'device_bay', 'virtual_chassis', 'vc_position', 'vc_priority', 'cluster', 'comments',
         ]
 
     def __init__(self, data=None, *args, **kwargs):
@@ -3164,12 +3179,6 @@ class InterfaceCreateForm(ComponentCreateForm, InterfaceCommonForm):
             'type': 'lag',
         }
     )
-    mtu = forms.IntegerField(
-        required=False,
-        min_value=INTERFACE_MTU_MIN,
-        max_value=INTERFACE_MTU_MAX,
-        label='MTU'
-    )
     mac_address = forms.CharField(
         required=False,
         label='MAC Address'
@@ -3369,13 +3378,18 @@ class InterfaceCSVForm(CustomFieldModelCSVForm):
                 Q(device=device) | Q(device__virtual_chassis=device.virtual_chassis),
                 type=InterfaceTypeChoices.TYPE_LAG
             )
+            self.fields['parent'].queryset = Interface.objects.filter(
+                Q(device=device) | Q(device__virtual_chassis=device.virtual_chassis)
+            )
         elif device:
             self.fields['lag'].queryset = Interface.objects.filter(
                 device=device,
                 type=InterfaceTypeChoices.TYPE_LAG
             )
+            self.fields['parent'].queryset = Interface.objects.filter(device=device)
         else:
             self.fields['lag'].queryset = Interface.objects.none()
+            self.fields['parent'].queryset = Interface.objects.none()
 
     def clean_enabled(self):
         # Make sure enabled is True when it's not included in the uploaded data
@@ -3838,11 +3852,32 @@ class InventoryItemCSVForm(CustomFieldModelCSVForm):
         to_field_name='name',
         required=False
     )
+    parent = CSVModelChoiceField(
+        queryset=Device.objects.all(),
+        to_field_name='name',
+        required=False,
+        help_text='Parent inventory item'
+    )
 
     class Meta:
         model = InventoryItem
         fields = InventoryItem.csv_headers
 
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+
+        # Limit parent choices to inventory items belonging to this device
+        device = None
+        if self.is_bound and 'device' in self.data:
+            try:
+                device = self.fields['device'].to_python(self.data['device'])
+            except forms.ValidationError:
+                pass
+        if device:
+            self.fields['parent'].queryset = InventoryItem.objects.filter(device=device)
+        else:
+            self.fields['parent'].queryset = InventoryItem.objects.none()
+
 
 class InventoryItemBulkCreateForm(
     form_from_model(InventoryItem, ['manufacturer', 'part_id', 'serial', 'asset_tag', 'discovered']),
@@ -3923,13 +3958,23 @@ class ConnectCableToDeviceForm(BootstrapMixin, CustomFieldModelForm):
             'group_id': '$termination_b_site_group',
         }
     )
+    termination_b_location = DynamicModelChoiceField(
+        queryset=Location.objects.all(),
+        label='Location',
+        required=False,
+        null_option='None',
+        query_params={
+            'site_id': '$termination_b_site'
+        }
+    )
     termination_b_rack = DynamicModelChoiceField(
         queryset=Rack.objects.all(),
         label='Rack',
         required=False,
         null_option='None',
         query_params={
-            'site_id': '$termination_b_site'
+            'site_id': '$termination_b_site',
+            'location_id': '$termination_b_location',
         }
     )
     termination_b_device = DynamicModelChoiceField(
@@ -3938,6 +3983,7 @@ class ConnectCableToDeviceForm(BootstrapMixin, CustomFieldModelForm):
         required=False,
         query_params={
             'site_id': '$termination_b_site',
+            'location_id': '$termination_b_location',
             'rack_id': '$termination_b_rack',
         }
     )

netbox/dcim/models/device_component_templates.py

@@ -290,19 +290,24 @@ class FrontPortTemplate(ComponentTemplateModel):
     def clean(self):
         super().clean()
 
-        # Validate rear port assignment
-        if self.rear_port.device_type != self.device_type:
-            raise ValidationError(
-                "Rear port ({}) must belong to the same device type".format(self.rear_port)
-            )
+        try:
 
-        # Validate rear port position assignment
-        if self.rear_port_position > self.rear_port.positions:
-            raise ValidationError(
-                "Invalid rear port position ({}); rear port {} has only {} positions".format(
-                    self.rear_port_position, self.rear_port.name, self.rear_port.positions
+            # Validate rear port assignment
+            if self.rear_port.device_type != self.device_type:
+                raise ValidationError(
+                    "Rear port ({}) must belong to the same device type".format(self.rear_port)
                 )
-            )
+
+            # Validate rear port position assignment
+            if self.rear_port_position > self.rear_port.positions:
+                raise ValidationError(
+                    "Invalid rear port position ({}); rear port {} has only {} positions".format(
+                        self.rear_port_position, self.rear_port.name, self.rear_port.positions
+                    )
+                )
+
+        except RearPortTemplate.DoesNotExist:
+            pass
 
     def instantiate(self, device):
         if self.rear_port:

netbox/dcim/models/device_components.py

@@ -230,6 +230,7 @@ class ConsolePort(ComponentModel, CableTermination, PathEndpoint):
     )
 
     csv_headers = ['device', 'name', 'label', 'type', 'speed', 'mark_connected', 'description']
+    clone_fields = ['device', 'type', 'speed']
 
     class Meta:
         ordering = ('device', '_name')
@@ -273,6 +274,7 @@ class ConsoleServerPort(ComponentModel, CableTermination, PathEndpoint):
     )
 
     csv_headers = ['device', 'name', 'label', 'type', 'speed', 'mark_connected', 'description']
+    clone_fields = ['device', 'type', 'speed']
 
     class Meta:
         ordering = ('device', '_name')
@@ -324,6 +326,7 @@ class PowerPort(ComponentModel, CableTermination, PathEndpoint):
     csv_headers = [
         'device', 'name', 'label', 'type', 'mark_connected', 'maximum_draw', 'allocated_draw', 'description',
     ]
+    clone_fields = ['device', 'maximum_draw', 'allocated_draw']
 
     class Meta:
         ordering = ('device', '_name')
@@ -434,6 +437,7 @@ class PowerOutlet(ComponentModel, CableTermination, PathEndpoint):
     )
 
     csv_headers = ['device', 'name', 'label', 'type', 'mark_connected', 'power_port', 'feed_leg', 'description']
+    clone_fields = ['device', 'type', 'power_port', 'feed_leg']
 
     class Meta:
         ordering = ('device', '_name')
@@ -483,7 +487,10 @@ class BaseInterface(models.Model):
     mtu = models.PositiveIntegerField(
         blank=True,
         null=True,
-        validators=[MinValueValidator(1), MaxValueValidator(65536)],
+        validators=[
+            MinValueValidator(INTERFACE_MTU_MIN),
+            MaxValueValidator(INTERFACE_MTU_MAX)
+        ],
         verbose_name='MTU'
     )
     mode = models.CharField(
@@ -574,6 +581,7 @@ class Interface(ComponentModel, BaseInterface, CableTermination, PathEndpoint):
         'device', 'name', 'label', 'parent', 'lag', 'type', 'enabled', 'mark_connected', 'mac_address', 'mtu',
         'mgmt_only', 'description', 'mode',
     ]
+    clone_fields = ['device', 'parent', 'lag', 'type', 'mgmt_only']
 
     class Meta:
         ordering = ('device', CollateAsChar('_name'))
@@ -708,6 +716,7 @@ class FrontPort(ComponentModel, CableTermination):
     csv_headers = [
         'device', 'name', 'label', 'type', 'mark_connected', 'rear_port', 'rear_port_position', 'description',
     ]
+    clone_fields = ['device', 'type']
 
     class Meta:
         ordering = ('device', '_name')
@@ -764,6 +773,7 @@ class RearPort(ComponentModel, CableTermination):
             MaxValueValidator(REARPORT_POSITIONS_MAX)
         ]
     )
+    clone_fields = ['device', 'type', 'positions']
 
     csv_headers = ['device', 'name', 'label', 'type', 'mark_connected', 'positions', 'description']
 
@@ -815,6 +825,7 @@ class DeviceBay(ComponentModel):
     )
 
     csv_headers = ['device', 'name', 'label', 'installed_device', 'description']
+    clone_fields = ['device']
 
     class Meta:
         ordering = ('device', '_name')
@@ -910,6 +921,7 @@ class InventoryItem(MPTTModel, ComponentModel):
     csv_headers = [
         'device', 'name', 'label', 'manufacturer', 'part_id', 'serial', 'asset_tag', 'discovered', 'description',
     ]
+    clone_fields = ['device', 'parent', 'manufacturer', 'part_id']
 
     class Meta:
         ordering = ('device__id', 'parent__id', '_name')

netbox/dcim/signals.py

@@ -146,14 +146,12 @@ def nullify_connected_endpoints(instance, **kwargs):
     # Disassociate the Cable from its termination points
     if instance.termination_a is not None:
         logger.debug(f"Nullifying termination A for cable {instance}")
-        instance.termination_a.cable = None
-        instance.termination_a._cable_peer = None
-        instance.termination_a.save()
+        model = instance.termination_a._meta.model
+        model.objects.filter(pk=instance.termination_a.pk).update(_cable_peer_type=None, _cable_peer_id=None)
     if instance.termination_b is not None:
         logger.debug(f"Nullifying termination B for cable {instance}")
-        instance.termination_b.cable = None
-        instance.termination_b._cable_peer = None
-        instance.termination_b.save()
+        model = instance.termination_b._meta.model
+        model.objects.filter(pk=instance.termination_b.pk).update(_cable_peer_type=None, _cable_peer_id=None)
 
     # Delete and retrace any dependent cable paths
     for cablepath in CablePath.objects.filter(path__contains=instance):

netbox/dcim/tables/devices.py

@@ -232,10 +232,6 @@ class DeviceComponentTable(BaseTable):
         linkify=True,
         order_by=('_name',)
     )
-    cable = tables.Column(
-        linkify=True
-    )
-    mark_connected = BooleanColumn()
 
     class Meta(BaseTable.Meta):
         order_by = ('device', 'name')
@@ -694,7 +690,7 @@ class InventoryItemTable(DeviceComponentTable):
     )
     cable = None  # Override DeviceComponentTable
 
-    class Meta(DeviceComponentTable.Meta):
+    class Meta(BaseTable.Meta):
         model = InventoryItem
         fields = (
             'pk', 'device', 'name', 'label', 'manufacturer', 'part_id', 'serial', 'asset_tag', 'description',
@@ -715,7 +711,7 @@ class DeviceInventoryItemTable(InventoryItemTable):
         buttons=('edit', 'delete')
     )
 
-    class Meta(DeviceComponentTable.Meta):
+    class Meta(BaseTable.Meta):
         model = InventoryItem
         fields = (
             'pk', 'name', 'label', 'manufacturer', 'part_id', 'serial', 'asset_tag', 'description', 'discovered',

netbox/dcim/tests/test_api.py

@@ -349,40 +349,36 @@ class RackReservationTest(APIViewTestCases.APIViewTestCase):
         user = User.objects.create(username='user1', is_active=True)
         site = Site.objects.create(name='Test Site 1', slug='test-site-1')
 
-        cls.racks = (
+        racks = (
             Rack(site=site, name='Rack 1'),
             Rack(site=site, name='Rack 2'),
         )
-        Rack.objects.bulk_create(cls.racks)
+        Rack.objects.bulk_create(racks)
 
         rack_reservations = (
-            RackReservation(rack=cls.racks[0], units=[1, 2, 3], user=user, description='Reservation #1'),
-            RackReservation(rack=cls.racks[0], units=[4, 5, 6], user=user, description='Reservation #2'),
-            RackReservation(rack=cls.racks[0], units=[7, 8, 9], user=user, description='Reservation #3'),
+            RackReservation(rack=racks[0], units=[1, 2, 3], user=user, description='Reservation #1'),
+            RackReservation(rack=racks[0], units=[4, 5, 6], user=user, description='Reservation #2'),
+            RackReservation(rack=racks[0], units=[7, 8, 9], user=user, description='Reservation #3'),
         )
         RackReservation.objects.bulk_create(rack_reservations)
 
-    def setUp(self):
-        super().setUp()
-
-        # We have to set creation data under setUp() because we need access to the test user.
-        self.create_data = [
+        cls.create_data = [
             {
-                'rack': self.racks[1].pk,
+                'rack': racks[1].pk,
                 'units': [10, 11, 12],
-                'user': self.user.pk,
+                'user': user.pk,
                 'description': 'Reservation #4',
             },
             {
-                'rack': self.racks[1].pk,
+                'rack': racks[1].pk,
                 'units': [13, 14, 15],
-                'user': self.user.pk,
+                'user': user.pk,
                 'description': 'Reservation #5',
             },
             {
-                'rack': self.racks[1].pk,
+                'rack': racks[1].pk,
                 'units': [16, 17, 18],
-                'user': self.user.pk,
+                'user': user.pk,
                 'description': 'Reservation #6',
             },
         ]
@@ -1215,8 +1211,9 @@ class InterfaceTest(Mixins.ComponentTraceMixin, APIViewTestCases.APIViewTestCase
             {
                 'device': device.pk,
                 'name': 'Interface 6',
-                'type': '1000base-t',
+                'type': 'virtual',
                 'mode': InterfaceModeChoices.MODE_TAGGED,
+                'parent': interfaces[0].pk,
                 'tagged_vlans': [vlans[0].pk, vlans[1].pk],
                 'untagged_vlan': vlans[2].pk,
             },

netbox/dcim/tests/test_views.py

@@ -1023,6 +1023,8 @@ class DeviceTestCase(ViewTestCases.PrimaryObjectViewTestCase):
 
         tags = create_tags('Alpha', 'Bravo', 'Charlie')
 
+        VirtualChassis.objects.create(name='Virtual Chassis 1')
+
         cls.form_data = {
             'device_type': devicetypes[1].pk,
             'device_role': deviceroles[1].pk,
@@ -1048,10 +1050,10 @@ class DeviceTestCase(ViewTestCases.PrimaryObjectViewTestCase):
         }
 
         cls.csv_data = (
-            "device_role,manufacturer,device_type,status,name,site,location,rack,position,face",
-            "Device Role 1,Manufacturer 1,Device Type 1,active,Device 4,Site 1,Location 1,Rack 1,10,front",
-            "Device Role 1,Manufacturer 1,Device Type 1,active,Device 5,Site 1,Location 1,Rack 1,20,front",
-            "Device Role 1,Manufacturer 1,Device Type 1,active,Device 6,Site 1,Location 1,Rack 1,30,front",
+            "device_role,manufacturer,device_type,status,name,site,location,rack,position,face,virtual_chassis,vc_position,vc_priority",
+            "Device Role 1,Manufacturer 1,Device Type 1,active,Device 4,Site 1,Location 1,Rack 1,10,front,Virtual Chassis 1,1,10",
+            "Device Role 1,Manufacturer 1,Device Type 1,active,Device 5,Site 1,Location 1,Rack 1,20,front,Virtual Chassis 1,2,20",
+            "Device Role 1,Manufacturer 1,Device Type 1,active,Device 6,Site 1,Location 1,Rack 1,30,front,Virtual Chassis 1,3,30",
         )
 
         cls.bulk_edit_data = {
@@ -1462,7 +1464,7 @@ class InterfaceTestCase(ViewTestCases.DeviceComponentViewTestCase):
             'enabled': False,
             'lag': interfaces[3].pk,
             'mac_address': EUI('01:02:03:04:05:06'),
-            'mtu': 2000,
+            'mtu': 65000,
             'mgmt_only': True,
             'description': 'A front port',
             'mode': InterfaceModeChoices.MODE_TAGGED,
@@ -1734,10 +1736,10 @@ class InventoryItemTestCase(ViewTestCases.DeviceComponentViewTestCase):
         }
 
         cls.csv_data = (
-            "device,name",
-            "Device 1,Inventory Item 4",
-            "Device 1,Inventory Item 5",
-            "Device 1,Inventory Item 6",
+            "device,name,parent",
+            "Device 1,Inventory Item 4,Inventory Item 1",
+            "Device 1,Inventory Item 5,Inventory Item 2",
+            "Device 1,Inventory Item 6,Inventory Item 3",
         )
 
 

netbox/dcim/views.py

@@ -695,6 +695,9 @@ class ManufacturerView(generic.ObjectView):
         ).annotate(
             instance_count=count_related(Device, 'device_type')
         )
+        inventory_items = InventoryItem.objects.restrict(request.user, 'view').filter(
+            manufacturer=instance
+        )
 
         devicetypes_table = tables.DeviceTypeTable(devicetypes)
         devicetypes_table.columns.hide('manufacturer')
@@ -702,6 +705,7 @@ class ManufacturerView(generic.ObjectView):
 
         return {
             'devicetypes_table': devicetypes_table,
+            'inventory_item_count': inventory_items.count(),
         }
 
 
@@ -1169,6 +1173,8 @@ class DeviceRoleView(generic.ObjectView):
 
         return {
             'devices_table': devices_table,
+            'device_count': Device.objects.filter(device_role=instance).count(),
+            'virtualmachine_count': VirtualMachine.objects.filter(role=instance).count(),
         }
 
 
@@ -2562,11 +2568,7 @@ class PowerConnectionsListView(generic.ObjectListView):
 
 
 class InterfaceConnectionsListView(generic.ObjectListView):
-    queryset = Interface.objects.filter(
-        # Avoid duplicate connections by only selecting the lower PK in a connected pair
-        _path__isnull=False,
-        pk__lt=F('_path__destination_id')
-    ).order_by('device')
+    queryset = Interface.objects.filter(_path__isnull=False).order_by('device')
     filterset = filtersets.InterfaceConnectionFilterSet
     filterset_form = forms.InterfaceConnectionFilterForm
     table = tables.InterfaceConnectionTable

netbox/extras/api/views.py

@@ -239,7 +239,7 @@ class ReportViewSet(ViewSet):
         Run a Report identified as "<module>.<script>" and return the pending JobResult as the result
         """
         # Check that the user has permission to run reports.
-        if not request.user.has_perm('extras.run_script'):
+        if not request.user.has_perm('extras.run_report'):
             raise PermissionDenied("This user does not have permission to run reports.")
 
         # Check that at least one RQ worker is running

netbox/extras/apps.py

@@ -5,4 +5,5 @@ class ExtrasConfig(AppConfig):
     name = "extras"
 
     def ready(self):
+        import extras.lookups
         import extras.signals

netbox/extras/context_managers.py

@@ -4,6 +4,7 @@ from django.db.models.signals import m2m_changed, pre_delete, post_save
 
 from extras.signals import _handle_changed_object, _handle_deleted_object
 from utilities.utils import curry
+from .webhooks import flush_webhooks
 
 
 @contextmanager
@@ -14,9 +15,11 @@ def change_logging(request):
 
     :param request: WSGIRequest object with a unique `id` set
     """
+    webhook_queue = []
+
     # Curry signals receivers to pass the current request
-    handle_changed_object = curry(_handle_changed_object, request)
-    handle_deleted_object = curry(_handle_deleted_object, request)
+    handle_changed_object = curry(_handle_changed_object, request, webhook_queue)
+    handle_deleted_object = curry(_handle_deleted_object, request, webhook_queue)
 
     # Connect our receivers to the post_save and post_delete signals.
     post_save.connect(handle_changed_object, dispatch_uid='handle_changed_object')
@@ -30,3 +33,7 @@ def change_logging(request):
     post_save.disconnect(handle_changed_object, dispatch_uid='handle_changed_object')
     m2m_changed.disconnect(handle_changed_object, dispatch_uid='handle_changed_object')
     pre_delete.disconnect(handle_deleted_object, dispatch_uid='handle_deleted_object')
+
+    # Flush queued webhooks to RQ
+    flush_webhooks(webhook_queue)
+    del webhook_queue

netbox/extras/lookups.py

@@ -0,0 +1,17 @@
+from django.db.models import CharField, Lookup
+
+
+class Empty(Lookup):
+    """
+    Filter on whether a string is empty.
+    """
+    lookup_name = 'empty'
+
+    def as_sql(self, qn, connection):
+        lhs, lhs_params = self.process_lhs(qn, connection)
+        rhs, rhs_params = self.process_rhs(qn, connection)
+        params = lhs_params + rhs_params
+        return 'CAST(LENGTH(%s) AS BOOLEAN) != %s' % (lhs, rhs), params
+
+
+CharField.register_lookup(Empty)

netbox/extras/models/customfields.py

@@ -149,7 +149,8 @@ class CustomField(BigIDModel):
         # Validate the field's default value (if any)
         if self.default is not None:
             try:
-                self.validate(self.default)
+                default_value = str(self.default) if self.type == CustomFieldTypeChoices.TYPE_TEXT else self.default
+                self.validate(default_value)
             except ValidationError as err:
                 raise ValidationError({
                     'default': f'Invalid default value "{self.default}": {err.message}'
@@ -280,15 +281,15 @@ class CustomField(BigIDModel):
         if value not in [None, '']:
 
             # Validate text field
-            if self.type == CustomFieldTypeChoices.TYPE_TEXT and self.validation_regex:
-                if not re.match(self.validation_regex, value):
+            if self.type == CustomFieldTypeChoices.TYPE_TEXT:
+                if type(value) is not str:
+                    raise ValidationError(f"Value must be a string.")
+                if self.validation_regex and not re.match(self.validation_regex, value):
                     raise ValidationError(f"Value must match regex '{self.validation_regex}'")
 
             # Validate integer
             if self.type == CustomFieldTypeChoices.TYPE_INTEGER:
-                try:
-                    int(value)
-                except ValueError:
+                if type(value) is not int:
                     raise ValidationError("Value must be an integer.")
                 if self.validation_minimum is not None and value < self.validation_minimum:
                     raise ValidationError(f"Value must be at least {self.validation_minimum}")

netbox/extras/models/models.py

@@ -431,9 +431,8 @@ class JournalEntry(ChangeLoggedModel):
         verbose_name_plural = 'journal entries'
 
     def __str__(self):
-        created_date = timezone.localdate(self.created)
-        created_time = timezone.localtime(self.created)
-        return f"{date_format(created_date)} - {time_format(created_time)} ({self.get_kind_display()})"
+        created = timezone.localtime(self.created)
+        return f"{date_format(created, format='SHORT_DATETIME_FORMAT')} ({self.get_kind_display()})"
 
     def get_absolute_url(self):
         return reverse('extras:journalentry', args=[self.pk])

netbox/extras/scripts.py

@@ -188,10 +188,10 @@ class ObjectVar(ScriptVariable):
 
     def __init__(self, model, query_params=None, null_option=None, *args, **kwargs):
 
-        # TODO: Remove display_field in v2.12
+        # TODO: Remove display_field in v3.0
         if 'display_field' in kwargs:
             warnings.warn(
-                "The 'display_field' parameter has been deprecated, and will be removed in NetBox v2.12. Object "
+                "The 'display_field' parameter has been deprecated, and will be removed in NetBox v3.0. Object "
                 "variables will now reference the 'display' attribute available on all model serializers by default."
             )
         display_field = kwargs.pop('display_field', 'display')

netbox/extras/signals.py

@@ -12,17 +12,27 @@ from prometheus_client import Counter
 
 from .choices import ObjectChangeActionChoices
 from .models import CustomField, ObjectChange
-from .webhooks import enqueue_webhooks
+from .webhooks import enqueue_object, get_snapshots, serialize_for_webhook
 
 
 #
 # Change logging/webhooks
 #
 
-def _handle_changed_object(request, sender, instance, **kwargs):
+def _handle_changed_object(request, webhook_queue, sender, instance, **kwargs):
     """
     Fires when an object is created or updated.
     """
+    def is_same_object(instance, webhook_data):
+        return (
+            ContentType.objects.get_for_model(instance) == webhook_data['content_type'] and
+            instance.pk == webhook_data['object_id'] and
+            request.id == webhook_data['request_id']
+        )
+
+    if not hasattr(instance, 'to_objectchange'):
+        return
+
     m2m_changed = False
 
     # Determine the type of change being made
@@ -53,8 +63,13 @@ def _handle_changed_object(request, sender, instance, **kwargs):
             objectchange.request_id = request.id
             objectchange.save()
 
-    # Enqueue webhooks
-    enqueue_webhooks(instance, request.user, request.id, action)
+    # If this is an M2M change, update the previously queued webhook (from post_save)
+    if m2m_changed and webhook_queue and is_same_object(instance, webhook_queue[-1]):
+        instance.refresh_from_db()  # Ensure that we're working with fresh M2M assignments
+        webhook_queue[-1]['data'] = serialize_for_webhook(instance)
+        webhook_queue[-1]['snapshots']['postchange'] = get_snapshots(instance, action)['postchange']
+    else:
+        enqueue_object(webhook_queue, instance, request.user, request.id, action)
 
     # Increment metric counters
     if action == ObjectChangeActionChoices.ACTION_CREATE:
@@ -68,10 +83,13 @@ def _handle_changed_object(request, sender, instance, **kwargs):
         ObjectChange.objects.filter(time__lt=cutoff)._raw_delete(using=DEFAULT_DB_ALIAS)
 
 
-def _handle_deleted_object(request, sender, instance, **kwargs):
+def _handle_deleted_object(request, webhook_queue, sender, instance, **kwargs):
     """
     Fires when an object is deleted.
     """
+    if not hasattr(instance, 'to_objectchange'):
+        return
+
     # Record an ObjectChange if applicable
     if hasattr(instance, 'to_objectchange'):
         objectchange = instance.to_objectchange(ObjectChangeActionChoices.ACTION_DELETE)
@@ -80,7 +98,7 @@ def _handle_deleted_object(request, sender, instance, **kwargs):
         objectchange.save()
 
     # Enqueue webhooks
-    enqueue_webhooks(instance, request.user, request.id, ObjectChangeActionChoices.ACTION_DELETE)
+    enqueue_object(webhook_queue, instance, request.user, request.id, ObjectChangeActionChoices.ACTION_DELETE)
 
     # Increment metric counters
     model_deletes.labels(instance._meta.model_name).inc()

netbox/extras/tests/test_webhooks.py

@@ -11,8 +11,8 @@ from rest_framework import status
 
 from dcim.models import Site
 from extras.choices import ObjectChangeActionChoices
-from extras.models import Webhook
-from extras.webhooks import enqueue_webhooks, generate_signature
+from extras.models import Tag, Webhook
+from extras.webhooks import enqueue_object, flush_webhooks, generate_signature
 from extras.webhooks_worker import process_webhook
 from utilities.testing import APITestCase
 
@@ -20,11 +20,10 @@ from utilities.testing import APITestCase
 class WebhookTest(APITestCase):
 
     def setUp(self):
-
         super().setUp()
 
         self.queue = django_rq.get_queue('default')
-        self.queue.empty()  # Begin each test with an empty queue
+        self.queue.empty()
 
     @classmethod
     def setUpTestData(cls):
@@ -34,38 +33,104 @@ class WebhookTest(APITestCase):
         DUMMY_SECRET = "LOOKATMEIMASECRETSTRING"
 
         webhooks = Webhook.objects.bulk_create((
-            Webhook(name='Site Create Webhook', type_create=True, payload_url=DUMMY_URL, secret=DUMMY_SECRET, additional_headers='X-Foo: Bar'),
-            Webhook(name='Site Update Webhook', type_update=True, payload_url=DUMMY_URL, secret=DUMMY_SECRET),
-            Webhook(name='Site Delete Webhook', type_delete=True, payload_url=DUMMY_URL, secret=DUMMY_SECRET),
+            Webhook(name='Webhook 1', type_create=True, payload_url=DUMMY_URL, secret=DUMMY_SECRET, additional_headers='X-Foo: Bar'),
+            Webhook(name='Webhook 2', type_update=True, payload_url=DUMMY_URL, secret=DUMMY_SECRET),
+            Webhook(name='Webhook 3', type_delete=True, payload_url=DUMMY_URL, secret=DUMMY_SECRET),
         ))
         for webhook in webhooks:
             webhook.content_types.set([site_ct])
 
+        Tag.objects.bulk_create((
+            Tag(name='Foo', slug='foo'),
+            Tag(name='Bar', slug='bar'),
+            Tag(name='Baz', slug='baz'),
+        ))
+
     def test_enqueue_webhook_create(self):
         # Create an object via the REST API
         data = {
-            'name': 'Test Site',
-            'slug': 'test-site',
+            'name': 'Site 1',
+            'slug': 'site-1',
+            'tags': [
+                {'name': 'Foo'},
+                {'name': 'Bar'},
+            ]
         }
         url = reverse('dcim-api:site-list')
         self.add_permissions('dcim.add_site')
         response = self.client.post(url, data, format='json', **self.header)
         self.assertHttpStatus(response, status.HTTP_201_CREATED)
         self.assertEqual(Site.objects.count(), 1)
+        self.assertEqual(Site.objects.first().tags.count(), 2)
 
         # Verify that a job was queued for the object creation webhook
         self.assertEqual(self.queue.count, 1)
         job = self.queue.jobs[0]
         self.assertEqual(job.kwargs['webhook'], Webhook.objects.get(type_create=True))
-        self.assertEqual(job.kwargs['data']['id'], response.data['id'])
-        self.assertEqual(job.kwargs['model_name'], 'site')
         self.assertEqual(job.kwargs['event'], ObjectChangeActionChoices.ACTION_CREATE)
+        self.assertEqual(job.kwargs['model_name'], 'site')
+        self.assertEqual(job.kwargs['data']['id'], response.data['id'])
+        self.assertEqual(len(job.kwargs['data']['tags']), len(response.data['tags']))
+        self.assertEqual(job.kwargs['snapshots']['postchange']['name'], 'Site 1')
+        self.assertEqual(job.kwargs['snapshots']['postchange']['tags'], ['Bar', 'Foo'])
+
+    def test_enqueue_webhook_bulk_create(self):
+        # Create multiple objects via the REST API
+        data = [
+            {
+                'name': 'Site 1',
+                'slug': 'site-1',
+                'tags': [
+                    {'name': 'Foo'},
+                    {'name': 'Bar'},
+                ]
+            },
+            {
+                'name': 'Site 2',
+                'slug': 'site-2',
+                'tags': [
+                    {'name': 'Foo'},
+                    {'name': 'Bar'},
+                ]
+            },
+            {
+                'name': 'Site 3',
+                'slug': 'site-3',
+                'tags': [
+                    {'name': 'Foo'},
+                    {'name': 'Bar'},
+                ]
+            },
+        ]
+        url = reverse('dcim-api:site-list')
+        self.add_permissions('dcim.add_site')
+        response = self.client.post(url, data, format='json', **self.header)
+        self.assertHttpStatus(response, status.HTTP_201_CREATED)
+        self.assertEqual(Site.objects.count(), 3)
+        self.assertEqual(Site.objects.first().tags.count(), 2)
+
+        # Verify that a webhook was queued for each object
+        self.assertEqual(self.queue.count, 3)
+        for i, job in enumerate(self.queue.jobs):
+            self.assertEqual(job.kwargs['webhook'], Webhook.objects.get(type_create=True))
+            self.assertEqual(job.kwargs['event'], ObjectChangeActionChoices.ACTION_CREATE)
+            self.assertEqual(job.kwargs['model_name'], 'site')
+            self.assertEqual(job.kwargs['data']['id'], response.data[i]['id'])
+            self.assertEqual(len(job.kwargs['data']['tags']), len(response.data[i]['tags']))
+            self.assertEqual(job.kwargs['snapshots']['postchange']['name'], response.data[i]['name'])
+            self.assertEqual(job.kwargs['snapshots']['postchange']['tags'], ['Bar', 'Foo'])
 
     def test_enqueue_webhook_update(self):
-        # Update an object via the REST API
         site = Site.objects.create(name='Site 1', slug='site-1')
+        site.tags.set(*Tag.objects.filter(name__in=['Foo', 'Bar']))
+
+        # Update an object via the REST API
         data = {
+            'name': 'Site X',
             'comments': 'Updated the site',
+            'tags': [
+                {'name': 'Baz'}
+            ]
         }
         url = reverse('dcim-api:site-detail', kwargs={'pk': site.pk})
         self.add_permissions('dcim.change_site')
@@ -76,13 +141,72 @@ class WebhookTest(APITestCase):
         self.assertEqual(self.queue.count, 1)
         job = self.queue.jobs[0]
         self.assertEqual(job.kwargs['webhook'], Webhook.objects.get(type_update=True))
-        self.assertEqual(job.kwargs['data']['id'], site.pk)
-        self.assertEqual(job.kwargs['model_name'], 'site')
         self.assertEqual(job.kwargs['event'], ObjectChangeActionChoices.ACTION_UPDATE)
+        self.assertEqual(job.kwargs['model_name'], 'site')
+        self.assertEqual(job.kwargs['data']['id'], site.pk)
+        self.assertEqual(len(job.kwargs['data']['tags']), len(response.data['tags']))
+        self.assertEqual(job.kwargs['snapshots']['prechange']['name'], 'Site 1')
+        self.assertEqual(job.kwargs['snapshots']['prechange']['tags'], ['Bar', 'Foo'])
+        self.assertEqual(job.kwargs['snapshots']['postchange']['name'], 'Site X')
+        self.assertEqual(job.kwargs['snapshots']['postchange']['tags'], ['Baz'])
+
+    def test_enqueue_webhook_bulk_update(self):
+        sites = (
+            Site(name='Site 1', slug='site-1'),
+            Site(name='Site 2', slug='site-2'),
+            Site(name='Site 3', slug='site-3'),
+        )
+        Site.objects.bulk_create(sites)
+        for site in sites:
+            site.tags.set(*Tag.objects.filter(name__in=['Foo', 'Bar']))
+
+        # Update three objects via the REST API
+        data = [
+            {
+                'id': sites[0].pk,
+                'name': 'Site X',
+                'tags': [
+                    {'name': 'Baz'}
+                ]
+            },
+            {
+                'id': sites[1].pk,
+                'name': 'Site Y',
+                'tags': [
+                    {'name': 'Baz'}
+                ]
+            },
+            {
+                'id': sites[2].pk,
+                'name': 'Site Z',
+                'tags': [
+                    {'name': 'Baz'}
+                ]
+            },
+        ]
+        url = reverse('dcim-api:site-list')
+        self.add_permissions('dcim.change_site')
+        response = self.client.patch(url, data, format='json', **self.header)
+        self.assertHttpStatus(response, status.HTTP_200_OK)
+
+        # Verify that a job was queued for the object update webhook
+        self.assertEqual(self.queue.count, 3)
+        for i, job in enumerate(self.queue.jobs):
+            self.assertEqual(job.kwargs['webhook'], Webhook.objects.get(type_update=True))
+            self.assertEqual(job.kwargs['event'], ObjectChangeActionChoices.ACTION_UPDATE)
+            self.assertEqual(job.kwargs['model_name'], 'site')
+            self.assertEqual(job.kwargs['data']['id'], data[i]['id'])
+            self.assertEqual(len(job.kwargs['data']['tags']), len(response.data[i]['tags']))
+            self.assertEqual(job.kwargs['snapshots']['prechange']['name'], sites[i].name)
+            self.assertEqual(job.kwargs['snapshots']['prechange']['tags'], ['Bar', 'Foo'])
+            self.assertEqual(job.kwargs['snapshots']['postchange']['name'], response.data[i]['name'])
+            self.assertEqual(job.kwargs['snapshots']['postchange']['tags'], ['Baz'])
 
     def test_enqueue_webhook_delete(self):
-        # Delete an object via the REST API
         site = Site.objects.create(name='Site 1', slug='site-1')
+        site.tags.set(*Tag.objects.filter(name__in=['Foo', 'Bar']))
+
+        # Delete an object via the REST API
         url = reverse('dcim-api:site-detail', kwargs={'pk': site.pk})
         self.add_permissions('dcim.delete_site')
         response = self.client.delete(url, **self.header)
@@ -92,9 +216,40 @@ class WebhookTest(APITestCase):
         self.assertEqual(self.queue.count, 1)
         job = self.queue.jobs[0]
         self.assertEqual(job.kwargs['webhook'], Webhook.objects.get(type_delete=True))
-        self.assertEqual(job.kwargs['data']['id'], site.pk)
-        self.assertEqual(job.kwargs['model_name'], 'site')
         self.assertEqual(job.kwargs['event'], ObjectChangeActionChoices.ACTION_DELETE)
+        self.assertEqual(job.kwargs['model_name'], 'site')
+        self.assertEqual(job.kwargs['data']['id'], site.pk)
+        self.assertEqual(job.kwargs['snapshots']['prechange']['name'], 'Site 1')
+        self.assertEqual(job.kwargs['snapshots']['prechange']['tags'], ['Bar', 'Foo'])
+
+    def test_enqueue_webhook_bulk_delete(self):
+        sites = (
+            Site(name='Site 1', slug='site-1'),
+            Site(name='Site 2', slug='site-2'),
+            Site(name='Site 3', slug='site-3'),
+        )
+        Site.objects.bulk_create(sites)
+        for site in sites:
+            site.tags.set(*Tag.objects.filter(name__in=['Foo', 'Bar']))
+
+        # Delete three objects via the REST API
+        data = [
+            {'id': site.pk} for site in sites
+        ]
+        url = reverse('dcim-api:site-list')
+        self.add_permissions('dcim.delete_site')
+        response = self.client.delete(url, data, format='json', **self.header)
+        self.assertHttpStatus(response, status.HTTP_204_NO_CONTENT)
+
+        # Verify that a job was queued for the object update webhook
+        self.assertEqual(self.queue.count, 3)
+        for i, job in enumerate(self.queue.jobs):
+            self.assertEqual(job.kwargs['webhook'], Webhook.objects.get(type_delete=True))
+            self.assertEqual(job.kwargs['event'], ObjectChangeActionChoices.ACTION_DELETE)
+            self.assertEqual(job.kwargs['model_name'], 'site')
+            self.assertEqual(job.kwargs['data']['id'], sites[i].pk)
+            self.assertEqual(job.kwargs['snapshots']['prechange']['name'], sites[i].name)
+            self.assertEqual(job.kwargs['snapshots']['prechange']['tags'], ['Bar', 'Foo'])
 
     def test_webhooks_worker(self):
 
@@ -125,13 +280,16 @@ class WebhookTest(APITestCase):
             return HttpResponse()
 
         # Enqueue a webhook for processing
+        webhooks_queue = []
         site = Site.objects.create(name='Site 1', slug='site-1')
-        enqueue_webhooks(
+        enqueue_object(
+            webhooks_queue,
             instance=site,
             user=self.user,
             request_id=request_id,
             action=ObjectChangeActionChoices.ACTION_CREATE
         )
+        flush_webhooks(webhooks_queue)
 
         # Retrieve the job from queue
         job = self.queue.jobs[0]

netbox/extras/views.py

@@ -202,15 +202,22 @@ class ObjectChangeView(generic.ObjectView):
         next_change = objectchanges.filter(time__gt=instance.time).order_by('time').first()
         prev_change = objectchanges.filter(time__lt=instance.time).order_by('-time').first()
 
-        if instance.prechange_data and instance.postchange_data:
+        if not instance.prechange_data and instance.action in ['update', 'delete'] and prev_change:
+            non_atomic_change = True
+            prechange_data = prev_change.postchange_data
+        else:
+            non_atomic_change = False
+            prechange_data = instance.prechange_data
+
+        if prechange_data and instance.postchange_data:
             diff_added = shallow_compare_dict(
-                instance.prechange_data or dict(),
+                prechange_data or dict(),
                 instance.postchange_data or dict(),
                 exclude=['last_updated'],
             )
             diff_removed = {
-                x: instance.prechange_data.get(x) for x in diff_added
-            } if instance.prechange_data else {}
+                x: prechange_data.get(x) for x in diff_added
+            } if prechange_data else {}
         else:
             diff_added = None
             diff_removed = None
@@ -221,7 +228,8 @@ class ObjectChangeView(generic.ObjectView):
             'next_change': next_change,
             'prev_change': prev_change,
             'related_changes_table': related_changes_table,
-            'related_changes_count': related_changes.count()
+            'related_changes_count': related_changes.count(),
+            'non_atomic_change': non_atomic_change
         }
 
 

netbox/extras/webhooks.py

@@ -1,5 +1,6 @@
 import hashlib
 import hmac
+from collections import defaultdict
 
 from django.contrib.contenttypes.models import ContentType
 from django.utils import timezone
@@ -12,6 +13,26 @@ from .models import Webhook
 from .registry import registry
 
 
+def serialize_for_webhook(instance):
+    """
+    Return a serialized representation of the given instance suitable for use in a webhook.
+    """
+    serializer_class = get_serializer_for_model(instance.__class__)
+    serializer_context = {
+        'request': None,
+    }
+    serializer = serializer_class(instance, context=serializer_context)
+
+    return serializer.data
+
+
+def get_snapshots(instance, action):
+    return {
+        'prechange': getattr(instance, '_prechange_snapshot', None),
+        'postchange': serialize_object(instance) if action != ObjectChangeActionChoices.ACTION_DELETE else None,
+    }
+
+
 def generate_signature(request_body, secret):
     """
     Return a cryptographic signature that can be used to verify the authenticity of webhook data.
@@ -24,10 +45,10 @@ def generate_signature(request_body, secret):
     return hmac_prep.hexdigest()
 
 
-def enqueue_webhooks(instance, user, request_id, action):
+def enqueue_object(queue, instance, user, request_id, action):
     """
-    Find Webhook(s) assigned to this instance + action and enqueue them
-    to be processed
+    Enqueue a serialized representation of a created/updated/deleted object for the processing of
+    webhooks once the request has completed.
     """
     # Determine whether this type of object supports webhooks
     app_label = instance._meta.app_label
@@ -35,41 +56,55 @@ def enqueue_webhooks(instance, user, request_id, action):
     if model_name not in registry['model_features']['webhooks'].get(app_label, []):
         return
 
-    # Retrieve any applicable Webhooks
-    content_type = ContentType.objects.get_for_model(instance)
-    action_flag = {
-        ObjectChangeActionChoices.ACTION_CREATE: 'type_create',
-        ObjectChangeActionChoices.ACTION_UPDATE: 'type_update',
-        ObjectChangeActionChoices.ACTION_DELETE: 'type_delete',
-    }[action]
-    webhooks = Webhook.objects.filter(content_types=content_type, enabled=True, **{action_flag: True})
+    queue.append({
+        'content_type': ContentType.objects.get_for_model(instance),
+        'object_id': instance.pk,
+        'event': action,
+        'data': serialize_for_webhook(instance),
+        'snapshots': get_snapshots(instance, action),
+        'username': user.username,
+        'request_id': request_id
+    })
 
-    if webhooks.exists():
 
-        # Get the Model's API serializer class and serialize the object
-        serializer_class = get_serializer_for_model(instance.__class__)
-        serializer_context = {
-            'request': None,
-        }
-        serializer = serializer_class(instance, context=serializer_context)
+def flush_webhooks(queue):
+    """
+    Flush a list of object representation to RQ for webhook processing.
+    """
+    rq_queue = get_queue('default')
+    webhooks_cache = {
+        'type_create': {},
+        'type_update': {},
+        'type_delete': {},
+    }
 
-        # Gather pre- and post-change snapshots
-        snapshots = {
-            'prechange': getattr(instance, '_prechange_snapshot', None),
-            'postchange': serialize_object(instance) if action != ObjectChangeActionChoices.ACTION_DELETE else None,
-        }
+    for data in queue:
+
+        action_flag = {
+            ObjectChangeActionChoices.ACTION_CREATE: 'type_create',
+            ObjectChangeActionChoices.ACTION_UPDATE: 'type_update',
+            ObjectChangeActionChoices.ACTION_DELETE: 'type_delete',
+        }[data['event']]
+        content_type = data['content_type']
+
+        # Cache applicable Webhooks
+        if content_type not in webhooks_cache[action_flag]:
+            webhooks_cache[action_flag][content_type] = Webhook.objects.filter(
+                **{action_flag: True},
+                content_types=content_type,
+                enabled=True
+            )
+        webhooks = webhooks_cache[action_flag][content_type]
 
-        # Enqueue the webhooks
-        webhook_queue = get_queue('default')
         for webhook in webhooks:
-            webhook_queue.enqueue(
+            rq_queue.enqueue(
                 "extras.webhooks_worker.process_webhook",
                 webhook=webhook,
-                model_name=instance._meta.model_name,
-                event=action,
-                data=serializer.data,
-                snapshots=snapshots,
+                model_name=content_type.model,
+                event=data['event'],
+                data=data['data'],
+                snapshots=data['snapshots'],
                 timestamp=str(timezone.now()),
-                username=user.username,
-                request_id=request_id
+                username=data['username'],
+                request_id=data['request_id']
             )

netbox/ipam/api/nested_serializers.py

@@ -102,10 +102,11 @@ class NestedVLANSerializer(WritableNestedSerializer):
 class NestedPrefixSerializer(WritableNestedSerializer):
     url = serializers.HyperlinkedIdentityField(view_name='ipam-api:prefix-detail')
     family = serializers.IntegerField(read_only=True)
+    _depth = serializers.IntegerField(read_only=True)
 
     class Meta:
         model = models.Prefix
-        fields = ['id', 'url', 'display', 'family', 'prefix']
+        fields = ['id', 'url', 'display', 'family', 'prefix', '_depth']
 
 
 #

netbox/ipam/api/serializers.py

@@ -197,12 +197,14 @@ class PrefixSerializer(PrimaryModelSerializer):
     vlan = NestedVLANSerializer(required=False, allow_null=True)
     status = ChoiceField(choices=PrefixStatusChoices, required=False)
     role = NestedRoleSerializer(required=False, allow_null=True)
+    children = serializers.IntegerField(read_only=True)
+    _depth = serializers.IntegerField(read_only=True)
 
     class Meta:
         model = Prefix
         fields = [
             'id', 'url', 'display', 'family', 'prefix', 'site', 'vrf', 'tenant', 'vlan', 'status', 'role', 'is_pool',
-            'description', 'tags', 'custom_fields', 'created', 'last_updated',
+            'description', 'tags', 'custom_fields', 'created', 'last_updated', 'children', '_depth',
         ]
         read_only_fields = ['family']
 
@@ -272,7 +274,7 @@ class IPAddressSerializer(PrimaryModelSerializer):
     )
     assigned_object = serializers.SerializerMethodField(read_only=True)
     nat_inside = NestedIPAddressSerializer(required=False, allow_null=True)
-    nat_outside = NestedIPAddressSerializer(read_only=True)
+    nat_outside = NestedIPAddressSerializer(required=False, read_only=True)
 
     class Meta:
         model = IPAddress
@@ -281,7 +283,7 @@ class IPAddressSerializer(PrimaryModelSerializer):
             'assigned_object_id', 'assigned_object', 'nat_inside', 'nat_outside', 'dns_name', 'description', 'tags',
             'custom_fields', 'created', 'last_updated',
         ]
-        read_only_fields = ['family']
+        read_only_fields = ['family', 'nat_outside']
 
     @swagger_serializer_method(serializer_or_field=serializers.DictField)
     def get_assigned_object(self, obj):

netbox/ipam/filtersets.py

@@ -209,6 +209,12 @@ class PrefixFilterSet(PrimaryModelFilterSet, TenancyFilterSet):
         method='search_contains',
         label='Prefixes which contain this prefix or IP',
     )
+    depth = MultiValueNumberFilter(
+        field_name='_depth'
+    )
+    children = MultiValueNumberFilter(
+        field_name='_children'
+    )
     mask_length = django_filters.NumberFilter(
         field_name='prefix',
         lookup_expr='net_mask_length'

netbox/ipam/forms.py

@@ -11,8 +11,8 @@ from tenancy.forms import TenancyFilterForm, TenancyForm
 from tenancy.models import Tenant
 from utilities.forms import (
     add_blank_choice, BootstrapMixin, BulkEditNullBooleanSelect, ContentTypeChoiceField, CSVChoiceField,
-    CSVModelChoiceField, DatePicker, DynamicModelChoiceField, DynamicModelMultipleChoiceField, ExpandableIPAddressField,
-    NumericArrayField, ReturnURLForm, SlugField, StaticSelect2, StaticSelect2Multiple, TagFilterField,
+    CSVContentTypeField, CSVModelChoiceField, DatePicker, DynamicModelChoiceField, DynamicModelMultipleChoiceField,
+    ExpandableIPAddressField, NumericArrayField, SlugField, StaticSelect2, StaticSelect2Multiple, TagFilterField,
     BOOLEAN_WITH_BLANK_CHOICES,
 )
 from virtualization.models import Cluster, ClusterGroup, VirtualMachine, VMInterface
@@ -682,7 +682,7 @@ class PrefixFilterForm(BootstrapMixin, TenancyFilterForm, CustomFieldFilterForm)
 # IP addresses
 #
 
-class IPAddressForm(BootstrapMixin, TenancyForm, ReturnURLForm, CustomFieldModelForm):
+class IPAddressForm(BootstrapMixin, TenancyForm, CustomFieldModelForm):
     device = DynamicModelChoiceField(
         queryset=Device.objects.all(),
         required=False,
@@ -1238,17 +1238,19 @@ class VLANGroupForm(BootstrapMixin, CustomFieldModelForm):
 
 
 class VLANGroupCSVForm(CustomFieldModelCSVForm):
-    site = CSVModelChoiceField(
-        queryset=Site.objects.all(),
-        required=False,
-        to_field_name='name',
-        help_text='Assigned site'
-    )
     slug = SlugField()
+    scope_type = CSVContentTypeField(
+        queryset=ContentType.objects.filter(model__in=VLANGROUP_SCOPE_TYPES),
+        required=False,
+        label='Scope type (app & model)'
+    )
 
     class Meta:
         model = VLANGroup
         fields = VLANGroup.csv_headers
+        labels = {
+            'scope_id': 'Scope ID',
+        }
 
 
 class VLANGroupBulkEditForm(BootstrapMixin, CustomFieldBulkEditForm):

netbox/ipam/management/commands/rebuild_prefixes.py

@@ -0,0 +1,27 @@
+from django.core.management.base import BaseCommand
+
+from ipam.models import Prefix, VRF
+from ipam.utils import rebuild_prefixes
+
+
+class Command(BaseCommand):
+    help = "Rebuild the prefix hierarchy (depth and children counts)"
+
+    def handle(self, *model_names, **options):
+        self.stdout.write(f'Rebuilding {Prefix.objects.count()} prefixes...')
+
+        # Reset existing counts
+        Prefix.objects.update(_depth=0, _children=0)
+
+        # Rebuild the global table
+        global_count = Prefix.objects.filter(vrf__isnull=True).count()
+        self.stdout.write(f'Global: {global_count} prefixes...')
+        rebuild_prefixes(None)
+
+        # Rebuild each VRF
+        for vrf in VRF.objects.all():
+            vrf_count = Prefix.objects.filter(vrf=vrf).count()
+            self.stdout.write(f'VRF {vrf}: {vrf_count} prefixes...')
+            rebuild_prefixes(vrf.pk)
+
+        self.stdout.write(self.style.SUCCESS('Finished.'))

netbox/ipam/migrations/0047_prefix_depth_children.py

@@ -0,0 +1,21 @@
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('ipam', '0046_set_vlangroup_scope_types'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='prefix',
+            name='_children',
+            field=models.PositiveBigIntegerField(default=0, editable=False),
+        ),
+        migrations.AddField(
+            model_name='prefix',
+            name='_depth',
+            field=models.PositiveSmallIntegerField(default=0, editable=False),
+        ),
+    ]

netbox/ipam/migrations/0048_prefix_populate_depth_children.py

@@ -0,0 +1,37 @@
+import sys
+from django.db import migrations
+
+from ipam.utils import rebuild_prefixes
+
+
+def populate_prefix_hierarchy(apps, schema_editor):
+    """
+    Populate _depth and _children attrs for all Prefixes.
+    """
+    Prefix = apps.get_model('ipam', 'Prefix')
+    VRF = apps.get_model('ipam', 'VRF')
+
+    total_count = Prefix.objects.count()
+    if 'test' not in sys.argv:
+        print(f'\nUpdating {total_count} prefixes...')
+
+    # Rebuild the global table
+    rebuild_prefixes(None)
+
+    # Iterate through all VRFs, rebuilding each
+    for vrf in VRF.objects.all():
+        rebuild_prefixes(vrf.pk)
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('ipam', '0047_prefix_depth_children'),
+    ]
+
+    operations = [
+        migrations.RunPython(
+            code=populate_prefix_hierarchy,
+            reverse_code=migrations.RunPython.noop
+        ),
+    ]

netbox/ipam/models/ip.py

@@ -181,7 +181,9 @@ class Aggregate(PrimaryModel):
         """
         queryset = Prefix.objects.filter(prefix__net_contained_or_equal=str(self.prefix))
         child_prefixes = netaddr.IPSet([p.prefix for p in queryset])
-        return int(float(child_prefixes.size) / self.prefix.size * 100)
+        utilization = int(float(child_prefixes.size) / self.prefix.size * 100)
+
+        return min(utilization, 100)
 
 
 @extras_features('custom_fields', 'custom_links', 'export_templates', 'webhooks')
@@ -293,6 +295,16 @@ class Prefix(PrimaryModel):
         blank=True
     )
 
+    # Cached depth & child counts
+    _depth = models.PositiveSmallIntegerField(
+        default=0,
+        editable=False
+    )
+    _children = models.PositiveBigIntegerField(
+        default=0,
+        editable=False
+    )
+
     objects = PrefixQuerySet.as_manager()
 
     csv_headers = [
@@ -306,6 +318,13 @@ class Prefix(PrimaryModel):
         ordering = (F('vrf').asc(nulls_first=True), 'prefix', 'pk')  # (vrf, prefix) may be non-unique
         verbose_name_plural = 'prefixes'
 
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+
+        # Cache the original prefix and VRF so we can check if they have changed on post_save
+        self._prefix = self.prefix
+        self._vrf = self.vrf
+
     def __str__(self):
         return str(self.prefix)
 
@@ -323,16 +342,6 @@ class Prefix(PrimaryModel):
                     'prefix': "Cannot create prefix with /0 mask."
                 })
 
-            # Disallow host masks
-            if self.prefix.version == 4 and self.prefix.prefixlen == 32:
-                raise ValidationError({
-                    'prefix': "Cannot create host addresses (/32) as prefixes. Create an IPv4 address instead."
-                })
-            elif self.prefix.version == 6 and self.prefix.prefixlen == 128:
-                raise ValidationError({
-                    'prefix': "Cannot create host addresses (/128) as prefixes. Create an IPv6 address instead."
-                })
-
             # Enforce unique IP space (if applicable)
             if (self.vrf is None and settings.ENFORCE_GLOBAL_UNIQUE) or (self.vrf and self.vrf.enforce_unique):
                 duplicate_prefixes = self.get_duplicates()
@@ -373,6 +382,14 @@ class Prefix(PrimaryModel):
             return self.prefix.version
         return None
 
+    @property
+    def depth(self):
+        return self._depth
+
+    @property
+    def children(self):
+        return self._children
+
     def _set_prefix_length(self, value):
         """
         Expose the IPNetwork object's prefixlen attribute on the parent model so that it can be manipulated directly,
@@ -385,6 +402,26 @@ class Prefix(PrimaryModel):
     def get_status_class(self):
         return PrefixStatusChoices.CSS_CLASSES.get(self.status)
 
+    def get_parents(self, include_self=False):
+        """
+        Return all containing Prefixes in the hierarchy.
+        """
+        lookup = 'net_contains_or_equals' if include_self else 'net_contains'
+        return Prefix.objects.filter(**{
+            'vrf': self.vrf,
+            f'prefix__{lookup}': self.prefix
+        })
+
+    def get_children(self, include_self=False):
+        """
+        Return all covered Prefixes in the hierarchy.
+        """
+        lookup = 'net_contained_or_equal' if include_self else 'net_contained'
+        return Prefix.objects.filter(**{
+            'vrf': self.vrf,
+            f'prefix__{lookup}': self.prefix
+        })
+
     def get_duplicates(self):
         return Prefix.objects.filter(vrf=self.vrf, prefix=str(self.prefix)).exclude(pk=self.pk)
 
@@ -426,8 +463,8 @@ class Prefix(PrimaryModel):
         child_ips = netaddr.IPSet([ip.address.ip for ip in self.get_child_ips()])
         available_ips = prefix - child_ips
 
-        # IPv6, pool, or IPv4 /31 sets are fully usable
-        if self.family == 6 or self.is_pool or self.prefix.prefixlen == 31:
+        # IPv6, pool, or IPv4 /31-/32 sets are fully usable
+        if self.family == 6 or self.is_pool or (self.family == 4 and self.prefix.prefixlen >= 31):
             return available_ips
 
         # For "normal" IPv4 prefixes, omit first and last addresses
@@ -467,14 +504,16 @@ class Prefix(PrimaryModel):
                 vrf=self.vrf
             )
             child_prefixes = netaddr.IPSet([p.prefix for p in queryset])
-            return int(float(child_prefixes.size) / self.prefix.size * 100)
+            utilization = int(float(child_prefixes.size) / self.prefix.size * 100)
         else:
             # Compile an IPSet to avoid counting duplicate IPs
             child_count = netaddr.IPSet([ip.address.ip for ip in self.get_child_ips()]).size
             prefix_size = self.prefix.size
             if self.prefix.version == 4 and self.prefix.prefixlen < 31 and not self.is_pool:
                 prefix_size -= 2
-            return int(float(child_count) / prefix_size * 100)
+            utilization = int(float(child_count) / prefix_size * 100)
+
+        return min(utilization, 100)
 
 
 @extras_features('custom_fields', 'custom_links', 'export_templates', 'tags', 'webhooks')
@@ -610,18 +649,15 @@ class IPAddress(PrimaryModel):
 
         # Check for primary IP assignment that doesn't match the assigned device/VM
         if self.pk:
-            device = Device.objects.filter(Q(primary_ip4=self) | Q(primary_ip6=self)).first()
-            if device:
-                if getattr(self.assigned_object, 'device', None) != device:
-                    raise ValidationError({
-                        'interface': f"IP address is primary for device {device} but not assigned to it!"
-                    })
-            vm = VirtualMachine.objects.filter(Q(primary_ip4=self) | Q(primary_ip6=self)).first()
-            if vm:
-                if getattr(self.assigned_object, 'virtual_machine', None) != vm:
-                    raise ValidationError({
-                        'vminterface': f"IP address is primary for virtual machine {vm} but not assigned to it!"
-                    })
+            for cls, attr in ((Device, 'device'), (VirtualMachine, 'virtual_machine')):
+                parent = cls.objects.filter(Q(primary_ip4=self) | Q(primary_ip6=self)).first()
+                if parent and getattr(self.assigned_object, attr) != parent:
+                    # Check for a NAT relationship
+                    if not self.nat_inside or getattr(self.nat_inside.assigned_object, attr) != parent:
+                        raise ValidationError({
+                            'interface': f"IP address is primary for {cls._meta.model_name} {parent} but "
+                                         f"not assigned to it!"
+                        })
 
         # Validate IP status selection
         if self.status == IPAddressStatusChoices.STATUS_SLAAC and self.family != 6:

netbox/ipam/querysets.py

@@ -1,27 +1,32 @@
 from django.contrib.contenttypes.models import ContentType
 from django.db.models import Q
+from django.db.models.expressions import RawSQL
 
 from utilities.querysets import RestrictedQuerySet
 
 
 class PrefixQuerySet(RestrictedQuerySet):
 
-    def annotate_tree(self):
+    def annotate_hierarchy(self):
         """
-        Annotate the number of parent and child prefixes for each Prefix. Raw SQL is needed for these subqueries
-        because we need to cast NULL VRF values to integers for comparison. (NULL != NULL).
+        Annotate the depth and number of child prefixes for each Prefix. Cast null VRF values to zero for
+        comparison. (NULL != NULL).
         """
-        return self.extra(
-            select={
-                'parents': 'SELECT COUNT(U0."prefix") AS "c" '
-                           'FROM "ipam_prefix" U0 '
-                           'WHERE (U0."prefix" >> "ipam_prefix"."prefix" '
-                           'AND COALESCE(U0."vrf_id", 0) = COALESCE("ipam_prefix"."vrf_id", 0))',
-                'children': 'SELECT COUNT(U1."prefix") AS "c" '
-                            'FROM "ipam_prefix" U1 '
-                            'WHERE (U1."prefix" << "ipam_prefix"."prefix" '
-                            'AND COALESCE(U1."vrf_id", 0) = COALESCE("ipam_prefix"."vrf_id", 0))',
-            }
+        return self.annotate(
+            hierarchy_depth=RawSQL(
+                'SELECT COUNT(DISTINCT U0."prefix") AS "c" '
+                'FROM "ipam_prefix" U0 '
+                'WHERE (U0."prefix" >> "ipam_prefix"."prefix" '
+                'AND COALESCE(U0."vrf_id", 0) = COALESCE("ipam_prefix"."vrf_id", 0))',
+                ()
+            ),
+            hierarchy_children=RawSQL(
+                'SELECT COUNT(U1."prefix") AS "c" '
+                'FROM "ipam_prefix" U1 '
+                'WHERE (U1."prefix" << "ipam_prefix"."prefix" '
+                'AND COALESCE(U1."vrf_id", 0) = COALESCE("ipam_prefix"."vrf_id", 0))',
+                ()
+            )
         )
 
 

netbox/ipam/signals.py

@@ -1,9 +1,52 @@
-from django.db.models.signals import pre_delete
+from django.db.models.signals import post_delete, post_save, pre_delete
 from django.dispatch import receiver
 
 from dcim.models import Device
 from virtualization.models import VirtualMachine
-from .models import IPAddress
+from .models import IPAddress, Prefix
+
+
+def update_parents_children(prefix):
+    """
+    Update depth on prefix & containing prefixes
+    """
+    parents = prefix.get_parents(include_self=True).annotate_hierarchy()
+    for parent in parents:
+        parent._children = parent.hierarchy_children
+    Prefix.objects.bulk_update(parents, ['_children'], batch_size=100)
+
+
+def update_children_depth(prefix):
+    """
+    Update children count on prefix & contained prefixes
+    """
+    children = prefix.get_children(include_self=True).annotate_hierarchy()
+    for child in children:
+        child._depth = child.hierarchy_depth
+    Prefix.objects.bulk_update(children, ['_depth'], batch_size=100)
+
+
+@receiver(post_save, sender=Prefix)
+def handle_prefix_saved(instance, created, **kwargs):
+
+    # Prefix has changed (or new instance has been created)
+    if created or instance.vrf != instance._vrf or instance.prefix != instance._prefix:
+
+        update_parents_children(instance)
+        update_children_depth(instance)
+
+        # If this is not a new prefix, clean up parent/children of previous prefix
+        if not created:
+            old_prefix = Prefix(vrf=instance._vrf, prefix=instance._prefix)
+            update_parents_children(old_prefix)
+            update_children_depth(old_prefix)
+
+
+@receiver(post_delete, sender=Prefix)
+def handle_prefix_deleted(instance, **kwargs):
+
+    update_parents_children(instance)
+    update_children_depth(instance)
 
 
 @receiver(pre_delete, sender=IPAddress)

netbox/ipam/tables.py

@@ -15,7 +15,7 @@ AVAILABLE_LABEL = mark_safe('<span class="label label-success">Available</span>'
 
 PREFIX_LINK = """
 {% load helpers %}
-{% for i in record.parents|as_range %}
+{% for i in record.depth|as_range %}
     <i class="mdi mdi-circle-small"></i>
 {% endfor %}
 <a href="{% if record.pk %}{% url 'ipam:prefix' pk=record.pk %}{% else %}{% url 'ipam:prefix_add' %}?prefix={{ record }}{% if object.vrf %}&vrf={{ object.vrf.pk }}{% endif %}{% if object.site %}&site={{ object.site.pk }}{% endif %}{% if object.tenant %}&tenant_group={{ object.tenant.group.pk }}&tenant={{ object.tenant.pk }}{% endif %}{% endif %}">{{ record.prefix }}</a>
@@ -65,7 +65,7 @@ VLAN_LINK = """
 {% if record.pk %}
     <a href="{{ record.get_absolute_url }}">{{ record.vid }}</a>
 {% elif perms.ipam.add_vlan %}
-    <a href="{% url 'ipam:vlan_add' %}?vid={{ record.vid }}&group={{ vlan_group.pk }}{% if vlan_group.site %}&site={{ vlan_group.site.pk }}{% endif %}" class="btn btn-xs btn-success">{{ record.available }} VLAN{{ record.available|pluralize }} available</a>
+    <a href="{% url 'ipam:vlan_add' %}?vid={{ record.vid }}{% if record.vlan_group %}&group={{ record.vlan_group.pk }}{% endif %}" class="btn btn-xs btn-success">{{ record.available }} VLAN{{ record.available|pluralize }} available</a>
 {% else %}
     {{ record.available }} VLAN{{ record.available|pluralize }} available
 {% endif %}
@@ -262,6 +262,24 @@ class PrefixTable(BaseTable):
         template_code=PREFIX_LINK,
         attrs={'td': {'class': 'text-nowrap'}}
     )
+    prefix_flat = tables.Column(
+        accessor=Accessor('prefix'),
+        linkify=True,
+        verbose_name='Prefix (Flat)'
+    )
+    depth = tables.Column(
+        accessor=Accessor('_depth'),
+        verbose_name='Depth'
+    )
+    children = LinkedCountColumn(
+        accessor=Accessor('_children'),
+        viewname='ipam:prefix_list',
+        url_params={
+            'vrf_id': 'vrf_id',
+            'within': 'prefix',
+        },
+        verbose_name='Children'
+    )
     status = ChoiceFieldColumn(
         default=AVAILABLE_LABEL
     )
@@ -287,7 +305,8 @@ class PrefixTable(BaseTable):
     class Meta(BaseTable.Meta):
         model = Prefix
         fields = (
-            'pk', 'prefix', 'status', 'children', 'vrf', 'tenant', 'site', 'vlan', 'role', 'is_pool', 'description',
+            'pk', 'prefix', 'prefix_flat', 'status', 'depth', 'children', 'vrf', 'tenant', 'site', 'vlan', 'role',
+            'is_pool', 'description',
         )
         default_columns = ('pk', 'prefix', 'status', 'vrf', 'tenant', 'site', 'vlan', 'role', 'description')
         row_attrs = {
@@ -300,15 +319,14 @@ class PrefixDetailTable(PrefixTable):
         accessor='get_utilization',
         orderable=False
     )
-    tenant = TenantColumn()
     tags = TagColumn(
         url_name='ipam:prefix_list'
     )
 
     class Meta(PrefixTable.Meta):
         fields = (
-            'pk', 'prefix', 'status', 'children', 'vrf', 'utilization', 'tenant', 'site', 'vlan', 'role', 'is_pool',
-            'description', 'tags',
+            'pk', 'prefix', 'prefix_flat', 'status', 'children', 'vrf', 'utilization', 'tenant', 'site', 'vlan', 'role',
+            'is_pool', 'description', 'tags',
         )
         default_columns = (
             'pk', 'prefix', 'status', 'children', 'vrf', 'utilization', 'tenant', 'site', 'vlan', 'role', 'description',

netbox/ipam/tests/test_api.py

@@ -186,7 +186,7 @@ class RoleTest(APIViewTestCases.APIViewTestCase):
 
 class PrefixTest(APIViewTestCases.APIViewTestCase):
     model = Prefix
-    brief_fields = ['display', 'family', 'id', 'prefix', 'url']
+    brief_fields = ['_depth', 'display', 'family', 'id', 'prefix', 'url']
     create_data = [
         {
             'prefix': '192.168.4.0/24',

netbox/ipam/tests/test_filtersets.py

@@ -400,7 +400,8 @@ class PrefixTestCase(TestCase, ChangeLoggedFilterSetTests):
             Prefix(prefix='10.0.0.0/16'),
             Prefix(prefix='2001:db8::/32'),
         )
-        Prefix.objects.bulk_create(prefixes)
+        for prefix in prefixes:
+            prefix.save()
 
     def test_family(self):
         params = {'family': '6'}
@@ -431,6 +432,18 @@ class PrefixTestCase(TestCase, ChangeLoggedFilterSetTests):
         params = {'contains': '2001:db8:0:1::/64'}
         self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
 
+    def test_depth(self):
+        params = {'depth': '0'}
+        self.assertEqual(self.filterset(params, self.queryset).qs.count(), 8)
+        params = {'depth__gt': '0'}
+        self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
+
+    def test_children(self):
+        params = {'children': '0'}
+        self.assertEqual(self.filterset(params, self.queryset).qs.count(), 8)
+        params = {'children__gt': '0'}
+        self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
+
     def test_mask_length(self):
         params = {'mask_length': '24'}
         self.assertEqual(self.filterset(params, self.queryset).qs.count(), 4)

netbox/ipam/tests/test_models.py

@@ -1,4 +1,4 @@
-import netaddr
+from netaddr import IPNetwork, IPSet
 from django.core.exceptions import ValidationError
 from django.test import TestCase, override_settings
 
@@ -10,27 +10,27 @@ class TestAggregate(TestCase):
 
     def test_get_utilization(self):
         rir = RIR.objects.create(name='RIR 1', slug='rir-1')
-        aggregate = Aggregate(prefix=netaddr.IPNetwork('10.0.0.0/8'), rir=rir)
+        aggregate = Aggregate(prefix=IPNetwork('10.0.0.0/8'), rir=rir)
         aggregate.save()
 
         # 25% utilization
         Prefix.objects.bulk_create((
-            Prefix(prefix=netaddr.IPNetwork('10.0.0.0/12')),
-            Prefix(prefix=netaddr.IPNetwork('10.16.0.0/12')),
-            Prefix(prefix=netaddr.IPNetwork('10.32.0.0/12')),
-            Prefix(prefix=netaddr.IPNetwork('10.48.0.0/12')),
+            Prefix(prefix=IPNetwork('10.0.0.0/12')),
+            Prefix(prefix=IPNetwork('10.16.0.0/12')),
+            Prefix(prefix=IPNetwork('10.32.0.0/12')),
+            Prefix(prefix=IPNetwork('10.48.0.0/12')),
         ))
         self.assertEqual(aggregate.get_utilization(), 25)
 
         # 50% utilization
         Prefix.objects.bulk_create((
-            Prefix(prefix=netaddr.IPNetwork('10.64.0.0/10')),
+            Prefix(prefix=IPNetwork('10.64.0.0/10')),
         ))
         self.assertEqual(aggregate.get_utilization(), 50)
 
         # 100% utilization
         Prefix.objects.bulk_create((
-            Prefix(prefix=netaddr.IPNetwork('10.128.0.0/9')),
+            Prefix(prefix=IPNetwork('10.128.0.0/9')),
         ))
         self.assertEqual(aggregate.get_utilization(), 100)
 
@@ -39,9 +39,9 @@ class TestPrefix(TestCase):
 
     def test_get_duplicates(self):
         prefixes = Prefix.objects.bulk_create((
-            Prefix(prefix=netaddr.IPNetwork('192.0.2.0/24')),
-            Prefix(prefix=netaddr.IPNetwork('192.0.2.0/24')),
-            Prefix(prefix=netaddr.IPNetwork('192.0.2.0/24')),
+            Prefix(prefix=IPNetwork('192.0.2.0/24')),
+            Prefix(prefix=IPNetwork('192.0.2.0/24')),
+            Prefix(prefix=IPNetwork('192.0.2.0/24')),
         ))
         duplicate_prefix_pks = [p.pk for p in prefixes[0].get_duplicates()]
 
@@ -54,11 +54,11 @@ class TestPrefix(TestCase):
             VRF(name='VRF 3'),
         ))
         prefixes = Prefix.objects.bulk_create((
-            Prefix(prefix=netaddr.IPNetwork('10.0.0.0/16'), status=PrefixStatusChoices.STATUS_CONTAINER),
-            Prefix(prefix=netaddr.IPNetwork('10.0.0.0/24'), vrf=None),
-            Prefix(prefix=netaddr.IPNetwork('10.0.1.0/24'), vrf=vrfs[0]),
-            Prefix(prefix=netaddr.IPNetwork('10.0.2.0/24'), vrf=vrfs[1]),
-            Prefix(prefix=netaddr.IPNetwork('10.0.3.0/24'), vrf=vrfs[2]),
+            Prefix(prefix=IPNetwork('10.0.0.0/16'), status=PrefixStatusChoices.STATUS_CONTAINER),
+            Prefix(prefix=IPNetwork('10.0.0.0/24'), vrf=None),
+            Prefix(prefix=IPNetwork('10.0.1.0/24'), vrf=vrfs[0]),
+            Prefix(prefix=IPNetwork('10.0.2.0/24'), vrf=vrfs[1]),
+            Prefix(prefix=IPNetwork('10.0.3.0/24'), vrf=vrfs[2]),
         ))
         child_prefix_pks = {p.pk for p in prefixes[0].get_child_prefixes()}
 
@@ -79,13 +79,13 @@ class TestPrefix(TestCase):
             VRF(name='VRF 3'),
         ))
         parent_prefix = Prefix.objects.create(
-            prefix=netaddr.IPNetwork('10.0.0.0/16'), status=PrefixStatusChoices.STATUS_CONTAINER
+            prefix=IPNetwork('10.0.0.0/16'), status=PrefixStatusChoices.STATUS_CONTAINER
         )
         ips = IPAddress.objects.bulk_create((
-            IPAddress(address=netaddr.IPNetwork('10.0.0.1/24'), vrf=None),
-            IPAddress(address=netaddr.IPNetwork('10.0.1.1/24'), vrf=vrfs[0]),
-            IPAddress(address=netaddr.IPNetwork('10.0.2.1/24'), vrf=vrfs[1]),
-            IPAddress(address=netaddr.IPNetwork('10.0.3.1/24'), vrf=vrfs[2]),
+            IPAddress(address=IPNetwork('10.0.0.1/24'), vrf=None),
+            IPAddress(address=IPNetwork('10.0.1.1/24'), vrf=vrfs[0]),
+            IPAddress(address=IPNetwork('10.0.2.1/24'), vrf=vrfs[1]),
+            IPAddress(address=IPNetwork('10.0.3.1/24'), vrf=vrfs[2]),
         ))
         child_ip_pks = {p.pk for p in parent_prefix.get_child_ips()}
 
@@ -102,16 +102,16 @@ class TestPrefix(TestCase):
     def test_get_available_prefixes(self):
 
         prefixes = Prefix.objects.bulk_create((
-            Prefix(prefix=netaddr.IPNetwork('10.0.0.0/16')),  # Parent prefix
-            Prefix(prefix=netaddr.IPNetwork('10.0.0.0/20')),
-            Prefix(prefix=netaddr.IPNetwork('10.0.32.0/20')),
-            Prefix(prefix=netaddr.IPNetwork('10.0.128.0/18')),
+            Prefix(prefix=IPNetwork('10.0.0.0/16')),  # Parent prefix
+            Prefix(prefix=IPNetwork('10.0.0.0/20')),
+            Prefix(prefix=IPNetwork('10.0.32.0/20')),
+            Prefix(prefix=IPNetwork('10.0.128.0/18')),
         ))
-        missing_prefixes = netaddr.IPSet([
-            netaddr.IPNetwork('10.0.16.0/20'),
-            netaddr.IPNetwork('10.0.48.0/20'),
-            netaddr.IPNetwork('10.0.64.0/18'),
-            netaddr.IPNetwork('10.0.192.0/18'),
+        missing_prefixes = IPSet([
+            IPNetwork('10.0.16.0/20'),
+            IPNetwork('10.0.48.0/20'),
+            IPNetwork('10.0.64.0/18'),
+            IPNetwork('10.0.192.0/18'),
         ])
         available_prefixes = prefixes[0].get_available_prefixes()
 
@@ -119,17 +119,17 @@ class TestPrefix(TestCase):
 
     def test_get_available_ips(self):
 
-        parent_prefix = Prefix.objects.create(prefix=netaddr.IPNetwork('10.0.0.0/28'))
+        parent_prefix = Prefix.objects.create(prefix=IPNetwork('10.0.0.0/28'))
         IPAddress.objects.bulk_create((
-            IPAddress(address=netaddr.IPNetwork('10.0.0.1/26')),
-            IPAddress(address=netaddr.IPNetwork('10.0.0.3/26')),
-            IPAddress(address=netaddr.IPNetwork('10.0.0.5/26')),
-            IPAddress(address=netaddr.IPNetwork('10.0.0.7/26')),
-            IPAddress(address=netaddr.IPNetwork('10.0.0.9/26')),
-            IPAddress(address=netaddr.IPNetwork('10.0.0.11/26')),
-            IPAddress(address=netaddr.IPNetwork('10.0.0.13/26')),
+            IPAddress(address=IPNetwork('10.0.0.1/26')),
+            IPAddress(address=IPNetwork('10.0.0.3/26')),
+            IPAddress(address=IPNetwork('10.0.0.5/26')),
+            IPAddress(address=IPNetwork('10.0.0.7/26')),
+            IPAddress(address=IPNetwork('10.0.0.9/26')),
+            IPAddress(address=IPNetwork('10.0.0.11/26')),
+            IPAddress(address=IPNetwork('10.0.0.13/26')),
         ))
-        missing_ips = netaddr.IPSet([
+        missing_ips = IPSet([
             '10.0.0.2/32',
             '10.0.0.4/32',
             '10.0.0.6/32',
@@ -145,39 +145,39 @@ class TestPrefix(TestCase):
     def test_get_first_available_prefix(self):
 
         prefixes = Prefix.objects.bulk_create((
-            Prefix(prefix=netaddr.IPNetwork('10.0.0.0/16')),  # Parent prefix
-            Prefix(prefix=netaddr.IPNetwork('10.0.0.0/24')),
-            Prefix(prefix=netaddr.IPNetwork('10.0.1.0/24')),
-            Prefix(prefix=netaddr.IPNetwork('10.0.2.0/24')),
+            Prefix(prefix=IPNetwork('10.0.0.0/16')),  # Parent prefix
+            Prefix(prefix=IPNetwork('10.0.0.0/24')),
+            Prefix(prefix=IPNetwork('10.0.1.0/24')),
+            Prefix(prefix=IPNetwork('10.0.2.0/24')),
         ))
-        self.assertEqual(prefixes[0].get_first_available_prefix(), netaddr.IPNetwork('10.0.3.0/24'))
+        self.assertEqual(prefixes[0].get_first_available_prefix(), IPNetwork('10.0.3.0/24'))
 
-        Prefix.objects.create(prefix=netaddr.IPNetwork('10.0.3.0/24'))
-        self.assertEqual(prefixes[0].get_first_available_prefix(), netaddr.IPNetwork('10.0.4.0/22'))
+        Prefix.objects.create(prefix=IPNetwork('10.0.3.0/24'))
+        self.assertEqual(prefixes[0].get_first_available_prefix(), IPNetwork('10.0.4.0/22'))
 
     def test_get_first_available_ip(self):
 
-        parent_prefix = Prefix.objects.create(prefix=netaddr.IPNetwork('10.0.0.0/24'))
+        parent_prefix = Prefix.objects.create(prefix=IPNetwork('10.0.0.0/24'))
         IPAddress.objects.bulk_create((
-            IPAddress(address=netaddr.IPNetwork('10.0.0.1/24')),
-            IPAddress(address=netaddr.IPNetwork('10.0.0.2/24')),
-            IPAddress(address=netaddr.IPNetwork('10.0.0.3/24')),
+            IPAddress(address=IPNetwork('10.0.0.1/24')),
+            IPAddress(address=IPNetwork('10.0.0.2/24')),
+            IPAddress(address=IPNetwork('10.0.0.3/24')),
         ))
         self.assertEqual(parent_prefix.get_first_available_ip(), '10.0.0.4/24')
 
-        IPAddress.objects.create(address=netaddr.IPNetwork('10.0.0.4/24'))
+        IPAddress.objects.create(address=IPNetwork('10.0.0.4/24'))
         self.assertEqual(parent_prefix.get_first_available_ip(), '10.0.0.5/24')
 
     def test_get_utilization(self):
 
         # Container Prefix
         prefix = Prefix.objects.create(
-            prefix=netaddr.IPNetwork('10.0.0.0/24'),
+            prefix=IPNetwork('10.0.0.0/24'),
             status=PrefixStatusChoices.STATUS_CONTAINER
         )
         Prefix.objects.bulk_create((
-            Prefix(prefix=netaddr.IPNetwork('10.0.0.0/26')),
-            Prefix(prefix=netaddr.IPNetwork('10.0.0.128/26')),
+            Prefix(prefix=IPNetwork('10.0.0.0/26')),
+            Prefix(prefix=IPNetwork('10.0.0.128/26')),
         ))
         self.assertEqual(prefix.get_utilization(), 50)
 
@@ -186,7 +186,7 @@ class TestPrefix(TestCase):
         prefix.save()
         IPAddress.objects.bulk_create(
             # Create 32 IPAddresses within the Prefix
-            [IPAddress(address=netaddr.IPNetwork('10.0.0.{}/24'.format(i))) for i in range(1, 33)]
+            [IPAddress(address=IPNetwork('10.0.0.{}/24'.format(i))) for i in range(1, 33)]
         )
         self.assertEqual(prefix.get_utilization(), 12)  # ~= 12%
 
@@ -196,36 +196,234 @@ class TestPrefix(TestCase):
 
     @override_settings(ENFORCE_GLOBAL_UNIQUE=False)
     def test_duplicate_global(self):
-        Prefix.objects.create(prefix=netaddr.IPNetwork('192.0.2.0/24'))
-        duplicate_prefix = Prefix(prefix=netaddr.IPNetwork('192.0.2.0/24'))
+        Prefix.objects.create(prefix=IPNetwork('192.0.2.0/24'))
+        duplicate_prefix = Prefix(prefix=IPNetwork('192.0.2.0/24'))
         self.assertIsNone(duplicate_prefix.clean())
 
     @override_settings(ENFORCE_GLOBAL_UNIQUE=True)
     def test_duplicate_global_unique(self):
-        Prefix.objects.create(prefix=netaddr.IPNetwork('192.0.2.0/24'))
-        duplicate_prefix = Prefix(prefix=netaddr.IPNetwork('192.0.2.0/24'))
+        Prefix.objects.create(prefix=IPNetwork('192.0.2.0/24'))
+        duplicate_prefix = Prefix(prefix=IPNetwork('192.0.2.0/24'))
         self.assertRaises(ValidationError, duplicate_prefix.clean)
 
     def test_duplicate_vrf(self):
         vrf = VRF.objects.create(name='Test', rd='1:1', enforce_unique=False)
-        Prefix.objects.create(vrf=vrf, prefix=netaddr.IPNetwork('192.0.2.0/24'))
-        duplicate_prefix = Prefix(vrf=vrf, prefix=netaddr.IPNetwork('192.0.2.0/24'))
+        Prefix.objects.create(vrf=vrf, prefix=IPNetwork('192.0.2.0/24'))
+        duplicate_prefix = Prefix(vrf=vrf, prefix=IPNetwork('192.0.2.0/24'))
         self.assertIsNone(duplicate_prefix.clean())
 
     def test_duplicate_vrf_unique(self):
         vrf = VRF.objects.create(name='Test', rd='1:1', enforce_unique=True)
-        Prefix.objects.create(vrf=vrf, prefix=netaddr.IPNetwork('192.0.2.0/24'))
-        duplicate_prefix = Prefix(vrf=vrf, prefix=netaddr.IPNetwork('192.0.2.0/24'))
+        Prefix.objects.create(vrf=vrf, prefix=IPNetwork('192.0.2.0/24'))
+        duplicate_prefix = Prefix(vrf=vrf, prefix=IPNetwork('192.0.2.0/24'))
         self.assertRaises(ValidationError, duplicate_prefix.clean)
 
 
+class TestPrefixHierarchy(TestCase):
+    """
+    Test the automatic updating of depth and child count in response to changes made within
+    the prefix hierarchy.
+    """
+    @classmethod
+    def setUpTestData(cls):
+
+        prefixes = (
+
+            # IPv4
+            Prefix(prefix='10.0.0.0/8', _depth=0, _children=2),
+            Prefix(prefix='10.0.0.0/16', _depth=1, _children=1),
+            Prefix(prefix='10.0.0.0/24', _depth=2, _children=0),
+
+            # IPv6
+            Prefix(prefix='2001:db8::/32', _depth=0, _children=2),
+            Prefix(prefix='2001:db8::/40', _depth=1, _children=1),
+            Prefix(prefix='2001:db8::/48', _depth=2, _children=0),
+
+        )
+        Prefix.objects.bulk_create(prefixes)
+
+    def test_create_prefix4(self):
+        # Create 10.0.0.0/12
+        Prefix(prefix='10.0.0.0/12').save()
+
+        prefixes = Prefix.objects.filter(prefix__family=4)
+        self.assertEqual(prefixes[0].prefix, IPNetwork('10.0.0.0/8'))
+        self.assertEqual(prefixes[0]._depth, 0)
+        self.assertEqual(prefixes[0]._children, 3)
+        self.assertEqual(prefixes[1].prefix, IPNetwork('10.0.0.0/12'))
+        self.assertEqual(prefixes[1]._depth, 1)
+        self.assertEqual(prefixes[1]._children, 2)
+        self.assertEqual(prefixes[2].prefix, IPNetwork('10.0.0.0/16'))
+        self.assertEqual(prefixes[2]._depth, 2)
+        self.assertEqual(prefixes[2]._children, 1)
+        self.assertEqual(prefixes[3].prefix, IPNetwork('10.0.0.0/24'))
+        self.assertEqual(prefixes[3]._depth, 3)
+        self.assertEqual(prefixes[3]._children, 0)
+
+    def test_create_prefix6(self):
+        # Create 2001:db8::/36
+        Prefix(prefix='2001:db8::/36').save()
+
+        prefixes = Prefix.objects.filter(prefix__family=6)
+        self.assertEqual(prefixes[0].prefix, IPNetwork('2001:db8::/32'))
+        self.assertEqual(prefixes[0]._depth, 0)
+        self.assertEqual(prefixes[0]._children, 3)
+        self.assertEqual(prefixes[1].prefix, IPNetwork('2001:db8::/36'))
+        self.assertEqual(prefixes[1]._depth, 1)
+        self.assertEqual(prefixes[1]._children, 2)
+        self.assertEqual(prefixes[2].prefix, IPNetwork('2001:db8::/40'))
+        self.assertEqual(prefixes[2]._depth, 2)
+        self.assertEqual(prefixes[2]._children, 1)
+        self.assertEqual(prefixes[3].prefix, IPNetwork('2001:db8::/48'))
+        self.assertEqual(prefixes[3]._depth, 3)
+        self.assertEqual(prefixes[3]._children, 0)
+
+    def test_update_prefix4(self):
+        # Change 10.0.0.0/24 to 10.0.0.0/12
+        p = Prefix.objects.get(prefix='10.0.0.0/24')
+        p.prefix = '10.0.0.0/12'
+        p.save()
+
+        prefixes = Prefix.objects.filter(prefix__family=4)
+        self.assertEqual(prefixes[0].prefix, IPNetwork('10.0.0.0/8'))
+        self.assertEqual(prefixes[0]._depth, 0)
+        self.assertEqual(prefixes[0]._children, 2)
+        self.assertEqual(prefixes[1].prefix, IPNetwork('10.0.0.0/12'))
+        self.assertEqual(prefixes[1]._depth, 1)
+        self.assertEqual(prefixes[1]._children, 1)
+        self.assertEqual(prefixes[2].prefix, IPNetwork('10.0.0.0/16'))
+        self.assertEqual(prefixes[2]._depth, 2)
+        self.assertEqual(prefixes[2]._children, 0)
+
+    def test_update_prefix6(self):
+        # Change 2001:db8::/48 to 2001:db8::/36
+        p = Prefix.objects.get(prefix='2001:db8::/48')
+        p.prefix = '2001:db8::/36'
+        p.save()
+
+        prefixes = Prefix.objects.filter(prefix__family=6)
+        self.assertEqual(prefixes[0].prefix, IPNetwork('2001:db8::/32'))
+        self.assertEqual(prefixes[0]._depth, 0)
+        self.assertEqual(prefixes[0]._children, 2)
+        self.assertEqual(prefixes[1].prefix, IPNetwork('2001:db8::/36'))
+        self.assertEqual(prefixes[1]._depth, 1)
+        self.assertEqual(prefixes[1]._children, 1)
+        self.assertEqual(prefixes[2].prefix, IPNetwork('2001:db8::/40'))
+        self.assertEqual(prefixes[2]._depth, 2)
+        self.assertEqual(prefixes[2]._children, 0)
+
+    def test_update_prefix_vrf4(self):
+        vrf = VRF(name='VRF A')
+        vrf.save()
+
+        # Move 10.0.0.0/16 to a VRF
+        p = Prefix.objects.get(prefix='10.0.0.0/16')
+        p.vrf = vrf
+        p.save()
+
+        prefixes = Prefix.objects.filter(vrf__isnull=True, prefix__family=4)
+        self.assertEqual(prefixes[0].prefix, IPNetwork('10.0.0.0/8'))
+        self.assertEqual(prefixes[0]._depth, 0)
+        self.assertEqual(prefixes[0]._children, 1)
+        self.assertEqual(prefixes[1].prefix, IPNetwork('10.0.0.0/24'))
+        self.assertEqual(prefixes[1]._depth, 1)
+        self.assertEqual(prefixes[1]._children, 0)
+
+        prefixes = Prefix.objects.filter(vrf=vrf)
+        self.assertEqual(prefixes[0].prefix, IPNetwork('10.0.0.0/16'))
+        self.assertEqual(prefixes[0]._depth, 0)
+        self.assertEqual(prefixes[0]._children, 0)
+
+    def test_update_prefix_vrf6(self):
+        vrf = VRF(name='VRF A')
+        vrf.save()
+
+        # Move 2001:db8::/40 to a VRF
+        p = Prefix.objects.get(prefix='2001:db8::/40')
+        p.vrf = vrf
+        p.save()
+
+        prefixes = Prefix.objects.filter(vrf__isnull=True, prefix__family=6)
+        self.assertEqual(prefixes[0].prefix, IPNetwork('2001:db8::/32'))
+        self.assertEqual(prefixes[0]._depth, 0)
+        self.assertEqual(prefixes[0]._children, 1)
+        self.assertEqual(prefixes[1].prefix, IPNetwork('2001:db8::/48'))
+        self.assertEqual(prefixes[1]._depth, 1)
+        self.assertEqual(prefixes[1]._children, 0)
+
+        prefixes = Prefix.objects.filter(vrf=vrf)
+        self.assertEqual(prefixes[0].prefix, IPNetwork('2001:db8::/40'))
+        self.assertEqual(prefixes[0]._depth, 0)
+        self.assertEqual(prefixes[0]._children, 0)
+
+    def test_delete_prefix4(self):
+        # Delete 10.0.0.0/16
+        Prefix.objects.filter(prefix='10.0.0.0/16').delete()
+
+        prefixes = Prefix.objects.filter(prefix__family=4)
+        self.assertEqual(prefixes[0].prefix, IPNetwork('10.0.0.0/8'))
+        self.assertEqual(prefixes[0]._depth, 0)
+        self.assertEqual(prefixes[0]._children, 1)
+        self.assertEqual(prefixes[1].prefix, IPNetwork('10.0.0.0/24'))
+        self.assertEqual(prefixes[1]._depth, 1)
+        self.assertEqual(prefixes[1]._children, 0)
+
+    def test_delete_prefix6(self):
+        # Delete 2001:db8::/40
+        Prefix.objects.filter(prefix='2001:db8::/40').delete()
+
+        prefixes = Prefix.objects.filter(prefix__family=6)
+        self.assertEqual(prefixes[0].prefix, IPNetwork('2001:db8::/32'))
+        self.assertEqual(prefixes[0]._depth, 0)
+        self.assertEqual(prefixes[0]._children, 1)
+        self.assertEqual(prefixes[1].prefix, IPNetwork('2001:db8::/48'))
+        self.assertEqual(prefixes[1]._depth, 1)
+        self.assertEqual(prefixes[1]._children, 0)
+
+    def test_duplicate_prefix4(self):
+        # Duplicate 10.0.0.0/16
+        Prefix(prefix='10.0.0.0/16').save()
+
+        prefixes = Prefix.objects.filter(prefix__family=4)
+        self.assertEqual(prefixes[0].prefix, IPNetwork('10.0.0.0/8'))
+        self.assertEqual(prefixes[0]._depth, 0)
+        self.assertEqual(prefixes[0]._children, 3)
+        self.assertEqual(prefixes[1].prefix, IPNetwork('10.0.0.0/16'))
+        self.assertEqual(prefixes[1]._depth, 1)
+        self.assertEqual(prefixes[1]._children, 1)
+        self.assertEqual(prefixes[2].prefix, IPNetwork('10.0.0.0/16'))
+        self.assertEqual(prefixes[2]._depth, 1)
+        self.assertEqual(prefixes[2]._children, 1)
+        self.assertEqual(prefixes[3].prefix, IPNetwork('10.0.0.0/24'))
+        self.assertEqual(prefixes[3]._depth, 2)
+        self.assertEqual(prefixes[3]._children, 0)
+
+    def test_duplicate_prefix6(self):
+        # Duplicate 2001:db8::/40
+        Prefix(prefix='2001:db8::/40').save()
+
+        prefixes = Prefix.objects.filter(prefix__family=6)
+        self.assertEqual(prefixes[0].prefix, IPNetwork('2001:db8::/32'))
+        self.assertEqual(prefixes[0]._depth, 0)
+        self.assertEqual(prefixes[0]._children, 3)
+        self.assertEqual(prefixes[1].prefix, IPNetwork('2001:db8::/40'))
+        self.assertEqual(prefixes[1]._depth, 1)
+        self.assertEqual(prefixes[1]._children, 1)
+        self.assertEqual(prefixes[2].prefix, IPNetwork('2001:db8::/40'))
+        self.assertEqual(prefixes[2]._depth, 1)
+        self.assertEqual(prefixes[2]._children, 1)
+        self.assertEqual(prefixes[3].prefix, IPNetwork('2001:db8::/48'))
+        self.assertEqual(prefixes[3]._depth, 2)
+        self.assertEqual(prefixes[3]._children, 0)
+
+
 class TestIPAddress(TestCase):
 
     def test_get_duplicates(self):
         ips = IPAddress.objects.bulk_create((
-            IPAddress(address=netaddr.IPNetwork('192.0.2.1/24')),
-            IPAddress(address=netaddr.IPNetwork('192.0.2.1/24')),
-            IPAddress(address=netaddr.IPNetwork('192.0.2.1/24')),
+            IPAddress(address=IPNetwork('192.0.2.1/24')),
+            IPAddress(address=IPNetwork('192.0.2.1/24')),
+            IPAddress(address=IPNetwork('192.0.2.1/24')),
         ))
         duplicate_ip_pks = [p.pk for p in ips[0].get_duplicates()]
 
@@ -237,44 +435,44 @@ class TestIPAddress(TestCase):
 
     @override_settings(ENFORCE_GLOBAL_UNIQUE=False)
     def test_duplicate_global(self):
-        IPAddress.objects.create(address=netaddr.IPNetwork('192.0.2.1/24'))
-        duplicate_ip = IPAddress(address=netaddr.IPNetwork('192.0.2.1/24'))
+        IPAddress.objects.create(address=IPNetwork('192.0.2.1/24'))
+        duplicate_ip = IPAddress(address=IPNetwork('192.0.2.1/24'))
         self.assertIsNone(duplicate_ip.clean())
 
     @override_settings(ENFORCE_GLOBAL_UNIQUE=True)
     def test_duplicate_global_unique(self):
-        IPAddress.objects.create(address=netaddr.IPNetwork('192.0.2.1/24'))
-        duplicate_ip = IPAddress(address=netaddr.IPNetwork('192.0.2.1/24'))
+        IPAddress.objects.create(address=IPNetwork('192.0.2.1/24'))
+        duplicate_ip = IPAddress(address=IPNetwork('192.0.2.1/24'))
         self.assertRaises(ValidationError, duplicate_ip.clean)
 
     def test_duplicate_vrf(self):
         vrf = VRF.objects.create(name='Test', rd='1:1', enforce_unique=False)
-        IPAddress.objects.create(vrf=vrf, address=netaddr.IPNetwork('192.0.2.1/24'))
-        duplicate_ip = IPAddress(vrf=vrf, address=netaddr.IPNetwork('192.0.2.1/24'))
+        IPAddress.objects.create(vrf=vrf, address=IPNetwork('192.0.2.1/24'))
+        duplicate_ip = IPAddress(vrf=vrf, address=IPNetwork('192.0.2.1/24'))
         self.assertIsNone(duplicate_ip.clean())
 
     def test_duplicate_vrf_unique(self):
         vrf = VRF.objects.create(name='Test', rd='1:1', enforce_unique=True)
-        IPAddress.objects.create(vrf=vrf, address=netaddr.IPNetwork('192.0.2.1/24'))
-        duplicate_ip = IPAddress(vrf=vrf, address=netaddr.IPNetwork('192.0.2.1/24'))
+        IPAddress.objects.create(vrf=vrf, address=IPNetwork('192.0.2.1/24'))
+        duplicate_ip = IPAddress(vrf=vrf, address=IPNetwork('192.0.2.1/24'))
         self.assertRaises(ValidationError, duplicate_ip.clean)
 
     @override_settings(ENFORCE_GLOBAL_UNIQUE=True)
     def test_duplicate_nonunique_nonrole_role(self):
-        IPAddress.objects.create(address=netaddr.IPNetwork('192.0.2.1/24'))
-        duplicate_ip = IPAddress(address=netaddr.IPNetwork('192.0.2.1/24'), role=IPAddressRoleChoices.ROLE_VIP)
+        IPAddress.objects.create(address=IPNetwork('192.0.2.1/24'))
+        duplicate_ip = IPAddress(address=IPNetwork('192.0.2.1/24'), role=IPAddressRoleChoices.ROLE_VIP)
         self.assertRaises(ValidationError, duplicate_ip.clean)
 
     @override_settings(ENFORCE_GLOBAL_UNIQUE=True)
     def test_duplicate_nonunique_role_nonrole(self):
-        IPAddress.objects.create(address=netaddr.IPNetwork('192.0.2.1/24'), role=IPAddressRoleChoices.ROLE_VIP)
-        duplicate_ip = IPAddress(address=netaddr.IPNetwork('192.0.2.1/24'))
+        IPAddress.objects.create(address=IPNetwork('192.0.2.1/24'), role=IPAddressRoleChoices.ROLE_VIP)
+        duplicate_ip = IPAddress(address=IPNetwork('192.0.2.1/24'))
         self.assertRaises(ValidationError, duplicate_ip.clean)
 
     @override_settings(ENFORCE_GLOBAL_UNIQUE=True)
     def test_duplicate_nonunique_role(self):
-        IPAddress.objects.create(address=netaddr.IPNetwork('192.0.2.1/24'), role=IPAddressRoleChoices.ROLE_VIP)
-        IPAddress.objects.create(address=netaddr.IPNetwork('192.0.2.1/24'), role=IPAddressRoleChoices.ROLE_VIP)
+        IPAddress.objects.create(address=IPNetwork('192.0.2.1/24'), role=IPAddressRoleChoices.ROLE_VIP)
+        IPAddress.objects.create(address=IPNetwork('192.0.2.1/24'), role=IPAddressRoleChoices.ROLE_VIP)
 
 
 class TestVLANGroup(TestCase):

netbox/ipam/tests/test_views.py

@@ -333,10 +333,10 @@ class VLANGroupTestCase(ViewTestCases.OrganizationalObjectViewTestCase):
         }
 
         cls.csv_data = (
-            "name,slug,description",
-            "VLAN Group 4,vlan-group-4,Fourth VLAN group",
-            "VLAN Group 5,vlan-group-5,Fifth VLAN group",
-            "VLAN Group 6,vlan-group-6,Sixth VLAN group",
+            f"name,slug,scope_type,scope_id,description",
+            f"VLAN Group 4,vlan-group-4,,,Fourth VLAN group",
+            f"VLAN Group 5,vlan-group-5,dcim.site,{sites[0].pk},Fifth VLAN group",
+            f"VLAN Group 6,vlan-group-6,dcim.site,{sites[1].pk},Sixth VLAN group",
         )
 
         cls.bulk_edit_data = {

netbox/ipam/utils.py

@@ -68,26 +68,102 @@ def add_available_ipaddresses(prefix, ipaddress_list, is_pool=False):
     return output
 
 
-def add_available_vlans(vlan_group, vlans):
+def add_available_vlans(vlans, vlan_group=None):
     """
     Create fake records for all gaps between used VLANs
     """
     if not vlans:
-        return [{'vid': VLAN_VID_MIN, 'available': VLAN_VID_MAX - VLAN_VID_MIN + 1}]
+        return [{
+            'vid': VLAN_VID_MIN,
+            'vlan_group': vlan_group,
+            'available': VLAN_VID_MAX - VLAN_VID_MIN + 1
+        }]
 
     prev_vid = VLAN_VID_MAX
     new_vlans = []
     for vlan in vlans:
         if vlan.vid - prev_vid > 1:
-            new_vlans.append({'vid': prev_vid + 1, 'available': vlan.vid - prev_vid - 1})
+            new_vlans.append({
+                'vid': prev_vid + 1,
+                'vlan_group': vlan_group,
+                'available': vlan.vid - prev_vid - 1,
+            })
         prev_vid = vlan.vid
 
     if vlans[0].vid > VLAN_VID_MIN:
-        new_vlans.append({'vid': VLAN_VID_MIN, 'available': vlans[0].vid - VLAN_VID_MIN})
+        new_vlans.append({
+            'vid': VLAN_VID_MIN,
+            'vlan_group': vlan_group,
+            'available': vlans[0].vid - VLAN_VID_MIN,
+        })
     if prev_vid < VLAN_VID_MAX:
-        new_vlans.append({'vid': prev_vid + 1, 'available': VLAN_VID_MAX - prev_vid})
+        new_vlans.append({
+            'vid': prev_vid + 1,
+            'vlan_group': vlan_group,
+            'available': VLAN_VID_MAX - prev_vid,
+        })
 
     vlans = list(vlans) + new_vlans
     vlans.sort(key=lambda v: v.vid if type(v) == VLAN else v['vid'])
 
     return vlans
+
+
+def rebuild_prefixes(vrf):
+    """
+    Rebuild the prefix hierarchy for all prefixes in the specified VRF (or global table).
+    """
+    def contains(parent, child):
+        return child in parent and child != parent
+
+    def push_to_stack(prefix):
+        # Increment child count on parent nodes
+        for n in stack:
+            n['children'] += 1
+        stack.append({
+            'pk': [prefix['pk']],
+            'prefix': prefix['prefix'],
+            'children': 0,
+        })
+
+    stack = []
+    update_queue = []
+    prefixes = Prefix.objects.filter(vrf=vrf).values('pk', 'prefix')
+
+    # Iterate through all Prefixes in the VRF, growing and shrinking the stack as we go
+    for i, p in enumerate(prefixes):
+
+        # Grow the stack if this is a child of the most recent prefix
+        if not stack or contains(stack[-1]['prefix'], p['prefix']):
+            push_to_stack(p)
+
+        # Handle duplicate prefixes
+        elif stack[-1]['prefix'] == p['prefix']:
+            stack[-1]['pk'].append(p['pk'])
+
+        # If this is a sibling or parent of the most recent prefix, pop nodes from the
+        # stack until we reach a parent prefix (or the root)
+        else:
+            while stack and not contains(stack[-1]['prefix'], p['prefix']):
+                node = stack.pop()
+                for pk in node['pk']:
+                    update_queue.append(
+                        Prefix(pk=pk, _depth=len(stack), _children=node['children'])
+                    )
+            push_to_stack(p)
+
+        # Flush the update queue once it reaches 100 Prefixes
+        if len(update_queue) >= 100:
+            Prefix.objects.bulk_update(update_queue, ['_depth', '_children'])
+            update_queue = []
+
+    # Clear out any prefixes remaining in the stack
+    while stack:
+        node = stack.pop()
+        for pk in node['pk']:
+            update_queue.append(
+                Prefix(pk=pk, _depth=len(stack), _children=node['children'])
+            )
+
+    # Final flush of any remaining Prefixes
+    Prefix.objects.bulk_update(update_queue, ['_depth', '_children'])

netbox/ipam/views.py

@@ -145,7 +145,6 @@ class RIRListView(generic.ObjectListView):
     filterset = filtersets.RIRFilterSet
     filterset_form = forms.RIRFilterForm
     table = tables.RIRTable
-    template_name = 'ipam/rir_list.html'
 
 
 class RIRView(generic.ObjectView):
@@ -238,7 +237,7 @@ class AggregateView(generic.ObjectView):
             'site', 'role'
         ).order_by(
             'prefix'
-        ).annotate_tree()
+        )
 
         # Add available prefixes to the table if requested
         if request.GET.get('show_available', 'true') == 'true':
@@ -352,7 +351,7 @@ class RoleBulkDeleteView(generic.BulkDeleteView):
 #
 
 class PrefixListView(generic.ObjectListView):
-    queryset = Prefix.objects.annotate_tree()
+    queryset = Prefix.objects.all()
     filterset = filtersets.PrefixFilterSet
     filterset_form = forms.PrefixFilterForm
     table = tables.PrefixDetailTable
@@ -377,7 +376,7 @@ class PrefixView(generic.ObjectView):
             prefix__net_contains=str(instance.prefix)
         ).prefetch_related(
             'site', 'role'
-        ).annotate_tree()
+        )
         parent_prefix_table = tables.PrefixTable(list(parent_prefixes), orderable=False)
         parent_prefix_table.exclude = ('vrf',)
 
@@ -407,7 +406,7 @@ class PrefixPrefixesView(generic.ObjectView):
         # Child prefixes table
         child_prefixes = instance.get_child_prefixes().restrict(request.user, 'view').prefetch_related(
             'site', 'vlan', 'role',
-        ).annotate_tree()
+        )
 
         # Add available prefixes to the table if requested
         if child_prefixes and request.GET.get('show_available', 'true') == 'true':
@@ -522,7 +521,7 @@ class IPAddressView(generic.ObjectView):
         # Parent prefixes table
         parent_prefixes = Prefix.objects.restrict(request.user, 'view').filter(
             vrf=instance.vrf,
-            prefix__net_contains=str(instance.address.ip)
+            prefix__net_contains_or_equals=str(instance.address.ip)
         ).prefetch_related(
             'site', 'role'
         )
@@ -551,6 +550,7 @@ class IPAddressView(generic.ObjectView):
             vrf=instance.vrf, address__net_contained_or_equal=str(instance.address)
         )
         related_ips_table = tables.IPAddressTable(related_ips, orderable=False)
+        paginate_table(related_ips_table, request)
 
         return {
             'parent_prefixes_table': parent_prefixes_table,
@@ -675,7 +675,7 @@ class VLANGroupView(generic.ObjectView):
             Prefetch('prefixes', queryset=Prefix.objects.restrict(request.user))
         ).order_by('vid')
         vlans_count = vlans.count()
-        vlans = add_available_vlans(instance, vlans)
+        vlans = add_available_vlans(vlans, vlan_group=instance)
 
         vlans_table = tables.VLANDetailTable(vlans)
         if request.user.has_perm('ipam.change_vlan') or request.user.has_perm('ipam.delete_vlan'):

netbox/netbox/api/authentication.py

@@ -25,6 +25,15 @@ class TokenAuthentication(authentication.TokenAuthentication):
         if not token.user.is_active:
             raise exceptions.AuthenticationFailed("User inactive")
 
+        # When LDAP authentication is active try to load user data from LDAP directory
+        if settings.REMOTE_AUTH_BACKEND == 'netbox.authentication.LDAPBackend':
+            from netbox.authentication import LDAPBackend
+            ldap_backend = LDAPBackend()
+            user = ldap_backend.populate_user(token.user.username)
+            # If the user is found in the LDAP directory use it, if not fallback to the local user
+            if user:
+                return user, token
+
         return token.user, token
 
 

netbox/netbox/authentication.py

@@ -11,7 +11,7 @@ from users.models import ObjectPermission
 from utilities.permissions import permission_is_exempt, resolve_permission, resolve_permission_ct
 
 
-class ObjectPermissionBackend(ModelBackend):
+class ObjectPermissionMixin():
 
     def get_all_permissions(self, user_obj, obj=None):
         if not user_obj.is_active or user_obj.is_anonymous:
@@ -20,13 +20,16 @@ class ObjectPermissionBackend(ModelBackend):
             user_obj._object_perm_cache = self.get_object_permissions(user_obj)
         return user_obj._object_perm_cache
 
+    def get_permission_filter(self, user_obj):
+        return Q(users=user_obj) | Q(groups__user=user_obj)
+
     def get_object_permissions(self, user_obj):
         """
         Return all permissions granted to the user by an ObjectPermission.
         """
         # Retrieve all assigned and enabled ObjectPermissions
         object_permissions = ObjectPermission.objects.filter(
-            Q(users=user_obj) | Q(groups__user=user_obj),
+            self.get_permission_filter(user_obj),
             enabled=True
         ).prefetch_related('object_types')
 
@@ -86,6 +89,10 @@ class ObjectPermissionBackend(ModelBackend):
         return model.objects.filter(constraints, pk=obj.pk).exists()
 
 
+class ObjectPermissionBackend(ObjectPermissionMixin, ModelBackend):
+    pass
+
+
 class RemoteUserBackend(_RemoteUserBackend):
     """
     Custom implementation of Django's RemoteUserBackend which provides configuration hooks for basic customization.
@@ -133,11 +140,27 @@ class RemoteUserBackend(_RemoteUserBackend):
         return False
 
 
+# Create a new instance of django-auth-ldap's LDAPBackend with our own ObjectPermissions
+try:
+    from django_auth_ldap.backend import LDAPBackend as LDAPBackend_
+
+    class NBLDAPBackend(ObjectPermissionMixin, LDAPBackend_):
+        def get_permission_filter(self, user_obj):
+            permission_filter = super().get_permission_filter(user_obj)
+            if (self.settings.FIND_GROUP_PERMS and
+                    hasattr(user_obj, "ldap_user") and
+                    hasattr(user_obj.ldap_user, "group_names")):
+                permission_filter = permission_filter | Q(groups__name__in=user_obj.ldap_user.group_names)
+            return permission_filter
+except ModuleNotFoundError:
+    pass
+
+
 class LDAPBackend:
 
     def __new__(cls, *args, **kwargs):
         try:
-            from django_auth_ldap.backend import LDAPBackend as LDAPBackend_, LDAPSettings
+            from django_auth_ldap.backend import LDAPSettings
             import ldap
         except ModuleNotFoundError as e:
             if getattr(e, 'name') == 'django_auth_ldap':
@@ -163,8 +186,7 @@ class LDAPBackend:
                 "Required parameter AUTH_LDAP_SERVER_URI is missing from ldap_config.py."
             )
 
-        # Create a new instance of django-auth-ldap's LDAPBackend
-        obj = LDAPBackend_()
+        obj = NBLDAPBackend()
 
         # Read LDAP configuration parameters from ldap_config.py instead of settings.py
         settings = LDAPSettings()

netbox/netbox/configuration.example.py

@@ -69,7 +69,7 @@ SECRET_KEY = ''
 # Specify one or more name and email address tuples representing NetBox administrators. These people will be notified of
 # application errors (assuming correct email settings are provided).
 ADMINS = [
-    # ['John Doe', 'jdoe@example.com'],
+    # ('John Doe', 'jdoe@example.com'),
 ]
 
 # URL schemes that are allowed within links in NetBox
@@ -89,8 +89,8 @@ BANNER_LOGIN = ''
 # BASE_PATH = 'netbox/'
 BASE_PATH = ''
 
-# Cache timeout in seconds. Set to 0 to dissable caching. Defaults to 900 (15 minutes)
-CACHE_TIMEOUT = 900
+# Cache timeout in seconds. Defaults to zero (disabled).
+CACHE_TIMEOUT = 0
 
 # Maximum number of days to retain logged changes. Set to 0 to retain changes indefinitely. (Default: 90)
 CHANGELOG_RETENTION = 90

netbox/netbox/constants.py

@@ -4,12 +4,12 @@ from circuits.filtersets import CircuitFilterSet, ProviderFilterSet, ProviderNet
 from circuits.models import Circuit, ProviderNetwork, Provider
 from circuits.tables import CircuitTable, ProviderNetworkTable, ProviderTable
 from dcim.filtersets import (
-    CableFilterSet, DeviceFilterSet, DeviceTypeFilterSet, PowerFeedFilterSet, RackFilterSet, LocationFilterSet,
-    SiteFilterSet, VirtualChassisFilterSet,
+    CableFilterSet, DeviceFilterSet, DeviceTypeFilterSet, PowerFeedFilterSet, RackFilterSet, RackReservationFilterSet,
+    LocationFilterSet, SiteFilterSet, VirtualChassisFilterSet,
 )
-from dcim.models import Cable, Device, DeviceType, PowerFeed, Rack, Location, Site, VirtualChassis
+from dcim.models import Cable, Device, DeviceType, Location, PowerFeed, Rack, RackReservation, Site, VirtualChassis
 from dcim.tables import (
-    CableTable, DeviceTable, DeviceTypeTable, PowerFeedTable, RackTable, LocationTable, SiteTable,
+    CableTable, DeviceTable, DeviceTypeTable, PowerFeedTable, RackTable, RackReservationTable, LocationTable, SiteTable,
     VirtualChassisTable,
 )
 from ipam.filtersets import AggregateFilterSet, IPAddressFilterSet, PrefixFilterSet, VLANFilterSet, VRFFilterSet
@@ -64,6 +64,12 @@ SEARCH_TYPES = OrderedDict((
         'table': RackTable,
         'url': 'dcim:rack_list',
     }),
+    ('rackreservation', {
+        'queryset': RackReservation.objects.prefetch_related('site', 'rack', 'user'),
+        'filterset': RackReservationFilterSet,
+        'table': RackReservationTable,
+        'url': 'dcim:rackreservation_list',
+    }),
     ('location', {
         'queryset': Location.objects.add_related_count(
             Location.objects.all(),

netbox/netbox/filtersets.py

@@ -89,13 +89,13 @@ class BaseFilterSet(django_filters.FilterSet):
             filters.MultiValueNumberFilter,
             filters.MultiValueTimeFilter
         )):
-            lookup_map = FILTER_NUMERIC_BASED_LOOKUP_MAP
+            return FILTER_NUMERIC_BASED_LOOKUP_MAP
 
         elif isinstance(existing_filter, (
             filters.TreeNodeMultipleChoiceFilter,
         )):
             # TreeNodeMultipleChoiceFilter only support negation but must maintain the `in` lookup expression
-            lookup_map = FILTER_TREENODE_NEGATION_LOOKUP_MAP
+            return FILTER_TREENODE_NEGATION_LOOKUP_MAP
 
         elif isinstance(existing_filter, (
             django_filters.ModelChoiceFilter,
@@ -103,7 +103,7 @@ class BaseFilterSet(django_filters.FilterSet):
             TagFilter
         )) or existing_filter.extra.get('choices'):
             # These filter types support only negation
-            lookup_map = FILTER_NEGATION_LOOKUP_MAP
+            return FILTER_NEGATION_LOOKUP_MAP
 
         elif isinstance(existing_filter, (
             django_filters.filters.CharFilter,
@@ -111,12 +111,9 @@ class BaseFilterSet(django_filters.FilterSet):
             filters.MultiValueCharFilter,
             filters.MultiValueMACAddressFilter
         )):
-            lookup_map = FILTER_CHAR_BASED_LOOKUP_MAP
+            return FILTER_CHAR_BASED_LOOKUP_MAP
 
-        else:
-            lookup_map = None
-
-        return lookup_map
+        return None
 
     @classmethod
     def get_filters(cls):

netbox/netbox/forms.py

@@ -11,12 +11,13 @@ OBJ_TYPE_CHOICES = (
     ('DCIM', (
         ('site', 'Sites'),
         ('rack', 'Racks'),
+        ('rackreservation', 'Rack reservations'),
         ('location', 'Locations'),
         ('devicetype', 'Device types'),
         ('device', 'Devices'),
-        ('virtualchassis', 'Virtual Chassis'),
+        ('virtualchassis', 'Virtual chassis'),
         ('cable', 'Cables'),
-        ('powerfeed', 'Power Feeds'),
+        ('powerfeed', 'Power feeds'),
     )),
     ('IPAM', (
         ('vrf', 'VRFs'),

netbox/netbox/settings.py

@@ -16,7 +16,7 @@ from django.core.validators import URLValidator
 # Environment setup
 #
 
-VERSION = '2.11.4'
+VERSION = '2.11.11'
 
 # Hostname
 HOSTNAME = platform.node()
@@ -29,10 +29,10 @@ if platform.python_version_tuple() < ('3', '6'):
     raise RuntimeError(
         "NetBox requires Python 3.6 or higher (current: Python {})".format(platform.python_version())
     )
-# TODO: Remove in NetBox v2.12
+# TODO: Remove in NetBox v3.0
 if platform.python_version_tuple() < ('3', '7'):
     warnings.warn(
-        "Support for Python 3.6 will be dropped in NetBox v2.12. Please upgrade to Python 3.7 or later at your "
+        "Support for Python 3.6 will be dropped in NetBox v3.0. Please upgrade to Python 3.7 or later at your "
         "earliest convenience."
     )
 
@@ -75,7 +75,7 @@ BANNER_TOP = getattr(configuration, 'BANNER_TOP', '')
 BASE_PATH = getattr(configuration, 'BASE_PATH', '')
 if BASE_PATH:
     BASE_PATH = BASE_PATH.strip('/') + '/'  # Enforce trailing slash only
-CACHE_TIMEOUT = getattr(configuration, 'CACHE_TIMEOUT', 900)
+CACHE_TIMEOUT = getattr(configuration, 'CACHE_TIMEOUT', 0)
 CHANGELOG_RETENTION = getattr(configuration, 'CHANGELOG_RETENTION', 90)
 CORS_ORIGIN_ALLOW_ALL = getattr(configuration, 'CORS_ORIGIN_ALLOW_ALL', False)
 CORS_ORIGIN_REGEX_WHITELIST = getattr(configuration, 'CORS_ORIGIN_REGEX_WHITELIST', [])
@@ -417,13 +417,7 @@ else:
         'ssl': CACHING_REDIS_SSL,
         'ssl_cert_reqs': None if CACHING_REDIS_SKIP_TLS_VERIFY else 'required',
     }
-
-if not CACHE_TIMEOUT:
-    CACHEOPS_ENABLED = False
-else:
-    CACHEOPS_ENABLED = True
-
-
+CACHEOPS_ENABLED = bool(CACHE_TIMEOUT)
 CACHEOPS_DEFAULTS = {
     'timeout': CACHE_TIMEOUT
 }

netbox/netbox/views/generic.py

@@ -18,9 +18,10 @@ from django_tables2.export import TableExport
 
 from extras.models import CustomField, ExportTemplate
 from utilities.error_handlers import handle_protectederror
-from utilities.exceptions import AbortTransaction
+from utilities.exceptions import AbortTransaction, PermissionsViolation
 from utilities.forms import (
-    BootstrapMixin, BulkRenameForm, ConfirmationForm, CSVDataField, ImportForm, TableConfigForm, restrict_form_fields,
+    BootstrapMixin, BulkRenameForm, ConfirmationForm, CSVDataField, CSVFileField, ImportForm, TableConfigForm,
+    restrict_form_fields,
 )
 from utilities.permissions import get_permission_for_model
 from utilities.tables import paginate_table
@@ -290,7 +291,8 @@ class ObjectEditView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View):
                     obj = form.save()
 
                     # Check that the new object conforms with any assigned object-level permissions
-                    self.queryset.get(pk=obj.pk)
+                    if not self.queryset.filter(pk=obj.pk).first():
+                        raise PermissionsViolation()
 
                 msg = '{} {}'.format(
                     'Created' if object_created else 'Modified',
@@ -304,21 +306,22 @@ class ObjectEditView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View):
                 messages.success(request, mark_safe(msg))
 
                 if '_addanother' in request.POST:
+                    redirect_url = request.path
+                    return_url = request.GET.get('return_url')
+                    if return_url is not None and is_safe_url(url=return_url, allowed_hosts=request.get_host()):
+                        redirect_url = f'{redirect_url}?return_url={return_url}'
 
                     # If the object has clone_fields, pre-populate a new instance of the form
                     if hasattr(obj, 'clone_fields'):
-                        url = '{}?{}'.format(request.path, prepare_cloned_fields(obj))
-                        return redirect(url)
+                        redirect_url += f"{'&' if return_url else '?'}{prepare_cloned_fields(obj)}"
 
-                    return redirect(request.get_full_path())
+                    return redirect(redirect_url)
 
-                return_url = form.cleaned_data.get('return_url')
-                if return_url is not None and is_safe_url(url=return_url, allowed_hosts=request.get_host()):
-                    return redirect(return_url)
-                else:
-                    return redirect(self.get_return_url(request, obj))
+                return_url = self.get_return_url(request, obj)
 
-            except ObjectDoesNotExist:
+                return redirect(return_url)
+
+            except PermissionsViolation:
                 msg = "Object save failed due to object-level permissions violation"
                 logger.debug(msg)
                 form.add_error(None, msg)
@@ -480,7 +483,7 @@ class BulkCreateView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View):
 
                     # Enforce object-level permissions
                     if self.queryset.filter(pk__in=[obj.pk for obj in new_objs]).count() != len(new_objs):
-                        raise ObjectDoesNotExist
+                        raise PermissionsViolation
 
                     # If we make it to this point, validation has succeeded on all new objects.
                     msg = "Added {} {}".format(len(new_objs), model._meta.verbose_name_plural)
@@ -494,7 +497,7 @@ class BulkCreateView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View):
             except IntegrityError:
                 pass
 
-            except ObjectDoesNotExist:
+            except PermissionsViolation:
                 msg = "Object creation failed due to object-level permissions violation"
                 logger.debug(msg)
                 form.add_error(None, msg)
@@ -565,7 +568,8 @@ class ObjectImportView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View):
                         obj = model_form.save()
 
                         # Enforce object-level permissions
-                        self.queryset.get(pk=obj.pk)
+                        if not self.queryset.filter(pk=obj.pk).first():
+                            raise PermissionsViolation()
 
                         logger.debug(f"Created {obj} (PK: {obj.pk})")
 
@@ -601,7 +605,7 @@ class ObjectImportView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View):
                 except AbortTransaction:
                     pass
 
-                except ObjectDoesNotExist:
+                except PermissionsViolation:
                     msg = "Object creation failed due to object-level permissions violation"
                     logger.debug(msg)
                     form.add_error(None, msg)
@@ -665,6 +669,22 @@ class BulkImportView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View):
                 from_form=self.model_form,
                 widget=Textarea(attrs=self.widget_attrs)
             )
+            csv_file = CSVFileField(
+                label="CSV file",
+                from_form=self.model_form,
+                required=False
+            )
+
+            def clean(self):
+                csv_rows = self.cleaned_data['csv'][1] if 'csv' in self.cleaned_data else None
+                csv_file = self.files.get('csv_file')
+
+                # Check that the user has not submitted both text data and a file
+                if csv_rows and csv_file:
+                    raise ValidationError(
+                        "Cannot process CSV text and file attachment simultaneously. Please choose only one import "
+                        "method."
+                    )
 
         return ImportForm(*args, **kwargs)
 
@@ -689,7 +709,7 @@ class BulkImportView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View):
     def post(self, request):
         logger = logging.getLogger('netbox.views.BulkImportView')
         new_objs = []
-        form = self._import_form(request.POST)
+        form = self._import_form(request.POST, request.FILES)
 
         if form.is_valid():
             logger.debug("Form validation was successful")
@@ -697,7 +717,10 @@ class BulkImportView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View):
             try:
                 # Iterate through CSV data and bind each row to a new model form instance.
                 with transaction.atomic():
-                    headers, records = form.cleaned_data['csv']
+                    if request.FILES:
+                        headers, records = form.cleaned_data['csv_file']
+                    else:
+                        headers, records = form.cleaned_data['csv']
                     for row, data in enumerate(records, start=1):
                         obj_form = self.model_form(data, headers=headers)
                         restrict_form_fields(obj_form, request.user)
@@ -712,7 +735,7 @@ class BulkImportView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View):
 
                     # Enforce object-level permissions
                     if self.queryset.filter(pk__in=[obj.pk for obj in new_objs]).count() != len(new_objs):
-                        raise ObjectDoesNotExist
+                        raise PermissionsViolation
 
                 # Compile a table containing the imported objects
                 obj_table = self.table(new_objs)
@@ -730,7 +753,7 @@ class BulkImportView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View):
             except ValidationError:
                 pass
 
-            except ObjectDoesNotExist:
+            except PermissionsViolation:
                 msg = "Object import failed due to object-level permissions violation"
                 logger.debug(msg)
                 form.add_error(None, msg)
@@ -774,9 +797,7 @@ class BulkEditView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View):
 
         # If we are editing *all* objects in the queryset, replace the PK list with all matched objects.
         if request.POST.get('_all') and self.filterset is not None:
-            pk_list = [
-                obj.pk for obj in self.filterset(request.GET, self.queryset.only('pk')).qs
-            ]
+            pk_list = self.filterset(request.GET, self.queryset.values_list('pk', flat=True)).qs
         else:
             pk_list = request.POST.getlist('pk')
 
@@ -847,7 +868,7 @@ class BulkEditView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View):
 
                         # Enforce object-level permissions
                         if self.queryset.filter(pk__in=[obj.pk for obj in updated_objects]).count() != len(updated_objects):
-                            raise ObjectDoesNotExist
+                            raise PermissionsViolation
 
                     if updated_objects:
                         msg = 'Updated {} {}'.format(len(updated_objects), model._meta.verbose_name_plural)
@@ -859,7 +880,7 @@ class BulkEditView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View):
                 except ValidationError as e:
                     messages.error(self.request, "{} failed validation: {}".format(obj, e))
 
-                except ObjectDoesNotExist:
+                except PermissionsViolation:
                     msg = "Object update failed due to object-level permissions violation"
                     logger.debug(msg)
                     form.add_error(None, msg)
@@ -954,7 +975,7 @@ class BulkRenameView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View):
 
                             # Enforce constrained permissions
                             if self.queryset.filter(pk__in=renamed_pks).count() != len(selected_objects):
-                                raise ObjectDoesNotExist
+                                raise PermissionsViolation
 
                             messages.success(request, "Renamed {} {}".format(
                                 len(selected_objects),
@@ -962,7 +983,7 @@ class BulkRenameView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View):
                             ))
                             return redirect(self.get_return_url(request))
 
-                except ObjectDoesNotExist:
+                except PermissionsViolation:
                     msg = "Object update failed due to object-level permissions violation"
                     logger.debug(msg)
                     form.add_error(None, msg)
@@ -1148,7 +1169,7 @@ class ComponentCreateView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View
 
                         # Enforce object-level permissions
                         if self.queryset.filter(pk__in=[obj.pk for obj in new_objs]).count() != len(new_objs):
-                            raise ObjectDoesNotExist
+                            raise PermissionsViolation
 
                     messages.success(request, "Added {} {}".format(
                         len(new_components), self.queryset.model._meta.verbose_name_plural
@@ -1158,7 +1179,7 @@ class ComponentCreateView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View
                     else:
                         return redirect(self.get_return_url(request))
 
-                except ObjectDoesNotExist:
+                except PermissionsViolation:
                     msg = "Component creation failed due to object-level permissions violation"
                     logger.debug(msg)
                     form.add_error(None, msg)
@@ -1231,7 +1252,7 @@ class BulkComponentCreateView(GetReturnURLMixin, ObjectPermissionRequiredMixin,
                                 component_form = self.model_form(component_data)
                                 if component_form.is_valid():
                                     instance = component_form.save()
-                                    logger.debug(f"Created {instance} on {instance.parent}")
+                                    logger.debug(f"Created {instance} on {instance.parent_object}")
                                     new_components.append(instance)
                                 else:
                                     for field, errors in component_form.errors.as_data().items():
@@ -1240,12 +1261,12 @@ class BulkComponentCreateView(GetReturnURLMixin, ObjectPermissionRequiredMixin,
 
                         # Enforce object-level permissions
                         if self.queryset.filter(pk__in=[obj.pk for obj in new_components]).count() != len(new_components):
-                            raise ObjectDoesNotExist
+                            raise PermissionsViolation
 
                 except IntegrityError:
                     pass
 
-                except ObjectDoesNotExist:
+                except PermissionsViolation:
                     msg = "Component creation failed due to object-level permissions violation"
                     logger.debug(msg)
                     form.add_error(None, msg)

netbox/secrets/tables.py

@@ -37,6 +37,7 @@ class SecretTable(BaseTable):
     )
     assigned_object = tables.Column(
         linkify=True,
+        orderable=False,
         verbose_name='Assigned object'
     )
     role = tables.Column(

netbox/secrets/views.py

@@ -92,7 +92,7 @@ def inject_deprecation_warning(request):
     """
     messages.warning(
         request,
-        mark_safe('<i class="mdi mdi-alert"></i> The secrets functionality will be moved to a plugin in NetBox v2.12. '
+        mark_safe('<i class="mdi mdi-alert"></i> The secrets functionality will be moved to a plugin in NetBox v3.0. '
                   'Please see <a href="https://github.com/netbox-community/netbox/issues/5278">issue #5278</a> for '
                   'more information.')
     )

netbox/templates/base.html

@@ -67,14 +67,14 @@
                     <p class="text-muted">{{ settings.HOSTNAME }} (v{{ settings.VERSION }})</p>
                 </div>
                 <div class="col-xs-4 text-center">
-                    <p class="text-muted">{% now 'Y-m-d H:i:s T' %}</p>
+                    <p class="text-muted">{% annotated_now %} {% now 'T' %}</p>
                 </div>
                 <div class="col-xs-4 text-right noprint">
                     <p class="text-muted">
                         <i class="mdi mdi-book-open-page-variant text-primary"></i> <a href="https://netbox.readthedocs.io/">Docs</a> &middot;
                         <i class="mdi mdi-cloud-braces text-primary"></i> <a href="{% url 'api_docs' %}">API</a> &middot;
                         <i class="mdi mdi-xml text-primary"></i> <a href="https://github.com/netbox-community/netbox">Code</a> &middot;
-                        <i class="mdi mdi-lifebuoy text-primary"></i> <a href="https://github.com/netbox-community/netbox/wiki">Help</a>
+                        <i class="mdi mdi-slack text-primary"></i> <a href="https://netdev.chat/">Community</a>
                     </p>
                 </div>
             </div>

netbox/templates/circuits/circuit.html

@@ -51,7 +51,7 @@
                 </tr>
                 <tr>
                     <td>Install Date</td>
-                    <td>{{ object.install_date|placeholder }}</td>
+                    <td>{{ object.install_date|annotated_date|placeholder }}</td>
                 </tr>
                 <tr>
                     <td>Commit Rate</td>

netbox/templates/dcim/cable_connect.html

@@ -35,13 +35,13 @@
                             <div class="form-group">
                                 <label class="col-md-3 control-label required">Region</label>
                                 <div class="col-md-9">
-                                    <p class="form-control-static">{{ termination_a.device.site.region }}</p>
+                                    <p class="form-control-static">{{ termination_a.device.site.region|placeholder }}</p>
                                 </div>
                             </div>
                             <div class="form-group">
                                 <label class="col-md-3 control-label required">Site Group</label>
                                 <div class="col-md-9">
-                                    <p class="form-control-static">{{ termination_a.device.site.group }}</p>
+                                    <p class="form-control-static">{{ termination_a.device.site.group|placeholder }}</p>
                                 </div>
                             </div>
                             <div class="form-group">
@@ -50,10 +50,16 @@
                                     <p class="form-control-static">{{ termination_a.device.site }}</p>
                                 </div>
                             </div>
+                            <div class="form-group">
+                                <label class="col-md-3 control-label required">Location</label>
+                                <div class="col-md-9">
+                                    <p class="form-control-static">{{ termination_a.device.location|placeholder }}</p>
+                                </div>
+                            </div>
                             <div class="form-group">
                                 <label class="col-md-3 control-label required">Rack</label>
                                 <div class="col-md-9">
-                                    <p class="form-control-static">{{ termination_a.device.rack|default:"None" }}</p>
+                                    <p class="form-control-static">{{ termination_a.device.rack|placeholder }}</p>
                                 </div>
                             </div>
                             <div class="form-group">

netbox/templates/dcim/device/config.html

@@ -39,9 +39,9 @@ $(document).ready(function() {
         url: "{% url 'dcim-api:device-napalm' pk=object.pk %}?method=get_config",
         dataType: 'json',
         success: function(json) {
-            $('#running_config').html($.trim(json['get_config']['running']));
-            $('#startup_config').html($.trim(json['get_config']['startup']));
-            $('#candidate_config').html($.trim(json['get_config']['candidate']));
+            $('#running_config').text($.trim(json['get_config']['running']));
+            $('#startup_config').text($.trim(json['get_config']['startup']));
+            $('#candidate_config').text($.trim(json['get_config']['candidate']));
         },
         error: function(xhr) {
             alert(xhr.responseText);

netbox/templates/dcim/device/status.html

@@ -1,7 +1,7 @@
 {% extends 'dcim/device/base.html' %}
 {% load static %}
 
-{% block title %}{{ device }} - Status{% endblock %}
+{% block title %}{{ object }} - Status{% endblock %}
 
 {% block content %}
     {% include 'inc/ajax_loader.html' %}

netbox/templates/dcim/device_component_add.html

@@ -1,4 +1,5 @@
 {% extends 'base.html' %}
+{% load helpers %}
 {% load form_helpers %}
 
 {% block title %}Create {{ component_type }}{% endblock %}
@@ -18,19 +19,34 @@
             {% endif %}
             <div class="panel panel-default">
                 <div class="panel-heading">
-                    <strong>{{ component_type|title }}</strong>
+                    <strong>{{ component_type|bettertitle }}</strong>
                 </div>
                 <div class="panel-body">
-                    {% render_form form %}
+                    {% for field in form.hidden_fields %}
+                        {{ field }}
+                    {% endfor %}
+                    {% for field in form.visible_fields %}
+                        {% if not form.custom_fields or field.name not in form.custom_fields %}
+                            {% render_field field %}
+                        {% endif %}
+                    {% endfor %}
                 </div>
             </div>
-		    <div class="form-group">
+            {% if form.custom_fields %}
+                <div class="panel panel-default">
+                    <div class="panel-heading"><strong>Custom Fields</strong></div>
+                    <div class="panel-body">
+                        {% render_custom_fields form %}
+                    </div>
+                </div>
+            {% endif %}
+            <div class="form-group">
                 <div class="col-md-9 col-md-offset-3">
                     <button type="submit" name="_create" class="btn btn-primary">Create</button>
                     <button type="submit" name="_addanother" class="btn btn-primary">Create and Add More</button>
                     <a href="{{ return_url }}" class="btn btn-default">Cancel</a>
                 </div>
-		    </div>
+            </div>
         </div>
     </div>
 </form>

netbox/templates/dcim/devicerole.html

@@ -42,7 +42,17 @@
         <tr>
           <td>Devices</td>
           <td>
-            <a href="{% url 'dcim:device_list' %}?role_id={{ object.pk }}">{{ devices_table.rows|length }}</a>
+            <a href="{% url 'dcim:device_list' %}?role_id={{ object.pk }}">{{ device_count }}</a>
+          </td>
+        </tr>
+        <tr>
+          <td>Virtual Machines</td>
+          <td>
+            {% if object.vm_role %}
+              <a href="{% url 'virtualization:virtualmachine_list' %}?role_id={{ object.pk }}">{{ virtualmachine_count }}</a>
+            {% else %}
+              &mdash;
+            {% endif %}
           </td>
         </tr>
       </table>

netbox/templates/dcim/interface.html

@@ -54,6 +54,16 @@
                             {% endif %}
                         </td>
                     </tr>
+                    <tr>
+                        <td>Management Only</td>
+                        <td>
+                            {% if object.mgmt_only %}
+                                <span class="text-success"><i class="mdi mdi-check-bold"></i></span>
+                            {% else %}
+                                <span class="text-danger"><i class="mdi mdi-close"></i></span>
+                            {% endif %}
+                        </td>
+                    </tr>
                     <tr>
                         <td>Parent</td>
                         <td>
@@ -88,7 +98,7 @@
                     </tr>
                     <tr>
                         <td>802.1Q Mode</td>
-                        <td>{{ object.get_mode_display }}</td>
+                        <td>{{ object.get_mode_display|placeholder }}</td>
                     </tr>
                 </table>
             </div>

netbox/templates/dcim/manufacturer.html

@@ -29,6 +29,12 @@
             <a href="{% url 'dcim:devicetype_list' %}?manufacturer_id={{ object.pk }}">{{ devicetypes_table.rows|length }}</a>
           </td>
         </tr>
+        <tr>
+          <td>Inventory Items</td>
+          <td>
+            <a href="{% url 'dcim:inventoryitem_list' %}?manufacturer_id={{ object.pk }}">{{ inventory_item_count }}</a>
+          </td>
+        </tr>
       </table>
     </div>
     {% plugin_left_page object %}

netbox/templates/dcim/powerfeed.html

@@ -7,10 +7,10 @@
 
 {% block breadcrumbs %}
   <li><a href="{% url 'dcim:powerfeed_list' %}">Power Feeds</a></li>
-  <li><a href="{{ object.power_panel.site.get_absolute_url }}">{{ object.power_panel.site }}</a></li>
-  <li><a href="{{ object.power_panel.get_absolute_url }}">{{ object.power_panel }}</a></li>
+  <li><a href="{% url 'dcim:powerfeed_list' %}?site_id={{ object.power_panel.site.pk }}">{{ object.power_panel.site }}</a></li>
+  <li><a href="{% url 'dcim:powerfeed_list' %}?power_panel_id={{ object.power_panel.pk }}">{{ object.power_panel }}</a></li>
   {% if object.rack %}
-    <li><a href="{{ object.rack.get_absolute_url }}">{{ object.rack }}</a></li>
+    <li><a href="{% url 'dcim:powerfeed_list' %}?rack_id={{ object.rack.pk }}">{{ object.rack }}</a></li>
   {% endif %}
   <li>{{ object }}</li>
 {% endblock %}

netbox/templates/dcim/powerpanel.html

@@ -5,7 +5,7 @@
 
 {% block breadcrumbs %}
   <li><a href="{% url 'dcim:powerpanel_list' %}">Power Panels</a></li>
-  <li><a href="{{ object.site.get_absolute_url }}">{{ object.site }}</a></li>
+  <li><a href="{% url 'dcim:powerpanel_list' %}?site_id={{ object.site.pk }}">{{ object.site }}</a></li>
   {% if object.location %}
     <li><a href="{{ object.location.get_absolute_url }}">{{ object.location }}</a></li>
   {% endif %}

netbox/templates/dcim/rack.html

@@ -9,11 +9,11 @@
 {% block breadcrumbs %}
   <li><a href="{% url 'dcim:rack_list' %}">Racks</a></li>
   <li><a href="{% url 'dcim:rack_list' %}?site_id={{ object.site.pk }}">{{ object.site }}</a></li>
-  {% if object.group %}
-    {% for group in object.group.get_ancestors %}
-      <li><a href="{{ group.get_absolute_url }}">{{ group }}</a></li>
+  {% if object.location %}
+    {% for location in object.location.get_ancestors %}
+      <li><a href="{{ location.get_absolute_url }}">{{ location }}</a></li>
     {% endfor %}
-    <li><a href="{{ object.group.get_absolute_url }}">{{ object.group }}</a></li>
+    <li><a href="{{ object.location.get_absolute_url }}">{{ object.location }}</a></li>
   {% endif %}
   <li>{{ object }}</li>
 {% endblock %}
@@ -268,7 +268,7 @@
                             </td>
                             <td>
                                 {{ resv.description }}<br />
-                                <small>{{ resv.user }} &middot; {{ resv.created }}</small>
+                                <small>{{ resv.user }} &middot; {{ resv.created|annotated_date }}</small>
                             </td>
                             <td class="text-right noprint">
                                 {% if perms.dcim.change_rackreservation %}

netbox/templates/dcim/rackreservation.html

@@ -39,10 +39,10 @@
                         </td>
                     </tr>
                     <tr>
-                        <td>Group</td>
+                        <td>Location</td>
                         <td>
-                            {% if rack.group %}
-                                <a href="{{ rack.group.get_absolute_url }}">{{ rack.group }}</a>
+                            {% if rack.location %}
+                                <a href="{{ rack.location.get_absolute_url }}">{{ rack.location }}</a>
                             {% else %}
                                 <span class="text-muted">None</span>
                             {% endif %}

netbox/templates/dcim/site.html

@@ -80,7 +80,11 @@
                     <td>
                         {% if object.time_zone %}
                             {{ object.time_zone }} (UTC {{ object.time_zone|tzoffset }})<br />
-                            <small class="text-muted">Site time: {% timezone object.time_zone %}{% now "SHORT_DATETIME_FORMAT" %}{% endtimezone %}</small>
+                            <small class="text-muted">Site time:
+                                {% timezone object.time_zone %}
+                                    {% annotated_now %}
+                                {% endtimezone %}
+                            </small>
                         {% else %}
                             <span class="text-muted">&mdash;</span>
                         {% endif %}

netbox/templates/extras/journalentry.html

@@ -25,7 +25,7 @@
                     <tr>
                         <td>Created</td>
                         <td>
-                            {{ object.created }}
+                            {{ object.created|annotated_date }}
                         </td>
                     </tr>
                     <tr>

netbox/templates/extras/objectchange.html

@@ -44,7 +44,7 @@
                     <tr>
                         <td>Time</td>
                         <td>
-                            {{ object.time }}
+                            {{ object.time|annotated_date }}
                         </td>
                     </tr>
                     <tr>
@@ -128,6 +128,8 @@
                       <span{% if k in diff_removed %} style="background-color: #ffdce0"{% endif %}>{{ k }}: {{ v|render_json }}</span>
                     {% endspaceless %}
 {% endfor %}</pre>
+                  {% elif non_atomic_change %}
+                    Warning: Comparing non-atomic change to previous change record (<a href="{% url 'extras:objectchange' pk=prev_change.pk %}">{{ prev_change.pk }}</a>)
                   {% else %}
                     <span class="text-muted">None</span>
                   {% endif %}

netbox/templates/extras/report.html

@@ -29,7 +29,7 @@
     {% endif %}
     <h1 class="title">{{ report.name }}</h1>
     {% if report.description %}
-        <p class="lead">{{ report.description }}</p>
+        <p class="lead">{{ report.description|render_markdown }}</p>
     {% endif %}
 {% endblock %}
 
@@ -38,7 +38,7 @@
         <div class="col-md-12">
             {% if report.result %}
                 Last run: <a href="{% url 'extras:report_result' job_result_pk=report.result.pk %}">
-                    <strong>{{ report.result.created }}</strong>
+                    <strong>{{ report.result.created|annotated_date }}</strong>
                 </a>
             {% endif %}
         </div>

netbox/templates/extras/report_list.html

@@ -29,10 +29,10 @@
                                     <td>
                                         {% include 'extras/inc/job_label.html' with result=report.result %}
                                     </td>
-                                    <td>{{ report.description|placeholder }}</td>
+                                    <td class="rendered-markdown">{{ report.description|render_markdown|placeholder }}</td>
                                     <td class="text-right">
                                         {% if report.result %}
-                                            <a href="{% url 'extras:report_result' job_result_pk=report.result.pk %}">{{ report.result.created }}</a>
+                                            <a href="{% url 'extras:report_result' job_result_pk=report.result.pk %}">{{ report.result.created|annotated_date }}</a>
                                         {% else %}
                                             <span class="text-muted">Never</span>
                                         {% endif %}