Merge pull request #4947 from netbox-community/develop

v2.8.9 - 2020-08-04

.github/stale.yml

@@ -4,19 +4,19 @@
 only: issues
 
 # Number of days of inactivity before an issue becomes stale
-daysUntilStale: 14
+daysUntilStale: 45
 
 # Number of days of inactivity before a stale issue is closed
-daysUntilClose: 7
+daysUntilClose: 15
 
 # Issues with these labels will never be considered stale
 exemptLabels:
   - "status: accepted"
-  - "status: gathering feedback"
   - "status: blocked"
+  - "status: needs milestone"
 
 # Label to use when marking an issue as stale
-staleLabel: wontfix
+staleLabel: "pending closure"
 
 # Comment to post when marking an issue as stale. Set to `false` to disable
 markComment: >

CONTRIBUTING.md

@@ -99,6 +99,10 @@ help prevent wasting time on something that might we might not be able to
 implement. When suggesting a new feature, also make sure it won't conflict with
 any work that's already in progress.
 
+* Once you've opened or identified an issue you'd like to work on, ask that it
+be assigned to you so that others are aware it's being worked on. A maintainer
+will then mark the issue as "accepted."
+
 * Any pull request which does _not_ relate to an accepted issue will be closed.
 
 * All major new functionality must include relevant tests where applicable.
@@ -132,18 +136,17 @@ accumulating a large backlog of work.
 The core maintainers group has chosen to make use of GitHub's [Stale bot](https://github.com/apps/stale)
 to aid in issue management.
 
-* Issues will be marked as stale after 14 days of no activity.
-* Then after 7 more days of inactivity, the issue will be closed.
+* Issues will be marked as stale after 45 days of no activity.
+* Then after 15 more days of inactivity, the issue will be closed.
 * Any issue bearing one of the following labels will be exempt from all Stale
   bot actions:
   * `status: accepted`
-  * `status: gathering feedback`
   * `status: blocked`
+  * `status: needs milestone`
 
-It is natural that some new issues get more attention than others. Often this
-is a metric of an issues's overall value to the project. In other cases in
-which issues merely get lost in the shuffle, notifications from Stale bot can
-bring renewed attention to potentially meaningful issues.
+It is natural that some new issues get more attention than others. Stale bot
+helps bring renewed attention to potentially valuable issues that may have been
+overlooked.
 
 ## Maintainer Guidance
 

docs/api/overview.md

@@ -279,6 +279,10 @@ http://localhost:8000/api/ipam/prefixes/ | jq ".actions.POST.status.choices"
 
 For most fields, when a filter is passed multiple times, objects matching _any_ of the provided values will be returned. For example, `GET /api/dcim/sites/?name=Foo&name=Bar` will return all sites named "Foo" _or_ "Bar". The exception to this rule is ManyToManyFields which may have multiple values assigned. Tags are the most common example of a ManyToManyField. For example, `GET /api/dcim/sites/?tag=foo&tag=bar` will return only sites tagged with both "foo" _and_ "bar".
 
+### Excluding Config Contexts
+
+The rendered config context for devices and VMs is included by default in all API results (list and detail views). Users with large amounts of context data will most likely observe a performance drop when returning multiple objects, particularly with page sizes in the high hundreds or more. To combat this, in cases where the rendered config context is not needed, the query parameter `?exclude=config_context` may be appended to the request URL to exclude the config context data from the API response.
+
 ### Custom Fields
 
 To filter on a custom field, prepend `cf_` to the field name. For example, the following query will return only sites where a custom field named `foo` is equal to 123:

docs/development/release-checklist.md

@@ -41,7 +41,14 @@ Create a file at `/docs/release-notes/X.Y.md` to establish the release notes for
 
 ### Manually Perform a New Install
 
-Create a new installation of NetBox by following [the current documentation](http://netbox.readthedocs.io/en/latest/). This should be a manual process, so that issues with the documentation can be identified and corrected.
+Install `mkdocs` in your local environment, then start the documentation server:
+
+```no-highlight
+$ pip install -r docs/requirements.txt
+$ mkdocs serve
+```
+
+Follow these instructions to perform a new installation of NetBox. This process must _not_ be automated: The goal of this step is to catch any errors or omissions in the documentation, and ensure that it is kept up-to-date for each release. Make any necessary changes to the documentation before proceeding with the release.
 
 ### Close the Release Milestone
 

docs/installation/3-netbox.md

@@ -74,12 +74,18 @@ Checking connectivity... done.
 
 Create a system user account named `netbox`. We'll configure the WSGI and HTTP services to run under this account. We'll also assign this user ownership of the media directory. This ensures that NetBox will be able to save local files.
 
-!!! note
-    CentOS users may need to create the `netbox` group first.
+#### Ubuntu
+
+```
+# adduser --system --group netbox
+# chown --recursive netbox /opt/netbox/netbox/media/
+```
+
+#### CentOS
 
 ```
 # groupadd --system netbox
-# adduser --system --gid netbox netbox
+# adduser --system -g netbox netbox
 # chown --recursive netbox /opt/netbox/netbox/media/
 ```
 

docs/installation/upgrading.md

@@ -30,6 +30,12 @@ Copy the 'configuration.py' you created when first installing to the new version
 # cp netbox-X.Y.Z/netbox/netbox/configuration.py netbox/netbox/netbox/configuration.py
 ```
 
+Copy your local requirements file if used:
+
+```no-highlight
+# cp netbox-X.Y.Z/local_requirements.txt netbox/local_requirements.txt
+```
+
 Also copy the LDAP configuration if using LDAP:
 
 ```no-highlight

docs/models/dcim/powerfeed.md

@@ -2,7 +2,7 @@
 
 A power feed identifies the power outlet/drop that goes to a rack and is terminated to a power panel. Power feeds have a supply type (AC/DC), voltage, amperage, and phase type (single/three).
 
-Power feeds are optionally assigned to a rack. In addition, a power port – and only one – can connect to a power feed; in the context of a PDU, the power feed is analogous to the power outlet that a PDU's power port/inlet connects to.
+Power feeds are optionally assigned to a rack. In addition, a power port may be connected to a power feed. In the context of a PDU, the power feed is analogous to the power outlet that a PDU's power port/inlet connects to.
 
 !!! info
     The power usage of a rack is calculated when a power feed (or multiple) is assigned to that rack and connected to a power port.

docs/models/extras/imageattachment.md

@@ -0,0 +1,3 @@
+# Image Attachments
+
+Certain objects in NetBox support the attachment of uploaded images. These will be saved to the NetBox server and made available whenever the object is viewed.

docs/plugins/development.md

@@ -110,6 +110,8 @@ NetBox looks for the `config` variable within a plugin's `__init__.py` to load i
 | `template_extensions` | The dotted path to the list of template extension classes (default: `template_content.template_extensions`) |
 | `menu_items` | The dotted path to the list of menu items provided by the plugin (default: `navigation.menu_items`) |
 
+All required settings must be configured by the user. If a configuration parameter is listed in both `required_settings` and `default_settings`, the default setting will be ignored.
+
 ### Install the Plugin for Development
 
 To ease development, it is recommended to go ahead and install the plugin at this point using setuptools' `develop` mode. This will create symbolic links within your Python environment to the plugin development directory. Call `setup.py` from the plugin's root directory with the `develop` argument (instead of `install`):

docs/release-notes/version-2.8.md

@@ -1,5 +1,28 @@
 # NetBox v2.8
 
+## v2.8.9 (2020-08-04)
+
+### Enhancements
+
+* [#4898](https://github.com/netbox-community/netbox/issues/4898) - Add MAC address search field to interfaces list
+* [#4899](https://github.com/netbox-community/netbox/issues/4899) - Add MAC address column to interfaces table
+
+### Bug Fixes
+
+* [#4455](https://github.com/netbox-community/netbox/issues/4455) - Fix ordering of prefixes beneath aggregate when available space is hidden
+* [#4875](https://github.com/netbox-community/netbox/issues/4875) - Fix documentation for image attachments
+* [#4876](https://github.com/netbox-community/netbox/issues/4876) - Fix labels for sites in staging or decommissioning status
+* [#4880](https://github.com/netbox-community/netbox/issues/4880) - Fix removal of tagged VLANs if not assigned in bulk interface editing
+* [#4887](https://github.com/netbox-community/netbox/issues/4887) - Don't disable NAPALM tabs when device has no primary IP
+* [#4894](https://github.com/netbox-community/netbox/issues/4894) - Fix display of device/VM counts on platforms list
+* [#4895](https://github.com/netbox-community/netbox/issues/4895) - Force UTF-8 encoding when embedding model documentation
+* [#4910](https://github.com/netbox-community/netbox/issues/4910) - Unpin redis dependency to fix exception in RQ worker
+* [#4926](https://github.com/netbox-community/netbox/issues/4926) - Fix ordering of VM interfaces in REST API endpoint
+* [#4927](https://github.com/netbox-community/netbox/issues/4927) - Fix validation error when updating an existing secret
+* [#4929](https://github.com/netbox-community/netbox/issues/4929) - Correct log message when creating a new object
+
+---
+
 ## v2.8.8 (2020-07-21)
 
 ### Enhancements

netbox/dcim/forms.py

@@ -2671,6 +2671,10 @@ class InterfaceFilterForm(DeviceComponentFilterForm):
             choices=BOOLEAN_WITH_BLANK_CHOICES
         )
     )
+    mac_address = forms.CharField(
+        required=False,
+        label='MAC address'
+    )
     tag = TagFilterField(model)
 
 

netbox/dcim/models/__init__.py

@@ -254,8 +254,10 @@ class Site(ChangeLoggedModel, CustomFieldModel):
     ]
 
     STATUS_CLASS_MAP = {
-        SiteStatusChoices.STATUS_ACTIVE: 'success',
         SiteStatusChoices.STATUS_PLANNED: 'info',
+        SiteStatusChoices.STATUS_STAGING: 'primary',
+        SiteStatusChoices.STATUS_ACTIVE: 'success',
+        SiteStatusChoices.STATUS_DECOMMISSIONING: 'warning',
         SiteStatusChoices.STATUS_RETIRED: 'danger',
     }
 

netbox/dcim/tables.py

@@ -94,6 +94,14 @@ MANUFACTURER_ACTIONS = """
 {% endif %}
 """
 
+DEVICEROLE_DEVICE_COUNT = """
+<a href="{% url 'dcim:device_list' %}?role={{ record.slug }}">{{ value|default:0 }}</a>
+"""
+
+DEVICEROLE_VM_COUNT = """
+<a href="{% url 'virtualization:virtualmachine_list' %}?role={{ record.slug }}">{{ value|default:0 }}</a>
+"""
+
 DEVICEROLE_ACTIONS = """
 <a href="{% url 'dcim:devicerole_changelog' slug=record.slug %}" class="btn btn-default btn-xs" title="Change log">
     <i class="fa fa-history"></i>
@@ -103,12 +111,12 @@ DEVICEROLE_ACTIONS = """
 {% endif %}
 """
 
-DEVICE_COUNT = """
-<a href="{% url 'dcim:device_list' %}?role={{ record.slug }}">{{ value|default:0 }}</a>
+PLATFORM_DEVICE_COUNT = """
+<a href="{% url 'dcim:device_list' %}?platform={{ record.slug }}">{{ value|default:0 }}</a>
 """
 
-VM_COUNT = """
-<a href="{% url 'virtualization:virtualmachine_list' %}?role={{ record.slug }}">{{ value|default:0 }}</a>
+PLATFORM_VM_COUNT = """
+<a href="{% url 'virtualization:virtualmachine_list' %}?platform={{ record.slug }}">{{ value|default:0 }}</a>
 """
 
 PLATFORM_ACTIONS = """
@@ -697,11 +705,11 @@ class DeviceBayTemplateTable(BaseTable):
 class DeviceRoleTable(BaseTable):
     pk = ToggleColumn()
     device_count = tables.TemplateColumn(
-        template_code=DEVICE_COUNT,
+        template_code=DEVICEROLE_DEVICE_COUNT,
         verbose_name='Devices'
     )
     vm_count = tables.TemplateColumn(
-        template_code=VM_COUNT,
+        template_code=DEVICEROLE_VM_COUNT,
         verbose_name='VMs'
     )
     color = tables.TemplateColumn(
@@ -728,11 +736,11 @@ class DeviceRoleTable(BaseTable):
 class PlatformTable(BaseTable):
     pk = ToggleColumn()
     device_count = tables.TemplateColumn(
-        template_code=DEVICE_COUNT,
+        template_code=PLATFORM_DEVICE_COUNT,
         verbose_name='Devices'
     )
     vm_count = tables.TemplateColumn(
-        template_code=VM_COUNT,
+        template_code=PLATFORM_VM_COUNT,
         verbose_name='VMs'
     )
     actions = tables.TemplateColumn(
@@ -950,8 +958,8 @@ class InterfaceDetailTable(DeviceComponentDetailTable):
 
     class Meta(InterfaceTable.Meta):
         order_by = ('parent', 'name')
-        fields = ('pk', 'parent', 'name', 'enabled', 'type', 'description', 'cable')
-        sequence = ('pk', 'parent', 'name', 'enabled', 'type', 'description', 'cable')
+        fields = ('pk', 'parent', 'name', 'enabled', 'type', 'mac_address', 'description', 'cable')
+        default_columns = ('pk', 'parent', 'name', 'enabled', 'type', 'description', 'cable')
 
 
 class FrontPortTable(BaseTable):

netbox/dcim/views.py

@@ -1060,8 +1060,8 @@ class DeviceRoleBulkDeleteView(PermissionRequiredMixin, BulkDeleteView):
 class PlatformListView(PermissionRequiredMixin, ObjectListView):
     permission_required = 'dcim.view_platform'
     queryset = Platform.objects.annotate(
-        device_count=get_subquery(Device, 'device_role'),
-        vm_count=get_subquery(VirtualMachine, 'role')
+        device_count=get_subquery(Device, 'platform'),
+        vm_count=get_subquery(VirtualMachine, 'platform')
     )
     table = tables.PlatformTable
 

netbox/ipam/views.py

@@ -327,6 +327,8 @@ class AggregateView(PermissionRequiredMixin, View):
             prefix__net_contained_or_equal=str(aggregate.prefix)
         ).prefetch_related(
             'site', 'role'
+        ).order_by(
+            'prefix'
         ).annotate_depth(
             limit=0
         )

netbox/netbox/settings.py

@@ -16,7 +16,7 @@ from django.core.validators import URLValidator
 # Environment setup
 #
 
-VERSION = '2.8.8'
+VERSION = '2.8.9'
 
 # Hostname
 HOSTNAME = platform.node()

netbox/secrets/forms.py

@@ -120,7 +120,7 @@ class SecretForm(BootstrapMixin, CustomFieldModelForm):
             device=self.cleaned_data['device'],
             role=self.cleaned_data['role'],
             name=self.cleaned_data['name']
-        ).exists():
+        ).exclude(pk=self.instance.pk).exists():
             raise forms.ValidationError(
                 "Each secret assigned to a device must have a unique combination of role and name"
             )

netbox/templates/base.html

@@ -80,8 +80,8 @@
             </div>
         </div>
     </footer>
-<script src="{% static 'jquery/jquery-3.4.1.min.js' %}"
-        onerror="window.location='{% url 'media_failure' %}?filename=jquery/jquery-3.4.1.min.js'"></script>
+<script src="{% static 'jquery/jquery-3.5.1.min.js' %}"
+        onerror="window.location='{% url 'media_failure' %}?filename=jquery/jquery-3.5.1.min.js'"></script>
 <script src="{% static 'jquery-ui-1.12.1/jquery-ui.min.js' %}"
         onerror="window.location='{% url 'media_failure' %}?filename=jquery-ui-1.12.1/jquery-ui.min.js'"></script>
 <script src="{% static 'bootstrap-3.4.1-dist/js/bootstrap.min.js' %}"

netbox/templates/dcim/device.html

@@ -108,8 +108,6 @@
                 {% include 'dcim/inc/device_napalm_tabs.html' with disabled_message='No platform assigned to this device' %}
             {% elif not device.platform.napalm_driver %}
                 {% include 'dcim/inc/device_napalm_tabs.html' with disabled_message='No NAPALM driver assigned for this platform' %}
-            {% elif not device.primary_ip %}
-                {% include 'dcim/inc/device_napalm_tabs.html' with disabled_message='No primary IP address assigned to this device' %}
             {% else %}
                 {% include 'dcim/inc/device_napalm_tabs.html' %}
             {% endif %}

netbox/utilities/templatetags/buttons.py

@@ -27,12 +27,12 @@ def _get_viewname(instance, action):
 
 @register.inclusion_tag('buttons/clone.html')
 def clone_button(instance):
-    viewname = _get_viewname(instance, 'add')
+    url = reverse(_get_viewname(instance, 'add'))
 
     # Populate cloned field values
     param_string = prepare_cloned_fields(instance)
     if param_string:
-        url = '{}?{}'.format(reverse(viewname), param_string)
+        url = f'{url}?{param_string}'
 
     return {
         'url': url,

netbox/utilities/templatetags/helpers.py

@@ -170,7 +170,7 @@ def get_docs(model):
         model._meta.model_name
     )
     try:
-        with open(path) as docfile:
+        with open(path, encoding='utf-8') as docfile:
             content = docfile.read()
     except FileNotFoundError:
         return "Unable to load documentation, file not found: {}".format(path)

netbox/utilities/views.py

@@ -267,9 +267,10 @@ class ObjectEditView(GetReturnURLMixin, View):
         if form.is_valid():
             logger.debug("Form validation was successful")
 
+            object_created = form.instance.pk is None
             obj = form.save()
             msg = '{} {}'.format(
-                'Created' if not form.instance.pk else 'Modified',
+                'Created' if object_created else 'Modified',
                 self.model._meta.verbose_name
             )
             logger.info(f"{msg} {obj} (PK: {obj.pk})")
@@ -721,8 +722,8 @@ class BulkEditView(GetReturnURLMixin, View):
 
                                 # ManyToManyFields
                                 elif isinstance(model_field, ManyToManyField):
-                                    getattr(obj, name).set(form.cleaned_data[name])
-
+                                    if form.cleaned_data[name].count() > 0:
+                                        getattr(obj, name).set(form.cleaned_data[name])
                                 # Normal fields
                                 elif form.cleaned_data[name] not in (None, ''):
                                     setattr(obj, name, form.cleaned_data[name])

netbox/virtualization/api/views.py

@@ -3,6 +3,7 @@ from django.db.models import Count
 from dcim.models import Device, Interface
 from extras.api.views import CustomFieldModelViewSet
 from utilities.api import ModelViewSet
+from utilities.query_functions import CollateAsChar
 from utilities.utils import get_subquery
 from virtualization import filters
 from virtualization.models import Cluster, ClusterGroup, ClusterType, VirtualMachine
@@ -74,7 +75,7 @@ class VirtualMachineViewSet(CustomFieldModelViewSet):
 class InterfaceViewSet(ModelViewSet):
     queryset = Interface.objects.filter(
         virtual_machine__isnull=False
-    ).prefetch_related(
+    ).order_by('virtual_machine', CollateAsChar('_name')).prefetch_related(
         'virtual_machine', 'tags'
     )
     serializer_class = serializers.InterfaceSerializer

requirements.txt

@@ -21,5 +21,4 @@ Pillow==7.1.1
 psycopg2-binary==2.8.5
 pycryptodome==3.9.7
 PyYAML==5.3.1
-redis==3.4.1
 svgwrite==1.4

upgrade.sh

@@ -40,11 +40,12 @@ echo "Installing core dependencies ($COMMAND)..."
 eval $COMMAND || exit 1
 
 # Install optional packages (if any)
-if [ -f "local_requirements.txt" ]
-then
+if [ -s "local_requirements.txt" ]; then
   COMMAND="pip3 install -r local_requirements.txt"
   echo "Installing local dependencies ($COMMAND)..."
   eval $COMMAND || exit 1
+elif [ -f "local_requirements.txt" ]; then
+  echo "Skipping local dependencies (local_requirements.txt is empty)"
 else
   echo "Skipping local dependencies (local_requirements.txt not found)"
 fi