starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-13 13:53:31 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

v3.5 reports & scripts cleanup #7805

New Issue
Closed
opened 2025-12-29 20:28:24 +01:00 by adam · 12 comments
adam commented 2025-12-29 20:28:24 +01:00
Owner
Copy Link

Originally created by @jeremystretch on GitHub (Mar 28, 2023).

Originally assigned to: @jeremystretch on GitHub.

Proposed Changes

Starting an issue to capture miscellaneous cleanup items relating to recent work around custom scripts and reports in preparation for the v3.5 release.

Justification

This issue will serve as a standing collector for miscellanea that don't quite warrant their own individual issue. Substantial work items should still have individual issues opened.

(Comments hidden below have been resolved in the 12081-cleanup branch.)

Originally created by @jeremystretch on GitHub (Mar 28, 2023). Originally assigned to: @jeremystretch on GitHub. ### Proposed Changes Starting an issue to capture miscellaneous cleanup items relating to recent work around custom scripts and reports in preparation for the v3.5 release. ### Justification This issue will serve as a standing collector for miscellanea that don't quite warrant their own individual issue. Substantial work items should still have individual issues opened. (Comments hidden below have been resolved in the `12081-cleanup` branch.)
adam added the status: acceptedtype: housekeepingbeta labels 2025-12-29 20:28:25 +01:00
adam closed this issue 2025-12-29 20:28:25 +01:00
adam commented 2025-12-29 20:28:25 +01:00
Author
Owner
Copy Link

@arthanson commented on GitHub (Mar 28, 2023):

When there are no reports the message should be changed as reports are no longer placed in the directory but need to be uploaded. There should also probably be a message here for if you upgrade - Note: If you are not seeing your reports here and have upgraded NetBox...

Monosnap Reports | NetBox 2023-03-28 13-04-14

Same thing for scripts.

@arthanson commented on GitHub (Mar 28, 2023): When there are no reports the message should be changed as reports are no longer placed in the directory but need to be uploaded. There should also probably be a message here for if you upgrade - Note: If you are not seeing your reports here and have upgraded NetBox... ![Monosnap Reports | NetBox 2023-03-28 13-04-14](https://user-images.githubusercontent.com/99642/228355607-d6d4fafb-a6a2-4bd4-bb62-72ee18ef489c.png) Same thing for scripts.
adam commented 2025-12-29 20:28:26 +01:00
Author
Owner
Copy Link

@arthanson commented on GitHub (Mar 28, 2023):

Somewhat of a side issue, but the docs for running RQ should probably be improved, I'd suggest adding a section to "Installation and Upgrade" for running "python netbox/manage.py rqworker", there is only one part in the documents I could find under "Plugins -> Developing Plugins -> Background Tasks" a note on "Configuring the RQ worker process" where it shows using the management command. The RQ docs just show "rq worker --with-scheduler" which is probably not what you want.

@arthanson commented on GitHub (Mar 28, 2023): Somewhat of a side issue, but the docs for running RQ should probably be improved, I'd suggest adding a section to "Installation and Upgrade" for running "python netbox/manage.py rqworker", there is only one part in the documents I could find under "Plugins -> Developing Plugins -> Background Tasks" a note on "Configuring the RQ worker process" where it shows using the management command. The RQ docs just show "rq worker --with-scheduler" which is probably not what you want.
adam commented 2025-12-29 20:28:26 +01:00
Author
Owner
Copy Link

@arthanson commented on GitHub (Mar 28, 2023):

I'm using the default report from: https://docs.netbox.dev/en/stable/customization/reports/ it is in the directory for 3.4 and uploaded for 3.5 with demo data the database should be pretty much the same but I am getting no results for the job run under 3.5:
report1
report2

@arthanson commented on GitHub (Mar 28, 2023): I'm using the default report from: https://docs.netbox.dev/en/stable/customization/reports/ it is in the directory for 3.4 and uploaded for 3.5 with demo data the database should be pretty much the same but I am getting no results for the job run under 3.5: ![report1](https://user-images.githubusercontent.com/99642/228358514-276fb1fd-ba1e-43b8-a1b3-bf53e772002e.png) ![report2](https://user-images.githubusercontent.com/99642/228358518-9bce1b2a-6f96-424e-92fd-e1a014cae351.png)
adam commented 2025-12-29 20:28:26 +01:00
Author
Owner
Copy Link

@arthanson commented on GitHub (Mar 28, 2023):

As reports are no longer in a directory to look at, it would be really useful to be able to look at the script itself once uploaded, I.E. on the report details page show the content of the file - otherwise you might have no idea what the script is if someone else uploaded it, or if you did a couple versions no idea which one it is:
report-script

Script run also shows no output using script from https://docs.netbox.dev/en/stable/customization/custom-scripts/ - on NetBox 3.4 it does show output.

@arthanson commented on GitHub (Mar 28, 2023): As reports are no longer in a directory to look at, it would be really useful to be able to look at the script itself once uploaded, I.E. on the report details page show the content of the file - otherwise you might have no idea what the script is if someone else uploaded it, or if you did a couple versions no idea which one it is: ![report-script](https://user-images.githubusercontent.com/99642/228359356-d0cf0e74-48aa-4a62-887b-ba91e7da7c1f.png) Script run also shows no output using script from https://docs.netbox.dev/en/stable/customization/custom-scripts/ - on NetBox 3.4 it does show output.
adam commented 2025-12-29 20:28:26 +01:00
Author
Owner
Copy Link

@arthanson commented on GitHub (Mar 28, 2023):

Reports detail should really have a "reports" or "script" tab like scripts to show the contents of the uploaded report script.

@arthanson commented on GitHub (Mar 28, 2023): Reports detail should really have a "reports" or "script" tab like scripts to show the contents of the uploaded report script.
adam commented 2025-12-29 20:28:27 +01:00
Author
Owner
Copy Link

@arthanson commented on GitHub (Mar 29, 2023):

I changed the device script and re-uploaded it and got the following error - was from having it the same filename but it should probably display a error modal instead of crash.:

Traceback (most recent call last):
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 56, in inner
    response = get_response(request)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/views/generic/base.py", line 103, in view
    return self.dispatch(request, *args, **kwargs)
  File "/Users/ahanson/dev/work/netbox/netbox/netbox/views/generic/object_views.py", line 167, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/Users/ahanson/dev/work/netbox/netbox/netbox/views/generic/base.py", line 26, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/Users/ahanson/dev/work/netbox/netbox/utilities/views.py", line 99, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/views/generic/base.py", line 142, in dispatch
    return handler(request, *args, **kwargs)
  File "/Users/ahanson/dev/work/netbox/netbox/netbox/views/generic/object_views.py", line 261, in post
    obj = form.save()
  File "/Users/ahanson/dev/work/netbox/netbox/core/forms/model_forms.py", line 111, in save
    return super().save(*args, **kwargs)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/forms/models.py", line 548, in save
    self.instance.save()
  File "/Users/ahanson/dev/work/netbox/netbox/extras/models/reports.py", line 68, in save
    return super().save(*args, **kwargs)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 812, in save
    self.save_base(
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 863, in save_base
    updated = self._save_table(
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 1006, in _save_table
    results = self._do_insert(
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 1047, in _do_insert
    return manager._insert(
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/query.py", line 1791, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1660, in execute_sql
    cursor.execute(sql, params)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/debug_toolbar/panels/sql/tracking.py", line 230, in execute
    return self._record(self.cursor.execute, sql, params)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/debug_toolbar/panels/sql/tracking.py", line 154, in _record
    return method(sql, params)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 102, in execute
    return super().execute(sql, params)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: duplicate key value violates unique constraint "core_managedfile_unique_root_path"
DETAIL:  Key (file_root, file_path)=(reports, devices.py) already exists.
@arthanson commented on GitHub (Mar 29, 2023): I changed the device script and re-uploaded it and got the following error - was from having it the same filename but it should probably display a error modal instead of crash.: ``` Traceback (most recent call last): File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 56, in inner response = get_response(request) File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/views/generic/base.py", line 103, in view return self.dispatch(request, *args, **kwargs) File "/Users/ahanson/dev/work/netbox/netbox/netbox/views/generic/object_views.py", line 167, in dispatch return super().dispatch(request, *args, **kwargs) File "/Users/ahanson/dev/work/netbox/netbox/netbox/views/generic/base.py", line 26, in dispatch return super().dispatch(request, *args, **kwargs) File "/Users/ahanson/dev/work/netbox/netbox/utilities/views.py", line 99, in dispatch return super().dispatch(request, *args, **kwargs) File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/views/generic/base.py", line 142, in dispatch return handler(request, *args, **kwargs) File "/Users/ahanson/dev/work/netbox/netbox/netbox/views/generic/object_views.py", line 261, in post obj = form.save() File "/Users/ahanson/dev/work/netbox/netbox/core/forms/model_forms.py", line 111, in save return super().save(*args, **kwargs) File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/forms/models.py", line 548, in save self.instance.save() File "/Users/ahanson/dev/work/netbox/netbox/extras/models/reports.py", line 68, in save return super().save(*args, **kwargs) File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 812, in save self.save_base( File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 863, in save_base updated = self._save_table( File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 1006, in _save_table results = self._do_insert( File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 1047, in _do_insert return manager._insert( File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/query.py", line 1791, in _insert return query.get_compiler(using=using).execute_sql(returning_fields) File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1660, in execute_sql cursor.execute(sql, params) File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/debug_toolbar/panels/sql/tracking.py", line 230, in execute return self._record(self.cursor.execute, sql, params) File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/debug_toolbar/panels/sql/tracking.py", line 154, in _record return method(sql, params) File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 102, in execute return super().execute(sql, params) File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 67, in execute return self._execute_with_wrappers( File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers return executor(sql, params, many, context) File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute with self.db.wrap_database_errors: File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute return self.cursor.execute(sql, params) django.db.utils.IntegrityError: duplicate key value violates unique constraint "core_managedfile_unique_root_path" DETAIL: Key (file_root, file_path)=(reports, devices.py) already exists. ```
adam commented 2025-12-29 20:28:27 +01:00
Author
Owner
Copy Link

@sleepinggenius2 commented on GitHub (Mar 29, 2023):

If reports/scripts are no longer going to be in a directory, how do we handle supporting files moving forward, Python or non-Python? For example, today we use a YAML configuration file that gets read by different reports/scripts, so that we can use the same reports/scripts in different environments (dev, staging, prod, etc.), but abstract out any variables that are specific to the particular environment. We also have some Python library files with shared code that is used by multiple reports/scripts. All of these files live within REPORTS_ROOT and SCRIPTS_ROOT today.

@sleepinggenius2 commented on GitHub (Mar 29, 2023): If reports/scripts are no longer going to be in a directory, how do we handle supporting files moving forward, Python or non-Python? For example, today we use a YAML configuration file that gets read by different reports/scripts, so that we can use the same reports/scripts in different environments (dev, staging, prod, etc.), but abstract out any variables that are specific to the particular environment. We also have some Python library files with shared code that is used by multiple reports/scripts. All of these files live within REPORTS_ROOT and SCRIPTS_ROOT today.
adam commented 2025-12-29 20:28:27 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Mar 29, 2023):

When creating a report/script from a DataFile, the path of the origin file is not replicated. For example, a DataFile with a path of foo/bar/myreport.py will create a file myreport.py in the reports root.

@jeremystretch commented on GitHub (Mar 29, 2023): When creating a report/script from a DataFile, the path of the origin file is not replicated. For example, a DataFile with a path of `foo/bar/myreport.py` will create a file `myreport.py` in the reports root.
adam commented 2025-12-29 20:28:27 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Mar 29, 2023):

@sleepinggenius2 this issue is only for recording observed bugs & limitations in ongoing work. Please hold such discussion until the beta release has been made available for testing.

@jeremystretch commented on GitHub (Mar 29, 2023): @sleepinggenius2 this issue is only for recording observed bugs & limitations in ongoing work. Please hold such discussion until the beta release has been made available for testing.
adam commented 2025-12-29 20:28:27 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Mar 29, 2023):

Attempting to render the reports list fails if a single report triggers an unhandled exception (such as an ImportError).

@jeremystretch commented on GitHub (Mar 29, 2023): Attempting to render the reports list fails if a single report triggers an unhandled exception (such as an `ImportError`).
adam commented 2025-12-29 20:28:27 +01:00
Author
Owner
Copy Link

@sleepinggenius2 commented on GitHub (Mar 29, 2023):

@sleepinggenius2 this issue is only for recording observed bugs & limitations in ongoing work. Please hold such discussion until the beta release has been made available for testing.

I apologize. I read through #12059 and it wasn't clear where this was being accounted for, so I thought it was a relevant question to ask in this context with respect to a potential limitation. I will wait for the beta release for further comments.

@sleepinggenius2 commented on GitHub (Mar 29, 2023): > @sleepinggenius2 this issue is only for recording observed bugs & limitations in ongoing work. Please hold such discussion until the beta release has been made available for testing. I apologize. I read through #12059 and it wasn't clear where this was being accounted for, so I thought it was a relevant question to ask in this context with respect to a potential limitation. I will wait for the beta release for further comments.
adam commented 2025-12-29 20:28:28 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Mar 29, 2023):

@kkthxbye-code has provided an example repo with a few scripts to help test upgrading & loading. There are currently two issues:

  1. Relative imports from within a module don't work. We likely just need to tweak the module loading logic.
  2. Empty modules get replicated as database objects. This isn't necessarily a problem, however it can be confusing. We'll need to figure out the ideal strategy for managing these files.
@jeremystretch commented on GitHub (Mar 29, 2023): @kkthxbye-code has provided an [example repo](https://github.com/kkthxbye-code/netboxscriptexample) with a few scripts to help test upgrading & loading. There are currently two issues: 1. Relative imports from within a module don't work. We likely just need to tweak the module loading logic. 2. Empty modules get replicated as database objects. This isn't necessarily a problem, however it can be confusing. We'll need to figure out the ideal strategy for managing these files.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label beta status: accepted type: housekeeping
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#7805
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?