starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Datasource git on local file system fails #9917

New Issue
Closed
opened 2025-12-29 21:24:21 +01:00 by adam · 5 comments
adam commented 2025-12-29 21:24:21 +01:00
Owner
Copy Link

Originally created by @thefriendlynet on GitHub (Jun 28, 2024).

Originally assigned to: @thefriendlynet on GitHub.

Deployment Type

Self-hosted

NetBox Version

v4.0.2

Python Version

3.11

Steps to Reproduce

  1. create Datasource type git, URL e.g. ///home/datasources
  2. try sync

Expected Behavior

files synced from local file system

Observed Behavior

SyncError("Fetching remote data failed (TypeError): LocalGitClient.__init__() got an unexpected keyword argument 'quiet'")

netbox/netbox/core/data_backends.py

        clone_args = {
            "branch": self.params.get('branch'),
            "config": self.config,
            "depth": 1,
            "errstream": porcelain.NoneStream(),
            "quiet": True,
        }

"quiet" and "depth" aren't supported by dulwich for local filesystem git repo's

Originally created by @thefriendlynet on GitHub (Jun 28, 2024). Originally assigned to: @thefriendlynet on GitHub. ### Deployment Type Self-hosted ### NetBox Version v4.0.2 ### Python Version 3.11 ### Steps to Reproduce 1. create Datasource type git, URL e.g. ///home/datasources 2. try sync ### Expected Behavior files synced from local file system ### Observed Behavior `SyncError("Fetching remote data failed (TypeError): LocalGitClient.__init__() got an unexpected keyword argument 'quiet'")` **netbox/netbox/core/data_backends.py** ```python clone_args = { "branch": self.params.get('branch'), "config": self.config, "depth": 1, "errstream": porcelain.NoneStream(), "quiet": True, } ``` "quiet" and "depth" aren't supported by dulwich for local filesystem git repo's
adam added the type: bugstatus: acceptedseverity: low labels 2025-12-29 21:24:21 +01:00
adam closed this issue 2025-12-29 21:24:21 +01:00
adam commented 2025-12-29 21:24:22 +01:00
Author
Owner
Copy Link

@thefriendlynet commented on GitHub (Jun 28, 2024):

This might be an issue in the dulwich LocalGitClient code, don't seem to be able to fix it from Netbox.

EDIT: see below for why it seemed my changes to the Netbox code didn't work.
(removing the dulwich code changes to avoid any further confusion)

@thefriendlynet commented on GitHub (Jun 28, 2024): This might be an issue in the dulwich LocalGitClient code, don't seem to be able to fix it from Netbox. EDIT: see below for why it seemed my changes to the Netbox code didn't work. (removing the dulwich code changes to avoid any further confusion)
adam commented 2025-12-29 21:24:22 +01:00
Author
Owner
Copy Link

@jeffgdotorg commented on GitHub (Jul 1, 2024):

Thanks for reporting a problem you've encountered in NetBox.

I've managed to reproduce the problem you describe in a fresh NetBox 4.0.6 installation. For the benefit of future workers of this issue, here's what I did to set up the bare local Git repo and otherwise prepare the system.

ubuntu@nb406-repro-16760:~$ sudo mkdir /home/datasources
ubuntu@nb406-repro-16760:~$ sudo chown netbox:netbox /home/datasources
ubuntu@nb406-repro-16760:~$ sudo -u netbox git init /home/datasources
hint: Using 'master' as the name for the initial branch. This default branch name
...
Initialized empty Git repository in /home/datasources/.git/
ubuntu@nb406-repro-16760:~$ sudo -u netbox /bin/bash
netbox@nb406-repro-16760:/home/ubuntu$ cd /home/datasources/
netbox@nb406-repro-16760:/home/datasources$ git status
On branch master

No commits yet

nothing to commit (create/copy files and use "git add" to track)
netbox@nb406-repro-16760:/home/datasources$ cat > README.txt
Hello world, this is an attempt to reproduce NetBox bug issue #16760.

Good luck!
netbox@nb406-repro-16760:/home/datasources$ git add .
netbox@nb406-repro-16760:/home/datasources$ git config --global user.email '***@netboxlabs.com'
netbox@nb406-repro-16760:/home/datasources$ git config --global user.name "Jeff Gehlbach"
netbox@nb406-repro-16760:/home/datasources$ git commit -m "Added a file"
[master (root-commit) f88787d] Added a file
 1 file changed, 3 insertions(+)
 create mode 100644 README.txt
ubuntu@nb406-repro-16760:~$ sudo -i
root@nb406-repro-16760:~# source /opt/netbox/venv/bin/activate
(venv) root@nb406-repro-16760:~# pip3 install dulwich
Collecting dulwich
  Downloading dulwich-0.22.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.4 kB)
Requirement already satisfied: urllib3>=1.25 in /opt/netbox/venv/lib/python3.10/site-packages (from dulwich) (2.2.2)
Downloading dulwich-0.22.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (979 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 979.1/979.1 kB 13.9 MB/s eta 0:00:00
Installing collected packages: dulwich
Successfully installed dulwich-0.22.1
(venv) root@nb406-repro-16760:~# systemctl restart netbox netbox-rq

I'm moving your issue along to needs owner status. If you would like to work it through to a PR, please say so and a maintainer will assign it to you.

@jeffgdotorg commented on GitHub (Jul 1, 2024): Thanks for reporting a problem you've encountered in NetBox. I've managed to reproduce the problem you describe in a fresh NetBox 4.0.6 installation. For the benefit of future workers of this issue, here's what I did to set up the bare local Git repo and otherwise prepare the system. ``` ubuntu@nb406-repro-16760:~$ sudo mkdir /home/datasources ubuntu@nb406-repro-16760:~$ sudo chown netbox:netbox /home/datasources ubuntu@nb406-repro-16760:~$ sudo -u netbox git init /home/datasources hint: Using 'master' as the name for the initial branch. This default branch name ... Initialized empty Git repository in /home/datasources/.git/ ubuntu@nb406-repro-16760:~$ sudo -u netbox /bin/bash netbox@nb406-repro-16760:/home/ubuntu$ cd /home/datasources/ netbox@nb406-repro-16760:/home/datasources$ git status On branch master No commits yet nothing to commit (create/copy files and use "git add" to track) netbox@nb406-repro-16760:/home/datasources$ cat > README.txt Hello world, this is an attempt to reproduce NetBox bug issue #16760. Good luck! netbox@nb406-repro-16760:/home/datasources$ git add . netbox@nb406-repro-16760:/home/datasources$ git config --global user.email '***@netboxlabs.com' netbox@nb406-repro-16760:/home/datasources$ git config --global user.name "Jeff Gehlbach" netbox@nb406-repro-16760:/home/datasources$ git commit -m "Added a file" [master (root-commit) f88787d] Added a file 1 file changed, 3 insertions(+) create mode 100644 README.txt ubuntu@nb406-repro-16760:~$ sudo -i root@nb406-repro-16760:~# source /opt/netbox/venv/bin/activate (venv) root@nb406-repro-16760:~# pip3 install dulwich Collecting dulwich Downloading dulwich-0.22.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.4 kB) Requirement already satisfied: urllib3>=1.25 in /opt/netbox/venv/lib/python3.10/site-packages (from dulwich) (2.2.2) Downloading dulwich-0.22.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (979 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 979.1/979.1 kB 13.9 MB/s eta 0:00:00 Installing collected packages: dulwich Successfully installed dulwich-0.22.1 (venv) root@nb406-repro-16760:~# systemctl restart netbox netbox-rq ``` I'm moving your issue along to `needs owner` status. If you would like to work it through to a PR, please say so and a maintainer will assign it to you.
adam commented 2025-12-29 21:24:23 +01:00
Author
Owner
Copy Link

@thefriendlynet commented on GitHub (Jul 2, 2024):

Thanks for taking the time to add these steps.

EDIT: I didn't restart the Netbox worker, only the Netbox process, that's why my changes to the Netbox code didn't work, duh

This fixes the issue, but is it acceptable if I rework it into a PR ?

--- a/core/data_backends.py
+++ b/core/data_backends.py
@@ -133,6 +133,10 @@
                         "password": self.params.get('password'),
                     }
                 )
+        elif not self.url_scheme:
+            clone_args["depth"] = None
+            del(clone_args["quiet"])
 
         logger.debug(f"Cloning git repo: {self.url}")
         try:
@thefriendlynet commented on GitHub (Jul 2, 2024): Thanks for taking the time to add these steps. EDIT: I didn't restart the Netbox worker, only the Netbox process, that's why my changes to the Netbox code didn't work, duh This fixes the issue, but is it acceptable if I rework it into a PR ? ```diff --- a/core/data_backends.py +++ b/core/data_backends.py @@ -133,6 +133,10 @@ "password": self.params.get('password'), } ) + elif not self.url_scheme: + clone_args["depth"] = None + del(clone_args["quiet"]) logger.debug(f"Cloning git repo: {self.url}") try: ```
adam commented 2025-12-29 21:24:23 +01:00
Author
Owner
Copy Link

@thefriendlynet commented on GitHub (Jul 10, 2024):

If you would like to work it through to a PR, please say so and a maintainer will assign it to you.

I'd like to submit a PR based on the diff I posted.

@thefriendlynet commented on GitHub (Jul 10, 2024): > If you would like to work it through to a PR, please say so and a maintainer will assign it to you. I'd like to submit a PR based on the diff I posted.
adam commented 2025-12-29 21:24:23 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Jul 10, 2024):

@thefriendlynet thanks, I've assigned this to you.

@jeremystretch commented on GitHub (Jul 10, 2024): @thefriendlynet thanks, I've assigned this to you.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label severity: low status: accepted type: bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#9917
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?