Offline Gradle build #44

New Issue

adam · 2025-12-30T01:19:53+01:00

adam commented

2025-12-30 01:19:53 +01:00

Originally created by @rafaelrc7 on GitHub (Feb 7, 2024).

Hello, I'm having trouble setting up an offline gradle build.

First I run gradle normally, using the command described in #66 to fetch all dependencies.

Then, I convert the downloaded dependencies into a local maven repository, that I'll call "deps".

And then, I edit the following files:

settings.gradle.kts
./buildSrc/src/main/kotlin/pklAllProjects.gradle.kts
./pkl-gradle/src/test/kotlin/org/pkl/gradle/JavaCodeGeneratorsTest.kt
./pkl-gradle/src/test/kotlin/org/pkl/gradle/KotlinCodeGeneratorsTest.kt
./buildSrc/settings.gradle.kts

Changing mavenCentral into maven { url = uri("/path/to/deps") }.

Then I try running gradle --offline --no-daemon build. It manages to run most of the configuration step, showing that it manages to find the dependencies when it looks into the local repo. However, it then fails when trying to resolve dependencies of the step :pkl-cli:runtimeClasspath with the following error:

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':pkl-cli:shadowJar'.
> Could not resolve all dependencies for configuration ':pkl-cli:runtimeClasspath'.
   > Did not resolve 'com.github.ajalt.clikt:clikt-jvm:3.5.1' which is part of the dependency lock state

By the looks of the error, it seems that, for some reason, this step is still trying to reach mavenCentral() instead of deps. How can I change the pkl-cli repositories into deps?

Originally created by @rafaelrc7 on GitHub (Feb 7, 2024). Hello, I'm having trouble setting up an offline gradle build. First I run gradle normally, using the command described in #66 to fetch all dependencies. Then, I convert the downloaded dependencies into a local maven repository, that I'll call "deps". And then, I edit the following files: - settings.gradle.kts - ./buildSrc/src/main/kotlin/pklAllProjects.gradle.kts - ./pkl-gradle/src/test/kotlin/org/pkl/gradle/JavaCodeGeneratorsTest.kt - ./pkl-gradle/src/test/kotlin/org/pkl/gradle/KotlinCodeGeneratorsTest.kt - ./buildSrc/settings.gradle.kts Changing `mavenCentral` into `maven { url = uri("/path/to/deps") }`. Then I try running `gradle --offline --no-daemon build`. It manages to run most of the configuration step, showing that it manages to find the dependencies when it looks into the local repo. However, it then fails when trying to resolve dependencies of the step `:pkl-cli:runtimeClasspath` with the following error: ``` FAILURE: Build failed with an exception. * What went wrong: Could not determine the dependencies of task ':pkl-cli:shadowJar'. > Could not resolve all dependencies for configuration ':pkl-cli:runtimeClasspath'. > Did not resolve 'com.github.ajalt.clikt:clikt-jvm:3.5.1' which is part of the dependency lock state ``` By the looks of the error, it seems that, for some reason, this step is still trying to reach `mavenCentral()` instead of deps. How can I change the `pkl-cli` repositories into `deps`?

adam closed this issue

2025-12-30 01:19:53 +01:00

adam commented

2025-12-30 01:19:54 +01:00

@bioball commented on GitHub (Feb 26, 2024):

The intricacies of Gradle aren't my area of expertise.

However, I tried building locally without editing any of the mavenCentral files, and it is passing on my machine.
Have you tried building this without making those source file changes?

@bioball commented on GitHub (Feb 26, 2024): The intricacies of Gradle aren't my area of expertise. However, I tried building locally without editing any of the `mavenCentral` files, and it is passing on my machine. Have you tried building this without making those source file changes?

adam commented

2025-12-30 01:19:54 +01:00

@StefMa commented on GitHub (Feb 26, 2024):

Some dependencies will be downloaded as part of a task. So running the dependencies gradle task will only download the dependencies that are "normally" declared in the build scripts. However it might be the case that some tasks download dependencies "on the fly". These are missing now, if you run the build task.

To download all required dependencies for the build task, run the build task with mavenCentral declared 😁 afterwards all of them "should be available"™ and you can run the build task offline as well...

I guess 😁 at least worth to try.

It could also happen that some plugins will add the mavenCentral repository by themself to download dependencies. In this case it might be not possible to build pkl without internet...

@StefMa commented on GitHub (Feb 26, 2024): Some dependencies will be downloaded as part of a task. So running the `dependencies` gradle task will only download the dependencies that are "normally" declared in the build scripts. However it might be the case that some tasks download dependencies "on the fly". These are missing now, if you run the `build` task. To download all required dependencies for the `build` task, run the `build` task with `mavenCentral` declared 😁 afterwards all of them "should be available"™ and you can run the `build` task offline as well... I guess 😁 at least worth to try. It could also happen that some plugins will add the `mavenCentral` repository by themself to download dependencies. In this case it might be not possible to build `pkl` without internet...

adam commented

2025-12-30 01:19:55 +01:00

@bioball commented on GitHub (Feb 26, 2024):

From someone more knowledgable than me about Gradle:

first of all, using dependencies is not a fully comprehensive way to determine all dependencies of a build. the dependencies output may skip some internal and invisible configurations which are nevertheless resolved during the build. For example, it is possible to create "detached" configurations to resolve and download dependencies in an ad-hoc way, and such configurations will not be visible in the dependencies output.

I think that the correct way to find all dependencies for a particular build execution is to run the build with completely empty caches (~/.gradle/caches) and then run the build in the offline mode, with the same caches. I've never done this myself so I don't know if there are other pitfalls here, but from what I understand, it should work as required

@bioball commented on GitHub (Feb 26, 2024): From someone more knowledgable than me about Gradle: > first of all, using dependencies is not a fully comprehensive way to determine all dependencies of a build. the dependencies output may skip some internal and invisible configurations which are nevertheless resolved during the build. For example, it is possible to create "detached" configurations to resolve and download dependencies in an ad-hoc way, and such configurations will not be visible in the dependencies output. > > I think that the correct way to find all dependencies for a particular build execution is to run the build with completely empty caches (~/.gradle/caches) and then run the build in the offline mode, with the same caches. I've never done this myself so I don't know if there are other pitfalls here, but from what I understand, it should work as required

adam commented

2025-12-30 01:19:55 +01:00

@rafaelrc7 commented on GitHub (Mar 8, 2024):

Thank you both for the input! However, I believe I did not explain fully what the issue is... To give a bit more of context, I am trying to package pkl for nixpkgs. In nixpkgs, reproducibility is a necessity, and so extra steps are taken to package software. The process with gradle generally involves first building a derivation (a nix package) that is a local maven repository with the package dependencies, that was done, partly thanks to @bioball by answering https://github.com/apple/pkl/issues/66 .

The dependencies derivation is then used as a local maven repo. To do so, the gradle files are patched to point to this repo (simply a path in the file system). However, it seems that even by editting any gradle file :pkl-cli:runtimeClasspath task still refuses to look up for the com.github.ajalt.clikt:clikt-jvm:3.5.1 dependency in the local repo. I checked and this particular dependency is downloaded and is available in the local repo... It just seems that the change to the list of repos is not affecting this task.

@rafaelrc7 commented on GitHub (Mar 8, 2024): Thank you both for the input! However, I believe I did not explain fully what the issue is... To give a bit more of context, I am trying to package pkl for nixpkgs. In nixpkgs, reproducibility is a necessity, and so extra steps are taken to package software. The process with gradle generally involves first building a derivation (a nix package) that is a local maven repository with the package dependencies, that was done, partly thanks to @bioball by answering https://github.com/apple/pkl/issues/66 . The dependencies derivation is then used as a local maven repo. To do so, the gradle files are patched to point to this repo (simply a path in the file system). However, it seems that even by editting any gradle file `:pkl-cli:runtimeClasspath` task still refuses to look up for the `com.github.ajalt.clikt:clikt-jvm:3.5.1` dependency in the local repo. I checked and this particular dependency is downloaded and is available in the local repo... It just seems that the change to the list of repos is not affecting this task.

adam commented

2025-12-30 01:19:55 +01:00

@bioball commented on GitHub (Mar 8, 2024):

Have you tried just running the dependencies command, and then re-building in offline mode? I've found that I can build in offline mode just fine. Or is that not good enough for nix?

@bioball commented on GitHub (Mar 8, 2024): Have you tried just running the dependencies command, and then re-building in offline mode? I've found that I can build in offline mode just fine. Or is that not good enough for nix?

adam commented

2025-12-30 01:19:55 +01:00

@rafaelrc7 commented on GitHub (Mar 9, 2024):

The issue is that during the nix derivation build process, the dependencies are downloaded in an isolated environment, then the dependencies, and only the dependencies, are extracted into a local maven repo. Finally, the package is built in other isolated environment, so the cache is a new one, like building pkl for the first time. In theory, all dependencies are available in the local repository, however the aforementioned task seems to ignore it...

I tried adding the local repo to:

settings.gradle.kts
./buildSrc/src/main/kotlin/pklAllProjects.gradle.kts
./pkl-gradle/src/test/kotlin/org/pkl/gradle/JavaCodeGeneratorsTest.kt
./pkl-gradle/src/test/kotlin/org/pkl/gradle/KotlinCodeGeneratorsTest.kt
./buildSrc/settings.gradle.kts

but it still does not apply to all tasks.

@rafaelrc7 commented on GitHub (Mar 9, 2024): The issue is that during the nix derivation build process, the dependencies are downloaded in an isolated environment, then the dependencies, and only the dependencies, are extracted into a local maven repo. Finally, the package is built in *other* isolated environment, so the cache is a new one, like building pkl for the first time. In theory, all dependencies are available in the local repository, however the aforementioned task seems to ignore it... I tried adding the local repo to: - settings.gradle.kts - ./buildSrc/src/main/kotlin/pklAllProjects.gradle.kts - ./pkl-gradle/src/test/kotlin/org/pkl/gradle/JavaCodeGeneratorsTest.kt - ./pkl-gradle/src/test/kotlin/org/pkl/gradle/KotlinCodeGeneratorsTest.kt - ./buildSrc/settings.gradle.kts but it still does not apply to all tasks.

adam commented

2025-12-30 01:19:56 +01:00

@rafaelrc7 commented on GitHub (Jul 20, 2024):

Ok, I finally understood the problem and why it seemed that offline builds were not working. As discussed in https://github.com/NixOS/nixpkgs/pull/286658#issuecomment-2240070303 some tasks as spotlessCheck depend on dynamic dependencies and thus the solution is to skip it.

Thanks for the help!

@rafaelrc7 commented on GitHub (Jul 20, 2024): Ok, I finally understood the problem and why it *seemed* that offline builds were not working. As discussed in https://github.com/NixOS/nixpkgs/pull/286658#issuecomment-2240070303 some tasks as `spotlessCheck` depend on dynamic dependencies and thus the solution is to skip it. Thanks for the help!

adam referenced this issue

2025-12-30 01:24:09 +01:00

[PR #44] [MERGED] Fix typos #384

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/pkl#44