package resolve does not save packages in cache #126

New Issue

adam · 2025-12-30T01:21:12+01:00

adam commented

2025-12-30 01:21:12 +01:00

Originally created by @StefMa on GitHub (Mar 28, 2024).

Right now resolving packages doesn't store packages in the cache.

This is in some situations problematic because "fully offline use" is not possible with the current implementation.
Except of the fact that it is also desired to cache packages to avoid unnecessary API calls.

Found out via this discussion.
See also this comment: https://github.com/apple/pkl/discussions/346#discussioncomment-8921705

Originally created by @StefMa on GitHub (Mar 28, 2024). Right now resolving packages doesn't store packages in the cache. This is in some situations problematic because "fully offline use" is not possible with the current implementation. Except of the fact that it is also desired to cache packages to avoid unnecessary API calls. Found out via [this discussion](https://github.com/apple/pkl/discussions/346). See also this comment: https://github.com/apple/pkl/discussions/346#discussioncomment-8921705

adam commented

2025-12-30 01:21:13 +01:00

@odenix commented on GitHub (Apr 3, 2024):

My understanding is that packages are always cached unless --no-cache is specified.
Nevertheless, something like pkl project resolve --download or pkl download-package (without arguments) could certainly be useful.

@odenix commented on GitHub (Apr 3, 2024): My understanding is that packages are always cached unless `--no-cache` is specified. Nevertheless, something like `pkl project resolve --download` or `pkl download-package` (without arguments) could certainly be useful.

adam commented

2025-12-30 01:21:13 +01:00

@holzensp commented on GitHub (Apr 3, 2024):

Instead of adding a --download argument, we're considering just always downloading (unless --no-cache is specified).

@holzensp commented on GitHub (Apr 3, 2024): Instead of adding a `--download` argument, we're considering just always downloading (unless `--no-cache` is specified).

adam commented

2025-12-30 01:21:14 +01:00

@odenix commented on GitHub (Apr 3, 2024):

Is there a benefit in keeping resolution and download decoupled? That was the idea behind pkl download-package without arguments. If resolution ever becomes non-deterministic, for example due to supporting some concept of “latest”, decoupling will be essential.

@odenix commented on GitHub (Apr 3, 2024): Is there a benefit in keeping resolution and download decoupled? That was the idea behind `pkl download-package` without arguments. If resolution ever becomes non-deterministic, for example due to supporting some concept of “latest”, decoupling will be essential.

adam commented

2025-12-30 01:21:14 +01:00

@holzensp commented on GitHub (Apr 3, 2024):

We did discuss it. When using pkl project resolve --no-cache, it would not download things. That does feel less that optimally discoverable, to be sure, but our thinking so far is that scenarios where download should actively be avoided are rare enough not to create a whole separate command for the downloading. Why would you say latest would make decoupling essential? latest would only be newer-than-what-we-have-downloaded-in-cache when you resolve (not on a normal eval; otherwise latest means you can never build without an internet connection).

@holzensp commented on GitHub (Apr 3, 2024): We did discuss it. When using `pkl project resolve --no-cache`, it would not download things. That does feel less that optimally discoverable, to be sure, but our thinking so far is that scenarios where download should actively be avoided are rare enough not to create a whole separate command for the downloading. Why would you say `latest` would make decoupling essential? `latest` would only be newer-than-what-we-have-downloaded-in-cache when you `resolve` (not on a normal `eval`; otherwise `latest` means you can never build without an internet connection).

adam commented

2025-12-30 01:21:14 +01:00

@odenix commented on GitHub (Apr 3, 2024):

It’s about actively avoiding resolution, not actively avoiding download. If all I want is to download, I should never have to worry about PklProject.deps.json changing as a side effect. According to the docs, the download-package command already exists, so I’m not sure what you mean by “whole separate command”.

@odenix commented on GitHub (Apr 3, 2024): It’s about actively avoiding resolution, not actively avoiding download. If all I want is to download, I should never have to worry about `PklProject.deps.json` changing as a side effect. According to the docs, the `download-package` command already exists, so I’m not sure what you mean by “whole separate command”.

adam commented

2025-12-30 01:21:14 +01:00

@holzensp commented on GitHub (Apr 3, 2024):

Avoiding resolution is fair enough. We were debating whether download-package should exist. Its current UI does not know about PklProject, through. This is why pkl download-package does not (yet?) work to download all project packages. Some more thinking required, but points taken.

@holzensp commented on GitHub (Apr 3, 2024): Avoiding resolution is fair enough. We were debating whether `download-package` _should_ exist. Its current UI does not know about `PklProject`, through. This is why `pkl download-package` does not (yet?) work to download all project packages. Some more thinking required, but points taken.

adam commented

2025-12-30 01:21:15 +01:00

@bioball commented on GitHub (Apr 3, 2024):

Good point about pkl project resolve maybe not being deterministic in the future. But, I don't think it relates to "pkl project resolve should not populate the cache dir". Downloading dependencies is a matter of saving network I/O, and making subsequent pkl eval executions faster.

If we do want to lean on this being the method for "I want to download all my project dependencies", we can maybe add a flag for "don't update already-resolved dependencies" in the future, if "latest" becomes a thing.

BTW: "I want to download all my dependencies" can also be a shell script:

cat PklProject.deps.json \
  | jq -c -r '.resolvedDependencies[] | select(.type == "remote") | (.uri[7:] + "::sha256:" + .checksums.sha256)' \
  | xargs -I {} pkl download-package {}

@bioball commented on GitHub (Apr 3, 2024): Good point about `pkl project resolve` maybe not being deterministic in the future. But, I don't think it relates to "`pkl project resolve` should not populate the cache dir". Downloading dependencies is a matter of saving network I/O, and making subsequent `pkl eval` executions faster. If we do want to lean on this being the method for "I want to download all my project dependencies", we can maybe add a flag for "don't update already-resolved dependencies" in the future, if "latest" becomes a thing. BTW: "I want to download all my dependencies" can also be a shell script: ```bash cat PklProject.deps.json \ | jq -c -r '.resolvedDependencies[] | select(.type == "remote") | (.uri[7:] + "::sha256:" + .checksums.sha256)' \ | xargs -I {} pkl download-package {} ```

adam commented

2025-12-30 01:21:15 +01:00

@odenix commented on GitHub (Apr 3, 2024):

I guess I don’t mind if pkl project resolve also downloads packages, although I don’t really see the need either. I don’t think it saves much network I/O as packages will only be downloaded once in any case. Also, downloading as part of resolving won’t help all the other folks that didn’t run the resolve command themselves. Overall, not downloading by default feels more consistent with Pkl’s lazy attitude.

My gut feeling is that there should be an easy way to download all package dependencies without resolving. If the download-package command is here to stay, I’d expect it to be capable of this.

@odenix commented on GitHub (Apr 3, 2024): I guess I don’t mind if `pkl project resolve` also downloads packages, although I don’t really see the need either. I don’t think it saves much network I/O as packages will only be downloaded once in any case. Also, downloading as part of resolving won’t help all the other folks that didn’t run the resolve command themselves. Overall, not downloading by default feels more consistent with Pkl’s lazy attitude. My gut feeling is that there should be an easy way to download all package dependencies without resolving. If the `download-package` command is here to stay, I’d expect it to be capable of this.

adam commented

2025-12-30 01:21:16 +01:00

@bioball commented on GitHub (Apr 4, 2024):

The resolve command does need to download in order to build the resolved dependency list. Since it's already downloading, it's actually a little strange that it's not writing them to the cache dir.

My gut feeling is that there should be an easy way to download all package dependencies without resolving. If the download-package command is here to stay, I’d expect it to be capable of this.

Fair; I can definitely see a mode in download-package that just downloads project dependencies. Maybe something like: pkl download-package --project=my-project/

@bioball commented on GitHub (Apr 4, 2024): The resolve command _does_ need to download in order to build the resolved dependency list. Since it's already downloading, it's actually a little strange that it's not writing them to the cache dir. > My gut feeling is that there should be an easy way to download all package dependencies without resolving. If the download-package command is here to stay, I’d expect it to be capable of this. Fair; I can definitely see a mode in `download-package` that just downloads project dependencies. Maybe something like: `pkl download-package --project=my-project/`

adam commented

2025-12-30 01:21:16 +01:00

@odenix commented on GitHub (Apr 4, 2024):

Oh I thought that pkl project resolve would just download metadata. If it has to download the packages, it should definitely store them in the cache, at least by default.

@odenix commented on GitHub (Apr 4, 2024): Oh I thought that `pkl project resolve` would just download metadata. If it has to download the packages, it should definitely store them in the cache, at least by default.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/pkl#126