Add support for HTTP rewrites (#1062)

This adds a new configuration option for the HTTP client to replace URI prefixes when making outbound calls.

Follows the design of https://github.com/apple/pkl-evolution/pull/17
This commit is contained in:
Daniel Chao
2025-07-16 15:53:31 -07:00
committed by GitHub
parent fea031a138
commit 99020bb79d
27 changed files with 607 additions and 47 deletions

View File

@@ -2154,6 +2154,9 @@ Optionally, the SHA-256 checksum of the package can also be specified:
Packages can be managed as dependencies within a _project_.
For more details, consult the <<projects,project>> section of the language reference.
Packages can also be downloaded from a mirror.
For more details, consult the <<mirroring_packages>> section of the language reference.
==== Standard Library URI
Example: `+pkl:math+`
@@ -3204,7 +3207,8 @@ This section discusses language features that are generally more relevant to tem
<<reserved-keywords,Reserved Keywords>> +
<<blank-identifiers,Blank Identifiers>> +
<<projects,Projects>> +
<<external-readers,External Readers>>
<<external-readers,External Readers>> +
<<mirroring_packages,Mirroring packages>>
[[meaning-of-new]]
=== Meaning of `new`
@@ -5755,3 +5759,25 @@ To support both schemes during evaluation, both would need to be registered expl
----
$ pkl eval <module> --external-resource-reader ldap=pkl-ldap --external-resource-reader ldaps=pkl-ldap
----
[[mirroring_packages]]
=== Mirroring packages
A package is a shareable archive of modules and resources that are published to the internet.
A package's URI tells two things:
1. The name of the package.
2. Where the package is downloaded from.
For example, given the package name `package://example.com/mypackage@1.0.0`, Pkl will make an HTTPS request to `\https://example.com/mypackage@1.0.0` to fetch package metadata.
In situations where internet access is restricted, a mirror can be set up to allow use of packages that are published to the internet.
To direct Pkl to a mirror, the `--http-rewrite` CLI option (and its equivalent options when using Pkl's other evaluator APIs) must be used.
For example, `--http-rewrite \https://pkg.pkl-lang.org/=\https://my.internal.mirror/` will tell Pkl to download packages from host `my.internal.mirror`.
NOTE: To effectively mirror packages from pkg.pkl-lang.org, there must be two rewrites; one for `\https://pkg.pkl-lang.org/` (where package metadata is downloaded), and one for `\https://github.com/` (where package zip files are downloaded).
NOTE: Pkl does not provide any tooling to run a mirror server.
To fully set up mirroring, an HTTP(s) server will need be running, and which mirrors the same assets byte-for-byte.

View File

@@ -152,3 +152,14 @@ Example: `example.com,169.254.0.0/16` +
Comma separated list of hosts to which all connections should bypass the proxy.
Hosts can be specified by name, IP address, or IP range using https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#CIDR_notation[CIDR notation].
====
.--http-rewrite
[%collapsible]
====
Default: (none) +
Example: `\https://pkg.pkl-lang.org/=https://my.internal.mirror/` +
Replace outbound HTTP(S) requests from one URL with another URL.
The left-hand side describes the source prefix, and the right-hand describes the target prefix.
This option is commonly used to enable package mirroring.
The above example will rewrite URL `\https://pkg.pkl-lang.org/pkl-k8s/k8s@1.0.0` to `\https://my.internal.mirror/pkl-k8s/k8s@1.0.0`.
====

View File

@@ -107,3 +107,14 @@ Example: `noProxy = ["example.com", "169.254.0.0/16"]` +
Hosts to which all connections should bypass the proxy.
Hosts can be specified by name, IP address, or IP range using https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#CIDR_notation[CIDR notation].
====
.httpRewrites: MapProperty<String, String>
[%collapsible]
====
Default: `null` +
Example: `httpRewrites = [uri("https://pkg.pkl-lang.org/"): uri("https://my.internal.mirror/")]` +
Replace outbound HTTP(S) requests from one URL with another URL.
The left-hand side describes the source prefix, and the right-hand describes the target prefix.
This option is commonly used to enable package mirroring.
The above example will rewrite URL `\https://pkg.pkl-lang.org/pkl-k8s/k8s@1.0.0` to `\https://my.internal.mirror/pkl-k8s/k8s@1.0.0`.
====