Add net stdlib module with URL support (#1834)

This commit is contained in:
Islon Scherer authored and GitHub committed 2026-10-06 15:23:32 +02:00
1 parent dbcecd9677
commit 862c61f2e2
46 files changed
+3682 -15

No files matched your search

@@ -74,6 +74,8 @@ endif::[]
:uri-stdlib-jsonnetModule: {uri-pkl-stdlib-docs}/jsonnet
:uri-stdlib-reflectModule: {uri-pkl-stdlib-docs}/reflect
:uri-stdlib-mathModule: {uri-pkl-stdlib-docs}/math
:uri-stdlib-netModule: {uri-pkl-stdlib-docs}/net
:uri-stdlib-net-Url: {uri-stdlib-netModule}/Url
:uri-stdlib-xmlModule: {uri-pkl-stdlib-docs}/xml
:uri-stdlib-protobufModule: {uri-pkl-stdlib-docs}/protobuf
:uri-stdlib-pklbinaryModule: {uri-pkl-stdlib-docs}/pklbinary
+115 -1
View File
@@ -9,7 +9,77 @@ include::partial$intro.adoc[]
== Highlights [small]#💖#
=== XXX
=== New standard library module: `pkl:net`
This release introduces a new standard library module, link:{uri-stdlib-netModule}[`pkl:net`], which brings native URL support to Pkl (https://github.com/apple/pkl/pull/1834[#1834]).
Its centerpiece is link:{uri-stdlib-net-Url}[`net.Url`], a URL as defined by https://www.rfc-editor.org/rfc/rfc3986[RFC 3986].
A string is parsed with `net.Url()`, which accepts an absolute URL as well as a relative reference, and which throws if the string cannot be parsed:
[source,pkl]
----
import "pkl:net"
local url = net.Url("https://user:pw@example.com:8080/a/b c?q=1&q=2#f")
scheme = url.scheme // "https"
rawUserInfo = url.rawUserInfo // "user:pw"
authority = url.authority // "user:pw@example.com:8080"
rawHost = url.rawHost // "example.com"
rawPort = url.rawPort // "8080"
port = url.port // 8080
rawPath = url.rawPath // "/a/b%20c"
path = url.path // "/a/b c"
pathSegments = url.pathSegments // List("a", "b c")
rawQuery = url.rawQuery // "q=1&q=2"
queryParameters = url.queryParameters // Mapping { ["q"] = Listing { "1"; "2" } }
rawFragment = url.rawFragment // "f"
----
Parsing is lenient: whatever cannot appear literally in a component is percent-encoded as its UTF-8 bytes, and whatever is already encoded is kept as written.
Every component is held percent-encoded in its `raw` property, whether the input was encoded or not; the percent-decoded form of each is available as `userInfo`, `host`, `path`, `query`, and `fragment`.
The derived `pathSegments` and `queryParameters` are percent-decoded too.
A URL can also be built from its components, or derived from another URL by amending it.
A component given this way has to be percent-encoded already:
[source,pkl]
----
built = new net.Url {
scheme = "https"
rawHost = "example.com"
rawPath = "/a%20b"
}.toString() // "https://example.com/a%20b"
local base = net.Url("https://example.com/a?q=1")
amended = (base) { rawQuery = null }.toString() // "https://example.com/a"
----
Each component is type-checked, so a URL built this way cannot be malformed.
For example, a `rawPort` can only hold digits, only a URL with a `rawHost` can have one, and a `rawPath` of `"/a b"` is rejected.
The decoded components are `fixed`, so they cannot be amended.
A relative reference can be resolved against an absolute URL, which also removes dot segments from the result's path:
[source,pkl]
----
resolved = net.Url("https://example.com/a/b").resolve("../c").toString() // "https://example.com/c"
----
Comparing two URLs with `==` compares their components as written.
To instead compare them as the *syntax-based normalization* of RFC 3986, where scheme and host are case-insensitive and percent-encoding is normalized, use `equals()`:
[source,pkl]
----
same = net.Url("HTTPS://Example.com/%7Euser").equals(net.Url("https://example.com/~user")) // true
----
The module also provides standalone helpers: `net.encodeUrlComponent()` and `net.decodeUrlComponent()` for percent-encoding, and `net.buildQuery()` and `net.parseQuery()` for `application/x-www-form-urlencoded` queries.
Alongside this module, `pkl:base` gains link:{uri-stdlib-String}#isValidUrl[`String.isValidUrl`], which tells whether a string is a valid absolute URL.
It is stricter than `net.Url()`, which also parses a relative reference, and which percent-encodes whatever cannot appear literally in a URL component.
The `Uri` typealias is now defined in terms of it, which is a <<uri-validates,breaking change>>.
== Noteworthy [small]#🎶#
@@ -51,6 +121,30 @@ For example, setting `XDG_CACHE_HOME` will configure the cache directory.
Note that the existing `~/.pkl/cache` directory is ignored, so Pkl will download packages to populate its cache if configured to do so.
=== Java Library Changes
==== `pkl.net#Url` maps to `java.net.URI`
The JVM bindings now understand the new `pkl.net#Url` class (https://github.com/apple/pkl/pull/1834[#1834]).
The Java and Kotlin code generators generate a `java.net.URI` for a property whose type is `net.Url`.
Two conversions are added to `org.pkl.config.java.mapper.Conversions`, both part of the preconfigured set:
* `pUrlToString` -- gives the same result as `Url.toString()`
* `pUrlToURI` -- throws `ConversionException` if the URL cannot be represented as a `java.net.URI`
=== Standard Library Changes
==== `pkl:net`
* New module introduced.
==== `pkl:base`
* New property: link:{uri-stdlib-String}#isValidUrl[`String.isValidUrl`]
* Changed typealias: link:{uri-stdlib-Uri}[`Uri`] is now `String(isValidUrl)` (<<uri-validates,breaking change>>)
== Breaking Changes [small]#💔#
=== Removal of macOS amd64 (Intel) Binaries
@@ -126,6 +220,26 @@ res = """
"""
----
[[uri-validates]]
=== `Uri` validates its value
The link:{uri-stdlib-Uri}[`Uri`] typealias in `pkl:base` is now `String(isValidUrl)` instead of a plain `String` (https://github.com/apple/pkl/pull/1834[#1834]).
A value typed as `Uri` must be a valid absolute URL according to https://www.rfc-editor.org/rfc/rfc3986[RFC 3986].
This means that a relative reference, or a URL containing a character that must be percent-encoded, no longer type-checks:
[source,pkl]
----
good: Uri = "https://example.com/some%20path"
bad1: Uri = "./some/path" // throws: not absolute
bad2: Uri = "https://example.com/some path" // throws: space must be percent-encoded
----
This affects every standard library member typed as `Uri`, among them link:{uri-stdlib-Resource}#uri[`Resource.uri`], link:{uri-stdlib-analyzeModule}#importGraph()[`analyze.importGraph()`], and the URI-valued properties of `pkl:Project`, `pkl:EvaluatorSettings` and `pkl:DocPackageInfo`.
As a consequence, an invalid URI is now reported as a type constraint violation at the point where it is set, rather than as an error raised later by whatever consumes it.
== Bug Fixes [small]#🐜#
* XXX
@@ -845,6 +845,7 @@ class JavaCodeGenerator(
PClassInfo.Class -> PCLASS.nullableIf(nullable)
PClassInfo.Regex -> PATTERN.nullableIf(nullable)
PClassInfo.Version -> VERSION.nullableIf(nullable)
PClassInfo.Url -> URI.nullableIf(nullable)
else ->
when {
!classInfo.isStandardLibraryClass -> pClass.toJavaPoetName().nullableIf(nullable)
@@ -1159,6 +1159,52 @@ class JavaCodeGeneratorTest {
)
}
@Test
fun `pkl_net Url`() {
val javaCode =
generateJavaCode(
"""
module mod
import "pkl:net"
url: net.Url
nullableUrl: net.Url?
listing: Listing<net.Url>
mapping: Mapping<String, net.Url>
class Foo {
url: net.Url
}
"""
.trimIndent()
)
assertThat(javaCode)
.compilesSuccessfully()
.contains("import java.net.URI;")
.contains(
"""
|public final class Mod {
| public final @NonNull URI url;
|
| public final URI nullableUrl;
|
| public final @NonNull List<@NonNull URI> listing;
|
| public final @NonNull Map<@NonNull String, @NonNull URI> mapping;
"""
.trimMargin()
)
.contains(
"""
| public static final class Foo {
| public final @NonNull URI url;
"""
.trimMargin()
)
}
@Test
fun `nullable properties`() {
var javaCode =
@@ -703,6 +703,7 @@ class KotlinCodeGenerator(
PClassInfo.Class -> PCLASS
PClassInfo.Regex -> REGEX
PClassInfo.Version -> VERSION
PClassInfo.Url -> URI
else ->
when {
!classInfo.isStandardLibraryClass -> pClass.toKotlinPoetName()
@@ -1186,6 +1186,52 @@ class KotlinCodeGeneratorTest {
)
}
@Test
fun `pkl_net Url`() {
val kotlinCode =
generateKotlinCode(
"""
module mod
import "pkl:net"
url: net.Url
nullableUrl: net.Url?
listing: Listing<net.Url>
mapping: Mapping<String, net.Url>
class Foo {
url: net.Url
}
"""
.trimIndent()
)
assertThat(kotlinCode)
.compilesSuccessfully()
.isEqualTo(
"""
import java.net.URI
import kotlin.String
import kotlin.collections.List
import kotlin.collections.Map
data class Mod(
val url: URI,
val nullableUrl: URI?,
val listing: List<URI>,
val mapping: Map<String, URI>
) {
data class Foo(
val url: URI
)
}
"""
.trimIndent()
)
}
@Test
fun `user defined type aliases`() {
val kotlinCode =
@@ -1,5 +1,5 @@
/*
* Copyright © 2024-2025 Apple Inc. and the Pkl project authors. All rights reserved.
* Copyright © 2024-2026 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -265,6 +265,36 @@ public final class Conversions {
}
});
/**
* Conversion from {@code pkl.net#Url} to {@link String}. Gives the same result as {@code
* Url.toString()}.
*/
public static final Conversion<PObject, String> pUrlToString =
Conversion.of(PClassInfo.Url, String.class, (value, mapper) -> serializeUrl(value));
/**
* Conversion from {@code pkl.net#Url} to {@link URI}. Throws {@link ConversionException} if the
* URL cannot be represented as a {@link URI}, which is the case for a host that is an IPvFuture
* address.
*/
public static final Conversion<PObject, URI> pUrlToURI =
Conversion.of(
PClassInfo.Url,
URI.class,
(value, mapper) -> {
var url = serializeUrl(value);
try {
return new URI(url);
} catch (URISyntaxException e) {
throw new ConversionException(
String.format("Failed to convert `pkl.net#Url` `%s` to `java.net.URI`.", url), e);
}
});
private static String serializeUrl(PObject url) {
return (String) url.getProperty("stringValue");
}
/**
* Identity conversions used when the Java representation of the Pkl type matches the target type
* or when the target type is {@link Object}.
@@ -323,7 +353,9 @@ public final class Conversions {
pDurationToDuration,
pVersionToVersion,
pVersionToString,
pStringToVersion);
pStringToVersion,
pUrlToString,
pUrlToURI);
/** All conversions defined in this class. */
public static final Collection<Conversion<?, ?>> all = collectAll();
@@ -0,0 +1,66 @@
/*
* Copyright © 2026 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.pkl.config.java.mapper;
import static org.assertj.core.api.Assertions.assertThat;
import static org.pkl.core.ModuleSource.modulePath;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.Test;
import org.pkl.core.Evaluator;
import org.pkl.core.PModule;
public class PUrlToStringTest {
private static final Evaluator evaluator = Evaluator.preconfigured();
private static final PModule module =
evaluator.evaluate(modulePath("org/pkl/config/java/mapper/PUrlToURITest.pkl"));
private static final ValueMapper mapper = ValueMapperBuilder.preconfigured().build();
@AfterAll
public static void afterAll() {
evaluator.close();
}
@Test
public void absolute() {
assertThat(map("absolute")).isEqualTo("https://user:pw@example.com:8080/a/b?q=1#frag");
}
@Test
public void percentEncoding() {
assertThat(map("literalSpace")).isEqualTo("https://example.com/a%20b");
assertThat(map("nonAscii")).isEqualTo("https://example.com/%C3%A9");
}
@Test
public void relative() {
assertThat(map("relative")).isEqualTo("/foo/bar");
assertThat(map("colonSegment")).isEqualTo("./a:b");
}
@Test
public void ipvFuture() {
assertThat(map("ipvFuture")).isEqualTo("http://[v1.fe80::a+en1]/");
}
private static String map(String propertyName) {
var url = module.get(propertyName);
assert url != null;
return mapper.map(url, String.class);
}
}
@@ -0,0 +1,92 @@
/*
* Copyright © 2026 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.pkl.config.java.mapper;
import static org.assertj.core.api.Assertions.*;
import static org.pkl.core.ModuleSource.modulePath;
import java.net.URI;
import java.net.URISyntaxException;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.Test;
import org.pkl.core.Evaluator;
import org.pkl.core.PModule;
public class PUrlToURITest {
private static final Evaluator evaluator = Evaluator.preconfigured();
private static final PModule module =
evaluator.evaluate(modulePath("org/pkl/config/java/mapper/PUrlToURITest.pkl"));
private static final ValueMapper mapper = ValueMapperBuilder.preconfigured().build();
@AfterAll
public static void afterAll() {
evaluator.close();
}
@Test
public void absolute() {
assertThat(map("absolute"))
.isEqualTo(URI.create("https://user:pw@example.com:8080/a/b?q=1#frag"));
}
@Test
public void percentEncoding() {
// characters that cannot appear literally are encoded, and what is already encoded is kept
assertThat(map("literalSpace")).isEqualTo(URI.create("https://example.com/a%20b"));
assertThat(map("encodedSpace")).isEqualTo(URI.create("https://example.com/a%20b"));
assertThat(map("encodedSlash")).isEqualTo(URI.create("https://example.com/a%2Fb"));
assertThat(map("nonAscii")).isEqualTo(URI.create("https://example.com/%C3%A9"));
}
@Test
public void relative() {
assertThat(map("relative")).isEqualTo(URI.create("/foo/bar"));
// a first segment holding a ":" has to be preceded by a dot-segment, or it would be read as a
// scheme
assertThat(map("colonSegment")).isEqualTo(URI.create("./a:b"));
}
@Test
public void authority() {
assertThat(map("rootless")).isEqualTo(URI.create("mailto:user@example.com"));
assertThat(map("noAuthority")).isEqualTo(URI.create("file:/tmp"));
assertThat(map("emptyAuthority")).isEqualTo(URI.create("file:///tmp"));
assertThat(map("emptyQuery")).isEqualTo(URI.create("https://example.com/?"));
assertThat(map("ipv6Zone")).isEqualTo(URI.create("http://[fe80::1%25eth0]/p"));
}
@Test
public void constructed() {
assertThat(map("constructed")).isEqualTo(URI.create("https://example.com/a%20b?x%20y"));
}
@Test
public void ipvFuture() {
// `java.net.URI` doesn't know IPvFuture addresses
assertThatThrownBy(() -> map("ipvFuture"))
.isInstanceOf(ConversionException.class)
.hasMessageContaining("http://[v1.fe80::a+en1]/")
.hasCauseInstanceOf(URISyntaxException.class);
}
private static URI map(String propertyName) {
var url = module.get(propertyName);
assert url != null;
return mapper.map(url, URI.class);
}
}
@@ -0,0 +1,16 @@
import "pkl:net"
absolute = net.Url("https://user:pw@example.com:8080/a/b?q=1#frag")
literalSpace = net.Url("https://example.com/a b")
encodedSpace = net.Url("https://example.com/a%20b")
encodedSlash = net.Url("https://example.com/a%2Fb")
nonAscii = net.Url("https://example.com/é")
relative = net.Url("/foo/bar")
rootless = net.Url("mailto:user@example.com")
noAuthority = net.Url("file:/tmp")
emptyAuthority = net.Url("file:///tmp")
emptyQuery = net.Url("https://example.com/?")
ipv6Zone = net.Url("http://[fe80::1%25eth0]/p")
ipvFuture = net.Url("http://[v1.fe80::a+en1]/")
constructed = new net.Url { scheme = "https"; rawHost = "example.com"; rawPath = "/a%20b"; rawQuery = "x%20y" }
colonSegment = new net.Url { rawPath = "./a:b" }
@@ -37,6 +37,7 @@ public final class PClassInfo<T> implements Serializable {
public static final URI pklBaseUri = URI.create("pkl:base");
public static final URI pklSemverUri = URI.create("pkl:semver");
public static final URI pklNetUri = URI.create("pkl:net");
public static final URI pklSettingsUri = URI.create("pkl:settings");
public static final URI pklProjectUri = URI.create("pkl:Project");
public static final URI pklRefUri = URI.create("pkl:ref");
@@ -82,6 +83,8 @@ public final class PClassInfo<T> implements Serializable {
pklBaseClassInfo("ModuleInfo", PObject.class);
public static final PClassInfo<PObject> Version =
new PClassInfo<>("pkl.semver", "Version", PObject.class, pklSemverUri);
public static final PClassInfo<PObject> Url =
new PClassInfo<>("pkl.net", "Url", PObject.class, pklNetUri);
public static final PClassInfo<PObject> Project =
new PClassInfo<>("pkl.Project", MODULE_CLASS_NAME, PObject.class, pklProjectUri);
public static final PClassInfo<PObject> Settings =
@@ -0,0 +1,44 @@
/*
* Copyright © 2026 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.pkl.core.ast.internal;
import com.oracle.truffle.api.dsl.Cached;
import com.oracle.truffle.api.dsl.NeverDefault;
import com.oracle.truffle.api.dsl.Specialization;
import com.oracle.truffle.api.nodes.IndirectCallNode;
import org.pkl.core.ast.PklNode;
import org.pkl.core.runtime.VmTyped;
import org.pkl.core.util.url.UrlFactory;
import org.pkl.core.util.url.UrlParser.Parsed;
public abstract class GetParsedUrlNode extends PklNode {
@NeverDefault
public static GetParsedUrlNode create() {
return GetParsedUrlNodeGen.create();
}
@Specialization(guards = "url.hasExtraStorage()")
protected Parsed evalCached(VmTyped url) {
return (Parsed) url.getExtraStorage();
}
@Specialization(replaces = "evalCached")
protected Parsed eval(VmTyped url, @Cached("create()") IndirectCallNode callNode) {
return UrlFactory.read(url, callNode);
}
public abstract Parsed execute(VmTyped url);
}
@@ -147,6 +147,15 @@ public final class Identifier implements Comparable<Identifier> {
// members of pkl.yaml
public static final Identifier MAX_COLLECTION_ALIASES = get("maxCollectionAliases");
// members of pkl.net
public static final Identifier RAW_USER_INFO = get("rawUserInfo");
public static final Identifier RAW_PATH = get("rawPath");
public static final Identifier RAW_QUERY = get("rawQuery");
public static final Identifier SCHEME = get("scheme");
public static final Identifier RAW_HOST = get("rawHost");
public static final Identifier RAW_PORT = get("rawPort");
public static final Identifier RAW_FRAGMENT = get("rawFragment");
// members of pkl.Command
public static final Identifier OPTIONS = get("options");
public static final Identifier PARENT = get("parent");
@@ -98,6 +98,8 @@ public final class ModuleCache {
return JsonnetModule.getModule();
case "math":
return MathModule.getModule();
case "net":
return NetModule.getModule();
case "pklbinary":
return PklBinaryModule.getModule();
case "platform":
@@ -0,0 +1,47 @@
/*
* Copyright © 2026 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.pkl.core.runtime;
import com.oracle.truffle.api.CompilerDirectives.TruffleBoundary;
import java.net.URI;
public final class NetModule extends StdLibModule {
private static final VmTyped instance = VmUtils.createEmptyModule();
static {
loadModule(URI.create("pkl:net"), instance);
}
private NetModule() {}
public static VmTyped getModule() {
return instance;
}
public static VmClass getUrlClass() {
return UrlClass.instance;
}
private static final class UrlClass {
static final VmClass instance = loadClass("Url");
}
@TruffleBoundary
private static VmClass loadClass(String className) {
var theModule = getModule();
return (VmClass) VmUtils.readMember(theModule, Identifier.get(className));
}
}
@@ -34,6 +34,7 @@ import org.pkl.core.util.GlobResolver;
import org.pkl.core.util.GlobResolver.InvalidGlobPatternException;
import org.pkl.core.util.Pair;
import org.pkl.core.util.StringUtils;
import org.pkl.core.util.url.UrlParser;
import org.pkl.parser.Lexer;
import org.pkl.parser.ParserError;
import org.pkl.parser.Token;
@@ -197,6 +198,14 @@ public final class StringNodes {
}
}
public abstract static class isValidUrl extends ExternalPropertyNode {
@Specialization
@TruffleBoundary
protected boolean eval(String self) {
return UrlParser.isValidUrl(self);
}
}
public abstract static class chars extends ExternalPropertyNode {
@Specialization
@TruffleBoundary
@@ -0,0 +1,87 @@
/*
* Copyright © 2026 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.pkl.core.stdlib.net;
import com.oracle.truffle.api.CompilerDirectives.TruffleBoundary;
import com.oracle.truffle.api.dsl.Specialization;
import org.pkl.core.runtime.VmListing;
import org.pkl.core.runtime.VmMapping;
import org.pkl.core.runtime.VmTyped;
import org.pkl.core.stdlib.ExternalMethod1Node;
import org.pkl.core.util.url.PercentEncoder;
import org.pkl.core.util.url.UrlFactory;
/** Backing nodes for {@code pkl:net}'s module-level members. */
public final class NetNodes {
private NetNodes() {}
public abstract static class Url extends ExternalMethod1Node {
@Specialization
protected Object eval(@SuppressWarnings("unused") VmTyped self, String input) {
return UrlFactory.create(UrlFactory.parseOrThrow(input, this));
}
}
public abstract static class encodeUrlComponent extends ExternalMethod1Node {
@Specialization
@TruffleBoundary
protected String eval(@SuppressWarnings("unused") VmTyped self, String value) {
var out = new StringBuilder(value.length());
PercentEncoder.encodeComponent(out, value);
return out.toString();
}
}
public abstract static class decodeUrlComponent extends ExternalMethod1Node {
@Specialization
protected String eval(@SuppressWarnings("unused") VmTyped self, String value) {
UrlFactory.checkPercentEncoding(value, "cannotDecodeUrlComponent", this);
return PercentEncoder.decode(value);
}
}
public abstract static class buildQuery extends ExternalMethod1Node {
@Specialization
@TruffleBoundary
protected String eval(@SuppressWarnings("unused") VmTyped self, VmMapping parameters) {
var out = new StringBuilder();
parameters.forceAndIterateMemberValues(
(name, def, values) -> {
((VmListing) values)
.forceAndIterateMemberValues(
(index, valueDef, value) -> {
if (!out.isEmpty()) {
out.append('&');
}
PercentEncoder.encodeForm(out, (String) name);
out.append('=');
PercentEncoder.encodeForm(out, (String) value);
return true;
});
return true;
});
return out.toString();
}
}
public abstract static class parseQuery extends ExternalMethod1Node {
@Specialization
protected VmMapping eval(@SuppressWarnings("unused") VmTyped self, String query) {
UrlFactory.checkPercentEncoding(query, "cannotParseQuery", this);
return UrlFactory.createQueryParameters(query);
}
}
}
@@ -0,0 +1,225 @@
/*
* Copyright © 2026 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.pkl.core.stdlib.net;
import com.oracle.truffle.api.CompilerDirectives.TruffleBoundary;
import com.oracle.truffle.api.dsl.Cached;
import com.oracle.truffle.api.dsl.Specialization;
import com.oracle.truffle.api.nodes.IndirectCallNode;
import org.jspecify.annotations.Nullable;
import org.pkl.core.ast.internal.GetParsedUrlNode;
import org.pkl.core.ast.internal.GetParsedUrlNodeGen;
import org.pkl.core.runtime.VmList;
import org.pkl.core.runtime.VmNull;
import org.pkl.core.runtime.VmTyped;
import org.pkl.core.stdlib.ExternalMethod0Node;
import org.pkl.core.stdlib.ExternalMethod1Node;
import org.pkl.core.stdlib.ExternalMethod3Node;
import org.pkl.core.stdlib.ExternalPropertyNode;
import org.pkl.core.util.url.UrlFactory;
import org.pkl.core.util.url.UrlParser;
import org.pkl.core.util.url.UrlParser.Parsed;
/** Backing nodes for {@code pkl:net}'s {@code Url} class. */
public final class UrlNodes {
private UrlNodes() {}
public abstract static class authority extends ExternalPropertyNode {
@Specialization(guards = "self.hasExtraStorage()")
protected Object evalCached(VmTyped self) {
var parsed = (Parsed) self.getExtraStorage();
return VmNull.lift(parsed.authority());
}
@Specialization(guards = "!self.hasExtraStorage()")
protected Object eval(VmTyped self, @Cached("create()") IndirectCallNode callNode) {
return VmNull.lift(UrlFactory.readAuthority(self, callNode));
}
}
public abstract static class hostKind extends ExternalPropertyNode {
@Specialization(guards = "self.hasExtraStorage()")
protected Object evalCached(VmTyped self) {
var parsed = (Parsed) self.getExtraStorage();
return getHostKind(parsed.host());
}
@Specialization(guards = "!self.hasExtraStorage()")
protected Object eval(VmTyped self, @Cached("create()") IndirectCallNode callNode) {
return getHostKind(UrlFactory.readHost(self, callNode));
}
private static Object getHostKind(@Nullable String host) {
return host == null ? VmNull.withoutDefault() : UrlParser.hostKind(host);
}
}
public abstract static class zoneId extends ExternalPropertyNode {
@Specialization(guards = "self.hasExtraStorage()")
protected Object evalCached(VmTyped self) {
var parsed = (Parsed) self.getExtraStorage();
return getZoneId(parsed.host());
}
@Specialization(guards = "!self.hasExtraStorage()")
protected Object eval(VmTyped self, @Cached("create()") IndirectCallNode callNode) {
return getZoneId(UrlFactory.readHost(self, callNode));
}
private static Object getZoneId(@Nullable String host) {
return VmNull.lift(host == null ? null : UrlParser.zoneId(host));
}
}
public abstract static class pathSegments extends ExternalPropertyNode {
@Specialization(guards = "self.hasExtraStorage()")
protected VmList evalCached(VmTyped self) {
var parsed = (Parsed) self.getExtraStorage();
return VmList.create(UrlParser.segments(parsed.path()));
}
@Specialization(guards = "!self.hasExtraStorage()")
protected VmList eval(VmTyped self, @Cached("create()") IndirectCallNode callNode) {
return VmList.create(UrlParser.segments(UrlFactory.readPath(self, callNode)));
}
}
public abstract static class queryParameters extends ExternalPropertyNode {
@Specialization(guards = "self.hasExtraStorage()")
protected Object evalCached(VmTyped self) {
var parsed = (Parsed) self.getExtraStorage();
return getQueryParameters(parsed.query());
}
@Specialization(guards = "!self.hasExtraStorage()")
protected Object eval(VmTyped self, @Cached("create()") IndirectCallNode callNode) {
return getQueryParameters(UrlFactory.readQuery(self, callNode));
}
private static Object getQueryParameters(@Nullable String query) {
return query == null ? VmNull.withoutDefault() : UrlFactory.createQueryParameters(query);
}
}
public abstract static class toString extends ExternalMethod0Node {
@Child private GetParsedUrlNode getParsedUrlNode = GetParsedUrlNodeGen.create();
@Specialization
protected String eval(VmTyped self) {
var parsed = getParsedUrlNode.execute(self);
return parsed.serialize();
}
}
public abstract static class resolve extends ExternalMethod1Node {
@Child private GetParsedUrlNode getParsedNode = GetParsedUrlNode.create();
@Specialization
protected Object evalString(VmTyped self, String ref) {
return resolve(getParsedNode.execute(self), UrlFactory.parseOrThrow(ref, this));
}
@Specialization
protected Object eval(VmTyped self, VmTyped ref) {
return resolve(getParsedNode.execute(self), getParsedNode.execute(ref));
}
@SuppressWarnings("MethodNameSameAsClassName")
@TruffleBoundary
private Object resolve(Parsed base, Parsed ref) {
if (base.scheme() == null) {
throw exceptionBuilder()
.evalError("cannotResolveAgainstRelativeUrl", base.serialize())
.build();
}
return UrlFactory.create(UrlParser.resolve(base, ref));
}
}
public abstract static class normalized extends ExternalPropertyNode {
@Child private GetParsedUrlNode getSelfNode = GetParsedUrlNode.create();
@Specialization
protected VmTyped eval(VmTyped self) {
return UrlFactory.create(UrlParser.normalize(getSelfNode.execute(self)));
}
}
public abstract static class equals extends ExternalMethod1Node {
@Child private GetParsedUrlNode getParsedNode = GetParsedUrlNode.create();
@Specialization
protected boolean eval(VmTyped self, VmTyped other) {
return UrlParser.isEquivalent(getParsedNode.execute(self), getParsedNode.execute(other));
}
}
// The constraints of `Url`. Each one only accepts a component that is already percent-encoded,
// and is otherwise the same check the parser makes, so that a URL written or amended by hand is
// held to exactly the standard of the parser's output.
public abstract static class isValidUserInfo extends ExternalMethod1Node {
@Specialization
@TruffleBoundary
protected boolean eval(@SuppressWarnings("unused") VmTyped self, String value) {
return UrlParser.isValidUserInfo(value);
}
}
public abstract static class isValidQuery extends ExternalMethod1Node {
@Specialization
@TruffleBoundary
protected boolean eval(@SuppressWarnings("unused") VmTyped self, String value) {
return UrlParser.isValidQueryOrFragment(value);
}
}
public abstract static class isValidFragment extends ExternalMethod1Node {
@Specialization
@TruffleBoundary
protected boolean eval(@SuppressWarnings("unused") VmTyped self, String value) {
return UrlParser.isValidQueryOrFragment(value);
}
}
public abstract static class isValidScheme extends ExternalMethod1Node {
@Specialization
@TruffleBoundary
protected boolean eval(@SuppressWarnings("unused") VmTyped self, String value) {
return UrlParser.isValidScheme(value);
}
}
public abstract static class isValidHost extends ExternalMethod1Node {
@Specialization
@TruffleBoundary
protected boolean eval(@SuppressWarnings("unused") VmTyped self, String value) {
return UrlParser.isValidHost(value);
}
}
public abstract static class isValidPath extends ExternalMethod3Node {
@Specialization
@TruffleBoundary
protected boolean eval(
@SuppressWarnings("unused") VmTyped self,
String value,
boolean hasScheme,
boolean hasAuthority) {
return UrlParser.isValidPath(value, hasScheme, hasAuthority);
}
}
}
@@ -0,0 +1,19 @@
/*
* Copyright © 2026 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
@NullMarked
package org.pkl.core.stdlib.net;
import org.jspecify.annotations.NullMarked;
@@ -0,0 +1,226 @@
/*
* Copyright © 2026 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.pkl.core.util.url;
import com.oracle.truffle.api.CompilerDirectives.TruffleBoundary;
import java.io.ByteArrayOutputStream;
import java.net.URLDecoder;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.util.function.IntPredicate;
/**
* Percent-encoding, and the character classes of <a
* href="https://www.rfc-editor.org/rfc/rfc3986#appendix-A">RFC 3986's ABNF</a>.
*
* <p>Each {@code IntPredicate} here holds for the characters that may appear literally in a
* component. Anything else has to be percent-encoded to appear there at all.
*/
public final class PercentEncoder {
private PercentEncoder() {}
/** unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~" */
static final IntPredicate UNRESERVED = PercentEncoder::isUnreserved;
/** userinfo = *( unreserved / pct-encoded / sub-delims / ":" ) */
static final IntPredicate USERINFO = c -> isUnreserved(c) || isSubDelim(c) || c == ':';
/** reg-name = *( unreserved / pct-encoded / sub-delims ) */
static final IntPredicate REG_NAME = c -> isUnreserved(c) || isSubDelim(c);
/** {@code *( pchar / "/" )}, which covers every shape of path. */
static final IntPredicate PATH = c -> isPchar(c) || c == '/';
/** {@code query} and {@code fragment} share a set: {@code *( pchar / "/" / "?" )}. */
static final IntPredicate QUERY_OR_FRAGMENT = c -> isPchar(c) || c == '/' || c == '?';
private static boolean isUnreserved(int c) {
return isAlpha(c) || isDigit(c) || c == '-' || c == '.' || c == '_' || c == '~';
}
/** sub-delims = "!" / "$" / "&" / "'" / "(" / ")" / "*" / "+" / "," / ";" / "=" */
private static boolean isSubDelim(int c) {
return switch (c) {
case '!', '$', '&', '\'', '(', ')', '*', '+', ',', ';', '=' -> true;
default -> false;
};
}
/** pchar = unreserved / pct-encoded / sub-delims / ":" / "@" */
private static boolean isPchar(int c) {
return isUnreserved(c) || isSubDelim(c) || c == ':' || c == '@';
}
static boolean isAlpha(int c) {
return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z');
}
static boolean isDigit(int c) {
return c >= '0' && c <= '9';
}
@SuppressWarnings("BooleanMethodIsAlwaysInverted")
static boolean isHexDigit(int c) {
return isDigit(c) || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F');
}
/**
* Percent-encodes every character of {@code value} that is not {@code allowed}.
*
* <p>A {@code %} is passed through rather than encoded, so this leaves an already percent-encoded
* value alone. Callers are expected to have checked that every {@code %} begins a percent-encoded
* octet; see {@link UrlParser#percentEncodingFailure}.
*/
static String encode(String value, IntPredicate allowed) {
if (isEncoded(value, allowed)) {
return value;
}
var out = new StringBuilder(value.length());
value
.codePoints()
.forEach(
codePoint -> {
if (codePoint == '%' || allowed.test(codePoint)) {
out.appendCodePoint(codePoint);
} else {
encodeUtf8(codePoint, out);
}
});
return out.toString();
}
static boolean isEncoded(String value, IntPredicate allowed) {
return value.codePoints().allMatch(c -> c == '%' || allowed.test(c));
}
/**
* Returns the percent-encoded {@code value} in the form section 6.2.2 compares it in.
*
* <p>This makes a component that was written in a different but equivalent way compare equal.
*/
static String normalize(String value) {
if (value.indexOf('%') < 0) {
return value;
}
var out = new StringBuilder(value.length());
var i = 0;
while (i < value.length()) {
var octet = octetAt(value, i);
if (octet < 0) {
out.append(value.charAt(i));
i++;
} else {
if (isUnreserved(octet)) {
out.append((char) octet);
} else {
out.append('%').append(toUpperHexDigit(octet >> 4)).append(toUpperHexDigit(octet & 0xF));
}
i += 3;
}
}
return out.toString();
}
private static int octetAt(String value, int index) {
if (value.charAt(index) != '%'
|| index + 2 >= value.length()
|| !isHexDigit(value.charAt(index + 1))
|| !isHexDigit(value.charAt(index + 2))) {
return -1;
}
return (Character.digit(value.charAt(index + 1), 16) << 4)
| Character.digit(value.charAt(index + 2), 16);
}
/** UTF-8 percent-encodes {@code codePoint} and appends the result to {@code out}. */
static void encodeUtf8(int codePoint, StringBuilder out) {
var bytes = new String(Character.toChars(codePoint)).getBytes(StandardCharsets.UTF_8);
for (var b : bytes) {
out.append('%');
out.append(toUpperHexDigit((b >> 4) & 0xF));
out.append(toUpperHexDigit(b & 0xF));
}
}
/**
* Appends {@code value} to {@code out}, percent-encoding everything that is not {@link
* #UNRESERVED}, {@code %} included.
*
* <p>Unlike {@link #encode}, this treats its input as entirely undecoded, which is what a value
* that is about to be interpolated into a URL component is.
*/
public static void encodeComponent(StringBuilder out, String value) {
value
.codePoints()
.forEach(
codePoint -> {
if (isUnreserved(codePoint)) {
out.appendCodePoint(codePoint);
} else {
encodeUtf8(codePoint, out);
}
});
}
private static char toUpperHexDigit(int value) {
return Character.toUpperCase(Character.forDigit(value, 16));
}
/**
* Percent-decodes {@code input} and interprets the decoded bytes as UTF-8.
*
* <p>Callers are expected to have checked that every {@code %} begins a percent-encoded octet;
* see {@link UrlParser#percentEncodingFailure}. One that does not is kept as-is.
*/
@TruffleBoundary
public static String decode(String input) {
if (input.indexOf('%') < 0) {
return input;
}
var in = input.getBytes(StandardCharsets.UTF_8);
var bytes = new ByteArrayOutputStream(in.length);
for (var i = 0; i < in.length; i++) {
if (in[i] != '%' || i + 2 >= in.length || !isHexDigit(in[i + 1]) || !isHexDigit(in[i + 2])) {
bytes.write(in[i]);
} else {
bytes.write((Character.digit(in[i + 1], 16) << 4) | Character.digit(in[i + 2], 16));
i += 2;
}
}
return bytes.toString(StandardCharsets.UTF_8);
}
/**
* Percent-decodes {@code input} as an {@code application/x-www-form-urlencoded}.
*
* <p>Callers are expected to have checked that every {@code %} begins a percent-encoded octet;
* see {@link UrlParser#percentEncodingFailure}. {@link URLDecoder} throws on one that does not.
*/
@TruffleBoundary
static String decodeForm(String input) {
return URLDecoder.decode(input, StandardCharsets.UTF_8);
}
/**
* Appends {@code value} to {@code out} as an {@code application/x-www-form-urlencoded}.
*
* <p>The inverse of {@link #decodeForm}.
*/
@TruffleBoundary
public static void encodeForm(StringBuilder out, String value) {
out.append(URLEncoder.encode(value, StandardCharsets.UTF_8));
}
}
@@ -0,0 +1,139 @@
/*
* Copyright © 2026 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.pkl.core.util.url;
import com.oracle.truffle.api.CompilerDirectives.TruffleBoundary;
import com.oracle.truffle.api.nodes.IndirectCallNode;
import com.oracle.truffle.api.nodes.Node;
import org.jspecify.annotations.Nullable;
import org.pkl.core.runtime.Identifier;
import org.pkl.core.runtime.NetModule;
import org.pkl.core.runtime.VmExceptionBuilder;
import org.pkl.core.runtime.VmMapping;
import org.pkl.core.runtime.VmNull;
import org.pkl.core.runtime.VmObjectBuilder;
import org.pkl.core.runtime.VmObjectLike;
import org.pkl.core.runtime.VmTyped;
import org.pkl.core.runtime.VmUtils;
import org.pkl.core.stdlib.VmObjectFactory;
import org.pkl.core.util.url.UrlParser.Parsed;
import org.pkl.core.util.url.UrlParser.Result;
/** Converts between a parsed URL and {@code pkl:net}'s {@code Url}. */
public final class UrlFactory {
private UrlFactory() {}
private static final VmObjectFactory<Parsed> factory =
new VmObjectFactory<Parsed>(NetModule::getUrlClass)
.addProperty("scheme", parsed -> VmNull.lift(parsed.scheme()))
.addProperty("rawUserInfo", parsed -> VmNull.lift(parsed.userInfo()))
.addProperty("rawHost", parsed -> VmNull.lift(parsed.host()))
.addProperty("rawPort", parsed -> VmNull.lift(parsed.port()))
.addStringProperty("rawPath", Parsed::path)
.addProperty("rawQuery", parsed -> VmNull.lift(parsed.query()))
.addProperty("rawFragment", parsed -> VmNull.lift(parsed.fragment()));
public static VmTyped create(Parsed parsed) {
return factory.create(parsed);
}
/** Parses {@code input} as a URI reference. */
@TruffleBoundary
public static Parsed parseOrThrow(String input, Node node) {
var result = UrlParser.parse(input);
if (result instanceof Result.Failure failure) {
throw new VmExceptionBuilder()
.withLocation(node)
.evalError("cannotParseUrl", input)
.withHint(failure.hint())
.build();
}
return ((Result.Success) result).url();
}
@TruffleBoundary
public static void checkPercentEncoding(String value, String messageKey, Node node) {
var failure = UrlParser.percentEncodingFailure(value);
if (failure != null) {
throw new VmExceptionBuilder()
.withLocation(node)
.evalError(messageKey, value)
.withHint(failure.hint())
.build();
}
}
/** Reads the components back off {@code url}. */
public static Parsed read(VmTyped url, IndirectCallNode callNode) {
if (url.hasExtraStorage()) {
return (Parsed) url.getExtraStorage();
}
var parsed =
new Parsed(
readNullableString(url, Identifier.SCHEME, callNode),
readNullableString(url, Identifier.RAW_USER_INFO, callNode),
readNullableString(url, Identifier.RAW_HOST, callNode),
readNullableString(url, Identifier.RAW_PORT, callNode),
(String) VmUtils.readMember(url, Identifier.RAW_PATH, callNode),
readNullableString(url, Identifier.RAW_QUERY, callNode),
readNullableString(url, Identifier.RAW_FRAGMENT, callNode));
url.setExtraStorage(parsed);
return parsed;
}
private static @Nullable String readNullableString(
VmObjectLike url, Identifier name, IndirectCallNode callNode) {
return (String) VmNull.unwrap(VmUtils.readMember(url, name, callNode));
}
public static String readPath(VmObjectLike url, IndirectCallNode callNode) {
return (String) VmUtils.readMember(url, Identifier.RAW_PATH, callNode);
}
public static @Nullable String readHost(VmObjectLike url, IndirectCallNode callNode) {
return readNullableString(url, Identifier.RAW_HOST, callNode);
}
public static @Nullable String readQuery(VmObjectLike url, IndirectCallNode callNode) {
return readNullableString(url, Identifier.RAW_QUERY, callNode);
}
/** The parameters of {@code query}, as {@code Mapping<String, Listing<String>>}. */
@TruffleBoundary
public static VmMapping createQueryParameters(String query) {
var builder = new VmObjectBuilder();
for (var parameter : UrlParser.queryParameters(query).entrySet()) {
var values = new VmObjectBuilder(parameter.getValue().size());
for (var value : parameter.getValue()) {
values.addElement(value);
}
builder.addEntry(parameter.getKey(), values.toListing());
}
return builder.toMapping();
}
/** Reads the authority off {@code url}, or {@code null} if it has none. */
public static @Nullable String readAuthority(VmObjectLike url, IndirectCallNode callNode) {
var host = readNullableString(url, Identifier.RAW_HOST, callNode);
if (host == null) {
return null;
}
return UrlParser.serializeAuthority(
readNullableString(url, Identifier.RAW_USER_INFO, callNode),
host,
readNullableString(url, Identifier.RAW_PORT, callNode));
}
}
@@ -0,0 +1,849 @@
/*
* Copyright © 2026 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.pkl.core.util.url;
import com.oracle.truffle.api.CompilerDirectives.TruffleBoundary;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.function.IntPredicate;
import org.jspecify.annotations.Nullable;
/**
* A parser for <a href="https://www.rfc-editor.org/rfc/rfc3986">RFC 3986</a> URI references.
*
* <p>Both shapes of {@code URI-reference} (section 4.1) are accepted: a {@code URI}, which states a
* scheme, and a {@code relative-ref}, which does not.
*
* <p>Reference resolution is not part of parsing; see {@link #resolve}.
*
* <p>An IPv6 address may carry the zone identifier of <a
* href="https://www.rfc-editor.org/rfc/rfc6874">RFC 6874</a>, as in {@code [fe80::1%25eth0]}..
*/
@SuppressWarnings("JavadocLinkAsPlainText")
public final class UrlParser {
private UrlParser() {}
/**
* The components of a URI reference, each percent-encoded, so that it holds nothing that cannot
* appear literally in it.
*
* <p>{@code scheme} is {@code null} for a relative reference. {@code host} is {@code null} when
* the reference has no authority. {@code userInfo} and {@code port} are only ever set alongside a
* host.
*/
public record Parsed(
@Nullable String scheme,
@Nullable String userInfo,
@Nullable String host,
@Nullable String port,
String path,
@Nullable String query,
@Nullable String fragment) {
/** Serializes these components (section 5.3). */
@TruffleBoundary
public String serialize() {
var sb = new StringBuilder();
if (scheme != null) {
sb.append(scheme).append(':');
}
if (host != null) {
sb.append("//").append(serializeAuthority(userInfo, host, port));
}
sb.append(path);
if (query != null) {
sb.append('?').append(query);
}
if (fragment != null) {
sb.append('#').append(fragment);
}
return sb.toString();
}
public @Nullable String authority() {
return host == null ? null : serializeAuthority(userInfo, host, port);
}
}
/** Serializes an authority (section 3.2) from its percent-encoded components. */
@TruffleBoundary
static String serializeAuthority(@Nullable String userInfo, String host, @Nullable String port) {
var sb = new StringBuilder();
if (userInfo != null) {
sb.append(userInfo).append('@');
}
sb.append(host);
if (port != null) {
sb.append(':').append(port);
}
return sb.toString();
}
// Parsing (https://www.rfc-editor.org/rfc/rfc3986#section-3)
/** The outcome of parsing. */
sealed interface Result {
record Success(Parsed url) implements Result {}
record Failure(String hint) implements Result {}
}
private static @Nullable String encodeOptional(@Nullable String component, IntPredicate allowed) {
return component == null ? null : PercentEncoder.encode(component, allowed);
}
/**
* Parses {@code input} as a URI reference, percent-encoding whatever its components cannot hold
* literally.
*/
static Result parse(String input) {
return parseAsWritten(input, true);
}
private static Result parseAsWritten(String input, boolean encode) {
var length = input.length();
// scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." ) ":"
// Absent a scheme, this is a relative reference and any ":" belongs to the path.
String scheme = null;
var pointer = 0;
if (length > 0 && PercentEncoder.isAlpha(input.charAt(0))) {
var end = 1;
while (end < length && isSchemeChar(input.charAt(end))) {
end++;
}
if (end < length && input.charAt(end) == ':') {
scheme = input.substring(0, end);
pointer = end + 1;
}
}
// authority = [ userinfo "@" ] host [ ":" port ]
String userInfo = null;
String host = null;
String port = null;
if (input.startsWith("//", pointer)) {
pointer += 2;
var end = pointer;
while (end < length && !isAuthorityTerminator(input.charAt(end))) {
end++;
}
var authority = input.substring(pointer, end);
pointer = end;
// the userinfo runs to the last "@", because "@" cannot appear literally in a host
var at = authority.lastIndexOf('@');
if (at >= 0) {
userInfo = authority.substring(0, at);
var failure = percentEncodingFailure(userInfo);
if (failure != null) {
return failure;
}
}
host = authority.substring(at + 1);
var colon = portSeparator(host);
if (colon >= 0) {
port = host.substring(colon + 1);
host = host.substring(0, colon);
if (!isPort(port)) {
return new Result.Failure("The port `" + port + "` can only hold digits.");
}
}
var failure = hostFailure(host);
if (failure != null) {
return failure;
}
}
var pathEnd = pointer;
while (pathEnd < length && input.charAt(pathEnd) != '?' && input.charAt(pathEnd) != '#') {
pathEnd++;
}
var path = input.substring(pointer, pathEnd);
var pathFailure = pathFailure(path, scheme != null, host != null);
if (pathFailure != null) {
return pathFailure;
}
pointer = pathEnd;
String query = null;
if (pointer < length && input.charAt(pointer) == '?') {
var end = input.indexOf('#', pointer + 1);
if (end < 0) {
end = length;
}
query = input.substring(pointer + 1, end);
var failure = percentEncodingFailure(query);
if (failure != null) {
return failure;
}
pointer = end;
}
String fragment = null;
if (pointer < length && input.charAt(pointer) == '#') {
fragment = input.substring(pointer + 1);
var failure = percentEncodingFailure(fragment);
if (failure != null) {
return failure;
}
}
if (encode) {
return new Result.Success(
new Parsed(
scheme,
encodeOptional(userInfo, PercentEncoder.USERINFO),
// an IP literal has already been validated, and none of its characters may be encoded
host == null || isIpLiteral(host)
? host
: PercentEncoder.encode(host, PercentEncoder.REG_NAME),
port,
PercentEncoder.encode(path, PercentEncoder.PATH),
encodeOptional(query, PercentEncoder.QUERY_OR_FRAGMENT),
encodeOptional(fragment, PercentEncoder.QUERY_OR_FRAGMENT)));
}
return new Result.Success(new Parsed(scheme, userInfo, host, port, path, query, fragment));
}
// Reference resolution (https://www.rfc-editor.org/rfc/rfc3986#section-5.2.2)
/**
* Resolves {@code ref} against {@code base}.
*
* <p>This is the strict form of the transform, which never reinterprets a reference that repeats
* the base's scheme as a relative one.
*
* <p>{@code base} is expected to state a scheme, as section 5.1 requires. Resolving against one
* that does not is not defined, and would only produce another relative reference.
*/
public static Parsed resolve(Parsed base, Parsed ref) {
if (ref.scheme() != null) {
return new Parsed(
ref.scheme(),
ref.userInfo(),
ref.host(),
ref.port(),
removeDotSegments(ref.path(), ref.host() != null),
ref.query(),
ref.fragment());
}
if (ref.host() != null) {
return new Parsed(
base.scheme(),
ref.userInfo(),
ref.host(),
ref.port(),
removeDotSegments(ref.path()),
ref.query(),
ref.fragment());
}
String path;
String query;
if (ref.path().isEmpty()) {
path = base.path();
query = ref.query() != null ? ref.query() : base.query();
} else {
path =
removeDotSegments(
ref.path().charAt(0) == '/' ? ref.path() : merge(base, ref.path()),
base.host() != null);
query = ref.query();
}
return new Parsed(
base.scheme(), base.userInfo(), base.host(), base.port(), path, query, ref.fragment());
}
/** https://www.rfc-editor.org/rfc/rfc3986#section-5.2.3 */
private static String merge(Parsed base, String path) {
if (base.host() != null && base.path().isEmpty()) {
return "/" + path;
}
var lastSlash = base.path().lastIndexOf('/');
return lastSlash < 0 ? path : base.path().substring(0, lastSlash + 1) + path;
}
private static String removeDotSegments(String path, boolean hasAuthority) {
var result = removeDotSegments(path);
return !hasAuthority && result.startsWith("//") ? "/." + result : result;
}
/** https://www.rfc-editor.org/rfc/rfc3986#section-5.2.4 */
static String removeDotSegments(String path) {
var out = new StringBuilder(path.length());
var length = path.length();
var pointer = 0;
while (pointer < length) {
if (path.startsWith("../", pointer)) {
pointer += 3;
} else if (path.startsWith("./", pointer)) {
pointer += 2;
} else if (path.startsWith("/./", pointer)) {
pointer += 2;
} else if (pointer + 2 == length && path.startsWith("/.", pointer)) {
out.append('/');
pointer = length;
} else if (path.startsWith("/../", pointer)) {
removeLastSegment(out);
pointer += 3;
} else if (pointer + 3 == length && path.startsWith("/..", pointer)) {
removeLastSegment(out);
out.append('/');
pointer = length;
} else if (pointer + 1 == length && path.charAt(pointer) == '.') {
pointer = length;
} else if (pointer + 2 == length && path.startsWith("..", pointer)) {
pointer = length;
} else {
// move the first segment, along with any leading "/", to the output
var end = path.indexOf('/', path.charAt(pointer) == '/' ? pointer + 1 : pointer);
if (end < 0) {
end = length;
}
out.append(path, pointer, end);
pointer = end;
}
}
return out.toString();
}
private static void removeLastSegment(StringBuilder out) {
var lastSlash = out.lastIndexOf("/");
out.setLength(Math.max(lastSlash, 0));
}
// Derived views
/** The percent-decoded segments of {@code path}. */
@TruffleBoundary
public static List<String> segments(String path) {
if (path.isEmpty()) {
return List.of();
}
var segments = new ArrayList<String>();
var start = path.charAt(0) == '/' ? 1 : 0;
while (true) {
var slash = path.indexOf('/', start);
if (slash < 0) {
segments.add(PercentEncoder.decode(path.substring(start)));
return segments;
}
segments.add(PercentEncoder.decode(path.substring(start, slash)));
start = slash + 1;
}
}
/**
* The percent-decoded parameters of {@code query}, which is read as {@code
* application/x-www-form-urlencoded}.
*
* <p>A name that repeats collects the values of all its occurrences, in the order they are
* written. A parameter that states no {@code =} holds no value, and contributes nothing.
*/
static Map<String, List<String>> queryParameters(String query) {
if (query.isEmpty()) {
return Map.of();
}
var parameters = new LinkedHashMap<String, List<String>>();
var start = 0;
while (start < query.length()) {
var end = query.indexOf('&', start);
if (end < 0) {
end = query.length();
}
var separator = query.indexOf('=', start);
// an empty pair ("a=1&&b=2") and a bare name ("a=1&b") both hold no parameter
if (end > start && separator >= 0 && separator < end) {
var name = PercentEncoder.decodeForm(query.substring(start, separator));
var value = PercentEncoder.decodeForm(query.substring(separator + 1, end));
parameters.computeIfAbsent(name, ignored -> new ArrayList<>()).add(value);
}
start = end + 1;
}
return parameters;
}
/**
* Whether {@code left} and {@code right} identify the same resource
* (https://www.rfc-editor.org/rfc/rfc3986#section-6).
*
* <p>Both are put through the syntax-based normalization of section 6.2.2 first, so that two ways
* of writing the same URL compare equal. A default port is kept.
*/
@TruffleBoundary
public static boolean isEquivalent(Parsed left, Parsed right) {
return normalize(left).equals(normalize(right));
}
/**
* The syntax-based normalization of {@code url}
* (https://www.rfc-editor.org/rfc/rfc3986#section-6.2.2).
*
* <p>Nothing that would take knowing the scheme is normalized, so a default port is kept.
*/
@TruffleBoundary
public static Parsed normalize(Parsed url) {
return new Parsed(
url.scheme() == null ? null : toLowerAscii(url.scheme()),
normalizeOptional(url.userInfo()),
url.host() == null ? null : normalizeHost(url.host()),
url.port(),
normalizePath(url),
normalizeOptional(url.query()),
normalizeOptional(url.fragment()));
}
private static @Nullable String normalizeOptional(@Nullable String component) {
return component == null ? null : PercentEncoder.normalize(component);
}
private static String normalizeHost(String host) {
if (!isIpLiteral(host)) {
// the hex digits of a percent-encoded octet are uppercase even in a component that is
// otherwise folded to lower case (section 6.2.2.1)
return toLowerAsciiOutsideOctets(PercentEncoder.normalize(host));
}
// an IP literal holds nothing that may be encoded, and its zone identifier, unlike the address
// in front of it, names an interface and is case-sensitive
var zone = host.indexOf("%25");
return zone < 0
? toLowerAscii(host)
: toLowerAscii(host.substring(0, zone)) + host.substring(zone);
}
private static String normalizePath(Parsed url) {
var path = PercentEncoder.normalize(url.path());
if (path.isEmpty()) {
// a URL with an authority and no path names the same resource as one whose path is "/"
// (section 6.2.3)
return url.host() == null ? path : "/";
}
// dot segments are only removable from an absolute path
return path.charAt(0) == '/' ? removeDotSegments(path, url.host() != null) : path;
}
static Result.@Nullable Failure percentEncodingFailure(String input) {
for (var i = input.indexOf('%'); i >= 0; i = input.indexOf('%', i + 3)) {
if (i + 2 >= input.length()
|| !PercentEncoder.isHexDigit(input.charAt(i + 1))
|| !PercentEncoder.isHexDigit(input.charAt(i + 2))) {
var octet = input.substring(i, Math.min(i + 3, input.length()));
return new Result.Failure("`" + octet + "` is not a percent-encoded octet.");
}
}
return null;
}
/**
* Whether {@code input} is already percent-encoded for a component that holds {@code allowed}
* literally: it holds nothing else, and every {@code %} begins a percent-encoded octet.
*/
private static boolean isEncodedComponent(String input, IntPredicate allowed) {
return PercentEncoder.isEncoded(input, allowed) && percentEncodingFailure(input) == null;
}
// The checks below are stricter than parsing, which encodes whatever a component cannot hold
// literally: each of them only accepts a component that is already encoded.
/** Whether {@code input} is a percent-encoded {@code userinfo}. */
public static boolean isValidUserInfo(String input) {
return isEncodedComponent(input, PercentEncoder.USERINFO);
}
/** Whether {@code input} is a percent-encoded {@code query} or {@code fragment}. */
public static boolean isValidQueryOrFragment(String input) {
return isEncodedComponent(input, PercentEncoder.QUERY_OR_FRAGMENT);
}
/**
* Whether {@code input} is an absolute URL that is already written the way RFC 3986 spells one
* out. Backs {@code String.isValidUrl}.
*
* <p>This is stricter than {@link #parse}, which is lenient in the two ways text taken from the
* outside world usually needs it to be:
*
* <ul>
* <li>a relative reference, such as {@code ./foo}, parses but is not a URL
* <li>nothing is encoded here, so a character that its component cannot hold literally has to
* already be percent-encoded: {@code http://example.com/some path} is not a URL, but {@code
* http://example.com/some%20path} is
* </ul>
*/
public static boolean isValidUrl(String input) {
if (!(parseAsWritten(input, false) instanceof Result.Success success)) {
return false;
}
var parsed = success.url();
return parsed.scheme() != null
&& (parsed.userInfo() == null || isValidUserInfo(parsed.userInfo()))
&& (parsed.host() == null || isValidHost(parsed.host()))
&& isValidPath(parsed.path(), true, parsed.host() != null)
&& (parsed.query() == null || isValidQueryOrFragment(parsed.query()))
&& (parsed.fragment() == null || isValidQueryOrFragment(parsed.fragment()));
}
/** Whether {@code input} is a scheme. Unlike parsing, the trailing {@code :} is not accepted. */
public static boolean isValidScheme(String input) {
if (input.isEmpty() || !PercentEncoder.isAlpha(input.charAt(0))) {
return false;
}
for (var i = 1; i < input.length(); i++) {
if (!isSchemeChar(input.charAt(i))) {
return false;
}
}
return true;
}
static Result.@Nullable Failure hostFailure(String host) {
if (isIpLiteral(host)) {
if (host.length() < 3) {
return new Result.Failure("The IP literal `" + host + "` is empty.");
}
var address = host.substring(1, host.length() - 1);
// "v" cannot begin an IPv6address, as it is not a hex digit, so it tells the two forms apart
if (address.charAt(0) == 'v' || address.charAt(0) == 'V') {
return isIpvFuture(address)
? null
: new Result.Failure("`" + address + "` is not an IPvFuture address.");
}
// IPv6addrz = IPv6address "%25" ZoneID (https://www.rfc-editor.org/rfc/rfc6874#section-2)
var zone = address.indexOf("%25");
if (zone < 0) {
if (isIpv6Address(address)) {
return null;
}
return address.indexOf('%') < 0
? new Result.Failure("`" + address + "` is not an IPv6 address.")
: new Result.Failure(
"A zone identifier must be preceded by `%25`, as in `[fe80::1%25eth0]`.");
}
if (zone == 0) {
return new Result.Failure(
"The IP literal `" + host + "` states a zone identifier but no address.");
}
var address6 = address.substring(0, zone);
if (!isIpv6Address(address6)) {
return new Result.Failure("`" + address6 + "` is not an IPv6 address.");
}
var zoneId = address.substring(zone + 3);
if (zoneId.isEmpty()) {
return new Result.Failure("The zone identifier of `" + host + "` is empty.");
}
return isZoneId(zoneId)
? null
: new Result.Failure(
"The zone identifier `"
+ zoneId
+ "` may only hold unreserved characters and percent-encoded octets.");
}
if (host.startsWith("[")) {
return host.indexOf(']') < 0
? new Result.Failure("The IP literal `" + host + "` is missing a closing `]`.")
: new Result.Failure(
"The IP literal `" + host + "` has trailing characters after its `]`.");
}
if (host.indexOf('[') >= 0 || host.indexOf(']') >= 0) {
return new Result.Failure(
"A registered name cannot contain `[` or `]`; percent-encode them, or enclose an IP"
+ " address in brackets.");
}
if (host.indexOf(':') >= 0) {
return new Result.Failure(
"A registered name cannot contain `:`; percent-encode it, or enclose an IPv6 address in brackets.");
}
return percentEncodingFailure(host);
}
/**
* What {@code host} is: {@code "ipv6"} or {@code "ipvFuture"} for an IP literal, {@code "ipv4"}
* for an {@code IPv4address}, and {@code "name"} for any other, registered, name (section 3.2.2).
*/
public static String hostKind(String host) {
if (isIpLiteral(host)) {
var c = host.charAt(1);
return c == 'v' || c == 'V' ? "ipvFuture" : "ipv6";
}
return isIpv4Address(host) ? "ipv4" : "name";
}
/**
* The percent-decoded zone identifier of {@code host}, or {@code null} if it is not an IPv6
* address that states one.
*/
@TruffleBoundary
public static @Nullable String zoneId(String host) {
if (!isIpLiteral(host)) {
return null;
}
// "%" cannot appear in an IPvFuture, so it can only begin a zone identifier
var zone = host.indexOf("%25");
return zone < 0 ? null : PercentEncoder.decode(host.substring(zone + 3, host.length() - 1));
}
/** Whether {@code host} is a percent-encoded {@code host}. */
public static boolean isValidHost(String host) {
return hostFailure(host) == null
&& (isIpLiteral(host) || PercentEncoder.isEncoded(host, PercentEncoder.REG_NAME));
}
/**
* Whether {@code address} is an {@code IPv6address}: eight 16-bit pieces, the last two of which
* may be written as an {@code IPv4address}, and a single {@code "::"} that stands for one or more
* pieces of zeros (section 3.2.2).
*/
private static boolean isIpv6Address(String address) {
var elision = address.indexOf("::");
if (elision < 0) {
return countIpv6Pieces(address) == 8;
}
if (address.indexOf("::", elision + 1) >= 0) {
return false;
}
var head = address.substring(0, elision);
var tail = address.substring(elision + 2);
// an IPv4address can only end the address, so it cannot sit in front of the "::"
var headPieces = head.isEmpty() ? 0 : head.indexOf('.') >= 0 ? -1 : countIpv6Pieces(head);
var tailPieces = tail.isEmpty() ? 0 : countIpv6Pieces(tail);
return headPieces >= 0 && tailPieces >= 0 && headPieces + tailPieces <= 7;
}
private static int countIpv6Pieces(String input) {
var pieces = 0;
var start = 0;
while (true) {
var colon = input.indexOf(':', start);
if (colon < 0) {
if (input.indexOf('.', start) >= 0) {
return isIpv4Address(input.substring(start)) ? pieces + 2 : -1;
}
return isH16(input, start, input.length()) ? pieces + 1 : -1;
}
if (!isH16(input, start, colon)) {
return -1;
}
pieces++;
start = colon + 1;
}
}
private static boolean isH16(String input, int start, int end) {
if (end <= start || end - start > 4) {
return false;
}
for (var i = start; i < end; i++) {
if (!PercentEncoder.isHexDigit(input.charAt(i))) {
return false;
}
}
return true;
}
/** IPv4address = dec-octet "." dec-octet "." dec-octet "." dec-octet */
@TruffleBoundary
private static boolean isIpv4Address(String input) {
var octets = 0;
var start = 0;
while (true) {
var dot = input.indexOf('.', start);
var end = dot < 0 ? input.length() : dot;
if (!isDecOctet(input, start, end)) {
return false;
}
octets++;
if (dot < 0) {
return octets == 4;
}
start = dot + 1;
}
}
/** A decimal number between 0 and 255, with no leading zeros. */
private static boolean isDecOctet(String input, int start, int end) {
var length = end - start;
if (length < 1 || length > 3 || (length > 1 && input.charAt(start) == '0')) {
return false;
}
var value = 0;
for (var i = start; i < end; i++) {
var c = input.charAt(i);
if (!PercentEncoder.isDigit(c)) {
return false;
}
value = value * 10 + (c - '0');
}
return value <= 255;
}
/** Whether {@code zoneId} is a {@code ZoneID}: {@code 1*( unreserved / pct-encoded )}. */
private static boolean isZoneId(String zoneId) {
if (zoneId.isEmpty()) {
return false;
}
for (var i = 0; i < zoneId.length(); i++) {
var c = zoneId.charAt(i);
if (c == '%') {
if (i + 2 >= zoneId.length()
|| !PercentEncoder.isHexDigit(zoneId.charAt(i + 1))
|| !PercentEncoder.isHexDigit(zoneId.charAt(i + 2))) {
return false;
}
i += 2;
} else if (!PercentEncoder.UNRESERVED.test(c)) {
return false;
}
}
return true;
}
/** IPvFuture = "v" 1*HEXDIG "." 1*( unreserved / sub-delims / ":" ) */
private static boolean isIpvFuture(String address) {
var dot = address.indexOf('.');
if (dot < 2 || dot == address.length() - 1) {
return false;
}
for (var i = 1; i < dot; i++) {
if (!PercentEncoder.isHexDigit(address.charAt(i))) {
return false;
}
}
for (var i = dot + 1; i < address.length(); i++) {
// the tail of an IPvFuture is drawn from the same set as a userinfo
if (!PercentEncoder.USERINFO.test(address.charAt(i))) {
return false;
}
}
return true;
}
static Result.@Nullable Failure pathFailure(
String path, boolean hasScheme, boolean hasAuthority) {
var failure = percentEncodingFailure(path);
if (failure != null) {
return failure;
}
if (hasAuthority) {
// path-abempty
return path.isEmpty() || path.charAt(0) == '/'
? null
: new Result.Failure("A path that follows an authority must start with `/`.");
}
if (path.startsWith("//")) {
return new Result.Failure("A path that follows no authority cannot start with `//`.");
}
if (!hasScheme && startsWithColonSegment(path)) {
// path-noscheme (section 4.2)
return new Result.Failure(
"The first segment of a relative path cannot contain `:`; precede it with `./`, as in"
+ " `./a:b`.");
}
// path-absolute / path-rootless / path-empty
return null;
}
/**
* Whether {@code path} is percent-encoded, and can sit next to a scheme and an authority, or next
* to no scheme and no authority, as {@code hasScheme} and {@code hasAuthority} say.
*/
public static boolean isValidPath(String path, boolean hasScheme, boolean hasAuthority) {
return pathFailure(path, hasScheme, hasAuthority) == null
&& PercentEncoder.isEncoded(path, PercentEncoder.PATH);
}
private static boolean isIpLiteral(String host) {
return host.startsWith("[") && host.endsWith("]");
}
/** Whether the first segment of {@code path} holds a {@code ":"}. */
private static boolean startsWithColonSegment(String path) {
var colon = path.indexOf(':');
if (colon < 0) {
return false;
}
var slash = path.indexOf('/');
return slash < 0 || colon < slash;
}
/**
* The index of the {@code ":"} that separates the host from the port in {@code hostAndPort}, or
* {@code -1} if it states no port.
*/
private static int portSeparator(String hostAndPort) {
if (hostAndPort.startsWith("[")) {
var close = hostAndPort.indexOf(']');
if (close < 0) {
// an unterminated IP literal; let host validation reject it
return -1;
}
var next = close + 1;
return next < hostAndPort.length() && hostAndPort.charAt(next) == ':' ? next : -1;
}
return hostAndPort.lastIndexOf(':');
}
/**
* Whether {@code input} is a {@code port}: {@code *DIGIT}, so it may be empty, and its leading
* zeros are part of it.
*/
private static boolean isPort(String input) {
for (var i = 0; i < input.length(); i++) {
if (!PercentEncoder.isDigit(input.charAt(i))) {
return false;
}
}
return true;
}
private static boolean isSchemeChar(int c) {
return PercentEncoder.isAlpha(c)
|| PercentEncoder.isDigit(c)
|| c == '+'
|| c == '-'
|| c == '.';
}
/** The gen-delims that end an authority and start the component after it. */
private static boolean isAuthorityTerminator(int c) {
return c == '/' || c == '?' || c == '#';
}
private static String toLowerAscii(String input) {
var out = new StringBuilder(input.length());
for (var i = 0; i < input.length(); i++) {
var c = input.charAt(i);
out.append((char) ((c >= 'A' && c <= 'Z') ? c + 0x20 : c));
}
return out.toString();
}
/** Lowercases {@code input}, leaving the hex digits of its percent-encoded octets as written. */
private static String toLowerAsciiOutsideOctets(String input) {
var out = new StringBuilder(input.length());
for (var i = 0; i < input.length(); i++) {
var c = input.charAt(i);
if (c == '%') {
out.append(input, i, i + 3);
i += 2;
} else {
out.append((char) ((c >= 'A' && c <= 'Z') ? c + 0x20 : c));
}
}
return out.toString();
}
}
@@ -809,6 +809,20 @@ Error parsing YAML document: The number of aliases for collection nodes exceeds
\n\
To increase the allowed maximum, set `YamlRenderer.maxCollectionAliases`.
cannotParseUrl=\
Cannot parse `{0}` as a URL.
cannotDecodeUrlComponent=\
Cannot percent-decode `{0}`.
cannotParseQuery=\
Cannot parse `{0}` as a query.
cannotResolveAgainstRelativeUrl=\
Cannot resolve a reference against `{0}`, which is a relative URL.\n\
\n\
Only a URL that states a scheme can be resolved against.
evaluationTimedOut=\
Evaluation timed out after {0,number,#.##} second{0,choice,0<s|1#|1.0<s}.
@@ -36,6 +36,30 @@ facts {
!"hello there".isBase64
}
["isValidUrl"] {
"mailto:user@example.com".isValidUrl
"urn:isbn:0451450523".isValidUrl
"file:///a/b".isValidUrl
"foo:".isValidUrl
"http://[fe80::1%25eth0]:8080/a?q#f".isValidUrl
"http://user:pw@EXAMPLE.com/!$&'()*+,;=:@?!$&'()*+,;=:@/?#!$&'()*+,;=:@/?".isValidUrl
// only absolute URLs are valid
!"./foo".isValidUrl
!"/a/b?q#f".isValidUrl
!"//example.com/a".isValidUrl
!"".isValidUrl
!"not a url".isValidUrl
// nothing is encoded
!"http://example.com/some path".isValidUrl
!"https://example.com/münchen?q=café".isValidUrl
"https://example.com/m%C3%BCnchen?q=caf%C3%A9".isValidUrl
// whatever the parser of `pkl:net` rejects is rejected here too
!"https://example.com:port/".isValidUrl
}
["lastIndex"] {
for (s in List(str1, str2, str3)) {
s.length == s.lastIndex + 1
@@ -0,0 +1,521 @@
amends "../snippetTest.pkl"
import "pkl:net"
local rfcBase = "http://a/b/c/d;p?q"
local function componentsOf(input: String): List =
let (result = net.Url(input))
List(
result.scheme,
result.rawUserInfo,
result.rawHost,
result.rawPort,
result.rawPath,
result.rawQuery,
result.rawFragment,
)
local function parsed(input: String): String = net.Url(input).toString()
/// Whether [input] parses to exactly itself.
local function roundTrips(input: String): Boolean = parsed(input) == input
local function throws(input: String): Boolean =
module.catch(() -> net.Url(input)) == "Cannot parse `\(input)` as a URL."
local function resolved(ref: String): String = net.Url(rfcBase).resolve(ref).toString()
local gClef = "\u{1D11E}"
facts {
["components"] {
componentsOf("https://user:pw@example.com:8080/a/b?q=1#f")
== List("https", "user:pw", "example.com", "8080", "/a/b", "q=1", "f")
componentsOf("https://example.com")
== List("https", null, "example.com", null, "", null, null)
componentsOf("https://example.com/a?#")
== List("https", null, "example.com", null, "/a", "", "")
componentsOf("mailto:user@example.com")
== List("mailto", null, null, null, "user@example.com", null, null)
// a null host is not an empty one
componentsOf("file:/tmp") == List("file", null, null, null, "/tmp", null, null)
componentsOf("file:///tmp") == List("file", null, "", null, "/tmp", null, null)
// an empty port is still one
componentsOf("http://example.com:/a") == List("http", null, "example.com", "", "/a", null, null)
// the userinfo runs to the last "@", so the extra one lands in it and is encoded
componentsOf("http://a@b@example.com/") == List("http", "a%40b", "example.com", null, "/", null, null)
}
["parse(): keeps what is already encoded as it was written"] {
net.Url("HTTPS://example.com").scheme == "HTTPS"
net.Url("http://EXAMPLE.com/A").rawHost == "EXAMPLE.com"
roundTrips("http://EXAMPLE.com/A")
net.Url("https://example.com:443/a").port == 443
roundTrips("https://example.com:443/a")
// a port is held as written, leading zeros and all
net.Url("x://foo:001").rawPort == "001"
net.Url("x://foo:001").port == 1
net.Url("http://example.com/a/b/../c/./d").rawPath == "/a/b/../c/./d"
roundTrips("http://example.com/a/b/../c/./d")
net.Url("http://example.com/%41%2Fb").rawPath == "/%41%2Fb"
roundTrips("http://example.com/%41%2fb")
roundTrips("http://0x7f.0.0.1/")
roundTrips("http://1.2.3.4./")
roundTrips("http://[2001:db8:0:0:0:0:0:1]/")
}
["parse(): round-trips valid URIs"] {
roundTrips("https://example.com/a?q=1#f")
roundTrips("http://1.2.3.4/")
roundTrips("http://[2001:db8::1]/")
roundTrips("http://[v7.future]/")
roundTrips("file:///a/b")
roundTrips("foo://example.com/a")
roundTrips("mailto:user@example.com")
roundTrips("urn:isbn:0451450523")
roundTrips("https://example.com/a%20b?q=%C3%A9#f%2F")
roundTrips("https://example.com")
roundTrips("https://example.com/a?#")
roundTrips("http://a/b/c/d;p?q")
roundTrips("foo:")
// every sub-delim, ":" and "@" may appear literally in a path, query and fragment
roundTrips("http://example.com/!$&'()*+,;=:@?!$&'()*+,;=:@/?#!$&'()*+,;=:@/?")
}
["parse(): encodes what it cannot keep"] {
net.Url("http://example.com/a b").rawPath == "/a%20b"
net.Url("http://example.com/a%20b").rawPath == "/a%20b"
parsed("http://example.com/a b") == "http://example.com/a%20b"
parsed("https://example.com/a<b>c") == "https://example.com/a%3Cb%3Ec"
parsed("https://example.com/?a b") == "https://example.com/?a%20b"
parsed("https://example.com/#a b") == "https://example.com/#a%20b"
parsed("https://example.com/münchen?q=café") == "https://example.com/m%C3%BCnchen?q=caf%C3%A9"
parsed("https://example.com/a" + gClef) == "https://example.com/a%F0%9D%84%9E"
parsed("http://münchen.de/") == "http://m%C3%BCnchen.de/"
parsed("http://exa mple.com/") == "http://exa%20mple.com/"
parsed("http://a@b@example.com/") == "http://a%40b@example.com/"
parsed("http://u:p@example.com/") == "http://u:p@example.com/"
// a "#" after the one that starts the fragment is part of it
net.Url("https://example.com/#a#b").rawFragment == "a%23b"
// the components are encoded, not just their serialization
componentsOf("https://u v@exa mple.com/a b?c d#e f")
== List("https", "u%20v", "exa%20mple.com", null, "/a%20b", "c%20d", "e%20f")
// once encoded, the result round-trips
roundTrips("http://example.com/a%20b")
parsed(parsed("http://example.com/a b")) == "http://example.com/a%20b"
net.Url("http://example.com/a b") == net.Url("http://example.com/a%20b")
}
["decoded components"] {
local url = net.Url("https://u%20v@m%C3%BCnchen.de/a%20b/c?q=a%20b+c#f%20g")
url.userInfo == "u v"
url.host == "münchen.de"
url.path == "/a b/c"
// a query is not read as a form here, so a "+" is kept
url.query == "q=a b+c"
url.fragment == "f g"
// it makes no difference whether the input was encoded
net.Url("https://example.com/a b").path == "/a b"
net.Url("https://example.com/a%20b").path == "/a b"
// an absent component has no decoded form either
local bare = net.Url("mailto:user@example.com")
bare.userInfo == null
bare.host == null
bare.path == "user@example.com"
bare.query == null
bare.fragment == null
net.Url("https://example.com").path == ""
// an encoded "/" decodes to one, which only pathSegments tells apart
net.Url("https://example.com/a%2Fb").path == "/a/b"
net.Url("https://example.com/a%2Fb").pathSegments == List("a/b")
net.Url("http://[fe80::1%25eth0]/").host == "[fe80::1%eth0]"
// the decoded components follow an amendment
(url) { rawPath = "/x%20y" }.path == "/x y"
(url) { rawQuery = null }.query == null
}
["parse(): failing"] {
// a "%" that does not begin a percent-encoded octet
throws("http://example.com/a%zz")
throws("http://example.com/a%2")
throws("http://example.com/a%")
throws("http://example.com/?a%zz")
throws("http://example.com/#a%zz")
throws("http://u%zz@example.com/")
throws("http://exa%zzmple.com/")
// a ":" in an authority always separates a port, so what follows has to be one
throws("https://example.com:port/")
throws("https://example.com:-1/")
throws("https://example.com:+1/")
// and only the last one separates it, so any earlier ":" is left in a registered name, which
// cannot hold one; an IPv6 address has to be bracketed
throws("http://::/foo")
throws("http://:::/foo")
throws("http://::1/foo")
throws("http://a:b:8080/")
// a single ":" still separates a port, which may be empty
roundTrips("http://:/")
net.Url("http://:/").port == null
roundTrips("http://[::]/foo")
roundTrips("http://[::1]/foo")
// an encoded ":" is a registered name that holds one, and is left alone
roundTrips("http://%3A%3A/foo")
// a malformed IP literal
throws("http://[::1")
throws("http://[::1x]/")
throws("http://[]/")
throws("http://[::1]x/")
// an IPv6 address is held to its grammar, not just to the characters it is built from
roundTrips("http://[::ffff:192.0.2.1]/")
throws("http://[1]/")
throws("http://[1::2::3]/")
throws("http://[1:2:3:4:5:6:7:8:9]/")
throws("http://[::192.0.2.256]/")
// the same holds for a relative reference, which is checked no less than an absolute URL
throws("a/%zz")
throws("//example.com:port/a")
throws("//[::1x]/a")
}
["parse(): RFC 6874 zone identifiers"] {
roundTrips("http://[fe80::1%25eth0]/")
roundTrips("http://[fe80::1%25eth0]:8080/a?q#f")
net.Url("http://[fe80::1%25eth0]/").rawHost == "[fe80::1%25eth0]"
roundTrips("http://[fe80::1%25a-b._~1]/")
roundTrips("http://[fe80::1%25%41]/")
throws("http://[fe80::1%eth0]/")
throws("http://[fe80::1%65th0]/")
throws("http://[fe80::1%25]/")
throws("http://[%25eth0]/")
throws("http://[fe80::1%25eth 0]/")
throws("http://[fe80::1%25eth%zz]/")
throws("http://[fe80::1%25a:b]/")
throws("http://[v7.future%25eth0]/")
roundTrips("http://fe80--1%25eth0/")
}
["parse(): relative references"] {
componentsOf("a/foo") == List(null, null, null, null, "a/foo", null, null)
componentsOf("../c") == List(null, null, null, null, "../c", null, null)
componentsOf("/a/b?q#f") == List(null, null, null, null, "/a/b", "q", "f")
componentsOf("//example.com/a") == List(null, null, "example.com", null, "/a", null, null)
componentsOf("?q") == List(null, null, null, null, "", "q", null)
componentsOf("#f") == List(null, null, null, null, "", null, "f")
componentsOf("") == List(null, null, null, null, "", null, null)
roundTrips("a/foo")
roundTrips("../c")
roundTrips("/a/b?q#f")
roundTrips("//example.com/a")
roundTrips("//user@example.com:8080")
roundTrips("?q")
roundTrips("#f")
roundTrips("")
// almost any text is a well-formed relative reference, so it is the scheme, and not parsing
// succeeding, that tells a URL apart from arbitrary text
net.Url("not a url").scheme == null
parsed("not a url") == "not%20a%20url"
// a hand-built relative URL is a parsed one
new net.Url { rawPath = "a/foo" } == net.Url("a/foo")
new net.Url { rawPath = "a/foo" }.toString() == "a/foo"
new net.Url { rawPath = "a/foo" }.pathSegments == List("a", "foo")
module.catch(() -> new net.Url { rawPath = "a:b" }.toString())
.startsWith("Type constraint `isValidPath(this, scheme != null, rawHost != null)` violated.")
}
["resolve(): RFC 3986, section 5.4.1 normal examples"] {
resolved("g:h") == "g:h"
resolved("g") == "http://a/b/c/g"
resolved("./g") == "http://a/b/c/g"
resolved("g/") == "http://a/b/c/g/"
resolved("/g") == "http://a/g"
resolved("//g") == "http://g"
resolved("?y") == "http://a/b/c/d;p?y"
resolved("g?y") == "http://a/b/c/g?y"
resolved("#s") == "http://a/b/c/d;p?q#s"
resolved("g#s") == "http://a/b/c/g#s"
resolved("g?y#s") == "http://a/b/c/g?y#s"
resolved(";x") == "http://a/b/c/;x"
resolved("g;x") == "http://a/b/c/g;x"
resolved("g;x?y#s") == "http://a/b/c/g;x?y#s"
resolved("") == "http://a/b/c/d;p?q"
resolved(".") == "http://a/b/c/"
resolved("./") == "http://a/b/c/"
resolved("..") == "http://a/b/"
resolved("../") == "http://a/b/"
resolved("../g") == "http://a/b/g"
resolved("../..") == "http://a/"
resolved("../../") == "http://a/"
resolved("../../g") == "http://a/g"
}
["resolve(): RFC 3986, section 5.4.2 abnormal examples"] {
// a path cannot be backed out of its root
resolved("../../../g") == "http://a/g"
resolved("../../../../g") == "http://a/g"
resolved("/./g") == "http://a/g"
resolved("/../g") == "http://a/g"
// only a complete "." or ".." segment is a dot segment
resolved("g.") == "http://a/b/c/g."
resolved(".g") == "http://a/b/c/.g"
resolved("g..") == "http://a/b/c/g.."
resolved("..g") == "http://a/b/c/..g"
resolved("./../g") == "http://a/b/g"
resolved("./g/.") == "http://a/b/c/g/"
resolved("g/./h") == "http://a/b/c/g/h"
resolved("g/../h") == "http://a/b/c/h"
resolved("g;x=1/./y") == "http://a/b/c/g;x=1/y"
resolved("g;x=1/../y") == "http://a/b/c/y"
// dot segments are a path notion; a query or fragment keeps them
resolved("g?y/./x") == "http://a/b/c/g?y/./x"
resolved("g?y/../x") == "http://a/b/c/g?y/../x"
resolved("g#s/./x") == "http://a/b/c/g#s/./x"
resolved("g#s/../x") == "http://a/b/c/g#s/../x"
resolved("http:g") == "http:g"
}
["resolve(): other cases"] {
resolved("../c") == "http://a/b/c"
net.Url("mailto:user@example.com").resolve("#f").toString()
== "mailto:user@example.com#f"
net.Url("foo://example.com/a/b").resolve("../c").toString() == "foo://example.com/c"
net.Url("file:///a/b").resolve("c").toString() == "file:///a/c"
// a reference that cannot be parsed cannot be resolved
module.catch(() -> net.Url(rfcBase).resolve("%zz")) == "Cannot parse `%zz` as a URL."
net.Url("http://a/b/c/").resolve(new net.Url { rawPath = "d%20e" }).toString()
== "http://a/b/c/d%20e"
// resolve a string or object should be the same
net.Url("http://a/b/c/").resolve("d e").rawPath == "/b/c/d%20e"
net.Url("http://a/b/c/").resolve(new net.Url { rawPath = "d%20e" }).rawPath == "/b/c/d%20e"
// resolving is the way to set a component without encoding it first
net.Url("http://a/b/c/").resolve("/x y").toString() == "http://a/x%20y"
// a path without an authority that would start with "//" is prefixed with "/."
net.Url("foo:/a").resolve("..//x").toString() == "foo:/.//x"
net.Url("foo:/a").resolve("..//x") == new net.Url { scheme = "foo"; rawPath = "/.//x" }
}
["new Url"] {
new net.Url { scheme = "https"; rawHost = "example.com" }.toString() == "https://example.com"
new net.Url { scheme = "https"; rawHost = "example.com"; rawPath = "/a/b" }.toString()
== "https://example.com/a/b"
new net.Url {
scheme = "https"
rawUserInfo = "user:pw"
rawHost = "example.com"
rawPort = "8080"
rawPath = "/a/b"
rawQuery = "q=1"
rawFragment = "f"
}.toString() == "https://user:pw@example.com:8080/a/b?q=1#f"
// a part has to be given encoded
new net.Url { scheme = "https"; rawHost = "example.com"; rawPath = "/a%20b" }.toString()
== "https://example.com/a%20b"
new net.Url { scheme = "https"; rawHost = "m%C3%BCnchen.de" }.toString() == "https://m%C3%BCnchen.de"
module.catch(() -> new net.Url { scheme = "https"; rawHost = "example.com"; rawPath = "/a b" }.toString())
.startsWith("Type constraint `isValidPath(this, scheme != null, rawHost != null)` violated.")
module.catch(() -> new net.Url { scheme = "https"; rawHost = "münchen.de" }.toString())
.startsWith("Type constraint `isValidHost(this)` violated.")
module.catch(() -> new net.Url { scheme = "https"; rawHost = "example.com"; rawUserInfo = "a@b" }.toString())
.startsWith("Type constraint `isValidUserInfo(this)` violated.")
module.catch(() -> new net.Url { scheme = "https"; rawHost = "example.com"; rawQuery = "a b" }.toString())
.startsWith("Type constraint `isValidQuery(this)` violated.")
module.catch(() -> new net.Url { scheme = "https"; rawHost = "example.com"; rawFragment = "a#b" }.toString())
.startsWith("Type constraint `isValidFragment(this)` violated.")
module.catch(() -> new net.Url { scheme = "https"; rawHost = "example.com"; rawPath = "/a%zz" }.toString())
.startsWith("Type constraint `isValidPath(this, scheme != null, rawHost != null)` violated.")
module.catch(() -> new net.Url { scheme = "https"; rawHost = "example.com"; rawPort = "8a" }.toString())
.startsWith(##"Type constraint `matches(Regex(#"\d*"#))` violated."##)
new net.Url { scheme = "x"; rawHost = "foo"; rawPort = "001" }.toString() == "x://foo:001"
// a URL without an authority
new net.Url { scheme = "mailto"; rawPath = "user@example.com" }.toString() == "mailto:user@example.com"
new net.Url { scheme = "foo" }.toString() == "foo:"
new net.Url { scheme = "https"; rawHost = "example.com"; rawPath = "/a" }
== net.Url("https://example.com/a")
// a host is held to the same rules as a parsed one: a registered name cannot hold a ":", but
// an IP literal and an encoded ":" are fine
module.catch(() -> new net.Url { scheme = "http"; rawHost = ":" }.toString())
.startsWith("Type constraint `isValidHost(this)` violated.")
module.catch(() -> new net.Url { scheme = "http"; rawHost = "::" }.toString())
.startsWith("Type constraint `isValidHost(this)` violated.")
new net.Url { scheme = "http"; rawHost = "[::]" }.toString() == "http://[::]"
new net.Url { scheme = "http"; rawHost = "%3A" }.toString() == "http://%3A"
}
["amending a Url"] {
local url = net.Url("https://user:pw@example.com:8080/a/b?q=1#f")
(url) { rawPath = "/c" }.toString() == "https://user:pw@example.com:8080/c?q=1#f"
(url) { rawQuery = null }.toString() == "https://user:pw@example.com:8080/a/b#f"
(url) { rawFragment = null; rawQuery = null }.toString() == "https://user:pw@example.com:8080/a/b"
(url) { rawPort = "0080" }.port == 80
(url) { rawUserInfo = null }.toString() == "https://example.com:8080/a/b?q=1#f"
(url) { scheme = "http" }.toString() == "http://user:pw@example.com:8080/a/b?q=1#f"
// the derived views follow the amendment
(url) { rawPath = "/c/d%20e" }.pathSegments == List("c", "d e")
// an amended part has to be encoded
module.catch(() -> (url) { rawPath = "/a b" }.toString())
.startsWith("Type constraint `isValidPath(this, scheme != null, rawHost != null)` violated.")
(url) { rawPath = "/a%20b" }.toString() == "https://user:pw@example.com:8080/a%20b?q=1#f"
}
["hostKind and zoneId"] {
net.Url("http://[v1.fe80::a+en1]/").hostKind == "ipvFuture"
net.Url("http://192.0.2.256/").hostKind == "name"
net.Url("file:///tmp").hostKind == "name"
net.Url("http://[fe80::1%25en%2F1]/").zoneId == "en/1"
new net.Url { scheme = "http"; rawHost = "[fe80::1%25eth0]" }.zoneId == "eth0"
}
["pathSegments"] {
net.Url("https://example.com/a/b").pathSegments == List("a", "b")
net.Url("https://example.com/").pathSegments == List("")
net.Url("https://example.com/a/").pathSegments == List("a", "")
net.Url("https://example.com").pathSegments == List()
net.Url("https://example.com/a/b%20c").pathSegments == List("a", "b c")
net.Url("https://example.com/m%C3%BCnchen").pathSegments == List("münchen")
// a path is split before its segments are decoded, so an encoded "/" stays in its segment
net.Url("https://example.com/a%2Fb").pathSegments == List("a/b")
// a rootless path has segments too
net.Url("mailto:user@example.com").pathSegments == List("user@example.com")
net.Url("urn:isbn:0451450523").pathSegments == List("isbn:0451450523")
}
["queryParameters"] {
net.Url("https://example.com?a=1&&b=2").queryParameters
== new Mapping<String, Listing<String>> { ["a"] { "1" }; ["b"] { "2" } }
net.Url("https://example.com?").queryParameters == new Mapping {}
net.Url("https://example.com").queryParameters == null
// a bare name holds no value, so it contributes nothing
net.Url("https://example.com?a").queryParameters == new Mapping {}
net.Url("https://example.com?a=").queryParameters == new Mapping<String, Listing<String>> { ["a"] { "" } }
net.Url("https://example.com?=1").queryParameters == new Mapping<String, Listing<String>> { [""] { "1" } }
// a name that repeats holds every value it is given, in order
net.Url("https://example.com?a=1&a=2").queryParameters == new Mapping<String, Listing<String>> { ["a"] { "1"; "2" } }
net.Url("https://example.com?a&a=2").queryParameters == new Mapping<String, Listing<String>> { ["a"] { "2" } }
net.Url("https://example.com?a=&a=2").queryParameters == new Mapping<String, Listing<String>> { ["a"] { ""; "2" } }
net.Url("https://example.com?a=1&b=2&a=3").queryParameters
== new Mapping<String, Listing<String>> { ["a"] { "1"; "3" }; ["b"] { "2" } }
net.Url("https://example.com?a%20b=c%20d").queryParameters == new Mapping<String, Listing<String>> { ["a b"] { "c d" } }
net.Url("https://example.com?a+b=c+d").queryParameters == new Mapping<String, Listing<String>> { ["a b"] { "c d" } }
}
["equality"] {
net.Url("https://example.com/a") == net.Url("https://example.com/a")
net.Url("https://example.com/a") != net.Url("https://example.com/b")
// nothing is normalized, so these are distinct URLs
net.Url("https://example.com/a") != net.Url("https://example.com:443/a")
net.Url("x://foo:001") != net.Url("x://foo:1")
net.Url("https://example.com/a") != net.Url("https://EXAMPLE.com/a")
net.Url("https://example.com/%41") != net.Url("https://example.com/A")
net.Url("https://example.com/a") != net.Url("HTTPS://example.com/a")
}
["normalized"] {
local function normalized(input: String): String = net.Url(input).normalized.toString()
normalized("HTTPS://EXAMPLE.com/A") == "https://example.com/A"
normalized("http://[2001:DB8::1]/") == "http://[2001:db8::1]/"
normalized("http://[FE80::1%25eTH0]/") == "http://[fe80::1%25eTH0]/"
normalized("https://example.com/%7Euser") == "https://example.com/~user"
normalized("https://U%7Ev@example.com/?q=%7E#%7E") == "https://U~v@example.com/?q=~#~"
normalized("https://example.com/a%2fb") == "https://example.com/a%2Fb"
normalized("https://münchen.de/") == "https://m%C3%BCnchen.de/"
normalized("https://example.com/a/./b/../c") == "https://example.com/a/c"
normalized("foo:/bar/baz") == "foo:/bar/baz"
normalized("file:///../a") == "file:///a"
normalized("file:///foo/%2E%2E/bar") == "file:///bar"
normalized("a/../b") == "a/../b"
normalized("mailto:user@example.com") == "mailto:user@example.com"
normalized("https://example.com") == "https://example.com/"
normalized("foo:") == "foo:"
normalized("https://example.com:443/") == "https://example.com:443/"
normalized("https://example.com/?#") == "https://example.com/?#"
normalized("//EXAMPLE.com/a/../b") == "//example.com/b"
normalized("") == ""
normalized("foo:/..//x") == "foo:/.//x"
// normalizing is idempotent
net.Url("HTTPS://Example.com/a/../%7Eb c").normalized.normalized
== net.Url("HTTPS://Example.com/a/../%7Eb c").normalized
net.Url("HTTPS://Example.com/%7Euser").normalized == net.Url("https://example.com/~user").normalized
net.Url("HTTPS://Example.com/%7Euser").normalized == net.Url("https://example.com/~user")
net.Url("https://example.com/a%2Fb").normalized != net.Url("https://example.com/a/b").normalized
net.Url("HTTPS://User%7E@EXAMPLE.com:8080/a/../%7Eb?q=%7E#%7E").normalized
== new net.Url {
scheme = "https"
rawUserInfo = "User~"
rawHost = "example.com"
rawPort = "8080"
rawPath = "/~b"
rawQuery = "q=~"
rawFragment = "~"
}
}
["equals()"] {
local function equal(left: String, right: String): Boolean =
net.Url(left).equals(net.Url(right))
// the scheme and host are case-insensitive
equal("FILE:///foo%20bar/baz", "file:///foo bar/baz")
equal("https://EXAMPLE.com/a", "https://example.com/a")
equal("http://[2001:DB8::1]/a", "http://[2001:db8::1]/a")
// but nothing else is
!equal("https://example.com/A", "https://example.com/a")
// percent-encoding is normalized in every component
equal("https://u%20v@example.com/?q=a b#a b", "https://u v@example.com/?q=a%20b#a%20b")
equal("https://example.com/m%C3%BCnchen", "https://example.com/münchen")
// an octet standing for a reserved character means something else decoded, so it is kept
!equal("https://example.com/a%2Fb", "https://example.com/a/b")
// dot segments are removed from an absolute path
equal("file:///foo/bar", "file:///foo/./bar")
equal("file:///foo/bar", "file:///foo/baz/../bar")
equal("file:///foo/bar/", "file:///foo/bar/baz/../")
// percent-encoding is normalized first, so an encoded dot segment is one too
equal("file:///foo/%2E%2E/bar", "file:///bar")
// a trailing "/" is part of the path, and a path ending in ".." keeps one
!equal("file:///foo/bar", "file:///foo/bar/")
// a rootless path is compared as written
!equal("../a", "a")
!equal("../a", "../../a")
equal("a/b", "a/b")
// a URL with an authority and no path is one whose path is "/"
equal("https://example.com", "https://example.com/")
// but a URL without an authority is not
!equal("foo:", "foo:/")
// a default port is only dropped by scheme-based normalization, which this does not do
!equal("https://example.com:443/a", "https://example.com/a")
}
}
@@ -0,0 +1,98 @@
amends "../snippetTest.pkl"
import "pkl:net"
local gClef = "\u{1D11E}"
facts {
["encodeUrlComponent()"] {
net.encodeUrlComponent("a b&c") == "a%20b%26c"
net.encodeUrlComponent("münchen") == "m%C3%BCnchen"
// only the unreserved characters are left alone
net.encodeUrlComponent("aZ09-._~") == "aZ09-._~"
net.encodeUrlComponent("!'()*") == "%21%27%28%29%2A"
net.encodeUrlComponent("/?#[]@:;=+$,%") == "%2F%3F%23%5B%5D%40%3A%3B%3D%2B%24%2C%25"
net.encodeUrlComponent("a" + gClef + "b") == "a%F0%9D%84%9Eb"
// an encoded component is safe to drop into any component of a URL
new net.Url {
scheme = "https"
rawHost = "example.com"
rawPath = "/" + net.encodeUrlComponent("a/b")
}.pathSegments == List("a/b")
}
["decodeUrlComponent()"] {
net.decodeUrlComponent("a%20b%26c") == "a b&c"
net.decodeUrlComponent("m%C3%BCnchen") == "münchen"
// "+" is not a space; that is an application/x-www-form-urlencoded convention
net.decodeUrlComponent("a+b") == "a+b"
// a literal "%" has to be written as "%25"
net.decodeUrlComponent("100%25") == "100%"
net.decodeUrlComponent("a%F0%9D%84%9Eb") == "a" + gClef + "b"
net.decodeUrlComponent("%FF") == "\u{FFFD}"
net.decodeUrlComponent(net.encodeUrlComponent("a b&c=d?e#f/münchen+%")) == "a b&c=d?e#f/münchen+%"
net.decodeUrlComponent(net.encodeUrlComponent(gClef)) == gClef
}
["buildQuery()"] {
net.buildQuery(new Mapping<String, Listing<String>> { ["a"] { "1" }; ["b"] { "2" } }) == "a=1&b=2"
net.buildQuery(new Mapping {}) == ""
// a name is written once per value it holds
net.buildQuery(new Mapping<String, Listing<String>> { ["a"] { "1"; "2" } }) == "a=1&a=2"
net.buildQuery(new Mapping<String, Listing<String>> { ["a"] { "1"; "2" }; ["b"] { "3" } }) == "a=1&a=2&b=3"
// one that holds no values is left out entirely
net.buildQuery(new Mapping<String, Listing<String>> { ["a"] {} }) == ""
net.buildQuery(new Mapping<String, Listing<String>> { ["a"] {}; ["b"] { "1" } }) == "b=1"
net.buildQuery(new Mapping<String, Listing<String>> { ["a"] { "" } }) == "a="
net.buildQuery(new Mapping<String, Listing<String>> { [""] { "1" } }) == "=1"
net.buildQuery(new Mapping<String, Listing<String>> { ["a b"] { "c d" } }) == "a+b=c+d"
net.buildQuery(new Mapping<String, Listing<String>> { ["a&b"] { "1=2" } }) == "a%26b=1%3D2"
// only the ASCII alphanumerics and "*-._" are left alone, as WHATWG's serializer does
net.buildQuery(new Mapping<String, Listing<String>> { ["q"] { "aZ09*-._" } }) == "q=aZ09*-._"
net.buildQuery(new Mapping<String, Listing<String>> { ["q"] { "~!'()" } }) == "q=%7E%21%27%28%29"
local parameters = new Mapping<String, Listing<String>> {
["a b"] { "c&d" }
["e"] { "1"; "2" }
["f"] { "" }
["münchen"] { "a+b" }
}
net.Url("https://example.com?" + net.buildQuery(parameters)).queryParameters == parameters
(net.Url("https://example.com/a?q=1")) {
rawQuery = net.buildQuery(new Mapping<String, Listing<String>> { ["a"] { "b c" } })
}.toString() == "https://example.com/a?a=b+c"
}
["parseQuery()"] {
net.parseQuery("a=1&b=2") == new Mapping<String, Listing<String>> { ["a"] { "1" }; ["b"] { "2" } }
net.parseQuery("") == new Mapping {}
net.parseQuery("a=1&&b=2") == new Mapping<String, Listing<String>> { ["a"] { "1" }; ["b"] { "2" } }
// a bare name holds no value, so it contributes nothing
net.parseQuery("a") == new Mapping {}
net.parseQuery("a&b=1") == new Mapping<String, Listing<String>> { ["b"] { "1" } }
net.parseQuery("a=") == new Mapping<String, Listing<String>> { ["a"] { "" } }
net.parseQuery("=1") == new Mapping<String, Listing<String>> { [""] { "1" } }
// a name that repeats holds every value it is given, in order
net.parseQuery("a=1&a=2") == new Mapping<String, Listing<String>> { ["a"] { "1"; "2" } }
net.parseQuery("a=1&b=2&a=3")
== new Mapping<String, Listing<String>> { ["a"] { "1"; "3" }; ["b"] { "2" } }
// only the first "=" separates a name from its value
net.parseQuery("a=1=2") == new Mapping<String, Listing<String>> { ["a"] { "1=2" } }
net.parseQuery("a%20b=c%20d") == new Mapping<String, Listing<String>> { ["a b"] { "c d" } }
net.parseQuery("a+b=c+d") == new Mapping<String, Listing<String>> { ["a b"] { "c d" } }
net.parseQuery("q=m%C3%BCnchen") == new Mapping<String, Listing<String>> { ["q"] { "münchen" } }
net.parseQuery("q=%7E*") == new Mapping<String, Listing<String>> { ["q"] { "~*" } }
// this is how a Url reads its own query
local url = net.Url("https://example.com?a=1&a=2&b=3")
net.parseQuery(url.rawQuery!!) == url.queryParameters
// and the inverse of buildQuery()
local parameters = new Mapping<String, Listing<String>> {
["a b"] { "c&d" }
["e"] { "1"; "2" }
["f"] { "" }
["münchen"] { "a+b" }
}
net.parseQuery(net.buildQuery(parameters)) == parameters
}
}
@@ -0,0 +1,3 @@
import "pkl:net"
res = net.decodeUrlComponent("100%")
@@ -0,0 +1,3 @@
import "pkl:net"
res = net.decodeUrlComponent("foo%zzbar")
@@ -0,0 +1,3 @@
import "pkl:net"
res = net.parseQuery("a=1&b=100%")
@@ -0,0 +1,3 @@
import "pkl:net"
res = net.Url("a/foo")!!.resolve("b")
@@ -959,6 +959,36 @@ alias {
allAnnotations = List(new {
version = "0.29.0"
})
}, "isValidUrl", new {
location {
line = XXXX
column = 3
displayUri = "https://github.com/apple/pkl/blob/$commitId/stdlib/base.pkl#LXXXX"
}
docComment = """
Tells if this is a valid absolute URL according to
[RFC 3986](https://www.rfc-editor.org/rfc/rfc3986).
Stricter than [net.Url()], which also parses a relative reference, and which percent-encodes
whatever cannot appear literally in a URL component.
Facts:
```
"https://example.com/a?q=1#f".isValidUrl
"mailto:user@example.com".isValidUrl
!"./foo".isValidUrl
!"http://example.com/some path".isValidUrl
```
"""
annotations = List(new {
version = "0.33.0"
})
modifiers = Set()
name = "isValidUrl"
allModifiers = Set()
allAnnotations = List(new {
version = "0.33.0"
})
}, "md5", new {
location {
line = XXXX
@@ -1324,6 +1354,36 @@ alias {
allAnnotations = List(new {
version = "0.29.0"
})
}, "isValidUrl", new {
location {
line = XXXX
column = 3
displayUri = "https://github.com/apple/pkl/blob/$commitId/stdlib/base.pkl#LXXXX"
}
docComment = """
Tells if this is a valid absolute URL according to
[RFC 3986](https://www.rfc-editor.org/rfc/rfc3986).
Stricter than [net.Url()], which also parses a relative reference, and which percent-encodes
whatever cannot appear literally in a URL component.
Facts:
```
"https://example.com/a?q=1#f".isValidUrl
"mailto:user@example.com".isValidUrl
!"./foo".isValidUrl
!"http://example.com/some path".isValidUrl
```
"""
annotations = List(new {
version = "0.33.0"
})
modifiers = Set()
name = "isValidUrl"
allModifiers = Set()
allAnnotations = List(new {
version = "0.33.0"
})
}, "md5", new {
location {
line = XXXX
@@ -24,6 +24,23 @@ facts {
true
true
}
["isValidUrl"] {
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
}
["lastIndex"] {
true
true
@@ -0,0 +1,335 @@
facts {
["components"] {
true
true
true
true
true
true
true
true
}
["parse(): keeps what is already encoded as it was written"] {
true
true
true
true
true
true
true
true
true
true
true
true
true
true
}
["parse(): round-trips valid URIs"] {
true
true
true
true
true
true
true
true
true
true
true
true
true
true
}
["parse(): encodes what it cannot keep"] {
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
}
["decoded components"] {
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
}
["parse(): failing"] {
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
}
["parse(): RFC 6874 zone identifiers"] {
true
true
true
true
true
true
true
true
true
true
true
true
true
true
}
["parse(): relative references"] {
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
}
["resolve(): RFC 3986, section 5.4.1 normal examples"] {
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
}
["resolve(): RFC 3986, section 5.4.2 abnormal examples"] {
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
}
["resolve(): other cases"] {
true
true
true
true
true
true
true
true
true
true
true
}
["new Url"] {
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
}
["amending a Url"] {
true
true
true
true
true
true
true
true
true
}
["hostKind and zoneId"] {
true
true
true
true
true
}
["pathSegments"] {
true
true
true
true
true
true
true
true
true
}
["queryParameters"] {
true
true
true
true
true
true
true
true
true
true
true
true
}
["equality"] {
true
true
true
true
true
true
true
}
["normalized"] {
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
}
["equals()"] {
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
}
}
@@ -0,0 +1,55 @@
facts {
["encodeUrlComponent()"] {
true
true
true
true
true
true
true
}
["decodeUrlComponent()"] {
true
true
true
true
true
true
true
true
}
["buildQuery()"] {
true
true
true
true
true
true
true
true
true
true
true
true
true
true
}
["parseQuery()"] {
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
true
}
}
@@ -1,12 +1,19 @@
–– Pkl Error ––
Module URI `foo <>` has invalid syntax.
Type constraint `isValidUrl` violated.
Value: "foo <>"
isValidUrl
│
false
xxxx | typealias Uri = String(isValidUrl)
^^^^^^^^^^
at pkl.analyze#importGraph (pkl:base)
x | result = analyze.importGraph(Set("foo <>"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at analyzeInvalidModuleUri#result (file:///$snippetsDir/input/errors/analyzeInvalidModuleUri.pkl)
Illegal character in path at index 3: foo <>
xxx | renderer.renderDocument(value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at pkl.base#Module.output.text (pkl:base)
@@ -1,5 +1,14 @@
–– Pkl Error ––
Cannot analyze relative module URI `foo.pkl`.
Type constraint `isValidUrl` violated.
Value: "foo.pkl"
isValidUrl
│
false
xxxx | typealias Uri = String(isValidUrl)
^^^^^^^^^^
at pkl.analyze#importGraph (pkl:base)
x | result = analyze.importGraph(Set("foo.pkl"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -0,0 +1,16 @@
–– Pkl Error ––
Cannot percent-decode `100%`.
x | res = net.decodeUrlComponent("100%")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at cannotDecodeUrlComponent#res (file:///$snippetsDir/input/errors/cannotDecodeUrlComponent.pkl)
`%` is not a percent-encoded octet.
xxx | renderer.renderDocument(value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at pkl.base#Module.output.text (pkl:base)
xxx | if (renderer is BytesRenderer) renderer.renderDocument(value) else text.encodeToBytes("UTF-8")
^^^^
at pkl.base#Module.output.bytes (pkl:base)
@@ -0,0 +1,16 @@
–– Pkl Error ––
Cannot percent-decode `foo%zzbar`.
x | res = net.decodeUrlComponent("foo%zzbar")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at cannotDecodeUrlComponent2#res (file:///$snippetsDir/input/errors/cannotDecodeUrlComponent2.pkl)
`%zz` is not a percent-encoded octet.
xxx | renderer.renderDocument(value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at pkl.base#Module.output.text (pkl:base)
xxx | if (renderer is BytesRenderer) renderer.renderDocument(value) else text.encodeToBytes("UTF-8")
^^^^
at pkl.base#Module.output.bytes (pkl:base)
@@ -16,6 +16,7 @@ pkl:EvaluatorSettings
pkl:json
pkl:jsonnet
pkl:math
pkl:net
pkl:pklbinary
pkl:platform
pkl:Project
@@ -0,0 +1,16 @@
–– Pkl Error ––
Cannot parse `a=1&b=100%` as a query.
x | res = net.parseQuery("a=1&b=100%")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at cannotParseQuery#res (file:///$snippetsDir/input/errors/cannotParseQuery.pkl)
`%` is not a percent-encoded octet.
xxx | renderer.renderDocument(value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at pkl.base#Module.output.text (pkl:base)
xxx | if (renderer is BytesRenderer) renderer.renderDocument(value) else text.encodeToBytes("UTF-8")
^^^^
at pkl.base#Module.output.bytes (pkl:base)
@@ -0,0 +1,16 @@
–– Pkl Error ––
Cannot resolve a reference against `a/foo`, which is a relative URL.
x | res = net.Url("a/foo")!!.resolve("b")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at cannotResolveAgainstRelativeUrl#res (file:///$snippetsDir/input/errors/cannotResolveAgainstRelativeUrl.pkl)
Only a URL that states a scheme can be resolved against.
xxx | renderer.renderDocument(value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at pkl.base#Module.output.text (pkl:base)
xxx | if (renderer is BytesRenderer) renderer.renderDocument(value) else text.encodeToBytes("UTF-8")
^^^^
at pkl.base#Module.output.bytes (pkl:base)
@@ -2,7 +2,7 @@
Expected value of type `String`, but got type `Int`.
Value: 42
xxxx | typealias Uri = String
xxxx | typealias Uri = String(isValidUrl)
^^^^^^
at intrinsifiedTypeAlias4#res1 (pkl:base)
@@ -1,6 +1,15 @@
–– Pkl Error ––
URI `file:///not a valid uri` has invalid syntax.
Type constraint `isValidUrl` violated.
Value: "file:///not a valid uri"
xxx | moduleCacheDir = resolvePath(enclosingUri, module.moduleCacheDir!!, forWindows)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at pkl.EvaluatorSettings#resolveForOs.moduleCacheDir (pkl:EvaluatorSettings)
isValidUrl
│
false
xxxx | typealias Uri = String(isValidUrl)
^^^^^^^^^^
at pkl.EvaluatorSettings#resolve (pkl:base)
xxx | fixed resolvedEvaluatorSettings: EvaluatorSettingsModule = evaluatorSettings.resolve(projectFileUri)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at pkl.Project#resolvedEvaluatorSettings (pkl:Project)
@@ -54,8 +54,8 @@ class PkldocGeneratorsTest : AbstractTest() {
version = "1.0.0"
importUri = "https://pkl-lang.org/"
authors { "publisher@apple.com" }
sourceCode = "sources.apple.com/"
issueTracker = "issues.apple.com"
sourceCode = "https://sources.apple.com/"
issueTracker = "https://issues.apple.com"
"""
.trimIndent(),
)
+18 -1
View File
@@ -23,6 +23,7 @@ module pkl.base
import "pkl:json"
import "pkl:jsonnet"
import "pkl:math"
import "pkl:net"
import "pkl:pklbinary"
// noinspection Deprecated
import "pkl:protobuf"
@@ -1340,6 +1341,22 @@ external class String extends Any {
@Since { version = "0.29.0" }
external isBase64: Boolean
/// Tells if this is a valid absolute URL according to
/// [RFC 3986](https://www.rfc-editor.org/rfc/rfc3986).
///
/// Stricter than [net.Url()], which also parses a relative reference, and which percent-encodes
/// whatever cannot appear literally in a URL component.
///
/// Facts:
/// ```
/// "https://example.com/a?q=1#f".isValidUrl
/// "mailto:user@example.com".isValidUrl
/// !"./foo".isValidUrl
/// !"http://example.com/some path".isValidUrl
/// ```
@Since { version = "0.33.0" }
external isValidUrl: Boolean
/// The [MD5](https://en.wikipedia.org/wiki/MD5)
/// hash of this string's UTF-8 byte sequence
/// as hexadecimal string.
@@ -1704,7 +1721,7 @@ external class String extends Any {
typealias Charset = "UTF-8" | "UTF-16" | "ISO-8859-1"
/// A string representing a [URI](https://en.wikipedia.org/wiki/Uniform_Resource_Identifier).
typealias Uri = String
typealias Uri = String(isValidUrl)
/// Constructs a regular expression described by [pattern].
///
+349
View File
@@ -0,0 +1,349 @@
//===----------------------------------------------------------------------===//
// Copyright © 2026 Apple Inc. and the Pkl project authors. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//===----------------------------------------------------------------------===//
/// Utilities for networking, like URL parsing.
@ModuleInfo { minPklVersion = "0.33.0" }
@Since { version = "0.33.0" }
module pkl.net
/// Constructs a Url as described by [input], following
/// [RFC 3986](https://www.rfc-editor.org/rfc/rfc3986).
/// Accepts a URL reference (one without a scheme).
///
/// This is lenient: whatever a component cannot hold literally is percent-encoded, and whatever is
/// already percent-encoded is kept as is.
///
/// Throws if [input] is not a valid URL.
///
/// To test if [input] is a valid URL, use [String.isValidUrl].
///
/// Facts:
/// ```
/// net.Url("https://example.com/foo?a=1").host == "example.com"
/// net.Url("https://example.com/foo?a=1").query == "a=1"
/// net.Url("https://example.com/a b").rawPath == "/a%20b"
/// net.Url("/foo").scheme == null
/// ```
external const function Url(input: String): Url
/// A URL, as defined by [RFC 3986](https://www.rfc-editor.org/rfc/rfc3986).
/// [RFC 6874](https://www.rfc-editor.org/rfc/rfc6874) IPv6 zone literals are also supported.
///
/// Every component is held percent-encoded, in its `raw` property, such as [rawPath]:
/// [Url()] encodes whatever a component cannot hold literally, and a component that is set by hand,
/// or amended, has to be encoded already.
/// The percent-decoded form of each component is available without the `raw` prefix, such as
/// [path].
///
/// Facts:
/// ```
/// net.Url("https://example.com/a b").rawPath == "/a%20b"
/// net.Url("https://example.com/a b").path == "/a b"
/// net.Url("https://example.com/a%20b").rawPath == "/a%20b"
/// ```
class Url {
/// The scheme, without the trailing `:`, or [null] if this is a relative reference.
scheme: String(isValidScheme(this))?
/// The percent-encoded userinfo, without the trailing `@`, or [null] if this URL has none.
///
/// Only a URL with a [rawHost] can have userinfo.
rawUserInfo: String(rawHost != null, isValidUserInfo(this))?
/// The percent-decoded [rawUserInfo].
hidden fixed userInfo: String? =
if (rawUserInfo != null) decodeUrlComponent(rawUserInfo) else null
/// The percent-encoded host, or [null] if this URL has no authority.
///
/// A [null] host is not the same as an empty one: `file:/tmp` has no host, `file:///tmp` has an
/// empty one.
///
/// An IPv6 address must be enclosed in brackets, as in `[::1]`.
/// It may carry a zone identifier, as described in RFC 6874.
/// Anything else is a registered name, which cannot contain `[`, `]` or `:` literally.
rawHost: String(isValidHost(this))?
/// The percent-decoded [rawHost].
hidden fixed host: String? = if (rawHost != null) decodeUrlComponent(rawHost) else null
/// The port, without the leading `:`, or [null] if this URL states none.
///
/// Only a URL with a [rawHost] can have a port.
rawPort: String(rawHost != null, matches(Regex(#"\d*"#)))?
/// The number that [rawPort] states.
///
/// This is [null] if [rawPort] is [null], or if [rawPort] is not parseable as a [UInt16].
///
/// Facts:
/// ```
/// net.Url("https://example.com:8080/").port == 8080
/// net.Url("https://example.com:/").port == null
/// ```
hidden fixed port: UInt16? =
let (parsed = rawPort?.toIntOrNull())
if (parsed is UInt16) parsed else null
/// The percent-encoded path, or the empty string if this URL has none.
///
/// With a [rawHost], the path is empty or starts with `/`.
/// Without one, it may be *rootless*, as in `mailto:user@example.com`.
/// Without a [scheme], its first segment cannot contain `:`.
///
/// To change the path without encoding it first, use [resolve()], which is as lenient as [Url()].
/// Unlike amending, this drops [rawQuery] and [rawFragment].
rawPath: String(isValidPath(this, scheme != null, rawHost != null)) = ""
/// The percent-decoded [rawPath].
///
/// An encoded `/` is decoded like any other octet, so it cannot be distinguished from a `/` that
/// separates segments.
/// [pathSegments] disambiguates between separator and literal `/` characters.
///
/// Facts:
/// ```
/// net.Url("https://example.com/a%2Fb/c").path == "/a/b/c"
/// net.Url("https://example.com/a%2Fb/c").pathSegments == List("a/b", "c")
/// ```
hidden fixed path: String = decodeUrlComponent(rawPath)
/// The percent-encoded query, without the leading `?`, or [null] if this URL has none.
///
/// The empty string is a URL ending in a bare `?`.
rawQuery: String(isValidQuery(this))?
/// The percent-decoded [rawQuery].
///
/// Unlike [queryParameters], this does not read [rawQuery] as
/// `application/x-www-form-urlencoded`, so a `+` is kept as is.
hidden fixed query: String? = if (rawQuery != null) decodeUrlComponent(rawQuery) else null
/// The percent-encoded fragment, without the leading `#`, or [null] if this URL has none.
///
/// The empty string is a URL ending in a bare `#`.
rawFragment: String(isValidFragment(this))?
/// The percent-decoded [rawFragment].
hidden fixed fragment: String? =
if (rawFragment != null) decodeUrlComponent(rawFragment) else null
/// The serialization of this URL.
fixed stringValue: String = toString()
/// Tells if this URL is absolute, that is, has a non-null scheme.
hidden fixed isAbsolute: Boolean = scheme != null
/// Tells if this URL's path is absolute, that is, starts with `/`.
hidden fixed hasAbsolutePath: Boolean = rawPath.startsWith("/")
/// The authority, without the leading `//`, or [null] if this URL has none.
///
/// A URL has an authority exactly when it has a [rawHost].
/// This is [rawUserInfo], [rawHost] and [rawPort] joined back together.
external fixed authority: String?
/// What [rawHost] is, or [null] if this URL has no authority.
///
/// - `"ipv6"`: an IPv6 address, such as `[::1]`
/// - `"ipvFuture"`: an IP address of a version that RFC 3986 leaves for the future
/// - `"ipv4"`: an IPv4 address in dotted-decimal form, such as `192.0.2.1`
/// - `"name"`: any other host, a *registered name*
///
/// Facts:
/// ```
/// net.Url("https://example.com/").hostKind == "name"
/// net.Url("https://192.0.2.1/").hostKind == "ipv4"
/// net.Url("https://[::1]/").hostKind == "ipv6"
/// net.Url("/foo").hostKind == null
/// ```
external fixed hostKind: ("name" | "ipv4" | "ipv6" | "ipvFuture")?
/// The percent-decoded zone identifier of an IPv6 [rawHost], as described in RFC 6874, or [null]
/// if it states none.
///
/// Facts:
/// ```
/// net.Url("http://[fe80::1%25eth0]/").zoneId == "eth0"
/// net.Url("http://[fe80::1]/").zoneId == null
/// ```
external fixed zoneId: String?
/// The percent-decoded segments of [rawPath].
///
/// [pathSegments] does not tell if the path is absolute or not.
/// A relative URL reference `"a/b"` also has path segments `List("a", "b")`.
/// To disambiguate between the two, use [hasAbsolutePath].
///
/// Facts:
/// ```
/// net.Url("https://example.com/a/b%20c").pathSegments == List("a", "b c")
/// net.Url("https://example.com/a/b%2Fc").pathSegments == List("a", "b/c")
/// ```
external fixed pathSegments: List<String>
/// The percent-decoded parameters of [rawQuery], or [null] if this URL has no query.
///
/// [rawQuery] is read as `application/x-www-form-urlencoded`.
/// A name that repeats holds the values of all its occurrences, in the order they are written.
/// A parameter that states no `=` holds no value at all, and is left out; this is not the same as
/// a bare `=`, which gives it the empty string.
///
/// Facts:
/// ```
/// net.Url("https://example.com/?q=a+b&q=c").queryParameters["q"] == new Listing { "a b"; "c" }
/// net.Url("https://example.com/?").queryParameters.isEmpty
/// net.Url("https://example.com/").queryParameters == null
/// ```
external fixed queryParameters: Mapping<String, Listing<String>>?
/// Resolves [ref] against this URL.
///
/// [ref] is either a relative reference, which is resolved against this URL, or an absolute URL,
/// which replaces it entirely.
/// Dot segments (`./` and `../`) are removed from the result's path, normalizing it.
///
/// Throws if [ref] is a string that cannot be parsed, or if this URL is itself a relative
/// reference.
///
/// Facts:
/// ```
/// net.Url("https://example.com/a/b").resolve("../c").toString() == "https://example.com/c"
/// net.Url("https://example.com/a/b").resolve("#top").toString() == "https://example.com/a/b#top"
/// net.Url("foo:/a").resolve("..//x").toString() == "foo:/.//x"
/// ```
external function resolve(ref: String | Url): Url
/// The normalized version of this URL, following *syntax-based normalization* as described in
/// [section 6.2.2](https://www.rfc-editor.org/info/rfc3986/#section-6.2.2) of RFC 3986.
///
/// [scheme] and [rawHost] are lowercased, as they are case-insensitive.
/// Percent-encoding is normalized: an octet standing for an unreserved character is decoded, and
/// any other octet is kept, with its hex digits uppercased.
///
/// Dot segments (`./` and `../`) are only removed from an absolute [rawPath].
/// A URL with a [rawHost] and no path is given the path `/`, which names the same resource.
///
/// Does not perform normalization that requires knowing the scheme, such as removing a default
/// port (see [section 6.2.3](https://www.rfc-editor.org/info/rfc3986/#section-6.2.3)).
///
/// Facts:
/// ```
/// net.Url("HTTPS://Example.com/%7Euser").normalized == net.Url("https://example.com/~user")
/// net.Url("https://example.com/a/b/../c").normalized.rawPath == "/a/c"
/// net.Url("https://example.com").normalized.rawPath == "/"
/// ```
external hidden fixed normalized: Url
/// Tells if this URL is equal to [other] according to *syntax-based normalization* as
/// described in [section 6.2.2](https://www.rfc-editor.org/info/rfc3986/#section-6.2.2) of
/// RFC 3986.
///
/// Returns [true] if the [normalized] versions of [this] and [other] are the
/// same, and [false] otherwise.
///
/// Facts:
/// ```
/// net.Url("HTTPS://Example.com/%7Euser").equals(net.Url("https://example.com/~user"))
/// !(net.Url("https://example.com/a%2Fb").equals(net.Url("https://example.com/a/b")))
/// !(net.Url("https://example.com:443/").equals(net.Url("https://example.com/")))
/// ```
external function equals(other: Url): Boolean
/// The serialization of this URL.
///
/// Facts:
/// ```
/// net.Url("https://example.com/a b").toString() == "https://example.com/a%20b"
/// net.Url("https://example.com/é").toString() == "https://example.com/%C3%A9"
/// net.Url("HTTPS://example.com").toString() == "HTTPS://example.com"
/// ```
external function toString(): String
/// Tells if [value] is a percent-encoded `userinfo`.
external function isValidUserInfo(value: String): Boolean
/// Tells if [value] is a `scheme`.
external function isValidScheme(value: String): Boolean
/// Tells if [value] is a percent-encoded `host`.
external function isValidHost(value: String): Boolean
/// Tells if [value] is a percent-encoded path that can sit next to a scheme and an authority, or,
/// when [hasScheme] or [hasAuthority] is [false], next to no scheme or no authority.
external function isValidPath(value: String, hasScheme: Boolean, hasAuthority: Boolean): Boolean
/// Tells if [value] is a percent-encoded `query`.
external function isValidQuery(value: String): Boolean
/// Tells if [value] is a percent-encoded `fragment`.
external function isValidFragment(value: String): Boolean
}
/// Percent-encodes [value] so that it can be interpolated into any URL component.
///
/// Every character other than an ASCII alphanumeric or one of `-._~`, the *unreserved* characters
/// of RFC 3986, is percent-encoded as its UTF-8 bytes.
///
/// Facts:
/// ```
/// net.encodeUrlComponent("a b&c") == "a%20b%26c"
/// net.encodeUrlComponent("50%") == "50%25"
/// ```
external const function encodeUrlComponent(value: String): String
/// Percent-decodes [value], interpreting the decoded bytes as UTF-8.
///
/// Throws if any `%` in [value] does not begin a percent-encoded octet.
/// To write a literal `%`, encode it as `%25`.
///
/// Facts:
/// ```
/// net.decodeUrlComponent("a%20b%26c") == "a b&c"
/// net.decodeUrlComponent("50%25") == "50%"
/// ```
external const function decodeUrlComponent(value: String): String
/// Builds a query from [parameters], for use as a [Url.rawQuery].
///
/// The result is `application/x-www-form-urlencoded`.
/// A name is written once per value it holds; one that holds no values is left out entirely.
///
/// [Url.queryParameters] reads back exactly the parameters given here, less any name that holds no
/// values.
///
/// Facts:
/// ```
/// net.buildQuery(new Mapping { ["q"] = new Listing { "a b" } }) == "q=a+b"
/// net.buildQuery(new Mapping { ["a"] = new Listing { "1"; "2" } }) == "a=1&a=2"
/// net.buildQuery(new Mapping { ["a"] = new Listing { "1" }; ["b"] = new Listing {} }) == "a=1"
/// ```
external const function buildQuery(parameters: Mapping<String, Listing<String>>): String
/// Reads [query] back into its parameters.
///
/// [query] is read as `application/x-www-form-urlencoded`.
/// A name that repeats holds the values of all its occurrences, in the order they are written.
/// A parameter that states no `=` holds no value at all, and is left out.
///
/// Throws if any `%` in [query] does not begin a percent-encoded octet.
///
/// Facts:
/// ```
/// net.parseQuery("a=1&a=2") == new Mapping { ["a"] = new Listing { "1"; "2" } }
/// net.parseQuery("q=a+b&bare") == new Mapping { ["q"] = new Listing { "a b" } }
/// ```
external const function parseQuery(query: String): Mapping<String, Listing<String>>