11 Commits
main ... 0.29.1

Author SHA1 Message Date
Jen Basch
6a3722a471 Prepare 0.29.1 release 2025-08-27 13:46:31 -07:00
Daniel Chao
2a7195125c Prepare 0.29.1 release (#1191) 2025-08-27 13:32:08 -07:00
Artem Yarmoliuk
f0896ba16f Fix shell completion for paths (#1161) 2025-08-26 10:50:08 -07:00
Daniel Chao
9f7997bbc4 Fix missing resources in native pkldoc, and disable test mode (#1175)
This fixes two issues:

1. Test mode is enabled in pkldoc without the ability to turn it off
2. Native pkldoc is missing required resources

This also adds tests for both `jpkldoc` and `pkldoc`.
2025-08-26 10:49:56 -07:00
Daniel Chao
245b53bc8a Add docs for installing via winget (#1171) 2025-08-26 10:49:45 -07:00
Daniel Chao
9a92d75fcb Fix escaping in yaml strings (#1165)
The backslash needs to be escaped when rendering double-quoted YAML strings.

In addition, this escapes the following characters:

* next line (0x85)
* nbsp (0xa0)
2025-08-26 10:49:39 -07:00
Daniel Chao
419127fb0d Fix encoding for mapping with local members (#1152)
Fixes an issue where local members are included in
the mapping entry count.

Also: avoid re-computing Mapping.length
2025-08-26 10:49:20 -07:00
Jen Basch
f7951510b9 Correctly handle EOF after unmatched backtick (#1187) 2025-08-25 14:35:12 -07:00
Daniel Chao
1b49ec9422 Fix download links (#1162)
* Snapshot repo has changed
* Fix a bug where pkl-codegen-java download link points to Sonatype instead of GitHub

Not in the PR: we also need to fix the snapshot download location;
but haven't figured out yet what the correct link is.
2025-07-31 08:44:22 -07:00
Dan Chao
bdabcea216 Add link to 0.29 release notes 2025-07-24 10:55:05 -07:00
Islon Scherer
5f00f9c82e Prepare 0.29.0 release 2025-07-24 19:07:40 +02:00
64 changed files with 854 additions and 270 deletions

View File

@@ -1,6 +1,6 @@
name: main
title: Main Project
version: 0.29.0-dev
prerelease: true
version: 0.29.1
prerelease: false
nav:
- nav.adoc

View File

@@ -3,10 +3,10 @@
// the following attributes must be updated immediately before a release
// pkl version corresponding to current git commit without -dev suffix or git hash
:pkl-version-no-suffix: 0.29.0
:pkl-version-no-suffix: 0.29.1
// tells whether pkl version corresponding to current git commit
// is a release version (:is-release-version: '') or dev version (:!is-release-version:)
:!is-release-version:
:is-release-version: ''
// the remaining attributes do not need to be updated regularly
@@ -23,9 +23,9 @@ endif::[]
:uri-maven-docsite: https://central.sonatype.com
:uri-sonatype: https://s01.oss.sonatype.org/content/groups/public
:uri-snapshot-repo: https://central.sonatype.com/repository/maven-snapshots
:uri-maven-repo: https://s01.oss.sonatype.org/content/groups/public
:uri-maven-repo: https://central.sonatype.com/repository/maven-snapshots
ifdef::is-release-version[]
:uri-maven-repo: https://repo1.maven.org/maven2
endif::[]
@@ -150,4 +150,5 @@ endif::[]
:uri-pkl-roadmap: https://github.com/orgs/apple/projects/12/views/1
// TODO: figure out what the correct URL should be
:uri-sonatype-snapshot-download: https://s01.oss.sonatype.org/service/local/artifact/maven/redirect?r=snapshots&g=org.pkl-lang&v={pkl-artifact-version}

View File

@@ -5,7 +5,7 @@ include::ROOT:partial$component-attributes.adoc[]
:uri-pkl-codegen-java-download: {uri-sonatype-snapshot-download}&a=pkl-cli-codegen-java&e=jar
ifdef::is-release-version[]
:uri-pkl-cli-codegen-java-download: {github-releases}/pkl-codegen-java
:uri-pkl-codegen-java-download: {github-releases}/pkl-codegen-java
endif::[]
The Java source code generator takes Pkl class definitions as an input, and generates corresponding Java classes with equally named properties.
@@ -33,7 +33,7 @@ The `pkl-codegen-java` library is available {uri-pkl-codegen-java-maven-module}[
It requires Java 17 or higher.
ifndef::is-release-version[]
NOTE: Snapshots are published to repository `{uri-sonatype}`.
NOTE: Snapshots are published to repository `{uri-snapshot-repo}`.
endif::[]
==== Gradle

View File

@@ -2,6 +2,7 @@
include::ROOT:partial$component-attributes.adoc[]
:uri-homebrew: https://brew.sh
:uri-mise: https://mise.jdx.dev
:uri-winget: https://learn.microsoft.com/en-us/windows/package-manager/
:uri-pkl-macos-amd64-download: {uri-sonatype-snapshot-download}&a=pkl-cli-macos-amd64&e=bin
:uri-pkl-macos-aarch64-download: {uri-sonatype-snapshot-download}&a=pkl-cli-macos-aarch64&e=bin
@@ -107,6 +108,31 @@ ifndef::is-release-version[]
For instructions, switch to a release version of this page.
endif::[]
[[winget]]
=== Windows Package Manager
On Windows, release versions can be installed with {uri-winget}[Windows Package Manager].
ifdef::is-release-version[]
To install Pkl, run:
[source,shell]
----
winget install Apple.Pkl
----
To update Pkl, run:
[source,shell]
----
winget upgrade Apple.Pkl
----
endif::[]
ifndef::is-release-version[]
For instructions, switch to a release version of this page.
endif::[]
[[download]]
=== Download

View File

@@ -94,7 +94,7 @@ The `pkl-doc` library is available {uri-pkl-doc-maven}[from Maven Central].
It requires Java 17 or higher.
ifndef::is-release-version[]
NOTE: Snapshots are published to repository `{uri-sonatype}`.
NOTE: Snapshots are published to repository `{uri-snapshot-repo}`.
endif::[]
==== Gradle

View File

@@ -25,7 +25,7 @@ It requires Java 17 or higher and Gradle 8.1 or higher.
Earlier Gradle versions are not supported.
ifndef::is-release-version[]
NOTE: Snapshots are published to repository `{uri-sonatype}`.
NOTE: Snapshots are published to repository `{uri-snapshot-repo}`.
endif::[]
The plugin is applied as follows:

View File

@@ -1,6 +1,6 @@
= Pkl 0.29 Release Notes
:version: 0.29
:version-minor: 0.29.0
:version-minor: 0.29.1
:release-date: July 24th, 2025
include::ROOT:partial$component-attributes.adoc[]

View File

@@ -1,6 +1,26 @@
= Changelog
include::ROOT:partial$component-attributes.adoc[]
[[release-0.29.1]]
== 0.29.1 (2025-08-27)
=== Fixes
* Fixes an issue where autocompletion in Bash and ZSH do noes not suggest filenames (https://github.com/apple/pkl/pull/1161[#1161]).
* Fixes an issue where `pkldoc` throws a runtime error about failing to load class path resources (https://github.com/apple/pkl/issues/1174[#1174]).
* Fixes an issue where `pkldoc` always runs with `testMode` set to true.
* Fixes an issue where evaluating a module that ends with an unmatched backtick throws `ArrayIndexOutOfBoundsException` (https://github.com/apple/pkl/issues/1182[#1182]).
* Fixes the formatting of YAML strings when emitting backslash characters within quoted strings (https://github.com/apple/pkl/pull/1165[#1165]).
* Fixes an issue where `local` members inside `Mapping` objects are incorrectly encoded into binary format (https://github.com/apple/pkl/issues/1151[#1151]).
=== Contributors ❤️
Thank you to all the contributors for this release!
* https://github.com/bioball[@bioball]
* https://github.com/gordonbondon[@gordonbondon]
* https://github.com/HT154[@HT154]
[[release-0.29.0]]
== 0.29.0 (2025-07-24)

View File

@@ -2,6 +2,7 @@
The Pkl team aims to release a new version of Pkl in February, June, and October of each year.
* xref:0.29.adoc[0.29 Release Notes]
* xref:0.28.adoc[0.28 Release Notes]
* xref:0.27.adoc[0.27 Release Notes]
* xref:0.26.adoc[0.26 Release Notes]

View File

@@ -1,7 +1,7 @@
# suppress inspection "UnusedProperty" for whole file
group=org.pkl-lang
version=0.29.0
version=0.29.1
# google-java-format requires jdk.compiler exports
org.gradle.jvmargs= \

View File

@@ -2,15 +2,15 @@
"catalogs": {},
"aliases": {
"pkl": {
"script-ref": "org.pkl-lang:pkl-cli-java:0.28.2",
"script-ref": "org.pkl-lang:pkl-cli-java:0.29.1",
"java-agents": []
},
"pkl-codegen-java": {
"script-ref": "org.pkl-lang:pkl-codegen-java:0.28.2",
"script-ref": "org.pkl-lang:pkl-codegen-java:0.29.1",
"java-agents": []
},
"pkl-codegen-kotlin": {
"script-ref": "org.pkl-lang:pkl-codegen-kotlin:0.28.2",
"script-ref": "org.pkl-lang:pkl-codegen-kotlin:0.29.1",
"java-agents": []
}
},

View File

@@ -15,6 +15,7 @@
*/
package org.pkl.cli.commands
import com.github.ajalt.clikt.completion.CompletionCandidates
import com.github.ajalt.clikt.parameters.options.default
import com.github.ajalt.clikt.parameters.options.flag
import com.github.ajalt.clikt.parameters.options.option
@@ -33,6 +34,7 @@ class EvalCommand : ModulesCommand(name = "eval", helpLink = helpLink) {
names = arrayOf("-o", "--output-path"),
metavar = "path",
help = "File path where the output file is placed.",
completionCandidates = CompletionCandidates.Path,
)
.single()
@@ -59,6 +61,7 @@ class EvalCommand : ModulesCommand(name = "eval", helpLink = helpLink) {
names = arrayOf("-m", "--multiple-file-output-path"),
metavar = "path",
help = "Directory where a module's multiple file output is placed.",
completionCandidates = CompletionCandidates.Path,
)
.single()
.validate {

View File

@@ -15,6 +15,7 @@
*/
package org.pkl.cli.commands
import com.github.ajalt.clikt.completion.CompletionCandidates
import com.github.ajalt.clikt.core.Context
import com.github.ajalt.clikt.core.NoOpCliktCommand
import com.github.ajalt.clikt.core.subcommands
@@ -116,6 +117,7 @@ class PackageCommand : BaseCommand(name = "package", helpLink = helpLink) {
names = arrayOf("--output-path"),
help = "The directory to write artifacts to",
metavar = "path",
completionCandidates = CompletionCandidates.Path,
)
.single()
.default(".out/%{name}@%{version}")

View File

@@ -15,6 +15,7 @@
*/
package org.pkl.cli.commands
import com.github.ajalt.clikt.completion.CompletionCandidates
import com.github.ajalt.clikt.parameters.arguments.argument
import com.github.ajalt.clikt.parameters.arguments.convert
import com.github.ajalt.clikt.parameters.arguments.multiple
@@ -30,7 +31,11 @@ class TestCommand : BaseCommand(name = "test", helpLink = helpLink) {
override val helpString = "Run tests within the given module(s)"
val modules: List<URI> by
argument(name = "modules", help = "Module paths or URIs to evaluate.")
argument(
name = "modules",
help = "Module paths or URIs to evaluate.",
completionCandidates = CompletionCandidates.Path,
)
.convert { BaseOptions.parseModuleName(it) }
.multiple()

View File

@@ -1,5 +1,5 @@
/*
* Copyright © 2024 Apple Inc. and the Pkl project authors. All rights reserved.
* Copyright © 2024-2025 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -64,4 +64,11 @@ class ReplMessagesTest {
startIndex = examples.indexOf("```", endIndex + 3)
}
}
@Test
fun `handle single backtick`() {
val responses = server.handleRequest(ReplRequest.Eval("1", "`", true, true))
assertThat(responses.size).isEqualTo(1)
assertThat(responses).hasOnlyElementsOfType(ReplResponse.EvalError::class.java)
}
}

View File

@@ -15,6 +15,7 @@
*/
package org.pkl.commons.cli.commands
import com.github.ajalt.clikt.completion.CompletionCandidates
import com.github.ajalt.clikt.parameters.groups.OptionGroup
import com.github.ajalt.clikt.parameters.options.*
import com.github.ajalt.clikt.parameters.types.enum
@@ -165,6 +166,7 @@ class BaseOptions : OptionGroup() {
option(
names = arrayOf("-f", "--format"),
help = "Output format to generate. <${output.joinToString()}>",
completionCandidates = CompletionCandidates.Fixed(output.toSet()),
)
.single()
@@ -187,9 +189,13 @@ class BaseOptions : OptionGroup() {
.splitAll(File.pathSeparator)
val settings: URI? by
option(names = arrayOf("--settings"), help = "Pkl settings module to use.").single().convert {
parseModuleName(it)
}
option(
names = arrayOf("--settings"),
help = "Pkl settings module to use.",
completionCandidates = CompletionCandidates.Path,
)
.single()
.convert { parseModuleName(it) }
val timeout: Duration? by
option(

View File

@@ -15,6 +15,7 @@
*/
package org.pkl.commons.cli.commands
import com.github.ajalt.clikt.completion.CompletionCandidates
import com.github.ajalt.clikt.parameters.arguments.argument
import com.github.ajalt.clikt.parameters.arguments.convert
import com.github.ajalt.clikt.parameters.arguments.multiple
@@ -24,7 +25,11 @@ import java.net.URI
abstract class ModulesCommand(name: String, helpLink: String) :
BaseCommand(name = name, helpLink = helpLink) {
open val modules: List<URI> by
argument(name = "modules", help = "Module paths or URIs to evaluate.")
argument(
name = "modules",
help = "Module paths or URIs to evaluate.",
completionCandidates = CompletionCandidates.Path,
)
.convert { BaseOptions.parseModuleName(it) }
.multiple(required = true)

View File

@@ -0,0 +1,85 @@
/*
* Copyright © 2024-2025 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.pkl.commons.test
import java.nio.file.Files
import java.nio.file.Path
import kotlin.io.path.exists
import org.pkl.commons.test.FileTestUtils.rootProjectDir
sealed class ExecutablePaths(protected val gradleProject: String) {
abstract val allNative: List<Path>
val existingNative: List<Path>
get() = allNative.filter(Files::exists)
val firstExistingNative: Path
get() =
existingNative.firstOrNull()
?: throw AssertionError(
"Native executable not found on system. " +
"To fix this problem, run `./gradlew $gradleProject:assembleNative`."
)
protected fun executable(name: String): Path =
rootProjectDir.resolve("$gradleProject/build/executable").resolve(name)
protected fun javaExecutable(name: String): Path {
val isWindows = System.getProperty("os.name").startsWith("Windows")
val effectiveName = if (isWindows) "$name.bat" else name
return rootProjectDir.resolve("$gradleProject/build/executable").resolve(effectiveName).also {
path ->
if (!path.exists()) {
throw AssertionError(
"Java executable not found on system. " +
"To fix this problem, run `./gradlew $gradleProject:javaExecutable`."
)
}
}
}
}
@Suppress("ClassName")
object Executables {
object pkl : ExecutablePaths("pkl-cli") {
val macAarch64: Path = executable("pkl-macos-aarch64")
val macAmd64: Path = executable("pkl-macos-amd64")
val linuxAarch64: Path = executable("pkl-linux-aarch64")
val linuxAmd64: Path = executable("pkl-linux-amd64")
val alpineAmd64: Path = executable("pkl-alpine-linux-amd64")
val windowsAmd64: Path = executable("pkl-windows-amd64.exe")
// order (aarch64 before amd64, linux before alpine) affects [firstExisting]
override val allNative: List<Path> =
listOf(macAarch64, macAmd64, linuxAarch64, linuxAmd64, alpineAmd64, windowsAmd64)
}
object pkldoc : ExecutablePaths("pkl-doc") {
val macAarch64: Path = executable("pkldoc-macos-aarch64")
val macAmd64: Path = executable("pkldoc-macos-amd64")
val linuxAarch64: Path = executable("pkldoc-linux-aarch64")
val linuxAmd64: Path = executable("pkldoc-linux-amd64")
val alpineAmd64: Path = executable("pkldoc-alpine-linux-amd64")
val windowsAmd64: Path = executable("pkldoc-windows-amd64.exe")
val javaExecutable: Path by lazy { javaExecutable("jpkldoc") }
// order (aarch64 before amd64, linux before alpine) affects [firstExisting]
override val allNative: List<Path> =
listOf(macAarch64, macAmd64, linuxAarch64, linuxAmd64, alpineAmd64, windowsAmd64)
}
}

View File

@@ -1,47 +0,0 @@
/*
* Copyright © 2024 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.pkl.commons.test
import java.nio.file.Files
import java.nio.file.Path
import org.pkl.commons.test.FileTestUtils.rootProjectDir
object PklExecutablePaths {
val macAarch64: Path = executablePath("pkl-macos-aarch64")
val macAmd64: Path = executablePath("pkl-macos-amd64")
val linuxAarch64: Path = executablePath("pkl-linux-aarch64")
val linuxAmd64: Path = executablePath("pkl-linux-amd64")
val alpineAmd64: Path = executablePath("pkl-alpine-linux-amd64")
val windowsAmd64: Path = executablePath("pkl-windows-amd64.exe")
// order (aarch64 before amd64, linux before alpine) affects [firstExisting]
val all: List<Path> =
listOf(macAarch64, macAmd64, linuxAarch64, linuxAmd64, alpineAmd64, windowsAmd64)
val existing: List<Path>
get() = all.filter(Files::exists)
val firstExisting: Path
get() =
existing.firstOrNull()
?: throw AssertionError(
"Native executable not found on system. " +
"To fix this problem, run `./gradlew assembleNative`."
)
private fun executablePath(name: String): Path =
rootProjectDir.resolve("pkl-cli/build/executable").resolve(name)
}

View File

@@ -33,7 +33,7 @@ import org.pkl.parser.syntax.Module;
@TruffleLanguage.Registration(
id = "pkl",
name = "Pkl",
version = "0.29.0-dev",
version = "0.29.1",
characterMimeTypes = VmLanguage.MIME_TYPE,
contextPolicy = ContextPolicy.SHARED)
public final class VmLanguage extends TruffleLanguage<VmContext> {

View File

@@ -1,5 +1,5 @@
/*
* Copyright © 2024 Apple Inc. and the Pkl project authors. All rights reserved.
* Copyright © 2024-2025 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -17,6 +17,7 @@ package org.pkl.core.runtime;
import com.oracle.truffle.api.CompilerDirectives.TruffleBoundary;
import com.oracle.truffle.api.frame.MaterializedFrame;
import java.util.HashSet;
import java.util.Map;
import javax.annotation.concurrent.GuardedBy;
import org.graalvm.collections.UnmodifiableEconomicMap;
@@ -25,10 +26,10 @@ import org.pkl.core.ast.member.ObjectMember;
import org.pkl.core.util.CollectionUtils;
import org.pkl.core.util.EconomicMaps;
import org.pkl.core.util.LateInit;
import org.pkl.core.util.MutableLong;
public final class VmMapping extends VmListingOrMapping {
private int cachedEntryCount = -1;
private long cachedLength = -1;
@GuardedBy("this")
private @LateInit VmSet __allKeys;
@@ -124,7 +125,7 @@ public final class VmMapping extends VmListingOrMapping {
// could use shallow force, but deep force is cached
force(false);
other.force(false);
if (getEntryCount() != other.getEntryCount()) return false;
if (getLength() != other.getLength()) return false;
var cursor = cachedValues.getEntries();
while (cursor.advance()) {
@@ -162,16 +163,21 @@ public final class VmMapping extends VmListingOrMapping {
return result;
}
// assumes mapping has been forced
public int getEntryCount() {
if (cachedEntryCount != -1) return cachedEntryCount;
var result = 0;
for (var key : cachedValues.getKeys()) {
if (key instanceof Identifier) continue;
result += 1;
}
cachedEntryCount = result;
return result;
@TruffleBoundary
public long getLength() {
if (cachedLength != -1) return cachedLength;
var count = new MutableLong(0);
var visited = new HashSet<>();
iterateMembers(
(key, member) -> {
var alreadyVisited = !visited.add(key);
// important to record hidden member as visited before skipping it
// because any overriding member won't carry a `hidden` identifier
if (alreadyVisited || member.isLocalOrExternalOrHidden()) return true;
count.getAndIncrement();
return true;
});
cachedLength = count.get();
return cachedLength;
}
}

View File

@@ -18,7 +18,6 @@ package org.pkl.core.stdlib.base;
import com.oracle.truffle.api.CompilerDirectives.TruffleBoundary;
import com.oracle.truffle.api.dsl.Specialization;
import com.oracle.truffle.api.nodes.IndirectCallNode;
import java.util.HashSet;
import org.pkl.core.ast.lambda.ApplyVmFunction1Node;
import org.pkl.core.ast.lambda.ApplyVmFunction2Node;
import org.pkl.core.ast.lambda.ApplyVmFunction2NodeGen;
@@ -31,7 +30,6 @@ import org.pkl.core.stdlib.ExternalMethod2Node;
import org.pkl.core.stdlib.ExternalPropertyNode;
import org.pkl.core.util.EconomicMaps;
import org.pkl.core.util.MutableBoolean;
import org.pkl.core.util.MutableLong;
import org.pkl.core.util.MutableReference;
public final class MappingNodes {
@@ -53,20 +51,8 @@ public final class MappingNodes {
public abstract static class length extends ExternalPropertyNode {
@Specialization
@TruffleBoundary
protected long eval(VmMapping self) {
var count = new MutableLong(0);
var visited = new HashSet<>();
self.iterateMembers(
(key, member) -> {
var alreadyVisited = !visited.add(key);
// important to record hidden member as visited before skipping it
// because any overriding member won't carry a `hidden` identifier
if (alreadyVisited || member.isLocalOrExternalOrHidden()) return true;
count.getAndIncrement();
return true;
});
return count.get();
return self.getLength();
}
}

View File

@@ -1,5 +1,5 @@
/*
* Copyright © 2024 Apple Inc. and the Pkl project authors. All rights reserved.
* Copyright © 2024-2025 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -19,33 +19,55 @@ import org.pkl.core.util.AbstractCharEscaper;
import org.pkl.core.util.IoUtils;
import org.pkl.core.util.Nullable;
// https://yaml.org/spec/1.2.2/#57-escaped-characters
/**
* Emits escape sequences for YAML. This is only used when emitting double-quoted strings.
*
* <p>Note: we don't need to escape space ({@code 0x20}) because we don't generate quoted multiline
* strings. We also don't need to escape forward slash ({@code 0x2f}) because a normal forward slash
* is also valid YAML.
*
* @see <a
* href="https://yaml.org/spec/1.2.2/#57-escaped-characters">https://yaml.org/spec/1.2.2/#57-escaped-characters</a>
*/
public final class YamlEscaper extends AbstractCharEscaper {
private static final String[] REPLACEMENTS;
static {
REPLACEMENTS = new String[0x22 + 1];
REPLACEMENTS = new String[0xA0 + 1];
for (var i = 0; i < 0x20; i++) {
REPLACEMENTS[i] = IoUtils.toHexEscape(i);
}
// ns-esc-null
REPLACEMENTS[0x00] = "\\0";
// ns-esc-bell
REPLACEMENTS[0x07] = "\\a";
// ns-esc-backspace
REPLACEMENTS[0x08] = "\\b";
// ns-esc-horizontal-tab
REPLACEMENTS[0x09] = "\\t";
// ns-esc-line-feed
REPLACEMENTS[0x0A] = "\\n";
// ns-esc-vertical-tab
REPLACEMENTS[0x0B] = "\\v";
// ns-esc-form-feed
REPLACEMENTS[0x0C] = "\\f";
// ns-esc-carriage-return
REPLACEMENTS[0x0D] = "\\r";
// ns-esc-escape
REPLACEMENTS[0x1B] = "\\e";
// we don't ever need to escape 0x20 because we don't generate quoted multiline strings
// ns-esc-double-quote
REPLACEMENTS[0x22] = "\\\"";
// ns-esc-backslash
REPLACEMENTS[0x5c] = "\\\\";
// ns-esc-next-line
REPLACEMENTS[0x85] = "\\N";
// ns-esc-non-breaking-space
REPLACEMENTS[0xA0] = "\\_";
}
@Override
protected @Nullable String findReplacement(char ch) {
//noinspection UnnecessaryUnicodeEscape
return ch <= '\u0022'
? REPLACEMENTS[ch]
: ch == '\u2028' ? "\\L" : ch == '\u2029' ? "\\P" : null;
return ch <= 0xA0 ? REPLACEMENTS[ch] : ch == '\u2028' ? "\\L" : ch == '\u2029' ? "\\P" : null;
}
}

View File

@@ -0,0 +1,36 @@
// test escaping in double quotes
// every string is prefixed with `\t` s.t. YamlRenderer will emit double-quoted strings.
`null` = "\t\u{00}"
bell = "\t\u{7}"
backspace = "\t\u{8}"
horizontalTab = "\t"
lineFeed = "\t\u{a}"
verticalTab = "\t\u{b}"
formFeed = "\t\u{c}"
carriageReturn = "\t\r"
escape = "\t\u{1b}"
doubleQuote = "\t\""
backslash = "\t\\"
nextLine = "\t\u{85}"
nbsp = "\t\u{a0}"
lineSep = "\t\u{2028}"
paragraphSep = "\t\u{2029}"
output {
renderer = new YamlRenderer {}
}

View File

@@ -114,12 +114,12 @@ examples {
render(".NAN")
render(".nAn") // never float
}
["tag like strings"] {
"!!bool true"
"!!str my string value"
}
["number like string keys"] {
render(new Dynamic {
`0` = "0"

View File

@@ -0,0 +1 @@
`

View File

@@ -0,0 +1,15 @@
'null': "\t\0"
bell: "\t\a"
backspace: "\t\b"
horizontalTab: "\t"
lineFeed: "\t\n"
verticalTab: "\t\v"
formFeed: "\t\f"
carriageReturn: "\t\r"
escape: "\t\e"
doubleQuote: "\t\""
backslash: "\t\\"
nextLine: "\t\N"
nbsp: "\t\_"
lineSep: "\t\L"
paragraphSep: "\t\P"

View File

@@ -0,0 +1,7 @@
Pkl Error
Unexpected character `
`. Did you mean `backquote`?
x | `
^
at singleBacktick (file:///$snippetsDir/input/errors/singleBacktick.pkl)

View File

@@ -27,10 +27,10 @@ import org.junit.platform.engine.EngineDiscoveryRequest
import org.junit.platform.engine.TestDescriptor
import org.junit.platform.engine.UniqueId
import org.junit.platform.engine.support.descriptor.EngineDescriptor
import org.pkl.commons.test.Executables
import org.pkl.commons.test.FileTestUtils
import org.pkl.commons.test.InputOutputTestEngine
import org.pkl.commons.test.PackageServer
import org.pkl.commons.test.PklExecutablePaths
import org.pkl.core.http.HttpClient
import org.pkl.core.project.Project
import org.pkl.core.util.IoUtils
@@ -298,27 +298,27 @@ abstract class AbstractNativeLanguageSnippetTestsEngine : AbstractLanguageSnippe
}
class MacAmd64LanguageSnippetTestsEngine : AbstractNativeLanguageSnippetTestsEngine() {
override val pklExecutablePath: Path = PklExecutablePaths.macAmd64
override val pklExecutablePath: Path = Executables.pkl.macAmd64
override val testClass: KClass<*> = MacLanguageSnippetTests::class
}
class MacAarch64LanguageSnippetTestsEngine : AbstractNativeLanguageSnippetTestsEngine() {
override val pklExecutablePath: Path = PklExecutablePaths.macAarch64
override val pklExecutablePath: Path = Executables.pkl.macAarch64
override val testClass: KClass<*> = MacLanguageSnippetTests::class
}
class LinuxAmd64LanguageSnippetTestsEngine : AbstractNativeLanguageSnippetTestsEngine() {
override val pklExecutablePath: Path = PklExecutablePaths.linuxAmd64
override val pklExecutablePath: Path = Executables.pkl.linuxAmd64
override val testClass: KClass<*> = LinuxLanguageSnippetTests::class
}
class LinuxAarch64LanguageSnippetTestsEngine : AbstractNativeLanguageSnippetTestsEngine() {
override val pklExecutablePath: Path = PklExecutablePaths.linuxAarch64
override val pklExecutablePath: Path = Executables.pkl.linuxAarch64
override val testClass: KClass<*> = LinuxLanguageSnippetTests::class
}
class AlpineLanguageSnippetTestsEngine : AbstractNativeLanguageSnippetTestsEngine() {
override val pklExecutablePath: Path = PklExecutablePaths.alpineAmd64
override val pklExecutablePath: Path = Executables.pkl.alpineAmd64
override val testClass: KClass<*> = AlpineLanguageSnippetTests::class
}
@@ -340,7 +340,7 @@ private val windowsNativeExcludedTests
)
class WindowsLanguageSnippetTestsEngine : AbstractNativeLanguageSnippetTestsEngine() {
override val pklExecutablePath: Path = PklExecutablePaths.windowsAmd64
override val pklExecutablePath: Path = Executables.pkl.windowsAmd64
override val testClass: KClass<*> = WindowsLanguageSnippetTests::class
override val excludedTests: List<Regex>
get() = super.excludedTests + windowsNativeExcludedTests + windowsExcludedTests

View File

@@ -66,6 +66,31 @@ publishing {
}
}
val testNativeExecutable by
tasks.registering(Test::class) {
inputs.dir("src/test/files/DocGeneratorTest/input")
outputs.dir("src/test/files/DocGeneratorTest/output")
systemProperty("org.pkl.doc.NativeExecutableTest", "true")
include(listOf("**/NativeExecutableTest.class"))
}
val testJavaExecutable by
tasks.registering(Test::class) {
dependsOn(tasks.javaExecutable)
inputs.dir("src/test/files/DocGeneratorTest/input")
outputs.dir("src/test/files/DocGeneratorTest/output")
systemProperty("org.pkl.doc.JavaExecutableTest", "true")
include(listOf("**/JavaExecutableTest.class"))
}
tasks.check { dependsOn(testJavaExecutable) }
tasks.testNative { dependsOn(testNativeExecutable) }
tasks.withType<NativeImageBuild> { extraNativeImageArgs.add("-H:IncludeResources=org/pkl/doc/.*") }
tasks.jar { manifest { attributes += mapOf("Main-Class" to "org.pkl.doc.Main") } }
htmlValidator { sources = files("src/test/files/DocGeneratorTest/output") }
tasks.validateHtml { mustRunAfter(testJavaExecutable) }

View File

@@ -69,6 +69,9 @@ class DocCommand : BaseCommand(name = "pkldoc", helpLink = helpLink) {
.single()
.flag(default = false)
private val isTestMode by
option(names = arrayOf("--test-mode"), help = "Internal test mode", hidden = true).flag()
private val projectOptions by ProjectOptions()
override val helpString: String = "Generate HTML documentation from Pkl modules and packages."
@@ -78,7 +81,7 @@ class DocCommand : BaseCommand(name = "pkldoc", helpLink = helpLink) {
CliDocGeneratorOptions(
baseOptions.baseOptions(modules, projectOptions),
outputDir,
true,
isTestMode,
noSymlinks,
)
CliDocGenerator(options).run()

View File

@@ -18,11 +18,9 @@ package org.pkl.doc
import com.google.common.jimfs.Configuration
import com.google.common.jimfs.Jimfs
import java.net.URI
import java.nio.file.FileSystem
import java.nio.file.Files
import java.nio.file.Path
import kotlin.io.path.*
import org.assertj.core.api.Assertions
import org.assertj.core.api.Assertions.assertThat
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.assertThrows
@@ -31,83 +29,20 @@ import org.junit.jupiter.params.ParameterizedTest
import org.junit.jupiter.params.provider.MethodSource
import org.pkl.commons.cli.CliBaseOptions
import org.pkl.commons.cli.CliException
import org.pkl.commons.readString
import org.pkl.commons.test.FileTestUtils
import org.pkl.commons.test.PackageServer
import org.pkl.commons.test.listFilesRecursively
import org.pkl.commons.toPath
import org.pkl.commons.walk
import org.pkl.core.Version
import org.pkl.core.util.IoUtils
import org.pkl.doc.DocGenerator.Companion.current
class CliDocGeneratorTest {
companion object {
private val tempFileSystem: FileSystem by lazy { Jimfs.newFileSystem(Configuration.unix()) }
private val tempFileSystem by lazy { Jimfs.newFileSystem(Configuration.unix()) }
private val tmpOutputDir by lazy {
private val tmpOutputDir: Path by lazy {
tempFileSystem.getPath("/work/output").apply { createDirectories() }
}
private val projectDir = FileTestUtils.rootProjectDir.resolve("pkl-doc")
private val inputDir: Path by lazy {
projectDir.resolve("src/test/files/DocGeneratorTest/input").apply { assert(exists()) }
}
private val docsiteModule: URI by lazy {
inputDir.resolve("docsite-info.pkl").apply { assert(exists()) }.toUri()
}
internal val package1PackageModule: URI by lazy {
inputDir.resolve("com.package1/doc-package-info.pkl").apply { assert(exists()) }.toUri()
}
private val package2PackageModule: URI by lazy {
inputDir.resolve("com.package2/doc-package-info.pkl").apply { assert(exists()) }.toUri()
}
internal val package1InputModules: List<URI> by lazy {
inputDir
.resolve("com.package1")
.listFilesRecursively()
.filter { it.fileName.toString() != "doc-package-info.pkl" }
.map { it.toUri() }
}
private val package2InputModules: List<URI> by lazy {
inputDir
.resolve("com.package2")
.listFilesRecursively()
.filter { it.fileName.toString() != "doc-package-info.pkl" }
.map { it.toUri() }
}
private val expectedOutputDir: Path by lazy {
projectDir.resolve("src/test/files/DocGeneratorTest/output").createDirectories()
}
private val expectedOutputFiles: List<Path> by lazy { expectedOutputDir.listFilesRecursively() }
private val actualOutputDir: Path by lazy { tempFileSystem.getPath("/work/DocGeneratorTest") }
private val actualOutputFiles: List<Path> by lazy { actualOutputDir.listFilesRecursively() }
private val expectedRelativeOutputFiles: List<String> by lazy {
expectedOutputFiles.map { path ->
IoUtils.toNormalizedPathString(expectedOutputDir.relativize(path)).let { str ->
// Git will by default clone symlinks as shortcuts on Windows, and shortcuts have a
// `.lnk` extension.
if (IoUtils.isWindows() && str.endsWith(".lnk")) str.dropLast(4) else str
}
}
}
private val actualRelativeOutputFiles: List<String> by lazy {
actualOutputFiles.map { IoUtils.toNormalizedPathString(actualOutputDir.relativize(it)) }
}
private val binaryFileExtensions = setOf("woff2", "png", "svg")
private val helper = DocGeneratorTestHelper()
private fun runDocGenerator(outputDir: Path, cacheDir: Path?, noSymlinks: Boolean = false) {
CliDocGenerator(
@@ -115,14 +50,14 @@ class CliDocGeneratorTest {
CliBaseOptions(
sourceModules =
listOf(
docsiteModule,
package1PackageModule,
package2PackageModule,
helper.docsiteModule,
helper.package1PackageModule,
helper.package2PackageModule,
URI("package://localhost:0/birds@0.5.0"),
URI("package://localhost:0/fruit@1.1.0"),
URI("package://localhost:0/unlisted@1.0.0"),
URI("package://localhost:0/deprecated@1.0.0"),
) + package1InputModules + package2InputModules,
) + helper.package1InputModules + helper.package2InputModules,
moduleCacheDir = cacheDir,
),
outputDir = outputDir,
@@ -135,19 +70,7 @@ class CliDocGeneratorTest {
@JvmStatic
private fun generateDocs(): List<String> {
val cacheDir = Files.createTempDirectory("cli-doc-generator-test-cache")
PackageServer.populateCacheDir(cacheDir)
runDocGenerator(actualOutputDir, cacheDir)
val missingFiles = expectedRelativeOutputFiles - actualRelativeOutputFiles.toSet()
if (missingFiles.isNotEmpty()) {
Assertions.fail<Unit>(
"The following expected files were not actually generated:\n" +
missingFiles.joinToString("\n")
)
}
return actualRelativeOutputFiles
return helper.generateDocs()
}
}
@@ -158,6 +81,7 @@ class CliDocGeneratorTest {
createParentDirectories()
createFile()
}
val descriptor2 =
tempFileSystem.getPath("/work/dir2/docsite-info.pkl").apply {
createParentDirectories()
@@ -220,49 +144,20 @@ class CliDocGeneratorTest {
@ParameterizedTest
@MethodSource("generateDocs")
fun test(relativeFilePath: String) {
val actualFile = actualOutputDir.resolve(relativeFilePath)
assertThat(actualFile)
.withFailMessage("Test bug: $actualFile should exist but does not.")
.exists()
// symlinks on Git and Windows is rather finnicky; they create shortcuts by default unless
// a core Git option is set. Also, by default, symlinks require administrator privileges to run.
// We'll just test that the symlink got created but skip verifying that it points to the right
// place.
if (actualFile.isSymbolicLink() && IoUtils.isWindows()) return
val expectedFile = expectedOutputDir.resolve(relativeFilePath)
if (expectedFile.exists()) {
when {
expectedFile.isSymbolicLink() -> {
assertThat(actualFile).isSymbolicLink
assertThat(expectedFile.readSymbolicLink().toString().toPath())
.isEqualTo(actualFile.readSymbolicLink().toString().toPath())
}
expectedFile.extension in binaryFileExtensions ->
assertThat(actualFile.readBytes()).isEqualTo(expectedFile.readBytes())
else -> assertThat(actualFile.readString()).isEqualTo(expectedFile.readString())
}
} else {
expectedFile.createParentDirectories()
if (actualFile.isSymbolicLink()) {
// needs special handling because `copyTo` can't copy symlinks between file systems
val linkTarget = actualFile.readSymbolicLink()
assertThat(linkTarget).isRelative
Files.createSymbolicLink(expectedFile, linkTarget.toString().toPath())
} else {
actualFile.copyTo(expectedFile)
}
Assertions.fail("Created missing expected file `$relativeFilePath`.")
}
DocTestUtils.testExpectedFile(
helper.expectedOutputDir,
helper.actualOutputDir,
relativeFilePath,
)
}
@Test
fun `creates a symlink called current by default`(@TempDir tempDir: Path) {
PackageServer.populateCacheDir(tempDir)
runDocGenerator(actualOutputDir, tempDir)
runDocGenerator(helper.actualOutputDir, tempDir)
val expectedSymlink = actualOutputDir.resolve("com.package1/current")
val expectedDestination = actualOutputDir.resolve("com.package1/1.2.3")
val expectedSymlink = helper.actualOutputDir.resolve("com.package1/current")
val expectedDestination = helper.actualOutputDir.resolve("com.package1/1.2.3")
assertThat(expectedSymlink).isSymbolicLink().matches {
Files.isSameFile(it, expectedDestination)
@@ -274,10 +169,10 @@ class CliDocGeneratorTest {
@TempDir tempDir: Path
) {
PackageServer.populateCacheDir(tempDir)
runDocGenerator(actualOutputDir, tempDir, noSymlinks = true)
runDocGenerator(helper.actualOutputDir, tempDir, noSymlinks = true)
val currentDirectory = actualOutputDir.resolve("com.package1/current")
val sourceDirectory = actualOutputDir.resolve("com.package1/1.2.3")
val currentDirectory = helper.actualOutputDir.resolve("com.package1/current")
val sourceDirectory = helper.actualOutputDir.resolve("com.package1/1.2.3")
assertThat(currentDirectory).isDirectory()
assertThat(currentDirectory.isSymbolicLink()).isFalse()

View File

@@ -0,0 +1,181 @@
/*
* Copyright © 2025 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.pkl.doc
import java.net.URI
import java.nio.file.Files
import java.nio.file.Path
import kotlin.io.path.createDirectories
import kotlin.io.path.exists
import org.assertj.core.api.Assertions
import org.junit.jupiter.api.fail
import org.pkl.commons.cli.CliBaseOptions
import org.pkl.commons.test.FileTestUtils
import org.pkl.commons.test.PackageServer
import org.pkl.commons.test.listFilesRecursively
import org.pkl.core.util.IoUtils
class DocGeneratorTestHelper {
internal val tempDir by lazy { Files.createTempDirectory("ExecutableCliDocGeneratorTest") }
internal val projectDir = FileTestUtils.rootProjectDir.resolve("pkl-doc")
internal val inputDir: Path by lazy {
projectDir.resolve("src/test/files/DocGeneratorTest/input").apply { assert(exists()) }
}
internal val docsiteModule: URI by lazy {
inputDir.resolve("docsite-info.pkl").apply { assert(exists()) }.toUri()
}
internal val package1PackageModule: URI by lazy {
inputDir.resolve("com.package1/doc-package-info.pkl").apply { assert(exists()) }.toUri()
}
internal val package2PackageModule: URI by lazy {
inputDir.resolve("com.package2/doc-package-info.pkl").apply { assert(exists()) }.toUri()
}
internal val package1InputModules: List<URI> by lazy {
inputDir
.resolve("com.package1")
.listFilesRecursively()
.filter { it.fileName.toString() != "doc-package-info.pkl" }
.map { it.toUri() }
}
internal val package2InputModules: List<URI> by lazy {
inputDir
.resolve("com.package2")
.listFilesRecursively()
.filter { it.fileName.toString() != "doc-package-info.pkl" }
.map { it.toUri() }
}
internal val expectedOutputDir: Path by lazy {
projectDir.resolve("src/test/files/DocGeneratorTest/output").createDirectories()
}
internal val expectedOutputFiles: List<Path> by lazy { expectedOutputDir.listFilesRecursively() }
internal val actualOutputDir: Path by lazy {
tempDir.resolve("work/DocGeneratorTest").createDirectories()
}
internal val actualOutputFiles: List<Path> by lazy { actualOutputDir.listFilesRecursively() }
internal val cacheDir: Path by lazy { tempDir.resolve("cache") }
internal val sourceModules =
listOf(
docsiteModule,
package1PackageModule,
package2PackageModule,
URI("package://localhost:0/birds@0.5.0"),
URI("package://localhost:0/fruit@1.1.0"),
URI("package://localhost:0/unlisted@1.0.0"),
URI("package://localhost:0/deprecated@1.0.0"),
) + package1InputModules + package2InputModules
internal val expectedRelativeOutputFiles: List<String> by lazy {
expectedOutputFiles.map { path ->
IoUtils.toNormalizedPathString(expectedOutputDir.relativize(path)).let { str ->
// Git will by default clone symlinks as shortcuts on Windows, and shortcuts have a
// `.lnk` extension.
if (IoUtils.isWindows() && str.endsWith(".lnk")) str.dropLast(4) else str
}
}
}
internal val actualRelativeOutputFiles: List<String> by lazy {
actualOutputFiles.map { IoUtils.toNormalizedPathString(actualOutputDir.relativize(it)) }
}
fun runPklDocCli(executable: Path, options: CliDocGeneratorOptions) {
val command = buildList {
add(executable.toString())
add("--output-dir")
add(options.normalizedOutputDir.toString())
add("--cache-dir")
add(options.base.normalizedModuleCacheDir.toString())
add("--test-mode")
addAll(sourceModules.map { it.toString() })
}
val process =
with(ProcessBuilder(command)) {
redirectErrorStream(true)
start()
}
try {
val out = process.inputStream.reader().readText()
val exitCode = process.waitFor()
if (exitCode != 0) {
fail(
"""
Process exited with $exitCode.
Output:
"""
.trimIndent() + out
)
}
} finally {
process.destroy()
}
}
private fun generateDocsWith(doGenerate: (CliDocGeneratorOptions) -> Unit): List<String> {
PackageServer.populateCacheDir(cacheDir)
val options =
CliDocGeneratorOptions(
CliBaseOptions(
sourceModules =
listOf(
docsiteModule,
package1PackageModule,
package2PackageModule,
URI("package://localhost:0/birds@0.5.0"),
URI("package://localhost:0/fruit@1.1.0"),
URI("package://localhost:0/unlisted@1.0.0"),
URI("package://localhost:0/deprecated@1.0.0"),
) + package1InputModules + package2InputModules,
moduleCacheDir = cacheDir,
),
outputDir = actualOutputDir,
isTestMode = true,
noSymlinks = false,
)
doGenerate(options)
val missingFiles = expectedRelativeOutputFiles - actualRelativeOutputFiles.toSet()
if (missingFiles.isNotEmpty()) {
Assertions.fail<Unit>(
"The following expected files were not actually generated:\n" +
missingFiles.joinToString("\n")
)
}
return actualRelativeOutputFiles
}
fun generateDocsWithCli(executable: Path): List<String> {
return generateDocsWith { runPklDocCli(executable, it) }
}
fun generateDocs(): List<String> {
return generateDocsWith { CliDocGenerator(it).run() }
}
}

View File

@@ -0,0 +1,73 @@
/*
* Copyright © 2025 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.pkl.doc
import java.nio.file.Files
import java.nio.file.Path
import kotlin.io.path.copyTo
import kotlin.io.path.createParentDirectories
import kotlin.io.path.exists
import kotlin.io.path.extension
import kotlin.io.path.isSymbolicLink
import kotlin.io.path.readBytes
import kotlin.io.path.readSymbolicLink
import org.assertj.core.api.Assertions
import org.assertj.core.api.Assertions.assertThat
import org.pkl.commons.readString
import org.pkl.commons.toPath
import org.pkl.core.util.IoUtils
object DocTestUtils {
private val binaryFileExtensions = setOf("woff2", "png", "svg")
fun testExpectedFile(expectedOutputDir: Path, actualOutputDir: Path, relativeFilePath: String) {
val actualFile = actualOutputDir.resolve(relativeFilePath)
assertThat(actualFile)
.withFailMessage("Test bug: $actualFile should exist but does not.")
.exists()
// symlinks on Git and Windows is rather finnicky; they create shortcuts by default unless
// a core Git option is set. Also, by default, symlinks require administrator privileges to run.
// We'll just test that the symlink got created but skip verifying that it points to the right
// place.
if (actualFile.isSymbolicLink() && IoUtils.isWindows()) return
val expectedFile = expectedOutputDir.resolve(relativeFilePath)
if (expectedFile.exists()) {
when {
expectedFile.isSymbolicLink() -> {
assertThat(actualFile).isSymbolicLink
assertThat(expectedFile.readSymbolicLink().toString().toPath())
.isEqualTo(actualFile.readSymbolicLink().toString().toPath())
}
expectedFile.extension in binaryFileExtensions ->
assertThat(actualFile.readBytes()).isEqualTo(expectedFile.readBytes())
else -> assertThat(actualFile.readString()).isEqualTo(expectedFile.readString())
}
} else {
expectedFile.createParentDirectories()
if (actualFile.isSymbolicLink()) {
// needs special handling because `copyTo` can't copy symlinks between file systems
val linkTarget = actualFile.readSymbolicLink()
assertThat(linkTarget).isRelative
Files.createSymbolicLink(expectedFile, linkTarget.toString().toPath())
} else {
actualFile.copyTo(expectedFile)
}
Assertions.fail("Created missing expected file `$relativeFilePath`.")
}
}
}

View File

@@ -0,0 +1,41 @@
/*
* Copyright © 2025 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.pkl.doc
import org.junit.jupiter.api.condition.DisabledIfSystemProperty
import org.junit.jupiter.api.condition.EnabledIfSystemProperty
import org.junit.jupiter.params.ParameterizedTest
import org.junit.jupiter.params.provider.MethodSource
import org.pkl.commons.test.Executables
// need both annotations for this to work (see https://stackoverflow.com/a/63252081)
@EnabledIfSystemProperty(named = "org.pkl.doc.JavaExecutableTest", matches = "true")
@DisabledIfSystemProperty(named = "org.pkl.doc.JavaExecutableTest", matches = "(?!true)")
class JavaExecutableTest {
companion object {
val helper = DocGeneratorTestHelper()
@JvmStatic
private fun generateDocs(): List<String> =
helper.generateDocsWithCli(Executables.pkldoc.javaExecutable)
}
@ParameterizedTest()
@MethodSource("generateDocs")
fun test(relativePath: String) {
DocTestUtils.testExpectedFile(helper.expectedOutputDir, helper.actualOutputDir, relativePath)
}
}

View File

@@ -0,0 +1,42 @@
/*
* Copyright © 2025 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.pkl.doc
import org.junit.jupiter.api.condition.DisabledIfSystemProperty
import org.junit.jupiter.api.condition.EnabledIfSystemProperty
import org.junit.jupiter.params.ParameterizedTest
import org.junit.jupiter.params.provider.MethodSource
import org.pkl.commons.test.Executables
// need both annotations for this to work (see https://stackoverflow.com/a/63252081)
@EnabledIfSystemProperty(named = "org.pkl.doc.NativeExecutableTest", matches = "true")
@DisabledIfSystemProperty(named = "org.pkl.doc.NativeExecutableTest", matches = "(?!true)")
class NativeExecutableTest {
companion object {
val helper = DocGeneratorTestHelper()
@JvmStatic
private fun generateDocs(): List<String> {
return helper.generateDocsWithCli(Executables.pkldoc.firstExistingNative)
}
}
@ParameterizedTest()
@MethodSource("generateDocs")
fun test(relativePath: String) {
DocTestUtils.testExpectedFile(helper.expectedOutputDir, helper.actualOutputDir, relativePath)
}
}

View File

@@ -26,17 +26,17 @@ import org.junit.jupiter.api.AfterAll
import org.junit.jupiter.api.Test
import org.pkl.commons.cli.CliBaseOptions
import org.pkl.commons.readString
import org.pkl.doc.CliDocGeneratorTest.Companion.package1InputModules
import org.pkl.doc.CliDocGeneratorTest.Companion.package1PackageModule
class SearchTest {
companion object {
private val tempFileSystem = lazy { Jimfs.newFileSystem(Configuration.unix()) }
private val helper = DocGeneratorTestHelper()
private val jsContext = lazy {
// reuse CliDocGeneratorTest's input files (src/test/files/DocGeneratorTest/input)
val packageModule: URI = package1PackageModule
val inputModules: List<URI> = package1InputModules
val packageModule: URI = helper.package1PackageModule
val inputModules: List<URI> = helper.package1InputModules
val pkldocDir = tempFileSystem.value.rootDirectories.first()

View File

@@ -0,0 +1,93 @@
/*
* Copyright © 2025 Apple Inc. and the Pkl project authors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.pkl.doc
import com.google.common.jimfs.Configuration
import com.google.common.jimfs.Jimfs
import java.net.URI
import java.nio.file.FileSystem
import java.nio.file.Path
import kotlin.io.path.createDirectories
import kotlin.io.path.exists
import org.pkl.commons.test.FileTestUtils
import org.pkl.commons.test.listFilesRecursively
import org.pkl.core.util.IoUtils
class TestUtils {
val tempFileSystem: FileSystem by lazy { Jimfs.newFileSystem(Configuration.unix()) }
val tmpOutputDir by lazy { tempFileSystem.getPath("/work/output").apply { createDirectories() } }
val projectDir = FileTestUtils.rootProjectDir.resolve("pkl-doc")
val inputDir: Path by lazy {
projectDir.resolve("src/test/files/DocGeneratorTest/input").apply { assert(exists()) }
}
val docsiteModule: URI by lazy {
inputDir.resolve("docsite-info.pkl").apply { assert(exists()) }.toUri()
}
internal val package1PackageModule: URI by lazy {
inputDir.resolve("com.package1/doc-package-info.pkl").apply { assert(exists()) }.toUri()
}
val package2PackageModule: URI by lazy {
inputDir.resolve("com.package2/doc-package-info.pkl").apply { assert(exists()) }.toUri()
}
internal val package1InputModules: List<URI> by lazy {
inputDir
.resolve("com.package1")
.listFilesRecursively()
.filter { it.fileName.toString() != "doc-package-info.pkl" }
.map { it.toUri() }
}
val package2InputModules: List<URI> by lazy {
inputDir
.resolve("com.package2")
.listFilesRecursively()
.filter { it.fileName.toString() != "doc-package-info.pkl" }
.map { it.toUri() }
}
val expectedOutputDir: Path by lazy {
projectDir.resolve("src/test/files/DocGeneratorTest/output").createDirectories()
}
val expectedOutputFiles: List<Path> by lazy { expectedOutputDir.listFilesRecursively() }
val actualOutputDir: Path by lazy { tempFileSystem.getPath("/work/DocGeneratorTest") }
val actualOutputFiles: List<Path> by lazy { actualOutputDir.listFilesRecursively() }
val expectedRelativeOutputFiles: List<String> by lazy {
expectedOutputFiles.map { path ->
IoUtils.toNormalizedPathString(expectedOutputDir.relativize(path)).let { str ->
// Git will by default clone symlinks as shortcuts on Windows, and shortcuts have a
// `.lnk` extension.
if (IoUtils.isWindows() && str.endsWith(".lnk")) str.dropLast(4) else str
}
}
}
val actualRelativeOutputFiles: List<String> by lazy {
actualOutputFiles.map { IoUtils.toNormalizedPathString(actualOutputDir.relativize(it)) }
}
val binaryFileExtensions = setOf("woff2", "png", "svg")
}

View File

@@ -489,7 +489,7 @@ public class Lexer {
}
private void lexQuotedIdentifier() {
while (lookahead != '`' && lookahead != '\n' && lookahead != '\r') {
while (lookahead != '`' && lookahead != '\n' && lookahead != '\r' && lookahead != EOF) {
nextChar();
}
if (lookahead == '`') {
@@ -705,6 +705,13 @@ public class Lexer {
}
private ParserError unexpectedChar(char got, String didYouMean) {
if (got == EOF) {
return unexpectedChar("EOF", didYouMean);
}
return lexError("unexpectedCharacter", got, didYouMean);
}
private ParserError unexpectedChar(String got, String didYouMean) {
return lexError("unexpectedCharacter", got, didYouMean);
}

View File

@@ -17,6 +17,7 @@ package org.pkl.parser
import org.assertj.core.api.Assertions.assertThat
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.assertThrows
class LexerTest {
@@ -46,4 +47,10 @@ class LexerTest {
assertThat(Lexer.maybeQuoteIdentifier("this")).isEqualTo("`this`")
assertThat(Lexer.maybeQuoteIdentifier("😀")).isEqualTo("`😀`")
}
@Test
fun lexSingleBacktick() {
val thrown = assertThrows<ParserError> { Lexer("`").next() }
assertThat(thrown).hasMessageContaining("Unexpected character `EOF`")
}
}

View File

@@ -221,7 +221,7 @@ internal class BinaryEvaluator(
override fun visitMapping(value: VmMapping) {
packer.packArrayHeader(2)
packer.packInt(CODE_MAPPING.toInt())
packer.packMapHeader(value.entryCount)
packer.packMapHeader(value.length.toInt())
value.iterateAlreadyForcedMemberValues { key, _, memberValue ->
visit(key)
visit(memberValue)

View File

@@ -13,3 +13,9 @@ res4: Mapping = new {
["bar"] = 2
}
}
// https://github.com/apple/pkl/issues/1151
res5: Mapping = new {
local self = this
["foo"] = new Dynamic { name = "foo" }
["bar"] = new Dynamic { name = self["foo"].name + "bar" }
}

View File

@@ -41,4 +41,28 @@
:
- 3
-
bar: 2
bar: 2
-
- 16
- res5
-
- 3
-
foo:
- 1
- Dynamic
- pkl:base
-
-
- 16
- name
- foo
bar:
- 1
- Dynamic
- pkl:base
-
-
- 16
- name
- foobar

View File

@@ -17,7 +17,7 @@ package org.pkl.server
import org.junit.jupiter.api.AfterEach
import org.junit.jupiter.api.BeforeEach
import org.pkl.commons.test.PklExecutablePaths
import org.pkl.commons.test.Executables
import org.pkl.core.messaging.MessageTransports
class NativeServerTest : AbstractServerTest() {
@@ -26,7 +26,7 @@ class NativeServerTest : AbstractServerTest() {
@BeforeEach
fun beforeEach() {
val executable = PklExecutablePaths.firstExisting.toString()
val executable = Executables.pkl.firstExistingNative.toString()
server = ProcessBuilder(executable, "server").start()
client =
TestTransport(

View File

@@ -36,7 +36,7 @@
///
/// Warning: Although this module is ready for initial use,
/// benchmark results may be inaccurate or inconsistent.
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
module pkl.Benchmark
import "pkl:platform" as _platform

View File

@@ -63,7 +63,7 @@
/// @Deprecated { message = "Use `com.example.Birds.Parrot` instead" }
/// amends "pkl:PackageInfo"
/// ```
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
module pkl.DocPackageInfo
import "pkl:reflect"

View File

@@ -31,7 +31,7 @@
///
/// title = "Title displayed in the header of each page"
/// ```
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
module pkl.DocsiteInfo
import "pkl:reflect"

View File

@@ -15,7 +15,7 @@
//===----------------------------------------------------------------------===//
/// Common settings for Pkl's own evaluator.
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
@Since { version = "0.26.0" }
module pkl.EvaluatorSettings

View File

@@ -64,7 +64,7 @@
/// value = project
/// }
/// ```
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
module pkl.Project
import "pkl:EvaluatorSettings" as EvaluatorSettingsModule

View File

@@ -19,7 +19,7 @@
/// These tools differentiate from [pkl:reflect][reflect] in that they parse Pkl modules, but do not
/// execute any code within these modules.
@Since { version = "0.27.0" }
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
module pkl.analyze
// used by doc comments

View File

@@ -17,7 +17,7 @@
/// Fundamental properties, methods, and classes for writing Pkl programs.
///
/// Members of this module are automatically available in every Pkl module.
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
module pkl.base
import "pkl:jsonnet"

View File

@@ -15,7 +15,7 @@
//===----------------------------------------------------------------------===//
/// A JSON parser.
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
module pkl.json
/// A JSON parser.

View File

@@ -15,7 +15,7 @@
//===----------------------------------------------------------------------===//
/// A [Jsonnet](https://jsonnet.org) renderer.
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
module pkl.jsonnet
/// Constructs an [ImportStr].

View File

@@ -18,7 +18,7 @@
///
/// Note that some mathematical functions, such as `sign()`, `abs()`, and `round()`,
/// are directly defined in classes [Number], [Int], and [Float].
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
module pkl.math
/// The minimum [Int] value: `-9223372036854775808`.

View File

@@ -15,7 +15,7 @@
//===----------------------------------------------------------------------===//
/// Information about the platform that the current program runs on.
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
module pkl.platform
/// The platform that the current program runs on.

View File

@@ -16,7 +16,7 @@
/// A renderer for [Protocol Buffers](https://developers.google.com/protocol-buffers).
/// Note: This module is _experimental_ and not ready for production use.
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
module pkl.protobuf
import "pkl:reflect"

View File

@@ -26,7 +26,7 @@
/// - Documentation generators (such as *Pkldoc*)
/// - Code generators (such as *pkl-codegen-java* and *pkl-codegen-kotlin*)
/// - Domain-specific schema validators
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
module pkl.reflect
import "pkl:base"

View File

@@ -15,7 +15,7 @@
//===----------------------------------------------------------------------===//
/// Information about the Pkl release that the current program runs on.
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
module pkl.release
import "pkl:semver"

View File

@@ -15,7 +15,7 @@
//===----------------------------------------------------------------------===//
/// Parsing, comparison, and manipulation of [semantic version](https://semver.org/spec/v2.0.0.html) numbers.
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
module pkl.semver
/// Tells whether [version] is a valid semantic version number.

View File

@@ -19,7 +19,7 @@
/// Every settings file must amend this module.
/// Unless CLI commands and build tool plugins are explicitly configured with a settings file,
/// they will use `~/.pkl/settings.pkl` or the defaults specified in this module.
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
module pkl.settings
import "pkl:EvaluatorSettings"

View File

@@ -15,7 +15,7 @@
//===----------------------------------------------------------------------===//
/// Utilities for generating shell scripts.
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
module pkl.shell
/// Escapes [str] by enclosing it in single quotes.

View File

@@ -18,7 +18,7 @@
///
/// To write tests, amend this module and define [facts] or [examples] (or both).
/// To run tests, evaluate the amended module.
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
open module pkl.test
/// Named groups of boolean expressions that are expected to evaluate to [true].

View File

@@ -15,7 +15,7 @@
//===----------------------------------------------------------------------===//
/// An XML renderer.
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
module pkl.xml
/// Renders values as XML.

View File

@@ -15,7 +15,7 @@
//===----------------------------------------------------------------------===//
/// A YAML 1.2 compliant YAML parser.
@ModuleInfo { minPklVersion = "0.29.0" }
@ModuleInfo { minPklVersion = "0.29.1" }
module pkl.yaml
/// A YAML parser.