23 Commits

Author SHA1 Message Date
ManuW
19c292f9a2 Improve let-expression examples (#680)
Because I did not understand what a `Let Expression` should be and the example code was more confusing as explanatory, I changed the format of the examples a little bit and changed the result.

- It should be clear that the example is just one code-line
- I tried out the example (`pkl eval...`) and get a little bit different result
2024-10-14 12:21:08 +01:00
Josh B
d720f21149 [docs] Add documentation for new keyword (#624) 2024-08-29 21:32:12 -07:00
Josh B
b28cdcd631 [docs] Improve searchability of "Methods" section (#625) 2024-08-29 21:32:12 -07:00
LamTrinh.Dev
825135a0f8 Correct the link for ANTLR 4 Documentation (#623) 2024-08-29 21:32:12 -07:00
Josh B
63b92ec72b [docs] Document the class-as-a-function pattern (#614) 2024-08-29 21:32:12 -07:00
Josh B
3125fc4678 [docs] Document hidden equality/hashing behavior (#618)
* [docs] Document `hidden` equality/hashing behavior

* Update docs/modules/language-reference/pages/index.adoc

---------

Co-authored-by: Daniel Chao <daniel.h.chao@gmail.com>
2024-08-29 21:32:12 -07:00
Josh B
4bb6890621 [docs] Add mention of "optional" to nullably types section (#613) 2024-08-29 21:32:12 -07:00
Dan Chao
8e15556201 Prepare 0.26.3 release 2024-08-06 09:58:31 -07:00
Daniel Chao
2fb17fd283 Add release notes for 0.26.3 (#612) 2024-08-06 09:56:00 -07:00
Daniel Chao
f4983c51be Fix: globbing--read extra storage from owner instead of receiver (#607)
This fixes an issue where a PklBugException is thrown when a globbed
read/import is amended.
2024-08-06 09:53:35 -07:00
Islon Scherer
2cd2712589 Fix property parsing bug in the cli (#596) 2024-08-06 09:53:28 -07:00
Daniel Chao
f9a3fc88fd Fix usage of file() notation with Pkl Gradle plugin on Windows (#611)
This is a port of a fix that was included in https://github.com/apple/pkl/pull/403.
2024-08-06 07:54:52 -07:00
Dan Chao
d097341abd Prepare 0.26.2 release 2024-07-18 09:13:58 -07:00
Daniel Chao
0eab5fb552 Add release notes for 0.26.2 (#586) 2024-07-18 09:13:11 -07:00
Daniel Chao
7c3787396e Fix race condition when concurrently downloading packages (#584)
This fixes a possible race condition where multiple processes download
the same package into the same temp dir.
2024-07-18 09:12:03 -07:00
Dan Chao
57df7995fd Prepare 0.26.1 release 2024-06-28 09:28:36 -07:00
Daniel Chao
6c97b09c29 Add notes for 0.26.1 (#556) 2024-06-28 09:28:23 -07:00
Daniel Chao
da19c3971e Do not enable TLS certificate revocation checks by default (#553)
This addresses an issue where network requests may fail if cert revocation checks
error, which may occur due to availability issues, or due to lack of internet access.

Revocation checking can still be enabled by setting JVM property com.sun.net.ssl.checkRevocation if on the JVM.

Also:
* Load built-in certs from resources, and move them to pkl-commons-cli
* Fix an issue where HttpInitException is not caught when loading a module
2024-06-28 09:02:44 -07:00
Daniel Chao
efad356b7b Only run Gradle compatibility tests against releases in CI (#554)
This fixes our CI tests on main. It mitigates an issue where the current RC is borked right now.
2024-06-28 09:02:37 -07:00
Daniel Chao
261a2260a1 Use compatible architecture in native executables (#551)
Use the most compatible architecture; for example, x86-64 instead of
x86-64-v3.
2024-06-28 09:02:25 -07:00
Philip K.F. Hölzenspies
204c6b16c3 Resolve project dirs from working dir by default 2024-06-28 09:02:02 -07:00
Daniel Chao
f91f91fd30 docs: add contributor for 0.26 release (#546)
Add a contributor name who was missing from acknowledgements.
2024-06-24 08:52:23 -07:00
Philip K.F. Hölzenspies
309fb49fa1 Prepare 0.26.0 release 2024-06-17 18:49:29 +01:00
47 changed files with 393 additions and 139 deletions

View File

@@ -132,8 +132,7 @@ jobs {
name = "gradle compatibility"
command = #"""
:pkl-gradle:build \
:pkl-gradle:compatibilityTestReleases \
:pkl-gradle:compatibilityTestCandidate
:pkl-gradle:compatibilityTestReleases
"""#
}.job
["deploy-snapshot"] = new DeployJob { command = "publishToSonatype" }.job

View File

@@ -610,8 +610,7 @@ jobs:
- run:
command: |-
./gradlew --info --stacktrace -DtestReportsDir=${HOME}/test-results :pkl-gradle:build \
:pkl-gradle:compatibilityTestReleases \
:pkl-gradle:compatibilityTestCandidate
:pkl-gradle:compatibilityTestReleases
name: gradle compatibility
- store_test_results:
path: ~/test-results

View File

@@ -81,7 +81,7 @@ For automated build setup examples see our https://github.com/apple/pkl/blob/mai
=== ANTLR
* https://github.com/antlr/antlr4/blob/main/doc/index.md[Documentation]
* https://github.com/antlr/antlr4/blob/master/doc/index.md[Documentation]
* https://groups.google.com/forum/#!forum/antlr-discussion[Forums]
* https://github.com/mobileink/lab.clj.antlr/tree/main/doc[Some third-party docs]

View File

@@ -1,6 +1,6 @@
name: main
title: Main Project
version: 0.26.0-dev
prerelease: true
version: 0.26.3
prerelease: false
nav:
- nav.adoc

View File

@@ -3,10 +3,10 @@
// the following attributes must be updated immediately before a release
// pkl version corresponding to current git commit without -dev suffix or git hash
:pkl-version-no-suffix: 0.26.0
:pkl-version-no-suffix: 0.26.3
// tells whether pkl version corresponding to current git commit
// is a release version (:is-release-version: '') or dev version (:!is-release-version:)
:!is-release-version:
:is-release-version: ''
// the remaining attributes do not need to be updated regularly

View File

@@ -915,6 +915,7 @@ pigeon = new Dynamic { // <1>
==== Hidden Properties
A property with the modifier `hidden` is omitted from the rendered output and object conversions.
Hidden properties are also ignored when evaluating equality or hashing (e.g. for `Mapping` or `Map` keys).
[source,{pkl}]
----
@@ -933,12 +934,19 @@ pigeon = new Bird { // <3>
pigeonInIndex = pigeon.nameAndLifespanInIndex // <4>
pigeonDynamic = pigeon.toDynamic() // <5>
favoritePigeon = (pigeon) {
nameAndLifespanInIndex = "Bettie, \(lifespan)"
}
samePigeon = pigeon == favoritePigeon // <6>
----
<1> Properties defined as `hidden` are accessible on any `Bird` instance, but not output by default.
<2> Non-hidden properties can refer to hidden properties as usual.
<3> `pigeon` is an object with _four_ properties, but is rendered with _three_ properties.
<4> Accessing a `hidden` property from outside the class and object is like any other property.
<5> Object conversions omit hidden properties, so the resulting `Dynamic` has three properties.
<6> Objects that differ only in `hidden` property values are considered equal
Invoking Pkl on this file produces the following result.
@@ -955,6 +963,12 @@ pigeonDynamic {
lifespan = 8
nameSignWidth = 9
}
favoritePigeon {
name = "Pigeon"
lifespan = 8
nameSignWidth = 9
}
samePigeon = true
----
==== Local properties
@@ -1992,7 +2006,8 @@ pigeon: ParentBird = new {
[[methods]]
== Methods
Modules and classes can define methods.
Pkl methods can be defined on classes and modules using the `function` keyword.
Methods may access properties of their containing type.
Submodules and subclasses can override them.
Like Java and most other object-oriented languages, Pkl uses _single dispatch_ -- methods are dynamically dispatched based on the receiver's runtime type.
@@ -2021,6 +2036,12 @@ greeting2 = greetPigeon(parrot) // <4>
<3> Call instance method on `pigeon`.
<4> Call module method (on `this`).
NOTE: Methods do not support named parameters or default parameter values.
The xref:blog:ROOT:class-as-a-function.adoc[Class-as-a-function] pattern may be a suitable replacement.
TIP: In most cases, methods without parameters should not be defined.
Instead, use <<fixed-properties,`fixed` properties>> on the module or class.
[[modules]]
== Modules
@@ -3156,6 +3177,7 @@ pkl: TRACE: num1 * num2 = 672 (at file:///some/module.pkl, line 42)
This section discusses language features that are generally more relevant to template and library authors than template consumers.
<<meaning-of-new,Meaning of `new`>> +
<<let-expressions,Let Expressions>> +
<<type-tests,Type Tests>> +
<<type-casts,Type Casts>> +
@@ -3179,6 +3201,162 @@ This section discusses language features that are generally more relevant to tem
<<blank-identifiers,Blank Identifiers>> +
<<projects,Projects>>
[[meaning-of-new]]
=== Meaning of `new`
Objects in Pkl always <<amending-objects, amends>> _some_ value.
The `new` keyword is a special case of amending where a contextual value is amended.
In Pkl, there are two forms of `new` objects:
* `new` with explicit type information, for example, `new Foo {}`.
* `new` without type information, for example, `new {}`.
==== Type defaults
To understand instantiation cases without explicit parent or type information, it's important to first understand implicit default values.
When a property is declared in a module or class but is not provided an explicit default value, the property's default value becomes the type's default value.
Similarly, when `Listing` and `Mapping` types are declared with explicit type arguments for their element or value, their `default` property amends that declared type.
When `Listing` and `Mapping` types are declared without type arguments, their `default` property amends an empty `Dynamic` object.
Some types, including `Pair` and primitives like `String`, `Number`, and `Boolean` have no default value; attempting to render such a property results in the error "Tried to read property `<name>` but its value is undefined".
[source,{pkl}]
----
class Bird {
name: String = "polly"
}
bird: Bird // <1>
birdListing: Listing<Bird> // <2>
birdMapping: Mapping<String, Bird> // <3>
----
<1> Without an explicit default value, this property has default value `new Bird { name = "polly" }`
<2> With an explicit element type argument, this property's default value is equivalent to `new Listing<Bird> { default = (_) -> new Bird { name = "polly" } }`
<3> With an explicit value type argument, this property's default value is equivalent to `new Mapping<String, Bird> { default = (_) -> new Bird { name = "polly" } }`
==== Explicitly Typed `new`
Instantiating an object with `new <type>` results in a value that amends the specified type's default value.
Notably, creating a `Listing` element or assigning a `Mapping` entry value with an explicitly typed `new` ignores the object's `default` value.
[source,{pkl}]
----
class Bird {
/// The name of the bird
name: String
/// Whether this is a bird of prey or not.
isPredatory: Boolean?
}
newProperty = new Bird { // <1>
name = "Warbler"
}
someListing = new Listing<Bird> {
default {
isPredatory = true
}
new Bird { // <2>
name = "Sand Piper"
}
}
someMapping = new Mapping<String, Bird> {
default {
isPredatory = true
}
["Penguin"] = new Bird { // <3>
name = "Penguin"
}
}
----
<1> Assigning a `new` explicitly-typed value to a property.
<2> Adding an `new` explicitly-typed `Listing` element.
The value will not have property `isPredatory = true` as the `default` property of the `Listing` is not used.
<3> Assigning a `new` explicitly-typed value to a `Mapping` entry.
The value will not have property `isPredatory = true` as the `default` property of the `Mapping` is not used.
==== Implicitly Typed `new`
When using the implicitly typed `new` invocation, there is no explicit parent value to amend.
In these cases, Pkl infers the amend operation's parent value based on context:
* When assigning to a declared property, the property's default value is amended (<<amend-null, including `null`>>).
If there is no type associated with the property, an empty `Dynamic` object is amended.
* When assigning to an entry (e.g. a `Mapping` member) or element (e.g. a `Listing` member), the enclosing object's `default` property is applied to the corresponding index or key, respectively, to produce the value to be amended.
* In other cases, evaluation fails with the error message "Cannot tell which parent to amend".
The type annotation of a <<methods,method>> parameter is not used for inference.
In this case, the argument's type should be specified explicitly.
[source,{pkl}]
----
class Bird {
name: String
function listHatchlings(items: Listing<String>): Listing<String> = new {
for (item in items) {
"\(name):\(item)"
}
}
}
typedProperty: Bird = new { // <1>
name = "Swift"
}
untypedProperty = new { // <2>
hello = "world"
}
typedListing: Listing<Bird> = new {
new { // <3>
name = "Kite"
}
}
untypedListing: Listing = new {
new { // <4>
hello = "there"
}
}
typedMapping: Mapping<String, Bird> = new {
default { entryKey ->
name = entryKey
}
["Saltmarsh Sparrow"] = new { // <5>
name = "Sharp-tailed Sparrow"
}
}
amendedMapping = (typedMapping) {
["Saltmarsh Sparrow"] = new {} // <6>
}
class Aviary {
birds: Listing<Bird> = new {
new { name = "Osprey" }
}
}
aviary: Aviary = new {
birds = new { // <7>
new { name = "Kiwi" }
}
}
swiftHatchlings = typedProperty.listHatchlings(new { "Poppy"; "Chirpy" }) // <8>
----
<1> Assignment to a property with an explicitly declared type, amending `new Bird {}`.
<2> Assignment to an undeclared property in module context, amending `new Dynamic {}`.
<3> `Listing` element creation, amending implicit `default`, `new Bird {}`.
<4> `Listing` element creation, amending implicit `default`, `new Dynamic {}`.
<5> `Mapping` value assignment, amdending the result of applying `default` to `"Saltmarsh Sparrow"`, `new Bird { name = "Saltmarsh Sparrow" }`.
<6> `Mapping` value assignment _replacing_ the parent's entry, amending the result of applying `default` to `"Saltmarsh Sparrow"`, `new Bird { name = "Saltmarsh Sparrow" }`.
<7> Admending the property default value `new Listing { new Bird { name = "Osprey" } }`; the result contains both birds.
<8> Error: Cannot tell which parent to amend.
[[let-expressions]]
=== Let Expressions
@@ -3200,10 +3378,11 @@ Here is an example:
[source%tested,{pkl}]
----
birdDiets = let (diets = List("Seeds", "Berries", "Mice"))
List(diets[2], diets[0]) // <1>
birdDiets =
let (diets = List("Seeds", "Berries", "Mice"))
List(diets[2], diets[0]) // <1>
----
<1> result: `List("Mice", "Seeds")`
<1> result: `birdDiets = List("Mice", "Seeds")`
`let` expressions serve two purposes:
@@ -3214,20 +3393,22 @@ List(diets[2], diets[0]) // <1>
[source%tested,{pkl}]
----
birdDiets = let (diets: List<String> = List("Seeds", "Berries", "Mice"))
diets[2] + diets[0] // <1>
birdDiets =
let (diets: List<String> = List("Seeds", "Berries", "Mice"))
diets[2] + diets[0] // <1>
----
<1> result: `List("Mice", "Seeds")`
<1> result: `birdDiets = List("Mice", "Seeds")`
`let` expressions can be stacked:
[source%tested,{pkl}]
----
birdDiets = let (birds = List("Pigeon", "Barn owl", "Parrot"))
let (diet = List("Seeds", "Mice", "Berries"))
birds.zip(diet) // <1>
birdDiets =
let (birds = List("Pigeon", "Barn owl", "Parrot"))
let (diet = List("Seeds", "Mice", "Berries"))
birds.zip(diet) // <1>
----
<1> result: `List(Pair("Pigeon", "Seeds"), Pair("Barn owl", "Mice"), Pair("Parrot", "Berries"))`
<1> result: `birdDiets = List(Pair("Pigeon", "Seeds"), Pair("Barn owl", "Mice"), Pair("Parrot", "Berries"))`
[[type-tests]]
=== Type Tests
@@ -3721,6 +3902,7 @@ bird2: Bird? = null // <2>
The only class types that admit `null` values despite not ending in `?` are `Any` and `Null`.
(`Null` is not very useful as a type because it _only_ admits `null` values.)
`Any?` and `Null?` are equivalent to `Any` and `Null`, respectively.
In some languages, nullable types are also known as _optional types_.
[[generic-types]]
==== Generic Types
@@ -5146,8 +5328,8 @@ It is defined by the presence of a `PklProject` file that amends the standard li
Defining a project serves the following purposes:
1. It allows defining common evaluator settings for Pkl modules within a logical project.
2. It helps with managing <<packages,package>> dependencies for Pkl modules within a logical project.
3. It enables packaging and sharing the contents of the project as a <<packages,package>>.
2. It helps with managing <<package-asset-uri,package>> dependencies for Pkl modules within a logical project.
3. It enables packaging and sharing the contents of the project as a <<package-asset-uri,package>>.
4. It allows importing packages via dependency notation.
[[project-dependencies]]

View File

@@ -1,6 +1,6 @@
= Pkl 0.26 Release Notes
:version: 0.26
:version-minor: 0.26.0
:version-minor: 0.26.3
:release-date: June 17th, 2024
include::ROOT:partial$component-attributes.adoc[]
@@ -508,6 +508,7 @@ We would like to thank the contributors to this release (in alphabetical order):
* https://github.com/MarkSRobinson[@MarkSRobinson]
* https://github.com/mitchcapper[@mitchcapper]
* https://github.com/mrs1669[@mrs1669]
* https://github.com/netvl[@netvl]
* https://github.com/nirinchev[@nirinchev]
* https://github.com/raj-j-shah[@raj-j-shah]
* https://github.com/sgammon[@sgammon]

View File

@@ -1,6 +1,43 @@
= Changelog
include::ROOT:partial$component-attributes.adoc[]
[[release-0.26.3]]
== 0.26.3 (2024-08-06)
=== Fixes
* Fixes an issue where CLI argument `--property foo=""` is effectively parsed as `--property foo="true"`. This is now parsed as an empty string (https://github.com/apple/pkl/pull/596[#596]).
* Fixes a regression where amending a globbed import or globbed read results in a PklBugException (https://github.com/apple/pkl/pull/607[#607]).
* Fixes an issue around using `file()` notation when using the pkl-gradle plugin on Windows (https://github.com/apple/pkl/pull/611[#611]).
[[release-0.26.2]]
== 0.26.2 (2024-07-18)
=== Fixes
* Fixes a possible race condition where multiple concurrent Pkl evaluations results in a thrown exception when downloading packages (https://github.com/apple/pkl/pull/584[#584]).
[[release-0.26.1]]
== 0.26.1 (2024-06-28)
=== Fixes
* Fixes a regression where native executables fail to run on some environments that don't support newer CPU features (https://github.com/apple/pkl/pull/551[#551]).
* Fixes a `PklBugException` when passing `.` as a project directory to `pkl project resolve` and `pkl project package` (https://github.com/apple/pkl/pull/544[#544]).
=== Changes
* Disable revocation checking of TLS certificates (https://github.com/apple/pkl/pull/553[#553]).
+
As part of HTTP improvements in 0.26, we unwittingly fixed a bug where Pkl does not actually perform cert revocation checks when making HTTPS requests.
This fix, unfortunately, caused a regression in some cases.
For example, this happens when connecting to a server that bears a public trust certificate, while in an environment with no internet access.
This is because the HTTP client needs to check the revocation status of all certificates in the chain.
+
Revocation checks are a nuanced topic with some benefits, and also with its own problem areas.
For this reason, revocation checking is disabled for Pkl's native CLIs.
Users of Pkl's Java APIs will respect the revocation settings set in the JVM.
[[release-0.26.0]]
== 0.26.0 (2024-06-17)

View File

@@ -1,7 +1,7 @@
# suppress inspection "UnusedProperty" for whole file
group=org.pkl-lang
version=0.26.0
version=0.26.3
# google-java-format requires jdk.compiler exports
org.gradle.jvmargs= \

View File

@@ -1,6 +1,3 @@
import java.security.KeyStore
import java.security.cert.CertificateFactory
plugins {
pklAllProjects
pklKotlinLibrary
@@ -38,8 +35,6 @@ val stagedLinuxAarch64Executable: Configuration by configurations.creating
val stagedAlpineLinuxAmd64Executable: Configuration by configurations.creating
val stagedWindowsAmd64Executable: Configuration by configurations.creating
val certs: SourceSet by sourceSets.creating
dependencies {
compileOnly(libs.svm)
@@ -148,38 +143,11 @@ tasks.check {
dependsOn(testStartJavaExecutable)
}
val trustStore = layout.buildDirectory.dir("generateTrustStore/PklCARoots.p12")
val trustStorePassword = "password" // no sensitive data to protect
// generate a trust store for Pkl's built-in CA certificates
val generateTrustStore by tasks.registering {
inputs.file(certs.resources.singleFile)
outputs.file(trustStore)
doLast {
val certificates = certs.resources.singleFile.inputStream().use { stream ->
CertificateFactory.getInstance("X.509").generateCertificates(stream)
}
KeyStore.getInstance("PKCS12").apply {
load(null, trustStorePassword.toCharArray()) // initialize empty trust store
for ((index, certificate) in certificates.withIndex()) {
setCertificateEntry("cert-$index", certificate)
}
val trustStoreFile = trustStore.get().asFile
trustStoreFile.parentFile.mkdirs()
trustStoreFile.outputStream().use { stream ->
store(stream, trustStorePassword.toCharArray())
}
}
}
}
fun Exec.configureExecutable(
graalVm: BuildInfo.GraalVm,
outputFile: Provider<RegularFile>,
extraArgs: List<String> = listOf()
) {
dependsOn(generateTrustStore)
inputs.files(sourceSets.main.map { it.output })
.withPropertyName("mainSourceSets")
.withPathSensitivity(PathSensitivity.RELATIVE)
@@ -210,14 +178,10 @@ fun Exec.configureExecutable(
// needed for messagepack-java (see https://github.com/msgpack/msgpack-java/issues/600)
add("--initialize-at-run-time=org.msgpack.core.buffer.DirectBufferAccess")
add("--no-fallback")
add("-Djavax.net.ssl.trustStore=${trustStore.get().asFile}")
add("-Djavax.net.ssl.trustStorePassword=$trustStorePassword")
add("-Djavax.net.ssl.trustStoreType=PKCS12")
// security property "ocsp.enable=true" is set in Main.kt
add("-Dcom.sun.net.ssl.checkRevocation=true")
add("-H:IncludeResources=org/pkl/core/stdlib/.*\\.pkl")
add("-H:IncludeResources=org/jline/utils/.*")
add("-H:IncludeResourceBundles=org.pkl.core.errorMessages")
add("-H:IncludeResources=org/pkl/commons/cli/PklCARoots.pem")
add("--macro:truffle")
add("-H:Class=org.pkl.cli.Main")
add("-H:Name=${outputFile.get().asFile.name}")
@@ -232,6 +196,7 @@ fun Exec.configureExecutable(
if (!buildInfo.isReleaseBuild) {
add("-Ob")
}
add("-march=compatibility")
// native-image rejects non-existing class path entries -> filter
add("--class-path")
val pathInput = sourceSets.main.get().output + configurations.runtimeClasspath.get()
@@ -316,7 +281,7 @@ val windowsExecutableAmd64: TaskProvider<Exec> by tasks.registering(Exec::class)
configureExecutable(
buildInfo.graalVmAmd64,
layout.buildDirectory.file("executable/pkl-windows-amd64"),
listOf("-Dfile.encoding=UTF-8", "-march=compatibility")
listOf("-Dfile.encoding=UTF-8")
)
}

View File

@@ -35,7 +35,7 @@ abstract class CliProjectCommand(cliOptions: CliBaseOptions, private val project
)
return@lazy listOf(projectFile.normalize())
}
projectDirs.map { dir ->
projectDirs.map(cliOptions.normalizedWorkingDir::resolve).map { dir ->
val projectFile = dir.resolve(PKL_PROJECT_FILENAME)
if (!Files.exists(projectFile)) {
throw CliException("Directory $dir does not contain a PklProject file.")

View File

@@ -171,8 +171,20 @@ abstract class CliCommand(protected val cliOptions: CliBaseOptions) {
private fun HttpClient.Builder.addDefaultCliCertificates() {
val caCertsDir = IoUtils.getPklHomeDir().resolve("cacerts")
var certsAdded = false
if (Files.isDirectory(caCertsDir)) {
Files.list(caCertsDir).filter { it.isRegularFile() }.forEach { addCertificates(it) }
Files.list(caCertsDir)
.filter { it.isRegularFile() }
.forEach { cert ->
certsAdded = true
addCertificates(cert)
}
}
if (!certsAdded) {
val defaultCerts =
javaClass.classLoader.getResourceAsStream("org/pkl/commons/cli/PklCARoots.pem")
?: throw CliException("Could not find bundled certificates")
addCertificates(defaultCerts.readAllBytes())
}
}

View File

@@ -16,7 +16,6 @@
package org.pkl.commons.cli
import java.io.PrintStream
import java.security.Security
import kotlin.system.exitProcess
/** Building block for CLIs. Intended to be called from a `main` method. */
@@ -30,9 +29,6 @@ fun cliMain(block: () -> Unit) {
// Force `native-image` to use system proxies (which does not happen with `-D`).
System.setProperty("java.net.useSystemProxies", "true")
// enable OCSP for default SSL context
Security.setProperty("ocsp.enable", "true")
try {
block()
} catch (e: CliTestException) {

View File

@@ -64,6 +64,16 @@ class BaseOptions : OptionGroup() {
throw CliException(message)
}
}
fun RawOption.associateProps():
OptionWithValues<Map<String, String>, Pair<String, String>, Pair<String, String>> {
return convert {
val parts = it.split("=")
if (parts.size <= 1) parts[0] to "true" else parts[0] to parts[1]
}
.multiple()
.toMap()
}
}
private val defaults = CliBaseOptions()
@@ -116,7 +126,7 @@ class BaseOptions : OptionGroup() {
metavar = "<name=value>",
help = "External property to set (repeatable)."
)
.associate()
.associateProps()
val noCache: Boolean by
option(names = arrayOf("--no-cache"), help = "Disable caching of packages")
@@ -215,7 +225,7 @@ class BaseOptions : OptionGroup() {
allowedModules = allowedModules.ifEmpty { null },
allowedResources = allowedResources.ifEmpty { null },
environmentVariables = envVars.ifEmpty { null },
externalProperties = properties.mapValues { it.value.ifBlank { "true" } }.ifEmpty { null },
externalProperties = properties.ifEmpty { null },
modulePath = modulePath.ifEmpty { null },
workingDir = workingDir,
settings = settings,

View File

@@ -45,10 +45,10 @@ class BaseCommandTest {
@Test
fun `external properties without value default to 'true'`() {
cmd.parse(arrayOf("-p", "flag1", "-p", "flag2", "-p", "FOO=bar"))
cmd.parse(arrayOf("-p", "flag1", "-p", "flag2=", "-p", "FOO=bar"))
val props = cmd.baseOptions.baseOptions(emptyList()).externalProperties
assertThat(props).isEqualTo(mapOf("flag1" to "true", "flag2" to "true", "FOO" to "bar"))
assertThat(props).isEqualTo(mapOf("flag1" to "true", "flag2" to "", "FOO" to "bar"))
}
@Test

View File

@@ -45,7 +45,7 @@ public final class ImportGlobMemberBodyNode extends ExpressionNode {
@Override
public Object executeGeneric(VirtualFrame frame) {
var mapping = VmUtils.getObjectReceiver(frame);
var mapping = VmUtils.getOwner(frame);
var path = (String) VmUtils.getMemberKey(frame);
return importModule(mapping, path);
}

View File

@@ -33,7 +33,7 @@ public class ReadGlobMemberBodyNode extends ExpressionNode {
@Override
public Object executeGeneric(VirtualFrame frame) {
var mapping = VmUtils.getObjectReceiver(frame);
var mapping = VmUtils.getOwner(frame);
var path = (String) VmUtils.getMemberKey(frame);
return readResource(mapping, path);
}

View File

@@ -31,22 +31,17 @@ import java.nio.file.Files;
import java.nio.file.NoSuchFileException;
import java.nio.file.Path;
import java.security.GeneralSecurityException;
import java.security.KeyStore;
import java.security.SecureRandom;
import java.security.cert.CertPathBuilder;
import java.security.cert.Certificate;
import java.security.cert.CertificateException;
import java.security.cert.CertificateFactory;
import java.security.cert.PKIXBuilderParameters;
import java.security.cert.PKIXRevocationChecker;
import java.security.cert.TrustAnchor;
import java.security.cert.X509CertSelector;
import java.security.cert.X509Certificate;
import java.time.Duration;
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import javax.annotation.concurrent.ThreadSafe;
import javax.net.ssl.CertPathTrustManagerParameters;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLException;
import javax.net.ssl.SSLHandshakeException;
@@ -130,43 +125,36 @@ final class JdkHttpClient implements HttpClient {
List<Path> certificateFiles, List<ByteBuffer> certificateBytes) {
try {
if (certificateFiles.isEmpty() && certificateBytes.isEmpty()) {
// use Pkl native executable's or JVM's built-in CA certificates
// use JVM's built-in CA certificates
return SSLContext.getDefault();
}
var certPathBuilder = CertPathBuilder.getInstance("PKIX");
// create a non-legacy revocation checker that is configured via setOptions() instead of
// security property "ocsp.enabled"
var revocationChecker = (PKIXRevocationChecker) certPathBuilder.getRevocationChecker();
revocationChecker.setOptions(Set.of()); // prefer OCSP, fall back to CRLs
var certFactory = CertificateFactory.getInstance("X.509");
Set<TrustAnchor> trustAnchors =
createTrustAnchors(certFactory, certificateFiles, certificateBytes);
var pkixParameters = new PKIXBuilderParameters(trustAnchors, new X509CertSelector());
// equivalent of "com.sun.net.ssl.checkRevocation=true"
pkixParameters.setRevocationEnabled(true);
pkixParameters.addCertPathChecker(revocationChecker);
List<Certificate> certs = gatherCertificates(certFactory, certificateFiles, certificateBytes);
var keystore = KeyStore.getInstance(KeyStore.getDefaultType());
keystore.load(null);
for (var i = 0; i < certs.size(); i++) {
keystore.setCertificateEntry("Certificate" + i, certs.get(i));
}
var trustManagerFactory = TrustManagerFactory.getInstance("PKIX");
trustManagerFactory.init(new CertPathTrustManagerParameters(pkixParameters));
trustManagerFactory.init(keystore);
var sslContext = SSLContext.getInstance("TLS");
sslContext.init(null, trustManagerFactory.getTrustManagers(), new SecureRandom());
return sslContext;
} catch (GeneralSecurityException e) {
} catch (GeneralSecurityException | IOException e) {
throw new HttpClientInitException(
ErrorMessages.create("cannotInitHttpClient", Exceptions.getRootReason(e)), e);
}
}
private static Set<TrustAnchor> createTrustAnchors(
private static List<Certificate> gatherCertificates(
CertificateFactory factory, List<Path> certificateFiles, List<ByteBuffer> certificateBytes) {
var anchors = new HashSet<TrustAnchor>();
var certificates = new ArrayList<Certificate>();
for (var file : certificateFiles) {
try (var stream = Files.newInputStream(file)) {
collectTrustAnchors(anchors, factory, stream, file);
collectCertificates(certificates, factory, stream, file);
} catch (NoSuchFileException e) {
throw new HttpClientInitException(ErrorMessages.create("cannotFindCertFile", file));
} catch (IOException e) {
@@ -176,13 +164,13 @@ final class JdkHttpClient implements HttpClient {
}
for (var byteBuffer : certificateBytes) {
var stream = new ByteArrayInputStream(byteBuffer.array());
collectTrustAnchors(anchors, factory, stream, "<unavailable>");
collectCertificates(certificates, factory, stream, "<unavailable>");
}
return anchors;
return certificates;
}
private static void collectTrustAnchors(
Collection<TrustAnchor> anchors,
private static void collectCertificates(
ArrayList<Certificate> anchors,
CertificateFactory factory,
InputStream stream,
Object source) {
@@ -197,8 +185,6 @@ final class JdkHttpClient implements HttpClient {
if (certificates.isEmpty()) {
throw new HttpClientInitException(ErrorMessages.create("emptyCertFile", source));
}
for (var certificate : certificates) {
anchors.add(new TrustAnchor(certificate, null));
}
anchors.addAll(certificates);
}
}

View File

@@ -455,10 +455,9 @@ final class PackageResolvers {
private byte[] downloadUriToPathAndComputeChecksum(URI downloadUri, Path path)
throws IOException, SecurityManagerException {
Files.createDirectories(path.getParent());
var inputStream = openExternalUri(downloadUri);
try (var digestInputStream = newDigestInputStream(inputStream)) {
Files.copy(digestInputStream, path);
Files.copy(digestInputStream, path, StandardCopyOption.REPLACE_EXISTING);
return digestInputStream.getMessageDigest().digest();
}
}
@@ -472,7 +471,7 @@ final class PackageResolvers {
}
try (var in = inputStream) {
Files.createDirectories(path.getParent());
Files.copy(in, path);
Files.copy(in, path, StandardCopyOption.REPLACE_EXISTING);
if (checksums != null) {
var digestInputStream = (DigestInputStream) inputStream;
var checksumBytes = digestInputStream.getMessageDigest().digest();
@@ -490,7 +489,10 @@ final class PackageResolvers {
if (Files.exists(cachePath)) {
return cachePath;
}
var tmpPath = tmpDir.resolve(metadataRelativePath);
Files.createDirectories(tmpDir);
var tmpPath =
Files.createTempFile(
tmpDir, IoUtils.encodePath(packageUri.toString().replaceAll("/", "-")), ".json");
try {
downloadMetadata(packageUri, requestUri, tmpPath, checksums);
Files.createDirectories(cachePath.getParent());
@@ -539,7 +541,10 @@ final class PackageResolvers {
if (Files.exists(cachePath)) {
return cachePath;
}
var tmpPath = tmpDir.resolve(relativePath);
Files.createDirectories(tmpDir);
var tmpPath =
Files.createTempFile(
tmpDir, IoUtils.encodePath(packageUri.toString().replaceAll("/", "-")), ".zip");
try {
var checksumBytes =
downloadUriToPathAndComputeChecksum(dependencyMetadata.getPackageZipUrl(), tmpPath);

View File

@@ -30,6 +30,7 @@ import java.util.stream.Collectors;
import org.pkl.core.Release;
import org.pkl.core.SecurityManager;
import org.pkl.core.SecurityManagerException;
import org.pkl.core.http.HttpClientInitException;
import org.pkl.core.module.ModuleKey;
import org.pkl.core.module.ModuleKeys;
import org.pkl.core.module.ResolvedModuleKey;
@@ -191,7 +192,7 @@ public final class ModuleCache {
ModuleKey module, SecurityManager securityManager, @Nullable Node importNode) {
try {
return module.resolve(securityManager);
} catch (SecurityManagerException | PackageLoadError e) {
} catch (SecurityManagerException | PackageLoadError | HttpClientInitException e) {
throw new VmExceptionBuilder().withOptionalLocation(importNode).withCause(e).build();
} catch (FileNotFoundException | NoSuchFileException e) {
var exceptionBuilder =

View File

@@ -33,7 +33,7 @@ import org.pkl.core.util.Nullable;
@TruffleLanguage.Registration(
id = "pkl",
name = "Pkl",
version = "0.26.0-dev",
version = "0.26.3",
characterMimeTypes = VmLanguage.MIME_TYPE,
contextPolicy = ContextPolicy.SHARED)
public final class VmLanguage extends TruffleLanguage<VmContext> {

View File

@@ -16,6 +16,16 @@ examples {
import*("../../input-helper/globtest/**.pkl").keys.toListing()
}
["amended"] {
(import*("../../input-helper/globtest/**.pkl")) {
[[true]] {
output {
renderer = new YamlRenderer {}
}
}
}.toMap().values.map((it) -> it.output.text).join("\n---\n")
}
["globstar then up one level"] {
import*("../../input-helper/globtest/**/../*.pkl").keys.toListing()
}

View File

@@ -8,6 +8,14 @@ examples {
read*("globtest/*.txt")
}
["amended"] {
(read*("../../input-helper/globtest/**.pkl")) {
[[true]] {
text = "hi"
}
}
}
["env:"] {
// doesn't match names that include slashes
read*("env:*")

View File

@@ -21,6 +21,21 @@ examples {
"../../input-helper/globtest/child/moduleC.pkl"
}
}
["amended"] {
"""
{}
---
name: moduleA
---
name: moduleB
---
name: child/moduleC
"""
}
["globstar then up one level"] {
new {
"../../input-helper/globtest/child/../module with [weird] ~!characters.pkl"

View File

@@ -50,6 +50,30 @@ examples {
}
}
}
["amended"] {
new {
["../../input-helper/globtest/module with [weird] ~!characters.pkl"] {
uri = "file:///$snippetsDir/input-helper/globtest/module%20with%20%5Bweird%5D%20~!characters.pkl"
text = "hi"
base64 = ""
}
["../../input-helper/globtest/moduleA.pkl"] {
uri = "file:///$snippetsDir/input-helper/globtest/moduleA.pkl"
text = "hi"
base64 = "bmFtZSA9ICJtb2R1bGVBIgo="
}
["../../input-helper/globtest/moduleB.pkl"] {
uri = "file:///$snippetsDir/input-helper/globtest/moduleB.pkl"
text = "hi"
base64 = "bmFtZSA9ICJtb2R1bGVCIgo="
}
["../../input-helper/globtest/child/moduleC.pkl"] {
uri = "file:///$snippetsDir/input-helper/globtest/child/moduleC.pkl"
text = "hi"
base64 = "bmFtZSA9ICJjaGlsZC9tb2R1bGVDIgo="
}
}
}
["env:"] {
new {
["env:NAME1"] = "value1"

View File

@@ -2,10 +2,9 @@ package org.pkl.core.packages
import org.assertj.core.api.Assertions.assertThat
import org.assertj.core.api.Assertions.assertThatCode
import org.junit.jupiter.api.AfterAll
import org.junit.jupiter.api.BeforeAll
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.assertThrows
import org.junit.jupiter.api.*
import org.junit.jupiter.api.parallel.Execution
import org.junit.jupiter.api.parallel.ExecutionMode
import org.pkl.commons.deleteRecursively
import org.pkl.commons.readString
import org.pkl.commons.test.FileTestUtils
@@ -44,7 +43,9 @@ class PackageResolversTest {
}
}
@Test
// execute test 3 times to check concurrent writes
@RepeatedTest(3)
@Execution(ExecutionMode.CONCURRENT)
fun `get module bytes`() {
val expectedBirdModule = packageRoot.resolve("birds@0.5.0/package/Bird.pkl").readString(StandardCharsets.UTF_8)
val assetUri = PackageAssetUri("package://localhost:0/birds@0.5.0#/Bird.pkl")

View File

@@ -134,6 +134,9 @@ public abstract class ModulesTask extends BasePklTask {
*/
private URI parsedModuleNotationToUri(Object notation) {
if (notation instanceof File file) {
if (file.isAbsolute()) {
return file.toPath().toUri();
}
return IoUtils.createUri(IoUtils.toNormalizedPathString(file.toPath()));
} else if (notation instanceof URI uri) {
return uri;

View File

@@ -36,7 +36,7 @@
///
/// Warning: Although this module is ready for initial use,
/// benchmark results may be inaccurate or inconsistent.
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
module pkl.Benchmark
import "pkl:platform" as _platform

View File

@@ -63,7 +63,7 @@
/// @Deprecated { message = "Use `com.example.Birds.Parrot` instead" }
/// amends "pkl:PackageInfo"
/// ```
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
module pkl.DocPackageInfo
import "pkl:reflect"

View File

@@ -31,7 +31,7 @@
///
/// title = "Title displayed in the header of each page"
/// ```
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
module pkl.DocsiteInfo
import "pkl:reflect"

View File

@@ -15,7 +15,7 @@
//===----------------------------------------------------------------------===//
/// Common settings for Pkl's own evaluator.
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
@Since { version = "0.26.0" }
module pkl.EvaluatorSettings

View File

@@ -64,7 +64,7 @@
/// value = project
/// }
/// ```
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
module pkl.Project
import "pkl:EvaluatorSettings" as EvaluatorSettingsModule

View File

@@ -17,7 +17,7 @@
/// Fundamental properties, methods, and classes for writing Pkl programs.
///
/// Members of this module are automatically available in every Pkl module.
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
module pkl.base
import "pkl:jsonnet"

View File

@@ -15,7 +15,7 @@
//===----------------------------------------------------------------------===//
/// A JSON parser.
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
module pkl.json
/// A JSON parser.

View File

@@ -15,7 +15,7 @@
//===----------------------------------------------------------------------===//
/// A [Jsonnet](https://jsonnet.org) renderer.
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
module pkl.jsonnet
/// Constructs an [ImportStr].

View File

@@ -18,7 +18,7 @@
///
/// Note that some mathematical functions, such as `sign()`, `abs()`, and `round()`,
/// are directly defined in classes [Number], [Int], and [Float].
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
module pkl.math
/// The minimum [Int] value: `-9223372036854775808`.

View File

@@ -15,7 +15,7 @@
//===----------------------------------------------------------------------===//
/// Information about the platform that the current program runs on.
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
module pkl.platform
/// The platform that the current program runs on.

View File

@@ -16,7 +16,7 @@
/// A renderer for [Protocol Buffers](https://developers.google.com/protocol-buffers).
/// Note: This module is _experimental_ and not ready for production use.
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
module pkl.protobuf
import "pkl:reflect"

View File

@@ -26,7 +26,7 @@
/// - Documentation generators (such as *Pkldoc*)
/// - Code generators (such as *pkl-codegen-java* and *pkl-codegen-kotlin*)
/// - Domain-specific schema validators
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
module pkl.reflect
import "pkl:base"

View File

@@ -15,7 +15,7 @@
//===----------------------------------------------------------------------===//
/// Information about the Pkl release that the current program runs on.
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
module pkl.release
import "pkl:semver"

View File

@@ -15,7 +15,7 @@
//===----------------------------------------------------------------------===//
/// Parsing, comparison, and manipulation of [semantic version](https://semver.org/spec/v2.0.0.html) numbers.
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
module pkl.semver
/// Tells whether [version] is a valid semantic version number.

View File

@@ -19,7 +19,7 @@
/// Every settings file must amend this module.
/// Unless CLI commands and build tool plugins are explicitly configured with a settings file,
/// they will use `~/.pkl/settings.pkl` or the defaults specified in this module.
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
module pkl.settings
import "pkl:EvaluatorSettings"

View File

@@ -15,7 +15,7 @@
//===----------------------------------------------------------------------===//
/// Utilities for generating shell scripts.
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
module pkl.shell
/// Escapes [str] by enclosing it in single quotes.

View File

@@ -18,7 +18,7 @@
///
/// To write tests, amend this module and define [facts] or [examples] (or both).
/// To run tests, evaluate the amended module.
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
open module pkl.test
/// Named groups of boolean expressions that are expected to evaluate to [true].

View File

@@ -15,7 +15,7 @@
//===----------------------------------------------------------------------===//
/// An XML renderer.
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
module pkl.xml
/// Renders values as XML.

View File

@@ -15,7 +15,7 @@
//===----------------------------------------------------------------------===//
/// A YAML 1.2 compliant YAML parser.
@ModuleInfo { minPklVersion = "0.26.0" }
@ModuleInfo { minPklVersion = "0.26.1" }
module pkl.yaml
/// A YAML parser.