This changes local asset path resolution so that `..` segments at the
dependency root just resolves to the dependency root.
This makes import resolution work the same between local and remote dependencies.
This changes the following:
1. If given a union type, the member must exist on each member of that type
2. If accessing a member off `Reference<D, Null>`, give a `Reference<D, Null>`
Also:
* Improve error messages thrown during member access.
* Add test around accessing members off of a function type
Stdlib modules are singletons that are shared across multiple evaluators.
This improves thread safety by evaluating its `output.bytes` during
initialization, which initializes truffle nodes (e.g. TypeTestNode),
and also initializes the member cache of the module output of `pkl:base`
This addresses many IDE warnings resulting from the switch to JSpecify.
Also, this changes the behavior of exporting VmObject; there's no place
in our code that does not force a VmObject prior to export, so
the existing logic around handling nullable values has been removed.
This adjusts the doc comments in ref.pkl
* Fix incorrect code snippets
* Clean up examples
* Remove some sections
* Make phrasing consistent with the rest of the stdlib
## What changed
This updates the GraalVM install task so Windows installs use ZIP extraction instead of the Unix tar path. Windows now publishes the extracted directory by moving it into place, while macOS and Linux keep the existing tar extraction and symlink-based install flow.
## Why
BuildInfo.GraalVm downloads .zip archives on Windows, but InstallGraalVm always ran tar --strip-components=1 -xzf and then tried to install by creating a symbolic link. That combination is fragile on Windows: the archive format does not match the extraction command, and symlink creation can require special privileges.
The changes made in this commit are good, but we're going to kick this out
to the next release.
This is because:
1. There's a couple more issues around the `abstract` modifier that is not
implemented yet, and need design considerations
2. These are breaking changes, and we want to minimize the amount of breakages
for users.
3. The main branch is still the develop branch for Pkl 0.32
We will apply a re-revert of this commit after Pkl 0.32 is released.
The eval task, unlike other tasks, creates the evaluator when the property is evaluated, not just when the task is executed (in the `getEffective*` properties). The problem here is that property evaluation can and will happen before the task is executed, because Gradle needs property values for caching and sometimes task dependency information.
Therefore, if due to misconfiguration or due to intentional configuration the evaluator cannot be created - for example, when the `PklProject` module doesn't exist, but the project directory is configured - then the task will fail with an exception which looks _very_ similar to a regular execution exception, but which actually is not, because it is thrown not during task execution, but during preparation for the execution.
This distinction matters a lot when you rely on conditional task execution. If you use the `onlyIf` predicate on tasks to define a condition for the task execution, this predicate will be evaluated before the task actions are run, but *after* properties are evaluated. Thus, if property evaluation fails with an exception, it will *look* as if the `onlyIf` predicate is completely ignored and not even evaluated, and the task action is run regardless of the predicate.
This error mode is extremely confusing and is actually wrong: if I use `onlyIf` to gate the task execution, I don't want *any* of its logic to run. In fact, in my case I specifically use `onlyIf` on the evaluation task to only execute it if a prerequisite is met, and the same predicate guards a bunch of other tasks which prepare the Pkl project, so my Pkl project isn't even generated if the predicate fails. But due to this behavior of properties evaluation which are not controlled by `onlyIf`, the project is still attempted to be evaluated, and this results in a very unexpected build failure.
The solution is to ignore the evaluation exception, because for all intents and purposes, if the project can't be properly evaluated, it will fail during the task execution as it should, so the values of output properties don't really matter as the task will never be run.
These methods aren't implemented in `Set`, and don't really make sense because `Set` types can't be accessed by index.
Note: although this removes methods, this actually isn't a breaking change:
1. Calling `Set.findIndex()` currently throws an error around "cannot invoke abstract method"
2. `List` and `Set` are the only subclasses of `Collection`.
The following code isn't breaking at runtime, although static analysis tooling (like our IDE plugins) will now flag this as an error:
```pkl
myCollection: Collection
idx = myCollection.indexOf(1)
```
Several `Facts:` examples in the `pkl:base` standard library docs assert
statements that are false when evaluated. Since these examples are not
run by any test, the mistakes went unnoticed and are rendered verbatim
into the generated API docs, where they mislead readers.
The corrected examples:
- `String.isNotBlank`: `"\t\n\r".isNotBlank` was listed as holding, but
a string of only whitespace is blank, so it is false. Negated it to
match the neighboring `!"".isNotBlank` and `!" ".isNotBlank` examples.
- `DataSize.toBinaryUnit` / `toDecimalUnit`: the `mb`/`mib` lines
mirrored the `kb`/`kib` lines, but the identity only holds for adjacent
units. `1024.kb == 1000.kib` (both 1,024,000 b), whereas `1024.mb` is
1,024,000,000 b and `1000.mib` is 1,048,576,000 b, so they are not
equal. There is no clean round-number equivalent at this magnitude, so I
removed the two false lines; the remaining examples still demonstrate
the conversion.
- `Collection.any`: `!List(1, 2, 3).any((n) -> n.isEven)` is false
because 2 is even. Changed the list to `List(1, 3, 5)` so the negation
holds.
- `IntSeq.end`: the example read `IntSeq(2, 5).start == 5`, which
documents the wrong property and is false (`start` is 2). Corrected it
to `IntSeq(2, 5).end == 5`.
- `List.isDistinctBy` / `distinctBy`: `List("a", "b", "abc")` is not
distinct by length, since `"a"` and `"b"` both have length 1. Switched
to `List("a", "bb", "ccc")` so the distinctness examples hold.
- `Map`: `Map(...).values` returns a `List`, not a `Set`. Corrected the
expected type.
I verified that each corrected example evaluates to `true`, and that the
neighboring examples I kept still pass, using the released Pkl 0.31.1
binary. The changes are confined to doc-comment text, so formatting is
unaffected.
---------
Signed-off-by: Aditya Singh <adisin650@gmail.com>
This makes various improvements to the handling of frame slot vars, and
includes some bug fixes introduced by
https://github.com/apple/pkl/pull/1622
* Refactor SymbolTable to track for-generator and parameter slots in
each scope
* Execute let expressions in their own root node in some places
* Unify how frame slots are managed; they are all represented as
`FrameSlotVariable`, created in `AstBuilder`, and passed into
`SymbolTable`.
* Fix how let expressions are executed in custom this scopes (introduce
a new root node when needed)
Fixes#1614.
## Context
A non-abstract `class` (or `module`) was allowed to declare `abstract`
properties and methods.
Because such an enclosing type is instantiable, an `abstract` member
there can never be guaranteed
an implementation — so the contradiction surfaced only as a runtime
error when the member was
accessed (`Cannot invoke abstract method`), or not at all.
This makes it a compile-time error to declare an `abstract` member
unless its enclosing class or
module is also `abstract`. This is consistent with how Pkl already
rejects instantiating an abstract
class, and mirrors how Java and Kotlin treat abstract members.
## Before
```pkl
class Foo {
abstract bar: Int
}
res = new Foo { bar = 5 } // evaluated successfully (should fail)
```
```pkl
class Foo {
abstract function bar(): Int
}
res = new Foo {} // evaluated successfully; res.bar() failed only at runtime
```
## After
```
–– Pkl Error ––
Cannot define an abstract member in a non-abstract class.
2 | abstract bar: Int
^^^^^^^^
at Foo
A member can only be `abstract` if its enclosing class is also `abstract`.
```
## Implementation
- `AstBuilder` now validates, while building the AST, that a
non-abstract class/module declares no
`abstract` members. The check runs in both `visitClass` and
`visitModule`, and the error points at
the `abstract` keyword.
- Adds the `abstractMemberInNonAbstractClass` error message.
## Scope: classes and modules
The issue describes classes; I applied the same rule to modules as well,
since a module is a class
in Pkl and a non-abstract module is likewise directly evaluatable. Happy
to narrow this to classes
only if you'd prefer — it's a one-line change either way.
The `moduleMethodModifiers` pkl-doc test fixture declared an abstract
method at non-abstract module
level (relying on the old behavior); it's updated to an `abstract
module`, and its expected
documentation output is regenerated.
## Tests
- New `LanguageSnippetTests` error cases: abstract property in a class,
abstract method in a class,
and abstract member in a module.
- `./gradlew build` passes (`pkl-core` and `pkl-doc` included).
---------
Co-authored-by: Vinayak <vinayak@vama.app>
Co-authored-by: Daniel Chao <daniel.h.chao@gmail.com>
This implements HTTP redirect following ourselves.
The goal is:
1. All I/O is checked against `--allowed-resources` and
`--allowed-modules`, including HTTP redirects
2. HTTP rewrite rules can affect redirect following
3. HTTP headers can affect redirect following
---------
Co-authored-by: Islon Scherer <islonscherer@gmail.com>
This reverts the commits that enabled Gradle's configuration cache
feature.
IMO: this feature is too hard to use. We don't know if a task is valid
for the configuration cache until it runs, and it's very hard to tell if
something is safe when authoring Gradle code.
For example, our publish tasks are currently failing; I don't know how I
would fix this without running the publish task again on my dev machine.
Also, some of our build scripts become more brittle because of this; for
example, see
https://github.com/apple/pkl/blob/bb07589eae0b3195a589559a3245cbc12c29b394/build-logic/src/main/kotlin/BuildInfo.kt#L291-L296
The added snippet test originally produced error "A value of type
`Function2` cannot be exported."
This PR actually fixes the bug twice:
* By marking `ConvertProperty.render` as `hidden` so that it is skipped
when the enclosing object is exported. This broke any attempts to obtain
the module schema because this requires exporting all annotations on all
class properties.
* By changing the way that `VmUndefinedValueException.fillInHint()`
obtains the module URI to avoid obtaining the module schema (and
triggering the more expensive module schema generation process).
It also makes function-typed annotation properties in `pkl:Command` hidden to avoid similar issues there.
Bumps `shadowPlugin` from 9.4.1 to 9.4.2.
Updates `com.gradleup.shadow:com.gradleup.shadow.gradle.plugin` from
9.4.1 to 9.4.2
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/GradleUp/shadow/releases">com.gradleup.shadow:com.gradleup.shadow.gradle.plugin's
releases</a>.</em></p>
<blockquote>
<h2>9.4.2</h2>
<h3>Changed</h3>
<ul>
<li>Update jdependency to support Java 27. (<a
href="https://redirect.github.com/GradleUp/shadow/pull/2033">#2033</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/GradleUp/shadow/commit/29c432a7b6824b6fef23d46dc067e5c14112ff90"><code>29c432a</code></a>
Prepare version 9.4.2</li>
<li><a
href="https://github.com/GradleUp/shadow/commit/8aa8e6c1fb2ac0c17f0ff77e521cbe599cf64245"><code>8aa8e6c</code></a>
Update dependency org.vafer:jdependency to v2.16 (<a
href="https://redirect.github.com/GradleUp/shadow/issues/2033">#2033</a>)</li>
<li>See full diff in <a
href="https://github.com/GradleUp/shadow/compare/9.4.1...9.4.2">compare
view</a></li>
</ul>
</details>
<br />
Updates `com.gradleup.shadow` from 9.4.1 to 9.4.2
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/GradleUp/shadow/releases">com.gradleup.shadow's
releases</a>.</em></p>
<blockquote>
<h2>9.4.2</h2>
<h3>Changed</h3>
<ul>
<li>Update jdependency to support Java 27. (<a
href="https://redirect.github.com/GradleUp/shadow/pull/2033">#2033</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/GradleUp/shadow/commit/29c432a7b6824b6fef23d46dc067e5c14112ff90"><code>29c432a</code></a>
Prepare version 9.4.2</li>
<li><a
href="https://github.com/GradleUp/shadow/commit/8aa8e6c1fb2ac0c17f0ff77e521cbe599cf64245"><code>8aa8e6c</code></a>
Update dependency org.vafer:jdependency to v2.16 (<a
href="https://redirect.github.com/GradleUp/shadow/issues/2033">#2033</a>)</li>
<li>See full diff in <a
href="https://github.com/GradleUp/shadow/compare/9.4.1...9.4.2">compare
view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This introduces breaking changes for external readers are loaded:
1. In PklProject, relative paths are resolved relative to the enclosing
PklProject file (make behavior consistent with how other settings work)
2. Make CLI flags blow away any settings set on a PklProject
3. Introduce a new `workingDir` property, which defaults to the
PklProject dir
The overall goal is to make this behavior consistent with how other
settings work.
For example, relative paths for other evaluator settings are already
relative to the project directory.
Additionally, in every other case, CLI flags will overwrite any setting
set within PklProject.
Previously, `VmDynamic.isHiddenOrLocalProperty` didn't correctly
identify locals whose cache key is an `ObjectMember` instead of an
`Identifier`, causing `VmDynamic. getRegularMemberCount` to return an
incorrect value. This caused some renderers to produce incorrect output.
Resolves#1631
This introduces an optimization to write let expressions to frame slots
of the current frame, rather than transforming them into lambdas. As a
result, let expressions are _much_ faster to execute.
Fixes the following error:
```
A problem was found with the configuration of task ':pkl-core:sourcesJar' (type 'Jar').
Deprecated Gradle features were used in this build, making it incompatible with Gradle 10.
- Gradle detected a problem with the following location: '/home/runner/work/pkl/pkl/pkl-core/build/generated/sources/baseModuleMembers'.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
Reason: Task ':pkl-core:sourcesJar' uses this output of task ':pkl-core:generateBaseModuleMembers' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.
For more on this, please refer to https://docs.gradle.org/9.5.1/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
Possible solutions:
94 actionable tasks: 76 executed, 18 from cache
1. Declare task ':pkl-core:generateBaseModuleMembers' as an input of ':pkl-core:sourcesJar'.
2. Declare an explicit dependency on ':pkl-core:generateBaseModuleMembers' from ':pkl-core:sourcesJar' using Task#dependsOn.
3. Declare an explicit dependency on ':pkl-core:generateBaseModuleMembers' from ':pkl-core:sourcesJar' using Task#mustRunAfter.
```
This replaces `ResolveVariableNode` and `ResolveMethodNode` with their resolution.
When we build the truffle node tree, we determine whether names resolve to:
* lexical scope
* base module
* implicit this
Then, we use this information to directly construct the underlying nodes (`ReadPropertyNode`, `ReadLocalPropertyNode`, etc).
Additionally, `AstBuilder` determines whether the property access must be const or not.
This introduces a `BaseModuleMembers` registry, which gets generated as part of Java compilation.
Use Locale.ROOT to apply the lowecase format. For URI scheme and host
locale-neutral casing is the semantically the correct choice. Added a
unit test that sets the default locale to tr-TR and that would fail
without the fix.
- Remove single usage of @immutable without replacement
- Remove HttpClient's usages of @threadsafe without replacement
- Replace javax.annotation.concurrent.GuardedBy
with com.google.errorprone.annotations.concurrent.GuardedBy
Also:
- Remove redundant final modifiers from members of a final class
---------
Co-authored-by: odenix <self@odenix.org>
With JSpecify now a dependency of pkl-config-java, this moves the
non-null annotation to jspecify's.
This makes it simpler for users to do nullness checks, as tooling
already understands JSpecify nullness annotations.
* Relax forbidden headers constraints
- remove restriction on browser-related headers
- allow any glob pattern (no need to end with `/` or `*`, because glob
patterns already require users to explicitly declare prefix matches if
that's the intention)
* Replace `List<Pair<, ...>>`; use `Map<String, ...>` instead
* Use glob pattern strings as an API throughout, instead of `Pattern`
(e.g. in `HttpClientBuilder`)
* Add HTTP headers to message passing API
* Add HTTP headers to executor API (introduces `ExecutorSpiOptions4`)
* Add tests for Gradle, CLI, and pkl-executor invocations
* Improve documentation
* Add `isGlobPattern` API to class `String` for in-language validation
of http headers
* Behavior change: make sure explicitly configured `User-Agent` in
`HttpClientBuilder` can be shadowed by headers (allows users to set
`--http-header "**=User-Agent: My User Agent"` and for this to be the
only user agent).
CC @kyokuping
IntelliJ can understand that some annotations on fields mean that they
are implicitly initialized, which means we don't get the "field XXX is
not initialized" warning for `@LateInit` fields.
This setting, unfortunately, is recorded into `.idea/misc.xml`, which
contains a bunch of arbitrary stuff that we don't want to check into
source control
This adds some logic to touch up that file to mark `@LateInit` as
implicitly initialized fields, so we don't get any editor warnings.
Also, suppress some warnings.
Replace pkl-core's local nullness annotations with JSpecify annotations.
Enable NullAway checking for pkl-core packages except org.pkl.core.ast
and org.pkl.core.stdlib.
Notable code changes:
- Add a dedicated late-init constructor to VmTyped
- Move VmExceptionBuilder's fallback message derivation from withCause()
to build()
- Split VmException rendering between builder-provided messages and
string-backed messages
- Initialize MessageTransport handlers with default throwing handlers
- Update JSON helper collection types to allow nullable values JSON
arrays and objects can contain JSON null,
so the Java Map/List element types need to model nullable elements
explicitly
- Make public command transform APIs accept nullable transformed values
Command transforms can produce null for optional/default handling,
so the BiFunction and options-map element types now model that
explicitly
- Make ExecutorSpiException accept nullable message and cause
Existing call sites can pass nullable causes from Throwable.getCause()
- Remove JSR-305 semantics from `@LateInit`
JSpecify does not support the same type-qualifier-nickname pattern,
so `@LateInit` is now documentation plus a NullAway
constructor-initialization exemption
Out of scope:
- NullAway checking of org.pkl.core.ast and org.pkl.core.stdlib
- IntelliJ warnings related to `@LateInit` fields
- Removing the JSR-305 dependency, since concurrency annotations are
still in use
Motivation:
Config.as() causes nullness warnings when its result is intentionally assigned
to a non-null variable
Changes:
* Introduce Config.asNullable(Class<T>), asNullable(JavaType<T>), and
asNullable(Type) to explicitly opt into nullable values
* Keep the signatures of Config.as(Class<T>) and Config.as(JavaType<T>)
unchanged from 0.31 by adding @NullUnmarked
* This gives users time to migrate from as() to asNullable() where appropriate
* Avoids introducing new spurious warnings
* Change `<T> T Config.as(Type)` to `<T extends @nullable Object> T Config.as(Type)`
* This overload is typically used by reflective code such as
pkl-config-kotlin's Config.to() rather than directly by user code
* Clarify that JavaType<T> represents a non-null top-level type whose type arguments may be nullable
* Restricting <T> to non-null keeps method signatures understandable for humans and tools
* Enables full symmetry between Class<T> and JavaType<T> overloads in Config and JavaType
* Enables future non-null runtime checks in both Config.as() overloads
* Simplify construction of `JavaType`s with nullable type arguments
* Add ofNullable() variants for most factory methods, e.g., JavaType.listOfNullable()
* Overhaul Javadoc of Config and JavaType
Result:
* Clear separation between accessing nullable and non-null values
* Config.as() is used for the common non-null case
* Config.as() can perform non-null runtime checks in a future release (breaking change)
* More ergonomic construction of types with nullable type arguments
* More detailed and consistent documentation
* Fix error message when an invalid test reporter is supplied in Gradle
* Fix Gradle property name in docs
* Fix Gradle property name in tasks
* Introduce `TestReporter.default`, and use it in places where default
is applied
* Remove calls to `convention()`; this is not required because the input
is optional anyways.
When a docsite has only one package name and no DocsiteInfo.overview,
treat it like Javadoc's single-module output: redirect the top-level
index to the package page and omit the site-title breadcrumb segment
from generated pages.
Add src/test/files/SinglePackageTest fixtures to cover multiple package
versions, redirect behavior, breadcrumb behavior, and unchanged site
structure.
Also:
- Shut down Executor used in test.
- Declare expected output fixtures of DocGenerator as test inputs, not
outputs.
- Fix IntelliJ warning by using a Set for the right-hand side of
collection subtraction.
The version of local project dependencies should _always_ exactly match
up with what's declared in a PklProject.deps.json; any package in the
transitive dependency tree should always be delcaring the same import
too.
Closes#1591
run() now creates and closes a default Executor per call. This is fine
because there is no good reason to call this method multiple times.
run(Executor) now lets callers provide their own Executor, which is
customary for a well-behaved library.
Also: Fix IntelliJ warning by calling toSet()
Closes#1583
This omission, in particular, prevents Gradle plugins (which rely on CLI
classes) from adding custom resource readers via the service loading
mechanism. This change seems benign, especially since this is already
done for module key factories.
Power assertions only work when the source section is available. If it
is unavailable, power assertions throw a ParserError (unexpected EOF on
an empty input) when re-parsing the expression for presentation.
This change allows `PklProject` files, usually loaded via the `Project`
static methods, to have references to external packages via `package://`
URIs.
This is helpful for centralizing and sharing common package
configuration via packages.
ktfmt has much improved how it formats Kotlin code. Unfortunately, this
means that whenever we touch a single line in a Kotlin file, we get a
_lot_ more changes thanks to ratcheting now picking up this file for
formatting.
This PR just reformats every single Kotlin file so we don't have to deal
with this churn in future PRs that touch Kotlin code.
This is a quality-of-life improvement; make our build logs more easy to
read through for the default case.
If we need more information, we can click on the "Enable debug logging"
checkbox when re-running a job, which then populates the `runner.debug`
context variable.
Modern versions of Gradle support configuration caching
to prevent the gradual increase of project size to affect
the overall developer experience of Gradle builds. To
prepare the PKL project, and specificall pkl-gradle, for
configuration support, we introduce an integration test to
vet configuration cache rules, and then perform the necessary
updates to provide configuration cache support.
Motivation:
- `Config` mixes configuration representation with decoding logic
- `Config.fromPklBinary()` does not scale as decoding gains options
(e.g., binary versions or formats)
- The decoding API is inconsistent with `ConfigEvaluator`
Changes:
- Introduce `ConfigDecoder` (with builder) and move
`Config.fromPklBinary()` logic into it
- Deprecate `Config.fromPklBinary()` methods for removal
- Add `ConfigDecoder.forKotlin()` extension function
- Update and improve tests
Result:
- Decoding is separated from `Config` and exposed via a dedicated API
- Decoding can evolve independently (e.g., adding options such as binary
versions or supporting new formats)
- Evaluation and decoding APIs follow a consistent design
Dependabot currently does not update lockfiles in multi-module projects
(see https://github.com/dependabot/dependabot-core/issues/14633)
To work around this issue, we will simply remove our lockfiles, and
change our version catalog to use fully specified versions.
The removal of lockfiles introduces two issues:
1. It is less visible what our dependency graph is
2. Our builds are potentially non-reproducible
To work around this, two mitigations are in place:
1. Enable `failOnDynamicVersions()`, which causes Gradle to fail the
build if any dependencies declare a version range
2. Enable GitHub dependency submission, which provides insight into the
project SBOM
Fixes the following pom.xml issues:
1. pkl-doc and pkl-codegen-java sets the wrong dependency scopes for
pkl-commons-cli/pkl-base
2. pkl-config-kotlin sets the wrong dependency scope for
pkl-config-java-all
Closes#1293Closes#1517
This doesn't really make sense as part of the `Config` API.
We can maybe make class `ConfigUtils` public, but, I don't know how useful it
is anyways; it's more of an implementation detail.
Motivation:
buildSrc is a special-case legacy mechanism.
Gradle recommends using an included build named build-logic instead:
https://docs.gradle.org/current/userguide/best_practices_structuring_builds.html#favor_composite_builds
Changes:
- Rename buildSrc/ to build-logic/
- triggers reformatting
- Replace occurrences of "buildSrc" with "build-logic"
- Include the build-logic build in the main build (via
settings.gradle.kts)
- Apply convention plugins via plugin IDs instead of type-safe accessors
- small tradeoff compared to buildSrc
Result:
- Faster and more isolated builds
- Build logic behaves like a normal build, making it easier to evolve
and reason about
---------
Co-authored-by: Daniel Chao <dan.chao@apple.com>
- Enforce Kotlin version via resolution rule (replaces BOM)
- fail if kotlin-stdlib/kotlin-reflect exceed target version
- Replace kotlin-stdlib-jdk8 with kotlin-stdlib (jdk7/8 are now shims)
- Port pkl-core annotation processor to Java (with Codex)
- removes kotlin-stdlib from its compile classpath for better dependency
hygiene (Java module)
- Downgrade clikt for Kotlin 2.2 compatibility
- Upgrade kotlinx-serialization
---------
Co-authored-by: Daniel Chao <dan.chao@apple.com>
Motivation
- Enable correct NullAway analysis
- Pick up toolchain fixes and improvements
Toolchains
- Require JDK 25 for JVM toolchain (keep Java 17 runtime compatibility)
- Require Kotlin 2.3.20 for Kotlin toolchain (keep Kotlin 2.2 runtime
compatibility)
- Require JDK 25 for Gradle daemon JVM (via
gradle-daemon-jvm.properties)
- Fix javac and kotlinc warnings from toolchain upgrades
CI
- Bump GitHub workflows to JDK 25
Building Kotlin
- Bump Kotlin language level to 2.2 to match stdlib version
- Consolidate build logic into pklKotlinBase.gradle.kts
- Adopt modern Kotlin plugin syntax
- Fix new kotlinc warnings
- Update ktfmt to 0.62
- first version compatible with Kotlin 2.3.20
- changes formatting compared to 0.61
- Replace dependency resolution rule with BOM alignment
- rule was too broad and interfered with toolchain/runtime separation
Testing
- Expand matrix to JDK 25 (LTS) and 26
- Ensure each matrix task can be run independently
- Fix KotlinCodeGeneratorsTest and EmbeddedExecutorsTest on affected
JDKs
- Disable one test in CliCommandTest on affected JDKs (failure cause
unknown)
Compatibility fixes
- Fix reflective access in DocGenerator on affected JDKs
Build fixes
- Fix misuse of `task.enabled` vs. `report.required`
- Fix `gradlew tasks` on Windows
- Downgrade Spotless to 8.3.0 to (hopefully) work around sporadic
NoClassDefFoundError
Result
- NullAway runs correctly
- Broader JDK test coverage
- More reproducible and potentially faster builds
This avoids an issue where, during the course of development, a file is
touched, thus modifying the copyright year.
Then, undoing the previous change does not undo the copyright year
change.
Motivation:
Facilitate the use of the NullAway checker as part of moving to
JSpecify.
Changes:
- represent "no children" as `List.of()` instead of null
- remove obsolete `children != null` assertions
- NullAway intentionally ignores such assertions
- remove "no children" special-casing where no longer necessary
Result:
- cleaner code with similar performance
- removed a barrier to using the NullAway checker
- Remove dependency org.fusesource.jansi:jansi
- In 4.x, org.fusesource.jansi:jansi was replaced with org.jline:jansi.
Instead of adding this new dependency, this commit replaces Pkl’s single
Jansi usage with custom code that preserves existing behavior. Fixing
existing ANSI quirks is left for a future PR.
- Replace jline-terminal-ansi with jline-terminal-jni
- In 4.x, only -jni and -ffm are available (-ffm requires Java 22+)
- Configure native-image build for jline-terminal-jni
As updating JLine is delicate, I manually tested `pkl repl` and `jpkl
repl` on Windows 11 (using Windows Terminal) and on Ubuntu, and found no
issues. However, I do not have access to a macOS machine.
- Make leaf AST classes final
- Make protected Lexer fields private and add getter
- Split Parser into Parser and ParserImpl
- Using a fresh ParserImpl instance per parse simplifies reasoning
(important) and makes the Parser API thread-safe (nice to have)
- Split GenericParser into GenericParser and GenericParserImpl
- Same motivation as for Parser
Some of these changes will facilitate the move to JSpecify, which has
proven challenging for this package.
- pass `GrammarVersion` to constructor instead of passing it to each
`format` method
- replace `format(Path): String` with `format(Reader, Appendable)`
- instead of picking which overloads besides `format(String): String`
might be useful, offer a single generalized method that streams input
and output
- add `@Throws(IOException::class)` to ensure that Java callers can
catch this exception
- deprecate old methods
Prior to this change, this code would activate powers assertions /
instrumentation permanently:
```pkl
foo: String(contains("a")) | String(contains("b")) = "boo"
```
This is because the `contains("a")` constraint would fail, triggering
power assertions, but the subsequent check of the union's
`contains("b")` branch would succeed.
As observed in #1419, once instrumentation is enabled, all subsequent
evaluation slows significantly.
As with #1419, the fix here is to disable power assertions via
`VmLocalContext` until we know that all union members failed type
checking; then, each member is re-executed with power assertions allowed
to provide the improved user-facing error.
Fixes broken links and grammar in DEVELOPMENT.adoc:
- Mailing list URL: openjdk.java.net → openjdk.org (old domain returns
301)
- Truffle FAQ link: old OpenJDK wiki is gone (404) — replaced with
current GraalVM Truffle docs
- Grammar: "enables to run" → "enables you to run"
- Grammar: "jenv use specific" → "jenv uses specific"
The loop unwraps nullables and constraints but breaks straight away
after a `typealias`. This means the nullable is missed. Removing the
`break` fixes it.
## Exception
```
org.pkl.core.PklException: –– Pkl Error ––
Command option property `foo` has unsupported type `String?`.
11 | foo: OptionalString
^^^^^^^^^^^^^^^^^^^
at <unknown> (file:///var/folders/xh/lmp1n6qj4m13t53cfmbqnkwh0000gn/T/junit-1378070630576324311/cmd.pkl)
Use a supported type or define a transformEach and/or transformAll function
```
The `choices` stream was consumed eagerly for metavar construction, then
captured in a lambda for later validation—which promptly fell over with
`IllegalStateException`. Materialise to a `List` straightaway.
based on version information from https://search.maven.org, https://plugins.gradle.org, and GitHub repos
. Run `gw updateDependencyLocks`
. Validate changes with `gw build buildNative`
. Review and commit the updated dependency lock files
== Code Generation
@@ -92,14 +90,9 @@ Example: `./gradlew test -Djvmdebug=true`
== Snippet Test Plugin
There is an IntelliJ plugin meant for development on the Pkl project itself.
This plugin provides a split pane window when viewing snippet tests such as LanguageSnippetTests and FormatterSnippetTests.
There is an IntelliJ plugin meant for development on the Pkl project itself located in https://github.com/apple/pkl-project-commons[pkl-project-commons].
To install:
1. Run `./gradlew pkl-internal-intellij-plugin:buildPlugin`.
2. Within IntelliJ, run the action "Install Plugin From Disk...".
3. Select the zip file within `pkl-internal-intellij-plugin/build/distributions`.
See https://github.com/apple/pkl-project-commons?tab=readme-ov-file#internal-intellij-plugin[its readme] for instructions on how to set it up.
== Resources
@@ -110,12 +103,12 @@ For automated build setup examples see our https://github.com/apple/pkl/blob/mai
NOTE: This is the final release that distributes binaries for macOS/amd64.
== Highlights [small]#💖#
=== New data type: `Reference`
This release introduces a new data type called `Reference` (https://github.com/apple/pkl/pull/1354[#1354], https://github.com/apple/pkl/pull/1691[#1691], https://github.com/apple/pkl/pull/1692[#1692], https://github.com/apple/pkl/pull/1693[#1693], https://github.com/apple/pkl/pull/1695[#1695]).
WARNING: This API is experimental. We are looking for feedback, and will aim to make this ready for production use in the next release.
A reference represents something whose actual value is unknown to Pkl, but nevertheless its type is checked.
The purpose of this data type is to improve how Pkl can be used to configure systems like GitHub Actions, Pulumi, Terraform, and more.
For example, let's assume that some system accepts YAML configuration that looks like so:
[source,yaml]
----
taskA:
uses: some-task
taskB:
uses: some-other-task
input: '${ taskA.output.number }'
----
In terms of YAML, `'${ taskA.output.number }'` is just a string.
However, semantically, it is an _expression_ in this target system, where `taskA.output.number` conveys some type.
Today, the only way to treat these values in Pkl is to also describe these as strings.
This introduces many shortcomings:
1. It leaves users vulnerable to syntax errors in this system.
2. It's hard to validate these beyond that they are a string.
3. Pkl has no understanding of the relationship between `taskA` and `taskB`.
With references, the above can be modeled in Pkl also as plain lookups:
[source,pkl]
----
taskA {
uses = "some-task"
}
taskB {
name = "some-other-task"
input = taskA.output.number
}
----
Library authors describe types as `ref.Reference<D, T>`, where `D` is the _domain_, and `T` is the _referent type_.
[source,pkl]
----
import "pkl:ref"
class MyDomain extends ref.Domain { // <1>
function renderReference(ref: ref.Reference<MyDomain, Any>): String =
<1> References exist inside a domain. The domain defines how these references should be stringified.
<2> A typealias can be used to improve ergonomics when declaring references that share the same domain.
Pkl will check that type arguments match up:
* The referent type must line up.
+
`MyReference<Int>` is not assignable to `MyReference<String>`.
* The domain must line up.
+
`ref.Reference<MyDomain, Int>` is not assignable to `ref.Reference<OtherDomain, Int>`.
Additionally, references give you synthetic members.
[source,pkl]
----
class Bird {
name: String
}
bird: MyReference<Bird>
birdName = bird.name // <1>
----
<1> Gives a `MyReference<String>`, because `class Bird` has property `name: String`
The synthetic member contains metadata about the path that was used to access it (`.name` in this
case).
This can be used to stringify this reference in that target domain.
To read more about references, consult the xref:language-reference:index.adoc#references[language reference].
To read through the design decisions made, consult the https://github.com/apple/pkl-evolution/blob/main/spices/SPICE-0020-type-safe-deferred-references.adoc[SPICE].
=== Custom HTTP Headers
Pkl can now attach custom HTTP headers to outbound requests, making it possible to access private or authenticated resources (https://github.com/apple/pkl/pull/1196[#1196], https://github.com/apple/pkl/pull/1584[#1584]).
For example, a CLI user can add an `Authorization` header either in settings.pkl, or the PklProject file:
[tabs]
====
~/.pkl/settings.pkl::
+
[source,pkl]
----
amends "pkl:settings"
http {
headers {
["https://my.private.server/**"] {
["Authorization"] = "Bearer my-secret-token"
}
}
}
----
PklProject::
+
[source,pkl]
----
amends "pkl:Project"
evaluatorSettings {
http {
headers {
["https://my.private.server/**"] {
["Authorization"] = "Bearer my-secret-token"
}
}
}
}
----
====
Each key in `headers` is a glob pattern matched against the request URL.
When a request is made, every matching pattern's headers are added to the request.
Headers can also be configured in other ways:
* In the pkl-gradle plugin
* As a CLI flag
* In the Java/Swift/Go/Kotlin evaluator APIs
* In the pkl-executor API
Header names must conform to RFC 7230 token syntax.
Certain reserved names (e.g. `host`, `connection`, `content-length`) and prefixes (`proxy-`, `sec-`) are forbidden.
Thank you to https://github.com/kyokuping[@kyokuping] for their contributions to this feature!
To learn more, consult https://github.com/apple/pkl-evolution/blob/main/spices/SPICE-0022-http-headers.adoc[SPICE-0022].
== Noteworthy [small]#🎶#
=== Resolved evaluator settings
PklProject files now resolve evaluator settings that are file paths (https://github.com/apple/pkl/pull/1394[#1394]).
Some of the settings within `pkl:EvaluatorSettings` represent file paths.
However, these paths are resolved inconsistently:
* Pkl is inconsistent about what relative paths mean. `rootDir`, `moduleCacheDir`, and `modulePath` are relative to the project dir, while `ExternalReader.executable` is relative to the PWD.
* External readers defined in a PklProject have different behavior depending on the PWD.
* The logic for resolving these paths is dependent on the caller, rather than Pkl.
For CLI users, this means that the current working directory can affect how evaluator settings are loaded.
In Pkl 0.32, these paths are resolved entirely within Pkl, and resolved against the enclosing directory.
A new method `resolve()` is added to `pkl:EvaluatorSettings`, and a new property `resolvedEvaluatorSettings` is added to `pkl:Project`.
Language bindings are expected to use `resolvedEvaluatorSettings` when configuring the evaluator.
To read more about this change, consult https://github.com/apple/pkl-evolution/blob/main/spices/SPICE-0027-resolved-paths-in-pkl-evaluatorsettings.adoc[SPICE-0027].
=== Line continuations in multiline strings
When authoring multiline strings, it is now possible to use a _line continuation_ to break a single line's value over multiple lines (https://github.com/apple/pkl/pull/1507[#1507], https://github.com/apple/pkl/pull/1564[#1564]).
These two string snippets are logically identical:
[source,pkl]
----
str =
"""
Although the Dodo is extinct, \
the species will be remembered.
"""
----
[source%parsed,pkl]
----
str = "Although the Dodo is extinct, the species will be remembered."
----
To read more about this design, consult https://github.com/apple/pkl-evolution/blob/main/spices/SPICE-0028-multiline-string-continuation.adoc[SPICE-0028].
[[parse-time-variable-resolution]]
=== Parse-time variable resolution
Pkl now resolves variables at parse time (https://github.com/apple/pkl/pull/1429[#1429], https://github.com/apple/pkl/pull/1622[#1622], https://github.com/apple/pkl/pull/1634[#1634], https://github.com/apple/pkl/pull/1706[#1706]).
Currently, Pkl defers the resolution of variable names to when the node is executed.
In Pkl 0.32, these are resolved when the file is parsed.
This is a pre-requisite feature for many upcoming features, such as flat member syntax and method varargs.
Additionally, this enables some runtime optimizations, including let expressions.
[[let-expression-improvements]]
=== Let expression performance improvements
Performance improvements were made to the handling of xref:language-reference:index.adoc#let-expressions[let expressions] (https://github.com/apple/pkl/pull/1622[#1622], https://github.com/apple/pkl/pull/1634[#1634]).
Here is a sample benchmark:
[source,pkl]
----
amends "pkl:Benchmark"
microbenchmarks {
["let"] {
expression =
let (a = 1)
let (b = 2)
let (c = 3)
let (d = 4)
let (e = 1)
let (f = 2)
let (g = 3)
let (h = 4)
let (i = 1)
let (j = 2)
let (k = 3)
let (l = 4)
let (m = 1)
let (n = 2)
let (o = 3)
let (p = 4)
let (q = 1)
let (r = 2)
let (s = 3)
let (t = 4)
a + b + c + d + e + f + g + h + i + j + k + l + m + n + o + p + q + r + s + t
And here are the results when run on a MacBook Pro M4 Max machine: `jpkl` is about 12x faster, while native `pkl` is about 4000x faster:
|===
|Variant |Results (mean)
|jpkl
|1208.06.ns -> 97.76.ns
|pkl
|1275.96.ns -> 0.32.ns
|===
=== Test reporter
A test reporter option is added when running tests (https://github.com/apple/pkl/pull/1563[#1563], https://github.com/apple/pkl/pull/1597[#1597]).
A new CLI flag, `--test-reporter`, is now available to the `pkl test` and `pkl project package` commands.
This flag accepts the name of a test reporter to be used to format test result output.
There are two possible reporters:
* `spec` (default) - the current reporter
* `minimal` - a reporter that only emits failing tests
This option is also available to pkl-gradle users.
=== Dependency notation improvements
Pkl's evaluator now accepts https://pkl-lang.org/main/current/language-reference/index.html#dependency-notation[dependency notation URIs] as source modules (https://github.com/apple/pkl/pull/1595[#1595]).
In Pkl 0.31, we introduced xref:0.31.adoc#cli-dependency-notation[CLI support for dependency notation].
Now, this support has been extended to the evaluator itself.
This means that users of Pkl's various language bindings can also use dependency notation.
Additionally, the limitation on local dependencies has been dropped.
=== pkldoc supports single-package docsites
pkldoc has improved support for single-package documentation websites (https://github.com/apple/pkl/pull/1592[#1592]).
When a docsite has only one package name, and also no overview from a `docsite-info.pkl`, the resulting docsite does not have a top-level package index page.
Instead, it redirects to the package page, and omits the site-level element from the breadcrumb.
=== Allowed package imports in `PklProject`
PklProject files can now import xref:language-reference:index.adoc#package-asset-uri[package asset URIs] (https://github.com/apple/pkl/pull/1547[#1547]).
This means that it is now possible to centralize and share common configurations across multiple projects.
=== Java Library Changes
==== Switch to JSpecify annotations
Pkl has switched to using https://jspecify.dev[JSpecify] for annotating nullability (https://github.com/apple/pkl/pull/1515[#1515], https://github.com/apple/pkl/pull/1527[#1527], https://github.com/apple/pkl/pull/1528[#1528], https://github.com/apple/pkl/pull/1530[#1530], https://github.com/apple/pkl/pull/1544[#1544], https://github.com/apple/pkl/pull/1601[#1601], https://github.com/apple/pkl/pull/1607[#1607]).
Currently, Pkl uses annotations from the `com.google.code.findbugs:jsr305` library.
This suffers from some issues:
* The https://jcp.org/en/jsr/detail?id=305[JSR-305 proposal] itself is dormant and unlikely to be adopted.
* The `com.google.code.findbugs:jsr305` library contributes packages to `javax`, which violates the https://www.oracle.com/downloads/licenses/binary-code-license.html[Oracle Binary Code License Agreement].
In contrast, JSpecify is widely seen as the standard for nullability annotations, and is already understood by most tools.
This impacts Pkl's Java APIs; methods previously annotated using JSR-305 nullability annotations now use JSpecify.
Additionally, the Java code generator now emits JSpecify annotations by default.
==== New API: `org.pkl.config.java.ConfigDecoder`
A new API is introduced called `org.pkl.config.java.ConfigDecoder` (https://github.com/apple/pkl/pull/1533[#1533]).
Currently, the `org.pkl.config.java.Config` interface mixes configuration representation with decoding logic.
To better separate these concerns, a new API is introduced for decoding.
Example:
[source,java]
----
class Main {
Config getConfig() {
var decoder = new ConfigDecoderBuilder().preconfigured().build();
return decoder.decode(bytes);
}
}
----
==== New `asNullable` methods in `pkl.config.java`
New methods are added for decoding into nullable types (https://github.com/apple/pkl/pull/1544[#1544]).
** Pass `GrammarVersion` to constructor instead of each `format` method
** Replace `format(Path): String` with `format(Reader, Appendable)`
** Mark as `throws IOException`
** Deprecate methods that accept `GrammarVersion` as an argument
=== Loading rule changes in `pkl:EvaluatorSettings`
Breaking changes have been made to how evaluator settings are loaded when using `PklProject` (https://github.com/apple/pkl/pull/1394[#1394]).
==== Loading rule changes for the external reader executable
The following changes have been made for the `executable` property in an external reader:
* If the executable does not contain a slash (`/` on POSIX, `\` on Windows) character, it is always resolved against the `PATH` environment variable.
* If it does contain a slash, it is resolved relative to the enclosing PklProject directory, instead of the current working directory.
=== Changes to `--external-module-reader` and `--external-resource-reader` CLI flags
The `--external-module-reader` and `--external-resource-reader` CLI flags will _replace_ any external readers otherwise configured within a PklProject, instead of add to it (https://github.com/apple/pkl/pull/1394[#1394]).
This makes this behavior consistent with how other settings work.
== Bug Fixes [small]#🐜#
The following bugs have been fixed.
* Data race in MessagePack encoder during concurrent server sends (https://github.com/apple/pkl/issues/1486[#1486])
* relativePathTo does not check if receiver is a module (https://github.com/apple/pkl/issues/1649[#1649])
* `Int` overflow on multiplication throws unexpected exception (https://github.com/apple/pkl/issues/1651[#1651])
* Incorrect eager check for `Map` type (https://github.com/apple/pkl/issues/1653[#1653])
* Incorrect `toRadixString()` for `math.minInt` (https://github.com/apple/pkl/issues/1655[#1655])
* `Evaluator.evaluateSchema()` throws if properties are annotated with `@ConvertProperty` (https://github.com/apple/pkl/issues/1657[#1657])
* `String.padStart` and `String.padEnd` return incorrect strings sometimes (https://github.com/apple/pkl/issues/1661[#1661])
* `String.getOrNull` crashes with certain strings (https://github.com/apple/pkl/issues/1662[#1662])
* Incorrect facts in `pkl:base` doc comments (https://github.com/apple/pkl/pull/1669[#1669])
* Index-based methods on Set aren't implemented (https://github.com/apple/pkl/issues/1682[#1682])
* pkl-gradle throws exception when a Pkl task can't create an evaluator during configuration time (https://github.com/apple/pkl/pull/1684[#1684])
* Unrelated exception gets thrown when validating elements/entries in objects (https://github.com/apple/pkl/issues/1697[#1697])
* Type argument is lost in constraint expressions within generic typealiases (https://github.com/apple/pkl/issues/1705[#1705])
* Typealiases with type var as root are broken (https://github.com/apple/pkl/issues/1711[#1711])
* Incorrect `List`/`Set`/`Map`/`Listing`/`Mapping`/union type check behavior through typealiases (https://github.com/apple/pkl/issues/1710[#1710])
* Type aliases with type var as root are broken (https://github.com/apple/pkl/issues/1711[#1711])
* Type checks through aliases are always run lazily even when they should be eager (https://github.com/apple/pkl/issues/1716[#1716])
* Improve thread safety when evaluating `pkl:base` (https://github.com/apple/pkl/pull/1719[#1719])
* Incorrect member links in doc comments (https://github.com/apple/pkl/pull/1723[#1723])
* pkl-gradle plugin does not allow `projectpackage:` reads by default (https://github.com/apple/pkl/issues/1734[#1734])
* `IntSeq` incorrectly emits on empty ``IntSeq``s (https://github.com/apple/pkl/issues/1738[#1738])
== Security Fixes [small]#🔒#
The following security vulnerabilities were fixed:
* Remote packages can read local filesystem items by escaping local dependency roots (https://github.com/apple/pkl/security/advisories/GHSA-fgvf-hh2w-cxff[GHSA-fgvf-hh2w-cxff], https://github.com/apple/pkl/pull/1737[#1737])
* Packages can be read/written outside the configured cache directory (https://github.com/apple/pkl/security/advisories/GHSA-87qh-25w9-mh34[GHSA-87qh-25w9-mh34], https://github.com/apple/pkl/pull/1683[#1683])
== Contributors [small]#🙏#
We would like to thank the contributors to this release (in alphabetical order):
@@ -249,14 +248,20 @@ internal class Repl(workingDir: Path, private val server: ReplServer, private va
}
privatefunhighlight(str:String):String{
valansi=Ansi.ansi()
varnormal=true
// Inserting ANSI codes into a string that may already contain ANSI codes is problematic.
// This code preserves existing behavior but should eventually be removed.
valbuilder=StringBuilder()
varbold=false
for(partinstr.split("`","```")){
ansi.a(part)
normal=!normal
if(!normal)ansi.bold()elseansi.boldOff()
if(bold){
builder.append("\u001B[1m")
builder.append(part)
builder.append("\u001B[22m")
}else{
builder.append(part)
}
ansi.reset()
returnansi.toString()
bold=!bold
}
returnbuilder.toString()
}
}
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
dependabot[bot]