ktfmt has much improved how it formats Kotlin code. Unfortunately, this
means that whenever we touch a single line in a Kotlin file, we get a
_lot_ more changes thanks to ratcheting now picking up this file for
formatting.
This PR just reformats every single Kotlin file so we don't have to deal
with this churn in future PRs that touch Kotlin code.
Modern versions of Gradle support configuration caching
to prevent the gradual increase of project size to affect
the overall developer experience of Gradle builds. To
prepare the PKL project, and specificall pkl-gradle, for
configuration support, we introduce an integration test to
vet configuration cache rules, and then perform the necessary
updates to provide configuration cache support.
Dependabot currently does not update lockfiles in multi-module projects
(see https://github.com/dependabot/dependabot-core/issues/14633)
To work around this issue, we will simply remove our lockfiles, and
change our version catalog to use fully specified versions.
The removal of lockfiles introduces two issues:
1. It is less visible what our dependency graph is
2. Our builds are potentially non-reproducible
To work around this, two mitigations are in place:
1. Enable `failOnDynamicVersions()`, which causes Gradle to fail the
build if any dependencies declare a version range
2. Enable GitHub dependency submission, which provides insight into the
project SBOM
Motivation:
buildSrc is a special-case legacy mechanism.
Gradle recommends using an included build named build-logic instead:
https://docs.gradle.org/current/userguide/best_practices_structuring_builds.html#favor_composite_builds
Changes:
- Rename buildSrc/ to build-logic/
- triggers reformatting
- Replace occurrences of "buildSrc" with "build-logic"
- Include the build-logic build in the main build (via
settings.gradle.kts)
- Apply convention plugins via plugin IDs instead of type-safe accessors
- small tradeoff compared to buildSrc
Result:
- Faster and more isolated builds
- Build logic behaves like a normal build, making it easier to evolve
and reason about
---------
Co-authored-by: Daniel Chao <dan.chao@apple.com>
- Enforce Kotlin version via resolution rule (replaces BOM)
- fail if kotlin-stdlib/kotlin-reflect exceed target version
- Replace kotlin-stdlib-jdk8 with kotlin-stdlib (jdk7/8 are now shims)
- Port pkl-core annotation processor to Java (with Codex)
- removes kotlin-stdlib from its compile classpath for better dependency
hygiene (Java module)
- Downgrade clikt for Kotlin 2.2 compatibility
- Upgrade kotlinx-serialization
---------
Co-authored-by: Daniel Chao <dan.chao@apple.com>
Motivation
- Enable correct NullAway analysis
- Pick up toolchain fixes and improvements
Toolchains
- Require JDK 25 for JVM toolchain (keep Java 17 runtime compatibility)
- Require Kotlin 2.3.20 for Kotlin toolchain (keep Kotlin 2.2 runtime
compatibility)
- Require JDK 25 for Gradle daemon JVM (via
gradle-daemon-jvm.properties)
- Fix javac and kotlinc warnings from toolchain upgrades
CI
- Bump GitHub workflows to JDK 25
Building Kotlin
- Bump Kotlin language level to 2.2 to match stdlib version
- Consolidate build logic into pklKotlinBase.gradle.kts
- Adopt modern Kotlin plugin syntax
- Fix new kotlinc warnings
- Update ktfmt to 0.62
- first version compatible with Kotlin 2.3.20
- changes formatting compared to 0.61
- Replace dependency resolution rule with BOM alignment
- rule was too broad and interfered with toolchain/runtime separation
Testing
- Expand matrix to JDK 25 (LTS) and 26
- Ensure each matrix task can be run independently
- Fix KotlinCodeGeneratorsTest and EmbeddedExecutorsTest on affected
JDKs
- Disable one test in CliCommandTest on affected JDKs (failure cause
unknown)
Compatibility fixes
- Fix reflective access in DocGenerator on affected JDKs
Build fixes
- Fix misuse of `task.enabled` vs. `report.required`
- Fix `gradlew tasks` on Windows
- Downgrade Spotless to 8.3.0 to (hopefully) work around sporadic
NoClassDefFoundError
Result
- NullAway runs correctly
- Broader JDK test coverage
- More reproducible and potentially faster builds
This adds power assertions to Pkl!
This implements the SPICE described in
https://github.com/apple/pkl-evolution/pull/29
This follows the power assertions style of reporting also found in
Groovy, Kotlin, and others.
* Literal values are not emitted in the diagram
* Stdlib constructors of literals like `List(1, 2)` are also considered
literals
Power assertions are added to:
* Failing type constraints
* Failing test facts
Power assertions are implemented as a truffle instrument to observe
execution.
When an assertion fails, the instrument is created and the assertion is
run again to observe facts.
This incurs runtime overhead to collect facts, but has no impact on code
in the non-error case.
---------
Co-authored-by: Islon Scherer <islonscherer@gmail.com>
Implements a binary renderer for Pkl values, which is a lossless capturing of Pkl data.
This follows the pkl binary format that is already used with `pkl server` calls, and is
made available as a Java API and also an in-language API.
Also, introduces a binary parser into the corresponding `PObject` types in Java.
The current version of the kotlin-gradle plugin is not compatible with
Gradle 9.1, causing error `java.lang.NoSuchMethodError:
'org.gradle.api.Project org.gradle.api.artifacts.ProjectDependency.getDependencyProject()'`
Also, the Kotlin 2.0 language target is deprecated as of Kotlin 2.2.
* Bump Gradle to 9.1.0
* Bump foojay resolver to 1.0.0
* Fix build logic on windows
Also, remove support for kotlin gradle plugin less than 1.8.x, because:
* Class `org.gradle.api.plugins.Convention` is no longer available in the classpath in Gradle
* Continued support for legacy plugin would require heavy reflection, which is brittle and hard to verify
* Kotlin 1.7 is 3 years old and no longer updated
Currently, in order to update a pkl-doc documentation site,
almost the entire existing site is read in order to update metadata
like known versions, known subtypes, and more.
For example, adding a new version of a package requires that the
existing runtime data of all existing versions be updated.
Eventually, this causes the required storage size to balloon
exponentially to the number of versions.
This addresses these limitations by:
1. Updating the runtime data structure to move "known versions" metadata
to the package level (the same JSON file is used for all versions).
2. Eliminating known subtype and known usage information at a
cross-package level.
3. Generating the search index by consuming the previously generated
search index.
4. Generating the main page by consuming the search index.
Because this changes how runtime data is stored, an existing docsite
needs to be migrated.
This also introduces a new migration command, `pkl-doc --migrate`,
which transforms an older version of the website into a newer version.
This adds logic so that the Kotlin code generator also supports the
"addGeneratedAnnotation" flag.
Also:
* Add Antora documentation
* Adjust names (generated-annotation -> add-generated-annotation)
* Adjust doc comments
JaCoCo automatically excludes methods and classes annotated with @Generated from the coverage reports. This is very important to us as generated code should not normally be included in the coverage report. We want to measure the coverage of the code that we actually wrote and maintain, not the code that was automatically generated by tools.
By introducing a property generatedAnnotation (default value false) one could enable writing @Generated on Java types to be generated.
Co-authored-by: Nullpointer <mike.schulze@tealium.com>
This introduces a new `Bytes` standard library class, for working with
binary data.
* Add Bytes class to the standard library
* Change CLI to eval `output.bytes`
* Change code generators to map Bytes to respective underlying type
* Add subscript and concat operator support
* Add binary encoding for Bytes
* Add PCF and Plist rendering for Bytes
Co-authored-by: Kushal Pisavadia <kushi.p@gmail.com>
Some systems require junit report to be in a single file. For example `bazel` https://bazel.build/reference/test-encyclopedia needs single file to be available in `XML_OUTPUT_FILE` path.
To avoid implementing junit aggregation in pkl wrappers in different places this PR instead adds a `--junit-aggregate-reports` flag to return all junit reports as a single file.
Additional flag `--junit-aggregate-suite-name` is added to allow overriding global test suite name from default `pkl-tests`
* Update dependencies
1. Remove */gradle.lockfile files
2. Run `gradle updateDependencyLocks` and commit
* Update multi-JDK testing to use simple Test task, add junit-platform-launcher to dependencies
- Don't use JvmTestSuite (we don't use another test runner, we use the same classpath)
* Add junit-platform-launcher to libs (prevent an issue where junit-engine and junit-launcher can fall out of sync)
Using native libraries in Gradle plugins is problematic; this adds some flags that causes Truffle to use a fallback runtime that avoids loading native libraries.
This bumps Clikt from version 3 to version 5, which, among other things, improves
the help text formatting with colors.
Also:
* Add `--version` flag to pkldoc, pkl-codegen-java, pkl-codegen-kotlin
* Add help text to pkldoc, pkl-codegen-java, pkl-codegen-kotlin
Some systems have trouble with handling symlinks, which breaks the current directory links created by Pkldoc. In this PR, we add an alternative mode which creates a full copy of the latest published version contents in the current directory instead.
Co-authored-by: Dan Chao <dan.chao@apple.com>
When we updated spotless's Java and Kotlin formatter, we changed the underlying
formatting rules.
However, due to spotless ratcheting, these formatting changes don't get applied unless a file
gets touched in a commit.
To avoid future PRs introducing lines of change that aren't related to the intention of the PR,
this is a one-time format of all files.
The Kotlin 2.0 version upgrade made us incompatible with Gradle 8.1.
Given that it is now over 1.5 years old, this drops support for it, and
raises the minimum version to 8.2.
- update Kotlin from 1.7.10 to 2.0.21
- Kotlin 1.6 dependencies in Gradle lock files are expected because kotlinc,
which is also used by some tests, internally uses some 1.6 dependencies
for backwards compatibility reasons.
- update kotlinx-html and kotlinx-serialization
- adapt Kotlin code where necessary
- use Kotlin stdlib Path APIs where possible
- fix IntelliJ Kotlin inspection warnings
- reformat code with `./gradlew spotlessApply`
- ktfmt adds lots of trailing commas
- Add workaround to fix IntelliJ "unresolved reference" errors
Motivation:
Spring Boot configuration classes neither require nor benefit from annotating constructor parameters with their name.
The same is true for pkl-config-java configuration classes compiled with `-parameter`.
Changes:
- Change CLI parameter `--params-annotation` to accept a `none` value.
This is recommended in https://clig.dev/#arguments-and-flags and is how the `-F` parameter of the `ssh` command works.
- Change `paramsAnnotation` property in Gradle plugin, CliJavaCodeGeneratorOptions, and JavaCodegenOptions as follows:
- Change meaning of `null` from "generate org.pkl.java.config.mapper.Named annotations" to "do not generate annotations".
This is a breaking change (only) affecting users who explicitly set the property to `null` instead of omitting it.
- Change property default from `null` to:
`null` if `generateSpringBootConfig` is `true` and `org.pkl.java.config.mapper.Named` otherwise
- add tests
- update docs of this and other codegen options
Result:
Generated code does not contain unnecessary annotations.
This fixes an issue where certain modules tasks fail due to the plugin
attempting to analyze their imports, but the arguments may not actually be
Pkl modules.
For example, the pkldoc task accepts entire packages in its "sourceMoules" property.
This changes the gather imports logic to only analyze file-based modules.
This is also a performance improvement; non file-based modules are unlikely to import
files due to insufficient trust levels.
Also: fix a bug when generating pkldoc on Windows
Any thrown Pkl Errors are colored in the simple test report!
Also:
* Refactor `TextFormatter` to be more generic; rename to `TextFormattingStringBuilder`
* Adjust test report slightly (no emojis, add more spacing).
* Introduce `ColorTheme` class.
* Make stack frame descriptors colored as "faint"
Also: this changes the summary so it summarizes _all_ modules, rather than a summary per module.
---------
Co-authored-by: Islon Scherer <islonscherer@gmail.com>
Co-authored-by: Philip K.F. Hölzenspies <holzensp@gmail.com>
Changes:
* Move class `TestResults` to package `org.pkl.core`, because it is a public class (it's the result of `Evaluator#evaluateTest`)
* Change examples to treat individual examples as assertions in the same test. Previously, they were considered different tests with an incrementing number. This better aligns with how facts are treated.
* Change `TestResults` to be a record, and introduce builders.
* Remove "module" test result section (it is not really a section).
* Add javadoc to `TestResults`
* Formatting fix: prefix ✍️ emoji just like we do the ❌ and ✅ emojis
* Consider writing examples as failures, not successes. `pkl test` will exit with code 10 if the only failing tests are due to writing examples.
Islon Scherer