ktfmt has much improved how it formats Kotlin code. Unfortunately, this
means that whenever we touch a single line in a Kotlin file, we get a
_lot_ more changes thanks to ratcheting now picking up this file for
formatting.
This PR just reformats every single Kotlin file so we don't have to deal
with this churn in future PRs that touch Kotlin code.
Motivation
- Enable correct NullAway analysis
- Pick up toolchain fixes and improvements
Toolchains
- Require JDK 25 for JVM toolchain (keep Java 17 runtime compatibility)
- Require Kotlin 2.3.20 for Kotlin toolchain (keep Kotlin 2.2 runtime
compatibility)
- Require JDK 25 for Gradle daemon JVM (via
gradle-daemon-jvm.properties)
- Fix javac and kotlinc warnings from toolchain upgrades
CI
- Bump GitHub workflows to JDK 25
Building Kotlin
- Bump Kotlin language level to 2.2 to match stdlib version
- Consolidate build logic into pklKotlinBase.gradle.kts
- Adopt modern Kotlin plugin syntax
- Fix new kotlinc warnings
- Update ktfmt to 0.62
- first version compatible with Kotlin 2.3.20
- changes formatting compared to 0.61
- Replace dependency resolution rule with BOM alignment
- rule was too broad and interfered with toolchain/runtime separation
Testing
- Expand matrix to JDK 25 (LTS) and 26
- Ensure each matrix task can be run independently
- Fix KotlinCodeGeneratorsTest and EmbeddedExecutorsTest on affected
JDKs
- Disable one test in CliCommandTest on affected JDKs (failure cause
unknown)
Compatibility fixes
- Fix reflective access in DocGenerator on affected JDKs
Build fixes
- Fix misuse of `task.enabled` vs. `report.required`
- Fix `gradlew tasks` on Windows
- Downgrade Spotless to 8.3.0 to (hopefully) work around sporadic
NoClassDefFoundError
Result
- NullAway runs correctly
- Broader JDK test coverage
- More reproducible and potentially faster builds
Currently, in order to update a pkl-doc documentation site,
almost the entire existing site is read in order to update metadata
like known versions, known subtypes, and more.
For example, adding a new version of a package requires that the
existing runtime data of all existing versions be updated.
Eventually, this causes the required storage size to balloon
exponentially to the number of versions.
This addresses these limitations by:
1. Updating the runtime data structure to move "known versions" metadata
to the package level (the same JSON file is used for all versions).
2. Eliminating known subtype and known usage information at a
cross-package level.
3. Generating the search index by consuming the previously generated
search index.
4. Generating the main page by consuming the search index.
Because this changes how runtime data is stored, an existing docsite
needs to be migrated.
This also introduces a new migration command, `pkl-doc --migrate`,
which transforms an older version of the website into a newer version.
This fixes two issues:
1. Test mode is enabled in pkldoc without the ability to turn it off
2. Native pkldoc is missing required resources
This also adds tests for both `jpkldoc` and `pkldoc`.
This bumps Clikt from version 3 to version 5, which, among other things, improves
the help text formatting with colors.
Also:
* Add `--version` flag to pkldoc, pkl-codegen-java, pkl-codegen-kotlin
* Add help text to pkldoc, pkl-codegen-java, pkl-codegen-kotlin
Some systems have trouble with handling symlinks, which breaks the current directory links created by Pkldoc. In this PR, we add an alternative mode which creates a full copy of the latest published version contents in the current directory instead.
Co-authored-by: Dan Chao <dan.chao@apple.com>
When we updated spotless's Java and Kotlin formatter, we changed the underlying
formatting rules.
However, due to spotless ratcheting, these formatting changes don't get applied unless a file
gets touched in a commit.
To avoid future PRs introducing lines of change that aren't related to the intention of the PR,
this is a one-time format of all files.
Making these classes caused native-image to statically initialize
them at build time, which included CLI argument default values
(like working dir).
This turns them back into classes.
Co-authored-by: Islon Scherer <islonscherer@gmail.com>
- update Kotlin from 1.7.10 to 2.0.21
- Kotlin 1.6 dependencies in Gradle lock files are expected because kotlinc,
which is also used by some tests, internally uses some 1.6 dependencies
for backwards compatibility reasons.
- update kotlinx-html and kotlinx-serialization
- adapt Kotlin code where necessary
- use Kotlin stdlib Path APIs where possible
- fix IntelliJ Kotlin inspection warnings
- reformat code with `./gradlew spotlessApply`
- ktfmt adds lots of trailing commas
- Add workaround to fix IntelliJ "unresolved reference" errors
This fixes an issue where certain modules tasks fail due to the plugin
attempting to analyze their imports, but the arguments may not actually be
Pkl modules.
For example, the pkldoc task accepts entire packages in its "sourceMoules" property.
This changes the gather imports logic to only analyze file-based modules.
This is also a performance improvement; non file-based modules are unlikely to import
files due to insufficient trust levels.
Also: fix a bug when generating pkldoc on Windows
* Encapsulate message transport by removing `ExternalReaderProcess.getTransport` and adding `getModuleResolver` and `getResourceResolver` methods
* Reuse `Random` instances within `ExternalReaderProcessImpl` and module/resource resolvers
* Externalize all `ExternalReaderProcessException` messages
* Add some missing doc comments to `ModuleKeyFactories` and `ResourceReaders` methods for external readers
* Move org.pkl.core.util.Readers to org.pkl.core.Readers
This adds a new feature, which allows Pkl to read resources and modules from external processes.
Follows the design laid out in SPICE-0009.
Also, this moves most of the messaging API into pkl-core
Write annotations to project metadata, and provide them to pkl-doc\
The following annotations have meaning for pkl-doc:
* `@Unlisted`: hide package from documentation site
* `@Deprecated`: add deprecated information
- Update dependencies by deleting lock files and regenerating them with `gw updateDependencyLocks`.
Deleting lock files avoids strange `some.library:some.older.version=default` entries.
Most updated dependencies are test dependencies.
- Handle breaking changes in library commonmark.
- Fix test to close PackageServer exactly once.
This problem surfaced because JUnit 5.11 changed override rules for lifecycle methods,
resulting in too many instead of too few close() calls.
- Bump msgpack version
- Bump clikt version
- Bump Gradle plugin versions
The current hard cutoff in the docs often results in people not
realising that the doc box can be expanded, and often resulting in
confusion because the most helpful examples are often in the module doc
box.
This change uses some simple CSS tweaks to replace the hard cut-off with
a visualfade, so it's obvious that there's content hidden out of view.
Doing this required removing the CSS transition, as it hard to correctly
transition the height property of CSS element of unknown target height.
But the improved discoverablility of the doc content seems like a
worthwhile tradeoff.
This adds support for Windows.
The in-language path separator is still `/`, to ensure Pkl programs are cross-platform.
Log lines are written using CRLF endings on Windows.
Modules that are combined with `--module-output-separator` uses LF endings to ensure
consistent rendering across platforms.
`jpkl` does not work on Windows as a direct executable.
However, it can work with `java -jar jpkl`.
Additional details:
* Adjust git settings for Windows
* Add native executable for pkl cli
* Add jdk17 windows Gradle check in CI
* Adjust CI test reports to be staged within Gradle rather than by shell script.
* Fix: encode more characters that are not safe Windows paths
* Skip running tests involving symbolic links on Windows (these require administrator privileges to run).
* Introduce custom implementation of `IoUtils.relativize`
* Allow Gradle to initialize ExecutableJar `Property` values
* Add Gradle flag to enable remote JVM debugging
Co-authored-by: Philip K.F. Hölzenspies <holzensp@gmail.com>
This changes the file paths to use characters that are safe for Windows.
Channges the output of the following:
* Package cache directory
* Generated pkl-doc files
* Kotlin generated code
Unsafe characters are encoded as (<hex>).
For example, the colon character `:` is encoded as `(3a)`.
Additionally, this changes the cache directory prefix (package-1 to
package-2).
Follows the design of https://github.com/apple/pkl-evolution/pull/3
Fixes an issue where source links are incorrectly URI encoded; i.e. `https%3A//github.com` instead of `https://github.com`.
This was causing the browser to resolve these as relative to the enclosing page.
- Fix and clean up the pkl-commons-test build script.
- Change tests to read test packages/certs directly from
the file system instead of packaging and reading them
from the class path.
- Update expected checksums of some test packages.
- Fix a conflict between Pkl's and Gradle's
Kotlin libraries in the pkl-gradle project.
- Fix build deprecation warnings.
- Ensure Gradle distribution integrity with `distributionSha256Sum`.
- Manually verify integrity of Gradle wrapper added by this commit.
Moving to java.net.http.HttpClient brings many benefits, including
HTTP/2 support and the ability to make asynchronous requests.
Major additions and changes:
- Introduce a lightweight org.pkl.core.http.HttpClient API.
This keeps some flexibility and allows to enforce behavior
such as setting the User-Agent header.
- Provide an implementation that delegates to java.net.http.HttpClient.
- Use HttpClient for all HTTP(s) requests across the codebase.
This required adding an HttpClient parameter to constructors and
factory methods of multiple classes, some of which are public APIs.
- Manage CA certificates per HTTP client instead of per JVM.
This makes it unnecessary to set JVM-wide system/security properties
and default SSLSocketFactory's.
- Add executor v2 options to the executor SPI
- Add pkl-certs as a new artifact, and remove certs from pkl-commons-cli artifact
Each HTTP client maintains its own connection pool and SSLContext.
For efficiency reasons, It's best to reuse clients whenever feasible.
To avoid memory leaks, clients are not stored in static fields.
HTTP clients are expensive to create. For this reason,
EvaluatorBuilder defaults to a "lazy" client that creates the underlying
java.net.http.HttpClient on the first send (which may never happen).
Daniel Chao