Replace pkl-core's local nullness annotations with JSpecify annotations.
Enable NullAway checking for pkl-core packages except org.pkl.core.ast
and org.pkl.core.stdlib.
Notable code changes:
- Add a dedicated late-init constructor to VmTyped
- Move VmExceptionBuilder's fallback message derivation from withCause()
to build()
- Split VmException rendering between builder-provided messages and
string-backed messages
- Initialize MessageTransport handlers with default throwing handlers
- Update JSON helper collection types to allow nullable values JSON
arrays and objects can contain JSON null,
so the Java Map/List element types need to model nullable elements
explicitly
- Make public command transform APIs accept nullable transformed values
Command transforms can produce null for optional/default handling,
so the BiFunction and options-map element types now model that
explicitly
- Make ExecutorSpiException accept nullable message and cause
Existing call sites can pass nullable causes from Throwable.getCause()
- Remove JSR-305 semantics from `@LateInit`
JSpecify does not support the same type-qualifier-nickname pattern,
so `@LateInit` is now documentation plus a NullAway
constructor-initialization exemption
Out of scope:
- NullAway checking of org.pkl.core.ast and org.pkl.core.stdlib
- IntelliJ warnings related to `@LateInit` fields
- Removing the JSR-305 dependency, since concurrency annotations are
still in use
Motivation:
Config.as() causes nullness warnings when its result is intentionally assigned
to a non-null variable
Changes:
* Introduce Config.asNullable(Class<T>), asNullable(JavaType<T>), and
asNullable(Type) to explicitly opt into nullable values
* Keep the signatures of Config.as(Class<T>) and Config.as(JavaType<T>)
unchanged from 0.31 by adding @NullUnmarked
* This gives users time to migrate from as() to asNullable() where appropriate
* Avoids introducing new spurious warnings
* Change `<T> T Config.as(Type)` to `<T extends @nullable Object> T Config.as(Type)`
* This overload is typically used by reflective code such as
pkl-config-kotlin's Config.to() rather than directly by user code
* Clarify that JavaType<T> represents a non-null top-level type whose type arguments may be nullable
* Restricting <T> to non-null keeps method signatures understandable for humans and tools
* Enables full symmetry between Class<T> and JavaType<T> overloads in Config and JavaType
* Enables future non-null runtime checks in both Config.as() overloads
* Simplify construction of `JavaType`s with nullable type arguments
* Add ofNullable() variants for most factory methods, e.g., JavaType.listOfNullable()
* Overhaul Javadoc of Config and JavaType
Result:
* Clear separation between accessing nullable and non-null values
* Config.as() is used for the common non-null case
* Config.as() can perform non-null runtime checks in a future release (breaking change)
* More ergonomic construction of types with nullable type arguments
* More detailed and consistent documentation
When a docsite has only one package name and no DocsiteInfo.overview,
treat it like Javadoc's single-module output: redirect the top-level
index to the package page and omit the site-title breadcrumb segment
from generated pages.
Add src/test/files/SinglePackageTest fixtures to cover multiple package
versions, redirect behavior, breadcrumb behavior, and unchanged site
structure.
Also:
- Shut down Executor used in test.
- Declare expected output fixtures of DocGenerator as test inputs, not
outputs.
- Fix IntelliJ warning by using a Set for the right-hand side of
collection subtraction.
run() now creates and closes a default Executor per call. This is fine
because there is no good reason to call this method multiple times.
run(Executor) now lets callers provide their own Executor, which is
customary for a well-behaved library.
Also: Fix IntelliJ warning by calling toSet()
Closes#1583
Motivation:
- `Config` mixes configuration representation with decoding logic
- `Config.fromPklBinary()` does not scale as decoding gains options
(e.g., binary versions or formats)
- The decoding API is inconsistent with `ConfigEvaluator`
Changes:
- Introduce `ConfigDecoder` (with builder) and move
`Config.fromPklBinary()` logic into it
- Deprecate `Config.fromPklBinary()` methods for removal
- Add `ConfigDecoder.forKotlin()` extension function
- Update and improve tests
Result:
- Decoding is separated from `Config` and exposed via a dedicated API
- Decoding can evolve independently (e.g., adding options such as binary
versions or supporting new formats)
- Evaluation and decoding APIs follow a consistent design
Motivation:
buildSrc is a special-case legacy mechanism.
Gradle recommends using an included build named build-logic instead:
https://docs.gradle.org/current/userguide/best_practices_structuring_builds.html#favor_composite_builds
Changes:
- Rename buildSrc/ to build-logic/
- triggers reformatting
- Replace occurrences of "buildSrc" with "build-logic"
- Include the build-logic build in the main build (via
settings.gradle.kts)
- Apply convention plugins via plugin IDs instead of type-safe accessors
- small tradeoff compared to buildSrc
Result:
- Faster and more isolated builds
- Build logic behaves like a normal build, making it easier to evolve
and reason about
---------
Co-authored-by: Daniel Chao <dan.chao@apple.com>
- Enforce Kotlin version via resolution rule (replaces BOM)
- fail if kotlin-stdlib/kotlin-reflect exceed target version
- Replace kotlin-stdlib-jdk8 with kotlin-stdlib (jdk7/8 are now shims)
- Port pkl-core annotation processor to Java (with Codex)
- removes kotlin-stdlib from its compile classpath for better dependency
hygiene (Java module)
- Downgrade clikt for Kotlin 2.2 compatibility
- Upgrade kotlinx-serialization
---------
Co-authored-by: Daniel Chao <dan.chao@apple.com>
Motivation
- Enable correct NullAway analysis
- Pick up toolchain fixes and improvements
Toolchains
- Require JDK 25 for JVM toolchain (keep Java 17 runtime compatibility)
- Require Kotlin 2.3.20 for Kotlin toolchain (keep Kotlin 2.2 runtime
compatibility)
- Require JDK 25 for Gradle daemon JVM (via
gradle-daemon-jvm.properties)
- Fix javac and kotlinc warnings from toolchain upgrades
CI
- Bump GitHub workflows to JDK 25
Building Kotlin
- Bump Kotlin language level to 2.2 to match stdlib version
- Consolidate build logic into pklKotlinBase.gradle.kts
- Adopt modern Kotlin plugin syntax
- Fix new kotlinc warnings
- Update ktfmt to 0.62
- first version compatible with Kotlin 2.3.20
- changes formatting compared to 0.61
- Replace dependency resolution rule with BOM alignment
- rule was too broad and interfered with toolchain/runtime separation
Testing
- Expand matrix to JDK 25 (LTS) and 26
- Ensure each matrix task can be run independently
- Fix KotlinCodeGeneratorsTest and EmbeddedExecutorsTest on affected
JDKs
- Disable one test in CliCommandTest on affected JDKs (failure cause
unknown)
Compatibility fixes
- Fix reflective access in DocGenerator on affected JDKs
Build fixes
- Fix misuse of `task.enabled` vs. `report.required`
- Fix `gradlew tasks` on Windows
- Downgrade Spotless to 8.3.0 to (hopefully) work around sporadic
NoClassDefFoundError
Result
- NullAway runs correctly
- Broader JDK test coverage
- More reproducible and potentially faster builds
Motivation:
Facilitate the use of the NullAway checker as part of moving to
JSpecify.
Changes:
- represent "no children" as `List.of()` instead of null
- remove obsolete `children != null` assertions
- NullAway intentionally ignores such assertions
- remove "no children" special-casing where no longer necessary
Result:
- cleaner code with similar performance
- removed a barrier to using the NullAway checker
- Remove dependency org.fusesource.jansi:jansi
- In 4.x, org.fusesource.jansi:jansi was replaced with org.jline:jansi.
Instead of adding this new dependency, this commit replaces Pkl’s single
Jansi usage with custom code that preserves existing behavior. Fixing
existing ANSI quirks is left for a future PR.
- Replace jline-terminal-ansi with jline-terminal-jni
- In 4.x, only -jni and -ffm are available (-ffm requires Java 22+)
- Configure native-image build for jline-terminal-jni
As updating JLine is delicate, I manually tested `pkl repl` and `jpkl
repl` on Windows 11 (using Windows Terminal) and on Ubuntu, and found no
issues. However, I do not have access to a macOS machine.
- Make leaf AST classes final
- Make protected Lexer fields private and add getter
- Split Parser into Parser and ParserImpl
- Using a fresh ParserImpl instance per parse simplifies reasoning
(important) and makes the Parser API thread-safe (nice to have)
- Split GenericParser into GenericParser and GenericParserImpl
- Same motivation as for Parser
Some of these changes will facilitate the move to JSpecify, which has
proven challenging for this package.
- pass `GrammarVersion` to constructor instead of passing it to each
`format` method
- replace `format(Path): String` with `format(Reader, Appendable)`
- instead of picking which overloads besides `format(String): String`
might be useful, offer a single generalized method that streams input
and output
- add `@Throws(IOException::class)` to ensure that Java callers can
catch this exception
- deprecate old methods
- move the following classes into package externalreader:
- ExternalModuleResolver
- ExternalResourceResolver
- MessageTransportModuleResolver (renamed to ExternalModuleResolverImpl, made package-private)
- MessageTransportResourceResolver (renamed to ExternalResourceResolverImpl, made package-private)
- replace interface ExternalModuleResolver.Spec with record ExternalModuleReaderSpec
- replace interface ExternalResourceResolver.Spec with record ExternalResourceReaderSpec
- translate between messaging.ResourceReaderSpec and ExternalResourceReaderSpec (eliminates dependency from messaging on higher layer)
- translate between messaging.ResourceResolverSpec and ExternalResourceResolverSpec (eliminates dependency from messaging on higher layer)
- add ServerMessages.ExternalReader and translate between this message component and the PklEvaluatorSettings.ExternalReader API
- add ServerMessages.Proxy and translate between this message component and the PklEvaluatorSettings.Proxy API
- change type of CreateEvaluatorRequest.allowedModules/allowedResources from List<Pattern>? to List<String>?
- removes a lot of code
- should not need to create a Pattern object to send a message
- deprecate method evaluatorSettings.PklEvaluatorSettings.Proxy.create()
- only seems useful internally, inlined
Co-authored-by: Dan Chao <dan.chao@apple.com>
Motivation:
* fix known bugs and limitations of for-generators
* improve code health by removing complex workarounds
Changes:
* simplify AstBuilder code related to for-generators
* track for-generators via `SymbolTable.enterForGenerator()`
* add `RestoreForBindingsNode` during initial AST construction
instead of calling `MemberNode.replaceBody()` later on
* simplify some unnecessarily complex code
* remove workarounds and band-aids such as:
* `isInIterable`
* `executeAndSetEagerly`
* adding dummy slots in `AmendFunctionNode`
* overhaul implementation of for-generators
* store keys and values of for-generator iterations in regular instead of auxiliary frame slots
* set them via `TypeNode.executeAndSet()`
* `ResolveVariableNode` no longer needs to search auxiliary slots
* `Read(Enclosing)AuxiliarySlot` is no longer needed
* at the start of each for-generator iteration, create a new `VirtualFrame`
that is a copy of the current frame (arguments + slots)
and stores the iteration key and value in additional slots.
* execute for-generator iteration with the newly created frame
* `childNode.execute(newFrame)`
* Pkl objects created during the iteration will materialize this frame
* store newly created frames in `owner.extraStorage`
if their for-generator slots may be accessed when a generated member is executed
* resolving variable names to for-generator variables at parse time would make this analysis more precise
* when a generated member is executed,
* retrieve the corresponding frame stored in `owner.extraStorage`
* copy the retrieved frame's for-generator slots into slots of the current frame
Result:
* for-generators are implemented in a correct, reasonably simple, and reasonably efficient way
* complexity is fully contained within package `generator` and `AstBuilder`
* for-generator keys and values can be accessed from all nested scopes:
* key and value expressions of generated members
* condition expressions of nested when-generators
* iterable expressions of nested for-generators
* for-generator keys and values can be accessed from within objects created by the expressions listed above
* sibling for-generators can use the same key/value variable names
* parent/child for-generators can use the same key/value variable names
* fixes https://github.com/apple/pkl/issues/741
- update Kotlin from 1.7.10 to 2.0.21
- Kotlin 1.6 dependencies in Gradle lock files are expected because kotlinc,
which is also used by some tests, internally uses some 1.6 dependencies
for backwards compatibility reasons.
- update kotlinx-html and kotlinx-serialization
- adapt Kotlin code where necessary
- use Kotlin stdlib Path APIs where possible
- fix IntelliJ Kotlin inspection warnings
- reformat code with `./gradlew spotlessApply`
- ktfmt adds lots of trailing commas
- Add workaround to fix IntelliJ "unresolved reference" errors
Motivation:
- Perform same exception handling for every implementation of PklRootNode.execute().
- Avoid code duplication.
Changes:
- Change PklRootNode.execute() to be a final method that performs exception handling
and calls abstract method executeImpl(), which is implemented by subclasses.
- Remove executeBody() methods, which served a similar purpose but were more limited.
- Remove duplicate exception handling code.
Result:
- More reliable exception handling.
This should fix known problems such as misclassifying stack overflows
as internal errors and displaying errors without a stack trace.
- Less code duplication.
Motivation:
Spring Boot configuration classes neither require nor benefit from annotating constructor parameters with their name.
The same is true for pkl-config-java configuration classes compiled with `-parameter`.
Changes:
- Change CLI parameter `--params-annotation` to accept a `none` value.
This is recommended in https://clig.dev/#arguments-and-flags and is how the `-F` parameter of the `ssh` command works.
- Change `paramsAnnotation` property in Gradle plugin, CliJavaCodeGeneratorOptions, and JavaCodegenOptions as follows:
- Change meaning of `null` from "generate org.pkl.java.config.mapper.Named annotations" to "do not generate annotations".
This is a breaking change (only) affecting users who explicitly set the property to `null` instead of omitting it.
- Change property default from `null` to:
`null` if `generateSpringBootConfig` is `true` and `org.pkl.java.config.mapper.Named` otherwise
- add tests
- update docs of this and other codegen options
Result:
Generated code does not contain unnecessary annotations.
Motivation:
- simplify implementation of lazy type checking
- fix correctness issues of lazy type checking (#785)
Changes:
- implement listing/mapping type cast via amendment (`parent`) instead of delegation (`delegate`)
- handle type checking of *computed* elements/entries in the same way as type checking of computed properties
- ElementOrEntryNode is the equivalent of TypeCheckedPropertyNode
- remove fields VmListingOrMapping.delegate/typeNodeFrame/cachedMembers/checkedMembers
- fix#785 by executing all type casts between a member's owner and receiver
- fix#823 by storing owner and receiver directly
instead of storing the mutable frame containing them (typeNodeFrame)
- remove overrides of VmObject methods that are no longer required
- good for Truffle partial evaluation and JVM inlining
- revert a85a173faa except for added tests
- move `VmUtils.setOwner` and `VmUtils.setReceiver` and make them private
- these methods aren't generally safe to use
Result:
- simpler code with greater optimization potential
- VmListingOrMapping can now have both a type node and new members
- fewer changes to surrounding code
- smaller memory footprint
- better performance in some cases
- fixes https://github.com/apple/pkl/issues/785
- fixes https://github.com/apple/pkl/issues/823
Potential future optimizations:
- avoid lazy type checking overhead for untyped listings/mappings
- improve efficiency of forcing a typed listing/mapping
- currently, lazy type checking will traverse the parent chain once per member,
reducing the performance benefit of shallow-forcing
a listing/mapping over evaluating each member individually
- avoid creating an intermediate untyped listing/mapping in the following cases:
- `new Listing<X> {...}`
- amendment of `property: Listing<X>`
odenix