This introduces a new `Bytes` standard library class, for working with
binary data.
* Add Bytes class to the standard library
* Change CLI to eval `output.bytes`
* Change code generators to map Bytes to respective underlying type
* Add subscript and concat operator support
* Add binary encoding for Bytes
* Add PCF and Plist rendering for Bytes
Co-authored-by: Kushal Pisavadia <kushi.p@gmail.com>
Some systems require junit report to be in a single file. For example `bazel` https://bazel.build/reference/test-encyclopedia needs single file to be available in `XML_OUTPUT_FILE` path.
To avoid implementing junit aggregation in pkl wrappers in different places this PR instead adds a `--junit-aggregate-reports` flag to return all junit reports as a single file.
Additional flag `--junit-aggregate-suite-name` is added to allow overriding global test suite name from default `pkl-tests`
The preconfigured evaluator currently adds module key factories from
service providers, but not resource readers.
This means that users of pkl-spring, for example, cannot add custom
resource readers.
This is also inconsistent (in the preconfigured evaluator, `import` can
use custom schemes, but not `read`).
* Update dependencies
1. Remove */gradle.lockfile files
2. Run `gradle updateDependencyLocks` and commit
* Update multi-JDK testing to use simple Test task, add junit-platform-launcher to dependencies
- Don't use JvmTestSuite (we don't use another test runner, we use the same classpath)
* Add junit-platform-launcher to libs (prevent an issue where junit-engine and junit-launcher can fall out of sync)
Opaque file URIs are URIs whose scheme-specific part does not start with `/`.
For example, `file:foo/bar.txt` is an opaque URI.
Currently, this has the unintentional behavior of: look for file `foo/bar.txt` from the process working directory.
These are effectively dynamics imports; from a single import, we can't statically analyze what it resolves as.
According to RFC-8089, File URIs must have paths that start with `/`. So, these are actually _not valid URIs_.
See the grammar defined in https://datatracker.ietf.org/doc/html/rfc8089#section-2
This changes Pkl's behavior so that these URIs are treated as errors.
This adds logic to build and publish the other executables related to Pkl.
These are:
* pkl-doc
* pkl-codegen-kotlin
* pkl-codegen-java
pkl-codegen-kotlin and pkl-codegen-java are published as executable JARs, whereas pkldoc is published both as an executable JAR, and also native executables (matching the set of os/arch supported by Pkl).
The reason this only publishes executable JARs for pkl-codegen-kotlin and pkl-codegen-java is because we expect that the Java requirement is not a problem for these users, and that the native executable provides negligible added value.
As part of this, the following changes are made:
* Introduce `pklJavaExecutable` plugin, which sets up building and publishing of executable JAR.
* Introduce `pklNativeExecutable` plugin, which sets up building and publishing of native executables.
* Introduce `NativeImageBuild` Gradle task, which knows how to build native-image executables.
* Introduce `ExecutableSpec` extension, for projects that publish executables to configure how those executables should be published.
* `./griddles buildNative`, by default, will only build the executable of the host OS/Arch, and will no longer cross-build.
* The target arch of `./gradlew buildNative` can be changed using `-Dpkl.targetArch=<aarch64|amd64>`.
* On linux/amd64 only, with `./gradlew buildNative`, a statically linked executable can be built using `-Dpkl.musl=true`
* Make `javaExecutable` a dependency of `assemble`
* Make `testStartJavaExecutable` a dependency of `check`
* Change name `pklNativeBuild` to `pklNativeLifecycle` to better match the plugin's purpose
* Remove Truffle SVM classes from main source set (don't publish these classes as part of the pkl-cli JAR)
* Change CircleCI definition to publish new executables
* Change CircleCI definition to call `buildNative` instead of individual task names
If a generator object literal doesn't add any object members, we
can simply use the parent in its place.
Co-authored-by: Kushal Pisavadia <kushi.p@gmail.com>
- move the following classes into package externalreader:
- ExternalModuleResolver
- ExternalResourceResolver
- MessageTransportModuleResolver (renamed to ExternalModuleResolverImpl, made package-private)
- MessageTransportResourceResolver (renamed to ExternalResourceResolverImpl, made package-private)
- replace interface ExternalModuleResolver.Spec with record ExternalModuleReaderSpec
- replace interface ExternalResourceResolver.Spec with record ExternalResourceReaderSpec
- translate between messaging.ResourceReaderSpec and ExternalResourceReaderSpec (eliminates dependency from messaging on higher layer)
- translate between messaging.ResourceResolverSpec and ExternalResourceResolverSpec (eliminates dependency from messaging on higher layer)
- add ServerMessages.ExternalReader and translate between this message component and the PklEvaluatorSettings.ExternalReader API
- add ServerMessages.Proxy and translate between this message component and the PklEvaluatorSettings.Proxy API
- change type of CreateEvaluatorRequest.allowedModules/allowedResources from List<Pattern>? to List<String>?
- removes a lot of code
- should not need to create a Pattern object to send a message
- deprecate method evaluatorSettings.PklEvaluatorSettings.Proxy.create()
- only seems useful internally, inlined
Co-authored-by: Dan Chao <dan.chao@apple.com>
This bumps Clikt from version 3 to version 5, which, among other things, improves
the help text formatting with colors.
Also:
* Add `--version` flag to pkldoc, pkl-codegen-java, pkl-codegen-kotlin
* Add help text to pkldoc, pkl-codegen-java, pkl-codegen-kotlin
This changes the language to check all types eagerly when within a type constraint.
This addresses two regressions in the language:
1. Type constraints are too relaxed (listing/mapping parameters may not be checked)
2. Failing type constraints hide members that were forced during execution of the constraint
When we updated spotless's Java and Kotlin formatter, we changed the underlying
formatting rules.
However, due to spotless ratcheting, these formatting changes don't get applied unless a file
gets touched in a commit.
To avoid future PRs introducing lines of change that aren't related to the intention of the PR,
this is a one-time format of all files.
This updates the GraalVM and Truffle libraries to 2024.1.2.
This also updates the build logic to compile Java sources using Java 21, due to some compile-only dependencies within GraalVM/Truffle using class file version 65. However, the produced artifact is still compatible with Java 17.
This also changes the Gradle build logic to use toolchains, and to test the Java libraries with JDK 17 and 21.
One consequence of this change is that Truffle is no longer shaded within the fat jars.
feat: support for jvm21+ toolchain
feat: support for gradle toolchains
feat: pass -PnativeArch=native to build with -march=native
test: multi-jdk testing support
test: support for jvm-test-suite plugin
test: add tasks to run jpkl eval on multiple jdks
test: make jdk exec tests respect multi-jdk flags and ranges
fix: remove mrjar classes at >jvm17 from fatjars
fix: use jdk21 to run the tests (needed for Unsafe.ensureInitialized)
fix: truffle svm dependency is required after graalvm 24.0.0
fix: warnings for gvm flag usage, renamed truffle svm macro
fix: build with --add-modules=jdk.unsupported where needed
fix: don't use gu tool for modern graalvm versions
fix: catch Throwable instead of deprecated-for-removal ThreadDeath
chore: buildinfo changes for JVM targets, toolchains
chore: enforce testing at exactly jdk21
chore: enforce build tooling at jdk21+
chore: bump graalvm/truffle libs → 24.1.2
chore: toolchains for buildSrc
Signed-off-by: Sam Gammon <sam@elide.dev>
If the object member is const, it only needs to be executed once, and all children in the prototype chain can use its cached value.
There is a possible other optimization here, not included in this PR: we can avoid adding these values to the child object's cachedMembers.
Motivation:
* fix known bugs and limitations of for-generators
* improve code health by removing complex workarounds
Changes:
* simplify AstBuilder code related to for-generators
* track for-generators via `SymbolTable.enterForGenerator()`
* add `RestoreForBindingsNode` during initial AST construction
instead of calling `MemberNode.replaceBody()` later on
* simplify some unnecessarily complex code
* remove workarounds and band-aids such as:
* `isInIterable`
* `executeAndSetEagerly`
* adding dummy slots in `AmendFunctionNode`
* overhaul implementation of for-generators
* store keys and values of for-generator iterations in regular instead of auxiliary frame slots
* set them via `TypeNode.executeAndSet()`
* `ResolveVariableNode` no longer needs to search auxiliary slots
* `Read(Enclosing)AuxiliarySlot` is no longer needed
* at the start of each for-generator iteration, create a new `VirtualFrame`
that is a copy of the current frame (arguments + slots)
and stores the iteration key and value in additional slots.
* execute for-generator iteration with the newly created frame
* `childNode.execute(newFrame)`
* Pkl objects created during the iteration will materialize this frame
* store newly created frames in `owner.extraStorage`
if their for-generator slots may be accessed when a generated member is executed
* resolving variable names to for-generator variables at parse time would make this analysis more precise
* when a generated member is executed,
* retrieve the corresponding frame stored in `owner.extraStorage`
* copy the retrieved frame's for-generator slots into slots of the current frame
Result:
* for-generators are implemented in a correct, reasonably simple, and reasonably efficient way
* complexity is fully contained within package `generator` and `AstBuilder`
* for-generator keys and values can be accessed from all nested scopes:
* key and value expressions of generated members
* condition expressions of nested when-generators
* iterable expressions of nested for-generators
* for-generator keys and values can be accessed from within objects created by the expressions listed above
* sibling for-generators can use the same key/value variable names
* parent/child for-generators can use the same key/value variable names
* fixes https://github.com/apple/pkl/issues/741
- update Kotlin from 1.7.10 to 2.0.21
- Kotlin 1.6 dependencies in Gradle lock files are expected because kotlinc,
which is also used by some tests, internally uses some 1.6 dependencies
for backwards compatibility reasons.
- update kotlinx-html and kotlinx-serialization
- adapt Kotlin code where necessary
- use Kotlin stdlib Path APIs where possible
- fix IntelliJ Kotlin inspection warnings
- reformat code with `./gradlew spotlessApply`
- ktfmt adds lots of trailing commas
- Add workaround to fix IntelliJ "unresolved reference" errors
Nested jars built by spring boot can possibly represent classpath
resource URIs as "jar:nested:".
This changes Pkl to by default trust them with the same level as
modulepath URIs.
Motivation:
- Perform same exception handling for every implementation of PklRootNode.execute().
- Avoid code duplication.
Changes:
- Change PklRootNode.execute() to be a final method that performs exception handling
and calls abstract method executeImpl(), which is implemented by subclasses.
- Remove executeBody() methods, which served a similar purpose but were more limited.
- Remove duplicate exception handling code.
Result:
- More reliable exception handling.
This should fix known problems such as misclassifying stack overflows
as internal errors and displaying errors without a stack trace.
- Less code duplication.
Motivation:
- simplify implementation of lazy type checking
- fix correctness issues of lazy type checking (#785)
Changes:
- implement listing/mapping type cast via amendment (`parent`) instead of delegation (`delegate`)
- handle type checking of *computed* elements/entries in the same way as type checking of computed properties
- ElementOrEntryNode is the equivalent of TypeCheckedPropertyNode
- remove fields VmListingOrMapping.delegate/typeNodeFrame/cachedMembers/checkedMembers
- fix#785 by executing all type casts between a member's owner and receiver
- fix#823 by storing owner and receiver directly
instead of storing the mutable frame containing them (typeNodeFrame)
- remove overrides of VmObject methods that are no longer required
- good for Truffle partial evaluation and JVM inlining
- revert a85a173faa except for added tests
- move `VmUtils.setOwner` and `VmUtils.setReceiver` and make them private
- these methods aren't generally safe to use
Result:
- simpler code with greater optimization potential
- VmListingOrMapping can now have both a type node and new members
- fewer changes to surrounding code
- smaller memory footprint
- better performance in some cases
- fixes https://github.com/apple/pkl/issues/785
- fixes https://github.com/apple/pkl/issues/823
Potential future optimizations:
- avoid lazy type checking overhead for untyped listings/mappings
- improve efficiency of forcing a typed listing/mapping
- currently, lazy type checking will traverse the parent chain once per member,
reducing the performance benefit of shallow-forcing
a listing/mapping over evaluating each member individually
- avoid creating an intermediate untyped listing/mapping in the following cases:
- `new Listing<X> {...}`
- amendment of `property: Listing<X>`
