This introduces a new `Bytes` standard library class, for working with
binary data.
* Add Bytes class to the standard library
* Change CLI to eval `output.bytes`
* Change code generators to map Bytes to respective underlying type
* Add subscript and concat operator support
* Add binary encoding for Bytes
* Add PCF and Plist rendering for Bytes
Co-authored-by: Kushal Pisavadia <kushi.p@gmail.com>
Opaque file URIs are URIs whose scheme-specific part does not start with `/`.
For example, `file:foo/bar.txt` is an opaque URI.
Currently, this has the unintentional behavior of: look for file `foo/bar.txt` from the process working directory.
These are effectively dynamics imports; from a single import, we can't statically analyze what it resolves as.
According to RFC-8089, File URIs must have paths that start with `/`. So, these are actually _not valid URIs_.
See the grammar defined in https://datatracker.ietf.org/doc/html/rfc8089#section-2
This changes Pkl's behavior so that these URIs are treated as errors.
If a generator object literal doesn't add any object members, we
can simply use the parent in its place.
Co-authored-by: Kushal Pisavadia <kushi.p@gmail.com>
This changes the language to check all types eagerly when within a type constraint.
This addresses two regressions in the language:
1. Type constraints are too relaxed (listing/mapping parameters may not be checked)
2. Failing type constraints hide members that were forced during execution of the constraint
If the object member is const, it only needs to be executed once, and all children in the prototype chain can use its cached value.
There is a possible other optimization here, not included in this PR: we can avoid adding these values to the child object's cachedMembers.
Motivation:
* fix known bugs and limitations of for-generators
* improve code health by removing complex workarounds
Changes:
* simplify AstBuilder code related to for-generators
* track for-generators via `SymbolTable.enterForGenerator()`
* add `RestoreForBindingsNode` during initial AST construction
instead of calling `MemberNode.replaceBody()` later on
* simplify some unnecessarily complex code
* remove workarounds and band-aids such as:
* `isInIterable`
* `executeAndSetEagerly`
* adding dummy slots in `AmendFunctionNode`
* overhaul implementation of for-generators
* store keys and values of for-generator iterations in regular instead of auxiliary frame slots
* set them via `TypeNode.executeAndSet()`
* `ResolveVariableNode` no longer needs to search auxiliary slots
* `Read(Enclosing)AuxiliarySlot` is no longer needed
* at the start of each for-generator iteration, create a new `VirtualFrame`
that is a copy of the current frame (arguments + slots)
and stores the iteration key and value in additional slots.
* execute for-generator iteration with the newly created frame
* `childNode.execute(newFrame)`
* Pkl objects created during the iteration will materialize this frame
* store newly created frames in `owner.extraStorage`
if their for-generator slots may be accessed when a generated member is executed
* resolving variable names to for-generator variables at parse time would make this analysis more precise
* when a generated member is executed,
* retrieve the corresponding frame stored in `owner.extraStorage`
* copy the retrieved frame's for-generator slots into slots of the current frame
Result:
* for-generators are implemented in a correct, reasonably simple, and reasonably efficient way
* complexity is fully contained within package `generator` and `AstBuilder`
* for-generator keys and values can be accessed from all nested scopes:
* key and value expressions of generated members
* condition expressions of nested when-generators
* iterable expressions of nested for-generators
* for-generator keys and values can be accessed from within objects created by the expressions listed above
* sibling for-generators can use the same key/value variable names
* parent/child for-generators can use the same key/value variable names
* fixes https://github.com/apple/pkl/issues/741
Motivation:
- simplify implementation of lazy type checking
- fix correctness issues of lazy type checking (#785)
Changes:
- implement listing/mapping type cast via amendment (`parent`) instead of delegation (`delegate`)
- handle type checking of *computed* elements/entries in the same way as type checking of computed properties
- ElementOrEntryNode is the equivalent of TypeCheckedPropertyNode
- remove fields VmListingOrMapping.delegate/typeNodeFrame/cachedMembers/checkedMembers
- fix#785 by executing all type casts between a member's owner and receiver
- fix#823 by storing owner and receiver directly
instead of storing the mutable frame containing them (typeNodeFrame)
- remove overrides of VmObject methods that are no longer required
- good for Truffle partial evaluation and JVM inlining
- revert a85a173faa except for added tests
- move `VmUtils.setOwner` and `VmUtils.setReceiver` and make them private
- these methods aren't generally safe to use
Result:
- simpler code with greater optimization potential
- VmListingOrMapping can now have both a type node and new members
- fewer changes to surrounding code
- smaller memory footprint
- better performance in some cases
- fixes https://github.com/apple/pkl/issues/785
- fixes https://github.com/apple/pkl/issues/823
Potential future optimizations:
- avoid lazy type checking overhead for untyped listings/mappings
- improve efficiency of forcing a typed listing/mapping
- currently, lazy type checking will traverse the parent chain once per member,
reducing the performance benefit of shallow-forcing
a listing/mapping over evaluating each member individually
- avoid creating an intermediate untyped listing/mapping in the following cases:
- `new Listing<X> {...}`
- amendment of `property: Listing<X>`
Motivation:
The following expression evaluates to 2 instead of 1:
new Listing { "value" } { [0 + 0] = "override" }.length
Changes:
- fix length computation in EntriesLiteralNode
- improve `api/listing` tests
- make snippet test failures diffable in IntelliJ
Result:
- fixes https://github.com/apple/pkl/issues/780
- improved dev experience in IntelliJ
This fixes issues when a mapping/listing delegates to another mapping/listing.
The core issue is that `cachedValues` is not guaranteed to be the complete set
of the object's members; they may be stored on the delegate instead.
Therefore, it's not correct compute hash code and equality based on `cachedValues`.
Instead, it's better to use the `getCachedValue()` method, which has the correct
logic for looking up the existing cached value.
There is currently a bug around resolving variables within the iterable of a for
generator or spread syntax (https://github.com/apple/pkl/issues/741)
Normally, mappings/listings are type-checked lazily. However, this results in the said
bug getting widened, for any object members declared in the iterable.
As a workaround for now, prevent the bug from being any worse by ensuring that these
object members are eagerly typechecked.
* Add `values` to `Mapping`
* Add `entries` to `Mapping`
* Add `containsValue` to `Mapping`
* Add `every` to `Mapping`
* Add `any` to `Mapping`
* Add `toDynamic` to `Mapping`
* Add `lastIndex` to `Listing`
* Add `getOrNull` to `Listing`
* Add `first` to `Listing`
* Add `firstOrNull` to `Listing`
* Add `last` to `Listing`
* Add `lastOrNull` to `Listing`
* Add `single` to `Listing`
* Add `singleOrNull` to `Listing`
* Add `contains` to `Listing`
* Add `any` to `Listing`
* Add `every` to `Listing`
* Fixup `any` to `Listing`
* Revert "Add `toDynamic` to `Mapping`"
This reverts commit 5551974ecd8110aa2eb8f546e992c32d3181df9b.
* Revert "Add `values` to `Mapping`"
This reverts commit 6fc78796
* Revert "Add `entries` to `Mapping`"
This reverts commit a7e8dfc4
* Annotate new members with `Since` 0.27
* Fix documentation in `base.pkl`
* Add location information to empty/single checks in `Listing` operations
* Remove additional variable for laziness preservation
* Apply spotless
* Apply suggestions from code review
Co-authored-by: Daniel Chao <daniel.h.chao@gmail.com>
---------
Co-authored-by: Daniel Chao <daniel.h.chao@gmail.com>
Fix a bug where overwriting `output` causes a PklBugException.
This now produces a more helpful message pointing to the actual problem.
Co-authored-by: translatenix <119817707+translatenix@users.noreply.github.com>
The addresses the following case:
local l1: Listing<String> = new { 1 }
l2: Listing<Int> = (l1) { 2 }
The member in `l1` should be checked against its owner; to check against `Listing<String>`.
This adds a new feature to build a dependency graph of Pkl programs, following the SPICE outlined in https://github.com/apple/pkl-evolution/pull/2.
It adds:
* CLI command `pkl analyze imports`
* Java API `org.pkl.core.Analyzer`
* Pkl stdlib module `pkl:analyze`
* pkl-gradle extension `analyze`
In addition, it also changes the Gradle plugin such that `transitiveModules` is by default computed from the import graph.
This adds support for adding the `const` modifier to object members.
Such object members are also required to have the `local` modifier applied.
This follows SPICE-0011.
This changes how the language performs typechecks for mappings and
listings.
Currently, Pkl will shallow-force any Mapping and Listing to check it
the type parameter (e.g. Listing<Person> means each element is checked
to be an instance of Person).
This changes the language to check each member's type when the member
is accessed.
This also adjust test runner to handle thrown errors from within tests.
With the change to make mapping/listing typechecks lazy, we can now
correctly handle thrown errors from within a single test case.
This adjusts the test runner to consider any thrown errors as a failure
for that specific test case.
* Add `--proxy` and `--no-proxy` CLI flags
* Add property `http` to `pkl:settings`
* Move `EvaluatorSettings` from `pkl:Project` to its own module and add property `http`
* Add support for proxying in server mode, and through Gradle
* Add `setProxy()` to `HttpClient`
* Add documentation
This adds a language change that requires references from typealiases
to the enclosing module to be `const`.
This is required because typealiases are not late-bound.
Rationale is laid out in SPICE-0007.
Also:
* Update documentation to reflect new rules.
* Fix `Project.pkl`; mark method `const` to not break said rule.
This adds changes to support loading project dependencies in non-file based projects.
The design for this feature can be found in SPICE-0005: https://github.com/apple/pkl-evolution/pull/6
Changes:
* Consider all imports prefixed with `@` as dependency notation.
* Bugfix: fix resolution of glob expressions in a local dependency.
* Adjust pkl.Project:
- Allow local dependencies from a scheme-local paths.
- Disallow certain evaluator settings if not loaded as a file-based module.
* Breaking API change: `ProjectDependenciesManager` constructor now requires `ModuleResolver` and `SecurityManager`.
* Split MemberNode into (Regular/Shared)MemberNode
SharedMemberNode enables generating non-constant object members
at run time without generating an unbounded number of Truffle root nodes.
* Invert shouldRunTypeCheck to match its name
* Introduce VmObjectBuilder
Introduce VmObjectBuilder, a uniform way to build `VmObject`s
whose `ObjectMember`s are determined at run time.
Replace some manual object building code with VmObjectBuilder.
Add some assertions to fix IntelliJ warnings.
* Improve implementation of globbed read/import nodes
- Leverage SharedMemberNode to have a single Truffle root node per globbed
read/import expression instead of one root node per resolved glob element.
- Remove caching in ReadGlobNode because it only works correctly if glob pattern is a string *constant*.
- Remove caching in StaticReadNode (now ReadGlobElementNode/ImportGlobElementNode)
because it seems unnecessary and the implementation doesn't look quite right.
* Simplify code
* Fix ClassCastException when reflecting on globbed import
* Fix caching of globbed reads
Problem:
The result of a globbed read depends not only on the glob pattern but also on the current module URI.
However, ResourceManager does not take this into account when caching globbed reads, causing wrong results.
Changes:
- Cache globbed reads per read expression.
This is also how globbed imports are cached, except that import URIs are constant.
- Make ReadGlobNode and ImportGlobNode code as similar as possible.
- Reduce code duplication by inheriting from AbstractReadNode.
- Add some tests.
This adds support for Windows.
The in-language path separator is still `/`, to ensure Pkl programs are cross-platform.
Log lines are written using CRLF endings on Windows.
Modules that are combined with `--module-output-separator` uses LF endings to ensure
consistent rendering across platforms.
`jpkl` does not work on Windows as a direct executable.
However, it can work with `java -jar jpkl`.
Additional details:
* Adjust git settings for Windows
* Add native executable for pkl cli
* Add jdk17 windows Gradle check in CI
* Adjust CI test reports to be staged within Gradle rather than by shell script.
* Fix: encode more characters that are not safe Windows paths
* Skip running tests involving symbolic links on Windows (these require administrator privileges to run).
* Introduce custom implementation of `IoUtils.relativize`
* Allow Gradle to initialize ExecutableJar `Property` values
* Add Gradle flag to enable remote JVM debugging
Co-authored-by: Philip K.F. Hölzenspies <holzensp@gmail.com>
Islon Scherer