Commit Graph

118 Commits

Author SHA1 Message Date
translatenix
1be1fe4863 Use VmObjectBuilder in Listing.distinct(By) implementations (#760) 2024-10-31 13:28:04 -07:00
translatenix
cc72f9d160 Polish external reader API/implementation (#759)
- keep implementation classes internal to their packages
- make classes final if possible
- make namespace classes non-instantiable
- throw IllegalStateException instead of ExternalReaderProcessException for use after close
  - common convention already used by HttpClient etc.
  - programming errors should be signaled with unchecked exceptions
- use private instead of public lock object
- polish Javadoc
- delete commented out code
- don't use star import for a single class
2024-10-31 13:12:19 -07:00
Philip K.F. Hölzenspies
a03827951c Add methods from List/Map to Listing/Mapping (#683)
* Add `values` to `Mapping`

* Add `entries` to `Mapping`

* Add `containsValue` to `Mapping`

* Add `every` to `Mapping`

* Add `any` to `Mapping`

* Add `toDynamic` to `Mapping`

* Add `lastIndex` to `Listing`

* Add `getOrNull` to `Listing`

* Add `first` to `Listing`

* Add `firstOrNull` to `Listing`

* Add `last` to `Listing`

* Add `lastOrNull` to `Listing`

* Add `single` to `Listing`

* Add `singleOrNull` to `Listing`

* Add `contains` to `Listing`

* Add `any` to `Listing`

* Add `every` to `Listing`

* Fixup `any` to `Listing`

* Revert "Add `toDynamic` to `Mapping`"

This reverts commit 5551974ecd8110aa2eb8f546e992c32d3181df9b.

* Revert "Add `values` to `Mapping`"

This reverts commit 6fc78796

* Revert "Add `entries` to `Mapping`"

This reverts commit a7e8dfc4

* Annotate new members with `Since` 0.27

* Fix documentation in `base.pkl`

* Add location information to empty/single checks in `Listing` operations

* Remove additional variable for laziness preservation

* Apply spotless

* Apply suggestions from code review

Co-authored-by: Daniel Chao <daniel.h.chao@gmail.com>

---------

Co-authored-by: Daniel Chao <daniel.h.chao@gmail.com>
2024-10-31 14:54:27 +00:00
Josh B
dd16f7469e pkl-executor: fix loading projects and fix incorrectly thrown PklException
* Load `PklProject` using the same security settings and env vars passed provided via ExecutorSPIOptions
* Handle thrown `PklException` from loading `PklProject` and throw `ExecutorSPIException`
* Handle thrown `PklException` of older Pkl distributions, converting them into `ExecutorSpiException`
2024-10-29 23:29:19 -07:00
Daniel Chao
acd2222534 Polish test result running and reporting (#738)
Changes:
* Move class `TestResults` to package `org.pkl.core`, because it is a public class (it's the result of `Evaluator#evaluateTest`)
* Change examples to treat individual examples as assertions in the same test. Previously, they were considered different tests with an incrementing number. This better aligns with how facts are treated.
* Change `TestResults` to be a record, and introduce builders.
* Remove "module" test result section (it is not really a section).
* Add javadoc to `TestResults`
* Formatting fix: prefix ✍️ emoji just like we do the  and  emojis 
* Consider writing examples as failures, not successes. `pkl test` will exit with code 10 if the only failing tests are due to writing examples.
2024-10-28 21:05:13 -07:00
Josh B
666f8c3939 Implement SPICE-0009 External Readers (#660)
This adds a new feature, which allows Pkl to read resources and modules from external processes.

Follows the design laid out in SPICE-0009.

Also, this moves most of the messaging API into pkl-core
2024-10-28 18:22:14 -07:00
Daniel Chao
466ae6fd4c Exclude non-ascii files from windows native snippet tests (#739)
There is an existing bug that is preventing the CLI from loading
these files.
2024-10-26 07:56:00 -07:00
Islon Scherer
8b0118fec5 Store PklProject annotations in the project metadata (#708)
Write annotations to project metadata, and provide them to pkl-doc\

The following annotations have meaning for pkl-doc:
* `@Unlisted`: hide package from documentation site
* `@Deprecated`: add deprecated information
2024-10-24 17:21:58 -07:00
Islon Scherer
93cc3253eb Report error on circular local dependencies (#731)
If a stack overflow is found during project evaluation, present any
circular imports found in the dependency graph.
2024-10-24 16:45:18 -07:00
translatenix
1ceb489d78 Fix lazy type checking of UInt types (#740)
For example, Listing<UInt16> is equivalent to Listing<UInt16>, but not to Listing<UInt32>.
2024-10-24 16:19:26 -07:00
Josh B
4b6bc7bb7c Produce more helpful output when module output is overwritten (#716)
Fix a bug where overwriting `output` causes a PklBugException.

This now produces a more helpful message pointing to the actual problem.

Co-authored-by: translatenix <119817707+translatenix@users.noreply.github.com>
2024-10-24 09:41:59 -07:00
Daniel Chao
22c9a6c9f4 Exclude thrown ascii filename error from native tests (#737)
The URI encoding is not predictable here.
2024-10-24 08:20:56 -07:00
Javier Maestro
86d870ba09 Improve testing with stats and errors per test section (#498)
* Emojis are moves to the left to be aligned
* A summary line is added with test counts
* Facts and Examples are grouped under their own section
2024-10-23 23:00:35 -07:00
Josh B
f9fe226eba Fix handling of file: module URIs with non-ASCII characters (#696)
Addresses an issue where Pkl cannot evaluate files with non-ASCII characters.
2024-10-23 20:52:40 -07:00
Daniel Chao
5a654e453c Fix formatting issues (#733) 2024-10-23 20:49:16 -07:00
Daniel Chao
9d10832ffc Fix typecheck error on Listing/Mapping (#725)
The addresses the following case:

    local l1: Listing<String> = new { 1 }
    l2: Listing<Int> = (l1) { 2 }

The member in `l1` should be checked against its owner; to check against `Listing<String>`.
2024-10-23 14:38:30 -07:00
Daniel Chao
ce25cb8ef0 Add analyze imports libs (SPICE-0001) (#695)
This adds a new feature to build a dependency graph of Pkl programs, following the SPICE outlined in https://github.com/apple/pkl-evolution/pull/2.

It adds:
* CLI command `pkl analyze imports`
* Java API `org.pkl.core.Analyzer`
* Pkl stdlib module `pkl:analyze`
* pkl-gradle extension `analyze`

In addition, it also changes the Gradle plugin such that `transitiveModules` is by default computed from the import graph.
2024-10-23 14:36:57 -07:00
Philip K.F. Hölzenspies
eb3891b21f Change license header from doc- to block-comment (#730)
* Change license header from doc to block comment

* Apply fixed license header throughout
2024-10-23 16:02:42 +01:00
Kushal Pisavadia
a7cc098925 Remove isMappingDefault on VmNull (#724)
This method is unused.
2024-10-22 10:41:31 -07:00
Daniel Chao
0ee3d37524 Add support for const object members (#678)
This adds support for adding the `const` modifier to object members.

Such object members are also required to have the `local` modifier applied.

This follows SPICE-0011.
2024-10-21 22:00:12 -07:00
translatenix
d271b62543 Update dependencies (#689)
- Update dependencies by deleting lock files and regenerating them with `gw updateDependencyLocks`.
  Deleting lock files avoids strange `some.library:some.older.version=default` entries.
  Most updated dependencies are test dependencies.
- Handle breaking changes in library commonmark.
- Fix test to close PackageServer exactly once.
  This problem surfaced because JUnit 5.11 changed override rules for lifecycle methods,
  resulting in too many instead of too few close() calls.
- Bump msgpack version
- Bump clikt version
- Bump Gradle plugin versions
2024-10-17 09:12:07 -07:00
Josh B
62c796a257 Fix double unary minus (#697)
Fix an issue where doubly unary minus (e.g. `--1`) on int/float literals are incorrectly
parsed as single unary minus.
2024-10-17 07:30:00 -07:00
translatenix
1022cf3ff3 Replace some usages of java.lang.String.replaceAll with String.replace (#701)
Both methods replace all occurrences.
The difference is that String.replaceAll interprets its first argument as a regex.
2024-10-16 15:00:45 -07:00
translatenix
475f29c896 Set lower limit for glob pattern resolution in test mode (#693)
Motivation:
Speed up the test that verifies enforcement of the limit for glob pattern resolution (invalidGlobImport6.pkl).

Changes:
- Set a lower limit if test mode is enabled.
- Change static field to static method to prevent compile-time evaluation by native-image.
2024-10-15 22:20:25 -07:00
translatenix
d00c466843 Fix dangling Javadoc reference (#687) 2024-10-15 08:26:35 -07:00
translatenix
7f26592360 Use case-insensitive comparison for module URI schemes (#688)
Previously, some factories used case-sensitive comparison, for no apparent reason.
2024-10-15 08:26:23 -07:00
Josh B
1e63c48ce4 Add String.splitLimit API (#666)
* Add String.splitLimit API

* Update stdlib/base.pkl

Co-authored-by: Daniel Chao <daniel.h.chao@gmail.com>

---------

Co-authored-by: Daniel Chao <daniel.h.chao@gmail.com>
2024-10-07 15:28:19 +01:00
Philip K.F. Hölzenspies
cc1e432a30 Add PklProject.deps.json file with newline for POSIX compliance (#664)
* Add Project.deps.json file with newline for POSIX compliance

* Fix test cases
2024-10-01 20:00:46 +01:00
Daniel Chao
47f2143e0d Fix PklBugException when reading local package assets (#642) 2024-09-10 05:51:34 -07:00
Daniel Chao
7868d9d9c8 Typecheck Mapping/Listing members lazily (#628)
This changes how the language performs typechecks for mappings and
listings.

Currently, Pkl will shallow-force any Mapping and Listing to check it
the type parameter (e.g. Listing<Person> means each element is checked
to be an instance of Person).

This changes the language to check each member's type when the member
is accessed.

This also adjust test runner to handle thrown errors from within tests.

With the change to make mapping/listing typechecks lazy, we can now
correctly handle thrown errors from within a single test case.

This adjusts the test runner to consider any thrown errors as a failure
for that specific test case.
2024-09-06 15:05:23 -07:00
Philip K.F. Hölzenspies
e21db76efe Remove const from reserved keywords (remnant) (#626) 2024-08-14 14:48:49 +01:00
Daniel Chao
d5e86ed077 Fix: globbing--read extra storage from owner instead of receiver (#607)
This fixes an issue where a PklBugException is thrown when a globbed
read/import is amended.
2024-08-05 10:27:50 -07:00
Daniel Chao
176ede0002 Fix race condition when concurrently downloading packages (#584)
This fixes a possible race condition where multiple processes download
the same package into the same temp dir.
2024-07-18 09:11:25 -07:00
Islon Scherer
24cc95abcc Support _ in String[toInt|toIntOrNull|toFloat|toFloatOrNull] (#580)
The logic is the same as the Pkl parser
2024-07-16 19:24:59 +02:00
Taichi Ishitani
b5e011dae1 Allow to toInt() to parse a string including "__" (#578)
Fix the issue where String#toInt() cannot parse a Int string
including sequence of "_" like "1_2__3___".
2024-07-15 10:39:17 +02:00
Philip K.F. Hölzenspies
5cc2ea2d00 Revert error coloring (for development on dedicated branch) (#565) 2024-07-03 17:25:06 +01:00
Dan Chao
8c1c10528f Run spotless apply 2024-07-01 09:24:04 -07:00
Dan Chao
7a9b571f6e Use spotless to format remaining Kotlin files
Also: fix issue where some commented out Kotlin files are invalid.
2024-07-01 09:24:04 -07:00
Thomas Purchas
3659ad8b7a Add test for colour outputs
Test two error conditions that cover all current colour outputs.
2024-07-01 10:13:16 +01:00
Thomas Purchas
0d7b95d3ff Add colours to Pkl errors in Cli output
To make error messages from Pkl eval easier to read, this change uses
the Jansi library to colour the output, making it quicker and easier to
scan error messages and understand what's happened.

The Jansi library also detects if the CLI output is a terminal capable
of handling colours, and will automatically strip out escape codes if
the output won't support them (e.g. piping the output somewhere else).
2024-07-01 10:13:16 +01:00
Daniel Chao
51d7265ec6 Do not enable TLS certificate revocation checks by default (#553)
This addresses an issue where network requests may fail if cert revocation checks
error, which may occur due to availability issues, or due to lack of internet access.

Revocation checking can still be enabled by setting JVM property com.sun.net.ssl.checkRevocation if on the JVM.

Also:
* Load built-in certs from resources, and move them to pkl-commons-cli
* Fix an issue where HttpInitException is not caught when loading a module
2024-06-28 08:57:46 -07:00
Philip K.F. Hölzenspies
0ef22a73fb Start next dev iteration 2024-06-17 19:53:30 +01:00
Philip K.F. Hölzenspies
4a27ac5bda Prepare 0.26.0 release 2024-06-17 19:53:30 +01:00
Philip K.F. Hölzenspies
d55866a04b Fix commitish detection in LanguageSnippetTest to also cover release versions (#537) 2024-06-17 18:21:22 +01:00
Daniel Chao
919de4838c Improve handling of CA certificates (#518)
Instead of bundling Pkl's built-in CA certificates as a class path resource and loading them at runtime,
pass them to the native image compiler as the default SSL context's trust store.
This results in faster SSL initialization and is more consistent with how default certificates
are handled when running on the JVM.

Further related improvements:
- Remove HttpClientBuilder methods `addDefaultCliCertificates` and `addBuiltInCertificates`.
- Remove pkl-certs subproject and the optional dependencies on it.
- Move `PklCARoots.pem` to `pkl-cli/src/certs`.
- Fix certificate related error messages that were missing an argument.
- Prevent PklBugException if initialization of `CliBaseOptions.httpClient` fails.
- Add ability to set CA certificates as a byte array
- Add CA certificates option to message passing API
2024-06-12 17:53:03 -07:00
Philip K.F. Hölzenspies
b03530ed1f Add support for HTTP proxying (#506)
* Add `--proxy` and `--no-proxy` CLI flags
* Add property `http` to `pkl:settings`
* Move `EvaluatorSettings` from `pkl:Project` to its own module and add property `http`
* Add support for proxying in server mode, and through Gradle
* Add `setProxy()` to `HttpClient`
* Add documentation
2024-06-12 11:54:22 -07:00
Daniel Chao
a4c0a271b4 Require references from typealiases to be const (#516)
This adds a language change that requires references from typealiases
to the enclosing module to be `const`.

This is required because typealiases are not late-bound.

Rationale is laid out in SPICE-0007.

Also:
* Update documentation to reflect new rules.
* Fix `Project.pkl`; mark method `const` to not break said rule.
2024-06-10 09:01:05 -07:00
Daniel Chao
d5ba8fa736 Support scheme-agnostic projects (#486)
This adds changes to support loading project dependencies in non-file based projects.

The design for this feature can be found in SPICE-0005: https://github.com/apple/pkl-evolution/pull/6

Changes:
* Consider all imports prefixed with `@` as dependency notation.
* Bugfix: fix resolution of glob expressions in a local dependency.
* Adjust pkl.Project:
  - Allow local dependencies from a scheme-local paths.
  - Disallow certain evaluator settings if not loaded as a file-based module.
* Breaking API change: `ProjectDependenciesManager` constructor now requires `ModuleResolver` and `SecurityManager`.
2024-06-04 16:52:20 -07:00
Daniel Chao
c0a7080287 Fix line endings (#513)
* Adjust gitattributes file (mark some files binary, mark bat files as using crlf endings)
* Fix remaining line endings issues
2024-06-04 07:55:33 -07:00
translatenix
207d0c78f0 Fix globbed read/import bugs (#449)
* Split MemberNode into (Regular/Shared)MemberNode

SharedMemberNode enables generating non-constant object members
at run time without generating an unbounded number of Truffle root nodes.

* Invert shouldRunTypeCheck to match its name

* Introduce VmObjectBuilder

Introduce VmObjectBuilder, a uniform way to build `VmObject`s
whose `ObjectMember`s are determined at run time.
Replace some manual object building code with VmObjectBuilder.
Add some assertions to fix IntelliJ warnings.

* Improve implementation of globbed read/import nodes

- Leverage SharedMemberNode to have a single Truffle root node per globbed
  read/import expression instead of one root node per resolved glob element.
- Remove caching in ReadGlobNode because it only works correctly if glob pattern is a string *constant*.
- Remove caching in StaticReadNode (now ReadGlobElementNode/ImportGlobElementNode)
  because it seems unnecessary and the implementation doesn't look quite right.

* Simplify code

* Fix ClassCastException when reflecting on globbed import

* Fix caching of globbed reads

Problem:
The result of a globbed read depends not only on the glob pattern but also on the current module URI.
However, ResourceManager does not take this into account when caching globbed reads, causing wrong results.

Changes:
- Cache globbed reads per read expression.
  This is also how globbed imports are cached, except that import URIs are constant.
- Make ReadGlobNode and ImportGlobNode code as similar as possible.
- Reduce code duplication by inheriting from AbstractReadNode.
- Add some tests.
2024-06-03 17:10:56 -07:00