Polish test result running and reporting (#738)

Changes:
* Move class `TestResults` to package `org.pkl.core`, because it is a public class (it's the result of `Evaluator#evaluateTest`)
* Change examples to treat individual examples as assertions in the same test. Previously, they were considered different tests with an incrementing number. This better aligns with how facts are treated.
* Change `TestResults` to be a record, and introduce builders.
* Remove "module" test result section (it is not really a section).
* Add javadoc to `TestResults`
* Formatting fix: prefix ✍️ emoji just like we do the  and  emojis 
* Consider writing examples as failures, not successes. `pkl test` will exit with code 10 if the only failing tests are due to writing examples.
This commit is contained in:
Daniel Chao
2024-10-28 21:05:13 -07:00
committed by GitHub
parent 666f8c3939
commit acd2222534
15 changed files with 851 additions and 708 deletions

View File

@@ -409,6 +409,10 @@ Evaluate the given `<modules>` as _tests_, producing a test report and appropria
Renderers defined in test files will be ignored by the `test` command.
Tests that result in writing `pkl-expected.pcf` files are considered failing tests.
If these are the only failures, the command exits with exit code 10.
Otherwise, failures result in exit code 1.
<modules>::
The absolute or relative URIs of the modules to test. Relative URIs are resolved against the working directory.