Add analyze imports libs (SPICE-0001) (#695)

This adds a new feature to build a dependency graph of Pkl programs, following the SPICE outlined in https://github.com/apple/pkl-evolution/pull/2.

It adds:
* CLI command `pkl analyze imports`
* Java API `org.pkl.core.Analyzer`
* Pkl stdlib module `pkl:analyze`
* pkl-gradle extension `analyze`

In addition, it also changes the Gradle plugin such that `transitiveModules` is by default computed from the import graph.
This commit is contained in:
Daniel Chao
2024-10-23 14:36:57 -07:00
committed by GitHub
parent eb3891b21f
commit ce25cb8ef0
53 changed files with 2054 additions and 53 deletions

View File

@@ -5407,7 +5407,7 @@ package {
packageZipUrl = "https://example.com/\(name)/\(name)@\(version).zip" // <4>
}
----
<1> The display name of the package. For display purposes only.
<1> The display name of the package.For display purposes only.
<2> The package URI, without the version part.
<3> The version of the package.
<4> The URL to download the package's ZIP file.
@@ -5415,8 +5415,9 @@ package {
The package itself is created by the command xref:pkl-cli:index.adoc#command-project-package[`pkl project package`].
This command only prepares artifacts to be published.
Once the artifacts are prepared, they are expected to be uploaded to an HTTPS server such that the ZIP asset can be downloaded at path `packageZipUrl`, and the metadata can be downloaded at `+https://<package uri>+`.
Once the artifacts are prepared, they are expected to be uploaded to an HTTPS server such that the ZIP asset can be downloaded at path `packageZipUrl`, and the metadata can be downloaded at `+https://<package uri>+`.
[[local-dependencies]]
==== Local dependencies
A project can depend on a local project as a dependency.

View File

@@ -558,14 +558,64 @@ package already exists in the cache directory, this command is a no-op.
This command accepts <<common-options,common options>>.
[[command-analyze-imports]]
=== `pkl analyze imports`
*Synopsis*: `pkl analyze imports [<modules>]`
This command builds a graph of imports declared in the provided modules.
This is a lower level command that is meant to be useful for Pkl-related tooling.
For example, this command feeds into the xref:pkl-gradle:index.adoc[] to determine if tasks are considered up-to-date or not.
This command produces an object with two properties, `imports` and `resolvedImports`.
The `imports` property is a mapping of a module's absolute URI, to the set of imports declared within that module.
The `resolvedImports` property is a mapping of a module's absolute URI (as stated in `imports`), to the resolved absolute URI that might be useful for fetching the module's contents.
For example, a xref:language-reference:index.adoc#local-dependencies[local dependency] import will have an in-language URI with scheme `projectpackage:`, and may have resolved URI with scheme `file:` (assuming that the project is file-based).
Examples:
[source,shell]
----
# Analyze the imports of a single module
pkl analyze imports myModule.pkl
# Same as the previous command, but output in JSON.
pkl analyze imports -f json myModule.pkl
# Analyze imports of all modules declared within src/
pkl analyze imports src/*.pkl
----
<modules>::
The absolute or relative URIs of the modules to analyze. Relative URIs are resolved against the working directory.
==== Options
.-f, --format
[%collapsible]
====
Same meaning as <<format>> in <<command-eval>>.
====
.-o, --output-path
[%collapsible]
====
Same meaning as <<output-path>> in <<command-eval>>.
====
This command also takes <<common-options,common options>>.
[[common-options]]
=== Common options
The <<command-eval>>, <<command-test>>, <<command-repl>>, <<command-project-resolve>>, <<command-project-package>>, and <<command-download-package>> commands support the following common options:
The <<command-eval>>, <<command-test>>, <<command-repl>>, <<command-project-resolve>>, <<command-project-package>>, <<command-download-package>>, and <<command-analyze-imports>> commands support the following common options:
include::../../pkl-cli/partials/cli-common-options.adoc[]
The <<command-eval>>, <<command-test>>, <<command-repl>>, and <<command-download-package>> commands also take the following options:
The <<command-eval>>, <<command-test>>, <<command-repl>>, <<command-download-package>>, and <<command-analyze-imports>> commands also take the following options:
include::../../pkl-cli/partials/cli-project-options.adoc[]

View File

@@ -7,7 +7,6 @@ Comma-separated list of URI patterns that determine which modules can be loaded
Patterns are matched against the beginning of module URIs.
(File paths have been converted to `file:` URLs at this stage.)
At least one pattern needs to match for a module to be loadable.
Both source modules and transitive modules are subject to this check.
====
[[allowed-resources]]

View File

@@ -102,7 +102,6 @@ pkl {
evaluators {
evalPkl {
sourceModules.add(file("module1.pkl"))
transitiveModules.from file("module2.pkl")
outputFile = layout.buildDirectory.file("module1.yaml")
outputFormat = "yaml"
}
@@ -118,7 +117,6 @@ pkl {
evaluators {
register("evalPkl") {
sourceModules.add(file("module1.pkl"))
transitiveModules.from(file("module2.pkl"))
outputFile.set(layout.buildDirectory.file("module1.yaml"))
outputFormat.set("yaml")
}
@@ -127,9 +125,6 @@ pkl {
----
====
To guarantee correct Gradle up-to-date behavior,
`transitiveModules` needs to contain all module files transitively referenced by `sourceModules`.
For each declared evaluator, the Pkl plugin creates an equally named task.
Hence the above evaluator can be run with:
@@ -691,3 +686,61 @@ The project directories to create packages for.
Common properties:
include::../partials/gradle-common-properties.adoc[]
[[analyze-imports]]
== Analyze Imports
This feature is the Gradle analogy for the xref:pkl-cli:index.adoc#command-analyze-imports[analyze imports] command in the CLI. It builds a graph of imports of the provided source modules.
=== Usage
[tabs]
====
build.gradle::
+
[source,groovy]
----
pkl {
analyzers {
imports {
appConfig {
sourceModules.add(file("src/main/resources/appConfig.pkl"))
}
}
}
}
----
build.gradle.kts::
+
[source,kotlin]
----
pkl {
analyzers {
imports {
register("appConfig") {
sourceModules.add(file("src/main/resources/appConfig.pkl"))
}
}
}
}
----
====
=== Configuration Options
.outputFormat: Property<String>
[%collapsible]
====
Same meaning as <<output-format,outputFormat>> in <<module-evaluation>>.
====
.outputFile: RegularFileProperty<String>
[%collapsible]
====
Same meaning as <<output-file,outputFile>> in <<module-evaluation>>.
====
Common properties:
include::../partials/gradle-modules-properties.adoc[]

View File

@@ -7,7 +7,6 @@ URI patterns that determine which modules can be loaded and evaluated.
Patterns are matched against the beginning of module URIs.
(File paths have been converted to `file:` URLs at this stage.)
At least one pattern needs to match for a module to be loadable.
Both source modules and transitive modules are subject to this check.
====
.allowedResources: ListProperty<String>

View File

@@ -20,11 +20,17 @@ This property accepts the following types to represent a module:
.transitiveModules: ConfigurableFileCollection
[%collapsible]
====
Default: `files()` (empty collection) +
Default: [computed by pkl-gradle] +
Example 1: `transitiveModules.from files("module1.pkl", "module2.pkl")` +
Example 2: `+transitiveModules.from fileTree("config").include("**/*.pkl")+` +
File paths of modules that are directly or indirectly used by source modules.
Setting this option enables correct Gradle up-to-date checks, which ensures that your Pkl tasks are executed if any of the transitive files are modified; it does not affect evaluation otherwise.
This property, along with `sourceModules`, is the set of input files used to determine whether this task is up-to-date or not.
By default, Pkl computes this property by analyzing the imports of the source modules.
Setting this property explicitly causes Pkl to skip the analyze imports step.
Including source modules in `transitiveModules` is permitted but not required.
Relative paths are resolved against the project directory.
====