[PR #204] [CLOSED] Upstream: Build Upgrades #448

New Issue

Closed

opened 2025-12-30 01:24:33 +01:00 by adam · 0 comments

adam commented 2025-12-30 01:24:33 +01:00

Owner

Copy Link

Copy Source

📋 Pull Request Information

Original PR: https://github.com/apple/pkl/pull/204
Author: @sgammon
Created: 2/19/2024
Status: ❌ Closed

Base: main ← Head: chore/build-upgrades

---

📝 Commits (10+)

• 2f12dbd Fix dropped implementSerializable flag
• f6c5bf3 Add setting for Kotlin package to codegen
• 2e45a9f Implement support for KotlinX Serialization
• 4a9a5ee General build upgrades and improvements
• 8fcf9cb GitHub Actions checks for PR review
• e3a1a2d feat: library api validation with kotlinx binary validator
• 2cc10a2 chore: add api pins for public modules
• 49a54cc fixup! cleanups before push
• 78a3534 fixup! drop buildless action in ci
• cdb3c66 fixup! pr checks, dependency graph

📊 Changes

166 files changed (+18060 additions, -1385 deletions)

View changed files

➕ .github/codeql/codeql-config.yml (+7 -0)
➕ .github/dependency-review-config.yml (+11 -0)
➕ .github/workflows/checks.apicheck.yml (+76 -0)
➕ .github/workflows/checks.codeql.yml (+95 -0)
➕ .github/workflows/checks.dependency-review.yml (+33 -0)
➕ .github/workflows/checks.detekt.yml (+73 -0)
➕ .github/workflows/checks.formatting.yml (+84 -0)
➕ .github/workflows/checks.gradle-wrapper.yml (+25 -0)
➕ .github/workflows/checks.scorecards.yml (+62 -0)
➕ .github/workflows/job.build.yml (+104 -0)
➕ .github/workflows/job.dependency-graph.yml (+75 -0)
➕ .github/workflows/job.native-build.yml (+200 -0)
➕ .github/workflows/job.tests.yml (+146 -0)
➕ .github/workflows/on.pr.yml (+195 -0)
➕ .github/workflows/on.push.yml (+80 -0)
📝 .gitignore (+4 -0)
📝 .idea/codeStyles/Project.xml (+0 -2)
📝 .idea/inspectionProfiles/Project_Default.xml (+1 -0)
📝 bench/bench.gradle.kts (+11 -12)
📝 bench/gradle.lockfile (+1 -40)

...and 80 more files

📄 Description

Summary

Not for merge

This is a massive PR that upgrades the Gradle build in a big way. I did this sort of for fun, over the weekend, while working on other PRs which are reasonable and not enormous.

We are intending to use and extend Pkl so we could maintain a fork, and I have no expectation that this will get merged as-is. That being said, since we've done the leg work on some of this stuff and it could be useful, I wanted to file and offer any of the below as a menu of changes the Pkl team could get upstreamed from us if desired.

There are a handful of test failures and checksums have changed, which I am fixing. Total broken tests number less than 10, I believe, and the breakages seem fixable.

Feedback is of course welcome but since this PR isn't designed to be merged, such feedback would be incorporated in another PR, which is smaller and reviewable.

Thank you to the Pkl team for a very cool project!

Update: Feb 21st

New batch of changes pushed with the following:

• chore: buildSrc refactored to includeBuild("build-logic") (here's why)
• feat: upgrade GraalVM → latest (23.1.2)
• feat: support for --strict-image-heap
• feat: support for GraalVM PGO in native mode
• feat: preinitialization of pkl truffle module
• feat: support for engine sharing/context re-use
• feat: native jpms support (module-info)
• feat: faster native builds with -Ob
• feat: light plugin system for embedded customization of pkl-core engine
• feat: use epsilon gc in native binary note: rolling this back bc pkl server

Initial native binary benchmarks show a modest improvement:

Download URLs to try it out

• macOS aarch64: zip, tar.gz

Update: Split-out PRs (Feb 20th)

• `is` operator does not take current scope into account (#213)
• Not possible to use `output` keyword even surrounded by backticks (#212)
• Support for future objects (#211)

Changes enclosed

0) Fun developer ergonomics stuff.

• Pkl icon for IDEA project
• Pkl syntax highlighting in some test fragments using IDEA Language Injections

1) Build upgrades.

• Support for Gradle toolchains
• Support for Gradle's jvm-test-suite plugins
• Separate toolchain and library target JVMs
• Unified test and coverage reporting
• Detekt, PMD, with baseline set
• Local + remote build caching support
• All build warnings fixed
• Transition to property = value syntax instead of property.set(value)
• Kotlin upgrade → 1.9.22 (for build)
• Gradle upgraded → 8.6
• Toolchain JVM → 21

2) Supply chain security.

• OWASP dependency checks
• Gradle dependency verification
• SPDX SBOM (not yet landed)
• Signing of release artifacts with Sigstore (not yet landed)

3) GitHub project integration/ergonomics

• Dependency Review on PRs
• Github Dependency Graph build and submission
• Enforcement of formatting (Spotless) on PRs
• Checking of published Kotlin library APIs for breakages on PRs

4) Library updates / upgrades

• Kotlin upgrade → 1.9.22 (for codegen and scripting)
• Includes Multiple file output does not work with symlinks (#196)

Full changelog

Expand to see full changelog

- chore: `buildSrc` refactored to `includeBuild("build-logic")`
- feat: upgrade GraalVM → latest (`23.1.2`)
- feat: support for `--strict-image-heap`
- feat: support for GraalVM PGO in native mode
- feat: preinitialization of `pkl` truffle module
- feat: support for engine sharing/context re-use
- feat: native jpms support (`module-info`)
- feat: faster native builds with `-Ob`
- feat: support for build scans with gradle enterprise
- feat: support for caching with buildless (inert without key)
- feat: support for gradle java toolchains
- feat: support for static analysis with detekt
- feat: support for toolchain vs. runtime target
- feat: support for dynamic provisioning of toolchains
- feat: support for static java checking with pmd
- feat: enable typed project accessors, use them project-wide
- feat: stricter repositories, locking for build classpath
- feat: property to retarget java or kotlin bytecode versions
- feat: parameter name integration between javac and kotlinc
- feat: kotlin coverage support via `kover` plugin
- feat: dependency verification for gradle build
- feat: support for new gradle `jvm-test-suite` plugin
- feat: reasonable local and remote build caching support
- feat: project icon in intellij new ui
- feat: aggregate reporting for tests, coverage, detekt
- fix: repeatable/consistent archives from gradle
- fix: don't list ephemeral spotless configurations in lockfiles
- fix: make version catalog symbols available in `buildSrc`
- fix: specify `rootProject.name` for `buildSrc`
- fix: error when running `gradlew tasks`
- fix: various java or gradle deprecations
- fix: move all tool (linter, etc) versions into version catalog
- chore: add testlogger for clearer test outcomes
- chore: check build configuration with gradle doctor plugin
- chore: generate initial suite of dependency verification material
- chore: transition to property set syntax (`property = xyz`)
- chore: cleanup uses of `buildDir` (becomes `layout.buildDirectory`)
- chore: add Gradle Versions plugin for update checks
- chore: upgrade Gradle → `8.6` (supports Java 21)
- chore: upgrade Kotlin → `1.9.22` (build-time)
- chore: upgrade KotlinX Serialization → `1.6.3`
- chore: upgrade KotlinX HTML → `0.11.0`
- chore: general dependency upgrades, where safe

Not yet completed

• feat: signing of artifacts with sigstore
• feat: embedding of SPDX SBOM in artifacts
• feat: dependency vulnerability checks with owasp
• test: checksum failures

---

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/apple/pkl/pull/204 **Author:** [@sgammon](https://github.com/sgammon) **Created:** 2/19/2024 **Status:** ❌ Closed **Base:** `main` ← **Head:** `chore/build-upgrades` --- ### 📝 Commits (10+) - [`2f12dbd`](https://github.com/apple/pkl/commit/2f12dbd887143a0a856faa536cccd22936dd21bd) Fix dropped `implementSerializable` flag - [`f6c5bf3`](https://github.com/apple/pkl/commit/f6c5bf3e47f2bc1b152b03fb7c029cdd373be3c6) Add setting for Kotlin package to codegen - [`2e45a9f`](https://github.com/apple/pkl/commit/2e45a9ffd751139352043779743dd72aff581990) Implement support for KotlinX Serialization - [`4a9a5ee`](https://github.com/apple/pkl/commit/4a9a5eebdfc12bf605ac155daeaaef0afc36062f) General build upgrades and improvements - [`8fcf9cb`](https://github.com/apple/pkl/commit/8fcf9cb166f89aef137952485df514032f998d91) GitHub Actions checks for PR review - [`e3a1a2d`](https://github.com/apple/pkl/commit/e3a1a2db005434f713146b7aabbb6ae4e0fcfd21) feat: library api validation with kotlinx binary validator - [`2cc10a2`](https://github.com/apple/pkl/commit/2cc10a2ba010b5993ce2044b384de2e9f876714d) chore: add api pins for public modules - [`49a54cc`](https://github.com/apple/pkl/commit/49a54cc7c222bc912a7e45bfc8ae2abd687d3a61) fixup! cleanups before push - [`78a3534`](https://github.com/apple/pkl/commit/78a3534226e008f47a163ac79bcdf7d56929b75c) fixup! drop buildless action in ci - [`cdb3c66`](https://github.com/apple/pkl/commit/cdb3c6601e656cff5edef73a5104d93014d2cd04) fixup! pr checks, dependency graph ### 📊 Changes **166 files changed** (+18060 additions, -1385 deletions) <details> <summary>View changed files</summary> ➕ `.github/codeql/codeql-config.yml` (+7 -0) ➕ `.github/dependency-review-config.yml` (+11 -0) ➕ `.github/workflows/checks.apicheck.yml` (+76 -0) ➕ `.github/workflows/checks.codeql.yml` (+95 -0) ➕ `.github/workflows/checks.dependency-review.yml` (+33 -0) ➕ `.github/workflows/checks.detekt.yml` (+73 -0) ➕ `.github/workflows/checks.formatting.yml` (+84 -0) ➕ `.github/workflows/checks.gradle-wrapper.yml` (+25 -0) ➕ `.github/workflows/checks.scorecards.yml` (+62 -0) ➕ `.github/workflows/job.build.yml` (+104 -0) ➕ `.github/workflows/job.dependency-graph.yml` (+75 -0) ➕ `.github/workflows/job.native-build.yml` (+200 -0) ➕ `.github/workflows/job.tests.yml` (+146 -0) ➕ `.github/workflows/on.pr.yml` (+195 -0) ➕ `.github/workflows/on.push.yml` (+80 -0) 📝 `.gitignore` (+4 -0) 📝 `.idea/codeStyles/Project.xml` (+0 -2) 📝 `.idea/inspectionProfiles/Project_Default.xml` (+1 -0) 📝 `bench/bench.gradle.kts` (+11 -12) 📝 `bench/gradle.lockfile` (+1 -40) _...and 80 more files_ </details> ### 📄 Description ## Summary **Not for merge** This is a massive PR that upgrades the Gradle build in a big way. I did this sort of for fun, over the weekend, while working on other PRs which are reasonable and not enormous. We are intending to use and extend Pkl so we could maintain a fork, and I have no expectation that this will get merged as-is. That being said, since we've done the leg work on some of this stuff and it could be useful, I wanted to file and **offer any of the below** as a menu of changes the Pkl team could get upstreamed from us if desired. There are a handful of test failures and checksums have changed, which I am fixing. Total broken tests number less than 10, I believe, and the breakages seem fixable. Feedback is of course welcome but since this PR isn't designed to be merged, such feedback would be incorporated in another PR, which is smaller and reviewable. Thank you to the Pkl team for a very cool project! ## Update: Feb 21st New batch of changes pushed with the following: - chore: `buildSrc` refactored to `includeBuild("build-logic")` ([here's why](https://proandroiddev.com/stop-using-gradle-buildsrc-use-composite-builds-instead-3c38ac7a2ab3)) - feat: upgrade GraalVM → latest (`23.1.2`) - feat: support for `--strict-image-heap` - feat: support for GraalVM PGO in native mode - feat: preinitialization of `pkl` truffle module - feat: support for engine sharing/context re-use - feat: native jpms support (`module-info`) - feat: faster native builds with `-Ob` - feat: light plugin system for embedded customization of `pkl-core` engine - ~~feat: use `epsilon` gc in native binary~~ _note: rolling this back bc `pkl server`_ Initial native binary benchmarks show a modest improvement: <img width="817" alt="Screenshot 2024-02-21 at 11 24 08 AM" src="https://github.com/apple/pkl/assets/171897/ea30d4bf-ae03-4ede-9475-cfb7bca0ba0a"> ### Download URLs to try it out - **macOS `aarch64`**: [zip](https://dl.elide.cloud/cli/pkl/labs/pkl-gvm-23.1.2.zip), [tar.gz](https://dl.elide.cloud/cli/pkl/labs/pkl-gvm-23.1.2.tar.gz) ## Update: Split-out PRs (Feb 20th) - #213 - #212 - #211 ## Changes enclosed **0) Fun developer ergonomics stuff.** - Pkl icon for IDEA project - Pkl syntax highlighting in some test fragments using IDEA [Language Injections](https://www.jetbrains.com/help/idea/using-language-injections.html) **1) Build upgrades.** - Support for Gradle toolchains - Support for Gradle's `jvm-test-suite` plugins - Separate toolchain and library target JVMs - Unified test and coverage reporting - Detekt, PMD, with baseline set - Local + remote build caching support - All build warnings fixed - Transition to `property = value` syntax instead of `property.set(value)` - Kotlin upgrade → `1.9.22` (for build) - Gradle upgraded → `8.6` - Toolchain JVM → `21` **2) Supply chain security.** - OWASP dependency checks - Gradle dependency verification - SPDX SBOM (not yet landed) - Signing of release artifacts with Sigstore (not yet landed) **3) GitHub project integration/ergonomics** - Dependency Review on PRs - Github Dependency Graph build and submission - Enforcement of formatting (Spotless) on PRs - Checking of published Kotlin library APIs for breakages on PRs **4) Library updates / upgrades** - Kotlin upgrade → `1.9.22` (for codegen and scripting) - Includes #196 ### Full changelog <details> <summary>Expand to see full changelog</summary> <pre> - chore: `buildSrc` refactored to `includeBuild("build-logic")` - feat: upgrade GraalVM → latest (`23.1.2`) - feat: support for `--strict-image-heap` - feat: support for GraalVM PGO in native mode - feat: preinitialization of `pkl` truffle module - feat: support for engine sharing/context re-use - feat: native jpms support (`module-info`) - feat: faster native builds with `-Ob` - feat: support for build scans with gradle enterprise - feat: support for caching with buildless (inert without key) - feat: support for gradle java toolchains - feat: support for static analysis with detekt - feat: support for toolchain vs. runtime target - feat: support for dynamic provisioning of toolchains - feat: support for static java checking with pmd - feat: enable typed project accessors, use them project-wide - feat: stricter repositories, locking for build classpath - feat: property to retarget java or kotlin bytecode versions - feat: parameter name integration between javac and kotlinc - feat: kotlin coverage support via `kover` plugin - feat: dependency verification for gradle build - feat: support for new gradle `jvm-test-suite` plugin - feat: reasonable local and remote build caching support - feat: project icon in intellij new ui - feat: aggregate reporting for tests, coverage, detekt - fix: repeatable/consistent archives from gradle - fix: don't list ephemeral spotless configurations in lockfiles - fix: make version catalog symbols available in `buildSrc` - fix: specify `rootProject.name` for `buildSrc` - fix: error when running `gradlew tasks` - fix: various java or gradle deprecations - fix: move all tool (linter, etc) versions into version catalog - chore: add testlogger for clearer test outcomes - chore: check build configuration with gradle doctor plugin - chore: generate initial suite of dependency verification material - chore: transition to property set syntax (`property = xyz`) - chore: cleanup uses of `buildDir` (becomes `layout.buildDirectory`) - chore: add Gradle Versions plugin for update checks - chore: upgrade Gradle → `8.6` (supports Java 21) - chore: upgrade Kotlin → `1.9.22` (build-time) - chore: upgrade KotlinX Serialization → `1.6.3` - chore: upgrade KotlinX HTML → `0.11.0` - chore: general dependency upgrades, where safe </pre> </details> ### Not yet completed - feat: signing of artifacts with sigstore - feat: embedding of SPDX SBOM in artifacts - feat: dependency vulnerability checks with owasp - test: checksum failures --- _{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

adam added the pull-request label 2025-12-30 01:24:33 +01:00

adam closed this issue 2025-12-30 01:24:33 +01:00

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

dependabot/github_actions/gradle/actions/dependency-submission-6.2.0

release/0.32

dependabot/gradle/com.squareup-kotlinpoet-2.3.0

truffle-object-model-clean

release/0.31

release/0.30

release/0.29

release/0.28

release/0.27

release/0.26

release/0.25

0.32.1

0.32.0

0.31.1

0.31.0

0.30.2

0.30.1

0.30.0

0.29.1

0.29.0

0.28.2

0.28.1

0.28.0

0.27.2

0.27.1

0.27.0

0.26.3

0.26.2

0.26.1

0.26.0

0.25.3

0.25.2

0.25.1

0.25.0

Labels

Clear labels

bug

duplicate

enhancement

good first issue

help wanted

pull-request

Mirrored from GitHub Pull Request

No labels pull-request

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

adam (Adam Melkus)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/pkl#448