starred/pkl
1
0
Fork 0
mirror of https://github.com/apple/pkl.git synced 2026-09-03 08:17:14 +02:00
Code Issues 198 Packages Projects Releases 10 Wiki Activity

[PR #1169] [MERGED] Introduce pkl-doc model version 2 #925

New Issue
Closed
opened 2025-12-30 01:27:47 +01:00 by adam · 0 comments
adam commented 2025-12-30 01:27:47 +01:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/apple/pkl/pull/1169
Author: @bioball
Created: 8/8/2025
Status: ✅ Merged
Merged: 9/30/2025
Merged by: @bioball

Base: main ← Head: pkldoc-improvements

📝 Commits (6)

  • 43a9ad0 Introduce pkl-doc model version 2
  • b00a035 Re-generate test output
  • 2a7941b Don't use jimfs when testing DocMigratorTest
  • 061d18f Fix test on Windows
  • 0bc6946 Fix DocMigratorTest and other misc
  • b09eba8 Adjust labels for known subtypes/usages within package

📊 Changes

1599 files changed (+129995 additions, -584 deletions)

View changed files

📝 buildSrc/src/main/kotlin/pklFatJar.gradle.kts (+3 -0)
📝 gradle/libs.versions.toml (+2 -0)
📝 pkl-commons-cli/src/main/kotlin/org/pkl/commons/cli/CliException.kt (+10 -2)
➕ pkl-commons-test/src/main/files/packages/birds@0.6.0/birds@0.6.0.json (+27 -0)
➕ pkl-commons-test/src/main/files/packages/birds@0.6.0/package/Bird.pkl (+7 -0)
➕ pkl-commons-test/src/main/files/packages/birds@0.6.0/package/allFruit.pkl (+4 -0)
➕ pkl-commons-test/src/main/files/packages/birds@0.6.0/package/catalog.pkl (+4 -0)
➕ pkl-commons-test/src/main/files/packages/birds@0.6.0/package/catalog/Ostrich.pkl (+7 -0)
➕ pkl-commons-test/src/main/files/packages/birds@0.6.0/package/catalog/Swallow.pkl (+7 -0)
➕ pkl-commons-test/src/main/files/packages/birds@0.6.0/package/some/dir/Bird.pkl (+7 -0)
➕ pkl-commons-test/src/main/files/packages/birds@0.7.0/birds@0.7.0.json (+27 -0)
➕ pkl-commons-test/src/main/files/packages/birds@0.7.0/package/Bird.pkl (+7 -0)
➕ pkl-commons-test/src/main/files/packages/birds@0.7.0/package/allFruit.pkl (+4 -0)
➕ pkl-commons-test/src/main/files/packages/birds@0.7.0/package/catalog.pkl (+4 -0)
➕ pkl-commons-test/src/main/files/packages/birds@0.7.0/package/catalog/Ostrich.pkl (+7 -0)
➕ pkl-commons-test/src/main/files/packages/birds@0.7.0/package/catalog/Swallow.pkl (+7 -0)
➕ pkl-commons-test/src/main/files/packages/birds@0.7.0/package/some/dir/Bird.pkl (+7 -0)
➕ pkl-commons/src/main/kotlin/org/pkl/commons/Control.kt (+33 -0)
📝 pkl-doc/gradle.lockfile (+6 -1)
📝 pkl-doc/pkl-doc.gradle.kts (+1 -0)

...and 80 more files

📄 Description

Implementation of https://github.com/apple/pkl-evolution/pull/20

Currently, in order to update a pkl-doc documentation site, almost the entire existing site is read in order to update metadata like known versions, known subtypes, and more.

For example, adding a new version of a package requires that the existing runtime data of all existing versions be updated. Eventually, this causes the required storage size to balloon exponentially to the number of versions.

This addresses these limitations by:

  1. Updating the runtime data structure to move "known versions" metadata to the package level (the same JSON file is used for all versions).
  2. Eliminating known subtype and known usage information at a cross-package level.
  3. Generating the search index by consuming the previously generated search index.
  4. Generating the main page by consuming the search index.

Because this changes how runtime data is stored, an existing docsite needs to be migrated.

This also introduces a new migration command, pkl-doc --migrate, which transforms an older version of the website into a newer version.

NOTE: most of the additions are due to new input/output tests. I split those off into a separate commit. To review this PR, just take a look at the first commit.

The generated output changes now need to be served via an HTTP server (because of ES6 module imports).
You can review the generated site using a command like python3 -m http.server -d pkl-doc/src/test/files/DocGeneratorTest/output/run-1/).

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/apple/pkl/pull/1169 **Author:** [@bioball](https://github.com/bioball) **Created:** 8/8/2025 **Status:** ✅ Merged **Merged:** 9/30/2025 **Merged by:** [@bioball](https://github.com/bioball) **Base:** `main` ← **Head:** `pkldoc-improvements` --- ### 📝 Commits (6) - [`43a9ad0`](https://github.com/apple/pkl/commit/43a9ad09d4095d17ba08fca5aa44ce797d05a009) Introduce pkl-doc model version 2 - [`b00a035`](https://github.com/apple/pkl/commit/b00a0354170f79ae63dc5005690864829c62f81e) Re-generate test output - [`2a7941b`](https://github.com/apple/pkl/commit/2a7941b70eebcd7ac93d8f589eca21f5ca27f17a) Don't use jimfs when testing DocMigratorTest - [`061d18f`](https://github.com/apple/pkl/commit/061d18fb7b3588e8ffd7bb1d877b8790468aa093) Fix test on Windows - [`0bc6946`](https://github.com/apple/pkl/commit/0bc69467f5804011164c8405c2c170fc5a36ed8b) Fix DocMigratorTest and other misc - [`b09eba8`](https://github.com/apple/pkl/commit/b09eba8339381d20807f27c9d4c3e98e1ca549d8) Adjust labels for known subtypes/usages within package ### 📊 Changes **1599 files changed** (+129995 additions, -584 deletions) <details> <summary>View changed files</summary> 📝 `buildSrc/src/main/kotlin/pklFatJar.gradle.kts` (+3 -0) 📝 `gradle/libs.versions.toml` (+2 -0) 📝 `pkl-commons-cli/src/main/kotlin/org/pkl/commons/cli/CliException.kt` (+10 -2) ➕ `pkl-commons-test/src/main/files/packages/birds@0.6.0/birds@0.6.0.json` (+27 -0) ➕ `pkl-commons-test/src/main/files/packages/birds@0.6.0/package/Bird.pkl` (+7 -0) ➕ `pkl-commons-test/src/main/files/packages/birds@0.6.0/package/allFruit.pkl` (+4 -0) ➕ `pkl-commons-test/src/main/files/packages/birds@0.6.0/package/catalog.pkl` (+4 -0) ➕ `pkl-commons-test/src/main/files/packages/birds@0.6.0/package/catalog/Ostrich.pkl` (+7 -0) ➕ `pkl-commons-test/src/main/files/packages/birds@0.6.0/package/catalog/Swallow.pkl` (+7 -0) ➕ `pkl-commons-test/src/main/files/packages/birds@0.6.0/package/some/dir/Bird.pkl` (+7 -0) ➕ `pkl-commons-test/src/main/files/packages/birds@0.7.0/birds@0.7.0.json` (+27 -0) ➕ `pkl-commons-test/src/main/files/packages/birds@0.7.0/package/Bird.pkl` (+7 -0) ➕ `pkl-commons-test/src/main/files/packages/birds@0.7.0/package/allFruit.pkl` (+4 -0) ➕ `pkl-commons-test/src/main/files/packages/birds@0.7.0/package/catalog.pkl` (+4 -0) ➕ `pkl-commons-test/src/main/files/packages/birds@0.7.0/package/catalog/Ostrich.pkl` (+7 -0) ➕ `pkl-commons-test/src/main/files/packages/birds@0.7.0/package/catalog/Swallow.pkl` (+7 -0) ➕ `pkl-commons-test/src/main/files/packages/birds@0.7.0/package/some/dir/Bird.pkl` (+7 -0) ➕ `pkl-commons/src/main/kotlin/org/pkl/commons/Control.kt` (+33 -0) 📝 `pkl-doc/gradle.lockfile` (+6 -1) 📝 `pkl-doc/pkl-doc.gradle.kts` (+1 -0) _...and 80 more files_ </details> ### 📄 Description Implementation of https://github.com/apple/pkl-evolution/pull/20 Currently, in order to update a pkl-doc documentation site, almost the entire existing site is read in order to update metadata like known versions, known subtypes, and more. For example, adding a new version of a package requires that the existing runtime data of all existing versions be updated. Eventually, this causes the required storage size to balloon exponentially to the number of versions. This addresses these limitations by: 1. Updating the runtime data structure to move "known versions" metadata to the package level (the same JSON file is used for all versions). 2. Eliminating known subtype and known usage information at a cross-package level. 3. Generating the search index by consuming the previously generated search index. 4. Generating the main page by consuming the search index. Because this changes how runtime data is stored, an existing docsite needs to be migrated. This also introduces a new migration command, `pkl-doc --migrate`, which transforms an older version of the website into a newer version. NOTE: most of the additions are due to new input/output tests. I split those off into a separate commit. To review this PR, just take a look at the first commit. The generated output changes now need to be served via an HTTP server (because of ES6 module imports). You can review the generated site using a command like `python3 -m http.server -d pkl-doc/src/test/files/DocGeneratorTest/output/run-1/`). --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
adam added the pull-request label 2025-12-30 01:27:47 +01:00
adam closed this issue 2025-12-30 01:27:47 +01:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
duplicate
enhancement
good first issue
help wanted
pull-request
Mirrored from GitHub Pull Request
No labels pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
adam (Adam Melkus)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/pkl#925