starred/pkl
1
0
Fork 0
mirror of https://github.com/apple/pkl.git synced 2026-01-11 14:20:35 +01:00
Code Issues 198 Packages Projects Releases 10 Wiki Activity

[Feature Request] Convert json/yaml to pkl object/class #2

New Issue
Open
opened 2025-12-30 01:19:12 +01:00 by adam · 10 comments
adam commented 2025-12-30 01:19:12 +01:00
Owner
Copy Link

Originally created by @bielikb on GitHub (Feb 3, 2024).

Problem statement:
The adoption of jsons/yamls is pretty much spread across the community/orgs. pkl goal's to provide configuration-as-code language with rich validation and tooling.
For the cases when pkl tool is the right fit this however entails a rather higher ramp up time for people converting the existing configurations -> pkl -> existing configuration format (json/yaml).

Feature Request:
A converter that would map the existing configurations into pkl templates + modules.
Aware of the complexity of the existing json/yaml configurations, the first versions of the converter could support the simplest mappings && let the configuration authors to convert more complex definitions manually.

Originally created by @bielikb on GitHub (Feb 3, 2024). **Problem statement:** The adoption of jsons/yamls is pretty much spread across the community/orgs. `pkl` goal's to provide **configuration-as-code language with rich validation and tooling**. For the cases when `pkl` tool is the right fit this however entails a rather higher ramp up time for people converting the existing configurations -> pkl -> existing configuration format (json/yaml). **Feature Request:** A converter that would map the existing configurations into pkl templates + modules. Aware of the complexity of the existing json/yaml configurations, the first versions of the converter could support the simplest mappings && let the configuration authors to convert more complex definitions manually.
adam commented 2025-12-30 01:19:13 +01:00
Author
Owner
Copy Link

@jackkleeman commented on GitHub (Feb 3, 2024):

Hey @bielikb! This should be fairly easy to do with a pkl file itself (very meta, I know). pkl files can read yaml files, and output pcf, which is a static subset of pkl that you can then just copy straight into a new file (in fact, you can use pkl to write that file for you, thus making a script that does the full conversion)

Here's a simplified example:

vars.yaml

varReference:
  - kind: HTTPRoute
    path: spec/rules/matches/headers/value
  - kind: HTTPRoute
    path: spec/hostnames
  - kind: HTTPRoute
    path: spec/rules/filters/urlRewrite/path/replaceFullPath
  - kind: HTTPRoute
    path: spec/rules/filters/requestRedirect/hostname
  - kind: HTTPRoute
    path: spec/rules/filters/requestHeaderModifier/set/value
  - kind: AuthenticationFilter
    path: spec/jwtProviders/audiences

converter.pkl

import "pkl:yaml"

file = read("file:vars.yaml")
parsed = new yaml.Parser {}.parse(file)

output {
  value = parsed
}

$ pkl eval converter.pkl

varReference {
  new {
    kind = "HTTPRoute"
    path = "spec/rules/matches/headers/value"
  }
  new {
    kind = "HTTPRoute"
    path = "spec/hostnames"
  }
  new {
    kind = "HTTPRoute"
    path = "spec/rules/filters/urlRewrite/path/replaceFullPath"
  }
  new {
    kind = "HTTPRoute"
    path = "spec/rules/filters/requestRedirect/hostname"
  }
  new {
    kind = "HTTPRoute"
    path = "spec/rules/filters/requestHeaderModifier/set/value"
  }
  new {
    kind = "AuthenticationFilter"
    path = "spec/jwtProviders/audiences"
  }
}

This is essentially a Dynamic format, but with a bit of work you can copy the fields over to the equivalent typed objects (eg, there are typed for all the Kubernetes objects) using reflection. In time hopefully we can provide tools that do this for you :)

@jackkleeman commented on GitHub (Feb 3, 2024): Hey @bielikb! This should be fairly easy to do with a pkl file itself (very meta, I know). pkl files can read yaml files, and output pcf, which is a static subset of pkl that you can then just copy straight into a new file (in fact, you can use pkl to write that file for you, thus making a script that does the full conversion) Here's a simplified example: `vars.yaml` ```yaml varReference: - kind: HTTPRoute path: spec/rules/matches/headers/value - kind: HTTPRoute path: spec/hostnames - kind: HTTPRoute path: spec/rules/filters/urlRewrite/path/replaceFullPath - kind: HTTPRoute path: spec/rules/filters/requestRedirect/hostname - kind: HTTPRoute path: spec/rules/filters/requestHeaderModifier/set/value - kind: AuthenticationFilter path: spec/jwtProviders/audiences ``` `converter.pkl` ```pkl import "pkl:yaml" file = read("file:vars.yaml") parsed = new yaml.Parser {}.parse(file) output { value = parsed } ``` ```bash $ pkl eval converter.pkl varReference { new { kind = "HTTPRoute" path = "spec/rules/matches/headers/value" } new { kind = "HTTPRoute" path = "spec/hostnames" } new { kind = "HTTPRoute" path = "spec/rules/filters/urlRewrite/path/replaceFullPath" } new { kind = "HTTPRoute" path = "spec/rules/filters/requestRedirect/hostname" } new { kind = "HTTPRoute" path = "spec/rules/filters/requestHeaderModifier/set/value" } new { kind = "AuthenticationFilter" path = "spec/jwtProviders/audiences" } } ``` This is essentially a [`Dynamic`](https://pkl-lang.org/package-docs/pkl/0.25.1/base/Dynamic) format, but with a bit of work you can copy the fields over to the equivalent typed objects (eg, there are typed for all the Kubernetes objects) using reflection. In time hopefully we can provide tools that do this for you :)
adam commented 2025-12-30 01:19:14 +01:00
Author
Owner
Copy Link

@holzensp commented on GitHub (Feb 9, 2024):

@jackkleeman is entirely right and this is the starting point of convert.pkl scripts we sometimes write for specific schema (see this example for k8s). Converting JSON/YAML files to a Pkl template can't be done, because the input does not specify the types. One might make educated guesses about the types, but getting it wrong will confuse people who then use the generated templates and believe them to be authoritative.

I wanted to highlight a couple of exiting tools that might help push this along, though;

@holzensp commented on GitHub (Feb 9, 2024): @jackkleeman is entirely right and this is the starting point of `convert.pkl` scripts we sometimes write for specific schema (see [this example for k8s](https://github.com/apple/pkl-pantry/blob/main/packages/k8s.contrib/convert.pkl)). Converting JSON/YAML files to a Pkl _template_ can't be done, because the input does not specify the types. One might make educated guesses about the types, but getting it wrong will confuse people who then use the generated templates and believe them to be authoritative. I wanted to highlight a couple of exiting tools that might help push this along, though; * if you have a JSON Schema, you can convert it to Pkl schema [with `org.json_schema`](https://github.com/apple/pkl-pantry/tree/main/packages/org.json_schema) * if you have a Pkl schema and a bunch of JSON/YAML data, you can use @jackkleeman's solution and [use `deepToTyped` to convert it](https://github.com/apple/pkl-pantry/tree/main/packages/pkl.experimental.deepToTyped) from `Dynamic` to `YourSchema`.
adam commented 2025-12-30 01:19:14 +01:00
Author
Owner
Copy Link

@HalilIbrahimAndic commented on GitHub (Feb 14, 2024):

Hi,
I wrote a short article about conversion between Pkl and other static languages and used this page as a reference. Thank you for your contribution @jackkleeman @holzensp
Here is the article: https://blog.protein.tech/pkl-from-apple-899310b43a96?source=friends_link&sk=286bfd74d0cd0e012098e198b9d18225

@HalilIbrahimAndic commented on GitHub (Feb 14, 2024): Hi, I wrote a short article about conversion between Pkl and other static languages and used this page as a reference. Thank you for your contribution @jackkleeman @holzensp Here is the article: [https://blog.protein.tech/pkl-from-apple-899310b43a96?source=friends_link&sk=286bfd74d0cd0e012098e198b9d18225](https://blog.protein.tech/pkl-from-apple-899310b43a96?source=friends_link&sk=286bfd74d0cd0e012098e198b9d18225)
adam commented 2025-12-30 01:19:14 +01:00
Author
Owner
Copy Link

@moritztim commented on GitHub (Feb 14, 2024):

I would love for this to be integrated directly into the cli. That would really speed up the process of migration

@moritztim commented on GitHub (Feb 14, 2024): I would love for this to be integrated directly into the cli. That would really speed up the process of migration
adam commented 2025-12-30 01:19:14 +01:00
Author
Owner
Copy Link

@moritztim commented on GitHub (Feb 15, 2024):

Easy access to this would also be a great way to learn pkl without having to read the whole tutorial or even know what the types you're dealing with are called.

@moritztim commented on GitHub (Feb 15, 2024): Easy access to this would also be a great way to learn pkl without having to read the whole tutorial or even know what the types you're dealing with are called.
adam commented 2025-12-30 01:19:15 +01:00
Author
Owner
Copy Link

@jackkleeman commented on GitHub (Feb 15, 2024):

I think in principle we could have a pkl-pantry module to help with this and it could be called with a one line like pkl eval module://... -p file=a.json

@jackkleeman commented on GitHub (Feb 15, 2024): I think in principle we could have a pkl-pantry module to help with this and it could be called with a one line like `pkl eval module://... -p file=a.json`
adam commented 2025-12-30 01:19:15 +01:00
Author
Owner
Copy Link

@bvalyou commented on GitHub (Apr 10, 2024):

I think adding Parser to the Language Reference would be valuable - I had to search for this issue to find the syntax to parse yaml to an object.

@bvalyou commented on GitHub (Apr 10, 2024): I think adding Parser to the [Language Reference](https://pkl-lang.org/main/current/language-reference/index.html) would be valuable - I had to search for this issue to find the syntax to parse yaml to an object.
adam commented 2025-12-30 01:19:15 +01:00
Author
Owner
Copy Link

@holzensp commented on GitHub (Apr 12, 2024):

It's not a general language construct; it's part of the standard library. We document individual modules (like "pkl:json" and "pkl:yaml") there.

As to CLI accessibility; it's not entirely clear what that would do / look like. You can't derive the schema from an untyped source. If it's only about data conversion, you can always:

pkl eval -x 'new Parser {}.parse("[0, 1, 2, 3]")' pkl:json

or, if you have foo.json:

pkl eval -p input="$(cat foo.json)" -x 'new Parser {}.parse(read("prop:input"))' pkl:json

That could be somewhat shorter, but the wins have a low ceiling here

@holzensp commented on GitHub (Apr 12, 2024): It's not a general language construct; it's part of the standard library. We document individual modules (like `"pkl:json"` and `"pkl:yaml"`) there. As to CLI accessibility; it's not entirely clear what that would do / look like. You can't derive the schema from an untyped source. If it's only about data conversion, you can always: ``` pkl eval -x 'new Parser {}.parse("[0, 1, 2, 3]")' pkl:json ``` or, if you have `foo.json`: ``` pkl eval -p input="$(cat foo.json)" -x 'new Parser {}.parse(read("prop:input"))' pkl:json ``` That could be _somewhat_ shorter, but the wins have a low ceiling here
adam commented 2025-12-30 01:19:16 +01:00
Author
Owner
Copy Link

@moritztim commented on GitHub (Apr 16, 2024):

I guess it's better in the long run anyways, for new users to learn the syntax instead of using shorthands.

PS. @holzensp pls check my issue template pr. I'm stuck there (I will likely not work on it til may tho, cuz of finals)

@moritztim commented on GitHub (Apr 16, 2024): I guess it's better in the long run anyways, for new users to learn the syntax instead of using shorthands. PS. @holzensp pls check my issue template pr. I'm stuck there (I will likely not work on it til may tho, cuz of finals)
adam commented 2025-12-30 01:19:16 +01:00
Author
Owner
Copy Link

@holzensp commented on GitHub (Apr 16, 2024):

PS. @holzensp pls check my issue template pr. I'm stuck there (I will likely not work on it til may tho, cuz of finals)

Done. Apologies for letting it sit and best of luck with finals.

@holzensp commented on GitHub (Apr 16, 2024): > PS. @holzensp pls check my issue template pr. I'm stuck there (I will likely not work on it til may tho, cuz of finals) Done. Apologies for letting it sit and best of luck with finals.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
duplicate
enhancement
good first issue
help wanted
pull-request
Mirrored from GitHub Pull Request
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/pkl#2
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?