Pain Point: Deeply Nested Amends (No Flat Member Syntax) #143

New Issue

2025-12-30T01:21:26+01:00

adam commented

2025-12-30 01:21:26 +01:00

Originally created by @dwbrite on GitHub (Apr 12, 2024).

I'm testing Pkl as a replacement for k8s tools like Kustomize/Helm. So far it's quite nice, but one pain point is modifying deeply nested properties.

As an example, say I have a HomeAssistant pkl package with a default volume claim of 1Gi, which I'd like to bump up to 2Gi.

import "ha-app.pkl" as homeassistant // extends AppEnvCluster, sets isLeafModule() = true
// assume the above is a third-party package

// default HomeAssistant application, with one modified property
local ha_app = (homeassistant) {
  statefulSets {
    ["core"] {
      spec {
        volumeClaimTemplates {
          [[metadata.name == "homeassistant-config"]] {
            spec {
              resources {
                requests {
                  ["storage"] = "2Gi"
                } // 9 lines of brackets :')
              }
            }
          }
        }
      }
    }
  }
}

output {
  files { // for use by argocd
    ["homeassistant.yaml"] = ha_app.output
  }
}

18 LoC for a single deeply nested amendment seems a little excessive.
One way I've found to improve this is to remove some newlines, but I don't find this ideal.

// I guess this is better?
local ha_app = (homeassistant) { statefulSets { ["core"] { spec { volumeClaimTemplates {
  [[metadata.name == "homeassistant-config"]] { spec { resources { requests {
    ["storage"] = "2Gi"
  }}}}
}}}}}

I could create late-binding variables in a new pkl module which extends ha-app further, but that seems like I'd just be moving the problem down the line.

Personally I'd like to see dot syntax supported for amendments. In my case, that might look something like:

local ha_app = (homeassistant) {
  statefulSets["core"].spec.volumeClaimTemplates {
    [[metadata.name == "homeassistant-config"]] {
      spec.resources.requests["storage"] = "2Gi"
    }
  }
}

Am I missing a more idiomatic solution?

Originally created by @dwbrite on GitHub (Apr 12, 2024). I'm testing Pkl as a replacement for k8s tools like Kustomize/Helm. So far it's quite nice, but one pain point is modifying deeply nested properties. As an example, say I have a HomeAssistant pkl package with a default volume claim of 1Gi, which I'd like to bump up to 2Gi. ``` Pkl import "ha-app.pkl" as homeassistant // extends AppEnvCluster, sets isLeafModule() = true // assume the above is a third-party package // default HomeAssistant application, with one modified property local ha_app = (homeassistant) { statefulSets { ["core"] { spec { volumeClaimTemplates { [[metadata.name == "homeassistant-config"]] { spec { resources { requests { ["storage"] = "2Gi" } // 9 lines of brackets :') } } } } } } } } output { files { // for use by argocd ["homeassistant.yaml"] = ha_app.output } } ``` 18 LoC for a single deeply nested amendment seems a little excessive. One way I've found to improve this is to remove some newlines, but I don't find this _ideal_. ``` Pkl // I guess this is better? local ha_app = (homeassistant) { statefulSets { ["core"] { spec { volumeClaimTemplates { [[metadata.name == "homeassistant-config"]] { spec { resources { requests { ["storage"] = "2Gi" }}}} }}}}} ``` I could create late-binding variables in a new pkl module which extends `ha-app` further, but that seems like I'd just be moving the problem down the line. --- Personally I'd like to see dot syntax supported for amendments. In my case, that might look something like: ``` Pkl local ha_app = (homeassistant) { statefulSets["core"].spec.volumeClaimTemplates { [[metadata.name == "homeassistant-config"]] { spec.resources.requests["storage"] = "2Gi" } } } ``` Am I missing a more idiomatic solution?

adam commented

2025-12-30 01:21:27 +01:00

@bioball commented on GitHub (Apr 12, 2024):

We'd like to add support for flat member syntax (the solution that you suggested) at some point. I definitely agree that this a pain point.

@bioball commented on GitHub (Apr 12, 2024): We'd like to add support for flat member syntax (the solution that you suggested) at some point. I definitely agree that this a pain point.

adam commented

2025-12-30 01:21:28 +01:00

@dwbrite commented on GitHub (Apr 12, 2024):

That's great to hear, and thanks for the quick response! Feel free to change the title to something more suitable for tracking flat member syntax.

Fwiw, Pkl has been fantastic for me, and I often find myself telling people how amazing it is :)
I'm hoping I can bring it into my day job soon 🤞

@dwbrite commented on GitHub (Apr 12, 2024): That's great to hear, and thanks for the quick response! Feel free to change the title to something more suitable for tracking flat member syntax. Fwiw, Pkl has been fantastic for me, and I often find myself telling people how amazing it is :) I'm hoping I can bring it into my day job soon 🤞

adam commented

2025-12-30 01:21:28 +01:00

@odenix commented on GitHub (Apr 12, 2024):

foo.bar.baz.qux {...} looks very similar to expressions foo.bar.baz.qux and (foo.bar.baz.qux) {...}, which complicates parsing and reading. I think a different syntax should at least be considered.

@odenix commented on GitHub (Apr 12, 2024): `foo.bar.baz.qux {...}` looks very similar to expressions `foo.bar.baz.qux` and `(foo.bar.baz.qux) {...}`, which complicates parsing and reading. I think a different syntax should at least be considered.

adam commented

2025-12-30 01:21:28 +01:00

@holzensp commented on GitHub (Apr 12, 2024):

Actually, the (...) in object amends syntax disambiguates that entirely. A slightly trickier thing about flat member syntax is; what do you do with consecutive amends;

foo.bar.baz = 0
foo.bar.qux.quux = "hello"
foo.bar.qux.corge = "world"

Is that the same as

foo {
  bar {
    baz = 0
    qux {
      quux = "hello"
      corge = "world"
    }
  }
}

or

foo {
  bar {
    baz = 0
  }
} {
  bar {
    qux {
      quux = "hello"
    }
  }
} {
  bar {
    qux {
      corge = "world"
    }
  }
}

That has implications for what super means (and for performance, but that's a shortcoming of the current implementation - albeit non-trivial). There are reasons why you're currently not allowed to amend the same property twice. Flat member syntax makes that ambiguous, because the overlap may be partial.

Or... should that ambiguity should syntactically be disallowed and should this require that properties are aggregated in notation, i.e.

foo.bar {
  baz = 0
  qux {
    quux = "hello"
    corge = "world"
  }
}

@holzensp commented on GitHub (Apr 12, 2024): Actually, the `(`...`)` in object amends syntax disambiguates that entirely. A slightly trickier thing about flat member syntax is; what do you do with consecutive amends; ``` foo.bar.baz = 0 foo.bar.qux.quux = "hello" foo.bar.qux.corge = "world" ``` Is that the same as ``` foo { bar { baz = 0 qux { quux = "hello" corge = "world" } } } ``` _or_ ``` foo { bar { baz = 0 } } { bar { qux { quux = "hello" } } } { bar { qux { corge = "world" } } } ``` That has implications for what `super` means (and for performance, but that's a shortcoming of the current implementation - albeit non-trivial). There are reasons why you're currently not allowed to amend the same property twice. Flat member syntax makes that ambiguous, because the overlap may be partial. Or... should that ambiguity should syntactically be disallowed and should this _require_ that properties are aggregated in notation, i.e. ``` foo.bar { baz = 0 qux { quux = "hello" corge = "world" } } ```

adam commented

2025-12-30 01:21:29 +01:00

@bioball commented on GitHub (Apr 12, 2024):

In both of those cases, super.foo should have the same meaning, no? It means the parent foo value in the prototype chain. super by itself isn't a valid expression.

An interesting problem here is what outer should mean. It would be surprising if these two were the same:

foo.bar.baz = outer.qux

foo {
  bar {
    baz = outer.qux
  }
}

Another one is this:

bar = 15

foo.bar.baz = bar

It would be surprising if it expands to the below snippet, because in the below snippet, the bar within baz = bar references the middle bar, rather than the module-level bar.

bar = 15

foo {
  bar {
    baz = bar
  }
}

@bioball commented on GitHub (Apr 12, 2024): In both of those cases, `super.foo` should have the same meaning, no? It means the parent `foo` value in the prototype chain. `super` by itself isn't a valid expression. An interesting problem here is what `outer` should mean. It would be surprising if these two were the same: ``` foo.bar.baz = outer.qux ``` ``` foo { bar { baz = outer.qux } } ``` Another one is this: ``` bar = 15 foo.bar.baz = bar ``` It would be surprising if it expands to the below snippet, because in the below snippet, the `bar` within `baz = bar` references the middle `bar`, rather than the module-level `bar`. ``` bar = 15 foo { bar { baz = bar } } ```

adam commented

2025-12-30 01:21:29 +01:00

@odenix commented on GitHub (Apr 12, 2024):

Actually, the (...) in object amends syntax disambiguates that entirely.

Distinguishing foo.bar.baz.qux {…} from foo.bar.baz.qux requires arbitrary lookahead. It’s also difficult to parse for a human, especially in the middle of a large program. The similarity with (foo.bar.baz.qux) {…} further complicates human parsing.

@odenix commented on GitHub (Apr 12, 2024): > Actually, the (...) in object amends syntax disambiguates that entirely. Distinguishing `foo.bar.baz.qux {…}` from `foo.bar.baz.qux` requires arbitrary lookahead. It’s also difficult to parse for a human, especially in the middle of a large program. The similarity with `(foo.bar.baz.qux) {…}` further complicates human parsing.

adam commented

2025-12-30 01:21:29 +01:00

@bioball commented on GitHub (Apr 12, 2024):

Distinguishing foo.bar.baz.qux {…} from foo.bar.baz.qux requires arbitrary lookahead. It’s also difficult to parse for a human, especially in the middle of a large program. The similarity with (foo.bar.baz.qux) {…} further complicates human parsing.

I see your point, but I'm not very convinced that this is a problem. This is unambiguous without needing lookahead:

foo.bar.baz.qux { ... }

This does require lookahead (because bar.baz.qux is a valid expression, and expressions are valid object members):

foo {
  bar.baz.qux { ... }
}

But:

Generally I wouldn't expect to see code that looks like this. I would more expect them to be flattened at the top level, as per the first snippet in my post.
Even in lookahead cases, it seems easy to distinguish.

Before shipping this feature, we'd definitely need to play around with this syntax in real world code.

@bioball commented on GitHub (Apr 12, 2024): > Distinguishing foo.bar.baz.qux {…} from foo.bar.baz.qux requires arbitrary lookahead. It’s also difficult to parse for a human, especially in the middle of a large program. The similarity with (foo.bar.baz.qux) {…} further complicates human parsing. I see your point, but I'm not very convinced that this is a problem. This is unambiguous without needing lookahead: ``` foo.bar.baz.qux { ... } ``` This does require lookahead (because `bar.baz.qux` is a valid expression, and expressions are valid object members): ``` foo { bar.baz.qux { ... } } ``` But: 1. Generally I wouldn't expect to see code that looks like this. I would more expect them to be flattened at the top level, as per the first snippet in my post. 2. Even in lookahead cases, it seems easy to distinguish. Before shipping this feature, we'd definitely need to play around with this syntax in real world code.

adam commented

2025-12-30 01:21:29 +01:00

@odenix commented on GitHub (Apr 12, 2024):

The lookahead here seems easy to me.

Arbitrary lookahead can cause problems with syntax highlighters and parser tools. It can also cause performance issues. It’s best avoided if possible.
https://github.com/elves/elvish/issues/664

This is unambiguous without needing lookahead:

I think that’s only the case because typed objects can’t have elements and entries. Is this limitation here to stay?

I would more expect them to be flattened at the top level, as per the first snippet in my post.

I think it’s best to avoid such assumptions when designing a feature. If the feature can be used inside objects, it will be used inside objects.

@odenix commented on GitHub (Apr 12, 2024): > The lookahead here seems easy to me. Arbitrary lookahead can cause problems with syntax highlighters and parser tools. It can also cause performance issues. It’s best avoided if possible. https://github.com/elves/elvish/issues/664 > This is unambiguous without needing lookahead: I think that’s only the case because typed objects can’t have elements and entries. Is this limitation here to stay? > I would more expect them to be flattened at the top level, as per the first snippet in my post. I think it’s best to avoid such assumptions when designing a feature. If the feature can be used inside objects, it will be used inside objects.

adam commented

2025-12-30 01:21:30 +01:00

@bioball commented on GitHub (Apr 12, 2024):

I think that’s only the case because typed objects can’t have elements and entries. Is this limitation here to stay?

No, it's because expressions are not permitted at the module level. And, this isn't likely to change.

@bioball commented on GitHub (Apr 12, 2024): > I think that’s only the case because typed objects can’t have elements and entries. Is this limitation here to stay? No, it's because expressions are not permitted at the module level. And, this isn't likely to change.

adam commented

2025-12-30 01:21:30 +01:00

@odenix commented on GitHub (Apr 12, 2024):

No, it's because expressions are not permitted at the module level.

But if typed objects could have elements, it would make a lot of sense to allow expressions (elements) at the module level. This would also work great for rendering formats that support top-level arrays (JSON, etc.). Same for entries.

@odenix commented on GitHub (Apr 12, 2024): > No, it's because expressions are not permitted at the module level. But if typed objects could have elements, it would make a lot of sense to allow expressions (elements) at the module level. This would also work great for rendering formats that support top-level arrays (JSON, etc.). Same for entries.

adam commented

2025-12-30 01:21:30 +01:00

@odenix commented on GitHub (Apr 12, 2024):

Or... should that ambiguity should syntactically be disallowed and should this require that properties are aggregated in notation,

This might be a good first (and possibly last) step. Good editor support could ease the strictness pain. Definitely worth prototyping.

It would be surprising if it expands to the below snippet, because in the below snippet, the bar within baz = bar references the middle bar, rather than the module-level bar.

Sounds like a simple expansion won't be good enough.

@odenix commented on GitHub (Apr 12, 2024): > Or... should that ambiguity should syntactically be disallowed and should this require that properties are aggregated in notation, This might be a good first (and possibly last) step. Good editor support could ease the strictness pain. Definitely worth prototyping. > It would be surprising if it expands to the below snippet, because in the below snippet, the bar within baz = bar references the middle bar, rather than the module-level bar. Sounds like a simple expansion won't be good enough.

adam commented

2025-12-30 01:21:30 +01:00

@bioball commented on GitHub (Apr 13, 2024):

But if typed objects could have elements, it would make a lot of sense to allow expressions (elements) at the module level. This would also work great for rendering formats that support top-level arrays (JSON, etc.). Same for entries.

Even if typed objects eventually allow entries and elements, I'm kind of bearish on this at the module level.

This starts to feel too much like a scripting language:

module foo

something()

somethingElse()

The way to render top-level arrays is to assign to output.value, e.g.

someListing: Listing<String> = new { "hello" }

output {
  value = someListing
  renderer = new JsonRenderer {}
}

Sounds like a simple expansion won't be good enough.

Yeah; we need to explore what the scoping rules should be. And, it might require that we start resolving variables at parse-time (within AstBuilder.java).

@bioball commented on GitHub (Apr 13, 2024): > But if typed objects could have elements, it would make a lot of sense to allow expressions (elements) at the module level. This would also work great for rendering formats that support top-level arrays (JSON, etc.). Same for entries. Even if typed objects eventually allow entries and elements, I'm kind of bearish on this at the module level. This starts to feel too much like a scripting language: ```groovy module foo something() somethingElse() ``` The way to render top-level arrays is to assign to `output.value`, e.g. ```groovy someListing: Listing<String> = new { "hello" } output { value = someListing renderer = new JsonRenderer {} } ``` > Sounds like a simple expansion won't be good enough. Yeah; we need to explore what the scoping rules should be. And, it might require that we start resolving variables at parse-time (within `AstBuilder.java`).

adam referenced this issue

2025-12-30 01:24:25 +01:00

[PR #143] [MERGED] Typo: Update use-cases.adoc #425

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/pkl#143