Type constraints are sometimes not enforced. #153

New Issue

2025-12-30T01:21:35+01:00

adam commented

2025-12-30 01:21:35 +01:00

Originally created by @jsundin on GitHub (Apr 28, 2024).

TypeConstrDef.pkl:

typealias RandomValue = String(this.matches(Regex("^4$")))  // secure random number generated using: https://xkcd.com/221/
 
listOfStrings: Listing<String>(!isEmpty)

TypeConstrImpl.pkl:

amends "TypeConstrDef.pkl"
 
local firstLocalStringVar = "(TODO: insert random number here)"
local secondLocalStringVar = "this-is-just-a-regular-old-string"
 
local randomValues = new Listing<RandomValue> {
  "i-love-pkl"              // this is enforced in IntelliJ, but is not enforced for "pkl eval" or through ConfigEvaluator
  firstLocalStringVar       // this is not enforced anywhere
}
 
listOfStrings {
  ...randomValues            // i'm ok with this, imo the problem is the lack of validation in previous block
  secondLocalStringVar      // this is actually fine, we can and shoule be able to add any string here
}

Expected results: the "localValues" variable contains two elements which do not validate against the RandomValue type, and should not be allowed.

Actual results: IntelliJ correctly picks up that the literal string ("i-love-pkl") is not valid, but fails to invalidate the second string reference. The "pkl eval" command (as well as the JDK library) allows for both values.

$ pkl eval TypeConstrImpl.pkl 
listOfStrings {
  "i-love-pkl"
  "(TODO: insert random number here)"
  "this-is-just-a-regular-old-string"
}

Without having any intimate knowledge about Pkl, I can only assume that this has to do with lazy evaluations and that the values in "randomValues" are actually only validated once they are rendered in "listOfStrings" (which is valid, "because string"). That seems a bit counter-intuitive, and would mean that the IntelliJ plugin is incorrect. Also, if we create a more basic TypeConstrImpl2.pkl we will get a different behaviour:

amends "TypeConstrDef.pkl"
 
local singleValue: RandomValue = "this-is-not-very-random"
 
listOfStrings {
  singleValue
  "just-another-string"
}

This results in a validation error from both IntelliJ and "pkl eval":

$ pkl eval TypeConstrImpl2.pkl 
-- Pkl Error --
Type constraint `this.matches(Regex("^4$"))` violated.
Value: "this-is-not-very-random"

1 | typealias RandomValue = String(this.matches(Regex("^4$")))  // secure random number generated using: https://xkcd.com/221/
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
at TypeConstrImpl2#singleValue (file:///tmp/issues/TypeConstrDef.pkl, line 1)

6 | singleValue
    ^^^^^^^^^^^
at TypeConstrImpl2#listOfStrings[#1] (file:///tmp/issues/TypeConstrImpl2.pkl, line 6)

5 | listOfStrings {
    ^^^^^^^^^^^^^^^
at TypeConstrImpl2#listOfStrings (file:///tmp/issues/TypeConstrImpl2.pkl, line 5)

106 | text = renderer.renderDocument(value)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at pkl.base#Module.output.text (https://github.com/apple/pkl/blob/0.25.3/stdlib/base.pkl#L106)

Notice that the error is on line 6, which could support the theory that this is related to lazy evaluation (otherwise I would expect the error to be on line 3, where the local declaration is). But if that's the case, I would not expect an error at all, as this should be equally wrong (or correct) as the Listing example.

(The example has been boiled down to bare essentials from our real configuration, the actual model is a bit more complex. But basically what we're trying to do is enforce strict formatting constraints on some parts of the configuration as an aid during configuration. The application using the list contains logic capable of dealing with these differences.)

The above has been tested using the following versions:

Pkl 0.25.3 (Linux 5.15.0-1053-aws, native)
Pkl 0.26.0-dev+3a31188 (Linux 5.4.0-177-generic, native)

Originally created by @jsundin on GitHub (Apr 28, 2024). TypeConstrDef.pkl: ``` typealias RandomValue = String(this.matches(Regex("^4$"))) // secure random number generated using: https://xkcd.com/221/ listOfStrings: Listing<String>(!isEmpty) ``` TypeConstrImpl.pkl: ``` amends "TypeConstrDef.pkl" local firstLocalStringVar = "(TODO: insert random number here)" local secondLocalStringVar = "this-is-just-a-regular-old-string" local randomValues = new Listing<RandomValue> { "i-love-pkl" // this is enforced in IntelliJ, but is not enforced for "pkl eval" or through ConfigEvaluator firstLocalStringVar // this is not enforced anywhere } listOfStrings { ...randomValues // i'm ok with this, imo the problem is the lack of validation in previous block secondLocalStringVar // this is actually fine, we can and shoule be able to add any string here } ``` Expected results: the "localValues" variable contains two elements which do not validate against the RandomValue type, and should not be allowed. Actual results: IntelliJ correctly picks up that the literal string ("i-love-pkl") is not valid, but fails to invalidate the second string reference. The "pkl eval" command (as well as the JDK library) allows for both values. ``` $ pkl eval TypeConstrImpl.pkl listOfStrings { "i-love-pkl" "(TODO: insert random number here)" "this-is-just-a-regular-old-string" } ``` Without having any intimate knowledge about Pkl, I can only assume that this has to do with lazy evaluations and that the values in "randomValues" are actually only validated once they are rendered in "listOfStrings" (which is valid, "because string"). That seems a bit counter-intuitive, and would mean that the IntelliJ plugin is incorrect. Also, if we create a more basic TypeConstrImpl2.pkl we will get a different behaviour: ``` amends "TypeConstrDef.pkl" local singleValue: RandomValue = "this-is-not-very-random" listOfStrings { singleValue "just-another-string" } ``` This results in a validation error from both IntelliJ and "pkl eval": ``` $ pkl eval TypeConstrImpl2.pkl -- Pkl Error -- Type constraint `this.matches(Regex("^4$"))` violated. Value: "this-is-not-very-random" 1 | typealias RandomValue = String(this.matches(Regex("^4$"))) // secure random number generated using: https://xkcd.com/221/ ^^^^^^^^^^^^^^^^^^^^^^^^^^ at TypeConstrImpl2#singleValue (file:///tmp/issues/TypeConstrDef.pkl, line 1) 6 | singleValue ^^^^^^^^^^^ at TypeConstrImpl2#listOfStrings[#1] (file:///tmp/issues/TypeConstrImpl2.pkl, line 6) 5 | listOfStrings { ^^^^^^^^^^^^^^^ at TypeConstrImpl2#listOfStrings (file:///tmp/issues/TypeConstrImpl2.pkl, line 5) 106 | text = renderer.renderDocument(value) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ at pkl.base#Module.output.text (https://github.com/apple/pkl/blob/0.25.3/stdlib/base.pkl#L106) ``` Notice that the error is on line 6, which could support the theory that this is related to lazy evaluation (otherwise I would expect the error to be on line 3, where the local declaration is). But if that's the case, I would not expect an error at all, as this should be equally wrong (or correct) as the Listing example. (The example has been boiled down to bare essentials from our real configuration, the actual model is a bit more complex. But basically what we're trying to do is enforce strict formatting constraints on some parts of the configuration as an aid during configuration. The application using the list contains logic capable of dealing with these differences.) The above has been tested using the following versions: - `Pkl 0.25.3 (Linux 5.15.0-1053-aws, native)` - `Pkl 0.26.0-dev+3a31188 (Linux 5.4.0-177-generic, native)`

adam added the duplicate label 2025-12-30 01:21:35 +01:00

adam closed this issue

2025-12-30 01:21:35 +01:00

adam commented

2025-12-30 01:21:35 +01:00

@odenix commented on GitHub (Apr 28, 2024):

The IntelliJ plugin uses a static analyzer and can only detect some constraint violations. To have the element type checked by the evaluator, you need to write local randomValues: Listing<RandomValue> = new { (see open issue).

@odenix commented on GitHub (Apr 28, 2024): The IntelliJ plugin uses a static analyzer and can only detect some constraint violations. To have the element type checked by the evaluator, you need to write `local randomValues: Listing<RandomValue> = new {` (see open issue).

adam commented

2025-12-30 01:21:35 +01:00

@holzensp commented on GitHub (Apr 29, 2024):

This is an instance of https://github.com/apple/pkl/issues/405

new Foo {} only derives what default value to amend from Foo; it does not check the amended result. This is one of the reasons we recommend to not foo = new Foo {}, but rather foo: Foo = new {}.

@holzensp commented on GitHub (Apr 29, 2024): This is an instance of https://github.com/apple/pkl/issues/405 `new Foo {}` only derives what default value to amend from `Foo`; it does not _check_ the amended result. This is one of the reasons we recommend to not `foo = new Foo {}`, but rather `foo: Foo = new {}`.

adam commented

2025-12-30 01:21:36 +01:00

@bioball commented on GitHub (Jun 18, 2024):

Closed as duplicate of #405

@bioball commented on GitHub (Jun 18, 2024): Closed as duplicate of #405

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/pkl#153