Support fetching secrets from cloud providers #61

New Issue

adam · 2025-12-30T01:20:15+01:00

adam commented

2025-12-30 01:20:15 +01:00

Originally created by @Clebbie on GitHub (Feb 13, 2024).

As a Developer I would like to store my application's secrets in my configuration.

I think we could add this feature with the read() interface. Supporting the cloud platforms could go a long way in making pkl the standard config language it wants to be.

i.e.
read(azvault:some.vault.url)

Originally created by @Clebbie on GitHub (Feb 13, 2024). As a Developer I would like to store my application's secrets in my configuration. I think we could add this feature with the read() interface. Supporting the cloud platforms could go a long way in making pkl the standard config language it wants to be. i.e. read(azvault:some.vault.url)

adam closed this issue

2025-12-30 01:20:16 +01:00

adam commented

2025-12-30 01:20:16 +01:00

@bioball commented on GitHub (Feb 14, 2024):

This is possible. To do this, you will need to drive Pkl execution through one of our language bindings. And, in the host language, you'll want to define a resource reader to implement reading from Vault, etc.

See:

@bioball commented on GitHub (Feb 14, 2024): This is possible. To do this, you will need to drive Pkl execution through one of our language bindings. And, in the host language, you'll want to define a resource reader to implement reading from Vault, etc. See: * https://javadocs.dev/org.pkl-lang/pkl-core/0.25.2/org/pkl/core/resource/ResourceReader.html * https://pkg.go.dev/github.com/apple/pkl-go@v0.5.3/pkl#ResourceReader * https://github.com/apple/pkl-swift/blob/main/Sources/PklSwift/Reader.swift#L89

adam commented

2025-12-30 01:20:17 +01:00

@Clebbie commented on GitHub (Feb 16, 2024):

Got it working in my go project. Ty for the documentation.

@Clebbie commented on GitHub (Feb 16, 2024): Got it working in my go project. Ty for the documentation.

adam commented

2025-12-30 01:20:17 +01:00

@khanakia commented on GitHub (Jul 27, 2025):

@Clebbie Can you share an example in Golang please how did you achieve this.

@khanakia commented on GitHub (Jul 27, 2025): @Clebbie Can you share an example in Golang please how did you achieve this.

adam commented

2025-12-30 01:20:17 +01:00

@HT154 commented on GitHub (Jul 27, 2025):

It's no longer necessary to drive evaluation via the language bindings to do this. The External Readers feature allows extending any Pkl usage (eg. the plain CLI) to allow reading or importing custom URI schemes. You can read more about external readers here: https://pkl-lang.org/main/current/language-reference/index.html. You can see a very basic example of implementing an external reader in golang here: https://pkl-lang.org/go/current/external-readers.html.

@HT154 commented on GitHub (Jul 27, 2025): It's no longer necessary to drive evaluation via the language bindings to do this. The External Readers feature allows extending any Pkl usage (eg. the plain CLI) to allow reading or importing custom URI schemes. You can read more about external readers here: https://pkl-lang.org/main/current/language-reference/index.html. You can see a very basic example of implementing an external reader in golang here: https://pkl-lang.org/go/current/external-readers.html.

adam commented

2025-12-30 01:20:17 +01:00

@khanakia commented on GitHub (Jul 27, 2025):

I checked this, but i think External Readers are to extend the pkl functionality

What I need is to be able to load Secrets from an External Secret Manager like AWS Secret Manager, and instead of defining secrets in the app.pkl file and commit with Docker Image file

I am using Docker Containers, so I need the best way to load secrets.

@khanakia commented on GitHub (Jul 27, 2025): I checked this, but i think External Readers are to extend the pkl functionality What I need is to be able to load Secrets from an External Secret Manager like AWS Secret Manager, and instead of defining secrets in the app.pkl file and commit with Docker Image file I am using Docker Containers, so I need the best way to load secrets.

adam commented

2025-12-30 01:20:18 +01:00

@Clebbie commented on GitHub (Jul 27, 2025):

Unfortunately I don’t have access to the code I wrote. I’m looking around to see if I don’t have another example somewhere… no promises though

On Sun, Jul 27, 2025 at 11:42 AM, Aman Bansal @.***(mailto:On Sun, Jul 27, 2025 at 11:42 AM, Aman Bansal < wrote:

khanakia left a comment (apple/pkl#148)

I checked this, but i think External Readers are to extend the pkl functionality

What I need is to be able to load Secrets from an External Secret Manager like AWS Secret Manager, and instead of defining secrets in the app.pkl file and commit with Docker Image file

I am using Docker Containers, so I need the best way to load secrets.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>

@Clebbie commented on GitHub (Jul 27, 2025): Unfortunately I don’t have access to the code I wrote. I’m looking around to see if I don’t have another example somewhere… no promises though On Sun, Jul 27, 2025 at 11:42 AM, Aman Bansal ***@***.***(mailto:On Sun, Jul 27, 2025 at 11:42 AM, Aman Bansal <<a href=)> wrote: > khanakia left a comment [(apple/pkl#148)](https://github.com/apple/pkl/issues/148#issuecomment-3124531608) > > I checked this, but i think External Readers are to extend the pkl functionality > > What I need is to be able to load Secrets from an External Secret Manager like AWS Secret Manager, and instead of defining secrets in the app.pkl file and commit with Docker Image file > > I am using Docker Containers, so I need the best way to load secrets. > > — > Reply to this email directly, [view it on GitHub](https://github.com/apple/pkl/issues/148#issuecomment-3124531608), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AIFS4CQ7JRVTHCE6FU7VPUL3KT6O7AVCNFSM6AAAAACCOQ476GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTCMRUGUZTCNRQHA). > You are receiving this because you were mentioned.Message ID: ***@***.***>

adam commented

2025-12-30 01:20:18 +01:00

@HT154 commented on GitHub (Jul 27, 2025):

I think I need a little more info to provide good guidance then.

Can you say more about how your app loads your config? Is your app.pkl included in your container image? Is it loaded directly by your application using pkl-go? Are you trying to configure which secrets to read in your app.pkl but defer actually reading them from the external secret manager until when your app loads the configuration?

@HT154 commented on GitHub (Jul 27, 2025): I think I need a little more info to provide good guidance then. Can you say more about how your app loads your config? Is your app.pkl included in your container image? Is it loaded directly by your application using pkl-go? Are you trying to configure which secrets to read in your app.pkl but defer actually reading them from the external secret manager until when your app loads the configuration?

adam commented

2025-12-30 01:20:18 +01:00

@khanakia commented on GitHub (Jul 27, 2025):

Currently, i commit to the github the app.pkl file along with codebase
Docker deploys the code, but it's not secure, and also secrets are hardcoded

So what i want is to store secrets to some remote service e.g AWS Secret Manager, and then do the following

Load Database from AWS Secret Manager
Override the secrets with dummy credentials defined in app.pkl

app.pkl

amends "../AppConfig.pkl"

appName = "saasbytee"

database {
  host = "host.docker.internal"
  port = 5432
  username = "postgres"
  password = "root"
  dbName = "saasbyteedb"
  sslmode= "disable"
}

Loading the file

func GetAppConfig() *appconfig.AppConfig {
	cfg, err := appconfig.LoadFromPath(context.Background(), "../config/pkl/default/app.pkl")
	if err != nil {
		panic(err)
	}

        // HERE Override the secrets somehow from Remote Secret API service

	return cfg
}

@khanakia commented on GitHub (Jul 27, 2025): - Currently, i commit to the github the app.pkl file along with codebase - Docker deploys the code, but it's not secure, and also secrets are hardcoded So what i want is to store secrets to some remote service e.g AWS Secret Manager, and then do the following - Load Database from AWS Secret Manager - Override the secrets with dummy credentials defined in app.pkl app.pkl ```pkl amends "../AppConfig.pkl" appName = "saasbytee" database { host = "host.docker.internal" port = 5432 username = "postgres" password = "root" dbName = "saasbyteedb" sslmode= "disable" } ``` Loading the file ```go func GetAppConfig() *appconfig.AppConfig { cfg, err := appconfig.LoadFromPath(context.Background(), "../config/pkl/default/app.pkl") if err != nil { panic(err) } // HERE Override the secrets somehow from Remote Secret API service return cfg } ```

adam commented

2025-12-30 01:20:19 +01:00

@HT154 commented on GitHub (Jul 27, 2025):

If you're looking to override the values loaded via Pkl in your application, that's somewhat out of scope for what this issue is originally asking about. It's up to you to decide how to interact with your secrets provider and override values in the configuration structs. You would need to consult the docs for AWS Secret Manager's golang SDK.

That said, the approach Clebbie referred to in this issue is indeed extending Pkl to read directly from a secrets provider. Usage would look like this:


amends "../AppConfig.pkl"

appName = "saasbytee"

database {
  host = "host.docker.internal"
  port = 5432
  username = "postgres"
  password = read("<uri for retrieving your secret>").text // eg. awssm:my_db_password
  dbName = "saasbyteedb"
  sslmode= "disable"
}

There are two ways to support this kind of extension:

External readers, like I mentioned above
Custom readers, like @bioball mentioned

In both cases you would need to implement the reader code in golang and configure your app's Pkl evaluator to use it. Since you're using pkl-go's evaluator API the Custom Reader approach is likely a better choice. Usage would look something like this:


func GetAppConfig() *appconfig.AppConfig {
  eval, err := pkl.NewEvaluator(context.Background(), pkl.PreconfiguredOptions, pkl.WithResourceReader(&awsSMReader{})) // where *awsSMReader implements pkl.ResourceReader
  if err != nil {
    panic(err)
  }

  var cfg appconfig.AppConfig
  err := eval.LoadModule(context.Background(), pkl.FileSource("../config/pkl/default/app.pkl"), &cfg)
  if err != nil {
    panic(err)
  }
  return &cfg
}

(I freehanded this so no promises it works as-is)

@HT154 commented on GitHub (Jul 27, 2025): If you're looking to override the values loaded via Pkl in your application, that's somewhat out of scope for what this issue is originally asking about. It's up to you to decide how to interact with your secrets provider and override values in the configuration structs. You would need to consult the docs for AWS Secret Manager's golang SDK. That said, the approach Clebbie referred to in this issue is indeed extending Pkl to read directly from a secrets provider. Usage would look like this: ```pkl amends "../AppConfig.pkl" appName = "saasbytee" database { host = "host.docker.internal" port = 5432 username = "postgres" password = read("<uri for retrieving your secret>").text // eg. awssm:my_db_password dbName = "saasbyteedb" sslmode= "disable" } ``` There are two ways to support this kind of extension: * External readers, like I mentioned above * Custom readers, like @bioball mentioned In both cases you would need to implement the reader code in golang and configure your app's Pkl evaluator to use it. Since you're using pkl-go's evaluator API the Custom Reader approach is likely a better choice. Usage would look something like this: ```go func GetAppConfig() *appconfig.AppConfig { eval, err := pkl.NewEvaluator(context.Background(), pkl.PreconfiguredOptions, pkl.WithResourceReader(&awsSMReader{})) // where *awsSMReader implements pkl.ResourceReader if err != nil { panic(err) } var cfg appconfig.AppConfig err := eval.LoadModule(context.Background(), pkl.FileSource("../config/pkl/default/app.pkl"), &cfg) if err != nil { panic(err) } return &cfg } ``` (I freehanded this so no promises it works as-is)

adam commented

2025-12-30 01:20:19 +01:00

@khanakia commented on GitHub (Jul 27, 2025):

Yes, i meant that reading directly from the secrets provider may be. I wrote it differently.

Thanks, will check

@khanakia commented on GitHub (Jul 27, 2025): Yes, i meant that reading directly from the secrets provider may be. I wrote it differently. Thanks, will check

adam referenced this issue

2025-12-30 01:24:16 +01:00

[PR #72] [CLOSED] Crash help text has mangled URL #61 #395

adam referenced this issue

2025-12-30 01:24:16 +01:00

[PR #73] [MERGED] fix mangled url #396

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/pkl#61