A value plugin feeds values into a config file at read time: a secret from gopass, Vault or AWS, the contents of a key file, or a base file to merge. You install it on a document; the file on disk keeps the reference (${vault:secret/data/app#password}) and only the expanded read sees the secret. Reach for one when a value must not live in the file itself, or lives somewhere better (a password store, a mounted secret, a shared base config).
Each plugin is its own Go module that requires only the textdata base, so you pull in only the ones you use:
go get github.com/ubgo/textdata/plugins/gopass@latest
go get github.com/ubgo/textdata/plugins/file@latest
go get github.com/ubgo/textdata/plugins/extends@latest
go get github.com/ubgo/textdata/plugins/vault@latest
go get github.com/ubgo/textdata/plugins/aws@latestYou also need a format: github.com/ubgo/textdata/dotenv or github.com/ubgo/textdata/jsonc. Every plugin works in both. The runnable programs for this section live in examples/plugins; each runs against a fake (an injected runner, a temp folder, an in-memory fs.FS or an httptest server), never a real store.
| Plugin | Reference | Ability | Guide | Example |
|---|---|---|---|---|
| gopass | ${gopass:path} |
Resolver, optional Lookuper | gopass.md | gopass-fake-runner |
| file | ${file:./key.pem} |
Resolver | file.md | file-confined |
| extends | "$extends": "base.jsonc" |
Includer | extends.md | extends-fs |
| vault | ${vault:secret/data/app#password} |
Resolver, optional Lookuper | vault.md | vault-httptest |
| aws | ${aws:sm/prod/db#password}, ${aws:ssm//app/db_url} |
Resolver, optional Lookuper | aws.md | aws-httptest |
| your own | anything | any of the above | write-a-value-plugin.md | custom-resolver |
The shared package github.com/ubgo/textdata/plugin defines what a plugin can do. A format detects each ability once, at install, by type assertion; a plugin implements only what it needs. Two of them supply values:
Lookuper |
Resolver |
|
|---|---|---|
| Answers | a plain ${NAME} the file does not define |
${scheme:ref} for the schemes it lists |
| Method | Lookup(name) (value, ok, err) |
Schemes() []string and Resolve(scheme, ref) (value, err) |
| "I don't have it" | ok=false, and the next Lookuper is asked |
an error wrapping plugin.ErrNotFound |
| Several installed | first one that answers wins | first one owning the scheme wins |
| Typical use | map DB_PASSWORD to a store path without changing the file |
spell the store in the file: ${gopass:personal/db} |
${scheme:ref} is the explicit form: the reader of the file sees where the value comes from. A Lookuper is the quiet form: the file says ${DB_PASSWORD} and the installed plugins decide. The gopass, vault and aws plugins hold both abilities over one client and one cache; their Lookuper is off until you map names (WithMapping, WithPrefix, WithLookup, WithLookupPrefix).
Two more abilities matter here. An Includer loads $extends targets (the extends plugin). ValueTransformer, CommandRunner, SaveGuard, ExpandObserver and Validator exist too; the write a value plugin guide lists them all.
Build the plugin once and pass the same instance to dotenv.WithPlugin and jsonc.WithPlugin. Here a confined file plugin reads a key next to both documents, and a reference that climbs out of the folder is refused:
readConfined · examples/plugins/file-confined/main.go:68-91
func readConfined(w io.Writer, cfg string) error {
p := fileplugin.New(cfg) // the root defaults to cfg itself
env, err := dotenv.Open(filepath.Join(cfg, envFile), dotenv.WithPlugin(p))
if err != nil {
return err
}
key, _, err := env.GetExpanded("TLS_KEY")
if err != nil {
return err
}
_, _, escape := env.GetExpanded("SHARED")
doc, err := jsonc.Open(filepath.Join(cfg, jsoncFile), jsonc.WithPlugin(p))
if err != nil {
return err
}
jkey, err := doc.GetExpanded("tls.key")
if err != nil {
return err
}
fmt.Fprintf(w, "dotenv TLS_KEY: %s\n", key)
fmt.Fprintf(w, "jsonc tls.key: %v\n", jkey)
fmt.Fprintf(w, "../shared.txt: denied=%t\n", errors.Is(escape, plugin.ErrDenied))
return nil
}Get, Render and Save still return and write the reference; only GetExpanded (dotenv) and GetExpanded / Expanded (jsonc) call the plugin. That split is structural: every value hook sits on the expansion path, which rendering never calls, so a decrypted secret can never be written back to disk.
In JSONC, jsonc.Read(path, jsonc.WithPlugin(p), jsonc.WithExpand()) expands before decoding; without WithExpand, Read returns raw values. An Includer is the exception: installing one is what makes Read merge $extends.
When several plugins are installed, install order decides who answers. The rules, all shown by the custom-resolver program:
- The file wins. A name the file defines never reaches a Lookuper, so adding a fallback cannot override a value somebody wrote down.
- Lookupers: first non-miss wins. Each is asked in install order;
ok=falsepasses to the next; an error stops expansion. - Resolvers: first owner of the scheme wins. A later plugin with the same scheme is never asked. To run two instances side by side (two Vault servers, two AWS accounts), give one another scheme with
WithScheme. - Includers: tried in install order; the first that does not answer
plugin.ErrNotFoundwins.
install · examples/plugins/custom-resolver/main.go:107-119
func install() []dotenv.Option {
primary := &Store{
name: "primary",
values: map[string]string{"prod/db": "pw-primary", "hosts/db": "db.primary"},
lookups: map[string]string{"DB_HOST": "hosts/db"},
}
fallback := &Store{
name: "fallback",
values: map[string]string{"prod/db": "pw-fallback", "hosts/db": "db.fallback", "hosts/cache": "cache.fallback"},
lookups: map[string]string{"DB_HOST": "hosts/db", "CACHE_HOST": "hosts/cache", "REGION": "hosts/db"},
}
return []dotenv.Option{dotenv.WithPlugin(primary), dotenv.WithPlugin(fallback)}
}Plugins() (dotenv) reports each installed plugin and the abilities it was detected as having, for example primary: Lookuper, Resolver.
The gopass, vault and aws plugins cache every successful read for the life of the instance: a document expanded twice, or the same secret read from a dotenv file and a JSONC file, costs one call. Two fields of one Vault secret, or two keys of one AWS JSON secret, share one request. Failures are never cached, so a transient outage is retried on the next reference.
A long-running process that wants rotated secrets calls ClearCache() (gopass) or Purge() (vault, aws), or builds a new instance. gopass can turn its cache off with WithCache(false). The file and extends plugins do not cache; they read the disk each time.
cd examples/plugins/custom-resolver && go run .installed: primary: Lookuper, Resolver
installed: fallback: Lookuper, Resolver
REGION: "eu-west-1"
ZONE: "eu-west-1-a"
DB_HOST: "db.primary"
CACHE_HOST: "cache.fallback"
DB_PASSWORD: "pw-primary"
TYPO: ""
BROKEN: error from plugin "primary", unavailable=true
jsonc: "pw-primary"
- A misspelt scheme reads as empty, not as an error.
${mme:prod/db}with no plugin owningmmeis read as an ordinary unset name, so it expands to""(theTYPOline above). That is deliberate (installing a resolver never changes how other values read) but it means a typo is silent. Guard required values with${NAME:?message}in the file, or check the expanded value. ${scheme:ref:-default}does not apply a default. Everything after the first colon is the ref, so the plugin receivesref:-default. Defaults work only on plain names.- A ref is not expanded.
${vault:secret/$ENV/app}sends the literal$ENVto the plugin, and the resolved value is not expanded again either. - Single-quoted dotenv values stay literal.
KEY='${gopass:x}'never calls the plugin, like every single-quoted dotenv value. - A store that is down is an error, never an empty value. Every plugin wraps
plugin.ErrUnavailable,plugin.ErrDeniedorplugin.ErrNotFound, and the format wraps that in a*plugin.Errornaming the plugin:dotenv: plugin "vault": resolve vault:secret/data/app#password: .... In JSONC the error also carriesfile:line:col. Branch witherrors.Is, not on the text. - A Lookuper's miss is
ok=false, a Resolver's miss is an error. A mapped name whose entry is missing falls through to the next Lookuper and finally reads as unset; the same entry spelled${gopass:...}fails the read. - Plugins never read the environment on their own. vault and aws read
VAULT_*/AWS_*only when you passFromEnv(); the formats never fall back toos.Getenv. Usedotenv.WithLookup(os.LookupEnv)orjsonc.WithLookup(os.LookupEnv)if you want that.
- Plugin guides: gopass, file, extends, vault, aws, write a value plugin
- Base interfaces: root README § plugin; dotenv's hooks: dotenv README § plugins; JSONC: jsonc README § expansion and plugins
- Examples:
examples/plugins - API: pkg.go.dev/github.com/ubgo/textdata/plugin