How to plug secret stores, decryption, sandboxed commands, save-time checks and auditing into the dotenv library without the library knowing about any of them, and how to write your own plugin type. You want a plugin when a value should read differently from how it is written (a password fetched from a vault, a ciphertext decrypted on read, an environment fallback), when a write must be refused (a plaintext secret, a missing key), or when you want to watch the file (an audit of disabled settings, a list of unused variables).
A plugin is any value with a Name() string method, installed with dotenv.WithPlugin. What it does depends on which of seven small capability interfaces it also implements. A plugin implements only the ones it needs, and one plugin may implement several, so a secret store can share one client and one cache between its lookup and its guard. For a one-function hook you do not need a type at all: WithLookup, WithValueTransform, WithSaveGuard and WithCommandRunner wrap a plain function.
go get github.com/ubgo/textdata/dotenv@latestSnippets come from two runnable examples: examples/dotenv/plugin-hooks (function options, a Resolver, observers, ordering, the firewall) and examples/dotenv/write-a-plugin (a plugin type with a Lookuper and a SaveGuard). Every store in them is an in-memory map. Ready-made value plugins (gopass, file, extends, vault, aws) live in their own modules under plugins/.
The seven hooks:
| Capability | Method | Runs | Several installed | Can fail | Function option (name it reports) |
|---|---|---|---|---|---|
EntryObserver |
ObserveEntry(*Entry) |
once per entry while parsing, in file order | all run, install order | no | none |
Lookuper |
Lookup(name) (string, bool, error) |
for a ${VAR} the file does not define |
first to answer wins | yes, aborts the read | WithLookup (lookup) |
Resolver |
Schemes() []string, Resolve(scheme, ref) (string, error) |
for a ${scheme:ref} whose scheme it owns |
first owner of the scheme wins | yes, aborts the read | none |
CommandRunner |
RunCommand(cmd) (string, error) |
for a $(cmd) in a value that interpolates |
last installed wins (a replacement) | yes, aborts the read with an *expand.CommandError naming the command |
WithCommandRunner (command-runner) |
ExpandObserver |
ObserveExpand(key, name, resolved) |
after every reference resolution | all run, install order | no | none |
ValueTransformer |
TransformValue(key, value) (string, error) |
after a value is expanded, on literal values too | chained, each sees the last output | yes, aborts the read | WithValueTransform (value-transform) |
SaveGuard |
GuardSave(*File) error |
before Save or SaveAs writes anything |
run in order; first error aborts | yes, the write is refused | WithSaveGuard (save-guard) |
Lookuper, Resolver, CommandRunner, ValueTransformer and ExpandObserver are aliases of the shared github.com/ubgo/textdata/plugin interfaces, so a plugin written once against plugin works in every textdata format. EntryObserver and SaveGuard are dotenv's own because they see dotenv types; a cross-format guard written against plugin.SaveGuard (GuardSave(plugin.Document) error) is accepted too (Step 3).
Expansion runs in GetExpanded, ExpandedMap and Read. Get, Map, Pairs, Render and Save never expand, so lookups, resolvers, runners, transformers and expand observers never touch them.
WithLookup takes a func(name string) (string, bool) and calls it for a ${VAR} the file does not define. The file always wins: a fallback cannot override a value somebody wrote down (ZONE in the map below is ignored). A ${VAR:-default} still falls back to its default when the lookup misses. In an application, pass os.LookupEnv to get Compose's environment fallback.
WithValueTransform takes a func(key, value string) (string, error) that rewrites each value after it is expanded; return the value unchanged when it is not yours. It also runs on single-quoted and backtick values: quoting controls interpolation, not transformation, and an encrypted value is often single-quoted to keep $ in the ciphertext literal. Its output is not expanded again, and an error aborts the read, so you never get ciphertext back where you expected a secret. Get and Render still return the ciphertext, so that is what gets saved.
WithSaveGuard takes a func(*File) error that runs before Save or SaveAs writes anything. Returning an error refuses the write: no temp file is created and the destination keeps its old content. The error wraps yours (errors.Is reaches it) and names the plugin and the path. WithCommandRunner is covered in expansion.md › Step 7.
functionOptions · examples/dotenv/plugin-hooks/main.go:72-103
func functionOptions(w io.Writer) error {
env := map[string]string{"REGION": "eu-west-1", "ZONE": "ignored"}
keys := map[string]string{"c2VjcmV0": "hunter2"}
f := dotenv.Parse("ZONE=${REGION}-a\nPASS='encrypted:c2VjcmV0'\n",
dotenv.WithLookup(func(name string) (string, bool) {
v, ok := env[name]
return v, ok
}),
dotenv.WithValueTransform(func(key, v string) (string, error) {
id, ok := strings.CutPrefix(v, cipherPrefix)
if !ok {
return v, nil
}
if plain, found := keys[id]; found {
return plain, nil
}
return "", fmt.Errorf("%s: no key for %q", key, id)
}),
dotenv.WithSaveGuard(func(*dotenv.File) error { return errors.New("read-only") }),
)
zone, _, err := f.GetExpanded("ZONE")
if err != nil {
return err
}
// PASS's error is printed, not returned: a nil error here is the point.
pass, _, passErr := f.GetExpanded("PASS")
raw, _ := f.Get("PASS")
fmt.Fprintln(w, "ZONE:", zone, "PASS:", pass, "Get(PASS):", raw, "err:", passErr)
f.Path = "never-written.env" // the guard refuses before any write
fmt.Fprintln(w, "save:", f.Save())
return nil
}A Resolver lists the schemes it owns and answers Resolve(scheme, ref). PASS=${mem:db/password} then reads as the stored value through GetExpanded, while Get, Render and Save keep the reference. Schemes match case-insensitively. A scheme no installed Resolver owns reads exactly as before (an ordinary unset name, so ${vault:x} is ""), which means installing one never changes how other values read. A single-quoted '${mem:x}' stays literal, and a Resolver that fails aborts with an error naming the plugin, never an empty value. Return plugin.ErrNotFound for a missing ref, so callers can match it with errors.Is.
The example's memStore is a map[string]string with Name() returning mem-store and Schemes() returning []string{"mem"}; its Resolve:
memStore.Resolve · examples/dotenv/plugin-hooks/main.go:113-118
func (m memStore) Resolve(scheme, ref string) (string, error) {
if v, ok := m[ref]; ok {
return v, nil
}
return "", fmt.Errorf("%w: %s:%s", plugin.ErrNotFound, scheme, ref)
}resolver · examples/dotenv/plugin-hooks/main.go:126-140
func resolver(w io.Writer) error {
store := memStore{"db/password": "pa55"}
src := "PASS=${mem:db/password}\nOTHER=[${vault:x}]\nLIT='${mem:db/password}'\nBAD=${mem:nope}\n"
f := dotenv.Parse(src, dotenv.WithPlugin(store))
for _, key := range []string{"PASS", "OTHER", "LIT"} {
v, _, err := f.GetExpanded(key)
if err != nil {
return err
}
fmt.Fprintf(w, "%s=%s\n", key, v)
}
_, _, err := f.GetExpanded("BAD")
fmt.Fprintln(w, err)
return nil
}When hooks share state, write a type. This vault implements Lookuper (fill ${DB_PASSWORD}) and SaveGuard (refuse a file that holds one of its secrets in plain text) over one store. Capabilities are detected once, at install time, by type assertion, so a plugin without a capability costs nothing on the expansion path.
vault · examples/dotenv/write-a-plugin/main.go:40-43
type vault struct {
secrets map[string]string
down bool
}Capabilities are optional interfaces, so a method with a wrong name or signature (LookUp instead of Lookup, or a Lookup without the error return) compiles, installs, and silently never runs. One assertion per capability turns that silence into a build error. Use (*T)(nil) for pointer receivers and T{} or T(nil) for value receivers.
_ · examples/dotenv/write-a-plugin/main.go:49-53
var (
_ dotenv.Plugin = (*vault)(nil)
_ dotenv.Lookuper = (*vault)(nil)
_ dotenv.SaveGuard = (*vault)(nil)
)A Lookuper returns ok=false for "I do not have this", and the next one is asked. It returns an error for "I could not check". An error aborts expansion, even when the reference has a default: an unreachable store must not look like an unset variable, or a deploy proceeds with an empty password. Wrap one of the base's named errors (plugin.ErrUnavailable, ErrDenied, ErrNotFound, ErrUnsupportedScheme) so callers can tell outages apart.
vault.Lookup · examples/dotenv/write-a-plugin/main.go:63-69
func (v *vault) Lookup(name string) (string, bool, error) {
if v.down {
return "", false, fmt.Errorf("%w: connection refused", plugin.ErrUnavailable)
}
s, ok := v.secrets[name]
return s, ok, nil
}A SaveGuard may only veto. The *File it receives is a copy made for that call, so it can read anything but cannot change what is written. Read raw values with Get: a key set to an unexpanded ${REF} is what will be saved.
vault.GuardSave · examples/dotenv/write-a-plugin/main.go:75-85
func (v *vault) GuardSave(f *dotenv.File) error {
for _, key := range f.Keys() {
raw, _ := f.Get(key)
for name, secret := range v.secrets {
if strings.Contains(raw, secret) {
return fmt.Errorf("%w: %s holds %s", errLeak, key, name)
}
}
}
return nil
}A guard written against the shared plugin.SaveGuard works in every format. *File implements plugin.Document: FormatName is "dotenv", Location the path, ValuePaths the active keys, ValueAt the raw value, Bytes exactly what Save would write. Both kinds of guard run in one install-ordered list. The example's requireKeys is a []string with Name() returning require-keys:
requireKeys.GuardSave · examples/dotenv/write-a-plugin/main.go:96-103
func (r requireKeys) GuardSave(doc plugin.Document) error {
for _, key := range r {
if _, ok := doc.ValueAt(key); !ok {
return fmt.Errorf("%s: %s is required", doc.FormatName(), key)
}
}
return nil
}Install both with one WithPlugin call. f.Plugins() returns one PluginInfo per installed plugin, in install order, with the capabilities it was detected as having; a capability missing from the list means the method does not match the interface. The Capability constants are CapLookup, CapResolver, CapValueTransformer, CapCommandRunner, CapSaveGuard, CapEntryObserver and CapExpandObserver, listed in that fixed order; PluginInfo.String renders name: Cap, Cap, or name: (none).
install · examples/dotenv/write-a-plugin/main.go:149-165
func install(w io.Writer, path string, v *vault) (*dotenv.File, error) {
f, err := dotenv.Open(path, dotenv.WithPlugin(v, requireKeys{"DB_URL"}))
if err != nil {
return nil, err
}
for _, info := range f.Plugins() {
fmt.Fprintln(w, "plugin", info)
}
url, _, err := f.GetExpanded("DB_URL")
if err != nil {
return nil, err
}
raw, _ := f.Get("DB_URL")
fmt.Fprintln(w, "GetExpanded:", url)
fmt.Fprintln(w, "Get: ", raw)
return f, nil
}A refused save writes nothing. The full error reads dotenv: plugin "<name>": guard save <path>: <your error>:
guard · examples/dotenv/write-a-plugin/main.go:172-197
func guard(w io.Writer, f *dotenv.File) error {
before, err := os.ReadFile(f.Path)
if err != nil {
return err
}
f.Set("DB_PASSWORD", "hush") // someone pastes the secret in
err = f.Save()
fmt.Fprintln(w, "leak refused:", errors.Is(err, errLeak))
fmt.Fprintln(w, strings.Replace(err.Error(), f.Path, envFile, 1))
f.Unset("DB_PASSWORD", true)
f.Unset("DB_URL", true)
err = f.Save()
fmt.Fprintln(w, strings.Replace(err.Error(), f.Path, envFile, 1))
after, err := os.ReadFile(f.Path)
if err != nil {
return err
}
entries, err := os.ReadDir(filepath.Dir(f.Path))
if err != nil {
return err
}
fmt.Fprintln(w, "file unchanged:", string(before) == string(after), "files in dir:", len(entries))
return nil
}When the store is down, the read fails even though ${DB_PASSWORD:-} has a default:
storeDown · examples/dotenv/write-a-plugin/main.go:203-209
func storeDown(w io.Writer, v *vault) {
v.down = true
f := dotenv.Parse("PASS=${DB_PASSWORD:-}\n", dotenv.WithPlugin(v))
_, _, err := f.GetExpanded("PASS")
fmt.Fprintln(w, err)
fmt.Fprintln(w, "unavailable:", errors.Is(err, plugin.ErrUnavailable))
}An EntryObserver sees every entry as it is parsed: comments, blank lines, pairs, commented-out pairs (KindDisabledPair), name-only declarations (KindInherited) and unrecognised lines (KindOther). A commented-out setting is a finding for an auditor, so nothing is filtered. A multi-line value is one entry. Each call gets a copy of the entry, so the observer cannot change the parsed file. It fires from Parse, ParseReader and Open, not from Set, Append or other edits, and it cannot fail; to reject a file, use a SaveGuard.
An ExpandObserver is told about every reference resolution: key is the entry being expanded, name the variable it referenced, resolved the result. Branches of ${VAR:-default} and ${VAR:+alt} are reported too; literal values report nothing. It fires only during expansion, so call ExpandedMap first. Recording the names gives dead-key detection.
The example's auditor has both: its ObserveExpand records used[name] = true, and its ObserveEntry collects disabled keys.
auditor.ObserveEntry · examples/dotenv/plugin-hooks/main.go:157-161
func (a *auditor) ObserveEntry(e *dotenv.Entry) {
if e.Kind == dotenv.KindDisabledPair {
a.disabled = append(a.disabled, e.Key)
}
}observers · examples/dotenv/plugin-hooks/main.go:170-185
func observers(w io.Writer) error {
a := &auditor{used: map[string]bool{}}
f := dotenv.Parse("HOST=db\nPORT=5432\nURL=pg://${HOST}\n# DEBUG=true\n", dotenv.WithPlugin(a))
if _, err := f.ExpandedMap(); err != nil {
return err
}
var unused []string
for _, key := range f.Keys() {
if !a.used[key] {
unused = append(unused, key)
}
}
sort.Strings(unused)
fmt.Fprintln(w, "disabled:", a.disabled, "never referenced:", unused)
return nil
}Each capability combines in its own way:
- Lookuper: the file's own keys first, then each
Lookuperin install order. The first that returnsok=truewins and later ones are not asked. A lookup error stops the chain. - Resolver: the first installed Resolver that lists the scheme answers.
- ValueTransformer: chained in install order. Each sees the previous one's output, so one plugin can dereference a value and another decode it.
- CommandRunner: a replacement, not a chain. The last runner installed is the only one used.
- SaveGuard: run in install order. The first error refuses the write and later guards do not run.
- EntryObserver and ExpandObserver: all run, in install order.
Install order is the order of the options, and of the plugins within one WithPlugin(a, b) call. The function options count as plugins in that order.
ordering · examples/dotenv/plugin-hooks/main.go:192-213
func ordering(w io.Writer) error {
look := func(answer string) func(string) (string, bool) {
return func(string) (string, bool) { return answer, answer != "" }
}
wrap := func(tag string) func(string, string) (string, error) {
return func(_, v string) (string, error) { return tag + "(" + v + ")", nil }
}
runner := func(tag string) func(string) (string, error) {
return func(string) (string, error) { return tag, nil }
}
f := dotenv.Parse("A=${X}\nB=$(cmd)\n",
dotenv.WithLookup(look("")), dotenv.WithLookup(look("second")), dotenv.WithLookup(look("third")),
dotenv.WithValueTransform(wrap("t1")), dotenv.WithValueTransform(wrap("t2")),
dotenv.WithCommandRunner(runner("first-runner")), dotenv.WithCommandRunner(runner("last-runner")),
)
m, err := f.ExpandedMap()
if err != nil {
return err
}
fmt.Fprintln(w, "A:", m["A"], "B:", m["B"])
return nil
}A hook may change what a value reads as. It may never change what gets written.
Lookups, resolvers, runners, transformers and expand observers all live in the expansion path, which Render and Save never call, and they are handed only strings. So a file round-trips byte for byte whatever is installed, and a decrypted secret cannot be written back in plaintext. The two hooks that receive structured state get copies: a SaveGuard is handed a clone of the file made for that call, and an EntryObserver a copy of each entry. A guard that calls Set, or an observer that rewrites e.Raw, changes only its copy, during the call or later through a kept pointer.
The example's meddler tries all three: its transformer upper-cases every value, its guard calls f.Set("A", "changed"), and its observer overwrites e.Value and e.Raw.
firewall · examples/dotenv/plugin-hooks/main.go:229-246
func firewall(w io.Writer, dir string) error {
const src = "A=secret\n# B=off\n"
f := dotenv.Parse(src, dotenv.WithPlugin(meddler{}))
read, _, err := f.GetExpanded("A")
if err != nil {
return err
}
f.Path = filepath.Join(dir, ".env")
if err := f.Save(); err != nil {
return err
}
saved, err := os.ReadFile(f.Path)
if err != nil {
return err
}
fmt.Fprintln(w, "reads as:", read, "render unchanged:", f.Render() == src, "saved unchanged:", string(saved) == src)
return nil
}The rule is also asserted by TestPlugins_NeverAffectBytes, which renders and edits one file with every value-changing hook and with none, and by the fuzz target FuzzPluginsNeverAffectBytes, which does the same for arbitrary input.
cd examples/dotenv/plugin-hooks && go run .== function options
ZONE: eu-west-1-a PASS: hunter2 Get(PASS): encrypted:c2VjcmV0 err: <nil>
save: dotenv: plugin "save-guard": guard save never-written.env: read-only
== Resolver
PASS=pa55
OTHER=[]
LIT=${mem:db/password}
dotenv: plugin "mem-store": resolve mem:nope: not found: mem:nope
== observers
disabled: [DEBUG] never referenced: [PORT URL]
== ordering
A: t2(t1(second)) B: t2(t1(last-runner))
== firewall
reads as: SECRET render unchanged: true saved unchanged: true
cd examples/dotenv/write-a-plugin && go run .== install and read
plugin vault: Lookuper, SaveGuard
plugin require-keys: SaveGuard
GetExpanded: postgres://app@db/app?password=hush
Get: postgres://${DB_USER}@db/app?password=${DB_PASSWORD}
== guard
leak refused: true
dotenv: plugin "vault": guard save .env: plain-text secret in file: DB_PASSWORD holds DB_PASSWORD
dotenv: plugin "require-keys": guard save .env: dotenv: DB_URL is required
file unchanged: true files in dir: 1
== store down
dotenv: plugin "vault": lookup DB_PASSWORD: plugin unavailable: connection refused
unavailable: true
More tested examples (ExampleWithPlugin, ExampleWithPlugin_lookupError, ExampleWithPlugin_guard, ExampleCommandRunner, ExampleEntryObserver, ExampleExpandObserver, ExampleFile_Plugins_ordering, ExampleFile_Plugins_firewall, ExampleCapability, ExamplePluginInfo) are in dotenv/example_plugin_test.go.
- A lookup error beats a default.
${DB_PASSWORD:-}still fails when theLookuperreturns an error. Returnok=false, not an error, for "not found". - A self-referencing default asks the
Lookuperonly when no earlier line defines the key. As in Docker Compose,ZONE=${ZONE:-default-zone}sees the definitions above it first, so an earlierZONE=line beats theLookuper(expansion.md › Step 4). - A runner error fails the read. A
CommandRunnerthat returns an error makesGetExpandedandExpandedMapreturn an*expand.CommandErrornaming the command and unwrapping to your error. To let a command fail harmlessly, return a fallback value with a nil error. - The function options name every hook the same. Two
WithLookupcalls both report aslookup, so an error does not tell you which one failed. Write a type with its ownName()when that matters. - Editing what a guard or observer is handed does nothing.
GuardSavereceives a copy of the file andObserveEntrya copy of the entry, so aSetor a changede.Valuethere is silently lost. To change a value before it is written, callSeton the file yourself beforeSave. - A typed-nil plugin panics.
WithPlugin((*MyPlugin)(nil))satisfies the interface and panics on first use. A plainnilis ignored, and so is anilfunction passed toWithLookup,WithValueTransformorWithSaveGuard.WithCommandRunner(nil)installs nothing but still turns substitution on with the defaultsh -crunner. - A plugin dotenv cannot use is refused. A plugin whose only abilities are ones dotenv never acts on (the
extendsIncluder, a bareValidator) is not installed:Open,ParseReaderandReadreturndotenv: plugin "<name>": no ability dotenv supports (<abilities>)(errors.Is(err, dotenv.ErrUnsupportedPlugin)), and on a file fromParsethe error comes fromGetExpanded,ExpandedMap,SaveandSaveAs. A plugin with no recognised ability at all still installs and shows asname: (none). - A wrong method signature is silent. Add the compile-time assertions from Step 3 to every plugin type, and check
f.Plugins()when a hook seems to do nothing. - Plugins do not reach the process environment by themselves. The library never reads it. Pass
WithLookup(os.LookupEnv)when you want that. Clonekeeps the plugins. A clone of a file with aSaveGuardis guarded too.
- expansion.md: the
${…}operators and$(…)substitution that lookups, runners and observers hook into. - entries.md: the entry kinds an
EntryObserversees. - saving.md:
SaveandSaveAs, where guards run. - typed-config.md: turning the expanded strings into a typed struct.
- Module README: Plugins, The seven hooks, The firewall, Writing a plugin.
- Shared interfaces: root README › plugin.
- Examples:
examples/dotenv/plugin-hooks,examples/dotenv/write-a-plugin. - API: pkg.go.dev/github.com/ubgo/textdata/dotenv#Plugin.