Writing a secret plugin
A secret plugin is a program of your own that reportd starts to read secrets from somewhere it can't read itself. A plugin named keeper answers every reference that starts secret:keeper:, wherever a secret: reference goes: a connection's password, a DSN, a sign-in provider's client secret. reportd keeps the plugin running beside it and talks to it over its standard input and output, one JSON line at a time, so you can write one in any language (choosing one).
This page is for the people who write one. Installing a plugin someone else wrote is in Installing one below, and how reportd uses plugins (the cache, rotation, licences) is in data connections.
- Write one when your secrets live in a store reportd has no plugin for, such as your password manager's API, a company secrets service, or a hardware module.
- You don't need one for secrets in environment variables or files (
secret:env:andsecret:file:), which cover Docker and Kubernetes secrets. Our plugins for HashiCorp Vault, Azure Key Vault and AWS Secrets Manager need a licence withsecret-vaults(licences). A plugin of your own declares no licence feature, and runs on any edition, Community included, unless it is of a kind reportd licenses: one namedvault,azureoraws(inconfig.yaml, its program's name or itsprovider) needssecret-vaultswhatever it declares, so give yours another name.
Two things in the public report-server-releases repository help:
- The example, a read-only plugin in Go of about 150 lines, with no dependencies beyond Go's standard library. It reads secrets from a JSON file. It is under the MIT licence: copy it and change where it reads from.
- The conformance kit, which starts your plugin as reportd does and checks it against the protocol. It is also MIT, and needs Go 1.24 or later to run, whatever your plugin is written in. Its README has the one-line command. See Testing it with the kit.
The protocol
Protocol version 1 is JSON lines on the plugin's standard input and output, in the style of git's and Docker's credential helpers.
- One message per line, in UTF-8, ended by a newline (
\r\nis accepted too). A line is at most 1 MiB, and a value at most 64 KiB. - reportd writes requests:
{"id": 7, "method": "get", "params": {"ref": "billing#password"}}. Theidis an integer. - The plugin writes answers:
{"id": 7, "result": {…}}or{"id": 7, "error": {"code": "not-found", "message": "…"}}. An answer carries its request'sid, and exactly one ofresult, an object, orerror, whosecodeis one of the codes below. - Requests may come before earlier ones are answered, and answers may go in any order. Answer each request once.
- Standard output is for answers only. Anything else there, a banner or a stray
print, breaks the protocol: reportd ends the plugin and starts it again. - Standard error is the plugin's log. reportd keeps it in its own log, each line prefixed with the plugin's name, at most 120 lines a minute and 2 KiB a line. It must never hold a value.
- Ignore fields you don't know, in params and in config, so a later reportd can add some.
| Method | reportd waits | Params | Result |
|---|---|---|---|
hello | 5 s | protocol, reportd | protocol, provider, version, feature, writable |
init | 30 s | config, dataDir | {} |
get | 10 s | ref | value, version, expiresAt |
put | 10 s | value, owner | ref, version |
replace | 10 s | ref, value | version |
delete | 10 s | ref | {} |
health | 10 s | none | state, message, expiresAt |
shutdown | 5 s | none | {}, then the plugin exits |
The examples below show what reportd sends (→) and what the plugin answers (←).
hello
The first request. reportd lists the protocol versions it speaks and its own version; the plugin chooses one and says who it is.
→ {"id": 1, "method": "hello", "params": {"protocol": [1], "reportd": "0.9.2"}}
← {"id": 1, "result": {"protocol": 1, "provider": "keeper", "version": "1.4.0", "writable": false}}protocolis the version you chose from the list. If the list has none you speak, answer the version you do speak: reportd then refuses the plugin, says which versions it speaks in its log, and doesn't start it again.provideris your plugin's own name, which needn't match the name the administrator gives it inconfig.yaml, andversionits version, which the list of providers shows.featureis a licence feature the plugin needs. Leave it out: it is for our commercial plugins. Leaving it out doesn't lift reportd's own table: aproviderofvault,azureorawsneedssecret-vaultsall the same.writableistrueonly for a plugin that keeps secrets people type in (put,replace,delete). Leave it out, orfalse, for one that only reads.
init
Sent once, after hello. config is the plugin's config from config.yaml, with each secret:env: and secret:file: reference in it already replaced by its value, so the plugin's own credentials never have to be written in the file. dataDir is the plugin's working folder, which is also its current folder: keep any state there.
→ {"id": 2, "method": "init", "params": {"config": {"address": "https://keeper.corp.example", "token": "…"}, "dataDir": "/data/plugins/keeper"}}
← {"id": 2, "result": {}}Sign in here if your store needs it, and check the config. Answer invalid-config for a config that can't work: the start fails, health says so, and reportd tries again later (see Restarts).
get
The value for a reference: ref is the part after secret:<name>:, so secret:keeper:billing#password sends billing#password. What a ref means is yours to define; it may hold :, / and #.
→ {"id": 3, "method": "get", "params": {"ref": "billing#password"}}
← {"id": 3, "result": {"value": "hunter2", "version": "3", "expiresAt": "2030-01-01T00:00:00Z"}}valueis the secret, a string of at most 64 KiB. An empty value counts as not found: answernot-foundinstead.version, optional, is the value's version in your store. reportd notes it, and a changed version or value reopens the connections that use it.expiresAt, optional, an RFC 3339 time, is when the value stops being good, such as a short-lived database credential's end. reportd keeps the value no longer than that, and asks again.- Don't keep values on disk to save calls: reportd keeps them in memory itself, for
secrets.cacheSec(300 seconds by default), and reads of one reference at the same time share one call.
put, replace and delete
Only for a plugin whose hello says writable: true: one that keeps secrets people type into a connection's form. reportd never sends them to a read-only plugin, which answers them with not-writable all the same.
→ {"id": 4, "method": "put", "params": {"value": "s3cret", "owner": {"connection": "billing", "environment": "dev", "field": "password"}}}
← {"id": 4, "result": {"ref": "9f2c41d0a1b2c3d4", "version": "1"}}
→ {"id": 5, "method": "replace", "params": {"ref": "9f2c41d0a1b2c3d4", "value": "s3cret-2"}}
← {"id": 5, "result": {"version": "2"}}
→ {"id": 6, "method": "delete", "params": {"ref": "9f2c41d0a1b2c3d4"}}
← {"id": 6, "result": {}}putkeeps a new secret and makes itsref: reportd storessecret:<name>:<ref>in the connection, never the value.ownersays which connection field it was set for.putandreplacemay also holdby, the name of the person who typed the value, for your own records. It is optional: ignore it if you keep none.replacegives a secret a new value, and must answer a newversion, so the connections using it are opened again.deleteremoves it. Agetof it afterwards answersnot-found.
health
reportd asks each running plugin every 30 seconds, and administrators see the answer in the list of secret providers.
→ {"id": 8, "method": "health", "params": {}}
← {"id": 8, "result": {"state": "degraded", "message": "token renewal failed; signing in again", "expiresAt": "2026-11-03T11:00:00Z"}}stateisok,degraded(working, but not for long, or not fully) orfailing(gets will fail).message, optional, says why, in words an administrator can act on. It is shown to them, so it must hold no value.expiresAt, optional, is when the plugin's own sign-in ends, if it has one.- A plugin that doesn't answer in 10 seconds shows as
failing. It isn't restarted for that.
shutdown
reportd is stopping. Answer, finish what you're doing, and exit.
→ {"id": 9, "method": "shutdown", "params": {}}
← {"id": 9, "result": {}}reportd then closes the plugin's input, and kills it if it is still running 5 seconds after the request. Exit also when your input closes, without a shutdown: on some systems that is the only sign that reportd has gone.
Other methods
Answer a method you don't have with unsupported. A plugin may have methods of its own, which reportd's commands send; a new protocol version may add more. They aren't part of protocol 1's required set, and the kit doesn't check them: it checks only that a method the plugin doesn't have answers unsupported.
Our store plugin has one, rotate, which reportd secrets rotate-key sends while reportd is stopped: {"key": "<the new master key>", "rekey": false} → {"keyId": "…", "previousKeyId": "…", "rekeyed": false, "secrets": 3}. reportd sends it only to the plugin the command names.
Error codes
An error's code decides what reportd does and what people see. Its message goes to reportd's log only, as a warning (or at debug level, for not-found): people never see it, so write it for the administrator reading the log, and never put a value in it.
| Code | Answer it when | What people see (a connection's test, health and errors) | A value kept from before |
|---|---|---|---|
not-found | There is no secret by that reference | the secret is not set | is dropped |
denied | Your store refused to give it (a policy, a permission) | the secret provider refused to read the secret | is dropped |
unavailable | Your store can't be reached, or is busy, for now | the secret provider can't be reached | keeps working |
invalid-ref | The reference isn't one your plugin can read | the secret provider can't read the reference | is dropped |
invalid-config | init's config can't work | the plugin doesn't start; health says why | – |
not-writable | A write to a plugin, or a secret, that is read-only | the secret provider failed | keeps working |
unsupported | A method you don't have | the secret provider failed | keeps working |
internal | Anything else | the secret provider failed | keeps working |
A code that isn't in the list counts as internal. "A value kept from before" is reportd's last value of that reference: it keeps working for up to secrets.staleSec (an hour by default) past its cache time while the plugin fails, so a short outage of your store goes unnoticed. The codes that say the secret is gone or barred drop it at once, so a revocation takes effect.
Starting, checking, restarting and stopping
At start
- reportd finds the program by file name in
secrets.pluginDirs, checks the folder's and the program's owner and permissions, and thesha256pin if there is one. - It starts the program with no arguments, in its working folder,
dataDir/plugins/<name>/(made with mode 0700, and kept across restarts), with only the environment variables the plugin'senvlists. On Windows the system addsSYSTEMROOT, which programs need to start. - The plugin is tied to reportd, so it never outlives it: on Linux it is killed when reportd ends, and on Windows a job object ends it. On other systems it must exit when its input closes.
- reportd sends
hello, checks the licence feature if the plugin names one, then sendsinit.
reportd starts the plugins before it reads its own secrets. A plugin that reportd's own config needs (say, a sign-in provider's client secret in your store) must come up, or reportd doesn't start, as with an environment variable that isn't set. state.postgres.url, the licence setting and renderer.token are read before the licence is, so only a plugin that declares no licence feature, as yours doesn't, can hold them.
While it runs
Each call has the time limit in the table above. A call that isn't answered in time fails as unavailable, and the plugin keeps running; its late answer is dropped. A plugin that stops reading its input until then is ended, and started again.
A value over 64 KiB, or a result of the wrong shape (a number where a string goes), fails only that call, as internal.
Restarts and outages
reportd ends the plugin and starts it again when it:
- exits or crashes;
- writes on standard output a line that isn't one JSON answer, an answer to an id reportd never sent or already had answered, a line over 1 MiB, or a last line without its newline.
The restarts wait 1, 2, 4… up to 60 seconds, and a run that lasted 60 seconds or more starts the wait at 1 second again. Calls in flight fail as unavailable, and health shows "restarting", with the crashes in the last 10 minutes.
A start that fails is tried again the same way, so a permission or a config file fixed while reportd runs is picked up: a missing program, an untrusted folder or program, a sha256 that doesn't match, a reference in its config that can't be read, or an init error. Two are final until reportd restarts: a plugin that needs a licence feature the licence lacks, and one that chose a protocol version reportd doesn't speak.
While a plugin is down, what fails is the connections whose values are past the stale limit, or were never read: they report secret-store until the plugin is back. Values read before keep working until their cache time plus secrets.staleSec. /readyz doesn't change: an outage fails only the connections that need the plugin.
At stop
reportd sends shutdown, closes the plugin's input, waits up to 5 seconds, then kills it.
Installing one
A plugin is installed by the administrator, in config.yaml: the studio can't add or change one, and Settings don't change secrets.*.
secrets:
providers: [env, file]
dir: /run/secrets
pluginDirs: [/plugins]
plugins:
- name: keeper # answers secret:keeper:…
program: reportd-secret-keeper # a file name in one of pluginDirs, never a path
sha256: 3b1f… # optional: the exact program
env: [KEEPER_CA_FILE] # the only environment variables it sees
config: # sent in init
address: https://keeper.corp.example
token: secret:file:keeper-token # read by reportd firstpluginDirsare absolute paths to the folders plugins are started from. Neither the folder nor the program may be writable by others, or owned by anyone but reportd's user or root (on Windows, reportd's user, SYSTEM, the Administrators group or TrustedInstaller). A program that is a symbolic link is refused.nameis the reference prefix: lower-case letters, digits and-, starting with a letter, at most 32 characters.envandfileare reportd's own.programis a plain file name. On Windows, a name without an extension also finds<name>.exe.sha256, optional, pins the program: reportd refuses to start any other. Set it when you install, and change it when you upgrade.envnames the environment variables the plugin sees, from reportd's own; aREPORTD_variable is never passed, so a plugin can't read reportd's settings.configis any YAML the plugin understands. The secrets it holds can be onlysecret:env:andsecret:file:references, not another plugin's.secrets.cacheSec(300) andsecrets.staleSec(3600) are the cache and stale times above.
In the container image, the plugin folders are /app/plugins, the image's own: it holds our store plugin, reportd-secret-store (data connections), our HashiCorp Vault plugin, reportd-secret-vault (HashiCorp Vault), our Azure Key Vault plugin, reportd-secret-azure (Azure Key Vault), and our AWS Secrets Manager plugin, reportd-secret-aws (AWS Secrets Manager), each of which runs only once config.yaml names it. For another plugin, mount a folder holding it, read-only, and add it to the plugin folders. The image sets them with a variable, which wins over config.yaml, so set the variable: REPORTD_SECRETS_PLUGINDIRS=/app/plugins,/plugins. reportd runs as user 65532, so the files must belong to root (or that user) and not be writable by others. The image's program is built for Linux, so the plugin must be too, for the image's platform (amd64 or arm64), and must run without anything the image lacks: a program with no dependencies on the system's libraries, such as a Go program built with CGO_ENABLED=0, is simplest.
services:
reportd:
environment:
REPORTD_SECRETS_PLUGINDIRS: /app/plugins,/plugins
volumes:
- ./plugins:/plugins:ro
- ./config.yaml:/config/config.yaml:roOutside the image, each release also has an archive per plugin and platform, such as reportd-secret-store-0.9.2-linux-amd64.tar.gz, listed in its SHA256SUMS, holding the program, the terms it is used under (LICENSE), and its third-party notices (THIRD_PARTY_NOTICES.md, with their licence files under licences/ when it has any). Unpack it into a folder that only root can write to, and name that folder in pluginDirs.
Then:
reportd config checkstarts the plugins, checks them against the licence and prints each one's state.GET /v1/secrets/providerslists every provider, built in or a plugin, with its state.- A studio connection may use the plugin's references only once an administrator adds a pattern for them, such as
secret:keeper:reporting/*, todata.connections.secrets(data connections).
What a plugin must do
- Never write a value to standard error, a file, a message or a health message, or anywhere on standard output but in a
getanswer. Values cross only the pipe; reportd keeps them in memory alone. - Keep your own credentials narrow. Sign in to your store with an account that can read only what reportd needs, and only read. Take the credential from
config, through asecret:file:orsecret:env:reference, never from a command line or a file of your own. - Answer
healthtruthfully.degradedbefore your sign-in runs out is worth more thanokuntil it does. - Be safe to restart at any time. reportd may kill the plugin mid-request. Keep state in
dataDir, and write it atomically (a temporary file, then a rename). - Check where you connect. A plugin's own network calls don't go through reportd's address guard: talk only to the addresses your config names, and verify TLS.
- Exit when your input closes, and open no port of your own.
Choosing a language
Any language that can read and write lines on standard input and output will do. Three things trip plugins up in most of them:
- Flush standard output after each answer. Most languages buffer output written to a pipe, so an unflushed answer waits in the buffer until reportd's time limit runs out: Python's
print, C'sstdioand Java'sSystem.outall do. - Write each answer whole. If you answer requests at the same time, from threads or tasks, write each line under one lock, so two answers never interleave.
- Read lines up to 1 MiB, past the default limit of some line readers.
Where reportd runs decides what the program may be:
In the container image, the plugin must be a native program for Linux. The image holds Debian 13's C library (glibc 2.41; before 0.10.0, Debian 12's 2.36), the C and C++ runtime libraries and CA certificates, and no shell or interpreter. These all work:
Language How Go Built with CGO_ENABLED=0, so it needs no library at all; the example is built this wayRust Built for the x86_64-unknown-linux-musl(oraarch64-…) target, which is static, or for glibc on Debian 13 or olderC, C++, Zig Linked statically, or against glibc 2.41 or older C#, F# .NET's Native AOT, for linux-x64orlinux-arm64, withInvariantGlobalizationon, since the image has no ICU libraryJava, Kotlin GraalVM's native-image; a program that needs a JVM won't startPython Bundled with its interpreter into one program, by PyInstaller or Nuitka JavaScript, TypeScript One program with its runtime inside, by Node's single executable applications, bun build --compileordeno compileA program linked against a glibc newer than 2.41 fails to start, with an error about
GLIBC_2.xxin reportd's log: build it on Debian 13 or older, or link it statically. To run in a 0.9 image too, build it on Debian 12 or older. Whatever the language, try the program in the image withreportd config checkbefore you ship it.Installed on a server of your own, a script works on Linux and macOS: make it executable, owned as
pluginDirsrequires, and start it with a#!line naming the interpreter by its absolute path, such as#!/usr/bin/python3, since the plugin gets noPATHunless itsenvlists it. The trust check and thesha256pin cover the script but not its interpreter, so a single program is the safer choice. On Windows, make the plugin an.exe.
The kit tests a plugin in any of these; only the kit itself needs Go.
Testing it with the kit
The kit starts your program as reportd does, with no arguments and only the environment you pass, and runs these checks:
- hello: it chooses protocol 1 when offered
[1]and names itself and its version; offered only a version that doesn't exist, it doesn't choose that. - init: it takes your config; given
--bad-config, it refuses that one withinvalid-config. - get: a hit (
--hit) answers a value, a miss answersnot-found, and a bad referenceinvalid-ref. - Pipelining: eight gets in flight at once each get their own answer, in any order.
- Other methods: an unknown method answers
unsupported;healthanswers a state. - Writes: for a writable plugin,
put,get,replace,get,deleteandgetagain, using therefyourputanswers and checking thatreplacemakes a new version. A read-only plugin must answer each write withnot-writable. - Ending: after
shutdownit answers and exits within 5 seconds; started again, it exits when its input closes. - Its output: nothing but answers on standard output, by the rules above, and no value the test read or wrote on standard error.
| Flag | What it is |
|---|---|
--config <file> | A JSON file holding the config sent in init, as reportd would send it, with the references already resolved |
--bad-config <file> | A config init must refuse |
--hit <ref> | A reference your plugin holds |
--miss <ref> | One it doesn't (a random one by default). When your references have a fixed form, such as an id of 16 hexadecimal digits, give one of that form that you don't hold: the pipelining check then sends it as it is, since anything else is a bad reference to your plugin |
--bad-ref <ref> | One it can't read (by default one holding a NUL character, which no reference may) |
--env <NAME> | An environment variable to pass through, as env does; repeat it for more |
--replay <script> | Replay one of the shared scripts instead; see below |
-v | Print each line sent and answered |
The kit exits 0 when every check passes, 1 when one fails, and 2 when it was run wrongly, so it can run in your CI. Point it at a test instance of your store, with a secret set aside for --hit. The values the kit writes start kit-value-.
--replay holds a plugin to the same scripted exchanges reportd's own tests use (exchanges.json, in the kit): store, vault and bad-config. The plugin must hold the scripts' data, such as billing#password with the value hunter2 at version 3, so it suits a test double more than a real store; the example's secrets.json holds it all. What is the plugin's own (its name, the refs and versions it makes, its health) is checked for shape only.
The kit is versioned on its own, as plugins/kit/v1.x.y: run @v1 for the latest kit of protocol version 1.
The example
The example is a whole plugin in one file. Its config names a JSON file of secrets:
{
"billing#password": {"value": "hunter2", "version": "3"},
"token": {"value": "t0k3n", "expiresAt": "2030-01-01T00:00:00Z"},
"forbidden": {"error": "denied"}
}An entry with error answers that code instead of a value, so you can see what reportd does with each one. It reads the file on each get, answers health with failing when it can't, and is read-only. To make it yours, change its lookup to read from your store, its init to sign in, and its name and version in hello. Its README says how to build it for the container image.