Skip to content

Writing a secret plugin ​

A secret plugin is a program of your own that reportd starts to read secrets from somewhere it can't read itself. A plugin named keeper answers every reference that starts secret:keeper:, wherever a secret: reference goes: a connection's password, a DSN, a sign-in provider's client secret. reportd keeps the plugin running beside it and talks to it over its standard input and output, one JSON line at a time, so you can write one in any language (choosing one).

This page is for the people who write one. Installing a plugin someone else wrote is in Installing one below, and how reportd uses plugins (the cache, rotation, licences) is in data connections.

  • Write one when your secrets live in a store reportd has no plugin for, such as your password manager's API, a company secrets service, or a hardware module.
  • You don't need one for secrets in environment variables or files (secret:env: and secret:file:), which cover Docker and Kubernetes secrets. Our plugins for HashiCorp Vault, Azure Key Vault and AWS Secrets Manager need a licence with secret-vaults (licences). A plugin of your own declares no licence feature, and runs on any edition, Community included, unless it is of a kind reportd licenses: one named vault, azure or aws (in config.yaml, its program's name or its provider) needs secret-vaults whatever it declares, so give yours another name.

Two things in the public report-server-releases repository help:

  • The example, a read-only plugin in Go of about 150 lines, with no dependencies beyond Go's standard library. It reads secrets from a JSON file. It is under the MIT licence: copy it and change where it reads from.
  • The conformance kit, which starts your plugin as reportd does and checks it against the protocol. It is also MIT, and needs Go 1.24 or later to run, whatever your plugin is written in. Its README has the one-line command. See Testing it with the kit.

The protocol ​

Protocol version 1 is JSON lines on the plugin's standard input and output, in the style of git's and Docker's credential helpers.

  • One message per line, in UTF-8, ended by a newline (\r\n is accepted too). A line is at most 1 MiB, and a value at most 64 KiB.
  • reportd writes requests: {"id": 7, "method": "get", "params": {"ref": "billing#password"}}. The id is an integer.
  • The plugin writes answers: {"id": 7, "result": {…}} or {"id": 7, "error": {"code": "not-found", "message": "…"}}. An answer carries its request's id, and exactly one of result, an object, or error, whose code is one of the codes below.
  • Requests may come before earlier ones are answered, and answers may go in any order. Answer each request once.
  • Standard output is for answers only. Anything else there, a banner or a stray print, breaks the protocol: reportd ends the plugin and starts it again.
  • Standard error is the plugin's log. reportd keeps it in its own log, each line prefixed with the plugin's name, at most 120 lines a minute and 2 KiB a line. It must never hold a value.
  • Ignore fields you don't know, in params and in config, so a later reportd can add some.
Methodreportd waitsParamsResult
hello5 sprotocol, reportdprotocol, provider, version, feature, writable
init30 sconfig, dataDir{}
get10 srefvalue, version, expiresAt
put10 svalue, ownerref, version
replace10 sref, valueversion
delete10 sref{}
health10 snonestate, message, expiresAt
shutdown5 snone{}, then the plugin exits

The examples below show what reportd sends (→) and what the plugin answers (←).

hello ​

The first request. reportd lists the protocol versions it speaks and its own version; the plugin chooses one and says who it is.

json
→ {"id": 1, "method": "hello", "params": {"protocol": [1], "reportd": "0.9.2"}}
← {"id": 1, "result": {"protocol": 1, "provider": "keeper", "version": "1.4.0", "writable": false}}
  • protocol is the version you chose from the list. If the list has none you speak, answer the version you do speak: reportd then refuses the plugin, says which versions it speaks in its log, and doesn't start it again.
  • provider is your plugin's own name, which needn't match the name the administrator gives it in config.yaml, and version its version, which the list of providers shows.
  • feature is a licence feature the plugin needs. Leave it out: it is for our commercial plugins. Leaving it out doesn't lift reportd's own table: a provider of vault, azure or aws needs secret-vaults all the same.
  • writable is true only for a plugin that keeps secrets people type in (put, replace, delete). Leave it out, or false, for one that only reads.

init ​

Sent once, after hello. config is the plugin's config from config.yaml, with each secret:env: and secret:file: reference in it already replaced by its value, so the plugin's own credentials never have to be written in the file. dataDir is the plugin's working folder, which is also its current folder: keep any state there.

json
→ {"id": 2, "method": "init", "params": {"config": {"address": "https://keeper.corp.example", "token": "…"}, "dataDir": "/data/plugins/keeper"}}
← {"id": 2, "result": {}}

Sign in here if your store needs it, and check the config. Answer invalid-config for a config that can't work: the start fails, health says so, and reportd tries again later (see Restarts).

get ​

The value for a reference: ref is the part after secret:<name>:, so secret:keeper:billing#password sends billing#password. What a ref means is yours to define; it may hold :, / and #.

json
→ {"id": 3, "method": "get", "params": {"ref": "billing#password"}}
← {"id": 3, "result": {"value": "hunter2", "version": "3", "expiresAt": "2030-01-01T00:00:00Z"}}
  • value is the secret, a string of at most 64 KiB. An empty value counts as not found: answer not-found instead.
  • version, optional, is the value's version in your store. reportd notes it, and a changed version or value reopens the connections that use it.
  • expiresAt, optional, an RFC 3339 time, is when the value stops being good, such as a short-lived database credential's end. reportd keeps the value no longer than that, and asks again.
  • Don't keep values on disk to save calls: reportd keeps them in memory itself, for secrets.cacheSec (300 seconds by default), and reads of one reference at the same time share one call.

put, replace and delete ​

Only for a plugin whose hello says writable: true: one that keeps secrets people type into a connection's form. reportd never sends them to a read-only plugin, which answers them with not-writable all the same.

json
→ {"id": 4, "method": "put", "params": {"value": "s3cret", "owner": {"connection": "billing", "environment": "dev", "field": "password"}}}
← {"id": 4, "result": {"ref": "9f2c41d0a1b2c3d4", "version": "1"}}
→ {"id": 5, "method": "replace", "params": {"ref": "9f2c41d0a1b2c3d4", "value": "s3cret-2"}}
← {"id": 5, "result": {"version": "2"}}
→ {"id": 6, "method": "delete", "params": {"ref": "9f2c41d0a1b2c3d4"}}
← {"id": 6, "result": {}}
  • put keeps a new secret and makes its ref: reportd stores secret:<name>:<ref> in the connection, never the value. owner says which connection field it was set for.
  • put and replace may also hold by, the name of the person who typed the value, for your own records. It is optional: ignore it if you keep none.
  • replace gives a secret a new value, and must answer a new version, so the connections using it are opened again.
  • delete removes it. A get of it afterwards answers not-found.

health ​

reportd asks each running plugin every 30 seconds, and administrators see the answer in the list of secret providers.

json
→ {"id": 8, "method": "health", "params": {}}
← {"id": 8, "result": {"state": "degraded", "message": "token renewal failed; signing in again", "expiresAt": "2026-11-03T11:00:00Z"}}
  • state is ok, degraded (working, but not for long, or not fully) or failing (gets will fail).
  • message, optional, says why, in words an administrator can act on. It is shown to them, so it must hold no value.
  • expiresAt, optional, is when the plugin's own sign-in ends, if it has one.
  • A plugin that doesn't answer in 10 seconds shows as failing. It isn't restarted for that.

shutdown ​

reportd is stopping. Answer, finish what you're doing, and exit.

json
→ {"id": 9, "method": "shutdown", "params": {}}
← {"id": 9, "result": {}}

reportd then closes the plugin's input, and kills it if it is still running 5 seconds after the request. Exit also when your input closes, without a shutdown: on some systems that is the only sign that reportd has gone.

Other methods ​

Answer a method you don't have with unsupported. A plugin may have methods of its own, which reportd's commands send; a new protocol version may add more. They aren't part of protocol 1's required set, and the kit doesn't check them: it checks only that a method the plugin doesn't have answers unsupported.

Our store plugin has one, rotate, which reportd secrets rotate-key sends while reportd is stopped: {"key": "<the new master key>", "rekey": false} → {"keyId": "…", "previousKeyId": "…", "rekeyed": false, "secrets": 3}. reportd sends it only to the plugin the command names.

Error codes ​

An error's code decides what reportd does and what people see. Its message goes to reportd's log only, as a warning (or at debug level, for not-found): people never see it, so write it for the administrator reading the log, and never put a value in it.

CodeAnswer it whenWhat people see (a connection's test, health and errors)A value kept from before
not-foundThere is no secret by that referencethe secret is not setis dropped
deniedYour store refused to give it (a policy, a permission)the secret provider refused to read the secretis dropped
unavailableYour store can't be reached, or is busy, for nowthe secret provider can't be reachedkeeps working
invalid-refThe reference isn't one your plugin can readthe secret provider can't read the referenceis dropped
invalid-configinit's config can't workthe plugin doesn't start; health says why–
not-writableA write to a plugin, or a secret, that is read-onlythe secret provider failedkeeps working
unsupportedA method you don't havethe secret provider failedkeeps working
internalAnything elsethe secret provider failedkeeps working

A code that isn't in the list counts as internal. "A value kept from before" is reportd's last value of that reference: it keeps working for up to secrets.staleSec (an hour by default) past its cache time while the plugin fails, so a short outage of your store goes unnoticed. The codes that say the secret is gone or barred drop it at once, so a revocation takes effect.

Starting, checking, restarting and stopping ​

At start ​

  1. reportd finds the program by file name in secrets.pluginDirs, checks the folder's and the program's owner and permissions, and the sha256 pin if there is one.
  2. It starts the program with no arguments, in its working folder, dataDir/plugins/<name>/ (made with mode 0700, and kept across restarts), with only the environment variables the plugin's env lists. On Windows the system adds SYSTEMROOT, which programs need to start.
  3. The plugin is tied to reportd, so it never outlives it: on Linux it is killed when reportd ends, and on Windows a job object ends it. On other systems it must exit when its input closes.
  4. reportd sends hello, checks the licence feature if the plugin names one, then sends init.

reportd starts the plugins before it reads its own secrets. A plugin that reportd's own config needs (say, a sign-in provider's client secret in your store) must come up, or reportd doesn't start, as with an environment variable that isn't set. state.postgres.url, the licence setting and renderer.token are read before the licence is, so only a plugin that declares no licence feature, as yours doesn't, can hold them.

While it runs ​

Each call has the time limit in the table above. A call that isn't answered in time fails as unavailable, and the plugin keeps running; its late answer is dropped. A plugin that stops reading its input until then is ended, and started again.

A value over 64 KiB, or a result of the wrong shape (a number where a string goes), fails only that call, as internal.

Restarts and outages ​

reportd ends the plugin and starts it again when it:

  • exits or crashes;
  • writes on standard output a line that isn't one JSON answer, an answer to an id reportd never sent or already had answered, a line over 1 MiB, or a last line without its newline.

The restarts wait 1, 2, 4… up to 60 seconds, and a run that lasted 60 seconds or more starts the wait at 1 second again. Calls in flight fail as unavailable, and health shows "restarting", with the crashes in the last 10 minutes.

A start that fails is tried again the same way, so a permission or a config file fixed while reportd runs is picked up: a missing program, an untrusted folder or program, a sha256 that doesn't match, a reference in its config that can't be read, or an init error. Two are final until reportd restarts: a plugin that needs a licence feature the licence lacks, and one that chose a protocol version reportd doesn't speak.

While a plugin is down, what fails is the connections whose values are past the stale limit, or were never read: they report secret-store until the plugin is back. Values read before keep working until their cache time plus secrets.staleSec. /readyz doesn't change: an outage fails only the connections that need the plugin.

At stop ​

reportd sends shutdown, closes the plugin's input, waits up to 5 seconds, then kills it.

Installing one ​

A plugin is installed by the administrator, in config.yaml: the studio can't add or change one, and Settings don't change secrets.*.

yaml
secrets:
  providers: [env, file]
  dir: /run/secrets
  pluginDirs: [/plugins]
  plugins:
    - name: keeper                      # answers secret:keeper:…
      program: reportd-secret-keeper    # a file name in one of pluginDirs, never a path
      sha256: 3b1f…                     # optional: the exact program
      env: [KEEPER_CA_FILE]             # the only environment variables it sees
      config:                           # sent in init
        address: https://keeper.corp.example
        token: secret:file:keeper-token # read by reportd first
  • pluginDirs are absolute paths to the folders plugins are started from. Neither the folder nor the program may be writable by others, or owned by anyone but reportd's user or root (on Windows, reportd's user, SYSTEM, the Administrators group or TrustedInstaller). A program that is a symbolic link is refused.
  • name is the reference prefix: lower-case letters, digits and -, starting with a letter, at most 32 characters. env and file are reportd's own.
  • program is a plain file name. On Windows, a name without an extension also finds <name>.exe.
  • sha256, optional, pins the program: reportd refuses to start any other. Set it when you install, and change it when you upgrade.
  • env names the environment variables the plugin sees, from reportd's own; a REPORTD_ variable is never passed, so a plugin can't read reportd's settings.
  • config is any YAML the plugin understands. The secrets it holds can be only secret:env: and secret:file: references, not another plugin's.
  • secrets.cacheSec (300) and secrets.staleSec (3600) are the cache and stale times above.

In the container image, the plugin folders are /app/plugins, the image's own: it holds our store plugin, reportd-secret-store (data connections), our HashiCorp Vault plugin, reportd-secret-vault (HashiCorp Vault), our Azure Key Vault plugin, reportd-secret-azure (Azure Key Vault), and our AWS Secrets Manager plugin, reportd-secret-aws (AWS Secrets Manager), each of which runs only once config.yaml names it. For another plugin, mount a folder holding it, read-only, and add it to the plugin folders. The image sets them with a variable, which wins over config.yaml, so set the variable: REPORTD_SECRETS_PLUGINDIRS=/app/plugins,/plugins. reportd runs as user 65532, so the files must belong to root (or that user) and not be writable by others. The image's program is built for Linux, so the plugin must be too, for the image's platform (amd64 or arm64), and must run without anything the image lacks: a program with no dependencies on the system's libraries, such as a Go program built with CGO_ENABLED=0, is simplest.

yaml
services:
  reportd:
    environment:
      REPORTD_SECRETS_PLUGINDIRS: /app/plugins,/plugins
    volumes:
      - ./plugins:/plugins:ro
      - ./config.yaml:/config/config.yaml:ro

Outside the image, each release also has an archive per plugin and platform, such as reportd-secret-store-0.9.2-linux-amd64.tar.gz, listed in its SHA256SUMS, holding the program, the terms it is used under (LICENSE), and its third-party notices (THIRD_PARTY_NOTICES.md, with their licence files under licences/ when it has any). Unpack it into a folder that only root can write to, and name that folder in pluginDirs.

Then:

  • reportd config check starts the plugins, checks them against the licence and prints each one's state.
  • GET /v1/secrets/providers lists every provider, built in or a plugin, with its state.
  • A studio connection may use the plugin's references only once an administrator adds a pattern for them, such as secret:keeper:reporting/*, to data.connections.secrets (data connections).

What a plugin must do ​

  • Never write a value to standard error, a file, a message or a health message, or anywhere on standard output but in a get answer. Values cross only the pipe; reportd keeps them in memory alone.
  • Keep your own credentials narrow. Sign in to your store with an account that can read only what reportd needs, and only read. Take the credential from config, through a secret:file: or secret:env: reference, never from a command line or a file of your own.
  • Answer health truthfully. degraded before your sign-in runs out is worth more than ok until it does.
  • Be safe to restart at any time. reportd may kill the plugin mid-request. Keep state in dataDir, and write it atomically (a temporary file, then a rename).
  • Check where you connect. A plugin's own network calls don't go through reportd's address guard: talk only to the addresses your config names, and verify TLS.
  • Exit when your input closes, and open no port of your own.

Choosing a language ​

Any language that can read and write lines on standard input and output will do. Three things trip plugins up in most of them:

  • Flush standard output after each answer. Most languages buffer output written to a pipe, so an unflushed answer waits in the buffer until reportd's time limit runs out: Python's print, C's stdio and Java's System.out all do.
  • Write each answer whole. If you answer requests at the same time, from threads or tasks, write each line under one lock, so two answers never interleave.
  • Read lines up to 1 MiB, past the default limit of some line readers.

Where reportd runs decides what the program may be:

  • In the container image, the plugin must be a native program for Linux. The image holds Debian 13's C library (glibc 2.41; before 0.10.0, Debian 12's 2.36), the C and C++ runtime libraries and CA certificates, and no shell or interpreter. These all work:

    LanguageHow
    GoBuilt with CGO_ENABLED=0, so it needs no library at all; the example is built this way
    RustBuilt for the x86_64-unknown-linux-musl (or aarch64-…) target, which is static, or for glibc on Debian 13 or older
    C, C++, ZigLinked statically, or against glibc 2.41 or older
    C#, F#.NET's Native AOT, for linux-x64 or linux-arm64, with InvariantGlobalization on, since the image has no ICU library
    Java, KotlinGraalVM's native-image; a program that needs a JVM won't start
    PythonBundled with its interpreter into one program, by PyInstaller or Nuitka
    JavaScript, TypeScriptOne program with its runtime inside, by Node's single executable applications, bun build --compile or deno compile

    A program linked against a glibc newer than 2.41 fails to start, with an error about GLIBC_2.xx in reportd's log: build it on Debian 13 or older, or link it statically. To run in a 0.9 image too, build it on Debian 12 or older. Whatever the language, try the program in the image with reportd config check before you ship it.

  • Installed on a server of your own, a script works on Linux and macOS: make it executable, owned as pluginDirs requires, and start it with a #! line naming the interpreter by its absolute path, such as #!/usr/bin/python3, since the plugin gets no PATH unless its env lists it. The trust check and the sha256 pin cover the script but not its interpreter, so a single program is the safer choice. On Windows, make the plugin an .exe.

The kit tests a plugin in any of these; only the kit itself needs Go.

Testing it with the kit ​

The kit starts your program as reportd does, with no arguments and only the environment you pass, and runs these checks:

  • hello: it chooses protocol 1 when offered [1] and names itself and its version; offered only a version that doesn't exist, it doesn't choose that.
  • init: it takes your config; given --bad-config, it refuses that one with invalid-config.
  • get: a hit (--hit) answers a value, a miss answers not-found, and a bad reference invalid-ref.
  • Pipelining: eight gets in flight at once each get their own answer, in any order.
  • Other methods: an unknown method answers unsupported; health answers a state.
  • Writes: for a writable plugin, put, get, replace, get, delete and get again, using the ref your put answers and checking that replace makes a new version. A read-only plugin must answer each write with not-writable.
  • Ending: after shutdown it answers and exits within 5 seconds; started again, it exits when its input closes.
  • Its output: nothing but answers on standard output, by the rules above, and no value the test read or wrote on standard error.
FlagWhat it is
--config <file>A JSON file holding the config sent in init, as reportd would send it, with the references already resolved
--bad-config <file>A config init must refuse
--hit <ref>A reference your plugin holds
--miss <ref>One it doesn't (a random one by default). When your references have a fixed form, such as an id of 16 hexadecimal digits, give one of that form that you don't hold: the pipelining check then sends it as it is, since anything else is a bad reference to your plugin
--bad-ref <ref>One it can't read (by default one holding a NUL character, which no reference may)
--env <NAME>An environment variable to pass through, as env does; repeat it for more
--replay <script>Replay one of the shared scripts instead; see below
-vPrint each line sent and answered

The kit exits 0 when every check passes, 1 when one fails, and 2 when it was run wrongly, so it can run in your CI. Point it at a test instance of your store, with a secret set aside for --hit. The values the kit writes start kit-value-.

--replay holds a plugin to the same scripted exchanges reportd's own tests use (exchanges.json, in the kit): store, vault and bad-config. The plugin must hold the scripts' data, such as billing#password with the value hunter2 at version 3, so it suits a test double more than a real store; the example's secrets.json holds it all. What is the plugin's own (its name, the refs and versions it makes, its health) is checked for shape only.

The kit is versioned on its own, as plugins/kit/v1.x.y: run @v1 for the latest kit of protocol version 1.

The example ​

The example is a whole plugin in one file. Its config names a JSON file of secrets:

json
{
  "billing#password": {"value": "hunter2", "version": "3"},
  "token": {"value": "t0k3n", "expiresAt": "2030-01-01T00:00:00Z"},
  "forbidden": {"error": "denied"}
}

An entry with error answers that code instead of a value, so you can see what reportd does with each one. It reads the file on each get, answers health with failing when it can't, and is read-only. To make it yours, change its lookup to read from your store, its init to sign in, and its name and version in hello. Its README says how to build it for the container image.