Skip to content

Data connections ​

A data source reads its rows through a connection: a database (PostgreSQL, MySQL, SQL Server or SQLite) or an HTTP service, with a binding in each environment that says where to connect there. There are two kinds:

  • The operator's connections, in config.yaml under connections (configuration). They change only there, with a restart, and the studio shows them as fixed.
  • Studio connections, which someone with the connection.manage permission adds, changes, tests, turns off and deletes from the studio's Data sources › Connections pane, with no restart. They are kept as data, like font and template sources.

Both work the same way once a source uses them: a source names its connection, and each environment's binding decides where it reads from there.

Who may change them ​

Studio connections need connection.manage, given per environment, so someone can look after the dev connections without touching prod. It is apart from source.manage because whoever adds a connection decides what reportd can reach on the network, and which secrets it sends there. An agent key never has it, whatever its roles, and there are no MCP tools for it.

  • The administrator role has it.
  • The built-in data-administrator role has studio.access, source.read, source.manage, data.live and connection.manage. It has no template.write, so it takes no designer seat (admin.md).

A change needs connection.manage in every environment whose binding it adds, changes or removes. A change to the driver or a limit reaches every binding, so it needs the permission in each environment that has one; so do turning a connection off or on and deleting it. Seeing a connection needs source.read or connection.manage; a binding's fields and references show only in the environments the person manages.

A binding outside the first environment needs a licence with environments: Community has one environment (licence.md). There is no limit on how many connections there are.

The policy comes first ​

An administrator sets what studio connections may reach in Settings › Studio connections (settings.md): the hosts (data.connections.allowHosts), private and loopback addresses, the one folder SQLite files come from, and the secret: reference patterns a connection may use (by default secret:env:CONN_* and secret:file:conn-*). Until allowHosts lists a host, no studio connection can be saved, and the pane says so.

Every binding is checked against the policy when it is saved, when it is tested, and again each time it is opened, so narrowing the policy takes effect at once. The address actually dialled is checked too, after DNS. Link-local addresses and the clouds' metadata services are never reached.

The operator's connections in config.yaml are trusted, so the policy doesn't apply to them; the address guard does (security.md).

Secrets are references ​

Every password, token and DSN from a secret is a secret: reference, such as secret:env:CONN_BILLING_PROD (an environment variable where reportd runs) or secret:file:conn-billing-prod (a file in secrets.dir). reportd reads the value when it connects; the value never comes to the studio, and no route, log line or audit event holds it.

  • Typing a secret's value where a reference goes is refused (secret-literal).
  • A reference outside the patterns is refused (policy).
  • A reference reportd itself uses is always refused, whatever the patterns say: the references in reportd's own settings (auth.*, viewer.linkSecret, the assistant's key, webhook and sign-in secrets, the operator's connections…), the tokens of the font and template sources, and any REPORTD_ variable. The refusal reads like one outside the patterns, so it doesn't tell which references reportd holds. Without this, someone could point an HTTP connection at their own server and have reportd send its secrets there.

Name the variables or files you set aside for connections so they match the patterns, and set them where reportd runs before saving the connection.

Secrets from plugins ​

A secret plugin is a separate program that reads secrets from somewhere else, such as a vault, and answers the references of its name: a plugin named vault answers secret:vault:kv/reporting/billing#password. reportd starts only the plugins config.yaml names; the studio can't add or change one, and Settings don't change secrets.*:

yaml
secrets:
  providers: [env, file]
  dir: /run/secrets
  pluginDirs: [/plugins]
  plugins:
    - name: vault                     # answers secret:vault:…
      program: reportd-secret-vault   # a file name in one of pluginDirs, never a path
      sha256: 3b1f…                   # optional: the exact program
      env: [VAULT_CACERT]             # the only environment variables it sees
      config:                         # sent to it at start; secret:env and secret:file references in it are read first
        address: https://vault.corp.example:8200
        auth: { method: approle, roleId: 7c1e…, secretId: secret:file:vault-secret-id }
  • env and file are reportd's own names. Each plugin runs in its own folder, dataDir/plugins/<name>/, as a child of reportd that ends when reportd does.
  • In the container image, the plugin folders come from the variable REPORTD_SECRETS_PLUGINDIRS, which the image sets to /app/plugins and which wins over pluginDirs in config.yaml. To add a folder of your own there, set the variable, such as REPORTD_SECRETS_PLUGINDIRS=/app/plugins,/plugins (secret-plugins.md); pluginDirs in the file is for installs outside the image.
  • A studio connection may use a plugin's references only once an administrator adds a pattern for them, such as secret:vault:kv/reporting/*, to data.connections.secrets.
  • reportd keeps a plugin's values in memory for secrets.cacheSec (300 seconds by default), or less when the plugin says a value expires sooner. Reads of one reference at the same time share one call.
  • Rotation. Once the cache time has passed, the secrets of each open connection are read again on its next use; a changed value or version reopens the connection, and reports already reading finish on the old one. A secret:file: secret is read again the same way, so a rotated Kubernetes secret volume works. When a database refuses the sign-in (PostgreSQL 28P01, MySQL 1045, SQL Server 18456) or an HTTP service answers 401, the connection's secrets are read once more past the cache, and the request is tried again if one changed.
  • When a plugin fails, the last value it gave keeps working for up to secrets.staleSec (an hour by default), and its health counts the stale values. A plugin that crashes, or writes anything on its output that isn't the protocol, is started again after 1, 2, 4… up to 60 seconds. What fails during an outage is the connections whose values are past the stale limit: they report secret-store until the plugin is back. /readyz doesn't change.
  • A plugin reportd's own config needs (an OIDC client secret in the vault, say) must come up at start, or reportd doesn't start, as with an environment variable that isn't set. One only connections use fails only them.
  • state.postgres.url, the licence setting and renderer.token are read before the licence is: a plugin that needs a licence feature can't hold them.
  • Licences. Our Vault (below), Azure (below) and AWS (below) plugins need a licence with secret-vaults (licence.md). reportd goes by the backend's kind, not by what the plugin says: a plugin named vault, azure or aws in secrets.plugins, one whose program is reportd-secret-vault, reportd-secret-azure or reportd-secret-aws, or one whose handshake names one of those as its provider, needs secret-vaults even if its handshake names no feature. Any other plugin that names no feature, such as one of your own, runs on any edition.
  • reportd config check starts the plugins and makes the checks reportd makes at start. reportd test --data live starts them too.
  • Writing a plugin of your own: the protocol, an example and a conformance kit are in writing a secret plugin.

Secrets typed into the form ​

With the store plugin, reportd-secret-store, a secret field can take the secret itself: the person types it once, and reportd keeps it, encrypted, in dataDir/plugins/store/store.json. It comes in the image and the Community edition, off until config.yaml names it:

yaml
secrets:
  providers: [env, file]
  dir: /run/secrets
  plugins:
    - name: store
      program: reportd-secret-store
      config: { key: secret:file:store-key }    # the master key: never written here
  • The master key is 32 random bytes in base64 (openssl rand -base64 32 makes one), from a secret:env: or secret:file: reference. reportd refuses a key written into config.yaml itself. Lose it and the stored secrets are gone: keep a copy apart from the backups (backup.md).
  • The encryption is AES-256-GCM, from Go's standard library: a random data key seals each secret, with its id and version bound in, and the master key wraps the data key. A MAC over the file means an entry can't be swapped, dropped or put back to an old version without the store refusing to open. The file holds no value in the clear.
  • In the form, once the plugin runs, each secret field has Set secret beside Reference. The value typed is sent once, with the save; reportd puts it in the store and keeps only the reference, secret:store:<id>. From then on the field shows Stored · set by pat, 2 Nov with Replace, never the value: no route returns it. Replace keeps the reference and makes a new version, so the connection opens again on the new value at once.
  • A stored secret belongs to the field it was set for: its reference is refused on any other field, binding or connection, whatever the patterns say. Copy from… leaves a stored secret behind, to be set again.
  • Deleted when unused. When a binding or a connection is deleted, or the field moves to a reference, its stored secret is deleted from the store.
  • One node only. On state.driver: postgres each node would keep its own store, so reportd refuses to start the store plugin there and says why: use a vault the nodes share, through the Vault, Azure or AWS plugin.
  • Rotating the master key: stop reportd, run reportd secrets rotate-key store --new-key secret:file:store-key-2, change the plugin's key to the new reference, and start it. --rekey also seals every secret again under a new data key. Until the next rotation the old key still opens the store, with a warning in the log and a degraded health, so a start on the old reference works; rotate to the same key again to forget the old one.
  • Backups hold store.json, still encrypted, and never the key (backup.md). A restore on a new machine needs the same key, under the same reference.

GET /v1/secrets/providers (with connection.manage in an environment, or settings.manage) lists each provider, built in or a plugin, with its state: ok, degraded or failing as a running plugin answers its health check, starting, restarting (with the crashes in the last 10 minutes), waiting for the licence check, refused (a licence feature the licence lacks, or no protocol version in common) or stopped.

Secrets from HashiCorp Vault ​

The Vault plugin, reportd-secret-vault, reads secrets from Vault's key/value secrets engine, version 2 or version 1. It comes in the image, off until config.yaml names it, and needs a licence with secret-vaults (licence.md). It is read-only: secrets are written in Vault, and connections name them.

yaml
secrets:
  plugins:
    - name: vault                           # answers secret:vault:…
      program: reportd-secret-vault
      config:
        address: https://vault.corp.example:8200
        namespace: reporting                # Vault Enterprise only; leave it out for the root namespace
        caFile: /run/secrets/vault-ca.pem   # when a private authority signs Vault's certificate
        mounts: { legacy: 1 }               # the KV version 1 mounts; every other mount is version 2
        auth: { method: approle, roleId: 7c1e…, secretId: secret:file:vault-secret-id }

References are secret:vault:<mount>/<path>#<key>, with ?version=<n> after the key for an older version of a version 2 secret:

ReferenceReads
secret:vault:kv/reporting/billing#passwordthe key password of the latest version of reporting/billing, in the mount kv
secret:vault:kv/reporting/billing#password?version=3the same key, of version 3
secret:vault:legacy/billing#passwordthe key password of billing, in legacy, a version 1 mount
  • The mount is the reference's first segment, unless mounts names a longer one: a mount whose path has a slash in it, such as teams/a/kv, must be listed (mounts: { teams/a/kv: 2 }).
  • A key isn't empty and holds no ?. A value that isn't text, such as a number, comes as its JSON.
  • Without ?version, a connection follows the latest version: write a new version in Vault and the connection opens again on it once reportd's cache time has passed, or at once when the database refuses the old password.
  • sys/, auth/, identity/ and cubbyhole/ are Vault's own paths: no reference reads them.
  • Keep the pattern in data.connections.secrets as narrow as the paths connections need, such as secret:vault:kv/reporting/*.

Signing in. auth.method is one of these. A credential is a secret:env: or secret:file: reference: reportd refuses one written into config.yaml itself.

MethodFieldsFor
tokentoken, a reference to a Vault tokenA token Vault Agent or an administrator gives reportd. A periodic token is renewed for as long as reportd runs; any other ends at its maximum TTL, and health then says to give reportd a new one
approleroleId; secretId, a reference (leave it out for a role that binds none); mount (approle by default)reportd on a machine or in a container of its own
kubernetesrole; mount (kubernetes by default); jwtFile, the service account token's file (by default the pod's own, /var/run/secrets/kubernetes.io/serviceaccount/token), or jwt, a reference to onereportd in Kubernetes
  • Renewal. The plugin renews its token at two thirds of its TTL. When a renewal fails, or the token has reached its maximum TTL, it signs in again; a sign-in that fails is tried again after 5 seconds, then 10, and so on up to a minute.
  • With kubernetes, the token file is read at each sign-in, since Kubernetes replaces a projected service account token within the hour.
  • When Vault refuses the token itself (someone revoked it, say), the read signs in again and tries once more, at most once every 30 seconds.
  • At stop, the plugin revokes the token it signed in for. The token method's token is yours, and stays.
  • The policy needs read on the paths connections use: for version 2, <mount>/data/<path>. Vault's default policy, which every token has unless you take it away, lets the plugin look up and renew its own token.
hcl
path "kv/data/reporting/*" { capabilities = ["read"] }

When Vault says no:

Vaultreportd
403 for the pathdenied: the policy doesn't allow it, and reportd drops the value it kept
404, a key the secret doesn't hold, or a deleted versionnot-found
Sealed, a standby that can't answer, a 5xx, or no answerunavailable: reportd keeps the last value for up to secrets.staleSec
A reference of the wrong shapeinvalid-ref

A connection's Test shows denied, not-found and invalid-ref as "The secret is not set.", and unavailable as "The secret store can't be reached."; reportd's log has Vault's reason.

  • Health (GET /v1/secrets/providers) is failing while Vault can't be reached, is sealed, or the token has expired, and degraded while renewing or signing in again fails but the token still works, with expiresAt, when the token ends. A token method's token that can't be renewed is degraded once two thirds of its life has gone.
  • Nothing on disk. The token lives in the plugin's memory only. The plugin writes nothing to dataDir/plugins/vault/, so backups hold nothing of it.
  • TLS. Use an https address; the plugin warns in the log when the address is plain http and not this machine. caFile is an absolute path. The VAULT_ variables Vault's own tools read, such as VAULT_CLIENT_CERT and VAULT_CLIENT_KEY for a client certificate, apply when the plugin's env lists them; VAULT_TOKEN doesn't, since the token comes from auth.

Secrets from Azure Key Vault ​

The Azure Key Vault plugin, reportd-secret-azure, reads secrets from the key vaults you name. It comes in the image, off until config.yaml names it, and needs a licence with secret-vaults (licence.md). It only reads: set, rotate and delete the secrets in Key Vault.

yaml
secrets:
  plugins:
    - name: azure                       # answers secret:azure:…
      program: reportd-secret-azure
      config:
        credential: managed-identity    # or workload-identity, or client-secret
        cloud: public                   # the default; or china, or usgov
        vaults: [billing-kv]            # the only key vaults it reads
  • References are secret:azure:<vault-name>/<secret-name>[/<version>][#key], such as secret:azure:billing-kv/db-password. Without a version, the current one is read, so a new version set in Key Vault reaches reportd once the cache time has passed, and reopens the connections that use it. #key picks one key out of a secret that holds a JSON object, such as secret:azure:billing-kv/billing-db#password: a string as it is, a number or anything else as its JSON. A secret's expiry date in Key Vault is the longest reportd keeps its value.
  • For studio connections, add a pattern for them to data.connections.secrets, such as secret:azure:billing-kv/*.
  • vaults is the whole list the plugin reads from: a reference to any other vault is refused as denied, without asking Azure. Give the identity it signs in as read access to those vaults' secrets alone: the Key Vault Secrets User role, or an access policy with Get on secrets.
  • cloud is where the vaults are: public (<name>.vault.azure.net, signing in at login.microsoftonline.com), china (vault.azure.cn, login.chinacloudapi.cn) or usgov (vault.usgovcloudapi.net, login.microsoftonline.us).

It signs in to Microsoft Entra ID with one of three credentials:

credentialWho it signs in asWhat it takes
managed-identityThe managed identity of the Azure VM, scale set, AKS node pool, App Service or Container App reportd runs onNothing for the system-assigned identity; clientId for a user-assigned one. On a VM it asks the instance metadata service, 169.254.169.254. On App Service and Container Apps, pass their identity endpoint: env: [IDENTITY_ENDPOINT, IDENTITY_HEADER]
workload-identityAn AKS workload identity: the pod's Kubernetes service account, federated with an app registration or a user-assigned identitytenantId, clientId and tokenFile, or the variables the workload identity webhook sets, passed with env: [AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_FEDERATED_TOKEN_FILE]. The plugin reads the token file each time it signs in, since Kubernetes renews it
client-secretAn app registration (a service principal)tenantId, clientId and clientSecret, a secret:env: or secret:file: reference: reportd refuses a client secret written into config.yaml itself
yaml
    - name: azure
      program: reportd-secret-azure
      config:
        credential: client-secret
        tenantId: 72f988bf-86f1-41af-91ab-2d7cd011db47
        clientId: 9c4e5b2a-0d1f-4f6e-8a3b-7c2d1e0f9a8b
        clientSecret: secret:file:azure-client-secret
        vaults: [billing-kv, shared-kv]
  • A field the credential doesn't take is refused, as an unknown field is, so a client secret can't be left beside a managed identity, unused.
  • Health is ok while the plugin can get a Key Vault token, with that token's expiry; failing, with what Entra ID or the identity endpoint answered (AADSTS7000215: Invalid client secret provided, say), while it can't; and degraded when it signs in but its last read failed, throttled or unreachable, until a read works. A sign-in that fails doesn't stop the plugin: it tries again on the next read.
  • What a failure means for a connection: Key Vault's 403 (no role or access policy, a disabled secret, the vault's firewall) is denied, and 404 not found: both drop the value reportd kept, and a connection's test shows secret-missing. Throttling, Key Vault or Entra ID unreachable or failing, and a refused sign-in are unavailable: the last value keeps working for up to secrets.staleSec, and a test shows secret-store. reportd's log says which it was.
  • Nothing on disk. Tokens stay in the plugin's memory; its folder, dataDir/plugins/azure/, stays empty, so a backup holds nothing of it.

AWS Secrets Manager ​

The AWS plugin, reportd-secret-aws, reads secrets from AWS Secrets Manager. It comes in the image, off until config.yaml names it, and needs a licence with secret-vaults (licence.md). It only reads: secrets are made, changed and rotated in AWS.

yaml
secrets:
  plugins:
    - name: aws                       # answers secret:aws:…
      program: reportd-secret-aws
      env: [AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY]   # what the default chain reads here; none on EC2, ECS or EKS
      config:
        region: eu-west-1
        roleArn: arn:aws:iam::123456789012:role/reportd-secrets   # optional
        externalId: secret:file:aws-external-id                   # optional, with roleArn

A studio connection uses its references once an administrator adds a pattern for them to data.connections.secrets, such as secret:aws:prod/reporting/*.

References name a secret, and optionally a key and a stage:

secret:aws:<name or ARN>[#key][?stage=AWSCURRENT|AWSPREVIOUS]
ReferenceReads
secret:aws:prod/billing-dsnThe whole secret: its text, as stored
secret:aws:prod/billing#passwordOne key of a secret that holds a JSON object. A number or true/false is read as written: #port gives 5432
secret:aws:rds!db-0f2c41d0-…#passwordThe password of a database whose master password RDS manages, under the name RDS gave its secret
secret:aws:arn:aws:secretsmanager:eu-west-1:123456789012:secret:prod/billing-AbCdEf#passwordA secret by its ARN, whole or without the six-character suffix: for a secret of another account, shared with yours
secret:aws:prod/billing#password?stage=AWSPREVIOUSThe version before the current one, which a rotation keeps for a while

A secret that holds binary data rather than text can't be read. A key that is missing, null or empty, or a stage the secret has no version for, is a secret not found; a key that holds an object or a list, or a secret that isn't JSON when a key is given, is a reference the plugin can't read.

Its config:

SettingWhat
regionRequired: the region the secrets are in, such as eu-west-1. The plugin doesn't take it from the environment.
roleArnA role to assume through STS, with the credentials below; the secrets are then read as that role. The role's keys are renewed a minute or so before they expire.
externalIdThe external id the role's trust policy asks for, if any: it may be a secret:env: or secret:file: reference
roleSessionNameThe session's name in CloudTrail: reportd by default
endpoint, stsEndpointOther addresses for Secrets Manager and STS, such as a VPC endpoint without private DNS, or LocalStack for a trial

An unknown setting is refused, so a misspelt one doesn't go unnoticed.

Credentials come from the AWS SDK's default chain, as for the AWS command line: the first it finds of access keys in AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN; a shared credentials or config file (AWS_SHARED_CREDENTIALS_FILE, AWS_CONFIG_FILE, AWS_PROFILE); a web identity token (AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN, as EKS's IAM roles for service accounts set them); the container's credentials (ECS task roles and EKS Pod Identity, through AWS_CONTAINER_CREDENTIALS_RELATIVE_URI or AWS_CONTAINER_CREDENTIALS_FULL_URI and AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE); and an EC2 instance's role, from its metadata service, which needs no variable. The plugin sees only the variables its env lists, so list the ones your chain needs, set where reportd runs:

  • On EC2, with an instance profile: no env at all. In a container there, the instance's metadata options need a hop limit of 2.
  • On ECS, with a task role: env: [AWS_CONTAINER_CREDENTIALS_RELATIVE_URI].
  • On EKS, with IAM roles for service accounts: env: [AWS_WEB_IDENTITY_TOKEN_FILE, AWS_ROLE_ARN]; with Pod Identity: env: [AWS_CONTAINER_CREDENTIALS_FULL_URI, AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE].
  • Elsewhere, an IAM user's access keys. To keep them out of the environment, put them in a credentials file, mounted as a secret, and name it: AWS_SHARED_CREDENTIALS_FILE=/run/secrets/aws-credentials where reportd runs, and env: [AWS_SHARED_CREDENTIALS_FILE].

The keys give the plugin nothing but what IAM grants them. Grant secretsmanager:GetSecretValue on the secrets reportd reads, and kms:Decrypt on their KMS key when it is a key of your own; with roleArn, the credentials need sts:AssumeRole on the role, and the role those grants:

json
{
  "Version": "2012-10-17",
  "Statement": [
    { "Effect": "Allow", "Action": "secretsmanager:GetSecretValue",
      "Resource": "arn:aws:secretsmanager:eu-west-1:123456789012:secret:prod/reporting/*" }
  ]
}
  • At start the plugin asks STS whose credentials it has, and the log says (reading Secrets Manager in eu-west-1 as arn:aws:sts::…:assumed-role/reportd-secrets/reportd). A start without credentials, or with ones AWS refuses, still comes up, with a warning: it tries again on each read, and its health says why it can't read.
  • Rotation works with no restart, RDS-managed rotation included. The plugin answers each secret's version id as its version, so once secrets.cacheSec has passed a new version reopens the connections that use it; and a database that refuses the sign-in has the secret read again past the cache at once (above).
  • Health is failing when the plugin has no credentials, the role can't be assumed, or AWS refused the credentials at the last try (an unknown key, an expired token), naming where the credentials came from and AWS's error. It is degraded for 5 minutes after Secrets Manager couldn't be reached or throttled the reads, and it gives the time the credentials are next renewed, when they expire. Health costs no call to Secrets Manager: it reports what the reads found. A secret the role may not read fails that secret only.
  • What fails, and how it shows: a secret that doesn't exist, is scheduled for deletion, or lacks the key or the stage, is "the secret is not set" (secret-missing in a test); a read IAM or KMS refuses is "refused to read the secret" (secret-missing too); throttling, a timeout or an address that doesn't answer is "can't be reached" (secret-store). Throttled reads are tried three times. The detail, with AWS's error code, is in reportd's log.
  • Its own network calls go to the region's Secrets Manager and STS addresses, and to the instance or container metadata service (169.254.169.254 or 169.254.170.2), not through the address guard. In a VPC without internet access, add interface endpoints for both services, or name them in endpoint and stsEndpoint.
  • Nothing is kept on disk: the plugin's folder, dataDir/plugins/aws/, stays empty, so backups hold no credentials of it. Its credentials and the values it reads are in memory only.

What a binding holds ​

DriverFields, per environment
PostgreSQLhost, port (5432), database, user, password (a secret), SSL mode (verify-full, the default, verify-ca, require or disable), and options from a short list: application_name, connect_timeout, target_session_attrs, search_path
MySQLhost, port (3306), database, user, password (a secret), TLS (true, the default, skip-verify or preferred), and options: charset, collation, timeout, readTimeout, loc
SQL Serverhost, and a port (1433) or a named instance, database, user, password (a secret), encrypt (true, the default, strict or disable), trust the server certificate, and options: app name, ApplicationIntent, connection timeout, dial timeout
SQLitea file in the policy's SQLite folder, opened read-only
HTTPthe base URL, its sign-in (none, a bearer token, an API key in a header, or basic, each with its secret), and extra headers, which are sent as written and are not for secrets

reportd builds the database's connection string from the fields, escaping the user, password and database name for each driver. The options are a short list on purpose: those that read files on reportd's machine (client certificates, password files, LOAD DATA LOCAL INFILE, Kerberos files) or widen what a query can do (several statements at once) are refused.

Each driver can instead take a whole DSN from a secret. Its host isn't known until the secret is read, so the binding lists the hosts it may reach (allowHosts, each within the policy), and the DSN is held to the same option lists when it is opened.

Each connection also has a description and limits: a fetch's timeout (30 s), its rows (100 000) and its size (50 MB).

Give connections a read-only database account. reportd reads in a read-only transaction that it always rolls back, except on SQL Server, which has none: there the account's own permissions are what stop a write.

The form ​

New connection (N) in Data sources › Connections, or Edit on a studio connection, opens the form: the name, which can't change later, the driver, a description, the limits, and a binding for each environment.

  • A tab per environment. Those you manage are editable, with Copy from… to start from another environment's binding, Remove the binding, and Test. The others show read-only whether there is a binding, and saving keeps theirs as it is. Community has one environment, so one form and no tabs.
  • Each secret field takes where the secret is (an environment variable or a file in secrets.dir) and its name, and builds the reference. The patterns are checked as you type. With the store plugin running, Set secret takes the secret itself instead (above).
  • Under the fields is the connection string reportd will open, each secret shown as its reference: postgres://report_ro:‹secret:env:CONN_BILLING_DEV›@db-dev.corp.example:5432/billing?sslmode=verify-full. A DSN from a secret shows as "DSN from ‹secret:…›", and a secret typed in or stored as ‹stored secret›.
  • SQLite picks a file from the policy's folder, and isn't offered when the policy has none.
  • A refused save says why under the field, on its environment's tab.

Connections from config.yaml show with a config mark, and only their Test. Without connection.manage, the pane is read-only.

Testing ​

Test opens the binding on a connection of its own, pings it within 5 seconds and closes it. It works for a saved binding (POST /v1/connections/{name}/test) and for one still in the form (POST /v1/connections/test), which is checked against the policy before anything is dialled. A secret typed for a test is used for that test only.

A failed test answers a category with a hint, never the database's or service's own message:

CategoryMeans
dnsThe host name wasn't found
blockedThe policy refused the address
timeoutNo answer in time
refusedNothing listens on that port
tlsThe secure connection failed
sign-inThe user, password or token was refused
databaseThe database wasn't found, or the user may not use it
secret-missingThe variable or file the reference names isn't set
secret-providerThe reference names a secret provider that isn't on
secret-storeThe secret plugin isn't answering, or failed
unboundNo binding in that environment, or the connection is off
failedAnything else

Tests count in the probe rate class, 10 a minute by default (rate-limits.md), and each is audited, so probing for open ports shows in the log.

Changing production ​

Sources pin a connection by name, so changing a binding changes, at once, where every published source using that connection reads from in that environment. reportd has no fixed "prod", so the rule is by dependents:

  • A change, a removal or turning off of a binding in an environment where sources use the connection is refused with 409 needs-confirm, naming the environments and the sources. Sent again with "confirm" naming those environments, it goes ahead. The studio shows "This changes where 4 published sources in prod read from: …" and a button that names the environment. Replacing a secret's reference counts as a change.
  • Adding a binding where no source uses the connection yet needs no confirm.
  • The audit event of a confirmed change ends (confirmed). With the connection's revision, that answers who changed production, and when.

Turning a connection off is the step that can be undone: it behaves as a connection with no binding, and a live fetch through it fails with "connection billing is turned off". Deleting is refused while any source uses it (409 in-use, with the sources).

Two people changing one connection ​

Every connection has a revision that goes up with each change, and a change names the revision it was made on. If someone else changed the connection meanwhile, it is refused with 409 changed and the current revision: load it again (the form offers Reload), and make the change on what is there now. This holds across several nodes as well.

Config and studio names ​

A studio connection can't take the name of one in config.yaml (409 fixed), and a name can't change once added. If config.yaml later adds a name a studio connection already has, config wins: its connection is used, the studio's shows as shadowed, and reportd's start-up log warns. Delete the shadowed one, or rename one of them, to tidy up.

Where they are kept ​

  • On state.driver: file: dataDir/connections.json.
  • On state.driver: postgres: the schema's connections table, so every node opens the same. A change on one node reaches the others at once, and within 5 seconds if a notice is lost (state.md). reportd state import and export move the connections with the rest of the administration data.

Either way they hold only secret: references, and for a stored secret who set it and when. A changed connection is reopened on its next use; a report already reading through the old binding finishes on it.

Backups include the connections, with their revisions (backup.md). The values behind the references are where you keep them (the environment, secrets.dir), which the backup leaves out, so set them on a new machine before restoring.

reportd test --data live opens studio connections as well as the operator's, under the same policy.

Audit ​

Each change is audited with the connection's name, and the environment where it has one:

ActionWhen
data.connection.add <name>Added, once per environment bound
data.connection.change <name>: <fields>Changed, once per environment touched, naming the fields and never their values (host, password (reference)), and ending (confirmed) after a confirm. Driver, limits and description are one event with no environment.
data.connection.disable <name>, data.connection.enable <name>Turned off or on
data.connection.remove <name>Deleted
data.connection.test <name>Tested; an unsaved one shows as (a new connection)
secret.set <reference> (<name>, <environment>, <field>)A secret typed in and put in the store; the change event names the field as password (stored secret)
secret.replace <reference> (…)A stored secret replaced
secret.delete <reference> (…)A stored secret deleted, with its binding, its connection, or the field moved to a reference

The routes ​

RouteDoes
GET /v1/connectionsEvery connection, its origin (config or studio), whether it is fixed, off or shadowed, how many sources use it in each environment, its health with check=true, and the bindings of the environments the caller manages
GET /v1/connections/{name}One connection, its revision and the sources using it
PUT /v1/connections/{name}Adds (201) or changes (200) a studio connection. A binding in an environment the caller doesn't manage, left out, stays as it is. A secret field may be {"set": "<value>"} while the store plugin runs (422 no-store otherwise, 503 secret-store when it fails the write, and nothing is saved); the answer holds only the reference, and the binding's stored names the fields the store keeps, with setBy and setAt
POST /v1/connections/{name}/disable, /enableTurns it off or on
DELETE /v1/connections/{name}Deletes it
POST /v1/connections/{name}/test?environment=Tests a saved binding
POST /v1/connections/testTests an unsaved one
GET /v1/connections/policyThe policy, the SQLite folder's files, the secret providers running (and whether one takes typed secrets) and the options, for the form
GET /v1/secrets/providersEach secret provider and its health

The HTTP API reference has every field and refusal.