Shared state: users and sign-ins on several nodes

reportd keeps users, API keys, roles and sign-in sessions in dataDir/auth.json, and the short-lived state of a sign-in (MFA tickets, OIDC and SAML flows, the sign-in throttle, rate limits, view sessions) in its memory. That suits one reportd. With state.driver: postgres, all of it is in a Postgres database instead, which several reportd nodes behind a load balancer share.

Only the sign-in side, the audit log and usage are shared yet. The template store, approvals, jobs and the git repository are still on each node's disk. Several nodes therefore can't serve the same templates until the Postgres template store, a later item. reportd says so in its log at every start on postgres.

What is shared, and how fast

State	On file	On postgres
Users, API keys, custom roles	auth.json	One row each. A change on one node reaches the others at once through NOTIFY, and within 5 seconds if a notice is lost.
Sign-in sessions	auth.json	One row each, read on every request: signing out, or a session an administrator ends, stops on every node at once. Disabling a user ends their sessions too.
MFA tickets and enrolments	memory	Rows. A ticket made on one node can be answered on another, and its 5 tries count across all of them. A TOTP secret waiting for its first code is sealed under auth.mfaKey.
OIDC and SAML flows	cookies sealed with a key made per process	The same cookies, sealed with a key the nodes share and replace daily; the previous day's still opens them. A sign-in started on one node finishes on another.
The SAML replay cache	memory	Rows: an assertion used on one node is refused on every other.
Rate limits and the sign-in throttle	memory	Token buckets in an unlogged table, one statement per request. While Postgres can't be reached, each node counts in its own memory and logs that once a minute.
The audit log	dataDir/audit.log, rotated to dataDir/audit/	Each node writes its own file and the audit_events table; the Audit page and GET /v1/audit read the table, so they show every node's events (admin.md).
Usage	dataDir/usage/	Every node's records in usage_records, each day's totals in usage_daily; one node at a time rolls up, under an advisory lock (admin.md).
Viewer sessions	memory	Their state (report, commit, parameters, view state) is in a row; their data is on the node that fetched it. A node without the data fetches it again with the same parameters, so the viewer carries on, perhaps with newer data. A session opened with data its caller gave carries on only on that node.

The viewer's data limits (viewer.maxBytes, memoryBytes, maxTotalBytes) stay per node, since each node holds its own data.

On postgres the sign-in throttle refills over its window instead of waiting for it to pass: five wrong passwords for a username block it, and then one try comes back every 3 minutes (one every 45 seconds for an address after twenty). On file it is as before.

When Postgres is down, sessions can't be checked, so sign-ins and every signed-in request answer 503 state-unavailable with Retry-After, never a guess. /readyz answers 503 too, so a load balancer (or the Helm chart's readiness probe) stops sending that node requests. The rate limits and the throttle carry on in each node's memory. A node keeps the users, keys and roles it has, and reloads them when Postgres answers.

Choosing postgres

In containers, compose's postgres.yml and the Helm chart's state.postgres.existingSecret set this up (docker.md, helm.md); the variables are REPORTD_STATE_DRIVER and REPORTD_STATE_POSTGRES_URL.

state:
  driver: postgres                      # file (the default) or postgres
  postgres:
    url: secret:env:REPORTD_STATE_URL   # a secret: reference; a literal is refused
    schema: reportd                     # default reportd
    maxConns: 10                        # default 10
auth:
  mfaKey: secret:env:REPORTD_MFA_KEY    # required on postgres (Shared secrets)
viewer:
  linkSecret: secret:env:REPORTD_LINK_SECRET   # required on postgres with the viewer on

• url is a Postgres connection URL (postgres://reportd:…@db.internal:5432/reports?sslmode=verify-full) behind a secret: reference, since it holds the database's password.
• reportd creates schema and its tables the first time, and upgrades them when a newer reportd needs it, under a lock, so nodes starting together don't race. The role needs CREATE on the database the first time, and on the schema afterwards.
• Every node of one setup names the same database and schema. Two setups can share a database with a schema each.
• Connect to Postgres directly, or through a pooler in session mode. reportd uses advisory locks and LISTEN, which a transaction-mode pooler (PgBouncer's default) breaks.
• Each node holds at most maxConns connections, and one more that listens for changes.
• A supported Postgres, 14 or later; reportd's own tests run on 17.

Shared secrets

With postgres, these must be secret: references every node reads, and reportd refuses to start without them, naming the setting:

Setting	When	Why
auth.mfaKey	always	It seals the users' TOTP secrets. A node with another key can't check their codes.
viewer.linkSecret	with viewer.enabled	It signs report links. A link signed on one node wouldn't open on another.
auth.saml.key	with SAML providers	It is the SP key pair each IdP pins. Each node would otherwise make its own.

On file, reportd makes these as files in dataDir when they aren't set (mfa.key, viewer-link.key, saml-sp.pem). Each holds its key as text, so moving it is copying its value into your secret store:

# For secrets.dir (secret:file:…): copy the files there.
cp /var/lib/reportd/mfa.key /var/lib/reportd/viewer-link.key /etc/reportd/secrets/
# Then, in the config:
#   auth.mfaKey: secret:file:mfa.key
#   viewer.linkSecret: secret:file:viewer-link.key

Keep the same values. A new auth.mfaKey makes every authenticator app fail (users sign in with a recovery code or a passkey, or an administrator resets them), and a new link secret breaks every link made before.

Moving to postgres

1. Move the key files into your secret store and point the settings at them, as above. Start reportd once on file with them, and check a code from an app and a report link still work.

2. Stop reportd.

3. Set state.driver: postgres and state.postgres.url.

4. Import auth.json, sessions included, so nobody has to sign in again:

   reportd state import --config /etc/reportd/config.yaml

   It writes the users, keys, roles and sessions into the schema, and renames auth.json to auth.json.imported-<time>, so a later start on file can't quietly use users that are out of date. It refuses a schema that already has users, and reminds you of any key file still in dataDir.

   It copies dataDir/usage into the schema's usage tables, unless it has usage already, and fills audit_events from this node's audit log, the rotated files too, unless the table has events. Both stay in dataDir. Import on one node: the others' audit logs stay their own files.

   It also moves the licence (licence.md): the one installed in dataDir/licence (unless the schema already has one, which stays), the licences installed with the features each granted and the 60-day transition, and the installation id. dataDir keeps its copies. A licence given by the licence setting isn't stored, so there is nothing to move. The same goes for what the administration site keeps: settings.yaml becomes the schema's settings, branding/ its branding, font-sources.json its font sources and template-sources.json its template sources, unless the schema has its own.

5. Start the nodes.

reportd refuses to start on an empty schema while auth.json beside it has users, and says to run reportd state import first. That keeps it from seeding a new administrator over the real ones.

Going back to file

reportd state export --config /etc/reportd/config.yaml --out /var/lib/reportd/auth.json

It writes the users, keys, roles and live sessions as an auth.json, and writes the schema's licence, its history and the installation id into the config's dataDir, so the licence still applies on file. A different licence already in dataDir is kept beside it as licence.before-export-<time>. The schema's usage goes to dataDir/usage, unless that has usage already; the audit log's files are left as they are. It writes the settings reportd reads (settings.yaml with the schema's settings over it), the branding and the font and template sources into dataDir too, keeping what they replace as settings.yaml.before-export-<time>, branding.before-export-<time>, font-sources.json.before-export-<time> and template-sources.json.before-export-<time>. Stop every node, put the file in dataDir, set state.driver: file, and start one reportd. To undo an import you just ran, rename auth.json.imported-<time> back instead.

Sessions

A session lasts auth.sessions.lifetime from sign-in (24 hours by default), whatever happens meanwhile. auth.sessions.idleTimeout, off by default, also ends one that has had no request for that long; it is checked to the minute, so it is at least 5 minutes.

auth:
  sessions:
    lifetime: 24h
    idleTimeout: 2h

Each session records when it was made and last used, how its user signed in (password, password+totp, password+passkey, password+recovery, oidc:<provider> or saml:<provider>), the browser and the client address. Last use is written at most once a minute. On file it is kept in memory and written with the next change, every 5 minutes, and at shutdown. A session from before reportd kept this detail shows "unknown" for it.

• Users see their sessions in the studio's Account dialog, "Where you're signed in", and sign any of them out, or all but this one.
• The API: GET /v1/me/sessions, DELETE /v1/me/sessions/{id}, DELETE /v1/me/sessions?others=true; and, with user.manage, GET /v1/users/{name}/sessions and DELETE /v1/users/{name}/sessions[/{id}].
• A session's id names it; it isn't its token, and signs no one in.
• Each end is audited as user.sign-out <name> (session <id>), with (by <admin>) when an administrator ended it.

Backups

reportd backup reads the users, keys and roles through the store on either driver and writes them as data/auth.json in the archive, still without sessions (backup.md). On postgres, reportd restore writes them into an empty schema, as state import does, and refuses a schema that has users before it writes anything. The usage records and totals are read from the schema into the archive's data/usage/ too, and a restore writes them back into it. The short-lived state and the audit_events table are never backed up: the archive has this node's audit log files.

The database's own backups (pg_dump, snapshots, point-in-time recovery) are yours. reportd's tables are all in its schema:

pg_dump --schema=reportd --format=custom --file=reportd-state.dump "$REPORTD_STATE_URL"

buckets is unlogged: a crash or a failover empties it, which only forgets how full the rate limit buckets were.

The tables

Table	Holds
auth_users, auth_keys, auth_roles	One row each, as JSON, stamped from the changes sequence; a deleted one stays a day as a deleted row, so a node that was away sees it
auth_identities	Which user each (issuer, subject) is: unique, so two nodes can't bind one identity twice
auth_sessions	Sessions by the SHA-256 of their token
mfa_tickets, mfa_enrolments	Sign-ins waiting for their second factor; factors being set up
flow_keys	The current and previous keys of the OIDC and SAML flow cookies
saml_assertions	Assertions used, until they expire
buckets	Rate limit and throttle buckets (unlogged)
view_sessions	Viewer sessions' state, without data
licence, licence_seen	The installed licence, the transition of an install from before licences, the shared installation id, and the licences installed with the features each granted (licence.md)
audit_events	Every node's audit events, with the fields the Audit page filters on as columns (admin.md)
usage_records, usage_daily	Every node's usage records, and each UTC day's totals (admin.md)
settings	The settings the administration site changed: one YAML document, read over each node's settings.yaml, and its version, which every node polls and restarts on
branding	The branding (branding.json) and its logo, favicon and background, by name
font_sources	The font sources declared, and what was changed on a preset, in the order they were added (fonts.md)
template_sources	The template sources added, and the example gallery's switch, in the order they were added (template-sources.md)
schema_version	The schema's version

A sweep every minute, on whichever node gets its lock, removes what has run out, and replaces the flow key once a day.