Introduction

FerroEHR is a pure-Rust openEHR Clinical Data Repository (CDR): a headless, API-first server that stores and queries structured health records through a vendor-neutral REST API and the Archetype Query Language. This book is the user-facing guide — how to run it, configure it, talk to its API, query it, and load the templates that give your data shape. If you build clinical applications, operate healthcare infrastructure, or are evaluating an openEHR back end, you are in the right place.

The name comes from ferrum, Latin for iron: Rust is iron oxide, so the project went straight to the element. Say it “FER-ro-E-H-R”.

What openEHR gives you

openEHR separates clinical knowledge from software. The structure and meaning of clinical data — a blood-pressure reading, a medication order, a discharge summary — live in shared, computable models called archetypes and templates, authored by clinicians and modellers rather than baked into application code. Applications then store and retrieve that data through a standard API, against a shared Reference Model, so the same record is portable across every conformant system.

FerroEHR implements that standard natively. It speaks the openEHR REST API (ITS-REST Release-1.1.0), executes Archetype Query Language (AQL 1.1), and holds data as canonical openEHR compositions with full, indelible version history. There is no proprietary data format in the middle: what you commit is what you query and what you read back.

What makes this implementation different

• Conformance you can run yourself. The openEHR conformance catalogue is executed by a committed runner against a live server, over both canonical JSON and canonical XML, and the profile verdicts are computed from the per-case outcomes. The run records live in the repository, and every number on the Conformance page is derived from them at build time, never hand-typed.
• The openEHR specifications, generated directly from the official machine-readable models: the Reference Model, the Archetype Model (1.4 and 2.4), the serialization schemas, and the REST API contract (Release-1.1.0). openEHR’s own published terminology (3.1) ships embedded, byte-identical to upstream. A specification update is a regeneration.
• Two selectable specification generations. One configuration key, spec_profile, chooses the whole generation set the server runs: development (Reference Model 1.2.0 with BASE 1.3.0 and LANG 1.1.0 — the default) or stable (the latest released generations, Reference Model 1.1.0 with BASE 1.2.0 and LANG 1.0.0). See spec_profile.
• One self-contained binary. No JVM, no language runtime, and a pure-Rust TLS stack, so there is nothing to provision beside PostgreSQL. The container image is shell-less and runs as non-root.
• PostgreSQL 18-native storage. Clinical documents are decomposed into an indexed node model with time-bounded, versioned rows; canonical openEHR JSON is stored verbatim so storage and API never disagree.

How the system is layered

FerroEHR is built in two layers. A specification layer is generated deterministically from openEHR’s published models: the Reference Model types, canonical JSON/XML serialization, the REST contract, and the AQL front end. On top of it sits the application: the server, the PostgreSQL-native storage, the AQL execution engine, validation, and security. The System architecture chapter walks through this in user terms; if you are new to openEHR itself, start with the openEHR primer.

Where to go next

• Wondering why this exists? Why FerroEHR exists is the project’s position: what openEHR is worth, what we commit to, what the licence lets you do, how building a product on it works, and why contributing back pays.
• Just want to try it? Getting started takes you from docker compose up to a stored composition and an AQL result in a few minutes.
• Deploying it? Installation covers Docker Compose, Kubernetes/Helm, building from source, and the configuration reference; Operations covers running it afterwards.
• Integrating an application? Using the API and Querying with AQL are the core reference for client developers.
• Modelling clinical data? Templates & validation explains how templates drive what the server will accept.
• Reviewing it for a deployment? Security and the Threat model state what is enforced and what is yours to enforce; the FerroEHR Viewer is the optional web UI over the same public API.

Note

FerroEHR began as a fork of EHRbase (by vitasystems and the Peter L. Reichertz Institute) and records that lineage in the labelled import commit at the root of its history, but it is an independent, from-scratch Rust implementation with no EHRbase code in this tree, and it is not affiliated with or endorsed by the EHRbase project. FerroEHR’s own code is source-available under the Business Source License 1.1; vendored openEHR material keeps its upstream terms: Apache-2.0 for the machine-readable artifacts, CC-BY-SA 3.0 for the specification text, and CC-BY-SA 3.0 and 4.0 for the clinical models (see Licensing & legal).

Why FerroEHR exists

This chapter is the project’s position: what makes openEHR worth implementing, what this implementation commits to, and what it offers the organisations that run it and build on it. Read it if you are deciding whether to depend on FerroEHR, or whether to contribute to it.

• openEHR is worth building for
• A specification is only as strong as the implementations you can run
• What we commit to
• Running FerroEHR commercially
• What we ask in return
• What contributing back can look like
• What this is not

openEHR is worth building for

openEHR does something almost nothing else in health IT does: it separates clinical knowledge from software, and then writes both down. What a blood pressure, a medication order or a discharge summary means lives in archetypes and templates authored by clinicians and modellers: published and computable. The Reference Model underneath them is specified. So is the query language, the REST interface, and the serialization, down to the shape of the JSON on the wire.

The consequence is determinism. Given the same template and the same composition, two conformant systems store the same record and answer the same AQL query the same way. A clinical record stops being one application’s private state and becomes data that outlives the application, the vendor and the procurement cycle. For a record that has to stay readable in twenty years, by software nobody has written yet, that is the whole game.

A specification is only as strong as the implementations you can run

A standard becomes real when there is something you can start with one command, read the source of, check against the specification yourself, and run for research, teaching or a non-profit deployment without a fee. Without that, an excellent specification stays an idea that only well-funded organisations can act on.

That is the gap this project set out to close: one complete, openly developed, source-available openEHR CDR whose conformance is measured and published.

What we commit to

• One licence for all of our own code, with no open-core tier. Role- and attribute-based access control, IHE ATNA audit, per-version digital signatures, change events and the viewer are in one repository under the Business Source License 1.1. Nothing is held back to be sold back to you. (Vendored openEHR material keeps its own upstream terms, and the spec crates that embed it say so in their own metadata; see Licensing & legal.)
• Every claim checkable. Conformance is executed by a runner against a live server, and the run records, per-case results and measured performance are committed to the repository. If a number appears on this site, the record it came from is in the tree, and a change that moves a verdict cannot land quietly.
• The specification is the authority. The normative openEHR text is vendored in the repository and cited decision by decision. Where we find it silent or self-contradictory, the finding is filed in public and reported upstream.
• The specification layer as reusable libraries. The generated openEHR model, the canonical codecs, the REST contract, the ADL engine and the AQL parser are published on crates.io as eight standalone crates, so the next Rust project does not have to re-model openEHR to get started. The five generated model crates are under the Apache License 2.0, the licence of the openEHR artifacts they are generated from, and need no licence conversation at all.
• Maintenance in the open. Public roadmap, public issue tracker, changelog-driven releases, signed artifacts, and a security policy with a private reporting channel.
• A compliance posture you can read before you buy. FerroEHR aims to be the first openly developed, source-available openEHR CDR with a published, tracker-backed EU compliance posture, and an EHDS conformity self-assessment is on its roadmap. The compliance overview states which controls ship today, which are planned and under which issue number, and which obligations stay with the organisation running the software. It claims no certification and no conformity, because a product cannot hold either on its own.

Running FerroEHR commercially

FerroEHR is source-available under the Business Source License 1.1, which is not an OSI-approved open-source licence. One licence covers the whole repository, and no feature is held back for a paid tier.

The last two rows hold whatever else you are: they need a commercial licence even for an organisation the rows above would otherwise leave free. Each version becomes Apache 2.0 four years after it is published, so what ships today opens up on a published schedule. Licensing & legal carries the full picture, including every vendored third-party tree.

Building products on FerroEHR is welcome, and it is how standards reach patients. Companies and care providers running FerroEHR are wanted here, and the commercial licence is the normal path for them, on terms meant to make those deployments and products possible on a sustainable footing. It is also how the shared, conformant foundation gets maintained by the people who build on it instead of every vendor rebuilding one in private. A commercial licence is arranged with Cadasto B.V., the Licensor, which handles the business side of FerroEHR: write to info@cadasto.com or use https://www.cadasto.com/contact/. Technical questions go to the maintainer named in MAINTAINERS.md. Talk to us early.

What we ask in return

Contribute back.

The licence does not oblige you. The arithmetic does:

• A private fork is the expensive option. Fork it and you inherit the whole maintenance surface: specification releases, security advisories, database upgrades, re-running conformance, and re-merging your changes at every release, forever. Upstream the same change and it is maintained once, by everyone who runs it.
• A defect found once should be fixed everywhere. In clinical software the validation gap you patched privately is still live in every other deployment of the same code. Sharing the fix is the difference between one organisation being safe and all of them being safe.
• Interoperability is a property of the population, not of any single implementation. Every conformance case contributed, and every ambiguity resolved in the open, makes it likelier that your system and the next one actually agree about a record.

When the same fix is made privately in five places, the standard is no stronger and five teams have paid for it. Nobody chose that outcome, and it is an easy one to avoid.

What contributing back can look like

You do not have to write Rust to make this project better:

• A bug report with the request that reproduces it, the most valuable thing most users will ever send.
• A conformance case for behaviour the catalogue does not cover yet, so the next release cannot regress it.
• A specification finding: the ambiguity you had to resolve in order to ship. We adjudicate those against the vendored text and report the genuine ones upstream to the openEHR Foundation.
• Documentation: a correction, deployment experience from your environment, or the paragraph that would have saved you two days.
• Measurement from your own hardware and workload. The instruments that produce our published numbers ship in the repository and run against any server, including one that is not FerroEHR.
• Code: fixes, features, connectors, packaging for your platform.
• Sponsorship, if your organisation depends on this and cannot spare engineering time.

Contributing is the practical starting point. Security issues have their own private channel; please use it rather than the public tracker.

What this is not

This is not an argument against commercial or proprietary software. A product with FerroEHR inside it, or a care provider running it for its patients, is a good outcome, and we would rather those succeed than not exist at all; a commercial licence is how they and this foundation support each other.

It is also not a claim to be the only good openEHR CDR. FerroEHR began as a fork of EHRbase and records that lineage in the labelled import commit at the root of its history; today it is an independent Rust implementation with none of that code left in the tree. FerroEHR is an independent implementation of the openEHR® specifications and is not affiliated with or endorsed by the openEHR Foundation.

And it is not a large team. The project is maintained by one person today, with the machine gates (conformance, the fidelity suites, the CI guards) standing in for review capacity a bigger group would have. The threat model and the repository’s own governance and maintainer documents say so plainly.

The same honesty covers how the code gets written: substantial parts of FerroEHR are built with AI coding tools, directed and reviewed by the maintainer, with every change held to the machine-enforced gates described above. The full statement — what that means, what bounds it, and what you can verify instead of trusting it — is the repository’s AI statement.

Maintained by Ruben Talstra and the FerroEHR contributors. If your organisation runs FerroEHR, or is thinking about building a product on it, we would like to hear about it early: open an issue or say hello on the tracker.

Getting started

This chapter takes you from nothing to a running server with a template loaded, a clinical composition stored, and an AQL query returning results in a few minutes, using Docker Compose. It is the fastest way to see FerroEHR work end to end and to get a feel for the API before reading the reference chapters. Everything here uses the built-in development credentials; do not use them outside local evaluation.

Warning

The steps below enable Basic auth with the throwaway user ferroehr / ferroehr, leave role-based access control off (so that one user reaches every enabled surface, admin API included), and use a permissive CORS policy. This is a development configuration only. See Security and the configuration reference before exposing a server.

1. Start the stack

You need Docker with the Compose plugin (2.23.1 or newer). Download docker-compose.yml (attached to every release) into an empty directory and start it:

docker compose up

This pulls and starts two services: the server (ferroehr) on port 8080, and a preconfigured PostgreSQL 18 database. The server runs its schema migrations automatically on first boot, so the database is ready as soon as it reports healthy. Nothing else is needed: the server’s configuration travels inside the Compose file, which also ships one Basic-auth user (ferroehr / ferroehr, holding both the ADMIN and USER roles) so the API authenticates out of the box.

Published ports bind 127.0.0.1 by default, so the stack is reachable from this machine and not from the network. Three things are optional and stay down until you ask for them: the viewer (docker compose --profile viewer up, then http://localhost:3000), a SeaweedFS S3 gateway for multimedia (--profile s3), and a ready-made Keycloak identity provider for bearer-token auth (a second downloadable overlay, docker-compose.keycloak.yml). See Docker Compose for all of them.

2. Probe the status endpoint

The status endpoint is public and confirms the server is up:

curl http://localhost:8080/ferroehr/rest/status

It answers a small JSON document: status, server_version, openehr_rest_api_version, a timestamp, the licence in force and the declared deployment profile. All clinical API routes live under the base path /ferroehr/rest/openehr/v1. Interactive OpenAPI documentation is served at http://localhost:8080/ferroehr/rest/swagger-ui; sign in with the API credential when the browser asks (the UI is served to authenticated users by default, see rest.swagger_ui). If you have no server yet, use the hosted sandbox: https://sandbox.ferroehr.eu opens the viewer over a live CDR, and the same server’s Swagger UI is at https://sandbox.ferroehr.eu/ferroehr/rest/swagger-ui. Both take the public demo credentials ferroehr / ferroehr. The sandbox is a shared demo server that resets every night, so treat anything you write there as temporary and expect it to slow down under heavy use.

What is already in the sandbox

The nightly reset reloads a fixed demo dataset, so the server always answers with something to look at:

• 8 EHRs, each with a mixed record and its own patient. The sandbox resolves patients through FerroPIX, the FerroHEALTH patient identity source, as the compose quickstart does: the patients are sandbox-patient-01 … -08 in the synthetic hospital domain urn:oid:2.999.1.1, and each EHR_STATUS names its patient by the pseudonym FerroPIX minted in the clinical domain urn:oid:2.999.7.1. POST /query/cohort with subjects named in the hospital domain finds their EHRs through FerroPIX. Two of them carry a second EHR_STATUS version: patient 07 is not queryable and so never appears in AQL results, patient 08 is not modifiable and refuses writes with a 409.
• 183 compositions across 17 ADL 1.4 templates: sixteen from the openEHR CKM (vital signs, problem and medicines lists, lab results, referrals, an International Patient Summary, several statutory case-report forms) and one terminology-binding template whose coded text resolves against the terminology server running beside the CDR. Six compositions have more than one version, so LATEST_VERSION and ALL_VERSIONS differ on real data.
• 233 ADL 2 archetypes and 5 ADL 2 templates, browsable under extensions/v1/definition/archetype/adl2 and openehr/v1/definition/template/adl2. Two of the templates are ADL 2 source templates whose slots the server flattens against that archetype library, and the compositions committed from them came out of the server’s own …/example generator.
• 11 demographic parties (persons, roles, party relationships) and a FOLDER directory in every EHR whose items reference real compositions.
• 5 stored AQL queries under eu.ferroehr.sandbox, all of which return rows: composition_index, blood_pressure, case_reports, composition_versions, and patient_record (takes an $ehr_id parameter). Run one with GET /query/eu.ferroehr.sandbox::composition_index, or open the viewer’s query screen.

There are also three always-on, unauthenticated health endpoints: /health, /health/liveness and /health/readiness; the last one reports each dependency it checked. See Operations → Health probes.

3. Create an EHR

An EHR is the container for one subject’s records. Create one with a POST (no body needed):

curl -u ferroehr:ferroehr -X POST -i \
  http://localhost:8080/ferroehr/rest/openehr/v1/ehr

The -i flag shows the response headers. On success you get 201 Created; the new EHR’s identifier is in the ETag header (as the weak form W/"<ehr_id>"), and Location points at the created resource. Copy the UUID; the examples below refer to it as EHR_ID.

By default the response body is empty. Add -H 'Prefer: return=representation' to have the server return the full EHR object instead, or -H 'Prefer: return=identifier' for just the uid; either way Preference-Applied echoes what the server honoured.

4. Upload a template

Before you can store a composition, the server needs the Operational Template (OPT 1.4) that the composition conforms to. Templates are XML documents; upload one with Content-Type: application/xml:

curl -u ferroehr:ferroehr \
  -H 'Content-Type: application/xml' \
  --data-binary @my-template.opt \
  http://localhost:8080/ferroehr/rest/openehr/v1/definition/template/adl1.4

A successful upload returns 201 Created. If you do not have a template to hand, the openEHR community publishes example OPTs (for instance the Vital Signs templates used in the openEHR training material), and the international Clinical Knowledge Manager is the source for the archetypes they are built from. List what is loaded, and inspect a template’s derived WebTemplate (a JSON description convenient for building forms), with:

# List templates
curl -u ferroehr:ferroehr \
  http://localhost:8080/ferroehr/rest/openehr/v1/definition/template/adl1.4

# Fetch the WebTemplate for one template
curl -u ferroehr:ferroehr \
  -H 'Accept: application/openehr.wt+json' \
  http://localhost:8080/ferroehr/rest/openehr/v1/definition/template/adl1.4/my_template_id

See Templates & validation for the full template lifecycle and the WebTemplate/FLAT/STRUCTURED formats.

5. Commit a composition

A composition is one clinical document, stored inside an EHR and validated against its template. Post the composition JSON (its archetype_details name the template it belongs to):

curl -u ferroehr:ferroehr \
  -H 'Content-Type: application/json' \
  -H 'Prefer: return=representation' \
  --data-binary @my-composition.json \
  http://localhost:8080/ferroehr/rest/openehr/v1/ehr/$EHR_ID/composition

On success you get 201 Created and (because of Prefer: return=representation) the stored composition in the body, now carrying a server-assigned version identifier in its uid. If the composition does not conform to its template you get 422 Unprocessable Entity with the validation errors; a malformed request gets 400 Bad Request. The composition walkthrough in Resource walkthroughs covers update and delete, which use the If-Match header for optimistic concurrency.

6. Query with AQL

Now query across the data with the Archetype Query Language. The simplest query lists the EHR ids the server holds:

curl -u ferroehr:ferroehr \
  -H 'Content-Type: application/json' \
  -d '{"q":"SELECT e/ehr_id/value FROM EHR e"}' \
  http://localhost:8080/ferroehr/rest/openehr/v1/query/aql

The response is a RESULT_SET: a columns array describing each selected value and a rows array of result tuples. To pull values out of the compositions you committed, select by their archetype path, for example every systolic blood pressure above 140:

curl -u ferroehr:ferroehr -H 'Content-Type: application/json' -d '{
  "q": "SELECT o/data[at0001]/events[at0006]/data[at0003]/items[at0004]/value/magnitude AS systolic FROM EHR e CONTAINS COMPOSITION c CONTAINS OBSERVATION o[openEHR-EHR-OBSERVATION.blood_pressure.v2] WHERE o/data[at0001]/events[at0006]/data[at0003]/items[at0004]/value/magnitude > 140"
}' http://localhost:8080/ferroehr/rest/openehr/v1/query/aql

Querying with AQL is the full language guide — parameters, stored queries, version scope, terminology, pagination, and the supported feature set.

7. Explore the API interactively

Open http://localhost:8080/ferroehr/rest/swagger-ui (signing in with the API credential when the browser asks) to browse and try every endpoint from your browser. The UI’s spec selector carries one entry per API family: the standardised openEHR groups (EHR, Query, Definition, Demographic, Admin) and the server’s own extensions (status & management, terminology, party relationships, messaging, event subscriptions, audit, SMART discovery), plus FerroEHR: Complete surface last, which is the whole server in one document. Every entry is filtered from that same document, which the server generates from its own handlers, so nothing here can drift from the routes it actually serves. When authentication is enabled the “Authorize” dialog shows the one scheme the server is configured for (HTTP Bearer/JWT when OIDC is set up, otherwise HTTP Basic). The hosted sandbox runs the same UI at https://sandbox.ferroehr.eu/ferroehr/rest/swagger-ui if you want to try it before running anything locally — and https://sandbox.ferroehr.eu itself opens the viewer on that server, with the demo credentials ferroehr / ferroehr.

Next steps

• Installation — running it for real (Compose, Kubernetes, or from source) and the configuration reference.
• Using the API — the per-resource reference with headers, status codes, and versioning.
• Operations — migrations, probes, observability and upgrades, plus the admin & messaging APIs.
• Concepts — the openEHR model and how FerroEHR is built, if the terms above were unfamiliar.

Installation

FerroEHR is one self-contained binary that connects to a PostgreSQL 18 database. There is no application server to install and no language runtime to provision: the binary links a pure-Rust TLS stack and needs no OpenSSL and no JVM, so all you choose is how to run it and where the database lives. This part covers the three paths and the full configuration surface.

• Try it in Codespaces: one click boots the published images in a GitHub Codespace in your browser, with the Swagger UI and the viewer forwarded. Nothing to install. Made for evaluating FerroEHR; a real deployment uses one of the paths below.
• Docker Compose: the fastest way to run the server plus a preconfigured PostgreSQL 18, for development and evaluation. One downloadable file that pulls the published images and needs no configuration. Optional viewer, OIDC, and observability overlays.
• Kubernetes & Helm: the production path, a hardened non-root workload that connects to an externally managed PostgreSQL 18. Its companion, Cluster hardening, covers what the chart cannot do for you.
• From source: building the binary yourself with the pinned Rust toolchain, and running the test suite.
• Configuration reference: how configuration loads and what every key means, split by area: server, database and telemetry, authentication and access, integrations, audit and subject proxy, and the CLI and production checklist.

What every path has in common

One configuration file, with environment overrides on top. The server reads a single ferroehr.toml covering every subsystem; FERROEHR__* environment variables override individual keys, and repeatable --set key=value flags override those. Anything the environment grammar cannot spell (the Basic-auth user store, which is an array of tables) is file-only. ferroehr config default writes an annotated template with every key at its default, and ferroehr config check validates the result without touching the database.

The schema is the binary’s, and who applies it is a choice. By default (db.migrate = "apply") the server applies its embedded migrations at boot, so an empty database self-provisions. Setting db.migrate = "verify" makes the server issue no DDL at all (it checks the schema and refuses to start if it is not this build’s) which lets the runtime role hold no DDL rights. Something else then runs ferroehr db migrate under a migrator role first. That check still reads every schema’s migration state, so a least-privilege deployment also sets db.migrate_url to the credential that can. Both postures are laid out in Operations → Applying migrations.

A deployment declares what it may hold. The top-level deployment_profile key is sandbox by default: the server starts whatever separations are missing and names every one of them on the banner, in the log and on GET /management/status. Set it to production and the server refuses to start while a separation is open and not accepted by name in deployment_accepts. Read deployment_profile before you point a deployment at real patient data.

Choosing a specification generation

One top-level key, spec_profile, selects which openEHR specification generation set the deployment runs: development (the default: Reference Model 1.2.0 with BASE 1.3.0 and LANG 1.1.0) or stable (the latest released generations: Reference Model 1.1.0 with BASE 1.2.0 and LANG 1.0.0). It is a single coupled choice, and stable → development is always safe while the reverse is not, so decide it before you commit clinical data. The full semantics, defaults, and the direction contract are in spec_profile.

Note

A Clinical Data Repository stores PHI. In production the database must be an externally managed, backed-up, point-in-time-recoverable PostgreSQL 18, never a throwaway sidecar. The Kubernetes chart deliberately ships no in-cluster database for this reason.

Try it in Codespaces

The fastest way to try FerroEHR is a GitHub Codespace: one click boots the published images in your browser, with nothing installed on your machine.

• What you get
• What it is, and what it is not
• The hosted sandbox
• If the stack is not up

What you get

Creating a Codespace on the FerroEHR repository starts a container that pulls the published quickstart images and runs docker compose up for you:

• the FerroEHR server with a preconfigured PostgreSQL 18,
• the viewer,
• the Swagger UI for the full REST API.

The stack boots automatically. When the terminal prints FerroEHR is up., open the PORTS panel and follow the forwarded ports:

Sign in with the quickstart credentials, ferroehr / ferroehr. From there the Getting started walkthrough applies unchanged: create an EHR, upload a template, commit a composition, and query it back with AQL, either from the Swagger UI or with curl in the Codespace terminal.

What it is, and what it is not

The Codespace is a tester sandbox and always runs the published release images pinned in the standalone docker-compose.yml. It does not build the checkout you opened it on: the COMPOSE_FILE environment variable inside the container pins every docker compose command to that file, so the repository’s development override (which switches to from-source builds) does not apply. To develop FerroEHR itself, use a local checkout as described under Repository development.

The Codespace runs on your own GitHub account. The smallest machine type (2 cores, 8 GB) is enough, and GitHub’s free monthly Codespaces allowance covers a long evaluation. Stop or delete the Codespace when you are done; a stopped Codespace restarts the stack automatically on resume.

The hosted sandbox

A public demo runs at https://sandbox.ferroehr.eu as the second zero-install path: no GitHub account needed, point any REST client at it with the demo credentials ferroehr / ferroehr. The Swagger UI is at https://sandbox.ferroehr.eu/ferroehr/rest/swagger-ui. So everyone knows what it runs on and what to expect from it:

The server does not scale to zero, so there is no cold start; it is still a single small machine shared by every visitor. It is a demo, never a place for real data.

The sandbox image is pinned to the latest release tag, so it always runs a released FerroEHR rather than a development snapshot.

Create, change and delete whatever you like: the nightly reset returns the sandbox to a small seeded corpus (a handful of demo EHRs with example compositions from published CKM templates), so nothing you do needs cleaning up and nothing you store survives the night.

If the stack is not up

The boot log is in the terminal that ran start-stack.sh. To restart the stack by hand:

bash .devcontainer/start-stack.sh

docker compose ps shows the three services; the server is healthy when curl http://localhost:8080/health answers 200.

Docker Compose

Docker Compose is the quickest way to run FerroEHR together with a preconfigured PostgreSQL 18, for local development and evaluation. The quickstart is one file and zero configuration: download docker-compose.yml, run docker compose up, and the published images are pulled and started: no repository checkout, no bind mounts, no environment variables. This chapter describes the three published images, the services, the authentication posture the quickstart ships with, the optional profiles and overlays, and the variables that tune them. For a step-by-step first run, see Getting started.

Note

This chapter applies equally to the standalone docker-compose.yml downloaded into an empty directory and to a bare docker compose up in a repository checkout — both run the published images. Building from source is an explicit opt-in (-f docker-compose.yml -f docker-compose.dev.yml) — see Repository development below.

• The three images
• Bringing up the stack
• Container engines: Docker and Podman
• The isolation posture
• The quickstart’s authentication posture
• The patient identity source (FerroPIX)
• The local EHR index
• The OIDC variant (Keycloak overlay)
• The terminology overlay (FerroTERM)
• Optional services (Compose profiles)
• Port already in use?
• Connecting to the database
• Variables the compose files read
• Repository development
• Build provenance
• Observability overlay
• Next

Note

The quickstart file carries the server configuration inline (a Compose configs entry with content:), which requires Docker Compose 2.23.1 or newer. Check with docker compose version.

The three images

FerroEHR publishes three container images to GHCR:

Each image is published under several tags:

The quickstart Compose file pins the exact release version it shipped with on all three images, so a downloaded file always runs one known-good, mutually compatible set; a CI guard (scripts/checks/compose-image-tags.sh) fails the build whenever those pinned tags disagree with the version being released, so they cannot be forgotten at a cut. To run something else, set the image variables in the table below.

The role the server connects as in the quickstart owns the database and is a member of ferroehr_migrator and ferroehr_clinical, which is what lets it apply migrations at boot, and of ferroehr_clinical and ferroehr_party, so the schema separation is exercised on one credential. That single credential is deliberate: one container with one DSN cannot demonstrate the credential separation honestly.

A least-privilege deployment sets db.migrate = "verify", runs ferroehr db migrate out of band under the migrator DSN, and serves on a narrow runtime credential. It also has to name the credential that prepares the schema, in [db] migrate_url: preparation reads all five _sqlx_migrations bookkeeping tables even under verify, which no least-privilege role can do, ferroehr_clinical included. Unset, migrate_url falls back to [db] url, which is what the quickstart runs. See Operations → Which credential prepares the schema and Applying migrations.

The PostgreSQL image is init-scripts only: it creates roles, schemas and btree_gist, and bakes in no migration state. That extension is the only one the schema needs, and the server installs it itself at boot (CREATE EXTENSION IF NOT EXISTS btree_gist), so a plain PostgreSQL 18 with a superuser-capable migration credential serves just as well. The server owns the schema content and applies its migrations idempotently at every boot, so a fresh database self-provisions and a restart is a no-op.

Note

PostgreSQL init scripts run only when the data volume is empty. If you see startup notices like skipping role creation (no CREATEROLE privilege) or roles absent, your volume predates the image’s role setup (or you are running a plain postgres image): either recreate the volume (docker compose down -v, which destroys data) or create the group roles once by hand as a superuser (Operations has the statements). The server runs fine either way: the grants are a defense-in-depth layer, not a functional requirement.

Bringing up the stack

Download docker-compose.yml (attached to every release) into an empty directory and start it:

docker compose up

This pulls and starts the core services (no profile needed): the database and the server below, and the patient identity source described in The patient identity source (FerroPIX).

• ferroehr-postgres: the database image, with a named data volume, a pg_isready healthcheck, pg_stat_statements preloaded, and a modest tuning floor (shared buffers, WAL size, work memory, lz4-compressed WAL full-page images — which cut per-commit WAL volume roughly in half — and a larger /dev/shm than Docker’s default, which otherwise starves parallel workers).
• ferroehr: the server, which waits for the database to report healthy (depends_on: condition: service_healthy), then boots, migrates, and serves on port 8080. Its healthcheck is the binary’s own healthcheck subcommand, because the image has no shell.

The API is then at http://localhost:8080/ferroehr/rest/openehr/v1, with Swagger UI at http://localhost:8080/ferroehr/rest/swagger-ui (the browser asks for the API credential; rest.swagger_ui chooses who may read it).

Important

Every published port binds the loopback interface by default (127.0.0.1:HOST:CONTAINER), and that is a security property rather than a style choice: a published port is DNAT’d by rules Docker inserts ahead of the host firewall’s own chains, so a ufw deny 8080 does not stop a port published on 0.0.0.0. To reach the stack from another machine, name the interface explicitly (FERROEHR_BIND_HOST=0.0.0.0 docker compose up, or a single address). The database publishes no host port at all — see Connecting to the database.

Note

Third-party base images are pinned by digest (name:tag@sha256:…), not by a mutable tag, so a pull always resolves the exact same image. Each service also declares a CPU and memory ceiling (deploy.resources.limits), so a local stack cannot exhaust the host; the server’s and the database’s are overridable for constrained machines.

Warning

PostgreSQL 18’s official image stores data in a major-version subdirectory, so the data volume mounts at /var/lib/postgresql (the parent), not the pre-18 /var/lib/postgresql/data. The bundled Compose file already does this correctly; keep the convention if you adapt it.

Container engines: Docker and Podman

The quickstart runs on Docker and on Podman. Both were verified first-hand on 2026-08-24: the core stack, the viewer and s3 profiles, and both overlays boot to healthy under podman compose exactly as under docker compose, with the same commands:

podman compose up -d --wait

Podman-specific notes:

• podman compose delegates to an external Compose provider. With Docker’s own Compose CLI installed (the common case on a machine that has ever run Docker Desktop) you get identical Compose behaviour; with the Python podman-compose package, flag coverage differs — prefer the Docker Compose provider.
• The default podman machine on macOS ships with 2048 MiB of memory. That is enough for the quickstart and both overlays, but not for building the server image from source inside the VM (a single spec-crate compile at release optimization holds more than that). If you want the build-from-source developer path under Podman, resize the machine first: podman machine set --memory 8192.
• Building from source under Podman (Buildah) is not part of the verified quickstart surface; the supported quickstart path is the published images.

The repository’s own CI and measurement lanes run Docker; the compose-driven scripts (scripts/conformance.sh, the deployment probe, the UI journey battery) are exercised against Docker only.

The isolation posture

Every service in every committed compose file drops all Linux capabilities and adds back only what its entrypoint provably needs, sets no-new-privileges:true, bounds its file descriptors, restarts unless stopped, and keeps Docker’s default seccomp profile (the posture is the absence of an unconfined override, so that absence is what a CI guard checks). No service is privileged and none mounts the Docker daemon socket.

Most services additionally run a read-only root filesystem with a tmpfs for the paths that must be writable. Two do not, and each says why in place: the S3 gateway writes its volume store, and the server container hits a real mutual exclusion rather than an oversight: Compose refuses an inline config in a read-only service, and that inline config is what makes this file standalone. A deployment that mounts its configuration from a file instead can add read_only: true and a tmpfs at /tmp. On Kubernetes the conflict does not arise: the configuration arrives as a mounted ConfigMap and the chart sets readOnlyRootFilesystem: true.

The quickstart’s authentication posture

The server configuration travels inside the Compose file and is written for evaluation, not for production:

• Basic auth with one user: ferroehr / ferroehr (stored as an Argon2id hash), carrying the ADMIN and USER roles.
• RBAC disabled: so any authenticated caller may use every enabled surface, including the admin API. The ADMIN / USER / READONLY separation is switched on by setting [authz.rbac] enabled = true and giving each user an explicit roles list; see Security.
• Admin API enabled: so the optional viewer’s panels work.
• Management introspection enabled, with info and metrics at private, prometheus at public, and env and loggers at admin_only.
• Permissive CORS: any origin may call the API from a browser.
• No TLS: plain HTTP on port 8080.
• The sandbox deployment profile, declared in the inline configuration: the binary’s default, stated so the file says what it runs. The server names every separation it has not made on the boot banner, in the log and on GET /management/status, and the viewer raises a notice saying the deployment must not hold real patient data. That is the quickstart telling the truth about itself, not a fault. See deployment_profile.

Warning

These are development credentials and development defaults. Before exposing a server, replace the user store (or point it at an identity provider), turn RBAC on, restrict CORS, close the admin API, and terminate TLS in front of the server. The configuration reference and Security cover each of these.

To change any of it without editing the Compose file, add FERROEHR__* variables to the ferroehr service’s environment: block; the environment layer wins over the inline file. The Basic-auth user store is the one setting the environment cannot express (it is an array of tables), so replacing the users means editing the inline config or mounting your own TOML file.

The patient identity source (FerroPIX)

FerroEHR keeps no patient-to-record cross-reference of its own by default. It asks an IHE patient identity source over PIXm (ITI-83) which pseudonym the clinical domain holds for a patient ([ehr_index]). The quickstart runs FerroPIX as that source, pinned by tag and image-index digest (CI runs the config check of every pinned image over its inline configuration), with four services beside the server:

• identity-issuer: a development OpenID Provider (NAV’s mock-oauth2-server) that signs any client credentials request, at http://localhost:8180/quickstart. It is not for anything real.
• ferropix: the PIX Manager, at http://localhost:8090/fhir. Its configuration declares the pseudonymisation domain urn:oid:2.999.7.1 with FerroEHR’s client ferroehr as its consumer, so it mints a pseudonym the first time FerroEHR asks, and a synthetic identifier domain urn:oid:2.999.1.1 fed by the client quickstart-source.
• ferropix-postgres and ferropix-key: FerroPIX’s own database, and a one-shot job that writes its sealing key into a volume, so the file carries no key.

The server’s [ehr_index] names the endpoint and the grant, and [privacy] subject_namespaces declares the clinical domain. The server reads FerroPIX’s CapabilityStatement at boot and does not start until it answers.

The issuer, FerroPIX and FerroEHR share the issuer’s network namespace. A server accepts a plain-http issuer only on loopback (RFC 8414 §2), and the mock issuer writes the host it was asked on into every token, so all three reach it at http://localhost:8180. The issuer service therefore publishes the ports of all three and answers to the names ferroehr and ferropix on the compose network.

To register a synthetic patient and ask for the pseudonym by hand:

SRC=$(curl -s -d grant_type=client_credentials -d client_id=quickstart-source \
  -d client_secret=any -d 'scope=system/Patient.cud' \
  http://localhost:8180/quickstart/token | jq -r .access_token)
curl -s -X PUT -H "Authorization: Bearer $SRC" \
  -H 'Content-Type: application/fhir+json' \
  'http://localhost:8090/fhir/Patient?identifier=urn:oid:2.999.1.1%7CH-1' \
  -d '{"resourceType":"Patient","identifier":[{"system":"urn:oid:2.999.1.1","value":"H-1"}],"name":[{"family":"Synthetic"}]}'

Any conformant PIX Manager can take FerroPIX’s place: point pixm_url, assigning_authority and [ehr_index.oauth] at it.

The local EHR index

The other supported posture keeps the subject-to-EHR cross-reference in FerroEHR ([ehr_index]): EHR_STATUS.subject names a patient by its own namespace and identifier, and GET /ehr?subject_id=…&subject_namespace=… finds the EHR by them. One variable selects it, with no copy of the configuration to make:

FERROEHR_EHR_INDEX=local docker compose up

The server then runs the file’s ferroehr-config-local block, which differs from ferroehr-config in [ehr_index] and [privacy] alone: the local map, and no pseudonym namespace, so a subject reference is stored as the client sends it:

curl -u ferroehr:ferroehr -X POST -i -H 'Content-Type: application/json' \
  http://localhost:8080/ferroehr/rest/openehr/v1/ehr -d '{
  "_type": "EHR_STATUS",
  "archetype_node_id": "openEHR-EHR-EHR_STATUS.generic.v1",
  "archetype_details": {"_type": "ARCHETYPED", "rm_version": "1.1.0",
    "archetype_id": {"_type": "ARCHETYPE_ID", "value": "openEHR-EHR-EHR_STATUS.generic.v1"}},
  "name": {"_type": "DV_TEXT", "value": "EHR Status"},
  "subject": {"_type": "PARTY_SELF", "external_ref": {"_type": "PARTY_REF",
    "namespace": "benchmark", "type": "PERSON",
    "id": {"_type": "GENERIC_ID", "value": "1793131381", "scheme": "benchmark"}}},
  "is_queryable": true, "is_modifiable": true}'
curl -u ferroehr:ferroehr -i \
  'http://localhost:8080/ferroehr/rest/openehr/v1/ehr?subject_id=1793131381&subject_namespace=benchmark'

The identity services still start, and nothing asks them. Start without the variable to return to FerroPIX; the EHRs created on the local index keep their subject references, and the database refuses no stored row.

The OIDC variant (Keycloak overlay)

A second standalone file adds a ready-made identity provider, so you can exercise bearer-token authentication without registering a client anywhere. Download docker-compose.keycloak.yml from the same release, beside the base file, and stack the two:

docker compose -f docker-compose.yml -f docker-compose.keycloak.yml up

That starts Keycloak with a small demo realm (ferroehr) defined inline and points the server’s bearer validation at it. Basic auth from the base file keeps working, so the server then advertises Basic, Bearer.

The server accepts a token issuer only over HTTPS, or plain HTTP on a loopback host, so Keycloak serves the issuer over TLS at https://keycloak.localhost:8443/auth/realms/ferroehr. The containers resolve keycloak.localhost through a network alias on the compose network, and a browser resolves every *.localhost name to the loopback address, where the published port lands, so both sides use one issuer URL with no hosts entry. On the first up, a one-shot keycloak-tls service generates a development CA and a certificate for that name into the keycloak-tls volume and discards the CA’s private key; the server ([auth] ca_file) and the viewer ([auth.oidc.tls] trust_roots_file) trust that CA beside the public ones. docker compose down --volumes deletes the volume, and the next up generates a new CA. Keycloak also answers plain HTTP on port 8081 for the token call below, and the tokens it mints there name the HTTPS issuer. Keycloak’s access tokens carry typ: JWT, so the overlay sets FERROEHR__AUTH__TOKEN_PROFILE=jwt.

Start the viewer too and sign in at http://localhost:3000 as any user below:

docker compose -f docker-compose.yml -f docker-compose.keycloak.yml --profile viewer up

The viewer takes the authorization-code flow with PKCE through the ferroehr-viewer client and calls the server with the user’s token. Your browser does not trust the development CA, so it warns once on the Keycloak page: accept the warning, or trust the CA, which docker compose cp keycloak:/opt/keycloak/conf/tls/ca/ca.crt . copies out. A browser that does not resolve *.localhost needs 127.0.0.1 keycloak.localhost in its hosts file.

Every route checks SMART on openEHR scopes, so the realm puts them in every token: user/composition-*.cruds, user/template-*.cruds and user/aql-*.cruds grant a user every resource family, the service client gets their system/ twins, and __ferroehr.platform is the product scope the ADMIN and DEMOGRAPHIC APIs need. The roles below then decide.

The overlay also turns RBAC on, deliberately: an identity provider whose roles decide nothing is a login screen, not authorization. So the realm ships four users whose roles are load-bearing:

Two confidential clients are defined, and both put ferroehr, the audience the server checks, in every access token:

Fetch a user’s token and call the API with it:

TOKEN=$(curl -s -d client_id=ferroehr-viewer -d client_secret=ferroehr-viewer-quickstart-secret \
  -d username=ferroehr -d password=ferroehr -d grant_type=password \
  http://localhost:8081/auth/realms/ferroehr/protocol/openid-connect/token \
  | jq -r .access_token)

curl -H "Authorization: Bearer $TOKEN" -X POST -i \
  http://localhost:8080/ferroehr/rest/openehr/v1/ehr

A service with no user behind it takes the client-credentials grant with ferroehr-service. Its service account holds USER, so the token reaches the clinical API and the admin routes refuse it with 403. Its system/aql-* grant counts because the overlay names the client in [smart] backend_clients (FERROEHR__SMART__BACKEND_CLIENTS):

TOKEN=$(curl -s -d client_id=ferroehr-service -d client_secret=ferroehr-service-quickstart-secret \
  -d grant_type=client_credentials \
  http://localhost:8081/auth/realms/ferroehr/protocol/openid-connect/token \
  | jq -r .access_token)

Warning

The realm, the client secret, the user passwords and the CA are demo values, served by a Keycloak in start-dev mode. For a real deployment, drop this overlay and point [auth] (or FERROEHR__AUTH__ISSUER and FERROEHR__AUTH__AUDIENCE) at your own issuer over HTTPS.

The terminology overlay (FerroTERM)

FerroTERM is the Ferro family’s FHIR terminology server: R4, R4B and R5 endpoints over a precomputed index, no JVM, no database, one distroless image. The overlay docker-compose.terminology.yml starts it beside the CDR and switches the CDR’s terminology server on, so archetype value-set bindings resolve at commit and AQL TERMINOLOGY() expands through it. It also switches on the read-only /terminology/* extension API (FERROEHR__TERMINOLOGY__API_ENABLED, off by default): a term, subsumption or value-set call there for a terminology the bundled openEHR terminology does not carry is answered by FerroTERM, while listing and describing terminologies stays with the bundle:

docker compose -f docker-compose.yml -f docker-compose.terminology.yml up
curl 'http://localhost:8090/r4b/metadata?mode=terminology'

It is an overlay rather than a profile for the same reason as the Keycloak and observability ones: enabling it has to change the CDR’s environment, which a profile cannot do. FerroTERM’s image carries its own health probe (the binary’s healthcheck subcommand, since the distroless image has no shell), and the CDR is declared to start only once that probe reports healthy, so docker compose up --wait returns with both serving.

Out of the box FerroTERM serves the licence-free shaped seed this repository ships (two code systems and two value sets under the reserved example.test domain, the content the conformance lane binds to), mounted read-only straight out of the ferroehr image. No licensed terminology is distributed with FerroEHR or FerroTERM. To serve a release you hold a licence for, build its index once into the named volume and tell the server to open it:

FERROTERM_RF2=/path/to/SnomedCT_Release.zip \
  docker compose -f docker-compose.yml -f docker-compose.terminology.yml \
  --profile terminology-build run --rm ferroterm-build
FERROTERM_INDEX=/data/index \
  docker compose -f docker-compose.yml -f docker-compose.terminology.yml up

The archive is mounted read-only and unpacked to a tmpfs that dies with the build container. FerroTERM’s loading page covers the other code systems and their build flags.

Three things the operator owns:

• The code-system licence. SNOMED CT needs an Affiliate Licence (free in Member countries, see snomed.org/get-snomed); LOINC, ICD and RxNorm have their own terms. Every surface that shows SNOMED CT content carries the notice its licence prescribes (clause 8.3.1) and the release version and date.
• Who can reach the server. The overlay publishes FerroTERM on the loopback interface at port 8090 (FERROEHR_TERMINOLOGY_PORT) so you can query it while developing. FerroTERM has no authentication and no rate limit of its own; a deployment that serves other machines puts a reverse proxy with both in front, or removes the ports: block and lets the CDR be the only caller, which is what the hosted sandbox does.
• FerroTERM’s licence. BUSL-1.1 from the same Licensor as FerroEHR: non-commercial production use is free, any other production use needs a commercial licence. Its README says which is which.

The CDR keeps its shipped fail-open posture: a binding the server cannot resolve is accepted. FERROEHR__TERMINOLOGY__SERVER__FAIL_ON_ERROR=true turns an unreachable server into a 422 refusal instead; the terminology page explains both. A code the server resolves as outside its value set is refused with 422 under either posture.

Optional services (Compose profiles)

The core services are profile-less and start on every up. Further services sit behind a Compose profile and stay down until you ask for them:

• ferroehr-viewer (--profile viewer): the viewer on port 3000, pointed at the server inside the Compose network. Start the stack with it:

  docker compose --profile viewer up
  # → http://localhost:3000  (log in with ferroehr / ferroehr)
  

• seaweedfs and seaweedfs-init (--profile s3): an S3 gateway for large DV_MULTIMEDIA externalization (development and test only). Point the server at it and bring the profile up:

  export FERROEHR__MULTIMEDIA__ENABLED=true
  export FERROEHR__MULTIMEDIA__ENDPOINT=http://seaweedfs:8333
  export FERROEHR__MULTIMEDIA__BUCKET=openehr-multimedia
  export FERROEHR__MULTIMEDIA__ALLOW_HTTP=true    # dev only; production S3 is HTTPS
  
  docker compose --profile s3 up -d --wait ferroehr seaweedfs
  

  The compose file passes the whole FERROEHR__MULTIMEDIA__* set through from your shell, so there is no file to edit, and seaweedfs-init creates the bucket by issuing an idempotent CreateBucket against the gateway, which ships with none, and answers an S3 write into a missing bucket with 403 AccessDenied, a reply that reads as a credentials problem and is not one. It reads the same bucket variable the server does, so the two cannot drift.

  To turn it off again, unset those variables (or set FERROEHR__MULTIMEDIA__ENABLED=false) and re-up: an unset variable is removed from the container’s environment rather than passed as empty, so the server falls back to its own default of enabled = false.

  Confirm the server took them with curl -s -u ferroehr:ferroehr http://localhost:8080/management/env | jq .multimedia ("enabled": true and a non-empty endpoint mean the wiring is right). In production, point the multimedia settings at a real, credentialed, HTTPS S3 endpoint instead; see S3 multimedia.

• ferroehr-backup-clinical, ferroehr-backup-party and ferroehr-backup-linkage (--profile backup): one pg_dump job per pseudonymisation domain, run on demand rather than started with the stack:

  docker compose --profile backup run --rm ferroehr-backup-clinical
  docker compose --profile backup run --rm ferroehr-backup-party
  docker compose --profile backup run --rm ferroehr-backup-linkage
  

  Each writes a timestamped custom-format dump into its own host directory (FERROEHR_BACKUP_CLINICAL_DIR, FERROEHR_BACKUP_PARTY_DIR, FERROEHR_BACKUP_LINKAGE_DIR, defaulting to ./backups/clinical, ./backups/demographic and ./backups/linkage). Set FERROEHR_BACKUP_USER="$(id -u):$(id -g)" and the files land owned by you; unset, the job runs as root inside the container with one capability, DAC_OVERRIDE, which is what writing a directory it does not own actually requires.

  Three dumps rather than one is the point: a single whole-database dump holds the clinical record, the identities of its subjects and the map between them, and whoever reads that file re-identifies every record in it. This stack runs one login credential, so it separates the artefacts and not the authority to produce them; Operations → Dump each domain separately is the production procedure, including why a backup credential needs BYPASSRLS.

Two things that used to be profiles of this file are not any more. Both have to change the server’s configuration to be useful, which a profile cannot do, so each is a separate file instead:

• OIDC / Keycloak is the docker-compose.keycloak.yml overlay above (and, in a repository checkout, a keycloak profile of the development override described below).
• A terminology server is the docker-compose.terminology.yml overlay above, FerroTERM beside the CDR; the repository’s conformance stack additionally runs its own seeded server for the acceptance lane, see Terminology servers.

Port already in use?

Every published port is a variable, so a conflict never requires editing any file. When docker compose up refuses with port is already allocated (or address already in use), pick a free port and pass it:

FERROEHR_PORT=8081 docker compose up

The same works for the other ports (FERROEHR_VIEWER_PORT, FERROEHR_PIX_PORT, FERROEHR_ISSUER_PORT, FERROEHR_S3_PORT; PostgreSQL publishes no host port — see Connecting to the database). The defaults stay fixed on purpose: every URL in this book assumes localhost:8080, and Docker’s automatic ephemeral-port allocation exists only when a mapping omits the host port entirely — a server that silently moved ports would break every printed URL, so here you always choose the port and always know it.

Note

On Windows the refusal reads differently: Ports are not available: exposing port TCP 127.0.0.1:8080 … bind: Only one usage of each socket address … is normally permitted when another program holds the port, or … An attempt was made to access a socket in a way forbidden by its access permissions when the port sits in a Windows reserved range. The remedy is the same variable; to see the reserved ranges, run netsh interface ipv4 show excludedportrange protocol=tcp. (The classic Windows collision — a natively installed PostgreSQL service holding 5432 — no longer affects the quickstart, which publishes no database port.)

Connecting to the database

The quickstart publishes no PostgreSQL host port: the server reaches the database over the compose network, so nothing needs one, and a published 5432 collided with natively installed PostgreSQL (on Windows the installer registers an auto-started postgresql-x64-* service). For a psql session, no port is needed at all:

docker compose exec ferroehr-postgres psql -U postgres -d ferroehr

For a GUI client on the host (pgAdmin, DBeaver), add the db-publish overlay, downloaded from the same release beside the base file:

docker compose -f docker-compose.yml -f docker-compose.db-publish.yml up

It publishes 127.0.0.1:5432 (retune with FERROEHR_DB_PORT, widen with FERROEHR_BIND_HOST — the warning above applies).

Variables the compose files read

Set these in your shell (or an .env file) to retune without editing anything:

A commercial licence token is not a passthrough variable: mount the signed token file into the server container read-only and name its path with FERROEHR__LICENCE__FILE (the compose file carries a commented example beside FERROEHR__SIGNING__ENABLED). Unset, the server runs under the embedded non-commercial grant and reports which on GET /management/status.

The Keycloak overlay adds KEYCLOAK_PORT (default 8081), KEYCLOAK_HOSTNAME, KEYCLOAK_ADMIN_USER and KEYCLOAK_ADMIN_PASSWORD (both admin); the observability overlay adds GRAFANA_PORT (default 3000). The FERROEHR__MULTIMEDIA__* set is declared with no defaults on purpose, so an unset key reaches the server as absent rather than as an empty string.

The server container is passed FERROEHR__DB__URL (assembled from the database variables), and its configuration file is the inline quickstart config, delivered to /etc/ferroehr/ferroehr.toml, which FERROEHR_CONFIG names. Any other setting from the configuration reference can be added under the ferroehr service’s environment: block and takes precedence over the file. Every variable uses the same FERROEHR__… grammar as the rest of that reference; a single-underscore spelling is rejected at startup, not ignored.

Repository development

A bare docker compose up in a checkout runs the same published-images quickstart as the downloaded file. Building from the current sources is an explicit opt-in: the docker-compose.dev.yml overlay, passed as a second file, switches the stack to the from-source developer posture:

docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build

That builds the server, database and viewer images from the current sources (the :local tags) instead of pulling published ones, and replaces the inline quickstart configuration with docker/ferroehr.dev.toml: three Basic users (ferroehr, ferroehr-admin, ferroehr-readonly, all with password ferroehr), RBAC enabled so the role separation is exercised, and trust for the development Keycloak realm. The override also defines a keycloak profile that imports the full development realm (docker compose --profile keycloak up).

Downloaders of the standalone quickstart file never see any of this; it is purely a convenience for working on FerroEHR itself, and it only ever applies when its -f is passed — nothing merges it in silently.

Tip

A cold from-source build compiles at two parallel rustc jobs by default, and a release-optimization compile can hold several gigabytes per job. If the build gets OOM-killed on a memory-constrained host (observed on a 24 GB machine that was also running another stack), rerun it serially — slower, but it completes:

CARGO_BUILD_JOBS=1 docker compose -f docker-compose.yml -f docker-compose.dev.yml build

Build provenance

Images built by CI (and any docker compose build you drive from a checkout) embed a build SHA reported at /management/info and on the ferroehr_build_info metric. The build does not read .git; instead the SHA is passed as the standard REVISION build argument, the same value that fills the org.opencontainers.image.revision image label (CI uses the commit SHA; the project’s own scripts export git rev-parse --short=12 HEAD). The argument is declared by the files that carry build: blocks (the development override and the conformance stack) not by the pull-only quickstart file. When no value is supplied the identity falls back to the workspace version with an unknown SHA; the build never fails for lack of it.

Observability overlay

A further standalone file adds a full local telemetry stack: an OTLP collector, Prometheus, Tempo, Loki and Grafana with a provisioned service-overview dashboard. Like the Keycloak overlay it travels inline, so downloading it beside docker-compose.yml is enough:

docker compose -f docker-compose.yml -f docker-compose.observability.yml up
# Grafana → http://localhost:3000

The overlay reconfigures the server for that stack: it exports traces and metrics over OTLP/gRPC (FERROEHR__TELEMETRY__OTLP__ENDPOINT with FERROEHR__TELEMETRY__OTLP__METRICS=true), switches stdout to JSON lines (FERROEHR__TELEMETRY__FORMAT=json), and enables the management surface on its own internal port 9464 with info, metrics and prometheus reachable there. That port is only published on the Compose network; Grafana’s 3000 is the sole published port.

Metrics are pushed, not scraped, and that is a property of the bundled image rather than a preference: grafana/otel-lgtm runs Prometheus with a config file carrying no scrape_configs at all, and receives metrics over OTLP from its own collector. A scrape job dropped into that image is read by nothing, which is what an earlier version of this overlay did: every metric panel was empty with no error anywhere. Since one meter provider feeds both readers, the push carries every metric family the scrape endpoint would, and /management/prometheus stays reachable on the Compose network for anyone who wants to compare the two.

Grafana’s port 3000 is the same one the viewer would use, but the two never collide by accident: the viewer only starts when you ask for the viewer profile. If you want both at once, move one of them (FERROEHR_VIEWER_PORT=3001 or GRAFANA_PORT=3001).

This is the easiest way to see the server’s metrics and traces without wiring up a collector by hand. See Operations for what the server exports and how to consume it in production.

Next

• Configuration reference — every setting you can pass.
• Kubernetes & Helm — the production deployment.
• Returning to the original state: resetting the stack to the configuration and empty database it shipped with.

Kubernetes & Helm

The ferroehr Helm chart deploys FerroEHR as a hardened, production-shaped Kubernetes workload (non-root, read-only root filesystem, a NetworkPolicy that admits its serving port only, its own user namespace) against an external PostgreSQL 18. This chapter covers installing it, verifying what you installed, the database role model it expects, the security posture it pins, the health probes, the optional integrations, and upgrades.

• Installing
  • Pin two versions, not one
  • Verifying what you installed
  • Your values file is checked before anything is applied
• Configuration: one key, one file
  • The deployment profile and the access log
  • The production overlay
• Database roles — who runs migrations
• Secrets and mounted config
• Security posture
  • TLS at the pod
• Health probes
• Authentication is required, and the chart says so before it installs
• Integrations
  • Any server setting is reachable
  • The common switches
  • FerroEHR Viewer (a second workload, off by default)
  • FerroPIX (the patient identity source, on by default)
  • FerroTERM (a terminology server beside the CDR, off by default)
  • FerroBRIDGE (the FHIR bridge, off by default)
  • Per-domain backups (one CronJob per domain, off by default)
• Staying available while things move
• Upgrades
  • Check your values before you deploy them

Important

The chart requires Kubernetes 1.36 or newer (kubeVersion: ">=1.36.0-0"). That is a compatibility floor, not a support opinion: 1.36 is the release where the newest field the chart renders (hostUsers, which gives every pod its own user namespace) became stable, so nothing has to be version-gated into silence. Your nodes must be Linux with containerd 2.0 or newer, or CRI-O 1.25 or newer; without that support the pod does not start, which is the loud failure rather than a silent downgrade. Set hostUsers: true to opt out and share the host’s user namespace. What the user namespace buys is in Cluster hardening.

Important

There is no in-chart PostgreSQL. A CDR stores PHI, so its database must be an externally managed, backed-up, point-in-time-recoverable PostgreSQL 18 (a managed service or an operator-run cluster). The chart carries only the connection string, preferably from an existing Secret.

Installing

The chart is published to GHCR as an OCI artifact, beside the images it deploys. Create a Secret holding the app-role connection string, then install:

kubectl create namespace ferroehr
kubectl -n ferroehr create secret generic ferroehr-db \
  --from-literal=FERROEHR__DB__URL='postgres://ferroehr_clinical:***@pg-host:5432/ferroehr?sslmode=verify-full'

helm install ferroehr oci://ghcr.io/ferrohealth/charts/ferroehr \
  --version 21.0.0-rc3 -n ferroehr \
  --set config.deployment_profile=sandbox \
  --set database.existingSecret=ferroehr-db \
  --set image.tag=5.6.0-rc3 \
  --set ferropix.enabled=false --set config.ehr_index.provider=local

config.deployment_profile=sandbox names the posture of a trial install, which must not hold real patient data; the chart has no default and refuses to render without one. A deployment holding patient data starts from the production overlay instead. The last line opts in to the in-instance EHR index, for a CDR run on its own. By default the chart runs FerroPIX, the IHE patient identity source, beside the CDR and points the EHR index at it, which needs FerroPIX’s own Secrets and an issuer: see FerroPIX. With neither, the render is refused and names both choices, as the server’s boot would be.

Important

helm repo add does not work for this chart, and never will. There is no HTTP chart repository and no index.yaml: OCI is the only publication path, so that there is exactly one place a chart version can exist. Every command takes the oci:// reference above. Helm has treated OCI registries as first-class since 3.8, so the cost of this choice is real but narrow: a client older than Helm 3.8 cannot install this chart at all.

helm show, helm pull, helm template and helm upgrade all take the same reference. To read the chart’s metadata without installing it:

helm show chart oci://ghcr.io/ferrohealth/charts/ferroehr --version 21.0.0-rc3

Pin two versions, not one

The chart version and the image tag move independently, and this is the thing people get wrong first. --version selects the chart (its templates and values schema); image.tag selects the server binary. The chart’s appVersion is only the default for the second, and it names the release the chart was cut against.

Always pin the image to an immutable version or, better, a @sha256 digest, never latest. Pin the two deliberately: the config tree is passed through to the server, so a key the chart’s defaults carry and your chosen image does not know is a boot refusal (unknown configuration key …), which presents as CrashLoopBackOff. A chart version is never republished with different content (the publish lane refuses to overwrite one) so a pinned chart version is a fixed artifact.

Note

A published chart version being immutable is a property of that refusal, not of the registry: an OCI tag is mutable, and helm push over an existing one would silently replace it. That is why a correction always ships as a new chart version, and why the 6.0.3 → 6.0.4 bump exists: the chart’s appVersion moved to a new release while its own version had already been published.

Verifying what you installed

Everything published here is signed keyless through Sigstore, bound to this repository’s build identity, so every claim is checkable rather than asserted. Two different artifacts answer two different questions, and you can ask both.

Who signed this chart: a cosign signature over the chart’s digest:

cosign verify ghcr.io/ferrohealth/charts/ferroehr:<chart-version> \
  --certificate-identity-regexp '^https://github\.com/FerroHEALTH/FerroEHR/\.github/workflows/build-chart\.yml@' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

Both flags matter. Without them cosign verify would accept a signature from any identity in the transparency log; with them you are requiring that this repository’s chart-publishing workflow, authenticated by GitHub’s OIDC issuer, is what signed the bytes you pulled.

What it was built from: a SLSA build provenance attestation:

# the chart
gh attestation verify oci://ghcr.io/ferrohealth/charts/ferroehr:<chart-version> -R FerroHEALTH/FerroEHR
# the image it deploys
gh attestation verify oci://ghcr.io/ferrohealth/ferroehr:<tag> -R FerroHEALTH/FerroEHR

The publish lane reads both back from the registry before it reports success, so a run that produced only one of them fails instead of going green.

Important

Both commands verify what the publishing lanes produce now. A published artifact is never replaced, so if one you pinned answers HTTP 404: Not Found it carries no attestation, which is the honest state rather than a verification failure, and the fix is to pin a current version. Image signing landed during the 3.17.4 cycle, so image tags from before it have nothing to verify.

Note

helm install --verify and helm verify do not apply: they check a PGP .prov file, and this chart ships none. That is deliberate: a .prov needs a long-lived private key in CI, which is the exact thing this project’s publishing lanes are built to avoid (the crates.io lane uses OIDC Trusted Publishing and holds no token at all). The two keyless commands above are what replace it: nothing to leak, and the same trust root as the images.

Your values file is checked before anything is applied

The chart ships a values.schema.json, so helm install, helm upgrade, helm lint and helm template refuse a values file that misspells one of the chart’s own keys, gets a type wrong, or names a value outside the permitted set, rather than rendering and silently ignoring it:

Error: values don't meet the specifications of the schema(s) in the following chart(s):
ferroehr:
- at '/image/pullPolicy': value must be one of 'Always', 'IfNotPresent', 'Never'

The config: tree is deliberately exempt. Those keys are the server’s (see the configuration reference), the binary validates them at boot, and copying that vocabulary into the chart’s schema would fork it: a new configuration key would then be rejected by the chart until someone remembered to widen the schema. So a mistake under config: is reported when the pod starts, not when the chart renders; --skip-schema-validation disables the check entirely if you ever need to bypass it.

The chart is also listed on Artifact Hub, which renders the chart’s metadata plus a security report over the four images the chart’s own metadata lists: the server, the optional viewer, the backup jobs’ pg_dump image, and FerroTERM.

Warning

Between releases the chart’s config defaults track development and can be ahead of appVersion’s image. Check the pairing before you install, with the image itself as the authority:

helm template ferroehr oci://ghcr.io/ferrohealth/charts/ferroehr --version 21.0.0-rc3 \
  -s templates/configmap.yaml --set database.existingSecret=ferroehr-db \
  | sed -n '/ferroehr.toml/,$p' | sed '1d;s/^    //' > /tmp/ferroehr.toml
docker run --rm -v /tmp/ferroehr.toml:/etc/ferroehr/ferroehr.toml:ro \
  -e FERROEHR_CONFIG=/etc/ferroehr/ferroehr.toml \
  -e FERROEHR__DB__URL=postgres://u:p@db:5432/ferroehr \
  --entrypoint /usr/local/bin/ferroehr ghcr.io/ferrohealth/ferroehr:<tag> config check

Exit 0 means the image accepts the rendered configuration. A reported unknown key means the image is older than the chart’s defaults: use a newer tag, or set that key to null for this deployment. A published chart is checked this way before it is published: the publish lane refuses to ship a chart whose defaults its own appVersion image rejects, and it repeats the check against every values overlay the chart carries, so this matters mainly when you install from a checkout of main, or pin an image older than the chart.

That install alone boots but answers 401 to everything, deliberately: config.auth.enabled is on and no mechanism is configured yet, and a server that authenticates nothing is not a safe default. Add a config.auth.issuer (with config.auth.audience) or a config.auth.basic.users entry before expecting a request to succeed. Every route also checks the SMART on openEHR scopes of the caller: a Basic user’s scopes, a token’s scope claim.

Configuration: one key, one file

The chart carries the server’s whole configuration under one key, config, rendered into a ferroehr.toml ConfigMap mounted at /etc/ferroehr/ferroehr.toml. Its keys are therefore exactly the TOML keys of the configuration reference (config.server.listen, config.authz.rbac.enabled, config.spec_profile, and so on) so anything that reference documents can be set without waiting for a bespoke chart key.

Secret-bearing keys are the one exception: a credential never reaches that ConfigMap. A ConfigMap is not a sensitive object: it is readable with namespace read, quoted wholesale into issues and support tickets, collected by backup tooling that skips Secrets, and not covered by Secret encryption at rest. The chart therefore classifies every key it renders and takes one of two actions.

A secret the chart routes: signing.key_passphrase, multimedia.secret_access_key, a Basic user’s password_hash, a terminology client_secret, and the three URL-shaped ones (db.url, events.url, audit.fhir_feed.url) has a secrets: key of its own, so a value under config: is a mistake and fails the render, naming the key that belongs there:

Error: execution error at (ferroehr/templates/deployment.yaml:…):
  refusing to render a secret into the ConfigMap (a ConfigMap is not a sensitive object …):
  - config.signing.key_passphrase: set `secrets.signingKeyPassphrase` instead

A secret the chart cannot route moves the whole rendered ferroehr.toml into the chart’s Secret, and no ConfigMap is created at all, the safe direction, taken automatically. No key reaches that branch today: every credential the server models now has either a *_file sibling or a Secret-borne environment route. The branch exists for the next one that does not, so that a secret key added upstream fails safe instead of landing in a ConfigMap. When it is taken, the install notes say which object your release used, and the configuration is read with:

kubectl -n ferroehr get secret ferroehr-config \
  -o jsonpath='{.data.ferroehr\.toml}' | base64 -d

Classification is by name shape, not by a list of today’s keys: any key whose name carries password, passphrase, secret, credential, private_key, api_key or a trailing token is treated as a credential unless it ends in _file, _path or _dir (those hold a path), and the four URL-shaped secrets are matched by path, because url carries no shape a classifier can see. That is what makes a secret key added to the server’s configuration tree in a future release move to the Secret rather than leak silently. extraEnv is the escape hatch for anything neither config: nor secrets: surfaces.

The deployment profile and the access log

The chart’s values declare three keys, so a rendered ferroehr.toml always says what it runs:

• config.deployment_profile: no default; the chart refuses to render until it names sandbox or production, so an installation that follows a quickstart instead of the instructions for use does not end up in the sandbox posture by omission (EHDS Annex II 1.2). sandbox must not hold real patient data: the server names every separation it has not made on its boot banner and on /management/status, and reports the profile on GET /ferroehr/rest/status. A deployment holding patient data sets production, which refuses to start while a separation is missing and not named in config.deployment_accepts. The chart defaults leave several open: one DSN for every domain, an empty config.privacy.subject_namespaces, the server’s open defaults for config.audit.fail_mode and config.authz.rbac.ehr_access_default, config.db.migrate: apply on the runtime credential, and a plaintext listener behind the ingress; see deployment_profile. The chart’s production overlay closes them and accepts the plaintext listener by name.
• config.audit.store.retention_years: unset. Set it in place of config.audit.store.retention_days when the horizon must be exact in calendar years, such as 3 for a German SGB V § 309 controller with config.audit.store.sgb_v_309_controller: true.
• config.audit.categories: an empty templates and archetypes map, under which the category map FerroEHR ships applies: it classifies the published CKM templates and archetypes of one priority category and records anything else as unclassified. Map your own templates and root archetypes, which replaces the shipped map; the keys are template ids and archetype ids, so quote a template id that contains spaces:

config:
  audit:
    categories:
      templates:
        "International Patient Summary": [patient-summary]
      archetypes:
        openEHR-EHR-COMPOSITION.report-result.v1: [test-results]

The Audit page has the category vocabulary and the retention floors and ceilings the server enforces at boot.

The production overlay

The chart ships values-production.yaml beside values.yaml. Layered over the defaults, it runs config.deployment_profile: production with every separation that profile checks made, except one it accepts by name:

The database hop is TLS with the server’s certificate checked: end every DSN in ?sslmode=verify-full&sslrootcert=/etc/ferroehr-db-ca/ca.crt, and put the CA your database certificates chain to in a ConfigMap under the key ca.crt. The overlay names it ferroehr-db-ca (database.caConfigMap), and the chart mounts it read-only at /etc/ferroehr-db-ca in the server, the migration Job and the backup CronJobs. A CA the image already trusts needs no ConfigMap; leave database.caConfigMap empty then.

Every Secret name, the CA ConfigMap name, the identity provider, the pseudonym namespace, the host, the TLS Secret and the ingress controller’s namespace in it are examples. Replace them in a values file of your own and pass both:

helm pull oci://ghcr.io/ferrohealth/charts/ferroehr --version 21.0.0-rc3 --untar
helm install ferroehr ./ferroehr -n ferroehr \
  -f ./ferroehr/values-production.yaml -f my-site-values.yaml

A name you forget to replace fails loudly: a Secret that does not exist keeps the pod from starting, and the server refuses to start under production while a separation is open, naming each gap and its remedy. The chart’s validation renders the overlay and asserts each separation above, and its boot check has the server evaluate the posture from the rendered configuration. Neither reaches a database. The deployment probe (scripts/deploy-probe-k8s.sh --production-only) installs the overlay on a real cluster against four PostgreSQL clusters that refuse a connection without encryption: it observes the migration Job prepare all four and the server start under production with plaintext_listener as its only gap, both over sslmode=verify-full against the CA the overlay mounts, and the server refuse all four domains on one cluster by naming shared_cluster. Against your own databases, shared_cluster is first judged when the pod starts.

Database roles — who runs migrations

The chart expects a PostgreSQL role model in which the runtime pod is never a superuser. Four roles cover the single-domain posture:

Migrations are DDL, so whoever applies them can rewrite the schema. Two flows, and config.db.migrate is where you choose:

• (a) The pod migrates itself (config.db.migrate: apply, the default). Simplest for single-tenant or small deployments, and the runtime DSN must then be a member of ferroehr_migrator, so the serving process holds DDL rights on the clinical schema for its whole life.
• (b) A separate migration step (config.db.migrate: verify) under a migrator DSN, with the pods on a narrow runtime DSN. The server issues no DDL at all and refuses to boot against a database that has not been migrated to its build, so the two versions cannot race the schema. Recommended for production. It needs database.migrateExistingSecret as well, for the reason the paragraph further down gives: verify reads all five bookkeeping tables, which a narrow runtime role cannot do.

Set migrations.job.enabled and the chart runs (b) for you as a pre-install,pre-upgrade hook Job that runs ferroehr db migrate. Helm creates it before the Deployment and waits for it, so a failed migration fails the release rather than rolling pods against a schema that was never applied. The Job authenticates from its own Secret (deliberately a different credential from database.existingSecret) and rendering is refused if you enable it without one:

database:
  existingSecret: ferroehr-db                 # postgres://ferroehr_clinical:…
  migrateExistingSecret: ferroehr-db-migrator # postgres://ferroehr_migrator:…
migrations:
  job:
    enabled: true
    existingSecret: ferroehr-db-migrator      # the same credential, its own Secret mount
config:
  db:
    migrate: verify

The demographic and linkage domains take their own credentials the same way, and that is what turns the schema separation into a credential separation. One more goes with them, because no runtime credential can prepare the schema:

database:
  existingSecret: ferroehr-db             # postgres://ferroehr_clinical:…
  party:
    existingSecret: ferroehr-db-party     # postgres://ferroehr_party:…
  linkage:
    existingSecret: ferroehr-db-linkage   # postgres://ferroehr_linkage:…
  migrateExistingSecret: ferroehr-db-migrator        # postgres://ferroehr_migrator:…

All four are mounted as files, so no DSN enters the pod’s environment. Unset the demographic and linkage ones and those two pools share the first, which is the schema-only posture, still separated and still verified at boot. Creating the five domain roles is a database step the chart cannot do for you; Operations has the statements and what ferroehr db verify reports.

database.migrateExistingSecret is what the pod prepares the schema with, and it is required as soon as database.existingSecret is anything narrower than a credential reaching every schema. Preparation spans them all at once: the DDL of all five migration sets under config.db.migrate: apply, all five _sqlx_migrations bookkeeping tables under verify. So verify is not the exception, and this is not only about the domain roles. ferroehr_clinical holds one pseudonymisation domain, ferroehr_clinical holds the clinical schemas, and neither can read the demographic or linkage bookkeeping at all; without this value the pod is refused on the first set it cannot read and does not start. It is normally the same credential migrations.job.existingSecret carries; the Job needs its own copy because it runs before the Deployment exists. Unset, preparation falls back to database.existingSecret, which is what a single-credential install has always done.

database.migrateExistingSecret reaches one database. A domain whose DSN reaches another database or cluster is prepared there, on that database’s own migrator, which database.<domain>.migrateExistingSecret names:

database:
  linkage:
    existingSecret: ferroehr-db-linkage            # postgres://ferroehr_linkage:…@linkage-db/…
    migrateExistingSecret: ferroehr-db-linkage-migrate   # postgres://ferroehr_migrator:…@linkage-db/…

The migration Job mounts it and applies the schema in that database; the server mounts it and verifies the schema there under config.db.migrate: verify. The domain’s own role cannot do either, because it holds one domain and cannot read the ext set’s bookkeeping. The server refuses to start when the two DSNs of one domain reach different databases. A database that holds the linkage domain without the party domain also receives the party migration set, its schema and grants and never a row, because the linkage set names a function the party set creates.

Give the migrator DSN a short lock_timeout (?options=-c%20lock_timeout%3D5s) so DDL blocked behind live traffic fails fast instead of queueing; migrations.job.activeDeadlineSeconds is the hard ceiling on the step either way. migrations.runByMigratorRole remains an informational marker surfaced in the install notes, which also tell you when the Job is enabled but config.db.migrate is still apply, a combination that buys nothing, because the server would migrate itself anyway.

You can check the posture from outside the cluster at any time: ferroehr db verify exits 0 only when the database carries exactly that build’s migrations, and issues no DDL doing it.

Secrets and mounted config

Some material is file-shaped rather than a value: ABAC policy files, ATNA TLS certificates, terminology-server client certificates, a JWKS blob, the PGP signing key, and a commercial licence token. Supply these under config.files, whose entries the chart mounts read-only from a Secret at /etc/ferroehr/<key> (and which is deliberately not part of the rendered TOML); point the matching in-TOML *_file / *_path key at the mounted path. A commercial licence goes the same way: put the token the licensor issued under config.files as licence.asc and set config.licence.file to /etc/ferroehr/licence.asc; GET /management/status then reports licence.use = "commercial". Without it the pod runs under the non-commercial grant every build embeds (see Licensing & legal). Secret-bearing scalar values go under secrets:: signingKeyPassphrase, eventsUrl, auditFhirFeedUrl, basicUserPasswordHashes, multimediaAccessKeyId, multimediaSecretAccessKey, terminologyClientSecret, and the database DSN comes from database.existingSecret (key database.existingSecretKey, default FERROEHR__DB__URL). None of these ever reach the ConfigMap, and all but two are delivered as mounted files rather than environment values.

How a secret reaches the process differs by whether the configuration key has a *_file sibling, and the difference is a security one: an environment variable is readable through /proc/<pid>/environ and is inherited by every child process, so the OWASP Kubernetes Security Cheat Sheet asks for a read-only volume instead.

The mount is read-only, 0440, owned root:65532 so the non-root process reads it through the group bit, and it is deliberately not a subPath mount, because a subPath-mounted Secret never receives updates and a rotation would not propagate.

Note

A Basic user’s Argon2id hash is delivered as a mounted file like the others: put it in secrets.basicUserPasswordHashes under the user-id, declare only the user, its scopes and its roles under config.auth.basic.users. Setting password_hash under config: is refused, and the error names the key that carries it. An Argon2id hash is not a plaintext password, but it is an offline cracking target, which is what the boot-time OWASP parameter floor exists to make expensive.

Security posture

The chart pins the following, and its render gate holds it: the Restricted fields are asserted per container for every workload in the render, the two isolation settings are asserted to agree across a release’s workloads, and the golden renders pin the exact bytes so a changed default fails a diff.

The whole set satisfies the Pod Security Restricted profile, and the deployment probe harness reads it back off a running pod rather than off the rendered manifest: the container runtime’s own spec for the security context, the API server for admission, the EndpointSlice for readiness.

Satisfying the profile and enforcing it are different things, and only one of them is yours to do. Enforcement comes from a label on the namespace, which a chart cannot set for a namespace it does not own, so label it, or the posture above is a convention nothing checks and nothing fails when a future change regresses it:

kubectl label --overwrite namespace ferroehr \
  pod-security.kubernetes.io/enforce=restricted \
  pod-security.kubernetes.io/enforce-version=latest

With the label in place the API server refuses a non-compliant pod outright (Pod Security Admission), which is a stronger guarantee than any check the chart can make about itself. The install notes print this as a prerequisite for the same reason.

enableServiceLinks: false is load-bearing, not hygiene. The kubelet injects a set of Service link environment variables for every Service in the namespace, and for a Service named ferroehr* those land inside the server’s reserved FERROEHR_ namespace, whose strict boot-time sweep rejects unknown variables and refuses to start. Leaving service links on makes every install crash-loop.

Egress restriction is opt-in (networkPolicy.egress.enabled) because egress targets (the database, broker, terminology server) are deployment-specific; when you enable it the chart always admits DNS and you add rules for the rest. In the default posture the server’s only outbound traffic is DNS and PostgreSQL, so those two rules suffice; each integration you switch on adds a target, and a blocked one can fail silently. The full destination table is in Namespaces, network & policy.

Two limits worth stating plainly. First, with networkPolicy.ingressFrom empty the rendered ingress rule carries no from selector, and a rule without from admits every source, other namespaces included. Only the port list is narrowed in that state, so set ingressFrom to your ingress controller for a PHI workload, and set networkPolicy.ingressAllowAll: false if an open ingress rule must never render at all; the full treatment is §Ingress. The viewer’s own policy carries the same pair under viewer.networkPolicy. Second, a NetworkPolicy is only as real as the CNI that implements it: on a cluster whose network plugin does not enforce NetworkPolicy the object is documentation rather than a control, and nothing in Kubernetes reports that. Confirm it by attempting a connection the policy should refuse.

The database-side controls (the server certificate behind sslmode=verify-full, pgaudit, at-rest encryption, WAL archiving / PITR) belong to whoever provisions PostgreSQL; the chart mounts the CA (database.caConfigMap) but cannot enforce the rest. See Operations.

TLS at the pod

The production overlay ends TLS at the Ingress and accepts plaintext_listener by name. To close that gap instead, the server terminates TLS itself (config.server.tls), and every client the chart wires to it follows: the viewer and FerroBRIDGE call https://<release>:<service.port>, the kubelet probes over HTTPS, the Service declares appProtocol: https, and a ServiceMonitor that scrapes the API port uses https too.

The clients check the server’s certificate against a CA you name in service.tls.caExistingSecret. The certificate must carry the Service name (the release’s full name, ferroehr for a release called ferroehr) as a DNS SAN, which no public CA issues, so the chart refuses to render a TLS listener beside an enabled viewer, FerroBRIDGE or API-port scrape while that value is empty. Name a Secret that holds the CA certificate alone: every pod and the ServiceMonitor that read it then read no private key, and a ServiceMonitor in another namespace reads its own copy there. A cert-manager Certificate writes the key pair and the CA into one Secret (tls.crt, tls.key, ca.crt); copy ca.crt out of it (trust-manager’s Bundle does this) rather than naming that Secret. The Ingress in front of the TLS listener must reach it over HTTPS too, and how a controller is told so is controller-specific, so the chart refuses to render the Ingress with server TLS on until ingress.backendTlsAnnotations names it:

config:
  deployment_accepts: []          # nothing left to accept for the listener
  server:
    tls:
      certificate_file: /etc/ferroehr-tls/tls.crt
      key_file: /etc/ferroehr-tls/tls.key
      reload_interval_s: 60
extraVolumes:
  - name: server-tls
    secret:
      secretName: ferroehr-server-tls
      defaultMode: 0440
      items:
        - { key: tls.crt, path: tls.crt }
        - { key: tls.key, path: tls.key }
extraVolumeMounts:
  - { name: server-tls, mountPath: /etc/ferroehr-tls, readOnly: true }
service:
  tls:
    caExistingSecret: ferroehr-server-ca    # ca.crt alone
ingress:
  backendTlsAnnotations:                    # ingress-nginx
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS

When cert-manager renews the certificate, the kubelet updates the mounted files and the server serves the new pair within config.server.tls.reload_interval_s (60 seconds above; unset, only SIGHUP reads the files again), with no restart; a renewed pair that fails to load is logged and the old one stays in service. A rotated client CA (client_ca_file, for client_auth) is picked up on the same interval, and the connections already open keep the trust anchor they were verified against.

The viewer reads the mounted CA through its own cdr.tls.trust_roots_file, beside the image’s roots. FerroBRIDGE reads it through SSL_CERT_DIR, which the chart sets to the image’s system roots plus the mounted CA, so its other peers that verify through the platform verifier stay trusted. For an OIDC issuer whose certificate comes from a private CA, name a Secret holding that CA in viewer.auth.oidc.caExistingSecret (key viewer.auth.oidc.caExistingSecretKey, ca.crt by default): the chart mounts it and points the viewer’s [auth.oidc.tls] trust_roots_file at it. For the server’s own [auth] issuer behind a private CA, name the Secret in auth.caExistingSecret (key auth.caExistingSecretKey, ca.crt by default): the chart mounts it and points the server’s [auth] ca_file at it. The ServiceMonitor reads the Secret from its own namespace, and the prometheus.io/scheme: https pod annotation follows the listener when metrics.enabled is set. With config.server.tls.client_auth: required the kubelet probes cannot present a client certificate, so the chart refuses to render it; keep it at none or optional. config.management.port stays plain HTTP on its own listener.

Health probes

All three probes use the always-on, unauthenticated, PHI-free health routes on the main HTTP port. They need no configuration at all: no management surface, no access level, nothing to forget:

That split is deliberate: a database outage must fail readiness (the pod stops receiving traffic) and never liveness (which would restart the container in a loop).

Note

There is no exec probe option. One existed and was removed: it ran the binary’s healthcheck subcommand, which then asked the openEHR status document rather than a health route, so readiness never touched the database and a pod with a dead database reported Ready and took clinical traffic. The subcommand now asks /health/readiness, and the httpGet probes above remain what the chart runs.

Warning

Migrations run only at boot, so a replaced or wiped database leaves the pods NotReady until something migrates it. Readiness reports the migrations indicator down (its detail on /management/health: core schema tables missing (migrations not applied)) while db reads up: the pod reaches PostgreSQL, finds no schema, and does not migrate again, because migration is a startup step. Liveness keeps passing (correctly: the process is healthy), so the kubelet never restarts the container and the Deployment sits with no ready replicas and no error in the logs after the first one:

kubectl -n ferroehr get pods                      # Running, not READY
curl -s http://ferroehr:8080/health/readiness     # 503, "migrations" down, "db" up
kubectl -n ferroehr rollout restart deploy/ferroehr

The check re-tests the schema on every probe, so a pod recovers on its own within one probes.readiness.periodSeconds of the schema existing; an untouched pod returns to ready, with no restart, once anything else has migrated. What needs the restart is the case where the only thing that would migrate is the pod itself.

That is what makes the interaction with flow (b) load-bearing: when migrations run out of band, the migration step must complete before the Deployment rolls, or the first pods sit unready waiting for a schema. migrations.job.enabled does exactly that, as a hook Helm waits on. Durable storage is the other half: an emptyDir-backed or otherwise disposable PostgreSQL puts a clinical repository one node eviction away from this state.

One recovery path does not self-heal, and it is worth knowing before you try it: restoring or dropping part of the schema set. A restore that brings back some of a domain’s relations without its migration bookkeeping makes the next migration run fail permanently with relation "version" already exists: the pod crash-loops, and restarting it retries the same failure. Recreate the whole database rather than one schema.

The management surface is independent of the probes and stays ops-only (/management/info, /prometheus, /metrics, /env, /loggers, /flamegraph). Unlike the bare binary, the chart ships config.management.enabled: true with every endpoint off, so nothing is exposed until you opt one in. Set config.management.endpoints.prometheus: public plus metrics.enabled=true to add the prometheus.io/* scrape annotations; set config.management.port to serve the surface on its own internal listener, so /management is never reachable on the clinical API port. The health probes stay on the main port regardless of all of this, and the install notes warn you if you add the scrape annotations while the endpoint is still off.

Those annotations are honoured by a Prometheus that discovers targets in its own scrape configuration. An operator-managed Prometheus (kube-prometheus-stack) ignores them and selects targets through ServiceMonitor objects instead, so set metrics.serviceMonitor.enabled=true there; metrics.serviceMonitor.labels is where the label your Prometheus’ serviceMonitorSelector matches on goes. It needs the monitoring.coreos.com CRDs installed first, or the install fails on an unknown kind.

Authentication is required, and the chart says so before it installs

config.auth.enabled defaults to true, and the server requires at least one mechanism with it: a 401 challenge has to name a scheme the server actually implements (RFC 9110 §11.6.1), so it exits rather than run as an openEHR API that can only refuse every request.

The chart therefore refuses to render a values file that enables authentication without configuring one, so helm install stops with an actionable error instead of reporting success and crash-looping:

Error: execution error at (ferroehr/templates/deployment.yaml:...):
config.auth.enabled is true but no authentication mechanism is configured...

Pick one:

config:
  auth:
    issuer: https://keycloak.example.org/realms/ferroehr
    audience: [ferroehr]

or Basic auth, whose password hashes go in secrets.basicUserPasswordHashes while the user, its scopes and its roles stay under config.auth.basic.users.

Warning

config.auth.enabled: false makes the chart render, and serves every request unauthenticated. On a repository holding patient data that is a development-only choice; the chart makes you state it explicitly rather than reaching it by omission.

Integrations

Any server setting is reachable

The chart renders its config tree verbatim into ferroehr.toml. So config.<the.toml.path> sets any key in the configuration reference, including keys this page and values.yaml never mention. There is no allow-list to extend and no chart release to wait for:

# values.yaml — [query] plan_cache_capacity, which values.yaml never names
config:
  query:
    plan_cache_capacity: 512

helm upgrade ferroehr oci://ghcr.io/ferrohealth/charts/ferroehr \
  --version 21.0.0-rc3 -n ferroehr --reuse-values \
  --set config.query.plan_cache_capacity=512

Turning something off is the same edit in reverse: remove the key (or set the integration’s enabled back to false) and upgrade. The tables below are a curated starting point for the switches most deployments want; they are not the boundary of what the chart supports.

Two things make this safe to rely on:

• A typo is a boot refusal, not a silent default. The server sweeps its configuration strictly and rejects an unknown key with a did-you-mean, so a misspelled path fails loudly at startup instead of quietly doing nothing. Check before you deploy with the schema and boot gates described under Check your values before you deploy them.
• Credentials do not belong in this tree. config becomes a mounted file; secrets have their own routes (secrets.*, existingSecret, and the *_file key siblings); see Secrets and mounted config.

The common switches

Every switch below lives in the chart’s config tree, so its key is the TOML key from the configuration reference. Most are off by default; the ones that ship on are marked, and enabling any of the others is an explicit, auditable decision:

Full detail on each is in Beyond the core, Security, and Operations.

The usage report is on by default, and usageReport.enabled=false switches it off for FerroEHR, for FerroTERM and for FerroBRIDGE. Each of those takes its own terminology.usageReport.enabled or bridge.usageReport.enabled when you want it to differ. FerroPIX runs from its own chart, which cannot read this chart’s switch: set ferropix.usageReport.enabled=false as well, or the chart refuses to render.

FerroEHR Viewer (a second workload, off by default)

viewer.enabled renders a second Deployment and Service for the viewer beside the CDR, from its own image (viewer.image.repository, tagged appVersion by default so the two move together). It is off by default: the viewer is a separate product surface with its own attack surface, and a CDR is complete without it.

Three properties are worth knowing before you switch it on:

• It reaches the CDR strictly over the REST API, which viewer.networkPolicy.enabled (on by default) enforces rather than assumes: the viewer’s egress admits the CDR Service, DNS and outbound HTTPS for an identity provider, and nothing else. It holds no database credential.
• It is a human-facing web UI, so viewer.ingress.enabled is the normal way to reach it, and viewer.auth.oidc.enabled with an issuer, client id and viewer.auth.oidc.publicBaseUrl is how you keep a person who should not see PHI out of it. Its client secret comes from viewer.existingSecret, mounted as a file exactly as the server’s DSN is.
• viewer.replicaCount is 1 deliberately. The viewer holds session state in process, so a second replica needs sticky sessions at the ingress or users are logged out on a reroute.

The viewer carries the same security context as the server, and the chart’s render gate holds it to the same Restricted profile and the same pod-isolation settings: a release whose two workloads disagreed about hostUsers would be a posture nobody could state in one sentence. The deployment probe harness then reads that back off the running viewer container, exactly as it does for the server: its uid, its empty capability bounding set, its read-only root and its seccomp filter come from the container runtime’s own spec, its admission from an enforce=restricted namespace, and its login page from an HTTP request made inside the cluster through the viewer Service, so the second workload is never vouched for by the first. What that run does not establish is stated in its own record: the viewer’s OIDC path, the screens behind a session, and whether a CNI enforces the viewer’s NetworkPolicy. Its screens are documented in the viewer chapter.

FerroPIX (the patient identity source, on by default)

The CDR keeps no patient-to-record cross-reference of its own by default: it asks an IHE patient identity source over PIXm (ITI-83) which pseudonym the clinical domain holds for a patient ([ehr_index]). The chart takes FerroPIX as the chart dependency ferropix (oci://ghcr.io/ferrohealth/charts, pinned in Chart.lock), so its Deployment, Service, NetworkPolicy and configuration are its own chart’s, and every key under ferropix: is that chart’s value. With ferropix.enabled (the default) the chart writes config.ehr_index.provider = "ihe" and config.ehr_index.pixm_url at the FerroPIX Service, and refuses a values file that sets either too. With the egress policy on, it adds the rule to FerroPIX’s pods.

You supply FerroPIX’s PostgreSQL URL and sealing key, its issuer, the pseudonymisation domain whose consumer is the CDR’s client, and the CDR’s side of the same facts:

ferropix:
  database: {existingSecret: ferropix-db}
  dataKey: {existingSecret: ferropix-data-key}
  config:
    public_base_url: https://pix.example.org
    auth:
      audience: [https://pix.example.org/fhir]
      issuer: [{issuer: https://idp.example.org/realms/pix}]
    trust:
      issuer: [{issuer: https://idp.example.org/realms/pix, clients: [hospital-source]}]
    pseudonym_domain:
      - {namespace: urn:oid:2.999.7.1, name: Clinical domain, consumer_client_id: ferroehr}
config:
  privacy:
    subject_namespaces: [urn:oid:2.999.7.1]
  ehr_index:
    assigning_authority: urn:oid:2.999.7.1
    oauth:
      token_endpoint: https://idp.example.org/realms/pix/protocol/openid-connect/token
      client_id: ferroehr
      scope: system/Patient.rs
secrets:
  ehrIndexClientSecret: <the client secret>

secrets.ehrIndexClientSecret is mounted at /etc/ferroehr-secrets/ehr_index.oauth.client_secret, and the chart injects the matching client_secret_file; the production overlay reads it from a Secret you own through extraEnv instead. Another PIX Manager: ferropix.enabled: false and config.ehr_index.pixm_url. A CDR run on its own: ferropix.enabled: false and config.ehr_index.provider: local.

The chart pins FerroPIX 0.0.12 by digest (ferropix.image) with the FerroPIX chart built for it. Since FerroPIX 0.0.8, public_base_url is a top-level key and the clients that may act as a Patient Identity Source sit in the [[trust.issuer]] table, apart from the [[auth.issuer]] entry that names the issuer; FerroPIX refuses the older server.public_base_url and auth.issuer[].clients, so an upgrade from a chart that pinned FerroPIX 0.0.7 moves those two keys in your values.

The restricted-profile gate holds FerroPIX’s pods to the Pod Security Standards restricted profile like every other pod in the release. FerroPIX’s chart sets the two properties this chart adds for its own pods itself: enableServiceLinks: false, and a user namespace per pod unless FerroPIX’s own hostUsers value, under ferropix:, is true.

FerroTERM (a terminology server beside the CDR, off by default)

FerroTERM is the Ferro family’s FHIR terminology server: R4, R4B and R5 endpoints over a precomputed index, no JVM, no database, one distroless image. terminology.enabled renders it as a third workload beside the CDR and the viewer, and points the CDR at it, so archetype value-set bindings resolve at commit and AQL TERMINOLOGY() expands through it. The CDR’s own /terminology/* extension routes are a separate switch: they answer only with config.terminology.api_enabled=true, which is false by default, and return 404 otherwise whether or not FerroTERM is running. It is the Helm equivalent of the compose terminology overlay.

helm upgrade --install ferroehr oci://ghcr.io/ferrohealth/charts/ferroehr \
  --version 21.0.0-rc3 -n ferroehr --reuse-values \
  --set terminology.enabled=true

That renders a Deployment, a ClusterIP Service, its own ServiceAccount, a ConfigMap holding the code systems, and a NetworkPolicy. It also writes three things into the CDR’s ferroehr.toml: config.terminology.server.enabled, config.terminology.server.fail_on_error from terminology.failOnError, and config.terminology.server.url at the Service address.

Because the chart writes those three, a values file that contradicts any of them is refused at render rather than silently overwritten. Three shapes are refused while terminology.wireCdr is on:

• config.terminology.server.url — two answers to “which server resolves a binding” is how resolution ends up pointed somewhere nobody chose.
• config.terminology.server.enabled: false — the rendered ferroehr.toml would say the opposite of your values file.
• config.terminology.server.fail_on_error disagreeing with terminology.failOnError — same reason, for the posture that decides whether an unresolvable binding is a 422.

Set terminology.wireCdr=false to keep your own server and run FerroTERM unwired. The CDR asks one terminology server, so a server of your own and the in-cluster FerroTERM are an either-or.

No Ingress is rendered, and no value renders one. The CDR is the only caller, which is the posture the hosted sandbox runs and what the SNOMED CT Affiliate Licence asks of a public deployment: clause 2.2.4 permits a public system to serve SNOMED-encoded data provided users cannot extract a substantial portion of the release and no fee is charged, and clause 2.7 requires measures so the release cannot be downloaded except by authorised users. A published FHIR terminology endpoint answers $expand and $lookup to every client that reaches it, and FerroTERM has neither authentication nor a rate limit of its own. Its NetworkPolicy is therefore the one in this chart that narrows sources by default: ingress admits the CDR’s pods on the terminology port and nothing else, egress admits DNS. terminology.networkPolicy.extraIngressFrom adds a peer you decide to admit. No boolean opens this port the way networkPolicy.ingressAllowAll opens the CDR’s — and a peer you add is admitted exactly as written, which includes admitting everything: a namespaceSelector: {} selects every namespace in the cluster (networking/v1: “if present but empty, it selects all namespaces”). The values schema therefore refuses an empty peer, and an empty podSelector or namespaceSelector inside one, so reaching that state takes labels you chose rather than a blank.

Out of the box it serves the shaped seed, which is synthetic content under the reserved example.test domain: two code systems and two value sets, the same content the conformance lane binds to. The chart carries those files and mounts them as a ConfigMap, so a fresh install from the registry serves terminology with no repository checkout. terminology.codeSystems.existingConfigMap swaps in a ConfigMap of your own FHIR JSON. No licensed terminology ships with FerroEHR or FerroTERM. One caveat with a ConfigMap of your own: the pod’s rollout annotation hashes the chart’s own seed files and the NAME you gave, not the contents of an object the chart does not own, so editing that ConfigMap updates the volume while the running pods keep serving what they read at boot. Restart the workload after such an edit (kubectl rollout restart deployment/ferroehr-terminology).

A real release is an index you build off-cluster. Run ferroterm-build --rf2 <release.zip> --out <dir> on a machine that holds the archive, put the output on a PersistentVolume, and name that claim in terminology.index.persistentVolumeClaim. The chart mounts it read-only at terminology.index.mountPath and sets FERROTERM_INDEX; with no claim named, the variable is absent and the seed is served alone. The chart provisions no storage and renders no build Job, for the same reason it provisions no database: unpacking a licensed archive is a step whose licence terms are yours, not a step a chart should start on its own.

Naming a claim also switches this Deployment to strategy: Recreate. A rolling update creates the replacement pod before the outgoing one goes away, and a ReadWriteOnce volume can only be attached by one node, so the new pod would sit Pending on the attachment while kubectl reported a progressing rollout. Recreate stops the old pods first, which costs a short terminology outage at every upgrade and finishes. If your index volume is ReadOnlyMany, override it with terminology.strategy={type: RollingUpdate}. FerroTERM’s loading page covers the other code systems and their build flags. Size the memory limit for the edition before you mount it: terminology.resources ships a 1536Mi limit, and the same FerroTERM page states the resident size of each edition, which the limit must cover.

The fail posture is the CDR’s shipped one. terminology.failOnError defaults to false, so a binding the server cannot resolve is accepted; true turns an unreachable terminology server into a 422 refusal. The terminology page explains both. A code the server resolves as outside its value set is refused with 422 under either posture.

FerroTERM’s licence. BUSL-1.1 from the same Licensor as FerroEHR: non-commercial production use is free, any other production use needs a commercial licence. Its README says which is which. The licence for whatever code systems you load is separate and yours to hold; SNOMED CT needs an Affiliate Licence (free in Member countries, see snomed.org/get-snomed), and every surface showing its content carries the notice clause 8.3.1 prescribes plus the release version and date.

More than one replica needs two other keys. terminology.replicaCount is a throughput choice — each replica loads its own copy of the index, so watch the memory limit — and above 1 the pods spread one per node by default, the same soft constraint the CDR gets. Turn terminology.podDisruptionBudget.enabled on with it: without a budget a node drain may take every terminology replica at once, and the CDR’s own budget does not cover these pods. It is off by default because the shipped replica count is 1 and a budget over a single pod blocks the drain it exists to survive.

FerroTERM carries the same security context as the CDR and the viewer, and the chart’s render gate holds all three to the same Restricted profile. The deployment probe harness reads this workload back off a running cluster too: its applied runtime posture, the shaped seed reaching the process from the ConfigMap volume, a refusal on the terminology port from a pod the policy does not name, and the CDR resolving a bound code through the Service. Its image is pinned by digest in terminology.image.digest and moves on FerroTERM’s own release line, so it does not follow appVersion the way the viewer image does. Because that digest is set, terminology.image.tag alone deploys nothing: the digest wins, and a tag pointing anywhere else is refused at render rather than ignored. Clear the digest to deploy by tag.

FerroBRIDGE (the FHIR bridge, off by default)

FerroBRIDGE maps between FHIR R4 and openEHR with FHIRconnect mappings and reaches FerroEHR over ITS-REST. FerroEHR itself serves no FHIR API for clinical data, so FerroBRIDGE is the component that does. bridge.enabled renders it as its own workload in the same release: a Deployment, a ClusterIP Service, its own ServiceAccount, a ConfigMap holding its configuration file, and a NetworkPolicy.

The chart writes three sections of that file for you:

• [cdr]: base_url is this release’s CDR Service with its REST base path (http://<release>:<service.port><config.rest.base_path>), and [cdr.credentials] names a file mounted from your Secret.
• [terminology]: with terminology.enabled on, base_url is the chart’s FerroTERM Service at /r4 and wire_version is r4. With it off the section is absent, and you may set your own under bridge.config.
• [mappings]: directory is where the mapping files below are mounted.

The CDR credential comes from a Secret you create. The chart never takes it as a value. The Secret carries the keys user and password, for a user of the CDR’s config.auth.basic.users whose scopes grant what FerroBRIDGE calls. The password is mounted as a file and named by its _file key, so it is in neither the ConfigMap nor the environment. FerroBRIDGE does not treat the username as a secret, and the chart passes it as FERROBRIDGE__CDR__CREDENTIALS__USER. FerroBRIDGE reads an environment value as TOML when it parses as TOML, so a username made only of digits arrives as a number and is refused at boot. A static bearer token is no longer a choice: machine-to-machine authentication is HTTP Basic or the OAuth 2.0 client credentials grant.

kubectl -n ferroehr create secret generic ferroehr-bridge-cdr \
  --from-literal=user=bridge --from-file=password=./bridge-password

The mapping files come from a ConfigMap you create, named in bridge.mappings.existingConfigMap, with each key listed in bridge.mappings.files. The chart mounts every key on its own: a whole ConfigMap volume also carries the kubelet’s ..data snapshot directories, and FerroBRIDGE 0.0.5 reads those as a second and third copy of every file and refuses the start with a duplicate mapping name. FerroBRIDGE fetches each template a mapping context names from the CDR at boot, so upload the templates first; a template the CDR does not hold stops the start. Without a mapping set FerroBRIDGE still starts, and $tofhir and $toopenehr answer 503.

bridge:
  enabled: true
  cdr:
    existingSecret: ferroehr-bridge-cdr
  mappings:
    existingConfigMap: ferroehr-bridge-mappings
    files:
      - vital_signs.context.yml
      - blood_pressure.yml
  networkPolicy:
    ingressFrom:
      - podSelector:
          matchLabels:
            app.kubernetes.io/name: my-fhir-client

It stays inside the cluster until FerroBRIDGE authenticates its callers. FerroBRIDGE has no inbound authentication yet (FerroBRIDGE#400), and whoever reaches its port acts on the CDR with the credential above. So the chart renders no Ingress for it and no value renders one, the Service is always ClusterIP, and the NetworkPolicy admits only the peers you list in bridge.networkPolicy.ingressFrom. An empty list is refused at render, and the values schema refuses a peer that selects everything (an empty peer, or an empty podSelector or namespaceSelector inside one). There is no key that admits every source. Egress admits this release’s CDR, FerroTERM when it runs, and DNS; bridge.networkPolicy.extraEgress adds a rule for an upstream you name in bridge.config. When you narrow the CDR’s own networkPolicy.ingressFrom, the chart adds FerroBRIDGE’s pods to it, and FerroTERM’s policy admits them too.

Anything else goes in bridge.config, rendered into the same file with the keys of FerroBRIDGE’s configuration reference. FerroBRIDGE refuses an unknown key at boot. The render refuses a key the chart writes (cdr.base_url, cdr.credentials, server.listen, terminology while FerroTERM runs, mappings.directory while the chart mounts the mappings) and any secret-shaped key, such as a password, a token or cdm.url: set its _file sibling and mount the file with bridge.extraVolumes and bridge.extraVolumeMounts. FerroBRIDGE’s FHIR facade ([facade]) keeps its identity map in a file it writes, and the root filesystem is read-only, so turning the facade on also takes a PersistentVolumeClaim of yours mounted with those two keys and named in facade.identity_store, with bridge.strategy={type: Recreate} when the claim is ReadWriteOnce. Back that volume up with the CDR’s database.

Probes use FerroBRIDGE’s own endpoints: liveness /health/liveness, readiness /health/readiness, which answers 503 while the CDR or FerroTERM is unreachable. FerroBRIDGE carries the same security context as the other workloads and the release-wide hostUsers setting, and the render gate holds it to the same Restricted profile. Its image is pinned by digest in bridge.image.digest and moves on FerroBRIDGE’s own release line, never with appVersion; a tag other than the pin is refused while the digest is set. The deployment probe harness boots it on a cluster and reads back its runtime posture, its admission, the credential being used for the template fetch, a refusal on its port from a pod outside ingressFrom, and its readiness from a pod inside it. FerroBRIDGE’s own deployment documentation is at https://ferrobridge.eu/docs/operate/container.html.

Per-domain backups (one CronJob per domain, off by default)

backup.enabled renders one CronJob per pseudonymisation domain: backup.clinical.schedule dumps the clinical schemas, backup.party.schedule dumps the identities, backup.linkage.schedule dumps the party-to-EHR map, and each writes to its own existing claim (backup.clinical.persistentVolumeClaim, backup.party.persistentVolumeClaim, backup.linkage.persistentVolumeClaim). The image is backup.image.repository, which carries pg_dump.

A fourth job renders for the audit domain when database.audit.existingSecret gives it a database of its own. While the trail shares the clinical database the clinical dump names its schema and nothing more is needed; once it is somewhere else, that dump cannot reach it and nothing else would take it, so backup.audit.schedule, backup.audit.existingSecret and backup.audit.persistentVolumeClaim become required the same way the other three are, and the clinical dump stops naming the audit schema.

The chart refuses to render rather than let the separation collapse quietly: a missing claim is an error naming the value, and naming the same claim for any two domains is an error too, because one volume holding two of them — the clinical record, the identities, or the map that joins them — is the state per-domain backups exist to prevent.

Each job needs its own backup credential, named by backup.clinical.existingSecret, backup.party.existingSecret, backup.linkage.existingSecret and, when it renders, backup.audit.existingSecret, and the render is refused without them. It cannot be the pool’s credential: each domain’s runtime role is revoked from the other domains, so a dump taken through one would be silently partial. Give each domain a read-only role on that domain’s schema alone. Every job reads its DSN from a mounted file, never an environment variable.

The chart provisions no storage. Create the claims yourself, and give them different access control; that part no chart can do for you. The restore procedure, and the ferroehr db verify gate that judges it, are on the Operating page.

Staying available while things move

Five defaults keep the API serving through the events that routinely interrupt it. None needs configuring; each is listed because the reason matters when you tune it.

Replicas land on different nodes. With nothing telling the scheduler otherwise, two replicas can share one node and one node failure is a total outage. The chart ships a soft spread constraint (maxSkew: 1 over kubernetes.io/hostname, whenUnsatisfiable: ScheduleAnyway) so replicas prefer separate nodes but a single-node or full cluster still schedules them rather than leaving a pod Pending. Setting topologySpreadConstraints replaces it wholesale, so give the complete constraint including its own labelSelector; add a topology.kubernetes.io/zone entry if your cluster spans zones.

A terminating pod stops receiving requests before it shuts down. Deleting a pod removes it from the EndpointSlice and sends SIGTERM concurrently, and the removal still has to propagate to every node. preStopSleepSeconds (default 5) holds the container for that window first. It uses the native sleep hook action rather than an exec hook, because the image ships no shell to run one.

A node drain does not hang on unhealthy pods. The PodDisruptionBudget sets podDisruptionBudget.unhealthyPodEvictionPolicy: AlwaysAllow, the documented recommendation. The API default, IfHealthyBudget, makes a drain wait for pods to become healthy, which never completes when they are unhealthy because of the drain.

A migration interrupted by a drain is not counted as a failure. The migration Job carries a podFailurePolicy that ignores pod failures caused by disruption, so ordinary cluster maintenance during a release cannot exhaust migrations.job.backoffLimit and fail the upgrade with no migration error anywhere in the logs.

A reused connection is not closed under a client. The server closes a kept-alive connection only after config.server.idle_timeout_ms (120 s by default) with no request, and counts config.server.header_read_timeout_ms from the first byte of each request. Keep the idle limit longer than the idle timeout of your clients (90 s for a Go client) and of the ingress controller or load balancer in front, so they close an idle connection first. A client that sends a request into a connection the server is closing gets no answer, and a client that never retries a POST reports the commit as failed.

Two more are available and off by default. service.trafficDistribution: PreferSameZone keeps traffic inside the caller’s zone, for lower latency and inter-zone cost, at the price of even load, so measure before setting it. And autoscaling.behavior passes scaling policies straight through; the documented defaults already scale up immediately and wait out a five-minute stabilization window before scaling down, so change it only to be more conservative.

Upgrades

Migrations are append-only (a schema change is a new file, never an edit to an applied one) so a rolling upgrade stays compatible with the previous schema during the window where both versions run: additive DDL first, destructive changes in a later release once all pods are on the new version. Keep replicaCount at two or more (or autoscaling.enabled) and the default PodDisruptionBudget so upgrades and node drains never fully interrupt the API; the default terminationGracePeriodSeconds covers the binary’s shutdown drain. Roll back by re-pinning the prior image tag or digest.

A change anywhere under config, config.files or secrets changes the checksum/config pod annotation, so helm upgrade rolls the pods for a configuration-only change too, including a rotated secret or an edited ABAC policy, both of which are read at boot and would otherwise reach the volume while every running pod kept using the old value.

helm uninstall removes everything the chart created; the chart declares no PersistentVolumeClaim, so nothing is left behind; your database, and the Secret holding its DSN, are yours and survive.

Preview an upgrade against what you have installed with helm diff, or render the new chart version and read it:

helm template ferroehr oci://ghcr.io/ferrohealth/charts/ferroehr --version 21.0.0-rc3 \
  -n ferroehr -f my-values.yaml | less

Working from a checkout instead, deploy/helm/validate.sh runs the chart’s full render gate: the helm-version pin, lint, YAML validity, the structural Restricted-profile and selector-immutability gates, the secret-leak gate, the values-schema probes, and the golden-render diff. It is the same gate CI runs on every change to the chart, so a local run and a pull request agree by construction.

Check your values before you deploy them

A render that lints is not a deployment that boots. validate.sh never runs the server, so it cannot see a configuration the server refuses: a missing authentication mechanism, an HS* algorithm, a password hash that is not a real Argon2id PHC string, SMART enabled without its public base URL. Every one of those renders perfectly and crash-loops the pod. The script prints exactly which properties it does not check, on success as well as on failure, so a green run is not mistaken for a working deployment.

The check that closes that gap runs the image against your rendered configuration:

FERROEHR_IMAGE=ghcr.io/ferrohealth/ferroehr:5.6.0-rc3 \
  deploy/helm/ci/boot-check.sh my-values.yaml

It renders the chart, mounts the ConfigMap and the Secret exactly as the Deployment does (with their real values, not placeholders) replays the declared environment, and runs ferroehr config check inside the image. Point FERROEHR_IMAGE at the tag you intend to deploy: the answer is specific to that image, since a key your values carry may be newer than the server that has to read it.

It validates configuration only; it opens no socket, so it cannot tell you the issuer resolves, the broker is reachable or the database accepts the DSN. CI runs the same script over every values overlay the chart ships.

Cluster hardening

The Kubernetes chapter documents the chart. This section is the other half: an audit of this deployment against the OWASP Kubernetes Security Cheat Sheet, split by who can actually apply each control.

That split is the point. A workload chart controls its pod security context, its resource bounds, its NetworkPolicy, its ServiceAccount and how it consumes secrets. It cannot patch a node, set an API-server flag, configure etcd, authenticate a kubelet, or install an admission controller. For those, the honest deliverable is not a setting but a statement of what you owe and what happens if you do not, because a deployment that inherits an unchecked assumption is not secured by the chart’s own hardening.

Important

Several controls here are ones no application-level hardening can compensate for. If the kubelet accepts anonymous requests, or anything untrusted can read etcd, then every control this project ships (non-root, read-only rootfs, dropped capabilities, RBAC, ABAC, the audit trail) is bypassable, because the attacker is beneath the layer they operate in. Those are marked where they appear.

Claims about what the chart renders are checked against the chart’s own render gates. Claims about what a cluster then applies come from the deployment probe harness, which reads each answer from the layer that actually decides it (the container runtime’s own spec for the security context, the API server for admission, the EndpointSlice for readiness) and which declares in its own record what it did not exercise, so silence is never read as coverage.

The five pages

The ownership map

Final thoughts: the three practices, checked

The cheat sheet closes with three practices rather than controls. Checking them against what this project actually ships, rather than claiming them:

“Embed security into the container lifecycle as early as possible”: evidenced. Security here is CI, not a review checklist: the chart’s render gate parses every rendered object and asserts the Restricted profile per container, the golden renders pin the exact bytes, the secret-leak gate refuses a credential that would reach a ConfigMap, the image scanners run at build and again on a schedule against the published tags, cargo deny runs on every change, and zizmor and CodeQL read the workflows themselves. Each of those refuses a merge rather than filing a comment, and each of the chart-side ones has been watched to fail deliberately; a gate nobody has seen fail is a gate nobody knows works.

“Use Kubernetes-native controls to reduce operational risk”: evidenced. The chart’s controls are the platform’s own: a NetworkPolicy rather than an in-app firewall, a security context and the Restricted profile rather than a hardening sidecar, resource limits rather than in-process throttling alone, probes rather than an external watchdog, a user namespace rather than a trusted UID, and no service mesh because what one would add is either already provided or not needed at this shape. The one place we did not take a native control is Pod Security Admission, and that is because labelling a namespace is not a chart’s call, stated as the operator’s rather than quietly skipped.

“Leverage the context Kubernetes provides to prioritize remediation”: partly, and here is the honest version. The generic form of this practice is to rank findings by whether the affected code is reachable in your deployment. This project’s answer is the OpenVEX documents: when a scanner reports advisories in a privilege-dropping helper that opens no socket and parses no untrusted input, the response is a machine-readable not_affected statement with a controlled-vocabulary justification and an impact_statement a reader can check, not a silenced ignore list, and not a rebuild that fixes nothing.

That mechanism has a cost, and stating it is what makes the claim honest: a VEX statement is a claim about today’s binary, and it must be re-checked on every base-image bump. When upstream rebuilds that helper, statements about it become obsolete, and a stale not_affected is worse than no VEX at all: it is an argument that has quietly stopped being true while still suppressing its finding. The scheduled scan is what surfaces a finding whose statement no longer matches, and the re-check itself is a human obligation, not an automated one.

The cluster: hosts, control plane, access

The controls on this page are almost all the operator’s, and several are ones no application-level hardening can compensate for. Read them as what you owe the CDR running on top, not as background.

• Host hardening and the version window
• Upgrades roll, they do not replace
• Security advisories
• The dashboard we do not ship
• etcd and what our secrets contain
• Ports, theirs and ours
• Cluster API access
• Cluster RBAC, and why this chart needs none
• Kubelet access

Host hardening and the version window

Operator’s. Keep the node OS patched, hardened and firewalled; a workload chart cannot reach any of it. The cheat sheet’s list applies unchanged.

The part worth stating precisely is the version window. Upstream Kubernetes maintains release branches for the three most recent minor releases, each receiving roughly a year of patch support (kubernetes.io/releases). A cluster below that window receives no security backports at all: a published CVE in the API server or kubelet simply stays open on it.

The chart’s kubeVersion: ">=1.36.0-0" is a compatibility floor, not a statement about that window. It sits at the window’s newest release because 1.36 is where the newest field the chart renders (hostUsers, user namespaces) became stable, which is what lets every field it renders apply unconditionally instead of being gated into silence on the clusters that most needed it. The cost is real and worth naming: an operator one minor behind cannot install this chart. See Beyond Restricted: the user namespace for what that buys.

A version gate is a silent absence; a floor is a loud refusal. For a workload holding PHI the loud one is correct, so the chart declares the floor and refuses below it rather than installing with a safety property quietly inapplicable.

If you run outside the supported window, you have accepted that the platform beneath this CDR is unpatched, and no setting in values.yaml changes that.

Upgrades roll, they do not replace

Chart’s. The cheat sheet asks that a new version arrive by rolling update rather than by mutating a running container. The chart sets the strategy explicitly rather than inheriting the API server’s percentages:

strategy:
  type: RollingUpdate
  rollingUpdate:
    maxSurge: 1
    maxUnavailable: 0        # served capacity never drops

maxUnavailable: 0 means a replacement pod must pass its readiness probe before an old one is removed. At the default two replicas this is identical to what Kubernetes computed from 25% (it rounds down to zero); above two replicas the percentage default would have taken a pod out of service, and this does not. The trade-off is real and fails in the safe direction: on a cluster with no spare scheduling capacity a maxUnavailable: 0 rollout stalls visibly instead of proceeding at reduced capacity.

Observed on a live cluster rather than argued: a probe pod drove a long run of sequential requests at the Service through a full helm upgrade, with Killing/Started events for both replicas inside the window, and recorded no failed request, while the ReplicaSet history showed the old revision scaled to zero as the new one scaled up (a roll, not a recreate) and the Deployment reported Available=True (MinimumReplicasAvailable) throughout.

Security advisories

Operator’s, with a decision recorded on our side.

You must follow kubernetes-announce and the official advisory feed. Kubernetes CVEs are announced there, and nothing in this project will tell you about them.

This project does not track Kubernetes platform advisories, deliberately. The decision, so it is not mistaken for an oversight: we run no cluster and cannot act on a node or control-plane CVE, and a watcher that opened issues we could only close as “the operator’s” would be noise that trains people to ignore it. What we do track is what we ship: dependency advisories on every change, our own container images on a schedule, and the openEHR specifications through release watchers.

A vulnerability in Kubernetes itself is reported to Kubernetes, not to this project. A vulnerability in FerroEHR, including in the chart, comes to us, through the security policy published with the source.

The dashboard we do not ship

Operator’s. This chart installs no Kubernetes Dashboard, and nothing in it depends on one. If you install one, the cheat sheet’s conditions apply: never expose it publicly, give it a limited-privilege ServiceAccount, and put an authenticating reverse proxy in front of it if it must be reachable at all.

This section stays because the same reasoning applies to two surfaces that are ours, and an operator hardening “the dashboard” should find them here:

• /management/*: the ops-introspection surface (info, prometheus, metrics, env, loggers, flamegraph). It is a privileged read onto the deployment: env renders the effective configuration and flamegraph profiles the live process. The chart ships the master switch on and every endpoint off, so nothing is exposed until you name an endpoint and a level. Set config.management.port to move the whole surface onto its own listener so it is never reachable on the clinical API port.
• The viewer: a separate image with its own Deployment, which this chart can render but leaves off (viewer.enabled). It consumes the CDR strictly over the REST API and holds no database credential, but it is a privileged UI and belongs behind the same authenticating edge you would put in front of a dashboard, which is what viewer.auth.oidc.enabled and viewer.ingress.enabled are for.

etcd and what our secrets contain

Operator’s, and this is one of the controls that cannot be compensated for.

The cheat sheet’s requirements stand: mutual TLS between the API server and etcd, etcd reachable from nothing else, and separate instances or ACLs to bound what a component can read.

What makes it concrete for this deployment: anything that can read etcd can read every Secret in the cluster, and this release’s Secrets are not incidental. They hold the database DSN (the credential that reaches patient data) plus the version-signing passphrase, any Basic user’s Argon2id hash, and any terminology-server client secret. So “etcd is a cluster concern” is true and insufficient: for this workload etcd is the confidentiality boundary of the credentials that reach PHI. The full inventory is what our Secrets contain.

Two mitigations you can apply without touching etcd’s network posture:

1. Encryption at rest for Secrets: not on by default in Kubernetes. See Kubernetes’ encryption-at-rest configuration, which is a cluster-level setting an operator applies once.
2. An external secret manager, which removes the credential from etcd entirely: every secret this chart carries has a *_file route or an existingSecret route, so the value can arrive from a CSI driver or an operator-synced Secret rather than from chart values.

Ports, theirs and ours

Operator’s, for the cluster’s ports. Block untrusted access to the control-plane ports 6443 (API server), 2379-2380 (etcd), 10250-10257 (kubelet and controller/scheduler), and the worker ports 10248-10250. An exposed 10250 is the kubelet case below.

Ours, for the workload’s ports, and in the default posture the surface is one port:

Read off the running pod rather than the template, from the listening sockets in the container’s own network namespace: the default posture binds one port and nothing else:

LISTENING TCP ports in the container netns: [8080]

The shipped NetworkPolicy narrows inbound traffic to that port list. Whether the narrowing is enforced is a property of your CNI, not of the object: on a cluster whose network plugin does not implement NetworkPolicy the object is accepted, stored and displayed with no effect and no warning. Verify it the way the managed-cluster section shows: attempt a connection the policy should refuse, from a pod in another namespace, and require it to fail.

Two limits stated in full under Namespaces, network & policy §Ingress and repeated because they matter here: with networkPolicy.ingressFrom empty the rule carries no from and therefore admits every source, including other namespaces (only the port list is narrowed in that state; set networkPolicy.ingressAllowAll: false to have the chart refuse to render that state at all, and the same pair exists for the viewer under viewer.networkPolicy); and a NetworkPolicy is only as real as the CNI that implements it.

Cluster API access

Operator’s. Control access to the Kubernetes API: authenticate, then authorize, and deny by default.

• Recommended routes: OIDC, a managed-IAM integration, or user impersonation, with MFA on the identities that can reach the cluster API.
• Not suitable for production: static token files, long-lived X.509 client certificates, and service-account tokens used as human credentials. They cannot be revoked individually, they do not expire usefully, and they carry no second factor.

Worth drawing explicitly, because it saves a translation: this server implements the same shape. Authentication then authorization, deny by default, OIDC preferred over long-lived credentials, and a missing credential distinguished from a refused one (401 with a challenge versus 403). The Security chapter is the detail. An operator who understands why a static token file is a poor cluster credential already understands why config.auth.basic is a machine and bootstrap mechanism here and a config.auth.issuer is the production one for people.

Cluster RBAC, and why this chart needs none

Operator’s, for the cluster: run the API server with --authorization-mode=Node,RBAC and enable the NodeRestriction admission plugin, so a compromised kubelet cannot edit objects belonging to other nodes.

Ours, and it is an absence on purpose. The chart creates a ServiceAccount and no Role, RoleBinding, ClusterRole or ClusterRoleBinding at all, with serviceAccount.automountServiceAccountToken: false. That is not an omission to be tidied up later: the workload never calls the Kubernetes API, so it needs no permissions, and it is not given a token with which to try. The same holds for the viewer’s own ServiceAccount when that workload is enabled. Checkable on a live release:

helm get manifest ferroehr | grep -cE '^kind: (Role|RoleBinding|ClusterRole|ClusterRoleBinding)'
kubectl -n ferroehr get role,rolebinding

The first prints zero; the second reports no resources. And because the token is not mounted, no service-account token file exists under the pod’s volumes on the node.

If you are reviewing this chart and reaching for a Role to add: don’t. The correct fix for a future feature that genuinely needs the Kubernetes API is a Role enumerating exactly the verbs and resources it needs, plus turning the token mount back on for that ServiceAccount alone, not a broad grant added speculatively.

Kubelet access

Operator’s, and the second control no application hardening can compensate for.

Run every kubelet with --anonymous-auth=false and --authorization-mode=Webhook so its HTTPS endpoint is not open. Left open, that endpoint permits arbitrary command execution in any container on the node.

For this deployment, spelled out: an attacker reaching an unauthenticated kubelet gets a process-level foothold in a running CDR: the ability to read the database DSN out of the mounted secret files, to read patient data straight from memory, and to do so beneath the layer where authentication, RBAC, ABAC and the ATNA audit trail operate, so none of them see it and none of them can stop it. Non-root, a read-only root filesystem, an empty capability set and a private user namespace raise the cost of what happens next; they do not prevent the entry.

This is a cluster-configuration control, and it is worth confirming rather than assuming; an exposed 10250 is a routine finding in real clusters.

Images: build, provenance, scanning

The image half of the audit. Building a minimal image and signing it is ours; requiring a cluster to check that signature before running the image is yours, and this page carries the copyable policies for both engines that can do it.

• The build phase, and what distroless costs
• Image provenance at admission
  • The identity to trust, read off a real artifact
  • Kyverno
  • sigstore-policy-controller
• Continuous scanning of published images
• The supply-chain map

The build phase, and what distroless costs

Ours. Each build-phase control and what actually satisfies it:

Three images are published, and they are not equivalent in risk. The server and the viewer are distroless and carry almost no OS package surface. The PostgreSQL image is a thin, COPY-only layer over the upstream postgres image (it adds initialization scripts and nothing else) so its package set is upstream’s, and its CVEs arrive on upstream’s schedule rather than ours. The chart deploys none of the second and third: it takes an external DSN and can optionally render the viewer.

What distroless costs, stated before an incident rather than during one: there is no shell in the image, so kubectl exec … -- sh does not work. That is the security property working as intended (an attacker who achieves command execution finds no interpreter, no curl, no package manager) but it changes how you debug. Use instead:

• kubectl logs (the server logs JSON by default, for a collector),
• the always-on /health/readiness body, which names the failing dependency,
• /management/* for the effective configuration, live log filters and an on-demand CPU flamegraph,
• kubectl debug -it <pod> --image=busybox:1.37 --target=ferroehr starts an ephemeral container shares the target’s namespaces without adding a shell to the image that ships.

The registry posture: the images are public on GHCR, so a pull needs no credential and there is nothing to leak. Public does not mean trusted, which is the point of the next section: nothing about a public registry stops a cluster pulling a different image with the same name from somewhere else.

Image provenance at admission

The operator’s, and this is provenance nobody in your cluster currently checks.

The publishing lanes attest their artifacts through keyless Sigstore, so a verifier can establish that an artifact came from this repository’s build: the published images carry a signed SLSA v1 provenance attestation, and the chart carries both an attestation and a cosign signature over its digest. Nothing in a cluster requires that check before running an image, and a signature nobody verifies changes nothing about what actually runs. That is what the policies below close.

The identity to trust, read off a real artifact

Each lane signs with a short-lived Fulcio certificate whose subject alternative name is the workflow that issued the token, and whose issuer is GitHub’s OIDC provider. Read it off a published artifact rather than deriving it from a workflow file:

gh attestation verify oci://ghcr.io/ferrohealth/ferroehr:main \
  -R FerroHEALTH/FerroEHR --format json \
  | jq '.[0].verificationResult.signature.certificate
        | {subjectAlternativeName, issuer, sourceRepositoryRef, runnerEnvironment}'

{
  "subjectAlternativeName": "https://github.com/FerroHEALTH/FerroEHR/.github/workflows/build-image.yml@refs/heads/main",
  "issuer": "https://token.actions.githubusercontent.com",
  "sourceRepositoryRef": "refs/heads/main",
  "runnerEnvironment": "github-hosted"
}

The SAN’s ref varies with the trigger, and that is the part a policy gets wrong. Each lane runs on more than one ref, so each issues more than one identity:

All three prefixed with https://github.com/FerroHEALTH/FerroEHR/.github/workflows/, and all with issuer https://token.actions.githubusercontent.com.

The release binaries are signed by release-build.yml rather than by the release workflow because the build lives in a reusable workflow: the certificate names the workflow that owns the build definition, which is what makes the --signer-workflow pin below meaningful.

Pick the ref set deliberately, because the choice is a refusal. A policy matching refs/tags/v… only admits released images and refuses ghcr.io/ferrohealth/ferroehr:main, correct for production, and the reason a policy tested against :main appears broken when it is working. A staging cluster that runs :main needs both refs. Nothing accepts an arbitrary branch: refs/heads/main is exact, not a prefix match.

Kyverno

The engine chosen here. Two details decide whether this policy works at all:

• type: SigstoreBundle. These attestations are GitHub Artifact Attestations, stored in the Sigstore bundle format as an OCI referrer. Kyverno reads that format only under this type; the field defaults to Cosign, which looks for a sha256-<digest>.sig tag that these images do not have (it returns 404: the bundle is a referrer, not a cosign tag). Requires Kyverno 1.13 or newer.
• attestations:, not attestors: alone. Kyverno’s own rule is that “each verifyImages rule can be used to verify signatures or attestations, but not both”, and what the image lane produces is a signed attestation: there is no detached image signature. A rule with attestors: at the top level therefore fails closed on a perfectly legitimate image.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: ferroehr-image-provenance
  annotations:
    pod-policies.kyverno.io/autogen-controllers: none
spec:
  background: false
  webhookTimeoutSeconds: 30
  rules:
    - name: verify-ferroehr-provenance
      match:
        any:
          - resources:
              kinds: [Pod]
              namespaces: [ferroehr]
      verifyImages:
        - imageReferences:
            - "ghcr.io/ferrohealth/ferroehr"
            - "ghcr.io/ferrohealth/ferroehr:*"
            - "ghcr.io/ferrohealth/ferroehr-viewer*"
          # Sigstore bundle format, GitHub Artifact Attestations. Omitting
          # this defaults to Cosign, which looks for a signature that does not
          # exist and refuses every image.
          type: SigstoreBundle
          failureAction: Enforce
          attestations:
            - type: https://slsa.dev/provenance/v1
              attestors:
                - count: 1
                  entries:
                    - keyless:
                        issuer: https://token.actions.githubusercontent.com
                        # Released images only. For a staging cluster that runs
                        # the development tag, make the group
                        # `(heads/main|tags/v.+)`.
                        subjectRegExp: '^https://github\.com/FerroHEALTH/FerroEHR/\.github/workflows/containers\.yml@refs/(tags/v.+)$'
                        rekor:
                          url: https://rekor.sigstore.dev
              conditions:
                - all:
                    - key: '{{ buildDefinition.buildType }}'
                      operator: Equals
                      value: https://actions.github.io/buildtypes/workflow/v1

Add ghcr.io/ferrohealth/ferroehr-postgres* to imageReferences only if you run that image in the namespace; the chart never installs it.

failureAction sits on the verifyImages entry: the spec-level validationFailureAction is deprecated in the CRD (“use validationFailureAction under the validate rule instead”), and a verifyImages rule has no validate block. mutateDigest, verifyDigest and required all default to true, which is what you want: a tag is rewritten to the digest that was verified, and an image with no attestation is refused rather than passed.

sigstore-policy-controller

If you already run it. The equivalent two details:

• signatureFormat: bundle on the authority. The default is legacy (cosign’s own), which cannot read these attestations. Requires policy-controller v0.13.0 or newer.
• an attestations: entry, because in bundle format policy-controller supports only attestations, not plain signatures.

apiVersion: policy.sigstore.dev/v1beta1
kind: ClusterImagePolicy
metadata:
  name: ferroehr-image-provenance
spec:
  images:
    - glob: "ghcr.io/ferrohealth/ferroehr**"
  authorities:
    - keyless:
        url: https://fulcio.sigstore.dev
        identities:
          - issuer: https://token.actions.githubusercontent.com
            # Same group as the Kyverno policy: `(heads/main|tags/v.+)` for a
            # staging cluster that runs the development tag.
            subjectRegExp: '^https://github\.com/FerroHEALTH/FerroEHR/\.github/workflows/containers\.yml@refs/(tags/v.+)$'
      signatureFormat: bundle
      attestations:
        - name: require-slsa-provenance
          predicateType: https://slsa.dev/provenance/v1

Note

What has been checked, and what has not. The identity, the issuer and the predicate type above are verified first-hand against the published images with cosign verify --certificate-identity-regexp …, which admits them and refuses both an unsigned image and, under a tags-only pattern, a :main image, so the matcher is neither vacuous nor accidentally permissive. The manifests are checked field by field against the published ClusterPolicy and ClusterImagePolicy CRDs. Neither policy has been exercised by a running admission controller, and the Kyverno CLI is no substitute: kyverno test reports a verifyImages rule as Excluded and returns the same verdict whichever result you assert. Before enforcing, run the policy in Audit (Kyverno) or as a warn policy (policy-controller) long enough to see one real deployment pass.

The chart is signed too, and verifying it is a release-time check rather than a per-pod one: an admission controller sees pods and images, never the Helm artifact a human pulled. The chart’s own commands are on the Kubernetes page; its identity is the build-chart.yml row in the table above.

Should the chart ship an admission policy? No, and the reason is structural: an admission policy is cluster-scoped and governs workloads the chart knows nothing about, while the chart deliberately renders no cluster-scoped object at all (see namespaces). A ClusterPolicy in this chart would mean helm uninstall removing a control that other releases had come to depend on. The chart’s contribution is the policy document, here, versioned with the lanes whose identity it encodes.

Without an admission controller, the manual equivalent is a release-time check. Add --signer-workflow to insist on the lane as well as the repository; without it you are trusting that some workflow here signed the image:

gh attestation verify oci://ghcr.io/ferrohealth/ferroehr:main \
  -R FerroHEALTH/FerroEHR \
  --signer-workflow FerroHEALTH/FerroEHR/.github/workflows/build-image.yml

Substitute a vX.Y.Z tag for main on a release; the signer workflow is the same.

Important

Signing landed in the publishing lanes during the 3.17.4 cycle, so image tags from before it answer HTTP 404: Not Found, and there is nothing to verify, which is the correct answer and not a verification failure to work around. Pin a current version instead.

Then deploy by digest (image.digest), so what you verified is what runs: a tag can be moved afterwards, a digest cannot.

Continuous scanning of published images

Ours. CI scans images at build time, which catches what was known when they were built and nothing after. A CVE published the week after a release applies to the image people are running, so the published tags are re-scanned on a weekly schedule: all three images at the tag a user pulls, with the same severity floor and the same adjudicated exceptions as the build-time scan, and the OpenVEX documents applied so an accepted finding stays accepted with its argument attached.

A finding does two things, because either alone fails quietly: it opens (or comments on) a tracking issue, and it fails the run. A red scheduled run nobody looks at is not a control, and an issue with no failing check can be closed without the finding being addressed.

The PostgreSQL image is what this lane exists for, and it has already fired: the 3.17.5 release moved that image onto a rebuilt PostgreSQL 18.4 base to pull in current Debian packages, changing nothing about FerroEHR itself. The distroless images carry almost no OS package surface, and when the lane was written none of the three published images carried a fixable HIGH or CRITICAL finding.

The remediation side is standing machinery, not a per-incident scramble. The PostgreSQL image applies Debian security updates at its own build time, so an OS-package fix published between upstream base rebuilds reaches the image at the next release rather than waiting on someone else’s cadence; a second weekly lane watches the pinned base itself and opens an issue when a newer PostgreSQL patch tag exists or the pinned tag is re-pointed upstream; and the exact published-image scan is reproducible locally against a rebuilt candidate, so a fix is proven clean before it merges.

The supply-chain map

Each cheat-sheet supply-chain control, and the artifact that satisfies it, so a reader can check rather than trust:

Two gaps remain, stated here rather than left out of a page that otherwise reads as complete:

1. Nothing verifies the signatures at admission. We sign; no cluster is required to check before running an image. The policies to close it are above, and neither has yet been exercised by a running admission controller, which is why the instruction there is to run them in audit mode first.
2. Provenance exists only from the 3.17.4 cycle onward. Images published before the signing lane landed carry no attestation and never will, because a published artifact is not replaced. gh attestation verify on those returns a 404, which is the correct answer and not a verification failure to work around.

The workload: security context & admission

The pod-level controls: what the chart asks for, what the runtime actually applies, and the one command that turns compliance into enforcement.

• The security context, and what keeps it true
• Beyond Restricted: the user namespace
• AppArmor, and why it is not on by default
• Pod Security Admission: complying versus being refused
• Sandboxing is not a substitute for instance separation
• Kernel modules: already impossible

The security context, and what keeps it true

Ours, and settled. Read back from the running container through the container runtime’s own view of the pod, not from values.yaml:

process.user     : {'additionalGids': [65532], 'gid': 65532, 'uid': 65532}
noNewPrivileges  : True
capabilities.bnd : None            ← an EMPTY bounding set, not "default minus ours"
root.readonly    : True
seccomp default  : SCMP_ACT_ERRNO
RW mounts        : /tmp  (plus the kernel/kubelet-managed /proc, /dev/*, /etc/hosts)
RO mounts        : /etc/ferroehr  /etc/ferroehr-secrets  /sys  /sys/fs/cgroup

Three things worth drawing out. The capability bounding set is empty, so the drop is total at the kernel level rather than a subtraction from a runtime’s default set. readOnlyRootFilesystem: true needed exactly one writable path (the chart’s own /tmp emptyDir) and no per-integration surprise, which is what makes it safe to keep rather than the setting an operator relaxes during the first incident. And the server pod carries no init containers and no sidecars, so the context above is the whole pod.

What keeps it true is not this page. deploy/helm/validate.sh parses the rendered objects and asserts the Restricted fields per container, for every workload in the render (including the optional viewer and the migration Job) and the golden renders pin the exact bytes, so even a changed default fails a diff. Both run in CI on any change to the chart. That structure is deliberate: the gate this replaced grepped the rendered file for field names, so one compliant container vouched for every other one, and a second workload could ship non-compliant while the gate stayed green. The gate also fails when it finds nothing to check, because a pod-less render reporting “all containers compliant” is a false green, and it fails when a render carries no NetworkPolicy, or when a multi-replica Deployment has neither spread constraints nor affinity.

A template edit that drops securityContext.readOnlyRootFilesystem or securityContext.allowPrivilegeEscalation therefore fails a job, not a review.

Beyond Restricted: the user namespace

The Restricted profile stops a container from asking for privilege. It does nothing about what a container’s UID means on the node, and under the Kubernetes default, uid 65532 in the pod is uid 65532 on the host, so a container escape arrives as a real host user with whatever that user can reach.

The chart closes that by default. Every pod it renders carries hostUsers: false, which puts the pod in its own user namespace and maps its UID range onto an unprivileged host range. Read off a running pod:

$ cat /proc/self/uid_map
         0     838860800      65536
$ id
uid=65532 gid=65532 groups=65532

The process still sees uid 65532; the kernel sees an offset host UID inside that mapped range. Root inside the pod (which this workload never uses, but a compromised process might reach for) maps to the base of the range, which owns nothing. Capabilities granted inside the namespace do not apply outside it.

This is why the chart’s kubeVersion floor is 1.36: that is the release where user namespaces went stable (KEP-127), and the floor is what lets the field render unconditionally instead of being gated and silently absent on the clusters that most needed it.

If your nodes cannot support it, the pod does not start, which is the failure mode you want rather than a silent downgrade. The requirement is a Linux node whose runtime implements idmapped mounts (containerd 2.0 or newer, CRI-O 1.25 or newer). Set hostUsers: true to opt out; the chart then omits the field entirely rather than stating the API default, so a future cluster-wide default can still apply.

The same reasoning drives podSecurityContext.supplementalGroupsPolicy: Strict, so the process gets only the groups the manifest names: a group baked into an image cannot widen file access. And the chart’s render gate asserts that this isolation set is identical across every workload of a release: a viewer that shared the host user namespace while the server did not would be a posture nobody could state in one sentence, and that is exactly the shape of drift a second workload introduces. That is also why hostUsers is a release-wide key rather than one per workload.

AppArmor, and why it is not on by default

securityContext.appArmorProfile (stable since Kubernetes 1.31) is a further confinement layer, and it is left off deliberately, because it is not free: a node without AppArmor rejects the pod outright rather than ignoring the field. Observed on a node that does not carry it:

STATUS: AppArmor
Warning  AppArmor  pod/…  Cannot enforce AppArmor: AppArmor is not enabled on the host

Turn it on once you know your nodes carry it: most Debian and Ubuntu nodes do, while Docker Desktop and several minimal distributions do not:

securityContext:
  appArmorProfile:
    type: RuntimeDefault

Pod Security Admission: complying versus being refused

Meeting the Restricted profile and being refused when you stop meeting it are different properties, and only the first is ours.

Restricted compliance, field by field, so the claim is checkable against the standard rather than asserted:

The enforcement half is the operator’s, and it is one command:

kubectl label --overwrite namespace ferroehr \
  pod-security.kubernetes.io/enforce=restricted \
  pod-security.kubernetes.io/audit=restricted \
  pod-security.kubernetes.io/warn=restricted

The install notes print the same command as a prerequisite, with enforce-version=latest so the profile does not silently loosen as the cluster moves.

Why the chart does not do this itself, recorded so it is not re-litigated: Helm installs into a namespace that already exists (or one helm --create-namespace creates, which is the CLI’s action and not a template), so a chart that declared its own release namespace would fight the tool, and helm uninstall would then delete a namespace holding objects the release does not own. More fundamentally, a PSA label is namespace-wide policy: it governs every workload in that namespace, including backup jobs, sidecars and any database an operator colocates. A single application chart is the wrong scope to claim it.

Observed on a live cluster, in a namespace labelled enforce=restricted, with the database genuinely external to it:

• the chart installs and serves unchanged: both replicas running with no restarts, readiness UP, and a real openEHR write accepted. No admission warnings for its pods.

• a regression of the chart’s own pod spec is refused. Upgrading with --set securityContext.privileged=true --set securityContext.allowPrivilegeEscalation=true produced a ReplicaSet that could create no pods, and:

  Error creating: pods "ferroehr-…" is forbidden: violates PodSecurity
  "restricted:latest": privileged (container "ferroehr" must not set
  securityContext.privileged=true), allowPrivilegeEscalation != false
  

  while the healthy ReplicaSet kept both replicas serving, because maxUnavailable: 0 means the rollout cannot retire a good pod before a replacement is ready. The two controls compose: the label refuses the regression, and the strategy means the refusal costs no availability.

What you get by not applying the label: the chart still complies, but nothing enforces it. A future chart change, a stray --set securityContext.privileged=true, or a sidecar injected by other tooling would be admitted, and the first sign would be a running privileged container in the namespace holding your PHI.

One practical note from that test: a colocated database fixture is unlikely to be Restricted-compliant (the upstream postgres entrypoint must start as root), so labelling a namespace that contains one will refuse it. That is another reason the production posture puts the database outside the cluster.

Sandboxing is not a substitute for instance separation

Not required, and the reason matters more than the conclusion. The cheat sheet scopes sandboxing (Kata, gVisor, Firecracker) to clusters running untrusted workloads. This is our own code, so the threat it addresses (a container escape by hostile software you chose to run) is not the one in front of us. And the escape it hardens against is already narrowed by the pod’s own user namespace, which is on by default.

Important

A container sandbox does nothing to separate organisations, and that is the misreading worth preventing. FerroEHR is single-tenant: one instance serves one organisation, with its own database and its own domain roles. A sandbox draws a stronger boundary around one container, which is a boundary a single organisation is already alone inside. What separates two organisations is running two instances; the namespace model is where that is arranged.

A sandbox is worth considering in one case: a cluster where this workload runs beside third-party or customer-supplied code, and you want to protect this workload’s node from that. Note the cost first: a sandboxed runtime changes the syscall surface and the performance profile of a database-bound server, and the gVisor/Kata runtimes need a RuntimeClass the chart does not set (add it through your platform’s pod defaults if you adopt one).

Kernel modules: already impossible

Satisfied by the chart’s own posture, which is unusual for a host-side control and worth recording precisely rather than deferring.

Loading a kernel module requires CAP_SYS_MODULE. The container’s capability bounding set is empty (read off the running container, not off the manifest’s request) and allowPrivilegeEscalation: false with noNewPrivileges set means no setuid binary can regain it. The pod’s own user namespace makes the point twice over: a capability held inside it does not apply outside it. So this container cannot trigger a module load at all, whatever /etc/modprobe.d/ on the host says. An attempt would still be a runtime-detection signal; it just cannot succeed.

Host-side blacklisting remains good practice for the node, and stays the operator’s: it constrains every other workload on that node, including ones with capabilities this one does not have, and a privileged pod anywhere on the node can still load modules that affect this container’s kernel.

Namespaces, network & policy

Where the release’s boundaries are drawn: namespace scoping, how organisations are separated, the decisions not to adopt a mesh or a second policy engine, resource bounds in four nested layers, the ingress policy that narrows ports before it narrows sources, and deny-by-default egress: the one control on these pages that is not free.

• Namespaces, and one instance per organisation
• Service mesh: a recorded decision
• Centralized policy, and which engine
• Resource bounds: four layers
• Ingress: ports are narrowed, sources are yours
• Egress: deny by default, and what it breaks

Namespaces, and one instance per organisation

Ours, and satisfied by construction. Every object the chart renders is namespace-scoped: Deployment, Service, ConfigMap, Secret, ServiceAccount, NetworkPolicy, PodDisruptionBudget, HorizontalPodAutoscaler, Ingress, ServiceMonitor, the migration Job, and the viewer’s own Deployment, Service, ServiceAccount, NetworkPolicy and Ingress when that workload is enabled. There is no ClusterRole, no ClusterRoleBinding, no CustomResourceDefinition, no cluster-scoped object of any kind, and no template hard-codes a namespace: every reference resolves within the release’s own namespace. So two releases in two namespaces cannot collide, and neither can reach the other’s Secrets.

Separating organisations is a release per organisation, each in its own namespace with its own database. FerroEHR carries no in-database tenancy, and that is where openEHR puts the boundary: an openEHR system is one organisation’s repository, and multi-tenancy is something the infrastructure beneath it does with several such systems (BASE architecture_overview master06-design_of_the_ehr.adoc §The EHR System).

What that buys, against the in-process alternative it replaces:

The cost is real and it is the price of the boundary. A shared process scoped by a row predicate is cheaper and is not equivalent, however similar the wire behaviour looks.

Service mesh: a recorded decision

Not adopted, deliberately. The cheat sheet presents a mesh as a trade-off, not a requirement, and for this workload the trade lands clearly.

What a mesh would provide, and what already provides it:

Against that: a mesh is a second control plane, a sidecar in every pod (which the Restricted profile and the empty capability set then have to accommodate), and an opinionated platform to upgrade in lockstep. For one workload plus an external database, the lateral-movement problem a mesh exists to solve barely exists.

A genuine gap, named rather than glossed: a mesh would give workload identity: SPIFFE-style cryptographic identity per pod, so the database could authenticate the client workload rather than a shared password held in a Secret. Nothing in this deployment provides that; the DSN is a bearer credential, and any process that can read the Secret can use it. The mitigations available without a mesh are an external secret manager plus short-lived credentials, for example cloud IAM database authentication, where the DSN carries a rotating token rather than a password, bound through serviceAccount.annotations. If you run a mesh anyway, expect the overlaps above rather than double-implementing them, and note that its sidecar will need its own Pod Security accommodation.

Centralized policy, and which engine

Operator’s, and only for one of the three use cases a general policy engine is usually proposed for.

• Application authorization: already solved, do not add a second engine. This server ships a policy-driven authorization layer: RBAC, plus ABAC with an embedded Cedar engine or an external policy decision point. Adding a second engine for application decisions would mean two policy engines disagreeing about one question, and the one that loses is whichever is consulted second.
• Service-mesh authorization: moot, no mesh (above).
• Admission control: applies, and it is the same lever image provenance and Pod Security enforcement need.

Decision: Kyverno, chosen for what it must enforce rather than on general merit. It is the only one of the three candidates that covers both levers with one controller: verifyImages does keyless signature and attestation verification, and its policies can enforce pod-security constraints beyond what namespace labels express. sigstore-policy-controller does provenance only, so it would have to be paired with something else. Kubernetes’ built-in ValidatingAdmissionPolicy (CEL, no external controller) is attractive for pod shape and is the right tool for cheap structural rules, but it cannot verify signatures at all, because CEL evaluation makes no network calls and cannot reach a transparency log, so it cannot be the answer to the control that matters most here.

If you already run OPA/Gatekeeper for other reasons, keep it and add sigstore-policy-controller alongside for provenance; do not run two general admission engines. The copyable provenance policies are in Images, and Pod Security enforcement needs no engine at all: it is the namespace label.

Resource bounds: four layers

Split exactly. Container requests and limits are the chart’s; namespace ResourceQuota and LimitRange are cluster-admin objects, and a workload chart that created one would be claiming the whole namespace for itself, wrong the moment anything else shares it.

The chart’s bounds, and where they come from:

Those figures are for a modest replica. Tune them from your own metrics rather than treating them as a recommendation, and remember that raising replicaCount multiplies the request, which is what a namespace quota will notice first. The viewer has its own, smaller bounds under viewer.resources.

The layering is the point, and an operator should see it as one story: four nested bounds, each catching what the next cannot:

1. Per request: config.server.body_limit_bytes and config.rest.body_limit_bytes (413 on an over-large body), config.server.request_timeout_ms (408), config.query.timeout_ms and config.query.max_result_rows (a query cannot return an unbounded row set), config.db.statement_timeout_ms as the backstop the HTTP timeout cannot be, because dropping a handler future does not cancel the statement PostgreSQL is running.
2. Per connection / per caller: config.server.header_read_timeout_ms bounds a socket before a request exists and config.server.idle_timeout_ms a kept-alive one between requests; config.server.caller_rate refuses 429; config.server.max_concurrent_requests refuses 503 when the server is full.
3. Per container: the requests and limits above.
4. Per namespace: the operator’s, and the only layer that protects other workloads from this one:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: ferroehr
  namespace: ferroehr
spec:
  hard:
    requests.cpu: "2"
    requests.memory: 4Gi
    limits.cpu: "12"
    limits.memory: 8Gi
    pods: "12"
    count/services: "4"
---
apiVersion: v1
kind: LimitRange
metadata:
  name: ferroehr-defaults
  namespace: ferroehr
spec:
  limits:
    - type: Container
      # A pod that arrives with no limits gets these, so nothing in the namespace
      # can be unbounded by omission.
      default: {cpu: "1", memory: 512Mi}
      defaultRequest: {cpu: 100m, memory: 128Mi}
      max: {cpu: "4", memory: 2Gi}

Size the quota above replicaCount × requests with headroom for a rolling upgrade: maxSurge: 1 means one extra pod exists mid-rollout, and a quota with no room for it makes upgrades stall rather than fail, which looks like a hung deployment. The migration Job needs its share too when migrations.job.enabled is on.

Ingress: ports are narrowed, sources are yours

The chart’s mechanism, your peers: and the one control on this page whose default looks stronger than it is.

networkPolicy.enabled ships on, and the policy it renders admits inbound traffic to the API port (plus the management port when that runs on its own listener) and to nothing else. That half is unconditional. The other half is not: the rule’s sources are narrowed only when you set networkPolicy.ingressFrom. While that list is empty the rule carries no from clause at all, and in the NetworkPolicy API a rule with no from admits every source, other namespaces and off-cluster clients included, not “any pod in this namespace” (NetworkPolicies).

So the shipped posture is: ports restricted, sources open. Nothing about kubectl get networkpolicy, the object’s own name, or a summary that says “NetworkPolicy: enabled” distinguishes that from a policy that restricts both: which is why the chart now states it as a value you can see and change rather than leaving it implicit in an empty list.

Narrow it to whatever fronts the CDR:

networkPolicy:
  enabled: true
  ingressFrom:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: ingress-nginx   # your controller's namespace
    # - ipBlock: {cidr: 10.0.0.0/16}                   # or a CIDR, for off-cluster callers
  ingressAllowAll: false                               # refuse to ever render the open rule

The viewer does not need an entry: when viewer.enabled is on, the chart appends the viewer’s own pod selector to whatever you list, because a narrowed policy that forgets it locks the viewer out of the CDR and presents as the CDR being down.

The viewer’s own policy works exactly the same way, under viewer.networkPolicy.ingressFrom and viewer.networkPolicy.ingressAllowAll, with the same refusal on false + empty. Its egress half is genuinely closed (the CDR Service and DNS, nothing else) but its ingress half ships open like the CDR’s, and what sits behind it is a login page. Narrow both.

Important

networkPolicy.ingressAllowAll: false makes “no open ingress” a checked fact. With it set, an empty ingressFrom is refused at render time, the same treatment egress gives a policy with no database destination, and for the same reason: an absent selector is a decision, and this one is invisible in the place an operator would look for it. Set it in the values file of any deployment where an open ingress rule would be a finding, and a later edit that drops your ingressFrom fails the install instead of quietly reopening the port.

It ships as true, which is the honest name for what the chart has always rendered: a stock helm install produces the open rule, and now says so, in the values file, in the object’s kubernetes.io/description, and in the notes printed at install. A non-empty ingressFrom always narrows, whatever this key is set to; it decides the empty case only.

Warning

A NetworkPolicy is only as real as the CNI that implements it. On a cluster whose network plugin does not enforce NetworkPolicy, the object is accepted, stored and displayed with no effect and no warning. Verify by attempting a connection the policy should refuse, from a pod in another namespace, since that is precisely the source an empty ingressFrom admits.

Egress: deny by default, and what it breaks

The mechanism is the chart’s; the destinations are yours: and this is the control where a NetworkPolicy stops being free.

Enabling networkPolicy.egress.enabled refuses all outbound traffic except what is listed. DNS is always included, and the database is a first-class value because it is the only destination that is never optional:

networkPolicy:
  egress:
    enabled: true
    database:
      to:
        - ipBlock: {cidr: 10.0.12.7/32}     # a managed database
        # or, in-cluster:
        # - podSelector: {matchLabels: {app.kubernetes.io/name: postgres}}
      port: 5432
    rules: []                                # one entry per integration, below

Important

Enabling egress with no destinations at all (neither networkPolicy.egress.database.to nor networkPolicy.egress.rules) is refused at render time rather than at rollout, because that mistake presents as a database fault: readiness reports the database down, the log shows a connect timeout, and nothing mentions the network policy. Supplying rules but omitting the database destination renders, so keep the database entry where it belongs rather than folding it into a raw rule where it is easy to lose.

The destination set, derived from the configuration tree rather than from what a default install happens to use. Every row except the usage report is off unless its key is set, so add only the rows you have switched on:

The viewer, when enabled, carries its own egress policy rather than appearing in this table: it admits the CDR Service, DNS, and outbound HTTPS for an identity provider. Narrow that last rule to your issuer’s address if you can.

Warning

A NetworkPolicy cannot match a DNS name. The API selects peers by pod, namespace or ipBlock CIDR only, so every off-cluster row above needs a CIDR you supply and keep current. A managed database that moves IP, or a terminology server behind a rotating CDN address, will break under a CIDR that was correct when written. Where a provider publishes no stable range, the honest options are an egress gateway with a fixed address, or leaving egress off and accepting that outbound traffic is unrestricted, not a 0.0.0.0/0 rule that pretends to be a policy.

Two failure modes worth knowing before you tighten this.

An over-tight policy silently stops observability. A blocked OTLP exporter does not fail the request that generated the span: it drops the span, with no log line and no error. So a policy that forgets the collector produces a server that is healthy by every check and has quietly stopped being observable. If you enable egress and traces disappear, look at the policy before the collector.

Tightening egress under a running pod appears to work when it has not. A NetworkPolicy is enforced on new connections; existing conntrack flows survive. So a pod whose connection pool is already established keeps serving after you remove its database rule, and fails at the next restart, which may be a node drain in the middle of the night. Observed on a live cluster:

policy: DNS + database  → readiness 200 {"status":"UP", db UP, migrations UP}, POST /ehr 201
remove the database rule (DNS only), pod untouched
                        → readiness STILL 200, db STILL "UP"   ← the pool survives
delete the pods so a fresh one must connect
                        → readiness 503, no replica available
                          kubelet: Readiness probe failed: HTTP probe failed with statuscode: 503
restore the rule        → ready again, with no restart

So: verify an egress policy by restarting a pod, not by watching the one that is already running. (The recovery in the last line needing no restart is the readiness check re-testing its dependencies on every probe, the same property described for migrations.)

Secrets, detection & response

What this deployment’s Secrets actually contain, how to notice a compromise on a shell-less image, what to do when you have one, and where a managed control plane takes controls out of your hands.

• Secrets at rest, and what ours contain
• Runtime detection on a shell-less image
• Replica deviation and the outbound inventory
• Breach containment and rotating credentials
  • Rotating each credential
  • Rotating the version signing key
• Logging: two streams that are not interchangeable
• On a managed control plane

Secrets at rest, and what ours contain

Operator’s: encryption at rest is an API-server flag (--encryption-provider-config), and Kubernetes does not encrypt Secrets by default. A Secret is base64-encoded, which is an encoding and not a protection: without that configuration, everything below sits readable in etcd, which is why the etcd section is the other half of this one.

The useful half is ours: exactly what this deployment’s Secrets contain, so you can judge the exposure rather than reading “secrets” generically.

The rendered ferroehr.toml itself is not in that list, and that is worth stating because it used to be: while a Basic user’s hash had nowhere secure to go, configuring one moved the whole configuration file into a Secret. Every credential the server models now has either a *_file sibling or a Secret-borne environment route, so no key takes that branch today; the chart keeps it only so that a secret key added upstream tomorrow fails safe instead of landing in a ConfigMap.

The first row is the one that matters most, and it has a property worth naming: the DSN is a bearer credential. Any process that can read it can use it, from anywhere the database is reachable: there is no binding to the workload that was issued it. That is the same gap named under service mesh (workload identity), reached from the other direction, and the mitigation is the same: a credential that is short-lived and issued to a workload identity rather than a long-lived password in an object.

Enable encryption at rest with an EncryptionConfiguration on every API server. Prefer a KMS provider over aescbc/secretbox with a local key: a key sitting in a file on the control-plane node is protected by the same boundary as the etcd data it encrypts. On a managed cluster this is usually one setting (envelope encryption with the provider’s KMS); check whether it is on, because it generally is not by default. Existing Secrets are only re-encrypted when rewritten, so follow the documented kubectl get secrets --all-namespaces -o json | kubectl replace -f - step, or the encryption applies to new writes only.

Or remove them from etcd entirely. Every secret this chart carries has a *_file route or an existingSecret route, so no code change is needed to source them from a secret manager:

• A CSI driver (Secrets Store CSI Driver with the Vault, AWS, Azure or GCP provider) mounts the value as a file. Point extraVolumes and extraVolumeMounts at it and set the matching *_file configuration key; config.signing.key_passphrase_file, config.multimedia.secret_access_key_file, the terminology grant’s client_secret_file. Nothing reaches a Kubernetes Secret at all.
• An operator that syncs into a Secret (External Secrets Operator, Vault Agent Injector) still lands in etcd, so it buys rotation rather than removal: worth having, but pair it with encryption at rest.
• The DSN is mounted too, from database.existingSecret, so a CSI-provided Secret carries it like any other. What a mount does not fix is that the DSN remains a bearer credential: cloud IAM database authentication is the route that removes the standing password (the DSN then carries a short-lived token), and serviceAccount.annotations exists for the IRSA or Workload Identity binding that needs.
• config.audit.fhir_feed.url is the only credential-bearing key with no *_file sibling, so it is the one value still passed as an environment variable.

Finding exposed secrets is already covered in CI rather than left to an operator: Trivy’s secret scanner runs over the whole tree and over every published image, so a credential committed to the repository or baked into a layer fails a job. The deliberate development credentials were checked against it and are not flagged, so nothing is exempted to make it pass.

The volume-versus-environment half of this control (mounting secrets as read-only files rather than passing them as environment variables) is covered on the Kubernetes page and is already the chart’s behaviour for every secret whose configuration key has a *_file sibling.

Runtime detection on a shell-less image

Operator’s tooling (Falco, Tetragon, or a managed equivalent), but the signals are unusually high-confidence here, and that is what makes it worth more than a recommendation.

The runtime image is distroless and shell-less. There is no sh, no bash, no curl, no package manager, and the container runs one process. So the usual heuristics stop being heuristics:

A starter Falco rule set exploiting exactly that:

- macro: ferroehr_container
  condition: container.image.repository endswith "/ferroehr"

- rule: FerroEHR unexpected process
  desc: Any process other than the server binary in a shell-less image
  condition: spawned_process and ferroehr_container and proc.exepath != "/usr/local/bin/ferroehr"
  output: Unexpected process in FerroEHR container (proc=%proc.exepath parent=%proc.pname container=%container.id)
  priority: CRITICAL

- rule: FerroEHR write outside tmp
  desc: The root filesystem is read-only; /tmp is the only writable mount
  condition: open_write and ferroehr_container and not fd.name startswith "/tmp/"
  output: Write outside /tmp in FerroEHR container (file=%fd.name proc=%proc.exepath container=%container.id)
  priority: CRITICAL

- rule: FerroEHR unexpected outbound connection
  desc: Outbound to a destination outside the configured inventory
  condition: >
    outbound and ferroehr_container and
    not fd.sport in (53) and not fd.sport in (5432, 4317, 4318, 443, 5671, 5672, 6514)
  output: Unexpected egress from FerroEHR container (dest=%fd.rip:%fd.rport proc=%proc.exepath)
  priority: WARNING

Tune the port list to the destinations you actually enabled; the point of the third rule is that the list is short enough to be worth writing. The macro matches the viewer’s image too, whose only expected process is its own server binary.

What each layer sees, and cannot. The syscall layer sees process execution, file writes and raw connections (a compromise of the container) but it has no idea which patient’s record was read, because to it every request is bytes on an established socket. The ATNA audit trail sees exactly that: which subject accessed which EHR, through which operation, under which authenticated identity: but it is emitted by the application, so a compromise deep enough to control the process can stop or falsify it. They are complementary and neither substitutes: runtime detection is how you learn the process is not itself any more; the audit trail is how you answer what was accessed while it still was. Forwarding audit records off-box to an external repository (config.audit.syslog, config.audit.fhir_feed) is what keeps the second answer available after the first alarm.

Replica deviation and the outbound inventory

Operator’s practice, from material the server already publishes: which makes it more actionable here than the generic advice.

Replicas of this Deployment are interchangeable: same image, same configuration, traffic distributed by the Service. So a metric that differs per pod is a signal, and the Prometheus surface is already per-pod (each pod is its own scrape target). Worth alerting on a divergence between pods rather than on an absolute value:

The management surface’s prometheus endpoint (opened with config.management.endpoints.prometheus) is the source, and metrics.serviceMonitor.enabled is how an operator-managed Prometheus discovers it. Two things worth knowing: the ATNA audit trail gives a second, independent view (an access pattern that deviates per pod is visible there at patient-and-operation granularity) and a pod that fails readiness leaves the Service, so “one pod has zero traffic” can mean it is unready rather than unreachable.

The outbound inventory the traffic half of this control asks for is the egress table, derived from the configuration tree. In the chart’s default posture, read from node conntrack on a running pod, the complete set is two destinations: TCP 5432 to the database and UDP 53 to cluster DNS. Nothing else. That is what makes a deny-by-default egress policy tractable: the base allowance is two rules, and every addition is a named, configured endpoint rather than an open range. Compare live traffic against the policy periodically: a connection the policy permits and nothing makes is a rule to remove.

Breach containment and rotating credentials

Scaling to zero is a clinical-safety decision, and it should be made before you need it. kubectl scale deploy/ferroehr --replicas=0 is the Kubernetes-native containment action, and for this workload it means clinical access stops immediately: no reads, no commits, for everyone. That is the point during a breach, and it is also an outage of a system clinicians may be depending on at that moment. Decide in advance who is authorized to make that call.

What scaling to zero does:

• stops all new requests, including whatever the attacker is doing through the API;
• leaves the database untouched: it is external, so its data, its contents and its own access controls are unaffected;
• preserves the pod’s evidence only partially: scaling to zero terminates the pods, so anything in memory is gone. To preserve a pod for forensics, cordon its node and remove the pod from the Service by editing its labels instead; the ReplicaSet then creates a replacement while the original keeps running, detached from traffic.

What it does not do:

• it does not undo committed data. openEHR change control is append-only, so a malicious commit is a new version, not an overwrite. The prior version is still there and still retrievable.
• it does not stop an attacker who has the DSN. The database is reachable independently of these pods; a leaked DSN is used from anywhere the database admits, which is why rotating it (below), not scaling, is the containment action for that particular compromise.
• it does not truncate the audit trail. Records already written to the local store, or already forwarded to an external repository, survive. Records still in the outbox at termination are drained during the grace period, so a scale-to-zero loses less than an abrupt kill; forwarding to an external repository (config.audit.syslog, config.audit.fhir_feed) is what makes the trail survive the pods entirely.

Rotating each credential

Every secret except two is a mounted file, so rotation is: update the Secret, then restart the pods. The restart is not optional: configuration is read at boot, and Kubernetes propagating a new Secret into the volume does not make a running process re-read it:

kubectl -n ferroehr create secret generic ferroehr-db \
  --from-literal=FERROEHR__DB__URL='postgres://…new…' --dry-run=client -o yaml \
  | kubectl apply -f -
kubectl -n ferroehr rollout restart deploy/ferroehr

(A rotation you make through helm upgrade needs no explicit restart: any change under config, config.files or secrets moves the checksum/config pod annotation and rolls the Deployment by itself.)

Rotating the version signing key

In the default digest mode there is no key: the version signature is a hash of the canonical form, recomputed at read time. Nothing to rotate, and nothing breaks.

In pgp mode there are two mechanisms, and the first is the one to reach for.

The ordinary path: rotate the signing subkey. OpenPGP is built for this. A certificate is a primary key plus its subkeys (RFC 9580 §10.1), the primary key certifies while a subkey signs, and rotating means issuing a new signing subkey on the same certificate. The retired subkey stays in the certificate, so every version it signed keeps verifying, with no configuration change, no second key file, and no window where history is unreadable. It is also the only path that keeps the primary key’s identity intact; replacing a primary key discards everything that has ever been said about it.

# add a fresh signing subkey to the existing certificate
gpg --quick-add-key <FINGERPRINT> ed25519 sign 1y
# revoke or expire the previous one, then re-export BOTH halves
gpg --export-secret-keys --armor <FINGERPRINT> > signing.asc

Update the mounted Secret and roll the pods. The server signs with the newest signing-capable subkey and verifies against every subkey the certificate carries, so the switch is invisible to readers.

The exception: a whole certificate is replaced. A compromised primary key, an organisational change, or migrating from a different signer means a genuinely new certificate, and then the old one must be retained, because a stored signature carries no key identifier and a version signature is an immutable committed fact that cannot be re-issued. Keep the retired public key:

[signing]
mode = "pgp"
key_path = "/etc/ferroehr/signing.asc"                  # the new certificate
retired_key_paths = ["/etc/ferroehr/signing-2025.pub.asc"]   # public, verify-only

Under Helm these are config.signing.key_path and config.signing.retired_key_paths, with both files mounted through config.files.

Note

Retired entries are public keys, which is what makes the safety property structural rather than a promise: no secret key is loaded for them, so a retired certificate can verify and can never sign again. Verification does not become permissive either: a signature matching neither the active certificate nor any retired one still fails, and tampered content still fails.

This is the same mechanism Debian uses for its archive: debian-archive-keyring ships current and retired keys so packages signed under an older key stay verifiable (Debian archive keys).

What neither mechanism covers. Key expiry and revocation are a different problem: a strict verifier arguably should accept a signature made while the key was valid and reject one made after, and nothing in a detached signature proves when it was made. Solving that needs trusted timestamps (RFC 3161), the approach Sigstore takes with short-lived keys and a timestamp authority. FerroEHR does not implement it, so treat an expired signing key as a certificate replacement and use config.signing.retired_key_paths.

If you are mid-rotation and need reads to keep working before either mechanism is in place, config.signing.verify_on_read: warn downgrades a verification failure from a 5xx to a logged and metered event (version_signature_invalid_total{verdict="pgp_invalid"}). That is a deliberate, recorded reduction in an integrity guarantee, not a setting to leave on.

Logging: two streams that are not interchangeable

Container logging is ours, and already the right shape. The server writes to stdout and stderr and never to a file inside the container, which is both the Kubernetes logging architecture’s expectation and a requirement of readOnlyRootFilesystem: true: there is nowhere to write a log file. Set config.telemetry.format: json (the chart’s default) for a collector; pretty is for a terminal.

The distinction that matters, because getting it wrong loses the accountability record:

Important

Do not treat the audit trail as “logs”. A collector configured to sample a noisy stream, or to drop under volume, is a reasonable policy for diagnostics and a compliance failure for the audit trail: it silently discards the record of who read which patient’s data. The two travel by different paths precisely so that one can be lossy: the audit trail does not go through stdout at all. If you also ship audit records to your log platform for convenience, that copy is a convenience, not the record.

The audit trail’s own failure behaviour is configurable, and the default is worth knowing rather than inheriting: config.audit.fail_mode defaults to open, so an operation whose audit record cannot be written still proceeds, and the failure is metered rather than refused. closed answers 503 instead, the stronger compliance posture, and one that turns an audit outage into a clinical outage. Which is correct depends on whether your regulatory position can tolerate an unaudited access more or less than a refused one; it is a policy choice either way, and the shipped default chooses availability.

Cluster API audit logging is the operator’s. Enable it on the API server with an audit policy (None / Metadata / Request / RequestResponse per rule). Metadata for most resources with Request-level detail for Secret and RBAC changes is a reasonable starting shape. Two things worth alerting on specifically: authorization failures (Forbidden responses, a principal probing what it can reach), and any read of Secrets in this namespace by a principal that is not the kubelet, since that is what reading the DSN looks like from the API side.

Kubernetes Events are a third source, and distinct from both: they are the cluster’s account of what happened to your objects, they expire (typically an hour), and they are where this chart’s failures show up first: Readiness probe failed: HTTP probe failed with statuscode: 503 when a dependency is down, FailedCreate … violates PodSecurity "restricted:latest" when a pod spec regresses under enforcement, Unhealthy and Killing during a rollout. Check kubectl get events --sort-by=.lastTimestamp before the application log when a pod will not start: the reason is usually there, and it is usually not in the log, because the container never ran.

On a managed control plane

On EKS, GKE, AKS or an equivalent, several controls in this audit stop being yours: you cannot set API-server flags, reach etcd, or configure kubelet authentication. In exchange you inherit the provider’s defaults, which may be stronger or weaker than this sheet assumes, and which you should verify rather than assume.

Check the CNI before relying on the shipped NetworkPolicy. This is the item that varies most and fails most silently: a NetworkPolicy on a cluster whose network plugin does not enforce it is an object the API accepts, stores and displays, with no effect and no warning. Provider defaults differ, versions change them, and some require enabling enforcement at cluster creation, which cannot be changed afterwards on some platforms. Do not read your provider’s documentation and conclude; test it:

kubectl create namespace ferroehr-probe
kubectl -n ferroehr-probe run probe --image=busybox:1.37 --restart=Never --command -- sleep 600
kubectl -n ferroehr-probe exec probe -- nc -w3 -z <ferroehr-pod-ip> 5432   # must fail
kubectl -n ferroehr-probe exec probe -- nc -w3 -z <ferroehr-pod-ip> 8080   # must succeed

If the first command succeeds, the policy is decoration and every claim in this section that rests on it is void for your cluster. This is also the reason the project’s own deployment probe harness declares NetworkPolicy enforcement as something it does not exercise: a green result on one cluster’s CNI would say nothing about yours.

Provider audit tooling exists: for EKS, hardeneks is the commonly cited one. We have not run it, so it is named as a starting point rather than a recommendation: treat its output as input to the same ownership question this section asks, not as a verdict.

From source

You can build the ferroehr binary yourself: for a platform without a published image, for local development, or to run the test suite. This chapter covers the prerequisites and the build. Most operators should prefer the published container images (Docker Compose, Kubernetes & Helm); build from source when you need to.

Prerequisites

• The pinned Rust toolchain. The repository pins Rust 1.99.1 (edition 2024) via rust-toolchain.toml, so rustup installs and selects it automatically the first time you build in the checkout; you do not choose a version by hand.
• Docker: required only for the integration tests, which spin up a real PostgreSQL 18 in a container.
• xmllint: required only for the canonical-XML tests.

Building

From the repository root:

cargo build --workspace

To build just the server binary in release mode (what the container image ships):

cargo build --release --locked -p ferroehr-server

The binary crate is ferroehr-server; the executable it produces is named ferroehr, so the resulting binary is target/release/ferroehr. It links a pure-Rust TLS stack (no OpenSSL, no JVM, no runtime dependencies) so it drops into a minimal base image or runs directly on the host.

Running the tests

cargo nextest run --workspace

Every database-backed test takes its database from a shared harness: one PostgreSQL 18 server, one migrated template database, and a fast clone per test. By default the harness starts (or re-adopts) a single reusable postgres:18 container, so Docker must be running; reclaim it afterwards with docker rm -f ferroehr-testkit-pg18. To use a PostgreSQL 18 server you already run instead, point the suite at it and Docker is not needed at all:

export FERROEHR_TEST_PG_URL='postgres://user:password@localhost:5432/postgres'
cargo nextest run --workspace

The role in that DSN must be allowed to CREATE DATABASE.

Running the binary

The binary is configured entirely through FERROEHR_* environment variables (see the configuration reference). At minimum it needs a database URL:

export FERROEHR__DB__URL='postgres://ferroehr:ferroehr@localhost:5432/ferroehr'
target/release/ferroehr

It runs its schema migrations at boot and then serves on the configured bind address (default 0.0.0.0:8080). That boot-time migration is a setting, not a fixture: db.migrate = "verify" makes the server issue no DDL at all, for a deployment whose database role has none of those rights (see Operations).

Two global flags and three subcommand groups round the CLI out:

• --config <path> points at a configuration file, overriding FERROEHR_CONFIG (with neither, the server reads no file);
• --set <key>=<value> is a repeatable dotted-path override with the highest precedence of all (--set db.max_connections=40);
• ferroehr config check validates the effective configuration and prints it redacted, exiting 0 when valid and 1 otherwise, the fastest way to test a deployment’s configuration before starting it;
• ferroehr db migrate applies the embedded migrations and exits, the out-of-band schema step, run under a ferroehr_migrator DSN;
• ferroehr db verify checks, without issuing any DDL, that the database carries exactly this build’s migrations, exiting 0 when it does;
• ferroehr healthcheck (used by the container healthcheck and Kubernetes exec probes) asks GET /health/readiness on the configured server.listen port over the loopback and exits 0 on 200, 1 otherwise.

Note

Building from source gives you the same binary the images use: the container Dockerfile pins its Rust version from the same rust-toolchain.toml, and CI cross-checks the two so they cannot drift.

Build features

The server builds with three additive cargo features, all on by default: fhir (the FHIR terminology providers, and the FHIR AuditEvent the audit store, the audit feed and the ITI-81 retrieval carry), events (contribution-outbox eventing and the AMQP transport), and multimedia (DV_MULTIMEDIA externalization to S3-compatible object storage). Their implementations live in a separate crate that the platform library pulls in only when the matching feature is on, so a build without them contains none of their code.

A slim build compiles them out entirely:

cargo build --release --locked -p ferroehr-server --no-default-features

A slim binary refuses loudly at boot if the configuration enables an integration it was built without: multimedia.enabled, events.enabled, audit.store.enabled, audit.fhir_feed.enabled, or a configured external FHIR terminology provider. The syslog ATNA feed and the in-process terminology bundle remain available in slim builds.

Warning

The local audit store is on in the shipped defaults, so a slim build refuses to start on an untouched configuration. To run one, disable it explicitly (FERROEHR__AUDIT__STORE__ENABLED=false) or configure the syslog feed as the audit sink; silently dropping the audit trail is not a boot mode.

Configuration reference

FerroEHR is configured by one file, ferroehr.toml, whose sections cover the entire server, with FERROEHR_* environment variables (and repeatable --set flags) as per-key overrides on top. This page is the entry point: the quickstart, how configuration loads and how environment names map onto the file, the one key that selects the openEHR specification generation set, and a map of which page documents which section.

• Quickstart
• How configuration loads
  • The environment-variable mapping
  • File discovery
  • Strict validation
• spec_profile
  • Why it is one key and not three
  • Seeing which profile is active
  • What changes on the wire
  • Changing the profile on an existing deployment
• deployment_profile
• Where each section is documented

Quickstart

Copy the annotated template, edit it, and run:

# A fully-commented ferroehr.toml with every key at its default.
curl -fsSLo ferroehr.toml \
  https://raw.githubusercontent.com/FerroHEALTH/FerroEHR/main/app/ferroehr/assets/ferroehr.default.toml

# Edit it — at minimum set db.url and an auth mechanism.
$EDITOR ferroehr.toml

# Validate without touching the database, then run.
ferroehr config check --config ferroehr.toml
ferroehr --config ferroehr.toml

A server started with no file and no environment still boots, with one exception you will hit immediately: authentication is on by default and a server with no mechanism configured refuses to start. See Zero-config boot and the production checklist.

How configuration loads

Configuration is assembled once at boot from four layers, lowest precedence to highest:

1. Built-in defaults: the values in the tables on the section pages.
2. The config file: ferroehr.toml (see file discovery).
3. FERROEHR_* environment variables: override individual keys.
4. --set key=value CLI flags (repeatable), which win over everything.

Some conventional names sit below their FERROEHR_ forms within layer 3:

• DATABASE_URL → db.url, and RUST_LOG → telemetry.filter.
• PORT → a 0.0.0.0:<PORT> value for server.listen (the port container platforms such as Vercel, Cloud Run and Heroku inject and route traffic to).
• The libpq environment set (PGHOST, PGUSER, PGPASSWORD, PGDATABASE, PGPORT, PGSSLMODE), which managed-Postgres integrations such as Neon inject: when PGHOST is set and no URL form is, the server assembles the DSN from them. An explicit DATABASE_URL beats the assembled form, and FERROEHR__DB__URL beats both.

Nothing else has a non-FERROEHR_ name.

The environment-variable mapping

Every key has one mechanical environment spelling: FERROEHR + the TOML path, upper-cased, with a double underscore (__) between every segment, including after the FERROEHR prefix. A single underscore only ever appears inside a key word.

Scalars are typed automatically (bool / int / float, else string). List-typed keys take comma-separated values (FERROEHR__AUTH__AUDIENCE=ferroehr,other). Map-keyed tables are reachable too: the map key is just another segment (FERROEHR__AUTHZ__ABAC__POLICY__COMPOSITION__NAME). Arrays of tables (the Basic-auth user store) are file-only, because the environment grammar has no way to spell an array index.

Note

Enum values are lowercase / snake_case tokens, exactly as the tables show. Secret-typed keys are redacted everywhere the configuration is rendered (the /management/env snapshot, GET /ferroehr/rest/extensions/v1/admin/config, ferroehr config check, ferroehr report, logs), and a secret-URL key shows its host and path with the embedded credentials masked. Each secret, and each database and broker URL, has a *_file sibling that reads the value from a file, for Kubernetes and Docker secret mounts. Setting a key and its *_file sibling at once is a boot error.

File discovery

The server reads the file --config <path> names, or else the file FERROEHR_CONFIG=<path> names; later layers still override its values. A named file is fatal if absent, unreadable or unparseable. With neither set, the server reads no file and searches no directory: the environment and the defaults make the whole configuration.

Strict validation

Configuration is validated at boot (and by ferroehr config check), and the server refuses to start on any error:

• Unknown keys are rejected: in the file, naming the key and the closest valid one, and in every FERROEHR__… override, so a misspelled security key can never be silently ignored. Only the uniform FERROEHR__<SECTION>__<KEY> form sets a key. Any other FERROEHR_ variable, such as the single-underscore FERROEHR_DB_URL, sets nothing and is left alone, because the viewer’s FERROEHR_VIEWER_ variables share the prefix.
• Type errors are boot errors, naming the key and what was expected.
• Semantic errors are aggregated: one pass reports every problem at once, so a broken configuration is fixed in a single iteration.

spec_profile

# The openEHR specification generation set the server runs.
spec_profile = "development"   # or "stable"

openEHR publishes released specification versions and keeps developing the next ones. FerroEHR generates both, and this key decides which set the running server serves.

Environment form: FERROEHR__SPEC_PROFILE=stable. The key is a top-level scalar, not a section; there is no [spec_profile] table.

Why it is one key and not three

The components’ generations are modelled against each other, not independently: RM 1.1.0’s own machine-readable model declares that it includes BASE 1.2.0. Letting you pick RM 1.1.0 with BASE 1.3.0 would offer a combination openEHR never published, so the profile is a single coupled choice and incoherent sets are unrepresentable.

Seeing which profile is active

The profile is reported in two places, so it is never a guess:

• the boot banner, on every start, alongside the RM version it serves;
• GET /management/info, which names the active profile with the RM and BASE generations it selects, next to the build provenance.

What changes on the wire

The profile is an acceptance boundary, and it is exact in both directions.

Under stable, a query that addresses specification surface the released generations do not define is refused with a typed error naming the active profile: an AQL FROM class or a path attribute RM 1.1.0 does not declare is rejected at planning time rather than answered as though it existed.

Released surface the development line later dropped stays accepted under stable, which is the half most implementations get wrong. A demographic party carrying the RM 1.1.0 PARTY.reverse_relationships attribute is read and accepted under stable; development refuses it as an undeclared key, because RM 1.2.0 removed it. The attribute is derived data the server recomputes from relationships, so the copy you send is validated and then dropped rather than stored.

Reading a stored object is bounded the same way. Every commit records whether the released generations can express the body it accepted, and under stable a stored version they cannot is refused with 409 Conflict naming the active profile, the version, and the remedy; never served under a generation set that does not define it, and never rewritten to fit one. Under development the same object reads normally. This is FerroEHR’s own extension: no openEHR specification governs runtime version selection, so the status follows HTTP itself (RFC 9110 §15.5.10, a conflict with the current state of the target resource, whose resolution the response describes).

Queries take the same refusal wherever they serve a version body. AQL is gated in two places, and they answer different questions. At planning time the query text is checked: a FROM class or path attribute the released generations do not declare is rejected. At result assembly the projection is checked: a whole-object projection (SELECT c FROM EHR e CONTAINS COMPOSITION c) returns stored version bodies, so if any row of the page comes from a version the released generations cannot express, the whole query answers 409 Conflict naming that version. The row is never quietly dropped from the result set instead: a RESULT_SET is columns and rows of values with nowhere to explain a missing row, so silently eliding one would be an answer you could not tell from “no such data”.

A leaf projection over the very same rows (SELECT c/name/value FROM EHR e CONTAINS COMPOSITION c) still answers 200. It serves data values rather than version bodies, over paths the planning gate has already bounded to the released generation’s declared surface. That is the honest boundary: the profile is an acceptance boundary on what is served as an openEHR object, not a content filter over the values inside one.

Under development none of this applies and it costs nothing: the assembly gate returns before it touches the database.

The exact additive delta between the two generation sets is pinned in the build, so a future openEHR re-vendoring cannot silently widen or narrow what a profile accepts.

Changing the profile on an existing deployment

Treat the profile as a deployment commitment. Both directions are defined, but they are not symmetric:

If you need to stay on released specifications, choose stable on day one rather than migrating into it later. Silently rewriting stored clinical content to fit an older generation would be data loss disguised as a setting, so no tool does it.

Objects committed before this stamp existed, and objects written by the verbatim-replay paths (EHR-Extract import, archive load), carry no recorded answer; they are assessed at read instead, which costs one extra parse per read of such an object and only under stable. Nothing is written back: a read stays a read.

Note

No openEHR specification governs runtime version selection; this key is FerroEHR’s own design. What the specifications do govern is the compatibility direction it relies on: minor releases within a major line are additive supersets.

deployment_profile

# The declared posture: what this deployment may hold.
deployment_profile = "sandbox"   # or "production"
deployment_accepts = []          # production only: gaps run without, by name

Every separation FerroEHR can make is a configuration key an operator can leave unset, and a deployment that has made none of them looks, from its own logs and its own API, like one that has made all of them. This key gives the server a declared posture and makes the two impossible to confuse.

The separations, each a real property the server checks rather than a box ticked by being present:

An accepted gap is stated on every boot and on /management/status; it can be run, not hidden. Environment form: FERROEHR__DEPLOYMENT_PROFILE=production, FERROEHR__DEPLOYMENT_ACCEPTS=shared_cluster. There is no research value: the key controls rigour, not purpose, and a secondary-use platform under an EHDS data permit runs on real patient data at production rigour whatever it is for.

This is FerroEHR’s own posture, not a legal requirement. GDPR Art. 4(5) asks that the additional information needed to re-identify a person be “kept separately and … subject to technical and organisational measures”, not that it sit on a separate server; one cluster with separated schemas and roles is a defensible reading. Two clusters are materially stronger, because a superuser, an instance-wide point-in-time recovery and a single compromise are bridges no grant can close, and that is a choice a deployment should make deliberately rather than inherit from a quickstart. See Compliance.

Where each section is documented

Server, database & telemetry

The listener ([server]), the REST surface ([rest]), the PostgreSQL connection ([db]), the observability section ([telemetry]), and the usage report ([usage_report]). Precedence, the environment-name grammar, and file discovery are on the Configuration reference index.

• [server]
  • [server.caller_rate]: per-caller request rates
  • [server.tls]: native TLS and mutual-TLS client authentication
• [rest]
  • base_path: shortening the REST base path
  • Behind a path-prefixed reverse proxy
  • system_id: the data-authoring identity
  • [rest.identity]
• [db]
• [storage]
• [telemetry]
• [usage_report]

[server]

The HTTP listener: the section every FerroHEALTH product shares (ferrohealth-server), so the keys and their behaviour are the same in each. It carries the stack every request passes through: the server’s request id, the request timeout, the body limit, the concurrency bound, compression, the request log, the health routes and the bounded drain on SIGTERM.

[server]
listen = "0.0.0.0:8080"
base_path = ""
request_timeout_ms = 30000
header_read_timeout_ms = 30000
idle_timeout_ms = 120000
body_limit_bytes = 67108864
drain_delay_ms = 0
shutdown_timeout_ms = 10000
# max_concurrent_requests = 256
retry_after_s = 1
trusted_proxies = []
forwarded_header = "forwarded"
compression = true
compression_min_bytes = 1024

A refusal the listener writes itself (408, 413, 429, 503) and a handler that panics (500) answer the standard openEHR { error, message } body. 413 and 429 are not in the ITS-REST status table; ITS-REST admits them as additional, non-conflicting codes, and they are what RFC 9110 §15.5.14 and RFC 6585 §4 define for these refusals. A panic’s message reaches neither the body nor the log.

Every request gets a request id the server mints, which the request log and the access record carry and the response returns in X-Request-Id. A client’s own X-Request-Id is returned to it unchanged and never recorded.

[server.caller_rate]: per-caller request rates

[server.caller_rate]
requests_per_second = 1024
burst = 2048

Unset, no caller is limited. The chart and the Compose quickstart set the values above.

This is not max_concurrent_requests, and you can tell them apart from the status alone. max_concurrent_requests protects capacity: too many requests in flight at once, from anyone, and the excess is refused 503 + Retry-After. The caller rate protects fairness: one caller asking too often over time, refused 429 + Retry-After. A 503 means the server is full; a 429 means you are asking too fast.

The limit is keyed on the authenticated subject, the only fair key for a clinical API: a hospital behind one NAT is a single address. It sits inside authentication, so the health routes, the status document and the discovery document are never refused by it.

Tip

Benchmarking this server? Lift the caller rate first, or you will measure it instead of the server. The project’s own measurement lanes compose an overlay that does exactly that, and both instruments refuse to write a record if the server answered any 429.

[server.tls]: native TLS and mutual-TLS client authentication

Native TLS termination on the main listener, unset by default; deployments commonly terminate TLS at an ingress. client_auth = "required" is the IHE ATNA ITI-19 mutually-authenticated-node posture (see the Audit trail chapter). The separate-port management listener always stays plain HTTP, so under deployment_profile = "production" setting [management] port opens the plaintext_listener gap.

[server.tls]
certificate_file = "/etc/ferroehr/server.pem"
key_file = "/etc/ferroehr/server.key"
client_auth = "none"
min_version = "1.3"
# reload_interval_s = 60

Before 5.5.0, client_ca_file alone meant that every client presents a certificate. From 5.5.0, client_auth decides it and defaults to none, so a configuration carried over from an earlier release set client_ca_file and no client_auth, and 5.5.0 served it without asking a client certificate. Such a configuration is now refused at load. Set client_auth = "required" beside client_ca_file to keep mutual TLS (GHSA-52mf-qg39-4q4j).

ferroehr healthcheck speaks TLS to a TLS listener over the loopback and accepts exactly the leaf certificate in certificate_file.

Note

min_version defaults to 1.3 only, as RFC 9325 §3.1.1 has it supported. Setting "1.2" enables 1.2 alongside 1.3, with the four forward-secret suites RFC 9325 §4.2 recommends; pick it only for a client that cannot do 1.3, such as an older integration engine or a pinned Java runtime.

Note

An EC certificate must name its curve. RFC 5480 §2.1.1 forbids explicit curve parameters, TLS clients refuse such a certificate, and the server refuses the pair at startup. LibreSSL (the openssl that ships with macOS) writes explicit parameters from genpkey and req -newkey ec unless you add -pkeyopt ec_param_enc:named_curve; openssl ecparam -name prime256v1 -genkey writes a named curve.

[rest]

The openEHR REST surface the listener serves.

[rest]
base_path = "/ferroehr/rest/openehr/v1"
swagger_ui = "private"
cors_permissive = false
system_id = "ferroehr.local"
# body_limit_bytes = 16777216

base_path: shortening the REST base path

ITS-REST leaves the API base to the deployment and fixes only the version segment at its end, so you may shorten base_path. The shape is checked at boot, and a value that breaks a rule stops the server with an error naming the key and every rule it broke:

• The first segment is ferroehr. /ferroehrx/v1 and /x/ferroehr/v1 are refused. This segment is not configurable away.
• The last segment is v1, the openEHR API version this server implements. The shortest accepted value is therefore /ferroehr/v1.
• No trailing slash, no empty segment (//), and every segment uses only the unreserved URL characters A-Z a-z 0-9 - . _ ~.

The status and documentation routes hang off the REST root, which the server derives from base_path by dropping the segments that name the openEHR API: the trailing v1, plus an openehr segment directly before it when you spell one.

The Swagger UI ({rest root}/swagger-ui), the OpenAPI documents ({rest root}/api-docs/…) and the SMART discovery document ({rest root}/.well-known/smart-configuration) move with it. The health family is served under server.base_path, which is empty by default, so it stays at the process root: /health, /health/liveness, /health/readiness.

Set it from the environment with:

FERROEHR__REST__BASE_PATH=/ferroehr/v1

Note

The ferroehr healthcheck subcommand, which the published container images run as their Docker health check, asks GET /health/readiness on the server.listen port over the loopback, so rest.base_path does not move it.

Behind a path-prefixed reverse proxy

A proxy that mounts the CDR under a prefix stacks that prefix on top of base_path. A proxy serving the server at /cdr with the default base path gives clients URLs like:

https://cdr.example.org/cdr/ferroehr/rest/openehr/v1/definition/template/adl1.4

Two ways to shorten that. Strip the prefix at the proxy, so the server sees the path it serves (nginx proxy_pass http://cdr:8080/; with the trailing slash, Caddy handle_path, Traefik StripPrefix). Or shorten the base path:

FERROEHR__REST__BASE_PATH=/ferroehr/v1

which gives https://cdr.example.org/cdr/ferroehr/v1/definition/template/adl1.4.

Whichever you choose, tell every client. The server builds its Location headers, its served OpenAPI paths and its SMART discovery document from its own rest.base_path, and it cannot see a prefix the proxy adds. The FerroEHR Viewer has its own mirror key, cdr.base_path, which must match.

system_id: the data-authoring identity

system_id is the identifier this CDR stamps into the data it authors. It appears on the wire in three places:

• EHR.system_id, recorded when an EHR is created. The openEHR RM (EHR Information Model, EHR Identifier Allocation) says the EHR.system_id “should be set to the value that would normally be used for locally created EHRs”: a value the deployment chooses, not a product constant.
• AUDIT_DETAILS.system_id on every commit for which the client did not supply one through the openehr-audit-details header. The openEHR REST API requires that “when system_id is not provided by the client, the server MUST set it to its own configured system identifier”.
• OBJECT_VERSION_ID.creating_system_id: the middle segment of every version identifier the server mints (<object_id>::<creating_system_id>::<version>).

Practical notes:

• The value must be a legal openEHR UID: a UUID, an ISO OID, or an internet id (a reverse-domain / DNS-style name) per the openEHR BASE identification grammar. The server judges it with the same validating constructor its reader uses and refuses to boot on an illegal value, because it becomes the creating_system_id segment of every version identifier the server mints and an illegal value would produce ids the server’s own reader rejects. A DNS-style name like cdr.hospital.example is valid; an empty value, or one containing the :: field separator, is not.
• Choose it before going live and keep it stable. The value is stored with each EHR and each version; changing it later affects only newly authored data: existing EHR ids, audit rows and version identifiers are never rewritten, and previously issued OBJECT_VERSION_IDs stay valid.
• Make it unique per system, so data exchanged between openEHR systems keeps unambiguous provenance. A node of a federation must hold a system_id no other member uses (Federation Tier N42a); the default ferroehr.local is the same on every node that leaves it unset. See FerroEHR as a federation node.
• system_id is not [rest.identity]. system_id says which system authored the data; [rest.identity] is the display identity of the OPTIONS System-Options manifest. Rebranding changes the manifest and nothing in stored data; changing system_id changes what new data says about its origin and leaves the manifest alone.

[rest.identity]

The System-Options manifest identity (OPTIONS on the API base path, e.g. OPTIONS /ferroehr/rest/openehr/v1, the System API’s one location). This is the deployment’s display identity only; the identifier stamped into authored data is system_id.

The defaults are derived from the build rather than typed into the handler, so the manifest never out-claims what was actually measured. Override them only to rebrand.

[db]

PostgreSQL connection.

[db]
url = "postgres://ferroehr:ferroehr@localhost:5432/ferroehr"
migrate = "apply"
max_connections = 20
min_connections = 2
acquire_timeout_secs = 30
statement_timeout_ms = 60000

statement_timeout_ms is the backstop for every statement. An AQL query (ad hoc or stored) and the composition-count statistics stop sooner when their caller goes away: a client that disconnects, or the AQL engine’s own query.timeout_ms, closes the statement’s connection, its pool slot frees at once, and every pooled session sets PostgreSQL’s client_connection_check_interval to one second, so the backend stops the statement within about a second. Other statements run to their end or to this timeout, so keep it above query.timeout_ms: the AQL engine’s typed refusal then fires first and this only catches what the engine does not govern.

[storage]

One DSN per storage domain. Every domain always lives in its own schema (clinical, party, linkage, audit) and every pool carries only its own on its search_path, so no statement can reach another domain’s relations without naming a schema it is not granted. Setting a url here adds the CREDENTIAL separation, and — if the DSN names another host — lets the domain live in a database or cluster of its own.

[storage.party]
url_file = "/run/secrets/db-party-dsn"

[storage.linkage]
url_file = "/run/secrets/db-linkage-dsn"

Why a schema is not always enough: a base backup, WAL archiving and physical replication carry every schema of a database together (PostgreSQL 18, Backup and Restore), so a separation that must survive those is a separation of databases. The Swiss EPDV Art. 10 Abs. 1 lit. b asks for storage “von anderen Datenbeständen getrennt” and the DSV Art. 4 Abs. 5 for the log to be kept “getrennt vom System, in welchem die Personendaten bearbeitet werden”; openEHR reads the same way for the identity cross-reference, which “could be located on different machines” (BASE architecture_overview/master07-security.adoc §Anonymity). No openEHR spec governs pools or database roles — this is our own design.

What the server checks at boot:

• a runtime role that can read another domain’s relations is refused, always;
• two domains configured on different DSNs that turn out to authenticate as the same database role are refused — a separation that exists only in the configuration is worse than none, because it reads as one that holds;
• a missing domain role is a warning under deployment_profile = "sandbox" and a refusal under production, where there are no grants to separate anything with.

Each database is prepared on its own migration credential. [db] migrate_url names one database; a domain whose DSN reaches another database is prepared there, and verified there under migrate = "verify", on the credential named in [storage.migrate.<domain>]:

[storage.linkage]
url_file = "/run/secrets/db-linkage-dsn"           # postgres://ferroehr_linkage:…@linkage-db/…
[storage.migrate.linkage]
url_file = "/run/secrets/db-linkage-migrate-dsn"   # postgres://ferroehr_migrator:…@linkage-db/…

The server refuses to start when a domain’s url and its storage.migrate URL reach different databases. A database that holds the linkage domain without the party domain also receives the party migration set, its schema and grants and never a row, because the linkage set revokes a function the party set creates.

[telemetry]

The console, the request log and the OpenTelemetry export: the section every FerroHEALTH product shares (ferrohealth-telemetry). With no [telemetry.otlp] table nothing is exported.

[telemetry]
format = "auto"
filter = "info,hyper=warn,tower=warn,h2=warn"
logged_query_parameters = []
# environment = "production"

# [telemetry.otlp]
# endpoint = "http://collector:4317"
# traces = true
# metrics = true
# sample_ratio = 1.0

The service.name of every span and metric is ferroehr, and service.version is the release and its commit.

The request log writes one line per request: the method, the matched route template, the status, the latency and the server’s request id, never a body, the raw path or a header value. It records the http.server.request.duration histogram, scraped as http_server_request_duration_seconds with the http_request_method, http_route, http_response_status_code and url_scheme labels.

Tip

The ASCII boot banner follows the rendering that is actually installed, not the configured word: it prints for pretty, and for auto only when stdout is a terminal. Under json, and under auto off a terminal (a container, a pipe into a log collector) stdout is parseable JSON from the first byte.

Note

Instrument names carry no Prometheus suffix; the exporter derives one. A counter named auth_failures is scraped as auth_failures_total, and units add _seconds/_bytes. Over OTLP the unsuffixed name is what a collector receives.

The scrape endpoint itself is not opened here; it is management.endpoints.prometheus, which is off until you name a level.

[usage_report]

The daily installation report to FerroPULSE, the collector Cadasto B.V. runs. It is on by default, and every boot logs one line saying whether it is on. What a report carries, when it is sent, the network it needs and the privacy notice are on the Usage report page.

[usage_report]
enabled = true
endpoint = "https://report.ferropulse.eu/v1/report"
deployment = "unknown"

slow_aql_ms is no longer a key: the report carries no AQL series. A configuration that still sets it is refused at boot; remove the line.

Under the Helm chart, set the switch with usageReport.enabled. The chart writes enabled and deployment = "helm" into the rendered file itself and refuses to render if either is also set under config.usage_report; endpoint passes through config.usage_report unchanged.

Authentication & access

Who may call the server, what they may do once identified, and the surfaces that are gated rather than always-on: [auth], [authz], [admin], [smart], [management], and [signing]. Precedence, the environment-name grammar, and file discovery are on the Configuration reference index.

• [auth]
  • Opaque tokens by introspection
  • The Basic-auth user store
• [authz]
  • [authz.assurance]: the assurance level patient data requires
  • [authz.professional]: the natural person behind a token
  • [authz.rbac]
  • [authz.abac]
• [admin]
• [smart]
• [management]
• [licence]
• [signing]
  • Choose an Ed25519 signing key

[auth]

Authentication: bearer tokens of one or more trusted issuers, Basic credentials, or both. [auth] is the section every FerroHEALTH product shares, checked by the ferrohealth-smart crate. FerroEHR is a resource server: it checks tokens and never issues them.

[auth]
enabled = true
issuer = "https://keycloak.example.com/realms/ferroehr"
audience = ["ferroehr"]

[[auth.basic.users]]
user = "clinician"
password_hash = "$argon2id$v=19$m=19456,t=2,p=1$…$…"   # never a plaintext password
scopes = ["user/composition-*.cruds", "user/template-*.cruds", "user/aql-*.cruds"]

Warning

auth.enabled = true with no mechanism is a boot error. Such a server could only refuse every request while advertising an authentication scheme it does not implement, which RFC 9110 §11.6.1 forbids: a 401 challenge must name a scheme applicable to the target resource. Configure an issuer, configure [[auth.basic.users]], or set auth.enabled = false for a development server.

The key-set fetches verify the identity provider’s certificate against the platform trust store and follow no redirects. An identity provider whose certificate comes from a private CA is trusted once that CA is in the image’s system store or in a directory or file named by SSL_CERT_DIR or SSL_CERT_FILE. A key set that cannot be read when a token needs it answers 503, never 401: the server cannot decide, and a 401 would tell a caller with a valid token that its credential was refused.

A validated token must also carry a non-blank sub claim: the authenticated subject is stamped into the audit trail, so a token without one is refused with 401 rather than recorded under a placeholder identity. A bearer token outside the RFC 6750 §2.1 b64token alphabet is refused the same way, with 401 and error="invalid_token".

Opaque tokens by introspection

An authorization server that issues opaque access tokens is served with mode = "introspection". Each bearer token is posted to the issuer’s introspection endpoint (RFC 7662 §2.1), and the active response stands in for the JWT claims: sub, scope, the roles of role_claims, aud against audience and iss against the issuer. An inactive token is a 401. An endpoint that cannot answer is a 503, because the server cannot decide. An active response is reused until its exp, at most cache_max_age_seconds.

[auth]
issuer = "https://idp.example.org/realms/ferroehr"
audience = ["ferroehr"]
mode = "introspection"

[auth.introspection.client]
client_id = "ferroehr-rs"
client_secret_file = "/run/secrets/introspection-client"

The Basic-auth user store

[[auth.basic.users]] is an array of tables and therefore file-only: the environment grammar cannot spell an array index.

[auth.basic] itself takes one key beside the users:

Warning

Every password_hash must meet the OWASP Argon2id floor: m>=19456 (19 MiB), t>=2, p>=1, algorithm argon2id. Anything weaker (or a non-argon2id PHC string, or an unparsable one) is a boot error naming the user. This is checked at startup because the verifier takes its cost parameters from the stored hash, so a deliberately cheap hash would otherwise verify happily and silently weaken every password in the store. The floor is the OWASP Password Storage Cheat Sheet §Argon2id minimum.

An unknown user-id is checked against a decoy hash of the same cost, so the time a refusal takes does not say whether the user exists.

[authz]

Role-based (RBAC) and attribute-based (ABAC) authorization, and the patient-data assurance gate. The full evaluation order and design rationale are in Security.

[authz.assurance]: the assurance level patient data requires

Off by default. What it refuses and the 401 challenge it sends are described in Security.

[authz.assurance]
minimum = "substantial"

[authz.assurance.levels]
"http://eidas.europa.eu/LoA/substantial" = "substantial"
"http://eidas.europa.eu/LoA/high" = "high"

Boot refuses a minimum with an empty levels table or with no value mapped at or above it (every patient-data request would be refused), a blank claim beside a declared table, and a levels value that is blank or carries whitespace, a quote or a backslash, because the refusal names the accepted values in one quoted, space-separated acr_values parameter (RFC 9470 §3).

[authz.professional]: the natural person behind a token

Absent by default, and tokens are then not judged on it. Present, a patient-data request whose token names no natural person is refused with 403, unless it is a client token acting for a named professional.

The access record then carries the acting mode and the professional; see Audit trail. Both tables are reachable from the environment too (FERROEHR__AUTHZ__ASSURANCE__MINIMUM, FERROEHR__AUTHZ__PROFESSIONAL__ACTING_FOR_CLAIM), except the levels map.

[authz.rbac]

Note

The management surface is not configured under [authz.rbac]. [management.endpoints] owns it, one level per endpoint, with no global default beside it: an endpoint you do not name is off and is not mounted. Only the admin_only level consults authz.rbac.admin_role.

[authz.abac]

The patient scope. With ABAC on and patient_claim set, a token without that claim is refused with 403. With the claim:

• A route that names an EHR (/ehr/{ehr_id}/…) answers 403 unless the claim names that EHR’s subject. An EHR that records no subject is in no patient’s scope and is refused.
• Without patient_namespace, an id held under two or more namespaces names none of those subjects, so it reaches none of their EHRs. Set patient_namespace to read the claim in one namespace.
• An AQL query returns rows only from EHRs the claim names, whatever the query selects, including SELECT e/ehr_id/value FROM EHR e.

After a query runs, the decision point receives the subject ids of every EHR the query touched as its patient attribute, so a patient-keyed policy applies to queries whether or not patient_claim is set. Without patient_namespace, a query that touches a subject id held under several namespaces is refused with 403, because a bare id would name both subjects.

• [authz.abac.cedar]: policy_dir (path, required when engine = "cedar" and ABAC is on) and reload_secs (int, unset, an optional hot-reload interval).
• [authz.abac.remote]: server (secret URL, required when engine = "remote", and it must end with /, because the policy name is appended; credentials embedded in it are redacted from every rendering), connect_timeout_ms (int, 2000), request_timeout_ms (int, 5000).
• [authz.abac.remote.oauth]: calls the policy service as an OAuth 2.0 client instead of with the Basic credentials of server: FerroEHR obtains a token with the client credentials grant and sends it as a bearer token. The keys are those of [audit.fhir_feed.oauth] (token_endpoint or issuer, client_id, client_secret or client_secret_file, client_auth, scope, audience, resource). When the policy service refuses the token with 401, a new one is fetched and the decision asked once more; a token that cannot be obtained, or that is refused again, is no decision, and the request fails closed with a 500, never a 403. server with credentials beside this table stops startup.
• [authz.abac.policy.<kind>]: one entry per resource kind, with kind ∈ ehr, ehr_status, composition, contribution, query, directory. Keys: name (string, the policy to evaluate) and parameters (list of enum{organization,patient,template}). A key that is not one of those six kinds is a boot error, and template is rejected on ehr and ehr_status, since neither carries a template.

Warning

With engine = "remote", every resource kind the enforcement point consults needs a policy entry: ehr, ehr_status, composition, contribution, query, plus directory when check_directory = true. A missing one is a boot error. At runtime a kind with no policy can only be denied, since there is no policy to ask and permitting would be a silent hole, so the misconfiguration is caught at startup rather than turning into blanket 403s on live traffic. The Cedar engine reads its policies from authz.abac.cedar.policy_dir and needs no entries here.

[admin]

Physical deletion is irreversible, so the group stays off by default. With it on, /admin also joins the group list the OPTIONS System-Options manifest advertises, so the manifest never names a group that answers 404.

[smart]

The SMART on openEHR scopes are enforced on every route whenever [auth] is on: each operation carries a requirement from FerroEHR’s route table, which the ferrohealth-smart crate checks against the token’s scopes or a Basic user’s scopes. There is no switch for it. This section decides what beside the route table: whether the discovery document is served, and whose grants count for what. See SMART App Launch.

[smart.episode]: enabled (bool, false) advertises episode context and accepts the launch/episode scope and episodeId claim, but applies no episode-scoped filtering.

No smart URL may carry a user name or password, whether or not the document is served: the discovery document publishes these URLs to every application, so a credential in one would be served to anyone who asks.

[management]

The ops-introspection surface: build info, Prometheus, metric views, the effective configuration, runtime log control, and the on-demand profiler. Off by default, and every endpoint off individually.

The health probes are not configured here: /health, /health/liveness and /health/readiness are always served on the main API port without authentication, whatever this section says (see Operations → Health probes).

[management]
enabled = false
base_path = "/management"

[management.endpoints]
info = "off"
metrics = "off"
prometheus = "off"
env = "off"
loggers = "off"
flamegraph = "off"

[management.profiling]
max_seconds = 30
max_frequency = 999

[management.endpoints]: info, metrics, prometheus, env, loggers and flamegraph, each enum{off,admin_only,private,public}, default off.

env renders the effective configuration and flamegraph starts a profiler on request, so the boot log prints exactly which endpoints a configuration turned on and at which level.

[management.profiling]: limits for the on-demand CPU flamegraph behind endpoints.flamegraph (see Operations → Profiling):

Warning

probes_enabled and endpoints.health do not exist. Configuration is strict, so a file or environment variable still setting either one fails at boot with an unknown-key error: delete the key; the probes are always on.

[licence]

The commercial licence token. Every build embeds the licensor’s non-commercial grant; a deployment that holds a commercial licence installs its token here. The server behaves identically under either grant and reports the one in force on GET /management/status (licence.use, licence.licensee, licence.not_after, licence.configured_token). See Licensing & legal.

[signing]

VERSION signing. On by default in digest mode, with read-time verification of the server’s own signatures strict by default.

retired_key_paths exists because a stored VERSION.signature records no key identifier and is an immutable committed fact that cannot be re-issued; keeping the retired public key is the only way history stays verifiable across a rotation, and a public key can verify but never sign. Its environment form takes a comma-separated list (FERROEHR__SIGNING__RETIRED_KEY_PATHS=/keys/a.pub.asc,/keys/b.pub.asc).

Note

verify_on_read resolves to strict when signing is enabled. On every read the server recomputes the signature of a version it signed and, on a mismatch, returns a 500 integrity fault rather than silently serving a provably corrupt record. Set it explicitly to warn (log + meter version_signature_invalid_total, still serve) or off (never check). Client-supplied signatures (an author’s own, or one carried by an imported version) are always stored verbatim and never re-verified, whatever this setting says, because the author may have signed a different agreed serialization.

Warning

pgp mode fails closed at boot if the key is missing or unusable: the server will not start. Verify the key and passphrase before switching modes.

Choose an Ed25519 signing key

The server refuses an ECDSA (P-256, P-384, P-521, secp256k1, Brainpool) or Ed448 signing key at boot: their private-key arithmetic runs on crypto-bigint 0.5, whose constant-time operations can branch on secret values (RUSTSEC-2026-0387), with no fixed release on that line. The boot error names the advisory. Move to Ed25519 with the steps below; the old certificate goes in retired_key_paths, where it keeps verifying, because verification uses only public values.

An RSA signing key is accepted, but it makes every commit perform an RSA private-key operation, the operation the Marvin timing sidechannel concerns (RUSTSEC-2023-0071 / CVE-2023-49092), for which the underlying rsa crate has no fixed release. An Ed25519 key keeps that code off the signing path entirely.

The server does not refuse an RSA key: a repository whose history is already RSA-signed needs that key to keep verifying, and signatures are immutable committed facts that cannot be re-issued. Instead it logs a warning at boot naming the advisory. To clear it:

1. Generate a new signing key: gpg --quick-generate-key "…" ed25519 sign, and export the armored secret key to key_path.
2. Export the public half of the old certificate and add it to retired_key_paths, so versions signed with it still verify (see above).
3. Restart. New versions are signed with Ed25519; old ones keep verifying.

Rotation inside one certificate is cheaper still: add an Ed25519 signing subkey to the existing certificate, export the armored secret key again to key_path, and restart. The server signs with the newest subkey whose binding grants signing and that is neither expired nor revoked (the rule), with no retired_key_paths entry needed: the certificate keeps the previous subkey, so past signatures continue to verify. For a refused ECDSA or Ed448 key this works when the new Ed25519 subkey is the one chosen. The steps above, with a new certificate, work in every case.

On Kubernetes, digest mode needs nothing; it is the default. pgp mode needs the key as a file and its passphrase as a secret, both of which the chart mounts for you:

# values.yaml
config:
  signing:
    enabled: true
    mode: pgp
    key_path: /etc/ferroehr/signing-key.asc
  files:
    signing-key.asc: |
      -----BEGIN PGP PRIVATE KEY BLOCK-----
      …
secrets:
  signingKeyPassphrase: "…"

Every config.files key becomes /etc/ferroehr/<key>, mounted read-only from a chart Secret at mode 0440, and that volume holds the private key, so it is never world-readable inside the container. To go back to digest, set mode: digest and drop the key material; versions already signed keep their signatures and still verify.

Integrations

Query execution, the patient identity source and the three optional integrations: [query], [ehr_index], [events], [terminology], [multimedia]. Everything except the query knobs and the identity source is off by default, and a disabled integration contacts nothing and mounts no routes. Precedence, the environment-name grammar, and file discovery are on the Configuration reference index.

• [query]
• [ehr_index]
• [events]
• [terminology]
  • Authenticating to the terminology server
  • Archetype value-set bindings at commit
• [multimedia]

[query]

AQL execution bounds.

[query]
plan_cache_capacity = 256
timeout_ms = 30000
max_result_rows = 10000

timeout_ms is on by default, and deliberately tighter than db.statement_timeout_ms. Keeping this budget the tighter of the two means an overrun surfaces as the engine’s own typed 408 rather than a driver error. When it fires, or when the client disconnects or the HTTP request timeout gives up first, the query’s connection is closed instead of returned to the pool, so its pool slot frees at once and PostgreSQL stops the statement within about a second (the session’s client_connection_check_interval).

max_result_rows is the largest page one execution serves. Without it, SELECT c FROM COMPOSITION c with no fetch generates SQL with no LIMIT and materialises every matching row: one request, unbounded allocation. A query that nothing else bounds takes the ceiling as its page. An explicit AQL LIMIT or a fetch parameter is honoured as written up to the ceiling; a page larger than the ceiling is refused with 400 naming the ceiling. The refusal is deliberate: a silently shortened page would let a client that pages with its own fetch as the stride skip the rows between the shortened page and its next offset, and the RESULT_SET has nothing to say so. ITS-REST leaves both the fetch default and its maximum to the implementation. A bulk consumer pages with offset and a fetch at or below the ceiling, or the operator raises the ceiling deliberately.

[ehr_index]

Who holds the cross-reference from a patient to the EHR that records them (the EHR index). No openEHR specification governs where it lives: the RM’s PARTY_SELF section says the association “may be done elsewhere for security reasons”, and FerroEHR’s design puts it in an IHE patient identity source.

The IHE patient identity source is the default. FerroEHR asks a PIX Manager, as a PIXm Patient Identifier Cross-reference Consumer (IHE ITI TF-2 §3.83, $ihe-pix), which identifier the clinical domain holds for a patient that another domain’s identifier names. That identifier is the pseudonym EHR_STATUS.subject carries. FerroEHR sends only what ITI-83 specifies, so any conformant PIX Manager serves; FerroPIX is the one the compose quickstart and the Helm chart run.

The in-instance map, provider = "local", is the other supported posture, for production as much as for development, tests and the conformance suites: the subject-id and subject-namespace model. A client names a patient in EHR_STATUS.subject.external_ref by any namespace and identifier (benchmark / 1793131381), as OBJECT_REF.namespace and a local GENERIC_ID.scheme allow, and the map answers the openEHR REST API’s GET /ehr?subject_id=…&subject_namespace=… lookup over EHR_STATUS content, matching both exactly. An EHR created for a subject through the service’s create_ehr_for_subject enters the map with its creation, as a Primary association with no location, so a subject cohort finds it; an EHR created over POST /ehr or PUT /ehr/{ehr_id} enters the map only through add_ehr_subject. With neither provider configured, the server refuses to start and the refusal names both choices.

What the local posture means for pseudonymisation: with no [privacy] subject_namespaces declared, the server stores a subject reference as the client sends it, a real identifier included. Pseudonymisation then belongs to the client, or to the demographic service the reference points at (GDPR Art. 4(5)). There is no mixed mode: a deployment that declares pseudonym namespaces keeps the rule in force under local too, so every subject reference must name a declared namespace and carry a UUID, and no other namespace is accepted beside them. A production deployment runs the subject-id posture by accepting the open_subject_namespace gap by name (deployment_profile), which boot and GET /management/status then state.

provider alone switches the index, in the file or as FERROEHR__EHR_INDEX__PROVIDER. The keys of the provider not chosen are kept and not used, and boot names them, so a configuration written for the IHE source runs on the local map with FERROEHR__EHR_INDEX__PROVIDER=local and no other change; its declared pseudonym namespaces stay in force. The compose quickstart selects its local configuration with one variable (The local EHR index).

[privacy]
subject_namespaces = ["urn:oid:2.999.7.1"]

[ehr_index]
provider = "ihe"
pixm_url = "https://pix.example.org/fhir"
assigning_authority = "urn:oid:2.999.7.1"

[ehr_index.oauth]
token_endpoint = "https://idp.example.org/realms/pix/protocol/openid-connect/token"
client_id = "ferroehr"
client_secret_file = "/run/secrets/pix-client"
scope = "system/Patient.rs"

[ehr_index.oauth] is the client credentials grant every ITI-83 request carries, with the keys of the terminology server’s grant; a 401 is answered once more under a new token. The client id is the one the identity source mints pseudonyms in the clinical domain for, and the scope is the one its PIX Manager takes from a Consumer. [ehr_index.tls] holds the client’s TLS material: trust_roots_file, a PEM bundle trusted beside the platform roots for a PIX Manager behind a private PKI, client_identity_file (or client_identity inline), the chain and key presented for mutual TLS, and min_version ("1.3", or "1.2").

What each operation asks:

No fallback. At start the server reads the source’s CapabilityStatement and refuses to start when it does not answer. A request the source then fails, refuses or times out on answers with an error that carries what the source said, and nothing is answered from the in-instance map. An answered pseudonym the [privacy] subject rule refuses, or more than one identifier in the clinical domain, is refused too.

FerroEHR does no identity work in either mode. It matches no patients, merges none and links no identifier domains; the identity source does. It asks only for the cross-reference ITI-83 answers. PDQm (ITI-78) is not used: its search answers with the demographic Patient, which FerroEHR would only discard, so asking for it breaks data minimisation (GDPR Art. 5(1)(c)), and its $match is patient matching.

[events]

Contribution-outbox eventing to an AMQP broker, plus its admin API. Off by default; the envelopes are PHI-free by design.

Note

Eventing needs the events build feature, which is on in the published binary and container images. A binary built with --no-default-features refuses at startup if events.enabled is set, rather than running with the publisher silently absent.

[terminology]

The terminology extension API and the one FHIR terminology server the CDR asks.

[terminology]: api_enabled (bool, false) mounts the terminology extension API.

[terminology.server]:

Every key is a scalar, so the whole section is reachable through the environment: FERROEHR__TERMINOLOGY__SERVER__ENABLED, FERROEHR__TERMINOLOGY__SERVER__URL, FERROEHR__TERMINOLOGY__SERVER__FAIL_ON_ERROR, FERROEHR__TERMINOLOGY__SERVER__REQUEST_TIMEOUT_MS.

[terminology.server]
enabled = true
url = "http://ferroterm:8080/r4b"
fail_on_error = false

The server answers every terminology the bundled openEHR terminology does not carry: the /terminology/* extension API, AQL TERMINOLOGY(…) resolution, the ADL 2 binding check and the composition-commit binding checks below. A response that is not a valid FHIR R4B Parameters or ValueSet resource (for example an $expand result missing the required ValueSet.status or expansion.timestamp) is treated as an upstream fault rather than partially read, so it takes the same path as an unreachable server and fail_on_error decides what the commit does.

Note

The terminology server needs the fhir build feature, which is on in the published binary and container images. A binary built with --no-default-features refuses at startup if terminology.server is enabled; the in-process openEHR terminology bundle remains available.

Authenticating to the terminology server

A server behind an authorization server takes the OAuth2 client-credentials grant under [terminology.server.oauth], with the same keys as [audit.fhir_feed.oauth] and [authz.abac.remote.oauth]. The access token is cached and re-requested shortly before it expires, and a request the server answers with 401 is sent once more under a new token.