Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Server, database & telemetry

The listener ([server]), the REST surface ([api]), the PostgreSQL connection ([database]), the observability section ([telemetry]), and the usage report ([usage_report]). Precedence, the environment-name grammar, and file discovery are on the Configuration reference index.

[server]

The HTTP listener: the section every FerroHEALTH product shares (ferrohealth-server), so the keys and their behaviour are the same in each. It carries the stack every request passes through: the server’s request id, the request timeout, the body limit, the concurrency bound, compression, the request log, the health routes and the bounded drain on SIGTERM.

[server]
listen = "0.0.0.0:8080"
base_path = ""
request_timeout = "30s"
header_read_timeout = "30s"
idle_timeout = "2min"
body_limit = "64MiB"
drain_delay = "0s"
shutdown_timeout = "10s"
max_concurrent_requests = 256
max_waiting_requests = 256
retry_after = "1s"
trusted_proxies = []
forwarded_header = "forwarded"
compression = true
compression_min_size = "1KiB"
# public_base_url = "https://cdr.example.com"
KeyTypeDefaultDescription
listensocket address0.0.0.0:8080The IP address and port the listener binds: every interface at the port the container image exposes. A host name is refused at startup. PORT alone gives 0.0.0.0:<PORT>, and FERROEHR__SERVER__LISTEN wins over both. Set 127.0.0.1:8080 to keep a bare binary on the loopback.
base_pathstringemptyThe path every route is served under, the health routes included. Empty serves them at the process root. The openEHR API path is api.base_path.
request_timeoutduration"30s"A request still running after this long is answered 408 with the openEHR error body.
header_read_timeoutduration"30s"How long a client may take to send its request head, counted from accept for a connection’s first request and from the first byte of each later request on a kept-alive connection; a client that has not sent it by then, one that sent nothing after connecting included, is disconnected. Bounds the slow-header and the silent connection, which no other limit reaches. "0s" is refused.
idle_timeoutduration"2min"How long a kept-alive HTTP/1.1 connection may wait for its next request; past it the server closes the connection with a clean end of stream. Keep it longer than the idle timeout of your clients (Go’s http.DefaultTransport keeps an idle connection 90 s) and of any load balancer, ingress controller or proxy in front, so they close an idle connection first and never send a request into one the server is closing. "0s" is refused.
body_limitsize"64MiB"The largest request body any route reads, refused 413 above it. The bulk routes (operational-template upload, /message/import) take up to this; api.body_limit narrows the rest of the clinical surface below it. The largest operational template in the vendored CKM corpus is several megabytes, well below this ceiling.
drain_delayduration"0s"On SIGTERM or SIGINT, readiness answers 503 this long while the listener still accepts, so an orchestrator stops routing first.
shutdown_timeoutduration"10s"Then the listener closes and the requests in flight get this long to finish; a connection still open after it is dropped.
max_concurrent_requestsint256The requests served at once. Past it a request waits for a slot (max_waiting_requests, admission_wait) and is answered 503 with Retry-After and the openEHR error body when the queue is full or the wait expires, so offered load beyond capacity cannot exhaust memory. The health routes are never refused or queued, so readiness holds while the server sheds. The bound is always on; 0 is refused. See sizing the bound.
max_waiting_requestsint256The requests past max_concurrent_requests that may wait for a slot at once, first come first served. A request that finds the queue full is answered 503 at once. The requests in flight and waiting never exceed the two bounds together. 0 keeps no queue and refuses every request past the bound at once.
admission_waitduration100 millisecondsHow long a request past max_concurrent_requests waits for a slot before it is answered 503, written with the ms or s unit. It rides out a short stall, such as a database WAL flush, that would otherwise refuse requests the server answers a moment later. The wait counts in the request’s latency. "0s" keeps no queue.
retry_afterduration"1s"The Retry-After of a 503 or 429 the listener writes, in whole seconds (RFC 9110 §10.2.3).
trusted_proxieslist[]The reverse proxies, as addresses or CIDR blocks, whose forwarded client address is believed. Empty, every request is named by its peer address.
forwarded_headerenum{forwarded,x-forwarded-for}forwardedThe header a trusted proxy names the client in (Forwarded is RFC 7239).
compressionbooltrueCompress a response with gzip or Brotli when the client accepts one.
compression_min_sizesize"1KiB"The smallest response body that is compressed.
public_base_urlstringunsetThe URL clients reach the server at, through any proxy in front of it: https://cdr.example.com, or with the path a proxy serves it under (https://gw.example.com/cdr). FerroEHR’s own key beside the shared section, and the one place a deployment names its public URL; see below. An absolute http(s) URL with no userinfo, query or fragment. Required with auth.smart.discovery, [auth.scim] or [auth.dpop] under auth.enabled.

Sizing max_concurrent_requests

The bound counts requests in flight, not requests per second. The memory a server holds grows with what each request in flight keeps (its body, its parsed query, its wait for a database connection), so the bound is what keeps a replica inside its memory when the offered load passes its capacity. Unbounded, one instance under the capacity test grew to 9.9 GB resident at about 4096 requests in flight and was killed by the kernel; with 256 it refused the excess 503 and kept serving.

To size it for your deployment:

  • In flight at your target rate. By Little’s law, requests in flight are the rate times the latency. The capacity test’s mix at 2000 requests per second held about 24 in flight on average and under 80 at the p99, so 256 refuses nothing there.
  • A multiple of the pool. Past a few times database.max_connections a request only waits for a connection. 256 is about 9 times the largest default pool of 30; raise the bound with the pool, not past it by much.
  • Memory per request. Allow a few megabytes per request in flight for clinical commits and AQL with large results, and keep the bound times that inside the replica’s memory limit.

A 503 with Retry-After under the bound tells a client to back off; add replicas when it shows at your normal load. The /metrics series ferrohealth_server_admission_waits_total, ferrohealth_server_admission_wait_duration_seconds (by outcome, admitted or expired) and ferrohealth_server_admission_refusals_total (by reason, queue_full or wait_expired) show how often requests wait and why they are refused. Many wait_expired refusals mean stalls longer than admission_wait; many queue_full refusals mean sustained load past capacity.

A refusal the listener writes itself (408, 413, 429, 503) and a handler that panics (500) answer the standard openEHR { error, message } body. 413 and 429 are not in the ITS-REST status table; ITS-REST admits them as additional, non-conflicting codes, and they are what RFC 9110 §15.5.14 and RFC 6585 §4 define for these refusals. A panic’s message reaches neither the body nor the log.

public_base_url: the one public URL

A server behind a proxy cannot read from a request the URL its client addressed, so three things that publish or check an absolute URL take it from public_base_url. Each derives its URL from it and server.base_path, the path the server mounts every route under:

ConsumerURL
The SMART discovery document’s services.org.openehr.rest.baseUrlpublic_base_url + server.base_path + api.base_path
Every SCIM meta.location and the SCIM discovery documentspublic_base_url + server.base_path + /scim/v2
The URL a DPoP proof’s htu is compared with (RFC 9449 §4.3, check 9)public_base_url + server.base_path + the request path

With public_base_url = "https://gw.example.com/cdr" and base_path = "/ehr", the discovery document names https://gw.example.com/cdr/ehr/ferroehr/rest/openehr/v1, a SCIM User lives at https://gw.example.com/cdr/ehr/scim/v2/Users/<id>, and a DPoP proof for GET /ehr/ferroehr/rest/openehr/v1/ehr names https://gw.example.com/cdr/ehr/ferroehr/rest/openehr/v1/ehr. A trailing / on either is ignored.

Every request gets a request id the server mints, which the request log and the access record carry and the response returns in X-Request-Id. A client’s own X-Request-Id is returned to it unchanged and never recorded.

[server.caller_rate]: per-caller request rates

[server.caller_rate]
requests_per_second = 1024
burst = 2048
KeyTypeDefaultDescription
requests_per_secondint(required in the table)Sustained requests per second one authenticated subject may send on the clinical API.
burstint(required in the table)How many requests one subject may send at once before the rate applies.

Unset, no caller is limited. The chart and the Compose quickstart set the values above.

This is not max_concurrent_requests, and you can tell them apart from the status alone. max_concurrent_requests protects capacity: too many requests in flight at once, from anyone, and the excess is refused 503 + Retry-After. The caller rate protects fairness: one caller asking too often over time, refused 429 + Retry-After. A 503 means the server is full; a 429 means you are asking too fast.

The limit is keyed on the authenticated subject, the only fair key for a clinical API: a hospital behind one NAT is a single address. It sits inside authentication, so the health routes, the status document and the discovery document are never refused by it.

Tip

Benchmarking this server? Lift the caller rate first, or you will measure it instead of the server. The project’s own measurement lanes compose an overlay that does exactly that, and both instruments refuse to write a record if the server answered any 429.

[server.tls]: native TLS and mutual-TLS client authentication

Native TLS termination on the main listener, unset by default; deployments commonly terminate TLS at an ingress. client_auth = "required" is the IHE ATNA ITI-19 mutually-authenticated-node posture (see the Audit trail chapter). The separate-port management listener always stays plain HTTP, so under deployment.profile = "production" setting management.listen opens the plaintext_listener gap.

[server.tls]
certificate_file = "/etc/ferroehr/server.pem"
key_file = "/etc/ferroehr/server.key"
client_auth = "none"
min_version = "1.3"
# reload_interval = "1min"
KeyTypeDefaultDescription
certificate_filepath(required in the table)Server certificate chain (PEM), the leaf first. A missing, unreadable or certificate-less file stops startup.
key_filepath(required in the table)Server private key (PEM), read as a secret file. A key that does not belong to the leaf stops startup.
client_authenum{none,optional,required}noneClient-certificate policy. optional verifies a certificate when one is presented and still accepts connections without; required rejects any client without a verified certificate at the handshake.
client_ca_filepathunsetThe explicit CA bundle client certificates must chain to, never the web PKI. Required when client_auth is optional or required, and refused at load when it is none, since a listener under none asks no client certificate.
min_versionenum{“1.3”,“1.2”}"1.3"The lowest TLS version this listener negotiates. "1.0" and "1.1" are refused when the configuration is read (RFC 8996).
reload_intervaldurationunsetHow often the certificate, key and client CA are read again while the server runs. The files are always read again on SIGHUP. A changed pair is served, and a changed client CA verifies, from the next handshake on without a restart; a read that fails is logged and the running configuration stays.
healthcheck_identity_filepathunsetThe client certificate chain and key, in one PEM, that ferroehr healthcheck presents to a listener requiring one.

Before 5.5.0, client_ca_file alone meant that every client presents a certificate. From 5.5.0, client_auth decides it and defaults to none, so a configuration carried over from an earlier release set client_ca_file and no client_auth, and 5.5.0 served it without asking a client certificate. Such a configuration is now refused at load. Set client_auth = "required" beside client_ca_file to keep mutual TLS (GHSA-52mf-qg39-4q4j).

ferroehr healthcheck speaks TLS to a TLS listener over the loopback and accepts exactly the leaf certificate in certificate_file.

Note

min_version defaults to 1.3 only, as RFC 9325 §3.1.1 has it supported. Setting "1.2" enables 1.2 alongside 1.3, with the four forward-secret suites RFC 9325 §4.2 recommends; pick it only for a client that cannot do 1.3, such as an older integration engine or a pinned Java runtime.

Note

An EC certificate must name its curve. RFC 5480 §2.1.1 forbids explicit curve parameters, TLS clients refuse such a certificate, and the server refuses the pair at startup. LibreSSL (the openssl that ships with macOS) writes explicit parameters from genpkey and req -newkey ec unless you add -pkeyopt ec_param_enc:named_curve; openssl ecparam -name prime256v1 -genkey writes a named curve.

[api]

The openEHR REST surface the listener serves.

[api]
base_path = "/ferroehr/rest/openehr/v1"
openapi_ui = "private"
system_id = "ferroehr.local"
# body_limit = "16MiB"
KeyTypeDefaultDescription
base_pathstring/ferroehr/rest/openehr/v1ITS-REST base path all API routes hang off. Shortening it is supported; see below. The served OpenAPI document describes whatever paths this setting produces, never the defaults.
openapi_uienum{off,admin_only,private,public}privateWho may read the Swagger UI and the OpenAPI documents under the REST root: off does not mount them, admin_only needs the admin role, private needs any authenticated principal (unauthenticated is 401 with the server’s WWW-Authenticate challenge, so a browser prompts for the Basic credential), public needs nothing. The documents list the whole enabled operation surface, admin and message groups included, so public discloses it to anyone who can reach the port; the hosted sandbox sets it deliberately. With authentication off the guard admits everyone, as the management endpoints do.
corstableabsent[api.cors], the CORS protocol answers (Fetch Standard §3.2.3): allow_origins, allow_methods, allow_headers, expose_headers, allow_credentials and max_age (a duration), each named for the header it sets. Absent, no CORS header is sent. allow_origins = ["*"] admits any origin, which the server warns about at boot and which refuses allow_credentials. Production lists its origins.
body_limitsizeunsetThe largest body the ordinary clinical surface accepts, below server.body_limit, which the bulk routes keep. Unset, every route takes the listener’s limit. A value at or above the listener’s is refused at startup. The largest example composition in the vendored corpus is 526 KB; raise it if your compositions embed large DV_MULTIMEDIA data.
adminboolfalseServe the ADMIN API group (physical delete); off, every admin route answers 405.
terminologyboolfalseMount the terminology extension API; the server it asks is [terminology].
system_idstringferroehr.localThis deployment’s own openEHR system identifier; see below. Set a stable, deployment-unique name in production (FERROEHR__API__SYSTEM_ID).

base_path: shortening the REST base path

ITS-REST leaves the API base to the deployment and fixes only the version segment at its end, so you may shorten base_path. The shape is checked at boot, and a value that breaks a rule stops the server with an error naming the key and every rule it broke:

  • The first segment is ferroehr. /ferroehrx/v1 and /x/ferroehr/v1 are refused. This segment is not configurable away.
  • The last segment is v1, the openEHR API version this server implements. The shortest accepted value is therefore /ferroehr/v1.
  • No trailing slash, no empty segment (//), and every segment uses only the unreserved URL characters A-Z a-z 0-9 - . _ ~.

The status and documentation routes hang off the REST root, which the server derives from base_path by dropping the segments that name the openEHR API: the trailing v1, plus an openehr segment directly before it when you spell one.

base_pathREST rootStatus document
/ferroehr/rest/openehr/v1 (default)/ferroehr/rest/ferroehr/rest/status
/ferroehr/openehr/v1/ferroehr/ferroehr/status
/ferroehr/v1/ferroehr/ferroehr/status
/ferroehr/cdr/v1/ferroehr/cdr/ferroehr/cdr/status

The Swagger UI ({rest root}/swagger-ui), the OpenAPI documents ({rest root}/api-docs/…) and the SMART discovery document ({rest root}/.well-known/smart-configuration) move with it. The health family is served under server.base_path, which is empty by default, so it stays at the process root: /health, /health/liveness, /health/readiness.

Set it from the environment with:

FERROEHR__API__BASE_PATH=/ferroehr/v1

Note

The ferroehr healthcheck subcommand, which the published container images run as their Docker health check, asks GET /health/readiness on the server.listen port over the loopback, so api.base_path does not move it.

Behind a path-prefixed reverse proxy

A proxy that mounts the CDR under a prefix stacks that prefix on top of base_path. A proxy serving the server at /cdr with the default base path gives clients URLs like:

https://cdr.example.org/cdr/ferroehr/rest/openehr/v1/definition/template/adl1.4

Two ways to shorten that. Strip the prefix at the proxy, so the server sees the path it serves (nginx proxy_pass http://cdr:8080/; with the trailing slash, Caddy handle_path, Traefik StripPrefix). Or shorten the base path:

FERROEHR__API__BASE_PATH=/ferroehr/v1

which gives https://cdr.example.org/cdr/ferroehr/v1/definition/template/adl1.4.

Whichever you choose, tell every client. The server builds its Location headers and its served OpenAPI paths from its own api.base_path, and it cannot see a prefix the proxy adds. Name the prefix in server.public_base_url, which the SMART discovery document, the SCIM locations and the DPoP check read. The FerroEHR Viewer has its own mirror key, cdr.base_path, which must match.

system_id: the data-authoring identity

system_id is the identifier this CDR stamps into the data it authors. It appears on the wire in three places:

  • EHR.system_id, recorded when an EHR is created. The openEHR RM (EHR Information Model, EHR Identifier Allocation) says the EHR.system_id “should be set to the value that would normally be used for locally created EHRs”: a value the deployment chooses, not a product constant.
  • AUDIT_DETAILS.system_id on every commit for which the client did not supply one through the openehr-audit-details header. The openEHR REST API requires that “when system_id is not provided by the client, the server MUST set it to its own configured system identifier”.
  • OBJECT_VERSION_ID.creating_system_id: the middle segment of every version identifier the server mints (<object_id>::<creating_system_id>::<version>).

Practical notes:

  • The value must be a legal openEHR UID: a UUID, an ISO OID, or an internet id (a reverse-domain / DNS-style name) per the openEHR BASE identification grammar. The server judges it with the same validating constructor its reader uses and refuses to boot on an illegal value, because it becomes the creating_system_id segment of every version identifier the server mints and an illegal value would produce ids the server’s own reader rejects. A DNS-style name like cdr.hospital.example is valid; an empty value, or one containing the :: field separator, is not.
  • Choose it before going live and keep it stable. The value is stored with each EHR and each version; changing it later affects only newly authored data: existing EHR ids, audit rows and version identifiers are never rewritten, and previously issued OBJECT_VERSION_IDs stay valid.
  • Make it unique per system, so data exchanged between openEHR systems keeps unambiguous provenance. A node of a federation must hold a system_id no other member uses (Federation Tier N42a); the default ferroehr.local is the same on every node that leaves it unset. See FerroEHR as a federation node.
  • system_id is not [api.options]. system_id says which system authored the data; [api.options] is the display identity of the OPTIONS System-Options manifest. Rebranding changes the manifest and nothing in stored data; changing system_id changes what new data says about its origin and leaves the manifest alone.

[api.options]

The System-Options manifest identity (OPTIONS on the API base path, e.g. OPTIONS /ferroehr/rest/openehr/v1, the System API’s one location). This is the deployment’s display identity only; the identifier stamped into authored data is system_id.

KeyTypeDefaultDescription
solutionstringFerroEHRProduct name.
solution_versionstringthe build’s versionProduct version.
vendorstringCadasto B.V.Providing organisation: by default the manufacturer of the release, the name ferroehr --version and GET /management/info give.
restapi_specs_versionstringthe ITS-REST release this build implements (1.1.0)The openEHR REST API edition advertised.

The defaults are derived from the build rather than typed into the handler. Override them only to rebrand. The manifest’s conformance_profile is no identity field and has no key here: the build derives it from the committed conformance record (see Using the API).

[database]

PostgreSQL connection.

[database]
url = "postgres://ferroehr:ferroehr@localhost:5432/ferroehr"
migrations = "apply"
connection_budget = 40
# max_connections and min_connections: unset, sized from the instance
acquire_timeout = "30s"
statement_timeout = "1min"
KeyTypeDefaultDescription
urlsecret URLpostgres://ferroehr:ferroehr@localhost:5432/ferroehrConnection DSN. The default suits a local from-source run against a localhost PostgreSQL, and the server logs a prominent warning at boot while it is in use; production MUST set it. Credentials are redacted from every rendering. DATABASE_URL is a recognized lower-priority alias.
url_filepathunsetRead the DSN from a file instead of the key above, for a mounted secret. Preferred over the environment form in Kubernetes: an environment value is readable through /proc/<pid>/environ and inherited by every child process. At most one of the pair, where the built-in development default does not count as “set”.
migrate_urlsecret URLunsetDSN that PREPARES the schema, used for that one boot step and then closed. Preparation spans every schema of a database at once, which no domain-scoped runtime credential can do, so a deployment separating the runtime roles names the credential that can here. It prepares every domain whose DSN reaches the SAME DATABASE — which is the whole of the separated-credential posture, where each domain has a login role of its own on one database; a domain whose DSN reaches a different database is prepared on that database’s database.<domain>.migrate_url (on the domain’s own DSN when that is unset). Which it is, is read from the server (pg_control_system() plus current_database()), never from the DSN text. Unset, it falls back to url. See Operations → Which credential prepares the schema.
migrate_url_filepathunsetRead migrate_url from a file instead, for a mounted secret. At most one of the pair.
migrateenum{apply,verify}applyWhether the server applies its embedded migrations at boot. apply is what makes an empty configuration boot against an empty database. verify issues no DDL at all: it checks that the database already carries exactly this build’s migrations and refuses to start otherwise, so the serving DSN can authenticate as a role with no DDL rights. That check still READS every _sqlx_migrations table of each database, which no least-privilege role can do, so pair it with migrate_url above and with ferroehr db migrate run out of band; see Operations.
connection_budgetint40The most connections the instance opens to PostgreSQL while it serves: its three pools, the audit runtime’s four and its listener connections together. The pools are sized inside it, and a max_connections whose pools and listeners exceed it is refused at startup. The default lets two instances share PostgreSQL’s default max_connections of 100. See Sizing the pool.
max_connectionsintunsetThe size of each domain’s pool (clinical, party and audit each have one). Unset, it is four per core the server may run on, at least 10, within the pool’s share of connection_budget, and never more than server.max_concurrent_requests. See Sizing the pool.
min_connectionsintunsetConnections each pool keeps open while idle, which the server opens before it answers ready. Unset, the server keeps one per core it may run on, at least 2 and at most the pool size. A one-shot command (ferroehr db migrate, party-binding refill) opens only what it uses.
acquire_timeoutduration"30s"How long to wait for a free connection before failing.
statement_timeoutduration"1min"statement_timeout applied to every pooled connection; "0s" leaves the server default.

Sizing the pool

A request checks a connection out of a pool at its first statement and gives it back when its handler has answered, so the audit record and the response’s way out hold none. How long that is shows in db_pool_hold_duration_seconds, and how long requests wait for a connection in db_pool_acquire_duration_seconds. A pool needs about the arrival rate times the hold time, and room for a burst above that: every request that arrives during a pause of the server or the database wants a connection when the pause ends. A request holds a connection about as long as it runs on a core, so four connections per core is about what the server keeps in use with every core busy, and the burst above it waits in the pool rather than failing.

A write takes a connection of the audit pool first, for the records it writes before it commits, and then a connection of its own domain’s pool, clinical or party. Its transaction therefore never waits for the audit pool, and no two requests wait for each other across the pools. The audit pool has the same size as each domain pool (max_connections, or the size derived from the cores), so it is never the smaller one. It also takes the access records of reads and the party domain’s writes, so a write that finds it busy waits for it before it takes its other connection, holding none. Watch db_pool_acquire_duration_seconds{pool="audit"}: a rising wait there means the pools need to grow together.

Every pool metric carries a pool label naming the pool it measures: clinical, party and audit for the three domain pools, and audit_runtime for the audit runtime’s own pool. db_pool_connections has a series per pool and state (idle, in_use), and db_pool_acquire_duration_seconds, db_pool_hold_duration_seconds and db_pool_nested_acquires_total record each wait, hold and nested acquire under the pool it was taken from. Read each pool by itself: a wait on the audit pool and one on the clinical pool have different causes. db_pool_nested_acquires_total also counts, with an order label, a request that took its audit connection after a domain one (audit_after_domain) and a request that kept a connection idle while it waited for a pool or for another request’s work (kept_while_waiting). Each is 0 on a server that works as designed; a count above 0 is a defect to report.

Each instance opens at most connection_budget connections: one listener connection per database its listeners reach (one when every domain shares a DSN), four for the audit runtime (the access-log chainer, the forwarders and the chain verification), and three pools that share the rest evenly. At the default of 40 with one listener connection, each pool holds at most 11, so an instance opens at most 3 × 11 + 4 + 1 = 38, and two instances, 76, fit PostgreSQL’s default max_connections of 100 less its three superuser_reserved_connections, with 21 left for your tools (Running several instances). Set the budget to PostgreSQL’s max_connections, less its reserved slots and what your tools use, over your instances; give a single instance at a high rate a larger budget and PostgreSQL the room for it. The server logs its pool size and connection count at startup, and warns when one instance alone may open more connections than the server it reaches admits.

A one-shot command (ferroehr db migrate, party-binding refill and the other subcommands) opens only the connections it uses, so it runs beside the instances inside what your tools are left.

Before it answers ready, the server opens the connections each pool keeps while idle, one per core unless min_connections says otherwise, so an instance restarted under load does not refuse its first requests while it opens them. The rest open when a burst asks for them. Keep the kept connections of every instance inside PostgreSQL’s budget too: an instance that cannot open them does not start.

statement_timeout is the backstop for every statement. An AQL query (ad hoc or stored) and the composition-count statistics stop sooner when their caller goes away: a client that disconnects, or the AQL engine’s own record.query.timeout, closes the statement’s connection, its pool slot frees at once, and every pooled session sets PostgreSQL’s client_connection_check_interval to one second, so the backend stops the statement within about a second. Other statements run to their end or to this timeout, so keep it above record.query.timeout: the AQL engine’s typed refusal then fires first and this only catches what the engine does not govern.

[database.<domain>]

One DSN per storage domain. Every domain always lives in its own schema (clinical, party, audit) and every pool carries only its own on its search_path, so no statement can reach another domain’s relations without naming a schema it is not granted. Setting a url here adds the CREDENTIAL separation, and — if the DSN names another host — lets the domain live in a database or cluster of its own.

[database.party]
url_file = "/run/secrets/db-party-dsn"
KeyTypeDefaultDescription
database.clinical.urlsecret URLunsetDSN the clinical domain connects on (role ferroehr_clinical). Unset, it uses database.url.
database.party.urlsecret URLunsetDSN the party (demographic) domain connects on (role ferroehr_party). Unset, it uses database.url.
database.audit.urlsecret URLunsetDSN the audit repository is written through. Unset, it uses database.url.
database.<domain>.url_filepathunsetRead that domain’s DSN from a file instead, for a mounted secret. At most one of the pair.

Why a schema is not always enough: a base backup, WAL archiving and physical replication carry every schema of a database together (PostgreSQL 18, Backup and Restore), so a separation that must survive those is a separation of databases. The Swiss EPDV Art. 10 Abs. 1 lit. b asks for storage “von anderen Datenbeständen getrennt” and the DSV Art. 4 Abs. 5 for the log to be kept “getrennt vom System, in welchem die Personendaten bearbeitet werden”; openEHR reads the same way for the identity cross-reference, which “could be located on different machines” (BASE architecture_overview/master07-security.adoc §Anonymity). No openEHR spec governs pools or database roles — this is our own design.

What the server checks at boot:

  • a runtime role that can read another domain’s relations is refused, always;
  • two domains configured on different DSNs that turn out to authenticate as the same database role are refused — a separation that exists only in the configuration is worse than none, because it reads as one that holds;
  • a missing domain role is a warning under deployment.profile = "sandbox" and a refusal under production, where there are no grants to separate anything with.

Each database is prepared on its own migration credential. [database] migrate_url names one database; a domain whose DSN reaches another database is prepared there, and verified there under migrations = "verify", on the credential named in [database.<domain>]:

[database.party]
url_file = "/run/secrets/db-party-dsn"                   # postgres://ferroehr_party:…@party-db/…
migrate_url_file = "/run/secrets/db-party-migrate-dsn"   # postgres://ferroehr_migrator:…@party-db/…
KeyTypeDefaultDescription
database.<domain>.migrate_urlsecret URLunsetDSN that prepares the database the domain lives in, when that is not the database database.migrate_url reaches. Used for that one step and closed again. Unset, such a database is prepared on the domain’s own url, which a DML-only runtime role cannot do: it is refused on the first bookkeeping table it cannot read.
database.<domain>.migrate_url_filepathunsetRead it from a file instead, for a mounted secret. At most one of the pair.

The server refuses to start when a domain’s url and its storage.migrate URL reach different databases. [storage.linkage] and [storage.migrate.linkage] are refused as unknown keys: the linkage domain is gone (Operations → Upgrading past the linkage domain).

[telemetry]

The console, the request log and the OpenTelemetry export: the section every FerroHEALTH product shares (ferrohealth-telemetry). With no [telemetry.otlp] table nothing is exported.

[telemetry]
format = "auto"
filter = "info,hyper=warn,tower=warn,h2=warn"
logged_query_parameters = []
# environment = "production"

# [telemetry.otlp]
# endpoint = "http://collector:4317"
# traces = true
# metrics = true
# sample_ratio = 1.0
KeyTypeDefaultDescription
formatenum{auto,json,pretty}autoStdout rendering. auto picks json when stdout is not a TTY and the human format when it is. NO_COLOR turns colour off.
filterstringinfo,hyper=warn,tower=warn,h2=warnConsole filter directives, refused at boot when they do not parse; also the value /management/loggers resets to. RUST_LOG is a lower-priority fallback. The filter decides what the console writes and never thins a trace.
logged_query_parameterslist of strings[]The query parameters whose values the request log may carry, each cut to 64 characters. Empty, the request log carries no query value.
environmentstringunsetThe deployment.environment.name resource attribute of every span and metric.
otlp.endpointsecret URL(required in the table)OTLP/gRPC collector endpoint; https:// speaks TLS. Credentials in it are masked in every rendering.
otlp.tracesbooltrueExport FerroEHR’s spans.
otlp.metricsbooltruePush the metrics over OTLP beside the Prometheus pull surface. One meter provider feeds both.
otlp.sample_ratiofloat1.0The share of traces sampled (0.1 is a common production start). A client’s traceparent is never read: every request starts a trace of the server’s own.
otlp.trust_roots_filepathunsetPEM trust roots the collector’s certificate may chain to, beside the platform’s. Refused at boot beside an endpoint that is not https://, and when the file holds no PEM certificate.
otlp.client_identity_filepathunsetPEM client certificate chain and key, for mutual TLS with the collector. Refused at boot beside an endpoint that is not https://, and when the file lacks a PEM certificate or a PEM private key.

The service.name of every span and metric is ferroehr, and service.version is the release and its commit.

The request log writes one line per request: the method, the matched route template, the status, the latency and the server’s request id, never a body, the raw path or a header value. It records the http.server.request.duration histogram, scraped as http_server_request_duration_seconds with the http_request_method, http_route, http_response_status_code and url_scheme labels, and the http.server.active_requests counter of the requests in flight, scraped as http_server_active_requests with the http_request_method and url_scheme labels.

Tip

The ASCII boot banner follows the rendering that is actually installed, not the configured word: it prints for pretty, and for auto only when stdout is a terminal. Under json, and under auto off a terminal (a container, a pipe into a log collector) stdout is parseable JSON from the first byte.

Note

Instrument names carry no Prometheus suffix; the exporter derives one. A counter named auth_failures is scraped as auth_failures_total, and units add _seconds/_bytes. Over OTLP the unsuffixed name is what a collector receives.

The scrape endpoint itself is not opened here; it is management.endpoints.prometheus, which is off until you name a level.

[usage_report]

The daily installation report to FerroPULSE, the collector Cadasto B.V. runs. It is on by default, and every boot logs one line saying whether it is on. What a report carries, when it is sent, the network it needs and the privacy notice are on the Usage report page.

[usage_report]
enabled = true
endpoint = "https://report.ferropulse.eu/v1/report"
deployment = "unknown"
KeyTypeDefaultDescription
enabledbooltrueSend the report. false sends nothing and writes nothing to the database for it. In a CI run (GITHUB_ACTIONS or CI set to true or 1) the report is off whatever this says. Environment: FERROEHR__USAGE_REPORT__ENABLED.
endpointURLhttps://report.ferropulse.eu/v1/reportWhere reports go. Must be an absolute https URL without credentials; plain http is accepted for a loopback host only. Anything else is refused when the configuration is read. Environment: FERROEHR__USAGE_REPORT__ENDPOINT.
deploymentenum{helm,compose,binary,unknown}unknownHow the instance was deployed, as the report states it. The Helm chart sets it to helm. Environment: FERROEHR__USAGE_REPORT__DEPLOYMENT.
state_dirpathunsetRefused. The section is the one every FerroHEALTH product shares, where this key names the directory of the instance id; FerroEHR keeps the instance id and the counts in the clinical database, so every replica reports as one installation.

Under the Helm chart, set the switch with usageReport.enabled. The chart writes enabled and deployment = "helm" into the rendered file itself and refuses to render if either is also set under config.usage_report; endpoint passes through config.usage_report unchanged.