keeper.yml

The config for one Keeper instance. Several instances with different kid sit behind shared PostgreSQL and Redis and form a cluster — each has its own config, differing at least by its identifier.

The file is typed YAML: at startup it is validated (structure, types, invariants). An unknown key, a wrong type, or a violated invariant is rejected with a diagnostic; the config is never applied partially.

Type conventions

Notation	Meaning
string	a UTF-8 string.
int	an integer.
bool	true / false.
duration	a Go duration (1s / 500ms / 1h30m) plus the <N>d suffix for days (30d). A composite form like 1d2h is not supported.
enum{a,b,c}	a string from the listed set.
string(host:port)	host:port; host is an IP or DNS name, port is 1..65535.
vault-ref	a vault:<path> string (for example vault:secret/keeper/postgres); resolved by Keeper’s Vault client at startup.
path	an absolute path on the Keeper host’s filesystem.

In the tables below, default: — marks a required field. Optional blocks can be omitted entirely — the default values apply.

kid — instance identifier

kid: keeper-eu-west-01

Field	Type	Default	Purpose
kid	string (kebab-case, unique in the cluster)	—	A stable, human-readable identifier for the instance. Used in leases on agents, in audit events, and in metric labels. Required.

listen — network listeners

Keeper brings up several independent listeners. gRPC is split into two sub-listeners: bootstrap (agent onboarding, server-only TLS — before onboarding the agent has no client certificate yet) and event_stream (a long-lived bidi stream, mTLS).

listen:
  grpc:
    bootstrap:
      addr: "0.0.0.0:9442"
      tls:
        cert: /etc/keeper/tls/server.crt
        key:  /etc/keeper/tls/server.key
    event_stream:
      addr: "0.0.0.0:8443"            # prod default 8443; dev often 9443
      max_apply_size_mb: 8
      tls:
        cert: /etc/keeper/tls/server.crt
        key:  /etc/keeper/tls/server.key
        ca:   /etc/keeper/tls/ca.crt
  openapi: { addr: "0.0.0.0:8080" }
  mcp:     { addr: "0.0.0.0:8081" }       # optional; remove the block to disable MCP
  metrics: { addr: "0.0.0.0:9090" }

Field	Type	Default	Purpose
listen.grpc.bootstrap.addr	string(host:port)	—	bind address of the onboarding listener (server-only TLS). Required. Typically :9442 (the same in dev and prod).
listen.grpc.bootstrap.tls.cert / .key	path	—	the server certificate and key for onboarding. No CA goes here (TLS is one-way).
listen.grpc.event_stream.addr	string(host:port)	—	bind address of the long-lived stream to agents (mTLS). Required; must differ from the bootstrap address. Prod default :8443, in dev usually :9443.
listen.grpc.event_stream.tls.cert / .key	path	—	the server certificate and key. May be the same as bootstrap.
listen.grpc.event_stream.tls.ca	path	—	the CA used to validate agents’ client certificates during the mTLS handshake.
listen.grpc.event_stream.max_apply_size_mb	int (MiB, ≥1)	8	The size ceiling for a single outbound message to an agent (chiefly a batch of rendered tasks). An attempt to send more fails fast with a clear error instead of a silent rejection on the agent side. Must be ≤ the agent’s recv limit (keeper.max_apply_size_mb in soul.yml); the defaults on both sides match (8 MiB).
listen.openapi.addr	string(host:port)	—	bind address of the Operator API (the operator’s primary interface, OpenAPI). A mandatory listener; there is no option to disable it. Typically :8080.
listen.mcp.addr	string(host:port)	—	bind address of the MCP server (an interface on par with OpenAPI). An optional listener: if addr is unset / the block is removed, the MCP server does not come up and MCP is disabled. Typically :8081.
listen.metrics.addr	string(host:port)	—	bind address of the dedicated Prometheus /metrics. A separate port (usually :9090), without the Operator API auth chain. A mandatory listener; there is no option to disable it. Optional protection — the metrics block.

postgres

postgres:
  dsn_ref: vault:secret/keeper/postgres
  pool: { min: 5, max: 50 }

PostgreSQL is the cluster’s only cold store of state: the registries of agents and operators, the service catalog, and the logs.

Field	Type	Default	Purpose
postgres.dsn_ref	vault-ref	—	A Vault reference to the full DSN. A plaintext DSN is never written to the file. The field in Vault KV is dsn.
postgres.pool.min	int (≥1)	2	The minimum pool size per instance.
postgres.pool.max	int (≥min)	20	The maximum pool size. Total flow to PG = max × number of instances — account for this when tuning PostgreSQL’s max_connections.

redis

Redis is the hot layer and the coordination bus between instances: presence/heartbeat, leases on agent identifiers, leader election for background tasks. The client supports three topologies natively — mode: standalone | sentinel | cluster; an empty/omitted mode = standalone (forward-compat). Choosing a mode for HA — prerequisites.md → Redis.

# Standalone (default): a single node.
redis:
  mode: standalone               # can be omitted — this is the default
  addr: "redis.example.com:6379"
  password_ref: vault:secret/keeper/redis

# Sentinel: HA with automatic failover (the recommended on-premise prod path).
redis:
  mode: sentinel
  master_name: mymaster
  sentinels:
    - "sentinel-1.example.com:26379"
    - "sentinel-2.example.com:26379"
    - "sentinel-3.example.com:26379"
  password_ref: vault:secret/keeper/redis                    # password of the Redis nodes
  sentinel_password_ref: vault:secret/keeper/redis#sentinel  # optional, password of the sentinel nodes

# Cluster: slot sharding (horizontal scaling).
redis:
  mode: cluster
  nodes:
    - "redis-1.example.com:6379"
    - "redis-2.example.com:6379"
    - "redis-3.example.com:6379"
  password_ref: vault:secret/keeper/redis

Field	Type	Default	Purpose
redis.mode	enum{standalone,sentinel,cluster}	standalone (empty/omitted)	The Redis topology. standalone — a single node; sentinel — Redis Sentinel HA (master discovery in the client); cluster — Redis Cluster (slot routing in the client).
redis.addr	string(host:port)	—	The node address. Required with mode: standalone; ignored with sentinel/cluster.
redis.master_name	string	— (optional)	The name of the monitored master group. Required with mode: sentinel.
redis.sentinels	list<string(host:port)>	— (optional)	Addresses of the sentinel nodes. Required (non-empty) with mode: sentinel.
redis.nodes	list<string(host:port)>	— (optional)	Addresses of the cluster nodes for bootstrap discovery (the client will pull the full topology itself). Required (non-empty) with mode: cluster.
redis.password_ref	vault-ref or string	—	The Redis password. vault:<mount>/<path>[#field] is resolved from Vault (default field password, override via #field); plaintext works as-is; empty — no password.
redis.sentinel_password_ref	vault-ref or string	— (optional)	A separate password for the sentinel nodes themselves. The same form as password_ref. Only meaningful with mode: sentinel.

vault

Vault is a mandatory dependency of Keeper: the block is checked at startup, and without a reachable Vault, Keeper will not start. It carries the PKI for issuing agents’ mTLS identity, the KV for secrets (DSNs, passwords), and the operator-token signing key.

Keeper reads KV secrets transparently from both KV v1 and KV v2 — the mount version is detected automatically, and it makes no difference to the operator which one is deployed. There is no need to specify the version in the config (see the optional kv_version override below).

# Dev: a static token
vault:
  addr: "http://127.0.0.1:8200"
  token: "dev-root-token"
  auth: { method: token }      # default; the auth block can be omitted
  pki_mount: "pki"

# Prod: AppRole
vault:
  addr: "https://vault.example.com:8200"
  auth:
    method: approle
    role_id: keeper-prod                          # not a secret — inline is fine
    secret_id_file: /etc/keeper/vault-secret-id   # mode 0400/0600
  pki_mount: "pki/soulstack"

vault.auth.method chooses how Keeper authenticates to Vault:

• token (default) — a static token from vault.token. Handy for local development. The entire auth block can be omitted — that is equivalent to method: token.
• approle — the prod path: Keeper does approle/login with role_id + secret_id and gets a renewable token that is refreshed in the background.

The AppRole credentials are not read from Vault itself (that would be a circular dependency: they are exactly what Keeper logs in with). The source is local: role_id is not a secret and is set inline; secret_id is a secret and is taken from a permission-restricted file (secret_id_file) or an environment variable (secret_id_env).

Field	Type	Default	Purpose
vault.addr	string (URL)	—	The Vault address.
vault.token	string	—	A static token for method: token. Must not be set with method: approle.
vault.kv_mount	string	secret	The KV mount point (without specifying a version). Works for both KV v1 and KV v2 — the version is detected automatically.
vault.kv_version	enum{"1","2"}	— (auto)	An optional override for the KV mount version. By default it is auto-detected and need not be set. Specify it only for a hardened Vault where auto-detection is blocked by ACL (the policy forbids reading the internal mount endpoint). A value outside the set is rejected.
vault.auth.method	enum{token,approle}	token	The authentication method. Empty = token.
vault.auth.role_id	string	—	The AppRole role_id (not a secret). Required with method: approle.
vault.auth.secret_id_file	path	—	A path to the file holding secret_id. Mutually exclusive with secret_id_env; exactly one is required with approle.
vault.auth.secret_id_env	string	—	The name of the environment variable holding secret_id.
vault.pki_mount	string	—	The PKI engine mount through which Keeper issues agents’ identity during onboarding.
vault.pki_role	string	(optional)	The PKI role name. Vault signs the CSR via <pki_mount>/sign/<pki_role>.

One config for KV v1 and v2

There is no need to specify the KV version in the config — the same vault block works on both KV v1 and KV v2. Keeper detects the mount version automatically (a probe via sys/internal/ui/mounts), so in an ordinary setup a single kv_mount is enough:

vault:
  addr: "https://vault.internal:8200"
  auth:
    method: approle
    role_id: keeper-prod
    secret_id_file: /etc/keeper/vault-secret-id
  kv_mount: "secret"          # the mount path; v1 or v2 — detected automatically
  pki_mount: "pki/soulstack"

The symmetry extends to secret references too. A vault: ref is written mount-relative and without the data/ segment — the path is the same for both versions, and the client inserts the data/ segment (needed only by KV v2) itself:

postgres:
  dsn_ref: vault:secret/keeper/postgres                 # takes the dsn field
redis:
  password_ref: vault:secret/keeper/redis#password      # a specific field via #
auth:
  jwt:
    signing_key_ref: vault:secret/keeper/jwt-signing-key

The same rule applies to explicitly reading a secret in a scenario via core.vault.kv-read: path is given without data/ and works the same on both versions.

- name: Read DB credentials from Vault
  on: keeper
  module: core.vault.kv-read
  register: db_creds
  params:
    path:   secret/redis/admin       # without data/; works on both v1 and v2
    fields: [username, password]     # optional; without it the whole payload is returned

The only place the version appears in the config is the optional kv_version override. It is needed only for a hardened Vault where the policy closes the internal probe endpoint sys/internal/ui/mounts and auto-detection fails:

vault:
  # ... the rest of the block ...
  kv_mount: "secret"
  kv_version: "1"             # pin the version explicitly since auto-detection is unavailable

Normally there is no kv_version line in the config: the field is optional and the default is auto (see the vault field table above).

auth

JWT authentication of operators for the Operator API and MCP. The block is responsible only for signing and the token format; the operator registry and the role catalog live in PostgreSQL and are managed through the API.

auth:
  jwt:
    signing_key_ref: vault:secret/keeper/jwt-signing-key
    issuer: keeper-eu-west-01
    ttl_default: 24h
    ttl_bootstrap: 720h        # 30 days

Field	Type	Default	Purpose
auth.jwt.signing_key_ref	vault-ref	vault:secret/keeper/jwt-signing-key	The Vault KV path to the operator-token signing key.
auth.jwt.issuer	string	<kid>	The value of the iss claim in issued tokens. Defaults to the instance’s kid; you can set a single name for the whole cluster.
auth.jwt.ttl_default	duration	24h	The TTL of ordinary operator tokens. A short TTL is a natural safeguard (a revoked operator stops working once the token expires).
auth.jwt.ttl_bootstrap	duration	720h (30 days)	The TTL of the first bootstrap token issued by keeper init.

The agent has no auth block — it authenticates to Keeper over mTLS (see soul.yml). JWT is for operators only.

metrics

An optional block that protects /metrics (the bind address itself is listen.metrics.addr). Without the block, the endpoint is served without authentication.

metrics:
  auth:
    basic:
      enabled: true
      username: scrape
      password_ref: vault:secret/keeper/metrics-password

Field	Type	Default	Purpose
metrics.auth.basic.enabled	bool	false	Enable HTTP Basic auth on /metrics.
metrics.auth.basic.username	string	—	The username. Required with enabled: true.
metrics.auth.basic.password_ref	vault-ref	—	A Vault reference to the password (the password field in KV). Plaintext is forbidden. Required with enabled: true.

The password is compared in constant time and never reaches the logs. The agent has no symmetric protection via Vault (soul has no Vault client) — there the password is given by a file, see soul.yml → metrics.

otel

otel:
  enabled: true
  exporter: otlp
  endpoint: "otel-collector.example.com:4317"

OpenTelemetry is push: when enabled, Keeper sends traces to the given OTLP collector itself. Keeper has no inbound listener port for OTel.

Field	Type	Default	Purpose
otel.enabled	bool	false	Enable export.
otel.exporter	enum{otlp}	otlp	The export format.
otel.endpoint	string(host:port)	—	The OTLP collector address (gRPC). Required with enabled: true.
otel.export_metrics	bool	false	Optionally push metrics over OTLP in addition to the Prometheus scrape. At this stage only traces are exported; by default metrics go through Prometheus /metrics.

logging

logging:
  level: info
  format: json
  file: /var/log/keeper/keeper.log    # empty → stderr without rotation
  rotation:
    max_size_mb: 100
    max_age_days: 7
    max_files: 10
    compress: true

Behavior depends on logging.file:

• unset → output to stderr without rotation (convenient under systemd/journald and in a container);
• set → writes to a file with built-in rotation (no dependency on an external logrotate), with archives alongside following the <file>-<timestamp>.<ext> pattern.

Field	Type	Default	Purpose
logging.level	enum{debug,info,warn,error}	info	The log level.
logging.format	enum{json,text}	json	json for machine processing, text for humans.
logging.file	path	— (stderr)	The path to the log file. Empty — stderr without rotation.
logging.rotation.max_size_mb	int (MB)	100	The rotation threshold for a single file.
logging.rotation.max_age_days	int (≥0)	7	How many days to keep an archive.
logging.rotation.max_files	int	10	How many archives to keep.
logging.rotation.compress	bool	true	Whether to compress archives.

The logging.rotation.* fields apply only when logging.file is set.

plugins

The catalog of plugins hosted by Keeper: CloudDriver plugins (for cloud provisioning) and SshProvider plugins (for push over SSH). Keeper resolves them from git repositories into a local cache at startup.

Cloud — what’s ready, what’s on the roadmap

The operator flow for cloud is available: Cloud-Provider and Cloud-Profile are registered through the Operator API (REST /v1/providers, /v1/profiles, MCP parity, web UI) under RBAC provider.* / profile.* with an audit log — see core.cloud → Registering providers and profiles. The plugins.cloud_drivers section below defines the resolution catalog for driver plugins (where Keeper pulls the soul-cloud-* binary from), which is separate from the provider registry. What remains on the roadmap are the ready-made soul-cloud-* drivers: they are not yet part of the base distribution. See core.cloud.

plugins:
  cache_root: /var/lib/soul-stack-keeper/plugins
  work_root:  /var/lib/soul-stack-keeper/plugin-src
  fetch_timeout: 120s
  max_artifact_size_mb: 256
  max_clone_size_mb: 1024
  cloud_drivers:
    - { name: aws, source: "git@github.com:example/soul-cloud-aws.git", ref: v2.0.0 }
  ssh_providers:
    - { name: vault-ssh, source: "git@github.com:example/soul-ssh-vault.git", ref: v1.0.0 }

Field	Type	Default	Purpose
plugins.cache_root	path (abs.)	/var/lib/soul-stack-keeper/plugins	A cache of built plugin artifacts.
plugins.work_root	path (abs.)	/var/lib/soul-stack-keeper/plugin-src	The root of the resolver’s working git clones. Must be outside cache_root.
plugins.fetch_timeout	duration	120s	The ceiling for one chain of git operations when resolving a plugin.
plugins.max_artifact_size_mb	int (MiB, ≥1)	256	The size ceiling for a single extracted plugin binary (disk protection against a hostile repository). Exceeding it fails closed.
plugins.max_clone_size_mb	int (MiB, ≥1)	1024	The size ceiling for the clone’s working tree. Exceeding it fails closed.
plugins.cloud_drivers[].name / .source / .ref	string / git URL / git ref	—	The provider name, the plugin’s git repository, and its ref (a tag or a branch).
plugins.ssh_providers[].name / .source / .ref	string / git URL / git ref	—	The same for push SSH providers.

An artifact’s version is a git ref (a tag or a branch), not a field in the manifest; semver ranges are not used.

plugin_runtime

The lifecycle of the plugin host process on the Keeper side: handshake and shutdown timeouts, the capabilities whitelist, the resource-conflict policy.

plugin_runtime:
  socket_dir: /var/run/soul-stack-keeper/plugins
  startup_timeout: 10s
  shutdown_grace: 10s
  allowed_capabilities:
    - run_as_root
    - network_outbound
    - network_inbound
    - vault_access
    - fs_write_root
    - exec_subprocess
  conflict_policy: warn
  enable_tls: false

Field	Type	Default	Purpose
plugin_runtime.socket_dir	path	/var/run/soul-stack-keeper/plugins/	The directory of the plugins’ Unix domain sockets (mode 0700, owned by Keeper’s service user).
plugin_runtime.startup_timeout	duration	10s	The time from launching the plugin process to the handshake. Exceeding it triggers SIGTERM, then SIGKILL.
plugin_runtime.shutdown_grace	duration	10s	The window from SIGTERM to SIGKILL when stopping a plugin.
plugin_runtime.allowed_capabilities	list<enum>	all 6	The whitelist of plugin capabilities. The linter rejects Destiny before a run if a plugin requests a capability outside the list. All six are allowed by default; narrow them per your security policy.
plugin_runtime.conflict_policy	enum{warn,fail}	warn	What to do if two plugins in one run claim the same resource: warn — record an audit entry and continue, fail — mark the step as failed.
plugin_runtime.enable_tls	bool	false	mTLS on the plugin socket. In the current release — only false; security is provided by the socket’s 0700 permissions.

audit

audit:
  enabled: true
  otel_export: true
  retention_days: 365

Field	Type	Default	Purpose
audit.enabled	bool	true	The global audit switch. With false the write path does not write to PostgreSQL. Keep it true in production (compliance).
audit.otel_export	bool	true	Duplicate the audit event into an OTel span as an attribute (the source of truth is PostgreSQL).
audit.retention_days	int (≥1)	365	The retention period for audit records (days). Cleanup is performed by a background task.

The agent has no audit block: Soul-side events (run reports) go through Keeper and are written by it.

reaper

A background task that cleans expired records out of PostgreSQL (one leader per cluster, elected via a Redis lease).

reaper:
  enabled: true
  interval: 1h
  dry_run: false
  batch_size: 500
  lock_ttl: 5m
  rules:
    expire_pending_seeds: { enabled: true, max_age: 24h, action: delete }
    mark_disconnected:    { enabled: true, stale_after: 90s, action: set_status, target_status: disconnected }
    purge_audit_old:      { enabled: true, max_age: 365d, action: delete }
    # … the rest of the rules

Field	Type	Default	Purpose
reaper.enabled	bool	true	Enable cleanup.
reaper.interval	duration	1h	The pass interval.
reaper.dry_run	bool	false	A dry run with no mutations (for checking).
reaper.batch_size	int	500	The batch size of a single pass.
reaper.lock_ttl	duration	5m	The TTL of the Redis lease on leadership.
reaper.rules	map<string, object>	—	Predefined cleanup rules (expired onboarding tokens, marking dropped agents, deleting old audit, and so on). A rule’s fields depend on its action.

cadence_scheduler

An optional block for the scheduler of recurring runs (one leader per cluster). If absent, it comes up with defaults.

cadence_scheduler:
  enabled: true        # omitted → ON by default; false → disable
  poll_floor: 30s
  poll_ceiling: 60s
  poll_idle: 120s
  lock_ttl: 5m

Field	Type	Default	Purpose
cadence_scheduler.enabled	bool (tri-state)	nil → ON	Enables the scheduler. Omitted/null → ON (enabled by default); false → disable; true → enable. Read at startup.
cadence_scheduler.poll_floor	duration	30s	The lower bound of the poll step. The absolute minimum is 30s.
cadence_scheduler.poll_ceiling	duration	60s	The upper bound of the poll step. Must be ≥ poll_floor.
cadence_scheduler.poll_idle	duration	120s	The poll step when the schedule registry is empty. Must be ≥ poll_ceiling.
cadence_scheduler.lock_ttl	duration	5m	The TTL of the Redis lease on the scheduler’s leadership.

tempo

An optional per-operator rate limiter for heavy write endpoints. Without the block, it is enabled with defaults.

tempo:
  enabled: true
  voyage_create:    # limit on creating batch runs
    rate: 10        # tokens per second
    burst: 20       # allowed burst
  voyage_preview:   # limit on preview (softer — no writes)
    rate: 30
    burst: 60

Field	Type	Default	Purpose
tempo.enabled	bool (tri-state)	true	Enable the limiter. Omitted/null → enabled.
tempo.voyage_create.rate	float	10	The token refill rate for creating a run (requests per second).
tempo.voyage_create.burst	int	20	The allowed burst for creation.
tempo.voyage_preview.rate	float	30	The refill rate for preview (softer — an operation with no writes).
tempo.voyage_preview.burst	int	60	The allowed burst for preview.

web_ui_enabled

A top-level toggle for the built-in operator web UI on the /ui route. The actual UI is compiled into the keeper binary and served by it out of the box — it needs no separate process, port, or backend; the static assets ride on the existing Operator API listener (listen.openapi.addr, usually :8080). More on access and working with the UI — Web interface.

# web_ui_enabled: true    # default (omitted / null → ON); false — opt-out

Field	Type	Default	Purpose
web_ui_enabled	*bool (tri-state)	true	Whether to mount the built-in UI on /ui. Omitted / null → true (default-ON, single-binary UI “out of the box”); an explicit false → opt-out: the /ui static assets are not mounted, while /v1/* and /docs are untouched. Unlike tempo / cadence_scheduler, it does not depend on infrastructure — the UI is baked into the binary, no external backend is needed.

Restart-required, not hot-reload

web_ui_enabled is read once at startup and fixed in the router; SIGHUP and reload through the API do not toggle it (symmetric to bringing subsystems up or down). To enable or disable the UI, change the key and restart keeper. Enabling the UI opens no new ports: the static assets share :8080 with the Operator API.

HA invariant: the worker pool with multiple instances

In a cluster of several live instances, run execution must go through a shared worker pool rather than being tied to the memory of a single instance. This is controlled by the acolytes key:

acolytes: 4    # number of execution-pool workers per instance; 0 — pool disabled

Field	Type	Default	Purpose
acolytes	int (≥0)	0	The number of run-execution pool workers per instance. 0 — the pool is not brought up, and execution goes the “classic” way in the memory of the owning instance.

acolytes: 0 — for a single Keeper only

The acolytes: 0 mode is correct only with a single live instance. In an HA cluster (several instances on shared PG + Redis), a run started on one instance but with the agent on another’s stream will execute on the host, yet the incarnation will hang in applying forever — the owning instance will not see the other’s report.

For that reason, with acolytes: 0 and other live instances detected, Keeper by default refuses to start. For multiple instances, set acolytes > 0. The refusal can be lifted by a deliberate opt-out (for example, during a migration when the “other” instance is on its way out) — via the allow_unsafe_single_path_multi_keeper: true key.

Sizing: how many acolytes to set

What it is. acolytes: N is the number of background “acolyte” workers in one Keeper instance that pull the shared run queue (apply jobs) from PostgreSQL and execute them. The value is per instance; the cluster’s total parallelism = acolytes × number of live instances.

What it scales with. With the number and width of concurrent runs, not with the number of souls: 10,000 agents is not 10,000 acolytes. Agents simply hang on their streams and do not occupy an acolyte on their own; an acolyte is active only when the queue holds jobs from a running run. One run over 50 hosts = 50 jobs in the queue, and all live acolytes of all cluster instances work through them in parallel.

How many to set — starting numbers:

Topology	acolytes (per instance)	Why
A single Keeper	0	The standard default: a run is executed by the very instance that accepted it. Correct and safe for a single node.
HA of 2–3 instances	10	A sensible start: half the default postgres.pool.max (20) with headroom; gives 20–30 parallel jobs across the cluster. acolytes > 0 is mandatory here (see HA invariant).
High load	20+	Only if under real load the queue of planned jobs steadily grows and applies lag behind — wide runs over hundreds of hosts or many simultaneous launches.

The 0 default is not “execution off forever”

acolytes: 0 does not turn off the scheduler; it selects the previous single-instance path: the owning instance drives the run to completion itself, without a shared queue. This path is unusable for any multi-keeper setup — see HA invariant.

The ceiling and the tie to postgres.pool.max. Keep acolytes ≤ postgres.pool.max − headroom (~8–10). Each worker takes PostgreSQL connections from the shared pool, and the Operator API, Reaper, and Voyage processing compete for the same connections. If there are more workers than connections in the pool, the extras simply wait for a free connection (starvation, rising latency) and add no parallelism. Need more workers — raise postgres.pool.max first, then acolytes. acolytes itself has no hard upper limit in the config (only ≥ 0 is validated); the actual ceiling is set by the PG pool size.

hot_reload

hot_reload:
  enable_signal: true
  enable_inotify: false
  audit_correlation_id: true

Controls the hot-reload triggers. The block is optional; if absent, the defaults from the table apply. All three fields require a restart when changed (they control the reload mechanism itself).

Field	Type	Default	Purpose
hot_reload.enable_signal	bool	true	Enable the SIGHUP trigger: the process re-reads keeper.yml from disk, validates it, and swaps it atomically.
hot_reload.enable_inotify	bool	false	Auto-reload on file change (Linux-only). Off by default.
hot_reload.audit_correlation_id	bool	true	Generate a correlation id for reload audit events.

Hot-reload

The config can be re-read without a full process restart. Two paths:

• File edit — the operator edits keeper.yml on the host and sends the process a SIGHUP. Pipeline: parse → validation → atomic swap → audit.
• API/MCP — a config mutation through the Operator API; in addition to the swap, a write-back happens: the changed value is written back into keeper.yml (preserving comments and key order, with an atomic file replacement).

Any validation error leaves the current state untouched — the file is not modified.

What applies without a restart: the log level, PostgreSQL pool parameters, background-task intervals and rules, rate-limiter thresholds, plugin policies (capabilities whitelist, conflict policy, timeouts for new launches).

What requires a restart: listener addresses and TLS certificates, the PostgreSQL DSN and Redis parameters, the Vault address/auth, the token signing key, log-file paths and rotation parameters, the OTel exporter, and bringing whole subsystems up or down (the scheduler, the worker pool).

Each cluster instance re-reads its own config independently — there is no centralized cross-host reload synchronization in the current release.

What moved to the DB (not in the config)

• RBAC (roles, operator bindings, permissions) — in PostgreSQL, managed through the API. An rbac: key in keeper.yml is rejected.
• The service registry and related well-known values — in PostgreSQL, managed through the API. The services: / default_destiny_source: keys in keeper.yml are rejected.

Full example

A minimal valid keeper.yml with all required fields:

kid: keeper-eu-west-01

listen:
  grpc:
    bootstrap:
      addr: "0.0.0.0:9442"
      tls:
        cert: /etc/keeper/tls/server.crt
        key:  /etc/keeper/tls/server.key
    event_stream:
      addr: "0.0.0.0:8443"
      tls:
        cert: /etc/keeper/tls/server.crt
        key:  /etc/keeper/tls/server.key
        ca:   /etc/keeper/tls/ca.crt
  openapi: { addr: "0.0.0.0:8080" }
  mcp:     { addr: "0.0.0.0:8081" }
  metrics: { addr: "0.0.0.0:9090" }

postgres:
  dsn_ref: vault:secret/keeper/postgres
  pool: { min: 5, max: 50 }

redis:
  addr: "redis.example.com:6379"
  password_ref: vault:secret/keeper/redis

vault:
  addr: "https://vault.example.com:8200"
  auth:
    method: approle
    role_id: keeper-prod
    secret_id_file: /etc/keeper/vault-secret-id
  pki_mount: "pki/soulstack"

auth:
  jwt:
    signing_key_ref: vault:secret/keeper/jwt-signing-key
    issuer: keeper-eu-west-01
    ttl_default: 24h
    ttl_bootstrap: 720h

otel:
  enabled: true
  exporter: otlp
  endpoint: "otel-collector.example.com:4317"

logging:
  level: info
  format: json
  rotation: { max_size_mb: 100, max_files: 10, compress: true }

audit:
  enabled: true
  otel_export: true
  retention_days: 365

reaper:
  enabled: true
  interval: 1h
  batch_size: 500
  lock_ttl: 5m
  rules:
    expire_pending_seeds: { enabled: true, max_age: 24h, action: delete }
    mark_disconnected:    { enabled: true, stale_after: 90s, action: set_status, target_status: disconnected }
    purge_audit_old:      { enabled: true, max_age: 365d, action: delete }

# For an HA cluster (several instances), set the worker pool:
# acolytes: 4

See also

• soul.yml — the agent config.
• Configuration → Overview — secrets via Vault, cross-cutting capabilities, hot-reload in brief.
• Operations — scaling, monitoring, upgrades.
• Keeper — the component’s role and ports.