Essence

Essence is a service’s parameters and values: application settings, sizes, flags, passwords, keys. Defaults live in git next to the service, and the operator overrides them in the incarnation’s spec. Secrets are resolved through Vault on the server side at render time and masked on output — they never reach logs and reports in cleartext.

A scenario has Essence available directly (as essence.<path>), while Destiny receives the values it needs only through an explicit hand-off on its input.

Hierarchical assembly

Essence is a hierarchical assembly, not a flat list: values are layered, and a more specific layer overrides a more general one. The default assembly order:

1. default            — the baseline for the whole service
2. OS family          — tweaks for the host's specific OS family
3. Coven tags         — tweaks for the groups the host belongs to
4. incarnation spec   — operator overrides (the strongest layer)

Each subsequent layer sees the values accumulated so far and overrides them. The operator layer (spec) is the strongest: it beats everything the hierarchy assembled. The database stores the original operator spec (not the merge result), so the audit trail shows exactly what the operator overrode.

Layout of the defaults in the service repository:

essence/
├── _default.yaml              # baseline
├── os/<os-family>.yaml        # tweaks for the OS family (if the file exists)
└── coven/<tag>.yaml           # tweaks for a Coven tag (if the file exists)

Essence does not layer by host role (master/replica): the role is volatile, and role-dependent parameters are passed into Destiny through its input by the live role, not through an Essence layer.

For most services the convention-based order above is enough — no separate description is needed.

Declarative pipeline

When you need conditions, iteration, or computed values, the assembly order is described explicitly in essence/_stack.yaml. At each step you have the host facts, the incarnation’s attributes, and the values accumulated so far:

essence/_stack.yaml

stack:
  # 1. Baseline — always
  - file: _default.yaml

  # 2. OS family; skip silently if the file is absent
  - file: "os/${ soulprint.self.os.family }.yaml"
    optional: true

  # 3. Iterate over all the host's Coven tags
  - foreach: "${ host.covens }"
    as: coven_name
    file: "coven/${ coven_name }.yaml"
    optional: true

  # 4. Value computed from the facts gathered so far
  - inline:
      redis_maxmemory: "${ int(soulprint.self.memory.total_mb * 0.6) }mb"
    when: vars.redis_maxmemory == null

Pipeline operators:

Operator	Purpose
file:	Include a file; the path may be an expression.
inline:	Include a set of values without a separate file.
when:	Condition for including the step.
optional: true	Don’t fail if the file named by file: is absent.
foreach: + as:	Iteration: the step repeats for each element.

Values are recomputed between steps, so a later step can build on what an earlier one assembled.

Accessing the values

The assembled Essence is available in scenario expressions as essence.<path>:

- name: Install redis on all cluster hosts
  apply:
    destiny: redis
    input:
      version:   "${ essence.redis_version }"
      maxmemory: "${ essence.redis.maxmemory }"

Destiny does not see essence.* directly (it’s isolated) — the scenario feeds the needed values into Destiny’s input: on the apply: call, as in the example above.

Secrets

Secret values are resolved through Vault at render time, on the server side. The expression engine treats a secret as an ordinary value — otherwise you could not legitimately pass it into a module parameter. Masking is applied on output: a secret never appears in cleartext in apply logs, traces, API responses, or run reports.

A field marked as a secret in the source schema is masked automatically. For tasks that pass secrets through, there is also no_log: on the step — it hides the step’s parameters and result from logs entirely.

What’s next

• Destiny — where Essence is passed through the input contract.
• Scenario — where Essence is available directly as essence.*.
• Architecture — the render phases and server-side secret resolution.