Your first souls service

This guide is a full walkthrough from a service’s git repository to state applied across several hosts, and verifying the result. Quick Start showed a minimal service on a single host; here the same logic expands to many souls bound to a Coven, and we break down what happens at each step.

This assumes several hosts are already connected (see Souls onboarding) and the first Archon’s JWT is in TOKEN.

What we’ll build

The chain of entities we’ll walk through:

flowchart LR
    R["service git repo<br/>scenario/ + essence/"] -->|register| C["service catalog"]
    C -->|create incarnation| S["create scenario"]
    S --> H["applied on the coven's hosts, by covens"]
    H --> P["incarnation.state in Postgres"]

Service — a service type: a git repository with scenarios (scenario/), default parameters (essence/), and a manifest. Registered in the catalog as a git source + ref.
Incarnation — a runtime instance of a service: a concrete deployment bound to a set of hosts. Creating an incarnation triggers the create scenario.
Scenario — an operation on an incarnation (create, restart, …). create brings the hosts to their initial state.

Step 1. Prepare the service repository

A service is an ordinary git repository with this layout:

my-service/
├── service.yml              # service manifest (state_schema, metadata)
├── essence/
│   └── _default.yaml        # default parameters (Essence)
└── scenario/
    └── create/
        └── main.yml         # the initial-deployment scenario

A minimal scenario/create/main.yml installs a package and drops a file on every host in the incarnation. With no orchestration delta (on: / serial: / where:), the scenario applies to all of the incarnation’s hosts at once:

input:
  banner_text:
    type: string
    default: "Managed by Soul Stack"

state_changes:
  sets:
    banner: "${ input.banner_text }"

tasks:
  - name: Install htop package
    module: core.pkg.installed
    params:
      name: htop

  - name: Write managed banner to motd
    module: core.file.present
    params:
      path: /etc/motd
      content: "${ input.banner_text }\n"
      mode: "0644"

Steps describe desired state (core.pkg.installed = “the package is installed”, core.file.present = “a file with this content exists”), not imperative commands. Each step is idempotent: a repeat run changes nothing if the state is already reached. The task grammar is in DSL → Destiny, the orchestration delta in DSL → Scenario.

Commit and push the repository, and note the ref (a tag or branch) you’ll reference — a service’s version in Soul Stack is set by a git ref, not by a field in the manifest.

Step 2. Register the service

For Keeper to resolve a service, it has to be in the service catalog. The catalog lives in Postgres and is populated through the Operator API — a git source + ref:

curl -s -X POST http://keeper.example.com:8080/v1/services \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-service",
    "git": "https://git.example.com/svc/my-service.git",
    "ref": "v1.0.0"
  }'

ref is a stable tag (v1.0.0) for production or a branch (main) for development. Keeper materializes the repository at this ref when it resolves the service.

Step 3. Create an incarnation on a coven

Creating an incarnation triggers the create scenario on the hosts. Let’s bind the incarnation to the prod Coven — it will roll out to all hosts carrying that label:

curl -s -X POST http://keeper.example.com:8080/v1/incarnations \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-service-prod",
    "service": "my-service",
    "covens": ["prod"],
    "input": { "banner_text": "Managed by Soul Stack — prod" }
  }'

The response is 202 Accepted with an apply_id: the operation is asynchronous. Keeper resolves the service at the ref, unrolls the create scenario, and applies it on every prod host.

The covens field defines the incarnation’s host set. If the steps in the create scenario don’t narrow the target (no on: / where:), they run on all of those hosts. Narrowing and rollout order are the subject of Orchestration.

Step 4. Wait for the ready status

Query the incarnation’s status (applying → ready on success, error_locked on failure):

curl -s http://keeper.example.com:8080/v1/incarnations/my-service-prod \
  -H "Authorization: Bearer $TOKEN"

The run history (state snapshots with per-host results):

curl -s http://keeper.example.com:8080/v1/incarnations/my-service-prod/history \
  -H "Authorization: Bearer $TOKEN"

The history shows which hosts the run passed on and where something changed (changed). The same run is available visually in the web UI (/ui) and through soulctl (soulctl).

Step 5. Verify the result on the hosts

At status: ready the committed state is available in incarnation.state (the banner field from state_changes). Check the facts directly on the hosts — for example, on host-01.example.com:

which htop && cat /etc/motd
# → /usr/bin/htop
# → Managed by Soul Stack — prod

The same holds on the other prod hosts: the create scenario applied to the entire set defined by covens.

What comes next: re-applying and evolving

Re-running the same scenario is idempotent: state that’s already reached stays put, and changes happen only where the actual state has drifted from the desired state.
New operations on an incarnation (restart, adding a user, updating a config) are other scenarios in the same service (scenario/restart/, scenario/add_user/, …), launched as runs against the existing incarnation.
A new service version is a new ref. Upgrading an incarnation to it, along with incarnation.state migrations, is a separate operator-initiated operation (see Operations → Incarnation upgrade).

Next steps

Scenario orchestration — control the rollout order: rolling, run_once, targeting by facts.
DSL → Scenario — the orchestration delta, state_changes, passing data between hosts.
DSL → Essence — where parameters come from and how to override them.
Monitoring out of the box — observe hosts after rollout.