Platform Technical Details

Technical details. You don't need this page to set up or run a platform: the Platform Operator Guide covers that with wbsp-platform setup and up. This page explains the machinery underneath them: where commands run, what the build creates, the settings the platform reads, and how to look inside the cluster directly when you need to.

Table of Contents

1. Where commands run: the toolbox image
2. Install homes
3. Running the stages of up one at a time
4. Running one operation in the foreground
5. Running the tool as a service
6. What the build creates
7. Operator values in .env
8. How the tools find the platform's coordinates
9. On-demand (Lambda) applications
10. How the API server checks sign-ins
11. The configuration store and its encryption key
12. Restoring the database
13. Permissions for the --backdoor paths
14. Looking inside the cluster
15. Technical troubleshooting
16. External reference documentation

Where commands run: the toolbox image

wbsp-platform carries the infrastructure definitions inside itself, and runs every tool it needs (Terraform, the AWS command-line tool, kubectl) from one versioned toolbox image, public.ecr.aws/wbsp-ai/wbsp-platform:<version>. The image tag always equals the binary's version. The only other thing the operator's machine needs is a container runtime.

The binary prints one line saying where a command runs and why. The first toolbox command pulls the image, which takes a few minutes once. Inside the image, the install home, ~/.aws and ~/.kube are mounted at the same paths, and files come back owned by you.

• --native runs a command on this machine with your own tools, and refuses, naming the tool, if one is missing.
• WBSP_INFRA_TOOL=tofu selects OpenTofu instead of Terraform.
• WBSP_TOOLBOX_IMAGE=<image> uses a locally built image instead of the published one.

Refusals name their remedy: no container runtime (with the install hint for your system), runtime not running, image not pullable, image and binary versions differ (with the exact pull command), account or region doesn't match the install's record, and an install home written by a newer binary.

Exit codes: 2 no install selected, account or region mismatch, or a home newer than the binary; 3 runtime missing or not running, image not pullable, version mismatch, or a tool missing under --native; 4 the recorded infrastructure backend doesn't match.

Install homes

Every command acts on an install home: a directory, ~/.wbsp-platform/<name>, holding

• the operator values file (.env),
• the install record (install.yaml),
• the unpacked infrastructure definitions (deploy/aws/),
• after a build, a cache of the infrastructure outputs, and
• the API server's own specification and values (wbsp-api/), including the configuration encryption key.

A home is selected, in order, by --install <name> (or --platform <name>), by WBSP_PLATFORM_HOME, or by the current pointer:

wbsp-platform installs                       # every home on this machine: account, region, layout, version
wbsp-platform installs use acme              # the default for commands that name none
wbsp-platform installs init acme --region ap-southeast-2 --aws-profile acme

installs init creates an empty home and prints the two files to write next: the .env with the required values, and deploy/aws/persistent/operator.auto.tfvars with your DNS zones. setup does init and writes both files for you, so you only need init to build a home by hand.

Each install's infrastructure state lives under its own prefix in the account's records bucket (<name>/persistent/terraform.tfstate and <name>/cluster/terraform.tfstate), so several platforms can share one account's bucket. If your bucket doesn't follow the wbsp-platform-tfstate-<account id> naming, set WBSP_TFSTATE_BUCKET.

Running the stages of up one at a time

up runs these stages in order, and each can be run on its own:

wbsp-platform run provision --plan-only --platform acme is the zero-change check: it plans both infrastructure layers and exits 0 only when nothing would change.

Running one operation in the foreground

run performs one operation, announcing each step as it starts and finishes, with the underlying tool's output streaming beneath. With a terminal attached, it asks for any value the install's .env doesn't have yet and writes your answer there, so it asks only once.

wbsp-platform run provision --platform acme
wbsp-platform run health --platform acme            # short operations answer directly, as JSON
wbsp-platform run destroy --platform acme --confirm acme   # destructive: repeat the install name

The operations are provision, destroy, state-migrate, health, inventory and installs. Exit codes: 0 succeeded, 1 failed, 2 usage error or missing --confirm, 5 another job is already running for that install, 6 a value is missing and there is no terminal to ask, 130 interrupted. Interrupting stops at the next safe point and leaves nothing locked. Every run is recorded as a job in the install home.

The image can be started directly with the operation as its arguments. The same works for setup:

docker run -it --rm \
  --user "$(id -u):$(id -g)" \
  -v "$HOME/.wbsp-platform:$HOME/.wbsp-platform" \
  -v "$HOME/.aws:$HOME/.aws:ro" \
  -e HOME="$HOME" -e AWS_PROFILE \
  public.ecr.aws/wbsp-ai/wbsp-platform:<version> \
  wbsp-platform run provision --install acme

Running the tool as a service

The same image can run as a long-lived service for one install. You can start a twenty-minute build, close your laptop, and ask later how it went:

docker run -d --name wbsp-platform-service \
  --user "$(id -u):$(id -g)" \
  -v "$HOME/.wbsp-platform:$HOME/.wbsp-platform" \
  -v "$HOME/.aws:$HOME/.aws:ro" \
  -e HOME="$HOME" -e AWS_PROFILE \
  -p 127.0.0.1:8470:8470 \
  public.ecr.aws/wbsp-ai/wbsp-platform:<version> \
  wbsp-platform serve --install acme

The -p 127.0.0.1:8470:8470 is deliberate: it publishes the service to this machine only. Anyone who can reach it holds your mounted AWS credentials, so it must not be reachable from the network.

Long operations are started as jobs and polled; short ones are answered in the request. A value missing from the install's .env is refused, naming the key, never prompted for, because nobody is there to answer. One service acts on one install and lists all of them.

curl localhost:8470/healthz
curl -X POST localhost:8470/installs/acme/jobs -d '{"operation":"provision"}'

Letting an AI agent drive it. The service also offers an agent interface at /mcp, with one tool per operation and nothing more: an agent can do exactly what you can through the plain interface. With Claude Code:

claude mcp add --transport http wbsp-platform http://localhost:8470/mcp

Ask it to list installs, start a build, watch the job, report health, or destroy. For a destroy it asks you to confirm the install name, because the tool requires it. An agent is never prompted for a secret.

What the build creates

The infrastructure is built from two layers, applied in order. The split is what lets a cluster be destroyed and rebuilt without touching the public address or any application data.

Persistent layer: survives a cluster rebuild

Cluster layer: destroyed and recreated with the cluster

The cluster layer reads the persistent layer's outputs, so both keep their state in the same records bucket.

To change the region, cluster name or node size, pass a config file to run provision --config:

# platform.yaml
target: aws
aws:
  region: ap-southeast-1
  cluster_name: acme
  node_size: t3.large     # node instance type
  node_count: 3

Operator values in .env

The build reads a few values from the install home's .env rather than from the infrastructure definitions. setup writes them for you. Four are required:

One more is optional:

Never run the infrastructure tool by hand against an install without these values. A raw terraform apply that bypasses wbsp-platform doesn't see them, and produces a destructive plan: it flips certificates to the practice issuer and removes routes. If you must, copy deploy/aws/cluster/secrets.auto.tfvars.example to secrets.auto.tfvars in the install home, fill in the real values, and confirm 0 to destroy before applying.

How the tools find the platform's coordinates

After a build, wbsp-platform reads the infrastructure outputs of both layers (merged; their names don't overlap) and caches them in the install home. You never need to set the database address, cluster details, registry address or Lambda settings by hand. Each comes from the output named below, and each has an environment-variable override that wins when set:

WBSP_RDS_USER, WBSP_RDS_PASSWORD and WBSP_EKS_OIDC_ISSUER have no output and come from the install's values.

wbsp never reads any of this. Only wbsp-platform does. The developer's tool talks to the platform's API server, and its --backdoor paths take coordinates from wbsp-platform env --backdoor or the developer's own settings. The platform's API server doesn't read infrastructure outputs either: api spec writes its coordinates into its deployment specification.

On-demand (Lambda) applications

An aws destination with mode: on-demand runs the application as a Lambda function instead of a cluster workload. The build creates what that needs:

The function's logs land in CloudWatch. With the AWS command-line tool:

aws lambda list-functions --query "Functions[].FunctionName" --output text
aws lambda get-function --function-name <function-name>
aws logs tail /aws/lambda/<function-name> --follow

How the API server checks sign-ins

The API server accepts one kind of credential: one issued by HAP for a tenant. Its HAP address is a single setting, WBSP_HAP_BASE_URL, which api spec fills in. Bringing a new tenant into service needs no change to it and no restart.

There is a second, optional setting for one rare situation: WBSP_HAP_DISCOVERY_BASE_URL. A well-built installation publishes a HAP whose address works from everywhere, from a browser and from inside the platform's own network, and needs nothing here. Leave it unset.

Set it only where that genuinely isn't true: where the platform reaches HAP at a different address from the one HAP's issuers name. Then it holds the address the platform can reach, WBSP_HAP_BASE_URL keeps naming the published one, verification keys are fetched from the first, and credentials are still checked against the second. Key material is only ever fetched from one of those two addresses; a HAP naming a key set at a third is refused.

The tempting alternative, rewriting the issuer to the address that works, makes every token that installation mints valid only on the network it was bent to fit. Don't do that; set this instead.

Every authorisation decision is logged under component authz, with stable outcome and reason fields, so a rise in one refusal reason can be counted without reading messages. The --backdoor recovery path drives AWS with your own credentials and has no dependency on HAP.

The configuration store and its encryption key

Every stored configuration value and every retained image-pull credential is sealed (AES-256-GCM) under WBSP_CONFIG_ENCRYPTION_KEY: base64 of exactly 32 random bytes. api spec generates it once, into the install home's wbsp-api/ values. It must never change under a running platform. To make one by hand:

openssl rand -base64 32

Delivery follows the platform's other durable secrets:

• Cloud platform: in the API server's protected settings, beside WBSP_RDS_PASSWORD.
• Appliance (a single-machine platform): through the API server's environment, sourced from outside the backed-up filesystem (an operating system keychain or similar). A key that rides in the same backup as the database makes the encryption decorative.

Without the key, the platform still starts and serves everything else. The configuration store refuses reads and writes with a sentence naming the variable, the startup log carries one warning, and /api/v1/healthz reports capabilities.config_store: "no_key". With a valid key, the startup log states the key's id (an 8-character fingerprint). The key itself is never logged, returned, or stored in the database it protects.

The store lives in the platform's database, with users, tokens, the deploy registry and hosting records. Consequences you can rely on:

• It rides the ordinary database backup with no extra step. Any pg_dump contains every installation's configuration, retained pull credentials (sealed), and placements.
• On an appliance, confirm your backups actually reach the database's storage. Don't assume it. Where the database lives inside a virtual machine whose disk is deliberately left out of the backups, no backup contains the store: it survives restarts, but not destruction of the machine.
• A lost store costs more than retyped values. Where an application runs (its enclave) and its registered sign-in client are recorded facts, not derived ones. With no recorded placement and no live deployment, a redeploy that names no enclave puts the application in its own new enclave and records that. An application that used to share an enclave then lands somewhere new, and application data restored from a separate backup is bound to the wrong database. Shared-enclave registrations live in the same store, so recover in this order: re-register the enclave, redeploy naming it explicitly, then restore the application data.
• Cluster teardown doesn't lose configuration. The cluster is the destroyable half of the platform; the store isn't in it.
• A backup is diagnosable without the key. Installation identity, mode, value names and placements are readable. No value and no credential can be recovered without the key. Restoring onto a platform holding a different key reports the values as unreadable, naming both key ids, never as absent.

Restoring the database

Restore a snapshot with AWS (RDS → Snapshots → Restore) into a new instance. Then set WBSP_RDS_HOST to the new instance's address in the API server's values (~/.wbsp-platform/<name>/wbsp-api/), and redeploy the API server with the command wbsp-platform api spec prints.

A disposable platform whose data is worthless can set skip_final_snapshot = true in its persistent-layer inputs, so removing it doesn't leave a snapshot behind.

Permissions for the --backdoor paths

Everyday wbsp use needs no AWS credentials: commands go through the platform's API server on the caller's wbsp login session. --backdoor is the exception. It drives AWS directly and needs real AWS permissions. aws-policy prints the least-privilege policy for a chosen set of commands, ready to paste into IAM → Policies → Create policy → JSON:

wbsp-platform aws-policy --cluster acme --deploy      # wbsp deploy/remove/stop/… --backdoor
wbsp-platform aws-policy --cluster acme --database    # wbsp db proxy/connect/run/url/dump/restore
wbsp-platform aws-policy --cluster acme --status      # read-only; safe for auditors and on-call
wbsp-platform aws-policy --cluster acme --platform    # build and remove the whole platform
wbsp-platform aws-policy --cluster acme --on-demand --repo --backup

Always pass --cluster <your platform's cluster name>. The policy names the platform's resources after it, and without it the policy is written for the original name, wbsp-platform. On this command --platform selects the build-and-remove permission set; it does not name a platform.

The policy goes to standard output and the notes to standard error, so redirecting gives a clean file. Account and region come from your AWS profile; override them with --account and --region. A selection larger than AWS's managed-policy size limit is split into several documents: create each as its own policy and attach them all. (wbsp aws-policy is the same command, for developers.)

Two things a policy can't do for you:

• Cluster access isn't an AWS permission. For every command that reaches the cluster, the person must also have an EKS access entry for the cluster. Without it, signing in to the cluster succeeds and every call is still refused.
• --platform isn't a day-to-day grant. It can create and delete the network, the cluster and the database. Grant it for a build, then remove it.

Looking inside the cluster

This is the one place you need tools of your own: kubectl, and for some steps psql. First point your shell at the platform. This exports a kubeconfig plus the platform's coordinates, including the database password, so never write its output to a file:

eval "$(wbsp-platform env --platform acme --backdoor)"

Applications. Each application has its own namespace, wbsp-<tenant>-<app>, and its workload is called <tenant>-<app>:

kubectl -n wbsp-<tenant>-<app> get pods,svc,deploy
kubectl -n wbsp-<tenant>-<app> get ingressroute            # its routes
kubectl -n wbsp-<tenant>-<app> logs deploy/<tenant>-<app> --tail=100 -f
kubectl -n wbsp-<tenant>-<app> exec -it deploy/<tenant>-<app> -- sh
kubectl -n wbsp-<tenant>-<app> set env deploy/<tenant>-<app> --list   # the settings it was given

The platform's own services.

kubectl get ns | grep wbsp-api                # find the API server's namespace
kubectl -n <that namespace> logs deploy/<its workload> --tail=100
kubectl -n traefik logs deploy/traefik --tail=100

Parallel-services environments live in wbsp-svc-<tenant>-<env>, and only allow-listed applications of the same tenant can reach them. See Accessing a parallel environment's database.

kubectl -n wbsp-svc-<tenant>-<env> get pods,svc,netpol
kubectl -n wbsp-svc-<tenant>-<env> port-forward svc/postgres 5544:5432

The platform database, as its administrator. Developers reach their own application's database with wbsp db (see How to reach an app's database). To connect as the administrator, run a throwaway client inside the cluster:

kubectl run psql-tmp --rm -it --image=postgres:16 --restart=Never -- \
  env PGPASSWORD="$WBSP_RDS_PASSWORD" \
  psql -h "$WBSP_RDS_HOST" -U wbsp_admin -d <tenant>_<app>

Technical troubleshooting

External reference documentation

AWS: EKS user guide · EKS networking and network policies · ECR user guide · Lambda developer guide · RDS for PostgreSQL · AWS CLI profiles

Kubernetes: documentation · kubectl cheat sheet · network policies · services and DNS

Terraform / OpenTofu: Terraform CLI · AWS provider · OpenTofu

Related pages: Architecture · The platform model · Certificates · Capability grants · Building an issuing authority