Infrastructure as Code

AlmaForge keeps infrastructure access configuration in the same places DevOps teams already review and roll back: daemon settings in ALMA_* environment variables, durable cluster state in YAML resource manifests, and rollout through CI or config management.

AlmaForge is configured from two inputs: daemon configuration and resource configuration.

Daemon configuration controls how each almad process starts. Use it when you are changing the local process shape: which services run on this host, which addresses it binds and advertises, which TLS and data paths it uses, and how it joins an existing cluster. These settings are read before the process can serve traffic.

Resource configuration is the cluster's desired state: infrastructure targets to manage access to, access policies and RBAC, SSO connectors, join tokens, storage, and other durable cluster objects. These objects are declared as Kubernetes-style YAML manifests with metadata and spec fields. API-managed resources are applied with alma apply and inspected with alma get, edit, and delete. Startup resources are placed in /etc/almaforge/almad.d, a conf.d-style directory managed by your configuration management system.

Choose a Workflow

Use the path that matches the change you are making:

Situation	Use	Why
Fresh cluster bootstrap	ALMA_* environment plus drop-in files	The Auth Service can load storage, SSO, roles, and tokens before an admin session exists.
Day-to-day policy or target changes	alma apply -f	The API validates and applies the change without a daemon restart.
Git-backed config management today	Render files into /etc/almaforge/almad.d	The Auth Service picks the change up on its own. This works with Ansible, Chef, Puppet, Nix, Salt, Kubernetes ConfigMaps, and plain rsync.
Full GitOps reconciliation	Not available in this release	Use drop-ins or alma apply -f today.

Daemon Configuration

Daemon configuration is process-local startup input for almad. It belongs with the process manager, container spec, shell environment, or Kubernetes Pod that starts the daemon. It answers questions like:

• Which control plane components should this instance run?
• Which address should it listen on?
• Which public address should clients and agents use?
• Where is local state stored?
• If this process is an agent, how does it join the cluster?

Use command-line flags for one-off runs and debugging:

Terminal

almad start \    --auth-service \    --proxy-service \    --listen=0.0.0.0:443 \    --advertise=almaforge.example.com:443

For packaged, container, and automation-driven deployments, set the same values with ALMA_* environment variables:

env

ALMA_AUTH_SERVICE=true
ALMA_PROXY_SERVICE=true
ALMA_SSH_SERVICE=false
ALMA_LISTEN=0.0.0.0:443
ALMA_ADVERTISE=almaforge.example.com:443

Most daemon CLI options have matching environment variables using the ALMA_<OPTION> convention. For example, --listen maps to ALMA_LISTEN. When both are set for the same option, the command-line flag wins.

Official RPM and DEB packages ship a systemd unit that reads /etc/almaforge/almad.env as an environment file and passes those variables to almad. Recommended Kubernetes manifests pass the same keys through the Pod environment. almad reads environment variables. It does not parse /etc/almaforge/almad.env itself.

Example /etc/almaforge/almad.env for a cluster node

examples/configuration/quick-example/almad-cluster-node.env

# Environment file for a single-host cluster node.
# Runs the Auth Service and the Proxy Service together.
# Place under /etc/almaforge/almad.env.

ALMA_AUTH_SERVICE=true
ALMA_PROXY_SERVICE=true

# The node is also reachable as an SSH target.
ALMA_SSH_SERVICE=true

# Public endpoint that clients and agents connect to.
ALMA_LISTEN=0.0.0.0:443
ALMA_ADVERTISE=almaforge.example.com:443

# ACME issues a TLS certificate for ALMA_ADVERTISE on first start.
# Replace with ALMA_TLS_CERT / ALMA_TLS_KEY for an organization CA.
ALMA_ACME_ENABLED=true
ALMA_ACME_EMAIL=ops@example.com

# Resource drop-ins and local runtime state.
ALMA_CONFIG_DIR=/etc/almaforge/almad.d
ALMA_DATA_DIR=/var/lib/almaforge

Example /etc/almaforge/almad.env for an agent node

examples/configuration/quick-example/almad-agent-node.env

# Environment file for an agent node that joins an existing cluster.
# This example runs the SSH Service. Swap the role for
# ALMA_DATABASE_SERVICE, ALMA_APP_SERVICE, or ALMA_KUBERNETES_SERVICE
# for other agents.
# Place under /etc/almaforge/almad.env.

ALMA_SSH_SERVICE=true

# Cluster Proxy address that the agent dials.
ALMA_PROXY=almaforge.example.com:443

# Static-token join credentials issued by the cluster admin.
# Keep the secret in a file mode 0600 and reference it from a secret
# manager when possible.
ALMA_CLIENT_ID=ssh-node-01
ALMA_CLIENT_SECRET=replace-with-the-issued-secret

# CA pin from `alma status` on the cluster, used to verify the Proxy.
ALMA_CA_PIN=sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef

# Labels attached to the node for role discovery and RBAC matching.
ALMA_LABELS=env=prod,team=infra

# Local runtime state for the agent.
ALMA_DATA_DIR=/var/lib/almaforge

See Environment Configuration for the complete ALMA_* catalog and CLI Reference for startup flags.

Resource Configuration

Resource configuration describes cluster state, not the local process that loaded it. The same manifest model works for drop-in files and for live API-managed objects. It answers questions like:

• Which roles and access policy should apply?
• Which SSO connectors and join tokens should exist?
• Which storage backends should the cluster use?
• Which SSH servers, Kubernetes clusters, databases, applications, and APIs should be available?

Resource manifests use the same top-level fields as Kubernetes resources. For example, role-editor.yaml:

examples/configuration/infrastructure-as-code/role-editor.yaml

# Role that grants SSH login to production hosts labeled env=prod.
# Apply it through the API or place it under /etc/almaforge/almad.d/.
apiVersion: almaforge.com/v1
kind: Role
metadata:
  name: editor
spec:
  allow:
    serverLogins: # OS accounts this role may assume on matched SSH hosts.
      - ubuntu
    serverLabels: # Host labels this role may access.
      env: prod

Every resource has a type and a stable name. alma get role/editor refers to the Role named editor, no matter which path created it.

See Declarative Configuration for the full list of resource kinds and fields.

Drop-in files

almad reads resource manifests from /etc/almaforge/almad.d (default /etc/almaforge/almad.d) during process startup. The same path can be set with ALMA_CONFIG_DIR, or with --config-dir for one-off runs. Drop a manifest in, edit one in place, or rsync the whole directory. The Auth Service parses the files and applies them to the cluster.

This is a natural fit for infrastructure-as-code tools that already manage filesystem state: Ansible, Chef, Puppet, Nix, Salt, Kubernetes ConfigMaps mounted into the daemon Pod, GitOps renders from Kustomize or Helm, or a plain rsync from a config repository. Your existing workflow over a directory of files becomes your workflow over cluster state.

text

/etc/almaforge/almad.d/
|-- 01-storage-state.yaml
|-- 02-storage-audit.yaml
|-- 10-oidc-google.yaml
|-- 20-role-editor.yaml
`-- 30-token-ssh-node-01.yaml

Files are numerically ordered by convention. Multiple manifests can share one file, separated by ---.

Treat drop-in resources as file-owned. alma get role/editor shows the live object, but the file should remain the source of truth. To change a drop-in resource, edit the file:

Terminal

sudo vim /etc/almaforge/almad.d/20-role-editor.yamlalma get role/editor -o yaml

The Auth Service watches /etc/almaforge/almad.d and re-applies it as soon as the directory changes, so an edit takes effect without a restart. Adding a file, editing one, and replacing the whole directory all work the same way. A Kubernetes ConfigMap mounted at that path behaves the same, with one extra delay: the kubelet republishes a changed ConfigMap into the Pod on its own sync period, up to about a minute, before AlmaForge sees it. The daemon log records each applied resource as Reloaded <kind> "<name>" from <path>.

Some changes need a daemon restart or an explicit deletion:

• The cluster name and the storage backends. These are read before the cluster is up, so a running daemon cannot switch to a new one.
• Targets served by an agent, such as KubernetesTarget, DatabaseTarget, and AppTarget in an agent's own /etc/almaforge/almad.d. An agent decides which services it runs at startup.
• Removing a resource. Deleting a file leaves the object in the cluster, so removal is a deliberate alma delete.

If any manifest fails to parse during a reload, the directory update is rejected and already loaded resources stay in place. Check the daemon log before assuming that another file in the same update applied.

alma apply -f

Once the cluster is running, declare resources locally and apply them through the API:

Terminal

alma apply -f role-editor.yamlrole/editor has been updated

Use read and edit commands separately when you want to inspect, round-trip, or remove an API-owned resource:

Terminal

alma get role/editor -o yaml > role-editor.yamlalma edit role/editor          # opens $EDITOR on the live objectalma delete role/editorrole/editor has been deleted

alma sends the manifest to the Auth Service over mTLS. The Auth Service validates it and persists it to the configured state backend, one of etcd, PostgreSQL, DynamoDB, Firestore, or SQLite (single-node evaluation only). Other components (Proxy, agents, controllers) watch the backend and pick up the change without a restart. The resource is API-owned: no file on disk, no daemon restart.

apply -f is idempotent: it creates the resource if absent and updates it if present. Use this path for day-to-day operations and for CI/CD pipelines that wrap alma. See CLI Reference for the full operation set.

Kubernetes Operator

A controller that watches AlmaForge custom resources in a Kubernetes cluster and reconciles them against the AlmaForge API. Resource definitions are written as YAML CRDs and applied with kubectl apply -f, suiting GitOps workflows where cluster config lives next to application manifests and is rolled out with Argo CD or Flux.

note

The Kubernetes Operator is not available in this release.

Terraform provider

The Terraform provider manages roles, locks, SSO connectors, and approval connectors through a plan/apply workflow. It authenticates with your saved alma profile and keeps managed resources in Terraform state.

Methods compared

Method	Owner	Change requires	Best for
Drop-in file	File	None (live update)	Filesystem-driven IaC: Ansible, Chef, Nix, K8s ConfigMaps, GitOps renders
alma apply -f	API	None (live update)	Day-to-day ops and CI/CD pipelines
Kubernetes Operator (not available)	CRD	None (reconcile)	GitOps with Argo CD or Flux
Terraform provider	State	terraform apply	Terraform shops

Use drop-in files, alma apply -f, or Terraform according to which tool owns the resource. The Kubernetes Operator is not available in this release.

Repository layout

Keep the environment file, resource manifests, and secrets as separate inputs. For a first repository, start with the files needed to bring up SSO, one role, and one agent token:

text

infra/almaforge/
|-- README.md
|-- almad.env.example
|-- almad.d/
|   |-- 10-oidc-google.yaml
|   |-- 20-role-editor.yaml
|   `-- 30-token-ssh-node-01.yaml

Commit examples and rendered manifests. Keep real almad.env files, OAuth client secrets, static join secrets, TLS keys, kubeconfigs, and database credentials in your secret manager or deployment system.

Use the examples from this page as the first contents:

• oidc-google.yaml
• role-editor.yaml
• ssh-node-token.yaml

Validate that directory before rolling it to an Auth node:

Terminal

cd infra/almaforgealmad start --test --config-dir ./almad.dConfiguration OK

As the deployment grows, split storage, SSO, roles, tokens, and agent environment examples into their own files:

text

infra/almaforge/
|-- README.md
|-- almad.env.example
|-- almad.d/
|   |-- 01-storage-state.yaml
|   |-- 02-storage-audit.yaml
|   |-- 10-oidc-google.yaml
|   |-- 20-role-editor.yaml
|   `-- 30-token-ssh-node-01.yaml
`-- agents/
    |-- app-prod.env.example
    `-- ssh-prod.env.example

Ownership rules

Pick one owner for each resource name:

• Use drop-in files for bootstrap resources and filesystem-driven IaC.
• Use alma apply -f for live day-to-day changes and CI jobs.
• Do not keep the same resource name in two files.
• Do not manage the same resource through both a file and the API.

The Auth Service applies the resources it reads from /etc/almaforge/almad.d on startup and again whenever the files change. If an operator changes the same object through the API, the file version is written back over it the next time the file changes or the daemon restarts. Removing a file from /etc/almaforge/almad.d does not delete the already persisted resource in this release. To remove a file-owned resource, remove the file from the rollout and then delete the live object deliberately with alma delete.

Common failure behavior:

Situation	Result	Operator action
Bad YAML or an invalid resource in /etc/almaforge/almad.d	On a running daemon the directory reload is rejected and the loaded resources stay in place. On a fresh start almad start --test fails, and the new process exits before serving.	Fix the file. Saving it triggers another directory reload.
Two files define the same resource kind and name	The files no longer represent a clear source of truth.	Keep one manifest per resource name. Split by file for ownership, not for override layers.
A file-owned object is edited through the API	The edit is overwritten the next time the file changes or Auth restarts.	Edit the file, or move the resource to API ownership first.
A file is removed from /etc/almaforge/almad.d	The persisted resource is not deleted in this release.	Remove the file from rollout, then delete the live object deliberately.
A CI job applies a file-owned resource with alma apply -f	The resource now has two writers.	Pick one owner and remove the other path.

To move a resource from API ownership to file ownership, export it with alma get <kind>/<name> -o yaml, commit the manifest, and deploy the file. To move the other direction, remove the file from the rollout first so a later reload cannot write the old version back.

Rollout and rollback

Run the startup parser before replacing a production node:

Terminal

almad start --test --config-dir ./almad.dConfiguration OK

Run the check with the same ALMA_* environment that the service will use in production. --test validates startup input and exits. It does not apply resources to a running cluster.

For HA clusters, deploy the file set to one Auth node first. That node reloads on its own and writes the change to the shared backend, so watch its logs for Reloaded <kind> "<name>" from <path>, then verify the object through the API:

Terminal

alma get role/editor -o yaml

If the live object is wrong, roll forward with a corrected manifest or roll back by deploying the previous manifest to the same node. Either lands the same way, without a restart.

Secrets

Daemon secrets belong in the process environment or in files mounted by the process manager. For example, agents receive static join secrets through ALMA_CLIENT_SECRET, and the control plane receives storage, TLS, and connector secret paths through ALMA_* variables or mounted files.

Resource secrets should stay out of committed YAML. Connector fields such as spec.clientSecret can read from the Auth Service environment with ${env:NAME} or from a file on the Auth Service host with ${file:/path}. See Resource value expansion for the supported fields and resolution rules.

Topology Configuration

Every AlmaForge daemon is almad. You do not install a different binary for Auth, Proxy, SSH agents, Kubernetes agents, application agents, or database agents. You start the same binary with different service flags.

The service flags answer one question: what should this process do?

• Auth Service stores cluster state and issues certificates.
• Proxy Service is the public endpoint for users and agents.
• Resource services run near the infrastructure they expose: SSH servers, Kubernetes clusters, applications, or databases.

One process can run Auth and Proxy together on a single host, or you can split components across hosts for high availability. The HA topology is documented for Linux and Kubernetes. Either way, additional almad processes join the cluster as agents to expose specific infrastructure. They authenticate to the Proxy using a join token issued by the cluster.

A join token is a Token resource. Create one with the role the agent will run as. For example, ssh-node-token.yaml:

examples/configuration/infrastructure-as-code/ssh-node-token.yaml

# Static join token for an SSH Service agent named ssh-node-01.
# Store the unhashed client secret outside this file and pass it to the
# agent as ALMA_CLIENT_SECRET.
apiVersion: almaforge.com/v1
kind: Token
metadata:
  name: ssh-node-01 # Agent ALMA_CLIENT_ID value.
spec:
  roles:
    - system:ssh # Service role granted to the joining SSH agent.
  static:
    # SHA-256 of the ALMA_CLIENT_SECRET value.
    secret: 9e7a4b3c6f2d1e8a5b0c4d9e2f1a3b6c7d8e9f0a1b2c3d4e5f6a7b8c9d0e1f2a

For a static join, the agent presents its client ID and client secret to the Proxy and verifies the Proxy's TLS certificate using the host's trust store. After joining, it holds a certificate that rotates on its own:

env

ALMA_SSH_SERVICE=true
ALMA_PROXY=almaforge.example.com:443
ALMA_CLIENT_ID=ssh-node-01
ALMA_CLIENT_SECRET=my-static-secret
ALMA_JOIN_METHOD=static

Use ALMA_KUBERNETES_SERVICE, ALMA_APP_SERVICE, or ALMA_DATABASE_SERVICE instead of ALMA_SSH_SERVICE for Kubernetes, application, or database agents.

Static joins use the secret stored with the token. Cloud and CI/CD joins replace the secret with platform identity, including AWS (aws, ec2), Azure, GCP, GitHub, GitLab, CircleCI, Spacelift, and Kubernetes service-account joins.

Use alma apply -f token.yaml to issue tokens against a running cluster, or write them as drop-in files. See CLI Reference for the full role flag list.

Network Configuration

Networking and TLS are daemon configuration. They control how the Proxy listens, what address clients and agents remember, and how TLS is terminated.

In production, the Proxy sits behind a load balancer. The Proxy binds a private address, presents an organization-issued certificate, and the load balancer hostname is what clients see:

env

ALMA_PROXY_SERVICE=true
ALMA_LISTEN=10.0.0.10:443
ALMA_ADVERTISE=almaforge.example.com:443
ALMA_TLS_CERT=/etc/almaforge/tls.crt
ALMA_TLS_KEY=/etc/almaforge/tls.key

For deployments without a load balancer, the Proxy can bind a public interface directly and obtain a Let's Encrypt certificate through ACME, provided TCP/443 is reachable from Let's Encrypt's validation servers and the Proxy can reach the ACME API:

env

ALMA_PROXY_SERVICE=true
ALMA_LISTEN=0.0.0.0:443
ALMA_ADVERTISE=almaforge.example.com:443
ALMA_ACME_ENABLED=true
ALMA_ACME_EMAIL=ops@example.com

Use Networking Guide for listener layout, load balancers, TLS files, ACME, wildcard DNS, and proxy protocol settings.

Storage Configuration

Storage can be set at daemon startup with environment variables, or declared as StorageConfig manifests. The env-var form is one backend per storage job and is the shortest path for a fresh cluster:

env

ALMA_STATE_URI=sqlite:///var/lib/almaforge/backend
ALMA_AUDIT_URI=file:///var/lib/almaforge/log
ALMA_RECORDINGS_URI=file:///var/lib/almaforge/records

Use StorageConfig manifests when you need file ownership or fan-out audit sinks. Declare them as drop-in files and restart the Auth Service to change storage settings. Storage configuration is read-only through the API. See storage configuration.

The three storage jobs are separate:

• Cluster state stores resources and certificate authorities.
• Audit events store the security event stream.
• Session recordings store larger recording artifacts.

Each job is its own StorageConfig instance (named state, audit, and recordings). They may use different backends:

text

/etc/almaforge/almad.d/
|-- 01-state-storage.yaml
|-- 02-audit-storage.yaml
`-- 03-recording-storage.yaml

Use Storage Guide for backend selection and the StorageConfig reference for fields.

Auth Configuration

Authentication is resource configuration. AlmaForge authenticates users through OIDC SSO. OIDCConnector defines the IdP integration: issuer, OAuth client settings, scopes, and claim-to-role mapping.

Declare OIDCConnector as a drop-in file or through alma apply. On a fresh cluster no admin session exists yet to apply against, so the initial connector is typically a drop-in. For example, oidc-google.yaml:

examples/configuration/infrastructure-as-code/oidc-google.yaml

# OIDC connector for Google Workspace.
# Inject GOOGLE_OIDC_CLIENT_SECRET into the Auth Service environment.
apiVersion: almaforge.com/v1
kind: OIDCConnector
metadata:
  name: google
spec:
  issuerURL: https://accounts.google.com # OIDC discovery issuer URL.
  clientID: 1234567890-abcdef.apps.googleusercontent.com # OAuth client ID from Google.
  clientSecret: ${env:GOOGLE_OIDC_CLIENT_SECRET} # OAuth client secret from Google.
  claimsToRoles: # At least one mapping is required, or nobody can log in.
    - claim: hd
      value: example.com
      roles: [access]

The connector handles authentication. Roles handle authorization. The claim-to-role mapping is the bridge between the IdP and AlmaForge access policy.

Use the OIDC guide for provider setup and the OIDCConnector reference for fields.

Policy Configuration

Role resources control who can do what. Each role grants access through allow rules and subtracts access through deny rules. Users gain roles from SSO claim mapping in OIDCConnector resources.

A typical role grants login on labeled servers. For example, role-editor.yaml:

examples/configuration/infrastructure-as-code/role-editor.yaml

# Role that grants SSH login to production hosts labeled env=prod.
# Apply it through the API or place it under /etc/almaforge/almad.d/.
apiVersion: almaforge.com/v1
kind: Role
metadata:
  name: editor
spec:
  allow:
    serverLogins: # OS accounts this role may assume on matched SSH hosts.
      - ubuntu
    serverLabels: # Host labels this role may access.
      env: prod

Label matchers (serverLabels, databaseLabels, appLabels, kubernetesLabels) decide which targets the role can reach. Login fields (serverLogins, kubernetesGroups, databaseUsers) decide which identity the session assumes on the target.

See RBAC for the access-policy guide.

Target Configuration

Each kind of protected resource has its own manifest. An agent running the matching service exposes the target through the Proxy. For example, app-grafana.yaml:

examples/configuration/infrastructure-as-code/app-grafana.yaml

# AppTarget for an internal Grafana deployment.
# The Application Service reaches spec.uri and users open spec.fqdn.
apiVersion: almaforge.com/v1
kind: AppTarget
metadata:
  name: grafana
  labels: # Labels matched by Role.appLabels.
    env: prod
    team: sre
spec:
  uri: https://grafana.internal:3000 # Internal URL reachable from the agent.
  fqdn: grafana.almaforge.example.com # Public hostname covered by proxy TLS.

The shapes by kind:

• AppTarget for HTTP applications. uri is the internal address. fqdn is the public hostname users open.
• DatabaseTarget for databases. Engine, host, port, and optional client certificate.
• KubernetesTarget for Kubernetes clusters. Kubeconfig or service-account credentials.
• SSH servers usually self-register through a running SSH agent rather than being declared individually.

Target manifests can be a drop-in either on the control plane host or on the agent host that exposes the target. Use agent-local files when each agent owns its own local target inventory. Use control-plane files or alma apply -f when the target inventory is central and agents should select work by label.

For central target inventory, set ALMA_MATCH_LABELS on each resource agent. An Application Service agent with ALMA_MATCH_LABELS=env=prod serves AppTarget resources whose metadata.labels match env=prod. Multiple agents can use the same matcher when each can reach the same upstream target.

The API object proves that the target is declared. It does not prove that an agent can reach the upstream service. For target rollouts, verify both the resource with alma get <kind>/<name> -o yaml and the serving agent logs.

See Declarative Configuration for the schemas and the Server, Kubernetes, Application, and Database access guides for setup.

Further Reading

• Planning for requirements and deployment choices.
• Installation for packages, containers, and Kubernetes manifests.
• High Availability Guide for the architecture overview.
• Cluster on Linux for the HA Linux walkthrough.
• Cluster on Kubernetes for the HA Kubernetes walkthrough.
• Environment Configuration for every supported ALMA_* variable.
• Declarative Configuration for resource schemas and examples.
• Storage Guide for storage backend selection.
• Networking Guide for listeners, advertise addresses, load balancers, TLS, and firewall rules.