unkin-agent 42307f285f
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/build Pipeline was successful
Adopt golib/pg for migrations and pool construction
encapi's schema was a cumulative IF NOT EXISTS blob re-executed inline on every
start: it grows forever, records nothing, and cannot express a change that is
not a fresh CREATE. golib owns that mechanism now, so encapi keeps the SQL and
drops the runner.

- Move the DDL verbatim into migrations/0001_init.sql, embedded via
  migrations.FS. It stays IF NOT EXISTS-guarded, so the first start against the
  live database re-runs it as a no-op and only lands the schema_migrations row.
- Build the pool with pg.NewMigrated under the advisory lock named
  encapi-migrations, and delete the inline migrate(). database.New now takes a
  context and a logger; main.go hands it the signal context so a start blocked
  on the migration lock still dies on SIGTERM.
- Render the DSN with pg.DSN. The env var contract is untouched — the fields are
  still resolved by internal/config, because encapi defaults DBUSER and DBNAME
  to "encapi" where pg.DSNFromEnv treats both as required.
- Guard the set: embedded files must match migrations/, every CREATE must be
  idempotent, the derived lock key is pinned, and a container test proves the
  adoption path over a database that predates schema_migrations.
- Plumb GOPRIVATE=git.unkin.net for the first cross-repo Go dependency:
  exported by the Makefile, set in the Dockerfile and the woodpecker Go steps,
  documented in the README.
2026-09-02 00:17:04 +10:00

encapi

A Postgres-backed External Node Classifier (ENC) for Puppet, written in Go. It replaces Cobbler as the source of truth for which role a host runs while keeping Puppet's node-classification contract byte-compatible.

Components

Binary What it is
encapi HTTP server: serves ENC documents + a read/write API. Postgres-backed.
encapi-cli CLI over the API. encapi-cli classify <host> is the Puppet ENC.
encapi-enc Thin wrapper (shipped in the RPM) for Puppet's external_nodes.

The Terraform provider lives in a sibling repo: terraform-provider-encapi.

Data model

  • status — a Puppet environment (Cobbler's "status": testing, production…).
  • role — a class assignment target (roles::infra::storage::vault) with inheritable default_params.
  • node — a host (certname) pinned to one role + one environment, with optional per-node params (which win over the role defaults).

Foreign keys guarantee a node can only reference a role/status that exists, and a role/status in use cannot be deleted.

Schema migrations

The SQL lives in migrations/, is embedded in the encapi binary, and is applied at startup before the server listens — there is no mirrored copy in the deployment to drift out of sync.

The runner is golib/pg's pg.NewMigrated: encapi owns the SQL, the shared library owns the mechanics. Each start takes pg_advisory_lock on a key derived from the lock name encapi-migrations, creates schema_migrations (version, applied_at) if missing, and applies every embedded file whose filename is not yet recorded — in lexical (version) order, each file's SQL and its tracking row in one transaction — then unlocks. Replicas starting together queue on the lock and then find nothing to do.

A file the live database already satisfies but schema_migrations does not record is re-run, which is how the pre-migration schema is adopted: migrations are IF NOT EXISTS-guarded, so the re-run is a no-op that only lands the tracking row. Add schema changes as a new numbered file; never edit an applied one.

ENC output

encapi renders two shapes from the same data:

  • GET /api/v1/nodes/{certname}/enc — the reshaped document Puppet's exec terminus consumes: classes as a list, environment dropped when it is testing, and parameters carrying enc_role (list) + enc_env. This is what encapi-cli classify prints.
  • GET /cblr/svc/op/puppet/hostname/{certname} — the cobbler-wire form (classes as a map, environment always present), so the legacy enc_direct_facts.rb fact can be repointed with only a URL change.

See docs/cutover.md for the migration from Cobbler.

API

Reads are open. Writes require Authorization: Bearer $ENCAPI_WRITE_TOKEN (set the token in the server's environment; manage it in Vault).

GET    /healthz
GET    /api/v1/nodes                      list nodes
GET    /api/v1/nodes/{certname}           get node
PUT    /api/v1/nodes/{certname}           upsert node        (token)
DELETE /api/v1/nodes/{certname}           delete node        (token)
GET    /api/v1/nodes/{certname}/enc       reshaped ENC (YAML)
GET    /cblr/svc/op/puppet/hostname/{h}   cobbler-wire ENC (YAML)
GET    /api/v1/roles       ... /roles/{name}      (PUT/DELETE token)
GET    /api/v1/statuses    ... /statuses/{name}   (PUT/DELETE token)

Configuration (server env)

Var Default Purpose
LISTEN_ADDR :8000 listen address
DBHOST/DBPORT/DBUSER/DBPASS/DBNAME/DBSSL localhost/5432/encapi/encapi/encapi/disable Postgres
ENCAPI_WRITE_TOKEN (unset) bearer token for writes; unset = read-only
ENCAPI_DISTRO_API_URL (unset) optional kickstart/distro API for provisioning params

CLI

export ENCAPI_URL=https://encapi.k8s.syd1.au.unkin.net
export ENCAPI_WRITE_TOKEN=# only needed for writes

encapi-cli status set production
encapi-cli role set roles::infra::storage::minio --param 'replicas=4' --param 'epel="9"'
encapi-cli node set ausyd1nxvm2100.main.unkin.net --role roles::infra::storage::minio --env production
encapi-cli classify ausyd1nxvm2100.main.unkin.net

# one-shot migration from the live Cobbler estate:
encapi-cli import-cobbler --dry-run
encapi-cli import-cobbler

Local development

docker compose up -d          # postgres + encapi
make test                     # full suite (Postgres via testcontainers)
make test-short               # skip container-backed DB tests

GOPRIVATE

encapi depends on git.unkin.net/unkin/golib, which is served by Gitea and is unknown to proxy.golang.org / sum.golang.org. Module resolution therefore needs:

export GOPRIVATE=git.unkin.net

The Makefile exports it for every target, and the Dockerfile and the woodpecker Go steps set it themselves, so make build|test|lint and CI work on a clean checkout. Only bare go commands run outside make need it in your shell — set it there (or in your shell profile) rather than with go env -w, which is machine state this repo cannot carry.

Releases

  • encapi server image — tagging vX.Y.Z builds and pushes artifactapi.k8s.syd1.au.unkin.net/docker-internal/encapi:{tag,latest} (distroless).
  • encapi-cli RPM — the same tag builds an RPM (nfpm) and publishes it to the ArtifactAPI rpm-internal repo. Installs encapi-cli, the encapi-enc Puppet wrapper, and /etc/encapi/enc.conf.
S
Description
Postgres-backed External Node Classifier (ENC) for Puppet, replacing Cobbler. Go API + encapi-cli.
Readme 111 KiB
Languages
Go 94.6%
Makefile 3%
Shell 1.8%
Dockerfile 0.6%