unkin-agent
42307f285f
encapi's schema was a cumulative IF NOT EXISTS blob re-executed inline on every start: it grows forever, records nothing, and cannot express a change that is not a fresh CREATE. golib owns that mechanism now, so encapi keeps the SQL and drops the runner. - Move the DDL verbatim into migrations/0001_init.sql, embedded via migrations.FS. It stays IF NOT EXISTS-guarded, so the first start against the live database re-runs it as a no-op and only lands the schema_migrations row. - Build the pool with pg.NewMigrated under the advisory lock named encapi-migrations, and delete the inline migrate(). database.New now takes a context and a logger; main.go hands it the signal context so a start blocked on the migration lock still dies on SIGTERM. - Render the DSN with pg.DSN. The env var contract is untouched — the fields are still resolved by internal/config, because encapi defaults DBUSER and DBNAME to "encapi" where pg.DSNFromEnv treats both as required. - Guard the set: embedded files must match migrations/, every CREATE must be idempotent, the derived lock key is pinned, and a container test proves the adoption path over a database that predates schema_migrations. - Plumb GOPRIVATE=git.unkin.net for the first cross-repo Go dependency: exported by the Makefile, set in the Dockerfile and the woodpecker Go steps, documented in the README.
137 lines
5.9 KiB
Markdown
137 lines
5.9 KiB
Markdown
# encapi
|
|
|
|
A Postgres-backed **External Node Classifier (ENC)** for Puppet, written in Go.
|
|
It replaces Cobbler as the source of truth for *which role a host runs* while
|
|
keeping Puppet's node-classification contract byte-compatible.
|
|
|
|
## Components
|
|
|
|
| Binary | What it is |
|
|
|-----------------|-----------------------------------------------------------------------|
|
|
| `encapi` | HTTP server: serves ENC documents + a read/write API. Postgres-backed. |
|
|
| `encapi-cli` | CLI over the API. `encapi-cli classify <host>` is the Puppet ENC. |
|
|
| `encapi-enc` | Thin wrapper (shipped in the RPM) for Puppet's `external_nodes`. |
|
|
|
|
The Terraform provider lives in a sibling repo:
|
|
[`terraform-provider-encapi`](https://git.unkin.net/unkin/terraform-provider-encapi).
|
|
|
|
## Data model
|
|
|
|
- **status** — a Puppet environment (Cobbler's "status": testing, production…).
|
|
- **role** — a class assignment target (`roles::infra::storage::vault`) with
|
|
inheritable `default_params`.
|
|
- **node** — a host (certname) pinned to one role + one environment, with
|
|
optional per-node `params` (which win over the role defaults).
|
|
|
|
Foreign keys guarantee a node can only reference a role/status that exists, and
|
|
a role/status in use cannot be deleted.
|
|
|
|
## Schema migrations
|
|
|
|
The SQL lives in [`migrations/`](migrations), is embedded in the `encapi`
|
|
binary, and is applied at startup before the server listens — there is no
|
|
mirrored copy in the deployment to drift out of sync.
|
|
|
|
The runner is [`golib/pg`](https://git.unkin.net/unkin/golib)'s
|
|
`pg.NewMigrated`: encapi owns the SQL, the shared library owns the mechanics.
|
|
Each start takes `pg_advisory_lock` on a key derived from the lock name
|
|
`encapi-migrations`, creates `schema_migrations` (`version`, `applied_at`) if
|
|
missing, and applies every embedded file whose filename is not yet recorded — in
|
|
lexical (version) order, each file's SQL and its tracking row in one transaction
|
|
— then unlocks. Replicas starting together queue on the lock and then find
|
|
nothing to do.
|
|
|
|
A file the live database already satisfies but `schema_migrations` does not
|
|
record is re-run, which is how the pre-migration schema is adopted: migrations
|
|
are `IF NOT EXISTS`-guarded, so the re-run is a no-op that only lands the
|
|
tracking row. Add schema changes as a new numbered file; never edit an applied
|
|
one.
|
|
|
|
## ENC output
|
|
|
|
`encapi` renders two shapes from the same data:
|
|
|
|
- **`GET /api/v1/nodes/{certname}/enc`** — the reshaped document Puppet's
|
|
`exec` terminus consumes: `classes` as a list, `environment` dropped when it
|
|
is `testing`, and `parameters` carrying `enc_role` (list) + `enc_env`. This is
|
|
what `encapi-cli classify` prints.
|
|
- **`GET /cblr/svc/op/puppet/hostname/{certname}`** — the cobbler-wire form
|
|
(`classes` as a map, `environment` always present), so the legacy
|
|
`enc_direct_facts.rb` fact can be repointed with only a URL change.
|
|
|
|
See [`docs/cutover.md`](docs/cutover.md) for the migration from Cobbler.
|
|
|
|
## API
|
|
|
|
Reads are open. Writes require `Authorization: Bearer $ENCAPI_WRITE_TOKEN`
|
|
(set the token in the server's environment; manage it in Vault).
|
|
|
|
```
|
|
GET /healthz
|
|
GET /api/v1/nodes list nodes
|
|
GET /api/v1/nodes/{certname} get node
|
|
PUT /api/v1/nodes/{certname} upsert node (token)
|
|
DELETE /api/v1/nodes/{certname} delete node (token)
|
|
GET /api/v1/nodes/{certname}/enc reshaped ENC (YAML)
|
|
GET /cblr/svc/op/puppet/hostname/{h} cobbler-wire ENC (YAML)
|
|
GET /api/v1/roles ... /roles/{name} (PUT/DELETE token)
|
|
GET /api/v1/statuses ... /statuses/{name} (PUT/DELETE token)
|
|
```
|
|
|
|
## Configuration (server env)
|
|
|
|
| Var | Default | Purpose |
|
|
|------------------------|---------------|------------------------------------------|
|
|
| `LISTEN_ADDR` | `:8000` | listen address |
|
|
| `DBHOST/DBPORT/DBUSER/DBPASS/DBNAME/DBSSL` | localhost/5432/encapi/encapi/encapi/disable | Postgres |
|
|
| `ENCAPI_WRITE_TOKEN` | *(unset)* | bearer token for writes; unset = read-only |
|
|
| `ENCAPI_DISTRO_API_URL`| *(unset)* | optional kickstart/distro API for provisioning params |
|
|
|
|
## CLI
|
|
|
|
```bash
|
|
export ENCAPI_URL=https://encapi.k8s.syd1.au.unkin.net
|
|
export ENCAPI_WRITE_TOKEN=… # only needed for writes
|
|
|
|
encapi-cli status set production
|
|
encapi-cli role set roles::infra::storage::minio --param 'replicas=4' --param 'epel="9"'
|
|
encapi-cli node set ausyd1nxvm2100.main.unkin.net --role roles::infra::storage::minio --env production
|
|
encapi-cli classify ausyd1nxvm2100.main.unkin.net
|
|
|
|
# one-shot migration from the live Cobbler estate:
|
|
encapi-cli import-cobbler --dry-run
|
|
encapi-cli import-cobbler
|
|
```
|
|
|
|
## Local development
|
|
|
|
```bash
|
|
docker compose up -d # postgres + encapi
|
|
make test # full suite (Postgres via testcontainers)
|
|
make test-short # skip container-backed DB tests
|
|
```
|
|
|
|
### `GOPRIVATE`
|
|
|
|
encapi depends on `git.unkin.net/unkin/golib`, which is served by Gitea and is
|
|
unknown to `proxy.golang.org` / `sum.golang.org`. Module resolution therefore
|
|
needs:
|
|
|
|
```
|
|
export GOPRIVATE=git.unkin.net
|
|
```
|
|
|
|
The `Makefile` exports it for every target, and the `Dockerfile` and the
|
|
woodpecker Go steps set it themselves, so `make build|test|lint` and CI work on
|
|
a clean checkout. Only bare `go` commands run outside `make` need it in your
|
|
shell — set it there (or in your shell profile) rather than with `go env -w`,
|
|
which is machine state this repo cannot carry.
|
|
|
|
## Releases
|
|
|
|
- **`encapi` server image** — tagging `vX.Y.Z` builds and pushes
|
|
`artifactapi.k8s.syd1.au.unkin.net/docker-internal/encapi:{tag,latest}` (distroless).
|
|
- **`encapi-cli` RPM** — the same tag builds an RPM (nfpm) and publishes it to
|
|
the ArtifactAPI `rpm-internal` repo. Installs `encapi-cli`, the `encapi-enc`
|
|
Puppet wrapper, and `/etc/encapi/enc.conf`.
|