unkin/terraform-incus
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Dual-write ENC data into encapi alongside Cobbler #39

Closed
unkinben wants to merge 2 commits from benvin/encapi-seed into master
pull from: benvin/encapi-seed
merge into: unkin:master
Conversation 1 Commits 2 Files Changed 13
+249 -1
unkinben commented 2026-07-24 22:47:34 +10:00
Owner
Copy Link
Copy Source

Why

Seed encapi so the k8s puppetserver ENC can cut over from Cobbler. encapi's DB is currently empty; nothing can point Puppet at it until the roles/statuses/nodes exist. This change makes terraform-incus dual-write the ENC data into encapi alongside the existing Cobbler/PuppetCA/PuppetDB resources. No Cobbler removal happens here — those resources are left fully intact until a later cutover PR.

Changes

  • Adds a config/encapi leaf backed by a new modules/encapi that seeds encapi from YAML:
    • statuses.yaml — 3 statuses (production, develop, testing). All 143 nodes currently report catalog_environment=develop; production is seeded because the instance module pins VMs there (matching puppetca_certificate), testing is the implicit/transient one.
    • roles.yaml — 51 distinct role classes: every cobbler_mgmt_classes[0] across config/instances/*/config.yaml (48) plus the 3 roles run by the physical prodnxsr* nodes (roles::infra::incus::node, roles::infra::k8s::compute, roles::infra::k8s::control).
    • nodes.yaml — encapi_node assignments for the 13 prodnxsr* bare-metal nodes (certname → role from the PuppetDB enc_role fact, environment: develop to match their current catalog_environment). These hosts are not managed by the instance module, so they are seeded explicitly here.
  • Extends modules/instance to also create an encapi_node for every VM: certname <name>.<cobbler_domain>, role var.cobbler_mgmt_classes[0], environment production (matches what cobbler_system/puppetca_certificate use today; overridable via new encapi_environment var). cobbler_system / puppetca_certificate / puppetdb_node are unchanged.
  • Wires the encapi provider (git.unkin.net/unkin/encapi v0.1.0) into config/root.hcl's generated required_providers and both module providers.tf files.
  • Plumbs ENCAPI_WRITE_TOKEN into terraform via Makefile vault_env (the provider reads the bearer token straight from that env var).

Apply order (important — FK ordering)

encapi nodes FK-require their role and status to already exist. Apply config/encapi before the instance leaves. Within a single terragrunt run --all apply (parallelism 2) this is not automatically ordered — the instance leaves have no terragrunt dependency on the encapi leaf (adding 130 of them is noisy and the encapi leaf has no outputs the instances consume). Recommended: run terragrunt apply in config/encapi first, or add a one-off ordering for the initial seed. The modules/encapi encapi_node resources already depends_on the roles/statuses within that leaf.

Prerequisites (orchestrator / follow-up PRs)

  1. Vault policy (terraform-vault): CI authenticates with the TERRAFORM_INCUS_VAULT_ROLEID approle, whose policy today reads kv/service/puppet/* and kv/data/service/terraform/incus. It does not grant read on kv/kubernetes/namespace/encapi/default/environment (where ENCAPI_WRITE_TOKEN lives). A terraform-vault change is required to add read on kv/data/kubernetes/namespace/encapi/default/environment (and metadata) to that approle's policy, or the make apply step will fail resolving the token. Not fixed in this PR.
  2. Provider registry resolution: the provider source git.unkin.net/unkin/encapi resolves via the ArtifactAPI Terraform registry (artifactapi.k8s.syd1.au.unkin.net, /terraform/v1/providers/), not via git.unkin.net directly. CI's terraform needs a CLI config (host "git.unkin.net" { services = { "providers.v1" = "https://artifactapi.k8s.syd1.au.unkin.net/terraform/v1/providers/" } }) or a network mirror so terraform init can fetch it. Local terraform init against git.unkin.net fails with "does not offer a Terraform provider registry" without this.

Validation done

  • terraform fmt clean on both modules; terragrunt hcl fmt clean on the leaf/root.
  • modules/encapi terraform init -backend=false + terraform validate = Success (against a locally built v0.1.0 provider via filesystem mirror). This exercises the same encapi_status/encapi_role/encapi_node schemas and provider block the instance module uses.
  • No plan/apply run — no Consul state writes, no encapi writes.
  • modules/instance cannot validate standalone (its ../../../../../../modules/* relative sources only resolve from a 6-deep leaf) — pre-existing, unaffected by this change.
## Why Seed encapi so the k8s puppetserver ENC can cut over from Cobbler. encapi's DB is currently empty; nothing can point Puppet at it until the roles/statuses/nodes exist. This change makes terraform-incus **dual-write** the ENC data into encapi alongside the existing Cobbler/PuppetCA/PuppetDB resources. **No Cobbler removal happens here** — those resources are left fully intact until a later cutover PR. ## Changes - Adds a `config/encapi` leaf backed by a new `modules/encapi` that seeds encapi from YAML: - `statuses.yaml` — 3 statuses (`production`, `develop`, `testing`). All 143 nodes currently report `catalog_environment=develop`; `production` is seeded because the instance module pins VMs there (matching `puppetca_certificate`), `testing` is the implicit/transient one. - `roles.yaml` — 51 distinct role classes: every `cobbler_mgmt_classes[0]` across `config/instances/*/config.yaml` (48) plus the 3 roles run by the physical `prodnxsr*` nodes (`roles::infra::incus::node`, `roles::infra::k8s::compute`, `roles::infra::k8s::control`). - `nodes.yaml` — encapi_node assignments for the 13 `prodnxsr*` bare-metal nodes (certname → role from the PuppetDB `enc_role` fact, `environment: develop` to match their current `catalog_environment`). These hosts are not managed by the instance module, so they are seeded explicitly here. - Extends `modules/instance` to **also** create an `encapi_node` for every VM: certname `<name>.<cobbler_domain>`, role `var.cobbler_mgmt_classes[0]`, environment `production` (matches what `cobbler_system`/`puppetca_certificate` use today; overridable via new `encapi_environment` var). `cobbler_system` / `puppetca_certificate` / `puppetdb_node` are unchanged. - Wires the `encapi` provider (`git.unkin.net/unkin/encapi` v0.1.0) into `config/root.hcl`'s generated `required_providers` and both module `providers.tf` files. - Plumbs `ENCAPI_WRITE_TOKEN` into terraform via `Makefile` `vault_env` (the provider reads the bearer token straight from that env var). ## Apply order (important — FK ordering) encapi nodes FK-require their role and status to already exist. **Apply `config/encapi` before the instance leaves.** Within a single `terragrunt run --all apply` (parallelism 2) this is not automatically ordered — the instance leaves have no terragrunt `dependency` on the encapi leaf (adding 130 of them is noisy and the encapi leaf has no outputs the instances consume). Recommended: run `terragrunt apply` in `config/encapi` first, or add a one-off ordering for the initial seed. The `modules/encapi` `encapi_node` resources already `depends_on` the roles/statuses within that leaf. ## Prerequisites (orchestrator / follow-up PRs) 1. **Vault policy (terraform-vault):** CI authenticates with the `TERRAFORM_INCUS_VAULT_ROLEID` approle, whose policy today reads `kv/service/puppet/*` and `kv/data/service/terraform/incus`. It does **not** grant read on `kv/kubernetes/namespace/encapi/default/environment` (where `ENCAPI_WRITE_TOKEN` lives). A terraform-vault change is required to add `read` on `kv/data/kubernetes/namespace/encapi/default/environment` (and metadata) to that approle's policy, or the `make apply` step will fail resolving the token. Not fixed in this PR. 2. **Provider registry resolution:** the provider source `git.unkin.net/unkin/encapi` resolves via the ArtifactAPI Terraform registry (`artifactapi.k8s.syd1.au.unkin.net`, `/terraform/v1/providers/`), not via `git.unkin.net` directly. CI's terraform needs a CLI config (`host "git.unkin.net" { services = { "providers.v1" = "https://artifactapi.k8s.syd1.au.unkin.net/terraform/v1/providers/" } }`) or a network mirror so `terraform init` can fetch it. Local `terraform init` against `git.unkin.net` fails with "does not offer a Terraform provider registry" without this. ## Validation done - `terraform fmt` clean on both modules; `terragrunt hcl fmt` clean on the leaf/root. - `modules/encapi` `terraform init -backend=false` + `terraform validate` = **Success** (against a locally built v0.1.0 provider via filesystem mirror). This exercises the same `encapi_status`/`encapi_role`/`encapi_node` schemas and provider block the instance module uses. - No plan/apply run — no Consul state writes, no encapi writes. - `modules/instance` cannot `validate` standalone (its `../../../../../../modules/*` relative sources only resolve from a 6-deep leaf) — pre-existing, unaffected by this change.
unkinben added 1 commit 2026-07-24 22:47:34 +10:00
Dual-write ENC data into encapi alongside Cobbler
Build / build (pull_request) Failing after 11m25s
Details
4b9a6de83b 
Seed encapi (statuses, roles, prodnxsr node assignments) and make the
instance module also create an encapi_node for every VM, so the k8s
puppetserver ENC can cut over from Cobbler. Cobbler/puppetca/puppetdb
resources are left untouched.

- Add config/encapi leaf + modules/encapi driving statuses/roles/nodes
  from YAML (3 statuses, 51 roles, 13 prodnxsr physical nodes).
- Add encapi_node to modules/instance (certname, role, environment).
- Wire encapi provider into root.hcl required_providers and the module
  providers; plumb ENCAPI_WRITE_TOKEN via Makefile vault_env.
unkinben added 1 commit 2026-07-24 22:53:11 +10:00
Use artifactapi registry source for encapi provider
Build / build (pull_request) Failing after 15m4s
Details
9aa43e17e9
unkinben commented 2026-07-24 23:01:49 +10:00
Author
Owner
Copy Link
Copy Source

Closing: superseded by the new dedicated terragrunt-enc repo (all encapi ENC data managed there; no changes to terraform-incus). Replacement PRs incoming.

Closing: superseded by the new dedicated `terragrunt-enc` repo (all encapi ENC data managed there; no changes to terraform-incus). Replacement PRs incoming.
unkinben closed this pull request 2026-07-24 23:01:49 +10:00
Some required checks failed
Build / build (pull_request) Failing after 15m4s
Required
Details

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
benvin (BenVincent) unkinben (Ben Vincent)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: unkin/terraform-incus#39
Delete Branch "benvin/encapi-seed"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?