unkin/terraform-infra
4
0
Fork 0
Code Issues 5 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

Initial terraform-infra: NetBox IPAM + networks + devices + Kea DHCP #5

Open
unkinben wants to merge 4 commits from benvin/initial-infra into main
pull from: benvin/initial-infra
merge into: unkin:main
Conversation 1 Commits 4 Files Changed 29
+860 -1
unkinben commented 2026-08-05 19:42:37 +10:00
Owner
Copy Link
Copy Source

Initial content for terraform-infra (renamed from terraform-ipam), bootstrapping YAML-driven infrastructure state for NetBox + Kea.

Schema

  • Subnets (config/au/syd1/subnets/*.yaml): one file per subnet; renders a NetBox prefix and a matching Kea DHCP scope (pools, reservations) from the same definition.
  • Networks (config/au/syd1/networks/*.yaml): logical network definitions carrying gateway/dns/search (capture only for now — consumer tracked in #3).
  • Devices (config/au/syd1/devices/*.yaml): per-host intent — site, role, model_hint, networks, bootstrap_mac. Six hosts (prodnxsr0014-0019) seeded; the 13 existing hosts are tracked in #2.
  • Managed IPs (config/au/syd1/managed-ips.yaml): static address assignments.
  • Module modules/infra/: NetBox (sites, prefixes, devices, interfaces, IPs) + Kea provider wiring; Terragrunt env under environments/au/syd1/.
  • CI .woodpecker/: plan (PR), apply (main), pre-commit.
  • Backfill ci/puppetdb_backfill.py: reality import from PuppetDB (in-cluster only — automation tracked in #1).

Discovery background

Follow-ups filed

  • #1 Automate PuppetDB reality backfill into NetBox
  • #2 Seed intent device YAML for existing hosts prodnxsr0001-0013
  • #3 Consume network gateway/dns/search into NetBox/provisioning
  • #4 Set device primary IP in NetBox

Known plan blockers (expected CI failures until resolved)

  • terraform-vault #111 (consul role/ACL/kv policy) not yet merged/applied
  • Vault KV kv/service/terraform/infra (netbox_token, kea_token) not seeded
  • NetBox + KeaAPI not yet deployed

https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT

Seeding required before plan/apply can pass

Even with the vault child-token fix, the plan reads two provider tokens from Vault KV v2 kv/service/terraform/infra. Only a human can seed real tokens today (the NetBox secrets engine has not landed):

vault kv put kv/service/terraform/infra \
  netbox_token=<NETBOX_API_TOKEN> \
  kea_token=<KEA_API_TOKEN>

Required keys: netbox_token, kea_token. The read + metadata-read policy is added in terraform-vault PR #111 (blocked on the unrelated litellm plan breakage, terraform-vault#112). Plan also needs NetBox + KeaAPI deployed and the NetBox site syd1 to exist.

Initial content for terraform-infra (renamed from terraform-ipam), bootstrapping YAML-driven infrastructure state for NetBox + Kea. ## Schema - **Subnets** (`config/au/syd1/subnets/*.yaml`): one file per subnet; renders a NetBox prefix and a matching Kea DHCP scope (pools, reservations) from the same definition. - **Networks** (`config/au/syd1/networks/*.yaml`): logical network definitions carrying gateway/dns/search (capture only for now — consumer tracked in #3). - **Devices** (`config/au/syd1/devices/*.yaml`): per-host intent — `site`, `role`, `model_hint`, `networks`, `bootstrap_mac`. Six hosts (prodnxsr0014-0019) seeded; the 13 existing hosts are tracked in #2. - **Managed IPs** (`config/au/syd1/managed-ips.yaml`): static address assignments. - **Module** `modules/infra/`: NetBox (sites, prefixes, devices, interfaces, IPs) + Kea provider wiring; Terragrunt env under `environments/au/syd1/`. - **CI** `.woodpecker/`: plan (PR), apply (main), pre-commit. - **Backfill** `ci/puppetdb_backfill.py`: reality import from PuppetDB (in-cluster only — automation tracked in #1). ## Discovery background - unkin/bootapi#3 - unkin/bootapi-templates#5 ## Follow-ups filed - #1 Automate PuppetDB reality backfill into NetBox - #2 Seed intent device YAML for existing hosts prodnxsr0001-0013 - #3 Consume network gateway/dns/search into NetBox/provisioning - #4 Set device primary IP in NetBox ## Known plan blockers (expected CI failures until resolved) - terraform-vault #111 (consul role/ACL/kv policy) not yet merged/applied - Vault KV `kv/service/terraform/infra` (`netbox_token`, `kea_token`) not seeded - NetBox + KeaAPI not yet deployed https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT --- ## Seeding required before plan/apply can pass Even with the vault child-token fix, the plan reads two provider tokens from Vault KV v2 `kv/service/terraform/infra`. Only a human can seed real tokens today (the NetBox secrets engine has not landed): ``` vault kv put kv/service/terraform/infra \ netbox_token=<NETBOX_API_TOKEN> \ kea_token=<KEA_API_TOKEN> ``` Required keys: `netbox_token`, `kea_token`. The read + metadata-read policy is added in terraform-vault PR #111 (blocked on the unrelated litellm plan breakage, terraform-vault#112). Plan also needs NetBox + KeaAPI deployed and the NetBox site `syd1` to exist.
unkinben added 2 commits 2026-08-05 19:42:37 +10:00
Initial terraform-ipam: NetBox IPAM + Kea DHCP from per-subnet YAML bfbe42f97e 
Manage NetBox prefixes and Kea DHCP scopes together from a single subnet
definition under config/<region>/<dc>/subnets/<name>.yaml.

- modules/ipam: netbox_prefix + netbox_ip_range (dhcp role) + gateway/managed
  netbox_ip_address + kea_subnet; tokens read from Vault KV via the vault provider
- environments/au/syd1 terragrunt env, Consul state backend
- config seed: five DHCP subnets (198.18.13-17) + local netbox-only 198.18.25.0/24
- .woodpecker pre-commit+plan on PR, apply on main; Makefile Vault auth pattern

Claude-Session: https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
Rename terraform-ipam -> terraform-infra; add devices + networks + puppetdb backfill
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/plan Pipeline failed
Details
9a6f775300 
Scope now spans devices and provisioning, not just IPAM.

- rename module/consul-path/role ipam -> infra
- networks config (subnet binding + gateway/dns/search constants); prefixes tagged net:<name>
- intent-only devices module: netbox_device + device_type/role/manufacturer, static or
  next-available IPs (sticky via ignore_changes), transitional bootstrap_mac interface for
  bootapi PXE keying
- seed 6 pending hosts prodnxsr0014-0019 (mgmt IPs .14-.19, optiplex-3070)
- ci/puppetdb_backfill.py: emit NetBox reality (serial/model/uuid/interfaces) for existing hosts

Claude-Session: https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
unkinben commented 2026-08-05 19:48:52 +10:00
Author
Owner
Copy Link
Copy Source

CI note: the plan step failure is the expected environmental blocker set (terraform-vault #111 not merged/applied, kv/service/terraform/infra unseeded, NetBox/Kea not deployed) — not an HCL problem. tofu init -backend=false && tofu validate passes locally on this branch (one deprecation warning for vault_kv_secret_v2 data source, non-blocking). Pre-commit is green.

https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT

CI note: the `plan` step failure is the expected environmental blocker set (terraform-vault #111 not merged/applied, `kv/service/terraform/infra` unseeded, NetBox/Kea not deployed) — not an HCL problem. `tofu init -backend=false && tofu validate` passes locally on this branch (one deprecation warning for `vault_kv_secret_v2` data source, non-blocking). Pre-commit is green. https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
unkinben added 1 commit 2026-08-06 23:09:01 +10:00
Fix vault provider auth in CI: skip_child_token + correct kv path
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/plan Pipeline failed
Details
9bdf9ce9fc 
The woodpecker_terraform_infra role cannot mint child tokens (auth/token/create
403), which failed the plan's vault_kv_secret_v2 read. Set skip_child_token so the
provider uses the login token directly (estate pattern). Also correct the KV path
missed in the ipam->infra rename: service/terraform/infra.

Claude-Session: https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
unkinben added 1 commit 2026-08-09 00:18:48 +10:00
Mint the NetBox token from the Vault netbox engine
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/plan Pipeline failed
Details
a025819fcd 
Why:
- The netbox provider authenticated with a static netbox_token seeded by hand
  into KV; the vault-plugin-secrets-netbox engine mints a per-run ephemeral
  token that is lease-revoked when the run ends, removing the manual seed.

How:
- Read netbox/creds/terraform-infra via vault_generic_secret and pass the minted
  netbox_token to the netbox provider.
- Keep kea_token from KV for now (follow-up: its own ephemeral-token engine).

Depends on the netbox engine mount + netbox/creds/terraform-infra role/policy in
terraform-vault being applied first.
Some required checks failed
ci/woodpecker/pr/pre-commit Pipeline was successful
Required
Details
ci/woodpecker/pr/plan Pipeline failed
Required
Details
Some required checks were not successful.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin benvin/initial-infra:benvin/initial-infra
git checkout benvin/initial-infra
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
bug
Something is not working
 duplicate
This issue or pull request already exists
 enhancement
New feature
 help wanted
Need some help
 invalid
Something is wrong
 question
More information is needed
 wontfix
This won't be fixed
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
benvin (BenVincent) unkin-agent (Unkin Agent) unkinben (Ben Vincent)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: unkin/terraform-infra#5
Delete Branch "benvin/initial-infra"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?