Derive the site from the config path and split out global objects
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/plan Pipeline failed

Stop repeating the site slug inside the YAML: terragrunt passes the dc as
site_slug and site.yaml carries only site-wide attributes. Declare device
types, roles, platforms, custom fields and the dhcp IPAM role once in
config/global via their own unit and state; the per-site unit looks them up
by slug or name.
This commit is contained in:
2026-10-03 18:34:40 +10:00
parent 17f9c2b7c7
commit be53170557
28 changed files with 280 additions and 185 deletions
+48 -27
View File
@@ -19,26 +19,50 @@ must exist before discovery can cable a host interface to them.
> [kea-operator](https://git.unkin.net/unkin/kea-operator) CRs in argocd-apps
> (`apps/base/dhcp-system/cr/`).
## Conventions
A per-site config path is `config/<region>/<dc>/`, and `<dc>` **is** the NetBox
site slug: terragrunt derives it from the directory and passes it in, so the
site is never named again inside the YAML.
Device types, manufacturers, device roles, platforms, custom-field definitions
and the `dhcp` IPAM role are global objects in NetBox, not site-scoped. They are
declared once in `config/global/` by their own terragrunt unit and state;
per-site units look them up by slug or name. Two per-site units creating them
would collide on name as soon as a second region/dc exists.
## Layout
```
config/<region>/<dc>/sites.yaml # site slug -> name/description
config/global/device-types/<slug>.yaml # manufacturer, model, port templates
config/global/device-roles.yaml
config/global/platforms.yaml # slug -> display name
config/global/custom-fields.yaml # custom-field DEFINITIONS
config/<region>/<dc>/site.yaml # site-wide attributes; slug is <dc>
config/<region>/<dc>/subnets/net-<a>-<b>-<c>-<d>-<len>.yaml
config/<region>/<dc>/device-types/<slug>.yaml # manufacturer, model, port templates
config/<region>/<dc>/device-roles.yaml
config/<region>/<dc>/network-devices.yaml # switches, keyed by device name
config/<region>/<dc>/platforms.yaml # slug -> display name
config/<region>/<dc>/vlans.yaml # keyed per site
config/<region>/<dc>/vlans.yaml
config/<region>/<dc>/network-profiles/<name>.yaml
config/<region>/<dc>/custom-fields.yaml # custom-field DEFINITIONS
config/<region>/<dc>/network-devices.yaml # switches, keyed by device name
config/<region>/<dc>/managed-ips.yaml
environments/global/terragrunt.hcl
environments/<region>/<dc>/terragrunt.hcl
modules/netbox-global/
modules/netbox/
```
`make plan` / `make apply` reconcile the YAML against NetBox. State lives in
Consul under `infra/terraform/netbox/<region>/<dc>/state`. A site that already
exists in NetBox must be imported, not re-created: site names are unique.
`make plan` / `make apply` reconcile the YAML against NetBox across both units.
State lives in Consul under `infra/terraform/netbox/global/state` and
`infra/terraform/netbox/<region>/<dc>/state`. A site that already exists in
NetBox must be imported, not re-created: site names are unique.
## Site (`site.yaml`)
Site-wide attributes only. The NetBox site name defaults to the slug; set
`name` only if the display name must differ from it.
```yaml
description: Sydney region, datacentre 1
```
## Subnets (`subnets/net-<a>-<b>-<c>-<d>-<len>.yaml`)
@@ -48,7 +72,6 @@ The filename carries the network and mask length; the human label lives in
```yaml
prefix: 198.18.15.0/24
description: syd1 production
site: syd1 # NetBox site slug
router: 254 # host-octet int -> gateway IP
dhcp: # omit for a NetBox-only prefix
enabled: true
@@ -64,7 +87,7 @@ gateway `netbox_ip_address`. `dns`, `domain` and `next_server` are read by no
resource today; they are retained so a later change can drive Kea from this
repo without re-deriving them.
## Device types (`device-types/<slug>.yaml`)
## Device types (`config/global/device-types/<slug>.yaml`)
`ports` declares interface templates as ranges, not 28 hand-written entries.
NetBox instantiates them on every device created from the type.
@@ -86,40 +109,38 @@ ports:
## Network devices (`network-devices.yaml`)
Switches only; host records come from autodiscovery. `chassis_mac` is written to
the custom field discovery matches an LLDP neighbour on.
Switches only; host records come from autodiscovery. `device_type` and `role`
name global objects, looked up by slug/name. `chassis_mac` is written to the
custom field discovery matches an LLDP neighbour on.
```yaml
sg3429x-m2-01:
device_type: sg3428x-m2 # device-types/ key
role: network-switch # device-roles.yaml key
site: syd1
device_type: sg3428x-m2 # config/global/device-types/ key
role: network-switch # config/global/device-roles.yaml key
serial: 225C578000245 # optional
chassis_mac: b8:fb:b3:a7:fa:24 # optional
```
## VLANs (`vlans.yaml`)
Keyed per site. Each site gets its own site-scoped `netbox_vlan_group`, so the
same VID may exist in more than one site.
The site gets its own site-scoped `netbox_vlan_group`, so the same VID may
exist in more than one site.
```yaml
syd1:
vid_ranges: [[1, 4094]]
vlans:
- vid: 101
name: fabric_common_01
vid_ranges: [[1, 4094]]
vlans:
- vid: 101
name: fabric_common_01
```
## Network profiles (`network-profiles/<name>.yaml`)
A profile describes the interfaces a logical host should have. It is published
as a `netbox_config_context` scoped to its sites (and roles, if given), so the
as a `netbox_config_context` scoped to its site (and roles, if given), so the
UI reads profiles from the NetBox API rather than from git.
```yaml
description: Dual-homed fabric host
sites: [syd1]
weight: 1000
interfaces:
- name: fabric0
@@ -128,7 +149,7 @@ interfaces:
subinterfaces: [] # optional, same shape
```
## Custom fields (`custom-fields.yaml`)
## Custom fields (`config/global/custom-fields.yaml`)
Field **definitions** for what bootapi reads: `pxe_enabled`,
`provision_template`, `domain`, `nameservers`, `gateway`; plus `chassis_mac`,
-2
View File
@@ -4,13 +4,11 @@
sg3429x-m2-01:
device_type: sg3428x-m2
role: network-switch
site: syd1
serial: 225C578000245
chassis_mac: b8:fb:b3:a7:fa:24
sg3429x-m2-02:
device_type: sg3428x-m2
role: network-switch
site: syd1
serial: 225C578000046
chassis_mac: b8:fb:b3:a7:f9:5d
@@ -1,7 +1,6 @@
# A network profile describes the interfaces a logical host should have. The UI
# reads it back from the NetBox config-context API when planning a host.
description: Dual-homed fabric host, one leg per common fabric VLAN
sites: [syd1]
weight: 1000
interfaces:
- name: fabric0
+3
View File
@@ -0,0 +1,3 @@
# Site-wide attributes. The slug is the config directory (config/<region>/<dc>)
# and is never repeated here; the NetBox site name defaults to it.
description: Sydney region, datacentre 1
-4
View File
@@ -1,4 +0,0 @@
# NetBox sites, keyed by slug.
syd1:
name: syd1
description: Sydney region, datacentre 1
@@ -1,6 +1,5 @@
prefix: 198.18.13.0/24
description: syd1 production
site: syd1
router: 254
dhcp:
enabled: true
@@ -1,6 +1,5 @@
prefix: 198.18.14.0/24
description: syd1 production
site: syd1
router: 254
dhcp:
enabled: true
@@ -1,6 +1,5 @@
prefix: 198.18.15.0/24
description: syd1 production
site: syd1
router: 254
dhcp:
enabled: true
@@ -1,6 +1,5 @@
prefix: 198.18.16.0/24
description: syd1 production
site: syd1
router: 254
dhcp:
enabled: true
@@ -1,6 +1,5 @@
prefix: 198.18.17.0/24
description: drw1-prod
site: syd1
router: 1
dhcp:
enabled: true
@@ -1,3 +1,2 @@
prefix: 198.18.25.0/24
description: syd1 netbox-only
site: syd1
+8 -9
View File
@@ -1,9 +1,8 @@
# VLANs keyed by site slug. Each site gets its own NetBox VLAN group scoped to
# the site, so the same VID may exist in more than one site.
syd1:
vid_ranges: [[1, 4094]]
vlans:
- vid: 101
name: fabric_common_01
- vid: 201
name: fabric_common_02
# The site's VLANs. The site gets one NetBox VLAN group scoped to it, so the
# same VID may exist in more than one site.
vid_ranges: [[1, 4094]]
vlans:
- vid: 101
name: fabric_common_01
- vid: 201
name: fabric_common_02
+9 -14
View File
@@ -3,6 +3,12 @@ include "root" {
expose = true
}
# Global objects (device types, roles, platforms, custom fields, the dhcp IPAM
# role) must exist before this unit can look them up by slug/name.
dependencies {
paths = ["../../global"]
}
locals {
parts = split("/", path_relative_to_include())
region = local.parts[0]
@@ -14,21 +20,13 @@ locals {
trimsuffix(basename(f), ".yaml") => yamldecode(file("${local.cfg_dir}/${f}"))
}
device_types = {
for f in fileset(local.cfg_dir, "device-types/*.yaml") :
trimsuffix(basename(f), ".yaml") => yamldecode(file("${local.cfg_dir}/${f}"))
}
network_profiles = {
for f in fileset(local.cfg_dir, "network-profiles/*.yaml") :
trimsuffix(basename(f), ".yaml") => yamldecode(file("${local.cfg_dir}/${f}"))
}
sites = yamldecode(file("${local.cfg_dir}/sites.yaml"))
device_roles = yamldecode(file("${local.cfg_dir}/device-roles.yaml"))
platforms = yamldecode(file("${local.cfg_dir}/platforms.yaml"))
site = yamldecode(file("${local.cfg_dir}/site.yaml"))
vlans = yamldecode(file("${local.cfg_dir}/vlans.yaml"))
custom_fields = yamldecode(file("${local.cfg_dir}/custom-fields.yaml"))
managed_ips = yamldecode(file("${local.cfg_dir}/managed-ips.yaml"))
network_devices = yamldecode(file("${local.cfg_dir}/network-devices.yaml"))
}
@@ -38,14 +36,11 @@ terraform {
}
inputs = {
sites = local.sites
site_slug = local.dc
site = local.site
subnets = local.subnets
device_types = local.device_types
device_roles = local.device_roles
platforms = local.platforms
vlans = local.vlans
network_profiles = local.network_profiles
custom_fields = local.custom_fields
managed_ips = local.managed_ips
network_devices = local.network_devices
}
+31
View File
@@ -0,0 +1,31 @@
# NetBox objects that are global rather than site-scoped. Own unit, own state:
# a per-site unit creating them would collide on name as soon as a second
# region/dc exists.
include "root" {
path = find_in_parent_folders("root.hcl")
expose = true
}
locals {
cfg_dir = "${get_repo_root()}/config/global"
device_types = {
for f in fileset(local.cfg_dir, "device-types/*.yaml") :
trimsuffix(basename(f), ".yaml") => yamldecode(file("${local.cfg_dir}/${f}"))
}
device_roles = yamldecode(file("${local.cfg_dir}/device-roles.yaml"))
platforms = yamldecode(file("${local.cfg_dir}/platforms.yaml"))
custom_fields = yamldecode(file("${local.cfg_dir}/custom-fields.yaml"))
}
terraform {
source = "${get_repo_root()}/modules/netbox-global"
}
inputs = {
device_types = local.device_types
device_roles = local.device_roles
platforms = local.platforms
custom_fields = local.custom_fields
}
@@ -1,3 +1,6 @@
# NetBox objects that are GLOBAL, not site-scoped: a second region/dc would
# collide on name if a per-site unit created them from its own state. They are
# declared once here and looked up by slug/name from the per-site module.
locals {
manufacturers = toset([for t in var.device_types : t.manufacturer])
@@ -58,20 +61,8 @@ resource "netbox_platform" "this" {
slug = each.key
}
# Network equipment is declared here, unlike host devices: nothing probes a
# switch, and its ports must already exist in NetBox for discovery to cable a
# host interface to one.
resource "netbox_device" "network" {
for_each = var.network_devices
name = each.key
device_type_id = tonumber(netbox_device_type.this[each.value.device_type].id)
role_id = tonumber(netbox_device_role.this[each.value.role].id)
site_id = tonumber(netbox_site.this[each.value.site].id)
serial = each.value.serial
asset_tag = each.value.asset_tag
custom_fields = each.value.chassis_mac != null ? { chassis_mac = lower(each.value.chassis_mac) } : null
# chassis_mac must exist as a field before a device can carry a value for it.
depends_on = [netbox_custom_field.this]
# Role tagging an IP range as DHCP-managed; shared by every site's ranges.
resource "netbox_ipam_role" "dhcp" {
name = "dhcp"
slug = "dhcp"
}
+19
View File
@@ -0,0 +1,19 @@
provider "vault" {
address = var.vault_address
# The woodpecker_terraform_netbox role cannot mint child tokens
# (auth/token/create is denied); use the login token directly.
skip_child_token = true
}
# NetBox API token: minted per run by the vault-plugin-secrets-netbox engine
# (netbox/creds/terraform-netbox), lease-revoked when the run ends. The engine
# returns the credential as "token" (alongside token_scheme/authorization); it is
# a bare nbt_ v2 value, which NetBox accepts under either Authorization keyword.
data "vault_generic_secret" "netbox" {
path = "netbox/creds/terraform-netbox"
}
provider "netbox" {
server_url = var.netbox_server_url
api_token = data.vault_generic_secret.netbox.data["token"]
}
+64
View File
@@ -0,0 +1,64 @@
variable "device_types" {
description = <<-EOT
Device-type catalogue keyed by NetBox slug. `ports` declares interface
templates as ranges: each block names `count` ports from `first`, formatted
with `name_format` (a single %d). NetBox instantiates them on every device
created from the type. Manufacturers are created from the distinct
`manufacturer` values.
EOT
type = map(object({
manufacturer = string
model = string
u_height = optional(number, 1)
ports = optional(list(object({
name_format = string
type = string
first = optional(number, 1)
count = number
mgmt_only = optional(bool, false)
})), [])
}))
default = {}
}
variable "device_roles" {
description = "NetBox device roles keyed by role name; the slug is derived from the name."
type = map(object({
description = optional(string, "")
color_hex = optional(string, "9e9e9e")
}))
default = {}
}
variable "platforms" {
description = "NetBox platforms, slug -> display name. A device's platform slug is bootapi's distro-catalog selector."
type = map(string)
default = {}
}
variable "custom_fields" {
description = <<-EOT
NetBox custom-field definitions keyed by field name. These are the field
DEFINITIONS only; values are set per device by the owning service.
`content_types` are NetBox "app_label.model" strings, e.g. "dcim.device".
EOT
type = map(object({
type = optional(string, "text")
content_types = list(string)
label = optional(string)
description = optional(string)
}))
default = {}
}
variable "vault_address" {
description = "Vault server address for the token data source."
type = string
default = "https://vault.service.consul:8200"
}
variable "netbox_server_url" {
description = "NetBox server base URL."
type = string
default = "https://netbox.k8s.syd1.au.unkin.net"
}
+13
View File
@@ -0,0 +1,13 @@
terraform {
required_version = ">= 1.10"
required_providers {
netbox = {
source = "e-breuninger/netbox"
version = "4.3.0"
}
vault = {
source = "hashicorp/vault"
version = "5.6.0"
}
}
}
+30
View File
@@ -0,0 +1,30 @@
# Device types and roles are global; look them up rather than create them here.
data "netbox_device_type" "this" {
for_each = toset([for d in var.network_devices : d.device_type])
slug = each.value
}
data "netbox_device_role" "this" {
for_each = toset(concat(
[for d in var.network_devices : d.role],
flatten([for p in var.network_profiles : p.roles]),
))
name = each.value
}
# Network equipment is declared here, unlike host devices: nothing probes a
# switch, and its ports must already exist in NetBox for discovery to cable a
# host interface to one.
resource "netbox_device" "network" {
for_each = var.network_devices
name = each.key
device_type_id = tonumber(data.netbox_device_type.this[each.value.device_type].id)
role_id = tonumber(data.netbox_device_role.this[each.value.role].id)
site_id = tonumber(netbox_site.this.id)
serial = each.value.serial
asset_tag = each.value.asset_tag
custom_fields = each.value.chassis_mac != null ? { chassis_mac = lower(each.value.chassis_mac) } : null
}
+9 -10
View File
@@ -9,14 +9,14 @@ locals {
gateways = { for k, v in var.subnets : k => v if v.router != null }
}
# The one site this unit owns. Its slug is the config directory
# (config/<region>/<dc>), never repeated inside the YAML; `name` defaults to it.
# A site that already exists in NetBox must be imported rather than re-created
# (NetBox site names are unique).
resource "netbox_site" "this" {
for_each = var.sites
name = each.value.name
slug = each.key
description = each.value.description
name = coalesce(var.site.name, var.site_slug)
slug = var.site_slug
description = var.site.description
}
resource "netbox_prefix" "this" {
@@ -25,13 +25,12 @@ resource "netbox_prefix" "this" {
prefix = each.value.prefix
status = "active"
description = each.value.description
site_id = each.value.site != null ? tonumber(netbox_site.this[each.value.site].id) : null
site_id = tonumber(netbox_site.this.id)
}
# Role tagging a range as DHCP-managed; created once and shared by every range.
resource "netbox_ipam_role" "dhcp" {
# Global object, created by the netbox-global unit.
data "netbox_ipam_role" "dhcp" {
name = "dhcp"
slug = "dhcp"
}
resource "netbox_ip_range" "dhcp" {
@@ -39,7 +38,7 @@ resource "netbox_ip_range" "dhcp" {
start_address = "${cidrhost(each.value.prefix, each.value.dhcp.start)}/${local.masks[each.key]}"
end_address = "${cidrhost(each.value.prefix, each.value.dhcp.stop)}/${local.masks[each.key]}"
role_id = netbox_ipam_role.dhcp.id
role_id = data.netbox_ipam_role.dhcp.id
status = "active"
description = "DHCP pool for ${each.key}"
}
+10 -18
View File
@@ -1,31 +1,23 @@
locals {
# "<site>:<vid>" -> {site, vid, name, description}.
site_vlans = merge([
for site, v in var.vlans : {
for vl in v.vlans : "${site}:${vl.vid}" => merge(vl, { site = site })
}
]...)
subnet_prefix = { for k, v in var.subnets : k => v.prefix }
}
# One VLAN group scoped to this site, so the same VID may exist in another site.
resource "netbox_vlan_group" "this" {
for_each = var.vlans
name = "${each.key} vlans"
slug = "${each.key}-vlans"
name = "${var.site_slug} vlans"
slug = "${var.site_slug}-vlans"
scope_type = "dcim.site"
scope_id = tonumber(netbox_site.this[each.key].id)
vid_ranges = each.value.vid_ranges
scope_id = tonumber(netbox_site.this.id)
vid_ranges = var.vlans.vid_ranges
}
resource "netbox_vlan" "this" {
for_each = local.site_vlans
for_each = { for v in var.vlans.vlans : v.vid => v }
name = each.value.name
vid = each.value.vid
site_id = tonumber(netbox_site.this[each.value.site].id)
group_id = tonumber(netbox_vlan_group.this[each.value.site].id)
site_id = tonumber(netbox_site.this.id)
group_id = tonumber(netbox_vlan_group.this.id)
status = "active"
description = each.value.description
}
@@ -38,8 +30,8 @@ resource "netbox_config_context" "network_profile" {
name = "network-profile:${each.key}"
description = each.value.description
weight = each.value.weight
sites = [for s in each.value.sites : tonumber(netbox_site.this[s].id)]
roles = [for r in each.value.roles : tonumber(netbox_device_role.this[r].id)]
sites = [tonumber(netbox_site.this.id)]
roles = [for r in each.value.roles : tonumber(data.netbox_device_role.this[r].id)]
data = jsonencode({
network_profile = {
+29 -78
View File
@@ -1,25 +1,29 @@
variable "sites" {
description = "NetBox sites keyed by slug."
type = map(object({
name = string
variable "site_slug" {
description = "NetBox site slug, derived from the config path (config/<region>/<dc>); the dc IS the slug."
type = string
}
variable "site" {
description = "Site-wide attributes. `name` defaults to the slug."
type = object({
name = optional(string)
description = optional(string, "")
}))
})
default = {}
}
variable "subnets" {
description = <<-EOT
Map of subnets keyed by name (config file basename,
net-<a>-<b>-<c>-<d>-<len>). `dhcp` records the pool as a NetBox ip_range
only; the DHCP server itself is the kea-operator's KeaSubnet CRs in
argocd-apps. `dns`, `domain` and `next_server` are not read by any resource
here -- they are retained so a later change can drive Kea from this repo
without re-deriving them.
net-<a>-<b>-<c>-<d>-<len>). Every subnet belongs to this unit's site.
`dhcp` records the pool as a NetBox ip_range only; the DHCP server itself is
the kea-operator's KeaSubnet CRs in argocd-apps. `dns`, `domain` and
`next_server` are not read by any resource here -- they are retained so a
later change can drive Kea from this repo without re-deriving them.
EOT
type = map(object({
prefix = string
description = optional(string, "")
site = optional(string)
router = optional(number)
dhcp = optional(object({
enabled = optional(bool, true)
@@ -33,57 +37,19 @@ variable "subnets" {
default = {}
}
variable "device_types" {
description = <<-EOT
Device-type catalogue keyed by NetBox slug. `ports` declares interface
templates as ranges: each block names `count` ports from `first`, formatted
with `name_format` (a single %d). NetBox instantiates them on every device
created from the type. Manufacturers are created from the distinct
`manufacturer` values.
EOT
type = map(object({
manufacturer = string
model = string
u_height = optional(number, 1)
ports = optional(list(object({
name_format = string
type = string
first = optional(number, 1)
count = number
mgmt_only = optional(bool, false)
})), [])
}))
default = {}
}
variable "device_roles" {
description = "NetBox device roles keyed by role name; the slug is derived from the name."
type = map(object({
description = optional(string, "")
color_hex = optional(string, "9e9e9e")
}))
default = {}
}
variable "platforms" {
description = "NetBox platforms, slug -> display name. A device's platform slug is bootapi's distro-catalog selector."
type = map(string)
default = {}
}
variable "vlans" {
description = <<-EOT
VLANs keyed by site slug. Each site gets one NetBox VLAN group scoped to the
site, so the same VID may exist in more than one site.
This site's VLANs. The site gets one NetBox VLAN group scoped to it, so the
same VID may exist in more than one site.
EOT
type = map(object({
type = object({
vid_ranges = optional(list(list(number)), [[1, 4094]])
vlans = list(object({
vlans = optional(list(object({
vid = number
name = string
description = optional(string, "")
}))
}))
})), [])
})
default = {}
}
@@ -92,12 +58,11 @@ variable "network_profiles" {
Network profiles keyed by profile name. A profile describes the interfaces a
logical host should have: one entry per interface, each bound to a VLAN VID
and a subnet key, optionally with subinterfaces. Published as a NetBox
config context scoped to `sites` (and `roles`, if given) so the UI reads
config context scoped to this site (and `roles`, if given) so the UI reads
profiles from the NetBox API.
EOT
type = map(object({
description = optional(string, "")
sites = list(string)
roles = optional(list(string), [])
weight = optional(number, 1000)
interfaces = list(object({
@@ -123,33 +88,19 @@ variable "managed_ips" {
default = []
}
variable "custom_fields" {
description = <<-EOT
NetBox custom-field definitions keyed by field name. These are the field
DEFINITIONS only; values are set per device by the owning service.
`content_types` are NetBox "app_label.model" strings, e.g. "dcim.device".
EOT
type = map(object({
type = optional(string, "text")
content_types = list(string)
label = optional(string)
description = optional(string)
}))
default = {}
}
variable "network_devices" {
description = <<-EOT
Network equipment keyed by device name. Switches are declared here rather
than left to autodiscovery: nothing probes one, and its interface templates
must be instantiated in NetBox before discovery can cable a host interface
to a port. `chassis_mac` is published as the chassis_mac custom field so an
LLDP neighbour can be matched without its SysName.
Network equipment keyed by device name, all of it in this unit's site.
Switches are declared here rather than left to autodiscovery: nothing probes
one, and its interface templates must be instantiated in NetBox before
discovery can cable a host interface to a port. `device_type` and `role`
name global objects created by the netbox-global unit. `chassis_mac` is
published as the chassis_mac custom field so an LLDP neighbour can be
matched without its SysName.
EOT
type = map(object({
device_type = string
role = string
site = string
serial = optional(string)
asset_tag = optional(string)
chassis_mac = optional(string)