unkinben
d3fb5dcd1a
Replace the ISC dhcpd PXE-boot VM with a Kea DHCP Kubernetes operator, modelled on bind-operator. The operator renders kea-dhcp4 config from CRs and runs an HA pair of kea-dhcp4 + kea-ctrl-agent servers behind an anycast Service. - add KeaCluster/KeaSubnet/KeaClientClass/KeaAPI CRDs (group kea.unkin.net) - render deterministic kea-dhcp4.conf + kea-ctrl-agent.conf into a ConfigMap and roll the StatefulSet via a config-hash annotation; best-effort hot-reload via the kea-ctrl-agent REST channel - run HA hot-standby (memfile leases) with stable per-peer DNS identity from a StatefulSet; expose an anycast LoadBalancer Service for PureLB - represent the full legacy dhcpd config: 198.18.13-17.0/24 pools, pool-less 198.18.25.0/24, and the Legacy/UEFI-64 PXE arch classes (option 93) - add the KeaAPI-spawned REST service: Terraform-friendly CRUD over subnet and client-class CRs (stable IDs, PUT upsert, 404 drift, bearer-token auth) - add Makefile (patch/minor/major tag targets), distroless operator/api images, an AlmaLinux+EPEL kea workload image, and woodpecker CI with k8s resources + serviceAccountName on every step - unit tests for config rendering, controller reconcile/config-hash, and the API Claude-Session: https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
88 lines
4.6 KiB
Markdown
88 lines
4.6 KiB
Markdown
# kea-operator
|
||
|
||
A Kubernetes operator that runs [ISC Kea](https://www.isc.org/kea/) DHCPv4
|
||
servers to replace the legacy ISC `dhcpd` VM used for PXE-booting physical
|
||
hosts. Modelled on the sibling `bind-operator`.
|
||
|
||
Namespace: `dhcp-system`. API group: `kea.unkin.net/v1alpha1`.
|
||
|
||
## Custom resources
|
||
|
||
| Kind | Purpose |
|
||
|------|---------|
|
||
| **KeaCluster** | Spawns a StatefulSet of `kea-dhcp4` + `kea-ctrl-agent` pods, an HA control channel, and an anycast DHCP `Service`. Holds global config (domain, lease times, NTP, PXE option-defs). |
|
||
| **KeaSubnet** | One DHCPv4 subnet referenced to a KeaCluster (`clusterRef`). CIDR, optional pools, routers, DNS, domain, next-server. Pool-less subnets are declared so relayed requests are still serviced. |
|
||
| **KeaClientClass** | PXE boot class matched on client architecture (option 93), e.g. `Legacy` → `/undionly.kpxe`, `UEFI-64` → `/ipxe.efi`. |
|
||
| **KeaAPI** | Spawns the Terraform-friendly REST API service (see below). |
|
||
|
||
The **KeaCluster** controller lists the matching subnets and client classes,
|
||
renders a deterministic `kea-dhcp4.conf` (+ `kea-ctrl-agent.conf`) into a
|
||
ConfigMap, and rolls the StatefulSet via a config-hash annotation stamped on the
|
||
pod template (identical trick to bind-operator). Ready pods are additionally
|
||
hot-reloaded best-effort via the kea-ctrl-agent REST control channel
|
||
(`config-reload`), analogous to `rndc reconfig`.
|
||
|
||
## HA design
|
||
|
||
**Mode: hot-standby, memfile lease DB (non-persistent).** PXE leases are
|
||
ephemeral, low-volume, and drawn from tiny pools (`.200–.220`). Hot-standby
|
||
keeps a single active primary answering all DHCP with a warm secondary that
|
||
syncs leases over the HA control channel and auto-promotes on primary failure —
|
||
avoiding the split-pool double-allocation two uncoordinated servers would cause.
|
||
Load-balancing mode is also selectable via `spec.ha.mode`.
|
||
|
||
Kea HA needs a **stable per-peer identity** (each server must know which peer it
|
||
is, and peers reference each other by stable URL). That is exactly why this
|
||
operator (like bind-operator) uses a **StatefulSet** rather than a bare
|
||
Deployment: pods get stable ordinals (`<cluster>-0`, `<cluster>-1`) and headless
|
||
DNS, the entrypoint derives `this-server-name` from the ordinal, and the peer
|
||
URLs are DNS names (never pod IPs, so the config hash never loops).
|
||
|
||
### Anycast routing caveat (deployment follow-up)
|
||
|
||
The DHCP `Service` is a `LoadBalancer` intended to receive a PureLB anycast IP;
|
||
routers relay unicast to it. Under hot-standby the standby does not answer while
|
||
the primary is up, so the anycast VIP should be pinned to the active peer. Pin
|
||
it operationally (PureLB local traffic policy / active-peer endpoint selection)
|
||
before production cutover. See follow-ups.
|
||
|
||
## REST API (KeaAPI)
|
||
|
||
A separate binary (`cmd/api`) spawned by the `KeaAPI` CRD. It is a Terraform-
|
||
friendly CRUD facade over the `KeaSubnet` / `KeaClientClass` CRs — an
|
||
alternative to argocd-managed CRs. Contract mirrors `encapi`:
|
||
|
||
- `PUT/GET/DELETE /api/v1/subnets/{name}` and `.../clientclasses/{name}`, plus
|
||
list endpoints. Stable client-supplied IDs (the CR name, from the URL path).
|
||
- `PUT` is an idempotent upsert returning the canonical object (200); `DELETE`
|
||
returns 204; `GET` on a missing resource returns 404 (drives provider drift
|
||
handling). Error bodies are `{"error": "..."}`.
|
||
- Auth: bearer token (constant-time compare, fail-closed) from `KEA_API_TOKEN`,
|
||
sourced from a k8s Secret the operator generates if absent (or pre-seeds).
|
||
|
||
The Terraform provider is a **separate queued task**; the API is designed so
|
||
wrapping it is trivial (full-object PUT, canonical GET, 404 semantics).
|
||
|
||
## Build & release
|
||
|
||
- `make test` — `go test -race` over `./api/... ./internal/...`.
|
||
- `make generate` — regenerates deepcopy, CRDs, RBAC, and the `install.yaml`
|
||
bundle (all committed).
|
||
- `make patch|minor|major` — tags `vX.Y.Z` and pushes; the tag triggers the
|
||
`.woodpecker/docker.yaml` image builds.
|
||
|
||
Images: `kea-operator` and `kea-api` are distroless Go binaries. The Kea
|
||
workload image (`Dockerfile.kea`) is built from AlmaLinux (reachable via the
|
||
artifactapi dockerhub remote) + EPEL kea packages.
|
||
|
||
## Follow-ups (not in this repo yet)
|
||
|
||
- **argocd-apps**: a `kea-operator-ci` ServiceAccount for the woodpecker CI
|
||
steps, and the operator deployment manifests.
|
||
- **artifactapi**: the Kea image build pulls kea RPMs from EPEL; if the CI build
|
||
network cannot reach EPEL, add an artifactapi rpm remote (or vendor kea via
|
||
rpmbuilder) and point `Dockerfile.kea` at it.
|
||
- **terraform-provider-kea**: wrap the KeaAPI REST contract.
|
||
- **Vault**: issue ephemeral API bearer tokens instead of a static k8s Secret.
|
||
- **Anycast**: pin the anycast VIP to the active HA peer (PureLB tuning).
|