Add mail edge and vpn hosts to the DMZ VPCs #2

Merged

benvin merged 9 commits from benvin/mail-gateways into main 2026-10-14 22:17:09 +11:00

Conversation 3 Commits 9 Files Changed 8

+470 -3

unkin-agent commented 2026-10-12 21:50:16 +11:00

Member

Copy Link

Copy Source

Each region (syd, mel) needs a postfix edge gateway that reaches the lab without carrying WireGuard, so each DMZ VPC also gets a vpn host peering OSPF with prodnxsr0020 over its own wg link.

• add dmz-syd 198.18.30.0/24 and dmz-mel 198.18.31.0/24 VPCs, each with an edge host (.10, PTR mta1/mta2.edge.unkin.net) and a vpn host (.11)
• route 198.18.0.0/16 and 10.10.12.200/32 via the vpn host; the VPC route table carries the /16 minus 198.18.30.0/23
• give each vpn host a /31 wg link from 198.18.2.176/30 and a dum0 loopback (198.18.2.161, .162)
• store router and host wg keys under kv/service/wireguard/binarylane/
• expose router_links for the prodnxsr0020 Puppet config
• bootstrap openvox via cloud-init and firewall each host

Each region (syd, mel) needs a postfix edge gateway that reaches the lab without carrying WireGuard, so each DMZ VPC also gets a vpn host peering OSPF with prodnxsr0020 over its own wg link. - add dmz-syd 198.18.30.0/24 and dmz-mel 198.18.31.0/24 VPCs, each with an edge host (.10, PTR mta1/mta2.edge.unkin.net) and a vpn host (.11) - route 198.18.0.0/16 and 10.10.12.200/32 via the vpn host; the VPC route table carries the /16 minus 198.18.30.0/23 - give each vpn host a /31 wg link from 198.18.2.176/30 and a dum0 loopback (198.18.2.161, .162) - store router and host wg keys under kv/service/wireguard/binarylane/ - expose `router_links` for the prodnxsr0020 Puppet config - bootstrap openvox via cloud-init and firewall each host

unkin-agent added 1 commit 2026-10-12 21:50:16 +11:00

Add mta1/mta2 DMZ mail gateways with WireGuard bootstrap 0ee9dac5b0

unkin-agent added 1 commit 2026-10-12 21:52:07 +11:00

Align comments to tofu 1.10 fmt fe6dc0bc95

unkin-agent commented 2026-10-12 21:55:30 +11:00

Author

Member

Copy Link

Copy Source

• modules/binarylane/main.tf:90 (binarylane_server.gateway) — no lifecycle block, and in this provider a user_data or image diff is an in-place rebuild/reimage (server_resource.go Update: rebuild on user_data/ssh_keys, resize+ChangeImage on image), which wipes the disk. Editing allowed_ips, the template, or the alma-9 slug rebuilds both MTAs → add lifecycle { ignore_changes = [user_data, image] }, plus prevent_destroy = true (replace is the other path: region is RequiresReplace and would change the public IP/PTR/IP reputation).
• modules/binarylane/cloud-init.yaml.tftpl:29-32 — systemctl enable --now wg-quick@wg0 succeeds without a handshake, then /etc/resolv.conf is pointed solely at 198.18.200.7 (tunnel-only) and Puppet is installed/configured. The hub peer lands in a separate PR, so on first boot the tunnel is down, the dnf install of the puppet release rpm fails on DNS, the later puppet config set/enable lines fail, and runcmd never retries → gateway never bootstraps. Add a bounded wait for a handshake after enable --now (e.g. loop on wg show wg0 latest-handshakes non-zero, ~5 min timeout, fail loudly) before touching resolv.conf, or keep the DHCP resolvers until the handshake is confirmed.
• nit: modules/binarylane/main.tf:98 — wg_private_key is in user_data, readable via the BinaryLane API (provider itself reads it back) and persisted in /var/lib/cloud/instance/user-data.txt; with ignore_changes above it will never rotate. Acceptable only because the API token already grants console/root reset; if kept, have Puppet re-key wg0 post-bootstrap and treat the Vault key as bootstrap-only.
• nit: modules/binarylane/.tflint.hcl — deletion is unrelated to the stated why → restore it or note it in the body.
• nit: duplicate identical .terraform.lock.hcl in environments/global/ and modules/binarylane/ → keep one (terragrunt copies the one beside terragrunt.hcl into the cache).

- modules/binarylane/main.tf:90 (`binarylane_server.gateway`) — no `lifecycle` block, and in this provider a `user_data` or `image` diff is an in-place rebuild/reimage (server_resource.go Update: rebuild on `user_data`/`ssh_keys`, resize+ChangeImage on `image`), which wipes the disk. Editing `allowed_ips`, the template, or the `alma-9` slug rebuilds both MTAs → add `lifecycle { ignore_changes = [user_data, image] }`, plus `prevent_destroy = true` (replace is the other path: `region` is RequiresReplace and would change the public IP/PTR/IP reputation). - modules/binarylane/cloud-init.yaml.tftpl:29-32 — `systemctl enable --now wg-quick@wg0` succeeds without a handshake, then `/etc/resolv.conf` is pointed solely at 198.18.200.7 (tunnel-only) and Puppet is installed/configured. The hub peer lands in a separate PR, so on first boot the tunnel is down, the `dnf install` of the puppet release rpm fails on DNS, the later `puppet config set`/`enable` lines fail, and runcmd never retries → gateway never bootstraps. Add a bounded wait for a handshake after `enable --now` (e.g. loop on `wg show wg0 latest-handshakes` non-zero, ~5 min timeout, fail loudly) before touching resolv.conf, or keep the DHCP resolvers until the handshake is confirmed. - nit: modules/binarylane/main.tf:98 — `wg_private_key` is in `user_data`, readable via the BinaryLane API (provider itself reads it back) and persisted in `/var/lib/cloud/instance/user-data.txt`; with `ignore_changes` above it will never rotate. Acceptable only because the API token already grants console/root reset; if kept, have Puppet re-key wg0 post-bootstrap and treat the Vault key as bootstrap-only. - nit: modules/binarylane/.tflint.hcl — deletion is unrelated to the stated why → restore it or note it in the body. - nit: duplicate identical `.terraform.lock.hcl` in `environments/global/` and `modules/binarylane/` → keep one (terragrunt copies the one beside `terragrunt.hcl` into the cache).

unkin-agent added 1 commit 2026-10-12 21:57:42 +11:00

Bootstrap gateways via a retrying wg handshake unit and guard servers against rebuilds 975ff15b6b

unkin-agent commented 2026-10-12 22:00:16 +11:00

Author

Member

Copy Link

Copy Source

• modules/binarylane/cloud-init.yaml.tftpl:28 — printf ... > /etc/resolv.conf writes through the EL9 NM symlink (/etc/resolv.conf -> /run/NetworkManager/resolv.conf); with dns=none NM never repopulates the tmpfs target, so after any reboot resolv.conf is empty/dangling and puppet/artifactapi lookups fail → rm -f /etc/resolv.conf before the printf so it is a persistent regular file.
• nit: cloud-init.yaml.tftpl:27 — systemctl reload NetworkManager is async, so the resolv.conf write can race the reload → use nmcli general reload (synchronous).
• nit: cloud-init.yaml.tftpl:50-51 — runcmd lines do not abort on failure; if the openvox dnf install fails, wg-bootstrap still switches DNS and then loops forever on enable --now puppet (unit missing), with no re-install since runcmd is first-boot only → add rpm -q openvox-agent (or systemctl cat puppet) as the first line of wg-bootstrap, before the DNS switch.

- modules/binarylane/cloud-init.yaml.tftpl:28 — `printf ... > /etc/resolv.conf` writes through the EL9 NM symlink (`/etc/resolv.conf` -> `/run/NetworkManager/resolv.conf`); with `dns=none` NM never repopulates the tmpfs target, so after any reboot resolv.conf is empty/dangling and puppet/artifactapi lookups fail → `rm -f /etc/resolv.conf` before the printf so it is a persistent regular file. - nit: cloud-init.yaml.tftpl:27 — `systemctl reload NetworkManager` is async, so the resolv.conf write can race the reload → use `nmcli general reload` (synchronous). - nit: cloud-init.yaml.tftpl:50-51 — runcmd lines do not abort on failure; if the openvox `dnf install` fails, wg-bootstrap still switches DNS and then loops forever on `enable --now puppet` (unit missing), with no re-install since runcmd is first-boot only → add `rpm -q openvox-agent` (or `systemctl cat puppet`) as the first line of wg-bootstrap, before the DNS switch.

unkin-agent added 1 commit 2026-10-12 22:08:11 +11:00

Split each DMZ into a postfix edge host and a WireGuard vpn host d4d62ffcf4

unkin-agent changed title from Add DMZ mail gateways mta1/mta2 to Add mail edge and vpn hosts to the DMZ VPCs 2026-10-12 22:08:16 +11:00

unkin-agent commented 2026-10-12 22:11:52 +11:00

Author

Member

Copy Link

Copy Source

• nit: modules/binarylane/.tflint.hcl:1 — deleted but not mentioned in the PR body and unrelated to the mail hosts → restore it, or drop it in a separate PR.

- nit: modules/binarylane/.tflint.hcl:1 — deleted but not mentioned in the PR body and unrelated to the mail hosts → restore it, or drop it in a separate PR.

unkin-agent added 1 commit 2026-10-12 22:13:02 +11:00

Route 198.18.0.0/16 and the management host through the vpn hosts cc72e6c8d2

unkin-agent added 1 commit 2026-10-12 22:32:04 +11:00

Give each vpn host its own p2p wg link to prodnxsr0020 29f3a53143

unkin-agent added 1 commit 2026-10-12 22:39:18 +11:00

Address vpn wg links and loopbacks from 198.18.2.160/27 for ospf 04f6fd69d3

unkin-agent added 1 commit 2026-10-13 21:29:10 +11:00

Number the DMZ VPCs 198.18.30.0/24 (syd) and 198.18.31.0/24 (mel) 6a0b6989ae

unkin-agent added 1 commit 2026-10-13 21:34:46 +11:00

Keep VPC route entries clear of the VPC ranges 88f50fc46a

benvin merged commit a8a2f50491 into main 2026-10-14 22:17:09 +11:00

benvin deleted branch benvin/mail-gateways 2026-10-14 22:17:09 +11:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

bug

Something is not working

duplicate

This issue or pull request already exists

enhancement

New feature

help wanted

Need some help

invalid

Something is wrong

question

More information is needed

wontfix

This won't be fixed

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

benvin (BenVincent) unkin-agent (Unkin Agent) unkinben (Ben Vincent)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unkin/terraform-binarylane#2