Replace the (unconfirmed) iplocate API expander with a database-backed one that
reads the iplocate ip-to-asn CSV (network,asn,...) proxied through the artifactapi
github remote. It downloads and indexes the whole DB once (ASN -> CIDRs), serves
every asn address group from the in-memory index, and rebuilds on a 24h TTL;
refresh failures keep the last-good index (fail-safe). No API key needed.
- Add IPLocateDB expander (zip + CSV parsing, ASN normalization).
- Wire it in main (TOMSWALLAPI_IPLOCATE_DB_URL overrides the default artifactapi
URL); remove the dead API client.
- Unit tests: CSV indexing (incl. quoted org fields), zip extraction, missing
columns, and ASN normalization.
The compiler now projects the global NAT intents through each device's bindings
into the rendered config:
- snat/masquerade: renders on a device that binds the egress zone (and, when the
source is a zone, that zone too), resolving the egress interface — this
auto-scopes masquerade to edge devices. A literal-CIDR source needs only the
egress binding.
- netmap: renders on the device its anchor (device:zone|interface) names,
resolving a zone anchor to its bound interface.
- 1:1 nat: renders on the device it is bound to.
Adds RenderedSNAT/RenderedNetmap/RenderedNAT to the rendered config, fetches the
tiers in Compile, and unit-tests binding-scoping (edge vs interior/other device).
Devices carry a reachable_prefixes set (migration 0004) reported by their agent
from the kernel FIB, via a new agent-authed POST /devices/{name}/routes endpoint.
This is scoping/observability data, so it does not bump the config generation and
is exposed on the device resource.
It deliberately does NOT limit which rules a device enforces: the compiler still
emits every applicable rule on every enforcing device. The interface-agnostic,
address-matched rule form is correct under ECMP precisely because it does not
depend on which device is on the path, and most routers hold a default route
anyway — so reachability could not meaningfully narrow the rule set. The reported
FIB is kept for fleet visibility and future zone-origin validation.
- Add snat/masquerade, netmap, and 1:1 nat as stored, terraformable resources:
migration 0003, model types, store CRUD (id-keyed, generation-bumping), and
REST handlers. These are the global-intent/device-resolved NAT tier; compiler
rendering of NAT into per-device configs is a tracked follow-up.
- Add a testcontainers-backed store integration suite exercising the CRUD
lifecycle, generation bumping, source/dest grammar validation, and FK cascade
against a real Postgres. It self-skips under 'go test -short' (the CI path) so
a container runtime is only needed for the full run.
Add GET /{name} and DELETE /{name} for zones, address-groups, portgroups,
fabrics, devices, and bindings, plus GET /rules/{id}, with the matching store
Get/Delete methods (deletes bump the generation and 404 on no-match). This gives
the resources full read/update/delete lifecycle so the Terraform provider can
manage them.
Expand asn address groups to concrete prefixes centrally (one iplocate key,
consistent fleet-wide) and refresh them on a per-group TTL (default 24h). A
background Refresher scans for due groups, unions each group's ASNs to a deduped
prefix set, and writes them to a new resolved/resolved_at column (migration
0002). Fail-safe: a lookup error or empty expansion keeps the last-good set,
never emptying it. The compiler folds resolved prefixes into the rendered set
members; membership churn bumps the generation but never rewrites rules. The
iplocate client is endpoint-configurable and response-tolerant, documented as
needing endpoint/key confirmation. Unit tests cover TTL parsing, due-checks, and
union/dedup/error propagation with a fake expander.
Project the fleet-global model through a device's bindings into a rendered,
interface-agnostic config: rules compile to saddr/daddr forward matches with no
iif/oif so they are correct under FRR/ECMP. Firewalls always enforce; routers
enforce only when their fabric opts into defense-in-depth. Referenced address
groups are emitted as named sets carrying their source (static CIDRs, dns FQDNs,
or asn numbers) so membership churns out-of-band without a rule reload. Wire
GET /devices/{name}/config to compile and serve YAML, generation-stamped. Add
portgroups/policies/settings store methods and portgroup CRUD. Pure Render is
unit-tested for enforcement gating, ASN set emission, and resolver precedence.
Add the store package (pgx-backed repository) with CRUD for the core resources
the compiler needs: fabrics, zones, address groups, portgroups, devices,
bindings, rules, policies. Every mutation bumps a global config generation.
Wire real JSON CRUD handlers with token auth, source/dest grammar validation on
rule create, and the agent status-report endpoint. Migration 0001 verified
against Postgres 17.
benvin
benvin
unkinben
gitadmin