Add central ASN address-group expander #2

2026-07-19T21:44:44+10:00

unkinben commented

2026-07-19 21:44:44 +10:00

Why

ASN-based rules (e.g. net:+asn_cloudflare) need the source ASNs turned into concrete prefixes to match on. Expansion is centralized so one iplocate key is used and every device gets identical, consistent membership; devices never call the upstream provider.

Changes

Add the asnexpand package: an Expander interface, an endpoint-configurable, response-tolerant iplocate client, and a background Refresher that re-expands each asn group on its TTL (default 24h, per-group configurable via refresh).
Union each group's ASNs into a deduped prefix set and store it in new resolved/resolved_at columns (migration 0002), verified against Postgres 17.
Fail-safe: a lookup error or a genuinely-empty expansion keeps the last-good set rather than emptying it.
Fold resolved prefixes into the compiler's rendered set members; membership churn bumps the generation but never rewrites rules.
Start the expander from main only when TOMSWALLAPI_IPLOCATE_API_KEY is set; otherwise asn sets stay empty and inert with a warning.
Unit tests: TTL parsing, due-checks, and union/dedup/error propagation.

Note

iplocate's documented ASN data is returned per IP lookup, so the exact ASN→prefixes endpoint depends on the account/plan. The client is written to be pointed at the correct endpoint once the key is provided; it is tolerant of prefixes/routes/cidrs response shapes.

## Why ASN-based rules (e.g. `net:+asn_cloudflare`) need the source ASNs turned into concrete prefixes to match on. Expansion is centralized so one iplocate key is used and every device gets identical, consistent membership; devices never call the upstream provider. ## Changes - Add the `asnexpand` package: an `Expander` interface, an endpoint-configurable, response-tolerant iplocate client, and a background `Refresher` that re-expands each asn group on its TTL (default 24h, per-group configurable via `refresh`). - Union each group's ASNs into a deduped prefix set and store it in new `resolved`/`resolved_at` columns (migration 0002), verified against Postgres 17. - Fail-safe: a lookup error or a genuinely-empty expansion keeps the last-good set rather than emptying it. - Fold resolved prefixes into the compiler's rendered set members; membership churn bumps the generation but never rewrites rules. - Start the expander from main only when `TOMSWALLAPI_IPLOCATE_API_KEY` is set; otherwise asn sets stay empty and inert with a warning. - Unit tests: TTL parsing, due-checks, and union/dedup/error propagation. ## Note iplocate's documented ASN data is returned per IP lookup, so the exact ASN→prefixes endpoint depends on the account/plan. The client is written to be pointed at the correct endpoint once the key is provided; it is tolerant of `prefixes`/`routes`/`cidrs` response shapes.

unkinben added 1 commit 2026-07-19 21:44:45 +10:00

Add central ASN address-group expander

ci/woodpecker/pr/pre-commit Pipeline was successful

Details

ci/woodpecker/pr/build Pipeline was successful

Details

ci/woodpecker/pr/test Pipeline was successful

Details

d52c3ee76e

Expand asn address groups to concrete prefixes centrally (one iplocate key,
consistent fleet-wide) and refresh them on a per-group TTL (default 24h). A
background Refresher scans for due groups, unions each group's ASNs to a deduped
prefix set, and writes them to a new resolved/resolved_at column (migration
0002). Fail-safe: a lookup error or empty expansion keeps the last-good set,
never emptying it. The compiler folds resolved prefixes into the rendered set
members; membership churn bumps the generation but never rewrites rules. The
iplocate client is endpoint-configurable and response-tolerant, documented as
needing endpoint/key confirmation. Unit tests cover TTL parsing, due-checks, and
union/dedup/error propagation with a fake expander.

benvin merged commit 3afaab7d6c into main

2026-07-19 22:10:33 +10:00

benvin deleted branch benvin/asn-expander

2026-07-19 22:10:33 +10:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unkin/tomswallapi#2