Expand ASN groups from the iplocate ip-to-asn database
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful

Replace the (unconfirmed) iplocate API expander with a database-backed one that
reads the iplocate ip-to-asn CSV (network,asn,...) proxied through the artifactapi
github remote. It downloads and indexes the whole DB once (ASN -> CIDRs), serves
every asn address group from the in-memory index, and rebuilds on a 24h TTL;
refresh failures keep the last-good index (fail-safe). No API key needed.

- Add IPLocateDB expander (zip + CSV parsing, ASN normalization).
- Wire it in main (TOMSWALLAPI_IPLOCATE_DB_URL overrides the default artifactapi
  URL); remove the dead API client.
- Unit tests: CSV indexing (incl. quoted org fields), zip extraction, missing
  columns, and ASN normalization.
This commit is contained in:
benvin
2026-07-21 22:36:19 +10:00
parent dc3b2ecdb9
commit 92213090fe
5 changed files with 272 additions and 106 deletions
+6 -1
View File
@@ -23,8 +23,12 @@ type Config struct {
// AgentToken guards the per-device config endpoint (used by tomswall agents).
AgentToken string
// IPLocateAPIKey is used to expand ASN address groups into prefixes.
// IPLocateAPIKey is used to expand ASN address groups into prefixes via the
// iplocate API (legacy path). The DB expander below is preferred.
IPLocateAPIKey string
// IPLocateDBURL points at the iplocate ip-to-asn CSV database (proxied via
// artifactapi). Empty uses the built-in default.
IPLocateDBURL string
}
// Load reads configuration from the environment, applying defaults.
@@ -40,6 +44,7 @@ func Load() (*Config, error) {
WriteToken: os.Getenv("TOMSWALLAPI_WRITE_TOKEN"),
AgentToken: os.Getenv("TOMSWALLAPI_AGENT_TOKEN"),
IPLocateAPIKey: os.Getenv("TOMSWALLAPI_IPLOCATE_API_KEY"),
IPLocateDBURL: os.Getenv("TOMSWALLAPI_IPLOCATE_DB_URL"),
}
if c.DBName == "" {
return nil, fmt.Errorf("TOMSWALLAPI_DB_NAME must not be empty")