Establish tomswall codebase and add the control-plane agent #1

2026-07-20T22:12:25+10:00

unkinben commented

2026-07-20 22:12:25 +10:00

Why

The tomswall codebase existed only locally; the remote had just an initial commit. This publishes the codebase and adds tomswall agent, the piece that lets a device consume its compiled config from the tomswallapi control plane — closing the loop so a single fleet-wide rule is actually enforced on every hop.

Changes

Import the existing tomswall codebase (config model, nftables differential engine, shorewall converter, CLI).
Add internal/agent: rendered-config types, an HTTP client (fetch config + report generation), an on-disk last-known-good cache, an on-device DNS resolver for dns sets (honors the device's configured resolver; fail-safe on lookup failure), and the pull-apply-report loop behind a mockable Applier.
Translate the interface-agnostic, address-matched rendered model into native tomswall config via the all:<cidr> any-interface source/dest form, reusing the existing differential engine. Named-set members are inlined as concrete addresses (native nftables set references are a tracked follow-up).
Add the tomswall agent subcommand (flags + TOMSWALL_* env, --once). It never fails closed: if the control plane is unreachable it keeps applying the cached config.
Add PR CI pipelines (build, test, pre-commit) and a pre-commit config — the repo had none.
Add DESIGN.md documenting the control-plane architecture.
Unit tests for translation, cache, and the don't-fail-closed fallback loop.

Follow-up

Release machinery (make patch|minor|major, nfpm RPM, release-on-tag pipeline) will land in a separate PR.

## Why The tomswall codebase existed only locally; the remote had just an initial commit. This publishes the codebase and adds `tomswall agent`, the piece that lets a device consume its compiled config from the tomswallapi control plane — closing the loop so a single fleet-wide rule is actually enforced on every hop. ## Changes - Import the existing tomswall codebase (config model, nftables differential engine, shorewall converter, CLI). - Add `internal/agent`: rendered-config types, an HTTP client (fetch config + report generation), an on-disk last-known-good cache, an on-device DNS resolver for dns sets (honors the device's configured resolver; fail-safe on lookup failure), and the pull-apply-report loop behind a mockable Applier. - Translate the interface-agnostic, address-matched rendered model into native tomswall config via the `all:<cidr>` any-interface source/dest form, reusing the existing differential engine. Named-set members are inlined as concrete addresses (native nftables set references are a tracked follow-up). - Add the `tomswall agent` subcommand (flags + TOMSWALL_* env, --once). It never fails closed: if the control plane is unreachable it keeps applying the cached config. - Add PR CI pipelines (build, test, pre-commit) and a pre-commit config — the repo had none. - Add DESIGN.md documenting the control-plane architecture. - Unit tests for translation, cache, and the don't-fail-closed fallback loop. ## Follow-up Release machinery (make patch|minor|major, nfpm RPM, release-on-tag pipeline) will land in a separate PR.

unkinben added 5 commits 2026-07-20 22:12:25 +10:00

Initial scaffold for tomswall 2a3eb3b04d

Spiritual successor to shorewall — manages nftables directly via
google/nftables. Reads a single YAML config covering zones, interfaces,
hosts, policy, rules, snat, and named portgroups. Computes differential
changes against the running nftables state and applies them atomically.
Supports detecting and purging rules added outside of tomswall.

Add comprehensive nftables compiler with shorewall feature parity 8d9a76c751

Rewrites the compiler from ~440 to ~1700 lines covering all major shorewall
firewall features: loopback, conntrack fast-path, anti-spoof, DHCP, intra-zone,
blacklist/whitelist, conntrack notrack, tunnels (13 types), rules with sections,
DNAT/redirect, SNAT/masquerade, static NAT, policies with zone exclusions,
MSS clamping, rate limiting, connection limiting, negated addresses, ICMP type
matching, TCP RST reject, user/UID matching, mark match/set, NFQUEUE, NONAT,
and policy-level rate/conn limiting.

Adds full config types for all shorewall subsystems (mangle, accounting, maclist,
netmap, providers, tunnels, conntrack, blrules, proxyarp/ndp, routes, tc, secmarks),
shorewall migration tooling, expanded CLI commands, expression-level diff engine,
and 49 unit tests.

Add tomswall agent (control-plane pull mode) e0f54ef320

Add `tomswall agent`: it pulls this device's compiled config from tomswallapi,
differentially applies it, and reports the applied generation. It caches the
last known-good config and, when the control plane is unreachable, keeps
applying that cache — it never fails closed.

- internal/agent: rendered-config types, HTTP client (fetch + status report),
  on-disk cache, on-device DNS resolver for dns sets (honors the device's
  configured resolver, fail-safe on lookup failure), and the pull-apply-report
  loop behind a mockable Applier.
- Translate the interface-agnostic, address-matched rendered model into native
  tomswall config using the "all:<cidr>" any-interface source/dest form, reusing
  the existing differential engine. Named-set members are inlined as concrete
  addresses (native nft set references are a tracked follow-up).
- cmd/tomswall: wire the `agent` subcommand (flags + TOMSWALL_* env, --once).
- Unit tests: translation, cache, and the don't-fail-closed fallback loop.
- Add DESIGN.md documenting the control-plane architecture.

Add PR CI pipelines (build, test, pre-commit) f61773ac88

Merge Gitea repo initialization

ci/woodpecker/pr/pre-commit Pipeline failed

Details

ci/woodpecker/pr/build Pipeline was successful

Details

ci/woodpecker/pr/test Pipeline was successful

Details

9993c36d74

unkinben added 1 commit 2026-07-20 22:19:12 +10:00

Fix end-of-file newline (pre-commit)

ci/woodpecker/pr/test Pipeline was successful

Details

ci/woodpecker/pr/build Pipeline was successful

Details

ci/woodpecker/pr/pre-commit Pipeline was successful

Details

174b2f93b9

benvin merged commit 5116fd83b4 into main

2026-07-20 22:24:44 +10:00

benvin deleted branch benvin/agent

2026-07-20 22:24:45 +10:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unkin/tomswall#1