Parity test harness: golden-file diffs against shorewall + netns integration tests #12

New Issue

2026-07-26T17:03:25+10:00

benvin commented

2026-07-26 17:03:25 +10:00

Problem. Commit 8d9a76c claims coverage of all major shorewall features
with 49 unit tests, but parity is asserted, not demonstrated. iptables→nftables
is not a clean mapping for policy first-match ordering, implicit intra-zone
accept, zone nesting, or ipsec zones — exactly the places subtle divergence
hides.

Proposal.

Golden-file suite: a corpus of shorewall configs (start from the shorewall
docs' worked examples) with the expected compiled nftables ruleset checked
in; CI diffs compiler output against them.
netns integration tests: bring up 2–3 namespaces, apply a config, assert
actual connectivity (accept/drop/dnat/masquerade paths) with real packets.
Priority order for scenarios: policy ordering + implicit intra-zone accept,
DNAT+masquerade interaction, nested zones, negated addresses, rate limiting.

Acceptance. CI fails when compiler output for any golden config changes
without the golden file being updated in the same PR.

**Problem.** Commit 8d9a76c claims coverage of all major shorewall features with 49 unit tests, but parity is asserted, not demonstrated. iptables→nftables is not a clean mapping for policy first-match ordering, implicit intra-zone accept, zone nesting, or ipsec zones — exactly the places subtle divergence hides. **Proposal.** - Golden-file suite: a corpus of shorewall configs (start from the shorewall docs' worked examples) with the expected compiled nftables ruleset checked in; CI diffs compiler output against them. - netns integration tests: bring up 2–3 namespaces, apply a config, assert actual connectivity (accept/drop/dnat/masquerade paths) with real packets. - Priority order for scenarios: policy ordering + implicit intra-zone accept, DNAT+masquerade interaction, nested zones, negated addresses, rate limiting. **Acceptance.** CI fails when compiler output for any golden config changes without the golden file being updated in the same PR.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unkin/tomswall#12