unkin/argocd-apps
3
0
Fork 0
Code Issues 7 Pull Requests 7 Actions Packages Projects Releases Wiki Activity

Deploy Stalwart into the mail namespace #539

Merged
benvin merged 9 commits from benvin/mail-stalwart into main 2026-10-13 20:58:09 +11:00
Conversation 3 Commits 9 Files Changed 13
+596 -2
unkin-agent commented 2026-10-12 00:43:02 +11:00
Member
Copy Link
Copy Source

Stalwart is the mailbox server for the mail migration and runs multi-node from the start, so it needs a coordinator and a shared in-memory store.

  • run Stalwart as a 3-replica StatefulSet with a headless Service, topology spread and a PDB
  • add the mail-valkey ValkeyCluster as in-memory store and coordinator
  • keep data in CNPG Postgres and blobs in cephrgw S3
  • converge settings from plan.ndjson in a PostSync Job, then roll the StatefulSet
  • add a global spam-to-junk Sieve script filing rspamd-tagged mail into Junk and stopping
  • exempt LMTP from the inbound throttles, as all inbound mail comes from the smarthost
  • label pods as smarthost clients and move the client LoadBalancer to 198.18.200.18
Stalwart is the mailbox server for the mail migration and runs multi-node from the start, so it needs a coordinator and a shared in-memory store. - run Stalwart as a 3-replica StatefulSet with a headless Service, topology spread and a PDB - add the `mail-valkey` ValkeyCluster as in-memory store and coordinator - keep data in CNPG Postgres and blobs in cephrgw S3 - converge settings from plan.ndjson in a PostSync Job, then roll the StatefulSet - add a global `spam-to-junk` Sieve script filing rspamd-tagged mail into Junk and stopping - exempt LMTP from the inbound throttles, as all inbound mail comes from the smarthost - label pods as smarthost clients and move the client LoadBalancer to 198.18.200.18
unkin-agent added 1 commit 2026-10-12 00:43:02 +11:00
Deploy Stalwart into the mail namespace
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/kubeconform Pipeline was successful
Details
92a29b4ca9
unkin-agent commented 2026-10-12 00:47:09 +11:00
Author
Member
Copy Link
Copy Source
  • apps/base/mail/stalwart-configmap.yaml:18 — MtaRoute relays to smarthost.mail.svc:25, but no such Service/workload exists in apps/base/mail (or anywhere in the repo) → all non-local outbound mail will fail to deliver; land the smarthost first or point at an existing relay.
  • apps/base/mail/stalwart-apply-job.yaml:3 — nothing restarts the pod after apply, yet the Job comment says listener and store changes only bind on restart. On first sync the lmtp:24, submission:587, metrics:9090 listeners and the S3 BlobStore are not live until someone bounces the pod (LMTP refused, VMPodScrape down, blobs in default store) → add an automated post-apply rollout (e.g. restart step with RBAC from this PR) and add configmap.reloader.stakater.com/reload: stalwart-config to stalwart-deployment.yaml:10 so config.json edits also roll the pod.
  • nit: apps/base/bind-internal/authoritative/unkin-net/a/mail.yaml — separate app and blast radius (bind-internal) from the mail deploy → split into its own PR; keep the wildcard reflection namespace edit here since the Deployment needs it.
- apps/base/mail/stalwart-configmap.yaml:18 — MtaRoute relays to `smarthost.mail.svc:25`, but no such Service/workload exists in `apps/base/mail` (or anywhere in the repo) → all non-local outbound mail will fail to deliver; land the smarthost first or point at an existing relay. - apps/base/mail/stalwart-apply-job.yaml:3 — nothing restarts the pod after `apply`, yet the Job comment says listener and store changes only bind on restart. On first sync the lmtp:24, submission:587, metrics:9090 listeners and the S3 BlobStore are not live until someone bounces the pod (LMTP refused, VMPodScrape down, blobs in default store) → add an automated post-apply rollout (e.g. restart step with RBAC from this PR) and add `configmap.reloader.stakater.com/reload: stalwart-config` to stalwart-deployment.yaml:10 so config.json edits also roll the pod. - nit: apps/base/bind-internal/authoritative/unkin-net/a/mail.yaml — separate app and blast radius (bind-internal) from the mail deploy → split into its own PR; keep the wildcard reflection namespace edit here since the Deployment needs it.
unkin-agent added 1 commit 2026-10-12 00:51:09 +11:00
Restart Stalwart after applying the plan; move mail DNS record out
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/kubeconform Pipeline was successful
Details
7fd1ce8404
unkin-agent commented 2026-10-12 00:54:26 +11:00
Author
Member
Copy Link
Copy Source
  • apps/base/mail/stalwart-configmap.yaml:14 — Certificate upsert uses matchOn: ["certificate"], a polymorphic object, not a scalar; stalwart-cli matchOn keys are scalar properties, so this either errors or fails to match and creates a duplicate Certificate on every sync (and #cert in SystemSettings then points at an unstable id) → give the Certificate a scalar label to match on, or verify against a live apply twice that it stays idempotent.
  • apps/base/mail/stalwart-apply-job.yaml:15 — no activeDeadlineSeconds; if Stalwart is not answering on :8080 the apply init container can hang and wedge the PostSync hook (and the Argo sync) indefinitely → set activeDeadlineSeconds (e.g. 300).
  • nit: apps/base/mail/stalwart-configmap.yaml:9 — Postgres connection uses allowInvalidCerts: true; the CNPG CA is available → trust it via the combined CA bundle and drop the flag.
  • nit: apps/base/mail/stalwart-configmap.yaml:15 — Application resourceUrl pulls webui.zip from releases/latest at runtime, unpinned and tied to GitHub egress → pin the release matching v0.16.25.
- apps/base/mail/stalwart-configmap.yaml:14 — Certificate upsert uses `matchOn: ["certificate"]`, a polymorphic object, not a scalar; stalwart-cli matchOn keys are scalar properties, so this either errors or fails to match and creates a duplicate Certificate on every sync (and `#cert` in SystemSettings then points at an unstable id) → give the Certificate a scalar label to match on, or verify against a live apply twice that it stays idempotent. - apps/base/mail/stalwart-apply-job.yaml:15 — no `activeDeadlineSeconds`; if Stalwart is not answering on :8080 the apply init container can hang and wedge the PostSync hook (and the Argo sync) indefinitely → set `activeDeadlineSeconds` (e.g. 300). - nit: apps/base/mail/stalwart-configmap.yaml:9 — Postgres connection uses `allowInvalidCerts: true`; the CNPG CA is available → trust it via the combined CA bundle and drop the flag. - nit: apps/base/mail/stalwart-configmap.yaml:15 — Application `resourceUrl` pulls `webui.zip` from `releases/latest` at runtime, unpinned and tied to GitHub egress → pin the release matching v0.16.25.
unkin-agent added 1 commit 2026-10-12 00:58:37 +11:00
Verify the CNPG CA for Stalwart's Postgres, pin webui, bound the apply Job
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/kubeconform Pipeline was successful
Details
2b992f771c
unkin-agent commented 2026-10-12 01:01:16 +11:00
Author
Member
Copy Link
Copy Source

No findings.

No findings.
unkin-agent added 1 commit 2026-10-12 21:48:59 +11:00
Run Stalwart as a 3-node StatefulSet coordinated through Valkey
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/kubeconform Pipeline was successful
Details
eb7dbca947
unkin-agent added 1 commit 2026-10-12 21:59:33 +11:00
Label Stalwart pods as smarthost clients
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/kubeconform Pipeline was successful
Details
9a12556841
unkin-agent added 1 commit 2026-10-12 22:16:21 +11:00
Add global spam-to-junk Sieve script
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/kubeconform Pipeline was successful
Details
1e15dc4a98
unkin-agent added 1 commit 2026-10-12 22:24:36 +11:00
Stop after filing spam and exempt LMTP from inbound throttles
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/kubeconform Pipeline was successful
Details
9476d369e1
unkin-agent added 1 commit 2026-10-12 22:32:22 +11:00
Merge remote-tracking branch 'origin/main' into benvin/mail-stalwart
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/kubeconform Pipeline was successful
Details
f6da274325 
# Conflicts:
#	apps/base/mail/kustomization.yaml
unkin-agent added 1 commit 2026-10-12 22:43:47 +11:00
Merge remote-tracking branch 'origin/main' into benvin/mail-stalwart
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/kubeconform Pipeline was successful
Details
869dfb7ccc 
# Conflicts:
#	apps/base/mail/kustomization.yaml
benvin merged commit 82b86072ee into main 2026-10-13 20:58:09 +11:00
benvin deleted branch benvin/mail-stalwart 2026-10-13 20:58:09 +11:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
benvin (BenVincent) unkin-agent (Unkin Agent) unkinben (Ben Vincent)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: unkin/argocd-apps#539
Delete Branch "benvin/mail-stalwart"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?