5 Commits

Author SHA1 Message Date
unkinben 5b3058e75e fix(postfix): use hash: transport_maps with postmap init container
- Splits hash-type map files into a separate postfix-maps ConfigMap
- Adds postmap init container that builds .db files from all maps into
  a postfix-db emptyDir, which is then subPath-mounted per map in the
  main container
- Updates transport_maps in main.cf to hash:/etc/postfix/transport
2026-05-24 20:38:27 +10:00
unkinben ebc21b9fa1 fix(postfix): use .unkin.net subdomain pattern in transport map
per transport(5), .domain matches all subdomains. main.unkin.net was
an exact-match entry; .unkin.net covers it and any future subdomains.
2026-05-24 20:35:19 +10:00
unkinben d57c1db5f8 fix(mailgateway): pin postfix-external LoadBalancer to 198.18.199.1 2026-05-24 19:54:08 +10:00
unkinben 942e23c146 fix(mailgateway): add purelb dmz service-group annotation to LoadBalancer 2026-05-24 19:52:16 +10:00
unkinben b09cd1628d feat(postfix): deploy postfix MTA and rspamd spam filter
- mailgateway namespace with Deployment + HPA (2-6 replicas)
- rspamd Deployment + HPA (2-6 replicas) with milter interface
- postfix configured to relay inbound mail to stalwart via transport maps
- rspamd milter on port 11332 for spam scanning and DKIM signing
- DKIM keys stored in Vault at kubernetes/namespace/mailgateway/default/dkim-keys
- TLS cert via cert-manager (vault-issuer) for mail.main.unkin.net
- rspamd web UI exposed via Traefik Gateway at rspamd.k8s.syd1.au.unkin.net
- postfix external LoadBalancer service for inbound MX on port 25
- Add full main.cf and master.cf as ConfigMap resources mounted via subPath
  - main.cf: relay-only gateway config, texthash: transport maps, rspamd milter
  - master.cf: standard smtp + submission (587, TLS required) + internal processes
- MAILNAME/MY_NETWORKS/MY_DESTINATION env vars kept in sync with main.cf
- LOG_TO_STDOUT=1 for k8s log collection
2026-05-24 12:46:28 +10:00
355 changed files with 1335 additions and 144665 deletions
-273
View File
@@ -1,273 +0,0 @@
---
description: Pull master, read open issues, pick one, branch, implement, test, commit, PR, and comment.
---
# Solve a Gitea Issue
## Current repo state
```!
git status --short
echo "Current branch: $(git branch --show-current)"
echo "Remote: $(git remote get-url origin 2>/dev/null || echo 'none')"
```
## Open issues (with full body)
```!
echo "Fetching open issues..."
issue_ids=$(tea issues list --output simple 2>/dev/null | awk 'NF && $1 ~ /^[0-9]+$/ {print $1}')
if [ -z "$issue_ids" ]; then
echo "No open issues found (or tea is not logged in)."
else
for id in $issue_ids; do
echo ""
echo "══════════════════════════════════════"
tea issues view "$id" --fields index,title,body 2>/dev/null \
|| tea issue "$id" 2>/dev/null \
|| echo " (could not read issue #$id)"
echo "══════════════════════════════════════"
done
fi
```
---
## Your task
Follow these steps **in order**. Do not skip steps.
### 1 — Choose an issue
Present the issues above to the user as a numbered list (index, one-line title). Ask which one to work on. Wait for the answer before continuing.
### 2 — Sync master
```bash
git checkout master
git pull
```
Confirm you are on master and up to date.
### 3 — Create a branch
Name the branch `benvin/issue-<N>-<short-slug>` where `<short-slug>` is 24 kebab-case words from the issue title.
```bash
git checkout -b benvin/issue-<N>-<slug>
```
### 4 — Read the issue in full
Re-read the full issue body shown above. If any part is ambiguous, state your interpretation before coding.
**If you discover other problems while working:** do NOT solve them inline. Create a new Gitea issue with `tea issues create --title "..." --description "..."` and stay focused on the assigned issue.
### 5 — Implement the solution
Make the code changes needed to resolve the issue. Follow the conventions already in the repo:
- `main.py` route handlers each contain a single function call; logic lives in submodules.
- No comments unless the WHY is non-obvious.
- No new files unless the issue or architecture requires it.
- Security: no command injection, XSS, SQL injection, or secrets in code.
- **For performance improvements:** implement at the most generic call site possible so the fix applies to all current and future implementations, not just the one being tested.
### 6 — Update tests
Add or update tests that cover the new behaviour. Tests live in `tests/`. Check existing test structure before writing new ones — mirror the style and fixture patterns already in use.
### 7 — Update README
If the feature introduces new config keys, endpoints, or user-facing behaviour, document it in `README.md`. Keep additions concise — follow the existing section style.
### 8 — Run the full test suite
```bash
make test
```
All tests must pass. If any fail, fix them before proceeding. Do not skip or suppress failing tests.
### 9 — Live Docker test (new package type only)
**Skip this step if the issue does not add a new remote package type.**
If the issue adds a new package type (e.g. `deb`, `conda`, `cargo`, `rubygems`, or any type not already in `remotes.yaml`), do the following before committing.
#### 9a — Add a real test remote to remotes.yaml
Append a valid, publicly accessible remote of the new type to `remotes.yaml`. Use a real upstream URL and patterns that cover both an immutable file (versioned artifact) and a mutable file (index/metadata). Add a comment explaining which URLs to use for manual testing.
#### 9b — Start the stack
```bash
make docker-up
```
Wait until `curl -s http://localhost:8000/health` returns `{"status":"healthy"}`.
#### 9c — Test a mutable file (first fetch — cache miss)
Download the index or metadata file for the new remote. Confirm:
- HTTP 200
- `X-Artifact-Source: remote` header (or equivalent log line confirming a cache miss)
- Content looks correct (not empty, not an error page)
```bash
curl -sv "http://localhost:8000/api/v1/remote/<new-remote>/<mutable-path>" 2>&1 | grep -E "< HTTP|X-Artifact"
```
#### 9d — Test a mutable file (second fetch — cache hit)
Repeat the exact same request. Confirm:
- HTTP 200
- `X-Artifact-Source: cache`
```bash
curl -sv "http://localhost:8000/api/v1/remote/<new-remote>/<mutable-path>" 2>&1 | grep -E "< HTTP|X-Artifact"
```
#### 9e — Test an immutable file (first fetch — cache miss)
Download a versioned/immutable artifact. Confirm HTTP 200 and a cache-miss log line.
```bash
curl -sv "http://localhost:8000/api/v1/remote/<new-remote>/<immutable-path>" 2>&1 | grep -E "< HTTP|X-Artifact"
```
#### 9f — Test an immutable file (second fetch — cache hit)
Repeat. Confirm `X-Artifact-Source: cache`.
#### 9g — Check container logs
```bash
make docker-logs
```
Scan for:
- `Cache MISS` on first fetches, `Cache HIT` on second fetches
- `Cache ADD SUCCESS` with correct sizes
- No unhandled exceptions or ERROR lines
#### 9h — Exercise package-type tooling against the proxy
Use the native tooling for this package type to verify end-to-end behaviour. Examples:
| Package type | Command |
|---|---|
| `pypi` | `uv run --index-url http://localhost:8000/api/v1/remote/<remote>/simple <tool>` |
| `npm` | `npm install --registry http://localhost:8000/api/v1/remote/<remote>/ <pkg>` |
| `helm` | `helm repo add test http://localhost:8000/api/v1/remote/<remote> && helm search repo test && helm template test/<chart>` |
| `alpine` | `apk fetch --repository http://localhost:8000/api/v1/remote/<remote>/<branch>/<arch> <pkg>` |
| `rpm` | `dnf install --repofrompath ... <pkg>` or `repoquery` |
| `generic` | `curl` / `wget` as appropriate |
Confirm the tool resolves and downloads correctly through the proxy.
#### 9i — Tear down
```bash
make docker-down
```
Fix any failures found during 9b9h before moving on.
### 9.5 — Performance issues: measure before/after and gate the PR
**Skip this step if the issue is not a performance improvement.**
For performance issues, a PR is only warranted if there is a measurable gain. Use the Docker stack to compare before and after.
#### 9.5a — Baseline measurement (before)
Start the stack with the **unmodified** code (temporarily revert your change):
```bash
make docker-up
```
Warm or clear the cache as appropriate, then measure the relevant metric — e.g. concurrent request latency during a slow operation, response time for a specific endpoint, or throughput. Record the numbers.
#### 9.5b — Apply your change and rebuild
```bash
make docker-up # rebuilds the image
```
Repeat exactly the same measurement. Record the numbers.
#### 9.5c — Decide
If the improvement is not clearly measurable, **do not open a PR**. Instead:
1. Update the issue with your findings.
2. Note any conditions under which the improvement would be observable.
3. Skip steps 1114.
If the improvement is clear, proceed with the commit and PR. Include the before/after numbers in the PR description and the issue comment.
#### 9.5d — Tear down
```bash
make docker-down
```
### 10 — Build the wheel (smoke check)
```bash
uv build --wheel
```
Confirm the build succeeds.
### 11 — Stage and commit
Stage only the files you changed. Do not use `git add -A` or `git add .` — list files explicitly. Run:
```bash
git add <file1> <file2> ...
git commit
```
The commit message must:
- Start with a conventional-commit prefix (`feat:`, `fix:`, `refactor:`, `chore:`, etc.)
- Summarise the change in ≤ 72 characters on the first line
- Optionally include a short body explaining *why* (not *what*)
If the pre-commit hook auto-fixes files, re-stage the fixed files and commit again.
### 12 — Push the branch
```bash
git push origin <branch-name>
```
### 13 — Open a pull request
```bash
tea pulls create \
--base master \
--head <branch-name> \
--title "<same as commit subject>" \
--description "Closes #<N>\n\n## Summary\n<bullet points>\n\n## Test plan\n<what was verified>"
```
### 14 — Comment on the issue
```bash
tea comment <N> "<resolution comment>"
```
The comment must cover:
- **How it was resolved** — what changed and why
- **Issues encountered** — any non-obvious problems hit during implementation
- **Potential future improvements** — what could be done next
### 15 — Return to master
```bash
git checkout master
```
Report the PR URL and a one-sentence summary to the user.
-2
View File
@@ -7,7 +7,6 @@ repos:
- id: check-json
- id: check-added-large-files
args: ['--maxkb=500']
exclude: '^schemas/'
- id: check-merge-conflict
- id: check-shebang-scripts-are-executable
- id: check-symlinks
@@ -20,7 +19,6 @@ repos:
- id: end-of-file-fixer
- id: forbid-new-submodules
- id: pretty-format-json
args: ['--autofix']
- id: trailing-whitespace
# YAML linting
+1 -1
View File
@@ -3,7 +3,7 @@ when:
steps:
- name: kubeconform
image: git.unkin.net/unkin/almalinux9-kubetest:20260606
image: git.unkin.net/unkin/almalinux9-kubetest:20260319
commands:
- make kubeconform
backend_options:
+1 -1
View File
@@ -3,7 +3,7 @@ when:
steps:
- name: pre-commit
image: git.unkin.net/unkin/almalinux9-base:20260606
image: git.unkin.net/unkin/almalinux9-base:20260308
commands:
- uvx pre-commit run --all-files
backend_options:
-261
View File
@@ -1,261 +0,0 @@
# AGENTS.md
## Project Overview
This is an **ArgoCD GitOps repository** that manages Kubernetes applications for the `au-syd1` cluster using a Kustomize + Helm pattern. Applications are deployed via ArgoCD ApplicationSets that watch directory patterns in this repo.
The migration pattern for this repo is: **Terragrunt/Terraform → ArgoCD** (see `migration.md` for full guide).
---
## Essential Commands
```bash
# Build and render manifests for a path (outputs to manifests/<path>/)
make build apps/overlays/au-syd1/<app-name>
make build clusters/au-syd1/bootstrap
# Validate all apps and clusters with kubeconform
make kubeconform
# Clean generated manifests
make clean
# Quick build + inspect without persisting output
kustomize build --enable-helm apps/overlays/au-syd1/<app-name>
# Check all resource kinds produced by an overlay
kustomize build --enable-helm apps/overlays/au-syd1/<app-name> | grep "^kind:" | sort | uniq -c
# Run pre-commit checks against all files
uvx pre-commit run --all-files
```
---
## Directory Structure
```
argocd-apps/
├── argocd/
│ ├── applicationsets/ # ArgoCD ApplicationSet definitions (platform.yaml, storage.yaml)
│ └── projects/ # ArgoCD AppProject definitions (platform.yaml, storage.yaml)
├── apps/
│ ├── base/ # Base Kustomize resources per app (no cluster-specific config)
│ │ └── <app-name>/
│ │ ├── kustomization.yaml
│ │ ├── namespace.yaml
│ │ ├── vaultauth.yaml # (if Vault-managed secrets)
│ │ └── vaultstaticsecret.yaml
│ └── overlays/
│ └── au-syd1/ # Cluster-specific overlays
│ └── <app-name>/
│ ├── kustomization.yaml # references base + helmCharts
│ └── values.yaml # Helm values for this cluster
├── clusters/
│ └── au-syd1/
│ ├── apps/ # Entry point: references apps/base (ArgoCD app-of-apps)
│ └── bootstrap/ # ArgoCD install + initial Application manifest
├── ci/
│ ├── validate-apps.sh # kubeconform over apps/overlays/*/kustomization.yaml
│ ├── validate-clusters.sh # kubeconform over clusters/*/kustomization.yaml
│ └── validate-no-secrets.sh # pre-commit hook: blocks plain Kubernetes Secrets
└── sources/ # Reference sources (Terraform configs, upstream charts, etc.)
└── terraform-k8s/ # Original Terraform configs — reference when migrating
```
---
## Adding a New Application
Follow these 10 steps (detailed in `migration.md`):
### 1. Create base resources
```
apps/base/<app-name>/
├── kustomization.yaml
├── namespace.yaml
├── vaultauth.yaml # if needed
└── vaultstaticsecret.yaml # if needed
```
### 2. Create cluster overlay
```
apps/overlays/au-syd1/<app-name>/
├── kustomization.yaml
└── values.yaml
```
**Overlay kustomization.yaml pattern:**
```yaml
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../../base/<app-name>
helmCharts:
- name: <chart-name>
repo: <helm-repo-url>
version: "<version>"
releaseName: <release-name>
namespace: <namespace>
valuesFile: values.yaml
```
### 3. Register in ApplicationSet
Add a directory entry to `argocd/applicationsets/platform.yaml` (or `storage.yaml` for `csi-*` apps):
```yaml
- path: apps/overlays/*/<app-name>
```
### 4. Update AppProject
In `argocd/projects/platform.yaml` (or `storage.yaml`):
- Add the Helm repo URL to `sourceRepos`
- Add the namespace to `destinations`
- Add any required cluster-scoped resource types to `clusterResourceWhitelist`
### 5. Validate
```bash
kustomize build --enable-helm apps/overlays/au-syd1/<app-name>
make kubeconform
```
---
## Secret Management
**Plain Kubernetes `Secret` objects are blocked** by the pre-commit hook. Use Vault Operator CRDs instead:
### VaultAuth template
```yaml
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
name: default
namespace: <namespace>
spec:
method: kubernetes
mount: k8s/au/syd1
vaultConnectionRef: vso-system/default
allowedNamespaces:
- <namespace>
kubernetes:
role: <role>
serviceAccount: <service-account>
audiences:
- vault
tokenExpirationSeconds: 600
```
### VaultStaticSecret template
```yaml
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: <secret-name>
namespace: <namespace>
spec:
vaultAuthRef: default
mount: kv
type: kv-v2
path: kubernetes/namespace/<namespace>/default/<secret-name>
refreshAfter: 5m
destination:
name: <k8s-secret-name>
create: true
overwrite: true
hmacSecretData: true
```
---
## YAML Conventions
- **2-space indentation** (enforced by yamllint)
- All files must end with a newline (`end-of-file-fixer`)
- No trailing whitespace
- YAML linting uses relaxed rules with `line-length: disable` (long base64/URLs are fine)
- yamllint ignores `chart` directories (vendored Helm charts)
- `---` document separator at top of every YAML file
- Multiple documents in one file are allowed (e.g., `vaultstaticsecret.yaml` often contains multiple secrets)
---
## Kubernetes Labels Pattern
Use standard `app.kubernetes.io/*` labels consistently:
```yaml
labels:
app.kubernetes.io/component: <component>
app.kubernetes.io/instance: <release-name>
app.kubernetes.io/name: <app-name>
app.kubernetes.io/version: <version>
```
---
## Resource Naming Conventions
Files in `apps/base/<app-name>/` follow the pattern:
```
<kind>_<name>.yaml
```
Examples:
- `deployment_puppetserver-master.yaml`
- `cronjob_g10k-code.yaml`
- `configmap_puppetboard-config.yaml`
- `horizontalpodautoscaler_puppetserver-compilers-autoscaler.yaml`
- `service_puppet-headless.yaml`
---
## Helm Chart Vendoring
Some overlays vendor Helm charts locally under `apps/overlays/au-syd1/<app-name>/charts/<chart-name>/`. When a chart is vendored, the overlay's `kustomization.yaml` references the local path. When not vendored, it references the OCI or HTTP repo directly.
Current Kubernetes target version: **1.33.7** (used by kubeconform in CI).
---
## Project Boundaries
| Project | ApplicationSet | App pattern |
|------------|---------------------------|--------------------------|
| `platform` | `argocd/applicationsets/platform.yaml` | Named apps (cert-manager, puppet, woodpecker, etc.) |
| `storage` | `argocd/applicationsets/storage.yaml` | `csi-*` apps |
The `clusters/au-syd1/apps/` entry-point is deployed as a standalone ArgoCD `Application` (not an ApplicationSet) called `au-syd1-apps`.
---
## CI / Pre-commit Hooks
Runs on every PR via Woodpecker CI (`.woodpecker/`):
| Check | Tool | Trigger |
|---|---|---|
| YAML lint + general file checks | `pre-commit` (yamllint + pre-commit-hooks) | PR |
| No plain Secrets | `ci/validate-no-secrets.sh` | PR (staged files) |
| Kubernetes manifest validation | `kubeconform` via `make kubeconform` | PR |
kubeconform skips: `CustomResourceDefinition`, `GpuDevicePlugin` (for apps validation).
---
## Git Workflow
- Branch naming: `benvin/<app-name>` (user prefix)
- **Never `git add .`** — add only relevant files explicitly
- If pre-commit modifies files, `git add -u` then `git commit --amend --no-edit`
- Use `git push --force-with-lease` after amending
---
## Security Policies
- `reloader.stakater.com/auto: "true"` annotation triggers rolling restarts on ConfigMap/Secret changes
- Security contexts follow least-privilege: `drop: [all]` then add only required capabilities
- `fsGroup: 999` on pod security context for Puppet workloads
- `runAsUser: 0` is used only for init containers that need to set file permissions, then regular containers run as non-root
+1 -5
View File
@@ -1,4 +1,4 @@
.PHONY: build clean schemas
.PHONY: build clean
# Build a kustomization path to manifests directory
# Usage: make build clusters/au-syd1/bootstrap
@@ -6,10 +6,6 @@ build:
@mkdir -p manifests/$(filter-out $@,$(MAKECMDGOALS))
@kustomize build --enable-helm $(filter-out $@,$(MAKECMDGOALS)) --output manifests/$(filter-out $@,$(MAKECMDGOALS))
# Generate JSON schemas from CRDs and Kubernetes swagger spec (run manually, results committed)
schemas:
@ci/generate-schemas.sh schemas
# kubeconform
kubeconform:
@ci/validate-apps.sh && \
-45
View File
@@ -1,45 +0,0 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: age-api
namespace: age-api
spec:
replicas: 1
selector:
matchLabels:
app: age-api
template:
metadata:
annotations:
reloader.stakater.com/auto: "true"
labels:
app: age-api
spec:
containers:
- name: age-api
image: git.unkin.net/unkin/age-api:v0.1.0
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
name: http
protocol: TCP
env:
- name: CONFIG_PATH
value: /etc/age-api/config.yaml
resources:
limits:
cpu: 100m
memory: 64Mi
requests:
cpu: 10m
memory: 32Mi
volumeMounts:
- mountPath: /etc/age-api/config.yaml
name: config
subPath: config.yaml
restartPolicy: Always
volumes:
- name: config
configMap:
name: age-api-config
-49
View File
@@ -1,49 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: age-api-http-redirect
namespace: age-api
spec:
hostnames:
- age-api.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: age-api
sectionName: http
rules:
- filters:
- type: RequestRedirect
requestRedirect:
scheme: https
statusCode: 301
matches:
- path:
type: PathPrefix
value: /
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: age-api
namespace: age-api
spec:
hostnames:
- age-api.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: age-api
sectionName: https
rules:
- backendRefs:
- group: ""
kind: Service
name: age-api
port: 80
weight: 1
matches:
- path:
type: PathPrefix
value: /
-17
View File
@@ -1,17 +0,0 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
- gateway.yaml
- httproute.yaml
- namespace.yaml
- service.yaml
configMapGenerator:
- name: age-api-config
files:
- config.yaml=resources/config.yaml
options:
disableNameSuffixHash: true
-7
View File
@@ -1,7 +0,0 @@
people:
- name: jaidi
birthtime: 1773135720
- name: ben
birthtime: 559663200
- name: sudaporn
birthtime: 686757600
-17
View File
@@ -1,17 +0,0 @@
---
apiVersion: v1
kind: Service
metadata:
name: age-api
namespace: age-api
spec:
internalTrafficPolicy: Cluster
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
selector:
app: age-api
sessionAffinity: None
type: ClusterIP
@@ -2,40 +2,24 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: api
name: artifactapi-deployment
namespace: artifactapi
annotations:
reloader.stakater.com/auto: "true"
spec:
selector:
matchLabels:
app: api
app: artifactapi
strategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
app: api
spec:
automountServiceAccountToken: true
initContainers:
- name: combine-certs
image: alpine:3
command:
- sh
- -c
- cat /etc/ssl/certs/ca-certificates.crt /custom-ca/ca.crt > /combined-certs/ca-certificates.crt
volumeMounts:
- name: vault-ca-cert
mountPath: /custom-ca
readOnly: true
- name: combined-certs
mountPath: /combined-certs
containers:
- name: api
image: git.unkin.net/unkin/artifactapi:v3.6.5
- name: artifactapi
image: git.unkin.net/unkin/artifactapi:v2.7.2
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8000
@@ -43,15 +27,11 @@ spec:
protocol: TCP
envFrom:
- configMapRef:
name: api-env
name: artifactapi-env
optional: false
- secretRef:
name: environment
optional: false
volumeMounts:
- name: combined-certs
mountPath: /etc/ssl/combined
readOnly: true
livenessProbe:
failureThreshold: 3
httpGet:
@@ -79,13 +59,34 @@ spec:
requests:
cpu: 100m
memory: 256Mi
volumes:
- name: vault-ca-cert
secret:
secretName: vault-ca-cert
items:
- key: ca.crt
path: ca.crt
- name: combined-certs
emptyDir: {}
volumeMounts:
- mountPath: /etc/artifactapi/conf.d/config.yaml
name: remotes-config
subPath: config.yaml
- mountPath: /etc/artifactapi/conf.d/local-generic.yaml
name: remotes-config
subPath: local-generic.yaml
- mountPath: /etc/artifactapi/conf.d/remote-alpine.yaml
name: remotes-config
subPath: remote-alpine.yaml
- mountPath: /etc/artifactapi/conf.d/remote-docker.yaml
name: remotes-config
subPath: remote-docker.yaml
- mountPath: /etc/artifactapi/conf.d/remote-generic.yaml
name: remotes-config
subPath: remote-generic.yaml
- mountPath: /etc/artifactapi/conf.d/remote-helm.yaml
name: remotes-config
subPath: remote-helm.yaml
- mountPath: /etc/artifactapi/conf.d/remote-rpm.yaml
name: remotes-config
subPath: remote-rpm.yaml
- mountPath: /etc/artifactapi/conf.d/virtual-helm.yaml
name: remotes-config
subPath: virtual-helm.yaml
restartPolicy: Always
volumes:
- configMap:
name: remotes-config
optional: false
name: remotes-config
@@ -2,13 +2,13 @@
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: api-hpa
name: artifactapi-hpa
namespace: artifactapi
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: api
name: artifactapi-deployment
minReplicas: 2
maxReplicas: 10
metrics:
-91
View File
@@ -1,91 +0,0 @@
---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: postgres
namespace: artifactapi
spec:
affinity:
podAntiAffinityType: preferred
bootstrap:
initdb:
database: artifacts
encoding: UTF8
localeCType: C
localeCollate: C
owner: artifacts
secret:
name: postgres-credentials
enablePDB: true
enableSuperuserAccess: false
failoverDelay: 0
imageName: ghcr.io/cloudnative-pg/postgresql:18.1-system-trixie
instances: 3
logLevel: info
maxSyncReplicas: 0
minSyncReplicas: 0
monitoring:
customQueriesConfigMap:
- key: queries
name: cnpg-default-monitoring
disableDefaultQueries: false
enablePodMonitor: false
postgresql:
parameters:
archive_mode: "on"
archive_timeout: 5min
dynamic_shared_memory_type: posix
effective_cache_size: 256MB
full_page_writes: "on"
log_destination: csvlog
log_directory: /controller/log
log_filename: postgres
log_rotation_age: "0"
log_rotation_size: "0"
log_truncate_on_rotation: "false"
logging_collector: "on"
max_connections: "200"
max_parallel_workers: "16"
max_replication_slots: "16"
max_worker_processes: "16"
shared_buffers: 128MB
shared_memory_type: mmap
ssl_max_protocol_version: TLSv1.3
ssl_min_protocol_version: TLSv1.3
wal_keep_size: 256MB
wal_level: logical
wal_log_hints: "on"
wal_receiver_timeout: 5s
wal_sender_timeout: 5s
syncReplicaElectionConstraint:
enabled: false
primaryUpdateMethod: restart
primaryUpdateStrategy: unsupervised
probes:
liveness:
isolationCheck:
connectionTimeout: 1000
enabled: true
requestTimeout: 1000
replicationSlots:
highAvailability:
enabled: true
slotPrefix: _cnpg_
synchronizeReplicas:
enabled: true
updateInterval: 30
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 250m
memory: 256Mi
smartShutdownTimeout: 180
startDelay: 3600
stopDelay: 1800
storage:
resizeInUseVolumes: true
size: 20Gi
storageClass: cephrbd-fast-delete
switchoverDelay: 3600
-33
View File
@@ -1,33 +0,0 @@
---
apiVersion: postgresql.cnpg.io/v1
kind: Pooler
metadata:
name: postgres-pooler
namespace: artifactapi
spec:
cluster:
name: postgres
instances: 2
pgbouncer:
parameters:
default_pool_size: "100"
max_client_conn: "400"
paused: false
poolMode: session
template:
metadata:
labels:
app: pooler
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- pooler
topologyKey: kubernetes.io/hostname
containers: []
type: rw
+16 -5
View File
@@ -2,15 +2,26 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: api-env
name: artifactapi-env
namespace: artifactapi
data:
DBHOST: postgres-pooler
CONFIG_PATH: /etc/artifactapi/conf.d/
DBHOST: postgres-service
DBNAME: artifacts
DBPORT: "5432"
DBUSER: artifacts
MINIO_BUCKET: artifactapi-prod-k8s-syd1-au
MINIO_BUCKET: artifactapi
MINIO_ENDPOINT: radosgw.service.consul
MINIO_SECURE: "true"
REDIS_URL: redis://redis:6379
SSL_CERT_FILE: /etc/ssl/combined/ca-certificates.crt
REDIS_URL: redis://redis-service:6379
REQUESTS_CA_BUNDLE: /etc/pki/tls/certs/ca-bundle.crt
SSL_CERT_FILE: /etc/pki/tls/certs/ca-bundle.crt
---
apiVersion: v1
kind: ConfigMap
metadata:
name: postgres-env
namespace: artifactapi
data:
POSTGRES_DB: artifacts
POSTGRES_USER: artifacts
+5 -22
View File
@@ -2,15 +2,13 @@
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: http-redirect
name: artifactapi-http-redirect
namespace: artifactapi
spec:
hostnames:
- artifactapi.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: artifactapi
- name: artifactapi
sectionName: http
rules:
- filters:
@@ -26,33 +24,18 @@ spec:
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: api-route
name: artifactapi
namespace: artifactapi
spec:
hostnames:
- artifactapi.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: artifactapi
- name: artifactapi
sectionName: https
rules:
- backendRefs:
- group: ""
kind: Service
name: ui
- name: artifactapi-api
port: 80
weight: 1
matches:
- path:
type: PathPrefix
value: /ui
- backendRefs:
- group: ""
kind: Service
name: artifactapi
port: 80
weight: 1
matches:
- path:
type: PathPrefix
+18 -6
View File
@@ -3,17 +3,29 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- api-deployment.yaml
- api-hpa.yaml
- artifactapi-deployment.yaml
- artifactapi-hpa.yaml
- configmap.yaml
- cnpg_cluster.yaml
- cnpg_pooler.yaml
- gateway.yaml
- httproute.yaml
- namespace.yaml
- postgres-deployment.yaml
- pvc.yaml
- redis-deployment.yaml
- services.yaml
- ui-deployment.yaml
- ui-hpa.yaml
- vaultauth.yaml
- vaultstaticsecret.yaml
configMapGenerator:
- name: remotes-config
files:
- resources/conf.d/config.yaml
- resources/conf.d/local-generic.yaml
- resources/conf.d/remote-generic.yaml
- resources/conf.d/remote-alpine.yaml
- resources/conf.d/remote-rpm.yaml
- resources/conf.d/remote-docker.yaml
- resources/conf.d/remote-helm.yaml
- resources/conf.d/virtual-helm.yaml
options:
disableNameSuffixHash: true
@@ -0,0 +1,76 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgres-deployment
namespace: artifactapi
annotations:
reloader.stakater.com/auto: "true"
spec:
replicas: 1
selector:
matchLabels:
app: postgres
strategy:
type: Recreate
template:
spec:
automountServiceAccountToken: true
containers:
- name: postgres
image: postgres:15-alpine
imagePullPolicy: IfNotPresent
ports:
- containerPort: 5432
name: postgres
protocol: TCP
envFrom:
- configMapRef:
name: postgres-env
optional: false
- secretRef:
name: postgres-password
optional: false
readinessProbe:
exec:
command:
- pg_isready
- -U
- artifacts
- -d
- artifacts
failureThreshold: 3
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
livenessProbe:
exec:
command:
- pg_isready
- -U
- artifacts
- -d
- artifacts
failureThreshold: 3
initialDelaySeconds: 30
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 5
resources:
limits:
cpu: 500m
memory: 1Gi
requests:
cpu: 50m
memory: 128Mi
volumeMounts:
- mountPath: /var/lib/postgresql/data
mountPropagation: None
name: pgdata
subPath: pgdata
restartPolicy: Always
volumes:
- name: pgdata
persistentVolumeClaim:
claimName: artifactapi-postgres-pgdata
+28
View File
@@ -0,0 +1,28 @@
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: artifactapi-postgres-pgdata
namespace: artifactapi
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
storageClassName: cephrbd-fast-delete
volumeMode: Filesystem
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: artifactapi-redis-data
namespace: artifactapi
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
storageClassName: cephrbd-fast-delete
volumeMode: Filesystem
+21 -11
View File
@@ -2,21 +2,23 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis
annotations:
deployment.kubernetes.io/revision: "1"
name: redis-deployment
namespace: artifactapi
spec:
replicas: 1
selector:
matchLabels:
app: redis
strategy:
type: Recreate
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: redis:7-alpine
imagePullPolicy: IfNotPresent
command:
- redis-server
- --save
@@ -26,13 +28,6 @@ spec:
- containerPort: 6379
name: redis
protocol: TCP
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 50m
memory: 128Mi
livenessProbe:
exec:
command:
@@ -53,4 +48,19 @@ spec:
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 50m
memory: 128Mi
volumeMounts:
- mountPath: /data
mountPropagation: None
name: data
restartPolicy: Always
volumes:
- name: data
persistentVolumeClaim:
claimName: artifactapi-redis-data
@@ -0,0 +1,3 @@
# Global artifactapi configuration.
# S3, Redis, and database connection settings are injected via environment variables.
# Add any top-level overrides here if needed.
@@ -0,0 +1,7 @@
locals:
local-generic:
package: "generic"
description: "Local generic file repository"
cache:
immutable_ttl: 0
mutable_ttl: 0
@@ -0,0 +1,10 @@
remotes:
alpine:
base_url: "https://dl-cdn.alpinelinux.org"
package: "alpine"
description: "Alpine Linux APK package repository"
immutable_patterns:
- ".*/x86_64/.*\\.apk$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
@@ -0,0 +1,98 @@
remotes:
ghcr:
base_url: "https://ghcr.io"
package: "docker"
description: "GitHub Container Registry"
immutable_patterns:
- "^cloudnative-pg/cloudnative-pg"
- "^emberstack/helm-charts"
- "^kanidm/"
- "^openvoxproject/"
- "^stakater/reloader"
- "^voxpupuli/puppetboard"
- "^woodpecker-ci/helm"
cache:
immutable_ttl: 0
mutable_ttl: 300
dockerhub:
base_url: "https://registry-1.docker.io"
package: "docker"
description: "Docker Hub registry"
immutable_patterns:
- "^library/almalinux"
- "^library/busybox"
- "^library/debian"
- "^library/fedora"
- "^library/nginx"
- "^library/postgres"
- "^library/redis"
- "^beats/filebeat"
- "^bitnami/"
- "^curlimages/curl"
- "^emberstack/kubernetes-reflector"
- "^hashicorp/consul"
- "^hashicorp/vault"
- "^jfrog/"
- "^rancher/"
- "^traefik/"
- "^ubi9/ubi-minimal"
- "^victoriametrics/"
- "^woodpeckerci/"
cache:
immutable_ttl: 0
mutable_ttl: 300
quay:
base_url: "https://quay.io"
package: "docker"
description: "Quay.io container registry"
immutable_patterns:
- "^brancz/kube-rbac-proxy"
- "^cephcsi/cephcsi"
- "^jetstack/cert-manager-"
cache:
immutable_ttl: 0
mutable_ttl: 300
k8s-registry:
base_url: "https://registry.k8s.io"
package: "docker"
description: "Kubernetes container registry"
immutable_patterns:
- "^external-dns/external-dns"
- "^sig-storage/"
cache:
immutable_ttl: 0
mutable_ttl: 300
gitlab:
base_url: "https://registry.gitlab.com"
package: "docker"
description: "GitLab container registry"
immutable_patterns:
- "^purelb/purelb"
cache:
immutable_ttl: 0
mutable_ttl: 300
elastic:
base_url: "https://docker.elastic.co"
package: "docker"
description: "Elastic container registry"
immutable_patterns:
- "^eck/eck-operator"
cache:
immutable_ttl: 0
mutable_ttl: 300
gcr:
base_url: "https://gcr.io"
package: "docker"
description: "Google Container Registry"
immutable_patterns:
- "^k8s-staging-nfd/charts"
- "^k8s-staging-nfd/node-feature-discovery"
cache:
immutable_ttl: 0
mutable_ttl: 300
@@ -0,0 +1,130 @@
remotes:
github:
base_url: "https://github.com"
package: "generic"
description: "GitHub releases and files"
mutable_patterns:
- ".*/archive/refs/heads/.*.tar.gz$"
immutable_patterns:
- ".*/archive/refs/tags/.*.tar.gz$"
- "ahmetb/kubectx/.*/kubectx_.*_linux_x86_64.tar.gz$"
- "ahmetb/kubectx/.*/kubens_.*_linux_x86_64.tar.gz$"
- "apple/foundationdb/.*/libfdb_c.x86_64.so$"
- "astral-sh/ruff/.*/ruff-x86_64-unknown-linux-gnu.tar.gz$"
- "astral-sh/uv/.*/uv-x86_64-unknown-linux-gnu.tar.gz$"
- "camptocamp/prometheus-puppetdb-exporter/.*/prometheus-puppetdb-exporter-.*.linux-amd64.tar.gz$"
- "coder/code-server/.*/code-server-.*-amd64.rpm$"
- "containernetworking/plugins/.*/cni-plugins-linux-amd64-.*.tgz"
- "dandavison/delta/.*/delta-.*-x86_64-unknown-linux-musl.tar.gz$"
- "ducaale/xh/.*/xh-.*-x86_64-unknown-linux-musl.tar.gz$"
- "etcd-io/etcd/.*/etcd-.*-linux-amd64.tar.gz$"
- "getsops/sops/.*/sops-v.*\\.linux\\.amd64$"
- "grafana/jsonnet-language-server/.*/jsonnet-language-server_.*_linux_amd64$"
- "gruntwork-io/boilerplate/.*/boilerplate_linux_amd64$"
- "gruntwork-io/terragrunt/.*terragrunt_linux_amd64.*"
- "hadolint/hadolint/.*/hadolint-linux-x86_64$"
- "helmfile/helmfile/.*/helmfile_.*_linux_amd64.tar.gz$"
- "helmfile/vals/.*/vals_.*_linux_amd64.tar.gz$"
- "jesseduffield/lazydocker/.*/lazydocker_.*_Linux_x86_64.tar.gz$"
- "kubecolor/kubecolor/.*/kubecolor_.*_linux_amd64.tar.gz$"
- "kubernetes-sigs/gateway-api/.*/standard-install.yaml$"
- "kubernetes-sigs/kustomize/.*/kustomize_.*_linux_amd64.tar.gz$"
- "lxc/incus/.*.tar.gz$"
- "mikefarah/yq/.*/yq_linux_amd64$"
- "neovim/neovim-releases/.*/nvim-linux-x86_64.tar.gz$"
- "neovim/neovim/.*/nvim-linux-x86_64.tar.gz$"
- "nzbgetcom/nzbget/.*/nzbget-.*.x86_64.rpm$"
- "onedr0p/exportarr/.*/exportarr_.*_linux_amd64.tar.gz$"
- "openbao/openbao-plugins/.*/openbao-plugin-secrets-consul_linux_amd64_.*.tar.gz$"
- "openbao/openbao-plugins/.*/openbao-plugin-secrets-nomad_linux_amd64_.*.tar.gz$"
- "prometheus-community/bind_exporter/.*/bind_exporter-.*.linux-amd64.tar.gz$"
- "prometheus-community/pgbouncer_exporter/.*/pgbouncer_exporter-.*.linux-amd64.tar.gz$"
- "prometheus-community/postgres_exporter/.*/postgres_exporter-.*.linux-amd64.tar.gz$"
- "prometheus/node_exporter/.*/node_exporter-.*.linux-amd64.tar.gz$"
- "rancher/rke2/.*/rke2-images.linux-amd64.tar.zst$"
- "stalwartlabs/stalwart/.*/stalwart-cli-x86_64-unknown-linux-gnu.tar.gz$"
- "stalwartlabs/stalwart/.*/stalwart-foundationdb-x86_64-unknown-linux-gnu.tar.gz$"
- "stalwartlabs/stalwart/.*/stalwart-x86_64-unknown-linux-gnu.tar.gz$"
- "starship/starship/.*/starship-x86_64-unknown-linux-musl.tar.gz$"
- "stern/stern/.*/stern_.*_linux_amd64.tar.gz$"
- "terraform-linters/tflint/.*/tflint_linux_amd64.zip$"
- "tynany/frr_exporter/.*/frr_exporter-.*.linux-amd64.tar.gz$"
- "VictoriaMetrics/VictoriaLogs/.*/victoria-logs-linux-amd64-.*.tar.gz$"
- "VictoriaMetrics/VictoriaLogs/.*/vlutils-linux-amd64-.*.tar.gz$"
- "VictoriaMetrics/VictoriaMetrics/.*/victoria-logs-linux-amd64-.*.tar.gz$"
- "VictoriaMetrics/VictoriaMetrics/.*/victoria-metrics-linux-amd64-.*-cluster.tar.gz$"
- "VictoriaMetrics/VictoriaMetrics/.*/vlutils-linux-amd64-.*.tar.gz$"
- "VictoriaMetrics/VictoriaMetrics/.*/vmutils-linux-amd64-.*.tar.gz$"
- "xorpaul/g10k/.*/g10k-.*-linux-amd64.zip$"
- "yannh/kubeconform/.*/kubeconform-linux-amd64.tar.gz$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
github_user:
base_url: "https://raw.githubusercontent.com"
package: "generic"
description: "GitHub User Content"
immutable_patterns:
- "argoproj/argo-cd/.*.yaml$"
- "yannh/kubernetes-json-schema/master/.*.json$"
- "datreeio/CRDs-catalog/main/.*.json$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
gitea-dl:
base_url: "https://dl.gitea.com"
package: "generic"
description: "Gitea download site"
immutable_patterns:
- "act_runner/.*/act_runner-.*-linux-amd64$"
- "tea/.*/tea-.*-linux-amd64$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
hashicorp-releases:
base_url: "https://releases.hashicorp.com"
package: "generic"
description: "HashiCorp product releases"
immutable_patterns:
- "terraform/.*terraform_.*_linux_amd64\\.zip$"
- "terraform/.*terraform_.*_windows_amd64\\.zip$"
- "terraform/.*terraform_.*_darwin_amd64\\.zip$"
- "vault/.*vault_.*_linux_amd64\\.zip$"
- "vault/.*vault_.*_windows_amd64\\.zip$"
- "vault/.*vault_.*_darwin_amd64\\.zip$"
- "consul-cni/.*/consul-cni_.*_linux_amd64\\.zip$"
- "consul/.*/consul_.*_linux_amd64\\.zip$"
- "nomad-autoscaler/.*/nomad-autoscaler_.*_linux_amd64\\.zip$"
- "nomad/.*/nomad_.*_linux_amd64\\.zip$"
- "packer/.*/packer_.*_linux_amd64\\.zip$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
rarlab:
base_url: "https://www.rarlab.com"
package: "generic"
description: "RARLab"
immutable_patterns:
- "rar/rarlinux-x64-.*.tar.gz"
cache:
immutable_ttl: 0
mutable_ttl: 7200
claude-ai:
base_url: "https://downloads.claude.ai"
package: "generic"
description: "Anthropic Claude Code binary releases"
mutable_patterns:
- "claude-code-releases/.*/manifest.json$"
immutable_patterns:
- "claude-code-releases/.*/linux-x64/claude$"
- "claude-code-releases/.*/linux-arm64/claude$"
- "claude-code-releases/.*/linux-x64-musl/claude$"
- "claude-code-releases/.*/linux-arm64-musl/claude$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
@@ -0,0 +1,143 @@
remotes:
ceph-csi:
base_url: "https://ceph.github.io/csi-charts"
package: "helm"
description: "Ceph CSI driver Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
cnpg:
base_url: "https://cloudnative-pg.github.io/charts"
package: "helm"
description: "CloudNativePG operator Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
elastic-helm:
base_url: "https://helm.elastic.co"
package: "helm"
description: "Elastic stack Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
external-dns:
base_url: "https://kubernetes-sigs.github.io/external-dns/"
package: "helm"
description: "ExternalDNS Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
hashicorp-helm:
base_url: "https://helm.releases.hashicorp.com"
package: "helm"
description: "HashiCorp Helm charts (Vault Secrets Operator, etc.)"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
intel-helm:
base_url: "https://intel.github.io/helm-charts/"
package: "helm"
description: "Intel Helm charts (device plugins)"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
jetstack:
base_url: "https://charts.jetstack.io"
package: "helm"
description: "Jetstack Helm charts (cert-manager)"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
purelb:
base_url: "https://gitlab.com/api/v4/projects/20400619/packages/helm/stable"
package: "helm"
description: "PureLB load balancer Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
rancher-stable:
base_url: "https://releases.rancher.com/server-charts/stable"
package: "helm"
description: "Rancher stable Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
stakater:
base_url: "https://stakater.github.io/stakater-charts"
package: "helm"
description: "Stakater Helm charts (Reloader)"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
traefik:
base_url: "https://traefik.github.io/charts"
package: "helm"
description: "Traefik Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
victoriametrics:
base_url: "https://victoriametrics.github.io/helm-charts/"
package: "helm"
description: "VictoriaMetrics observability Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
argo-helm:
base_url: "https://argoproj.github.io/argo-helm"
package: "helm"
description: "Argo Project Helm charts (ArgoCD, Image Updater, Rollouts, etc.)"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
@@ -0,0 +1,154 @@
remotes:
almalinux:
base_url: "https://gsl-syd.mm.fcix.net/almalinux"
package: "rpm"
description: "AlmaLinux RPM package repository"
immutable_patterns:
- ".*/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*\\.sqlite.*$"
- ".*/repodata/.*\\.xml.*$"
- ".*/repodata/.*\\.yaml.*$"
- ".*/install.img"
- ".*/squashfs.img"
- ".*/updates.img"
- ".*/RPM-GPG-KEY-.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
ceph-reef:
base_url: "https://download.ceph.com/rpm-reef/"
package: "rpm"
description: "Ceph Reef 18"
immutable_patterns:
- ".*/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
ceph-squid:
base_url: "https://download.ceph.com/rpm-squid/"
package: "rpm"
description: "Ceph Squid 19"
immutable_patterns:
- ".*/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
ceph-tentacle:
base_url: "https://download.ceph.com/rpm-tentacle/"
package: "rpm"
description: "Ceph Tentacle 20"
immutable_patterns:
- ".*/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
epel:
base_url: "https://gsl-syd.mm.fcix.net/epel"
package: "rpm"
description: "EPEL (Extra Packages for Enterprise Linux)"
immutable_patterns:
- ".*/Everything/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*\\.sqlite.*$"
- ".*/repodata/.*\\.xml.*$"
- ".*/repodata/.*\\.yaml.*$"
- "RPM-GPG-KEY-.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
fedora:
base_url: "https://gsl-syd.mm.fcix.net/fedora/linux"
package: "rpm"
description: "Fedora Linux RPM package repository"
immutable_patterns:
- "releases/.*/Everything/x86_64/.*\\.rpm$"
- "updates/.*/Everything/x86_64/.*\\.rpm$"
- "development/.*/Everything/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
frr:
base_url: "https://rpm.frrouting.org/repo"
package: "rpm"
description: "FRR RPM package repository"
immutable_patterns:
- ".*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
mariadb:
base_url: "http://mariadb.mirror.digitalpacific.com.au/yum"
package: "rpm"
description: "MariaDB RPM package repository"
immutable_patterns:
- ".*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
- ".*/RPM-GPG-KEY-.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
openvox:
base_url: "https://yum.voxpupuli.org"
package: "rpm"
description: "OpenVox RPM package repository"
immutable_patterns:
- ".*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
- "GPG-KEY-.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
postgresql:
base_url: "https://download.postgresql.org/pub/repos/yum"
package: "rpm"
description: "PostgreSQL RPM package repository"
immutable_patterns:
- ".*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
- ".*/RPM-GPG-KEY-.*$"
- ".*/PGDG-RPM-GPG-KEY-.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
rke2:
base_url: "https://rpm.rancher.io"
package: "rpm"
description: "RKE2 RPM package repository"
immutable_patterns:
- ".*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
- "public.key$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
zfs:
base_url: "http://download.zfsonlinux.org"
package: "rpm"
description: "ZFS RPM package repository"
immutable_patterns:
- ".*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
@@ -0,0 +1,18 @@
virtuals:
helm:
package: "helm"
description: "Virtual repository merging all helm remotes — member order is priority order for duplicate chart+version"
members:
- ceph-csi
- cnpg
- elastic-helm
- external-dns
- hashicorp-helm
- intel-helm
- jetstack
- purelb
- rancher-stable
- stakater
- traefik
- victoriametrics
- argo-helm
+8 -8
View File
@@ -2,7 +2,7 @@
apiVersion: v1
kind: Service
metadata:
name: artifactapi
name: artifactapi-api
namespace: artifactapi
spec:
internalTrafficPolicy: Cluster
@@ -12,31 +12,31 @@ spec:
protocol: TCP
targetPort: http
selector:
app: api
app: artifactapi
sessionAffinity: None
type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
name: ui
name: postgres-service
namespace: artifactapi
spec:
internalTrafficPolicy: Cluster
ports:
- name: http
port: 80
- name: postgres
port: 5432
protocol: TCP
targetPort: http
targetPort: postgres
selector:
app: ui
app: postgres
sessionAffinity: None
type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
name: redis
name: redis-service
namespace: artifactapi
spec:
internalTrafficPolicy: Cluster
-58
View File
@@ -1,58 +0,0 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: ui
namespace: artifactapi
annotations:
reloader.stakater.com/auto: "true"
spec:
selector:
matchLabels:
app: ui
strategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
app: ui
spec:
automountServiceAccountToken: true
containers:
- name: ui
image: git.unkin.net/unkin/artifactapi-ui:v3.6.5
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
name: http
protocol: TCP
livenessProbe:
failureThreshold: 3
httpGet:
path: /ui
port: http
scheme: HTTP
initialDelaySeconds: 15
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 5
readinessProbe:
failureThreshold: 3
httpGet:
path: /ui
port: http
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 5
successThreshold: 1
timeoutSeconds: 5
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 50m
memory: 128Mi
restartPolicy: Always
+1 -1
View File
@@ -10,7 +10,7 @@ spec:
kubernetes:
audiences:
- vault
role: default
role: artifactapi
serviceAccount: default
tokenExpirationSeconds: 600
method: kubernetes
+19 -19
View File
@@ -1,23 +1,6 @@
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: postgres-credentials
namespace: artifactapi
spec:
destination:
create: true
name: postgres-credentials
overwrite: true
hmacSecretData: true
mount: kv
path: kubernetes/namespace/artifactapi/default/postgres-credentials
refreshAfter: 5m
type: kv-v2
vaultAuthRef: default
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: environment
namespace: artifactapi
@@ -25,10 +8,27 @@ spec:
destination:
create: true
name: environment
overwrite: true
overwrite: false
hmacSecretData: true
mount: kv
path: kubernetes/namespace/artifactapi/default/environment
path: service/artifactapi/environment
refreshAfter: 5m
type: kv-v2
vaultAuthRef: default
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: postgres-password
namespace: artifactapi
spec:
destination:
create: true
name: postgres-password
overwrite: true
hmacSecretData: true
mount: kv
path: service/artifactapi/postgres-password
refreshAfter: 5m
type: kv-v2
vaultAuthRef: default
+3 -10
View File
@@ -8,9 +8,7 @@ spec:
hostnames:
- rancher.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: rancher
- name: rancher
sectionName: http
rules:
- filters:
@@ -32,17 +30,12 @@ spec:
hostnames:
- rancher.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: rancher
- name: rancher
sectionName: https
rules:
- backendRefs:
- group: ""
kind: Service
name: rancher
- name: rancher
port: 80
weight: 1
matches:
- path:
type: PathPrefix
+5 -17
View File
@@ -11,9 +11,7 @@ spec:
hostnames:
- consul.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: consul
- name: consul
sectionName: http
rules:
- filters:
@@ -38,17 +36,12 @@ spec:
hostnames:
- consul.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: consul
- name: consul
sectionName: https
rules:
- backendRefs:
- group: ""
kind: Service
name: consul-ui
- name: consul-ui
port: 80
weight: 1
matches:
- path:
type: PathPrefix
@@ -66,17 +59,12 @@ spec:
hostnames:
- consul.service.consul
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: consul
- name: consul
sectionName: consul-svc
rules:
- backendRefs:
- group: ""
kind: Service
name: consul-ui
- name: consul-ui
port: 80
weight: 1
matches:
- path:
type: PathPrefix
-51
View File
@@ -1,51 +0,0 @@
# kanidm
Three-replica kanidm identity server with Vault-managed replication certificates.
## Architecture
- Per-pod `server-N.toml` in `resources/` — each has its own replication origin hardcoded
- `config-init` busybox init container copies the right config and injects peer certs from the
vault-synced `kanidm-repl-certs` Secret at pod startup
- `reloader.stakater.com/auto: "true"` triggers a rolling restart when the ConfigMap or Secret changes
- Vault path: `kv/kubernetes/namespace/kanidm/default/repl-certs`
- Keys: `kanidm-0`, `kanidm-1`, `kanidm-2` — each holds that pod's replication certificate
## Initial setup
After the first pod starts, generate the admin credentials:
```bash
kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd recover-account -c /config/server.toml admin
kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd recover-account -c /config/server.toml idm_admin
```
## Replication certificate rotation
When certs need to be renewed, update vault and reloader will roll the StatefulSet:
```bash
# Get new cert from a pod
kubectl exec -it -n kanidm kanidm-N -- /sbin/kanidmd renew-replication-certificate -c /config/server.toml
# Write updated cert to vault (reloader triggers restart automatically)
vault kv patch kv/kubernetes/namespace/kanidm/default/repl-certs "kanidm-N=<cert>"
```
## Resolving domain UUID mismatch
If pods initialized independently (each with a different domain UUID), replication will fail with
`Consumer Domain UUID does not match`. Fix by resetting kanidm-1 and kanidm-2 to sync from
kanidm-0 (the authoritative node):
```bash
# Scale down to avoid split-brain during reset
kubectl scale statefulset -n kanidm kanidm --replicas=1
# Delete the stale PVCs for the replica pods
kubectl delete pvc -n kanidm data-kanidm-1 data-kanidm-2
# Scale back up — replicas start with empty DBs and automatic_refresh=true
# will trigger a full sync from kanidm-0 once TLS peer certs are verified
kubectl scale statefulset -n kanidm kanidm --replicas=3
```
-26
View File
@@ -1,26 +0,0 @@
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: kanidm-tls
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
secretName: kanidm-tls
issuerRef:
kind: ClusterIssuer
name: vault-issuer
commonName: auth.unkin.net
dnsNames:
- auth.unkin.net
- au.auth.unkin.net
- kanidm.k8s.syd1.au.unkin.net
- kanidm.kanidm.svc.cluster.local
- kanidm-0.kanidm-headless.kanidm.svc.cluster.local
- kanidm-1.kanidm-headless.kanidm.svc.cluster.local
- kanidm-2.kanidm-headless.kanidm.svc.cluster.local
privateKey:
algorithm: RSA
size: 4096
-30
View File
@@ -1,30 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: kanidm
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
traefik.io/instance: internal
annotations:
external-dns.alpha.kubernetes.io/hostname: kanidm.k8s.syd1.au.unkin.net
external-dns.alpha.kubernetes.io/target: 198.18.200.4
spec:
gatewayClassName: traefik-internal
listeners:
- name: http
port: 80
protocol: HTTP
allowedRoutes:
namespaces:
from: Same
- name: https-passthrough
port: 443
protocol: TLS
tls:
mode: Passthrough
allowedRoutes:
namespaces:
from: Same
-29
View File
@@ -1,29 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: kanidm-http-redirect
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
hostnames:
- kanidm.k8s.syd1.au.unkin.net
- auth.unkin.net
- au.auth.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: kanidm
sectionName: http
rules:
- filters:
- type: RequestRedirect
requestRedirect:
scheme: https
statusCode: 301
matches:
- path:
type: PathPrefix
value: /
-27
View File
@@ -1,27 +0,0 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
- serviceaccount.yaml
- vaultauth.yaml
- vaultstaticsecret.yaml
- certificate.yaml
- service.yaml
- statefulset.yaml
- poddisruptionbudget.yaml
- gateway.yaml
- httproute.yaml
- tlsroute.yaml
configMapGenerator:
- name: kanidm-config
namespace: kanidm
options:
disableNameSuffixHash: true
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
files:
- server-0.toml=resources/server-0.toml
-5
View File
@@ -1,5 +0,0 @@
---
apiVersion: v1
kind: Namespace
metadata:
name: kanidm
-15
View File
@@ -1,15 +0,0 @@
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: kanidm
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
maxUnavailable: 1
selector:
matchLabels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
-37
View File
@@ -1,37 +0,0 @@
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: kanidm-repl
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create"]
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["kanidm-repl-certs"]
verbs: ["get", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: kanidm-repl
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
subjects:
- kind: ServiceAccount
name: kanidm
namespace: kanidm
roleRef:
kind: Role
name: kanidm-repl
apiGroup: rbac.authorization.k8s.io
-15
View File
@@ -1,15 +0,0 @@
version = "2"
domain = "auth.unkin.net"
origin = "https://auth.unkin.net"
bindaddress = "[::]:8443"
db_path = "/data/kanidm.db"
db_arc_size = 2048
tls_chain = "/data/tls/tls.crt"
tls_key = "/data/tls/tls.key"
log_level = "info"
[online_backup]
path = "/data/backups/"
schedule = "0 22 * * *"
versions = 7
-43
View File
@@ -1,43 +0,0 @@
---
apiVersion: v1
kind: Service
metadata:
name: kanidm
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
type: ClusterIP
sessionAffinity: ClientIP
sessionAffinityConfig:
clientIP:
timeoutSeconds: 10800
ports:
- name: https
port: 8443
targetPort: https
protocol: TCP
selector:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
---
apiVersion: v1
kind: Service
metadata:
name: kanidm-headless
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
type: ClusterIP
clusterIP: None
ports:
- name: https
port: 8443
targetPort: https
protocol: TCP
selector:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
-9
View File
@@ -1,9 +0,0 @@
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: kanidm
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
-85
View File
@@ -1,85 +0,0 @@
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: kanidm
namespace: kanidm
annotations:
reloader.stakater.com/auto: "true"
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
serviceName: kanidm-headless
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
template:
metadata:
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
serviceAccountName: kanidm
securityContext:
runAsUser: 1000
runAsGroup: 1000
runAsNonRoot: true
fsGroup: 1000
containers:
- name: kanidm
image: kanidm/server:1.10.3
command: ["/sbin/kanidmd"]
args: ["server", "-c", "/config/server.toml"]
ports:
- name: https
containerPort: 8443
protocol: TCP
volumeMounts:
- name: data
mountPath: /data
- name: config
mountPath: /config/server.toml
subPath: server-0.toml
readOnly: true
- name: tls
mountPath: /data/tls
readOnly: true
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: false
resources:
requests:
memory: 256Mi
cpu: 100m
limits:
memory: 1Gi
cpu: 500m
readinessProbe:
tcpSocket:
port: 8443
initialDelaySeconds: 15
periodSeconds: 10
livenessProbe:
tcpSocket:
port: 8443
initialDelaySeconds: 30
periodSeconds: 30
volumes:
- name: config
configMap:
name: kanidm-config
- name: tls
secret:
secretName: kanidm-tls
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ReadWriteOnce]
storageClassName: cephrbd-fast-delete
resources:
requests:
storage: 10Gi
-26
View File
@@ -1,26 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: TLSRoute
metadata:
name: kanidm
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
hostnames:
- kanidm.k8s.syd1.au.unkin.net
- auth.unkin.net
- au.auth.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: kanidm
sectionName: https-passthrough
rules:
- backendRefs:
- group: ""
kind: Service
name: kanidm
port: 8443
weight: 1
-23
View File
@@ -1,23 +0,0 @@
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: repl-certs
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
vaultAuthRef: default
mount: kv
type: kv-v2
path: kubernetes/namespace/kanidm/default/repl-certs
refreshAfter: 5m
destination:
name: kanidm-repl-certs
create: true
overwrite: true
hmacSecretData: true
rolloutRestartTargets:
- kind: StatefulSet
name: kanidm
+2 -2
View File
@@ -76,8 +76,8 @@ spec:
updateInterval: 30
resources:
limits:
cpu: "1"
memory: 1Gi
cpu: 1
memory: 1024Mi
requests:
cpu: 250m
memory: 512Mi
+3 -10
View File
@@ -8,9 +8,7 @@ spec:
hostnames:
- litellm.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: litellm
- name: litellm
sectionName: http
rules:
- filters:
@@ -32,17 +30,12 @@ spec:
hostnames:
- litellm.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: litellm
- name: litellm
sectionName: https
rules:
- backendRefs:
- group: ""
kind: Service
name: litellm
- name: litellm
port: 4000
weight: 1
matches:
- path:
type: PathPrefix
+18
View File
@@ -0,0 +1,18 @@
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: postfix-smtp-tls
namespace: mailgateway
spec:
secretName: postfix-smtp-tls
issuerRef:
name: vault-issuer
kind: ClusterIssuer
commonName: mail.main.unkin.net
dnsNames:
- mail.main.unkin.net
- smtp-in.main.unkin.net
privateKey:
size: 4096
algorithm: RSA
@@ -6,26 +6,26 @@ metadata:
traefik.io/instance: internal
annotations:
cert-manager.io/cluster-issuer: vault-issuer
cert-manager.io/common-name: age-api.k8s.syd1.au.unkin.net
cert-manager.io/common-name: rspamd.k8s.syd1.au.unkin.net
cert-manager.io/private-key-size: "4096"
external-dns.alpha.kubernetes.io/hostname: age-api.k8s.syd1.au.unkin.net
external-dns.alpha.kubernetes.io/hostname: rspamd.k8s.syd1.au.unkin.net
external-dns.alpha.kubernetes.io/target: 198.18.200.4
name: age-api
namespace: age-api
name: rspamd
namespace: mailgateway
spec:
gatewayClassName: traefik-internal
listeners:
- allowedRoutes:
namespaces:
from: Same
hostname: age-api.k8s.syd1.au.unkin.net
hostname: rspamd.k8s.syd1.au.unkin.net
name: http
port: 80
protocol: HTTP
- allowedRoutes:
namespaces:
from: Same
hostname: age-api.k8s.syd1.au.unkin.net
hostname: rspamd.k8s.syd1.au.unkin.net
name: https
port: 443
protocol: HTTPS
@@ -33,5 +33,5 @@ spec:
certificateRefs:
- group: ""
kind: Secret
name: age-api-tls
name: rspamd-tls
mode: Terminate
+16
View File
@@ -0,0 +1,16 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: rspamd
namespace: mailgateway
spec:
parentRefs:
- name: rspamd
namespace: mailgateway
hostnames:
- rspamd.k8s.syd1.au.unkin.net
rules:
- backendRefs:
- name: rspamd
port: 11334
+36
View File
@@ -0,0 +1,36 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- certificate.yaml
- gateway.yaml
- httproute.yaml
- namespace.yaml
- postfix-deployment.yaml
- postfix-hpa.yaml
- rspamd-deployment.yaml
- rspamd-hpa.yaml
- services.yaml
- vaultauth.yaml
- vaultstaticsecret.yaml
configMapGenerator:
- name: postfix-config
files:
- main.cf=resources/postfix/main.cf
- master.cf=resources/postfix/master.cf
options:
disableNameSuffixHash: true
- name: postfix-maps
files:
- transport=resources/postfix/transport
options:
disableNameSuffixHash: true
- name: rspamd-config
files:
- worker-proxy.inc=resources/rspamd/local.d/worker-proxy.inc
- dkim_signing.conf=resources/rspamd/local.d/dkim_signing.conf
- milter_headers.conf=resources/rspamd/local.d/milter_headers.conf
options:
disableNameSuffixHash: true
@@ -2,4 +2,4 @@
apiVersion: v1
kind: Namespace
metadata:
name: age-api
name: mailgateway
@@ -0,0 +1,105 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: postfix
namespace: mailgateway
spec:
selector:
matchLabels:
app: postfix
template:
metadata:
annotations:
reloader.stakater.com/auto: "true"
labels:
app: postfix
spec:
initContainers:
- name: postmap
image: tozd/postfix:alpine-322
command: ["/bin/sh", "-c"]
args:
- |
for f in /etc/postfix/maps/*; do
base=$(basename "$f")
cp "$f" /tmp/"$base"
postmap hash:/tmp/"$base"
cp /tmp/"${base}.db" /etc/postfix/db/
done
volumeMounts:
- name: postfix-maps
mountPath: /etc/postfix/maps
readOnly: true
- name: postfix-db
mountPath: /etc/postfix/db
containers:
- name: postfix
image: tozd/postfix:alpine-322
ports:
- containerPort: 25
name: smtp
protocol: TCP
- containerPort: 587
name: submission
protocol: TCP
env:
# Keep these in sync with main.cf so the tozd startup postconf calls are no-ops
- name: MAILNAME
value: "mail.main.unkin.net"
- name: MY_NETWORKS
value: "127.0.0.0/8 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16"
- name: MY_DESTINATION
value: "localhost.localdomain, localhost"
- name: LOG_TO_STDOUT
value: "1"
livenessProbe:
tcpSocket:
port: 25
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
tcpSocket:
port: 25
initialDelaySeconds: 15
periodSeconds: 10
timeoutSeconds: 3
failureThreshold: 3
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: "1"
memory: 512Mi
volumeMounts:
- name: postfix-config
mountPath: /etc/postfix/main.cf
subPath: main.cf
- name: postfix-config
mountPath: /etc/postfix/master.cf
subPath: master.cf
- name: postfix-db
mountPath: /etc/postfix/transport.db
subPath: transport.db
- name: postfix-tls
mountPath: /etc/postfix/tls
readOnly: true
- name: spool
mountPath: /var/spool/postfix
volumes:
- name: postfix-config
configMap:
name: postfix-config
- name: postfix-maps
configMap:
name: postfix-maps
- name: postfix-db
emptyDir: {}
- name: postfix-tls
secret:
secretName: postfix-smtp-tls
- name: spool
emptyDir: {}
@@ -2,22 +2,22 @@
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: ui-hpa
namespace: artifactapi
name: postfix-hpa
namespace: mailgateway
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: ui
name: postfix
minReplicas: 2
maxReplicas: 10
maxReplicas: 6
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 60
averageUtilization: 70
behavior:
scaleUp:
stabilizationWindowSeconds: 0
@@ -25,7 +25,7 @@ spec:
policies:
- type: Percent
value: 100
periodSeconds: 30
periodSeconds: 60
- type: Pods
value: 4
periodSeconds: 30
@@ -34,8 +34,5 @@ spec:
selectPolicy: Min
policies:
- type: Percent
value: 10
periodSeconds: 60
- type: Pods
value: 2
value: 30
periodSeconds: 60
@@ -0,0 +1,46 @@
# Basic identity — kept in sync with MAILNAME/MY_NETWORKS/MY_DESTINATION env vars
# so the tozd startup script's postconf calls are no-ops
myhostname = mail.main.unkin.net
myorigin = main.unkin.net
mydestination = localhost.localdomain, localhost
mynetworks = 127.0.0.0/8 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16
inet_protocols = ipv4
inet_interfaces = all
# No local delivery — we're a relay-only gateway
local_transport = error:no local delivery
alias_maps =
alias_database =
# Relay inbound mail for these domains to Stalwart
relay_domains = main.unkin.net unkin.net
transport_maps = hash:/etc/postfix/transport
# rspamd milter (same namespace — short DNS name resolves)
smtpd_milters = inet:rspamd:11332
non_smtpd_milters = inet:rspamd:11332
milter_default_action = accept
milter_protocol = 6
milter_mail_macros = i {mail_addr} {client_addr} {client_name} {auth_authen}
# Inbound TLS (cert from cert-manager Certificate resource)
smtpd_use_tls = yes
smtpd_tls_security_level = may
smtpd_tls_cert_file = /etc/postfix/tls/tls.crt
smtpd_tls_key_file = /etc/postfix/tls/tls.key
smtpd_tls_loglevel = 1
# Outbound TLS (opportunistic)
smtp_tls_security_level = may
smtp_tls_loglevel = 1
# Message size limit (50 MiB)
message_size_limit = 52428800
mailbox_size_limit = 0
# Queue retention
maximal_queue_lifetime = 5d
bounce_queue_lifetime = 1d
# Log to stdout for k8s log collection
maillog_file = /dev/stdout
@@ -0,0 +1,42 @@
# ==========================================================================
# service type private unpriv chroot wakeup maxproc command + args
# (yes) (yes) (yes) (never) (100)
# ==========================================================================
# SMTP inbound (port 25) — runs rspamd milter, relays to Stalwart via transport_maps
smtp inet n - n - - smtpd
# Submission (port 587) — TLS required, relay from trusted mynetworks only
submission inet n - n - - smtpd
-o syslog_name=postfix/submission
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=no
-o smtpd_reject_unlisted_recipient=no
-o smtpd_relay_restrictions=permit_mynetworks,reject
-o milter_macro_daemon_name=ORIGINATING
# Internal postfix processes
pickup unix n - n 60 1 pickup
cleanup unix n - n - 0 cleanup
qmgr unix n - n 300 1 qmgr
tlsmgr unix - - n 1000? 1 tlsmgr
rewrite unix - - n - - trivial-rewrite
bounce unix - - n - 0 bounce
defer unix - - n - 0 bounce
trace unix - - n - 0 bounce
verify unix - - n - 1 verify
flush unix n - n 1000? 0 flush
proxymap unix - - n - - proxymap
proxywrite unix - - n - 1 proxymap
smtp unix - - n - - smtp
relay unix - - n - - smtp
showq unix n - n - - showq
error unix - - n - - error
retry unix - - n - - error
discard unix - - n - - discard
local unix - n n - - local
virtual unix - n n - - virtual
lmtp unix - - n - - lmtp
anvil unix - - n - 1 anvil
scache unix - - n - 1 scache
postlog unix-dgram n - n - 1 postlogd
@@ -0,0 +1,2 @@
.unkin.net smtp:[stalwart.stalwart.svc.cluster.local]:25
unkin.net smtp:[stalwart.stalwart.svc.cluster.local]:25
@@ -0,0 +1,13 @@
enabled = true;
selector = "mail";
domain {
main.unkin.net {
privkey = "/etc/rspamd/dkim/private_key";
selector = "mail";
}
unkin.net {
privkey = "/etc/rspamd/dkim/private_key";
selector = "mail";
}
}
@@ -0,0 +1,2 @@
extended_spam_headers = true;
use = ["x-spam-status", "x-spam-score", "authentication-results"];
@@ -0,0 +1,7 @@
milter = yes;
bind_socket = "*:11332";
upstream "local" {
default = yes;
self_scan = yes;
}
@@ -0,0 +1,75 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: rspamd
namespace: mailgateway
spec:
selector:
matchLabels:
app: rspamd
template:
metadata:
annotations:
reloader.stakater.com/auto: "true"
labels:
app: rspamd
spec:
securityContext:
runAsUser: 11333
runAsGroup: 11333
fsGroup: 11333
containers:
- name: rspamd
image: rspamd/rspamd:4.0.1
ports:
- containerPort: 11332
name: milter
protocol: TCP
- containerPort: 11333
name: worker
protocol: TCP
- containerPort: 11334
name: controller
protocol: TCP
livenessProbe:
httpGet:
path: /ping
port: 11334
initialDelaySeconds: 15
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
httpGet:
path: /ping
port: 11334
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 3
failureThreshold: 3
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: "1"
memory: 512Mi
volumeMounts:
- name: rspamd-config
mountPath: /etc/rspamd/local.d
readOnly: true
- name: dkim-keys
mountPath: /etc/rspamd/dkim
readOnly: true
- name: rspamd-data
mountPath: /var/lib/rspamd
volumes:
- name: rspamd-config
configMap:
name: rspamd-config
- name: dkim-keys
secret:
secretName: dkim-keys
- name: rspamd-data
emptyDir: {}
+38
View File
@@ -0,0 +1,38 @@
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: rspamd-hpa
namespace: mailgateway
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: rspamd
minReplicas: 2
maxReplicas: 6
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
behavior:
scaleUp:
stabilizationWindowSeconds: 0
selectPolicy: Max
policies:
- type: Percent
value: 100
periodSeconds: 30
- type: Pods
value: 4
periodSeconds: 30
scaleDown:
stabilizationWindowSeconds: 300
selectPolicy: Min
policies:
- type: Percent
value: 30
periodSeconds: 60
+64
View File
@@ -0,0 +1,64 @@
---
# Internal service for rspamd - used by postfix pods in the same namespace
apiVersion: v1
kind: Service
metadata:
name: rspamd
namespace: mailgateway
spec:
selector:
app: rspamd
ports:
- name: milter
port: 11332
targetPort: 11332
protocol: TCP
- name: worker
port: 11333
targetPort: 11333
protocol: TCP
- name: controller
port: 11334
targetPort: 11334
protocol: TCP
---
# Internal ClusterIP for postfix - used by stalwart for outbound relay
apiVersion: v1
kind: Service
metadata:
name: postfix
namespace: mailgateway
spec:
selector:
app: postfix
ports:
- name: smtp
port: 25
targetPort: 25
protocol: TCP
- name: submission
port: 587
targetPort: 587
protocol: TCP
---
# External LoadBalancer for inbound MX (internet → postfix)
apiVersion: v1
kind: Service
metadata:
name: postfix-external
namespace: mailgateway
annotations:
external-dns.alpha.kubernetes.io/hostname: smtp-in.main.unkin.net
external-dns.alpha.kubernetes.io/target: 198.18.199.1
purelb.io/service-group: dmz
purelb.io/addresses: 198.18.199.1
spec:
type: LoadBalancer
externalTrafficPolicy: Local
selector:
app: postfix
ports:
- name: smtp
port: 25
targetPort: 25
protocol: TCP
@@ -3,10 +3,10 @@ apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
name: default
namespace: kanidm
namespace: mailgateway
spec:
allowedNamespaces:
- kanidm
- mailgateway
kubernetes:
audiences:
- vault
@@ -0,0 +1,17 @@
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: dkim-keys
namespace: mailgateway
spec:
destination:
create: true
name: dkim-keys
overwrite: true
hmacSecretData: true
mount: kv
path: kubernetes/namespace/mailgateway/default/dkim-keys
refreshAfter: 5m
type: kv-v2
vaultAuthRef: default
+3 -10
View File
@@ -8,9 +8,7 @@ spec:
hostnames:
- paperclip.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: paperclip
- name: paperclip
sectionName: http
rules:
- filters:
@@ -32,17 +30,12 @@ spec:
hostnames:
- paperclip.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: paperclip
- name: paperclip
sectionName: https
rules:
- backendRefs:
- group: ""
kind: Service
name: paperclip
- name: paperclip
port: 3100
weight: 1
matches:
- path:
type: PathPrefix
@@ -1,6 +0,0 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- priorityclasses.yaml
@@ -1,36 +0,0 @@
---
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
name: low
value: 100
preemptionPolicy: Never
globalDefault: false
description: "Low-importance workloads. Can be evicted under pressure but will not preempt other pods."
---
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
name: power
value: 100
preemptionPolicy: Never
globalDefault: false
description: "Compute-heavy workloads with low scheduling importance. Evictable under pressure."
---
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
name: medium
value: 10000
preemptionPolicy: PreemptLowerPriority
globalDefault: false
description: "Standard workloads. Will preempt low-priority pods if the cluster is under pressure."
---
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
name: high
value: 100000
preemptionPolicy: PreemptLowerPriority
globalDefault: false
description: "High-importance services. Will preempt medium- and low-priority pods if necessary."
+1 -1
View File
@@ -28,7 +28,7 @@ spec:
imagePullSecrets: null
containers:
- name: g10k-code
image: git.unkin.net/unkin/almalinux9-g10k:20260606
image: git.unkin.net/unkin/almalinux9-g10k:20260308
imagePullPolicy: IfNotPresent
resources:
requests:
+1 -1
View File
@@ -50,7 +50,7 @@ spec:
cpu: 20m
memory: 32Mi
- name: cert-generator
image: git.unkin.net/unkin/almalinux9-base:20260606
image: git.unkin.net/unkin/almalinux9-base:20260308
imagePullPolicy: IfNotPresent
command:
- sh
@@ -181,7 +181,7 @@ spec:
name: puppet-puppet-volume
- name: setup-shared-bins
image: git.unkin.net/unkin/almalinux9-base:20260606
image: git.unkin.net/unkin/almalinux9-base:20260308
command:
- sh
- -c
+3 -10
View File
@@ -13,9 +13,7 @@ spec:
hostnames:
- puppetboard.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: puppetboard
- name: puppetboard
sectionName: http
rules:
- filters:
@@ -42,17 +40,12 @@ spec:
hostnames:
- puppetboard.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: puppetboard
- name: puppetboard
sectionName: https
rules:
- backendRefs:
- group: ""
kind: Service
name: puppetboard
- name: puppetboard
port: 80
weight: 1
matches:
- path:
type: PathPrefix
+2 -7
View File
@@ -13,17 +13,12 @@ spec:
hostnames:
- puppetdb.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: puppetdb
- name: puppetdb
sectionName: https
rules:
- backendRefs:
- group: ""
kind: Service
name: puppetdb
- name: puppetdb
port: 8080
weight: 1
matches:
- path:
type: PathPrefix
+1
View File
@@ -9,6 +9,7 @@ metadata:
name: puppetdb
namespace: puppet
spec:
clusterIP: null
ports:
- name: pdb-http
port: 8080
+5 -17
View File
@@ -11,9 +11,7 @@ spec:
hostnames:
- vault.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: vault
- name: vault
sectionName: http
rules:
- filters:
@@ -38,17 +36,12 @@ spec:
hostnames:
- vault.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: vault
- name: vault
sectionName: https
rules:
- backendRefs:
- group: ""
kind: Service
name: vault
- name: vault
port: 8200
weight: 1
matches:
- path:
type: PathPrefix
@@ -67,17 +60,12 @@ spec:
- vault.service.consul
- vault.query.consul
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: vault
- name: vault
sectionName: vault-direct
rules:
- backendRefs:
- group: ""
kind: Service
name: vault
- name: vault
port: 8200
weight: 1
matches:
- path:
type: PathPrefix
-1
View File
@@ -6,4 +6,3 @@ resources:
- namespace.yaml
- gateway.yaml
- httproute.yaml
- role_k8s-service-registration.yaml
@@ -1,24 +0,0 @@
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: vault-k8s-service-registration
namespace: vault
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: vault-k8s-service-registration
namespace: vault
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: vault-k8s-service-registration
subjects:
- kind: ServiceAccount
name: vault
namespace: vault
-2
View File
@@ -6,8 +6,6 @@ resources:
- namespace.yaml
- cnpg_cluster.yaml
- cnpg_pooler.yaml
- serviceaccount_terraform_artifactapi.yaml
- serviceaccount_terraform_git.yaml
- serviceaccount_terraform_vault.yaml
- vaultauth.yaml
- vaultstaticsecret.yaml
@@ -1,6 +0,0 @@
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: terraform-artifactapi
namespace: woodpecker
@@ -1,6 +0,0 @@
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: terraform-git
namespace: woodpecker
+1 -1
View File
@@ -37,7 +37,7 @@ server:
cpu: 100m
limits:
memory: 2Gi
cpu: "1"
cpu: 1000m
client:
enabled: false
@@ -1,6 +0,0 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../../base/kanidm
@@ -3,4 +3,4 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../../base/age-api
- ../../../base/mailgateway
@@ -1,6 +0,0 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../../base/priority-classes
+3 -9
View File
@@ -40,7 +40,9 @@ server:
}
}
service_registration "kubernetes" {}
service_registration "consul" {
address = "consul-server.consul.svc.cluster.local:8500"
}
dataStorage:
enabled: true
@@ -48,14 +50,6 @@ server:
storageClass: cephrbd-fast-delete
accessMode: ReadWriteOnce
extraEnv:
- name: VAULT_K8S_NAMESPACE
value: vault
- name: VAULT_K8S_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
statefulSet:
securityContext:
container:
@@ -2,7 +2,6 @@ agent:
replicaCount: 3
env:
WOODPECKER_MAX_WORKFLOWS: "8"
WOODPECKER_BACKEND_K8S_PRIORITY_CLASS: power
WOODPECKER_BACKEND_K8S_STORAGE_CLASS: cephrbd-fast-delete
WOODPECKER_BACKEND_K8S_VOLUME_SIZE: 10G
WOODPECKER_BACKEND_K8S_STORAGE_RWX: false
+1 -9
View File
@@ -11,7 +11,6 @@ spec:
revision: HEAD
directories:
- path: apps/overlays/*/artifactapi
- path: apps/overlays/*/age-api
- path: apps/overlays/*/cattle-system
- path: apps/overlays/*/cert-manager
- path: apps/overlays/*/certificates
@@ -21,9 +20,8 @@ spec:
- path: apps/overlays/*/externaldns
- path: apps/overlays/*/inteldeviceplugins-system
- path: apps/overlays/*/jfrog
- path: apps/overlays/*/kanidm
- path: apps/overlays/*/node-feature-discovery
- path: apps/overlays/*/priority-classes
- path: apps/overlays/*/mailgateway
- path: apps/overlays/*/puppet
- path: apps/overlays/*/purelb
- path: apps/overlays/*/reflector-system
@@ -46,12 +44,6 @@ spec:
destination:
server: https://kubernetes.default.svc
namespace: '{{path[3]}}' # Use directory name as namespace
ignoreDifferences:
- group: ""
kind: ConfigMap
name: kanidm-repl-certs
jsonPointers:
- /data
syncPolicy:
automated:
prune: true
-6
View File
@@ -17,8 +17,6 @@ spec:
server: https://kubernetes.default.svc
- namespace: 'artifactapi'
server: https://kubernetes.default.svc
- namespace: 'age-api'
server: https://kubernetes.default.svc
- namespace: 'cert-manager'
server: https://kubernetes.default.svc
- namespace: 'certificates'
@@ -29,12 +27,8 @@ spec:
server: https://kubernetes.default.svc
- namespace: 'jfrog'
server: https://kubernetes.default.svc
- namespace: 'kanidm'
server: https://kubernetes.default.svc
- namespace: 'node-feature-discovery'
server: https://kubernetes.default.svc
- namespace: 'priority-classes'
server: https://kubernetes.default.svc
- namespace: 'purelb'
server: https://kubernetes.default.svc
- namespace: 'puppet'
-50
View File
@@ -1,50 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
SCHEMA_DIR="${1:-schemas}"
rm -rf "$SCHEMA_DIR"
mkdir -p "$SCHEMA_DIR"
echo "==> Fetching CRDs from cluster..." >&2
kubectl get crds -o json | python3 -c "
import sys, json, os
def write_schema(schema, schema_dir, group, kind, version):
def strip_descriptions(obj):
if isinstance(obj, dict):
return {k: strip_descriptions(v) for k, v in obj.items() if k != 'description'}
if isinstance(obj, list):
return [strip_descriptions(i) for i in obj]
return obj
schema = strip_descriptions(schema)
group_dir = os.path.join(schema_dir, group)
os.makedirs(group_dir, exist_ok=True)
fname = f'{kind}_{version}.json'.lower()
with open(os.path.join(group_dir, fname), 'w') as f:
json.dump(schema, f, indent=2, sort_keys=True)
f.write('\n')
print(f' Generated: {group}/{fname}', file=sys.stderr)
data = json.load(sys.stdin)
for crd in data.get('items', []):
spec = crd.get('spec', {})
group = spec.get('group', '')
kind = spec.get('names', {}).get('kind', '')
for ver in spec.get('versions', []):
version = ver.get('name', '')
openapi = ver.get('schema', {}).get('openAPIV3Schema', {})
if not openapi:
continue
schema = dict(openapi)
schema['\$schema'] = 'http://json-schema.org/draft-07/schema#'
schema['type'] = 'object'
schema.setdefault('properties', {})
schema['properties'].setdefault('apiVersion', {'type': 'string'})
schema['properties'].setdefault('kind', {'type': 'string'})
schema['properties'].setdefault('metadata', {'type': 'object'})
write_schema(schema, '$SCHEMA_DIR', group, kind, version)
"
total=$(find "$SCHEMA_DIR" -name '*.json' | wc -l)
echo "==> Schema generation complete: $total schemas in $SCHEMA_DIR" >&2
+3 -6
View File
@@ -3,12 +3,9 @@ set -euo pipefail
KUBE_VERSION="1.33.7"
SCHEMA_DIR="${SCHEMA_DIR:-schemas}"
schema_args=(
-schema-location "$SCHEMA_DIR/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json"
-schema-location "https://artifactapi.k8s.syd1.au.unkin.net/api/v1/remote/github_user/yannh/kubernetes-json-schema/master/{{.NormalizedKubernetesVersion}}-standalone{{.StrictSuffix}}/{{.ResourceKind}}{{.KindSuffix}}.json"
-schema-location "https://artifactapi.k8s.syd1.au.unkin.net/api/v1/remote/github_user/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json"
-schema-location "https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/{{.NormalizedKubernetesVersion}}-standalone{{.StrictSuffix}}/{{.ResourceKind}}{{.KindSuffix}}.json"
-schema-location "https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json"
)
while IFS= read -r -d "" k; do
@@ -21,6 +18,6 @@ while IFS= read -r -d "" k; do
-summary \
-output pretty \
-verbose \
\
-skip CustomResourceDefinition,GpuDevicePlugin,LBNodeAgent,ServiceGroup \
"${schema_args[@]}"
done < <(find apps/overlays -name kustomization.yaml -print0)
+3 -6
View File
@@ -3,12 +3,9 @@ set -euo pipefail
KUBE_VERSION="1.33.7"
SCHEMA_DIR="${SCHEMA_DIR:-schemas}"
schema_args=(
-schema-location "$SCHEMA_DIR/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json"
-schema-location "https://artifactapi.k8s.syd1.au.unkin.net/api/v1/remote/github_user/yannh/kubernetes-json-schema/master/{{.NormalizedKubernetesVersion}}-standalone{{.StrictSuffix}}/{{.ResourceKind}}{{.KindSuffix}}.json"
-schema-location "https://artifactapi.k8s.syd1.au.unkin.net/api/v1/remote/github_user/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json"
-schema-location "https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/{{.NormalizedKubernetesVersion}}-standalone{{.StrictSuffix}}/{{.ResourceKind}}{{.KindSuffix}}.json"
-schema-location "https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json"
)
while IFS= read -r -d "" k; do
@@ -21,6 +18,6 @@ while IFS= read -r -d "" k; do
-summary \
-output pretty \
-verbose \
\
-skip CustomResourceDefinition \
"${schema_args[@]}"
done < <(find clusters -name kustomization.yaml -print0)
-145
View File
@@ -1,145 +0,0 @@
# Migration Guide: Terragrunt to ArgoCD
## Prerequisites
- Examine existing Terraform configuration in `sources/terraform-k8s/config/<app-name>/`
- Identify Helm charts, values, storage classes, secrets, and other resources
## Migration Steps
### 1. Branch Creation
- Change back to main branch: `git checkout main`
- Create new branch: `git checkout -b benvin/<app-name>`
### 2. Create Base Application Structure
- Create directory: `apps/base/<app-name>/`
- Create `apps/base/<app-name>/kustomization.yaml` with resources:
- `namespace.yaml`
- `vaultauth.yaml` (if needed)
- `vaultstaticsecret.yaml` (if needed)
- Additional resources as required (storageclass.yaml, etc.)
### 3. Create Namespace
- Create `apps/base/<app-name>/namespace.yaml` with app namespace
### 4. Create Vault Integration (if needed)
- Create `apps/base/<app-name>/vaultauth.yaml` from Terraform `vault_auth.yaml`
- Convert snake_case to camelCase for Kubernetes
- Map Terraform fields to VaultAuth spec
- Create `apps/base/<app-name>/vaultstaticsecret.yaml` from `vault_static_secret.yaml`
- Convert to VaultStaticSecret spec format
### 5. Create Additional Resources
- Create any additional resources (StorageClass, etc.) from Terraform config
- Maintain exact parameter parity with Terraform
### 6. Create Overlay Structure
- Create directory: `apps/overlays/au-syd1/<app-name>/`
- Create `apps/overlays/au-syd1/<app-name>/kustomization.yaml`:
- Reference base: `../../../base/<app-name>`
- Add helmCharts section with repo, version, valuesFile
- Create `apps/overlays/au-syd1/<app-name>/values.yaml` from Terraform helm_release values
### 7. Update Project Configuration
- Add Helm repository to appropriate project in `argocd/projects/`
- Add namespace to project destinations (if needed)
- Add required cluster resource permissions
### 8. Update ApplicationSet
- Add directory pattern to appropriate ApplicationSet in `argocd/applicationsets/`
- Use existing patterns like `apps/overlays/*/csi-*` or `apps/overlays/*/<app-name>`
### 9. Validation
- Run `kustomize build --enable-helm apps/overlays/au-syd1/<app-name>` to generate all resources
- Check resource types: `kustomize build --enable-helm apps/overlays/au-syd1/<app-name> | grep "^kind:" | sort | uniq -c`
- Verify all resource types are permitted in the target project's `clusterResourceWhitelist` and `namespaceResourceWhitelist`
- Run `make kubeconform` to validate all resources
- Fix any validation errors
### 10. Git Workflow
- Add only created/modified files: `git add apps/base/<app-name>/ apps/overlays/au-syd1/<app-name>/ argocd/projects/<project>.yaml argocd/applicationsets/<project>.yaml`
- **Never use `git add .`**
- Create commit with descriptive message following existing patterns
- Push branch: `git push -u origin benvin/<app-name>`
## Project Organization
### Platform Project
- Core infrastructure and system components
- Examples: cert-manager, external-dns, cnpg-system, reflector-system, etc.
### Storage Project
- Storage-related components
- Pattern: `csi-*` applications
- Examples: csi-cephfs, csi-cephrbd
### Application-Specific Projects
- Create new projects for logical groupings as needed
## Common Patterns
### Helm Chart Integration
```yaml
helmCharts:
- name: <chart-name>
repo: <helm-repo-url>
version: "<version>"
releaseName: <release-name>
namespace: <namespace>
valuesFile: values.yaml
```
### VaultAuth Template
```yaml
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
name: <auth-name>
namespace: <namespace>
spec:
method: kubernetes
mount: k8s/au/syd1
vaultConnectionRef: vso-system/default
allowedNamespaces:
- <namespace>
kubernetes:
role: <role>
serviceAccount: <service-account>
audiences:
- vault
tokenExpirationSeconds: 600
```
### VaultStaticSecret Template
```yaml
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: <secret-name>
namespace: <namespace>
spec:
vaultAuthRef: <auth-ref>
mount: kv
type: kv-v2
path: <vault-path>
refreshAfter: 5m
destination:
name: <k8s-secret-name>
create: true
```
## Troubleshooting
### Pre-commit Hook Issues
- If hooks modify files, add changes: `git add -u`
- Amend commit if safe: `git commit --amend --no-edit`
- Use `git push --force-with-lease` for amended commits
### Validation Failures
- Check Helm chart compatibility with Kubernetes version
- Verify all required fields are present in resources
- Ensure proper YAML formatting
### Missing Permissions
- Add required cluster resources to project clusterResourceWhitelist
- Add namespaces to project destinations
- Verify Helm repository is in project sourceRepos
File diff suppressed because it is too large Load Diff
-162
View File
@@ -1,162 +0,0 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"apiVersion": {
"type": "string"
},
"kind": {
"type": "string"
},
"metadata": {
"type": "object"
},
"spec": {
"properties": {
"commonName": {
"type": "string"
},
"dnsNames": {
"items": {
"type": "string"
},
"type": "array",
"x-kubernetes-list-type": "atomic"
},
"duration": {
"type": "string"
},
"ipAddresses": {
"items": {
"type": "string"
},
"type": "array",
"x-kubernetes-list-type": "atomic"
},
"issuerRef": {
"properties": {
"group": {
"type": "string"
},
"kind": {
"type": "string"
},
"name": {
"type": "string"
}
},
"required": [
"name"
],
"type": "object"
},
"profile": {
"type": "string"
},
"request": {
"format": "byte",
"type": "string"
}
},
"required": [
"issuerRef",
"request"
],
"type": "object"
},
"status": {
"properties": {
"authorizations": {
"items": {
"properties": {
"challenges": {
"items": {
"properties": {
"token": {
"type": "string"
},
"type": {
"type": "string"
},
"url": {
"type": "string"
}
},
"required": [
"token",
"type",
"url"
],
"type": "object"
},
"type": "array",
"x-kubernetes-list-type": "atomic"
},
"identifier": {
"type": "string"
},
"initialState": {
"enum": [
"valid",
"ready",
"pending",
"processing",
"invalid",
"expired",
"errored"
],
"type": "string"
},
"url": {
"type": "string"
},
"wildcard": {
"type": "boolean"
}
},
"required": [
"url"
],
"type": "object"
},
"type": "array",
"x-kubernetes-list-type": "atomic"
},
"certificate": {
"format": "byte",
"type": "string"
},
"failureTime": {
"format": "date-time",
"type": "string"
},
"finalizeURL": {
"type": "string"
},
"reason": {
"type": "string"
},
"state": {
"enum": [
"valid",
"ready",
"pending",
"processing",
"invalid",
"expired",
"errored"
],
"type": "string"
},
"url": {
"type": "string"
}
},
"type": "object"
}
},
"required": [
"metadata",
"spec"
],
"type": "object"
}
@@ -1,130 +0,0 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"apiVersion": {
"type": "string"
},
"kind": {
"type": "string"
},
"metadata": {
"type": "object"
},
"spec": {
"properties": {
"clusterSelector": {
"properties": {
"matchExpressions": {
"items": {
"properties": {
"key": {
"type": "string"
},
"operator": {
"type": "string"
},
"values": {
"items": {
"type": "string"
},
"type": "array",
"x-kubernetes-list-type": "atomic"
}
},
"required": [
"key",
"operator"
],
"type": "object"
},
"type": "array",
"x-kubernetes-list-type": "atomic"
},
"matchLabels": {
"additionalProperties": {
"type": "string"
},
"type": "object"
}
},
"type": "object",
"x-kubernetes-map-type": "atomic"
},
"resources": {
"items": {
"properties": {
"kind": {
"enum": [
"Secret",
"ConfigMap"
],
"type": "string"
},
"name": {
"minLength": 1,
"type": "string"
}
},
"required": [
"kind",
"name"
],
"type": "object"
},
"type": "array"
},
"strategy": {
"enum": [
"ApplyOnce"
],
"type": "string"
}
},
"required": [
"clusterSelector"
],
"type": "object"
},
"status": {
"properties": {
"conditions": {
"items": {
"properties": {
"lastTransitionTime": {
"format": "date-time",
"type": "string"
},
"message": {
"type": "string"
},
"reason": {
"type": "string"
},
"severity": {
"type": "string"
},
"status": {
"type": "string"
},
"type": {
"type": "string"
}
},
"required": [
"status",
"type"
],
"type": "object"
},
"type": "array"
},
"observedGeneration": {
"format": "int64",
"type": "integer"
}
},
"type": "object"
}
},
"type": "object"
}
@@ -1,130 +0,0 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"apiVersion": {
"type": "string"
},
"kind": {
"type": "string"
},
"metadata": {
"type": "object"
},
"spec": {
"properties": {
"clusterSelector": {
"properties": {
"matchExpressions": {
"items": {
"properties": {
"key": {
"type": "string"
},
"operator": {
"type": "string"
},
"values": {
"items": {
"type": "string"
},
"type": "array",
"x-kubernetes-list-type": "atomic"
}
},
"required": [
"key",
"operator"
],
"type": "object"
},
"type": "array",
"x-kubernetes-list-type": "atomic"
},
"matchLabels": {
"additionalProperties": {
"type": "string"
},
"type": "object"
}
},
"type": "object",
"x-kubernetes-map-type": "atomic"
},
"resources": {
"items": {
"properties": {
"kind": {
"enum": [
"Secret",
"ConfigMap"
],
"type": "string"
},
"name": {
"minLength": 1,
"type": "string"
}
},
"required": [
"kind",
"name"
],
"type": "object"
},
"type": "array"
},
"strategy": {
"enum": [
"ApplyOnce"
],
"type": "string"
}
},
"required": [
"clusterSelector"
],
"type": "object"
},
"status": {
"properties": {
"conditions": {
"items": {
"properties": {
"lastTransitionTime": {
"format": "date-time",
"type": "string"
},
"message": {
"type": "string"
},
"reason": {
"type": "string"
},
"severity": {
"type": "string"
},
"status": {
"type": "string"
},
"type": {
"type": "string"
}
},
"required": [
"status",
"type"
],
"type": "object"
},
"type": "array"
},
"observedGeneration": {
"format": "int64",
"type": "integer"
}
},
"type": "object"
}
},
"type": "object"
}

Some files were not shown because too many files have changed in this diff Show More