52 Commits

Author SHA1 Message Date
unkinben 06c3c59ea2 Add JSON schema generation for kubeconform CRD validation
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/kubeconform Pipeline was successful
- ci/generate-schemas.sh generates schemas from CRD manifests and K8s swagger
- Sources: ArgoCD v3.3.2, Gateway API v1.5.1, Kubernetes v1.33.7
- Optionally fetches live cluster CRDs via kubectl when available
- Generated schemas committed to schemas/ for CI use
- Run `make schemas` to regenerate after CRD version bumps
- validate-apps.sh and validate-clusters.sh check local schemas first
2026-06-28 17:03:21 +10:00
unkinben cfca1e5278 Add age-api deployment (#210)
## Summary
- Deploy age-api to the au-syd1 cluster
- Uses configMapGenerator for people config with jaidi, ben, and sudaporn
- Includes gateway, httproute, service, and deployment
- Image: git.unkin.net/unkin/age-api:v0.1.0

Reviewed-on: #210
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-28 12:19:38 +10:00
benvin 99a95f4e57 chore: source schema source for kubeconform (#209)
cache frequent lookups to prevent 400 errors from github. the schemas
are available via artifactapi.

---------

Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #209
2026-06-27 22:39:31 +10:00
unkinben feaec2c8a9 chore: bump artifactapi + ui to v3.6.5 (#208)
Adds bandwidth saved stat to dashboard.

Reviewed-on: #208
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-27 22:27:55 +10:00
unkinben d1cc467455 chore: bump artifactapi + ui to v3.6.4 (#207)
Fixes helm chart URL path duplication for same-host repos (stakater).

Reviewed-on: #207
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-27 08:06:26 +10:00
unkinben 0e9ac4d390 chore: bump artifactapi + ui to v3.6.3 (#206)
Includes Docker Accept header forwarding, Content-Type fix, nginx base path fix, and version endpoint fix.

Reviewed-on: #206
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-27 07:51:13 +10:00
unkinben 722ced3256 chore: bump artifactapi + ui to v3.6.2 (#205)
Includes Docker Bearer token auth (#60) and UI BASE_PATH build_args fix (#59).

Reviewed-on: #205
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-27 00:20:27 +10:00
unkinben 92e6f0f13b chore: bump artifactapi + ui to v3.6.1 (#204)
Rebuilds UI with BASE_PATH=/ui so assets serve under /ui/.

Reviewed-on: #204
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-27 00:03:58 +10:00
unkinben 825c46c91b chore: bump artifactapi + ui to v3.6.0 (#203)
Bumps API and UI images from v3.5.0 to v3.6.0.

Reviewed-on: #203
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-26 23:57:52 +10:00
unkinben 2c9c79d8f1 fix: update UI health check paths to /ui (#202)
The UI now serves under /ui (artifactapi#58). Health probes need /ui instead of /.

Reviewed-on: #202
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-26 23:57:27 +10:00
unkinben f695657d9d refactor: simplify artifactapi routes (#201)
Route /ui → UI service, everything else → API service.

Replaces the growing list of per-prefix rules (/api, /v2, /health) with a single catch-all to the API. No more needing to add a route rule every time the API adds a new top-level path.

Reviewed-on: #201
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-26 23:39:24 +10:00
unkinben 5dee768170 fix: route /v2 and /health to artifactapi API service (#200)
The v3 route migration (#198) split routes into /api → API and / → UI, but /v2/ (Docker Registry V2 API) and /health now hit the UI catch-all instead of the API backend.

This breaks `docker pull artifactapi.k8s.syd1.au.unkin.net/...` with context deadline exceeded.

Adds /v2 and /health prefix rules before the UI catch-all.

Reviewed-on: #200
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-26 23:31:47 +10:00
benvin f120f3b426 fix: rename environment2 to environment (#199)
update the environment secret reference to match what has been
 deployed. this prevents a containerconfigerror

---------

Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #199
2026-06-26 22:55:24 +10:00
benvin f6d60bd02d feat: artifactapi route change (#198)
complete cutover to artifactapi 3

---------

Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #198
2026-06-26 22:50:27 +10:00
benvin aac1b654bb feat: migrate to artifactapi 3+ (#197)
What changed:
- Adds new v3 API and UI deployments (separate api-deployment.yaml, ui-deployment.yaml) alongside the existing monolithic artifactapi-deployment.yaml
- Adds CNPG PostgreSQL cluster + pooler to replace the standalone postgres deployment
- Adds new api-env configmap, new Vault secrets (postgres-credentials, environment), and a second VaultAuth (default1)
- Adds new services targeting the split api and ui selectors
- Adds HPAs for both new deployments
- Updates kustomization to include all new resources

---------

Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #197
2026-06-26 22:18:07 +10:00
benvin 1c6e087116 chore: cleanup artifactory3 mess (#196)
attempted to let claude deploy a new version of artifactory with
terrible results. this change is to remove that mess so I can start
again.

---------

Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #196
2026-06-21 17:40:17 +10:00
benvin 9e6efb7c78 🤦 (#195)
Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #195
2026-06-21 17:30:47 +10:00
benvin cae42b4896 feat: manage postgres-credentials for artifactapi3 (#194)
pull credentials for postgres/cnpg from vault

---------

Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #194
2026-06-21 17:26:26 +10:00
benvin 349dc5fd01 chore: remove middleware resource (#193)
there is no crd for this, preventing the deployment of artifactapi 3

---------

Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #193
2026-06-21 09:10:49 +10:00
benvin 8cbd645332 feat: deploy artifactapi3 (#192)
just-enough to test terraform deployment and begin migration. have
change to cnpg for the database and a new bucket for storage

---------

Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #192
2026-06-20 12:22:22 +10:00
benvin ad2cdd3b63 fix: update woodpecker kustomization (#191)
Reviewed-on: #191
2026-06-17 21:34:02 +10:00
benvin 17782d716c feat: enable terraform-artifactapi jobs (#190)
woodpecker jobs for terraform-artifactapi use the service account of the
same name to run jobs, so that it can access specific secrets

- add terraform-artifactapi serviceaccount

---------

Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #190
2026-06-17 21:23:49 +10:00
unkinben 188c39f85d feat: add terraform-git service account for woodpecker CI (#189)
## Summary
- Add ServiceAccount terraform-git in woodpecker namespace for terraform-git CI pipelines
- Add to kustomization.yaml

## Test plan
- [ ] Verify ArgoCD syncs the new service account
- [ ] Verify woodpecker CI can use the service account

Reviewed-on: #189
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-07 20:36:55 +10:00
unkinben 0b7819bda3 chore: bump almalinux9 image tags (#188)
Bump almalinux9 image tags to 20260606

Reviewed-on: #188
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-07 00:35:12 +10:00
benvin 3c6330ebfd benvin/gitea (#187)
Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #187
2026-06-06 19:47:16 +10:00
unkinben a3a56d0c2b chore: add almalinux-vault repos (#186)
- 9.7 is end of life, ensure that we can still query packages

Reviewed-on: #186
2026-06-02 23:13:45 +10:00
unkinben 4b1fbe1fe1 feat(kanidm): scale down to single replica, remove replication (#185)
Drop from 3 replicas to 1. Remove init container, repl-certs secret,
replication port, podAntiAffinity, server-1/2 configs, and replication
stanza from server-0.toml. Mount configmap directly via subPath.

Reviewed-on: #185
2026-06-02 22:41:28 +10:00
unkinben 666f3d055c feat: add sessionaffinity to kanidm service (#184)
- required as traefik is in passthrough mode

Reviewed-on: #184
2026-06-02 21:17:51 +10:00
unkinben 3dc8801070 fix(kanidm): fix automatic_refresh TOML generation in init container (#182)
## Summary

- The `\n` escape in a shell variable wasn't interpreted as a newline when passed as a `printf %s` argument
- This caused `automatic_refresh = true` to be appended to the `partner_cert` string value on the same line, breaking TOML parsing on kanidm-2
- Fixed by using separate `printf` calls per peer type, with `\n` in the format string (not a variable) where it is correctly interpreted

## Test plan

- [ ] kanidm-2 init container generates valid TOML with `automatic_refresh = true` on its own line under the kanidm-0 peer section
- [ ] kanidm-1 and kanidm-2 start successfully and auto-refresh domain UUID from kanidm-0

Reviewed-on: #182
2026-05-31 00:25:21 +10:00
unkinben 60f1f3130b fix(kanidm): replicate 1/2 from 0 only with automatic_refresh (#181)
kanidm-0 is the authoritative supplier; kanidm-1 and kanidm-2 pull
from kanidm-0 only. automatic_refresh = true on the kanidm-0 peer
entry for kanidm-1/2 so fresh nodes auto-sync domain UUID on restart.

Reviewed-on: #181
2026-05-31 00:20:30 +10:00
unkinben b6f8cb0633 feat: autorestart statefulset (#180)
- ensure kanidm is restarted with vault secrets

Reviewed-on: #180
2026-05-30 23:40:07 +10:00
unkinben f11ec1056d fix(kanidm): remove invalid automatic_refresh from replication config (#179)
Reviewed-on: #179
2026-05-30 23:20:48 +10:00
unkinben ed7feaf19a Update apps/base/kanidm/vaultauth.yaml (#177)
Fix the VaultAuth object

Reviewed-on: #177
2026-05-30 23:11:38 +10:00
unkinben 4d594fbde7 feat(kanidm): vault-managed replication certs with auto-restart (#176)
- Store per-pod replication certs in Vault (kv/kubernetes/namespace/kanidm/default/repl-certs)
- VaultAuth + VaultStaticSecret sync certs to kanidm-repl-certs Secret
- busybox config-init init container injects peer certs from Secret into server.toml at startup
- Remove hardcoded partner_cert entries from per-pod server.toml templates
- Add automatic_refresh = true to all replication configs
- Add reloader.stakater.com/auto annotation to trigger rolling restart on ConfigMap/Secret changes
- Document domain UUID mismatch resolution and cert rotation in README

Reviewed-on: #176
2026-05-30 23:00:46 +10:00
unkinben 1b781e0885 feat(woodpecker): set workflow pod priority class to power (#175)
## Summary
Sets `WOODPECKER_BACKEND_K8S_PRIORITY_CLASS: power` on the Woodpecker agent so all CI pipeline pods are scheduled with the `power` PriorityClass (value 100, preemptionPolicy: Never).

This means pipeline pods can be evicted when the cluster is under pressure but won't preempt other workloads.

## Dependency
Requires the `power` PriorityClass to exist on the cluster — deploy PR #174 (priority-classes app) first.

## Test plan
- Trigger a pipeline run and confirm pods are created with `priorityClassName: power`
- `kubectl get pod -n woodpecker -o jsonpath='{.items[*].spec.priorityClassName}'`

Reviewed-on: #175
2026-05-26 23:58:57 +10:00
unkinben ede25a3858 feat(platform): add priority-classes app with low/power/medium/high classes (#174)
## Summary
- New `apps/base/priority-classes/` app with four `PriorityClass` objects managed via the `platform` ArgoCD project
- Adds `apps/overlays/*/priority-classes` to the platform ApplicationSet generator
- Adds `priority-classes` namespace to platform AppProject destinations (required even for cluster-scoped resources)

| Class | Value | PreemptionPolicy | Intent |
|---|---|---|---|
| `low` | 100 | Never | Background work; evictable, won't preempt others |
| `power` | 100 | Never | Compute-heavy but expendable (e.g. AI/ML workloads) |
| `medium` | 10000 | PreemptLowerPriority | Standard services |
| `high` | 100000 | PreemptLowerPriority | Critical services; preempts lower-priority pods |

`PriorityClass` is already in the platform project's `clusterResourceWhitelist` so no project policy changes were needed.

## Test plan
- ArgoCD syncs `platform-priority-classes` successfully
- `kubectl get priorityclasses low power medium high` shows all four classes

Reviewed-on: #174
2026-05-26 23:41:54 +10:00
unkinben f5f713fe86 feat(artifactapi): add open-webui/open-webui to ghcr immutable patterns (#173)
Part of #155 (prerequisite for open-webui deployment PR #172).

## Summary
- Adds `^open-webui/open-webui` to the `ghcr` remote's `immutable_patterns` in `remote-docker.yaml` so version-pinned open-webui image pulls are cached indefinitely through artifactapi

## Test plan
- artifactapi serves `ghcr.io/open-webui/open-webui:<version>` with `X-Artifact-Source: cache` on second fetch

Reviewed-on: #173
2026-05-26 23:28:27 +10:00
unkinben 3990fbfe06 feat(vault): switch to Kubernetes service registration (#171)
Replaces Consul service registration with the native Kubernetes provider so Vault labels its own pods with active/standby/perf-standby status without requiring a Consul dependency.

## Changes
- `values.yaml`: swap `service_registration "consul"` for `service_registration "kubernetes" {}`, add `VAULT_K8S_NAMESPACE` and `VAULT_K8S_POD_NAME` env vars via downward API
- `role_k8s-service-registration.yaml`: Role + RoleBinding granting the `vault` service account `get`/`update`/`patch` on pods
- `kustomization.yaml`: include new RBAC file

Reviewed-on: #171
2026-05-26 00:06:56 +10:00
unkinben d358098fff chore: update replication certs (#170)
- add replication certs for kanidm-0, kanidm-1 and kanidm-2

Reviewed-on: #170
2026-05-25 23:52:06 +10:00
unkinben 201e601737 feat: update kanidm replicaiton (#169)
- split to per-server configs
- remove init containers that attempted to automate the replication config
- add README.md

Reviewed-on: #169
2026-05-25 23:25:48 +10:00
unkinben d230d87ec9 feat(artifactapi): add conftest to GitHub generic remote cache (#168)
## Summary

- Adds `open-policy-agent/conftest/.*/conftest_.*_Linux_x86_64.tar.gz$` to the `github` remote immutable patterns in artifactapi

## Why

conftest v0.68.2 (https://github.com/open-policy-agent/conftest/releases/tag/v0.68.2) is now used for OPA policy checks in CI (see #167). Caching the release tarball in artifactapi reduces external dependency on GitHub during builds.

Reviewed-on: #168
2026-05-25 22:44:57 +10:00
unkinben 6497dab25e fix(puppet): remove explicit clusterIP: null from puppetdb Service (#166)
## Summary

- Removes `clusterIP: null` from the `puppetdb` Service spec

## Why

Setting `clusterIP: null` makes ArgoCD's desired state explicit about the field being null. Kubernetes assigns a real IP on creation and the field is immutable afterward. The null vs assigned-IP mismatch causes permanent OutOfSync on the puppetdb Service. Removing the field means ArgoCD no longer claims ownership of `clusterIP`, so the API server's value is authoritative.

Reviewed-on: #166
2026-05-25 22:44:24 +10:00
unkinben f403c6b05d fix(kanidm): add explicit group/kind/weight to TLSRoute refs (#165)
## Summary

- Adds `group: gateway.networking.k8s.io` and `kind: Gateway` to `parentRefs`
- Adds `group: ""`, `kind: Service`, and `weight: 1` to `backendRefs`

## Why

The Gateway API controller defaults these fields when creating/updating TLSRoute objects, so the live state always has them. ArgoCD diffs desired vs live by string comparison, causing the `kanidm` TLSRoute to show permanent OutOfSync. Same root cause as #162 (HTTPRoutes).

Reviewed-on: #165
2026-05-25 22:43:52 +10:00
unkinben ac8b8212bd fix(consul): normalize cpu limit to canonical string form (#164)
## Summary

- Changes `server.resources.limits.cpu` from `1000m` to `"1"` in consul Helm values

## Why

`1000m` (1000 milliCPU) is equivalent to `1` CPU, but Kubernetes normalizes the value to `"1"` when storing. ArgoCD diffs desired vs live by string comparison, so the mismatch causes a permanent OutOfSync on the `consul-server` StatefulSet. Same root cause as #163.

Reviewed-on: #164
2026-05-25 22:43:35 +10:00
unkinben dd282f59fb fix(litellm): normalize postgres cluster resource values (#163)
## Summary

- Changes `limits.memory` from `1024Mi` to `1Gi` (same value, canonical form)
- Changes `limits.cpu` from `1` (integer) to `"1"` (string, canonical form)

## Why

Kubernetes normalizes resource quantities on write — `1024Mi` becomes `1Gi` and integer `1` becomes string `"1"`. ArgoCD diffs by string comparison, so these equivalent values cause a permanent OutOfSync on the `litellm-postgres` Cluster.

Reviewed-on: #163
2026-05-24 23:30:10 +10:00
unkinben 1890dd4bda fix(gateways): add explicit group/kind/weight to all HTTPRoute refs (#162)
## Summary

- Adds `group: gateway.networking.k8s.io` and `kind: Gateway` to all `parentRefs` entries
- Adds `group: ""`, `kind: Service`, and `weight: 1` to all `backendRefs` entries
- Affects 9 HTTPRoute files across artifactapi, cattle-system, consul, kanidm, litellm, paperclip, puppet, and vault

## Why

ArgoCD diffs the desired manifest against the live Kubernetes object. The Gateway API controller defaults these fields when creating/updating objects, so the live state always has them — causing persistent OutOfSync for every HTTPRoute. Same root cause as #153 (certificateRefs).

## Test plan

- [ ] All affected ArgoCD applications show Synced after merge

Reviewed-on: #162
2026-05-24 20:32:37 +10:00
unkinben 6815b66010 fix(kanidm): use dockerhub image instead of ghcr.io (#161)
## Summary

- Changes both `config-init` init container and `kanidm` container images from `ghcr.io/kanidm/server:1.10.3` to `kanidm/server:1.10.3`

## Why

`kanidm/server` is published on Docker Hub, not ghcr.io. RKE2 rewrites dockerhub pulls through the artifactapi mirror automatically.

## Test plan

- [ ] Pods roll successfully after ArgoCD sync
- [ ] Verify kanidm cluster replication still healthy

Reviewed-on: #161
2026-05-24 20:27:21 +10:00
unkinben 7cbec33588 fix(artifactapi): move kanidm to dockerhub remote (#160)
## Summary

- Removes `^kanidm/` from the `ghcr` remote immutable_patterns
- Adds `^kanidm/` to the `dockerhub` remote immutable_patterns

## Why

`kanidm/server` is published on Docker Hub, not ghcr.io. Pulling via the `ghcr` cache was failing with 403 on anonymous token fetch → 502 Bad Gateway.

## Test plan

- [ ] `docker pull artifactapi.k8s.syd1.au.unkin.net/dockerhub/kanidm/server:1.10.3` succeeds after artifactapi redeploys

Reviewed-on: #160
2026-05-24 20:24:33 +10:00
unkinben 3756208ccd benvin/kanidm (#159)
Reviewed-on: #159
2026-05-24 19:55:22 +10:00
unkinben 6ce92e8ead benvin/artifactapi-mail-images (#158)
Reviewed-on: #158
2026-05-24 14:44:38 +10:00
unkinben af79d86db6 feat(artifactapi): cache stalwart webadmin zip (#157)
## Summary

- Adds \`stalwartlabs/webadmin/releases/latest/download/webadmin.zip\` to \`mutable_patterns\` in the \`github\` generic remote so the stalwart webadmin UI can be fetched through artifactapi rather than directly from GitHub.

## Notes

- Uses \`mutable_patterns\` (not \`immutable\`) because \`releases/latest\` resolves to whichever release is current and changes over time.
- Access URL: \`https://artifactapi.k8s.syd1.au.unkin.net/generic/github/stalwartlabs/webadmin/releases/latest/download/webadmin.zip\`

Reviewed-on: #157
2026-05-24 12:55:16 +10:00
unkinben 5f4c9225bb feat(artifactapi): add mail stack images to docker registry cache (#156)
- ghcr: stalwartlabs/stalwart (Stalwart mail server)
- dockerhub: rspamd/rspamd (spam filter), tozd/postfix (MTA gateway)

Reviewed-on: #156
2026-05-24 12:42:27 +10:00
656 changed files with 152239 additions and 1330 deletions
+273
View File
@@ -0,0 +1,273 @@
---
description: Pull master, read open issues, pick one, branch, implement, test, commit, PR, and comment.
---
# Solve a Gitea Issue
## Current repo state
```!
git status --short
echo "Current branch: $(git branch --show-current)"
echo "Remote: $(git remote get-url origin 2>/dev/null || echo 'none')"
```
## Open issues (with full body)
```!
echo "Fetching open issues..."
issue_ids=$(tea issues list --output simple 2>/dev/null | awk 'NF && $1 ~ /^[0-9]+$/ {print $1}')
if [ -z "$issue_ids" ]; then
echo "No open issues found (or tea is not logged in)."
else
for id in $issue_ids; do
echo ""
echo "══════════════════════════════════════"
tea issues view "$id" --fields index,title,body 2>/dev/null \
|| tea issue "$id" 2>/dev/null \
|| echo " (could not read issue #$id)"
echo "══════════════════════════════════════"
done
fi
```
---
## Your task
Follow these steps **in order**. Do not skip steps.
### 1 — Choose an issue
Present the issues above to the user as a numbered list (index, one-line title). Ask which one to work on. Wait for the answer before continuing.
### 2 — Sync master
```bash
git checkout master
git pull
```
Confirm you are on master and up to date.
### 3 — Create a branch
Name the branch `benvin/issue-<N>-<short-slug>` where `<short-slug>` is 24 kebab-case words from the issue title.
```bash
git checkout -b benvin/issue-<N>-<slug>
```
### 4 — Read the issue in full
Re-read the full issue body shown above. If any part is ambiguous, state your interpretation before coding.
**If you discover other problems while working:** do NOT solve them inline. Create a new Gitea issue with `tea issues create --title "..." --description "..."` and stay focused on the assigned issue.
### 5 — Implement the solution
Make the code changes needed to resolve the issue. Follow the conventions already in the repo:
- `main.py` route handlers each contain a single function call; logic lives in submodules.
- No comments unless the WHY is non-obvious.
- No new files unless the issue or architecture requires it.
- Security: no command injection, XSS, SQL injection, or secrets in code.
- **For performance improvements:** implement at the most generic call site possible so the fix applies to all current and future implementations, not just the one being tested.
### 6 — Update tests
Add or update tests that cover the new behaviour. Tests live in `tests/`. Check existing test structure before writing new ones — mirror the style and fixture patterns already in use.
### 7 — Update README
If the feature introduces new config keys, endpoints, or user-facing behaviour, document it in `README.md`. Keep additions concise — follow the existing section style.
### 8 — Run the full test suite
```bash
make test
```
All tests must pass. If any fail, fix them before proceeding. Do not skip or suppress failing tests.
### 9 — Live Docker test (new package type only)
**Skip this step if the issue does not add a new remote package type.**
If the issue adds a new package type (e.g. `deb`, `conda`, `cargo`, `rubygems`, or any type not already in `remotes.yaml`), do the following before committing.
#### 9a — Add a real test remote to remotes.yaml
Append a valid, publicly accessible remote of the new type to `remotes.yaml`. Use a real upstream URL and patterns that cover both an immutable file (versioned artifact) and a mutable file (index/metadata). Add a comment explaining which URLs to use for manual testing.
#### 9b — Start the stack
```bash
make docker-up
```
Wait until `curl -s http://localhost:8000/health` returns `{"status":"healthy"}`.
#### 9c — Test a mutable file (first fetch — cache miss)
Download the index or metadata file for the new remote. Confirm:
- HTTP 200
- `X-Artifact-Source: remote` header (or equivalent log line confirming a cache miss)
- Content looks correct (not empty, not an error page)
```bash
curl -sv "http://localhost:8000/api/v1/remote/<new-remote>/<mutable-path>" 2>&1 | grep -E "< HTTP|X-Artifact"
```
#### 9d — Test a mutable file (second fetch — cache hit)
Repeat the exact same request. Confirm:
- HTTP 200
- `X-Artifact-Source: cache`
```bash
curl -sv "http://localhost:8000/api/v1/remote/<new-remote>/<mutable-path>" 2>&1 | grep -E "< HTTP|X-Artifact"
```
#### 9e — Test an immutable file (first fetch — cache miss)
Download a versioned/immutable artifact. Confirm HTTP 200 and a cache-miss log line.
```bash
curl -sv "http://localhost:8000/api/v1/remote/<new-remote>/<immutable-path>" 2>&1 | grep -E "< HTTP|X-Artifact"
```
#### 9f — Test an immutable file (second fetch — cache hit)
Repeat. Confirm `X-Artifact-Source: cache`.
#### 9g — Check container logs
```bash
make docker-logs
```
Scan for:
- `Cache MISS` on first fetches, `Cache HIT` on second fetches
- `Cache ADD SUCCESS` with correct sizes
- No unhandled exceptions or ERROR lines
#### 9h — Exercise package-type tooling against the proxy
Use the native tooling for this package type to verify end-to-end behaviour. Examples:
| Package type | Command |
|---|---|
| `pypi` | `uv run --index-url http://localhost:8000/api/v1/remote/<remote>/simple <tool>` |
| `npm` | `npm install --registry http://localhost:8000/api/v1/remote/<remote>/ <pkg>` |
| `helm` | `helm repo add test http://localhost:8000/api/v1/remote/<remote> && helm search repo test && helm template test/<chart>` |
| `alpine` | `apk fetch --repository http://localhost:8000/api/v1/remote/<remote>/<branch>/<arch> <pkg>` |
| `rpm` | `dnf install --repofrompath ... <pkg>` or `repoquery` |
| `generic` | `curl` / `wget` as appropriate |
Confirm the tool resolves and downloads correctly through the proxy.
#### 9i — Tear down
```bash
make docker-down
```
Fix any failures found during 9b9h before moving on.
### 9.5 — Performance issues: measure before/after and gate the PR
**Skip this step if the issue is not a performance improvement.**
For performance issues, a PR is only warranted if there is a measurable gain. Use the Docker stack to compare before and after.
#### 9.5a — Baseline measurement (before)
Start the stack with the **unmodified** code (temporarily revert your change):
```bash
make docker-up
```
Warm or clear the cache as appropriate, then measure the relevant metric — e.g. concurrent request latency during a slow operation, response time for a specific endpoint, or throughput. Record the numbers.
#### 9.5b — Apply your change and rebuild
```bash
make docker-up # rebuilds the image
```
Repeat exactly the same measurement. Record the numbers.
#### 9.5c — Decide
If the improvement is not clearly measurable, **do not open a PR**. Instead:
1. Update the issue with your findings.
2. Note any conditions under which the improvement would be observable.
3. Skip steps 1114.
If the improvement is clear, proceed with the commit and PR. Include the before/after numbers in the PR description and the issue comment.
#### 9.5d — Tear down
```bash
make docker-down
```
### 10 — Build the wheel (smoke check)
```bash
uv build --wheel
```
Confirm the build succeeds.
### 11 — Stage and commit
Stage only the files you changed. Do not use `git add -A` or `git add .` — list files explicitly. Run:
```bash
git add <file1> <file2> ...
git commit
```
The commit message must:
- Start with a conventional-commit prefix (`feat:`, `fix:`, `refactor:`, `chore:`, etc.)
- Summarise the change in ≤ 72 characters on the first line
- Optionally include a short body explaining *why* (not *what*)
If the pre-commit hook auto-fixes files, re-stage the fixed files and commit again.
### 12 — Push the branch
```bash
git push origin <branch-name>
```
### 13 — Open a pull request
```bash
tea pulls create \
--base master \
--head <branch-name> \
--title "<same as commit subject>" \
--description "Closes #<N>\n\n## Summary\n<bullet points>\n\n## Test plan\n<what was verified>"
```
### 14 — Comment on the issue
```bash
tea comment <N> "<resolution comment>"
```
The comment must cover:
- **How it was resolved** — what changed and why
- **Issues encountered** — any non-obvious problems hit during implementation
- **Potential future improvements** — what could be done next
### 15 — Return to master
```bash
git checkout master
```
Report the PR URL and a one-sentence summary to the user.
+2
View File
@@ -7,6 +7,7 @@ repos:
- id: check-json
- id: check-added-large-files
args: ['--maxkb=500']
exclude: '^schemas/'
- id: check-merge-conflict
- id: check-shebang-scripts-are-executable
- id: check-symlinks
@@ -19,6 +20,7 @@ repos:
- id: end-of-file-fixer
- id: forbid-new-submodules
- id: pretty-format-json
args: ['--autofix']
- id: trailing-whitespace
# YAML linting
+1 -1
View File
@@ -3,7 +3,7 @@ when:
steps:
- name: kubeconform
image: git.unkin.net/unkin/almalinux9-kubetest:20260319
image: git.unkin.net/unkin/almalinux9-kubetest:20260606
commands:
- make kubeconform
backend_options:
+1 -1
View File
@@ -3,7 +3,7 @@ when:
steps:
- name: pre-commit
image: git.unkin.net/unkin/almalinux9-base:20260308
image: git.unkin.net/unkin/almalinux9-base:20260606
commands:
- uvx pre-commit run --all-files
backend_options:
+261
View File
@@ -0,0 +1,261 @@
# AGENTS.md
## Project Overview
This is an **ArgoCD GitOps repository** that manages Kubernetes applications for the `au-syd1` cluster using a Kustomize + Helm pattern. Applications are deployed via ArgoCD ApplicationSets that watch directory patterns in this repo.
The migration pattern for this repo is: **Terragrunt/Terraform → ArgoCD** (see `migration.md` for full guide).
---
## Essential Commands
```bash
# Build and render manifests for a path (outputs to manifests/<path>/)
make build apps/overlays/au-syd1/<app-name>
make build clusters/au-syd1/bootstrap
# Validate all apps and clusters with kubeconform
make kubeconform
# Clean generated manifests
make clean
# Quick build + inspect without persisting output
kustomize build --enable-helm apps/overlays/au-syd1/<app-name>
# Check all resource kinds produced by an overlay
kustomize build --enable-helm apps/overlays/au-syd1/<app-name> | grep "^kind:" | sort | uniq -c
# Run pre-commit checks against all files
uvx pre-commit run --all-files
```
---
## Directory Structure
```
argocd-apps/
├── argocd/
│ ├── applicationsets/ # ArgoCD ApplicationSet definitions (platform.yaml, storage.yaml)
│ └── projects/ # ArgoCD AppProject definitions (platform.yaml, storage.yaml)
├── apps/
│ ├── base/ # Base Kustomize resources per app (no cluster-specific config)
│ │ └── <app-name>/
│ │ ├── kustomization.yaml
│ │ ├── namespace.yaml
│ │ ├── vaultauth.yaml # (if Vault-managed secrets)
│ │ └── vaultstaticsecret.yaml
│ └── overlays/
│ └── au-syd1/ # Cluster-specific overlays
│ └── <app-name>/
│ ├── kustomization.yaml # references base + helmCharts
│ └── values.yaml # Helm values for this cluster
├── clusters/
│ └── au-syd1/
│ ├── apps/ # Entry point: references apps/base (ArgoCD app-of-apps)
│ └── bootstrap/ # ArgoCD install + initial Application manifest
├── ci/
│ ├── validate-apps.sh # kubeconform over apps/overlays/*/kustomization.yaml
│ ├── validate-clusters.sh # kubeconform over clusters/*/kustomization.yaml
│ └── validate-no-secrets.sh # pre-commit hook: blocks plain Kubernetes Secrets
└── sources/ # Reference sources (Terraform configs, upstream charts, etc.)
└── terraform-k8s/ # Original Terraform configs — reference when migrating
```
---
## Adding a New Application
Follow these 10 steps (detailed in `migration.md`):
### 1. Create base resources
```
apps/base/<app-name>/
├── kustomization.yaml
├── namespace.yaml
├── vaultauth.yaml # if needed
└── vaultstaticsecret.yaml # if needed
```
### 2. Create cluster overlay
```
apps/overlays/au-syd1/<app-name>/
├── kustomization.yaml
└── values.yaml
```
**Overlay kustomization.yaml pattern:**
```yaml
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../../base/<app-name>
helmCharts:
- name: <chart-name>
repo: <helm-repo-url>
version: "<version>"
releaseName: <release-name>
namespace: <namespace>
valuesFile: values.yaml
```
### 3. Register in ApplicationSet
Add a directory entry to `argocd/applicationsets/platform.yaml` (or `storage.yaml` for `csi-*` apps):
```yaml
- path: apps/overlays/*/<app-name>
```
### 4. Update AppProject
In `argocd/projects/platform.yaml` (or `storage.yaml`):
- Add the Helm repo URL to `sourceRepos`
- Add the namespace to `destinations`
- Add any required cluster-scoped resource types to `clusterResourceWhitelist`
### 5. Validate
```bash
kustomize build --enable-helm apps/overlays/au-syd1/<app-name>
make kubeconform
```
---
## Secret Management
**Plain Kubernetes `Secret` objects are blocked** by the pre-commit hook. Use Vault Operator CRDs instead:
### VaultAuth template
```yaml
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
name: default
namespace: <namespace>
spec:
method: kubernetes
mount: k8s/au/syd1
vaultConnectionRef: vso-system/default
allowedNamespaces:
- <namespace>
kubernetes:
role: <role>
serviceAccount: <service-account>
audiences:
- vault
tokenExpirationSeconds: 600
```
### VaultStaticSecret template
```yaml
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: <secret-name>
namespace: <namespace>
spec:
vaultAuthRef: default
mount: kv
type: kv-v2
path: kubernetes/namespace/<namespace>/default/<secret-name>
refreshAfter: 5m
destination:
name: <k8s-secret-name>
create: true
overwrite: true
hmacSecretData: true
```
---
## YAML Conventions
- **2-space indentation** (enforced by yamllint)
- All files must end with a newline (`end-of-file-fixer`)
- No trailing whitespace
- YAML linting uses relaxed rules with `line-length: disable` (long base64/URLs are fine)
- yamllint ignores `chart` directories (vendored Helm charts)
- `---` document separator at top of every YAML file
- Multiple documents in one file are allowed (e.g., `vaultstaticsecret.yaml` often contains multiple secrets)
---
## Kubernetes Labels Pattern
Use standard `app.kubernetes.io/*` labels consistently:
```yaml
labels:
app.kubernetes.io/component: <component>
app.kubernetes.io/instance: <release-name>
app.kubernetes.io/name: <app-name>
app.kubernetes.io/version: <version>
```
---
## Resource Naming Conventions
Files in `apps/base/<app-name>/` follow the pattern:
```
<kind>_<name>.yaml
```
Examples:
- `deployment_puppetserver-master.yaml`
- `cronjob_g10k-code.yaml`
- `configmap_puppetboard-config.yaml`
- `horizontalpodautoscaler_puppetserver-compilers-autoscaler.yaml`
- `service_puppet-headless.yaml`
---
## Helm Chart Vendoring
Some overlays vendor Helm charts locally under `apps/overlays/au-syd1/<app-name>/charts/<chart-name>/`. When a chart is vendored, the overlay's `kustomization.yaml` references the local path. When not vendored, it references the OCI or HTTP repo directly.
Current Kubernetes target version: **1.33.7** (used by kubeconform in CI).
---
## Project Boundaries
| Project | ApplicationSet | App pattern |
|------------|---------------------------|--------------------------|
| `platform` | `argocd/applicationsets/platform.yaml` | Named apps (cert-manager, puppet, woodpecker, etc.) |
| `storage` | `argocd/applicationsets/storage.yaml` | `csi-*` apps |
The `clusters/au-syd1/apps/` entry-point is deployed as a standalone ArgoCD `Application` (not an ApplicationSet) called `au-syd1-apps`.
---
## CI / Pre-commit Hooks
Runs on every PR via Woodpecker CI (`.woodpecker/`):
| Check | Tool | Trigger |
|---|---|---|
| YAML lint + general file checks | `pre-commit` (yamllint + pre-commit-hooks) | PR |
| No plain Secrets | `ci/validate-no-secrets.sh` | PR (staged files) |
| Kubernetes manifest validation | `kubeconform` via `make kubeconform` | PR |
kubeconform skips: `CustomResourceDefinition`, `GpuDevicePlugin` (for apps validation).
---
## Git Workflow
- Branch naming: `benvin/<app-name>` (user prefix)
- **Never `git add .`** — add only relevant files explicitly
- If pre-commit modifies files, `git add -u` then `git commit --amend --no-edit`
- Use `git push --force-with-lease` after amending
---
## Security Policies
- `reloader.stakater.com/auto: "true"` annotation triggers rolling restarts on ConfigMap/Secret changes
- Security contexts follow least-privilege: `drop: [all]` then add only required capabilities
- `fsGroup: 999` on pod security context for Puppet workloads
- `runAsUser: 0` is used only for init containers that need to set file permissions, then regular containers run as non-root
+4
View File
@@ -6,6 +6,10 @@ build:
@mkdir -p manifests/$(filter-out $@,$(MAKECMDGOALS))
@kustomize build --enable-helm $(filter-out $@,$(MAKECMDGOALS)) --output manifests/$(filter-out $@,$(MAKECMDGOALS))
# Generate JSON schemas from CRDs and Kubernetes swagger spec (run manually, results committed)
schemas:
@ci/generate-schemas.sh schemas
# kubeconform
kubeconform:
@ci/validate-apps.sh && \
+45
View File
@@ -0,0 +1,45 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: age-api
namespace: age-api
spec:
replicas: 1
selector:
matchLabels:
app: age-api
template:
metadata:
annotations:
reloader.stakater.com/auto: "true"
labels:
app: age-api
spec:
containers:
- name: age-api
image: git.unkin.net/unkin/age-api:v0.1.0
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
name: http
protocol: TCP
env:
- name: CONFIG_PATH
value: /etc/age-api/config.yaml
resources:
limits:
cpu: 100m
memory: 64Mi
requests:
cpu: 10m
memory: 32Mi
volumeMounts:
- mountPath: /etc/age-api/config.yaml
name: config
subPath: config.yaml
restartPolicy: Always
volumes:
- name: config
configMap:
name: age-api-config
@@ -6,26 +6,26 @@ metadata:
traefik.io/instance: internal
annotations:
cert-manager.io/cluster-issuer: vault-issuer
cert-manager.io/common-name: rspamd.k8s.syd1.au.unkin.net
cert-manager.io/common-name: age-api.k8s.syd1.au.unkin.net
cert-manager.io/private-key-size: "4096"
external-dns.alpha.kubernetes.io/hostname: rspamd.k8s.syd1.au.unkin.net
external-dns.alpha.kubernetes.io/hostname: age-api.k8s.syd1.au.unkin.net
external-dns.alpha.kubernetes.io/target: 198.18.200.4
name: rspamd
namespace: mailgateway
name: age-api
namespace: age-api
spec:
gatewayClassName: traefik-internal
listeners:
- allowedRoutes:
namespaces:
from: Same
hostname: rspamd.k8s.syd1.au.unkin.net
hostname: age-api.k8s.syd1.au.unkin.net
name: http
port: 80
protocol: HTTP
- allowedRoutes:
namespaces:
from: Same
hostname: rspamd.k8s.syd1.au.unkin.net
hostname: age-api.k8s.syd1.au.unkin.net
name: https
port: 443
protocol: HTTPS
@@ -33,5 +33,5 @@ spec:
certificateRefs:
- group: ""
kind: Secret
name: rspamd-tls
name: age-api-tls
mode: Terminate
+49
View File
@@ -0,0 +1,49 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: age-api-http-redirect
namespace: age-api
spec:
hostnames:
- age-api.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: age-api
sectionName: http
rules:
- filters:
- type: RequestRedirect
requestRedirect:
scheme: https
statusCode: 301
matches:
- path:
type: PathPrefix
value: /
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: age-api
namespace: age-api
spec:
hostnames:
- age-api.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: age-api
sectionName: https
rules:
- backendRefs:
- group: ""
kind: Service
name: age-api
port: 80
weight: 1
matches:
- path:
type: PathPrefix
value: /
+17
View File
@@ -0,0 +1,17 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
- gateway.yaml
- httproute.yaml
- namespace.yaml
- service.yaml
configMapGenerator:
- name: age-api-config
files:
- config.yaml=resources/config.yaml
options:
disableNameSuffixHash: true
@@ -2,4 +2,4 @@
apiVersion: v1
kind: Namespace
metadata:
name: mailgateway
name: age-api
+7
View File
@@ -0,0 +1,7 @@
people:
- name: jaidi
birthtime: 1773135720
- name: ben
birthtime: 559663200
- name: sudaporn
birthtime: 686757600
+17
View File
@@ -0,0 +1,17 @@
---
apiVersion: v1
kind: Service
metadata:
name: age-api
namespace: age-api
spec:
internalTrafficPolicy: Cluster
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
selector:
app: age-api
sessionAffinity: None
type: ClusterIP
@@ -2,24 +2,40 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: artifactapi-deployment
name: api
namespace: artifactapi
annotations:
reloader.stakater.com/auto: "true"
spec:
selector:
matchLabels:
app: artifactapi
app: api
strategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
app: api
spec:
automountServiceAccountToken: true
initContainers:
- name: combine-certs
image: alpine:3
command:
- sh
- -c
- cat /etc/ssl/certs/ca-certificates.crt /custom-ca/ca.crt > /combined-certs/ca-certificates.crt
volumeMounts:
- name: vault-ca-cert
mountPath: /custom-ca
readOnly: true
- name: combined-certs
mountPath: /combined-certs
containers:
- name: artifactapi
image: git.unkin.net/unkin/artifactapi:v2.7.2
- name: api
image: git.unkin.net/unkin/artifactapi:v3.6.5
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8000
@@ -27,11 +43,15 @@ spec:
protocol: TCP
envFrom:
- configMapRef:
name: artifactapi-env
name: api-env
optional: false
- secretRef:
name: environment
optional: false
volumeMounts:
- name: combined-certs
mountPath: /etc/ssl/combined
readOnly: true
livenessProbe:
failureThreshold: 3
httpGet:
@@ -59,34 +79,13 @@ spec:
requests:
cpu: 100m
memory: 256Mi
volumeMounts:
- mountPath: /etc/artifactapi/conf.d/config.yaml
name: remotes-config
subPath: config.yaml
- mountPath: /etc/artifactapi/conf.d/local-generic.yaml
name: remotes-config
subPath: local-generic.yaml
- mountPath: /etc/artifactapi/conf.d/remote-alpine.yaml
name: remotes-config
subPath: remote-alpine.yaml
- mountPath: /etc/artifactapi/conf.d/remote-docker.yaml
name: remotes-config
subPath: remote-docker.yaml
- mountPath: /etc/artifactapi/conf.d/remote-generic.yaml
name: remotes-config
subPath: remote-generic.yaml
- mountPath: /etc/artifactapi/conf.d/remote-helm.yaml
name: remotes-config
subPath: remote-helm.yaml
- mountPath: /etc/artifactapi/conf.d/remote-rpm.yaml
name: remotes-config
subPath: remote-rpm.yaml
- mountPath: /etc/artifactapi/conf.d/virtual-helm.yaml
name: remotes-config
subPath: virtual-helm.yaml
restartPolicy: Always
volumes:
- configMap:
name: remotes-config
optional: false
name: remotes-config
- name: vault-ca-cert
secret:
secretName: vault-ca-cert
items:
- key: ca.crt
path: ca.crt
- name: combined-certs
emptyDir: {}
restartPolicy: Always
@@ -2,13 +2,13 @@
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: artifactapi-hpa
name: api-hpa
namespace: artifactapi
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: artifactapi-deployment
name: api
minReplicas: 2
maxReplicas: 10
metrics:
+91
View File
@@ -0,0 +1,91 @@
---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: postgres
namespace: artifactapi
spec:
affinity:
podAntiAffinityType: preferred
bootstrap:
initdb:
database: artifacts
encoding: UTF8
localeCType: C
localeCollate: C
owner: artifacts
secret:
name: postgres-credentials
enablePDB: true
enableSuperuserAccess: false
failoverDelay: 0
imageName: ghcr.io/cloudnative-pg/postgresql:18.1-system-trixie
instances: 3
logLevel: info
maxSyncReplicas: 0
minSyncReplicas: 0
monitoring:
customQueriesConfigMap:
- key: queries
name: cnpg-default-monitoring
disableDefaultQueries: false
enablePodMonitor: false
postgresql:
parameters:
archive_mode: "on"
archive_timeout: 5min
dynamic_shared_memory_type: posix
effective_cache_size: 256MB
full_page_writes: "on"
log_destination: csvlog
log_directory: /controller/log
log_filename: postgres
log_rotation_age: "0"
log_rotation_size: "0"
log_truncate_on_rotation: "false"
logging_collector: "on"
max_connections: "200"
max_parallel_workers: "16"
max_replication_slots: "16"
max_worker_processes: "16"
shared_buffers: 128MB
shared_memory_type: mmap
ssl_max_protocol_version: TLSv1.3
ssl_min_protocol_version: TLSv1.3
wal_keep_size: 256MB
wal_level: logical
wal_log_hints: "on"
wal_receiver_timeout: 5s
wal_sender_timeout: 5s
syncReplicaElectionConstraint:
enabled: false
primaryUpdateMethod: restart
primaryUpdateStrategy: unsupervised
probes:
liveness:
isolationCheck:
connectionTimeout: 1000
enabled: true
requestTimeout: 1000
replicationSlots:
highAvailability:
enabled: true
slotPrefix: _cnpg_
synchronizeReplicas:
enabled: true
updateInterval: 30
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 250m
memory: 256Mi
smartShutdownTimeout: 180
startDelay: 3600
stopDelay: 1800
storage:
resizeInUseVolumes: true
size: 20Gi
storageClass: cephrbd-fast-delete
switchoverDelay: 3600
+33
View File
@@ -0,0 +1,33 @@
---
apiVersion: postgresql.cnpg.io/v1
kind: Pooler
metadata:
name: postgres-pooler
namespace: artifactapi
spec:
cluster:
name: postgres
instances: 2
pgbouncer:
parameters:
default_pool_size: "100"
max_client_conn: "400"
paused: false
poolMode: session
template:
metadata:
labels:
app: pooler
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- pooler
topologyKey: kubernetes.io/hostname
containers: []
type: rw
+5 -16
View File
@@ -2,26 +2,15 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: artifactapi-env
name: api-env
namespace: artifactapi
data:
CONFIG_PATH: /etc/artifactapi/conf.d/
DBHOST: postgres-service
DBHOST: postgres-pooler
DBNAME: artifacts
DBPORT: "5432"
DBUSER: artifacts
MINIO_BUCKET: artifactapi
MINIO_BUCKET: artifactapi-prod-k8s-syd1-au
MINIO_ENDPOINT: radosgw.service.consul
MINIO_SECURE: "true"
REDIS_URL: redis://redis-service:6379
REQUESTS_CA_BUNDLE: /etc/pki/tls/certs/ca-bundle.crt
SSL_CERT_FILE: /etc/pki/tls/certs/ca-bundle.crt
---
apiVersion: v1
kind: ConfigMap
metadata:
name: postgres-env
namespace: artifactapi
data:
POSTGRES_DB: artifacts
POSTGRES_USER: artifacts
REDIS_URL: redis://redis:6379
SSL_CERT_FILE: /etc/ssl/combined/ca-certificates.crt
+22 -5
View File
@@ -2,13 +2,15 @@
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: artifactapi-http-redirect
name: http-redirect
namespace: artifactapi
spec:
hostnames:
- artifactapi.k8s.syd1.au.unkin.net
parentRefs:
- name: artifactapi
- group: gateway.networking.k8s.io
kind: Gateway
name: artifactapi
sectionName: http
rules:
- filters:
@@ -24,18 +26,33 @@ spec:
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: artifactapi
name: api-route
namespace: artifactapi
spec:
hostnames:
- artifactapi.k8s.syd1.au.unkin.net
parentRefs:
- name: artifactapi
- group: gateway.networking.k8s.io
kind: Gateway
name: artifactapi
sectionName: https
rules:
- backendRefs:
- name: artifactapi-api
- group: ""
kind: Service
name: ui
port: 80
weight: 1
matches:
- path:
type: PathPrefix
value: /ui
- backendRefs:
- group: ""
kind: Service
name: artifactapi
port: 80
weight: 1
matches:
- path:
type: PathPrefix
+6 -18
View File
@@ -3,29 +3,17 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- artifactapi-deployment.yaml
- artifactapi-hpa.yaml
- api-deployment.yaml
- api-hpa.yaml
- configmap.yaml
- cnpg_cluster.yaml
- cnpg_pooler.yaml
- gateway.yaml
- httproute.yaml
- namespace.yaml
- postgres-deployment.yaml
- pvc.yaml
- redis-deployment.yaml
- services.yaml
- ui-deployment.yaml
- ui-hpa.yaml
- vaultauth.yaml
- vaultstaticsecret.yaml
configMapGenerator:
- name: remotes-config
files:
- resources/conf.d/config.yaml
- resources/conf.d/local-generic.yaml
- resources/conf.d/remote-generic.yaml
- resources/conf.d/remote-alpine.yaml
- resources/conf.d/remote-rpm.yaml
- resources/conf.d/remote-docker.yaml
- resources/conf.d/remote-helm.yaml
- resources/conf.d/virtual-helm.yaml
options:
disableNameSuffixHash: true
@@ -1,76 +0,0 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgres-deployment
namespace: artifactapi
annotations:
reloader.stakater.com/auto: "true"
spec:
replicas: 1
selector:
matchLabels:
app: postgres
strategy:
type: Recreate
template:
spec:
automountServiceAccountToken: true
containers:
- name: postgres
image: postgres:15-alpine
imagePullPolicy: IfNotPresent
ports:
- containerPort: 5432
name: postgres
protocol: TCP
envFrom:
- configMapRef:
name: postgres-env
optional: false
- secretRef:
name: postgres-password
optional: false
readinessProbe:
exec:
command:
- pg_isready
- -U
- artifacts
- -d
- artifacts
failureThreshold: 3
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
livenessProbe:
exec:
command:
- pg_isready
- -U
- artifacts
- -d
- artifacts
failureThreshold: 3
initialDelaySeconds: 30
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 5
resources:
limits:
cpu: 500m
memory: 1Gi
requests:
cpu: 50m
memory: 128Mi
volumeMounts:
- mountPath: /var/lib/postgresql/data
mountPropagation: None
name: pgdata
subPath: pgdata
restartPolicy: Always
volumes:
- name: pgdata
persistentVolumeClaim:
claimName: artifactapi-postgres-pgdata
-28
View File
@@ -1,28 +0,0 @@
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: artifactapi-postgres-pgdata
namespace: artifactapi
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
storageClassName: cephrbd-fast-delete
volumeMode: Filesystem
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: artifactapi-redis-data
namespace: artifactapi
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
storageClassName: cephrbd-fast-delete
volumeMode: Filesystem
+11 -21
View File
@@ -2,23 +2,21 @@
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
name: redis-deployment
name: redis
namespace: artifactapi
spec:
replicas: 1
selector:
matchLabels:
app: redis
strategy:
type: Recreate
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: redis:7-alpine
imagePullPolicy: IfNotPresent
command:
- redis-server
- --save
@@ -28,6 +26,13 @@ spec:
- containerPort: 6379
name: redis
protocol: TCP
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 50m
memory: 128Mi
livenessProbe:
exec:
command:
@@ -48,19 +53,4 @@ spec:
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 50m
memory: 128Mi
volumeMounts:
- mountPath: /data
mountPropagation: None
name: data
restartPolicy: Always
volumes:
- name: data
persistentVolumeClaim:
claimName: artifactapi-redis-data
@@ -1,3 +0,0 @@
# Global artifactapi configuration.
# S3, Redis, and database connection settings are injected via environment variables.
# Add any top-level overrides here if needed.
@@ -1,7 +0,0 @@
locals:
local-generic:
package: "generic"
description: "Local generic file repository"
cache:
immutable_ttl: 0
mutable_ttl: 0
@@ -1,10 +0,0 @@
remotes:
alpine:
base_url: "https://dl-cdn.alpinelinux.org"
package: "alpine"
description: "Alpine Linux APK package repository"
immutable_patterns:
- ".*/x86_64/.*\\.apk$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
@@ -1,98 +0,0 @@
remotes:
ghcr:
base_url: "https://ghcr.io"
package: "docker"
description: "GitHub Container Registry"
immutable_patterns:
- "^cloudnative-pg/cloudnative-pg"
- "^emberstack/helm-charts"
- "^kanidm/"
- "^openvoxproject/"
- "^stakater/reloader"
- "^voxpupuli/puppetboard"
- "^woodpecker-ci/helm"
cache:
immutable_ttl: 0
mutable_ttl: 300
dockerhub:
base_url: "https://registry-1.docker.io"
package: "docker"
description: "Docker Hub registry"
immutable_patterns:
- "^library/almalinux"
- "^library/busybox"
- "^library/debian"
- "^library/fedora"
- "^library/nginx"
- "^library/postgres"
- "^library/redis"
- "^beats/filebeat"
- "^bitnami/"
- "^curlimages/curl"
- "^emberstack/kubernetes-reflector"
- "^hashicorp/consul"
- "^hashicorp/vault"
- "^jfrog/"
- "^rancher/"
- "^traefik/"
- "^ubi9/ubi-minimal"
- "^victoriametrics/"
- "^woodpeckerci/"
cache:
immutable_ttl: 0
mutable_ttl: 300
quay:
base_url: "https://quay.io"
package: "docker"
description: "Quay.io container registry"
immutable_patterns:
- "^brancz/kube-rbac-proxy"
- "^cephcsi/cephcsi"
- "^jetstack/cert-manager-"
cache:
immutable_ttl: 0
mutable_ttl: 300
k8s-registry:
base_url: "https://registry.k8s.io"
package: "docker"
description: "Kubernetes container registry"
immutable_patterns:
- "^external-dns/external-dns"
- "^sig-storage/"
cache:
immutable_ttl: 0
mutable_ttl: 300
gitlab:
base_url: "https://registry.gitlab.com"
package: "docker"
description: "GitLab container registry"
immutable_patterns:
- "^purelb/purelb"
cache:
immutable_ttl: 0
mutable_ttl: 300
elastic:
base_url: "https://docker.elastic.co"
package: "docker"
description: "Elastic container registry"
immutable_patterns:
- "^eck/eck-operator"
cache:
immutable_ttl: 0
mutable_ttl: 300
gcr:
base_url: "https://gcr.io"
package: "docker"
description: "Google Container Registry"
immutable_patterns:
- "^k8s-staging-nfd/charts"
- "^k8s-staging-nfd/node-feature-discovery"
cache:
immutable_ttl: 0
mutable_ttl: 300
@@ -1,130 +0,0 @@
remotes:
github:
base_url: "https://github.com"
package: "generic"
description: "GitHub releases and files"
mutable_patterns:
- ".*/archive/refs/heads/.*.tar.gz$"
immutable_patterns:
- ".*/archive/refs/tags/.*.tar.gz$"
- "ahmetb/kubectx/.*/kubectx_.*_linux_x86_64.tar.gz$"
- "ahmetb/kubectx/.*/kubens_.*_linux_x86_64.tar.gz$"
- "apple/foundationdb/.*/libfdb_c.x86_64.so$"
- "astral-sh/ruff/.*/ruff-x86_64-unknown-linux-gnu.tar.gz$"
- "astral-sh/uv/.*/uv-x86_64-unknown-linux-gnu.tar.gz$"
- "camptocamp/prometheus-puppetdb-exporter/.*/prometheus-puppetdb-exporter-.*.linux-amd64.tar.gz$"
- "coder/code-server/.*/code-server-.*-amd64.rpm$"
- "containernetworking/plugins/.*/cni-plugins-linux-amd64-.*.tgz"
- "dandavison/delta/.*/delta-.*-x86_64-unknown-linux-musl.tar.gz$"
- "ducaale/xh/.*/xh-.*-x86_64-unknown-linux-musl.tar.gz$"
- "etcd-io/etcd/.*/etcd-.*-linux-amd64.tar.gz$"
- "getsops/sops/.*/sops-v.*\\.linux\\.amd64$"
- "grafana/jsonnet-language-server/.*/jsonnet-language-server_.*_linux_amd64$"
- "gruntwork-io/boilerplate/.*/boilerplate_linux_amd64$"
- "gruntwork-io/terragrunt/.*terragrunt_linux_amd64.*"
- "hadolint/hadolint/.*/hadolint-linux-x86_64$"
- "helmfile/helmfile/.*/helmfile_.*_linux_amd64.tar.gz$"
- "helmfile/vals/.*/vals_.*_linux_amd64.tar.gz$"
- "jesseduffield/lazydocker/.*/lazydocker_.*_Linux_x86_64.tar.gz$"
- "kubecolor/kubecolor/.*/kubecolor_.*_linux_amd64.tar.gz$"
- "kubernetes-sigs/gateway-api/.*/standard-install.yaml$"
- "kubernetes-sigs/kustomize/.*/kustomize_.*_linux_amd64.tar.gz$"
- "lxc/incus/.*.tar.gz$"
- "mikefarah/yq/.*/yq_linux_amd64$"
- "neovim/neovim-releases/.*/nvim-linux-x86_64.tar.gz$"
- "neovim/neovim/.*/nvim-linux-x86_64.tar.gz$"
- "nzbgetcom/nzbget/.*/nzbget-.*.x86_64.rpm$"
- "onedr0p/exportarr/.*/exportarr_.*_linux_amd64.tar.gz$"
- "openbao/openbao-plugins/.*/openbao-plugin-secrets-consul_linux_amd64_.*.tar.gz$"
- "openbao/openbao-plugins/.*/openbao-plugin-secrets-nomad_linux_amd64_.*.tar.gz$"
- "prometheus-community/bind_exporter/.*/bind_exporter-.*.linux-amd64.tar.gz$"
- "prometheus-community/pgbouncer_exporter/.*/pgbouncer_exporter-.*.linux-amd64.tar.gz$"
- "prometheus-community/postgres_exporter/.*/postgres_exporter-.*.linux-amd64.tar.gz$"
- "prometheus/node_exporter/.*/node_exporter-.*.linux-amd64.tar.gz$"
- "rancher/rke2/.*/rke2-images.linux-amd64.tar.zst$"
- "stalwartlabs/stalwart/.*/stalwart-cli-x86_64-unknown-linux-gnu.tar.gz$"
- "stalwartlabs/stalwart/.*/stalwart-foundationdb-x86_64-unknown-linux-gnu.tar.gz$"
- "stalwartlabs/stalwart/.*/stalwart-x86_64-unknown-linux-gnu.tar.gz$"
- "starship/starship/.*/starship-x86_64-unknown-linux-musl.tar.gz$"
- "stern/stern/.*/stern_.*_linux_amd64.tar.gz$"
- "terraform-linters/tflint/.*/tflint_linux_amd64.zip$"
- "tynany/frr_exporter/.*/frr_exporter-.*.linux-amd64.tar.gz$"
- "VictoriaMetrics/VictoriaLogs/.*/victoria-logs-linux-amd64-.*.tar.gz$"
- "VictoriaMetrics/VictoriaLogs/.*/vlutils-linux-amd64-.*.tar.gz$"
- "VictoriaMetrics/VictoriaMetrics/.*/victoria-logs-linux-amd64-.*.tar.gz$"
- "VictoriaMetrics/VictoriaMetrics/.*/victoria-metrics-linux-amd64-.*-cluster.tar.gz$"
- "VictoriaMetrics/VictoriaMetrics/.*/vlutils-linux-amd64-.*.tar.gz$"
- "VictoriaMetrics/VictoriaMetrics/.*/vmutils-linux-amd64-.*.tar.gz$"
- "xorpaul/g10k/.*/g10k-.*-linux-amd64.zip$"
- "yannh/kubeconform/.*/kubeconform-linux-amd64.tar.gz$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
github_user:
base_url: "https://raw.githubusercontent.com"
package: "generic"
description: "GitHub User Content"
immutable_patterns:
- "argoproj/argo-cd/.*.yaml$"
- "yannh/kubernetes-json-schema/master/.*.json$"
- "datreeio/CRDs-catalog/main/.*.json$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
gitea-dl:
base_url: "https://dl.gitea.com"
package: "generic"
description: "Gitea download site"
immutable_patterns:
- "act_runner/.*/act_runner-.*-linux-amd64$"
- "tea/.*/tea-.*-linux-amd64$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
hashicorp-releases:
base_url: "https://releases.hashicorp.com"
package: "generic"
description: "HashiCorp product releases"
immutable_patterns:
- "terraform/.*terraform_.*_linux_amd64\\.zip$"
- "terraform/.*terraform_.*_windows_amd64\\.zip$"
- "terraform/.*terraform_.*_darwin_amd64\\.zip$"
- "vault/.*vault_.*_linux_amd64\\.zip$"
- "vault/.*vault_.*_windows_amd64\\.zip$"
- "vault/.*vault_.*_darwin_amd64\\.zip$"
- "consul-cni/.*/consul-cni_.*_linux_amd64\\.zip$"
- "consul/.*/consul_.*_linux_amd64\\.zip$"
- "nomad-autoscaler/.*/nomad-autoscaler_.*_linux_amd64\\.zip$"
- "nomad/.*/nomad_.*_linux_amd64\\.zip$"
- "packer/.*/packer_.*_linux_amd64\\.zip$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
rarlab:
base_url: "https://www.rarlab.com"
package: "generic"
description: "RARLab"
immutable_patterns:
- "rar/rarlinux-x64-.*.tar.gz"
cache:
immutable_ttl: 0
mutable_ttl: 7200
claude-ai:
base_url: "https://downloads.claude.ai"
package: "generic"
description: "Anthropic Claude Code binary releases"
mutable_patterns:
- "claude-code-releases/.*/manifest.json$"
immutable_patterns:
- "claude-code-releases/.*/linux-x64/claude$"
- "claude-code-releases/.*/linux-arm64/claude$"
- "claude-code-releases/.*/linux-x64-musl/claude$"
- "claude-code-releases/.*/linux-arm64-musl/claude$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
@@ -1,143 +0,0 @@
remotes:
ceph-csi:
base_url: "https://ceph.github.io/csi-charts"
package: "helm"
description: "Ceph CSI driver Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
cnpg:
base_url: "https://cloudnative-pg.github.io/charts"
package: "helm"
description: "CloudNativePG operator Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
elastic-helm:
base_url: "https://helm.elastic.co"
package: "helm"
description: "Elastic stack Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
external-dns:
base_url: "https://kubernetes-sigs.github.io/external-dns/"
package: "helm"
description: "ExternalDNS Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
hashicorp-helm:
base_url: "https://helm.releases.hashicorp.com"
package: "helm"
description: "HashiCorp Helm charts (Vault Secrets Operator, etc.)"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
intel-helm:
base_url: "https://intel.github.io/helm-charts/"
package: "helm"
description: "Intel Helm charts (device plugins)"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
jetstack:
base_url: "https://charts.jetstack.io"
package: "helm"
description: "Jetstack Helm charts (cert-manager)"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
purelb:
base_url: "https://gitlab.com/api/v4/projects/20400619/packages/helm/stable"
package: "helm"
description: "PureLB load balancer Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
rancher-stable:
base_url: "https://releases.rancher.com/server-charts/stable"
package: "helm"
description: "Rancher stable Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
stakater:
base_url: "https://stakater.github.io/stakater-charts"
package: "helm"
description: "Stakater Helm charts (Reloader)"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
traefik:
base_url: "https://traefik.github.io/charts"
package: "helm"
description: "Traefik Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
victoriametrics:
base_url: "https://victoriametrics.github.io/helm-charts/"
package: "helm"
description: "VictoriaMetrics observability Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
argo-helm:
base_url: "https://argoproj.github.io/argo-helm"
package: "helm"
description: "Argo Project Helm charts (ArgoCD, Image Updater, Rollouts, etc.)"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
@@ -1,154 +0,0 @@
remotes:
almalinux:
base_url: "https://gsl-syd.mm.fcix.net/almalinux"
package: "rpm"
description: "AlmaLinux RPM package repository"
immutable_patterns:
- ".*/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*\\.sqlite.*$"
- ".*/repodata/.*\\.xml.*$"
- ".*/repodata/.*\\.yaml.*$"
- ".*/install.img"
- ".*/squashfs.img"
- ".*/updates.img"
- ".*/RPM-GPG-KEY-.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
ceph-reef:
base_url: "https://download.ceph.com/rpm-reef/"
package: "rpm"
description: "Ceph Reef 18"
immutable_patterns:
- ".*/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
ceph-squid:
base_url: "https://download.ceph.com/rpm-squid/"
package: "rpm"
description: "Ceph Squid 19"
immutable_patterns:
- ".*/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
ceph-tentacle:
base_url: "https://download.ceph.com/rpm-tentacle/"
package: "rpm"
description: "Ceph Tentacle 20"
immutable_patterns:
- ".*/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
epel:
base_url: "https://gsl-syd.mm.fcix.net/epel"
package: "rpm"
description: "EPEL (Extra Packages for Enterprise Linux)"
immutable_patterns:
- ".*/Everything/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*\\.sqlite.*$"
- ".*/repodata/.*\\.xml.*$"
- ".*/repodata/.*\\.yaml.*$"
- "RPM-GPG-KEY-.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
fedora:
base_url: "https://gsl-syd.mm.fcix.net/fedora/linux"
package: "rpm"
description: "Fedora Linux RPM package repository"
immutable_patterns:
- "releases/.*/Everything/x86_64/.*\\.rpm$"
- "updates/.*/Everything/x86_64/.*\\.rpm$"
- "development/.*/Everything/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
frr:
base_url: "https://rpm.frrouting.org/repo"
package: "rpm"
description: "FRR RPM package repository"
immutable_patterns:
- ".*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
mariadb:
base_url: "http://mariadb.mirror.digitalpacific.com.au/yum"
package: "rpm"
description: "MariaDB RPM package repository"
immutable_patterns:
- ".*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
- ".*/RPM-GPG-KEY-.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
openvox:
base_url: "https://yum.voxpupuli.org"
package: "rpm"
description: "OpenVox RPM package repository"
immutable_patterns:
- ".*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
- "GPG-KEY-.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
postgresql:
base_url: "https://download.postgresql.org/pub/repos/yum"
package: "rpm"
description: "PostgreSQL RPM package repository"
immutable_patterns:
- ".*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
- ".*/RPM-GPG-KEY-.*$"
- ".*/PGDG-RPM-GPG-KEY-.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
rke2:
base_url: "https://rpm.rancher.io"
package: "rpm"
description: "RKE2 RPM package repository"
immutable_patterns:
- ".*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
- "public.key$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
zfs:
base_url: "http://download.zfsonlinux.org"
package: "rpm"
description: "ZFS RPM package repository"
immutable_patterns:
- ".*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
cache:
immutable_ttl: 0
mutable_ttl: 7200
@@ -1,18 +0,0 @@
virtuals:
helm:
package: "helm"
description: "Virtual repository merging all helm remotes — member order is priority order for duplicate chart+version"
members:
- ceph-csi
- cnpg
- elastic-helm
- external-dns
- hashicorp-helm
- intel-helm
- jetstack
- purelb
- rancher-stable
- stakater
- traefik
- victoriametrics
- argo-helm
+8 -8
View File
@@ -2,7 +2,7 @@
apiVersion: v1
kind: Service
metadata:
name: artifactapi-api
name: artifactapi
namespace: artifactapi
spec:
internalTrafficPolicy: Cluster
@@ -12,31 +12,31 @@ spec:
protocol: TCP
targetPort: http
selector:
app: artifactapi
app: api
sessionAffinity: None
type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
name: postgres-service
name: ui
namespace: artifactapi
spec:
internalTrafficPolicy: Cluster
ports:
- name: postgres
port: 5432
- name: http
port: 80
protocol: TCP
targetPort: postgres
targetPort: http
selector:
app: postgres
app: ui
sessionAffinity: None
type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
name: redis-service
name: redis
namespace: artifactapi
spec:
internalTrafficPolicy: Cluster
+58
View File
@@ -0,0 +1,58 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: ui
namespace: artifactapi
annotations:
reloader.stakater.com/auto: "true"
spec:
selector:
matchLabels:
app: ui
strategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
app: ui
spec:
automountServiceAccountToken: true
containers:
- name: ui
image: git.unkin.net/unkin/artifactapi-ui:v3.6.5
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
name: http
protocol: TCP
livenessProbe:
failureThreshold: 3
httpGet:
path: /ui
port: http
scheme: HTTP
initialDelaySeconds: 15
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 5
readinessProbe:
failureThreshold: 3
httpGet:
path: /ui
port: http
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 5
successThreshold: 1
timeoutSeconds: 5
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 50m
memory: 128Mi
restartPolicy: Always
@@ -2,22 +2,22 @@
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: rspamd-hpa
namespace: mailgateway
name: ui-hpa
namespace: artifactapi
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: rspamd
name: ui
minReplicas: 2
maxReplicas: 6
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
averageUtilization: 60
behavior:
scaleUp:
stabilizationWindowSeconds: 0
@@ -34,5 +34,8 @@ spec:
selectPolicy: Min
policies:
- type: Percent
value: 30
value: 10
periodSeconds: 60
- type: Pods
value: 2
periodSeconds: 60
+1 -1
View File
@@ -10,7 +10,7 @@ spec:
kubernetes:
audiences:
- vault
role: artifactapi
role: default
serviceAccount: default
tokenExpirationSeconds: 600
method: kubernetes
+18 -18
View File
@@ -1,6 +1,23 @@
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: postgres-credentials
namespace: artifactapi
spec:
destination:
create: true
name: postgres-credentials
overwrite: true
hmacSecretData: true
mount: kv
path: kubernetes/namespace/artifactapi/default/postgres-credentials
refreshAfter: 5m
type: kv-v2
vaultAuthRef: default
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: environment
namespace: artifactapi
@@ -8,27 +25,10 @@ spec:
destination:
create: true
name: environment
overwrite: false
hmacSecretData: true
mount: kv
path: service/artifactapi/environment
refreshAfter: 5m
type: kv-v2
vaultAuthRef: default
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: postgres-password
namespace: artifactapi
spec:
destination:
create: true
name: postgres-password
overwrite: true
hmacSecretData: true
mount: kv
path: service/artifactapi/postgres-password
path: kubernetes/namespace/artifactapi/default/environment
refreshAfter: 5m
type: kv-v2
vaultAuthRef: default
+10 -3
View File
@@ -8,7 +8,9 @@ spec:
hostnames:
- rancher.k8s.syd1.au.unkin.net
parentRefs:
- name: rancher
- group: gateway.networking.k8s.io
kind: Gateway
name: rancher
sectionName: http
rules:
- filters:
@@ -30,12 +32,17 @@ spec:
hostnames:
- rancher.k8s.syd1.au.unkin.net
parentRefs:
- name: rancher
- group: gateway.networking.k8s.io
kind: Gateway
name: rancher
sectionName: https
rules:
- backendRefs:
- name: rancher
- group: ""
kind: Service
name: rancher
port: 80
weight: 1
matches:
- path:
type: PathPrefix
+17 -5
View File
@@ -11,7 +11,9 @@ spec:
hostnames:
- consul.k8s.syd1.au.unkin.net
parentRefs:
- name: consul
- group: gateway.networking.k8s.io
kind: Gateway
name: consul
sectionName: http
rules:
- filters:
@@ -36,12 +38,17 @@ spec:
hostnames:
- consul.k8s.syd1.au.unkin.net
parentRefs:
- name: consul
- group: gateway.networking.k8s.io
kind: Gateway
name: consul
sectionName: https
rules:
- backendRefs:
- name: consul-ui
- group: ""
kind: Service
name: consul-ui
port: 80
weight: 1
matches:
- path:
type: PathPrefix
@@ -59,12 +66,17 @@ spec:
hostnames:
- consul.service.consul
parentRefs:
- name: consul
- group: gateway.networking.k8s.io
kind: Gateway
name: consul
sectionName: consul-svc
rules:
- backendRefs:
- name: consul-ui
- group: ""
kind: Service
name: consul-ui
port: 80
weight: 1
matches:
- path:
type: PathPrefix
+51
View File
@@ -0,0 +1,51 @@
# kanidm
Three-replica kanidm identity server with Vault-managed replication certificates.
## Architecture
- Per-pod `server-N.toml` in `resources/` — each has its own replication origin hardcoded
- `config-init` busybox init container copies the right config and injects peer certs from the
vault-synced `kanidm-repl-certs` Secret at pod startup
- `reloader.stakater.com/auto: "true"` triggers a rolling restart when the ConfigMap or Secret changes
- Vault path: `kv/kubernetes/namespace/kanidm/default/repl-certs`
- Keys: `kanidm-0`, `kanidm-1`, `kanidm-2` — each holds that pod's replication certificate
## Initial setup
After the first pod starts, generate the admin credentials:
```bash
kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd recover-account -c /config/server.toml admin
kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd recover-account -c /config/server.toml idm_admin
```
## Replication certificate rotation
When certs need to be renewed, update vault and reloader will roll the StatefulSet:
```bash
# Get new cert from a pod
kubectl exec -it -n kanidm kanidm-N -- /sbin/kanidmd renew-replication-certificate -c /config/server.toml
# Write updated cert to vault (reloader triggers restart automatically)
vault kv patch kv/kubernetes/namespace/kanidm/default/repl-certs "kanidm-N=<cert>"
```
## Resolving domain UUID mismatch
If pods initialized independently (each with a different domain UUID), replication will fail with
`Consumer Domain UUID does not match`. Fix by resetting kanidm-1 and kanidm-2 to sync from
kanidm-0 (the authoritative node):
```bash
# Scale down to avoid split-brain during reset
kubectl scale statefulset -n kanidm kanidm --replicas=1
# Delete the stale PVCs for the replica pods
kubectl delete pvc -n kanidm data-kanidm-1 data-kanidm-2
# Scale back up — replicas start with empty DBs and automatic_refresh=true
# will trigger a full sync from kanidm-0 once TLS peer certs are verified
kubectl scale statefulset -n kanidm kanidm --replicas=3
```
+26
View File
@@ -0,0 +1,26 @@
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: kanidm-tls
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
secretName: kanidm-tls
issuerRef:
kind: ClusterIssuer
name: vault-issuer
commonName: auth.unkin.net
dnsNames:
- auth.unkin.net
- au.auth.unkin.net
- kanidm.k8s.syd1.au.unkin.net
- kanidm.kanidm.svc.cluster.local
- kanidm-0.kanidm-headless.kanidm.svc.cluster.local
- kanidm-1.kanidm-headless.kanidm.svc.cluster.local
- kanidm-2.kanidm-headless.kanidm.svc.cluster.local
privateKey:
algorithm: RSA
size: 4096
+30
View File
@@ -0,0 +1,30 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: kanidm
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
traefik.io/instance: internal
annotations:
external-dns.alpha.kubernetes.io/hostname: kanidm.k8s.syd1.au.unkin.net
external-dns.alpha.kubernetes.io/target: 198.18.200.4
spec:
gatewayClassName: traefik-internal
listeners:
- name: http
port: 80
protocol: HTTP
allowedRoutes:
namespaces:
from: Same
- name: https-passthrough
port: 443
protocol: TLS
tls:
mode: Passthrough
allowedRoutes:
namespaces:
from: Same
+29
View File
@@ -0,0 +1,29 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: kanidm-http-redirect
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
hostnames:
- kanidm.k8s.syd1.au.unkin.net
- auth.unkin.net
- au.auth.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: kanidm
sectionName: http
rules:
- filters:
- type: RequestRedirect
requestRedirect:
scheme: https
statusCode: 301
matches:
- path:
type: PathPrefix
value: /
+27
View File
@@ -0,0 +1,27 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
- serviceaccount.yaml
- vaultauth.yaml
- vaultstaticsecret.yaml
- certificate.yaml
- service.yaml
- statefulset.yaml
- poddisruptionbudget.yaml
- gateway.yaml
- httproute.yaml
- tlsroute.yaml
configMapGenerator:
- name: kanidm-config
namespace: kanidm
options:
disableNameSuffixHash: true
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
files:
- server-0.toml=resources/server-0.toml
+5
View File
@@ -0,0 +1,5 @@
---
apiVersion: v1
kind: Namespace
metadata:
name: kanidm
+15
View File
@@ -0,0 +1,15 @@
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: kanidm
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
maxUnavailable: 1
selector:
matchLabels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
+37
View File
@@ -0,0 +1,37 @@
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: kanidm-repl
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create"]
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["kanidm-repl-certs"]
verbs: ["get", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: kanidm-repl
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
subjects:
- kind: ServiceAccount
name: kanidm
namespace: kanidm
roleRef:
kind: Role
name: kanidm-repl
apiGroup: rbac.authorization.k8s.io
+15
View File
@@ -0,0 +1,15 @@
version = "2"
domain = "auth.unkin.net"
origin = "https://auth.unkin.net"
bindaddress = "[::]:8443"
db_path = "/data/kanidm.db"
db_arc_size = 2048
tls_chain = "/data/tls/tls.crt"
tls_key = "/data/tls/tls.key"
log_level = "info"
[online_backup]
path = "/data/backups/"
schedule = "0 22 * * *"
versions = 7
+43
View File
@@ -0,0 +1,43 @@
---
apiVersion: v1
kind: Service
metadata:
name: kanidm
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
type: ClusterIP
sessionAffinity: ClientIP
sessionAffinityConfig:
clientIP:
timeoutSeconds: 10800
ports:
- name: https
port: 8443
targetPort: https
protocol: TCP
selector:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
---
apiVersion: v1
kind: Service
metadata:
name: kanidm-headless
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
type: ClusterIP
clusterIP: None
ports:
- name: https
port: 8443
targetPort: https
protocol: TCP
selector:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
+9
View File
@@ -0,0 +1,9 @@
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: kanidm
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
+85
View File
@@ -0,0 +1,85 @@
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: kanidm
namespace: kanidm
annotations:
reloader.stakater.com/auto: "true"
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
serviceName: kanidm-headless
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
template:
metadata:
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
serviceAccountName: kanidm
securityContext:
runAsUser: 1000
runAsGroup: 1000
runAsNonRoot: true
fsGroup: 1000
containers:
- name: kanidm
image: kanidm/server:1.10.3
command: ["/sbin/kanidmd"]
args: ["server", "-c", "/config/server.toml"]
ports:
- name: https
containerPort: 8443
protocol: TCP
volumeMounts:
- name: data
mountPath: /data
- name: config
mountPath: /config/server.toml
subPath: server-0.toml
readOnly: true
- name: tls
mountPath: /data/tls
readOnly: true
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: false
resources:
requests:
memory: 256Mi
cpu: 100m
limits:
memory: 1Gi
cpu: 500m
readinessProbe:
tcpSocket:
port: 8443
initialDelaySeconds: 15
periodSeconds: 10
livenessProbe:
tcpSocket:
port: 8443
initialDelaySeconds: 30
periodSeconds: 30
volumes:
- name: config
configMap:
name: kanidm-config
- name: tls
secret:
secretName: kanidm-tls
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ReadWriteOnce]
storageClassName: cephrbd-fast-delete
resources:
requests:
storage: 10Gi
+26
View File
@@ -0,0 +1,26 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: TLSRoute
metadata:
name: kanidm
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
hostnames:
- kanidm.k8s.syd1.au.unkin.net
- auth.unkin.net
- au.auth.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: kanidm
sectionName: https-passthrough
rules:
- backendRefs:
- group: ""
kind: Service
name: kanidm
port: 8443
weight: 1
@@ -3,10 +3,10 @@ apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
name: default
namespace: mailgateway
namespace: kanidm
spec:
allowedNamespaces:
- mailgateway
- kanidm
kubernetes:
audiences:
- vault
+23
View File
@@ -0,0 +1,23 @@
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: repl-certs
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
vaultAuthRef: default
mount: kv
type: kv-v2
path: kubernetes/namespace/kanidm/default/repl-certs
refreshAfter: 5m
destination:
name: kanidm-repl-certs
create: true
overwrite: true
hmacSecretData: true
rolloutRestartTargets:
- kind: StatefulSet
name: kanidm
+2 -2
View File
@@ -76,8 +76,8 @@ spec:
updateInterval: 30
resources:
limits:
cpu: 1
memory: 1024Mi
cpu: "1"
memory: 1Gi
requests:
cpu: 250m
memory: 512Mi
+10 -3
View File
@@ -8,7 +8,9 @@ spec:
hostnames:
- litellm.k8s.syd1.au.unkin.net
parentRefs:
- name: litellm
- group: gateway.networking.k8s.io
kind: Gateway
name: litellm
sectionName: http
rules:
- filters:
@@ -30,12 +32,17 @@ spec:
hostnames:
- litellm.k8s.syd1.au.unkin.net
parentRefs:
- name: litellm
- group: gateway.networking.k8s.io
kind: Gateway
name: litellm
sectionName: https
rules:
- backendRefs:
- name: litellm
- group: ""
kind: Service
name: litellm
port: 4000
weight: 1
matches:
- path:
type: PathPrefix
-18
View File
@@ -1,18 +0,0 @@
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: postfix-smtp-tls
namespace: mailgateway
spec:
secretName: postfix-smtp-tls
issuerRef:
name: vault-issuer
kind: ClusterIssuer
commonName: mail.main.unkin.net
dnsNames:
- mail.main.unkin.net
- smtp-in.main.unkin.net
privateKey:
size: 4096
algorithm: RSA
-16
View File
@@ -1,16 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: rspamd
namespace: mailgateway
spec:
parentRefs:
- name: rspamd
namespace: mailgateway
hostnames:
- rspamd.k8s.syd1.au.unkin.net
rules:
- backendRefs:
- name: rspamd
port: 11334
-36
View File
@@ -1,36 +0,0 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- certificate.yaml
- gateway.yaml
- httproute.yaml
- namespace.yaml
- postfix-deployment.yaml
- postfix-hpa.yaml
- rspamd-deployment.yaml
- rspamd-hpa.yaml
- services.yaml
- vaultauth.yaml
- vaultstaticsecret.yaml
configMapGenerator:
- name: postfix-config
files:
- main.cf=resources/postfix/main.cf
- master.cf=resources/postfix/master.cf
options:
disableNameSuffixHash: true
- name: postfix-maps
files:
- transport=resources/postfix/transport
options:
disableNameSuffixHash: true
- name: rspamd-config
files:
- worker-proxy.inc=resources/rspamd/local.d/worker-proxy.inc
- dkim_signing.conf=resources/rspamd/local.d/dkim_signing.conf
- milter_headers.conf=resources/rspamd/local.d/milter_headers.conf
options:
disableNameSuffixHash: true
@@ -1,105 +0,0 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: postfix
namespace: mailgateway
spec:
selector:
matchLabels:
app: postfix
template:
metadata:
annotations:
reloader.stakater.com/auto: "true"
labels:
app: postfix
spec:
initContainers:
- name: postmap
image: tozd/postfix:alpine-322
command: ["/bin/sh", "-c"]
args:
- |
for f in /etc/postfix/maps/*; do
base=$(basename "$f")
cp "$f" /tmp/"$base"
postmap hash:/tmp/"$base"
cp /tmp/"${base}.db" /etc/postfix/db/
done
volumeMounts:
- name: postfix-maps
mountPath: /etc/postfix/maps
readOnly: true
- name: postfix-db
mountPath: /etc/postfix/db
containers:
- name: postfix
image: tozd/postfix:alpine-322
ports:
- containerPort: 25
name: smtp
protocol: TCP
- containerPort: 587
name: submission
protocol: TCP
env:
# Keep these in sync with main.cf so the tozd startup postconf calls are no-ops
- name: MAILNAME
value: "mail.main.unkin.net"
- name: MY_NETWORKS
value: "127.0.0.0/8 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16"
- name: MY_DESTINATION
value: "localhost.localdomain, localhost"
- name: LOG_TO_STDOUT
value: "1"
livenessProbe:
tcpSocket:
port: 25
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
tcpSocket:
port: 25
initialDelaySeconds: 15
periodSeconds: 10
timeoutSeconds: 3
failureThreshold: 3
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: "1"
memory: 512Mi
volumeMounts:
- name: postfix-config
mountPath: /etc/postfix/main.cf
subPath: main.cf
- name: postfix-config
mountPath: /etc/postfix/master.cf
subPath: master.cf
- name: postfix-db
mountPath: /etc/postfix/transport.db
subPath: transport.db
- name: postfix-tls
mountPath: /etc/postfix/tls
readOnly: true
- name: spool
mountPath: /var/spool/postfix
volumes:
- name: postfix-config
configMap:
name: postfix-config
- name: postfix-maps
configMap:
name: postfix-maps
- name: postfix-db
emptyDir: {}
- name: postfix-tls
secret:
secretName: postfix-smtp-tls
- name: spool
emptyDir: {}
-38
View File
@@ -1,38 +0,0 @@
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: postfix-hpa
namespace: mailgateway
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: postfix
minReplicas: 2
maxReplicas: 6
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
behavior:
scaleUp:
stabilizationWindowSeconds: 0
selectPolicy: Max
policies:
- type: Percent
value: 100
periodSeconds: 60
- type: Pods
value: 4
periodSeconds: 30
scaleDown:
stabilizationWindowSeconds: 300
selectPolicy: Min
policies:
- type: Percent
value: 30
periodSeconds: 60
@@ -1,46 +0,0 @@
# Basic identity — kept in sync with MAILNAME/MY_NETWORKS/MY_DESTINATION env vars
# so the tozd startup script's postconf calls are no-ops
myhostname = mail.main.unkin.net
myorigin = main.unkin.net
mydestination = localhost.localdomain, localhost
mynetworks = 127.0.0.0/8 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16
inet_protocols = ipv4
inet_interfaces = all
# No local delivery — we're a relay-only gateway
local_transport = error:no local delivery
alias_maps =
alias_database =
# Relay inbound mail for these domains to Stalwart
relay_domains = main.unkin.net unkin.net
transport_maps = hash:/etc/postfix/transport
# rspamd milter (same namespace — short DNS name resolves)
smtpd_milters = inet:rspamd:11332
non_smtpd_milters = inet:rspamd:11332
milter_default_action = accept
milter_protocol = 6
milter_mail_macros = i {mail_addr} {client_addr} {client_name} {auth_authen}
# Inbound TLS (cert from cert-manager Certificate resource)
smtpd_use_tls = yes
smtpd_tls_security_level = may
smtpd_tls_cert_file = /etc/postfix/tls/tls.crt
smtpd_tls_key_file = /etc/postfix/tls/tls.key
smtpd_tls_loglevel = 1
# Outbound TLS (opportunistic)
smtp_tls_security_level = may
smtp_tls_loglevel = 1
# Message size limit (50 MiB)
message_size_limit = 52428800
mailbox_size_limit = 0
# Queue retention
maximal_queue_lifetime = 5d
bounce_queue_lifetime = 1d
# Log to stdout for k8s log collection
maillog_file = /dev/stdout
@@ -1,42 +0,0 @@
# ==========================================================================
# service type private unpriv chroot wakeup maxproc command + args
# (yes) (yes) (yes) (never) (100)
# ==========================================================================
# SMTP inbound (port 25) — runs rspamd milter, relays to Stalwart via transport_maps
smtp inet n - n - - smtpd
# Submission (port 587) — TLS required, relay from trusted mynetworks only
submission inet n - n - - smtpd
-o syslog_name=postfix/submission
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=no
-o smtpd_reject_unlisted_recipient=no
-o smtpd_relay_restrictions=permit_mynetworks,reject
-o milter_macro_daemon_name=ORIGINATING
# Internal postfix processes
pickup unix n - n 60 1 pickup
cleanup unix n - n - 0 cleanup
qmgr unix n - n 300 1 qmgr
tlsmgr unix - - n 1000? 1 tlsmgr
rewrite unix - - n - - trivial-rewrite
bounce unix - - n - 0 bounce
defer unix - - n - 0 bounce
trace unix - - n - 0 bounce
verify unix - - n - 1 verify
flush unix n - n 1000? 0 flush
proxymap unix - - n - - proxymap
proxywrite unix - - n - 1 proxymap
smtp unix - - n - - smtp
relay unix - - n - - smtp
showq unix n - n - - showq
error unix - - n - - error
retry unix - - n - - error
discard unix - - n - - discard
local unix - n n - - local
virtual unix - n n - - virtual
lmtp unix - - n - - lmtp
anvil unix - - n - 1 anvil
scache unix - - n - 1 scache
postlog unix-dgram n - n - 1 postlogd
@@ -1,2 +0,0 @@
.unkin.net smtp:[stalwart.stalwart.svc.cluster.local]:25
unkin.net smtp:[stalwart.stalwart.svc.cluster.local]:25
@@ -1,13 +0,0 @@
enabled = true;
selector = "mail";
domain {
main.unkin.net {
privkey = "/etc/rspamd/dkim/private_key";
selector = "mail";
}
unkin.net {
privkey = "/etc/rspamd/dkim/private_key";
selector = "mail";
}
}
@@ -1,2 +0,0 @@
extended_spam_headers = true;
use = ["x-spam-status", "x-spam-score", "authentication-results"];
@@ -1,7 +0,0 @@
milter = yes;
bind_socket = "*:11332";
upstream "local" {
default = yes;
self_scan = yes;
}
@@ -1,75 +0,0 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: rspamd
namespace: mailgateway
spec:
selector:
matchLabels:
app: rspamd
template:
metadata:
annotations:
reloader.stakater.com/auto: "true"
labels:
app: rspamd
spec:
securityContext:
runAsUser: 11333
runAsGroup: 11333
fsGroup: 11333
containers:
- name: rspamd
image: rspamd/rspamd:4.0.1
ports:
- containerPort: 11332
name: milter
protocol: TCP
- containerPort: 11333
name: worker
protocol: TCP
- containerPort: 11334
name: controller
protocol: TCP
livenessProbe:
httpGet:
path: /ping
port: 11334
initialDelaySeconds: 15
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
httpGet:
path: /ping
port: 11334
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 3
failureThreshold: 3
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: "1"
memory: 512Mi
volumeMounts:
- name: rspamd-config
mountPath: /etc/rspamd/local.d
readOnly: true
- name: dkim-keys
mountPath: /etc/rspamd/dkim
readOnly: true
- name: rspamd-data
mountPath: /var/lib/rspamd
volumes:
- name: rspamd-config
configMap:
name: rspamd-config
- name: dkim-keys
secret:
secretName: dkim-keys
- name: rspamd-data
emptyDir: {}
-64
View File
@@ -1,64 +0,0 @@
---
# Internal service for rspamd - used by postfix pods in the same namespace
apiVersion: v1
kind: Service
metadata:
name: rspamd
namespace: mailgateway
spec:
selector:
app: rspamd
ports:
- name: milter
port: 11332
targetPort: 11332
protocol: TCP
- name: worker
port: 11333
targetPort: 11333
protocol: TCP
- name: controller
port: 11334
targetPort: 11334
protocol: TCP
---
# Internal ClusterIP for postfix - used by stalwart for outbound relay
apiVersion: v1
kind: Service
metadata:
name: postfix
namespace: mailgateway
spec:
selector:
app: postfix
ports:
- name: smtp
port: 25
targetPort: 25
protocol: TCP
- name: submission
port: 587
targetPort: 587
protocol: TCP
---
# External LoadBalancer for inbound MX (internet → postfix)
apiVersion: v1
kind: Service
metadata:
name: postfix-external
namespace: mailgateway
annotations:
external-dns.alpha.kubernetes.io/hostname: smtp-in.main.unkin.net
external-dns.alpha.kubernetes.io/target: 198.18.199.1
purelb.io/service-group: dmz
purelb.io/addresses: 198.18.199.1
spec:
type: LoadBalancer
externalTrafficPolicy: Local
selector:
app: postfix
ports:
- name: smtp
port: 25
targetPort: 25
protocol: TCP
@@ -1,17 +0,0 @@
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: dkim-keys
namespace: mailgateway
spec:
destination:
create: true
name: dkim-keys
overwrite: true
hmacSecretData: true
mount: kv
path: kubernetes/namespace/mailgateway/default/dkim-keys
refreshAfter: 5m
type: kv-v2
vaultAuthRef: default
+10 -3
View File
@@ -8,7 +8,9 @@ spec:
hostnames:
- paperclip.k8s.syd1.au.unkin.net
parentRefs:
- name: paperclip
- group: gateway.networking.k8s.io
kind: Gateway
name: paperclip
sectionName: http
rules:
- filters:
@@ -30,12 +32,17 @@ spec:
hostnames:
- paperclip.k8s.syd1.au.unkin.net
parentRefs:
- name: paperclip
- group: gateway.networking.k8s.io
kind: Gateway
name: paperclip
sectionName: https
rules:
- backendRefs:
- name: paperclip
- group: ""
kind: Service
name: paperclip
port: 3100
weight: 1
matches:
- path:
type: PathPrefix
@@ -3,4 +3,4 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../../base/mailgateway
- priorityclasses.yaml
@@ -0,0 +1,36 @@
---
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
name: low
value: 100
preemptionPolicy: Never
globalDefault: false
description: "Low-importance workloads. Can be evicted under pressure but will not preempt other pods."
---
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
name: power
value: 100
preemptionPolicy: Never
globalDefault: false
description: "Compute-heavy workloads with low scheduling importance. Evictable under pressure."
---
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
name: medium
value: 10000
preemptionPolicy: PreemptLowerPriority
globalDefault: false
description: "Standard workloads. Will preempt low-priority pods if the cluster is under pressure."
---
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
name: high
value: 100000
preemptionPolicy: PreemptLowerPriority
globalDefault: false
description: "High-importance services. Will preempt medium- and low-priority pods if necessary."
+1 -1
View File
@@ -28,7 +28,7 @@ spec:
imagePullSecrets: null
containers:
- name: g10k-code
image: git.unkin.net/unkin/almalinux9-g10k:20260308
image: git.unkin.net/unkin/almalinux9-g10k:20260606
imagePullPolicy: IfNotPresent
resources:
requests:
+1 -1
View File
@@ -50,7 +50,7 @@ spec:
cpu: 20m
memory: 32Mi
- name: cert-generator
image: git.unkin.net/unkin/almalinux9-base:20260308
image: git.unkin.net/unkin/almalinux9-base:20260606
imagePullPolicy: IfNotPresent
command:
- sh
@@ -181,7 +181,7 @@ spec:
name: puppet-puppet-volume
- name: setup-shared-bins
image: git.unkin.net/unkin/almalinux9-base:20260308
image: git.unkin.net/unkin/almalinux9-base:20260606
command:
- sh
- -c
+10 -3
View File
@@ -13,7 +13,9 @@ spec:
hostnames:
- puppetboard.k8s.syd1.au.unkin.net
parentRefs:
- name: puppetboard
- group: gateway.networking.k8s.io
kind: Gateway
name: puppetboard
sectionName: http
rules:
- filters:
@@ -40,12 +42,17 @@ spec:
hostnames:
- puppetboard.k8s.syd1.au.unkin.net
parentRefs:
- name: puppetboard
- group: gateway.networking.k8s.io
kind: Gateway
name: puppetboard
sectionName: https
rules:
- backendRefs:
- name: puppetboard
- group: ""
kind: Service
name: puppetboard
port: 80
weight: 1
matches:
- path:
type: PathPrefix
+7 -2
View File
@@ -13,12 +13,17 @@ spec:
hostnames:
- puppetdb.k8s.syd1.au.unkin.net
parentRefs:
- name: puppetdb
- group: gateway.networking.k8s.io
kind: Gateway
name: puppetdb
sectionName: https
rules:
- backendRefs:
- name: puppetdb
- group: ""
kind: Service
name: puppetdb
port: 8080
weight: 1
matches:
- path:
type: PathPrefix
-1
View File
@@ -9,7 +9,6 @@ metadata:
name: puppetdb
namespace: puppet
spec:
clusterIP: null
ports:
- name: pdb-http
port: 8080
+17 -5
View File
@@ -11,7 +11,9 @@ spec:
hostnames:
- vault.k8s.syd1.au.unkin.net
parentRefs:
- name: vault
- group: gateway.networking.k8s.io
kind: Gateway
name: vault
sectionName: http
rules:
- filters:
@@ -36,12 +38,17 @@ spec:
hostnames:
- vault.k8s.syd1.au.unkin.net
parentRefs:
- name: vault
- group: gateway.networking.k8s.io
kind: Gateway
name: vault
sectionName: https
rules:
- backendRefs:
- name: vault
- group: ""
kind: Service
name: vault
port: 8200
weight: 1
matches:
- path:
type: PathPrefix
@@ -60,12 +67,17 @@ spec:
- vault.service.consul
- vault.query.consul
parentRefs:
- name: vault
- group: gateway.networking.k8s.io
kind: Gateway
name: vault
sectionName: vault-direct
rules:
- backendRefs:
- name: vault
- group: ""
kind: Service
name: vault
port: 8200
weight: 1
matches:
- path:
type: PathPrefix
+1
View File
@@ -6,3 +6,4 @@ resources:
- namespace.yaml
- gateway.yaml
- httproute.yaml
- role_k8s-service-registration.yaml
@@ -0,0 +1,24 @@
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: vault-k8s-service-registration
namespace: vault
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: vault-k8s-service-registration
namespace: vault
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: vault-k8s-service-registration
subjects:
- kind: ServiceAccount
name: vault
namespace: vault
+2
View File
@@ -6,6 +6,8 @@ resources:
- namespace.yaml
- cnpg_cluster.yaml
- cnpg_pooler.yaml
- serviceaccount_terraform_artifactapi.yaml
- serviceaccount_terraform_git.yaml
- serviceaccount_terraform_vault.yaml
- vaultauth.yaml
- vaultstaticsecret.yaml
@@ -0,0 +1,6 @@
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: terraform-artifactapi
namespace: woodpecker
@@ -0,0 +1,6 @@
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: terraform-git
namespace: woodpecker
@@ -0,0 +1,6 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../../base/age-api
+1 -1
View File
@@ -37,7 +37,7 @@ server:
cpu: 100m
limits:
memory: 2Gi
cpu: 1000m
cpu: "1"
client:
enabled: false
@@ -0,0 +1,6 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../../base/kanidm
@@ -0,0 +1,6 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../../base/priority-classes
+9 -3
View File
@@ -40,9 +40,7 @@ server:
}
}
service_registration "consul" {
address = "consul-server.consul.svc.cluster.local:8500"
}
service_registration "kubernetes" {}
dataStorage:
enabled: true
@@ -50,6 +48,14 @@ server:
storageClass: cephrbd-fast-delete
accessMode: ReadWriteOnce
extraEnv:
- name: VAULT_K8S_NAMESPACE
value: vault
- name: VAULT_K8S_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
statefulSet:
securityContext:
container:
@@ -2,6 +2,7 @@ agent:
replicaCount: 3
env:
WOODPECKER_MAX_WORKFLOWS: "8"
WOODPECKER_BACKEND_K8S_PRIORITY_CLASS: power
WOODPECKER_BACKEND_K8S_STORAGE_CLASS: cephrbd-fast-delete
WOODPECKER_BACKEND_K8S_VOLUME_SIZE: 10G
WOODPECKER_BACKEND_K8S_STORAGE_RWX: false
+9 -1
View File
@@ -11,6 +11,7 @@ spec:
revision: HEAD
directories:
- path: apps/overlays/*/artifactapi
- path: apps/overlays/*/age-api
- path: apps/overlays/*/cattle-system
- path: apps/overlays/*/cert-manager
- path: apps/overlays/*/certificates
@@ -20,8 +21,9 @@ spec:
- path: apps/overlays/*/externaldns
- path: apps/overlays/*/inteldeviceplugins-system
- path: apps/overlays/*/jfrog
- path: apps/overlays/*/kanidm
- path: apps/overlays/*/node-feature-discovery
- path: apps/overlays/*/mailgateway
- path: apps/overlays/*/priority-classes
- path: apps/overlays/*/puppet
- path: apps/overlays/*/purelb
- path: apps/overlays/*/reflector-system
@@ -44,6 +46,12 @@ spec:
destination:
server: https://kubernetes.default.svc
namespace: '{{path[3]}}' # Use directory name as namespace
ignoreDifferences:
- group: ""
kind: ConfigMap
name: kanidm-repl-certs
jsonPointers:
- /data
syncPolicy:
automated:
prune: true
+6
View File
@@ -17,6 +17,8 @@ spec:
server: https://kubernetes.default.svc
- namespace: 'artifactapi'
server: https://kubernetes.default.svc
- namespace: 'age-api'
server: https://kubernetes.default.svc
- namespace: 'cert-manager'
server: https://kubernetes.default.svc
- namespace: 'certificates'
@@ -27,8 +29,12 @@ spec:
server: https://kubernetes.default.svc
- namespace: 'jfrog'
server: https://kubernetes.default.svc
- namespace: 'kanidm'
server: https://kubernetes.default.svc
- namespace: 'node-feature-discovery'
server: https://kubernetes.default.svc
- namespace: 'priority-classes'
server: https://kubernetes.default.svc
- namespace: 'purelb'
server: https://kubernetes.default.svc
- namespace: 'puppet'
+127
View File
@@ -0,0 +1,127 @@
#!/usr/bin/env bash
set -euo pipefail
SCHEMA_DIR="${1:-schemas}"
rm -rf "$SCHEMA_DIR"
mkdir -p "$SCHEMA_DIR"
CRD_URLS=(
"https://artifactapi.k8s.syd1.au.unkin.net/api/v1/remote/github_user/argoproj/argo-cd/refs/tags/v3.3.2/manifests/ha/install.yaml"
"https://artifactapi.k8s.syd1.au.unkin.net/api/v1/remote/github/kubernetes-sigs/gateway-api/releases/download/v1.5.1/standard-install.yaml"
)
SWAGGER_URL="https://artifactapi.k8s.syd1.au.unkin.net/api/v1/remote/github_user/kubernetes/kubernetes/refs/tags/v1.33.7/api/openapi-spec/swagger.json"
write_schema='
import json, os
def write_schema(schema, schema_dir, group, kind, version):
"""Strip descriptions and write compact JSON with trailing newline."""
def strip_descriptions(obj):
if isinstance(obj, dict):
return {k: strip_descriptions(v) for k, v in obj.items() if k != "description"}
if isinstance(obj, list):
return [strip_descriptions(i) for i in obj]
return obj
schema = strip_descriptions(schema)
group_dir = os.path.join(schema_dir, group) if group else schema_dir
os.makedirs(group_dir, exist_ok=True)
fname = f"{kind}_{version}.json".lower()
with open(os.path.join(group_dir, fname), "w") as f:
json.dump(schema, f, indent=2, sort_keys=True)
f.write("\n")
print(f" Generated: {group}/{fname}" if group else f" Generated: {fname}", file=__import__("sys").stderr)
'
if command -v kubectl &>/dev/null && kubectl cluster-info &>/dev/null 2>&1; then
echo "==> Fetching CRDs from cluster..." >&2
kubectl get crds -o json | python3 -c "
import sys, json, os
$write_schema
data = json.load(sys.stdin)
for crd in data.get('items', []):
spec = crd.get('spec', {})
group = spec.get('group', '')
kind = spec.get('names', {}).get('kind', '')
for ver in spec.get('versions', []):
version = ver.get('name', '')
openapi = ver.get('schema', {}).get('openAPIV3Schema', {})
if not openapi:
continue
schema = dict(openapi)
schema['\$schema'] = 'http://json-schema.org/draft-07/schema#'
schema['type'] = 'object'
schema.setdefault('properties', {})
schema['properties'].setdefault('apiVersion', {'type': 'string'})
schema['properties'].setdefault('kind', {'type': 'string'})
schema['properties'].setdefault('metadata', {'type': 'object'})
write_schema(schema, '$SCHEMA_DIR', group, kind, version)
"
else
echo "==> kubectl not available, skipping cluster CRDs" >&2
fi
echo "==> Downloading CRD manifests..." >&2
for url in "${CRD_URLS[@]}"; do
echo " Fetching: $url" >&2
curl -sSfL "$url"
done | python3 -c "
import sys, json, yaml, os
$write_schema
for doc in yaml.safe_load_all(sys.stdin):
if doc is None or doc.get('kind') != 'CustomResourceDefinition':
continue
spec = doc.get('spec', {})
group = spec.get('group', '')
kind = spec.get('names', {}).get('kind', '')
for ver in spec.get('versions', []):
version = ver.get('name', '')
openapi = ver.get('schema', {}).get('openAPIV3Schema', {})
if not openapi:
continue
group_dir = os.path.join('$SCHEMA_DIR', group)
fname = f'{kind}_{version}.json'.lower()
if os.path.exists(os.path.join(group_dir, fname)):
continue
schema = dict(openapi)
schema['\$schema'] = 'http://json-schema.org/draft-07/schema#'
schema['type'] = 'object'
schema.setdefault('properties', {})
schema['properties'].setdefault('apiVersion', {'type': 'string'})
schema['properties'].setdefault('kind', {'type': 'string'})
schema['properties'].setdefault('metadata', {'type': 'object'})
write_schema(schema, '$SCHEMA_DIR', group, kind, version)
"
echo "==> Downloading Kubernetes swagger spec..." >&2
curl -sSfL "$SWAGGER_URL" | python3 -c "
import sys, json, os
$write_schema
swagger = json.load(sys.stdin)
definitions = swagger.get('definitions', {})
for defn_name, defn in definitions.items():
for gvk in defn.get('x-kubernetes-group-version-kind', []):
group = gvk.get('group', '')
version = gvk.get('version', '')
kind = gvk.get('kind', '')
schema = {
'\$schema': 'http://json-schema.org/draft-07/schema#',
'type': 'object',
'properties': {},
'additionalProperties': True,
}
for prop_name, prop_val in defn.get('properties', {}).items():
prop_copy = {k: v for k, v in prop_val.items() if k != '\$ref'}
if not prop_copy.get('type') and len(prop_copy) > 0:
prop_copy['type'] = 'object'
prop_copy['additionalProperties'] = True
schema['properties'][prop_name] = prop_copy
write_schema(schema, '$SCHEMA_DIR', group, kind, version)
"
total=$(find "$SCHEMA_DIR" -name '*.json' | wc -l)
echo "==> Schema generation complete: $total schemas in $SCHEMA_DIR" >&2
+5 -2
View File
@@ -3,9 +3,12 @@ set -euo pipefail
KUBE_VERSION="1.33.7"
SCHEMA_DIR="${SCHEMA_DIR:-schemas}"
schema_args=(
-schema-location "https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/{{.NormalizedKubernetesVersion}}-standalone{{.StrictSuffix}}/{{.ResourceKind}}{{.KindSuffix}}.json"
-schema-location "https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json"
-schema-location "$SCHEMA_DIR/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json"
-schema-location "https://artifactapi.k8s.syd1.au.unkin.net/api/v1/remote/github_user/yannh/kubernetes-json-schema/master/{{.NormalizedKubernetesVersion}}-standalone{{.StrictSuffix}}/{{.ResourceKind}}{{.KindSuffix}}.json"
-schema-location "https://artifactapi.k8s.syd1.au.unkin.net/api/v1/remote/github_user/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json"
)
while IFS= read -r -d "" k; do
+5 -2
View File
@@ -3,9 +3,12 @@ set -euo pipefail
KUBE_VERSION="1.33.7"
SCHEMA_DIR="${SCHEMA_DIR:-schemas}"
schema_args=(
-schema-location "https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/{{.NormalizedKubernetesVersion}}-standalone{{.StrictSuffix}}/{{.ResourceKind}}{{.KindSuffix}}.json"
-schema-location "https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json"
-schema-location "$SCHEMA_DIR/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json"
-schema-location "https://artifactapi.k8s.syd1.au.unkin.net/api/v1/remote/github_user/yannh/kubernetes-json-schema/master/{{.NormalizedKubernetesVersion}}-standalone{{.StrictSuffix}}/{{.ResourceKind}}{{.KindSuffix}}.json"
-schema-location "https://artifactapi.k8s.syd1.au.unkin.net/api/v1/remote/github_user/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json"
)
while IFS= read -r -d "" k; do
+145
View File
@@ -0,0 +1,145 @@
# Migration Guide: Terragrunt to ArgoCD
## Prerequisites
- Examine existing Terraform configuration in `sources/terraform-k8s/config/<app-name>/`
- Identify Helm charts, values, storage classes, secrets, and other resources
## Migration Steps
### 1. Branch Creation
- Change back to main branch: `git checkout main`
- Create new branch: `git checkout -b benvin/<app-name>`
### 2. Create Base Application Structure
- Create directory: `apps/base/<app-name>/`
- Create `apps/base/<app-name>/kustomization.yaml` with resources:
- `namespace.yaml`
- `vaultauth.yaml` (if needed)
- `vaultstaticsecret.yaml` (if needed)
- Additional resources as required (storageclass.yaml, etc.)
### 3. Create Namespace
- Create `apps/base/<app-name>/namespace.yaml` with app namespace
### 4. Create Vault Integration (if needed)
- Create `apps/base/<app-name>/vaultauth.yaml` from Terraform `vault_auth.yaml`
- Convert snake_case to camelCase for Kubernetes
- Map Terraform fields to VaultAuth spec
- Create `apps/base/<app-name>/vaultstaticsecret.yaml` from `vault_static_secret.yaml`
- Convert to VaultStaticSecret spec format
### 5. Create Additional Resources
- Create any additional resources (StorageClass, etc.) from Terraform config
- Maintain exact parameter parity with Terraform
### 6. Create Overlay Structure
- Create directory: `apps/overlays/au-syd1/<app-name>/`
- Create `apps/overlays/au-syd1/<app-name>/kustomization.yaml`:
- Reference base: `../../../base/<app-name>`
- Add helmCharts section with repo, version, valuesFile
- Create `apps/overlays/au-syd1/<app-name>/values.yaml` from Terraform helm_release values
### 7. Update Project Configuration
- Add Helm repository to appropriate project in `argocd/projects/`
- Add namespace to project destinations (if needed)
- Add required cluster resource permissions
### 8. Update ApplicationSet
- Add directory pattern to appropriate ApplicationSet in `argocd/applicationsets/`
- Use existing patterns like `apps/overlays/*/csi-*` or `apps/overlays/*/<app-name>`
### 9. Validation
- Run `kustomize build --enable-helm apps/overlays/au-syd1/<app-name>` to generate all resources
- Check resource types: `kustomize build --enable-helm apps/overlays/au-syd1/<app-name> | grep "^kind:" | sort | uniq -c`
- Verify all resource types are permitted in the target project's `clusterResourceWhitelist` and `namespaceResourceWhitelist`
- Run `make kubeconform` to validate all resources
- Fix any validation errors
### 10. Git Workflow
- Add only created/modified files: `git add apps/base/<app-name>/ apps/overlays/au-syd1/<app-name>/ argocd/projects/<project>.yaml argocd/applicationsets/<project>.yaml`
- **Never use `git add .`**
- Create commit with descriptive message following existing patterns
- Push branch: `git push -u origin benvin/<app-name>`
## Project Organization
### Platform Project
- Core infrastructure and system components
- Examples: cert-manager, external-dns, cnpg-system, reflector-system, etc.
### Storage Project
- Storage-related components
- Pattern: `csi-*` applications
- Examples: csi-cephfs, csi-cephrbd
### Application-Specific Projects
- Create new projects for logical groupings as needed
## Common Patterns
### Helm Chart Integration
```yaml
helmCharts:
- name: <chart-name>
repo: <helm-repo-url>
version: "<version>"
releaseName: <release-name>
namespace: <namespace>
valuesFile: values.yaml
```
### VaultAuth Template
```yaml
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
name: <auth-name>
namespace: <namespace>
spec:
method: kubernetes
mount: k8s/au/syd1
vaultConnectionRef: vso-system/default
allowedNamespaces:
- <namespace>
kubernetes:
role: <role>
serviceAccount: <service-account>
audiences:
- vault
tokenExpirationSeconds: 600
```
### VaultStaticSecret Template
```yaml
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: <secret-name>
namespace: <namespace>
spec:
vaultAuthRef: <auth-ref>
mount: kv
type: kv-v2
path: <vault-path>
refreshAfter: 5m
destination:
name: <k8s-secret-name>
create: true
```
## Troubleshooting
### Pre-commit Hook Issues
- If hooks modify files, add changes: `git add -u`
- Amend commit if safe: `git commit --amend --no-edit`
- Use `git push --force-with-lease` for amended commits
### Validation Failures
- Check Helm chart compatibility with Kubernetes version
- Verify all required fields are present in resources
- Ensure proper YAML formatting
### Missing Permissions
- Add required cluster resources to project clusterResourceWhitelist
- Add namespaces to project destinations
- Verify Helm repository is in project sourceRepos
File diff suppressed because it is too large Load Diff
+162
View File
@@ -0,0 +1,162 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"apiVersion": {
"type": "string"
},
"kind": {
"type": "string"
},
"metadata": {
"type": "object"
},
"spec": {
"properties": {
"commonName": {
"type": "string"
},
"dnsNames": {
"items": {
"type": "string"
},
"type": "array",
"x-kubernetes-list-type": "atomic"
},
"duration": {
"type": "string"
},
"ipAddresses": {
"items": {
"type": "string"
},
"type": "array",
"x-kubernetes-list-type": "atomic"
},
"issuerRef": {
"properties": {
"group": {
"type": "string"
},
"kind": {
"type": "string"
},
"name": {
"type": "string"
}
},
"required": [
"name"
],
"type": "object"
},
"profile": {
"type": "string"
},
"request": {
"format": "byte",
"type": "string"
}
},
"required": [
"issuerRef",
"request"
],
"type": "object"
},
"status": {
"properties": {
"authorizations": {
"items": {
"properties": {
"challenges": {
"items": {
"properties": {
"token": {
"type": "string"
},
"type": {
"type": "string"
},
"url": {
"type": "string"
}
},
"required": [
"token",
"type",
"url"
],
"type": "object"
},
"type": "array",
"x-kubernetes-list-type": "atomic"
},
"identifier": {
"type": "string"
},
"initialState": {
"enum": [
"valid",
"ready",
"pending",
"processing",
"invalid",
"expired",
"errored"
],
"type": "string"
},
"url": {
"type": "string"
},
"wildcard": {
"type": "boolean"
}
},
"required": [
"url"
],
"type": "object"
},
"type": "array",
"x-kubernetes-list-type": "atomic"
},
"certificate": {
"format": "byte",
"type": "string"
},
"failureTime": {
"format": "date-time",
"type": "string"
},
"finalizeURL": {
"type": "string"
},
"reason": {
"type": "string"
},
"state": {
"enum": [
"valid",
"ready",
"pending",
"processing",
"invalid",
"expired",
"errored"
],
"type": "string"
},
"url": {
"type": "string"
}
},
"type": "object"
}
},
"required": [
"metadata",
"spec"
],
"type": "object"
}
@@ -0,0 +1,130 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"apiVersion": {
"type": "string"
},
"kind": {
"type": "string"
},
"metadata": {
"type": "object"
},
"spec": {
"properties": {
"clusterSelector": {
"properties": {
"matchExpressions": {
"items": {
"properties": {
"key": {
"type": "string"
},
"operator": {
"type": "string"
},
"values": {
"items": {
"type": "string"
},
"type": "array",
"x-kubernetes-list-type": "atomic"
}
},
"required": [
"key",
"operator"
],
"type": "object"
},
"type": "array",
"x-kubernetes-list-type": "atomic"
},
"matchLabels": {
"additionalProperties": {
"type": "string"
},
"type": "object"
}
},
"type": "object",
"x-kubernetes-map-type": "atomic"
},
"resources": {
"items": {
"properties": {
"kind": {
"enum": [
"Secret",
"ConfigMap"
],
"type": "string"
},
"name": {
"minLength": 1,
"type": "string"
}
},
"required": [
"kind",
"name"
],
"type": "object"
},
"type": "array"
},
"strategy": {
"enum": [
"ApplyOnce"
],
"type": "string"
}
},
"required": [
"clusterSelector"
],
"type": "object"
},
"status": {
"properties": {
"conditions": {
"items": {
"properties": {
"lastTransitionTime": {
"format": "date-time",
"type": "string"
},
"message": {
"type": "string"
},
"reason": {
"type": "string"
},
"severity": {
"type": "string"
},
"status": {
"type": "string"
},
"type": {
"type": "string"
}
},
"required": [
"status",
"type"
],
"type": "object"
},
"type": "array"
},
"observedGeneration": {
"format": "int64",
"type": "integer"
}
},
"type": "object"
}
},
"type": "object"
}
@@ -0,0 +1,130 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"apiVersion": {
"type": "string"
},
"kind": {
"type": "string"
},
"metadata": {
"type": "object"
},
"spec": {
"properties": {
"clusterSelector": {
"properties": {
"matchExpressions": {
"items": {
"properties": {
"key": {
"type": "string"
},
"operator": {
"type": "string"
},
"values": {
"items": {
"type": "string"
},
"type": "array",
"x-kubernetes-list-type": "atomic"
}
},
"required": [
"key",
"operator"
],
"type": "object"
},
"type": "array",
"x-kubernetes-list-type": "atomic"
},
"matchLabels": {
"additionalProperties": {
"type": "string"
},
"type": "object"
}
},
"type": "object",
"x-kubernetes-map-type": "atomic"
},
"resources": {
"items": {
"properties": {
"kind": {
"enum": [
"Secret",
"ConfigMap"
],
"type": "string"
},
"name": {
"minLength": 1,
"type": "string"
}
},
"required": [
"kind",
"name"
],
"type": "object"
},
"type": "array"
},
"strategy": {
"enum": [
"ApplyOnce"
],
"type": "string"
}
},
"required": [
"clusterSelector"
],
"type": "object"
},
"status": {
"properties": {
"conditions": {
"items": {
"properties": {
"lastTransitionTime": {
"format": "date-time",
"type": "string"
},
"message": {
"type": "string"
},
"reason": {
"type": "string"
},
"severity": {
"type": "string"
},
"status": {
"type": "string"
},
"type": {
"type": "string"
}
},
"required": [
"status",
"type"
],
"type": "object"
},
"type": "array"
},
"observedGeneration": {
"format": "int64",
"type": "integer"
}
},
"type": "object"
}
},
"type": "object"
}

Some files were not shown because too many files have changed in this diff Show More