1 Commits

Author SHA1 Message Date
unkinben 2254a39d77 feat: add artifact-keeper
- converted the artifact-keeper helm-chart into kustomization manifests
- converted postgres to cnpg
- moved secrets to vault
2026-04-19 18:43:56 +10:00
442 changed files with 1945 additions and 147405 deletions
-273
View File
@@ -1,273 +0,0 @@
---
description: Pull master, read open issues, pick one, branch, implement, test, commit, PR, and comment.
---
# Solve a Gitea Issue
## Current repo state
```!
git status --short
echo "Current branch: $(git branch --show-current)"
echo "Remote: $(git remote get-url origin 2>/dev/null || echo 'none')"
```
## Open issues (with full body)
```!
echo "Fetching open issues..."
issue_ids=$(tea issues list --output simple 2>/dev/null | awk 'NF && $1 ~ /^[0-9]+$/ {print $1}')
if [ -z "$issue_ids" ]; then
echo "No open issues found (or tea is not logged in)."
else
for id in $issue_ids; do
echo ""
echo "══════════════════════════════════════"
tea issues view "$id" --fields index,title,body 2>/dev/null \
|| tea issue "$id" 2>/dev/null \
|| echo " (could not read issue #$id)"
echo "══════════════════════════════════════"
done
fi
```
---
## Your task
Follow these steps **in order**. Do not skip steps.
### 1 — Choose an issue
Present the issues above to the user as a numbered list (index, one-line title). Ask which one to work on. Wait for the answer before continuing.
### 2 — Sync master
```bash
git checkout master
git pull
```
Confirm you are on master and up to date.
### 3 — Create a branch
Name the branch `benvin/issue-<N>-<short-slug>` where `<short-slug>` is 24 kebab-case words from the issue title.
```bash
git checkout -b benvin/issue-<N>-<slug>
```
### 4 — Read the issue in full
Re-read the full issue body shown above. If any part is ambiguous, state your interpretation before coding.
**If you discover other problems while working:** do NOT solve them inline. Create a new Gitea issue with `tea issues create --title "..." --description "..."` and stay focused on the assigned issue.
### 5 — Implement the solution
Make the code changes needed to resolve the issue. Follow the conventions already in the repo:
- `main.py` route handlers each contain a single function call; logic lives in submodules.
- No comments unless the WHY is non-obvious.
- No new files unless the issue or architecture requires it.
- Security: no command injection, XSS, SQL injection, or secrets in code.
- **For performance improvements:** implement at the most generic call site possible so the fix applies to all current and future implementations, not just the one being tested.
### 6 — Update tests
Add or update tests that cover the new behaviour. Tests live in `tests/`. Check existing test structure before writing new ones — mirror the style and fixture patterns already in use.
### 7 — Update README
If the feature introduces new config keys, endpoints, or user-facing behaviour, document it in `README.md`. Keep additions concise — follow the existing section style.
### 8 — Run the full test suite
```bash
make test
```
All tests must pass. If any fail, fix them before proceeding. Do not skip or suppress failing tests.
### 9 — Live Docker test (new package type only)
**Skip this step if the issue does not add a new remote package type.**
If the issue adds a new package type (e.g. `deb`, `conda`, `cargo`, `rubygems`, or any type not already in `remotes.yaml`), do the following before committing.
#### 9a — Add a real test remote to remotes.yaml
Append a valid, publicly accessible remote of the new type to `remotes.yaml`. Use a real upstream URL and patterns that cover both an immutable file (versioned artifact) and a mutable file (index/metadata). Add a comment explaining which URLs to use for manual testing.
#### 9b — Start the stack
```bash
make docker-up
```
Wait until `curl -s http://localhost:8000/health` returns `{"status":"healthy"}`.
#### 9c — Test a mutable file (first fetch — cache miss)
Download the index or metadata file for the new remote. Confirm:
- HTTP 200
- `X-Artifact-Source: remote` header (or equivalent log line confirming a cache miss)
- Content looks correct (not empty, not an error page)
```bash
curl -sv "http://localhost:8000/api/v1/remote/<new-remote>/<mutable-path>" 2>&1 | grep -E "< HTTP|X-Artifact"
```
#### 9d — Test a mutable file (second fetch — cache hit)
Repeat the exact same request. Confirm:
- HTTP 200
- `X-Artifact-Source: cache`
```bash
curl -sv "http://localhost:8000/api/v1/remote/<new-remote>/<mutable-path>" 2>&1 | grep -E "< HTTP|X-Artifact"
```
#### 9e — Test an immutable file (first fetch — cache miss)
Download a versioned/immutable artifact. Confirm HTTP 200 and a cache-miss log line.
```bash
curl -sv "http://localhost:8000/api/v1/remote/<new-remote>/<immutable-path>" 2>&1 | grep -E "< HTTP|X-Artifact"
```
#### 9f — Test an immutable file (second fetch — cache hit)
Repeat. Confirm `X-Artifact-Source: cache`.
#### 9g — Check container logs
```bash
make docker-logs
```
Scan for:
- `Cache MISS` on first fetches, `Cache HIT` on second fetches
- `Cache ADD SUCCESS` with correct sizes
- No unhandled exceptions or ERROR lines
#### 9h — Exercise package-type tooling against the proxy
Use the native tooling for this package type to verify end-to-end behaviour. Examples:
| Package type | Command |
|---|---|
| `pypi` | `uv run --index-url http://localhost:8000/api/v1/remote/<remote>/simple <tool>` |
| `npm` | `npm install --registry http://localhost:8000/api/v1/remote/<remote>/ <pkg>` |
| `helm` | `helm repo add test http://localhost:8000/api/v1/remote/<remote> && helm search repo test && helm template test/<chart>` |
| `alpine` | `apk fetch --repository http://localhost:8000/api/v1/remote/<remote>/<branch>/<arch> <pkg>` |
| `rpm` | `dnf install --repofrompath ... <pkg>` or `repoquery` |
| `generic` | `curl` / `wget` as appropriate |
Confirm the tool resolves and downloads correctly through the proxy.
#### 9i — Tear down
```bash
make docker-down
```
Fix any failures found during 9b9h before moving on.
### 9.5 — Performance issues: measure before/after and gate the PR
**Skip this step if the issue is not a performance improvement.**
For performance issues, a PR is only warranted if there is a measurable gain. Use the Docker stack to compare before and after.
#### 9.5a — Baseline measurement (before)
Start the stack with the **unmodified** code (temporarily revert your change):
```bash
make docker-up
```
Warm or clear the cache as appropriate, then measure the relevant metric — e.g. concurrent request latency during a slow operation, response time for a specific endpoint, or throughput. Record the numbers.
#### 9.5b — Apply your change and rebuild
```bash
make docker-up # rebuilds the image
```
Repeat exactly the same measurement. Record the numbers.
#### 9.5c — Decide
If the improvement is not clearly measurable, **do not open a PR**. Instead:
1. Update the issue with your findings.
2. Note any conditions under which the improvement would be observable.
3. Skip steps 1114.
If the improvement is clear, proceed with the commit and PR. Include the before/after numbers in the PR description and the issue comment.
#### 9.5d — Tear down
```bash
make docker-down
```
### 10 — Build the wheel (smoke check)
```bash
uv build --wheel
```
Confirm the build succeeds.
### 11 — Stage and commit
Stage only the files you changed. Do not use `git add -A` or `git add .` — list files explicitly. Run:
```bash
git add <file1> <file2> ...
git commit
```
The commit message must:
- Start with a conventional-commit prefix (`feat:`, `fix:`, `refactor:`, `chore:`, etc.)
- Summarise the change in ≤ 72 characters on the first line
- Optionally include a short body explaining *why* (not *what*)
If the pre-commit hook auto-fixes files, re-stage the fixed files and commit again.
### 12 — Push the branch
```bash
git push origin <branch-name>
```
### 13 — Open a pull request
```bash
tea pulls create \
--base master \
--head <branch-name> \
--title "<same as commit subject>" \
--description "Closes #<N>\n\n## Summary\n<bullet points>\n\n## Test plan\n<what was verified>"
```
### 14 — Comment on the issue
```bash
tea comment <N> "<resolution comment>"
```
The comment must cover:
- **How it was resolved** — what changed and why
- **Issues encountered** — any non-obvious problems hit during implementation
- **Potential future improvements** — what could be done next
### 15 — Return to master
```bash
git checkout master
```
Report the PR URL and a one-sentence summary to the user.
-2
View File
@@ -7,7 +7,6 @@ repos:
- id: check-json
- id: check-added-large-files
args: ['--maxkb=500']
exclude: '^schemas/'
- id: check-merge-conflict
- id: check-shebang-scripts-are-executable
- id: check-symlinks
@@ -20,7 +19,6 @@ repos:
- id: end-of-file-fixer
- id: forbid-new-submodules
- id: pretty-format-json
args: ['--autofix']
- id: trailing-whitespace
# YAML linting
+1 -11
View File
@@ -3,16 +3,6 @@ when:
steps:
- name: kubeconform
image: git.unkin.net/unkin/almalinux9-kubetest:20260606
image: git.unkin.net/unkin/almalinux9-kubetest:20260319
commands:
- make kubeconform
backend_options:
kubernetes:
serviceAccountName: default
resources:
requests:
memory: 512Mi
cpu: 1
limits:
memory: 2Gi
cpu: 2
+1 -11
View File
@@ -3,16 +3,6 @@ when:
steps:
- name: pre-commit
image: git.unkin.net/unkin/almalinux9-base:20260606
image: git.unkin.net/unkin/almalinux9-base:20260308
commands:
- uvx pre-commit run --all-files
backend_options:
kubernetes:
serviceAccountName: default
resources:
requests:
memory: 256Mi
cpu: 250m
limits:
memory: 1Gi
cpu: 1
-261
View File
@@ -1,261 +0,0 @@
# AGENTS.md
## Project Overview
This is an **ArgoCD GitOps repository** that manages Kubernetes applications for the `au-syd1` cluster using a Kustomize + Helm pattern. Applications are deployed via ArgoCD ApplicationSets that watch directory patterns in this repo.
The migration pattern for this repo is: **Terragrunt/Terraform → ArgoCD** (see `migration.md` for full guide).
---
## Essential Commands
```bash
# Build and render manifests for a path (outputs to manifests/<path>/)
make build apps/overlays/au-syd1/<app-name>
make build clusters/au-syd1/bootstrap
# Validate all apps and clusters with kubeconform
make kubeconform
# Clean generated manifests
make clean
# Quick build + inspect without persisting output
kustomize build --enable-helm apps/overlays/au-syd1/<app-name>
# Check all resource kinds produced by an overlay
kustomize build --enable-helm apps/overlays/au-syd1/<app-name> | grep "^kind:" | sort | uniq -c
# Run pre-commit checks against all files
uvx pre-commit run --all-files
```
---
## Directory Structure
```
argocd-apps/
├── argocd/
│ ├── applicationsets/ # ArgoCD ApplicationSet definitions (platform.yaml, storage.yaml)
│ └── projects/ # ArgoCD AppProject definitions (platform.yaml, storage.yaml)
├── apps/
│ ├── base/ # Base Kustomize resources per app (no cluster-specific config)
│ │ └── <app-name>/
│ │ ├── kustomization.yaml
│ │ ├── namespace.yaml
│ │ ├── vaultauth.yaml # (if Vault-managed secrets)
│ │ └── vaultstaticsecret.yaml
│ └── overlays/
│ └── au-syd1/ # Cluster-specific overlays
│ └── <app-name>/
│ ├── kustomization.yaml # references base + helmCharts
│ └── values.yaml # Helm values for this cluster
├── clusters/
│ └── au-syd1/
│ ├── apps/ # Entry point: references apps/base (ArgoCD app-of-apps)
│ └── bootstrap/ # ArgoCD install + initial Application manifest
├── ci/
│ ├── validate-apps.sh # kubeconform over apps/overlays/*/kustomization.yaml
│ ├── validate-clusters.sh # kubeconform over clusters/*/kustomization.yaml
│ └── validate-no-secrets.sh # pre-commit hook: blocks plain Kubernetes Secrets
└── sources/ # Reference sources (Terraform configs, upstream charts, etc.)
└── terraform-k8s/ # Original Terraform configs — reference when migrating
```
---
## Adding a New Application
Follow these 10 steps (detailed in `migration.md`):
### 1. Create base resources
```
apps/base/<app-name>/
├── kustomization.yaml
├── namespace.yaml
├── vaultauth.yaml # if needed
└── vaultstaticsecret.yaml # if needed
```
### 2. Create cluster overlay
```
apps/overlays/au-syd1/<app-name>/
├── kustomization.yaml
└── values.yaml
```
**Overlay kustomization.yaml pattern:**
```yaml
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../../base/<app-name>
helmCharts:
- name: <chart-name>
repo: <helm-repo-url>
version: "<version>"
releaseName: <release-name>
namespace: <namespace>
valuesFile: values.yaml
```
### 3. Register in ApplicationSet
Add a directory entry to `argocd/applicationsets/platform.yaml` (or `storage.yaml` for `csi-*` apps):
```yaml
- path: apps/overlays/*/<app-name>
```
### 4. Update AppProject
In `argocd/projects/platform.yaml` (or `storage.yaml`):
- Add the Helm repo URL to `sourceRepos`
- Add the namespace to `destinations`
- Add any required cluster-scoped resource types to `clusterResourceWhitelist`
### 5. Validate
```bash
kustomize build --enable-helm apps/overlays/au-syd1/<app-name>
make kubeconform
```
---
## Secret Management
**Plain Kubernetes `Secret` objects are blocked** by the pre-commit hook. Use Vault Operator CRDs instead:
### VaultAuth template
```yaml
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
name: default
namespace: <namespace>
spec:
method: kubernetes
mount: k8s/au/syd1
vaultConnectionRef: vso-system/default
allowedNamespaces:
- <namespace>
kubernetes:
role: <role>
serviceAccount: <service-account>
audiences:
- vault
tokenExpirationSeconds: 600
```
### VaultStaticSecret template
```yaml
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: <secret-name>
namespace: <namespace>
spec:
vaultAuthRef: default
mount: kv
type: kv-v2
path: kubernetes/namespace/<namespace>/default/<secret-name>
refreshAfter: 5m
destination:
name: <k8s-secret-name>
create: true
overwrite: true
hmacSecretData: true
```
---
## YAML Conventions
- **2-space indentation** (enforced by yamllint)
- All files must end with a newline (`end-of-file-fixer`)
- No trailing whitespace
- YAML linting uses relaxed rules with `line-length: disable` (long base64/URLs are fine)
- yamllint ignores `chart` directories (vendored Helm charts)
- `---` document separator at top of every YAML file
- Multiple documents in one file are allowed (e.g., `vaultstaticsecret.yaml` often contains multiple secrets)
---
## Kubernetes Labels Pattern
Use standard `app.kubernetes.io/*` labels consistently:
```yaml
labels:
app.kubernetes.io/component: <component>
app.kubernetes.io/instance: <release-name>
app.kubernetes.io/name: <app-name>
app.kubernetes.io/version: <version>
```
---
## Resource Naming Conventions
Files in `apps/base/<app-name>/` follow the pattern:
```
<kind>_<name>.yaml
```
Examples:
- `deployment_puppetserver-master.yaml`
- `cronjob_g10k-code.yaml`
- `configmap_puppetboard-config.yaml`
- `horizontalpodautoscaler_puppetserver-compilers-autoscaler.yaml`
- `service_puppet-headless.yaml`
---
## Helm Chart Vendoring
Some overlays vendor Helm charts locally under `apps/overlays/au-syd1/<app-name>/charts/<chart-name>/`. When a chart is vendored, the overlay's `kustomization.yaml` references the local path. When not vendored, it references the OCI or HTTP repo directly.
Current Kubernetes target version: **1.33.7** (used by kubeconform in CI).
---
## Project Boundaries
| Project | ApplicationSet | App pattern |
|------------|---------------------------|--------------------------|
| `platform` | `argocd/applicationsets/platform.yaml` | Named apps (cert-manager, puppet, woodpecker, etc.) |
| `storage` | `argocd/applicationsets/storage.yaml` | `csi-*` apps |
The `clusters/au-syd1/apps/` entry-point is deployed as a standalone ArgoCD `Application` (not an ApplicationSet) called `au-syd1-apps`.
---
## CI / Pre-commit Hooks
Runs on every PR via Woodpecker CI (`.woodpecker/`):
| Check | Tool | Trigger |
|---|---|---|
| YAML lint + general file checks | `pre-commit` (yamllint + pre-commit-hooks) | PR |
| No plain Secrets | `ci/validate-no-secrets.sh` | PR (staged files) |
| Kubernetes manifest validation | `kubeconform` via `make kubeconform` | PR |
kubeconform skips: `CustomResourceDefinition`, `GpuDevicePlugin` (for apps validation).
---
## Git Workflow
- Branch naming: `benvin/<app-name>` (user prefix)
- **Never `git add .`** — add only relevant files explicitly
- If pre-commit modifies files, `git add -u` then `git commit --amend --no-edit`
- Use `git push --force-with-lease` after amending
---
## Security Policies
- `reloader.stakater.com/auto: "true"` annotation triggers rolling restarts on ConfigMap/Secret changes
- Security contexts follow least-privilege: `drop: [all]` then add only required capabilities
- `fsGroup: 999` on pod security context for Puppet workloads
- `runAsUser: 0` is used only for init containers that need to set file permissions, then regular containers run as non-root
+1 -5
View File
@@ -1,4 +1,4 @@
.PHONY: build clean schemas
.PHONY: build clean
# Build a kustomization path to manifests directory
# Usage: make build clusters/au-syd1/bootstrap
@@ -6,10 +6,6 @@ build:
@mkdir -p manifests/$(filter-out $@,$(MAKECMDGOALS))
@kustomize build --enable-helm $(filter-out $@,$(MAKECMDGOALS)) --output manifests/$(filter-out $@,$(MAKECMDGOALS))
# Generate JSON schemas from CRDs and Kubernetes swagger spec (run manually, results committed)
schemas:
@ci/generate-schemas.sh schemas
# kubeconform
kubeconform:
@ci/validate-apps.sh && \
-45
View File
@@ -1,45 +0,0 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: age-api
namespace: age-api
spec:
replicas: 1
selector:
matchLabels:
app: age-api
template:
metadata:
annotations:
reloader.stakater.com/auto: "true"
labels:
app: age-api
spec:
containers:
- name: age-api
image: git.unkin.net/unkin/age-api:v0.1.0
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
name: http
protocol: TCP
env:
- name: CONFIG_PATH
value: /etc/age-api/config.yaml
resources:
limits:
cpu: 100m
memory: 64Mi
requests:
cpu: 10m
memory: 32Mi
volumeMounts:
- mountPath: /etc/age-api/config.yaml
name: config
subPath: config.yaml
restartPolicy: Always
volumes:
- name: config
configMap:
name: age-api-config
-37
View File
@@ -1,37 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
labels:
traefik.io/instance: internal
annotations:
cert-manager.io/cluster-issuer: vault-issuer
cert-manager.io/common-name: age-api.k8s.syd1.au.unkin.net
cert-manager.io/private-key-size: "4096"
external-dns.alpha.kubernetes.io/hostname: age-api.k8s.syd1.au.unkin.net
external-dns.alpha.kubernetes.io/target: 198.18.200.4
name: age-api
namespace: age-api
spec:
gatewayClassName: traefik-internal
listeners:
- allowedRoutes:
namespaces:
from: Same
hostname: age-api.k8s.syd1.au.unkin.net
name: http
port: 80
protocol: HTTP
- allowedRoutes:
namespaces:
from: Same
hostname: age-api.k8s.syd1.au.unkin.net
name: https
port: 443
protocol: HTTPS
tls:
certificateRefs:
- group: ""
kind: Secret
name: age-api-tls
mode: Terminate
-49
View File
@@ -1,49 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: age-api-http-redirect
namespace: age-api
spec:
hostnames:
- age-api.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: age-api
sectionName: http
rules:
- filters:
- type: RequestRedirect
requestRedirect:
scheme: https
statusCode: 301
matches:
- path:
type: PathPrefix
value: /
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: age-api
namespace: age-api
spec:
hostnames:
- age-api.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: age-api
sectionName: https
rules:
- backendRefs:
- group: ""
kind: Service
name: age-api
port: 80
weight: 1
matches:
- path:
type: PathPrefix
value: /
-17
View File
@@ -1,17 +0,0 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
- gateway.yaml
- httproute.yaml
- namespace.yaml
- service.yaml
configMapGenerator:
- name: age-api-config
files:
- config.yaml=resources/config.yaml
options:
disableNameSuffixHash: true
-5
View File
@@ -1,5 +0,0 @@
---
apiVersion: v1
kind: Namespace
metadata:
name: age-api
-7
View File
@@ -1,7 +0,0 @@
people:
- name: jaidi
birthtime: 1773135720
- name: ben
birthtime: 559663200
- name: sudaporn
birthtime: 686757600
-17
View File
@@ -1,17 +0,0 @@
---
apiVersion: v1
kind: Service
metadata:
name: age-api
namespace: age-api
spec:
internalTrafficPolicy: Cluster
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
selector:
app: age-api
sessionAffinity: None
type: ClusterIP
@@ -2,20 +2,23 @@
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: litellm-postgres
namespace: litellm
name: artifact-keeper-postgres
namespace: artifact-keeper
spec:
affinity:
podAntiAffinityType: preferred
bootstrap:
initdb:
database: litellm
database: artifact_registry
encoding: UTF8
localeCType: C
localeCollate: C
owner: litellm
owner: registry
secret:
name: postgres-credentials
postInitSQL:
- CREATE DATABASE dependency_track OWNER registry;
- GRANT ALL PRIVILEGES ON DATABASE dependency_track TO registry;
enablePDB: true
enableSuperuserAccess: false
failoverDelay: 0
@@ -80,12 +83,12 @@ spec:
memory: 1Gi
requests:
cpu: 250m
memory: 512Mi
memory: 256Mi
smartShutdownTimeout: 180
startDelay: 3600
stopDelay: 1800
storage:
resizeInUseVolumes: true
size: 10Gi
size: 20Gi
storageClass: cephrbd-fast-delete
switchoverDelay: 3600
@@ -2,11 +2,11 @@
apiVersion: postgresql.cnpg.io/v1
kind: Pooler
metadata:
name: postgres-pooler
namespace: artifactapi
name: artifact-keeper-postgres-pooler
namespace: artifact-keeper
spec:
cluster:
name: postgres
name: artifact-keeper-postgres
instances: 2
pgbouncer:
parameters:
@@ -17,7 +17,7 @@ spec:
template:
metadata:
labels:
app: pooler
app: artifact-keeper-pooler
spec:
affinity:
podAntiAffinity:
@@ -27,7 +27,7 @@ spec:
- key: app
operator: In
values:
- pooler
- artifact-keeper-pooler
topologyKey: kubernetes.io/hostname
containers: []
type: rw
@@ -0,0 +1,20 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: config
namespace: artifact-keeper
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/part-of: artifact-keeper
data:
BIND_ADDRESS: "0.0.0.0:8080"
LOG_LEVEL: "info,artifact_keeper=debug"
STORAGE_BACKEND: "s3"
MEILISEARCH_URL: "http://meilisearch:7700"
TRIVY_URL: "http://trivy:8090"
DEPENDENCY_TRACK_URL: "http://dtrack:8080"
DEPENDENCY_TRACK_ENABLED: "true"
SCAN_WORKSPACE_PATH: "/scan-workspace"
PLUGINS_DIR: "/data/plugins"
@@ -0,0 +1,15 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: s3-env
namespace: artifact-keeper
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/part-of: artifact-keeper
data:
S3_ENDPOINT: "https://radosgw.service.consul"
S3_BUCKET: "artifact-keeper"
S3_REGION: "ap-southeast-2"
S3_PATH_STYLE: "true"
@@ -0,0 +1,171 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: backend
namespace: artifact-keeper
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/part-of: artifact-keeper
app.kubernetes.io/component: backend
annotations:
reloader.stakater.com/auto: "true"
spec:
replicas: 2
selector:
matchLabels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/component: backend
template:
metadata:
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/component: backend
spec:
serviceAccountName: backend
automountServiceAccountToken: false
securityContext:
runAsNonRoot: true
runAsUser: 1000
fsGroup: 1000
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app.kubernetes.io/component
operator: In
values:
- backend
topologyKey: kubernetes.io/hostname
initContainers:
- name: wait-for-postgres
image: postgres:16-alpine
securityContext:
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 100m
memory: 64Mi
command: ["/bin/sh", "-c"]
args:
- |
echo "Waiting for PostgreSQL..."
until pg_isready -h artifact-keeper-postgres-pooler -p 5432 -U registry; do
sleep 3
done
echo "PostgreSQL is ready"
- name: wait-for-meilisearch
image: alpine:3.20
securityContext:
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
resources:
requests:
cpu: 10m
memory: 16Mi
ephemeral-storage: 32Mi
limits:
cpu: 100m
memory: 64Mi
ephemeral-storage: 64Mi
command: ["/bin/sh", "-c"]
args:
- |
echo "Waiting for Meilisearch..."
until wget -qO- http://meilisearch:7700/health >/dev/null 2>&1; do
sleep 3
done
echo "Meilisearch is ready"
containers:
- name: backend
image: "ghcr.io/artifact-keeper/artifact-keeper-backend:dev"
imagePullPolicy: Always
securityContext:
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
command: ["/bin/sh", "-c"]
args:
- |
if [ -f /shared/dtrack-api-key ] && [ -s /shared/dtrack-api-key ]; then
export DEPENDENCY_TRACK_API_KEY="$(cat /shared/dtrack-api-key)"
fi
exec /usr/local/bin/artifact-keeper
ports:
- name: http
containerPort: 8080
protocol: TCP
- name: grpc
containerPort: 9090
protocol: TCP
envFrom:
- configMapRef:
name: config
- configMapRef:
name: s3-env
- secretRef:
name: s3-credentials
- secretRef:
name: app-secrets
resources:
limits:
cpu: "2"
memory: 2Gi
requests:
cpu: 250m
memory: 256Mi
livenessProbe:
httpGet:
path: /livez
port: http
periodSeconds: 15
timeoutSeconds: 5
failureThreshold: 5
volumeMounts:
- name: tmp
mountPath: /tmp
- name: storage
mountPath: /data/storage
subPath: storage
- name: storage
mountPath: /data/backups
subPath: backups
- name: storage
mountPath: /data/plugins
subPath: plugins
- name: scan-workspace
mountPath: /scan-workspace
- name: shared-config
mountPath: /shared
readOnly: true
volumes:
- name: tmp
emptyDir:
sizeLimit: 256Mi
- name: storage
persistentVolumeClaim:
claimName: storage
- name: scan-workspace
persistentVolumeClaim:
claimName: scan-workspace
- name: shared-config
persistentVolumeClaim:
claimName: shared-config
@@ -0,0 +1,111 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: dtrack
namespace: artifact-keeper
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/part-of: artifact-keeper
app.kubernetes.io/component: dependency-track
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/component: dependency-track
template:
metadata:
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/component: dependency-track
spec:
automountServiceAccountToken: false
securityContext:
runAsNonRoot: true
runAsUser: 1000
fsGroup: 1000
initContainers:
- name: wait-for-postgres
image: postgres:16-alpine
securityContext:
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 100m
memory: 64Mi
command: ["/bin/sh", "-c"]
args:
- |
echo "Waiting for PostgreSQL..."
until pg_isready -h artifact-keeper-postgres-pooler -p 5432 -U registry; do
sleep 3
done
echo "PostgreSQL is ready"
containers:
- name: dtrack-api
image: "dependencytrack/apiserver:4.11.4"
imagePullPolicy: IfNotPresent
securityContext:
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
ports:
- name: http
containerPort: 8080
protocol: TCP
env:
- name: ALPINE_DATABASE_MODE
value: "external"
- name: ALPINE_DATABASE_URL
value: "jdbc:postgresql://artifact-keeper-postgres-pooler:5432/dependency_track"
- name: ALPINE_DATABASE_DRIVER
value: "org.postgresql.Driver"
- name: ALPINE_DATABASE_USERNAME
value: "registry"
- name: ALPINE_DATABASE_PASSWORD
valueFrom:
secretKeyRef:
name: postgres-credentials
key: password
- name: ALPINE_DATA_DIRECTORY
value: "/data"
- name: ALPINE_ENFORCE_AUTHENTICATION
value: "true"
- name: ALPINE_CORS_ENABLED
value: "true"
- name: ALPINE_CORS_ALLOW_ORIGIN
value: "*"
- name: JAVA_OPTIONS
value: "-Xmx4g"
resources:
limits:
cpu: "2"
memory: 6Gi
requests:
cpu: 250m
memory: 4Gi
volumeMounts:
- name: dtrack-data
mountPath: /data
- name: tmp
mountPath: /tmp
volumes:
- name: tmp
emptyDir:
sizeLimit: 256Mi
- name: dtrack-data
persistentVolumeClaim:
claimName: dtrack
@@ -0,0 +1,154 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: meilisearch
namespace: artifact-keeper
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/part-of: artifact-keeper
app.kubernetes.io/component: meilisearch
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/component: meilisearch
template:
metadata:
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/component: meilisearch
spec:
automountServiceAccountToken: false
securityContext:
runAsNonRoot: true
runAsUser: 1000
fsGroup: 1000
initContainers:
- name: fix-ownership
image: busybox:1.37
securityContext:
runAsNonRoot: false
runAsUser: 0
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
add:
- CHOWN
- FOWNER
resources:
requests:
cpu: 10m
memory: 16Mi
ephemeral-storage: 32Mi
limits:
cpu: 100m
memory: 64Mi
ephemeral-storage: 64Mi
command: ["sh", "-c", "chown -R 1000:1000 /meili_data"]
volumeMounts:
- name: meilisearch-data
mountPath: /meili_data
- name: version-guard
image: busybox:1.37
securityContext:
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 100m
memory: 64Mi
command: ["sh", "-c"]
args:
- |
EXPECTED="v1.12"
VERSION_FILE="/meili_data/data.ms/VERSION"
if [ ! -f "$VERSION_FILE" ]; then
echo "No existing database, fresh start"
exit 0
fi
CURRENT=$(cat "$VERSION_FILE" 2>/dev/null || echo "unknown")
echo "Current DB version: $CURRENT, expected image: $EXPECTED"
if echo "$CURRENT" | grep -qv "$(echo $EXPECTED | sed 's/^v//')"; then
echo "Version mismatch — wiping data.ms for clean re-index"
rm -rf /meili_data/data.ms
echo "Done. Backend will re-index automatically."
else
echo "Versions match, keeping existing data"
fi
volumeMounts:
- name: meilisearch-data
mountPath: /meili_data
containers:
- name: meilisearch
image: "getmeili/meilisearch:v1.12"
imagePullPolicy: IfNotPresent
securityContext:
readOnlyRootFilesystem: false
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
ports:
- name: http
containerPort: 7700
protocol: TCP
env:
- name: MEILI_MASTER_KEY
valueFrom:
secretKeyRef:
name: app-secrets
key: MEILISEARCH_API_KEY
- name: MEILI_ENV
value: "production"
- name: MEILI_MAX_INDEXING_THREADS
value: "4"
resources:
limits:
cpu: "1"
memory: 8Gi
requests:
cpu: 250m
memory: 512Mi
readinessProbe:
httpGet:
path: /health
port: 7700
initialDelaySeconds: 5
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 5
livenessProbe:
httpGet:
path: /health
port: 7700
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 5
volumeMounts:
- name: meilisearch-data
mountPath: /meili_data
- name: tmp
mountPath: /tmp
volumes:
- name: tmp
emptyDir:
sizeLimit: 256Mi
- name: meilisearch-data
persistentVolumeClaim:
claimName: meilisearch
@@ -0,0 +1,87 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: trivy
namespace: artifact-keeper
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/part-of: artifact-keeper
app.kubernetes.io/component: trivy
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/component: trivy
template:
metadata:
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/component: trivy
spec:
automountServiceAccountToken: false
securityContext:
runAsNonRoot: true
runAsUser: 10000
fsGroup: 10000
containers:
- name: trivy
image: "aquasec/trivy:0.62.1"
imagePullPolicy: IfNotPresent
securityContext:
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
command: ["trivy"]
args: ["server", "--listen", "0.0.0.0:8090", "--cache-dir", "/home/trivy/.cache"]
ports:
- name: http
containerPort: 8090
protocol: TCP
resources:
limits:
cpu: "1"
memory: 2Gi
requests:
cpu: 250m
memory: 256Mi
readinessProbe:
tcpSocket:
port: 8090
initialDelaySeconds: 15
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 5
livenessProbe:
tcpSocket:
port: 8090
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
volumeMounts:
- name: trivy-cache
mountPath: /home/trivy/.cache
- name: tmp
mountPath: /tmp
- name: scan-workspace
mountPath: /scan-workspace
readOnly: true
volumes:
- name: tmp
emptyDir:
sizeLimit: 256Mi
- name: trivy-cache
persistentVolumeClaim:
claimName: trivy-cache
- name: scan-workspace
persistentVolumeClaim:
claimName: scan-workspace
@@ -0,0 +1,98 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: web
namespace: artifact-keeper
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/part-of: artifact-keeper
app.kubernetes.io/component: web
spec:
replicas: 2
selector:
matchLabels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/component: web
template:
metadata:
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/component: web
spec:
automountServiceAccountToken: false
securityContext:
runAsNonRoot: true
runAsUser: 1000
fsGroup: 1000
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app.kubernetes.io/component
operator: In
values:
- web
topologyKey: kubernetes.io/hostname
containers:
- name: web
image: "ghcr.io/artifact-keeper/artifact-keeper-web:dev"
imagePullPolicy: Always
securityContext:
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
ports:
- name: http
containerPort: 3000
protocol: TCP
env:
- name: NEXT_PUBLIC_API_URL
value: ""
- name: BACKEND_URL
value: "http://backend:8080"
- name: NODE_ENV
value: "production"
resources:
limits:
cpu: "1"
memory: 1Gi
requests:
cpu: 250m
memory: 256Mi
readinessProbe:
httpGet:
path: /
port: http
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 5
livenessProbe:
httpGet:
path: /
port: http
initialDelaySeconds: 20
periodSeconds: 15
timeoutSeconds: 5
failureThreshold: 5
volumeMounts:
- name: tmp
mountPath: /tmp
- name: nextjs-cache
mountPath: /app/.next/cache
volumes:
- name: tmp
emptyDir:
sizeLimit: 256Mi
- name: nextjs-cache
emptyDir:
sizeLimit: 1Gi
+286
View File
@@ -0,0 +1,286 @@
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: artifact-keeper
namespace: artifact-keeper
annotations:
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/proxy-body-size: 10g
nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
cert-manager.io/cluster-issuer: vault-issuer
cert-manager.io/common-name: artifacts.k8s.syd1.au.unkin.net
cert-manager.io/private-key-size: "4096"
external-dns.alpha.kubernetes.io/hostname: artifacts.k8s.syd1.au.unkin.net
external-dns.alpha.kubernetes.io/target: 198.18.200.0
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
ingressClassName: nginx
rules:
- host: artifacts.k8s.syd1.au.unkin.net
http:
paths:
- path: /api
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /health
pathType: Exact
backend:
service:
name: backend
port:
number: 8080
- path: /ready
pathType: Exact
backend:
service:
name: backend
port:
number: 8080
- path: /v2
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /maven
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /npm
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /pypi
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /nuget
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /cargo
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /gems
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /go
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /helm
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /debian
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /rpm
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /alpine
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /composer
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /conan
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /conda
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /swift
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /terraform
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /cocoapods
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /hex
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /pub
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /lfs
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /ivy
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /chef
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /puppet
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /ansible
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /cran
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /huggingface
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /jetbrains
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /vscode
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /proto
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /incus
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /ext
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
- path: /dtrack
pathType: Prefix
backend:
service:
name: dtrack
port:
number: 8080
- path: /
pathType: Prefix
backend:
service:
name: web
port:
number: 3000
tls:
- hosts:
- artifacts.k8s.syd1.au.unkin.net
secretName: artifacts-tls
@@ -0,0 +1,70 @@
---
apiVersion: batch/v1
kind: Job
metadata:
name: dtrack-init
namespace: artifact-keeper
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/part-of: artifact-keeper
app.kubernetes.io/component: dependency-track
spec:
backoffLimit: 3
template:
metadata:
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/component: dependency-track
spec:
restartPolicy: OnFailure
automountServiceAccountToken: false
securityContext:
runAsNonRoot: true
runAsUser: 1000
fsGroup: 1000
containers:
- name: dtrack-init
image: alpine:3.20
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
resources:
requests:
cpu: 10m
memory: 32Mi
ephemeral-storage: 64Mi
limits:
cpu: 200m
memory: 128Mi
ephemeral-storage: 128Mi
command: ["/bin/sh", "-c"]
args:
- |
apk add --no-cache curl jq >/dev/null 2>&1
/bin/sh /scripts/init-dtrack.sh
env:
- name: DEPENDENCY_TRACK_URL
value: "http://dtrack:8080"
- name: DEPENDENCY_TRACK_ADMIN_PASSWORD
valueFrom:
secretKeyRef:
name: app-secrets
key: DEPENDENCY_TRACK_ADMIN_PASSWORD
volumeMounts:
- name: init-script
mountPath: /scripts
readOnly: true
- name: shared-config
mountPath: /shared
volumes:
- name: init-script
configMap:
name: dtrack-init
defaultMode: 0755
- name: shared-config
persistentVolumeClaim:
claimName: shared-config
@@ -0,0 +1,33 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
- serviceaccount_backend.yaml
- cnpg_cluster.yaml
- cnpg_pooler.yaml
- vaultauth.yaml
- vaultstaticsecret.yaml
- configmap_app-config.yaml
- configmap_s3-env.yaml
- persistentvolumeclaims.yaml
- service_backend.yaml
- service_dtrack.yaml
- service_meilisearch.yaml
- service_trivy.yaml
- service_web.yaml
- deployment_backend.yaml
- deployment_dtrack.yaml
- deployment_meilisearch.yaml
- deployment_trivy.yaml
- deployment_web.yaml
- job_dtrack-init.yaml
- ingress.yaml
configMapGenerator:
- name: dtrack-init
files:
- resources/init-dtrack.sh
options:
disableNameSuffixHash: true
+7
View File
@@ -0,0 +1,7 @@
---
apiVersion: v1
kind: Namespace
metadata:
name: artifact-keeper
labels:
app.kubernetes.io/name: artifact-keeper
@@ -0,0 +1,78 @@
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: storage
namespace: artifact-keeper
spec:
accessModes:
- ReadWriteMany
storageClassName: cephfs-raid5-delete
resources:
requests:
storage: 10Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: scan-workspace
namespace: artifact-keeper
spec:
accessModes:
- ReadWriteMany
storageClassName: cephfs-raid5-delete
resources:
requests:
storage: 2Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: shared-config
namespace: artifact-keeper
spec:
accessModes:
- ReadWriteMany
storageClassName: cephfs-raid5-delete
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: dtrack
namespace: artifact-keeper
spec:
accessModes:
- ReadWriteOnce
storageClassName: cephrbd-fast-delete
resources:
requests:
storage: 5Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: meilisearch
namespace: artifact-keeper
spec:
accessModes:
- ReadWriteOnce
storageClassName: cephrbd-fast-delete
resources:
requests:
storage: 5Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: trivy-cache
namespace: artifact-keeper
spec:
accessModes:
- ReadWriteOnce
storageClassName: cephrbd-fast-delete
resources:
requests:
storage: 5Gi
+43
View File
@@ -0,0 +1,43 @@
#!/bin/sh
set -e
DT_URL="${DEPENDENCY_TRACK_URL:-http://ak-artifact-keeper-dtrack:8080}"
DT_ADMIN_USER="admin"
DT_DEFAULT_PASS="admin"
DT_NEW_PASS="${DEPENDENCY_TRACK_ADMIN_PASSWORD}"
API_KEY_FILE="/shared/dtrack-api-key"
echo "[dtrack-init] Waiting for Dependency-Track at $DT_URL ..."
for i in $(seq 1 60); do
if curl -sf "$DT_URL/api/version" > /dev/null 2>&1; then break; fi
if [ "$i" -eq 60 ]; then echo "[dtrack-init] ERROR: timeout"; exit 1; fi
sleep 5
done
if [ -f "$API_KEY_FILE" ] && [ -s "$API_KEY_FILE" ]; then
echo "[dtrack-init] API key already provisioned -- skipping"
exit 0
fi
TOKEN=$(curl -sf -X POST "$DT_URL/api/v1/user/login" \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "username=${DT_ADMIN_USER}&password=${DT_NEW_PASS}" 2>/dev/null || true)
if [ -z "$TOKEN" ] || echo "$TOKEN" | grep -qi "FORCE_PASSWORD_CHANGE"; then
curl -sf -o /dev/null -X POST "$DT_URL/api/v1/user/forceChangePassword" \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "username=${DT_ADMIN_USER}&password=${DT_DEFAULT_PASS}&newPassword=${DT_NEW_PASS}&confirmPassword=${DT_NEW_PASS}"
TOKEN=$(curl -sf -X POST "$DT_URL/api/v1/user/login" \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "username=${DT_ADMIN_USER}&password=${DT_NEW_PASS}" 2>/dev/null || true)
fi
if [ -z "$TOKEN" ]; then echo "[dtrack-init] ERROR: auth failed"; exit 1; fi
API_KEY=$(curl -sf "$DT_URL/api/v1/team" \
-H "Authorization: Bearer $TOKEN" | \
jq -r '.[] | select(.name == "Automation") | .apiKeys[0].key // empty')
if [ -z "$API_KEY" ]; then echo "[dtrack-init] ERROR: no API key"; exit 1; fi
echo "$API_KEY" > "$API_KEY_FILE"
echo "[dtrack-init] Done"
@@ -0,0 +1,26 @@
---
apiVersion: v1
kind: Service
metadata:
name: backend
namespace: artifact-keeper
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/part-of: artifact-keeper
app.kubernetes.io/component: backend
spec:
type: ClusterIP
ports:
- name: http
port: 8080
targetPort: http
protocol: TCP
- name: grpc
port: 9090
targetPort: grpc
protocol: TCP
selector:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/component: backend
@@ -0,0 +1,22 @@
---
apiVersion: v1
kind: Service
metadata:
name: dtrack
namespace: artifact-keeper
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/part-of: artifact-keeper
app.kubernetes.io/component: dependency-track
spec:
type: ClusterIP
ports:
- name: http
port: 8080
targetPort: http
protocol: TCP
selector:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/component: dependency-track
@@ -0,0 +1,22 @@
---
apiVersion: v1
kind: Service
metadata:
name: meilisearch
namespace: artifact-keeper
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/part-of: artifact-keeper
app.kubernetes.io/component: meilisearch
spec:
type: ClusterIP
ports:
- name: http
port: 7700
targetPort: http
protocol: TCP
selector:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/component: meilisearch
@@ -0,0 +1,22 @@
---
apiVersion: v1
kind: Service
metadata:
name: trivy
namespace: artifact-keeper
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/part-of: artifact-keeper
app.kubernetes.io/component: trivy
spec:
type: ClusterIP
ports:
- name: http
port: 8090
targetPort: http
protocol: TCP
selector:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/component: trivy
@@ -0,0 +1,22 @@
---
apiVersion: v1
kind: Service
metadata:
name: web
namespace: artifact-keeper
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/part-of: artifact-keeper
app.kubernetes.io/component: web
spec:
type: ClusterIP
ports:
- name: http
port: 3000
targetPort: http
protocol: TCP
selector:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/component: web
@@ -0,0 +1,11 @@
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: backend
namespace: artifact-keeper
labels:
app.kubernetes.io/name: artifact-keeper
app.kubernetes.io/instance: ak
app.kubernetes.io/part-of: artifact-keeper
app.kubernetes.io/component: backend
@@ -3,10 +3,10 @@ apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
name: default
namespace: authentik
namespace: artifact-keeper
spec:
allowedNamespaces:
- authentik
- artifact-keeper
kubernetes:
audiences:
- vault
@@ -3,7 +3,7 @@ apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: postgres-credentials
namespace: authentik
namespace: artifact-keeper
spec:
destination:
create: true
@@ -11,7 +11,7 @@ spec:
overwrite: true
hmacSecretData: true
mount: kv
path: kubernetes/namespace/authentik/default/postgres-credentials
path: kubernetes/namespace/artifact-keeper/default/postgres-credentials
refreshAfter: 5m
type: kv-v2
vaultAuthRef: default
@@ -19,16 +19,16 @@ spec:
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: authentik-credentials
namespace: authentik
name: app-secrets
namespace: artifact-keeper
spec:
destination:
create: true
name: authentik-credentials
name: app-secrets
overwrite: true
hmacSecretData: true
mount: kv
path: kubernetes/namespace/authentik/default/authentik-credentials
path: kubernetes/namespace/artifact-keeper/default/app-secrets
refreshAfter: 5m
type: kv-v2
vaultAuthRef: default
@@ -37,7 +37,7 @@ apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: s3-credentials
namespace: authentik
namespace: artifact-keeper
spec:
destination:
create: true
@@ -45,7 +45,7 @@ spec:
overwrite: true
hmacSecretData: true
mount: kv
path: kubernetes/namespace/authentik/default/s3-credentials
path: kubernetes/namespace/artifact-keeper/default/s3-credentials
refreshAfter: 5m
type: kv-v2
vaultAuthRef: default
@@ -2,56 +2,36 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: api
name: artifactapi-deployment
namespace: artifactapi
annotations:
reloader.stakater.com/auto: "true"
spec:
selector:
matchLabels:
app: api
app: artifactapi
strategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
app: api
spec:
automountServiceAccountToken: true
initContainers:
- name: combine-certs
image: alpine:3
command:
- sh
- -c
- cat /etc/ssl/certs/ca-certificates.crt /custom-ca/ca.crt > /combined-certs/ca-certificates.crt
volumeMounts:
- name: vault-ca-cert
mountPath: /custom-ca
readOnly: true
- name: combined-certs
mountPath: /combined-certs
containers:
- name: api
image: git.unkin.net/unkin/artifactapi:v3.6.5
imagePullPolicy: IfNotPresent
- name: artifactapi
image: git.unkin.net/unkin/almalinux9-artifactapi:latest
imagePullPolicy: Always
ports:
- containerPort: 8000
name: http
protocol: TCP
envFrom:
- configMapRef:
name: api-env
name: artifactapi-env
optional: false
- secretRef:
name: environment
optional: false
volumeMounts:
- name: combined-certs
mountPath: /etc/ssl/combined
readOnly: true
livenessProbe:
failureThreshold: 3
httpGet:
@@ -79,13 +59,14 @@ spec:
requests:
cpu: 100m
memory: 256Mi
volumes:
- name: vault-ca-cert
secret:
secretName: vault-ca-cert
items:
- key: ca.crt
path: ca.crt
- name: combined-certs
emptyDir: {}
volumeMounts:
- mountPath: /app/remotes.yaml
mountPropagation: None
name: remotes-config
subPath: remotes.yaml
restartPolicy: Always
volumes:
- configMap:
name: remotes-config
optional: false
name: remotes-config
@@ -2,13 +2,13 @@
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: api-hpa
name: artifactapi-hpa
namespace: artifactapi
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: api
name: artifactapi-deployment
minReplicas: 2
maxReplicas: 10
metrics:
-91
View File
@@ -1,91 +0,0 @@
---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: postgres
namespace: artifactapi
spec:
affinity:
podAntiAffinityType: preferred
bootstrap:
initdb:
database: artifacts
encoding: UTF8
localeCType: C
localeCollate: C
owner: artifacts
secret:
name: postgres-credentials
enablePDB: true
enableSuperuserAccess: false
failoverDelay: 0
imageName: ghcr.io/cloudnative-pg/postgresql:18.1-system-trixie
instances: 3
logLevel: info
maxSyncReplicas: 0
minSyncReplicas: 0
monitoring:
customQueriesConfigMap:
- key: queries
name: cnpg-default-monitoring
disableDefaultQueries: false
enablePodMonitor: false
postgresql:
parameters:
archive_mode: "on"
archive_timeout: 5min
dynamic_shared_memory_type: posix
effective_cache_size: 256MB
full_page_writes: "on"
log_destination: csvlog
log_directory: /controller/log
log_filename: postgres
log_rotation_age: "0"
log_rotation_size: "0"
log_truncate_on_rotation: "false"
logging_collector: "on"
max_connections: "200"
max_parallel_workers: "16"
max_replication_slots: "16"
max_worker_processes: "16"
shared_buffers: 128MB
shared_memory_type: mmap
ssl_max_protocol_version: TLSv1.3
ssl_min_protocol_version: TLSv1.3
wal_keep_size: 256MB
wal_level: logical
wal_log_hints: "on"
wal_receiver_timeout: 5s
wal_sender_timeout: 5s
syncReplicaElectionConstraint:
enabled: false
primaryUpdateMethod: restart
primaryUpdateStrategy: unsupervised
probes:
liveness:
isolationCheck:
connectionTimeout: 1000
enabled: true
requestTimeout: 1000
replicationSlots:
highAvailability:
enabled: true
slotPrefix: _cnpg_
synchronizeReplicas:
enabled: true
updateInterval: 30
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 250m
memory: 256Mi
smartShutdownTimeout: 180
startDelay: 3600
stopDelay: 1800
storage:
resizeInUseVolumes: true
size: 20Gi
storageClass: cephrbd-fast-delete
switchoverDelay: 3600
+16 -5
View File
@@ -2,15 +2,26 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: api-env
name: artifactapi-env
namespace: artifactapi
data:
DBHOST: postgres-pooler
CONFIG_PATH: /app/remotes.yaml
DBHOST: postgres-service
DBNAME: artifacts
DBPORT: "5432"
DBUSER: artifacts
MINIO_BUCKET: artifactapi-prod-k8s-syd1-au
MINIO_BUCKET: artifactapi
MINIO_ENDPOINT: radosgw.service.consul
MINIO_SECURE: "true"
REDIS_URL: redis://redis:6379
SSL_CERT_FILE: /etc/ssl/combined/ca-certificates.crt
REDIS_URL: redis://redis-service:6379
REQUESTS_CA_BUNDLE: /etc/pki/tls/certs/ca-bundle.crt
SSL_CERT_FILE: /etc/pki/tls/certs/ca-bundle.crt
---
apiVersion: v1
kind: ConfigMap
metadata:
name: postgres-env
namespace: artifactapi
data:
POSTGRES_DB: artifacts
POSTGRES_USER: artifacts
-37
View File
@@ -1,37 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
labels:
traefik.io/instance: internal
annotations:
cert-manager.io/cluster-issuer: vault-issuer
cert-manager.io/common-name: artifactapi.k8s.syd1.au.unkin.net
cert-manager.io/private-key-size: "4096"
external-dns.alpha.kubernetes.io/hostname: artifactapi.k8s.syd1.au.unkin.net
external-dns.alpha.kubernetes.io/target: 198.18.200.4
name: artifactapi
namespace: artifactapi
spec:
gatewayClassName: traefik-internal
listeners:
- allowedRoutes:
namespaces:
from: Same
hostname: artifactapi.k8s.syd1.au.unkin.net
name: http
port: 80
protocol: HTTP
- allowedRoutes:
namespaces:
from: Same
hostname: artifactapi.k8s.syd1.au.unkin.net
name: https
port: 443
protocol: HTTPS
tls:
certificateRefs:
- group: ""
kind: Secret
name: artifactapi-tls
mode: Terminate
-59
View File
@@ -1,59 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: http-redirect
namespace: artifactapi
spec:
hostnames:
- artifactapi.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: artifactapi
sectionName: http
rules:
- filters:
- type: RequestRedirect
requestRedirect:
scheme: https
statusCode: 301
matches:
- path:
type: PathPrefix
value: /
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: api-route
namespace: artifactapi
spec:
hostnames:
- artifactapi.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: artifactapi
sectionName: https
rules:
- backendRefs:
- group: ""
kind: Service
name: ui
port: 80
weight: 1
matches:
- path:
type: PathPrefix
value: /ui
- backendRefs:
- group: ""
kind: Service
name: artifactapi
port: 80
weight: 1
matches:
- path:
type: PathPrefix
value: /
+32
View File
@@ -0,0 +1,32 @@
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
cert-manager.io/cluster-issuer: vault-issuer
cert-manager.io/common-name: artifactapi.k8s.syd1.au.unkin.net
cert-manager.io/private-key-size: "4096"
external-dns.alpha.kubernetes.io/hostname: artifactapi.k8s.syd1.au.unkin.net
external-dns.alpha.kubernetes.io/target: 198.18.200.0
nginx.ingress.kubernetes.io/proxy-body-size: 10g
nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
name: artifactapi-ingress
namespace: artifactapi
spec:
ingressClassName: nginx
rules:
- host: artifactapi.k8s.syd1.au.unkin.net
http:
paths:
- backend:
service:
name: artifactapi-api
port:
number: 80
path: /
pathType: Prefix
tls:
- hosts:
- artifactapi.k8s.syd1.au.unkin.net
secretName: artifactapi-tls
+12 -8
View File
@@ -3,17 +3,21 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- api-deployment.yaml
- api-hpa.yaml
- artifactapi-deployment.yaml
- artifactapi-hpa.yaml
- configmap.yaml
- cnpg_cluster.yaml
- cnpg_pooler.yaml
- gateway.yaml
- httproute.yaml
- ingress.yaml
- namespace.yaml
- postgres-deployment.yaml
- pvc.yaml
- redis-deployment.yaml
- services.yaml
- ui-deployment.yaml
- ui-hpa.yaml
- vaultauth.yaml
- vaultstaticsecret.yaml
configMapGenerator:
- name: remotes-config
files:
- resources/remotes.yaml
options:
disableNameSuffixHash: true
@@ -2,66 +2,75 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis
namespace: litellm
name: postgres-deployment
namespace: artifactapi
annotations:
reloader.stakater.com/auto: "true"
spec:
replicas: 1
selector:
matchLabels:
app: redis
app: postgres
strategy:
type: Recreate
template:
metadata:
labels:
app: redis
spec:
automountServiceAccountToken: true
containers:
- name: redis
image: redis:7-alpine
- name: postgres
image: postgres:15-alpine
imagePullPolicy: IfNotPresent
command:
- redis-server
- --save
- "20"
- "1"
ports:
- containerPort: 6379
name: redis
- containerPort: 5432
name: postgres
protocol: TCP
livenessProbe:
exec:
command:
- redis-cli
- ping
failureThreshold: 3
initialDelaySeconds: 30
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 5
envFrom:
- configMapRef:
name: postgres-env
optional: false
- secretRef:
name: postgres-password
optional: false
readinessProbe:
exec:
command:
- redis-cli
- ping
- pg_isready
- -U
- artifacts
- -d
- artifacts
failureThreshold: 3
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
livenessProbe:
exec:
command:
- pg_isready
- -U
- artifacts
- -d
- artifacts
failureThreshold: 3
initialDelaySeconds: 30
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 5
resources:
limits:
cpu: 500m
memory: 512Mi
memory: 1Gi
requests:
cpu: 50m
memory: 128Mi
volumeMounts:
- mountPath: /data
- mountPath: /var/lib/postgresql/data
mountPropagation: None
name: data
name: pgdata
subPath: pgdata
restartPolicy: Always
volumes:
- name: data
- name: pgdata
persistentVolumeClaim:
claimName: litellm-redis-data
claimName: artifactapi-postgres-pgdata
+28
View File
@@ -0,0 +1,28 @@
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: artifactapi-postgres-pgdata
namespace: artifactapi
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
storageClassName: cephrbd-fast-delete
volumeMode: Filesystem
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: artifactapi-redis-data
namespace: artifactapi
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
storageClassName: cephrbd-fast-delete
volumeMode: Filesystem
+21 -11
View File
@@ -2,21 +2,23 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis
annotations:
deployment.kubernetes.io/revision: "1"
name: redis-deployment
namespace: artifactapi
spec:
replicas: 1
selector:
matchLabels:
app: redis
strategy:
type: Recreate
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: redis:7-alpine
imagePullPolicy: IfNotPresent
command:
- redis-server
- --save
@@ -26,13 +28,6 @@ spec:
- containerPort: 6379
name: redis
protocol: TCP
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 50m
memory: 128Mi
livenessProbe:
exec:
command:
@@ -53,4 +48,19 @@ spec:
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 50m
memory: 128Mi
volumeMounts:
- mountPath: /data
mountPropagation: None
name: data
restartPolicy: Always
volumes:
- name: data
persistentVolumeClaim:
claimName: artifactapi-redis-data
@@ -0,0 +1,287 @@
remotes:
github:
base_url: "https://github.com"
type: "remote"
package: "generic"
description: "GitHub releases and files"
include_patterns:
- "apple/foundationdb/.*/libfdb_c.x86_64.so$"
- "astral-sh/ruff/.*/ruff-x86_64-unknown-linux-gnu.tar.gz$"
- "astral-sh/uv/.*/uv-x86_64-unknown-linux-gnu.tar.gz$"
- "camptocamp/prometheus-puppetdb-exporter/.*/prometheus-puppetdb-exporter-.*.linux-amd64.tar.gz$"
- "containernetworking/plugins/.*/cni-plugins-linux-amd64-.*.tgz"
- "ducaale/xh/.*/xh-.*-x86_64-unknown-linux-musl.tar.gz$"
- "etcd-io/etcd/.*/etcd-.*-linux-amd64.tar.gz$"
- "grafana/jsonnet-language-server/.*/jsonnet-language-server_.*_linux_amd64$"
- "gruntwork-io/boilerplate/.*/boilerplate_linux_amd64$"
- "gruntwork-io/terragrunt/.*terragrunt_linux_amd64.*"
- "helmfile/helmfile/.*/helmfile_.*_linux_amd64.tar.gz$"
- "helmfile/vals/.*/vals_.*_linux_amd64.tar.gz$"
- "lxc/incus/.*.tar.gz$"
- "nzbgetcom/nzbget/.*/nzbget-.*.x86_64.rpm$"
- "onedr0p/exportarr/.*/exportarr_.*_linux_amd64.tar.gz$"
- "openbao/openbao-plugins/.*/openbao-plugin-secrets-consul_linux_amd64_.*.tar.gz$"
- "openbao/openbao-plugins/.*/openbao-plugin-secrets-nomad_linux_amd64_.*.tar.gz$"
- "prometheus/node_exporter/.*/node_exporter-.*.linux-amd64.tar.gz$"
- "prometheus-community/bind_exporter/.*/bind_exporter-.*.linux-amd64.tar.gz$"
- "prometheus-community/pgbouncer_exporter/.*/pgbouncer_exporter-.*.linux-amd64.tar.gz$"
- "prometheus-community/postgres_exporter/.*/postgres_exporter-.*.linux-amd64.tar.gz$"
- "rancher/rke2/.*/rke2-images.linux-amd64.tar.zst$"
- "stalwartlabs/stalwart/.*/stalwart-cli-x86_64-unknown-linux-gnu.tar.gz$"
- "stalwartlabs/stalwart/.*/stalwart-foundationdb-x86_64-unknown-linux-gnu.tar.gz$"
- "stalwartlabs/stalwart/.*/stalwart-x86_64-unknown-linux-gnu.tar.gz$"
- "terraform-linters/tflint/.*/tflint_linux_amd64.zip$"
- "tynany/frr_exporter/.*/frr_exporter-.*.linux-amd64.tar.gz$"
- "VictoriaMetrics/VictoriaLogs/.*/victoria-logs-linux-amd64-.*.tar.gz$"
- "VictoriaMetrics/VictoriaLogs/.*/vlutils-linux-amd64-.*.tar.gz$"
- "VictoriaMetrics/VictoriaMetrics/.*/victoria-logs-linux-amd64-.*.tar.gz$"
- "VictoriaMetrics/VictoriaMetrics/.*/victoria-metrics-linux-amd64-.*-cluster.tar.gz$"
- "VictoriaMetrics/VictoriaMetrics/.*/vlutils-linux-amd64-.*.tar.gz$"
- "VictoriaMetrics/VictoriaMetrics/.*/vmutils-linux-amd64-.*.tar.gz$"
- "xorpaul/g10k/.*/g10k-.*-linux-amd64.zip$"
cache:
file_ttl: 0
index_ttl: 0
github_user:
base_url: "https://raw.githubusercontent.com"
type: "remote"
package: "generic"
description: "GitHub User Content"
include_patterns:
- "argoproj/argo-cd/.*.yaml$"
- "yannh/kubernetes-json-schema/master/.*.json$"
- "datreeio/CRDs-catalog/main/.*.json$"
cache:
file_ttl: 0
index_ttl: 0
gitea-dl:
base_url: "https://dl.gitea.com"
type: "remote"
package: "generic"
description: "Gitea download site"
include_patterns:
- "act_runner/.*/act_runner-.*-linux-amd64$"
- "tea/.*/tea-.*-linux-amd64$"
cache:
file_ttl: 0
index_ttl: 0
hashicorp-releases:
base_url: "https://releases.hashicorp.com"
type: "remote"
package: "generic"
description: "HashiCorp product releases"
include_patterns:
- "terraform/.*terraform_.*_linux_amd64\\.zip$"
- "terraform/.*terraform_.*_windows_amd64\\.zip$"
- "terraform/.*terraform_.*_darwin_amd64\\.zip$"
- "vault/.*vault_.*_linux_amd64\\.zip$"
- "vault/.*vault_.*_windows_amd64\\.zip$"
- "vault/.*vault_.*_darwin_amd64\\.zip$"
- "consul-cni/.*/consul-cni_.*_linux_amd64\\.zip$"
- "consul/.*/consul_.*_linux_amd64\\.zip$"
- "nomad-autoscaler/.*/nomad-autoscaler_.*_linux_amd64\\.zip$"
- "nomad/.*/nomad_.*_linux_amd64\\.zip$"
- "packer/.*/packer_.*_linux_amd64\\.zip$"
cache:
file_ttl: 0
index_ttl: 0
rarlab:
base_url: "https://www.rarlab.com"
type: "remote"
package: "generic"
description: "RARLab"
include_patterns:
- "rar/rarlinux-x64-.*.tar.gz"
cache:
file_ttl: 0
index_ttl: 0
alpine:
base_url: "https://dl-cdn.alpinelinux.org"
type: "remote"
package: "alpine"
description: "Alpine Linux APK package repository"
include_patterns:
- ".*/x86_64/.*\\.apk$"
cache:
file_ttl: 0
index_ttl: 7200
almalinux:
base_url: "https://gsl-syd.mm.fcix.net/almalinux"
type: "remote"
package: "rpm"
description: "AlmaLinux RPM package repository"
include_patterns:
- ".*/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*\\.sqlite.*$"
- ".*/repodata/.*\\.xml.*$"
- ".*/repodata/.*\\.yaml.*$"
- ".*/install.img"
- ".*/squashfs.img"
- ".*/updates.img"
- ".*/RPM-GPG-KEY-.*$"
cache:
file_ttl: 0
index_ttl: 7200
ceph-reef:
base_url: "https://download.ceph.com/rpm-reef/"
type: "remote"
package: "rpm"
description: "Ceph Reef 18"
include_patterns:
- ".*/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
cache:
file_ttl: 0
index_ttl: 7200
ceph-squid:
base_url: "https://download.ceph.com/rpm-squid/"
type: "remote"
package: "rpm"
description: "Ceph Squid 19"
include_patterns:
- ".*/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
cache:
file_ttl: 0
index_ttl: 7200
ceph-tentacle:
base_url: "https://download.ceph.com/rpm-tentacle/"
type: "remote"
package: "rpm"
description: "Ceph Tentacle 20"
include_patterns:
- ".*/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
cache:
file_ttl: 0
index_ttl: 7200
epel:
base_url: "https://gsl-syd.mm.fcix.net/epel"
type: "remote"
package: "rpm"
description: "EPEL (Extra Packages for Enterprise Linux)"
include_patterns:
- ".*/Everything/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*\\.sqlite.*$"
- ".*/repodata/.*\\.xml.*$"
- ".*/repodata/.*\\.yaml.*$"
- "RPM-GPG-KEY-.*$"
cache:
file_ttl: 0
index_ttl: 7200
fedora:
base_url: "https://gsl-syd.mm.fcix.net/fedora/linux"
type: "remote"
package: "rpm"
description: "Fedora Linux RPM package repository"
include_patterns:
- "releases/.*/Everything/x86_64/.*\\.rpm$"
- "updates/.*/Everything/x86_64/.*\\.rpm$"
- "development/.*/Everything/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
cache:
file_ttl: 0
index_ttl: 7200
frr:
base_url: "https://rpm.frrouting.org/repo"
type: "remote"
package: "rpm"
description: "FRR RPM package repository"
include_patterns:
- ".*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
cache:
file_ttl: 0
index_ttl: 7200
mariadb:
base_url: "http://mariadb.mirror.digitalpacific.com.au/yum"
type: "remote"
package: "rpm"
description: "MariaDB RPM package repository"
include_patterns:
- ".*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
- ".*/RPM-GPG-KEY-.*$"
cache:
file_ttl: 0
index_ttl: 7200
openvox:
base_url: "https://yum.voxpupuli.org"
type: "remote"
package: "rpm"
description: "OpenVox RPM package repository"
include_patterns:
- ".*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
- "GPG-KEY-.*$"
cache:
file_ttl: 0
index_ttl: 7200
postgresql:
base_url: "https://download.postgresql.org/pub/repos/yum"
type: "remote"
package: "rpm"
description: "PostgreSQL RPM package repository"
include_patterns:
- ".*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
- ".*/RPM-GPG-KEY-.*$"
- ".*/PGDG-RPM-GPG-KEY-.*$"
cache:
file_ttl: 0
index_ttl: 7200
rke2:
base_url: "https://rpm.rancher.io"
type: "remote"
package: "rpm"
description: "RKE2 RPM package repository"
include_patterns:
- ".*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
- "public.key$"
cache:
file_ttl: 0
index_ttl: 7200
zfs:
base_url: "http://download.zfsonlinux.org"
type: "remote"
package: "rpm"
description: "ZFS RPM package repository"
include_patterns:
- ".*\\.rpm$"
- ".*/repodata/.*\\.xml.*$"
cache:
file_ttl: 0
index_ttl: 7200
local-generic:
type: "local"
package: "generic"
description: "Local generic file repository"
cache:
file_ttl: 0
index_ttl: 0
+8 -8
View File
@@ -2,7 +2,7 @@
apiVersion: v1
kind: Service
metadata:
name: artifactapi
name: artifactapi-api
namespace: artifactapi
spec:
internalTrafficPolicy: Cluster
@@ -12,31 +12,31 @@ spec:
protocol: TCP
targetPort: http
selector:
app: api
app: artifactapi
sessionAffinity: None
type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
name: ui
name: postgres-service
namespace: artifactapi
spec:
internalTrafficPolicy: Cluster
ports:
- name: http
port: 80
- name: postgres
port: 5432
protocol: TCP
targetPort: http
targetPort: postgres
selector:
app: ui
app: postgres
sessionAffinity: None
type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
name: redis
name: redis-service
namespace: artifactapi
spec:
internalTrafficPolicy: Cluster
-58
View File
@@ -1,58 +0,0 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: ui
namespace: artifactapi
annotations:
reloader.stakater.com/auto: "true"
spec:
selector:
matchLabels:
app: ui
strategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
app: ui
spec:
automountServiceAccountToken: true
containers:
- name: ui
image: git.unkin.net/unkin/artifactapi-ui:v3.6.5
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
name: http
protocol: TCP
livenessProbe:
failureThreshold: 3
httpGet:
path: /ui
port: http
scheme: HTTP
initialDelaySeconds: 15
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 5
readinessProbe:
failureThreshold: 3
httpGet:
path: /ui
port: http
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 5
successThreshold: 1
timeoutSeconds: 5
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 50m
memory: 128Mi
restartPolicy: Always
-41
View File
@@ -1,41 +0,0 @@
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: ui-hpa
namespace: artifactapi
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: ui
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 60
behavior:
scaleUp:
stabilizationWindowSeconds: 0
selectPolicy: Max
policies:
- type: Percent
value: 100
periodSeconds: 30
- type: Pods
value: 4
periodSeconds: 30
scaleDown:
stabilizationWindowSeconds: 300
selectPolicy: Min
policies:
- type: Percent
value: 10
periodSeconds: 60
- type: Pods
value: 2
periodSeconds: 60
+1 -1
View File
@@ -10,7 +10,7 @@ spec:
kubernetes:
audiences:
- vault
role: default
role: artifactapi
serviceAccount: default
tokenExpirationSeconds: 600
method: kubernetes
+19 -19
View File
@@ -1,23 +1,6 @@
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: postgres-credentials
namespace: artifactapi
spec:
destination:
create: true
name: postgres-credentials
overwrite: true
hmacSecretData: true
mount: kv
path: kubernetes/namespace/artifactapi/default/postgres-credentials
refreshAfter: 5m
type: kv-v2
vaultAuthRef: default
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: environment
namespace: artifactapi
@@ -25,10 +8,27 @@ spec:
destination:
create: true
name: environment
overwrite: true
overwrite: false
hmacSecretData: true
mount: kv
path: kubernetes/namespace/artifactapi/default/environment
path: service/artifactapi/environment
refreshAfter: 5m
type: kv-v2
vaultAuthRef: default
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: postgres-password
namespace: artifactapi
spec:
destination:
create: true
name: postgres-password
overwrite: true
hmacSecretData: true
mount: kv
path: service/artifactapi/postgres-password
refreshAfter: 5m
type: kv-v2
vaultAuthRef: default
-91
View File
@@ -1,91 +0,0 @@
---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: postgres
namespace: authentik
spec:
affinity:
podAntiAffinityType: preferred
bootstrap:
initdb:
database: authentik
encoding: UTF8
localeCType: C
localeCollate: C
owner: authentik
secret:
name: postgres-credentials
enablePDB: true
enableSuperuserAccess: false
failoverDelay: 0
imageName: ghcr.io/cloudnative-pg/postgresql:18.1-system-trixie
instances: 3
logLevel: info
maxSyncReplicas: 0
minSyncReplicas: 0
monitoring:
customQueriesConfigMap:
- key: queries
name: cnpg-default-monitoring
disableDefaultQueries: false
enablePodMonitor: false
postgresql:
parameters:
archive_mode: "on"
archive_timeout: 5min
dynamic_shared_memory_type: posix
effective_cache_size: 256MB
full_page_writes: "on"
log_destination: csvlog
log_directory: /controller/log
log_filename: postgres
log_rotation_age: "0"
log_rotation_size: "0"
log_truncate_on_rotation: "false"
logging_collector: "on"
max_connections: "200"
max_parallel_workers: "16"
max_replication_slots: "16"
max_worker_processes: "16"
shared_buffers: 128MB
shared_memory_type: mmap
ssl_max_protocol_version: TLSv1.3
ssl_min_protocol_version: TLSv1.3
wal_keep_size: 256MB
wal_level: logical
wal_log_hints: "on"
wal_receiver_timeout: 5s
wal_sender_timeout: 5s
syncReplicaElectionConstraint:
enabled: false
primaryUpdateMethod: restart
primaryUpdateStrategy: unsupervised
probes:
liveness:
isolationCheck:
connectionTimeout: 1000
enabled: true
requestTimeout: 1000
replicationSlots:
highAvailability:
enabled: true
slotPrefix: _cnpg_
synchronizeReplicas:
enabled: true
updateInterval: 30
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 250m
memory: 256Mi
smartShutdownTimeout: 180
startDelay: 3600
stopDelay: 1800
storage:
resizeInUseVolumes: true
size: 20Gi
storageClass: cephrbd-fast-delete
switchoverDelay: 3600
-66
View File
@@ -1,66 +0,0 @@
---
apiVersion: postgresql.cnpg.io/v1
kind: Pooler
metadata:
name: postgres-pooler-rw
namespace: authentik
spec:
cluster:
name: postgres
instances: 2
pgbouncer:
parameters:
default_pool_size: "100"
max_client_conn: "400"
paused: false
poolMode: session
template:
metadata:
labels:
app: pooler-rw
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- pooler-rw
topologyKey: kubernetes.io/hostname
containers: []
type: rw
---
apiVersion: postgresql.cnpg.io/v1
kind: Pooler
metadata:
name: postgres-pooler-ro
namespace: authentik
spec:
cluster:
name: postgres
instances: 2
pgbouncer:
parameters:
default_pool_size: "100"
max_client_conn: "400"
paused: false
poolMode: session
template:
metadata:
labels:
app: pooler-ro
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- pooler-ro
topologyKey: kubernetes.io/hostname
containers: []
type: ro
-57
View File
@@ -1,57 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
labels:
traefik.io/instance: internal
annotations:
cert-manager.io/cluster-issuer: vault-issuer
cert-manager.io/common-name: identity.unkin.net
cert-manager.io/private-key-size: "4096"
external-dns.alpha.kubernetes.io/hostname: identity.unkin.net,identity.k8s.syd1.au.unkin.net
external-dns.alpha.kubernetes.io/target: 198.18.200.4
name: authentik
namespace: authentik
spec:
gatewayClassName: traefik-internal
listeners:
- allowedRoutes:
namespaces:
from: Same
hostname: identity.unkin.net
name: http
port: 80
protocol: HTTP
- allowedRoutes:
namespaces:
from: Same
hostname: identity.unkin.net
name: https
port: 443
protocol: HTTPS
tls:
certificateRefs:
- group: ""
kind: Secret
name: authentik-tls
mode: Terminate
- allowedRoutes:
namespaces:
from: Same
hostname: identity.k8s.syd1.au.unkin.net
name: http-internal
port: 80
protocol: HTTP
- allowedRoutes:
namespaces:
from: Same
hostname: identity.k8s.syd1.au.unkin.net
name: https-internal
port: 443
protocol: HTTPS
tls:
certificateRefs:
- group: ""
kind: Secret
name: authentik-tls
mode: Terminate
-59
View File
@@ -1,59 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: authentik-http-redirect
namespace: authentik
spec:
hostnames:
- identity.unkin.net
- identity.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: authentik
sectionName: http
- group: gateway.networking.k8s.io
kind: Gateway
name: authentik
sectionName: http-internal
rules:
- filters:
- type: RequestRedirect
requestRedirect:
scheme: https
statusCode: 301
matches:
- path:
type: PathPrefix
value: /
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: authentik
namespace: authentik
spec:
hostnames:
- identity.unkin.net
- identity.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: authentik
sectionName: https
- group: gateway.networking.k8s.io
kind: Gateway
name: authentik
sectionName: https-internal
rules:
- backendRefs:
- group: ""
kind: Service
name: authentik-server
port: 80
weight: 1
matches:
- path:
type: PathPrefix
value: /
-19
View File
@@ -1,19 +0,0 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- cnpg_cluster.yaml
- cnpg_pooler.yaml
- gateway.yaml
- httproute.yaml
- ldap-gateway.yaml
- ldap-httproute.yaml
- ldap-service.yaml
- ldap-tlsroute.yaml
- namespace.yaml
- redis-deployment.yaml
- redis-pvc.yaml
- redis-service.yaml
- vaultauth.yaml
- vaultstaticsecret.yaml
-47
View File
@@ -1,47 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
labels:
traefik.io/instance: internal
annotations:
cert-manager.io/cluster-issuer: vault-issuer
cert-manager.io/common-name: ldap.k8s.syd1.au.unkin.net
cert-manager.io/private-key-size: "4096"
name: authentik-ldap
namespace: authentik
spec:
gatewayClassName: traefik-internal
listeners:
- allowedRoutes:
namespaces:
from: Same
hostname: ldap.k8s.syd1.au.unkin.net
name: ldaps-internal
port: 636
protocol: TLS
tls:
mode: Passthrough
- allowedRoutes:
namespaces:
from: Same
hostname: ldap.main.unkin.net
name: ldaps-main
port: 636
protocol: TLS
tls:
mode: Passthrough
- allowedRoutes:
namespaces:
from: Same
hostname: ldap.k8s.syd1.au.unkin.net
name: http-dns
port: 80
protocol: HTTP
- allowedRoutes:
namespaces:
from: Same
hostname: ldap.main.unkin.net
name: http-dns-main
port: 80
protocol: HTTP
-32
View File
@@ -1,32 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: authentik-ldap-dns
namespace: authentik
annotations:
external-dns.alpha.kubernetes.io/hostname: ldap.k8s.syd1.au.unkin.net,ldap.main.unkin.net
external-dns.alpha.kubernetes.io/target: 198.18.200.4
spec:
hostnames:
- ldap.k8s.syd1.au.unkin.net
- ldap.main.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: authentik-ldap
sectionName: http-dns
- group: gateway.networking.k8s.io
kind: Gateway
name: authentik-ldap
sectionName: http-dns-main
rules:
- filters:
- type: RequestRedirect
requestRedirect:
scheme: https
statusCode: 301
matches:
- path:
type: PathPrefix
value: /
-18
View File
@@ -1,18 +0,0 @@
---
apiVersion: v1
kind: Service
metadata:
name: authentik-ldap
namespace: authentik
spec:
internalTrafficPolicy: Cluster
ports:
- name: ldaps
port: 6636
protocol: TCP
targetPort: 6636
selector:
app.kubernetes.io/name: authentik
app.kubernetes.io/component: ldap
sessionAffinity: None
type: ClusterIP
-26
View File
@@ -1,26 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: TLSRoute
metadata:
name: authentik-ldaps
namespace: authentik
spec:
hostnames:
- ldap.k8s.syd1.au.unkin.net
- ldap.main.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: authentik-ldap
sectionName: ldaps-internal
- group: gateway.networking.k8s.io
kind: Gateway
name: authentik-ldap
sectionName: ldaps-main
rules:
- backendRefs:
- group: ""
kind: Service
name: authentik-ldap
port: 6636
weight: 1
-5
View File
@@ -1,5 +0,0 @@
---
apiVersion: v1
kind: Namespace
metadata:
name: authentik
-58
View File
@@ -1,58 +0,0 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis
namespace: authentik
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: redis
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: redis:7-alpine
imagePullPolicy: IfNotPresent
args:
- --save
- "20"
- "1"
ports:
- containerPort: 6379
name: redis
protocol: TCP
livenessProbe:
exec:
command:
- redis-cli
- ping
initialDelaySeconds: 5
periodSeconds: 10
readinessProbe:
exec:
command:
- redis-cli
- ping
initialDelaySeconds: 5
periodSeconds: 10
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 50m
memory: 128Mi
volumeMounts:
- mountPath: /data
name: redis-data
volumes:
- name: redis-data
persistentVolumeClaim:
claimName: redis-data
-13
View File
@@ -1,13 +0,0 @@
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: redis-data
namespace: authentik
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
storageClassName: cephrbd-fast-delete
-17
View File
@@ -1,17 +0,0 @@
---
apiVersion: v1
kind: Service
metadata:
name: redis
namespace: authentik
spec:
internalTrafficPolicy: Cluster
ports:
- name: redis
port: 6379
protocol: TCP
targetPort: redis
selector:
app: redis
sessionAffinity: None
type: ClusterIP
-37
View File
@@ -1,37 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
labels:
traefik.io/instance: internal
annotations:
cert-manager.io/cluster-issuer: vault-issuer
cert-manager.io/common-name: rancher.k8s.syd1.au.unkin.net
cert-manager.io/private-key-size: "4096"
external-dns.alpha.kubernetes.io/hostname: rancher.k8s.syd1.au.unkin.net
external-dns.alpha.kubernetes.io/target: "198.18.200.4"
name: rancher
namespace: cattle-system
spec:
gatewayClassName: traefik-internal
listeners:
- allowedRoutes:
namespaces:
from: Same
hostname: rancher.k8s.syd1.au.unkin.net
name: http
port: 80
protocol: HTTP
- allowedRoutes:
namespaces:
from: Same
hostname: rancher.k8s.syd1.au.unkin.net
name: https
port: 443
protocol: HTTPS
tls:
certificateRefs:
- group: ""
kind: Secret
name: rancher-tls
mode: Terminate
-49
View File
@@ -1,49 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: rancher-http-redirect
namespace: cattle-system
spec:
hostnames:
- rancher.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: rancher
sectionName: http
rules:
- filters:
- type: RequestRedirect
requestRedirect:
scheme: https
statusCode: 301
matches:
- path:
type: PathPrefix
value: /
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: rancher
namespace: cattle-system
spec:
hostnames:
- rancher.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: rancher
sectionName: https
rules:
- backendRefs:
- group: ""
kind: Service
name: rancher
port: 80
weight: 1
matches:
- path:
type: PathPrefix
value: /
+29
View File
@@ -0,0 +1,29 @@
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: rancher
namespace: cattle-system
annotations:
cert-manager.io/cluster-issuer: vault-issuer
cert-manager.io/common-name: rancher.k8s.syd1.au.unkin.net
cert-manager.io/private-key-size: "4096"
external-dns.alpha.kubernetes.io/hostname: rancher.k8s.syd1.au.unkin.net
external-dns.alpha.kubernetes.io/target: "198.18.200.0"
spec:
ingressClassName: nginx
tls:
- hosts:
- rancher.k8s.syd1.au.unkin.net
secretName: rancher-tls
rules:
- host: rancher.k8s.syd1.au.unkin.net
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: rancher
port:
number: 80
+1 -2
View File
@@ -6,5 +6,4 @@ resources:
- namespace.yaml
- vaultauth.yaml
- vaultstaticsecret.yaml
- gateway.yaml
- httproute.yaml
- ingress.yaml
-53
View File
@@ -1,53 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: consul
namespace: consul
labels:
app.kubernetes.io/name: consul
app.kubernetes.io/instance: consul
traefik.io/instance: internal
annotations:
cert-manager.io/cluster-issuer: vault-issuer
cert-manager.io/common-name: consul.k8s.syd1.au.unkin.net
cert-manager.io/private-key-size: "4096"
cert-manager.io/alt-names: consul.service.consul
external-dns.alpha.kubernetes.io/hostname: consul.k8s.syd1.au.unkin.net
external-dns.alpha.kubernetes.io/target: 198.18.200.4
spec:
gatewayClassName: traefik-internal
listeners:
- name: http
port: 80
protocol: HTTP
hostname: consul.k8s.syd1.au.unkin.net
allowedRoutes:
namespaces:
from: Same
- name: https
port: 443
protocol: HTTPS
hostname: consul.k8s.syd1.au.unkin.net
allowedRoutes:
namespaces:
from: Same
tls:
mode: Terminate
certificateRefs:
- group: ""
kind: Secret
name: consul-tls
- name: consul-svc
port: 443
protocol: HTTPS
hostname: consul.service.consul
allowedRoutes:
namespaces:
from: Same
tls:
mode: Terminate
certificateRefs:
- group: ""
kind: Secret
name: consul-tls
-83
View File
@@ -1,83 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: consul-http-redirect
namespace: consul
labels:
app.kubernetes.io/name: consul
app.kubernetes.io/instance: consul
spec:
hostnames:
- consul.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: consul
sectionName: http
rules:
- filters:
- type: RequestRedirect
requestRedirect:
scheme: https
statusCode: 301
matches:
- path:
type: PathPrefix
value: /
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: consul
namespace: consul
labels:
app.kubernetes.io/name: consul
app.kubernetes.io/instance: consul
spec:
hostnames:
- consul.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: consul
sectionName: https
rules:
- backendRefs:
- group: ""
kind: Service
name: consul-ui
port: 80
weight: 1
matches:
- path:
type: PathPrefix
value: /
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: consul-svc
namespace: consul
labels:
app.kubernetes.io/name: consul
app.kubernetes.io/instance: consul
spec:
hostnames:
- consul.service.consul
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: consul
sectionName: consul-svc
rules:
- backendRefs:
- group: ""
kind: Service
name: consul-ui
port: 80
weight: 1
matches:
- path:
type: PathPrefix
value: /
-8
View File
@@ -1,8 +0,0 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
- gateway.yaml
- httproute.yaml
-5
View File
@@ -1,5 +0,0 @@
---
apiVersion: v1
kind: Namespace
metadata:
name: consul
@@ -7,12 +7,12 @@ resources:
helmCharts:
- name: intel-device-plugins-operator
repo: https://artifactapi.k8s.syd1.au.unkin.net/api/v1/virtual/helm
repo: https://intel.github.io/helm-charts/
version: "0.35.0"
releaseName: intel-device-plugins-operator
namespace: inteldeviceplugins-system
- name: intel-device-plugins-gpu
repo: https://artifactapi.k8s.syd1.au.unkin.net/api/v1/virtual/helm
repo: https://intel.github.io/helm-charts/
version: "0.34.1"
releaseName: intel-gpu-plugin
namespace: inteldeviceplugins-system
-51
View File
@@ -1,51 +0,0 @@
# kanidm
Three-replica kanidm identity server with Vault-managed replication certificates.
## Architecture
- Per-pod `server-N.toml` in `resources/` — each has its own replication origin hardcoded
- `config-init` busybox init container copies the right config and injects peer certs from the
vault-synced `kanidm-repl-certs` Secret at pod startup
- `reloader.stakater.com/auto: "true"` triggers a rolling restart when the ConfigMap or Secret changes
- Vault path: `kv/kubernetes/namespace/kanidm/default/repl-certs`
- Keys: `kanidm-0`, `kanidm-1`, `kanidm-2` — each holds that pod's replication certificate
## Initial setup
After the first pod starts, generate the admin credentials:
```bash
kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd recover-account -c /config/server.toml admin
kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd recover-account -c /config/server.toml idm_admin
```
## Replication certificate rotation
When certs need to be renewed, update vault and reloader will roll the StatefulSet:
```bash
# Get new cert from a pod
kubectl exec -it -n kanidm kanidm-N -- /sbin/kanidmd renew-replication-certificate -c /config/server.toml
# Write updated cert to vault (reloader triggers restart automatically)
vault kv patch kv/kubernetes/namespace/kanidm/default/repl-certs "kanidm-N=<cert>"
```
## Resolving domain UUID mismatch
If pods initialized independently (each with a different domain UUID), replication will fail with
`Consumer Domain UUID does not match`. Fix by resetting kanidm-1 and kanidm-2 to sync from
kanidm-0 (the authoritative node):
```bash
# Scale down to avoid split-brain during reset
kubectl scale statefulset -n kanidm kanidm --replicas=1
# Delete the stale PVCs for the replica pods
kubectl delete pvc -n kanidm data-kanidm-1 data-kanidm-2
# Scale back up — replicas start with empty DBs and automatic_refresh=true
# will trigger a full sync from kanidm-0 once TLS peer certs are verified
kubectl scale statefulset -n kanidm kanidm --replicas=3
```
-26
View File
@@ -1,26 +0,0 @@
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: kanidm-tls
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
secretName: kanidm-tls
issuerRef:
kind: ClusterIssuer
name: vault-issuer
commonName: auth.unkin.net
dnsNames:
- auth.unkin.net
- au.auth.unkin.net
- kanidm.k8s.syd1.au.unkin.net
- kanidm.kanidm.svc.cluster.local
- kanidm-0.kanidm-headless.kanidm.svc.cluster.local
- kanidm-1.kanidm-headless.kanidm.svc.cluster.local
- kanidm-2.kanidm-headless.kanidm.svc.cluster.local
privateKey:
algorithm: RSA
size: 4096
-30
View File
@@ -1,30 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: kanidm
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
traefik.io/instance: internal
annotations:
external-dns.alpha.kubernetes.io/hostname: kanidm.k8s.syd1.au.unkin.net
external-dns.alpha.kubernetes.io/target: 198.18.200.4
spec:
gatewayClassName: traefik-internal
listeners:
- name: http
port: 80
protocol: HTTP
allowedRoutes:
namespaces:
from: Same
- name: https-passthrough
port: 443
protocol: TLS
tls:
mode: Passthrough
allowedRoutes:
namespaces:
from: Same
-29
View File
@@ -1,29 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: kanidm-http-redirect
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
hostnames:
- kanidm.k8s.syd1.au.unkin.net
- auth.unkin.net
- au.auth.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: kanidm
sectionName: http
rules:
- filters:
- type: RequestRedirect
requestRedirect:
scheme: https
statusCode: 301
matches:
- path:
type: PathPrefix
value: /
-27
View File
@@ -1,27 +0,0 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
- serviceaccount.yaml
- vaultauth.yaml
- vaultstaticsecret.yaml
- certificate.yaml
- service.yaml
- statefulset.yaml
- poddisruptionbudget.yaml
- gateway.yaml
- httproute.yaml
- tlsroute.yaml
configMapGenerator:
- name: kanidm-config
namespace: kanidm
options:
disableNameSuffixHash: true
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
files:
- server-0.toml=resources/server-0.toml
-5
View File
@@ -1,5 +0,0 @@
---
apiVersion: v1
kind: Namespace
metadata:
name: kanidm
-15
View File
@@ -1,15 +0,0 @@
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: kanidm
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
maxUnavailable: 1
selector:
matchLabels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
-37
View File
@@ -1,37 +0,0 @@
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: kanidm-repl
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create"]
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["kanidm-repl-certs"]
verbs: ["get", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: kanidm-repl
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
subjects:
- kind: ServiceAccount
name: kanidm
namespace: kanidm
roleRef:
kind: Role
name: kanidm-repl
apiGroup: rbac.authorization.k8s.io
-15
View File
@@ -1,15 +0,0 @@
version = "2"
domain = "auth.unkin.net"
origin = "https://auth.unkin.net"
bindaddress = "[::]:8443"
db_path = "/data/kanidm.db"
db_arc_size = 2048
tls_chain = "/data/tls/tls.crt"
tls_key = "/data/tls/tls.key"
log_level = "info"
[online_backup]
path = "/data/backups/"
schedule = "0 22 * * *"
versions = 7
-43
View File
@@ -1,43 +0,0 @@
---
apiVersion: v1
kind: Service
metadata:
name: kanidm
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
type: ClusterIP
sessionAffinity: ClientIP
sessionAffinityConfig:
clientIP:
timeoutSeconds: 10800
ports:
- name: https
port: 8443
targetPort: https
protocol: TCP
selector:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
---
apiVersion: v1
kind: Service
metadata:
name: kanidm-headless
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
type: ClusterIP
clusterIP: None
ports:
- name: https
port: 8443
targetPort: https
protocol: TCP
selector:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
-9
View File
@@ -1,9 +0,0 @@
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: kanidm
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
-85
View File
@@ -1,85 +0,0 @@
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: kanidm
namespace: kanidm
annotations:
reloader.stakater.com/auto: "true"
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
serviceName: kanidm-headless
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
template:
metadata:
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
serviceAccountName: kanidm
securityContext:
runAsUser: 1000
runAsGroup: 1000
runAsNonRoot: true
fsGroup: 1000
containers:
- name: kanidm
image: kanidm/server:1.10.3
command: ["/sbin/kanidmd"]
args: ["server", "-c", "/config/server.toml"]
ports:
- name: https
containerPort: 8443
protocol: TCP
volumeMounts:
- name: data
mountPath: /data
- name: config
mountPath: /config/server.toml
subPath: server-0.toml
readOnly: true
- name: tls
mountPath: /data/tls
readOnly: true
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: false
resources:
requests:
memory: 256Mi
cpu: 100m
limits:
memory: 1Gi
cpu: 500m
readinessProbe:
tcpSocket:
port: 8443
initialDelaySeconds: 15
periodSeconds: 10
livenessProbe:
tcpSocket:
port: 8443
initialDelaySeconds: 30
periodSeconds: 30
volumes:
- name: config
configMap:
name: kanidm-config
- name: tls
secret:
secretName: kanidm-tls
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ReadWriteOnce]
storageClassName: cephrbd-fast-delete
resources:
requests:
storage: 10Gi
-26
View File
@@ -1,26 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: TLSRoute
metadata:
name: kanidm
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
hostnames:
- kanidm.k8s.syd1.au.unkin.net
- auth.unkin.net
- au.auth.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: kanidm
sectionName: https-passthrough
rules:
- backendRefs:
- group: ""
kind: Service
name: kanidm
port: 8443
weight: 1
-18
View File
@@ -1,18 +0,0 @@
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
name: default
namespace: kanidm
spec:
allowedNamespaces:
- kanidm
kubernetes:
audiences:
- vault
role: default
serviceAccount: default
tokenExpirationSeconds: 600
method: kubernetes
mount: k8s/au/syd1
vaultConnectionRef: vso-system/default
-23
View File
@@ -1,23 +0,0 @@
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: repl-certs
namespace: kanidm
labels:
app.kubernetes.io/name: kanidm
app.kubernetes.io/instance: kanidm
spec:
vaultAuthRef: default
mount: kv
type: kv-v2
path: kubernetes/namespace/kanidm/default/repl-certs
refreshAfter: 5m
destination:
name: kanidm-repl-certs
create: true
overwrite: true
hmacSecretData: true
rolloutRestartTargets:
- kind: StatefulSet
name: kanidm
-33
View File
@@ -1,33 +0,0 @@
---
apiVersion: postgresql.cnpg.io/v1
kind: Pooler
metadata:
name: litellm-postgres-pooler
namespace: litellm
spec:
cluster:
name: litellm-postgres
instances: 2
pgbouncer:
parameters:
default_pool_size: "100"
max_client_conn: "400"
paused: false
poolMode: session
template:
metadata:
labels:
app: pooler
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- pooler
topologyKey: kubernetes.io/hostname
containers: []
type: rw
-71
View File
@@ -1,71 +0,0 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: litellm
namespace: litellm
spec:
selector:
matchLabels:
app: litellm
template:
metadata:
annotations:
reloader.stakater.com/auto: "true"
labels:
app: litellm
spec:
containers:
- name: litellm
image: docker.litellm.ai/berriai/litellm-database:main-stable
imagePullPolicy: Always
args:
- --config
- /app/config.yaml
- --port
- "4000"
- --num_workers
- "8"
ports:
- containerPort: 4000
name: http
protocol: TCP
envFrom:
- secretRef:
name: litellm-credentials
- configMapRef:
name: litellm-env
livenessProbe:
httpGet:
path: /health/liveliness
port: 4000
failureThreshold: 3
initialDelaySeconds: 30
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 5
readinessProbe:
httpGet:
path: /health/readiness
port: 4000
failureThreshold: 3
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
resources:
limits:
cpu: "2"
memory: 8Gi
requests:
cpu: 250m
memory: 6Gi
volumeMounts:
- mountPath: /app/config.yaml
name: config
subPath: config.yaml
restartPolicy: Always
volumes:
- name: config
configMap:
name: litellm-config
-37
View File
@@ -1,37 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
labels:
traefik.io/instance: internal
annotations:
cert-manager.io/cluster-issuer: vault-issuer
cert-manager.io/common-name: litellm.k8s.syd1.au.unkin.net
cert-manager.io/private-key-size: "4096"
external-dns.alpha.kubernetes.io/hostname: litellm.k8s.syd1.au.unkin.net
external-dns.alpha.kubernetes.io/target: 198.18.200.4
name: litellm
namespace: litellm
spec:
gatewayClassName: traefik-internal
listeners:
- allowedRoutes:
namespaces:
from: Same
hostname: litellm.k8s.syd1.au.unkin.net
name: http
port: 80
protocol: HTTP
- allowedRoutes:
namespaces:
from: Same
hostname: litellm.k8s.syd1.au.unkin.net
name: https
port: 443
protocol: HTTPS
tls:
certificateRefs:
- group: ""
kind: Secret
name: litellm-tls
mode: Terminate
-41
View File
@@ -1,41 +0,0 @@
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: litellm-hpa
namespace: litellm
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: litellm
minReplicas: 2
maxReplicas: 4
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 80
behavior:
scaleUp:
stabilizationWindowSeconds: 0
selectPolicy: Max
policies:
- type: Percent
value: 100
periodSeconds: 60
- type: Pods
value: 4
periodSeconds: 30
scaleDown:
stabilizationWindowSeconds: 300
selectPolicy: Min
policies:
- type: Percent
value: 30
periodSeconds: 60
- type: Pods
value: 2
periodSeconds: 60
-49
View File
@@ -1,49 +0,0 @@
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: litellm-http-redirect
namespace: litellm
spec:
hostnames:
- litellm.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: litellm
sectionName: http
rules:
- filters:
- type: RequestRedirect
requestRedirect:
scheme: https
statusCode: 301
matches:
- path:
type: PathPrefix
value: /
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: litellm
namespace: litellm
spec:
hostnames:
- litellm.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: litellm
sectionName: https
rules:
- backendRefs:
- group: ""
kind: Service
name: litellm
port: 4000
weight: 1
matches:
- path:
type: PathPrefix
value: /
-29
View File
@@ -1,29 +0,0 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- cnpg_cluster.yaml
- cnpg_pooler.yaml
- deployment.yaml
- hpa.yaml
- gateway.yaml
- httproute.yaml
- namespace.yaml
- redis-deployment.yaml
- redis-pvc.yaml
- services.yaml
- vaultauth.yaml
- vaultstaticsecret.yaml
configMapGenerator:
- name: litellm-config
files:
- config.yaml=resources/config.yaml
options:
disableNameSuffixHash: true
- name: litellm-env
literals:
- STORE_MODEL_IN_DB=True
options:
disableNameSuffixHash: true
-5
View File
@@ -1,5 +0,0 @@
---
apiVersion: v1
kind: Namespace
metadata:
name: litellm
-14
View File
@@ -1,14 +0,0 @@
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: litellm-redis-data
namespace: litellm
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
storageClassName: cephrbd-fast-delete
volumeMode: Filesystem
-15
View File
@@ -1,15 +0,0 @@
model_list: []
router_settings:
redis_host: redis-service
redis_port: 6379
general_settings:
use_redis_transaction_buffer: true
litellm_settings:
cache: true
cache_params:
type: redis
host: redis-service
port: 6379
-34
View File
@@ -1,34 +0,0 @@
---
apiVersion: v1
kind: Service
metadata:
name: litellm
namespace: litellm
spec:
internalTrafficPolicy: Cluster
ports:
- name: http
port: 4000
protocol: TCP
targetPort: http
selector:
app: litellm
sessionAffinity: None
type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
name: redis-service
namespace: litellm
spec:
internalTrafficPolicy: Cluster
ports:
- name: redis
port: 6379
protocol: TCP
targetPort: redis
selector:
app: redis
sessionAffinity: None
type: ClusterIP

Some files were not shown because too many files have changed in this diff Show More