Commit Graph

58 Commits

Author SHA1 Message Date
unkinben c2d23aaeae refactor: convert puppetserver compilers to deployment with configmap integration (#57)
- Convert StatefulSet to Deployment for better scaling flexibility
- Add initContainer to copy configmaps to shared RWX volume (10GB)
- Integrate puppetserver-compiler-config configmap for environment variables
- Configure configMapGenerator with stable names (disableNameSuffixHash)
- Update HPA to target Deployment instead of StatefulSet
- Simplify puppetboard SSL config to skip verification for internal connections

Reviewed-on: #57
2026-03-20 20:47:36 +11:00
unkinben f25117ab7f testing via ingress for puppetdb (#56)
Reviewed-on: #56
2026-03-20 00:00:41 +11:00
unkinben 47b894c450 enable debugging for puppetboard (#55)
Reviewed-on: #55
2026-03-19 23:56:49 +11:00
unkinben 059992f6a3 fix: external access to puppetdb (#53) (#54)
- use vault cert for puppetdb ingress

Reviewed-on: #53

Reviewed-on: #54
2026-03-19 23:32:27 +11:00
unkinben 6ffb0898a4 fix: external access to puppetdb (#53)
- use vault cert for puppetdb ingress

Reviewed-on: #53
2026-03-19 23:26:02 +11:00
unkinben 30d56030b5 fix: increase number of cnpg_pooler_connections (#52)
in previous puppet installs, the puppetdb api service opens MANY
connections. we need to increase the number to greater than 300.

Reviewed-on: #52
2026-03-19 18:37:03 +11:00
unkinben 504d4ae7c9 fix: enable PuppetDB HTTPS support with automatic SSL certificate generation (#51)
This enables secure HTTPS communication to PuppetDB, required for other puppet related services

- make use of USE_OPENVOXSERVER flag

Reviewed-on: #51
2026-03-19 17:06:49 +11:00
unkinben 24d09744e3 git commit -m "fix: configure PuppetDB HTTPS connections and add Puppetboard SSL support (#50)
- Update PuppetDB connections from HTTP (8080) to HTTPS (8081)
- Add automatic certificate generation for Puppetboard using Puppet CA
- Implement initContainers for proper certificate provisioning before app start
- Add dedicated PVC for Puppetboard certificates with RWX access
- Configure SSL verification and client authentication for secure PuppetDB access

Reviewed-on: #50
2026-03-19 16:34:41 +11:00
unkinben 301f8dcc1a fix: add NodeFeatureRule and Intel device plugin permissions to platform project (#49)
- Add nfd.k8s-sigs.io/NodeFeatureRule for node-feature-discovery
- Add deviceplugin.intel.com/* for Intel device plugins (GpuDevicePlugin, etc.)
- Add cert-manager.io resources (Certificate, Issuer) for Intel device plugins

Reviewed-on: #49
2026-03-19 02:20:32 +11:00
unkinben dfbb315522 feat: migrate node-feature-discovery and inteldeviceplugins-system to platform project (#48)
- Add node-feature-discovery and inteldeviceplugins-system to platform project
- Convert intel-nfd-rules from local Helm chart to static NodeFeatureRule manifests
- Add required Helm repositories (NFD OCI registry and Intel charts)
- Create base configurations with Helm charts and overlay structures
- Update platform ApplicationSet and project permissions

Reviewed-on: #48
2026-03-19 02:14:45 +11:00
unkinben d641f630e9 fix: change puppet compilers to use HTTP for internal puppetdb connections (#47)
This resolves SSL certificate verification failures preventing puppetdb access

- Update OPENVOXDB_SERVER_URLS from https://puppetdb:8081 to http://puppetdb:8080
- External access to puppetdb will still use HTTPS via ingress
- Internal cluster communication does not require encryption

Reviewed-on: #47
2026-03-19 01:51:11 +11:00
unkinben c157774033 fix: enable ServerSideApply for ArgoCD ApplicationSets (#46)
- resolve CRD annotation size limit errors by enabling server-side apply
- add storage ApplicationSet and project to kustomization files

Reviewed-on: #46
2026-03-19 01:37:56 +11:00
unkinben 90f793464b feat: migrate CSI drivers to dedicated storage project (#45)
- Migrate csi-cephfs from Terraform to ArgoCD
- Migrate csi-cephrbd from Terraform to ArgoCD
- Create dedicated storage project and ApplicationSet for CSI drivers
- Add csi-* pattern matching in storage ApplicationSet
- Remove CSI apps from platform project to separate concerns

Reviewed-on: #45
2026-03-19 01:29:31 +11:00
unkinben 06a8f98b5c feat: migrate cnpg-system from Terraform to ArgoCD (#44)
- Add cnpg-system base ArgoCD application with namespace
- Create cnpg-system overlay for au-syd1 with CloudNativePG Helm chart
- Update platform ApplicationSet to include cnpg-system deployment
- Configure cloudnative-pg operator v0.27.0 with HA and resource limits
- Maintain one-to-one migration from Terraform configuration

Reviewed-on: #44
2026-03-19 01:25:50 +11:00
unkinben 0bf6e80d6f feat: migrate externaldns from Terraform to ArgoCD (#43)
- Add externaldns base ArgoCD application with namespace and Vault integration
- Create externaldns overlay for au-syd1 with Helm chart configuration
- Update platform ApplicationSet to include externaldns deployment
- Configure external-dns v1.19.0 with RFC2136 provider for DNS updates
- Maintain one-to-one migration from Terraform configuration including TSIG secrets

Reviewed-on: #43
2026-03-19 01:22:39 +11:00
unkinben ed300fabed feat: migrate cert-manager from Terraform to ArgoCD (#42)
- Add cert-manager base ArgoCD application with namespace, RBAC resources
- Create cert-manager overlay for au-syd1 with Helm chart configuration
- Update platform ApplicationSet to include cert-manager deployment
- Configure cert-manager v1.19.2 with jetstack Helm repository
- Maintain one-to-one migration from Terraform configuration

Reviewed-on: #42
2026-03-19 01:18:19 +11:00
unkinben 656aedfc53 fix: enable unscoped permissions (#41)
- add access to create priorityclass resourcees in platform applicationset

Reviewed-on: #41
2026-03-19 01:03:54 +11:00
unkinben ea71ebb55b feat: migrate cattle-system (Rancher) from Terraform to ArgoCD (#39)
- Add cattle-system base ArgoCD application with namespace, Vault integration, and ingress
- Create cattle-system overlay for au-syd1 with Rancher Helm chart configuration
- Update platform ApplicationSet to include cattle-system deployment
- Update platform project to include Rancher Helm repository as source
- Configure Rancher v2.13.1 with HA, TLS, audit logging, and bootstrap secret from Vault
- Maintain one-to-one migration from Terraform configuration

Reviewed-on: #39
2026-03-19 00:56:39 +11:00
unkinben 5255c78927 chore: bump kubetest container (#40)
unkin/packer-images#43

Error: Error: chart requires kubeVersion: < 1.35.0-0 which is incompatible with Kubernetes v1.35.0

Reviewed-on: #40
2026-03-19 00:55:30 +11:00
unkinben 8207935d36 fix: cannot write to certificates namespace (#38)
- enable the platform application to write to certificates namespace

Reviewed-on: #38
2026-03-19 00:20:39 +11:00
unkinben 3f282fbdc2 feat: migrate certificates from Terraform to ArgoCD (#37)
- Add certificates base ArgoCD application with namespace and Vault CA certificate secret
- Create certificates overlay for au-syd1 with static certificate configuration
- Update platform ApplicationSet to include certificates deployment
- Configure Vault CA certificate with reflector annotations for cross-namespace replication
- Maintain one-to-one migration from Terraform configuration

Note: Skip no_plain_secrets hook as this is a public CA certificate that needs
to be replicated via reflector, not a sensitive secret

Reviewed-on: #37
2026-03-19 00:16:33 +11:00
unkinben 3961fe4e68 fix: annotations, not labels (#36)
<picard face palm gif>

- purelb requires annotations not labels

Reviewed-on: #36
2026-03-18 15:17:58 +11:00
unkinben e86cd7a6ae feat: ensure puppet is available externally (#35)
- change puppet/puppetca -> LoadBalancer
- dedicate ip's for puppet and puppetca loadbalancers
- name the puppetserver port
- remove puppet/puppetca ingress

Reviewed-on: #35
2026-03-18 15:07:25 +11:00
unkinben 88fe895409 fix: puppetboard port issues (#34)
service / ingres / deployment mismatch, attempt 2

Reviewed-on: #34
2026-03-18 14:31:43 +11:00
unkinben 687a7f1ffd fix: svc/puppetboard forwarding to wrong port (#33)
puppetboard uses `PUPPETBOARD_PORT` to specify the port, otherwise it
listens on tcp/80

```
ENV PUPPETBOARD_PORT 80
ENV PUPPETBOARD_HOST 0.0.0.0
ENV PUPPETBOARD_STATUS_ENDPOINT /status
ENV PUPPETBOARD_SETTINGS docker_settings.py
EXPOSE 80
```

- change svc/puppetboard to use tcp/80

Reviewed-on: #33
2026-03-18 14:25:00 +11:00
unkinben 64fb4da04c fix: puppetboard tcp is not a valid port (#32)
puppetdb_port has tcp:// in it, even though we pass the correct variable
in from a configmap.

```
ben@metabox ~/s/p/argocd-apps> kubectl --context admin run debug-pod --image=busybox --rm -it --restart=Never -n puppet -- env | grep -i puppetdb_port
PUPPETDB_PORT_8081_TCP_PORT=8081
PUPPETDB_PORT_8081_TCP_PROTO=tcp
PUPPETDB_PORT=tcp://10.43.101.142:8080
PUPPETDB_PORT_8080_TCP=tcp://10.43.101.142:8080
PUPPETDB_PORT_8080_TCP_ADDR=10.43.101.142
PUPPETDB_PORT_8081_TCP=tcp://10.43.101.142:8081
PUPPETDB_PORT_8080_TCP_PROTO=tcp
PUPPETDB_PORT_8081_TCP_ADDR=10.43.101.142
PUPPETDB_PORT_8080_TCP_PORT=8080
```

Reviewed-on: #32
2026-03-18 12:51:54 +11:00
unkinben 35f00858ae fix: puppet-compiler cant find ca (#31)
the puppetca is not pointing to the puppetmasters which prevents the
puppet-compilers from starting, preventing puppetdb/puppetboard from
starting.

- point puppetca service -> puppetserver-master

Reviewed-on: #31
2026-03-18 12:39:38 +11:00
unkinben 276d8c1d78 fix: update service names and references (#30)
updating all the names of services and their respective filenames to
better match the way puppet infra is used in my lab.

- puppet -> the compilers
- puppetca -> the master(s)
- puppetdb -> the puppetdb
- puppetboard -> puppetboard

updated references to these services in all other definitions I could find

note: need a good way to test these changes with argocd

Reviewed-on: #30
2026-03-18 12:19:57 +11:00
unkinben df1b9a5685 feat: complete puppet infrastructure (#29)
complete the implementation of puppet in kubernetes, taking many
features from the openvox helm chart and improving on them. changes from
helm are:
- using vault for storing secrets
- using g10k instead of r10k
- using a single shared g10k cronjob for all masters/compilers
- using a single shared /etc/puppetlabs/code directory (shared, cephfs)

changes:
- deploy puppet master and compiler servers with statefulset/deployment
- deploy puppetdb with postgresql backend, taking advantage of cnpg cluster and pooler
- deploy puppetboard
- all supporting configmaps, services, ingresses, and hpas
- added vaultstaticsecret for eyaml private keys
- configured secure mounting of eyaml keys at /var/lib/puppet/keys/
- updated base kustomization to include all 23 new puppet resource files

Reviewed-on: #29
2026-03-17 20:25:11 +11:00
unkinben 13de81a192 chore: cleanup r10k cache (#28)
g10k hardlinks, so reqires that the cache and code be in the same pvc.
updated r10k repository with cachedir in same pvc, and so now I can
remove these unused pvcs from argo.

unkin/puppet-r10k#4

Reviewed-on: #28
2026-03-17 19:05:21 +11:00
unkinben 02877b6385 fix: include puppet pvc yaml (#27)
- ensure the persistentvolumeclaims.yaml is included in kustomize

Reviewed-on: #27
2026-03-09 01:33:40 +11:00
unkinben b4d6fede98 chore: use specific images for ci tests (#26)
- kubetest contains required rpms
- base contains uv/make

Reviewed-on: #26
2026-03-09 01:13:33 +11:00
unkinben 14e3946d4b feat: initial puppet deployment (#25)
working towards a larger, redundant, autoscaling and simple puppet
implementation in kubernetes. this was originally based on the openvox
helm chart with several improvements (not all in this pr)

- use of cnpg instead of single bitnamilegacy postgres container
- use for g10k instead of r10k
- run one instance of g10k per namespace, instead of per-pod
- store only keep one copy of the environments/branches (instead of per-pod)
- change g10k to native cronjob instead of hacky implementation
- use vault secrets

part one adds:

- cnpg puppetdb pgsql cluster
- cnpg puppetdb pgpooler
- persistent volume claims for puppet, puppetdb, the code repository, etc

Reviewed-on: #25
2026-03-09 01:10:30 +11:00
unkinben 68b753d7fa chore: reload woodpecker (#24)
- add reloader annotations to woodpecker agent/server

Reviewed-on: #24
2026-03-07 16:02:39 +11:00
unkinben d7b661a619 chore: set WOODPECKER_ADMIN (#23)
- enable admin features for myself

Reviewed-on: #23
2026-03-07 15:47:42 +11:00
unkinben 2f6a56d15e chore: add rarlab remote (#22)
- cache rarlab packages
- found they disappear when a new release is available

Reviewed-on: #22
2026-03-07 12:14:04 +11:00
unkinben 563b81c5d2 feat: updates for artifactapi (#21)
- remove replicas (rely on horizontal-pod-scaler)
- add raw.githubusercontent.com remote

Reviewed-on: #21
2026-03-07 00:49:30 +11:00
unkinben e2ada738f8 fix: remove configmap hash (#20)
prevent the automatic hashing of configmaps

Reviewed-on: #20
2026-03-06 22:11:11 +11:00
unkinben 61b3546c2c fix: copy/paste error (#19)
- use correct role for artifactapi to access vault

Reviewed-on: #19
2026-03-06 21:46:01 +11:00
unkinben 05a88459a5 chore: migrate artifactapi to kustomize (#18)
- migrate terraform deployment to kustomize

Reviewed-on: #18
2026-03-06 21:35:47 +11:00
unkinben 0894e51ad5 feat: manage woodpecker-agent-secret in vault (#17)
- unkin/terraform-vault#60

Reviewed-on: #17
2026-03-06 18:33:21 +11:00
unkinben f9a8dca060 chore: change max workflows to string (#16)
WOODPECKER_MAX_WORKFLOWS shows no value in the pods environment, trying
as a string instead

Reviewed-on: #16
2026-03-03 23:14:05 +11:00
unkinben 46e11dd05e chore: increase agents to 3 (#15)
- increase woodpecker agents to 3 for parallel jobs

Reviewed-on: #15
2026-03-03 23:02:15 +11:00
unkinben 244d1b5baa fix: remove revision for pooler (#14)
- artifact from migrating yaml from k8s to argocd

Reviewed-on: #14
2026-03-03 22:50:45 +11:00
unkinben dbd8914013 feat: migrate woodpecker to argocd (#13)
- move woodpecker helm chart deployment to argocd
- move cnpg resources
- move vault resources

Reviewed-on: #13
2026-03-03 22:24:17 +11:00
unkinben 68d872e36f feat: speed up pre-commit (#12)
- move kubeconform job to makefile
- add CI job to run make kubeconform

Reviewed-on: #12
2026-03-02 23:08:04 +11:00
unkinben be9d485bfe feat: testing jfrog-container-registry (#11)
- trialing jfrog container registry

Reviewed-on: #11
2026-03-02 23:07:47 +11:00
unkinben 0daa026f01 feat: add pre-commit workflow (#10)
- enforce pre-commit is run for all pull-requests

Reviewed-on: #10
2026-03-02 00:19:04 +11:00
unkinben 72a892eb14 feat: add pre-commit configuration (#9)
- add pre-commit-config
- add yamllint config
- add ci/validate-* custom scripts
  - verify no secrets added
  - verify clusters with kustomize and kubeconform
  - verify apps with kustomize and kubeconform

Reviewed-on: #9
2026-03-02 00:09:21 +11:00
unkinben ebb47348fe fix: resolve issues with helm deployments (#8)
- remove helm-patch files that are unused
- change platform namespaces allowed to *-system
- change chart name

Reviewed-on: #8
2026-03-01 18:55:47 +11:00