a4808f55288e4c70d9927b077d7eedaadef8b29d
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 unkin-agent
|
a4808f5528 |
ghp: deploy service-token fork image + wire GHP_AUTH_SERVICE_TOKENS (#372)
Why: P4 step 3. Now that the CA-baked buildx plugin lets `docker-internal/ghp` push succeed, deploy the in-house ghp fork (`v0.20.0-unkin4`) which adds static admin service tokens — the mechanism the forthcoming Vault ghp secrets engine uses to authenticate as admin and mint scoped tokens. How: - image: ghcr.io/goodtune/ghp:0.20.0 -> artifactapi.k8s.syd1.au.unkin.net/docker-internal/ghp:v0.20.0-unkin4 (a LOCAL registry image, not the ghcr pull-through that previously ImagePullBackOff-ed). - add env GHP_AUTH_SERVICE_TOKENS from Secret ghp-app key `service_token`, `optional: true` — ghp runs unchanged until the token is seeded. Requires (Ben, runbook): seed `service_token` into kv/kubernetes/namespace/ghp/default/app (same pattern as encryption_key/github-app). The default role's templated policy already grants read; no terraform-vault change. The existing ghp-app reloader annotation rolls the Deployment once the key appears. This same token value later becomes the Vault ghp engine admin_token (P4 steps 5-6). Reviewed-on: #372 Co-authored-by: unkin-agent <unkin-agent@unkin.net> Co-committed-by: unkin-agent <unkin-agent@unkin.net> |
||
|
unkin-agent
|
7a1e8351a7 |
ghp: serve plain HTTP behind the gateway (fix redirect loop) (#361)
https://ghp.unkin.net/ 308-loops onto itself. The traefik gateway terminates TLS and forwards cleartext to the ghp Service port 80 -> container :8080, but :8080 was `GHP_SERVER_HTTP_LISTEN` — ghp's http->https **308 redirect** listener. So ghp bounced every request back to https, the gateway re-forwarded it to :8080, and it looped forever. ## Root cause (confirmed against ghp source) `internal/server/server.go` `Run()` picks the serving mode: ```go hasTLS := s.cfg.Server.HTTPSListen != "" || (systemd socket + certs) if hasTLS { return s.serveTLS(...) } // app on HTTPSListen (TLS); HTTPListen = 308 redirect; Listen IGNORED return s.servePlain(...) // app (full handler: mgmt UI + API) on Listen, cleartext ``` It is **strictly either/or**. `serveTLS` serves the app on `HTTPSListen` and gives `HTTPListen` only `httpsRedirectHandler()` (`redirect.go`: `http.StatusPermanentRedirect` = 308). `servePlain` serves the same full handler on `Listen` in cleartext (`createListener()` uses `cfg.Server.Listen`). The mgmt UI is the same `handler` in both modes, so it IS served on the plain `Listen` port. Behind a TLS-terminating gateway that forwards cleartext to :8080, ghp therefore has to run in **plain mode**. Keeping `GHP_SERVER_HTTPS_LISTEN` would keep `hasTLS` true, leave `GHP_SERVER_LISTEN` ignored, and nothing would serve cleartext on :8080. ## Change - **configmap**: drop `GHP_SERVER_HTTPS_LISTEN` and `GHP_SERVER_HTTP_LISTEN`; set `GHP_SERVER_LISTEN: ":8080"` so :8080 SERVES the app; add `GHP_SERVER_TRUST_PROXY_HEADERS: "true"` so ghp trusts the gateway's `X-Forwarded-*`/`Forwarded` for scheme/host (`GHP_SERVER_BASE_URL` already set). - **deployment + vmservicescrape**: the metrics server only wraps TLS when `hasTLS` is true (`Run()` gates `loadTLSConfig` on `hasTLS`); in plain mode it is cleartext, so the `/metrics` liveness/readiness probes and the VMServiceScrape switch from HTTPS/https to HTTP/http. Service, HTTPRoute and Gateway are unchanged. configmap+deployment carry the stakater reloader annotation, so pods roll on the change. ## Deviation from the brief The brief said to keep `GHP_SERVER_HTTPS_LISTEN: ":8443"`. Source shows that is incompatible with serving cleartext on :8080 (the two modes are mutually exclusive), so this drops it. The unused `GHP_TLS_CERT_FILE`/`KEY_FILE`, the `tls` volume, and containerPort 8443 are left in place (harmless) for an easy revert to TLS mode. The alternative — gateway -> Service 443 -> :8443 with a BackendTLSPolicy — is the bigger change flagged in the brief and is NOT taken here. Validated: `kustomize build apps/overlays/au-syd1/ghp` clean, kubeconform 0 invalid/0 errors, pre-commit clean. Not applied. --------- Co-authored-by: unkin-agent <agent@unkin.net> Reviewed-on: #361 Co-authored-by: Unkin Agent <unkin-agent@unkin.net> Co-committed-by: Unkin Agent <unkin-agent@unkin.net> |
||
|
unkin-agent
|
0130d538f5 |
ghp: use direct ghcr.io image + set GHP_ADMINS (#359)
## Why The `ghp` app is deployed but its pods are stuck 0/1 Ready (and were ImagePullBackOff), for three separate reasons this PR fixes: - **ImagePullBackOff:** kubelet anonymous pulls fail on the artifactapi ghcr pull-through because ghcr.io's per-scope token auth is not proxied. The direct public image pulls anonymously, so switch to it. - **Pods never Ready:** ghp serves its metrics endpoint over **HTTPS** (TLS is configured globally), but the liveness/readiness probes used the default HTTP scheme, so the kubelet probe got an HTTPS-server error and the pods never went Ready. - **Scrape failure:** the VMServiceScrape hits that same HTTPS endpoint and needs a matching scheme/TLS config, or VM scraping of ghp fails. - The `GHP_ADMINS` value was still a placeholder. ## How - `deployment.yaml`: image -> `ghcr.io/goodtune/ghp:0.20.0`; liveness + readiness probe `scheme: HTTP` -> `HTTPS` (kubelet does not verify the probe cert). - `migrate-job.yaml`: image -> `ghcr.io/goodtune/ghp:0.20.0` (shared image). - `vmservicescrape.yaml`: endpoint `scheme: https` + `tlsConfig.insecureSkipVerify: true` (internal-CA cert; pod-IP target not in SANs). - `configmap.yaml`: `GHP_ADMINS` -> `neoloc`. Validated: `kustomize build apps/overlays/au-syd1/ghp` renders clean, kubeconform + pre-commit pass. Not applied. ## Follow-up (not fixed here) The artifactapi ghcr pull-through does not proxy ghcr.io's per-scope token auth for anonymous kubelet pulls — worth closing that gap so estate images can go back through artifactapi. Reviewed-on: #359 Co-authored-by: unkin-agent <unkin-agent@unkin.net> Co-committed-by: unkin-agent <unkin-agent@unkin.net> |
||
|
unkin-agent
|
fa1f3e7756 |
add ghp (GitHub proxy) deployment (#357)
## Why Deploy `goodtune/ghp` (a GitHub proxy) so estate agents have a controlled GitHub access path. Mirrors the artifactapi app structure and the estate's templated `default` Vault convention. ## How (this PR) - Namespace `ghp`. Runs as the namespace's auto-created `default` ServiceAccount (no custom SA). - CNPG `Cluster` (named `postgres`, db/owner `ghp`) + dedicated `cnpg-ghp` Ceph RGW backup bucket + nightly ScheduledBackup. - Deployment `command: ["/ghp","serve"]`; `GHP_DATABASE_DSN` assembled from the CNPG-generated `postgres-app` Secret via `$(VAR)` expansion pointing at `postgres-rw.ghp.svc`. - **Sync-phase migrate hook Job** `ghp-migrate` (`/ghp migrate`) at sync-wave 1 so serve replicas never race migrations. - VSO wiring on the templated `default` role: `VaultAuth` role/SA `default`; `VaultStaticSecret`s source `kv/kubernetes/namespace/ghp/default/github-app` → Secret `ghp-github-app` and `kv/kubernetes/namespace/ghp/default/app` → Secret `ghp-app`. `private_key` mounted as a file; `encryption_key` → `GHP_ENCRYPTION_KEY`. - cert-manager `Certificate` `ghp-tls` off ClusterIssuer `vault-issuer`, SANs github.com/api.github.com/codeload.github.com/*.githubcopilot.com/ghp.k8s.syd1.au.unkin.net; mounted for `GHP_TLS_CERT_FILE`/`GHP_TLS_KEY_FILE`. - Service (443->8443, 80->8080, metrics 9136), PDB, VMServiceScrape, internal Traefik Gateway + HTTPRoute for `ghp.k8s.syd1.au.unkin.net`. - Sync-wave ordering: DB + VSO/cert (wave 0, wait Healthy) -> migrate Job (wave 1) -> serve Deployment + Service/Gateway/HTTPRoute (wave 2). Writable `/tmp` emptyDir on both pods (root FS is read-only). - Registered in the `platform` ApplicationSet + AppProject. - Validation: `kustomize build` clean, kubeconform 17/17 valid (0 skipped), yamllint + pre-commit clean. ## NOTE — prerequisites before ghp goes Healthy - **`GHP_ADMINS` placeholder**: `apps/base/ghp/configmap.yaml` has `GHP_ADMINS: "REPLACE_ME_ben_github_username"` — set it to Ben's GitHub username. - Ben's GitHub-App runbook + kv population at the templated paths `kv/kubernetes/namespace/ghp/default/github-app` (app_id/client_id/client_secret/private_key) and `kv/kubernetes/namespace/ghp/default/app` (encryption_key). - tf-artifactapi #25: the `ghcr` remote pattern for pulling `ghcr/goodtune/ghp`. - **No terraform-vault change needed** — the estate's templated `default` policy already grants read on `kv/kubernetes/namespace/<ns>/default/*` (the bespoke ghp role PR terraform-vault#120 is closed). - DNS overrides (pointing github.com etc. at ghp) are intentionally **out of scope** — separate deferred bind-operator PR. Assumptions flagged: metrics env var `GHP_METRICS_LISTEN`; two Secrets (`ghp-github-app` + `ghp-app`) rather than one merged Secret; DSN assumes the CNPG-generated password is URL-safe. Reviewed-on: #357 Co-authored-by: unkin-agent <unkin-agent@unkin.net> Co-committed-by: unkin-agent <unkin-agent@unkin.net> |