ghp: serve plain HTTP behind the gateway (fix redirect loop) #361

2026-08-13T23:16:37+10:00

unkin-agent commented

2026-08-13 23:16:37 +10:00

https://ghp.unkin.net/ 308-loops onto itself. The traefik gateway terminates TLS and forwards cleartext to the ghp Service port 80 -> container :8080, but :8080 was GHP_SERVER_HTTP_LISTEN — ghp's http->https 308 redirect listener. So ghp bounced every request back to https, the gateway re-forwarded it to :8080, and it looped forever.

Root cause (confirmed against ghp source)

internal/server/server.go Run() picks the serving mode:

hasTLS := s.cfg.Server.HTTPSListen != "" || (systemd socket + certs)
if hasTLS { return s.serveTLS(...) }  // app on HTTPSListen (TLS); HTTPListen = 308 redirect; Listen IGNORED
return s.servePlain(...)              // app (full handler: mgmt UI + API) on Listen, cleartext

It is strictly either/or. serveTLS serves the app on HTTPSListen and gives HTTPListen only httpsRedirectHandler() (redirect.go: http.StatusPermanentRedirect = 308). servePlain serves the same full handler on Listen in cleartext (createListener() uses cfg.Server.Listen). The mgmt UI is the same handler in both modes, so it IS served on the plain Listen port.

Behind a TLS-terminating gateway that forwards cleartext to :8080, ghp therefore has to run in plain mode. Keeping GHP_SERVER_HTTPS_LISTEN would keep hasTLS true, leave GHP_SERVER_LISTEN ignored, and nothing would serve cleartext on :8080.

Change

configmap: drop GHP_SERVER_HTTPS_LISTEN and GHP_SERVER_HTTP_LISTEN; set GHP_SERVER_LISTEN: ":8080" so :8080 SERVES the app; add GHP_SERVER_TRUST_PROXY_HEADERS: "true" so ghp trusts the gateway's X-Forwarded-*/Forwarded for scheme/host (GHP_SERVER_BASE_URL already set).
deployment + vmservicescrape: the metrics server only wraps TLS when hasTLS is true (Run() gates loadTLSConfig on hasTLS); in plain mode it is cleartext, so the /metrics liveness/readiness probes and the VMServiceScrape switch from HTTPS/https to HTTP/http.

Service, HTTPRoute and Gateway are unchanged. configmap+deployment carry the stakater reloader annotation, so pods roll on the change.

Deviation from the brief

The brief said to keep GHP_SERVER_HTTPS_LISTEN: ":8443". Source shows that is incompatible with serving cleartext on :8080 (the two modes are mutually exclusive), so this drops it. The unused GHP_TLS_CERT_FILE/KEY_FILE, the tls volume, and containerPort 8443 are left in place (harmless) for an easy revert to TLS mode. The alternative — gateway -> Service 443 -> :8443 with a BackendTLSPolicy — is the bigger change flagged in the brief and is NOT taken here.

Validated: kustomize build apps/overlays/au-syd1/ghp clean, kubeconform 0 invalid/0 errors, pre-commit clean. Not applied.

https://ghp.unkin.net/ 308-loops onto itself. The traefik gateway terminates TLS and forwards cleartext to the ghp Service port 80 -> container :8080, but :8080 was `GHP_SERVER_HTTP_LISTEN` — ghp's http->https **308 redirect** listener. So ghp bounced every request back to https, the gateway re-forwarded it to :8080, and it looped forever. ## Root cause (confirmed against ghp source) `internal/server/server.go` `Run()` picks the serving mode: ```go hasTLS := s.cfg.Server.HTTPSListen != "" || (systemd socket + certs) if hasTLS { return s.serveTLS(...) } // app on HTTPSListen (TLS); HTTPListen = 308 redirect; Listen IGNORED return s.servePlain(...) // app (full handler: mgmt UI + API) on Listen, cleartext ``` It is **strictly either/or**. `serveTLS` serves the app on `HTTPSListen` and gives `HTTPListen` only `httpsRedirectHandler()` (`redirect.go`: `http.StatusPermanentRedirect` = 308). `servePlain` serves the same full handler on `Listen` in cleartext (`createListener()` uses `cfg.Server.Listen`). The mgmt UI is the same `handler` in both modes, so it IS served on the plain `Listen` port. Behind a TLS-terminating gateway that forwards cleartext to :8080, ghp therefore has to run in **plain mode**. Keeping `GHP_SERVER_HTTPS_LISTEN` would keep `hasTLS` true, leave `GHP_SERVER_LISTEN` ignored, and nothing would serve cleartext on :8080. ## Change - **configmap**: drop `GHP_SERVER_HTTPS_LISTEN` and `GHP_SERVER_HTTP_LISTEN`; set `GHP_SERVER_LISTEN: ":8080"` so :8080 SERVES the app; add `GHP_SERVER_TRUST_PROXY_HEADERS: "true"` so ghp trusts the gateway's `X-Forwarded-*`/`Forwarded` for scheme/host (`GHP_SERVER_BASE_URL` already set). - **deployment + vmservicescrape**: the metrics server only wraps TLS when `hasTLS` is true (`Run()` gates `loadTLSConfig` on `hasTLS`); in plain mode it is cleartext, so the `/metrics` liveness/readiness probes and the VMServiceScrape switch from HTTPS/https to HTTP/http. Service, HTTPRoute and Gateway are unchanged. configmap+deployment carry the stakater reloader annotation, so pods roll on the change. ## Deviation from the brief The brief said to keep `GHP_SERVER_HTTPS_LISTEN: ":8443"`. Source shows that is incompatible with serving cleartext on :8080 (the two modes are mutually exclusive), so this drops it. The unused `GHP_TLS_CERT_FILE`/`KEY_FILE`, the `tls` volume, and containerPort 8443 are left in place (harmless) for an easy revert to TLS mode. The alternative — gateway -> Service 443 -> :8443 with a BackendTLSPolicy — is the bigger change flagged in the brief and is NOT taken here. Validated: `kustomize build apps/overlays/au-syd1/ghp` clean, kubeconform 0 invalid/0 errors, pre-commit clean. Not applied.

unkin-agent added 1 commit 2026-08-13 23:16:37 +10:00

ghp: serve plain HTTP behind the gateway (fix redirect loop)

ci/woodpecker/pr/vector-test Pipeline was successful

Details

ci/woodpecker/pr/pre-commit Pipeline was successful

Details

ci/woodpecker/pr/kubeconform Pipeline was successful

Details

51243d8145

The traefik gateway terminates TLS for ghp.unkin.net and forwards
cleartext to the ghp Service port 80 -> container :8080. But :8080 was
GHP_SERVER_HTTP_LISTEN, ghp's http->https 308 REDIRECT listener, so ghp
bounced every request back to https, which the gateway forwarded to
:8080 again: an infinite ghp.unkin.net -> ghp.unkin.net 308 loop.

Per ghp source (internal/server/server.go Run/serveTLS/servePlain,
redirect.go), the app is served on either GHP_SERVER_LISTEN (plain, full
handler incl. mgmt UI + API) OR GHP_SERVER_HTTPS_LISTEN (own TLS) - it is
strictly either/or: any non-empty https_listen sets hasTLS and runs
serveTLS, in which GHP_SERVER_LISTEN is ignored and http_listen only ever
redirects. To serve cleartext on :8080 behind the TLS-terminating
gateway, ghp must run in plain mode:

- configmap: drop GHP_SERVER_HTTPS_LISTEN + GHP_SERVER_HTTP_LISTEN; set
  GHP_SERVER_LISTEN ":8080" so :8080 SERVES the app; add
  GHP_SERVER_TRUST_PROXY_HEADERS so ghp trusts the gateway's
  X-Forwarded-*/Forwarded for scheme/host.
- deployment + vmservicescrape: the metrics server only wraps TLS when
  hasTLS is true, so in plain mode it is cleartext - switch the /metrics
  probes and the scrape from HTTPS/https to HTTP/http.

Service, HTTPRoute and Gateway are unchanged.

benvin merged commit 7a1e8351a7 into main

2026-08-13 23:19:54 +10:00

benvin deleted branch benvin/ghp-reverse-proxy

2026-08-13 23:19:55 +10:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unkin/argocd-apps#361