Add gocache serve Deployment with nginx stream sidecar #496

Merged

benvin merged 2 commits from benvin/gocache-serve into main 2026-09-27 00:32:33 +10:00

Conversation 1 Commits 2 Files Changed 4

+196

unkin-agent commented 2026-09-27 00:25:15 +10:00

Member

Copy Link

Copy Source

go-cache-plugin serve binds 127.0.0.1 only, so nothing outside the pod can reach it and laptops have no way to use the S3-backed Go cache without holding RGW credentials.

• Run go-cache-plugin serve against the gocache bucket, path-style, explicit region to skip the GetBucketLocation probe
• Add an nginx sidecar stream-proxying :9090 to the loopback plugin port, proxy_timeout 2h
• Publish it on PureLB 198.18.200.11, externalTrafficPolicy: Local so the client IP reaches the allow rules
• Restrict to workstation + pod CIDRs: GOCACHEPROG is unauthenticated and a poisoned entry runs in every consuming build

Merge only after docker-internal/go-cache-plugin:v0.1.0 is published.

`go-cache-plugin serve` binds `127.0.0.1` only, so nothing outside the pod can reach it and laptops have no way to use the S3-backed Go cache without holding RGW credentials. - Run `go-cache-plugin serve` against the `gocache` bucket, path-style, explicit region to skip the GetBucketLocation probe - Add an nginx sidecar stream-proxying `:9090` to the loopback plugin port, `proxy_timeout 2h` - Publish it on PureLB `198.18.200.11`, `externalTrafficPolicy: Local` so the client IP reaches the allow rules - Restrict to workstation + pod CIDRs: GOCACHEPROG is unauthenticated and a poisoned entry runs in every consuming build Merge only after `docker-internal/go-cache-plugin:v0.1.0` is published.

unkin-agent added 1 commit 2026-09-27 00:25:16 +10:00

Add gocache serve Deployment for the shared Go build cache ... 693f541840

go-cache-plugin serve binds 127.0.0.1 only, so a Service cannot reach it
directly and CI/developer builds have no way to use the S3-backed cache.

- Run go-cache-plugin serve against the gocache RGW bucket, path-style, with
  an explicit region to skip the GetBucketLocation probe
- Add an nginx sidecar stream-proxying 9090 to the loopback plugin port
- Restrict the listener to the workstation and pod CIDRs: GOCACHEPROG is
  unauthenticated and a poisoned entry runs in every consuming build
- Stage the cache on an emptyDir; loss costs a repopulate from S3

unkin-agent added 1 commit 2026-09-27 00:29:30 +10:00

Expose gocache over PureLB instead of ClusterIP ... d83735b3e3

A ClusterIP is only reachable via kubectl port-forward, which needs a
create grant on pods/portforward that the read-only operator context does
not have; laptops reach the cache over wireguard.

- Publish the Service as a LoadBalancer on 198.18.200.11 in the common pool
- Set externalTrafficPolicy Local so the client IP survives to the nginx
  allow rules, matching the bind LoadBalancers

benvin merged commit 02f877540c into main 2026-09-27 00:32:33 +10:00

benvin deleted branch benvin/gocache-serve 2026-09-27 00:32:33 +10:00

unkin-agent commented 2026-09-27 00:38:41 +10:00

Author

Member

Copy Link

Copy Source

No findings.

No findings.

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

benvin (BenVincent) unkin-agent (Unkin Agent) unkinben (Ben Vincent)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unkin/argocd-apps#496