Add the golib scaffold and the pg module #1

2026-08-31T22:20:27+10:00

unkin-agent commented

2026-08-31 22:20:27 +10:00

Why

Every service reimplements the same Postgres plumbing; golib gives it one home and one set of tests.

How

Add pg.DSNFromEnv(prefix), consolidating the byte-identical DSN builders in encapi, artifactapi and forgebot: <PREFIX>DATABASE_URL and DATABASE_URL pass through verbatim, then prefixed DB* vars, then libpq PG* vars, then defaults, resolved per field. User and database name have no default and error instead. User and password are percent-escaped, so a password containing @ or / no longer truncates the host.
Add pg.New(ctx, dsn, log), which pings before returning so an unreachable server fails at startup rather than on the first query, and pg.NewMigrated, which migrates before handing the pool back.
Add pg.Migrate(ctx, pool, fsys, pg.MigrateOptions{LockName, Logger}), preserving arrproxy's runner semantics exactly — advisory lock on a dedicated pooled connection, one transaction per file covering both the SQL and its schema_migrations row, connection discarded when the unlock does not land — and generalising the hardcoded lock key to FNV-1a/64 of LockName and the embedded set to an fs.FS. pg.LockKey exposes the derivation so a service can assert it matches its old hardcoded key before switching.
Add pg/pgtest, unifying the encapi and artifactapi testcontainers helpers behind StartPostgres/MustStartPostgres on postgres:17-alpine, with the Ryuk-disable workaround and SkipIfShort so container-backed tests self-skip on the Docker-less Kubernetes runners.
Cover the pgx-facing code with fakes rather than a database: pgMigrator runs against a session interface, so the exact SQL, arguments and transaction ordering are unit-tested. The migration runner keeps arrproxy's fake-DB suite (ordering, lock exclusion, concurrent starters, mid-set failure, unlock failure). Container-backed integration tests cover the real path behind -short.
Add the Makefile: build compiles every package, test runs the unit suite, test-all includes the integration tests, and cover gates on 90% statement coverage over the shipped packages. The gate runs -short, so it clears without a database; pg/pgtest is excluded as test scaffolding. Measured coverage is 93.2%.
Add test, pre-commit and build pull-request pipelines on golib-ci, mirroring mediamark's pre-commit lanes (gofmt/vet, golangci-lint, uvx pre-commit), plus the encapi pre-commit config. No docker.yaml: golib ships no image.
Fill in the README: what golib is, the module list, DSN precedence, migration guarantees, go get git.unkin.net/unkin/golib@vX.Y.Z consumption, and the 90% bar.

make build, make cover (93.2%), golangci-lint, go vet, gofmt and uvx pre-commit run --all-files all pass locally, as do the container-backed integration tests against real Postgres.

## Why Every service reimplements the same Postgres plumbing; golib gives it one home and one set of tests. ## How - Add `pg.DSNFromEnv(prefix)`, consolidating the byte-identical DSN builders in encapi, artifactapi and forgebot: `<PREFIX>DATABASE_URL` and `DATABASE_URL` pass through verbatim, then prefixed `DB*` vars, then libpq `PG*` vars, then defaults, resolved per field. User and database name have no default and error instead. User and password are percent-escaped, so a password containing `@` or `/` no longer truncates the host. - Add `pg.New(ctx, dsn, log)`, which pings before returning so an unreachable server fails at startup rather than on the first query, and `pg.NewMigrated`, which migrates before handing the pool back. - Add `pg.Migrate(ctx, pool, fsys, pg.MigrateOptions{LockName, Logger})`, preserving arrproxy's runner semantics exactly — advisory lock on a dedicated pooled connection, one transaction per file covering both the SQL and its `schema_migrations` row, connection discarded when the unlock does not land — and generalising the hardcoded lock key to FNV-1a/64 of `LockName` and the embedded set to an `fs.FS`. `pg.LockKey` exposes the derivation so a service can assert it matches its old hardcoded key before switching. - Add `pg/pgtest`, unifying the encapi and artifactapi testcontainers helpers behind `StartPostgres`/`MustStartPostgres` on `postgres:17-alpine`, with the Ryuk-disable workaround and `SkipIfShort` so container-backed tests self-skip on the Docker-less Kubernetes runners. - Cover the pgx-facing code with fakes rather than a database: `pgMigrator` runs against a `session` interface, so the exact SQL, arguments and transaction ordering are unit-tested. The migration runner keeps arrproxy's fake-DB suite (ordering, lock exclusion, concurrent starters, mid-set failure, unlock failure). Container-backed integration tests cover the real path behind `-short`. - Add the Makefile: `build` compiles every package, `test` runs the unit suite, `test-all` includes the integration tests, and `cover` gates on 90% statement coverage over the shipped packages. The gate runs `-short`, so it clears without a database; `pg/pgtest` is excluded as test scaffolding. Measured coverage is 93.2%. - Add `test`, `pre-commit` and `build` pull-request pipelines on `golib-ci`, mirroring mediamark's pre-commit lanes (gofmt/vet, golangci-lint, `uvx pre-commit`), plus the encapi pre-commit config. No `docker.yaml`: golib ships no image. - Fill in the README: what golib is, the module list, DSN precedence, migration guarantees, `go get git.unkin.net/unkin/golib@vX.Y.Z` consumption, and the 90% bar. `make build`, `make cover` (93.2%), `golangci-lint`, `go vet`, `gofmt` and `uvx pre-commit run --all-files` all pass locally, as do the container-backed integration tests against real Postgres.

unkin-agent added 1 commit 2026-08-31 22:20:27 +10:00

Add the golib scaffold and the pg module

ci/woodpecker/pr/build Pipeline was successful

Details

ci/woodpecker/pr/test Pipeline was successful

Details

ci/woodpecker/pr/pre-commit Pipeline was successful

Details

d59dcbe74e

Stand up the shared library with its first module: the Postgres plumbing
every service currently copy-pastes — the DSN builder, the pool
constructor, and the migration runner arrproxy proved out.

- Add pg.DSNFromEnv, generalising the identical Sprintf builders in
  encapi, artifactapi and forgebot into one prefixed lookup with
  DATABASE_URL passthrough and libpq fallbacks.
- Add pg.New and pg.NewMigrated, which ping before returning so an
  unreachable server fails at startup rather than on the first query.
- Add pg.Migrate, lifting arrproxy's runner verbatim in semantics and
  generalising the hardcoded advisory-lock key to FNV-1a/64 of a
  caller-supplied name and the embedded set to an fs.FS.
- Add pg/pgtest, unifying the encapi and artifactapi testcontainers
  helpers, with SkipIfShort so container-backed tests self-skip on the
  Docker-less Kubernetes runners.
- Add the Makefile, README and pre-commit config, plus test, pre-commit
  and build pipelines on golib-ci.

unkin-agent referenced this pull request

2026-08-31 22:44:45 +10:00

Add read/write splitting to the pg module #2

benvin merged commit 5a23e0d069 into main

2026-08-31 23:08:38 +10:00

benvin deleted branch benvin/initial-pg

2026-08-31 23:08:38 +10:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unkin/golib#1