golib

Shared Go library for the unkin estate: the plumbing that was being copy-pasted between services, kept in one place with one set of tests.

golib is a library only. It ships no binaries and no container images, holds no service configuration, and takes on dependencies grudgingly — every service that imports it inherits them.

Modules

Import	What it does
git.unkin.net/unkin/golib/pg	Postgres: DSN from the environment, pgxpool construction, read/write splitting across a primary and a replica, and the estate's migration runner.
git.unkin.net/unkin/golib/pg/pgtest	A throwaway Postgres container for a consumer's own _test.go files. Test-only.

pg

dsn, err := pg.DSNFromEnv("ENCAPI_")
if err != nil {
    return err
}
pool, err := pg.NewMigrated(ctx, dsn, migrations.FS, pg.MigrateOptions{
    LockName: "encapi-migrations",
    Logger:   log,
})

pg.DSNFromEnv(prefix) resolves a connection string from the environment. Precedence, highest first:

1. <PREFIX>DATABASE_URL — used verbatim.
2. DATABASE_URL — likewise.
3. <PREFIX>DBHOST, <PREFIX>DBPORT, <PREFIX>DBUSER, <PREFIX>DBPASS, <PREFIX>DBNAME, <PREFIX>DBSSL.
4. libpq's PGHOST, PGPORT, PGUSER, PGPASSWORD, PGDATABASE, PGSSLMODE.
5. Defaults: localhost, 5432, sslmode=disable.

Levels 3 to 5 resolve per field, so a deployment can set the password from a secret and leave the rest to PG*. User and database name have no default — an unset one is an error naming the variables that were checked. An empty prefix reads the bare DBHOST/DBPORT/… names the estate's services already use, so the rendered DSN is unchanged from the fmt.Sprintf builders this replaces.

pg.New opens a pgxpool and pings it, so an unreachable server fails at startup rather than on the first query. pg.NewMigrated does that and then migrates.

pg.Migrate(ctx, pool, fsys, opts) applies the .sql files at the root of fsys in lexical filename order. Every replica calls it at startup:

• The run holds a cluster-wide pg_advisory_lock keyed on FNV-1a/64 of opts.LockName, on one dedicated pooled connection, because the lock is session-scoped. Replicas that queue behind the winner find the set already recorded and do nothing.
• Each file is applied together with its schema_migrations row in a single transaction, so a failure leaves neither a half-tracked migration nor a tracking row that would skip it next time.
• A file missing from schema_migrations is re-applied even if the live database already has it, which is how a schema applied out of band is adopted. Write migrations IF NOT EXISTS-guarded so that re-run is a no-op.
• If the unlock does not land, the connection is discarded rather than returned to the pool, so a session that may still hold the lock cannot be reused.

pg.LockKey(name) exposes the derivation, so a service migrating off a hardcoded key can assert the two agree before switching over.

Read/write splitting

pg.Cluster wraps a primary pool and an optional read-replica pool:

primaryDSN, replicaDSN, err := pg.ClusterDSNsFromEnv("ENCAPI_")
if err != nil {
    return err
}
db, err := pg.NewCluster(ctx, pg.ClusterConfig{
    PrimaryDSN: primaryDSN,
    ReplicaDSN: replicaDSN,
    Logger:     log,
})
if err != nil {
    return err
}
defer db.Close()

if err := db.Migrate(ctx, migrations.FS, pg.MigrateOptions{LockName: "encapi-migrations"}); err != nil {
    return err
}

rows, err := db.Read().Query(ctx, "SELECT id, name FROM nodes")   // replica when healthy
tag, err := db.Write().Exec(ctx, "UPDATE nodes SET ...")          // always the primary
row := db.Primary().QueryRow(ctx, "SELECT ... WHERE id = $1", id) // read-your-writes

Call	Goes to
Write()	The primary, always.
Read()	The replica when one is configured and healthy; the primary otherwise.
Primary()	The primary. Same pool as Write(), named for reads that must not be stale.

Routing is explicit; nothing inspects SQL. Deciding from the statement text gets it wrong in both directions: a CTE with an INSERT in it reads as a SELECT, and SELECT ... FOR UPDATE takes row locks a replica cannot grant. The caller knows which it wants, so the caller picks.

Replica lag is real. A replica serves a slightly stale snapshot, so a read that has to observe a write this process just made goes to Primary(). The usual shape is a handler that writes and then re-reads what it wrote, or a redirect straight into a GET of the row just created — both need the primary. Everything else (list endpoints, dashboards, reports, background aggregation) can take the replica.

A missing replica is not an error. With ReplicaDSN empty, Read() returns the primary and the Cluster is an ordinary single-pool handle, so a service can use Cluster unconditionally and let the deployment decide whether reads split. A ReplicaDSN equal to PrimaryDSN does the same rather than opening a second pool to the same place.

An unhealthy replica falls back. The Cluster keeps a small circuit over the replica:

• A replica that fails its startup ping does not fail startup; reads begin on the primary and move over once it answers a probe.
• db.ReportReplicaError(err) — pass the error from a query run on the pool Read() handed out — trips the circuit, and reads move to the primary.
• While tripped, Read() issues no probes until the backoff window expires; the next Read() after that pays for one probe that either promotes the replica or doubles the window. The window runs from ReplicaRetryMin to ReplicaRetryMax (5s to 2m by default) and resets on recovery. A healthy replica is never probed at all, so the split costs nothing on the read path.
• Reporting is advisory. A caller that never reports still routes correctly; it just does not react to a replica that dies mid-flight.

Migrations always target the primary. Cluster.Migrate runs through Write(). A replica is physically read-only, and a schema change has to originate on the primary to reach the replica at all.

pg.ClusterDSNsFromEnv(prefix) returns both connection strings. The primary follows DSNFromEnv exactly. The replica resolves, highest first:

1. <PREFIX>DATABASE_RO_URL — used verbatim.
2. DATABASE_RO_URL — likewise.
3. <PREFIX>DB_RO_HOST, or bare DB_RO_HOST — the primary's port, user, password, database and sslmode with that host substituted.
4. Nothing set — an empty replica DSN, so the Cluster runs single-pool.

Nothing is derived. The read-only host is never rewritten out of the primary's, because a wrong guess silently sends reads somewhere unintended. Setting only DB_RO_HOST while the primary comes from a whole DATABASE_URL is an error, not a guess: the fields to substitute into are not known.

CloudNativePG publishes exactly the two endpoints this expects — <cluster>-rw routes to the primary and <cluster>-ro to the replicas — so a Deployment names both:

env:
  - name: ENCAPI_DBHOST
    value: encapi-db-rw
  - name: ENCAPI_DB_RO_HOST
    value: encapi-db-ro
  - name: ENCAPI_DBNAME
    value: encapi
  - name: ENCAPI_DBUSER
    valueFrom: { secretKeyRef: { name: encapi-db-app, key: username } }
  - name: ENCAPI_DBPASS
    valueFrom: { secretKeyRef: { name: encapi-db-app, key: password } }

Dropping ENCAPI_DB_RO_HOST turns the split off without a code change.

pgtest

pgtest starts postgres:17-alpine via testcontainers. Import it only from _test.go files. The estate's CI runs on Kubernetes with no Docker socket, so container-backed tests must skip themselves under -short:

func TestSomething(t *testing.T) {
    ctx := context.Background()
    dsn := pgtest.MustStartPostgres(ctx, t) // skips under -short, cleans up after
    ...
}

Consuming

golib is versioned with semver tags and consumed like any Go module. Pin a tag:

go get git.unkin.net/unkin/golib@v0.1.0

Nothing is released until a v* tag exists; make patch, make minor and make major cut and push the next one.

Because everything shares one module path, a consumer that imports only pg still resolves golib's full dependency set in its module graph. That is the reason to keep the dependency list short, and the reason pgtest's testcontainers dependency is the exception rather than the pattern.

Development

make build     # compile every package
make test      # unit tests (-short: no container needed)
make test-all  # everything, including the container-backed integration tests
make cover     # unit tests with the coverage gate
make lint      # golangci-lint

New code needs meaningful tests. make cover fails below 90% statement coverage, measured over the shipped packages from the unit tests alone — the integration tests do not count towards it, so the bar has to be cleared without a database. pg/pgtest is excluded: it is test scaffolding for other repos, and is covered by the integration tests the gate does not run.

CI runs test, pre-commit and build on every pull request.