Adopt golib/pg for migrations and pool construction (#125)
ci/woodpecker/tag/docker Pipeline was successful
ci/woodpecker/tag/docker Pipeline was successful
## Why The Postgres schema was a ~180-line DDL blob inlined in `internal/database/postgres.go` and re-executed on every start. It could only grow — each change appended another `ALTER TABLE ... ADD COLUMN IF NOT EXISTS` — and nothing recorded what had ever been applied. `golib/pg` already owns that mechanic for the estate (arrproxy#10), so artifactapi keeps the SQL and hands over the machinery. ## How - Depend on `git.unkin.net/unkin/golib` v0.1.0. - Move the DDL **verbatim** into `migrations/0001_init.sql`, embedded by the new `migrations` package, and build the pool with `pg.NewMigrated` (`LockName: "artifactapi-migrations"`). The runner takes a cluster-wide advisory lock the old blob never took, so replicas starting together queue instead of racing each other through the DDL. - **No-op on the live database.** Production has the schema but no `schema_migrations`, so its first start on this build re-runs `0001`. Every statement is `IF NOT EXISTS`-guarded, so that run changes nothing and only lands the tracking row. `TestMigratingAnAlreadyPopulatedSchemaIsANoOp` drops the row from a migrated database and asserts exactly that; `TestMigrationsAreIdempotent` keeps future migrations additive and idempotent. - **Env var contract unchanged.** `config.DatabaseDSN` stays the DSN source rather than `pg.DSNFromEnv`: the variable names match, but golib deliberately has no default user or database name, and artifactapi documents and ships `DBUSER`/`DBNAME` defaults of `artifacts`. - Drift guards: the embedded set is asserted equal to `migrations/`, and the derived advisory key is pinned so a rename cannot silently let two builds migrate at once. - Plumb `GOPRIVATE=git.unkin.net` for the first cross-repo Go dependency — exported by the `Makefile`, set in the `Dockerfile` and both woodpecker Go steps, documented in the README. `database.New`'s signature and every caller are untouched. Transitive bumps come with golib's `go.mod` (testcontainers-go 0.42 → 0.44, otel, x/crypto, x/net); `go build`, `go vet`, `golangci-lint`, `go test -race ./pkg/... ./internal/...` and `pre-commit run --all-files` are clean. Reviewed-on: #125 Co-authored-by: unkin-agent <unkin-agent@unkin.net> Co-committed-by: unkin-agent <unkin-agent@unkin.net>
This commit was merged in pull request #125.
This commit is contained in:
@@ -321,6 +321,28 @@ S3/MinIO ─── content-addressable blob storage (blobs/sha256/{hash})
|
||||
|
||||
S3 client supports MinIO, Ceph RGW, and AWS S3 (via minio-go).
|
||||
|
||||
### Schema migrations
|
||||
|
||||
The SQL schema lives in `migrations/` as numbered `.sql` files, embedded into
|
||||
the binary and applied at startup before the server listens, so there is no
|
||||
mirrored copy of the schema in the deployment to drift out of sync.
|
||||
|
||||
The runner is [`golib/pg`](https://git.unkin.net/unkin/golib)'s `pg.NewMigrated`
|
||||
— artifactapi owns the SQL, the shared library owns the mechanics.
|
||||
|
||||
Each start takes `pg_advisory_lock` on a fixed key (FNV-1a/64 of the lock name
|
||||
`artifactapi-migrations`), creates `schema_migrations` (`version`, `applied_at`)
|
||||
if missing, and applies every embedded file whose filename is not yet recorded —
|
||||
in lexical (version) order, each file's SQL and its tracking row in one
|
||||
transaction — then unlocks. Replicas starting together queue on the lock and
|
||||
then find nothing to do.
|
||||
|
||||
A file absent from `schema_migrations` is re-run even when the database already
|
||||
has the schema, which is how a database migrated by the old untracked inline DDL
|
||||
picks `0001_init.sql` up: the statements are `IF NOT EXISTS`-guarded, so the
|
||||
re-run is a no-op that only lands the tracking row. New migrations must stay
|
||||
additive and idempotent for the same reason; a test enforces it.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
@@ -331,6 +353,7 @@ S3 client supports MinIO, Ceph RGW, and AWS S3 (via minio-go).
|
||||
| `DBUSER` | `artifacts` | PostgreSQL user |
|
||||
| `DBPASS` | | PostgreSQL password |
|
||||
| `DBNAME` | `artifacts` | PostgreSQL database |
|
||||
| `DBSSL` | `disable` | PostgreSQL `sslmode` |
|
||||
| `REDIS_URL` | `redis://localhost:6379` | Redis URL |
|
||||
| `MINIO_ENDPOINT` | `localhost:9000` | S3 endpoint |
|
||||
| `MINIO_ACCESS_KEY` | | S3 access key |
|
||||
@@ -358,6 +381,22 @@ make lint # golangci-lint + go vet
|
||||
make fmt # gofmt + goimports
|
||||
```
|
||||
|
||||
### `GOPRIVATE`
|
||||
|
||||
artifactapi depends on `git.unkin.net/unkin/golib`, which is served by Gitea and
|
||||
is unknown to `proxy.golang.org` / `sum.golang.org`. Module resolution therefore
|
||||
needs:
|
||||
|
||||
```
|
||||
export GOPRIVATE=git.unkin.net
|
||||
```
|
||||
|
||||
The `Makefile` exports it for every target, and the `Dockerfile` and the
|
||||
woodpecker Go steps set it themselves, so `make build|test|lint` and CI work on
|
||||
a clean checkout. Only bare `go` commands run outside `make` need it in your
|
||||
shell — set it there (or in your shell profile) rather than with `go env -w`,
|
||||
which is machine state this repo cannot carry.
|
||||
|
||||
### TUI
|
||||
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user