unkin-agent
415bf0cce1
Single Go binary for the ClickHouse log store (logs.raw): chlog with cat/tail/grep subcommands, plus chcat/chtail/chgrep argv[0]-dispatched symlink entrypoints. Every query is time-bounded and fully parameterized; chgrep guards wide unfiltered scans. Ships nfpm RPM with completions and woodpecker PR/tag pipelines mirroring node-lookup.
66 lines
2.4 KiB
Markdown
66 lines
2.4 KiB
Markdown
# clickhouse-tools
|
|
|
|
CLI tools for the ClickHouse log store (`logs.raw`): one binary, `chlog`, with
|
|
three entrypoints installed as symlinks:
|
|
|
|
| Command | Also as | Does |
|
|
|----------|---------------|------|
|
|
| `chcat` | `chlog cat` | Print logs oldest-first over a bounded time range |
|
|
| `chtail` | `chlog tail` | Follow logs live (2s poll, overlap + dedupe so nothing is lost or repeated) |
|
|
| `chgrep` | `chlog grep` | Search log messages (substring, `-i`, `--regex`) |
|
|
|
|
## Why time bounds everywhere
|
|
|
|
`logs.raw` has no text index and holds ~281M rows/day (3-day TTL). An unbounded
|
|
message scan takes ~1 minute and the server kills queries at 120s. Every query
|
|
these tools issue is therefore time-bounded — the default range is the last
|
|
hour (`--since 1h`) — and `chgrep` refuses a search wider than 6h with no
|
|
`--namespace`/`--host`/`--app` filter unless you pass `--force`.
|
|
|
|
All user input travels as ClickHouse HTTP `{name:Type}` parameters; nothing is
|
|
ever interpolated into SQL text.
|
|
|
|
## Usage
|
|
|
|
```sh
|
|
chcat -n logging --since 30m
|
|
chcat --host web01 --since 2h --until 1h --format logfmt
|
|
chtail -n media --app jellyfin
|
|
chgrep -n kube-system -i "connection refused" --since 4h
|
|
chgrep --app vector --regex 'timed? ?out' --since 1d
|
|
chgrep --fields req_id=42 -n api "payment"
|
|
```
|
|
|
|
### Common flags
|
|
|
|
- `--since` / `--until` — duration ago (`15m`, `1h`, `2d`, `1w`) or RFC3339;
|
|
default `--since 1h`, `--until` now
|
|
- `-n/--namespace`, `--host`, `--pod`, `--container`, `--app` (labels['app']),
|
|
`--severity` (case-insensitive), `--stream`, `--source`
|
|
- `--limit` — max rows (default 10000 for cat/grep; tail is unlimited)
|
|
- `--format text|json|logfmt` — text is `ts ns/pod msg` (host for vm rows),
|
|
colored only on a TTY (`NO_COLOR` respected)
|
|
|
|
### chgrep extras
|
|
|
|
- pattern is a substring by default; `-i` case-insensitive; `--regex` RE2 (`match()`)
|
|
- `--fields key=value` (repeatable) filters the structured `fields` map
|
|
- `--force` overrides the wide-unfiltered-search guard
|
|
|
|
## Connection
|
|
|
|
| Env | Default |
|
|
|-----|---------|
|
|
| `CH_URL` | `http://clickhouse-logs.logging.svc.cluster.local:8123` |
|
|
| `CH_USER` | `logreader` |
|
|
| `CH_PASSWORD` | (empty) |
|
|
|
|
## Build and release
|
|
|
|
```sh
|
|
make build # dist/chlog + symlinks
|
|
make test # go test -race ./...
|
|
make rpm # nfpm RPM with binary, symlinks, bash/zsh/fish completions
|
|
make patch # tag + push next vX.Y.Z → CI releases RPM to artifactapi rpm-internal
|
|
```
|