Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,6 @@
/go/cmd/vserver/ @GreenNodeHub/vserver
/go/internal/vserverclient/ @GreenNodeHub/vserver
/docs/commands/vserver/ @GreenNodeHub/vserver
/go/cmd/vdb/ @GreenNodeHub/vdb
/go/internal/vdbclient/ @GreenNodeHub/vdb
/docs/commands/vdb/ @GreenNodeHub/vdb
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,5 @@ build/

# Internal design docs (local only, not for public repo)
docs/superpowers/
docs/api-docs-local/

76 changes: 74 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,38 @@ go/
│ │ ├── helpers.go # mustLoadConfig / newAuthProvider
│ │ ├── tools.go # tools group: built-in browsers (+ sessions, streams, recording; delete has confirm/--force/--dry-run)
│ │ └── tools_code_interpreter.go # tools code-interpreter + sessions + session invoke (+ async task-wait)
│ ├── vdb/ # grn vdb (behind !vks_only tag, in development)
│ │ ├── vdb.go # VdbCmd parent + --user-type persistent flag
│ │ ├── CLAUDE.md # vDB API quirks (differ from BOTH vks and vserver)
│ │ ├── relational/ # /vdb-relational — one subpackage per engine family
│ │ │ ├── instance/ # 18 commands: read + lifecycle (create/resize/start/
│ │ │ │ # stop/reboot/delete) + replicas + settings/secrules
│ │ │ ├── catalog/ # read-only lookups: engines/datastores/flavors/families/
│ │ │ │ # codes/zones/networks/subnets/volume-types/config-groups
│ │ │ ├── backup/ # list/get/create/delete/restore/get-free-storage
│ │ │ ├── configuration/ # config groups: list/get/create/update/delete/list-params
│ │ │ └── backupstorage/ # quota: list/list-packages/create/resize/delete
│ │ ├── memorystore/ # MemoryStore (Redis): 42 commands, same 5 nouns as relational
│ │ │ │ # but almost every path differs; no volume/resize-storage
│ │ │ ├── instance/ # 16 commands (master password instead of a db user)
│ │ │ ├── catalog/ # 9 lookups under /database/* (no zones/subnets)
│ │ │ ├── backup/ configuration/ backupstorage/
│ │ ├── postgresql/ # PostgreSQL Cluster: /vdb-postgresql + 7 relational endpoints
│ │ │ ├── cluster/ # list/get/histories/secrules/create/resize/settings/
│ │ │ │ # config-group/reboot/delete (+ pg- ID guard)
│ │ │ ├── catalog/ # versions/flavors/volume-types/config-groups/backup opts
│ │ │ └── backup/ # list/get/list-restore-points/create (keyed by CLUSTER id)
│ │ └── kafka/ # Kafka: 36 commands / 35 endpoints, /vdb-kafka (NO /v1)
│ │ │ # envelope on 9/36 only, no pagination, no backups
│ │ ├── cluster/ # 14: read + create/delete + resize-brokers/resize-storage/
│ │ │ # update-volume-type + authentication/config-group/
│ │ │ # public-access + secrule create/delete (list-secrules
│ │ │ # reads cluster get — the API has no endpoint)
│ │ ├── topic/ # list/get/create/update/delete (under a cluster)
│ │ ├── user/ # + get-creds/generate-creds; permissions by topic NAME
│ │ ├── configuration/ # VERSIONED config groups: create-version instead of update
│ │ └── catalog/ # flavors/families/codes/volume-types (versions come from
│ │ # /database/configs, which has no command of its own)
│ ├── configure/
│ │ ├── configure.go # Interactive setup
│ │ ├── list.go # grn configure list
Expand Down Expand Up @@ -77,6 +109,10 @@ go/
│ ├── resources/
│ │ └── vserver/
│ │ └── vserver.go # vserver resource completers (vpc/subnet/ssh-key/security-group/disk-type)
│ ├── vdbclient/
│ │ ├── client.go # BuildClient(cmd) + user-type header
│ │ ├── query.go # BuildListQuery — 1-based pageNumber, flat filters
│ │ └── output.go # Unwrap({code,message,data}) + Output/OutputWithColumns
│ ├── agentbase/ # self-contained agentbase stack (own auth/config/client)
│ │ ├── auth/ # OAuth2 v2 clientcredentials
│ │ ├── client/ # bearer-token HTTP client (401 refresh + retry-once, mirrors internal/client)
Expand Down Expand Up @@ -110,8 +146,24 @@ scripts/
## GreenNode API quirks

- **IAM API uses camelCase**: `grantType`, `accessToken`, `expiresIn`
- **VKS API pagination is 0-based**: page 0 = first page
- **`--version` conflict**: Use `--k8s-version` for Kubernetes version
- **Pagination differs per service** — check before writing a list command:

| Service | Params | First page |
|---------|--------|------------|
| vks | `page`, `pageSize` | `0` |
| vserver | `page`, `size` | `1` |
| vdb | `pageNumber`, `pageSize` | `1` |
| vdb Kafka | — | **no pagination at all** |

- **vDB wraps every response** in `{code, message, data}`, with a second `data`
nested inside for paginated lists — except Kafka, which has no consistent
envelope. Use `vdbclient.Unwrap`; see `go/cmd/vdb/CLAUDE.md`.
- **`client.GreennodeClient` has `Request`/`RequestRaw`** for method+params+body in
one call, and for responses that must not be JSON-parsed. Both were added for vDB
Kafka, whose mutating endpoints take their arguments in the query string of a
body-less PUT and answer with an empty body. The verb helpers are still the right
choice everywhere else.

## Adding a new command

Expand All @@ -132,7 +184,25 @@ Static command/flag completion is automatic (cobra: `grn completion <shell>`). F
- API-backed: `cli.FlagFromAPI(func(ctx, cmd) ([]string,error))` — bounded timeout, fails silently
- Cross-service resource: consumer uses `cli.ResourceCompletion("<svc>:<resource>")`; the owning service registers a provider via `cli.RegisterResourceCompleter("<svc>:<resource>", ...)` (e.g. `internal/resources/vserver/`, blank-imported in `cmd/register.go`)

VKS wires its flags centrally in `cmd/vks/completion.go` `registerCompletions()`.
VKS wires its flags centrally in `cmd/vks/completion.go` `registerCompletions()`,
called from `vks.go`'s `init()`. That works because every direct subcommand of
`VksCmd` lives in a file sorted before `vks.go`, so its flags already exist —
`init()` functions run in filename order within a package. (`wait.go` sorts after,
but its flags are on *sub*subcommands, which `registerCompletions` does not walk;
`grn vks wait … --cluster-id` therefore has no value completion.)

**Prefer binding a flag to its completer in the `init()` of the file that defines
the flag** — that is what `cmd/vdb/**` does. A central binding that
runs too early fails inside `RegisterFlagCompletionFunc`, and the error is
discarded by the usual `//nolint:errcheck`, leaving completion that looks wired
but does nothing. Assert bindings with `cmd.GetFlagCompletionFunc` in a test.

Better still, assert the whole tree: `cmd/vdb/completion_test.go` walks every command
under `VdbCmd` and fails for any id/version/zone/package flag with no completer, with an
explicit exception list. It caught 11 missing bindings that per-file tests had not covered.
Note that a bound completer can still return nothing — ids that arrive as JSON numbers need
`vdbclient.ExtractIDValues`, since `cli.ExtractIDs` collects only strings — so confirm new
completers with `grn __complete …`.

## Adding a new service

Expand Down Expand Up @@ -236,6 +306,8 @@ Code without docs is not done.
| `cmd/root.go` | Root command, global flags, --version |
| `cmd/vks/helpers.go` | Client creation, output formatting, label/taint parsing |
| `internal/config/config.go` | Config loading from ~/.greennode/, REGIONS map |
| `internal/vdbclient/` | vDB client + error enrichment, envelope unwrap, 1-based list query, ID guard, action/resize bodies, security-rule parsing, dry-run preview |
| `go/cmd/vdb/CLAUDE.md` | vDB API quirks — read before touching `grn vdb` |
| `internal/config/writer.go` | INI file writer with 0600 perms |
| `internal/auth/token.go` | TokenManager — OAuth2 with IAM (camelCase) |
| `internal/client/client.go` | HTTP client with retry (3x backoff) + 401 refresh |
Expand Down
134 changes: 134 additions & 0 deletions docs/commands/vdb/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# vDB commands

`grn vdb` — manage GreenNode Database (vDB) resources.

```bash
grn vdb <group> <resource> <command> [options]
```

vDB covers four database products that share a gateway but not an API, so the
command tree keeps them apart:

| Group | Product | Status |
|-------|---------|--------|
| [relational](relational-instance.md) | Relational Database (MySQL, PostgreSQL, MariaDB) | Available |
| [postgresql](postgresql-cluster.md) | PostgreSQL Cluster (2–10 nodes) | Available |
| [memorystore](memorystore-instance.md) | MemoryStore (Redis) | Available |
| [kafka](kafka-cluster.md) | Kafka | Available |

## Commands

| Command | Description |
|---------|-------------|
| [relational instance](relational-instance.md) | Manage Relational Database instances |
| [relational catalog](relational-catalog.md) | Look up engines, flavors, zones, networks and volume types |
| [relational backup](relational-backup.md) | Manage backups and restore them into new instances |
| [relational configuration](relational-configuration.md) | Manage database config groups |
| [relational backup-storage](relational-backup-storage.md) | Buy and manage backup storage quota |
| [postgresql cluster](postgresql-cluster.md) | Manage PostgreSQL Clusters |
| [postgresql catalog](postgresql-catalog.md) | Look up cluster versions, flavors, volume types and backup options |
| [postgresql backup](postgresql-backup.md) | Manage PostgreSQL Cluster backups |
| [memorystore instance](memorystore-instance.md) | Manage MemoryStore (Redis) instances |
| [memorystore catalog](memorystore-catalog.md) | Look up Redis versions, flavors, networks and config groups |
| [memorystore backup](memorystore-backup.md) | Manage MemoryStore backups and restore them into new instances |
| [memorystore configuration](memorystore-configuration.md) | Manage Redis config groups |
| [memorystore backup-storage](memorystore-backup-storage.md) | Buy and manage backup storage quota |
| [kafka cluster](kafka-cluster.md) | Manage Kafka clusters |
| [kafka topic](kafka-topic.md) | Manage the topics of a cluster |
| [kafka user](kafka-user.md) | Manage users and their credentials |
| [kafka configuration](kafka-configuration.md) | Manage versioned Kafka config groups |
| [kafka catalog](kafka-catalog.md) | Look up broker flavors and volume types |

## Choosing between `relational` and `postgresql`

Both run PostgreSQL, but they are separate products with separate APIs:

- **`grn vdb relational`** — single instances (also MySQL and MariaDB). IDs start
with `db-`.
- **`grn vdb postgresql`** — clusters of 2–10 nodes, with their own flavors, volume
types and backup service. IDs start with `pg-`.

`grn vdb relational instance list` returns **both**, because the cluster product
has no listing of its own; `grn vdb postgresql cluster list` shows only the
clusters. Values are not interchangeable between the two: a cluster flavor
(`pgp-...`) is rejected by a relational create and vice versa.

## Region availability

vDB is currently available in **HCM-3 only**. Running a `grn vdb` command with
`--region HAN` fails with an endpoint-not-found error naming the service and
region.

## Global options

`--user-type` (string)
: Billing flow used by chargeable operations (create, resize, restore).
Possible values: `ROOT_USER` (Checkout flow), `IAM_USER` (Auto Payment flow).
When omitted, the API applies its own default of `ROOT_USER`.

All [global options](../../usage/global-options.md) apply as well, including
`--output`, `--query`, `--profile` and `--region`.

## MemoryStore vs Relational Database

MemoryStore has the same five nouns as Relational Database, and two product differences
that change what the commands offer:

- **No volume.** A MemoryStore instance's capacity is its flavor's RAM, so there are no
storage flags and no `resize-storage` — growing means a larger flavor.
- **A master password, not a database user.** Access is guarded by
`redisPasswordEnabled` + `redisPassword` on the instance, and public access requires
it. See [Redis authentication](memorystore-instance.md#redis-authentication).

Zones and subnets come from the **relational** catalog: MemoryStore has no endpoint for
either, and they are project-wide.

Instance IDs use the `db-` prefix in both products, so an ID alone does not tell you
which one it belongs to — the listings do, and they are separate.

### Paths that differ

Useful when porting a script or reading the API directly. Each row is the same
operation spelled two ways:

| Operation | relational | memorystore |
|---|---|---|
| get instance | `/database-instances/id/{id}` | `/database-instances/{id}` |
| update settings | `/{id}/update/setting` | `/{id}/update-setting` |
| update config group | `/{id}/update/config-group` | `/{id}/update-config-group` |
| catalog root | `/database-instances/*` | `/database/*` |
| flavor codes | `/database-instances/flavor_zones/codes` | `/database/codes` |
| volume types | `/database-instances/volume/types` | `/database/volume-types` |
| backup detail | `/backups/detail/{id}` | `/backups/{id}/detail` |
| backup delete | `DELETE /backups/{id}/delete` | `POST /backups/delete` (no ID in the path) |
| backups of an instance | `/backups/insId/{id}` | `/database-instances/{id}/backups` |
| config group detail | `GET /configurations/id?id=` | `GET /configurations/{id}/detail` |
| config group delete | `DELETE /configurations/delete` | `POST /configurations/delete` |
| **storage you own** | `/backup-storages/information` | **`/backup-storages`** |
| **storage you can buy** | `/backup-storages` | **`/backup-storages/packages`** |
| release storage | `/actions/deletions` | `/actions/delete` |

The last three invert: `GET /backup-storages` means "what I can buy" in one product and
"what I own" in the other.

## Kafka is not shaped like the other three

`grn vdb kafka` shares the gateway and almost nothing else. Do not carry habits from
the other groups over:

- **No backups, no replicas.** Kafka is the one vDB product with no backup service, so
deleting a cluster or a topic loses the data outright.
- **No pagination and no server-side filters** on any listing. `cluster list --name` and
`--status` are applied by the CLI to a complete response.
- **The nouns are different**: topics, users with per-topic permissions, and config
groups that are **versioned** — a cluster attaches to a config group *version*, and a
group is changed by adding a version, never edited in place.
- **Security rules are per rule**, added with `create-secrule` and removed with
`delete-secrule`, where relational and memorystore replace the whole set at once. There
is no endpoint that lists them; `cluster list-secrules` reads them out of the cluster.
- **Identifiers are opaque strings**: `clus-`, `topic-`, `user-`, `cgroup-`,
`cgroupver-`, and — for `--flavor-id` and `--volume-type` — `flav-` and `vtype-`.
Relational Database's numeric flavor id and named volume type have no equivalent here.
- **Broker count, volume size and volume type are three separate chargeable
operations** (`resize-brokers`, `resize-storage`, `update-volume-type`), not one
resize.
113 changes: 113 additions & 0 deletions docs/commands/vdb/kafka-catalog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# kafka catalog

Read-only lookups for what a Kafka cluster can be built from.

```bash
grn vdb kafka catalog <command> [options]
```

## Commands

| Command | Description |
|---------|-------------|
| [list-flavors](#list-flavors) | Broker flavors |
| [list-families](#list-families) | Instance families |
| [list-flavor-codes](#list-flavor-codes) | CPU platform codes |
| [list-volume-types](#list-volume-types) | Volume types |

Kafka's catalog is smaller than the other products'. It has **no** engine, datastore,
zone, network, subnet or config-group lookup:

- **Versions** come from shell completion on `--kafka-version` — see
[Kafka versions](#kafka-versions) below.
- **Networks and subnets** come from
[`grn vdb relational catalog list-networks`](relational-catalog.md) and
`list-subnets`; they are project-wide and Kafka has no endpoint for them.
- **Config groups** have their own noun,
[`kafka configuration`](kafka-configuration.md), because they are versioned.

None of these endpoints takes a `--zone-id`, unlike their relational and memorystore
counterparts.

---

## list-flavors

```
grn vdb kafka catalog list-flavors [--datastore-type kafka] [--datastore-version <value>]
```

Columns: `flavorId`, `id`, `name`, `vcpus`, `ram`, `priceKey`, `description`,
`locateZoneId`.

!!! important "`cluster create --flavor-id` takes `flavorId`, not `id`"
The value is the opaque `flav-…` string. The numeric `id` is the identifier
Relational Database's `--package-id` uses; passing it here is wrong.

`--datastore-type` filters, and `kafka` is the only value that matches anything
(`?type=bogus` returns none; `kafka` and `Kafka` both return everything — the case
inconsistency vDB has elsewhere too).

`--datastore-version` is accepted by the API and currently has **no effect**: the same
flavors come back for a real version and for a made-up one, and the rows carry no
version field to filter on client-side (verified live 2026-08-17). Completion for the
flag still offers the real versions.

---

## list-families

```
grn vdb kafka catalog list-families
```

Columns: `group`, `key`, `value`, `name`, `description`. In practice these are zone/
family records with `key` and `value` null.

---

## list-flavor-codes

```
grn vdb kafka catalog list-flavor-codes
```

The CPU platform codes flavors are grouped by (`code-a`, `code-s`, `code-s2`).
Columns: `key`, `value`, `familyType`, `description`.

---

## list-volume-types

```
grn vdb kafka catalog list-volume-types
```

Columns: `kafkaUuid`, `type`, `displayName`, `minVolumeSize`, `maxVolumeSize`, `iops`,
`zoneId`.

!!! important "`cluster create --volume-type` takes `kafkaUuid`"
The value is the opaque `vtype-…` string, **not** the display `type`
(`kafka.Gen2-NVMe2-IOPS3000`) and not the numeric `id`. This is the opposite of
Relational Database, where the volume type is passed by name.

This is the one Kafka catalog endpoint whose payload is not a bare array: the envelope's
`data` is `{projectId, data: [...]}`. The CLI strips that second layer, so the output is
the volume types themselves.

---

## Kafka versions

There is no command that lists Kafka versions, because there is no user-facing endpoint
that carries them. The only source is `GET /vdb-kafka/database/configs`, which returns
the service's own settings — regexes, per-user quotas, forced broker properties — and is
deliberately not exposed as a command.

The CLI reads that endpoint for one value, `kafkaVersions`, and uses it for shell
completion of `--kafka-version` and `--datastore-version`:

```bash
grn vdb kafka cluster create --kafka-version <TAB>
3.6.0 3.6.1 3.7.0
```
Loading
Loading