# Kubernetes If your PostgreSQL runs in Kubernetes, under CloudNativePG, Crunchy, Zalando, or a StatefulSet you wrote yourself, run the collector there too, with the official Helm chart. For a database on your laptop see [Local development with Docker](./local-docker.md). For RDS or Aurora see [AWS RDS and Aurora](./aws-rds-aurora.md); the AWS path there deploys a Fargate task and does not involve Kubernetes. ## Before you start - **Helm 3.8 or newer.** The chart is distributed as an OCI artifact, which older Helm cannot pull. - **Kubernetes 1.25 or newer.** - **Collector credentials.** In the DBGorilla app, open **Collectors → New Collector**. It issues an agent ID, a tenant ID and a secret. The secret is shown once and cannot be retrieved again. - **A database login with the right grants**. See [Grants](#grants) below. - **A namespace.** Not an arbitrary choice; see [Choosing a namespace](#choosing-a-namespace). The collector opens **outbound connections only**. It needs your databases on 5432, and `otlp.dbgorilla.com` plus `auth.dbgorilla.com` on 443. The chart renders no Service and no inbound ports. ## The chart models nothing about your database You supply a complete `collector.toml` and the chart writes it to a ConfigMap verbatim, mounted at `/etc/dbg-collector/collector.toml`. The chart owns the image, the pod spec, the ServiceAccount and secret injection, and nothing else. Anything under `config:` other than `inline` is rejected at install rather than quietly ignored. :::danger[Never leave the secret inline in `config.inline`] `config.inline` renders into a **ConfigMap**, which is readable by anything that can `get configmap` in that namespace. The app can show you a `collector.toml` with the secret already filled in. That is convenient for the `docker run` path, wrong here. Before using it with this chart, replace the literal with an environment reference: ```toml secret = { env = "DBG_SERVER_SECRET" } ``` and pass the real value through `secrets.serverSecret`, which the chart puts in a Secret. The same applies to every database password: reference `{ env = "…" }` and supply the values under `secrets.databasePasswords`. ::: ## Grants Create the `dbg_readonly` login on **each** database server the collector watches, exactly as in [Collector Installation](./overview.md#database-grants). `USAGE` and `SELECT` are per-database. The example below watches `app` and `orders`, so those two grants have to be run twice, once connected to each, and once per schema within them. Run them in `postgres` only and the collector reports metrics normally while capturing nothing. :::caution[`pg_monitor` is what makes replicas visible] Without it, `pg_stat_replication.client_addr` reads back **NULL** for a non-superuser. The collector finds no cluster members, reports your three-node cluster as a single node, and logs nothing wrong, because from where it sits, nothing is wrong. If your topology shows one node where you expect several, check this grant before anything else. ::: ## Point at the primary Address the **read-write** service, not a read-only or round-robin one. For a CloudNativePG cluster named `app-pg`, that is `app-pg-rw`, not `app-pg-ro` or `app-pg-r`. Two things break otherwise. `pg_stat_replication` is empty on a standby, so the collector discovers no cluster members. And a node in recovery has frozen statistics. Replicas apply changes through WAL replay, which never moves a counter, so the collector will not use one as its source of truth. The collector finds the replicas itself, from the primary. You do not list them. ## TLS against an internal CA Operator-managed PostgreSQL almost always presents a certificate from the operator's own authority. Mount that CA into the collector and point `ca_cert` at it: ```yaml extraVolumes: - name: pg-ca secret: secretName: app-pg-ca # CloudNativePG names it -ca extraVolumeMounts: - name: pg-ca mountPath: /etc/dbg-collector-certs readOnly: true ``` ```toml ssl_mode = "verify-full" ca_cert = "/etc/dbg-collector-certs/ca.crt" ``` :::caution[Do not mount under `/etc/dbg-collector`] That path is the ConfigMap volume, mounted read-only. Kubernetes cannot create a nested mountpoint inside it, so a CA mounted at `/etc/dbg-collector/certs` fails to start the pod. Use a sibling path. ::: If you would rather not manage a CA bundle, `ssl_mode = "require"` is the documented fallback: encrypted, unverified, and reasonable inside a cluster you control. ## Choosing a namespace Two constraints usually decide this for you, and both are easier to satisfy than to work around. **A Secret cannot be mounted across namespaces.** If you mount the database CA, the collector must run in the namespace holding it, or you copy that Secret, which then drifts when the operator rotates its CA. **NetworkPolicy.** If your database namespace runs a default-deny policy, the collector must be admitted by it. Many policies admit by namespace; some admit only pods carrying a particular label. :::note[The chart cannot set pod labels] It exposes `podAnnotations` but not `podLabels`, so a NetworkPolicy that admits traffic by **pod label** cannot be satisfied by the collector. Use a rule that admits by **namespace**, and run the collector in a namespace that rule already allows. ::: Running the collector in the same namespace as the database satisfies both constraints at once, and is the shortest path for a single cluster. ## Turn off `node_exporter` for containerized databases A database in a pod has nothing listening on port 9100, and the default `auto` mode spends the full probe timeout on every node on every discovery pass. Turn it off unless your databases run on hosts you have put `node_exporter` on: ```toml [component.provider.node_exporter] mode = "disabled" ``` ## Install `collector.toml`: ```toml [dbgorilla] agent_id = "" tenant_id = "" secret = { env = "DBG_SERVER_SECRET" } [topology] interval = "60s" [commands] enabled = false [[component]] name = "app-pg" engine = "postgres" [component.provider] type = "self_hosted" [component.provider.node_exporter] mode = "disabled" [component.auth] method = "password" user = "dbg_readonly" password = { env = "APP_PG_PASSWORD" } [component.connect] host = "app-pg-rw.databases.svc.cluster.local" port = 5432 ssl_mode = "verify-full" ca_cert = "/etc/dbg-collector-certs/ca.crt" databases = ["app", "orders"] ``` `values.yaml`: ```yaml secrets: serverSecret: "" databasePasswords: APP_PG_PASSWORD: "" extraVolumes: - name: pg-ca secret: secretName: app-pg-ca extraVolumeMounts: - name: pg-ca mountPath: /etc/dbg-collector-certs readOnly: true ``` Then: ```bash helm install dbg-collector oci://dbgorillapublic.azurecr.io/charts/dbg-collector \ --namespace databases \ --values ./values.yaml \ --set-file config.inline=./collector.toml ``` `values.yaml` holds live secrets. `collector.toml` does not, as long as you kept the `{ env = … }` references above rather than pasting literals into it. Keep `values.yaml` out of version control, or supply `secrets.existingSecret` and manage the Secret yourself, with an ExternalSecret, for instance. If you do, it must contain the `DBG_SERVER_SECRET` key plus every database password key your TOML references; the chart cannot read your config, so it cannot derive those names for you. The image comes from `dbgorillapublic.azurecr.io`, which allows anonymous pulls. No image pull secret is needed. ## Check it worked ```bash kubectl -n databases logs -f deploy/dbg-collector ``` Four lines say it is healthy: ```text opamp websocket connected opamp handshake complete otelcol started discovered component ``` Then look at what it found. For a three-node cluster you want three nodes: one writer and two readers. One node where you expect three points at the [`pg_monitor` grant](#grants). This warning means one node cannot be reached: ```text no instance baseline and a node is unreachable; deferring topology post until all nodes are probed ``` The collector holds back the whole component until every node answers, so a single unreachable replica means **no data at all**, including from the primary. Usual causes are a NetworkPolicy that admits the collector to the primary's service but not to individual pods, or a `ca_cert` path that does not exist in the container. ## Several databases, one collector Add a `[[component]]` block per database server. One collector can watch as many as it can reach, each with its own credentials and TLS settings: ```toml [[component]] name = "app-pg" # … [[component]] name = "billing-pg" # … ``` Give each component's password its own environment variable name and list them all under `secrets.databasePasswords`. ## Upgrading and removing ```bash helm upgrade dbg-collector oci://dbgorillapublic.azurecr.io/charts/dbg-collector \ --namespace databases \ --values ./values.yaml \ --set-file config.inline=./collector.toml helm uninstall dbg-collector --namespace databases ``` `helm uninstall` removes the workload but does **not** deprovision the collector's identity in DBGorilla. To retire a collector permanently, delete it in the app as well, and note that a new install always means a new collector, never the old one resumed.