Skip to content

Kubernetes

Last updated

View as Markdown

Use the official Helm chart to run the collector in Kubernetes alongside PostgreSQL managed by CloudNativePG, Crunchy, Zalando, or a StatefulSet you wrote yourself.

CloudNativePG has a dedicated provider that tracks failover and scrapes every instance. Install the chart as described here, then make the changes on CloudNativePG. For a database on your laptop, use Local development with Docker. For RDS or Aurora, use AWS RDS and Aurora. That setup deploys a Fargate task and does not involve Kubernetes.

  • Helm 3.8 or newer. The chart is distributed as an OCI artifact, which older Helm versions cannot pull.
  • Kubernetes 1.25 or newer. amd64 and arm64 nodes both work.
  • Collector credentials. In the DBGorilla app, open Settings → Collectors → New Collector and choose Create credentials under Deploy with your infrastructure as code. You get an agent ID, a tenant ID and a secret. The secret is shown once and cannot be retrieved again.
  • A database login with the grants in Grants.
  • A namespace that meets the two constraints in Choosing a namespace.

Network requirements are listed in Choose your setup. The chart renders no Service and no inbound ports. What the chart leaves to your config is under The chart models nothing about your database.

Create the dbg_readonly login on each database server the collector watches, exactly as described in Prepare PostgreSQL.

USAGE and SELECT are per-database. The example below watches app and orders, so run those two grants twice, once connected to each database, and once per schema within them. If you run them in postgres only, the collector reports metrics normally while capturing nothing.

Two constraints determine where the collector runs.

A Secret cannot be mounted across namespaces. If you mount the database CA, the collector must run in the namespace that holds it. The alternative is to copy that Secret, and the copy drifts when the operator rotates its CA.

If your database namespace runs a default-deny NetworkPolicy, that policy must admit the collector. Many policies admit by namespace. Some admit only pods with a particular label.

Running the collector in the same namespace as the database meets both constraints. For a single cluster, this is the shortest path.

The example watches two databases on a cluster named app-pg in the databases namespace over TLS with the operator’s CA.

Set host to the read-write service. For a CloudNativePG cluster named app-pg, that is app-pg-rw. Do not use a read-only or round-robin service such as app-pg-ro or app-pg-r.

A standby breaks two things. pg_stat_replication is empty on a standby, so the collector discovers no cluster members. A node in recovery also has frozen statistics: replicas apply changes through WAL replay, which never moves a counter, so the collector takes its statistics from the primary.

The collector finds the replicas from the primary, so you do not list them.

This applies to type = "self_hosted". With the cnpg provider, the collector finds the primary through the Kubernetes API.

An operator usually signs PostgreSQL’s certificate with its own certificate authority. Mount that CA into the collector and point ca_cert at it:

extraVolumes:
- name: pg-ca
secret:
secretName: app-pg-ca # CloudNativePG names it <cluster>-ca
extraVolumeMounts:
- name: pg-ca
mountPath: /etc/dbg-collector-certs
readOnly: true
ssl_mode = "verify-full"
ca_cert = "/etc/dbg-collector-certs/ca.crt"

Without a CA bundle, use ssl_mode = "require". The connection is encrypted and unverified, which is reasonable inside a cluster you control. Then drop ca_cert and the two extra blocks from the files below.

Turn off node_exporter for containerized databases

Section titled “Turn off node_exporter for containerized databases”

A database in a pod has nothing listening on port 9100. In the default auto mode, the collector spends the full probe timeout on every node during each discovery pass. Turn it off unless your databases run on hosts with node_exporter:

[component.provider.node_exporter]
mode = "disabled"

collector.toml:

[dbgorilla]
agent_id = "<agent id from the app>"
tenant_id = "<tenant id from the app>"
secret = { env = "DBG_SERVER_SECRET" }
[topology]
interval = "60s"
[commands]
enabled = true
allowed = ["explain", "collect_statistics"] # see https://www.dbgorilla.com/docs/reference/collector-commands/#database-commands
[[component]]
name = "app-pg"
engine = "postgres"
[component.provider]
type = "self_hosted"
[component.provider.node_exporter]
mode = "disabled"
[component.auth]
method = "password"
user = "dbg_readonly"
password = { env = "APP_PG_PASSWORD" }
[component.connect]
host = "app-pg-rw.databases.svc.cluster.local"
port = 5432
ssl_mode = "verify-full"
ca_cert = "/etc/dbg-collector-certs/ca.crt"
databases = ["app", "orders"]

The [commands] block allows explain and collect_statistics. To turn either off, remove it from allowed or set enabled = false, then run helm upgrade with the edited file. See Database commands.

values.yaml:

secrets:
serverSecret: "<secret from the app>"
databasePasswords:
APP_PG_PASSWORD: "<the dbg_readonly password>"
extraVolumes:
- name: pg-ca
secret:
secretName: app-pg-ca
extraVolumeMounts:
- name: pg-ca
mountPath: /etc/dbg-collector-certs
readOnly: true

Run:

Terminal window
helm install dbg-collector oci://dbgorillapublic.azurecr.io/charts/dbg-collector \
--namespace databases \
--values ./values.yaml \
--set-file config.inline=./collector.toml

values.yaml holds live secrets. collector.toml does not, as long as it keeps the { env = … } references above instead of literal values. Keep values.yaml out of version control, or supply secrets.existingSecret and manage the Secret yourself, with an ExternalSecret, for instance.

An existing Secret must contain the DBG_SERVER_SECRET key and every database password key your TOML references. The chart injects only DBG_SERVER_SECRET on its own. Declare each password key as well, either as a name under secrets.databasePasswords with the value "" or as an extraEnv entry. A key that is in the Secret but not declared never reaches the collector.

The image comes from dbgorillapublic.azurecr.io, which allows anonymous pulls. No image pull secret is needed.

Add a [[component]] block for each database server. One collector can watch as many as it can reach, each with its own credentials and TLS settings:

[[component]]
name = "app-pg"
# …
[[component]]
name = "billing-pg"
# …

Give each component’s password its own environment variable name and list them all under secrets.databasePasswords.

Follow Check it worked for Kubernetes.

The helm upgrade command is under Upgrading and removing in Manage collectors.

helm uninstall and what it leaves behind are under Upgrading and removing in Manage collectors.

To manage this install from Terraform, see Deploying from Terraform.