Skip to content

CloudNativePG

Last updated

View as Markdown

If your PostgreSQL runs under the CloudNativePG operator, the collector has a provider written for it. Set type = "cnpg", name the Cluster, and the collector asks Kubernetes which pod is the primary, which pods are standbys, and where their metrics ports are. It keeps that answer current, so a failover changes nothing in your config and loses no history.

This page covers what is different about CloudNativePG. Everything about the chart itself, the namespace, secrets and the install command is on the Kubernetes page and applies here unchanged.

What you get over the generic Kubernetes path

Section titled “What you get over the generic Kubernetes path”

The Kubernetes page also works against a CloudNativePG cluster, with type = "self_hosted" pointed at the -rw service. The cnpg provider does three things that path cannot.

It survives failover without noticing. The component’s identity is the namespace and Cluster name, so when CloudNativePG promotes a standby the collector picks up the new primary on its next discovery pass and history continues under the same component.

It scrapes every instance rather than only the primary. Each PostgreSQL pod exposes metrics on port 9187 through the operator’s instance manager, and the collector reads all of them, so per-standby replication lag arrives as its own series.

It knows what is not an instance. PgBouncer pods created by a Pooler carry the same cnpg.io/cluster label as the database pods. The provider filters on cnpg.io/podRole=instance, so they are left out instead of being scraped forever with nothing to show.

Everything on the Kubernetes page, plus:

  • The collector runs inside the same Kubernetes cluster as the CloudNativePG Cluster. It reaches the Kubernetes API with its own ServiceAccount token and dials pod IPs directly for the metrics scrape. Neither works from outside.
  • A read-only Role in the namespace that holds the Cluster. The chart creates it when you ask. See Kubernetes read access.
  • pg_stat_statements loaded through the Cluster spec. See pg_stat_statements.
  • A database login with the right grants. The same dbg_readonly role as everywhere else. See Grants.

The provider reads three kinds of object: the Cluster itself, to learn the current primary and whether monitoring is served over TLS; the Cluster’s pods, to find instance addresses; and pod metrics. It gets three verbs, get, list and watch. No writes, no exec, and no access to Secrets. Nothing the collector is granted can change a database or a Kubernetes object.

The chart renders this when rbac.create is on. The only decision is where to bind it:

rbac:
create: true
scope: namespaced
namespaces:
- prod-db # the namespace that HOLDS the Cluster

scope: cluster binds the same permissions across every namespace. Use it only when you really do discover Clusters across the whole cluster.

rbac.nodeMetrics is on by default and is separate from scope. The collector reads each database pod’s CPU, memory and volume usage from the kubelet on the node that runs it, and the objects that authorise that read, nodes, nodes/stats and nodes/pods, are cluster scoped. So this one is always a ClusterRoleBinding, whatever scope says. It reads node statistics and grants nothing on any database object.

Set it to false and you lose exactly those three metrics for your database pods. The collector checks the grant at startup and skips the pipeline when it is missing; nothing else is affected.

Mode What it does Use it when
auto (default) Tries the API. Full mode if it can read, metrics-only if it cannot. You want an install that never fails on permissions
enabled Requires the API. Fails loudly if the Role is missing. You want to know the moment the Role is wrong
disabled Never calls the API. No Kubernetes permissions needed. You cannot grant read access at all

auto makes installs painless and problems quiet: a missing binding looks like a healthy collector with fewer numbers. If you have granted the Role and want proof it works, set mode = "enabled" and read the logs.

You do not run ALTER SYSTEM on an operator-managed cluster. Declare the library in the Cluster spec and CloudNativePG restarts the instances for you:

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: app-db
namespace: prod-db
spec:
postgresql:
shared_preload_libraries:
- pg_stat_statements
parameters:
shared_buffers: 256MB

Then, connected to each database you will list under databases:

CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

Adding a library to shared_preload_libraries triggers a rolling restart of the instances. Plan it the way you would plan any restart; it is the only disruptive step on this page.

The provider adds no SQL requirement of its own. Create dbg_readonly exactly as in Collector Installation, connected to the primary through the -rw service, and repeat the per-database USAGE and SELECT grants in every database you list.

One difference from the generic Kubernetes page is worth knowing. There, a missing pg_monitor grant collapses a three-node cluster to one node, because members are discovered through pg_stat_replication. Here, members come from the Kubernetes API, so the topology stays correct without it. pg_monitor is still required: without it the collector cannot read pg_stat_activity or pg_stat_statements, and every query-level feature goes dark.

SQL goes to the -rw service and nowhere else. Each standby is scraped on its metrics port and is never given a SQL connection.

The reason is the certificate. CloudNativePG issues one server certificate per Cluster, shared by every instance, whose names cover the -rw, -ro and -r services and nothing else: no pod names, no IP addresses. The collector verifies certificates by default, so a SQL connection to a pod can never verify. A metrics scrape can dial a pod IP while checking the certificate against the -rw name, so that is how per-instance data is read.

In practice: the primary is the SQL source and one metrics target; each standby is a metrics target only. In the app, a standby shows as a member you cannot run a query against, which is correct.

Point host at the short service name when the collector runs in the same namespace, or the fully qualified one otherwise. Both are on the certificate:

host = "app-db-rw" # same namespace
host = "app-db-rw.prod-db.svc.cluster.local" # anywhere in the cluster

Metrics on 9187 are plain HTTP unless the Cluster sets .spec.monitoring.tls.enabled. When it does, the collector switches the scrape to TLS and verifies against the Cluster CA on its own. You do not configure that.

collector.toml, one [[component]] per Cluster:

[dbgorilla]
agent_id = "<agent id from the app>"
tenant_id = "<tenant id from the app>"
secret = { env = "DBG_SERVER_SECRET" }
[topology]
interval = "60s"
[commands]
enabled = false
[[component]]
name = "prod-cnpg"
engine = "postgres"
[component.provider]
type = "cnpg"
namespace = "prod-db" # required
cluster = "app-db" # required
[component.provider.kubernetes]
mode = "auto"
[component.provider.metrics]
port = 9187
[component.auth]
method = "password"
user = "dbg_readonly"
password = { env = "APP_DB_PASSWORD" }
[component.connect]
host = "app-db-rw"
port = 5432
ssl_mode = "verify-full"
databases = ["app"]

values.yaml:

rbac:
create: true
scope: namespaced
namespaces:
- prod-db
secrets:
serverSecret: "<secret from the app>"
databasePasswords:
APP_DB_PASSWORD: "<the dbg_readonly password>"

Then:

Terminal window
helm install dbg-collector oci://dbgorillapublic.azurecr.io/charts/dbg-collector \
--namespace prod-db \
--values ./values.yaml \
--set-file config.inline=./collector.toml

Two things you would expect from the Kubernetes page are missing on purpose. There is no ca_cert and no mounted CA: SQL goes to the -rw name on the Cluster certificate, and the metrics scrape carries its own trust. And there is no node_exporter block: the cnpg provider has none, because there is no host to probe.

Terminal window
kubectl -n prod-db logs -f deploy/dbg-collector

The same four lines as on the Kubernetes page say it is healthy: opamp websocket connected, opamp handshake complete, otelcol started, discovered component.

Then confirm full mode actually happened. A three-instance Cluster should show three members, one writer and two readers, each with its own metrics. Three members but standby metrics missing means the pod IPs are not reachable on 9187, usually a NetworkPolicy that admits the service and not the pods. One member, or a Cluster that shows no primary, means discovery fell back to metrics-only: check rbac.namespaces first, then set mode = "enabled" to make the failure loud.

Add a [[component]] block per Cluster, each with its own namespace and cluster, and list every namespace under rbac.namespaces. One stanza is always one Cluster. The provider does not discover across a namespace, by design, so that a component’s identity is settled before discovery runs.

Same as the Kubernetes page. helm uninstall removes the Role and the binding with the rest of the release.