# Apache Cassandra

Monitor Apache Cassandra 4.0, 4.1 or 5.0 that you run yourself.  You create one
read-only role, write `collector.toml` by hand, and run the collector in a Docker container that
can reach every node.  The collector installs nothing on the nodes.

NetApp Instaclustr managed Cassandra is not covered here.

## Before you start

- Check the version on any node.  The collector supports 4.0, 4.1 and 5.0.

  ```sql
  SELECT release_version FROM system.local;
  ```

  Cassandra 3.11 is refused during discovery, and nothing is sent:

  ```text
  Cassandra 3.11.19 is not supported: the collector needs 4.0 or later (3.x has no virtual tables and reached end of life with 5.0)
  ```

- Turn on password authentication and authorization in `cassandra.yaml`, if they are not already
  enabled.  The grants below require the authorizer.  Changing either setting requires a restart of
  each node.

  ```yaml
  authenticator: PasswordAuthenticator
  authorizer: CassandraAuthorizer
  ```

- Let the collector reach port 9042 on every node at the address that each node advertises to
  clients.  You give the collector one node.  It finds the others and connects to each of them
  directly.
- Let the collector reach `otlp.dbgorilla.com` and `auth.dbgorilla.com` on port 443.  See
  [What the collector is](/docs/connect/#what-the-collector-is) in Choose your setup.
- Install Docker on the machine that will run the collector.
- Add a collector in the app, and keep the agent ID, tenant ID and secret it shows.  The secret is
  shown only once.

## Prepare the cluster

Create the role as a superuser on any node.  Roles and grants apply to the whole cluster.

```sql
CREATE ROLE dbg_collector WITH LOGIN = true AND PASSWORD = '<strong password>';
GRANT SELECT ON KEYSPACE system_views TO dbg_collector;
GRANT SELECT ON TABLE system.prepared_statements TO dbg_collector;
GRANT DESCRIBE ON ALL ROLES TO dbg_collector;
```

None of these grants read your tables.  `system_views` contains node statistics, and
`system.prepared_statements` contains the text of the statements that your applications prepare.

| Grant | What it adds | Without it |
|---|---|---|
| `SELECT ON KEYSPACE system_views` | Node metrics, the workload, in-flight requests and node settings | Only the keyspaces, tables and node uptime arrive.  The collector log names the missing grant: `User dbg_collector has no SELECT permission on <table system_views.thread_pools>`. |
| `SELECT ON TABLE system.prepared_statements` | The statement text behind each workload entry | The workload lists each table operation, such as `READ shop.customers`, without its statements. |
| `DESCRIBE ON ALL ROLES` | Every role in the topology | The topology lists only `dbg_collector`. |

With the login alone, the collector still reads the schema: keyspaces, tables, columns,
secondary indexes and user-defined types.

### If the keyspace grant fails

Some older patch releases refuse the keyspace grant, including 5.0.4:

```text
Resource <keyspace system_views> doesn't exist
```

Grant each table instead:

```sql
GRANT SELECT ON TABLE system_views.thread_pools TO dbg_collector;
GRANT SELECT ON TABLE system_views.caches TO dbg_collector;
GRANT SELECT ON TABLE system_views.internode_inbound TO dbg_collector;
GRANT SELECT ON TABLE system_views.internode_outbound TO dbg_collector;
GRANT SELECT ON TABLE system_views.clients TO dbg_collector;
GRANT SELECT ON TABLE system_views.sstable_tasks TO dbg_collector;
GRANT SELECT ON TABLE system_views.pending_hints TO dbg_collector;
GRANT SELECT ON TABLE system_views.cql_metrics TO dbg_collector;
GRANT SELECT ON TABLE system_views.queries TO dbg_collector;
GRANT SELECT ON TABLE system_views.coordinator_read_latency TO dbg_collector;
GRANT SELECT ON TABLE system_views.coordinator_write_latency TO dbg_collector;
GRANT SELECT ON TABLE system_views.coordinator_scan_latency TO dbg_collector;
GRANT SELECT ON TABLE system_views.local_read_latency TO dbg_collector;
GRANT SELECT ON TABLE system_views.local_write_latency TO dbg_collector;
GRANT SELECT ON TABLE system_views.local_scan_latency TO dbg_collector;
GRANT SELECT ON TABLE system_views.disk_usage TO dbg_collector;
GRANT SELECT ON TABLE system_views.max_partition_size TO dbg_collector;
GRANT SELECT ON TABLE system_views.tombstones_per_read TO dbg_collector;
GRANT SELECT ON TABLE system_views.settings TO dbg_collector;
GRANT SELECT ON TABLE system_views.gossip_info TO dbg_collector;
```

## Install

Write `collector.toml`.  `host` can be any one node.

```toml
[dbgorilla]
agent_id  = "<agent id from the app>"
tenant_id = "<tenant id from the app>"
secret    = "${DBG_SERVER_SECRET}"

[[component]]
name   = "orders-ring"
engine = "cassandra"

[component.provider]
type = "self_hosted"

[component.auth]
method   = "password"
user     = "dbg_collector"
password = { env = "CASSANDRA_COLLECTOR_PASSWORD" }

[component.connect]
host     = "cass-1.internal.example.com"
port     = 9042
ssl_mode = "disable"
```

Keep the secrets in a separate file with mode 0600, and keep that file out of version control:

```ini
# secrets.env
DBG_SERVER_SECRET=your-client-secret
CASSANDRA_COLLECTOR_PASSWORD=your-role-password
```

Run the collector from the directory that contains both files:

```sh
docker run -d --name dbg-collector --restart unless-stopped \
  -v "$PWD/collector.toml:/etc/dbg-collector/collector.toml:ro" \
  --env-file secrets.env \
  dbgorillapublic.azurecr.io/dbg-collector:0.16.0 \
  --config-file /etc/dbg-collector/collector.toml
```

Cassandra requires collector 0.16.0 or later.  Pin the release tag as shown here so that a restart
cannot change the running version.

### Client encryption

If the cluster has `client_encryption_options` enabled, set `ssl_mode` and mount the CA that signed
the node certificates:

```toml
[component.connect]
host     = "cass-1.internal.example.com"
port     = 9042
ssl_mode = "verify_full"
ca_cert  = "/etc/dbgorilla/cassandra-ca.pem"
```

```sh
docker run -d --name dbg-collector --restart unless-stopped \
  -v "$PWD/collector.toml:/etc/dbg-collector/collector.toml:ro" \
  -v "$PWD/cassandra-ca.pem:/etc/dbgorilla/cassandra-ca.pem:ro" \
  --env-file secrets.env \
  dbgorillapublic.azurecr.io/dbg-collector:0.16.0 \
  --config-file /etc/dbg-collector/collector.toml
```

| `ssl_mode` | What it checks |
|---|---|
| `disable` | Nothing.  The default.  Use it only when client encryption is off. |
| `require` | Encrypts, but does not check the certificate. |
| `verify_ca` | The certificate chains to `ca_cert`.  The name in it is not checked.  `ca_cert` is required. |
| `verify_full` | The chain, and that the certificate names the address the collector connects to.  Without `ca_cert`, the image's own trust store is used. |

The collector connects to each node at the address that the node advertises, usually an IP
address.  With `verify_full`, every node's certificate needs that address as a subject alternative
name.  If the certificates contain only DNS names, use `verify_ca`.  Otherwise, the seed connects
and every node then fails with:

```text
x509: cannot validate certificate for 10.250.50.10 because it doesn't contain any IP SANs
```

If the cluster sets `require_client_auth: true`, add a client certificate and its key, and mount
them in the same way:

```toml
client_cert = "/etc/dbgorilla/collector.pem"
client_key  = "/etc/dbgorilla/collector.key"
```

### Commands

Cassandra takes one command, `execute_query`: a single CQL `SELECT` with at most 1,000 rows and no
`ALLOW FILTERING`.  It reads only the keyspaces granted to the role.  To allow it, add this to
`collector.toml`:

```toml
[commands]
enabled = true
allowed = ["execute_query"]
```

Then grant `SELECT` on each keyspace it may read:

```sql
GRANT SELECT ON KEYSPACE shop TO dbg_collector;
```

`explain` and `collect_statistics` do not apply to Cassandra.  If they are listed, the collector
leaves them out.

### Host metrics

For CPU, memory and disk metrics, run Prometheus `node_exporter` on each node on its default port
9100.  The collector finds it there.  See [node_exporter](/docs/connect/self-managed/#node_exporter)
in Self-managed server.

## Check it is working

Read the collector log:

```sh
docker logs dbg-collector 2>&1 | grep -E 'opamp handshake complete|discovered component|snapshot serialized|WARN|ERROR'
```

A healthy start has these three messages:

```text
opamp handshake complete
discovered component
topology snapshot serialized for OTLP post
```

The first `discovered component` line carries `"engine":"cassandra"` and a `"nodes"` count.  The
count is the number of nodes in the cluster.

Then open Home in DBGorilla.  The cluster appears under the name you gave the component, marked
**Monitored**, with its engine and version, such as `Cassandra 5.0.9`.  Its keyspaces and tables
arrive within a minute of startup, and node metrics arrive within two.  The workload arrives within
a minute of the cluster taking traffic.  Each workload entry is a table operation, such as
`READ shop.customers`, with its call count, mean time and the statements behind it.

In-flight requests and the workload's times require Cassandra 4.1.5 or later.  On 4.0, each
workload entry has its call count and statements, and node metrics arrive as usual.

If nothing arrives, the log names the cause:

| Log line | Fix |
|---|---|
| `Cassandra 3.11.19 is not supported` | Upgrade to 4.0 or later. |
| `unsupported combination (engine=cassandra, provider=self_hosted, auth=password)` | The collector is older than 0.16.0.  Use 0.16.0 or later. |
| `bad credentials (0x0100): Provided username dbg_collector and/or password are incorrect` | Check the role name and `CASSANDRA_COLLECTOR_PASSWORD`. |
| `no SELECT permission on <table system_views.thread_pools>` | Run the grants in [Prepare the cluster](#prepare-the-cluster). |
| `read the response header: Connection reset by peer` | Client encryption is on.  Set `ssl_mode`. |
| `invalid peer certificate: UnknownIssuer` | Set `ca_cert` to the CA that signed the node certificates. |
| `x509: cannot validate certificate for` (an IP address) | Use `verify_ca`, or reissue the node certificates with their IP addresses. |
| `received fatal alert: CertificateRequired` | The cluster requires client certificates.  Set `client_cert` and `client_key`. |

## Upgrade

Change the tag in the `docker run` command to the new release, then replace the container:

```sh
docker pull dbgorillapublic.azurecr.io/dbg-collector:<new version>
docker rm -f dbg-collector
# run the docker run command again, with the new tag
```

The agent ID and secret stay the same, so it reconnects as the same collector.

## Remove

```sh
docker rm -f dbg-collector
```

The cluster stays on Home, marked **Not monitored**, with its history, until you delete it.  To
delete it, follow [Removing the database from DBGorilla](/docs/administer/manage-collectors/#removing-the-database-from-dbgorilla)
in Manage collectors.

Then drop the role on any node:

```sql
DROP ROLE dbg_collector;
```

## Next

- [Topology](/docs/using-dbgorilla/topology/) shows what the collector found.
- [Organization Management](/docs/administer/members-and-roles/) lets you bring in your team.