# My pg_wal directory filled the disk, what is safe to delete?

> **Quick answer:** Nothing in `pg_wal` is safe to delete by hand.  Removing WAL
> files can leave the database unable to recover and can permanently break replicas
> and backups.  WAL piles up because something is *pinning* it, usually an inactive
> replication slot, a failing `archive_command`, or `wal_keep_size` set too high.
> Fix the cause and PostgreSQL recycles the files itself at the next checkpoint.

## Rule zero: do not `rm` anything in pg_wal

Say it out loud before you touch the shell.

The write-ahead log is not a cache and it is not a log file in the syslog sense.  It
is the durability mechanism.  On crash recovery PostgreSQL replays WAL from the last
checkpoint forward; if a segment it needs is missing, the cluster **will not start**,
and the remediation is restoring from backup.  Delete the wrong file and you have
turned a full disk into data loss.

`pg_wal` is also not a directory that "grows out of control" on its own.  Under
normal operation PostgreSQL removes or recycles segments at every checkpoint.  If
files are accumulating, some component has told the server *"do not remove those
yet."* Your job in this incident is to find out which one and revoke the hold, not
to fight the filesystem.

(On PostgreSQL 9.6 and earlier this directory is named `pg_xlog`. It was renamed
specifically because too many people assumed a directory called "log" was safe to
clear out.)

## Step 1: Confirm pg_wal is actually what filled the disk

Do not assume.  Bloated tables, runaway temp files in `base/pgsql_tmp`, and unrotated
server logs fill disks too.

```bash
df -h /var/lib/postgresql          # adjust to your data directory mount
du -sh /var/lib/postgresql/*/main/*  | sort -h | tail -20
```

From inside the database, without shell access:

```sql
SELECT pg_size_pretty(sum(size)) AS pg_wal_total,
       count(*)                  AS segment_count
FROM pg_ls_waldir();
```

Default segment size is 16 MB, so segment count × 16 MB should match.  A steady-state
cluster typically holds somewhere around `max_wal_size` worth plus a margin:

```sql
SHOW max_wal_size;      -- soft target for WAL between checkpoints
SHOW min_wal_size;
SHOW wal_keep_size;     -- extra segments held for replicas without slots
SHOW archive_mode;
SHOW archive_command;
SHOW archive_library;   -- if set, archive_command is IGNORED
```

Check `archive_library` before you spend any time on `archive_command`. The two
are mutually exclusive: when `archive_library` is set to a non-empty value that
module does the archiving and `archive_command` is ignored entirely.  Debugging a
shell command the server never runs is a classic way to lose twenty minutes at
3am.  If `archive_library` is set, the failure and its logging belong to that
module, check its own diagnostics, not `archive_command`.

If total `pg_wal` size is dramatically larger than `max_wal_size`, something is
pinning WAL. Move to step 2.

Also check the archive backlog directly.  This is a fast tell:

```sql
SELECT count(*) FROM pg_ls_dir('pg_wal/archive_status') WHERE pg_ls_dir ~ '\.ready$';
```

(`pg_ls_dir` and `pg_ls_waldir` need superuser, or membership in `pg_monitor`
or `pg_read_server_files`. The unaliased `pg_ls_dir` in the `WHERE` is the
function's own output column, valid, though aliasing reads better.)

Thousands of `.ready` files means archiving is failing or far behind, and every one
of those segments is being retained until it succeeds.

## Step 2: Find what is pinning the WAL

There are three common holds.  Check all three.  They can coexist.

### 2a.  inactive replication slots

This is the number one cause.

```sql
SELECT
    slot_name,
    slot_type,
    database,
    active,
    active_pid,
    restart_lsn,
    pg_size_pretty(
        pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)
    ) AS wal_retained
FROM pg_replication_slots
ORDER BY pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) DESC NULLS LAST;
```

| What you see | What it means |
|---|---|
| `active = false`, `wal_retained` in the tens of GB | A dead consumer is holding your disk.  This is your culprit |
| `active = true`, `wal_retained` large and growing | A live but badly lagging consumer, a replica that cannot keep up, or a stalled CDC pipeline |
| `wal_retained` small on every slot | Slots are not the problem; check archiving |

On versions that expose them, `wal_status` and `safe_wal_size` are more direct:

```sql
SELECT slot_name, active, wal_status, pg_size_pretty(safe_wal_size) AS safe_wal_size
FROM pg_replication_slots;
```

`wal_status` progresses `reserved` → `extended` → `unreserved` → `lost`. Anything
past `reserved` means the slot is beyond the normal retention limit; `lost` means
required WAL has already been removed and that consumer **cannot** resume, it needs
rebuilding regardless of what you do next.

**Why does a slot pin WAL at all?** A replication slot is a durable promise.  The
consumer, a physical standby, or a logical decoding client like Debezium, tells
the primary "I have processed up to LSN X." The primary must then retain every WAL
segment from that point onward, because the consumer is allowed to disconnect and
reconnect later and resume exactly where it left off with no gap.  That guarantee is
the entire point of slots; it is what makes replication safe against a replica
rebooting.  The cost is that a consumer which *never* comes back never advances its
confirmed position, so the promise is never released, and WAL accumulates forever
unless you have configured a ceiling.

### 2b.  archiving is failing

If `archive_mode = on`, PostgreSQL will not remove a WAL segment until it has
been successfully archived.  Whatever is doing that archiving, a shell
`archive_command`, or an archive module named by `archive_library`, a failure
means WAL retention grows without bound.  Typical causes: expired cloud
credentials, a full archive destination, a mistyped path, an unmounted NFS share.

The `pg_stat_archiver` view below works either way.  Only the remediation differs:
if `archive_library` is set, `archive_command` is ignored, so fix the module's
configuration rather than the shell command.

```sql
SELECT archived_count,
       last_archived_wal,
       last_archived_time,
       failed_count,
       last_failed_wal,
       last_failed_time,
       stats_reset
FROM pg_stat_archiver;
```

| Signal | Diagnosis |
|---|---|
| `failed_count` high, `last_failed_time` recent | Archiving is broken right now.  This is your cause |
| `last_archived_time` hours old while writes continue | Archiving is stalled or extremely slow |
| `last_failed_wal` equals a segment that still exists | It is retrying the same segment and failing |

Then look at the server log for the actual error text.  `archive_command failed with
exit code N` lines tell you whether it is permissions, network, or disk.

### 2c. wal_keep_size held too high

```sql
SHOW wal_keep_size;   -- named wal_keep_segments on older versions
```

This is a flat floor of WAL retained **at all times**, independently of `max_wal_size`, and regardless of whether any replica exists or uses a slot.  Its purpose is to let a slotless replica catch up, but the retention itself is
unconditional, so do not rule it out just because all your replicas use slots.

If someone set it to a large value to paper over a replica that kept falling
behind, that value is now a permanent tax on your disk.  It is the least common of
the three but the easiest to miss, because it looks like a deliberate setting.

### 2d.  less common: checkpoints not completing

If checkpoints are stalled, WAL since the last checkpoint cannot be released.

{/* sql-check: pg 17+ */}
```sql
-- PostgreSQL 17+: checkpoint counters moved out of pg_stat_bgwriter into
-- their own view. num_done was added in PostgreSQL 18, omit it on 17.
SELECT num_timed, num_requested, write_time, sync_time, stats_reset
FROM pg_stat_checkpointer;
```

{/* sql-check: pg -16 */}
```sql
-- PostgreSQL 16 and earlier: the same counters live in pg_stat_bgwriter
SELECT checkpoints_timed, checkpoints_req, checkpoint_write_time,
       checkpoint_sync_time, stats_reset
FROM pg_stat_bgwriter;
```

```sql
SELECT pg_current_wal_lsn(), pg_walfile_name(pg_current_wal_lsn());
```

A cluster with very large `max_wal_size` and a heavy write burst can also legitimately
hold a lot of WAL without anything being wrong.  It is just sized bigger than the
volume it lives on.  That is a capacity mistake, not an incident cause, but it is worth
ruling in before you go hunting.

## Step 3: The safe remediation for each cause

### Dead replication slot

If you have confirmed the consumer is gone permanently:

```sql
-- DANGEROUS for the consumer. Safe for the database.
SELECT pg_drop_replication_slot('old_standby_slot');
```

**What this breaks, explicitly:**

- A **physical standby** using that slot loses its retention guarantee.  If it comes
  back and the WAL it needs has since been recycled, it cannot catch up and must be
  re-seeded from a fresh base backup (or `pg_rewind`, if applicable).
- A **logical replication / CDC consumer** loses its decoding position entirely.
  Slots are the only record of where a logical consumer was.  Dropping it means that
  pipeline must be re-snapshotted, for a large table that can be hours, and
  downstream systems will see a gap or a full reload.

You cannot drop a slot while it is `active`; you must stop the consumer first.  Do
**not** reflexively terminate the consumer's backend to force the drop, if it is a
live replica that is simply lagging, killing it makes the problem worse.  Confirm the
consumer is truly abandoned first.

A middle path for a logical slot you want to *keep* but need to unpin:

:::danger[Do not copy the LSN below]
`0/A0000000` is a **placeholder**, not a value to run.  The target LSN is the
decision, every change between the slot's current `restart_lsn` and wherever you
advance it is discarded, permanently, with no way to replay it.  Pasting a
stranger's LSN at 3am is how you silently drop a day of CDC events.

Read the slot's current position first, and choose a target deliberately:

```sql
SELECT slot_name, restart_lsn, confirmed_flush_lsn,
       pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS retained
FROM pg_replication_slots WHERE slot_name = 'cdc_slot';
```
:::

```sql
-- DANGEROUS: skips ahead; the consumer will never see the skipped changes.
-- Replace the LSN with one you chose from the query above.
SELECT pg_replication_slot_advance('cdc_slot', '<target_lsn>'::pg_lsn);
```

This releases the retained WAL without dropping the slot, at the cost of a data gap
for that consumer.  Only do this if you can reconcile downstream, for an analytics
sink you can backfill, for a system of record you usually cannot.

Either way, WAL is not freed the instant you run the command.  A checkpoint has to
happen.  You can force one:

```sql
CHECKPOINT;   -- safe; causes an I/O burst, so expect a latency bump
```

### Broken archive_command

The fix is to make archiving succeed.  If the destination is reachable and it is a
credentials or permissions problem, fix that and archiving drains on its own.  WAL
frees at the following checkpoint.

If you cannot fix the destination fast enough and the disk is about to fill, you can
neutralize the command temporarily:

```conf
# postgresql.conf, DANGEROUS: creates a hole in your archive
archive_command = '/bin/true'
```

```sql
SELECT pg_reload_conf();
```

Understand exactly what you are trading: every segment "archived" this way is
**never actually stored**, which means your point-in-time recovery window is broken
from this moment until you take a **fresh base backup**. That is a real, permanent
gap in your recovery capability, not a temporary inconvenience.  Do it only to keep
production alive, write down the exact time you did it, restore the real command
immediately after, and take a new base backup as the very next task.

Turning `archive_mode = off` requires a restart and has the same recovery
consequence, so the `/bin/true` route is usually preferred under pressure.

### Wal_keep_size too high

```conf
wal_keep_size = 1GB    # or 0 if every replica uses a slot
```

```sql
SELECT pg_reload_conf();
```

**Safe**, provided your replicas use replication slots (which give them a stronger
guarantee anyway).  If a slotless replica is currently behind by more than the new
value, lowering it can strand that replica, check replica lag first.

## Step 4: Emergency space when the disk is already full

If the volume is at 100%, PostgreSQL may have already PANICed and shut down, and it
may refuse to start because it cannot write WAL. You need headroom *before* anything
else works.

**In rough order of preference:**

1. **Delete things that are not PostgreSQL data.** Old server logs (not WAL), rotated
   logs, package caches, core dumps, forgotten tarballs in `/tmp`. Always the first
   move.
2. **Delete a ballast file if you have one.** A pre-allocated multi-GB dummy file on
   the data volume exists precisely for this moment.  If you do not have one, create
   one *after* this incident.`fallocate -l 10G /var/lib/postgresql/BALLAST` costs
   nothing and buys you a guaranteed escape hatch.
3. **Grow the volume.** On cloud block storage this is often an online operation and
   is by far the safest real fix.  Do this while you diagnose, not after.
4. **Move the archive destination or old backups off the volume**, if they
   inadvisedly share it.
5. **Last resort: relocate `pg_wal` to another filesystem.** With the server
   **cleanly stopped**, move the directory and symlink it back:

   ```bash
   # Server MUST be stopped. Verify with pg_ctl status. Do not do this while running.
   # Get the real path from the cluster itself, do not assume a distribution
   # layout, and make sure it is the cluster you think it is. Run this BEFORE
   # stopping the server:
   #   psql -Atc 'SHOW data_directory'
   PGDATA=$(psql -Atc 'SHOW data_directory')   # e.g. /var/lib/postgresql/18/main

   mv "$PGDATA/pg_wal" /bigdisk/pg_wal
   ln -s /bigdisk/pg_wal "$PGDATA/pg_wal"
   chown -h postgres:postgres "$PGDATA/pg_wal"
   ```

   This is a supported layout, but you have now added a second filesystem to your
   durability path, if `/bigdisk` disappears, so does your database.  Treat it as
   temporary unless you plan it properly.

**Why you do not just `rm` the oldest segments:** the oldest segments are precisely
the ones being retained deliberately.  They are the file the archiver has not stored,
or the file a slot is still pointing at, or the file recovery would need to replay
from.  "Oldest" is a proxy for "most load-bearing" here, not "most disposable."

There is a tool, `pg_archivecleanup`, that removes WAL, its intended target is the
**archive destination**, cleaning segments older than a restart point.  Pointing it at
a live cluster's `pg_wal` is an expert operation with the same failure modes as
`rm`, and it is not the move to make at 3am on the basis of a search result.

Once you have space, start the server, then work through steps 2 and 3 to fix the
actual cause.  Space alone just resets the timer.

## How do I stop it coming back?

**`max_slot_wal_keep_size` is the guard that matters.** Left unset (unlimited), a
single forgotten slot can fill any disk.  Set, PostgreSQL will invalidate a slot that
exceeds the limit rather than let it take the cluster down.

```conf
max_slot_wal_keep_size = 100GB
```

(Available in PostgreSQL 13 and later.  On older versions there is no such ceiling, which is exactly why slot monitoring is non-negotiable there.)

**How to size it.** It is a trade between two failure modes: too small and a replica
that reboots for 20 minutes gets invalidated and needs a full re-seed; too large and
it cannot prevent the disk from filling.  Work it out from your actual numbers:

1. **Measure your WAL generation rate.** Sample `pg_current_wal_lsn()` an hour apart:

   ```sql
   -- run, wait an hour, run again, then:
   SELECT pg_size_pretty(pg_wal_lsn_diff('<later_lsn>', '<earlier_lsn>')) AS wal_per_hour;
   ```

   Do this during a *busy* hour, and again during your heaviest batch job, a nightly
   bulk load can generate more WAL in 20 minutes than the whole rest of the day.

2. **Decide your tolerable consumer outage.** How long should a replica or CDC pipeline
   be able to be down and still resume without a rebuild?  For a physical standby with
   fast automated re-seeding, a couple of hours may be plenty.  For a logical slot
   feeding a warehouse where re-snapshotting is a six-hour ordeal, you want a day or
   more.

3. **Multiply, using the peak rate**, then sanity-check against free disk.  If
   peak WAL is 10 GB/hour and you want to survive a 12-hour outage.  That is 120 GB, and you need to actually *have* 120 GB of headroom on the WAL volume plus room for
   `max_wal_size` and archiving lag on top.  If you do not, the honest answer is that
   your retention target is a storage purchase, not a config value.  Set the limit to
   what the disk can genuinely absorb (say 60–70% of free space) and accept that a
   long outage means a rebuild.

**Alert on all four signals**, because each is cheap and each one is a 3am call
avoided:

```sql
-- Slots pinning significant WAL, or inactive at all
SELECT slot_name, active,
       pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) AS bytes_retained
FROM pg_replication_slots
WHERE active = false
   OR pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) > 10 * 1024^3;

-- Archiver failing
SELECT failed_count, last_failed_wal, last_failed_time FROM pg_stat_archiver;

-- pg_wal total size
SELECT pg_size_pretty(sum(size)) FROM pg_ls_waldir();
```

Plus plain **free disk percentage on the WAL volume**, from outside the database, so
you still get the page when PostgreSQL itself is the thing that is stuck.

**Give `pg_wal` its own volume** if you can.  Then a WAL surge degrades replication
instead of taking down the entire cluster, and you get an isolated metric to alarm on.

**Clean up slots as part of decommissioning.** The single most common origin story
for this incident is a replica or CDC connector that was torn down months ago and
whose slot nobody dropped.  Make "drop the replication slot" an explicit line item in
the runbook for removing any WAL consumer.

## Why does aI-generated code cause this?

Less directly than most database incidents, but it happens.  An AI coding agent
(Claude Code, Cursor) will happily scaffold a logical replication setup or a CDC
connector.`CREATE_REPLICATION_SLOT`, a Debezium config, a `pg_recvlogical` sidecar, because that is a well-documented, locally-correct pattern.  What it does not add is
the operational half: a `max_slot_wal_keep_size` ceiling, monitoring on slot lag, or
teardown logic that drops the slot when the consumer is removed.

The same gap shows up in generated infrastructure code: an experimental replica
spun up in a branch environment, torn down by deleting the pod or the VM, with the
slot left behind on the primary quietly retaining WAL forever.  The code that created
the problem is correct.  The code that would have prevented it was never in scope.

## How DBGorilla helps

DBGorilla connects read-only and gives your AI coding agent (Claude Code, Cursor)
the state that makes this diagnosable instead of guessable: which replication slots
exist, which are inactive, how much WAL each is pinning, whether the archiver is
failing and on which segment, and how `pg_wal` size compares to your retention
settings.  Your agent can then tell you which of the three causes you are actually
looking at, and which of your services owns the slot that is doing it.  It surfaces
and explains the data; it does not delete files, drop replication slots, change
configuration, or touch your WAL.
[Get started free →](https://app.dbgorilla.com/signup)

## FAQ

**Can I delete files from `pg_wal` to free space?**
No.  Deleting WAL can leave the cluster unable to recover after a crash and can
permanently break replicas and point-in-time recovery.  Remove the cause of the
retention instead; PostgreSQL then recycles the files itself at the next checkpoint.

**Why is `pg_wal` growing?**
Almost always an inactive replication slot, a failing `archive_command`, or
`wal_keep_size` set high.  Check `pg_replication_slots`, `pg_stat_archiver`, and your
config in that order.

**Why does a replication slot pin WAL?**
The slot is a durable promise that the consumer can disconnect and resume without a
gap, so the server must retain WAL from the slot's `restart_lsn` until the consumer
confirms it has processed past it.  A consumer that never returns never releases it.

**Is dropping a replication slot safe?**
It is safe for the database and destructive for the consumer.  A physical standby may
need a full re-seed; a logical/CDC consumer loses its position and must be
re-snapshotted.  Only drop a slot whose consumer is genuinely gone.

**Does WAL free up immediately after I fix the cause?**
No, removal and recycling happen at a checkpoint.  You can run `CHECKPOINT;` to
force one; it is safe but causes an I/O burst.

**What is the one setting that prevents this?**
`max_slot_wal_keep_size` (PostgreSQL 13 and later).  It caps how much WAL a slot may
pin, invalidating the slot instead of filling the disk.  Size it from your measured
peak WAL rate times the consumer outage you want to survive, bounded by real free
disk.

## Related

- [Why is autovacuum not keeping up (and how do I safely make it more aggressive)?](https://www.dbgorilla.com/learn/postgres/why-is-autovacuum-not-keeping-up/)
- [How to fix idle in transaction connections in PostgreSQL](https://www.dbgorilla.com/learn/postgres/how-to-fix-idle-in-transaction-connections/)
- [What is table bloat in PostgreSQL and how do I fix it?](https://www.dbgorilla.com/learn/postgres/what-is-table-bloat-in-postgresql-and-how-to-fix-it/)
- Product: [DBGorilla docs](https://www.dbgorilla.com/docs/)