Why is my pgvector HNSW index taking forever to build?

Quick answer: Look for the NOTICE that says the graph no longer fits into maintenance_work_mem. If it is there, raise the setting for the build session and it goes away: on the table measured below, a build that took 354 s to 440 s at the default took 74 s to 99 s at 2GB, PostgreSQL 17.11 and pgvector 0.8.6 in a container, 2026-09-20. If it is not there, memory is not the problem, and the index parameters, the worker count and the build mode are where to look.

What was measured, and where

Every number on this page came from one container, so you can tell which claims are about PostgreSQL and which are about this table.

server	PostgreSQL 17.11, pgvector 0.8.6, image pgvector/pgvector:0.8.6-pg17
container	Docker, --shm-size=4g, on an 8 vCPU host
table	100,000 rows, vector(768), 403 MB total, 6,144 kB main relation, 768 main pages
data	one fixed set of vectors, reused for every run
date	2026-09-20

Identical runs of the same statement did not return identical times on this host, so the timings that carry an argument are ranges across four runs rather than single numbers: the memory and worker table, the index parameter table and the load order table. Differences smaller than a range are not differences.

The rest are single runs, and they are single runs because the thing they show is not a duration. The lock timings show which statement was let through and which was made to wait. The shared memory failure shows an error. The uniform-random comparison is two runs and says so where it appears. Nothing on this page asks you to compare one single run against another.

The shared memory failure is also the one exception to the single container, and it is labelled where it appears: it needed a second container left on the default 64MB of /dev/shm, same PostgreSQL 17.11 and pgvector 0.8.6, same day, 2026-09-20.

What is the problem?

HNSW builds a graph, and the build is much faster while that graph fits in the memory the operation is allowed. When it stops fitting, the build does not fail and does not stop. It carries on more slowly, and the only sign is a notice at the top of your psql output that scrolls away.

pgvector says so directly:

Indexes build significantly faster when the graph fits into maintenance_work_mem

That sentence is from the pgvector README, read 2026-09-20.

Check it first because it is visible and cheap to check, not because it is always the answer. A build with no NOTICE is slow for some other reason, and the later sections cover those: the m and ef_construction you chose, whether any parallel workers started, and whether you are loading into an index instead of building one.

How do I confirm it?

Look for the notice. Building the index below at maintenance_work_mem of 64MB, on PostgreSQL 17.11 and pgvector 0.8.6 in the container above, 2026-09-20, printed:

NOTICE:  hnsw graph no longer fits into maintenance_work_mem after 17702 tuples
DETAIL:  Building will take significantly more time.
HINT:  Increase maintenance_work_mem to speed up builds.

17,702 of 100,000 rows, on that same PostgreSQL 17.11 and pgvector 0.8.6 container, 2026-09-20. Most of the table was inserted after the graph stopped fitting, which is what the NOTICE tells you and all it tells you. It does not say what share of the elapsed time went to the slow path, because HNSW insertion is not a fixed cost per row.

Do not read the count as a constant either. It is where these particular vectors stopped fitting, and it moves: the same build with four workers on the same table reported 16,754 tuples.

Check what the setting actually is, which is not always what the config file says, and read the worker limits at the same time, because requested and launched are different quantities:

SELECT name, setting, unit, source
FROM pg_settings
WHERE name IN ('maintenance_work_mem', 'autovacuum_work_mem',
               'max_parallel_maintenance_workers', 'max_parallel_workers',
               'max_worker_processes', 'min_parallel_table_scan_size',
               'shared_buffers')
ORDER BY name;

PostgreSQL’s own default for maintenance_work_mem is 64MB, per Resource Consumption in the PostgreSQL 17 documentation, read 2026-09-20. A managed service, an image or an existing postgresql.conf can all have changed it, so read it rather than assuming it.

What the measurements said

Same table, same data, same container, PostgreSQL 17.11 and pgvector 0.8.6 on 2026-09-20. Only the settings changed. Four runs of each, in milliseconds as psql reported them:

maintenance_work_mem	workers asked for	workers launched	run 1	run 2	run 3	run 4
64MB (default)	0	0	354,318.560	409,143.802	425,233.660	440,406.759
2GB	0	0	85,296.640	77,652.457	99,274.600	73,945.844
2GB	4, the usual way	0	83,979.853	74,846.878	100,861.138	77,187.336
2GB	4, plus parallel_workers on the table	4	40,869.252	27,593.054	43,312.814	26,049.636
64MB	4, plus parallel_workers on the table	4	137,217.622	143,445.886	152,356.866	168,213.244

As ranges: 354 s to 440 s at the default, 74 s to 99 s once the graph fit, 75 s to 101 s when four workers were asked for the usual way and none started, 26 s to 43 s when four actually started, and 137 s to 168 s with four workers but not enough memory. Every one of those runs was PostgreSQL 17.11 and pgvector 0.8.6 in the one container, 2026-09-20.

Every one of those five configurations produced an index of 409,608,192 bytes, measured once per row on PostgreSQL 17.11 and pgvector 0.8.6 in that container, 2026-09-20. Nothing about the result changed, only how long you waited for it.

Two things follow from that table. Memory is the bigger lever here: the last row throws four workers at the problem and is still slower than a serial build with enough memory, which is the second row and launched no workers at all. The third row is the trap, which the next section is about.

The worker counts came from pg_stat_progress_create_index joined to the backends whose leader is that build, sampled every 50 milliseconds against builds of 26 seconds and longer, PostgreSQL 17.11 in the same container on 2026-09-20. A cluster-wide count would not have told you whose workers they were.

How do I fix it?

Raise maintenance_work_mem for the build, not for the server

maintenance_work_mem is a per-operation limit rather than a reservation, and autovacuum inherits it by default, so a large global value raises the memory the server can be using at once by more than it looks.

The PostgreSQL 17 resource configuration page puts the autovacuum half like this:

Note that when autovacuum runs, up to autovacuum_max_workers times this memory may be allocated, so be careful not to set the default value too high.

That is from Resource Consumption in the PostgreSQL 17 documentation, read 2026-09-20.

The same page says why autovacuum inherits it, under autovacuum_work_mem:

It defaults to -1, indicating that the value of maintenance_work_mem should be used instead.

Also from Resource Consumption in the PostgreSQL 17 documentation, read 2026-09-20. On the container measured here autovacuum_work_mem was -1 and autovacuum_max_workers was 3.

So set it on the session doing the build:

SET maintenance_work_mem = '2GB';

The right amount depends on your data. Raise it until the NOTICE stops appearing, and stay within what the host can spare. pgvector is explicit about the second half:

Note: Do not set maintenance_work_mem so high that it exhausts the memory on the server

From the pgvector README, read 2026-09-20.

The finished index size is a poor substitute for that experiment. On this table the index was 409,608,192 bytes whether it was built with m = 8 or with m = 32, while the build times for those two settings ran 33,842.310 ms to 37,143.947 ms and 164,689.934 ms to 169,928.906 ms across four runs each, PostgreSQL 17.11 and pgvector 0.8.6 in the container above, 2026-09-20. A number that does not move while the build time does cannot be the thing you size memory against.

Let it build in parallel

SET max_parallel_maintenance_workers = 4;

On this table that line did nothing on its own. It launched zero workers, sampled throughout the build, and the times matched the serial ones: 75 s to 101 s against 74 s to 99 s, PostgreSQL 17.11 and pgvector 0.8.6 in the container above, 2026-09-20.

The reason is layout. The cost model sizes an index build from the main relation, and a vector(768) value is too wide to keep there, so this 403 MB table has a 6,144 kB main relation of 768 pages, against a min_parallel_table_scan_size of 1,024 pages, read off PostgreSQL 17.11 in that container on 2026-09-20. A later section measures that directly.

So check the main relation, not the total:

CREATE TABLE items (
  id        bigserial PRIMARY KEY,
  embedding vector(768)
);

SELECT pg_size_pretty(pg_relation_size('items'))       AS main_relation,
       pg_size_pretty(pg_total_relation_size('items')) AS total,
       pg_relation_size('items') / 8192                AS main_pages;

When the main relation is too small for the cost model to ask for workers, set the value on the table:

ALTER TABLE items SET (parallel_workers = 4);

ALTER TABLE items RESET (parallel_workers);

PostgreSQL documents that this is not a hint:

Setting a value for parallel_workers via ALTER TABLE directly controls how many parallel worker processes will be requested by a CREATE INDEX against the table. This bypasses the cost model completely, and prevents maintenance_work_mem from affecting how many parallel workers are requested.

From CREATE INDEX in the PostgreSQL 17 documentation, read 2026-09-20.

That setting moved the 2GB build from 74 s to 99 s down to 26 s to 43 s on PostgreSQL 17.11 with pgvector 0.8.6 in the container above, 2026-09-20. Reset it when the build is done, because it applies to ordinary queries against the table as well.

Treat that speedup as one result on one machine. Four workers do not mean four times, and whether they help depends on cores, storage and what else the server is doing.

Requested is not launched

max_parallel_maintenance_workers = 4 does not mean four workers. The cluster worker pool is shared, and the leader does its own share of the work.

Measured here on PostgreSQL 17.11 and pgvector 0.8.6 in the container above, 2026-09-20: with parallel_workers = 4 on the table and max_parallel_maintenance_workers = 4, setting max_parallel_workers = 2 launched two workers, not four. max_worker_processes can cap it the same way, and so can memory, because PostgreSQL rations the operation’s budget across the processes:

Note that maintenance_work_mem may influence the number of worker processes requested, since parallel workers must have at least a 32MB share of the total maintenance_work_mem budget. There must also be a remaining 32MB share for the leader process.

From CREATE INDEX in the PostgreSQL 17 documentation, read 2026-09-20.

The workers share one budget rather than each getting a copy of it, and the measurements agree: the serial build at 64MB reported the graph stopped fitting after 17,702 tuples, and the four-worker build at the same 64MB reported 16,754, both on PostgreSQL 17.11 and pgvector 0.8.6 in that container, 2026-09-20. More processes on the same budget spilled slightly sooner rather than four times later.

Confirm what actually started, tied to this build rather than to the cluster:

SELECT p.pid,
       p.index_relid::regclass AS building,
       p.phase,
       (SELECT count(*) FROM pg_stat_activity a WHERE a.leader_pid = p.pid) AS workers
FROM pg_stat_progress_create_index p;

A count of pg_stat_activity rows where backend_type = 'parallel worker' is cluster-wide. It counts workers belonging to other sessions, ordinary queries included, and it can report four while your build is running with none.

If the parallel build fails on shared memory

ERROR:  could not resize shared memory segment "/PostgreSQL.3882895838" to 2144407008 bytes: No space left on device

The disk is fine. Parallel index builds pass data through POSIX shared memory, which in a container is /dev/shm, and Docker’s default for that is 64MB, as it was on the PostgreSQL 17.11 and pgvector 0.8.6 container this came from on 2026-09-20. On Kubernetes the same failure comes from whatever the pod mounts at /dev/shm, which CloudNativePG makes a memory-backed emptyDir, and on a bare host from the system’s own shared memory limits. Check /dev/shm first because it is the common one, not because it is the only one.

docker run --shm-size=4g ...

The size it asks for comes from maintenance_work_mem, not from your data. That error came from a table of 5,000 rows and 20 MB, at maintenance_work_mem of 2GB, in a second container left on the default 64MB of /dev/shm, running the same PostgreSQL 17.11 and pgvector 0.8.6 on 2026-09-20. It asked for 2,144,407,008 bytes on PostgreSQL 17.11 and pgvector 0.8.6 in that container, 2026-09-20. The identical index built serially in the same container in 943.077 ms, one run, which is here to show that the build succeeds without workers rather than to time it.

So a small table is not safe from this, and shrinking the table will not fix it. Either give the container more /dev/shm or build serially. The four-worker measurements on this page ran in a container created with 4GB, PostgreSQL 17.11 and pgvector 0.8.6, 2026-09-20.

Build the index after the load, not before

Creating the index on a populated table is one build. Loading into a table that already has an HNSW index pays graph maintenance on every row. Eight runs, four of each order alternating, on copies of the same 100,000 vectors at maintenance_work_mem of 2GB with max_parallel_maintenance_workers at 0, PostgreSQL 17.11 and pgvector 0.8.6 in the container above, 2026-09-20:

order	run 1	run 2	run 3	run 4
load into an unindexed table, then build	6,057.105 + 81,443.288	4,055.707 + 111,057.057	3,622.957 + 101,949.906	3,681.128 + 95,298.621
build on the empty table, then load	497,427.152	474,261.015	518,178.214	412,997.199

Milliseconds, and the first row is the load followed by the build. As ranges on PostgreSQL 17.11 and pgvector 0.8.6 in that container, 2026-09-20: 3,622.957 ms to 6,057.105 ms to load, then 81,443.288 ms to 111,057.057 ms to build, against 412,997.199 ms to 518,178.214 ms to load into the indexed table. The ranges do not come close to overlapping.

Do not compare that build range against the index parameter table’s. These eight runs each built a fresh copy of the rows, so the table was newly written every time and the parameter table’s was not.

All eight runs ended with an index of 409,608,192 bytes. For a bulk import, drop the index, load, and create it again.

Choose the build mode deliberately

A plain CREATE INDEX blocks writes to the table for the whole build:

Normally PostgreSQL locks the table to be indexed against writes and performs the entire index build with a single scan of the table.

From CREATE INDEX in the PostgreSQL 17 documentation, read 2026-09-20.

Measured on this table, PostgreSQL 17.11 and pgvector 0.8.6 in the container above, 2026-09-20: while a plain build ran it held a ShareLock, a SELECT count(*) returned in 13.475 ms, and an INSERT with lock_timeout = '20s' waited until the timeout cancelled it at 20,000.487 ms.

CREATE INDEX CONCURRENTLY is the alternative. On the same table, same PostgreSQL 17.11 and pgvector 0.8.6 container on 2026-09-20, it held a ShareUpdateExclusiveLock, let the same SELECT through in 13.312 ms and the same INSERT through in 5.781 ms. Those four timings are one run each. What they establish is which statement was let through and which was made to wait, and a second run would not change that: the INSERT under a plain build did not run at all, it was cancelled at its timeout.

The concurrent build scans the table more than once, takes longer, and can leave an invalid index behind if it fails, which then has to be dropped and built again.

It also cannot be wrapped in a transaction:

a regular CREATE INDEX command can be performed within a transaction block, but CREATE INDEX CONCURRENTLY cannot

From CREATE INDEX in the PostgreSQL 17 documentation, read 2026-09-20. Running it inside BEGIN returns ERROR: CREATE INDEX CONCURRENTLY cannot run inside a transaction block.

CREATE INDEX items_embedding_hnsw ON items
USING hnsw (embedding vector_l2_ops)
WITH (m = 16, ef_construction = 64);

The index parameters are part of the build cost

m is how many connections each node in the graph keeps, and ef_construction is how hard the build searches while placing each one. Both raise recall and both cost build time. Twelve builds on this table, four per setting run round robin, at maintenance_work_mem of 2GB with max_parallel_maintenance_workers at 0, PostgreSQL 17.11 and pgvector 0.8.6 in the container above, 2026-09-20:

parameters	run 1	run 2	run 3	run 4	index size
m = 8, ef_construction = 32	35,072.828	37,143.947	35,483.416	33,842.310	409,608,192 bytes
m = 16, ef_construction = 64 (the defaults)	80,843.626	73,788.178	78,932.539	76,163.471	409,608,192 bytes
m = 32, ef_construction = 128	164,689.934	169,928.906	166,606.816	164,832.888	409,608,192 bytes

As ranges, in milliseconds on PostgreSQL 17.11 and pgvector 0.8.6 in that container, 2026-09-20: 33,842.310 to 37,143.947 at the smaller setting, 73,788.178 to 80,843.626 at the defaults, and 164,689.934 to 169,928.906 at the larger one. The defaults row here is the same configuration as the second row of the memory and worker table, which ran 73,945.844 ms to 99,274.600 ms on that same server, container and day, and the two ranges overlap. That is how you can tell the two tables are measuring the same thing.

The index size did not move across those twelve builds. At 768 dimensions, m bought graph connectivity without a measurable footprint here. At smaller dimensions that would not hold.

If you raised memory and workers and the build is still slow, check this setting.

Why is the main relation so small?

The vectors are not in it, which makes a vector table behave differently from most large tables you have indexed.

Because the vectors are not in it, a vector table behaves differently from most large tables you have indexed.

SELECT c.relname,
       pg_size_pretty(pg_relation_size(c.oid)) AS main_relation,
       pg_size_pretty(COALESCE(pg_total_relation_size(c.reltoastrelid), 0)) AS toast
FROM pg_class c
WHERE c.relname = 'items';

Two tables of 10,000 rows each, measured on PostgreSQL 17.11 and pgvector 0.8.6 in the container above on 2026-09-20, differing only in the width of the vector:

column	main relation	TOAST	attstorage
vector(3)	592 kB	8,192 bytes	e
vector(768)	592 kB	40 MB	e

Both columns have the same type and storage strategy, with opposite results. The strategy permits out-of-line storage rather than forcing it:

EXTERNAL allows out-of-line storage but not compression.

From TOAST in the PostgreSQL 17 documentation, read 2026-09-20.

What decides is the width of the row, per TOAST in the PostgreSQL 17 documentation, read 2026-09-20:

The TOAST management code is triggered only when a row value to be stored in a table is wider than TOAST_TUPLE_THRESHOLD bytes (normally 2 kB).

Also from TOAST in the PostgreSQL 17 documentation, read 2026-09-20. A vector(3) row is nowhere near that. A vector(768) row is well past it, so the value goes out of line and the main relation keeps a pointer.

That is why the cost model asked for no workers on this 403 MB table on PostgreSQL 17.11 and pgvector 0.8.6 in the container above, 2026-09-20, and why the parallel_workers override is the answer rather than a smaller min_parallel_table_scan_size.

What else makes a build slow

NULL vectors are not indexed, so your row count is not your element count. Measured on two tables of 10,000 rows, PostgreSQL 17.11 and pgvector 0.8.6 in the container above, 2026-09-20: with every vector present the index was 40,968,192 bytes, and with half of them NULL it was 20,488,192 bytes. Counting rows overstates the work when the column is sparsely populated.

The vectors themselves. Build time depends on the data, not only on its shape. The same 100,000 row, 768 dimension build at the same settings took 16,747.287 ms and 16,206.863 ms on two runs over freshly generated uniform random vectors, against 73,945.844 ms to 99,274.600 ms on the fixed set used for the rest of this page, PostgreSQL 17.11 and pgvector 0.8.6 in that container, 2026-09-20. The uniform index also came out at a different size, 407,158,784 bytes. Benchmark on your own embeddings, because a timing from other data does not carry over.

The other index type. pgvector offers IVFFlat as well, and describes the trade-off:

It has better query performance than IVFFlat (in terms of speed-recall tradeoff), but has slower build times and uses more memory.

That is HNSW being described, in the pgvector README, read 2026-09-20. If your recall budget allows it, IVFFlat changes the build problem rather than tuning it.

How do I stop it coming back?

Keep the notice. If your deployment tooling swallows psql’s stderr, that NOTICE is the one message that would have told you what was wrong, and you have thrown it away.

Write the build settings into the migration that creates the index, with the m and ef_construction spelled out, so the next person on a fresh database gets the same build and the same index.

Re-check as the table grows. A value that held the graph at a hundred thousand rows will not at a million, and the symptom is a build that quietly got slower rather than an error.

Set --shm-size on every container that runs PostgreSQL, not only the one where you hit this. The default is small enough to cause problems for parallel work generally, not only for vector indexes.

How DBGorilla helps

DBGorilla can help

DBGorilla connects read-only and records each index’s size on disk over time, so you can see what an index actually grew to rather than estimating it. It also keeps topology snapshots, so comparing two either side of a build shows the index appear. And it samples sessions and wait events over time, so a long-running build is visible as what it was doing rather than as a gap. All of that was verified in a live session.

Start free

FAQ

• Why is my HNSW build so slow?

  Check first whether pgvector printed the NOTICE saying the graph no longer fits into maintenance_work_mem. If it did, the build left its fast path. Measured 2026-09-20 on 100,000 rows of vector(768), PostgreSQL 17.11 and pgvector 0.8.6 in a Docker container, four runs each: the same index took 354 s to 440 s at the 64MB default and 74 s to 99 s at 2GB. If the NOTICE never appeared, memory is not your problem, and the HNSW parameters, the worker count and the build mode are where to look.

• How much maintenance_work_mem does HNSW need?

  Enough that the NOTICE stops appearing, found by trying it, and no more than the host can spare. The finished index size is not a sizing rule: measured 2026-09-20 on PostgreSQL 17.11 and pgvector 0.8.6 in a container, the index came out at 409,608,192 bytes in all twelve builds, at m=8, at m=16 and at m=32, while the build time for those same three settings ran 33,842.310 ms to 37,143.947 ms, 73,788.178 ms to 80,843.626 ms and 164,689.934 ms to 169,928.906 ms across four runs each. Set it for the session that builds the index, not globally.

• Do parallel workers help?

  They did here, once they started. Measured 2026-09-20 on PostgreSQL 17.11 and pgvector 0.8.6 in a container: at 2GB the build took 74 s to 99 s with no workers and 26 s to 43 s with four. Setting max_parallel_maintenance_workers to 4 on its own launched zero workers and changed nothing, 75 s to 101 s, because the cost model sizes the build from the main relation and a vector(768) column lives in TOAST. ALTER TABLE items SET (parallel_workers = 4) is what started them.

• Why did my build fail with no space left on device when the disk is empty?

  It is shared memory, not disk. Docker gives a container 64MB of /dev/shm by default and a parallel build asks for about maintenance_work_mem. The size of your table does not come into it: measured 2026-09-20 on PostgreSQL 17.11 and pgvector 0.8.6, a 5,000 row table of 20 MB asked for 2,144,407,008 bytes at maintenance_work_mem of 2GB and failed, while the same index built serially in 943.077 ms. Run the container with a larger --shm-size, or build serially. On Kubernetes the same failure comes from the pod rather than from Docker.

• Should I index before or after loading the data?

  After. Measured 2026-09-20 on PostgreSQL 17.11 and pgvector 0.8.6 in a container, four runs of each order: loading 100,000 rows of vector(768) into a table that already had the HNSW index took 412,997.199 ms to 518,178.214 ms. Loading the same rows into a table with no index took 3,622.957 ms to 6,057.105 ms, and building the index afterwards took 81,443.288 ms to 111,057.057 ms. All eight runs ended with an index of 409,608,192 bytes.

Related

• When to leave pgvector for a dedicated vector database