# Why is my pgvector HNSW index taking forever to build? > **Quick answer:** Look for the NOTICE that says the graph no longer fits into > `maintenance_work_mem`. If it is there, raise the setting for the build session > and it goes away: on the table measured below, a build that took 354 s to 440 s > at the default took 74 s to 99 s at 2GB, PostgreSQL 17.11 and pgvector 0.8.6 in > a container, 2026-09-20. If it is not there, memory is not the > problem, and the index parameters, the worker count and the build mode are > where to look. ## What was measured, and where Every number on this page came from one container, so you can tell which claims are about PostgreSQL and which are about this table. | | | |---|---| | server | PostgreSQL 17.11, pgvector 0.8.6, image `pgvector/pgvector:0.8.6-pg17` | | container | Docker, `--shm-size=4g`, on an 8 vCPU host | | table | 100,000 rows, `vector(768)`, 403 MB total, 6,144 kB main relation, 768 main pages | | data | one fixed set of vectors, reused for every run | | date | 2026-09-20 | Identical runs of the same statement did not return identical times on this host, so the timings that carry an argument are ranges across four runs rather than single numbers: the memory and worker table, the index parameter table and the load order table. Differences smaller than a range are not differences. The rest are single runs, and they are single runs because the thing they show is not a duration. The lock timings show which statement was let through and which was made to wait. The shared memory failure shows an error. The uniform-random comparison is two runs and says so where it appears. Nothing on this page asks you to compare one single run against another. The shared memory failure is also the one exception to the single container, and it is labelled where it appears: it needed a second container left on the default 64MB of `/dev/shm`, same PostgreSQL 17.11 and pgvector 0.8.6, same day, 2026-09-20. ## What is the problem? HNSW builds a graph, and the build is much faster while that graph fits in the memory the operation is allowed. When it stops fitting, the build does not fail and does not stop. It carries on more slowly, and the only sign is a notice at the top of your psql output that scrolls away. pgvector says so directly: > Indexes build significantly faster when the graph fits into > `maintenance_work_mem` That sentence is from the [pgvector README](https://github.com/pgvector/pgvector), read 2026-09-20. Check it first because it is visible and cheap to check, not because it is always the answer. A build with no NOTICE is slow for some other reason, and the later sections cover those: the `m` and `ef_construction` you chose, whether any parallel workers started, and whether you are loading into an index instead of building one. ## How do I confirm it? Look for the notice. Building the index below at `maintenance_work_mem` of 64MB, on PostgreSQL 17.11 and pgvector 0.8.6 in the container above, 2026-09-20, printed: ``` NOTICE: hnsw graph no longer fits into maintenance_work_mem after 17702 tuples DETAIL: Building will take significantly more time. HINT: Increase maintenance_work_mem to speed up builds. ``` 17,702 of 100,000 rows, on that same PostgreSQL 17.11 and pgvector 0.8.6 container, 2026-09-20. Most of the table was inserted after the graph stopped fitting, which is what the NOTICE tells you and all it tells you. It does not say what share of the elapsed time went to the slow path, because HNSW insertion is not a fixed cost per row. Do not read the count as a constant either. It is where these particular vectors stopped fitting, and it moves: the same build with four workers on the same table reported 16,754 tuples. Check what the setting actually is, which is not always what the config file says, and read the worker limits at the same time, because requested and launched are different quantities: ```sql SELECT name, setting, unit, source FROM pg_settings WHERE name IN ('maintenance_work_mem', 'autovacuum_work_mem', 'max_parallel_maintenance_workers', 'max_parallel_workers', 'max_worker_processes', 'min_parallel_table_scan_size', 'shared_buffers') ORDER BY name; ``` PostgreSQL's own default for `maintenance_work_mem` is 64MB, per [Resource Consumption in the PostgreSQL 17 documentation](https://www.postgresql.org/docs/17/runtime-config-resource.html), read 2026-09-20. A managed service, an image or an existing `postgresql.conf` can all have changed it, so read it rather than assuming it. ## What the measurements said Same table, same data, same container, PostgreSQL 17.11 and pgvector 0.8.6 on 2026-09-20. Only the settings changed. Four runs of each, in milliseconds as psql reported them: | `maintenance_work_mem` | workers asked for | workers launched | run 1 | run 2 | run 3 | run 4 | |---|---|---|---|---|---|---| | 64MB (default) | 0 | 0 | 354,318.560 | 409,143.802 | 425,233.660 | 440,406.759 | | 2GB | 0 | 0 | 85,296.640 | 77,652.457 | 99,274.600 | 73,945.844 | | 2GB | 4, the usual way | **0** | 83,979.853 | 74,846.878 | 100,861.138 | 77,187.336 | | 2GB | 4, plus `parallel_workers` on the table | 4 | 40,869.252 | 27,593.054 | 43,312.814 | 26,049.636 | | 64MB | 4, plus `parallel_workers` on the table | 4 | 137,217.622 | 143,445.886 | 152,356.866 | 168,213.244 | As ranges: 354 s to 440 s at the default, 74 s to 99 s once the graph fit, 75 s to 101 s when four workers were asked for the usual way and none started, 26 s to 43 s when four actually started, and 137 s to 168 s with four workers but not enough memory. Every one of those runs was PostgreSQL 17.11 and pgvector 0.8.6 in the one container, 2026-09-20. Every one of those five configurations produced an index of 409,608,192 bytes, measured once per row on PostgreSQL 17.11 and pgvector 0.8.6 in that container, 2026-09-20. Nothing about the result changed, only how long you waited for it. Two things follow from that table. Memory is the bigger lever here: the last row throws four workers at the problem and is still slower than a serial build with enough memory, which is the second row and launched no workers at all. The third row is the trap, which the next section is about. The worker counts came from `pg_stat_progress_create_index` joined to the backends whose leader is that build, sampled every 50 milliseconds against builds of 26 seconds and longer, PostgreSQL 17.11 in the same container on 2026-09-20. A cluster-wide count would not have told you whose workers they were. ## How do I fix it? ### Raise `maintenance_work_mem` for the build, not for the server `maintenance_work_mem` is a per-operation limit rather than a reservation, and autovacuum inherits it by default, so a large global value raises the memory the server can be using at once by more than it looks. The PostgreSQL 17 resource configuration page puts the autovacuum half like this: > Note that when autovacuum runs, up to autovacuum_max_workers times this memory > may be allocated, so be careful not to set the default value too high. That is from [Resource Consumption in the PostgreSQL 17 documentation](https://www.postgresql.org/docs/17/runtime-config-resource.html), read 2026-09-20. The same page says why autovacuum inherits it, under `autovacuum_work_mem`: > It defaults to -1, indicating that the value of maintenance_work_mem should be > used instead. Also from [Resource Consumption in the PostgreSQL 17 documentation](https://www.postgresql.org/docs/17/runtime-config-resource.html), read 2026-09-20. On the container measured here `autovacuum_work_mem` was `-1` and `autovacuum_max_workers` was 3. So set it on the session doing the build: ```sql SET maintenance_work_mem = '2GB'; ``` The right amount depends on your data. Raise it until the NOTICE stops appearing, and stay within what the host can spare. pgvector is explicit about the second half: > Note: Do not set `maintenance_work_mem` so high that it exhausts the memory on > the server From the [pgvector README](https://github.com/pgvector/pgvector), read 2026-09-20. The finished index size is a poor substitute for that experiment. On this table the index was 409,608,192 bytes whether it was built with `m = 8` or with `m = 32`, while the build times for those two settings ran 33,842.310 ms to 37,143.947 ms and 164,689.934 ms to 169,928.906 ms across four runs each, PostgreSQL 17.11 and pgvector 0.8.6 in the container above, 2026-09-20. A number that does not move while the build time does cannot be the thing you size memory against. ### Let it build in parallel ```sql SET max_parallel_maintenance_workers = 4; ``` On this table that line did nothing on its own. It launched zero workers, sampled throughout the build, and the times matched the serial ones: 75 s to 101 s against 74 s to 99 s, PostgreSQL 17.11 and pgvector 0.8.6 in the container above, 2026-09-20. The reason is layout. The cost model sizes an index build from the main relation, and a `vector(768)` value is too wide to keep there, so this 403 MB table has a 6,144 kB main relation of 768 pages, against a `min_parallel_table_scan_size` of 1,024 pages, read off PostgreSQL 17.11 in that container on 2026-09-20. A later section measures that directly. So check the main relation, not the total: ```sql CREATE TABLE items ( id bigserial PRIMARY KEY, embedding vector(768) ); ``` ```sql SELECT pg_size_pretty(pg_relation_size('items')) AS main_relation, pg_size_pretty(pg_total_relation_size('items')) AS total, pg_relation_size('items') / 8192 AS main_pages; ``` When the main relation is too small for the cost model to ask for workers, set the value on the table: ```sql ALTER TABLE items SET (parallel_workers = 4); ALTER TABLE items RESET (parallel_workers); ``` PostgreSQL documents that this is not a hint: > Setting a value for `parallel_workers` via `ALTER TABLE` directly controls how > many parallel worker processes will be requested by a `CREATE INDEX` against > the table. This bypasses the cost model completely, and prevents > `maintenance_work_mem` from affecting how many parallel workers are requested. From [CREATE INDEX in the PostgreSQL 17 documentation](https://www.postgresql.org/docs/17/sql-createindex.html), read 2026-09-20. That setting moved the 2GB build from 74 s to 99 s down to 26 s to 43 s on PostgreSQL 17.11 with pgvector 0.8.6 in the container above, 2026-09-20. Reset it when the build is done, because it applies to ordinary queries against the table as well. Treat that speedup as one result on one machine. Four workers do not mean four times, and whether they help depends on cores, storage and what else the server is doing. ### Requested is not launched `max_parallel_maintenance_workers = 4` does not mean four workers. The cluster worker pool is shared, and the leader does its own share of the work. Measured here on PostgreSQL 17.11 and pgvector 0.8.6 in the container above, 2026-09-20: with `parallel_workers = 4` on the table and `max_parallel_maintenance_workers = 4`, setting `max_parallel_workers = 2` launched two workers, not four. `max_worker_processes` can cap it the same way, and so can memory, because PostgreSQL rations the operation's budget across the processes: > Note that `maintenance_work_mem` may influence the number of worker processes > requested, since parallel workers must have at least a `32MB` share of the > total `maintenance_work_mem` budget. There must also be a remaining `32MB` > share for the leader process. From [CREATE INDEX in the PostgreSQL 17 documentation](https://www.postgresql.org/docs/17/sql-createindex.html), read 2026-09-20. The workers share one budget rather than each getting a copy of it, and the measurements agree: the serial build at 64MB reported the graph stopped fitting after 17,702 tuples, and the four-worker build at the same 64MB reported 16,754, both on PostgreSQL 17.11 and pgvector 0.8.6 in that container, 2026-09-20. More processes on the same budget spilled slightly sooner rather than four times later. Confirm what actually started, tied to this build rather than to the cluster: ```sql SELECT p.pid, p.index_relid::regclass AS building, p.phase, (SELECT count(*) FROM pg_stat_activity a WHERE a.leader_pid = p.pid) AS workers FROM pg_stat_progress_create_index p; ``` A count of `pg_stat_activity` rows where `backend_type = 'parallel worker'` is cluster-wide. It counts workers belonging to other sessions, ordinary queries included, and it can report four while your build is running with none. ### If the parallel build fails on shared memory ``` ERROR: could not resize shared memory segment "/PostgreSQL.3882895838" to 2144407008 bytes: No space left on device ``` The disk is fine. Parallel index builds pass data through POSIX shared memory, which in a container is `/dev/shm`, and Docker's default for that is 64MB, as it was on the PostgreSQL 17.11 and pgvector 0.8.6 container this came from on 2026-09-20. On Kubernetes the same failure comes from whatever the pod mounts at `/dev/shm`, which CloudNativePG makes a memory-backed `emptyDir`, and on a bare host from the system's own shared memory limits. Check `/dev/shm` first because it is the common one, not because it is the only one. ``` docker run --shm-size=4g ... ``` The size it asks for comes from `maintenance_work_mem`, not from your data. That error came from a table of 5,000 rows and 20 MB, at `maintenance_work_mem` of 2GB, in a second container left on the default 64MB of `/dev/shm`, running the same PostgreSQL 17.11 and pgvector 0.8.6 on 2026-09-20. It asked for 2,144,407,008 bytes on PostgreSQL 17.11 and pgvector 0.8.6 in that container, 2026-09-20. The identical index built serially in the same container in 943.077 ms, one run, which is here to show that the build succeeds without workers rather than to time it. So a small table is not safe from this, and shrinking the table will not fix it. Either give the container more `/dev/shm` or build serially. The four-worker measurements on this page ran in a container created with 4GB, PostgreSQL 17.11 and pgvector 0.8.6, 2026-09-20. ### Build the index after the load, not before Creating the index on a populated table is one build. Loading into a table that already has an HNSW index pays graph maintenance on every row. Eight runs, four of each order alternating, on copies of the same 100,000 vectors at `maintenance_work_mem` of 2GB with `max_parallel_maintenance_workers` at 0, PostgreSQL 17.11 and pgvector 0.8.6 in the container above, 2026-09-20: | order | run 1 | run 2 | run 3 | run 4 | |---|---|---|---|---| | load into an unindexed table, then build | 6,057.105 + 81,443.288 | 4,055.707 + 111,057.057 | 3,622.957 + 101,949.906 | 3,681.128 + 95,298.621 | | build on the empty table, then load | 497,427.152 | 474,261.015 | 518,178.214 | 412,997.199 | Milliseconds, and the first row is the load followed by the build. As ranges on PostgreSQL 17.11 and pgvector 0.8.6 in that container, 2026-09-20: 3,622.957 ms to 6,057.105 ms to load, then 81,443.288 ms to 111,057.057 ms to build, against 412,997.199 ms to 518,178.214 ms to load into the indexed table. The ranges do not come close to overlapping. Do not compare that build range against the index parameter table's. These eight runs each built a fresh copy of the rows, so the table was newly written every time and the parameter table's was not. All eight runs ended with an index of 409,608,192 bytes. For a bulk import, drop the index, load, and create it again. ### Choose the build mode deliberately A plain `CREATE INDEX` blocks writes to the table for the whole build: > Normally PostgreSQL locks the table to be indexed against writes and performs > the entire index build with a single scan of the table. From [CREATE INDEX in the PostgreSQL 17 documentation](https://www.postgresql.org/docs/17/sql-createindex.html), read 2026-09-20. Measured on this table, PostgreSQL 17.11 and pgvector 0.8.6 in the container above, 2026-09-20: while a plain build ran it held a `ShareLock`, a `SELECT count(*)` returned in 13.475 ms, and an `INSERT` with `lock_timeout = '20s'` waited until the timeout cancelled it at 20,000.487 ms. `CREATE INDEX CONCURRENTLY` is the alternative. On the same table, same PostgreSQL 17.11 and pgvector 0.8.6 container on 2026-09-20, it held a `ShareUpdateExclusiveLock`, let the same `SELECT` through in 13.312 ms and the same `INSERT` through in 5.781 ms. Those four timings are one run each. What they establish is which statement was let through and which was made to wait, and a second run would not change that: the `INSERT` under a plain build did not run at all, it was cancelled at its timeout. The concurrent build scans the table more than once, takes longer, and can leave an invalid index behind if it fails, which then has to be dropped and built again. It also cannot be wrapped in a transaction: > a regular `CREATE INDEX` command can be performed within a transaction block, > but `CREATE INDEX CONCURRENTLY` cannot From [CREATE INDEX in the PostgreSQL 17 documentation](https://www.postgresql.org/docs/17/sql-createindex.html), read 2026-09-20. Running it inside `BEGIN` returns `ERROR: CREATE INDEX CONCURRENTLY cannot run inside a transaction block`. ```sql CREATE INDEX items_embedding_hnsw ON items USING hnsw (embedding vector_l2_ops) WITH (m = 16, ef_construction = 64); ``` ### The index parameters are part of the build cost `m` is how many connections each node in the graph keeps, and `ef_construction` is how hard the build searches while placing each one. Both raise recall and both cost build time. Twelve builds on this table, four per setting run round robin, at `maintenance_work_mem` of 2GB with `max_parallel_maintenance_workers` at 0, PostgreSQL 17.11 and pgvector 0.8.6 in the container above, 2026-09-20: | parameters | run 1 | run 2 | run 3 | run 4 | index size | |---|---|---|---|---|---| | `m = 8, ef_construction = 32` | 35,072.828 | 37,143.947 | 35,483.416 | 33,842.310 | 409,608,192 bytes | | `m = 16, ef_construction = 64` (the defaults) | 80,843.626 | 73,788.178 | 78,932.539 | 76,163.471 | 409,608,192 bytes | | `m = 32, ef_construction = 128` | 164,689.934 | 169,928.906 | 166,606.816 | 164,832.888 | 409,608,192 bytes | As ranges, in milliseconds on PostgreSQL 17.11 and pgvector 0.8.6 in that container, 2026-09-20: 33,842.310 to 37,143.947 at the smaller setting, 73,788.178 to 80,843.626 at the defaults, and 164,689.934 to 169,928.906 at the larger one. The defaults row here is the same configuration as the second row of the memory and worker table, which ran 73,945.844 ms to 99,274.600 ms on that same server, container and day, and the two ranges overlap. That is how you can tell the two tables are measuring the same thing. The index size did not move across those twelve builds. At 768 dimensions, `m` bought graph connectivity without a measurable footprint here. At smaller dimensions that would not hold. If you raised memory and workers and the build is still slow, check this setting. ## Why is the main relation so small? The vectors are not in it, which makes a vector table behave differently from most large tables you have indexed. Because the vectors are not in it, a vector table behaves differently from most large tables you have indexed. ```sql SELECT c.relname, pg_size_pretty(pg_relation_size(c.oid)) AS main_relation, pg_size_pretty(COALESCE(pg_total_relation_size(c.reltoastrelid), 0)) AS toast FROM pg_class c WHERE c.relname = 'items'; ``` Two tables of 10,000 rows each, measured on PostgreSQL 17.11 and pgvector 0.8.6 in the container above on 2026-09-20, differing only in the width of the vector: | column | main relation | TOAST | `attstorage` | |---|---|---|---| | `vector(3)` | 592 kB | 8,192 bytes | `e` | | `vector(768)` | 592 kB | 40 MB | `e` | Both columns have the same type and storage strategy, with opposite results. The strategy permits out-of-line storage rather than forcing it: > EXTERNAL allows out-of-line storage but not compression. From [TOAST in the PostgreSQL 17 documentation](https://www.postgresql.org/docs/17/storage-toast.html), read 2026-09-20. What decides is the width of the row, per [TOAST in the PostgreSQL 17 documentation](https://www.postgresql.org/docs/17/storage-toast.html), read 2026-09-20: > The TOAST management code is triggered only when a row value to be stored in a > table is wider than TOAST_TUPLE_THRESHOLD bytes (normally 2 kB). Also from [TOAST in the PostgreSQL 17 documentation](https://www.postgresql.org/docs/17/storage-toast.html), read 2026-09-20. A `vector(3)` row is nowhere near that. A `vector(768)` row is well past it, so the value goes out of line and the main relation keeps a pointer. That is why the cost model asked for no workers on this 403 MB table on PostgreSQL 17.11 and pgvector 0.8.6 in the container above, 2026-09-20, and why the `parallel_workers` override is the answer rather than a smaller `min_parallel_table_scan_size`. ## What else makes a build slow NULL vectors are not indexed, so your row count is not your element count. Measured on two tables of 10,000 rows, PostgreSQL 17.11 and pgvector 0.8.6 in the container above, 2026-09-20: with every vector present the index was 40,968,192 bytes, and with half of them NULL it was 20,488,192 bytes. Counting rows overstates the work when the column is sparsely populated. The vectors themselves. Build time depends on the data, not only on its shape. The same 100,000 row, 768 dimension build at the same settings took 16,747.287 ms and 16,206.863 ms on two runs over freshly generated uniform random vectors, against 73,945.844 ms to 99,274.600 ms on the fixed set used for the rest of this page, PostgreSQL 17.11 and pgvector 0.8.6 in that container, 2026-09-20. The uniform index also came out at a different size, 407,158,784 bytes. Benchmark on your own embeddings, because a timing from other data does not carry over. The other index type. pgvector offers IVFFlat as well, and describes the trade-off: > It has better query performance than IVFFlat (in terms of speed-recall > tradeoff), but has slower build times and uses more memory. That is HNSW being described, in the [pgvector README](https://github.com/pgvector/pgvector), read 2026-09-20. If your recall budget allows it, IVFFlat changes the build problem rather than tuning it. ## How do I stop it coming back? Keep the notice. If your deployment tooling swallows psql's stderr, that NOTICE is the one message that would have told you what was wrong, and you have thrown it away. Write the build settings into the migration that creates the index, with the `m` and `ef_construction` spelled out, so the next person on a fresh database gets the same build and the same index. Re-check as the table grows. A value that held the graph at a hundred thousand rows will not at a million, and the symptom is a build that quietly got slower rather than an error. Set `--shm-size` on every container that runs PostgreSQL, not only the one where you hit this. The default is small enough to cause problems for parallel work generally, not only for vector indexes. ## How DBGorilla helps DBGorilla connects read-only and records each index's size on disk over time, so you can see what an index actually grew to rather than estimating it. It also keeps topology snapshots, so comparing two either side of a build shows the index appear. And it samples sessions and wait events over time, so a long-running build is visible as what it was doing rather than as a gap. All of that was verified in a live session. [Get started free →](https://app.dbgorilla.com/signup) ## Related - [When to leave pgvector for a dedicated vector database](https://www.dbgorilla.com/learn/postgres/when-to-leave-pgvector-for-a-dedicated-vector-database/)