Vector search benchmark
These benchmarks were taken from the eShopOnWeb data set (524 vectors of 1024 dimensions) and from the 20 golden questions, which ran the speed test.
The queries are read from ~/ForClaude/evalkit/questions_golden.json.
sources
- results
results:pipeline= eshoponweb - results
results:rows= 524 - results
results:dimension= 1024 - results
results:queryCount= 20 - consolidated
consolidated:queries.recorded= golden: 20 labelled questions from ~/ForClaude/evalkit/questions_golden.json, embedded with qwen3-emb-0.6b (cached in ~/gvb-data/bench-cache/golden-eshoponweb-68df42ca69efa079.json, no embedding calls)
Multi search, 8 searchers at once: database engines in order
Most searches per second first. Searches per second is the median of the 6 runs. The ms per search is worked out as one second divided by the searches per second, so it is not a timed latency. Each difference is from the row above it.
| Engine | Searches per second | ms per search | Difference from the row above, ms per search |
|---|---|---|---|
| mariadb | 5,923 | 0.169 | first in the list |
| redis within run-to-run spread of the row above | 5,179 | 0.193 | +0.024 |
redis holds its data in memory its saved docs page says 'Redis is an in-memory but persistent on disk database', and its compose file sets save "300 1" and appendonly no. Read this row with that in mind. sources
| |||
| pgvector within run-to-run spread of the row above | 4,004 | 0.250 | +0.057 |
| qdrant-hnsw within run-to-run spread of the row above | 3,460 | 0.289 | +0.039 |
| qdrant within run-to-run spread of the row above | 3,284 | 0.305 | +0.016 |
| oracle within run-to-run spread of the row above | 2,803 | 0.357 | +0.052 |
| elasticsearch within run-to-run spread of the row above | 2,701 | 0.370 | +0.013 |
| mongodb within run-to-run spread of the row above | 2,123 | 0.471 | +0.101 |
| vespa within run-to-run spread of the row above | 1,560 | 0.641 | +0.170 |
| opensearch within run-to-run spread of the row above | 1,492 | 0.670 | +0.029 |
| milvus within run-to-run spread of the row above | 1,264 | 0.791 | +0.121 |
| sqlitevec within run-to-run spread of the row above | 1,137 | 0.879 | +0.088 |
| chroma within run-to-run spread of the row above | 778 | 1.286 | +0.406 |
| sql-diskann within run-to-run spread of the row above | 687 | 1.457 | +0.171 |
| sql within run-to-run spread of the row above | 604 | 1.655 | +0.198 |
| duckdb within run-to-run spread of the row above | 577 | 1.733 | +0.078 |
| typesense within run-to-run spread of the row above | 574 | 1.741 | +0.008 |
| weaviate | 306 | 3.270 | +1.530 |
| Not ranked | |||
| clickhouse | not held, not ranked | ||
A row marked within run-to-run spread is not separated from the row above by the report's test. The order shown is the order of the medians.
A row marked not held has figures from a timed pass the tool recorded as not held: the engine was still changing when it was timed. The row is shown and is not ranked.
The figures of a row that is not ranked are in the full results.
Full results: every table, the flags and every run
Build that measured the runs: v7 not recorded, v8 9b924200abc3. Set: published-2026-10-08.
In this public copy, machine names, network addresses and home-folder paths are replaced; the checksums in the results refer to the unredacted run files.