Vector engine benchmark: adventureworks
Run started 3 Oct 2026, 15:15 UTC. Data: AdventureWorks, 50,000 vectors. Queries: 200 random stored vectors.
Machine: CPU Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz; Logical CPUs 8; RAM (GiB) 62.7; OS Ubuntu 24.04.5 LTS, kernel 6.8.0-139-generic; Governor not recorded; Partition not recorded; Debug build.
Warm-up, as this run's notes record it: Latency is client-side wall time around each search (network and driver included), one query at a time, after 20 warm-up queries; at least 200 samples (small query sets are repeated).
| Engine | p50 (ms) | Searches per second, one searcher | Searches per second, eight searchers at once | Exact mode p50 (ms) | Client CPU per search, one searcher (ms) | Recall in hits | Flags |
|---|---|---|---|---|---|---|---|
| Qdrant (HNSW) | 1.61 | 462 | 1,467 | 10.87 | - | 1997 of 2000 | |
| Qdrant (exact) | 12.24 | 88 | 124 | - | - | 2000 of 2000 | |
| SQL Server 2025 | 66.77 | 14 | 12 | - | - | 2000 of 2000 | |
| SQL Server 2025 + DiskANN | 6.87 | 147 | 471 | 86.64 | - | 1995 of 2000 |
Engines are listed in alphabetical order. The table does not rank them.
A dash means the table has no figure there.
Full results
This report is printed as the run wrote it, except for any sentence a note above it says was left out. This page does not check the engine texts in it. The summary page's facts table gives the label of each fact it uses.
Run 2026-10-03T15:15:06Z (run-all). Command line:
~/tmp/gvblane-bench/src/GenericVectorBuilder.Bench/bin/Debug/net10.0/GenericVectorBuilder.Bench.dll run-all --pipeline adventureworks --limit 50000 --targets sql,qdrant,qdrant-hnsw,sql-diskann --queries random:200 --repo ~/ForClaude/GenericVectorBuilder
- Machine: bench-host, Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz (8 logical CPUs), 62.7 GiB RAM, GPU NVIDIA GeForce RTX 3060, 12288 MiB, Ubuntu 24.04.5 LTS, kernel 6.8.0-139-generic, .NET 10.0.12, load average at start 14.58 19.60 23.60
- Data: 50,000 vectors x 1024 dims, collection
gvbbench_adventureworks. Read READ-ONLY from GenericVectorBuilder.dbo.gvb_adventureworks in ChunkId order, vectors as native SqlVector<float>, 2.2 s. - Queries: random:200 (stored vectors picked with seed 20261003; each query's own row is left out of the exact answer and removed from every engine's hits). Top 10. Throughput at concurrency 1, 8 for 20 s each.
- Ground truth: brute force over every vector in memory, 1.2 s for all 200 queries.
| engine | index | load rows/s | p50 ms | p95 ms | p99 ms | QPS@1 | QPS@8 | recall@10 | RAM | disk |
|---|---|---|---|---|---|---|---|---|---|---|
| sql | exact VECTOR_DISTANCE cosine, no vector index (full scan) | 364 | 66.8 | 79.9 | 87.5 | 14.1 | 11.9 | 1.000 | 4.13 GiB | 398.64 MiB |
| qdrant | exact search (the builder's setting), HNSW m=16 ef_construct=100 built but not used | 2,755 | 12.2 | 14.5 | 16.2 | 88.4 | 124.2 | 1.000 | 6.19 GiB | 459.1 MiB |
| qdrant-hnsw | HNSW m=16 ef_construct=100, hnsw_ef=server default, cosine | 3,104 | 1.61 | 2.14 | 2.88 | 462.3 | 1466.6 | 0.999 | 6.23 GiB | 491.09 MiB |
| sql-diskann | DiskANN (preview) via VECTOR_SEARCH, cosine, build {"StartId":"30588", "L":"48", "M":"8", "R":"48"} | 699 | 6.87 | 10.3 | 11.3 | 146.7 | 471.4 | 0.998 | 4.13 GiB | 404.58 MiB |
Details per target
SQL Server 2025 sql
- Engine: Microsoft SQL Server 2025 (RTM-CU9) (KB5122048) - 17.0.5005.3 (X64) (always-on)
- Index: exact VECTOR_DISTANCE cosine, no vector index (full scan)
- Load: 50,000 rows in batches of 1000, 137.4 s of upserts (364 rows/s); count matched 0.0 s after the last upsert; index step n/a
- Search: 200 latency samples, target held 50,000 rows, 0 errors
- RAM: 4.13 GiB (whole SQL Server process); disk: 398.64 MiB (table and its indexes, reserved pages)
- WARNING: load average 8.02 15.31 21.42 (1/5/15 min) on 8 logical CPUs when searching began; the box was busy, so latency and QPS are inflated by other work.
- Benchmark copy gvbbench_adventureworks dropped afterwards.
Qdrant (exact) qdrant
- Engine: Qdrant 1.17.0 (systemd, local) (always-on)
- Index: exact search (the builder's setting), HNSW m=16 ef_construct=100 built but not used
- Load: 50,000 rows in batches of 1000, 18.2 s of upserts (2,755 rows/s); count matched 0.0 s after the last upsert; index step 7.0 s (status Green, 50,000 of 50,000 vectors in HNSW segments (4 segments))
- Search: 200 latency samples, target held 50,000 rows, 0 errors
- RAM: 6.19 GiB (whole Qdrant process, every collection); disk: 459.1 MiB (collection folder)
- WARNING: load average 9.12 13.81 20.38 (1/5/15 min) on 8 logical CPUs when searching began; the box was busy, so latency and QPS are inflated by other work.
- Benchmark copy gvbbench_adventureworks dropped afterwards.
Qdrant (HNSW) qdrant-hnsw
- Engine: Qdrant 1.17.0 (systemd, local) (always-on)
- Index: HNSW m=16 ef_construct=100, hnsw_ef=server default, cosine
- Load: 50,000 rows in batches of 1000, 16.1 s of upserts (3,104 rows/s); count matched 0.0 s after the last upsert; index step 4.0 s (status Green, 50,000 of 50,000 vectors in HNSW segments (4 segments))
- Search: 200 latency samples, target held 50,000 rows, 0 errors
- Exact mode: 200 queries, p50 10.9 ms, p95 13.4 ms, recall 1.000
- RAM: 6.23 GiB (whole Qdrant process, every collection); disk: 491.09 MiB (collection folder)
- WARNING: load average 8.99 12.93 19.63 (1/5/15 min) on 8 logical CPUs when searching began; the box was busy, so latency and QPS are inflated by other work.
- Benchmark copy gvbbench_adventureworks dropped afterwards.
SQL Server 2025 + DiskANN sql-diskann
- Engine: Microsoft SQL Server 2025 (RTM-CU9) (KB5122048) - 17.0.5005.3 (X64) (always-on)
- Index: DiskANN (preview) via VECTOR_SEARCH, cosine, build {"StartId":"30588", "L":"48", "M":"8", "R":"48"}
- Load: 50,000 rows in batches of 1000, 71.5 s of upserts (699 rows/s); count matched 0.0 s after the last upsert; index step 126.7 s (copied to dbo.gvb_gvbbench_adventureworks_ann (INT key) and built DiskANN, parameters {"StartId":"30588", "L":"48", "M":"8", "R":"48"})
- Search: 200 latency samples, target held 50,000 rows, 0 errors
- Exact mode: 200 queries, p50 86.6 ms, p95 130.5 ms, recall 1.000
- RAM: 4.13 GiB (whole SQL Server process); disk: 404.58 MiB (table and its indexes, reserved pages)
- WARNING: load average 8.72 10.19 16.95 (1/5/15 min) on 8 logical CPUs when searching began; the box was busy, so latency and QPS are inflated by other work.
- Benchmark copy gvbbench_adventureworks dropped afterwards.
Notes
- WARNING: load average 14.58 19.60 23.60 on 8 logical CPUs when the run started. Other work was competing for the CPU, so absolute latency and QPS are worse than this box can do; compare engines only within one run, and rerun on a quiet box before quoting numbers.
- Load rows/s counts only time inside each target's upsert calls: one writer, batches of 1000, rows already in memory, the collection dropped and created fresh first.
- Latency is client-side wall time around each search (network and driver included), one query at a time, after 20 warm-up queries; at least 200 samples (small query sets are repeated).
- QPS: N workers searching back to back for 20 s per level; completed searches divided by elapsed time.
- Recall@10: share of the exact top 10 (brute force in memory) that the engine returned. A hit whose exact similarity ties the 10th best (within 1e-5) also counts, because duplicate rows embed to identical vectors.
- RAM of always-on servers (SQL Server, Qdrant) is the whole process, including every other database or collection it serves; for compose engines it is docker stats of the engine's containers. Disk of compose engines is the whole engine data folder.
- The empty benchmark database GvbBench was dropped at the end.