redis had the lowest p50 latency: every other engine took at least 1.45 times as long, in each session; redis holds its data in memory.

Data: eshoponweb, 524 vectors of 1024 dimensions; 20 queries, top 10 hits each.

Machine: Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz, 8 logical CPUs, 62.7 GiB of RAM.

At 524 vectors each figure is the cost of one request through the benchmark's client for that engine, and does not show how an index scales.

Of the 19 engines, 8 are reached through the benchmark's own HttpClient REST code: elasticsearch, vespa, opensearch, chroma, milvus, typesense, clickhouse and weaviate.

Session v7: runs 20261006-130619-eshoponweb, 20261006-142724-eshoponweb and 20261006-154837-eshoponweb, started from 2026-10-06T13:06:19Z to 2026-10-06T15:48:37Z.

Session v8: runs 20261007-205106-eshoponweb, 20261007-221429-eshoponweb and 20261007-233551-eshoponweb, started from 2026-10-07T20:51:06Z to 2026-10-07T23:35:51Z.

The runs of v8 were measured by build 9b924200abc3.

This report was consolidated by build 9e5878ce5dc0, not by the build that measured the runs of v8.

The runs of v8 read a question file with the same SHA256 hash as design/bench-inputs/questions_golden.json.

The runs of v7 record no question-file hash, and truthNdcg reads 0.5181699774768911 in all 6 claim runs.

golden: 20 labelled questions from ~/ForClaude/evalkit/questions_golden.json, embedded with qwen3-emb-0.6b (cached in ~/gvb-data/bench-cache/golden-eshoponweb-68df42ca69efa079.json, no embedding calls)

p50 is the median latency of one searcher's searches, in ms; rows are in order of the median of 6 runs.

QPS@1 is searches per second of the same one-searcher pass as p50, so it is no second confirmation of the p50 order.

no sources recorded

QPS@8 is searches per second with 8 searchers at once.

clickhouse is shown and not ranked in this table: its timed eight-searcher pass was recorded NOT HELD in 3 of the 6 runs.

Exact p50 is the median latency of each engine's own exact mode, which is a different operation per engine; the why table's CPU figures come from the other passes.

no sources recorded

sql has no exact pass; its search fact says: no index used, exact scan by design.

qdrant has no exact pass; its search fact says: no index used, exact scan by design.

milvus has no exact pass in these runs.

no sources recorded

weaviate has no exact pass in these runs.

no sources recorded

chroma has no exact pass in these runs.

no sources recorded

Recall counts hits of the exact top 10 over 20 queries, 200 per run; it is printed, never ranked.

The recall hits of v7 are the recall of each run times 20 queries times 10 hits, rounded; those runs record no hit count.

clickhouse's recall hits differ between runs: v7 199, 193, 198; v8 199, 199, 199.

CPU per search is cost summed over every thread.

no sources recorded

It can exceed the time per search, so it is not a split of the latency.

no sources recorded

This test did not isolate causes.

no sources recorded

Costs are medians over the 3 runs of v8, and a cost is left blank unless every one of them recorded it.

At one searcher, client CPU per search ran from 0.303 ms (mongodb) to 4.985 ms (duckdb).

At one searcher, engine CPU per search ran from 0.258 ms (redis) to 8.246 ms (weaviate) where measured.

Hosting embedded is recorded for sqlitevec and duckdb: each runs inside the test's own process, so its client CPU per search includes the engine's own work.

sql-diskann recorded no setting for how hard a query searches.

An engine is shown ahead of another when, in each session separately, its slowest run beat the other's fastest run by at least 1.45 times; every other pair is not separated by this test.

The threshold is 45%: the smallest multiple of 5% at least 5% above the largest move in the basis, 35.82%, and never below 35%.

The basis holds the 12 runs of sessions v5, v6, v7 and v8, all with machine control on.

The clock was held in the runs of v7 and v8 and not in those of v5 and v6.

Basis moves are taken between runs in which the engine recorded the same setup: its engine text, index text, search settings, durability text, engine files, engine settings, image id and hosting.

no sources recorded

A field other than the engine text and the index text that one of two runs did not record is not compared.

no sources recorded

The largest move was 35.82%, the QPS@8 ratio of clickhouse/oracle between runs 20261006-142724-eshoponweb (seed 702) and 20261007-205106-eshoponweb (seed 801).

Each move is rounded up to the next basis point before it is printed: the largest, 35.815% unrounded, prints as 35.82%.

Those two runs differ in build.

Run 20261007-205106-eshoponweb: the timed eight-searcher pass of clickhouse read 336 QPS, 24% from the settled 418 QPS (limit 10%), and the run recorded it as NOT HELD.

The largest move includes a NOT HELD pass; with clickhouse left out of the basis, the largest move is 29.08%, the QPS@8 ratio of mongodb/oracle, and the threshold would be 35%.

p50, between basis runs of one recorded setup: largest moves 15.28% (redis) for one engine and 26.34% (mongodb/redis) for a pair.

p50, between those runs that also share turbo, uncore, warm-up and rehearsal: largest moves 12.02% (mongodb) for one engine and 18.88% (mongodb/redis) for a pair.

QPS@1, between basis runs of one recorded setup: largest moves 11.49% (mongodb) for one engine and 19.33% (mongodb/pgvector) for a pair.

QPS@1, between those runs that also share turbo, uncore, warm-up and rehearsal: largest moves 11.49% (mongodb) for one engine and 14.01% (mongodb/sqlitevec) for a pair.

QPS@8, between basis runs of one recorded setup: largest moves 27.36% (clickhouse) for one engine and 35.82% (clickhouse/oracle) for a pair.

QPS@8, between those runs that also share turbo, uncore, warm-up and rehearsal: largest moves 27.36% (clickhouse) for one engine and 35.82% (clickhouse/oracle) for a pair.

exact p50, between basis runs of one recorded setup: largest moves 20.23% (redis) for one engine and 19.66% (qdrant-hnsw/redis) for a pair.

exact p50, between those runs that also share turbo, uncore, warm-up and rehearsal: largest moves 7.74% (opensearch) for one engine and 8.41% (opensearch/sqlitevec) for a pair.

Changes of recorded setup left out of the basis moves: 4, in clickhouse, duckdb, mariadb and sqlitevec.

Each is listed with the field that changed, the move it hides and the threshold it would give.

no sources recorded

Between its v6 runs and its v7 and v8 runs, clickhouse recorded a different engine.

Counted across that change, the largest one-engine move is 47.36% (QPS@8) and the largest pair move 52.08% (clickhouse/redis QPS@8), and the threshold would be 60%.

The two runs of that one-engine move also differ in turbo, uncore, warmup, build and median MHz.

Between its v5 runs and its v6, v7 and v8 runs, duckdb recorded a different index.

Counted across that change, the largest one-engine move is 115.56% (QPS@8) and the largest pair move 144.95% (duckdb/oracle QPS@8), and the threshold would be 150%.

The two runs of that one-engine move also differ in warmup, rehearsal, build, median MHz and whether searchSettings was recorded.

Between its v5 runs and its v6, v7 and v8 runs, mariadb recorded a different durability.

Counted across that change, the largest one-engine move is 4.09% (exact p50) and the largest pair move 17.85% (mariadb/oracle QPS@8), and the threshold would be 45%.

The two runs of that one-engine move also differ in turbo, uncore, warmup, rehearsal, build, median MHz and whether searchSettings was recorded.

Between its v5 runs and its v6, v7 and v8 runs, sqlitevec recorded a different index.

Counted across that change, the largest one-engine move is 119.36% (QPS@8) and the largest pair move 155.08% (oracle/sqlitevec QPS@8), and the threshold would be 165%.

The two runs of that one-engine move also differ in warmup, rehearsal, build, median MHz and whether searchSettings was recorded.

vespa QPS@8, seed list 502 and 503, is left out of the basis (kind: method defect); design/verdicts/v5-verdict.txt, item 2, holds the words: The @8 pass ran first in runs 502 and 503.

elasticsearch QPS@8, seed list 503, is left out of the basis (kind: method defect); design/verdicts/v5-verdict.txt, item 2, holds the words: run 503, the one run where @8 was its first pass.

clickhouse every metric, seed list 501, 502 and 503, is left out of the basis (kind: unrecorded config change); design/verdicts/v6-verdict.md, item 1, holds the words: after the v5 runs ended at 04:38.

With the method defect rows kept in, the largest moves would be 74.42% for one engine and 95.64% for a pair, and the threshold 105%.

With the unrecorded config change rows kept in, the largest moves would be 27.36% for one engine and 35.82% for a pair, and the threshold 45%.

In the 6 runs without machine control, one engine moved up to 112.82% and a pair up to 148.82%; the threshold rests on runs with machine control on.

6 other runs of this pipeline are not in the basis; each is listed with its reason.

From v7 to v8, a ranked figure's session median moved 1.43% in the middle case and 4.66% at most (opensearch, exact p50).

The NOT HELD clickhouse QPS@8 cell is left out of these figures; its session median moved 13.36%, with v7 runs at 375 to 428 and v8 runs at 336 to 380.

6 orders held in v7 and not in v8, and 9 the other way; they are listed and not published.

20 unordered pairs sat between 1.35 and 1.45 times in each session, always in one direction; they are listed as close to the line.

1 ordered pair cleared 1.45 times by 25 bp or less in its lowest session; it is listed as on the line.

The sessions differ in build, day and outside load, and this test does not separate those from the engines' own drift.

The median outside load of a pass, per run, was 0.2095 to 0.2135 CPUs in v7 and 0.14 to 0.1435 in v8.

Both sessions ran with the clock held; drift under other conditions is not measured here.

no sources recorded

In v7, every run held the clock: turbo off, MSR 0x620 at 0x1e1e, top ratio 35 (nominally 3500 MHz); the median of the kernel's pass medians is 3492 MHz; the ceiling before was 3600 MHz.

The median rule: a pass is off when the median, over its engine CPUs or over its client CPUs, of each CPU's median MHz is more than 1% from the pinned 3500 MHz.

In v7, by the median rule, 0 of 156 passes were off the pinned clock and 0 had a CPU group not read.

In v8, every run held the clock: turbo off, MSR 0x620 at 0x1e1e, top ratio 35 (nominally 3500 MHz); the median of the kernel's pass medians is 3492 MHz; the ceiling before was 3600 MHz.

In v8, by the median rule, 0 of 156 passes were off the pinned clock and 0 had a CPU group not read.

Run 20261006-130619-eshoponweb flagged oracle's eight-searcher pass as off its clock by a mean; the medians read 3492 MHz on engine CPUs and 3492 MHz on client CPUs, within 1%.

Run 20261006-142724-eshoponweb flagged oracle's eight-searcher pass as off its clock by a mean; the medians read 3492 MHz on engine CPUs and 3492 MHz on client CPUs, within 1%.

Run 20261006-154837-eshoponweb flagged oracle's eight-searcher pass as off its clock by a mean; the medians read 3492 MHz on engine CPUs and 3492 MHz on client CPUs, within 1%.

Every claim run used the performance governor, with the client on CPUs 0-1,4-5.

The container engines clickhouse, vespa, oracle, elasticsearch, sql, pgvector, qdrant, opensearch, qdrant-hnsw, redis, milvus, mariadb, weaviate, mongodb, chroma, typesense and sql-diskann ran on CPUs 2-3,6-7.

The embedded engines sqlitevec and duckdb ran inside the client process on CPUs 0-1,4-5.

Every v8 run recorded boot a2f94313-eeda-460c-8474-8a9ee898f123 at 2026-10-03T15:53:39Z, before the first v7 run started at 2026-10-06T13:06:19Z, so v7 ran on that boot too.

Each of the 17 container targets ran one image id, equal to its pin, in every v8 run; the ids are in the images table.

Each of those images was last tagged on this box before the first v7 run started at 2026-10-06T13:06:19Z.

redis holds its data in memory: its saved docs page says 'Redis is an in-memory but persistent on disk database'; its compose file sets save "300 1" and appendonly no.

Each engine's durability text, as run 20261007-205106-eshoponweb recorded it, follows clause by clause with how each clause is backed; the program wrote the text, and its figures were not re-derived for this report.

Strace logs of measurements cited in those texts are saved for clickhouse, milvus, mongodb, qdrant and qdrant-hnsw, in design/engine-docs/durability-logs-2026-10-04, each listed with its hash in SHA256SUMS.

No log is saved for the measurements cited in the durability texts of oracle, pgvector, sqlitevec, redis, weaviate, chroma, typesense and duckdb.

clickhouse: its recorded durability text, clause by clause, with how each clause is backed.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

Acknowledged inserts are not fsynced. The sink creates plain MergeTree tables, so the MergeTree settings are the defaults: fsync_after_insert 0 and fsync_part_directory 0 (system.merge_tree_settings), and async_insert 1 with wait_for_async_insert 1 is the server default for the gvb login (system.settings), so an insert is acknowledged once its part is written, not once it is on disk.

Typed in the program's own text; it cites a measurement that these runs did not repeat, and no saved source backs this clause:

no sources recorded

Measured 2026-10-04 with strace over 30 acknowledged single-row inserts (30 Ok rows in system.asynchronous_insert_log):

Backed by a saved log or measurement, not read back from the engine in these runs:

zero fsync, fdatasync or sync_file_range calls,

while the control, a table created with fsync_after_insert=1, made 61 fdatasync calls

Typed in the program's own text; it cites a measurement that these runs did not repeat, and no saved source backs this clause:

no sources recorded

for 5 inserts.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

A host power loss or kernel crash can lose acknowledged rows that are still in the OS page cache; a ClickHouse process crash alone should not (inferred, not tested).

vespa: its recorded durability text, clause by clause, with how each clause is backed.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

Vespa's transaction log server fsyncs after each commit: searchlib.translogserver usefsync=true, and an operation is searchable when it is acknowledged: proton documentdb visibilitydelay=0. Neither is overridden by this sink or by the services.xml it deploys;

Described by a test saved in this repository, which these runs did not run:

VespaReadinessTests reads both from the live config server.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

After a crash the node replays the transaction log over its last flushed data and rebuilds the in-memory HNSW graph. By those settings an acknowledged write survives a crash; this is read from the settings, not shown by pulling power.

Set by a line of code or a compose file saved in this repository, not read back from the engine:

One node and min-redundancy 1,

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

so a lost disk loses the data.

oracle: its recorded durability text, clause by clause, with how each clause is backed.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

By these settings a committed row should survive a crash or power loss. The sink uses a plain COMMIT and commit_logging, commit_wait and commit_write are unset in the database, so every commit waits for its redo to be written.

Typed in the program's own text; it cites a measurement that these runs did not repeat, and no saved source backs this clause:

no sources recorded

Measured 2026-10-04: 50 separate client commits raised V$SYSSTAT 'redo synch writes' by 55 (the 50 commits plus the CREATE and DROP of the probe table), and the log writer and the datafile writer hold their files open with O_DSYNC (open flags 02110002, filesystemio_options none).

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

The database runs NOARCHIVELOG (V$DATABASE.LOG_MODE), so redo serves crash recovery only and there is no point-in-time restore.

Set by a line of code or a compose file saved in this repository, not read back from the engine:

The HNSW graph lives in the 768 MB vector memory pool (oracle-init/01-vector-memory.sh)

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

and is not the durable copy; the table is.

Not tested by cutting power; whether the disk's own write cache reaches the media was not checked.

Read from the running engine or the machine by the benchmark's own code, as the cited line shows:

CPU: Oracle Free caps itself at 2 CPUs (cpu_count 2 in V$PARAMETER, edition FREE in V$INSTANCE, 8 host CPUs in V$OSSTAT NUM_CPUS;

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

the 2 CPU thread limit is Oracle's documented Free edition limit).

elasticsearch: its recorded durability text, clause by clause, with how each clause is backed.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

Every acknowledged bulk request is fsynced to the translog before the answer: index.translog.durability=request, the Elasticsearch default, which neither this sink nor elasticsearch.compose.yaml overrides

Described by a test saved in this repository, which these runs did not run:

(ElasticsearchReadinessTests reads it back from the live index).

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

By that setting a process crash or power loss loses no acknowledged write; this is read from the setting, not shown by pulling power.

Set by a line of code or a compose file saved in this repository, not read back from the engine:

One node and no replicas,

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

so a lost disk loses the data.

sql: its recorded durability text, clause by clause, with how each clause is backed.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

A commit returns after its transaction-log records are written to disk (SQL Server write-ahead logging;

Read from the running engine or the machine by the benchmark's own code, as the cited line shows:

delayed durability is DISABLED);

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

a database created here copies the model database:

Read from the running engine or the machine by the benchmark's own code, as the cited line shows:

recovery model FULL, page_verify CHECKSUM; no global trace flags are enabled;

mssql.conf of container gvb-mssql (/var/opt/mssql/mssql.conf, on the host at ~/gvb-data/engines/mssql/mssql.conf) does not exist, so it sets: nothing; it has no [control] or [traceflag] entry,

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

so SQL Server's own Linux defaults for flushing writes apply (Microsoft's Linux performance guide names trace flag 3982 as that default; read from the guide, not tested here).

Not tested by cutting power; whether the disk's own write cache reaches the media was not checked.

Read from the running engine or the machine by the benchmark's own code, as the cited line shows:

Container settings from its environment (names only): MSSQL_AGENT_ENABLED, MSSQL_MEMORY_LIMIT_MB, MSSQL_PID, MSSQL_RPC_PORT, MSSQL_SA_PASSWORD.

pgvector: its recorded durability text, clause by clause, with how each clause is backed.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

fsync on, synchronous_commit on, full_page_writes on, wal_sync_method fdatasync (PostgreSQL defaults;

Set by a line of code or a compose file saved in this repository, not read back from the engine:

pgvector.compose.yaml sets only shared_buffers, maintenance_work_mem and max_wal_size):

Typed in the program's own text; it cites a measurement that these runs did not repeat, and no saved source backs this clause:

no sources recorded

every commit is flushed to the write-ahead log before it returns and HNSW index changes are WAL-logged, so a crash loses no committed row (max_wal_size 4GB only spaces out checkpoints; measured with docker kill, which is SIGKILL, right after a 2,000-vector load and a restart: all 2,000 rows were there and the HNSW index was valid and used by the default search)

sqlitevec: its recorded durability text, clause by clause, with how each clause is backed.

Set by a line of code or a compose file saved in this repository; the v8 engine settings also list journal_mode, read from the running engine:

PRAGMA journal_mode=WAL and PRAGMA synchronous=NORMAL (set by this sink when it opens the file):

Typed in the program's own text; it cites a measurement that these runs did not repeat, and no saved source backs this clause:

no sources recorded

each commit is appended to the -wal file and handed to the operating system, but the WAL is fsynced only when SQLite checkpoints it (measured with strace: 40 single-record commits caused 3 WAL fsyncs), so a crash of the process loses nothing (measured with kill -9 and a reopen: all 50, 300 and 2,000 rows were there), while an operating-system crash or power cut can lose the newest commits since the last checkpoint and leaves the database consistent

qdrant: its recorded durability text, clause by clause, with how each clause is backed.

Backed by a saved log or measurement, not read back from the engine in these runs:

What was measured (strace -f on the Qdrant server process, 30 single-point REST upserts per setting, every sync call timed against the request that caused it): with wait=true each upsert had exactly one msync(MS_SYNC) of the write-ahead-log segment inside the request, before the reply (30 of 30);

with wait=false the 30 upserts were acknowledged within 0.26 s with no sync call, and the first WAL msync (one call covering all 30 records) came 2.7 s after the last reply, followed by the segment-file flushes.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

So the log is flushed to disk under both settings, but with wait=false the flush comes after the acknowledgement: an operating-system crash or power cut in that gap loses acknowledged writes, while a crash of the Qdrant process alone should not, because the bytes are already in the kernel's page cache (inferred, not tested).

Set by a line of code or a compose file saved in this repository, not read back from the engine:

Every upsert here is sent with wait=true in batches of 256 over gRPC;

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

the strace used one point per request, so one flush per batch is inferred, not measured.

Read from the running engine or the machine by the benchmark's own code, as the cited line shows:

Segment files are flushed every 5 s; log segments 32 MB, 0 created ahead. /qdrant/config/config.yaml in container gvb-qdrant (the image's own file; its storage keys are listed) sets: storage.collection.quantization = null, storage.collection.replication_factor = 1, storage.collection.vectors.on_disk = null, storage.collection.write_consistency_factor = 1, storage.hnsw_index.ef_construct = 100, storage.hnsw_index.full_scan_threshold_kb = 10000, storage.hnsw_index.m = 16, storage.hnsw_index.max_indexing_threads = 0, storage.hnsw_index.on_disk = false, storage.hnsw_index.payload_m = null, storage.max_collections = null, storage.node_type = Normal, storage.on_disk_payload = true, storage.optimizers.default_segment_number = 0, storage.optimizers.deleted_threshold = 0.2, storage.optimizers.flush_interval_sec = 5, storage.optimizers.indexing_threshold_kb = 10000, storage.optimizers.max_optimization_threads = null, storage.optimizers.max_segment_size_kb = null, storage.optimizers.vacuum_min_vector_number = 1000, storage.performance.max_search_threads = 0, storage.performance.optimizer_cpu_budget = 0, storage.performance.update_rate_limit = null, storage.shard_transfer_method = null, storage.snapshots_config.snapshots_storage = local, storage.snapshots_path = ./snapshots, storage.storage_path = ./storage, storage.temp_path = null, storage.update_concurrency = null, storage.wal.wal_capacity_mb = 32, storage.wal.wal_segments_ahead = 0; no QDRANT__ environment overrides in the server process.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

Not tested by cutting power; whether the disk's own write cache reaches the media was not checked.

opensearch: its recorded durability text, clause by clause, with how each clause is backed.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

Every acknowledged bulk request is fsynced to the translog before the answer: index.translog.durability=request, the OpenSearch default, which neither this sink nor opensearch.compose.yaml overrides

Described by a test saved in this repository, which these runs did not run:

(OpenSearchReadinessTests reads it back from the live index).

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

By that setting a process crash or power loss loses no acknowledged write; this is read from the setting, not shown by pulling power.

Set by a line of code or a compose file saved in this repository, not read back from the engine:

One node and no replicas,

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

so a lost disk loses the data.

qdrant-hnsw: its recorded durability text, clause by clause, with how each clause is backed.

Backed by a saved log or measurement, not read back from the engine in these runs:

What was measured (strace -f on the Qdrant server process, 30 single-point REST upserts per setting, every sync call timed against the request that caused it): with wait=true each upsert had exactly one msync(MS_SYNC) of the write-ahead-log segment inside the request, before the reply (30 of 30);

with wait=false the 30 upserts were acknowledged within 0.26 s with no sync call, and the first WAL msync (one call covering all 30 records) came 2.7 s after the last reply, followed by the segment-file flushes.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

So the log is flushed to disk under both settings, but with wait=false the flush comes after the acknowledgement: an operating-system crash or power cut in that gap loses acknowledged writes, while a crash of the Qdrant process alone should not, because the bytes are already in the kernel's page cache (inferred, not tested).

Set by a line of code or a compose file saved in this repository, not read back from the engine:

Every upsert here is sent with wait=true in batches of 256 over gRPC;

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

the strace used one point per request, so one flush per batch is inferred, not measured.

Read from the running engine or the machine by the benchmark's own code, as the cited line shows:

Segment files are flushed every 5 s; log segments 32 MB, 0 created ahead. /qdrant/config/config.yaml in container gvb-qdrant (the image's own file; its storage keys are listed) sets: storage.collection.quantization = null, storage.collection.replication_factor = 1, storage.collection.vectors.on_disk = null, storage.collection.write_consistency_factor = 1, storage.hnsw_index.ef_construct = 100, storage.hnsw_index.full_scan_threshold_kb = 10000, storage.hnsw_index.m = 16, storage.hnsw_index.max_indexing_threads = 0, storage.hnsw_index.on_disk = false, storage.hnsw_index.payload_m = null, storage.max_collections = null, storage.node_type = Normal, storage.on_disk_payload = true, storage.optimizers.default_segment_number = 0, storage.optimizers.deleted_threshold = 0.2, storage.optimizers.flush_interval_sec = 5, storage.optimizers.indexing_threshold_kb = 10000, storage.optimizers.max_optimization_threads = null, storage.optimizers.max_segment_size_kb = null, storage.optimizers.vacuum_min_vector_number = 1000, storage.performance.max_search_threads = 0, storage.performance.optimizer_cpu_budget = 0, storage.performance.update_rate_limit = null, storage.shard_transfer_method = null, storage.snapshots_config.snapshots_storage = local, storage.snapshots_path = ./snapshots, storage.storage_path = ./storage, storage.temp_path = null, storage.update_concurrency = null, storage.wal.wal_capacity_mb = 32, storage.wal.wal_segments_ahead = 0; no QDRANT__ environment overrides in the server process.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

Not tested by cutting power; whether the disk's own write cache reaches the media was not checked.

redis: its recorded durability text, clause by clause, with how each clause is backed.

Set by a line of code or a compose file saved in this repository; the v8 engine settings also list appendonly, read from the running engine:

save "300 1" and appendonly no (redis.compose.yaml):

Typed in the program's own text; it cites a measurement that these runs did not repeat, and no saved source backs this clause:

no sources recorded

an RDB snapshot is written every 5 minutes if at least one key changed, and on a clean stop; there is no append-only log, so a crash loses every write since the last snapshot (up to 5 minutes plus the time a snapshot takes; measured with docker kill, which is SIGKILL, right after a 2,000-vector load: none of the 2,000 hashes were there after the restart, and the restart loaded an older snapshot that still held keys of a collection that had been dropped since)

milvus: its recorded durability text, clause by clause, with how each clause is backed.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

Writes are not fsynced. Standalone uses the default message queue rocksmq (RocksDB, /var/lib/milvus/rdb_data, mq.type default) and flushed segments go to local-disk object storage

Set by a line of code or a compose file saved in this repository, not read back from the engine:

(COMMON_STORAGETYPE=local in milvus.compose.yaml).

Backed by a saved log or measurement, not read back from the engine in these runs:

Measured 2026-10-04 with strace over 30 acknowledged single-row upserts and one flush: fdatasync ran only on the embedded etcd files (member/wal, member/snap/db), never on rdb_data or the segment files.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

A host power loss or kernel crash can lose acknowledged rows that are still in the OS page cache; a Milvus process crash alone should not (inferred, not tested). Metadata in embedded etcd is fdatasynced on every commit.

mariadb: its recorded durability text, clause by clause, with how each clause is backed.

Backed by a saved log or measurement; the v8 engine settings also list innodb_flush_log_at_trx_commit, read from the running engine:

innodb_flush_log_at_trx_commit=2 (set in mariadb-bench.compose.yaml for the benchmark's own container gvbbench-mariadb, and in mariadb.compose.yaml for the daily gvb-mariadb, which has the same server settings):

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

the InnoDB redo log is written to the operating system at every commit but fsynced about once a second, so a crash of the mariadbd process loses nothing, while an operating-system crash or power cut can lose the last second of commits;

Backed by a saved log or measurement, not read back from the engine in these runs:

innodb_doublewrite is on

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

(default),

Backed by a saved log or measurement, not read back from the engine in these runs:

the binary log is off,

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

and the vector graph is an InnoDB table under the same log

Backed by a saved log or measurement, not read back from the engine in these runs:

(settings read from the running server with SHOW VARIABLES;

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

the crash behaviour is InnoDB's documented behaviour for this setting and was not tested on either container)

weaviate: its recorded durability text, clause by clause, with how each clause is backed.

The recorded weaviate text says 'sets no persistence variable'; deploy/engines/weaviate.compose.yaml sets PERSISTENCE_DATA_PATH. That clause is not printed.

Typed in the program's own text; it cites a measurement that these runs did not repeat, and no saved source backs this clause:

no sources recorded

every object is appended to the LSM write-ahead log with a plain write and no fsync, and the log is fsynced only when its memtable is flushed, 60 seconds after the last write (PERSISTENCE_MEMTABLES_FLUSH_IDLE_AFTER_SECONDS default; measured with strace: 2,080 writes into the objects log during a 2,000-object load, the first fsync 60 s after the last write), so a power cut can lose the last minute of writes; the HNSW commit log is buffered inside the process (77 writes for 2,000 vectors), so a killed process loses the newest vectors from the vector index while their objects survive (measured with docker kill, which is SIGKILL, one second after the load and a restart, two runs each: 0 to 1 of 50, 264 to 267 of 300 and 1,992 to 1,997 of 2,000 stored objects were still reachable through a vector search)

mongodb: its recorded durability text, clause by clause, with how each clause is backed.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

Acknowledged writes are journaled before the acknowledgement. The sink sets no write concern, so the server default applies: getDefaultRWConcern gives w majority and the one-member replica set (--replSet gvbmongo in the image) has writeConcernMajorityJournalDefault true, with WiredTiger journaling on (journalCommitInterval 100 ms is only the interval for unacknowledged work).

Typed in the program's own text; it cites a measurement that these runs did not repeat, and no saved source backs this clause:

no sources recorded

Measured 2026-10-04: 30 acknowledged single-document inserts raised WiredTiger 'log sync operations' by 30 and

Backed by a saved log or measurement, not read back from the engine in these runs:

strace showed fdatasync on /data/db/journal/WiredTigerLog files.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

A crash loses no acknowledged write. mongot's search index is not part of that promise: it follows the collection asynchronously and is rebuilt from it.

chroma: its recorded durability text, clause by clause, with how each clause is backed.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

SQLite rollback journal with its default synchronous=FULL under the data directory

Set by a line of code or a compose file saved in this repository, not read back from the engine:

(chroma.compose.yaml sets IS_PERSISTENT=1 and PERSIST_DIRECTORY=/data and no sync setting):

Typed in the program's own text; it cites a measurement that these runs did not repeat, and no saved source backs this clause:

no sources recorded

every write is committed to chroma.sqlite3 with fsync of the journal, the directory and the database file before the call returns (strace: 15 database fsyncs and 38 journal fsyncs for one create, four 500-vector upserts and one delete), and the HNSW files are written every sync_threshold=1000 vectors and rebuilt from the SQLite log after a crash; measured with kill -9 and a restart: all 50, 300 and 2,000 vectors were still stored and searchable

typesense: its recorded durability text, clause by clause, with how each clause is backed.

Typed in the program's own text; it cites a measurement that these runs did not repeat, and no saved source backs this clause:

no sources recorded

Every acknowledged write is appended to Typesense's raft log and fsynced before the answer: braft raft_sync=true with raft_sync_policy=0 (sync immediately), read from the running container's brpc /flags page on 2026-10-04. A restart replays the log from the last snapshot

Set by a line of code or a compose file saved in this repository, not read back from the engine:

(typesense.compose.yaml sets TYPESENSE_SNAPSHOT_INTERVAL_SECONDS=300)

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

and rebuilds the in-memory HNSW graph, so by those settings a crash loses no acknowledged write, but the restart is slow after a big load. This is read from the settings, not shown by pulling power. One node and no replicas, so a lost disk loses the data.

sql-diskann: its recorded durability text, clause by clause, with how each clause is backed.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

A commit returns after its transaction-log records are written to disk (SQL Server write-ahead logging;

Read from the running engine or the machine by the benchmark's own code, as the cited line shows:

delayed durability is DISABLED);

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

a database created here copies the model database:

Read from the running engine or the machine by the benchmark's own code, as the cited line shows:

recovery model FULL, page_verify CHECKSUM; no global trace flags are enabled;

mssql.conf of container gvb-mssql (/var/opt/mssql/mssql.conf, on the host at ~/gvb-data/engines/mssql/mssql.conf) does not exist, so it sets: nothing; it has no [control] or [traceflag] entry,

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

so SQL Server's own Linux defaults for flushing writes apply (Microsoft's Linux performance guide names trace flag 3982 as that default; read from the guide, not tested here).

Not tested by cutting power; whether the disk's own write cache reaches the media was not checked.

Read from the running engine or the machine by the benchmark's own code, as the cited line shows:

Container settings from its environment (names only): MSSQL_AGENT_ENABLED, MSSQL_MEMORY_LIMIT_MB, MSSQL_PID, MSSQL_RPC_PORT, MSSQL_SA_PASSWORD.

duckdb: its recorded durability text, clause by clause, with how each clause is backed.

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

DuckDB's write-ahead log is fsynced at every commit and no DuckDbSink setting changes that

Set by a line of code or a compose file saved in this repository; the v8 engine settings also list checkpoint_threshold, read from the running engine:

(checkpoint_threshold=256MB

Typed in the program's own text; it cites a measurement that these runs did not repeat, and no saved source backs this clause:

no sources recorded

only spaces out the checkpoints that write the database file; measured with strace: 42 WAL fsyncs for 40 single-record commits plus 2 setup statements), so an operating-system crash or power cut loses no committed row; measured with kill -9 right after the last commit and a reopen: all 50, 300 and 2,000 rows and an HNSW index that counted the same number were recovered from the log; not tested and documented by DuckDB: the HNSW index is file-backed only through hnsw_enable_experimental_persistence = true,

Set by a line of code or a compose file saved in this repository, not read back from the engine:

which this sink turns on,

Typed in the program's own text; not read back, measured or backed by a saved source:

no sources recorded

and WAL recovery for such custom indexes is not complete, so a crash during a checkpoint or a later commit can damage the index while the rows survive

The engine settings table lists what run 20261007-205106-eshoponweb read from each running engine or set from a repository file, as each row's how column says.

qdrant's measured search fact says: no index used, exact scan by design.

oracle's recorded index text says: Oracle Free caps itself at 2 CPUs.

elasticsearch's measured search fact says: no HNSW graph, searches scan all 524 vectors.

sqlitevec's measured search fact says: no index, exact scan by design.

sql's measured search fact says: no index used, exact scan by design.

mongodb's search fact is the engine's own report; its segments held at most 278 vectors, below the 1,043 at which elasticsearch's recorded text says a segment gets a graph, and no graph was checked.

Run 20261006-130619-eshoponweb: clickhouse's whole engine data folder was recorded as 4.21 GiB, against 99.17 KiB before; the run's page prints a correction to its recorded "added by this load" figure.

Run 20261006-142724-eshoponweb: clickhouse's whole engine data folder was recorded as 6.79 GiB, against 4.22 GiB before; the run's page prints a correction to its recorded "added by this load" figure.

Run 20261006-154837-eshoponweb: clickhouse's whole engine data folder was recorded as 6.79 GiB, against 6.79 GiB before; the run's page prints a correction to its recorded "added by this load" figure.

Run 20261007-205106-eshoponweb: clickhouse's data folder held 4559215474 bytes at its start, 14691380 bytes after its system log tables were truncated and it stopped changing, and 5699801237 bytes at its end.

Run 20261007-205106-eshoponweb: the reset truncated 9 MergeTree log tables, whose active bytes by clickhouse's own count were 2.09 GiB before and 0 B after.

Run 20261007-221429-eshoponweb: clickhouse's data folder held 5707585756 bytes at its start, 2917655627 bytes after its system log tables were truncated and it stopped changing, and 7408848771 bytes at its end.

Run 20261007-221429-eshoponweb: the reset truncated 9 MergeTree log tables, whose active bytes by clickhouse's own count were 183.97 KiB before and 0 B after.

Run 20261007-233551-eshoponweb: clickhouse's data folder held 7426117183 bytes at its start, 2246745378 bytes after its system log tables were truncated and it stopped changing, and 7261702692 bytes at its end.

Run 20261007-233551-eshoponweb: the reset truncated 9 MergeTree log tables, whose active bytes by clickhouse's own count were 185.06 KiB before and 0 B after.

Segment layouts differed between runs for mongodb; each run's layout is in its row flags.

no sources recorded

An observer process ran beside the v8 runs; its own CPU, at most 0.138 CPUs in any one pass, counts as outside load, and its summary is observer/summary.json beside this report.

By APERF and MPERF, the largest deviation of one CPU in a pass was 0.66%: oracle's eight-searcher pass on CPU 7, at 3477.0 MHz against the pin of 3500 MHz.

The observer read MSR 0x620 as 0x1e1e.

oracle's eight-searcher pass ran 0.27% to 0.66% under the pin on every CPU in all 3 runs of v8; no other pass was more than 0.04% from it, and why is not known.

Run 20261007-205106-eshoponweb: oracle's eight-searcher pass averaged 3085.5 MHz on the engine CPUs and 3189.8 MHz on the client CPUs, as in v7; a mean rule flags it and the median rule does not.

Run 20261007-221429-eshoponweb: oracle's eight-searcher pass averaged 3206.2 MHz on the engine CPUs and 3299.0 MHz on the client CPUs, as in v7; a mean rule flags it and the median rule does not.

Run 20261007-233551-eshoponweb: oracle's eight-searcher pass averaged 3063.0 MHz on the engine CPUs and 3189.2 MHz on the client CPUs, as in v7; a mean rule flags it and the median rule does not.

The observer was not pinned: in the v8 runs 34.8, 33.8 and 33.5 percent of its resident-thread ticks were seen on engine CPUs 2-3,6-7.

Averaged over a whole run, the observer's CPU by cgroup was at most 0.0755 CPUs in any of the v8 runs.

The observer saw a systemd timer fire during 14 timed passes of the v8 runs.

An observer process ran beside the v7 runs; its own CPU, at most 0.123 CPUs in any one pass, counts as outside load, and its summary is observer/summary-v7.json beside this report.

By APERF and MPERF, the largest deviation of one CPU in a pass was 0.73%: oracle's eight-searcher pass on CPU 2, at 3474.3 MHz against the pin of 3500 MHz.

The observer read MSR 0x620 as 0x1e1e.

oracle's eight-searcher pass ran 0.41% to 0.73% under the pin on every CPU in all 3 runs of v7; no other pass was more than 0.05% from it, and why is not known.

The observer summary covers systemd timers for no v7 run.

CPU idle states were recorded and left as found: driver intel_idle, governor menu.

Outside load is busy CPU less this client's and the followed engine cgroups'; dockerd's cgroup is one of them for a compose engine, so its CPU counts as benchmark work, not outside load.

The runs of v7 did not record how dockerd's CPU was counted.

no sources recorded

engine CPU per search is the CPU time (cpu.stat usage_usec) of the cgroups each target's turn followed (conditions.engines[].cgroups, dockerd's left out), read with every 250 ms sample, interpolated to the pass's timed window and divided by its completed searches; engineCpusBusy is that CPU time over the window's length. A followed cgroup counts every process in it. dockerd's cgroup (/sys/fs/cgroup/system.slice/docker.service, holding docker-proxy, dockerd, for every container on the box) is left out of engine CPU per search and given as dockerdCpuMsPerSearch; it is subtracted from outside load as benchmark work during the turns of oracle, mongodb, redis, typesense, clickhouse, sql, opensearch, milvus, sql-diskann, qdrant-hnsw, weaviate, pgvector, vespa, elasticsearch, mariadb, chroma, qdrant, and counts as outside load during the turns of duckdb, sqlitevec. containerd-shim runs in /sys/fs/cgroup/system.slice/containerd.service, which is in neither engine figure and counts as outside load. An embedded engine runs in this client process, so its CPU is in clientCpuMsPerSearch and its engine figure is null with the reason; a target whose engine has no cgroup followed, or whose containers could not all be pinned, also gets null with the reason, never 0.

The test OutsideLoad_IsTheV7AccountingBitForBit holds the formula that turns CPU counters into outside load equal to the formula of v7, on fixed counters; it does not make the measured load of two sessions equal.

Every order here was measured at the search settings each run recorded for the engine, listed in the why section; this report makes no claim at other settings.

no sources recorded

A route each run recorded in conditions.connections reads: container address on its Docker network, not the published port (no docker-proxy).

A route each run recorded in conditions.connections reads: in this process (embedded), no network.