Warm-up method as the runs recorded it

• Preparation, untimed, before any timed pass: a rehearsal of every pass type at its own concurrency for 30 s each, through the same code the passes use.
• Warm-up and settle check, untimed, right before every timed pass: the pass's own search at the pass's own number of searchers for at least 15 s and at least 20 searches (at most 120 s), read in windows of at least 2 s and 100 searches; then a 3 s trial of the same pass. The trial's figure (p50 with one searcher, QPS with several) must lie within 10% of the warm-up's settled figure (the median of its last 3 windows, which must agree within 5%); if not, the warm-up is extended once (at least 30 s, until its windows agree, at most 120 s), where an extension's windows agree when its level test passes: the older and the newer half of its latest windows, 5 to 10 windows a half, each half read as the pass reads it (the p50 of all its searches with one searcher, its searches per second with several), agree within 5%, judged on the windows that stopped it (an extension whose cap runs out with fewer than 10 windows is judged by its last 3 instead); then a second trial is taken, which must lie inside the range of the newer half's windows or within 10% of the newer half's figure, and the warm-up runs again before the pass. After the pass its own figure is held against the settled figure, within 10%. In the level test, the second trial and the hold, two one-searcher p50s no more than 0.1 ms apart agree whatever their percentage (below what this box resolves at one searcher). Each target's notes give every check, and a pass that still disagrees, or whose own figure did not hold, flags its target as unsettled. With machine control on, the check for a quiet box is made when the warm-up is announced, before it starts (a wait between the warm-up and the timed pass let the engine go cold), so by the time the clock opens that check is as old as the warm-up and the trial (about 18 s, more after an extension); each pass's conditions record that lead (quietCheckLeadSeconds). Failed untimed searches are counted per target (warmupErrors) and are not in the timed error counts.

Runs used

Runs dropped

(none)

Targets not in this report

(none)

Targets

Every column is what the runs recorded for the engine: its description, its index and search settings, its non-default engine settings and configuration files (when a run recorded them) and its durability statement. Runs that differ in any of them are refused for that target, never merged; a column that reads missing was not recorded by any run, so a setting changed there would not show.

Request speed on a small collection (524 vectors)

Measured end to end through each engine's .NET client; at this size it reflects per-request cost including the client library, not index scaling.

Engines in different bands never overlap: every run of an engine in a faster band beat every run of an engine in a slower band, and the medians on either side of a band boundary are at least 3% apart. Engines in one band are linked by overlapping slowest-to-fastest ranges or by neighboring medians less than 3% apart (an engine varies about 2% from run to run), so these runs do not separate them cleanly. Inside a band they are listed by median, and that order is not a ranking.

p50 latency, one search at a time (lower is faster)

QPS with 8 searchers at once (higher is faster)

QPS with one searcher (higher is faster)

Client CPU per search is the CPU time the test's .NET client itself used for each search, measured in the same pass as the figure beside it.

The engine notes column lists the notes under Per-engine notes and, after "flags:", every flag the engine carries (an unsettled engine, a spread above 15%, a shared core, a segment layout that differs between runs, and the rest). The number is still shown; read it with its flag, whose evidence is under Flags.

Per-engine notes

Per target: median [min, max]

Per run: p50 ms

Per run: QPS@1

Per run: QPS@8

Per run: QPS ratio 8/1

Per run: client CPU ms per search@1

Per run: client CPU ms per search@8

Rank per run: p50 (1 = lowest)

Rank per run: QPS@8 (1 = highest)

Rank per run: QPS@1 (1 = highest)

Paired per run: exact p50 / default p50

Exact / default summary

Recorded per run: order, passes, index state, durability

Flags

Notes

• Load rows/s is reported but not ranked and does not feed any comparison: each load wrote 524 rows, which is too few to separate engines from connection set-up and first-call cost.
• Runs were used together only when their build configuration, CPU governor, CPU partition, warm-up method (the warm-up time and count, the settle trial, the extension and the rehearsal as the runs recorded them), exact-mode seconds, seconds per level and every listed target's index description and search settings matched; runs that differed are listed under Runs dropped. Spread is flagged when the largest value is more than 1.15 times the smallest across runs.
• Engines in different bands never overlap: every run of an engine in a faster band beat every run of an engine in a slower band, and the medians on either side of a band boundary are at least 3% apart. Engines in one band are linked by overlapping slowest-to-fastest ranges or by neighboring medians less than 3% apart (an engine varies about 2% from run to run), so these runs do not separate them cleanly. Inside a band they are listed by median, and that order is not a ranking. Bands are drawn only from two runs or more.
• Runs of different commands (run-all against bench) are refused, never merged, and nothing is written. A target whose engine hosting differs between the runs (container against native) is refused for that target alone, never merged: it has no row and is named under "Targets not in this report" with the reason. The largest-group rule does not apply to either.
• A target whose engine description, recorded engine settings, configuration files (compose file and read-only mounted config, by SHA-256) or durability statement differs between the runs is refused for that target alone, like a different hosting: it has no row and is named under "Targets not in this report" with the differing values. A setting no run recorded cannot be compared and shows as missing.
• A segment layout that differs between the runs (the engine's own count after the load or after the searches) does not refuse or drop anything: the target stays in every table and carries the segment-layout-differs-between-runs flag, which names each layout and the runs that had it.
• Every target the runs hold that has no row in this report (left out of --targets, or refused for a different hosting or engine setup) is named under "Targets not in this report" with its reason (0 in this report), so no measured engine is left out unsaid.