Index
Preprint · version 2 · 3 Nov 2026 · CC BY 4.0

The 1/k Ceiling in Reciprocal Rank Fusion: A Zero-Overlap Test for Hybrid Retrieval

Benjamin Meadows · Independent Researcher

pdf doi 10.5281/zenodo.23920744 source orcid

Abstract

Hybrid search runs a keyword retriever and a dense retriever over the same corpus and fuses their ranked lists, often with reciprocal rank fusion (RRF). If the keyword retriever silently returns nothing, the fused list still looks full, because the dense retriever fills it. This paper shows that, at per-retriever depth K ≤ k, the maximum fused score reveals whether the two lists shared any document. With ranks starting at zero, RRF gives a document 1/(k+r) from each retriever that ranks it at position r. With per-retriever depth K ≤ k, a document returned by two retrievers always scores above 1/k, and a document returned by one scores at most 1/k. So a result list whose maximum fused score is 1/k, a list at the ceiling, contains no document that both retrievers returned. The test measures overlap, not health: two nonempty lists that share nothing produce the same ceiling as an empty list, and per-retriever row counts tell the two cases apart. I describe a production system whose keyword query builder quoted every token of an SQLite FTS5 query, which FTS5 reads as an AND over all of them. Replayed on ten real queries, the builder returned nothing for every conversational one, and the fused list was at the ceiling on all ten. On six BEIR collections with three encoders, across 22 362 hybrid query runs at K = 100 > k = 60, the buggy builder returns nothing for 50.8 to 98.0% of queries on five collections, and the ceiling missed none of those empty lists, as the derivation requires with two retrievers and a dense list that was never empty. Across both builders, the fused list was also at the ceiling in 174 query runs where the keyword list was nonempty but shared nothing with the dense list, mostly short lists on the larger corpora, and in two runs under the repaired builder where the lists shared one document at deep ranks, the case K > k leaves open. On the sixth collection, ArguAna, each query is itself a corpus document, so the buggy keyword list is never empty and almost never relevant, and the ceiling never appears. Repairing the keyword retriever raises overlap in every collection-encoder pair. At nDCG@10 the repaired hybrid beats the dense retriever alone in 5 of 18 pairs and loses to it in 13; two of the gains and five of the losses are significant under a paired randomization test with Holm's correction.

Code, results, per-query diagnostics and this paper's source: https://github.com/TheBenMeadows/dead-retriever

Version 2 rewrites version 1 (12 Oct 2026, doi 10.5281/zenodo.23331139, titled “A Score Ceiling in Reciprocal Rank Fusion Detects Zero Overlap, and Found a Dead Retriever”): six BEIR collections and three encoders instead of two and two, paired significance tests, and a corrected evaluation protocol.

Cite

@misc{meadows_2026_23920744,
  author    = {Meadows, Benjamin},
  title     = {The 1/k Ceiling in Reciprocal Rank Fusion: A Zero-Overlap Test for Hybrid Retrieval},
  month     = nov,
  year      = 2026,
  publisher = {Zenodo},
  version   = {2},
  doi       = {10.5281/zenodo.23920744},
  url       = {https://doi.org/10.5281/zenodo.23920744},
}