SkewFuse: Unsupervised Confidence-Guided Rank Fusion for Multi-View Image Retrieval

In image retrieval systems, the use of multiple embedding models to capture diverse visual features often improves retrieval performance. However, traditional rank aggregation methods generally assume uniform reliability across retrieval models, neglecting the variability in model confidence across different queries. This paper introduces SkewFuse, an unsupervised rank fusion approach that dynamically weights retrieval results based on statistical skewness of similarity score distributions, effectively capturing query-specific confidence. We extend the Dowdall positional voting method by integrating these confidence weights, achieving adaptive and robust fusion. Evaluations conducted on the In-Shop Clothes Retrieval dataset demonstrate that SkewFuse consistently outperforms existing unsupervised and supervised fusion methods, notably improving early precision metrics, including a 4.7% relative gain in Recall@1 over the strongest baseline. SkewFuse offers an efficient, parameter-light solution ideal for real-world applications where top-ranked retrieval precision is critical.