Open-Vocabulary Product Discovery via Fusion Based Visual Retrieval in E-Commerce

In e-commerce, users often seek complementary products that appear together in the same scene, but existing visual search systems face scalability challenges. Traditional approaches rely on class-specific object detectors and supervised metric-learning models that require re-training and re-annotation for each new product category. Consequently, this reliance on manually labeled datasets limits adaptability in dynamic marketplace environments. We propose an end-to-end pipeline that addresses these constraints through open-vocabulary object detection combined with a novel freeze-weighted reciprocal rank fusion (FWRRF) retrieval strategy. Our system employs YOLO-World for prompt-driven product detection, enabling zero-shot recognition of unseen categories without additional training or annotations. The retrieval module extracts dual-view embeddings from both detected object crops and full scene images, then combines them using FWRRF, a lightweight fusion method that preserves high-precision object-centric results while incorporating contextual information through weighted reciprocal ranking. Evaluation on a curated living-room dataset demonstrates strong performance: 0.851 [email protected] for detection and Recall@3 = 0.580 for retrieval, surpassing single-view baselines and conventional fusion methods. This framework enables scalable, prompt-driven product discovery with continuous category expansion capabilities, eliminating costly retraining cycles in dynamic e-commerce environments.