Visual Recognition in Product Search

Ever seen a beautiful perfume bottle online (on TikTok or Instagram) through a review by an influencer or in a shop, but couldn't get a sense of its scent? As a perfume enthusiast, I've been there! That challenge inspired my latest project: a computer vision pipeline that provides deep insights without a single sniff. I wanted to build a system that goes beyond simple recognition. In the messy reality of social media posts and retail displays, perfume bottles are often surrounded by clutter. My pipeline aims to mimic human perception: first, finding the bottle; second, recognizing it; and finally, inferring its core characteristics like its fragrance family (or notes). Here's a look at how this end-to-end system works: Precise Object Detection: I leveraged YOLOv5 to train a model that expertly locates perfume bottles in diverse, real-world images, outputting exact bounding boxes. Fine-Grained Product Identification: The cropped bottle images are then fed into a fine-tuned ResNet50 classifier, capable of distinguishing specific products (e.g., "Chanel No. 5," "YSL Black Opium") even among similar-looking bottles. Fragrance Family Classification: The identified product is then mapped to its corresponding fragrance family (e.g., Floral, Woody, Oriental) using a second-stage ResNet classifier trained on curated scent metadata. To ensure this sophisticated backend was easily accessible, I architected the deployment in two key parts: The core inference logic is exposed via a Flask API, providing a robust and scalable way to integrate the computer vision models. The entire user experience, from image upload to results display, is powered by a Streamlit app, seamlessly hosted on Streamlit Cloud for global accessibility and ease of use. This project showcases how we can bridge the gap between clean datasets and real-world image complexity, leveraging modern deployment practices to deliver a tangible solution. It excites me because it touches on real-world problems in visual search and AI-powered retail, all while helping fellow perfume lovers explore new scents! PS: Still in demo phase so let me know about the bugs, what you think or if you have any feedback? Explore the live demo here: https://lnkd.in/gxp53uCK

Amazon Rufus has changed the rules In July 2024, Amazon launched an Ai assistant that doesn’t look for product keywords. → It sees them. Rufus reads the label inside your product photo. Understands the lifestyle image beside it. And when a shopper uploads a picture of something they want, it finds the product, fast. This isn’t futuristic. It’s happening now and it’s rewriting how Amazon product discovery works in 2025. Rufus doesn’t use filters or categories to make suggestions. It uses Visual AI (vision-language models + OCR ) to match behavior, context, and needs. This new tech made one thing clear: Great product visuals aren’t decoration. → They’re data. And 35% of Amazon’s revenue now comes from these visual-first systems. Rufus shows where discovery is heading → here’s how to start moving there now: 1. Design images for information → not just appeal → Use overlays (e.g., “Machine Washable”) to feed OCR 2. Build diverse photo sets → every image should answer a different customer question Use these tools: → Nosto – Visual discovery + behavior-based placement → Syte – “Shop the look” + visual similarity engine → Visenze – Contextual visual recommendations → Algolia Recommend – AI ranking with visual enrichment If your product discovery still starts with search bars and filters, you're ready for product discovery with Visual AI. P.S. Are your product visuals built to sell or just to impress?

Google just announced the visual fan-out technique‼️ Visual Search Fan-Out marks a significant leap in how AI interprets and interacts with images in search. Instead of simply identifying objects, the system now understands images in full context, combining visual details with natural language to deliver richer and more relevant results. ✅ Interactive visual exploration You can describe what you’re looking for in plain language, and the AI turns vague ideas into clear visual results. It learns from your follow-up questions to improve and refine the images it shows. ✅ Context-aware shopping Finding products is easier. Instead of using complicated filters, you just describe what you want—like style, fit, or color—and the AI shows matching items using up-to-date product data. ✅ Advanced image understanding The AI looks at the main subject of an image as well as smaller details and background elements. It combines all this information to give more accurate and relevant results. ✅ Flexible, multimodal input You can start your search with text, an image, or a photo. The AI blends these inputs to guide you to the most useful results.

🚀 Amazon has just taken visual search to another level. They’ve launched Lens Live - an AI-powered tool that lets shoppers point their phone at an item and instantly see real-time matches in a swipeable carousel. No more snapping, uploading, or scanning barcodes. What makes this bigger is the integration with Rufus, Amazon’s generative AI shopping assistant. That means shoppers don’t just see “similar items” - they get product summaries, comparisons, and answers to their questions in the moment. 🔎 Think about what this means: - Faster product discovery - Less friction from inspiration to purchase - AI shaping the buying decision inside Amazon’s ecosystem For brands, this isn’t just a feature update - it’s a reminder that Amazon is redefining how shoppers discover products. If your content, imagery, and product data aren’t optimised, you’re already behind.