CRO Lab - Image-based product recommendations

Monday, 17 January 2022

We know replicating the in-store experience online isn’t easy. Being able to browse through hangers of clothes, see jewellery up close with your own eyes, touch furniture with your own hands or sit on a sofa is just impossible to reproduce through ecommerce, but there’s more and more tools to help bring your products to life and give users greater help to purchase online. Image-based product recommendations are one such option.

When a consumer visits a physical store, they can see everything with their own eyes, taking in every colour, style, option and feel of whatever it is they are looking at. They say a picture paints a thousand words and people are often drawn to things they like the look of, particularly in the world of fashion, jewellery, furniture, homeware etc… So why not harness that power to show your customers other things they might like the look of and increase your chance of conversion? Or upsell them similar looking items that will increase your margins?

The Issue

A good ecommerce site will have many tools aimed at helping users find what they want and at showing them other similar items. Things like mega menus with different types of categorisation, multi-check faceted navigation, intuitive on-site search, flexible sort options and metric-driven product recommendations all help users discover your products. However, all have their limitations and all rely heavily on user input or action.

For instance, when using a site's search tool, it's very difficult for a user to describe the exact appearance of what they are looking for in any great level of detail using a text query as they'll likely return zero results. Instead their search may just extend to something generic like 'red dress', but on a site with a large catalogue they'll likely have hundreds of results returned and have to sift through them to find things they like, or start using the facets to filter down their search results to get a closer match. Everything just becomes quite manual and slow, and conversions fall. The difference with image-based product recommendations is that as soon as the user sees something they like, they can see all the other items on your site of similar appearance with the click of a button. It's a much closer experience to scanning the shelves or flicking through the racks of a physical store.

How it works

Through integration with Google’s Cloud Vision API Product Search, all product images (including multiple images of each product) are analysed and stored. Any new products added to your site are also uploaded and stored each time. The API applies machine learning to compare any product the user wants to look at, with all of the images in your product catalogue, and then returns a ranked list of the nine most visually and semantically similar results starting with the closest match.

The API runs on pre-existing models that examine different features of an image, such as colour, shape, and pattern. The images with similar features are clustered together to form a set so that when a user searches for similar images to the one they are looking at, the API extracts features of the chosen image and identifies the set it most likely belongs to. It then identifies the most similar products from that set based on its appearance.

There are a couple of different scenarios where these image-based product recommendations can be really effective.

  • Firstly, if you stock hundreds or thousands of products it's not always that easy for customers to see everything they like. Something may catch their eye and they'd like to see other similar looking things but your merchandising will be based on different metrics so won't be as accurate as image-based product recommendations. Displaying similar looking products to every single item they want gives them a much better chance of finding something they like, and of you making a sale.
  • The other scenario is where items are out of stock. If a user sees an item they really like but it is not currently in stock, they can see a selection of the closest looking items in your store. The results can even be configured to return only those items that you currently have in stock. Showing them similar alternative options could be the difference between losing and closing a sale.

In practice

F.Hinds the jewellers have thousands of different products across a number of different categories and understand that purchasing jewellery is a very emotive process. Most people buy jewellery based on what it looks like, and the differences between some items can often be quite subtle. As a result, they were keen to give users the option to use image-based product recommendations wherever they could, in the hope of closing more sales.

“The application has completely transformed the customer journey and their user experience. We’re now looking to leverage the technology in other ways to further enhance how the website performs and allow the customer to explore the diverse range of products we sell, historically something that would take numerous clicks that has now been reduced down to just one.”

The functionality is shown across a number of different pages where product images are displayed including the product listings pages, product search results, product details pages, products in the basket and products in any carousels. When a user sees a product they like they can click the find similar option and a popup is launched showing the closest matches.

image-based product recommendations

Each product shows the 'Find Similar' icon. When selecting the gold ring, items below are shown.

image-based product recommendations

Product the user is comparing is shown with the nine closest matches displayed in a carousel.


Since F.Hinds implemented the functionality last month, results are extremely positive. Over 10% of transactions already come from customers who have used the Find Similar tool, with conversions almost three times as high for those customers that do use it versus those that don't.

“This has to be the single biggest boost we have given to conversion rate in the 25 years we’ve been trading online.”

The Average Order Value has also increased by a small percentage. The project has subsequently been shortlisted for Online Innovation of the Year at the Retail Systems Awards 2022, Best UX in eCommerce 100+ Employees at the eCommerce Awards 2022 and the European Ecommerce Innovation Award at the European Ecommerce Awards 2022.

Frequently Asked Questions

Retail systems awards 22 logoEcommerce Awards 22 logoEuropean Ecommerce Awards 22 logo

Speak to us to learn more about how we can configure image-based product recommendations for your ecommerce site.