insights

CRO Lab - Image-based product recommendations

Monday, 17 January 2022

We know replicating the in-store experience online isn’t easy. Being able to browse through hangers of clothes, see jewellery up close with your own eyes, touch furniture with your own hands or sit on a sofa is just impossible to reproduce through ecommerce, but there’s more and more tools to help bring your products to life and give users greater help to purchase online. Image-based product recommendations are one such option.

When a consumer visits a physical store, they can see everything with their own eyes, taking in every colour, style, option and feel of whatever it is they are looking at. They say a picture paints a thousand words and people are often drawn to things they like the look of, particularly in the world of fashion, jewellery, furniture, homeware etc… So why not harness that power to show your customers other things they might like the look of and increase your chance of conversion? Or upsell them similar looking items that will increase your margins?

The Issue

A good ecommerce site will have many tools aimed at helping users find what they want and at showing them other similar items. Things like mega menus with different types of categorisation, multi-check faceted navigation, intuitive on-site search, flexible sort options and metric-driven product recommendations all help users discover your products. However, all have their limitations and all rely heavily on user input or action.

For instance, when using a site's search tool, it's very difficult for a user to describe the exact appearance of what they are looking for in any great level of detail using a text query as they'll likely return zero results. Instead their search may just extend to something generic like 'red dress', but on a site with a large catalogue they'll likely have hundreds of results returned and have to sift through them to find things they like, or start using the facets to filter down their search results to get a closer match. Everything just becomes quite manual and slow, and conversions fall. The difference with image-based product recommendations is that as soon as the user sees something they like, they can see all the other items on your site of similar appearance with the click of a button. It's a much closer experience to scanning the shelves or flicking through the racks of a physical store.

How it works

Through integration with Google’s Cloud Vision API Product Search, all product images (including multiple images of each product) are analysed and stored. Any new products added to your site are also uploaded and stored each time. The API applies machine learning to compare any product the user wants to look at, with all of the images in your product catalogue, and then returns a ranked list of the nine most visually and semantically similar results starting with the closest match.

The API runs on pre-existing models that examine different features of an image, such as colour, shape, and pattern. The images with similar features are clustered together to form a set so that when a user searches for similar images to the one they are looking at, the API extracts features of the chosen image and identifies the set it most likely belongs to. It then identifies the most similar products from that set based on its appearance.

There are a couple of different scenarios where these image-based product recommendations can be really effective.

Firstly, if you stock hundreds or thousands of products it's not always that easy for customers to see everything they like. Something may catch their eye and they'd like to see other similar looking things but your merchandising will be based on different metrics so won't be as accurate as image-based product recommendations. Displaying similar looking products to every single item they want gives them a much better chance of finding something they like, and of you making a sale.
The other scenario is where items are out of stock. If a user sees an item they really like but it is not currently in stock, they can see a selection of the closest looking items in your store. The results can even be configured to return only those items that you currently have in stock. Showing them similar alternative options could be the difference between losing and closing a sale.

In practice

F.Hinds the jewellers have thousands of different products across a number of different categories and understand that purchasing jewellery is a very emotive process. Most people buy jewellery based on what it looks like, and the differences between some items can often be quite subtle. As a result, they were keen to give users the option to use image-based product recommendations wherever they could, in the hope of closing more sales.

“The application has completely transformed the customer journey and their user experience. We’re now looking to leverage the technology in other ways to further enhance how the website performs and allow the customer to explore the diverse range of products we sell, historically something that would take numerous clicks that has now been reduced down to just one.”

The functionality is shown across a number of different pages where product images are displayed including the product listings pages, product search results, product details pages, products in the basket and products in any carousels. When a user sees a product they like they can click the find similar option and a popup is launched showing the closest matches.

image-based product recommendations

Each product shows the 'Find Similar' icon. When selecting the gold ring, items below are shown.

image-based product recommendations

Product the user is comparing is shown with the nine closest matches displayed in a carousel.

Results

Since F.Hinds implemented the functionality last month, results are extremely positive. Over 10% of transactions already come from customers who have used the Find Similar tool, with conversions almost three times as high for those customers that do use it versus those that don't.

“This has to be the single biggest boost we have given to conversion rate in the 25 years we’ve been trading online.”

The Average Order Value has also increased by a small percentage. The project has subsequently been shortlisted for Online Innovation of the Year at the Retail Systems Awards 2022, Best UX in eCommerce 100+ Employees at the eCommerce Awards 2022 and the European Ecommerce Innovation Award at the European Ecommerce Awards 2022.

Frequently Asked Questions

Won't product recommendations show users the same items? Well yes, and no. It's likely that they will show some other similar items based on whatever method you use to populate them, be it manual, via algorithm or using tradeit's metric driven system, but it won't be as accurate. Remember, we are showing items based entirely on what they look like. For example, manually populated recommendations might show items that are the same colour, whilst metric-driven recommendations might show something that other users bought or that sells well, but image-based product recommendations will show the absolute closest matches in your entire product catalogue, based on their appearance. + -

Don’t facets do the same thing? Multi-check facets are great, and you should definitely offer them to your customers, but they rely on the customer populating the different options, are only as good as your data, and normally just refine by common characteristics like colour, flavour, size, price etc… they don’t really achieve the same depth of granularity or similarity as our image-based product recommendations.

For example, say the user finds a dress they like, by using facets they might be able to find other similar dresses based on the colour, size and maybe even style, but they couldn’t go any deeper into finding a closer match than that. With image-based product recommendations they can because they are driven by exactly what the dress looks like, rather than a few common characteristics. So as well as colour, size and style, this could include all those with a similar pattern, cut, neck line, hem line, sleeve style, and buttons for instance. + -

We have a small product catalogue – is it worth it? In all honesty, probably not. With a small catalogue of products, users can just scroll through without needing any really advanced tools to help them find what they are looking for as long as you have a strong search tool. However, if your catalogue begins to increase in size and you stock many similar types of product, it’s definitely worth considering image-based product recommendations to help your customers. It's also useful for showing similar items that are in stock, regardless of your catalogue size. + -

Is this suitable for all industries? No, as it is a visual-based medium it's entirely focused on the appearance of an item rather than it's functionality. Google's API actually limits product types to Home Goods (appliances, furniture, furnishings etc...), Apparel (bags, shoes, clothes, jewellery etc...), Toys and Packaged Goods (Any products that are shown through a wrapper, container, box, etc...) so these are clearly the most suitable items.

We suggest that it suits design-led products where people are attracted to them, and buy them, because of what they look like. That way, showing other similar looking items is likely to be most effective. + -