The concealed cost of ‘perfect’ recommendation systems

Photo by Charles Deluvio on Unsplash

Software products nowadays have really solid recommendation systems backed by data from millions of users. Business analysts and data scientists too work tirelessly to build systems in order to optimize and refresh recommendations constantly and make users spend a lot more time with their product and drive them to come back for more.

But is there a KPI that is hard to optimize? Perhaps something that is hard to quantify and even harder to measure? This article explores one such KPI and product feature that tries to overcome this issue

The way I personally look at the current recommendation systems is that it helps users from making easy exclusion decisions while asking the users to make tough decisions. If a person is a fan of sci-fi movies and he/she has only consumed their movies on Netflix, it is an easy decision to not recommend them romcom movies (assuming recommendations are content-based for simplicity). But you ask them to choose between Interstellar & Gravity which is a much harder decision to make (assuming he/she likes them both). You might ask, isn’t this what you expect from the system? Showing users what they want to see from the myriad of options available? Well yes, but actually no.

Decision paralysis & decision fatigue :

The problem with multiple good recommendations is that they can stop the user from making a decision altogether. When a user is stuck and cannot choose between two movies or two products recommended, he/she is said to be in a state of decision paralysis.

Let's say you’re planning to buy a watch. You visit Amazon and start searching for it. Amazon recommends you top-2 watches and you like them both equally. Now you’re stuck as you cannot make a decision. You might get frustrated and postpone your decision or even reconsider it. This is a very common scenario in online retailers where users struggle with last-mile decision making and decision paralysis is one cause for this.

On the other side, a user makes more and more decisions to reach their final result and may end up with decision fatigue. Rejection of product/service is also ultimately a decision, hence poor recommendations also lead to a higher probability of decision fatigue.

Recommendation systems, on a broader perspective, tries to find the user optimum between decision fatigue and decision paralysis.

Factors leading to decision paralysis

There are a lot of factors that could lead to decision paralysis. I’ve compiled 4 of the most obvious factors of the product service which could lead to decision paralysis:

  1. The urgency of product/service: If someone needs a product/service urgently, it is unlikely he/she will spend a lot of time going through all positives/negatives/reviews. If you were stuck in a software bug/error, you’d likely rush through Google provided recommendations to an article/video that provides an accurate solution
  2. The upfront cost of product/service: If the upfront cost of the product is high, one might spend some more time analyzing the product thoroughly before making a decision
  3. Product/Feature interaction time to understand decision outcome: After making a decision, if the outcome of the decision takes a large amount of time, there is a higher chance of decision paralysis. Ex: To understand if you like or dislike a movie or a series, you’ll have to spend at least a few hours understanding the outcome of your decision. Google search in this case is close to the best-case scenario because you either get your answer or not. Amazon is not hugely impacted by this since after you place an order, you do not actively interact with Amazon (unless to check status, get a refund, etc.) to interact with your product.
  4. Brand/product Image: the probability of decision paralysis is inversely proportional to brand/product perception and image. Stackoverflow for answering programming questions, Google for search engine, Chrome for web browser, and so on.

Features that help

Following features have proven to be effective against decision paralysis:

  1. More data points: Going back to our Amazon example above, to prevent users from getting stuck, the reviews and ratings feature helps tremendously. You compare the ratings and reviews of the watches and since one of the watches had much positive review, you decide to go for it. Similar such features are built on top of recommendation systems to assist users to make informed decisions on top of provided recommendations
  2. Marketing/advertising: Trying to convince your user to make purchase decisions by showing their search result in advertisements also helps in conversion. Amazon is notorious for Google ads on recent amazon searches. It also establishes your brand/product image in case you’re not well established

Personally, this is where I feel the Netflix recommendation system still lags behind. It has no metrics to help users make faster, more informed decisions. A user cannot sit through multiple trailers to decide if he likes a movie/series and if he/she will watch an unknown or lesser-known series. The following features might help users make more informed decisions:

  • Netflix could ask for user ratings on their completed movie/series. This helps understand past user experience who’ve watched the movie/series before. Currently, there is only the option of like/dislike
  • Collaborate with rating agencies like IMDB or Rotten Tomatoes to provide users ratings from reliable review/rating aggregators. This will also help viewers understand experienced critics reviews

Netflix has tried to minimize this to some extent by providing a shuffle feature. If a user is unable to decide which movie/series to watch, Netflix can help make it by providing random movies or series based on your past preference and similar behavior user pattern.

A similar problem is also present to some extent in food delivery aggregators like Swiggy, Uber Eats, Zomato, etc. Users are bombarded with options of cuisines, restaurants, and food combinations to choose from. It is very easy for the user to feel overburdened with decisions and ultimately not make one.

Another feature that could help is knowing user behavior with respect to the exploration vs exploitation tradeoff. We know users' content patterns and can also derive other similar things to suggest from other similar behaving user patterns. Recommendation systems are already in a position to exploit user behavior and patterns. What we can do is create a decision momentum by providing easy exclusions (add a bit more exploration in your recommendations) or showing content that has lesser overlap with user behavior and hence pushing the user to make a decision to view content that he/she thinks they would like to watch. This has to be done carefully as too much exploration also leads to decision fatigue which too much exploitation has a higher probability of leading to decision paralysis.

Recommendation systems do help users tremendously and prevent them from experiencing decision fatigue. Ultimately, we have to move from the idea that recommendation systems are meant to show users what they want to see, to the fact that recommendation systems should help users make decisions faster & effectively.




Data Scientist @Royal Dutch Shell | Deep Learning | NLP | TensorFlow 2.0 | Python | Astrophysics ❤

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

An MVP Design Guide

Framework for designing and defining Product Success metrics

A few thoughts on “How can I become a product manager?”

The Painted Door approach allows a Product Manager to avoid wasting time building something nobody…

How to make online shopping delightful? Learn from our VP-Product, Ankit Uppal

Book summary: “ Inspired: How To Create Tech Products Customers Love” — Part 4.1

One article about Minimum Lovable Products

No more “user”, say “person” instead

Scene from Wall-E movie where “users” blindly follow a system’s instructions on behaviour

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Sundaresh Chandran

Sundaresh Chandran

Data Scientist @Royal Dutch Shell | Deep Learning | NLP | TensorFlow 2.0 | Python | Astrophysics ❤

More from Medium

Conquering A/B Testing (1): When should I conduct A/B testing?

Machine learning and product analytics: Navigating the hype

Verify This! How Cedar Uses Data to Optimize the Check-In Experience