Tom Hoy, co-Founder and Partner at Stripe Partners, explores why Spotify is making recommendations more meaningful

According to Michael Schrage, 31% of ecommerce is now driven by automated recommendation. Think about that. Almost one third of every t-shirt, novel or takeaway purchased online is prompted by an algorithm. Netflix claims that 75% of what people watch on its platform stems from personalized suggestions.

But recommendation extends beyond consumer choices. Suggestions are shaping the way we understand ourselves. There is evidence to suggest that Tik-Tok is predicting sexuality and conditions such as ADHD in advance of the individual themselves. By informing self-discovery, self-awareness and self-knowledge, the future of recommendation systems is also “the future of the self”.

By informing self-discovery, self-awareness and self-knowledge, the future of recommendation systems is also “the future of the self”.

It incumbent on those of us informing how these recommendation systems are built that we support their development in thoughtful ways. One dimension of this is thinking through how recommendations can go beyond just reflecting or triggering “needs” and create truly meaningful experiences. This may sound like an impossible task, but our project with Spotify — published at the EPIC conference last year — started to sketch out what meaningful recommendation actually looks like.

The limitations of needs-driven recommendation

Several years ago Spotify made a bet that the future of recommendation would be algorithmic. As machine learning improves, the role of human editors will gradually recede as predictive systems become smarter. Central to this bet was a belief that it is possible to accurately map the musical needs of users. In essence, good recommendation maps content to need.

The key issue is that, while music may serve these needs, needs are not the real reason why users connect with music and artists. People love a band or a musician because they have become meaningful to them.

Several major projects ensued, with the objective of observing and mapping user needs in painstaking detail. Hundreds of discrete ‘music needs’ were identified from ‘relaxing when cooking’ to ‘singing with the family’ to ‘getting pumped for the gym’. This is the mindset in which music is ‘hired’ for a ‘Job To Be Done’.

Hundreds of playlists have been generated to serve these needs. Engineers have worked tirelessly to recommend these playlists in the right context, for the right occasion. But it became clear these playlists were only addressing the tip of the iceberg. The key issue is that, while music may serve these needs, needs are not the real reason why users connect with music and artists. Drake’s music may help people relax, or provide a solid tempo for their workout. But that’s not why they love Drake and his music. They love Drake because he has become meaningful to them.

Why the value of music cannot be reduced to needs

Economic sociologist Lucien Karpik calls cultural goods like music, wine, novels, and movies, ‘singularities‘. Singularities are complex, multidimensional goods the value of which can’t be reduced to their specific features. It would be foolish to claim one song has more value because it is longer or because the singer hits higher notes. Or that a glass of red wine should be more expensive because it is a darker hue. Focusing on features in isolation misses the point.

The true meaning of these goods only emerges to the user when they experience it themselves.

In these complex markets, expert and/or common opinion replaces the comparison of features. We rely on the movie critic or wine connoisseur to tell us what to expect. Or use popularity (charts, social recommendation) as a proxy for value.

But the true meaning of these goods only emerges to the user when they experience it themselves. Unlike user needs or goals, meaning can’t be fully anticipated in advance. And two users may derive entirely different sorts of value from the same good. For example, one person could connect to a song because it soundtracked a breakup. Someone else could love the same song because it gets people dancing.

How to trigger meaningful experiences

Spotify have found the concept of “cues” useful for thinking about how to trigger more meaningful experiences for their users. Cues are the way the music is packaged and presented to the individual user, so that the content is more likely to trigger meaning that is relevant for them (without dictating it literally).

For example, one study participant, 32, liked to listen to artists she could identify with (e.g. with Beyoncé as a mother, or with P!nk as a strong woman who’s ‘her own boss’). The recommender system might therefore suggest the artist Lizzo, prioritising visuals that celebrate Lizzo’s body positivity as part of the recommendation.

In many ways this approach reflects the traditional role of radio hosts to cue meaning before playing a song. When a host shares the story behind some lyrics, for instance, the listener is then more likely to find a meaningful connection with the music. The difference is that on Spotify these cues can be personalised to specific users rather than generalised to an audience.

The value of Meaning-creation

For companies looking to match users with content — whether music or movies or news — there is also a business risk of focusing on needs. Because needs are pre-determined, users optimise for efficiency: which system can serve my need most conveniently? As soon as another system serves that need more efficiently then, all other aspects being equal, a user will switch.

However, if a system consistently creates meaning then the value of the system is much greater. Because the meaning isn’t anticipatable the user doesn’t know what it is they need: they put their trust in the system to create meaning. This is why, for example, people follow the advice of specific movie critics even if they otherwise don’t like the ‘look’ of a film. The value of the critic becomes innate and unquantifiable: she just ‘gets’ me.

This is why, for example, people follow the advice of specific movie critics even if they otherwise don’t like the ‘look’ of a film

And as researchers, exploring what meaning can be created by a system is a more fruitful question than asking what needs can be addressed. Designing for needs can create unintended consequences and addictive behaviours. Designing for meaning also raises ethical questions but truly meaningful experiences are generally judged to have fewer downsides (at the level of the experience, at least).

About Tom Hoy

Tom has spent twelve years advising some of the world’s leading organisations on strategy and innovation. Prior to co-founding Stripe Partners he was a leader in the social innovation field, growing a hackathon network in South London to several hundred members and nurturing multiple successful initiatives.

Tom’s expertise lies in applying social science theory to interrogate technology in consumer and business cultures. The frameworks and concepts developed by Tom’s teams continue to guide the activity of world-class clients including Spotify, Facebook, Google and Intel. His work has appeared in the FT and the Guardian.

Related articles

Anna Leggett and Morgan Williams, Stripe Partners, explore how social science can steer AI towards better outcomes.