This post originally appeared on the Yelp Data Science Medium blog, Locally Optimal.
Some of my friends who rely on Yelp reviews are amazed at the variety of opinions: Many can only imagine writing a review about a terrible experience or a transcendent one. And terrible experiences, like transcendent ones, are rare.
We love users who use Yelp to find businesses, make reservations, request quotes and check in without ever writing reviews. But not all of our users are like them. Some people feel a strong impulse to tell the world about the interactions they have with local businesses.This impulse is exactly what spurred Yelp’s co-founder Jeremy Stoppelman to focus on fostering public recommendations.
These public recommendations can run the gamut from extremely positive to extremely negative experiences. After all, it’s incredibly gratifying to share when you’ve found a hidden gem, and, conversely, warn people about the tragic duds.
But most experiences are neither terrible nor transcendent. If one dish is extraordinary at an otherwise ordinary restaurant, the reviewer who unearths that diamond in the rough for others to enjoy can feel joy herself. And a bank, pharmacy or solar installer that got the job done can be worth rating even if not worth raving about. Many of our users in turn understand that a business with three or four stars can often be excellent, especially if any critical reviews reflect parts of the experience that don’t matter to the reader. A higher review count also signals to users that more people are choosing the business.
A review — of any star count — can provide users with information about what services the business offers, and can provide businesses with actionable suggestions to improve the customer experience.
Having millions of reviews throughout our five-star scale — at the top, bottom, and in between — makes it possible to differentiate between businesses more easily than if all of our ratings were bunched up near the top end of the scale, a problem that has reduced the usefulness of some other five-star scales in the digital economy. If every business you’re considering is between a 4.5 and a 5, the differences shrink and it becomes hard to distinguish two businesses.
Today the average rating on Yelp sits in between “A-OK.” (3) and “Yay! I’m a fan.” (4). Our average rating across all places and types of businesses is 3.77. (Throughout the post, we’re using data from reviews in the U.S. and Canada.) And 40 percent of ratings are neither one star nor five star.
Let’s focus on restaurants, the category with the most total reviews and the one most commonly associated with Yelp, to illustrate what our reviews look like and who’s writing them. Restaurant reviews show a similar pattern to reviews as a whole: Half of reviews accompany ratings between two stars and four, and the average rating is 3.71. (We’ll look at other categories in a separate post.)
And just about any way you slice our restaurant ratings, they look similar: above the average rating of 3, below 4. Whoever is writing reviews, wherever and whenever, their ratings add up roughly the same, with plenty of ordinary experiences leavening the extraordinary.
Wherever our reviewers are eating and writing, they’re pretty consistent. In each state and province, the average rating is between 3.57 (Ontario) and 3.90 (Prince Edward Island).
When our reviewers sit down — or stand up — to write about their latest restaurant experience, they’re also remarkably consistent no matter the time of year, day of week, time of day, or year.
Reviewers also score similarly no matter whether they’re writing their first restaurant review or their 500th. We can see this by grouping reviewers into cohorts based on how many restaurants they’ve reviewed. So everyone who’s written between one and 10 restaurant reviews is in a group, and everyone who’s written 501 or more is in a different group. If you average the ratings of reviews in every group, you get an average rating that is greater than or equal to 3.63 and less than or equal to 3.77.
This remarkable consistency of restaurant ratings demonstrates the average sentiment of our food-loving contributors; that average is made up of plenty of fond recollections of best meals ever, lots of recording of adequate sandwiches and salads, and a fair few regrets of meals that can’t be undone, too. Every restaurant’s Yelp rating is the sum of experiences of all kinds, from all sorts of reviewers. And the average rating of open restaurants, whether they have 1 review or more than 10,000? 3.47.
There is at least one way you can slice restaurant reviews to find a big gap. The longer the review, the lower the rating. That’s true whether the review is being tapped out on a smartphone or typed on a desktop keyboard. Take the top 25 percent of mobile reviews by length, and the average rating is 3.31. The 25 percent shortest? A generous 4.10.
Shorter and more positive reviews are more likely than longer and more negative reviews to use adjectives such as “great,” “best,” “awesome,” and “amazing,” summing up the experience in a word. Imagine a review that has to fit in a text message or a tweet: succinct and to the point. One review like this ends with, “Great outdoor seating area. Ocean view. Cash only.”
Longer, more downbeat reviews tell unhappy stories: “ordered,” “came,” “minutes,” “asked,” and “wanted.” Consider this review that precedes a thorough, dated, price-included breakdown of every dish ordered with a sentence summing it all up: “Bottom Line — The restaurant (both the food and the service) needs quite a bit of improvement.”
I think it was one of Leo Tolstoy’s children who said, “All happy meals are alike; each unhappy meal is unhappy in its own way.”
That is the story for restaurants. Next time we’re going to look at local services, auto, professional services and home services, which has an entirely different ratings landscape.
