I just did a little research on Yelp’s filtering methods this morning, and I’m not sure how I feel about it as a business or a user of the site.
I have a tendency to obsess over my Yelp and TripAdviser scores at work. I have them pinned as app tabs on my Firefox browser so that I can check them quickly and regularly. After checking so frequently, I’ve noticed that often times reviews, both positive and negative disappear occasionally. We got slammed on Yelp the last week or so and two of those horrible reviews just dropped off our score and the site. I had noticed this before, but this morning it stirred my curiosity enough to research it.
First off, if enough users flag a review as abusive or inappropriate in someway, like they’re threatening violence or using crude language, the review can get taken down. Yelp evaluates those reviews that are flagged and determine if they should be removed. That makes sense, free speech issues aside, Yelp is a private entity with the right (in my opinion) to set their own bar on appropriate content. Any reviews that get flagged for those issues really have no place in that forum. Yelp is supposed to be a community review site, but those community reviewers need to maintain a certain level of decorum otherwise they’re no better than the establishment they’re slinging out a 1 star review too.
The other way a review disappears, is Yelp’s automatic filtering algorithm. The program is designed to help prevent abuses of the system, like current employees or owners going on to write a glowing review about themselves or their work. Or competitors or disgruntled employees going on to write negative reviews. That makes sense. However, the computer program that can actually review the content a review and determine its veracity just hasn’t been invented. So what does the filter look for? First off it looks at the user information, number of reviews, amount of profile information (and a bio pic), and length of time that you’ve been a member, so more established users that have been Yelping for awhile have a higher authenticity rating in the eyes of their filter and their reviews are likely to stick around whether they’re positive or negative. The filter also isn’t static, so if you’re a new user that goes on and writes your first review and it gets filtered out in a couple of days, Yelp says to keep on Yelping, once the filter can be more confident that you’re authentically reviewing establishments your previously filtered reviews might pop back up. This all makes sense to me and seems relatively fair, though I can see how new users would find it frustrating at first. It also cuts down on those people who I’ve encountered that created an account just to write the one review about my establishment because they had a horrible experience and they want to trash us (legitimately or not). They have no real interest in providing useful or constructive feedback, and Yelp recognizes that some people are just crazy customers.
Here’s where it gets tricky. The filter also looks at how your review, the score you gave, measures up to the business’s over all average. Basically it’s trying to eliminate outliers both positive and negative. Once again this isn’t a static filter, it just takes a certain amount of critical mass for reviews to start effecting the overall average. However, it means that unusual experiences get dropped out, a business with a strong 4 star average, that gets one 1 star review out of six 5 star reviews is likely to have that 1 star review filtered out in a couple of days. This once again protects businesses from vindictive reviewers. However, what about users? If a business suddenly takes a dramatic downturn, what is the point of critical mass where those filtered reviews will return? It also means a business with a low or middle rating business like a 2 or 3 star average that is trying to turn things around will have a much harder time doing so, because their positive 4 and 5 star reviews are getting filtered out. The only saving grace to this, is it takes about a week for the review filter to catch on, so regardless of whether they’re going to get filtered out or not, they’ll at least be up there and visible for a few days. Also, one star reviews still show up on places with a 4 star average as well, so this filter is weighted along with the Yelp reviewers user history, so someone with a strong Yelp reputation who writes and outlier review is less likely to get filtered.
Now some of this is conjecture and supposition because is very tight-lipped about their filtering process, because the more they explain it to the public, the easier it is to trick the filter. There are very little news and blog posts out there that I’ve seen in a quick Google search, but the Yelp message boards was understandably filled with this question from users and in some cases there were official Yelp responses.
At first glance it seems like a fishy practice, but the more I’ve thought about it, the more it kind of makes sense. Yelp is trying to protect both users and businesses from outlier experiences and from users who think that because they’re anonymous they can be horrible and vindictive human beings. On the other hand, I can see it’s frustrating for reviewers to get filtered out initially. Making it seem like their voice, opinions and experiences don’t really matter to Yelp. The only saving grace is that filtered reviews can return if the reason that it was filtered no longer qualifies. I’m curious what other people think though. Is Yelp abusing their power by filtering reviews? Or are they doing the responsible thing? Are they pandering to businesses that they might want to attract as advertisers? Or are they just trying to prevent their site for being used for fraudulent and abusive purposes?
***Update*** Yelp has addressed this issue somewhat on their own blog. Worth checking out.