What's up with the ratings on Google Play?

[Underseer]

https://play.google.com/store/movies/details/Pixels?id=qy1W3Ss2_kM

Pixels is widely maligned as a really terrible recent movie, yet it has a user rating of 4.1 stars out of 5. In fact most things at Google Play seem to have about the same user rating, around 4.1 to 4.7 or so. Every single thing, good or terrible, seems to have around the same rating. Something about that seems awfully suspicious to me. Are moviemakers and app developers spamming the ratings with bots or something?

 

[Underseer]

Battlefield Earth: 4.2

Battlefield Earth is a trashy scifi novel. One of my friends goaded me into reading it a long time ago, and it's horrible. Utterly disposable, and yet it gets a 4.2. Now contrast that with[ent]hellip[/ent]

Watchmen: 4.5

This is arguably the greatest western comic book ever made. It cleverly deconstructs golden age comic books, silver age comic books, and the bizarrely fascistic fantasy behind the superhero genre. It does genuinely new and interesting things with the medium, including new and unique ways of weaving multiple nonlinear storylines together that just wouldn't work the same in any other medium. It's the only comic book to make Time magazine's 100 most important novels.

And yet here it is at 4.5, just slightly higher than Battlefield Earth.

I'm telling you, something is not right with the Google Play user ratings.

 

[Malintent]

There are two problems... the users, and the developers.

The users are terrible. Most things either get 1 or 5 stars... 1 star if anything bothers you at all, and 5 stars if it is "good". Much of that is due to how and when rating opportunity is presented to the user. and developers exploit that...
The developers game the system... perfect example, the game "dungeon master". A popup asks you to rate the game. There are two buttons, one button says "rate 5 stars", which brings you to the Google play review page and fills in 5 stars for you. The other button says "rate 1-4 stars", which brings you to EA's support page to provide private feedback.

Don't trust a rating system that lets you just click the number of stars you want to give it, or allows other apps and web services to rate items for you. Only trust rating systems that require an actual review... only trust reviews that appear thoughtfully composed.

Another (user) problem is Fake Reviews. Bots, and the like. There are ways to spot them:

1. Use of "Marketing Language". Phraseology lifted right off the advertisement of the product or service. A review that sounds more like a commercial than an individual's opinion.
2. User review is only one on site. If a review of a product on a site is the first and last review a user has made, then they are either a Bot, an angry person, or this is just their first review... look at the date. is it recent? If so, then they are just new (and maybe not reliable).
3. Two or more reviews are overly similar (or copy-paste). Bot.
4. Overuse of product-specific technical information that is not important to the review (such as model number with color option, or the brand name of the family of products)... i.e. "This model 2000 x400 in BLACK is the best Model 2000 Acme company has ever made in their Home Products line of Family Entertainment, bla, bla, bla. These reviews are just search engine bait.
4. The user has other reviews on site, but all are 5 star (or all are 1 star, or all are just for one manufacturer).. agenda.
5. My favorite dead giveaway - the user says they hated the product for some reason that has gotten the product negative reviews on the site, but for some unexplainable reason, they continued to use or re-tried the product, and they now love it, and everyone else is wrong / impatient / stupid.
6. The user name has a suspicious number of digits at the end of their name. Not a perfect measure, but computer-generated bots making reviews pretty much need to append at least 3 digits to the end of their generated names, or their scripts will often collide. Normal people do put digits at the end of their user names often enough (especially the decade of their birth), but 80% use only 1 or 2. 3 or more digits at the end is uncommon for a person to do, but common for a bot to do.

How does this information help? Well, if you want to just trust the average rating at the top of the page and not read through the actual reviews, it doesn't... it only implies you cannot rely on that, especially on sites where you can just click the 5 star button and do nothing else.
If you are reading reviews, then try googling a sentence from the review to see if it is found elsewhere. Click the reviewer's name to bring up all of their reviews. look at the reviewer's name to see if it is bot-ish.

My best piece of advice - seek reviews independently of the merchant. Do your own research.

 

[Jimmy Higgins]

I'm not going on to any site and skimming all the movies to right the internet on their rating. So there is that issue.

 

[Underseer]

I'm not going on to any site and skimming all the movies to right the internet on their rating. So there is that issue.

But that's just it, the user ratings on Google Play are next to useless because of whatever is causing this. I refuse to believe that Battlefield Earth is only 0.3/5 better than Watchmen.

 

[braces_for_impact]

Personally, I believe it's because the general population is stupid.

 

[Cheerful Charlie]

Battlefield Earth: 4.2

Battlefield Earth is a trashy scifi novel. One of my friends goaded me into reading it a long time ago, and it's horrible. Utterly disposable, and yet it gets a 4.2. Now contrast that with[ent]hellip[/ent]

Watchmen: 4.5

This is arguably the greatest western comic book ever made. It cleverly deconstructs golden age comic books, silver age comic books, and the bizarrely fascistic fantasy behind the superhero genre. It does genuinely new and interesting things with the medium, including new and unique ways of weaving multiple nonlinear storylines together that just wouldn't work the same in any other medium. It's the only comic book to make Time magazine's 100 most important novels.

And yet here it is at 4.5, just slightly higher than Battlefield Earth.

I'm telling you, something is not right with the Google Play user ratings.

The Church of Scientology has long been notorious for gaming the system to make it appear Hubbard's books are bestsellers. For example buying large numbers of his books and recycling them through whole sellers to inflate the supposed popularity of his damned garbage books. So it would not surprise me to find that with Google Play they are back to their old tricks.


http://www.latimes.com/local/la-scientology062890-story.html


In some cases, sales of Hubbard's books apparently got an extra boost from Scientology followers and employees of the publishing firm. Showing up at major book outlets like B. Dalton and Waldenbooks, they purchased armloads of Hubbard's works, according to former employees.
As a writer, Hubbard was extremely prolific. He wrote short stories. He wrote books. He wrote screenplays. And, for more than 30 years, he wrote thousands of directives and scores of personal improvement courses that form the doctrine of Scientology.


The promotion of Hubbard's books is part of a costly and calculated campaign by the movement to gain respect, influence and, ultimately, new members. In the process, Hubbard's followers hope to refurbish his controversial image and position him as one of the world's great humanitarians and thinkers.

 

[The Colourful Jester]

Personally, I believe it's because the general population is stupid.

And quick to act on that. I can’t help but wish customer reviewers could only vote a year after release.

 

[Underseer]

Battlefield Earth: 4.2

Battlefield Earth is a trashy scifi novel. One of my friends goaded me into reading it a long time ago, and it's horrible. Utterly disposable, and yet it gets a 4.2. Now contrast that with[ent]hellip[/ent]

Watchmen: 4.5

This is arguably the greatest western comic book ever made. It cleverly deconstructs golden age comic books, silver age comic books, and the bizarrely fascistic fantasy behind the superhero genre. It does genuinely new and interesting things with the medium, including new and unique ways of weaving multiple nonlinear storylines together that just wouldn't work the same in any other medium. It's the only comic book to make Time magazine's 100 most important novels.

And yet here it is at 4.5, just slightly higher than Battlefield Earth.

I'm telling you, something is not right with the Google Play user ratings.

The Church of Scientology has long been notorious for gaming the system to make it appear Hubbard's books are bestsellers. For example buying large numbers of his books and recycling them through whole sellers to inflate the supposed popularity of his damned garbage books. So it would not surprise me to find that with Google Play they are back to their old tricks.


http://www.latimes.com/local/la-scientology062890-story.html


In some cases, sales of Hubbard's books apparently got an extra boost from Scientology followers and employees of the publishing firm. Showing up at major book outlets like B. Dalton and Waldenbooks, they purchased armloads of Hubbard's works, according to former employees.
As a writer, Hubbard was extremely prolific. He wrote short stories. He wrote books. He wrote screenplays. And, for more than 30 years, he wrote thousands of directives and scores of personal improvement courses that form the doctrine of Scientology.


The promotion of Hubbard's books is part of a costly and calculated campaign by the movement to gain respect, influence and, ultimately, new members. In the process, Hubbard's followers hope to refurbish his controversial image and position him as one of the world's great humanitarians and thinkers.

Sure, that's possible, but it was the first bad novel that came to mind. The thing is, I've seen plenty of other bad movies, novels, apps, etc. that had roughly similar ratings.

- - - Updated - - -

Personally, I believe it's because the general population is stupid.

Hanlon's razor is a possible solution to this dilemma.

 

[Cheerful Charlie]

With Scientology, I suspect more than possible. This sort of crap is SOP with them. I am still waiting for Battlefield Earth Part 2 to make it to the silver screen.

 

[Hermit]

The users are terrible. Most things either get 1 or 5 stars...

Is there a breakdown of how users voted that I have missed somehow, or have you just made an ex recto assertion?

It's also worth pointing out that per the link Underseer provided, the rating for Battlefield Earth is based on 6 votes, i.e. statistically negligible.

 

[Malintent]

The users are terrible. Most things either get 1 or 5 stars...

Is there a breakdown of how users voted that I have missed somehow, or have you just made an ex recto assertion?

It's also worth pointing out that per the link Underseer provided, the rating for Battlefield Earth is based on 6 votes, i.e. statistically negligible.

The post you extracted that quote from was related to how rating systems work in general. That information was gotten from internet usage research that revealed this information, and is not an unexpected finding, unless you are disagreeing... is it your position that those statistics are in error because people actually DO exercise subtlety and nuance more instinctively than black-and-white, fight or flight extremes? I'd be curious about your findings in that area.

Specific to Google play was how the developers implemented "in-app ratings" that redirect 1-4 stars to private feedback, thus resulting in 5 star ratings posted on Google play exclusively. Evidence for that is documented in their API whitepapers.

Are you employing a specific bias you would like to share, in challenging the unsurprising finding that "people are generally obliviously thoughtless"?

 

[Hermit]

The users are terrible. Most things either get 1 or 5 stars...

Is there a breakdown of how users voted that I have missed somehow, or have you just made an ex recto assertion?

It's also worth pointing out that per the link Underseer provided, the rating for Battlefield Earth is based on 6 votes, i.e. statistically negligible.

The post you extracted that quote from was related to how rating systems work in general. That information was gotten from internet usage research that revealed this information, and is not an unexpected finding, unless you are disagreeing... is it your position that those statistics are in error because people actually DO exercise subtlety and nuance more instinctively than black-and-white, fight or flight extremes? I'd be curious about your findings in that area.

Specific to Google play was how the developers implemented "in-app ratings" that redirect 1-4 stars to private feedback, thus resulting in 5 star ratings posted on Google play exclusively. Evidence for that is documented in their API whitepapers.

Are you employing a specific bias you would like to share, in challenging the unsurprising finding that "people are generally obliviously thoughtless"?

There are many sites people can vote using a star system. Where a breakdown is provided, for instance in Facebook reviews, the result is usually a bell curve, though the peak is rarely dead centre. I wrote my post in light of the ones I've seen, and I would really like to see the raw data for the votes cast in the cases of Battlefield Earth and Watchmen, though with a sample rate n=6 the latter would be statistically insignificant.

There are exceptions, of course, usually when a vote war beaks out. It can get quite amusing. The Avid Reader Bookshop, for instance had been trading in Brisbane for almost 20 years without attracting much attention. It was rated a couple of dozen times on Facebook. The spread was concentrated on four and three stars. In June last year the owners announced that they will be stocking Boys Will Be Boys, a book written by the outspoken feminist, Clementine Freud. All hell broke loose. First, some MRA type group, Anti-Feminism Australia, called on its members to hit the shop with one-star reviews. The call was made on their publicly viewable website, so word got around and hundreds of one-star reviews piled up very quickly. Sane people soon reacted and thousands of five-star reviews flooded in just as quickly. After a week or so everything went quiet again.

The score as of today:

5 stars: 4700
4 stars: 38
3 stars: 4
2 stars: 7
1 star: 409

Well, I thought it was amusing anyway - and exceptional.

 

[Underseer]

Battlefield Earth: 4.2

Battlefield Earth is a trashy scifi novel. One of my friends goaded me into reading it a long time ago, and it's horrible. Utterly disposable, and yet it gets a 4.2. Now contrast that with[ent]hellip[/ent]

Watchmen: 4.5

This is arguably the greatest western comic book ever made. It cleverly deconstructs golden age comic books, silver age comic books, and the bizarrely fascistic fantasy behind the superhero genre. It does genuinely new and interesting things with the medium, including new and unique ways of weaving multiple nonlinear storylines together that just wouldn't work the same in any other medium. It's the only comic book to make Time magazine's 100 most important novels.

And yet here it is at 4.5, just slightly higher than Battlefield Earth.

I'm telling you, something is not right with the Google Play user ratings.

The Church of Scientology has long been notorious for gaming the system to make it appear Hubbard's books are bestsellers. For example buying large numbers of his books and recycling them through whole sellers to inflate the supposed popularity of his damned garbage books. So it would not surprise me to find that with Google Play they are back to their old tricks.


http://www.latimes.com/local/la-scientology062890-story.html


In some cases, sales of Hubbard's books apparently got an extra boost from Scientology followers and employees of the publishing firm. Showing up at major book outlets like B. Dalton and Waldenbooks, they purchased armloads of Hubbard's works, according to former employees.
As a writer, Hubbard was extremely prolific. He wrote short stories. He wrote books. He wrote screenplays. And, for more than 30 years, he wrote thousands of directives and scores of personal improvement courses that form the doctrine of Scientology.


The promotion of Hubbard's books is part of a costly and calculated campaign by the movement to gain respect, influence and, ultimately, new members. In the process, Hubbard's followers hope to refurbish his controversial image and position him as one of the world's great humanitarians and thinkers.

OK, so Battlefield Earth was a bad example. Sorry.

I just wanted an arbitrarily bad work to cite, and Battlefield Earth was the first thing that came to mind.

How about the movie Battleship? That got a 4.0 out of 7,382 ratings. That seems to be a rating in the same general ballpark despite the fact that nearly everyone I've ever talked to about this movie thought it sucked.

 

[Underseer]

Fifty Shades of Grey
4.4 out of 31,191

Twilight
4.3 out of 5,212

300
4.2 out of 96
This is from after Miller started to decline and became a shadow. Notable only for being the basis for the equally stupid movie. I wanted a bad comic book to contrast with the ratings for Watchmen, but most of the bad comic books I could think of (including Miller's sequel to Dark Knight Returns) didn't have very many reviews.

Hopefully the above are better examples than Battlefield Earth. I don't think anything on the above list should score so close to Watchmen, which is objectively a classic and the only comic book to make Time magazine's list of most important novels.