This is a question I get from authors:
I get some reviews that love something, another review says that they hate that exact same thing. Who should I listen to?
Well, I’m not an author. But I do know that authors obsessively read everything related to themselves and their work: Goodreads, blog posts, Amazon Reviews, etc. They obsess over them.
In my World, I am a web developer. We also obsess over our apps, performance, feedback, etc. and this sounds like A/B Testing.
A/B Testing means that you produce two versions of something, an “A” version and a “B” version, and see which works better. From this, you know how to optimize your app to improve conversion rates, user engagement, or whatever other metric you’re testing. Sounds great in theory, right? But like most things in theory, practice proves more difficult.
An example:
Statistical significance is often the thing that most most testing software and people running tests focus on but there’s an equally important thing that is overlooked: statistical power. Or running the test with enough traffic so the result is meaningful and not chance. A page conversion change from 5% to 5.5% (an increase of 10%) could mean thousands of dollars more to the business, but to be certain about that result 62,000 users would need to go through the A/B test
The “Three Jellybeans” approach
Google invented the “three” jellybeans approach, and it’s simple. Google figured that most people reacted more strongly to things they dislike than things they like. Given this, if they implemented a new feature, they needed three “white” jellybeans (user likes) vs. “black” jellybeans (user dislikes) to implement a feature. If they received anything less, they rolled back the feature.
You can read the entire article here.
So what’s the problem?
The first one is something I already reviewed on my reason’s why Amazon Unlimited can be a problem for readers. Amazon’s information is “low information” signals, reviews don’t give any useful information.
Amazon lacks a lot of features that we see in Facebook. For example, Facebook lets you break down advertising based upon similar likes, age ranges, interests, gender, location, etc. Amazon has no such breakdown. Thus you can’t tell if the reviewer is someone who falls into your demographic or if they just randomly picked up the book. From an optimization standpoint, it makes more sense to go after a specific demographic than trying to hit as broad a demographic as possible.
At Google, the same sort of advanced breakdown is available. It’s why Google Analytics is one of their most popular programs. Amazon has nothing like Analytics, no breakdown of average pages read, average reading speed, which sections were skipped over, what page people stop reading on (bounce rate), etc.
Because Google was able to supplement their “likes” with lots of hard data, they could make an informed decision. Amazon lacks any hard data.
Find good beta-readers
Join a writer’s group or groups
The Fantasy Writers group, the Indie Authors Group, and many others are dedicated towards writers. You can find other authors on there and the advice you get from authors is what I would call high-information signals. You can also Facebook follow your favorite authors and see what websites they recommend and follow.
Hire a Developmental Editor
Developmental editors offer specific suggestions about the core intentions and goals of the book, the underlying premise, the story, character development, use of dialogue and sensory description, the polish, narrative voice, pacing, style, language – the craft and literary art of the book.
As with talking to other authors, a developmental editor is a form of high-quality signals.
GameLit reviews 