There are many problems with reviews online. Many people use reviews as a way to filter out the fodder and find good products. Talking about and discussing tea is also an important and necessary part of the learning process and a part that reviews can help to facilitate. Learning in a vacuum is usually a terrible idea and reviews can be a formative part of learning. People who sell products and vendors have realized that online reviews and opinions affect sales and have reacted in a number of ways. On the most basic level, yelp, ebay, and amazon vendors openly ask for and encourage reviews of their product. Fair enough. On a slightly more sinister level, there are paid reviewers of Amazon products that are on the payroll of the producer and are required to give 5 star reviews. This becomes a huge problem when there are only a few reviews of similar products online, meaning one or two five star reviews can propel a significant amount of sales! Many of these problems persist into tea, to varying degrees. While there isn’t necessarily malintent on the reviewer or the vendor’s part, bias and aspects beyond the quality of tea play frequently heavily into the actual review and people’s perception. This article will specifically examine points of bias in the online tea world.
Reviews are a Filtered Selection
What teas are reviewed? Reviews are a highly filtered and biased selection of teas. TeaDB videos fall into this category. The teas brought on for review aren’t a random selection. They’re usually picked out for one reason or another. Occasionally we’ll do blind tastings, but usually the teas brought onto the show are teas that we feel confident in recommending. This isn’t necessarily a bad thing, but the motivation for reviewing a tea should be taken into account by any potential consumer. Is it a random review of a random tea or is it an actual recommendation? What happens to the other teas? Well, they simply won’t be talked about by the reviewer.
Filtered reviews often lack vision of the whole picture. This problem is a major purpose of the Tea of the Month Series, which focuses more on the larger picture of certain tea genres.
Note #1: Ever notice how yelp/amazon reviews tend to veer towards the extremes? Restaurants aren’t necessarily love/hate, but the reviews tend to be. Perhaps, people are more likely to talk about these things if they have a strong opinion on it. The people talking about the restaurant haven’t necessarily picked it at random, they’re reviewing for a reason!
Free Tea, Sugar Coated Reviews & Reciprocation
This is a primary complaint about steepster and to a lesser extent reddit. Free tea is given out and reviews are solicited. While, there’s seemingly nothing wrong with this, it hits one of our basic human characteristics of reciprocation and despite often good intentions will ultimately end up biasing the review. There is a mindset difference between reviewing a free tea vs. a tea that has been purchased. It is difficult to talk poorly about something you’ve been given. If a tea is bad and you’ve been given it for free, you’re less likely to talk about it than if you actually spent money. Similarly if a tea is good and you’ve been given it for free, you’re more likely to write/talk about it than if you actually purchased it.
Another aspect of this is when the reviewer develops some sort of relationship/connection to the person selling the tea. Many people that talk about/review tea inevitably get introduced to vendors. This is good and bad. For the purpose of reviews, it adds another element of bias. It’s hard to talk publicly poorly about teas from someone you know!
Note #1: TeaDB definitely is not immune to any of the above. Much of the tea we and consciously or unconsciously we are affected by how we have received it.
Note #2: Finding bloggers like Marshaln or Hster that are willing to openly call out vendors has become increasingly rare.
Note #3: If you can’t say anything nice, don’t say anything at all! Many opinions, especially negative/more controversial ones are never published. If you want to hear crap tossed around behind the scenes, start a blog!
Note #4: Social Proof and group think also play heavily into teas that people choose to review. How many White Whale reviews came out after Hobbes gave his famous review on it!?
Ratings & Flavors/Tea Quality
If you read or listen to 90% of English tea reviews, the impact of wine culture on western tea culture is pretty obvious. Reviews often grasp pretty far for weird and odd flavors. However, good tea isn’t necessarily about the initial, up-front flavors (important Marshaln posts: It’s Not About the Flavours and Drinking With Your Body). It is often more about the nuances of the tea. The lingering flavor, texture, qi, the body effect.
Brewing method, water, water temperature, storage, and social situations can also all cause major differences in the end product! This can be mitigated with controlled conditions, such as competition style brewing and distilled water. However, a rigorous setup can also eliminate much of the personal enjoyment of tea itself. Distilled water also significantly reduces the mineral content of the water, making an overall worst cup of tea.
Ratings are another very flawed metric. Overall, everyone rates stuff differently. Some people define 50 as the average/median, for others 50 could be a pretty bad cup of tea. This is most apparent in the rating system of steepster or in the standard ratings systems of other beverages.
Note #1: Certain flavors may also not be the same person to person. Apricots can mean one thing to Steve, the tea reviewer, and another to Bob, another tea reviewer.
This article isn’t intended to dismiss tea reviews as useless, but to simply put them in the proper context. Shopping online is tricky and tea reviews can play an important part in the simplifying the purchasing equation to sort through much of the fodder. That being said, there are many flaws and biases inherent within the online review system. Tea reviews should be taken within the proper context with all the necessary grains of salt.