We get a lot email and comments from our readers debating the value ratings that our staff put on our various specialty audio/video product reviews. Some think the ratings are a little too high. Others think we might have shorted a specific product. By all means, our readers are entitled to their thoughts and encouraged to share them (best in the comments section so everyone can read their thoughts too). In an ongoing staff dialog, a lot of consideration goes into the Performance and Value ratings so that our content is both fair and useful. Here's a little insight into how we come up with our ratings.
First off: we never, ever set out to review crap gear. There might be other AV publications looking for that "gotcha moment," but we realistically only have 104 reviews (52 featured and 52 of the quick variety) per calendar year to cover products from roughly 28 categories. Those 104 reviews are nowhere near enough to cover the thousands upon thousands of components that we'd like to review, but we simply don't have the resources and space to publish more. For this exact reason, we try to seek out what we imagine will be the best of the best when it comes to AV products to evaluate. When we say the "best of the best," we mean both super-high-value products as well as super-high-performance products. Road and Track puts Ferraris on the cover of their magazine for a reason. They also review an occasional Prius or a Volkswagen GTI for a more real-world taste of the types of cars actual people buy. We try to balance things that way, too. Of course you want to read about the $3,500 Sony Masters Series reference level OLED 4K TV, but you also might want to hear how the company's $1,295 LED compares (it's scary close when the picture is calibrated) which creates some interesting conversations and buying decisions.
Audiophiles know that they have to pay for the Nth degree of performance. The fact is: we all do. Products that push the performance barrier are often much more expensive than more mainstream products. If you need to be on the bleeding edge of performance, you are going to have to pay to get there.
As such, value calculations have to be made on an individual basis. Does this component do something that no other product does? Does it break performance barriers? Does it have good distribution? Will it have a reasonable resale value down the road? How does its performance compare with other products in its class (if there are even competitors)? Many of these factors go into the consideration for a value rating.
Beyond just posting arbitrary star ratings on our 5-star scale for value, our reviews must have a well thought out Competition and Comparison section. Writers are expected to look at the review product in its competitive matrix. A $2,000 pair of floorstanding speakers are going to have a lot of competition. What makes this pair a 4-star value versus a 5-star or even a 3-star? That's up to the reviewer, but a lot of discussion goes into that rating. Going forward, we are trying to get the reviewers to explain in even more detail as to why they selected their value and performance ratings so that you, the reader, have an idea of where they are coming from. By all means you might disagree, and who's to say that either of you are wrong? Likely, neither of you are that off-base. The fact is: we aren't here to sell gear like a traditional e-commerce site (although you can buy stuff using our Amazon.com buy-it-now buttons as a convenience, which helps fund our free content model a little bit) as much as we are here to start a conversation about your next upgrade and to help you with your AV research. That's our goal.
One thing readers likely never see is when we receive a true stinker of a product. First of all, there aren't very many lousy AV products anymore. Today's AV gear in the specialty space is almost always really strong performers. We also spend a lot of time at trade shows hand-picking the products to be reviewed in coming months. That's not to say there isn't an outlier from time to time. If a product is defective (it happens) we generally rush to get it replaced and start the review process over. It's painful and costly but well worth the effort. If the product just outright sucks, sometimes we just send it back. We can come back to said company some other time and give them another shot at ink later. This is rare instance but it does happen.
Overall, I think review values tend to trend to the high side and that is by design. With the small sample size of reviews that we are able to publish, we just aren't looking for mediocre products. We ask the companies which products from their offerings are the best value and they often are more than willing to suggest where the sweet spot is for their gear and that's often what we review. I just don't see as much value dragging mediocre products through the process when we can hunt down the gems up front.
How valuable are HomeTheaterReview.com's review ratings for your buying decisions? Comment below as we always care to hear your thoughts on these important matters.