How We Evaluate and Measure HDTVs

By |

How We Evaluate and Measure HDTVs


How-we-measure-TVs-small.jpgAs the lead video reviewer on the HomeTheaterReview.com team, I evaluate the vast majority of TVs that are made available to us for review. I recently adopted a new review methodology for both TVs and projectors, so now is a good time to talk about how we review and measure displays. More importantly, it's a good time to go over some the basic terms that we often use in our reviews. If you're new to the video enthusiast world, you've probably heard the word calibration mentioned on more than one occasion, but what does it mean to calibrate a display? What do all the technical terms and numbers mean in terms of real-world performance, and why do we recommend calibration so strongly?

Calibration is the process of adjusting the display's advanced picture controls to get attributes like color, white balance, and gamma close to reference standards, as defined by the Society of Motion Picture and Television Engineers (SMPTE). These standards are the same ones used on the production side of the equation. When your TV is calibrated to the same standard as the professional monitors used during the mastering of a film or TV show, you can be confident that you are seeing exactly what the director intended. If a movie's color tone looks overly blue or green (think The Matrix), or the colors seem muted or exaggerated, it's because that is how the director wanted the movie to look, not because your TV is inaccurate.

During the review process, we measure the TV with a meter and look at several performance attributes: color temperature, color balance, color points, gamma and light output. Without going too deep into tech speak, color temperature and color balance deal with the neutrality of the image - specifically, the neutrality of white. Is the tone or "color" of white too blue, too red, or right at the standard that SMPTE has set for film and TV programming: around 6,500 Kelvin (or D65)? In terms of color, we measure the TV's primary (red, green, and blue) and secondary (cyan, magenta, and yellow) color points to see if they meet the current Rec. 709 standard. In other words, are the colors the correct hue, the correct level of saturation, and the correct brightness? Do reds look more orange? Do greens have a neon quality?

A very basic definition of gamma is that it describes how quickly a display comes out of black, or moves from black to white. A lower number means that the display's black level gets brighter faster (which is good for bright-room viewing), while a higher number means that the blacks get brighter more slowly (which is good for dark-room viewing of film content). For HD content, we use a gamma target of 2.2 for TVs and a slightly darker 2.4 for projectors.

In terms of overall brightness of the HD display, THX recommends an image brightness of about 35 foot-lamberts for direct-view TVs (14 ftL for projectors), while SMPTE suggests anywhere from 30 to 40 ftL (12 to 16 ftL for projectors).

An ISF or THX video calibrator will use the TV's suite of advanced picture controls to bring these various picture attributes as close as they can to the reference standards. The phrase Delta Error describes how close the attribute is to the standard, with zero being right on the mark. Anything under 10 is considered tolerable, anything under five is considered very good for a consumer-grade TV, and anything under three is considered to be imperceptible to the human eye. So, in our reviews, if we describe something as having a DE under three, then you know it's as accurate as it needs to be.

SpectraCal-logo.jpgMy evaluation equipment consists of Portrait Displays' Spectracal CalMAN software running on a Lenovo PC, an X-rite i1Pro 2 spectrophotometer, and a DVDO iScan Duo HD pattern generator.

Whether we like it or not, the fact is that the majority of TV owners will never have their TVs calibrated and probably won't even take the time to adjust basic controls like color and contrast. With that truth in mind, my first step in evaluating a TV is to measure several of the picture modes exactly as they come out of the box. I don't adjust any of the basic controls, nor do I turn off automatic functions like de-judder, dynamic black/contrast, etc. This way, I can tell the readers which mode(s) will give them the best performance with no further work required on their part, beyond changing the picture mode. (Spoiler alert: in most cases, the mode labeled Cinema, Movie, or THX is going to be the most accurate, but sometimes it helps to know what the second most accurate mode is, too.) If significant improvement can be made just by tweaking basic controls like contrast, brightness, color, tint and color temperature, I will try to point that out. A number of discs are available that will teach you how to adjust these basic controls, including DVE: HD Basics, Disney's World of Wonder, and Spears & Munsil: HD Benchmark 2nd Edition.

My next step is to calibrate the best picture mode(s) to see how close to accurate they can be. It's very rare that a TV's performance can't be improved by calibration - unless it was perfect to begin with, and that's rare, too. Even if the TV's main setup menu lacks advanced picture controls to do a full calibration, a certified calibrator usually has access to the necessary code to enter the TV's service menu and make the needed adjustments. (Attempting to access the service menu on your own is a bad idea and can void the TV's warranty, so don't try this at home.) What we want to provide in our reviews is a clear idea of the before and after. Knowing just how much improvement you can get, just how accurate your TV can be, will help you decide whether to invest a few hundred dollars extra to get it calibrated. On occasion, circumstance may mandate that someone else on the HTR team does a display review, in which case the method may change slightly; however, even in those instances, we will make every effort to measure and calibrate the TV as part of the review process, even if it requires hiring a local calibrator to do it.

Our display evaluations don't end there. Plenty of other attributes must be considered as well. One of the most important attributes of TV performance is black level, or the TV's ability to produce a truly deep black. A deep black level produces a richer, more saturated image, especially in a dark room, whereas a lighter black level can make the picture look flat and washed out. We also look at the TV's ability to render the finest, most subtle details in dark scenes to make sure important information isn't being "crushed" right out of the picture. Beyond test patterns that help set black level, I have a few real-world favorites to help evaluate black level and detail: chapter one of The Bourne Supremacy (DVD, Universal), chapter four of The Pirates of the Caribbean: The Curse of the Black Pearl (BD, Buena Vista), and chapter two of Flags of Our Fathers (BD, Paramount), among others. Screen uniformity is another important factor: are certain areas of the screen brighter than others? This will be most apparent (and most distracting) when viewing dark images. How does the TV perform in a brighter viewing environment: does the screen still produce a well-saturated image, or does the picture get washed out by the room's lighting? How reflective is the screen? How detailed is the image, and does the detail remain intact during fast-moving scenes? I use a Blu-ray disc called FPD Benchmark that was made available to reviewers many trade shows ago to evaluate a TV's motion resolution. One scene in particular shows resolution patterns at DVD480, HD720, and HD1080 resolutions; when the pattern begins to move, you can see how much detail is lost or preserved. Finally, we test the TV's processing abilities. Using discs like the HQV Benchmark DVD and HQV HD Benchmark BD from Silicon Optix, we can see if a TV correctly handles different film and video cadences to produce an image free of digital artifacts like jaggies and moiré. One of my pet peeves is a noisy TV, one in which you can see lots of pixellation, odd color shifting, and/or banding in backgrounds and light-to-dark transitions.

All of these attributes contribute to the success or failure of a TV's performance. While many of them can be evaluated by objective standards, the sum of all the parts definitely has a subjective element. In the audio world, not everyone likes the sound of a speaker that measures perfectly flat; the same can be true in the video realm, where you might prefer a slightly cooler image (lots of people do), more light output, a higher gamma, etc. Also, how enjoyable is the TV to use on a daily basis? No matter how well a TV measures on paper, the point can be rendered moot if it has some quirky flaw that annoys you every time you turn it on. So, after the meters and test DVDs have been put away, it simply comes down to living with each TV for several weeks to see whether or not it's a product we'd want to live with for a whole lot longer.

Additional Resources
• Read more original content like this in our Feature News Stories section.
• Explore more news in our LED HDTV, LCD HDTV, and Plasma HDTV news sections.
• See reviews in our HDTV Review section.


  • Comment on this article

Post a Comment
comments powered by Disqus

Latest Feature News Stories

Oct 09
HDMI 2.1 Is Coming Soon to AV Gear Near You We should begin to see AV gear with HDMI 2.1 hit the market soon. What does the latest HDMI spec offer, and how will it affect your current system? Adrienne Maxwell answers those questions in this week's featured news story.
HDMI 2.1 Is Coming Soon to AV Gear Near You

Oct 02
Are AV Manufacturers Finding Success Via Crowdfunding? Is crowdfunding the best way for a small or new AV manufacturer to finance a product? Have crowdfunded AV products enjoyed success? Jeff Berman explores the topic in this week's featured news story.
Are AV Manufacturers Finding Success Via Crowdfunding?

Sep 25
Room Correction Revisited Dennis Burger revisits the topic of room correction, expanding on some of the original explanations from his 2013 primer and providing an updated look at the major room correction systems in use today.
Room Correction Revisited

Sep 18
CEDIA 2017 Show Wrap-up The annual CEDIA Expo--which celebrates high-end home theater, whole-house AV, and all things custom-oriented--took place recently in San Diego. Adrienne Maxwell and Dennis Burger discuss the latest trends and new products on display.
CEDIA 2017 Show Wrap-up

Sep 11
Do You Own an AV Boat Anchor? AV technology is always advancing (sometimes quite quickly), and that means some pieces of gear inevitably get left behind. Do you own a product that became obsolete far sooner than you would've preferred?
Do You Own an AV Boat Anchor?