How Tom's Guide Tests and Reviews TVs

Reviewing televisions can easily be a subjective affair overly influenced by the eyes and ears of the reviewer. To mitigate such subjectivity, at Tom's Guide we put each television we review through a series of instrument tests to measure color accuracy and contrast. We use the results as a reality check on reviewers' subjective impressions of color, contrast and detail in the tests with real-world content.

Testing Equipment

We take measurements in a completely dark room using a Klein K-10A colorimeter. This cool device can measure light output from as low as 0.0006 nits (virtually pitch black) to 10,000 nits (full sunlight on a cloudless day). One glitch: The Klein K-10A's accuracy can "drift" over time. To ensure accuracy, we calibrate the colorimeter with an X-Rite i1 Basic Pro 2 spectrophotometer. This device can't handle low light as well as a colorimeter can, but it's significantly more accurate in the long term.

Objective Instrument Tests

We use SpectraCal's Client 3 software on a laptop to generate screen test-patterns for the device to read. We record the readings to the same laptop using SpectraCal's Calman 5 software, which analyzes the data and makes pretty tables and graphs of it.

We test all the preset image modes that a TV offers, such as Vivid, Movie or Sports, and use the best-performing mode for our subjective tests (though our reviewer may also try out other modes).

MORE: Best TVs Available Now

Here are the main measurements we use to objectively evaluate TV performance.

Color Accuracy and Gamut

A gamut represents every color a television is capable of producing, which is still a subset of every color the human eye can see. For Full HD TVs (720p and 1,080p) that gamut is called Rec. 709. New UHD (4K) televisions can reproduce even more colors in a larger gamut known as Rec. 2020.

We compare colors in these gamuts to what the TVs we're testing can reproduce. This means measuring test patterns on the TV that use all three primary colors (red, green and blue) and all three secondary colors (cyan, magenta and yellow).

We also measure dozens of lighter shades in both primary and secondary colors. From this we can determine, for example, if what should be a medium magenta comes out a bit purple, or if a light yellow is too orange.

In the chart above, the light triangle represents the target gamut for the type of TV we are reviewing (in this case, Rec. 709 for HDTVs). The squares represent the colors of the test patterns we send to the TV, and the dots represent the color the TV actually displayed.

Color Temperature

When the color temperature on your TV is too high, skies will appear vibrantly blue, but people may seem gray, or even purple. If the color temperature is too low, a sunset will look gorgeous, but the people on screen will look Oompa-Loompa orange. Every film you watch and video game you play is created on a screen with a color temperature calibrated to 6,500 Kelvin (K), which is roughly the color temperature of sunlight. Thus your TV screen should reproduce images at or close to the 6,500-K ideal. However, many people actually prefer higher temperatures (bluer images). Thus many televisions can ship with a preset temperature of 7,000 K; 8,000 K; or even 10,000 K!

We measure color temperature in 10-percent brightness increments from black to brightest white, and average the readings to report overall color temperature for the TV.

On/Off Contrast Ratio

When you're in the store comparing two TVs, contrast ratio will likely be the first thing you notice. A high contrast ratio makes images appear richer and more saturated. The problem: There is no universal standard for measuring contrast ratio, and every television manufacturer uses a different method. You can't compare the contrast ratio specs of, for example, a Vizio and a Samsung TV.

MORE: Best 32-inch TVs

We test all TVs using the same method, so we can make direct comparisons among different models. We use the same pitch-black room for each test and then compare brightness of the whitest white and blackest black the TV can display. A higher number is better, with anything above 500:1 being comparable to the contrast ratio found on a typical movie theater screen.

Gamma vs. Contrast

Ever watched a scary movie and been annoyed because everything is so bright you can too easily see the monster lurking in the shadows? That's probably because the TV's gamma is too low.

Whereas contrast ratio compares the whitest white to the darkest black, gamma concerns the "mid-tones" in between. The lower the gamma, the more washed out the shadows may appear, and the higher the gamma, the less detail found in those shadows. The ideal gamma for a TV is generally accepted as 2.2, though in our tests, we've seen that many TV makers set it lower.

To determine the gamma when testing, we use what's called a 10-IRE test point pattern. This displays 10 shades of gray from white to black and averages the gamma of all 10 points.

Subjective Evaluation by Reviewer

After lab testing, we wheel the TV to the reviewing facility — an acoustically treated room with flexible lighting to simulate home viewing. 

Our reviewer watches a variety of content, including Blu-ray versions of movies and TV shows, and also plays current video games — all on a Sony PS4. To see how the TV handles less-than-perfect content, we also watch recorded television, as well as 480p DVDs with the PS4's upscaling disabled. To replicate online services such as Netflix or Hulu, we stream 1,080p material over a Roku 3 from a computer running Plex media server software. This program allows us to vary the data rate of the stream to better replicate fluctuating real-world Internet speeds.

Content passes through an amplified HDMI signal splitter, so the reviewer can compare several TVs at once.

Related Buying Guide:
Best Soundbar Speakers
Best Streaming Media Players
Best Headphones and Earbuds
Create a new thread in the Streaming Video & TVs forum about this subject
This thread is closed for comments
1 comment
    Your comment
  • Anonymous
    Like may professionals I use LightSpace CMS for all display verification and calibration.

    It is the most common display calibration system in use in the professional (Film and TV) industry.

    There is also a free 'verification' version that can be used to check any display.
    0