7. Intel's DHCAT Benchmark For HTPCs: What Is It

This review features our first use of a new benchmarking tool. Built and provided by Intel, the Digital Home Capabilities Assessment Tool, aka DHCAT, seeks to measure the capabilities of a digital home entertainment platform. These tests are based on how people actually use such machines in the home to deal with all kinds of media and standard playback and usage scenarios.

DHCAT is based on standards from the Digital Living Network Alliance (DNLA). This tool exercise a platform's hardware and software components by running typical tasks that users perform. Thus, for example, the tool will work equally well with Windows Media Video, MPEG-2, and DIVX video formats and related codecs and software tools. Beginning with simple, single-use workloads, DHCAT presents test machines with increasingly tougher workloads, and ultimately piles on video playback, streaming, recoding and transcoding activities. Systems that handle such workloads earn higher scores, but only if they also produce a good experience for the people using them at the time.

DHCAT consists of a collection of usage primitives, each of which corresponds to some typical media task: recording, playing back or transcoding audio, playing back or transcoding TV and/or HDTV streams, and so forth. Usage primitives are organized into scenarios, where anywhere from 1 to 4 simultaneous primitives are run. Scenarios may then be combined into situations that represent typical connected use on a local platform and a local network, HDTV recording, playback, and transcoding, and basic TV recording, playback and transcoding.

Intel uses some sophisticated tricks to level the playing field among multiple systems, such as a virtual TV tuner that hands off MPEG-2 streams for both SDTV and HDTV to the video decoder as much as possible like a physical tuner would. A virtual model for a digital media adapter (DMA) likewise lets the tests compensate for variations in network latency that might otherwise skew test results by handing off a video stream locally that would otherwise have to travel across a real network. Finally, DHCAT also uses a technique called perceptual modeling to measure subjective perceptions of video quality using standardized, objective metrics. Video playback, recoding, and streaming are modeled and dropped frames and frame quality subjected to close scrutiny and measurement throughout. The technique uses a mean opinion score (MOS) very similar to that used to measure voice quality in VoIP systems.

Measuring The Performance Of Our Four Mini PCs

The following chart shows a summary value for each of our mini HTPCs. To understand this value you need to appreciate what we had to say in the previous section and what we say here. However, knowing that higher values indicate PCs better suited to multimedia applications, it is pretty obvious that whatever differences there are between our four mini PCs are related mostly to CPU power. Notice, for example, that the Shuttle's fancy AIT Mobility Radeon graphics processor didn't help raise the PC out of last place. Read on for more on how we got the summary values shown in the following chart.

The DHCAT main results appear in a single graph, broken up into individual areas for Basic, HD, and Connected usage groups. These are shown in Figure 1, where the green areas inside each of the three regions indicates how the test machine compares to a theoretically perfect platform (Basic looks okay here, but HD and connected appear somewhat below average). The yellow bar at the bottom of the figure (which Intel calls a "wavelet" states a Platform Capability Score, which represents MOS values for the system when running extra credit scenarios. This metric also provides a good way to compare systems to one another. This is the summary score in the chart above.

Figure 1: The DHCAT main results screen shows how a platform does in each of the three usage groups, along with a single Platform Capability Score, or PCS (201 in the example shown).

Clicking the details button produces what Intel calls a Platform Capability Matrix, which provides individual measures for each constituent scenario in the usage group. In figure 2, green indicates an excellent experience, yellow a fair to good experience, and red a poor experience or some failure condition in the scenario. The individual elements support the relative rankings for the Basic, HD, and Connected usage group elements shown in Figure 1.

Figure 2: The detail results provide MOS information for each constituent scenario in the three usage groups, where green is best, yellow intermediate, and red worst.

An additional detailed report provides information down the lowest level of granularity, including even individual scores for specific video codecs (WMV, MPEG-2, or DivX) for each task in any given scenario. All scores are stored in an XML file, which makes it easy to present the results in a Web browser, or to transport the data into Excel or a database for further analysis or interpretation. The bottom line is that DHCAT offers a better way to evaluate a system's media handling capabilities than other standard benchmarks we've used. That said, we're not enamored of the requirement to emulate a DLNA media server to complete these tests (not part of MCE 2005, and implemented with varying degrees of fidelity in other media environments such as MythTV, SageTV, and Beyond TV) but it's the first standard test battery we've found that lets us dig into this area, so we're taking it under advisement for the foreseeable future.

Join our discussion on this topic