6. Routing Performance

Testing Notes:
• WAN to LAN tests are all run with LAN endpoint in DMZ
• LAN to WAN tests are run with LAN endpoint not in DMZ, except UDP Stream

The truth be known, I've been having a hell of a time with my normal testing tool, Qcheck, since I've started testing these ADMtek-based routers. Although routers have claimed to be using SPI (Stateful Packet Inspection) before, it seems that the ADMtek-based boxes really are using SPI, or at least doing something different in their implementation.

At any rate, these routers sent me off on a few weeks of experimentation with Qcheck and Chariot, the two NetIQ testing tools that I rely on for my most of my testing. After a lot of back-and-forth with NetIQ Tech Support, experimentation with various Endpoint versions (an Endpoint is a little application that runs in the background of your machine, waiting for test instructions from the Qcheck or Chariot console), and a nice new flat spot on my forehead from banging it against the wall, I finally think I've come up with a new test method for SPI+NAT routers... and I have the VBR to thank.

Turns out that the VBR's ability to fine-tune its NAT settings (Intrusion Detection section) allowed me to finally decide that the reduced throughput that I was seeing on LAN-WAN tests when I put the LAN machine in DMZ really could be attributed to the interaction between Qcheck / Chariot and the SPI in the router. The VBR allowed me to leave the LAN endpoint in DMZ, and disable SPI, therefore separating the DMZ and SPI effects. With other products all I could do was move the LAN endpoint in and out of DMZ.

So that my test results are as apples-to-apples as I can get them for SPI+NAT routers and given Chariot's "can't handle SPI+NAT" warning, I went back and retested the D-Link DI-604 [reviewed here] and NETGEAR FR114P [reviewed here] using the same settings. I strongly suggest you read the new test method info, so that you don't misinterpret the results from my new test method.

As I said, the SPI-tuning features allowed me to explore the VBR a little more, so I'll present my findings in another handy chart:

Since WAN-LAN tests won't work unless the LAN endpoint is in DMZ, I could only compare results with SPI enabled and disabled, but the results in Cases 1 and 2 show a 5X throughput improvement with SPI disabled. Comparing Cases 3 and 6 shows an almost 30% throughput improvement from disabling SPI with the LAN endpoint not in DMZ, and an almost 4X improvement with the endpoint in DMZ (Cases 4 and 5). The last point to note is that the UDP LAN-WAN test would run, but would not report back results.

What's all this mean for real-world performance? I'd say that either putting a LAN client in DMZ (or forwarding specific ports to a LAN-side server) with SPI enabled will result in the lower throughput performance, although the actual numbers you get may vary from my Chariot-based measurements. But before you turn off SPI in search of higher throughput, consider the typical 1 to 1.5Mbps speed of most broadband connections, and that the "exposed" computer will be protected against nasty stuff like fragmented packet, syn flood, and DoS type attacks, which is protection well-worth the throughput hit.

Routing Performance Test Results

See details of how we test.