Archived from groups: comp.dcom.lans.ethernet (More info?)

Hi,

I was wondering if anyone has used Iperf or Netperf for testing network
performance over GigE? The reason for my question is that I've been
doing some testing, initially with Iperf, and recently with Netperf, of
GigE LAN links, and I've been finding results in the 300Mbit/sec range.
The server vendor is implying that these results are not valid, and is
suggesting that I do a file copy of a 36GB file instead and time it,
subtracting the time for a local file copy. I don't mind doing the test
they're suggesting, but I'm just wondering if there's a possibility that
the numbers that I'm getting from both Iperf and Netperf are really
'off'?

Thanks,
Jim

Archived from groups: comp.dcom.lans.ethernet (More info?)

ohaya <ohaya@cox.net> wrote:
> I was wondering if anyone has used Iperf or Netperf for testing network
> performance over GigE?

Yes

> The reason for my question is that I've been doing some testing,
> initially with Iperf, and recently with Netperf, of GigE LAN links,
> and I've been finding results in the 300Mbit/sec range. The server
> vendor is implying that these results are not valid, and is
> suggesting that I do a file copy of a 36GB file instead and time it,
> subtracting the time for a local file copy. I don't mind doing the
> test they're suggesting, but I'm just wondering if there's a
> possibility that the numbers that I'm getting from both Iperf and
> Netperf are really 'off'?

I suspect they are not off, but they may be using TCP settings that
are not optimal for GigE. For example, what are you using for -s and
-S as test-specific parameters in the netperf TCP_STREAM test?

Also, what sort of system are you using, the GigE card, the bus speeds
and feeds all that stuff.

rick jones
--
firebug n, the idiot who tosses a lit cigarette out his car window
these opinions are mine, all mine; HP might not want them anyway...
feel free to post, OR email to raj in cup.hp.com but NOT BOTH...

Archived from groups: comp.dcom.lans.ethernet (More info?)

Ya. Two Dell Poweredge 2650 servers connected to a Nortel Baystack 5510
will run 996Mb/s all day long, jumbo frames enabled. Servers were running
RedHat Enterprise. Needless to say, we use Iperf for performance tuning and
testing all the time. The multicast and udp support is great for QoS
testing.

-mike


"ohaya" <ohaya@cox.net> wrote in message news:416DA35F.A0F52E20@cox.net...
> Hi,
>
> I was wondering if anyone has used Iperf or Netperf for testing network
> performance over GigE? The reason for my question is that I've been
> doing some testing, initially with Iperf, and recently with Netperf, of
> GigE LAN links, and I've been finding results in the 300Mbit/sec range.
> The server vendor is implying that these results are not valid, and is
> suggesting that I do a file copy of a 36GB file instead and time it,
> subtracting the time for a local file copy. I don't mind doing the test
> they're suggesting, but I'm just wondering if there's a possibility that
> the numbers that I'm getting from both Iperf and Netperf are really
> 'off'?
>
> Thanks,
> Jim

Archived from groups: comp.dcom.lans.ethernet (More info?)

ohaya <ohaya@cox.net> wrote:
> Hi,

> I was wondering if anyone has used Iperf or Netperf for testing network
> performance over GigE? The reason for my question is that I've been
> doing some testing, initially with Iperf, and recently with Netperf, of
> GigE LAN links, and I've been finding results in the 300Mbit/sec range.
> The server vendor is implying that these results are not valid, and is
> suggesting that I do a file copy of a 36GB file instead and time it,
> subtracting the time for a local file copy. I don't mind doing the test
> they're suggesting, but I'm just wondering if there's a possibility that
> the numbers that I'm getting from both Iperf and Netperf are really
> 'off'?

As always, benchmarks measure the speed of the particular benchmark.

As regards for network performance, netperf _is_ a very good tool, allowing
to compare different systems. If your system only comes to 300Mbit/sec then
your system is limited to that speed. A vendor that is unable to tune up
might react with bullshit and try to shift your focus.

If you need more speed, you might need radical changes, it could be
anouther OS, or other networking gear. But remember, changing measurment
tools won't give you more performance from your application!



> Thanks,
> Jim

--
Peter Håkanson
IPSec Sverige ( At Gothenburg Riverside )
Sorry about my e-mail address, but i'm trying to keep spam out,
remove "icke-reklam" if you feel for mailing me. Thanx.

Archived from groups: comp.dcom.lans.ethernet (More info?)

Hi Rick,

Comments below...

> > The reason for my question is that I've been doing some testing,
> > initially with Iperf, and recently with Netperf, of GigE LAN links,
> > and I've been finding results in the 300Mbit/sec range. The server
> > vendor is implying that these results are not valid, and is
> > suggesting that I do a file copy of a 36GB file instead and time it,
> > subtracting the time for a local file copy. I don't mind doing the
> > test they're suggesting, but I'm just wondering if there's a
> > possibility that the numbers that I'm getting from both Iperf and
> > Netperf are really 'off'?
>
> I suspect they are not off, but they may be using TCP settings that
> are not optimal for GigE. For example, what are you using for -s and
> -S as test-specific parameters in the netperf TCP_STREAM test?
>
> Also, what sort of system are you using, the GigE card, the bus speeds
> and feeds all that stuff.


With the netperf testing so far, I just used the default settings. I
was assuming that this should give us at least "an idea" of what the
actual throughput was?

I've been using iperf more extensively, because I couldn't find netperf
until a couple of days ago.

Needless to say, I was surprised with the results I got from iperf, and
then when I finally got a working netperf, those numbers came in about
the same.

System under test consisted of two IBM blade servers (HS40) with 4 x
Xeon 3.0 GHz CPUs, 16GB of memory, and 4 x Intel/1000 NICs onboard.
Connection between the blades (for these tests with netperf) was simply
a fiber cross-over cable.

Jim

Archived from groups: comp.dcom.lans.ethernet (More info?)

ohaya wrote:
>
> Hi Rick,
>
> Comments below...
>
> > > The reason for my question is that I've been doing some testing,
> > > initially with Iperf, and recently with Netperf, of GigE LAN links,
> > > and I've been finding results in the 300Mbit/sec range. The server
> > > vendor is implying that these results are not valid, and is
> > > suggesting that I do a file copy of a 36GB file instead and time it,
> > > subtracting the time for a local file copy. I don't mind doing the
> > > test they're suggesting, but I'm just wondering if there's a
> > > possibility that the numbers that I'm getting from both Iperf and
> > > Netperf are really 'off'?
> >
> > I suspect they are not off, but they may be using TCP settings that
> > are not optimal for GigE. For example, what are you using for -s and
> > -S as test-specific parameters in the netperf TCP_STREAM test?
> >
> > Also, what sort of system are you using, the GigE card, the bus speeds
> > and feeds all that stuff.
>
> With the netperf testing so far, I just used the default settings. I
> was assuming that this should give us at least "an idea" of what the
> actual throughput was?
>
> I've been using iperf more extensively, because I couldn't find netperf
> until a couple of days ago.
>
> Needless to say, I was surprised with the results I got from iperf, and
> then when I finally got a working netperf, those numbers came in about
> the same.
>
> System under test consisted of two IBM blade servers (HS40) with 4 x
> Xeon 3.0 GHz CPUs, 16GB of memory, and 4 x Intel/1000 NICs onboard.
> Connection between the blades (for these tests with netperf) was simply
> a fiber cross-over cable.


Hi,

Sorry, forgot to mention that both systems are running Windows 2003
Server.

Jim

Archived from groups: comp.dcom.lans.ethernet (More info?)

Michael Roberts wrote:
>
> Ya. Two Dell Poweredge 2650 servers connected to a Nortel Baystack 5510
> will run 996Mb/s all day long, jumbo frames enabled. Servers were running
> RedHat Enterprise. Needless to say, we use Iperf for performance tuning and
> testing all the time. The multicast and udp support is great for QoS
> testing.
>


Mike,

Thanks for the info. Actually, that gives me an idea. We have some
Dell PowerEdges with GigE NICs sitting around somewhere. I'll see if I
can try out Iperf and/or Netperf on them and see what I get.

Jim

Archived from groups: comp.dcom.lans.ethernet (More info?)

> As always, benchmarks measure the speed of the particular benchmark.
>
> As regards for network performance, netperf _is_ a very good tool, allowing
> to compare different systems. If your system only comes to 300Mbit/sec then
> your system is limited to that speed. A vendor that is unable to tune up
> might react with bullshit and try to shift your focus.
>
> If you need more speed, you might need radical changes, it could be
> anouther OS, or other networking gear. But remember, changing measurment
> tools won't give you more performance from your application!


Peter,

Thanks for the advice.

I think/hope that you're aware of what I've been attempting to do, based
on my earlier thread, and personally, I agree that at this point, the
vendor is reacting with "b....".

Nevertheless, it looks like I'm going to have to do their "manual copy"
test to satisfy them that there's a problem in the first place, even
though I think that tools like Iperf and Netperf do a better job because
they're specifically designed for what they do. Otherwise, so far, it
doesn't look like they're going to even look into this problem.

I guess that we've all "been there, and done that" with our vendors
...

Jim

Archived from groups: comp.dcom.lans.ethernet (More info?)

> I suspect they are not off, but they may be using TCP settings that
> are not optimal for GigE. For example, what are you using for -s and
> -S as test-specific parameters in the netperf TCP_STREAM test?


Hi Rick,

I can't see any -s or -S parameters? What I'm going by is a man page
at:

http://carol.science.uva.nl/~jblom [...] tperf.html

Also tried a "-h" and didn't see any -s or -S there?

FYI, the binaries that I have are 2.1pl1. The www.netperf.org site
doesn't seem to be working anymore, so these were the only binaries I
could find for Win32, on a site in Japan, I think.

Jim

P.S. Are you "the" Rick Jones, the originator of Netperf?

Archived from groups: comp.dcom.lans.ethernet (More info?)

ohaya wrote:

> Hi Rick,
>
> Comments below...
>
>> > The reason for my question is that I've been doing some testing,
>> > initially with Iperf, and recently with Netperf, of GigE LAN links,
>> > and I've been finding results in the 300Mbit/sec range. The server
>> > vendor is implying that these results are not valid, and is
>> > suggesting that I do a file copy of a 36GB file instead and time it,
>> > subtracting the time for a local file copy. I don't mind doing the
>> > test they're suggesting, but I'm just wondering if there's a
>> > possibility that the numbers that I'm getting from both Iperf and
>> > Netperf are really 'off'?
>>
>> I suspect they are not off, but they may be using TCP settings that
>> are not optimal for GigE. For example, what are you using for -s and
>> -S as test-specific parameters in the netperf TCP_STREAM test?
>>
>> Also, what sort of system are you using, the GigE card, the bus speeds
>> and feeds all that stuff.
>
>
> With the netperf testing so far, I just used the default settings. I
> was assuming that this should give us at least "an idea" of what the
> actual throughput was?
>
> I've been using iperf more extensively, because I couldn't find netperf
> until a couple of days ago.
>
> Needless to say, I was surprised with the results I got from iperf, and
> then when I finally got a working netperf, those numbers came in about
> the same.
>
> System under test consisted of two IBM blade servers (HS40) with 4 x
> Xeon 3.0 GHz CPUs, 16GB of memory, and 4 x Intel/1000 NICs onboard.
> Connection between the blades (for these tests with netperf) was simply
> a fiber cross-over cable.

I can't find a match on the Serverworks site for the chipset that is
supposed to be on that board, but one possible would be the "HE" chipset,
which has onboard 32/33 PCI and an IMB link that allows connection of a
64/66 or PCI-X southbridge. If the Ethernet is on the 32/33 PCI that would
explain the poor performance you're seeing. Just for hohos, try each
Ethernet port in turn, using the same port on both blades--it may be that
one or two are on 32/33 and the others are on the fast bus. I realize it's
a long shot, but it's simple and obvious.

Also, are you _sure_ you've got a good cable.

And is there any possibility that there's a duplex mismatch? Did you
connect the cable with both blades powered down? If not, it may be that
the NICs did not handshake properly--they're _supposed_ to I know but
what's supposed to happen and what does happen aren't always the same.

> Jim

--
--John
Reply to jclarke at ae tee tee global dot net
(was jclarke at eye bee em dot net)

Archived from groups: comp.dcom.lans.ethernet (More info?)

John,

Comments below...

Jim


> I can't find a match on the Serverworks site for the chipset that is
> supposed to be on that board, but one possible would be the "HE" chipset,
> which has onboard 32/33 PCI and an IMB link that allows connection of a
> 64/66 or PCI-X southbridge. If the Ethernet is on the 32/33 PCI that would
> explain the poor performance you're seeing. Just for hohos, try each
> Ethernet port in turn, using the same port on both blades--it may be that
> one or two are on 32/33 and the others are on the fast bus. I realize it's
> a long shot, but it's simple and obvious.

I asked IBM specifically about the interface, and they said they were
PCI-X. Of course, they could be wrong. Also, I've tried between combos
among 4 servers already.

Re. cables, I've tried several fiber cables.

Here's the page listing the driver:

http://www-307.ibm.com/pc/support/ [...] MIGR-54801

I used the one:

"Intel-based Gigabit and 10/100 Ethernet adapter drivers for Microsoft
Windows 2000 and Microsoft Windows Server 2003"


> Also, are you _sure_ you've got a good cable.

See above.


>
> And is there any possibility that there's a duplex mismatch? Did you
> connect the cable with both blades powered down? If not, it may be that
> the NICs did not handshake properly--they're _supposed_ to I know but
> what's supposed to happen and what does happen aren't always the same.


That's a good hint. For the tests via a GigE switch, the servers were
connected to the switch prior to power-on (no choice). For the tests
via fiber cross-over cable, I plugged the fiber together after power on.

I'll try some tests powering the servers off, connecting the cables, the
powering the servers on, if I can.

Archived from groups: comp.dcom.lans.ethernet (More info?)

ohaya <ohaya@cox.net> wrote:
> I asked IBM specifically about the interface, and they
> said they were PCI-X. Of course, they could be wrong.
> Also, I've tried between combos among 4 servers already.

OK, I'm a little late to this thread, but have you
tried UDP? MS-Windows used to have horrible problems
setting adequate TCP-Rcv-Windows sizes.

Personally, I use `ttcp` for bandwidth testing.

-- Robert

Archived from groups: comp.dcom.lans.ethernet (More info?)

ohaya <ohaya@cox.net> wrote:
> With the netperf testing so far, I just used the default settings. I
> was assuming that this should give us at least "an idea" of what the
> actual throughput was?

"Typically" (as if there really is such a thing) one wants 64KB or
larger TCP windows for local gigabit. Default netperf settings simply
take the system's defaults which may not be large enough for
maximizing GbE throughput.

> System under test consisted of two IBM blade servers (HS40) with 4 x
> Xeon 3.0 GHz CPUs, 16GB of memory, and 4 x Intel/1000 NICs onboard.
> Connection between the blades (for these tests with netperf) was simply
> a fiber cross-over cable.

Is that 4X Intel/1000 on each blade, or are they on the chassis?
Windows or Linux? I'd check CPU util if possible - although don't put
_tooo_ much faith in top.

rick jones
--
a wide gulf separates "what if" from "if only"
these opinions are mine, all mine; HP might not want them anyway...
feel free to post, OR email to raj in cup.hp.com but NOT BOTH...

Archived from groups: comp.dcom.lans.ethernet (More info?)

ohaya <ohaya@cox.net> wrote:
> I can't see any -s or -S parameters? What I'm going by is a man page
> at:

> http://carol.science.uva.nl/~jblom [...] tperf.html

> Also tried a "-h" and didn't see any -s or -S there?

-s and -S are "test specific" options. Help for test-specific options is diplayed when you specify a test type and the -- -h:

$ ./netperf -t TCP_STREAM -- -h

Usage: netperf [global options] -- [test options]

TCP/UDP BSD Sockets Test Options:
-C Set TCP_CORK when available
-D [L][,R] Set TCP_NODELAY locally and/or remotely (TCP_*)
-h Display this text
-I local[,remote] Set the local/remote IP addresses for the data socket
-m bytes Set the send size (TCP_STREAM, UDP_STREAM)
-M bytes Set the recv size (TCP_STREAM, UDP_STREAM)
-p min[,max] Set the min/max port numbers for TCP_CRR, TCP_TRR
-P local[,remote] Set the local/remote port for the data socket
-r req,[rsp] Set request/response sizes (TCP_RR, UDP_RR)
-s send[,recv] Set local socket send/recv buffer sizes
-S send[,recv] Set remote socket send/recv buffer sizes

For those options taking two parms, at least one must be specified;
specifying one value without a comma will set both parms to that
value, specifying a value with a leading comma will set just the second
parm, a value with a trailing comma will set just the first. To set
each parm to unique values, specify both and separate them with a
comma.

> FYI, the binaries that I have are 2.1pl1. The www.netperf.org site
> doesn't seem to be working anymore, so these were the only binaries I
> could find for Win32, on a site in Japan, I think.

www.netperf.org should be up - i'll double check it. While netperf
sources are up to 2.3pl1 now, which includes some non-trivial Windows
re-integration, there aren't binaries for it from netperf.org/ftp.cup.

> P.S. Are you "the" Rick Jones, the originator of Netperf?

Yes. These days I call myself the "Contributing Editor"

rick jones
--
Wisdom Teeth are impacted, people are affected by the effects of events.
these opinions are mine, all mine; HP might not want them anyway...
feel free to post, OR email to raj in cup.hp.com but NOT BOTH...