Skip to main content

Testing 10G network with iperf

 

I am testing the real bandwidth I can get with a 10G network connection. My server has an intel X540-AT2 network card with 2 10G interfaces.

The server is configured to use bonding in balance-alb mode, but in this test only one interface comes into play because the iperf client only gets a connection from one MAC address.

$ sudo cat /proc/net/bonding/bond0 
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: adaptive load balancing
Primary Slave: None
Currently Active Slave: eth0
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth0
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:25:90:f9:3d:54
Slave queue ID: 0

Slave Interface: eth1
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:25:90:f9:3d:55
Slave queue ID: 0

 

Frist run in the server iperf in server mode:

$ sudo iperf -s
 ------------------------------------------------------------
 Server listening on TCP port 5001
 TCP window size: 85.3 KByte (default)
 ------------------------------------------------------------

 

An from one client run iperf connecting to the server IP

The first test is using a max transmision unit of 1500 bytes (default)):

$ sudo iperf -c 172.17.16.78 -i1 -t 10 -m
------------------------------------------------------------
Client connecting to 172.17.16.78, TCP port 5001
TCP window size: 92.9 KByte (default)
------------------------------------------------------------
[  3] local 172.17.16.79 port 52458 connected with 172.17.16.78 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 1.0 sec   535 MBytes  4.49 Gbits/sec
[  3]  1.0- 2.0 sec   594 MBytes  4.98 Gbits/sec
[  3]  2.0- 3.0 sec   554 MBytes  4.64 Gbits/sec
[  3]  3.0- 4.0 sec   553 MBytes  4.64 Gbits/sec
[  3]  4.0- 5.0 sec   565 MBytes  4.74 Gbits/sec
[  3]  5.0- 6.0 sec   605 MBytes  5.07 Gbits/sec
[  3]  6.0- 7.0 sec   597 MBytes  5.01 Gbits/sec
[  3]  7.0- 8.0 sec   587 MBytes  4.92 Gbits/sec
[  3]  8.0- 9.0 sec   602 MBytes  5.05 Gbits/sec
[  3]  0.0-10.0 sec  5.67 GBytes  4.87 Gbits/sec
[  3] MSS size 1448 bytes (MTU 1500 bytes, ethernet)

With this test I can barely get around 5Gbits/sec. It seems too poor for a 10G network card.

In the next test I’ve changed the MTU to 9000 (jumbo frames) in all network devices (server, client and switch):

 

$ sudo iperf -c 172.17.16.78 -i1 -t 10 -m
------------------------------------------------------------
Client connecting to 172.17.16.78, TCP port 5001
TCP window size: 92.9 KByte (default)
------------------------------------------------------------
[ 3] local 172.17.16.79 port 52101 connected with 172.17.16.78 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 971 MBytes 8.14 Gbits/sec
[ 3] 1.0- 2.0 sec 1.09 GBytes 9.37 Gbits/sec
[ 3] 2.0- 3.0 sec 1.05 GBytes 9.04 Gbits/sec
[ 3] 3.0- 4.0 sec 1.06 GBytes 9.14 Gbits/sec
[ 3] 4.0- 5.0 sec 1.10 GBytes 9.43 Gbits/sec
[ 3] 5.0- 6.0 sec 1.11 GBytes 9.51 Gbits/sec
[ 3] 6.0- 7.0 sec 1009 MBytes 8.46 Gbits/sec
[ 3] 7.0- 8.0 sec 1.04 GBytes 8.94 Gbits/sec
[ 3] 8.0- 9.0 sec 1.11 GBytes 9.56 Gbits/sec
[ 3] 9.0-10.0 sec 1.07 GBytes 9.21 Gbits/sec
[ 3] 0.0-10.0 sec 10.6 GBytes 9.08 Gbits/sec
[ 3] MSS size 8948 bytes (MTU 8988 bytes, unknown interface)

 

And now, it looks very different, I can saturate the network link.
To change MTU, I’ve changed it in the server and in the client, in all network devices: bond interface and its slaves. Also the switch must support it. You can change the device MTU easly with this command:

$ sudo ip link set dev eth0 mtu 1500

3 thoughts to “Testing 10G network with iperf”

  1. I see you don’t monetize your website, don’t waste your traffic, you can earn extra
    bucks every month because you’ve got hi quality content.
    If you want to know how to make extra money, search for: Boorfe’s tips best adsense
    alternative

  2. I have noticed you don’t monetize your website, don’t waste your traffic, you can earn extra cash every month because you’ve got hi quality content.
    If you want to know how to make extra $$$, search for: Ercannou’s
    essential adsense alternative

Leave a Reply to FirstEula Cancel reply

Your email address will not be published. Required fields are marked *

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.