network – Page 2 – Elkano.org Blog!

Check if jumbo frames are working

August 27, 2014August 27, 2014 sysadmin Leave a comment

This is a simple post showing how to test if large MTU is working in your network using ping tool:

When jumbo frames are not enabled in all networking devs:

# ping -s 8000 -M do -c 5 172.16.64.75
PING 172.16.64.75 (172.16.64.75) 2000(2028) bytes of data.
From 172.16.64.68 icmp_seq=1 Frag needed and DF set (mtu = 1500)
From 172.16.64.68 icmp_seq=1 Frag needed and DF set (mtu = 1500)
From 172.16.64.68 icmp_seq=1 Frag needed and DF set (mtu = 1500)
From 172.16.64.68 icmp_seq=1 Frag needed and DF set (mtu = 1500)
From 172.16.64.68 icmp_seq=1 Frag needed and DF set (mtu = 1500)

--- 172.16.64.75 ping statistics ---
0 packets transmitted, 0 received, +5 errors

And with enabled jumbo frames:

# ping -s 8000 -M do -c 5 172.16.64.75
PING 172.16.64.75 (172.16.64.75) 8000(8028) bytes of data.
8008 bytes from 172.16.64.75: icmp_seq=1 ttl=64 time=0.461 ms
8008 bytes from 172.16.64.75: icmp_seq=2 ttl=64 time=0.360 ms
8008 bytes from 172.16.64.75: icmp_seq=3 ttl=64 time=0.402 ms
8008 bytes from 172.16.64.75: icmp_seq=4 ttl=64 time=0.410 ms
8008 bytes from 172.16.64.75: icmp_seq=5 ttl=64 time=0.347 ms

--- 172.16.64.75 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 3998ms
rtt min/avg/max/mdev = 0.347/0.396/0.461/0.040 ms

Testing 10G network with iperf

July 4, 2014February 22, 2018 sysadmin 6 Comments

I am testing the real bandwidth I can get with a 10G network connection. My server has an intel X540-AT2 network card with 2 10G interfaces.

The server is configured to use bonding in balance-alb mode, but in this test only one interface comes into play because the iperf client only gets a connection from one MAC address.

$ sudo cat /proc/net/bonding/bond0 
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: adaptive load balancing
Primary Slave: None
Currently Active Slave: eth0
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth0
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:25:90:f9:3d:54
Slave queue ID: 0

Slave Interface: eth1
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:25:90:f9:3d:55
Slave queue ID: 0

Frist run in the server iperf in server mode:

$ sudo iperf -s
 ------------------------------------------------------------
 Server listening on TCP port 5001
 TCP window size: 85.3 KByte (default)
 ------------------------------------------------------------

An from one client run iperf connecting to the server IP

The first test is using a max transmision unit of 1500 bytes (default)):

$ sudo iperf -c 172.17.16.78 -i1 -t 10 -m
------------------------------------------------------------
Client connecting to 172.17.16.78, TCP port 5001
TCP window size: 92.9 KByte (default)
------------------------------------------------------------
[  3] local 172.17.16.79 port 52458 connected with 172.17.16.78 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 1.0 sec   535 MBytes  4.49 Gbits/sec
[  3]  1.0- 2.0 sec   594 MBytes  4.98 Gbits/sec
[  3]  2.0- 3.0 sec   554 MBytes  4.64 Gbits/sec
[  3]  3.0- 4.0 sec   553 MBytes  4.64 Gbits/sec
[  3]  4.0- 5.0 sec   565 MBytes  4.74 Gbits/sec
[  3]  5.0- 6.0 sec   605 MBytes  5.07 Gbits/sec
[  3]  6.0- 7.0 sec   597 MBytes  5.01 Gbits/sec
[  3]  7.0- 8.0 sec   587 MBytes  4.92 Gbits/sec
[  3]  8.0- 9.0 sec   602 MBytes  5.05 Gbits/sec
[  3]  0.0-10.0 sec  5.67 GBytes  4.87 Gbits/sec
[  3] MSS size 1448 bytes (MTU 1500 bytes, ethernet)

With this test I can barely get around 5Gbits/sec. It seems too poor for a 10G network card.

In the next test I’ve changed the MTU to 9000 (jumbo frames) in all network devices (server, client and switch):

$ sudo iperf -c 172.17.16.78 -i1 -t 10 -m
------------------------------------------------------------
Client connecting to 172.17.16.78, TCP port 5001
TCP window size: 92.9 KByte (default)
------------------------------------------------------------
[ 3] local 172.17.16.79 port 52101 connected with 172.17.16.78 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 971 MBytes 8.14 Gbits/sec
[ 3] 1.0- 2.0 sec 1.09 GBytes 9.37 Gbits/sec
[ 3] 2.0- 3.0 sec 1.05 GBytes 9.04 Gbits/sec
[ 3] 3.0- 4.0 sec 1.06 GBytes 9.14 Gbits/sec
[ 3] 4.0- 5.0 sec 1.10 GBytes 9.43 Gbits/sec
[ 3] 5.0- 6.0 sec 1.11 GBytes 9.51 Gbits/sec
[ 3] 6.0- 7.0 sec 1009 MBytes 8.46 Gbits/sec
[ 3] 7.0- 8.0 sec 1.04 GBytes 8.94 Gbits/sec
[ 3] 8.0- 9.0 sec 1.11 GBytes 9.56 Gbits/sec
[ 3] 9.0-10.0 sec 1.07 GBytes 9.21 Gbits/sec
[ 3] 0.0-10.0 sec 10.6 GBytes 9.08 Gbits/sec
[ 3] MSS size 8948 bytes (MTU 8988 bytes, unknown interface)

And now, it looks very different, I can saturate the network link.
To change MTU, I’ve changed it in the server and in the client, in all network devices: bond interface and its slaves. Also the switch must support it. You can change the device MTU easly with this command:

$ sudo ip link set dev eth0 mtu 1500

VLAN tagging on Linux for KVM

January 10, 2012January 10, 2012 sysadmin Leave a comment

Today, I’m going to explain my config for KVM server to get network connectivity on guests machines using tagged vlans to get independent networks. As virtual platform I am using Proxmox ve. Proxmox is a great platform to administer KVM and OpenVZ machines, actually it is based on Debian Lenny, but very soon will be available the 2.0 version based on Debian Squeeze and with many great features.

I have connected my kvm server network interfaces to two different switches and the switch ports configured in trunk mode only accepting traffic for my tagged vlans. For vlan configuration I am using vlan package in debian, rather than specify them like eth0.X, I prefer to configure them using this tool.

To install vlan package simply run:

 # apt-get install vlan

Above the two network interfaces I have configured a bond interface in active-backup mode. My /etc/network/interfaces file looks like this:

iface eth0 inet manual
iface eth1 inet manual


auto bond0
iface bond0 inet manual
        slaves eth0 eth1
        bond_miimon 100
        bond_mode active-backup

auto vlan50
iface vlan50 inet manual
        vlan_raw_device bond0

auto vlan60
iface vlan60 inet manual
       vlan_raw_device bond0

auto vlan100
iface vlan100 inet manual
       vlan_raw_device bond0


auto vmbr0
iface vmbr0 inet static
        address  172.17.16.5
        netmask  255.255.240.0
        gateway  172.17.16.1
        bridge_ports vlan100
        bridge_stp off
        bridge_fd 0

auto vmbr50
iface vmbr50 inet static
        address 0.0.0.0
        netmask 255.255.255.255
        bridge_ports vlan50
        bridge_stp off
        bridge_fd 0

auto vmbr60
iface vmbr60 inet static
        address 0.0.0.0
        netmask 255.255.255.255
        bridge_ports vlan60
        bridge_stp off
        bridge_fd 0

I have three bridges configured, vmbr0 (with vlan 100), required to access proxmox web interface, and vmbr50 and vmbr60, each of them accessing to their vlans to provide access to guests. The bridge vmbr0 is the only bridge that has an IP address configured, because is the only interface I’m going to use to access to the kvm server.

Now, it is easy to provide network connectivity to the kvm guests machines, simply you have to link their network interfaces to the bridge you want depending on, to that vlan you want they get access.

For example, part of one of my kvm machine config file looks like this:


vlan60: virtio=DE:17:7C:C3:CE:B2
vlan50: virtio=B2:0A:19:3E:72:4D

This is automatically added using proxmox ve web interface.

Rename linux network interfaces

January 9, 2012January 10, 2012 sysadmin Leave a comment

Sometimes it can be useful to have network interfaces with an recognizable name, rather than eth0, eth1,… with which you can easily identify to which networks they are connected. For example, if you have a server connected to many networks and vlans you can identify the network/vlan based on interface name, eth50 for vlan50, eth100 for vlan100,… or you can simply rename interface to vlan50, vlan100 directly.

In Debian you should install ifrename package and create the file /etc/iftab with this format:

$ cat /etc/iftab 
eth50  mac F2:A8:C0:77:82:94
eth55  mac 4E:DF:22:FD:71:EA

In Redhat/CentOS you can rename /etc/sysconfig/network-scripts/ifcfg-eth0 to /etc/sysconfig/network-scripts/ifcfg-XXXX and edit file specifying this name. For example, I have my ifcfg-eth50 script like this:

DEVICE=eth50 NM_CONTROLLED=yes ONBOOT=yes HWADDR=9c:83:4f:af:a1:ec TYPE=Ethernet BOOTPROTO=none IPADDR=172.16.0.19 PREFIX=23 DEFROUTE=yes IPV4_FAILURE_FATAL=yes IPV6INIT=no NAME=eth50 UUID=5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03 NETMASK=255.255.254.0 USERCTL=no

In Ubuntu, I did not try it but I think you can edit the file /etc/udev/rules.d/70-persistent-net.rules and change the name of interface according to its mac address.