Move data up to 6X faster than you used to with LACP 802.3ad.

If you have a Cisco or other programmable switch capable of LACP 802.3ad then you're most of the way to having faster data transfer than you may have thought possible.  The simplicity of this process still throws me for how powerful it is.  LACP is simply bonding two or more network interfaces at the switch and at the computer so that they act as a single virtual network interface.  This increases the speed and availability by two, three, four or up to six times.  It is not recommended to use more than six physical interfaces in one Link Aggregate.

FreeBSD refers to the link aggregate as laggN where N is the ID of the virtual interface.  For example your first lagg would be lagg0 followed by lagg1 and so on.  The server I am working on is a fresh install of FreeBSD 8.2 amd64 with four physical network interfaces em0 em1 em2 and em3.  It is going to be the permanent replacement of a virtual server that currently has three aggregated network interfaces.  I am using a Cisco 3750 in this specific case, but I have used this same process on a variety of other Cisco switches.  We're going to do the switch first, then we'll set up the server.  This example assumes you are using Gigabit ports 0/10 through 0/13.

After you get into enable mode on the switch type:

# config t
(config)# int range gi0/10-13
(config-if-range)# channel-group 1 mode active
(config-if-range)# channel-protocol lacp
(config-if-range)# exit
(config)# exit

Then type show run to make sure your new interfaces have the new channel- switches in them.

interface GigabitEthernet0/10
 description admin
 switchport access vlan 20
 switchport mode access
 channel-protocol lacp
 channel-group 1 mode active
 spanning-tree portfast

The group number is pretty arbitrary; you can make the number whatever you want for documentation.  Just make sure it's not already used somewhere else on that switch.  Sometimes I have issues getting the channel-protocol line to stick when I choose an int range instead of calling the interface separately.  If this happens to you, simply go back into config t and call each interface separately, give it the channel-protocol line and exit back to config t for the next one, like this:

# config t
(config)# int gi0/10
(config-if)#channel-protocol lacp
(config-if)#exit
(config)#int gi0/11
(config-if)#channel-protocol lacp
(config-if)#exit
(config)#int gi0/12

..And so on.

The switch is done.  We're going to add a few lines to our /etc/rc.conf file and that'll do it for the server.  Pull up /etc/rc.conf with an editor and add:

ifconfig_em0="up"
ifconfig_em1="up"
ifconfig_em2="up"
ifconfig_em3="up"
cloned_interfaces="lagg0"
ifconfig_lagg0="laggproto lacp laggport em0 laggport em1 laggport em2 laggport em3"
ipv4_addrs_lagg0="10.0.10.149/24"
defaultrouter="10.0.10.1"

The ifconfig_lagg0="laggproto lacp laggport em0 laggport em1 laggport em2 laggport em3" entry is all one line.

When you reboot or activate your new network stack your server will now listen on all four network interfaces making your data transfers four times faster than they used to be.  When you're satisfied that everything is good, go back to your Cisco and copy run start in enable mode to ensure you keep your link aggregation after a switch reboot.


Make sure your disk I/O is fast enough to handle the new speeds by enabling ZFS on your server.  This example makes the network so fast that there is no way you could possibly feed information to it from disk fast enough to use it without some major arrays.