HOW-TO: Create an Aggregated Link in Solaris

Nowadays, servers ship with multiple network interface cards (NIC). The rationale behind this, is you will use them for high availability implementations or utilize the other NIC as a private lan for backup. If the box is not designed this way, it doesn't usually get deployed in the enterprise.

Previously, we featured a procedure in deploying IPMP or IP Multipathing. Starting with Solaris 10, there is this feature that is much more robust than IPMP. It is called link aggregation; others call it trunking, but they pertain to the same technology. The equivalent in Linux is NIC bonding.

Link aggregation only works on full-duplex, point-to-point links that operate at identical speeds. Make sure that the interfaces in your aggregation conform to this foremost requirement. If you are using a switch in your aggregation topology, make sure that you have configured the ports to be used as an aggregation. Modern switches can support Link Aggregation Control Protocol or LACP. If so, have it configured in either active mode or passive mode.

[1] Assume the Primary Administrator role, or become superuser.

[2] Determine which interfaces are currently installed on your system.

# dladm show-link 
bge0         link: up        speed: 1000  Mbps       duplex: full
bge1         link: up        speed: 1000  Mbps       duplex: full

The command dladm show-dev can also be used here. However, this command will not only show the NIC but also the HBAs (host bus adapters) that are installed on the host. Seasoned veterans know the prefixes of Solaris NIC naming convention so this should not pose an issue.

[3] Determine which interfaces have been plumbed.

# ifconfig -a

[4] Create an aggregation. Syntax is:

# dladm create-aggr -d interface key
interface
  represents the device name of the interface to become part of the aggregation.
key
  the number that identifies the aggregation. The lowest key number is 1. Zeroes are not allowed as keys.
# dladm create-aggr -d bge0 -d bge1 1

The above command is not limited to two (2) NICs. As many NICs can be used as there are physical NICs on the server. Just separate each NIC with -d to join them in the aggregate creation.

[5] Configure and plumb the newly created aggregation.

# ifconfig aggr[key] plumb IP-address up
# ifconfig aggr1 plumb 192.168.0.11 netmask + broadcast + up

[6] Check the status of the aggregation you just created.

# dladm show-aggr
key: 1 (0x0001) policy: L4      address: 0:3:ba:77:84:e (auto)
       device   address           speed         duplex  link    state
       bge0     0:3:ba:77:84:e    1000  Mbps    full    up      attached
       bge1     0:3:ba:77:84:e    1000  Mbps    full    up      attached
# ifconfig aggr1
aggr1: flags=2008843 mtu 1500 index 5
inet 192.168.0.11 netmask ffffff00 broadcast 192.168.0.255
ether 0:3:ba:77:84:e

The above configuration is not yet persistent across reboots. To make it persistent, create a file in this manner: /etc/hostname.aggr[key]. In this example, that translates to /etc/hostname.aggr1.

Add the IP address (192.168.0.11) to the file /etc/hostname.aggr1. Another option is to add the hostname of the server as the content of /etc/hostname.aggr1, but the IP address has to be added to the hosts file. The approach of adding the IP address is simpler and very straight forward.

Do a reboot to test if the aggregation persists across reboots.

Link Aggregation is very picky on both the hardware, driver, speed, duplex and on the switch port. So in most cases where aggregation doesn't work, IPMP is being deployed. To know more about the advantages and disadvantages of each, check out this comparison betweek Link Aggregation vs IPMP.