INFO: IPMP or IP Multipathing to Improve Resiliency

High availability (HA) is a system design approach and associated service implementation that ensures a prearranged level of operational performance will be met during a contractual measurement period. (source: wikipedia).

Tweaks about high availability are not taught in school. Either you learn it on the job or seek knowledge from sites on the net. Then, you will have to try it out and see for yourself how it gets implemented. High availability ensures minimal downtime or impact from outage. Others define it as elimination of single points of failure.

HA can be very costly but if implemented right, tweaking for HA may not be as expensive. This is where a simple tweak like IPMP (short for IP Multipathing) comes in. It is not as expensive. And just covers the single point of failure on the network part.

I found this tweak from Sun Blueprints a long while ago, but it seems that site is no longer in existence, I'm putting up a similar procedure.

Solaris supports IPMP right out of the box. And implementation is not as complex as one might think. Only basic knowledge of Solaris network configuration is required. In this example, implementation of IPMP is for a failover of a public IP with two private IPs.

Let's assume your server has two network interfaces (NICs). In Solaris, the network interface is recognized as ce0 and ce1 (for copper ethernet).

[1] Ensure that each network card has a unique MAC address flag set to true.

On the OBP: setenv local-mac-address?=true
On the CLI: eeprom "local-mac-address?=true"

[2] Reserve three IP addresses on the same subnet or VLAN. Add them to /etc/hosts.

root@host# cat /etc/hosts
#
# Internet host table
#
127.0.0.1       localhost       
192.168.0.10   host-name  loghost 
192.168.0.11   host-priv1 
192.168.0.12   host-priv2

[3] Configure the individual interfaces with the corresponding IP addresses assigned (assuming ce0 and ce1, could be any NIC of your choice).

root@host# cat /etc/hostname.ce0
host-priv1 netmask + broadcast + group production deprecated -failover up \
addif host-name netmask + broadcast + up
root@host# cat /etc/hostname.ce1
host-priv2 netmask + broadcast + group production deprecated -failover up

There are many variations to this configuration. But what this particular configuration does is sets up two private IP addresses that are fixed to each interface (host-priv1 and host-priv2); it sets the interfaces under a group named production; and, it adds an IP (host-name) that is assigned to ce0 but migrates between interfaces in cases of failure.

In the above configuration, receiving of packets is done by the active host-name but transmission is shared by each, thereby "doubling" the uplink bandwidth.

root@host# ifconfig -a
...
ce0: flags=1000843 mtu 1500 index 3
inet 192.168.0.11 netmask ffffff00 broadcast 192.168.0.255
ether 00:14:4f:2e:94:43
groupname production
ce0:1: flags=21000843 mtu 1500 index 4
inet 192.168.0.10 netmask ffffff00 broadcast 192.168.0.255
ce1: flags=29040843 mtu 1500 index 4
inet 192.168.0.12 netmask ffffff00 broadcast 192.168.0.255
groupname production
ether 00:14:4f:62:51:3b

Administration best practice is to try out the failover to test resiliency to failure. To simulate failure, use the command below. Execute tail -f /var/adm/messages to view the informative and error messages.

root@host# if_mpadm -d ce0
Oct 10 16:28:23 host in.mpathd[428]: Successfully failed over from NIC ce0 to NIC ce1
root@host# if_mpadm -r ce0
Oct 10 16:37:45 host in.mpathd[428]: Successfully failed back to NIC ce0

With the configuration tried and tested, the single point of failure on the network is eliminated. To further improve resiliency, it is recommended to plug each NIC connection to different switches.

NOTE: The secondary network card can also be setup in standby mode with only one data address between the cards. However, by assigning a data address for each card, the outbound load is spread by IPMP. But using only one shared IP address, if the card with a single data address fails during boot up, the address may not be failed over, leaving both interfaces inactive.