Dynamic Reconfiguration

Dynamic reconfiguration (DR) is the ability to reconfigure a system while the system is running, with little or no impact on existing operations. Not all Sun platforms support DR. Some Sun platforms might only support DR of certain types of hardware. On platforms that support DR of NIC's, IPMP can be used to transparently fail over network access, providing uninterrupted network access to the system.

For more information on how IPMP supports DR, refer to IPMP and Dynamic Reconfiguration.

Basic Requirements of IPMP

IPMP is built into the Solaris Operating System (Solaris OS) and does not require any special hardware. Any interface that is supported by the Solaris OS can be used with IPMP. However, IPMP does impose the following requirements on your network configuration and topology:

• All interfaces in an IPMP group must have unique MAC addresses.

  Note that by default, the network interfaces on SPARC based systems all share a single MAC address. Thus, you must explicitly change the default in order to use IPMP on SPARC based systems. For more information, refer to How to Plan for an IPMP Group.

• All interfaces in an IPMP group must be of the same media type. For more information, refer to IPMP Group.

• All interfaces in an IPMP group must be on the same IP link. For more information, refer to IPMP Group.

• Depending on your failure detection requirements, you might need to either use specific types of network interfaces or configure additional IP addresses on each network interface. Refer to Link-Based Failure Detection and Probe-Based Failure Detection.

IPMP Addressing

You can configure IPMP failure detection on both IPv4 networks and dual-stack, IPv4 and IPv6 networks. Interfaces that are configured with IPMP support two types of addresses: data addresses and test addresses.

Data Addresses

Data addresses are the conventional IPv4 and IPv6 addresses that are assigned to an interface of a NIC at boot time or manually, through the ifconfig command. The standard IPv4 and, if applicable, IPv6 packet traffic through an interface is considered to be data traffic.

Test Addresses

Test addresses are IPMP-specific addresses that are used by the in.mpathd daemon. For an interface to use probe-based failure and repair detection, that interface must be configured with at least one test address.

Note - You need to configure test addresses only if you want to use probe-based failure detection.

The in.mpathd daemon uses test addresses to exchange ICMP probes, also called probe traffic, with other targets on the IP link. Probe traffic helps to determine the status of the interface and its NIC, including whether an interface has failed. The probes verify that the send and receive path to the interface is working correctly.

Each interface can be configured with an IP test address. For an interface on a dual-stack network, you can configure an IPv4 test address, an IPv6 test address, or both IPv4 and IPv6 test addresses.

After an interface fails, the test addresses remain on the failed interface so that in.mpathd can continue to send probes to check for subsequent repair. You must specifically configure test addresses so that applications do not accidentally use them. For more information, refer to Preventing Applications From Using Test Addresses.

For more information on probe-based failure detection, refer to Probe-Based Failure Detection.

IPv4 Test Addresses

In general, you can use any IPv4 address on your subnet as a test address. IPv4 test addresses do not need to be routeable. Because IPv4 addresses are a limited resource for many sites, you might want to use non-routeable RFC 1918 private addresses as test addresses. Note that the in.mpathd daemon exchanges only ICMP probes with other hosts on the same subnet as the test address. If you do use RFC 1918-style test addresses, be sure to configure other systems, preferably routers, on the IP link with addresses on the appropriate RFC 1918 subnet. The in.mpathd daemon can then successfully exchange probes with target systems.

The IPMP examples use RFC 1918 addresses from the 192.168.0/24 network as IPv4 test addresses. For more information about RFC 1918 private addresses, refer to RFC 1918, Address Allocation for Private Internets.

To configure IPv4 test addresses, refer to the task How to Configure an IPMP Group With Multiple Interfaces.

IPv6 Test Addresses

The only valid IPv6 test address is the link-local address of a physical interface. You do not need a separate IPv6 address to serve as an IPMP test address. The IPv6 link-local address is based on the Media Access Control (MAC ) address of the interface. Link-local addresses are automatically configured when the interface becomes IPv6-enabled at boot time or when the interface is manually configured through ifconfig.

To identify the link-local address of an interface, run the ifconfig interface command on an IPv6-enabled node. Check the output for the address that begins with the prefix fe80, the link-local prefix. The NOFAILOVER flag in the following ifconfig output indicates that the link-local address fe80::a00:20ff:feb9:17fa/10 of the hme0 interface is used as the test address.

hme0: flags=a000841<UP,RUNNING,MULTICAST,IPv6,NOFAILOVER> mtu 1500 index 2
        	inet6 fe80::a00:20ff:feb9:17fa/10 

For more information on link-local addresses, refer to Link-Local Unicast Address.

When an IPMP group has both IPv4 and IPv6 plumbed on all the group's interfaces, you do not need to configure separate IPv4 test addresses. The in.mpathd daemon can use the IPv6 link-local addresses as test addresses.

To create an IPv6 test address, refer to the task How to Configure an IPMP Group With Multiple Interfaces.

Preventing Applications From Using Test Addresses

After you have configured a test address, you need to ensure that this address is not used by applications. Otherwise, if the interface fails, the application is no longer reachable because test addresses do not fail over during the failover operation. To ensure that IP does not choose the test address for normal applications, mark the test address as deprecated.

IPv4 does not use a deprecated address as a source address for any communication, unless an application explicitly binds to the address. The in.mpathd daemon explicitly binds to such an address in order to send and receive probe traffic.

Because IPv6 link-local addresses are usually not present in a name service, DNS and NIS applications do not use link-local addresses for communication. Consequently, you must not mark IPv6 link-local addresses as deprecated.

IPv4 test addresses should not be placed in the DNS and NIS name service tables. In IPv6, link-local addresses are not normally placed in the name service tables.

IPMP Interface Configurations

An IPMP configuration typically consists of two or more physical interfaces on the same system that are attached to the same IP link. These physical interfaces might or might not be on the same NIC. The interfaces are configured as members of the same IPMP group. If the system has additional interfaces on a second IP link, you must configure these interfaces as another IPMP group.

A single interface can be configured in its own IPMP group. The single interface IPMP group has the same behavior as an IPMP group with multiple interfaces. However, failover and failback cannot occur for an IPMP group with only one interface.

Standby Interfaces in an IPMP Group

The standby interface in an IPMP group is not used for data traffic unless some other interface in the group fails. When a failure occurs, the data addresses on the failed interface migrate to the standby interface. Then, the standby interface is treated the same as other active interfaces until the failed interface is repaired. Some failovers might not choose a standby interface. Instead, these failovers might choose an active interface with fewer data addresses that are configured as UP than the standby interface.

You should configure only test addresses on a standby interface. IPMP does not permit you to add a data address to an interface that is configured through the ifconfig command as standby. Any attempt to create this type of configuration will fail. Similarly, if you configure as standby an interface that already has data addresses, these addresses automatically fail over to another interface in the IPMP group. Due to these restrictions, you must use the ifconfig command to mark any test addresses as deprecated and -failover prior to setting the interface as standby. To configure standby interfaces, refer to How to Configure a Standby Interface for an IPMP Group.

Common IPMP Interface Configurations

As mentioned in IPMP Addressing, interfaces in an IPMP group handle regular data traffic and probe traffic, depending on the interfaces' configuration. You use IPMP options of the ifconfig command to create the configuration.

An active interface is a physical interface that transmits both data traffic and probe traffic. You configure the interface as "active" by performing either the task How to Configure an IPMP Group With Multiple Interfaces or the task How to Configure a Single Interface IPMP Group.

The following are two common types of IPMP configurations:

Checking the Status of an Interface

You can check the status of an interface by issuing the ifconfig interface command. For general information on ifconfig status reporting, refer to How to Get Information About a Specific Interface.

For example, you can use the ifconfig command to obtain the status of a standby interface. When the standby interface is not hosting any data address, the interface has the INACTIVE flag for its status. You can observe this flag in the status lines for the interface in the ifconfig output.

IPMP Failure Detection and Recovery Features

The in.mpathd daemon handles the following types of failure detection:

• Link-based failure detection, if supported by the NIC driver

• Probe-based failure detection, when test addresses are configured

• Detection of interfaces that were missing at boot time

The in.mpathd(1M) man page completely describes how the in.mpathd daemon handles the detection of interface failures.