Chapter 3. Setting Up LVS
LVS consists of two basic groups: the LVS routers and the real servers. To prevent a single point of failure, each groups should contain at least two member systems.
The LVS router group should consist of two identical or very similar systems running Red Hat Enterprise Linux. One will act as the active LVS router while the other stays in hot standby mode, so they need to have as close to the same capabilities as possible.
Before choosing and configuring the hardware for the real server group, determine which of the three LVS topologies to use.
3.1. The NAT LVS Network
The NAT topology allows for great latitude in utilizing existing hardware, but it is limited in its ability to handle large loads because all packets going into and coming out of the pool pass through the LVS router.
- Network Layout
- The topology for LVS using NAT routing is the easiest to configure from a network layout perspective because only one access point to the public network is needed. The real servers pass all requests back through the LVS router so they are on their own private network.
- Hardware
- The NAT topology is the most flexible in regards to hardware because the real servers do not need to be Linux machines to function correctly. In a NAT topology, each real server only needs one NIC since it will only be responding to the LVS router. The LVS routers, on the other hand, need two NICs each to route traffic between the two networks. Because this topology creates a network bottleneck at the LVS router, gigabit Ethernet NICs can be employed on each LVS router to increase the bandwidth the LVS routers can handle. If gigabit Ethernet is employed on the LVS routers, any switch connecting the real servers to the LVS routers must have at least two gigabit Ethernet ports to handle the load efficiently.
- Software
- Because the NAT topology requires the use of
iptablesfor some configurations, there can be a fair amount of software configuration outside of Piranha Configuration Tool. In particular, FTP services and the use of firewall marks requires extra manual configuration of the LVS routers to route requests properly.
3.1.1. Configuring Network Interfaces for LVS with NAT
To set up LVS with NAT, you must first configure the network interfaces for the public network and the private network on the LVS routers. In this example, the LVS routers' public interfaces (
eth0) will be on the 192.168.26/24 network (I know, I know, this is not a routable IP, but let us pretend there is a firewall in front of the LVS router for good measure) and the private interfaces which link to the real servers (eth1) will be on the 10.11.12/24 network.
So on the active or primary LVS router node, the public interface's network script,
/etc/sysconfig/network-scripts/ifcfg-eth0, could look something like this:
DEVICE=eth0 BOOTPROTO=static ONBOOT=yes IPADDR=192.168.26.9 NETMASK=255.255.255.0 GATEWAY=192.168.26.254
The
/etc/sysconfig/network-scripts/ifcfg-eth1 for the private NAT interface on the LVS router could look something like this:
DEVICE=eth1 BOOTPROTO=static ONBOOT=yes IPADDR=10.11.12.9 NETMASK=255.255.255.0
In this example, the VIP for the LVS router's public interface will be 192.168.26.10 and the VIP for the NAT or private interface will be 10.11.12.10. So, it is essential that the real servers route requests back to the VIP for the NAT interface.
Important
The sample Ethernet interface configuration settings in this section are for the real IP addresses of an LVS router and not the floating IP addresses. To configure the public and private floating IP addresses the administrator should use the Piranha Configuration Tool, as shown in Section 4.4, “GLOBAL SETTINGS” and Section 4.6.1, “The VIRTUAL SERVER Subsection”.
After configuring the primary LVS router node's network interfaces, configure the backup LVS router's real network interfaces — taking care that none of the IP address conflict with any other IP addresses on the network.
Important
Be sure each interface on the backup node services the same network as the interface on primary node. For instance, if eth0 connects to the public network on the primary node, it must also connect to the public network on the backup node as well.
3.1.2. Routing on the Real Servers
The most important thing to remember when configuring the real servers network interfaces in a NAT topology is to set the gateway for the NAT floating IP address of the LVS router. In this example, that address is 10.11.12.10.
Note
Once the network interfaces are up on the real servers, the machines will be unable to ping or connect in other ways to the public network. This is normal. You will, however, be able to ping the real IP for the LVS router's private interface, in this case 10.11.12.8.
So the real server's
/etc/sysconfig/network-scripts/ifcfg-eth0 file could look similar to this:
DEVICE=eth0 ONBOOT=yes BOOTPROTO=static IPADDR=10.11.12.1 NETMASK=255.255.255.0 GATEWAY=10.11.12.10
Warning
If a real server has more than one network interface configured with a
GATEWAY= line, the first one to come up will get the gateway. Therefore if both eth0 and eth1 are configured and eth1 is used for LVS, the real servers may not route requests properly.
It is best to turn off extraneous network interfaces by setting
ONBOOT=no in their network scripts within the /etc/sysconfig/network-scripts/ directory or by making sure the gateway is correctly set in the interface which comes up first.
3.1.3. Enabling NAT Routing on the LVS Routers
In a simple NAT LVS configuration where each clustered service uses only one port, like HTTP on port 80, the administrator needs only to enable packet forwarding on the LVS routers for the requests to be properly routed between the outside world and the real servers. See Section 2.5, “Turning on Packet Forwarding” for instructions on turning on packet forwarding. However, more configuration is necessary when the clustered services require more than one port to go to the same real server during a user session. For information on creating multi-port services using firewall marks, see Section 3.4, “Multi-port Services and LVS”.
Once forwarding is enabled on the LVS routers and the real servers are set up and have the clustered services running, use the Piranha Configuration Tool to configure LVS as shown in Chapter 4, Configuring the LVS Routers with Piranha Configuration Tool.
Warning
Do not configure the floating IP for
eth0:1 or eth1:1 by manually editing network scripts or using a network configuration tool. Instead, use the Piranha Configuration Tool as shown in Section 4.4, “GLOBAL SETTINGS” and Section 4.6.1, “The VIRTUAL SERVER Subsection”.
When finished, start the
pulse service as shown in Section 4.8, “Starting LVS”. Once pulse is up and running, the active LVS router will begin routing requests to the pool of real servers.
