20.3. Adding a Network Device
Network device driver modules are loaded automatically by udev.
You can add a network interface on IBM Z dynamically or persistently.
- Dynamically
- Load the device driver
- Remove the network devices from the list of ignored devices.
- Create the group device.
- Configure the device.
- Set the device online.
- Persistently
- Create a configuration script.
- Activate the interface.
The following sections provide basic information for each task of each IBM Z network device driver. Section 20.3.1, “Adding a qeth Device” describes how to add a qeth device to an existing instance of Red Hat Enterprise Linux. Section 20.3.2, “Adding an LCS Device” describes how to add an lcs device to an existing instance of Red Hat Enterprise Linux.
20.3.1. Adding a qeth Device Copy linkLink copied to clipboard!
Copy linkLink copied to clipboard!
The
qeth
network device driver supports IBM Z OSA-Express features in QDIO mode, HiperSockets, z/VM guest LAN, and z/VM VSWITCH.
The
qeth
device driver assigns the same interface name for Ethernet and Hipersockets devices: enccw
bus_ID. The bus ID is composed of the channel subsystem ID, subchannel set ID, and device number, for example enccw0.0.0a00
.
20.3.1.1. Dynamically Adding a qeth Device Copy linkLink copied to clipboard!
Copy linkLink copied to clipboard!
To add a
qeth
device dynamically, follow these steps:
- Determine whether the
qeth
device driver modules are loaded. The following example shows loadedqeth
modules:Copy to Clipboard Copied! Toggle word wrap Toggle overflow If the output of thelsmod
command shows that theqeth
modules are not loaded, run themodprobe
command to load them:modprobe qeth
# modprobe qeth
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Use the
cio_ignore
utility to remove the network channels from the list of ignored devices and make them visible to Linux:cio_ignore -r read_device_bus_id,write_device_bus_id,data_device_bus_id
# cio_ignore -r read_device_bus_id,write_device_bus_id,data_device_bus_idcio_ignore -r read_device_bus_id,write_device_bus_id,data_device_bus_idcio_ignore -r read_device_bus_id,write_device_bus_id,data_device_bus_idcio_ignore -r read_device_bus_id,write_device_bus_id,data_device_bus_idcio_ignore -r read_device_bus_id,write_device_bus_id,data_device_bus_idcio_ignore -r read_device_bus_id,write_device_bus_id,data_device_bus_id
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace read_device_bus_id,write_device_bus_id,data_device_bus_id with the three device bus IDs representing a network device. For example, if the read_device_bus_id is0.0.f500
, the write_device_bus_id is0.0.f501
, and the data_device_bus_id is0.0.f502
:cio_ignore -r 0.0.f500,0.0.f501,0.0.f502
# cio_ignore -r 0.0.f500,0.0.f501,0.0.f502
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Use the znetconf utility to sense and list candidate configurations for network devices:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Select the configuration you want to work with and use znetconf to apply the configuration and to bring the configured group device online as network device.
znetconf -a f500
# znetconf -a f500 Scanning for network devices... Successfully configured device 0.0.f500 (enccw0.0.f500)
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Optionally, you can also pass arguments that are configured on the group device before it is set online:
znetconf -a f500 -o portname=myname
# znetconf -a f500 -o portname=myname Scanning for network devices... Successfully configured device 0.0.f500 (enccw0.0.f500)
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Now you can continue to configure theenccw0.0.f500
network interface.
Alternatively, you can use
sysfs
attributes to set the device online as follows:
- Create a
qeth
group device:echo read_device_bus_id,write_device_bus_id,data_device_bus_id > /sys/bus/ccwgroup/drivers/qeth/group
# echo read_device_bus_id,write_device_bus_id,data_device_bus_id > /sys/bus/ccwgroup/drivers/qeth/groupecho read_device_bus_id,write_device_bus_id,data_device_bus_id > /sys/bus/ccwgroup/drivers/qeth/groupecho read_device_bus_id,write_device_bus_id,data_device_bus_id > /sys/bus/ccwgroup/drivers/qeth/groupecho read_device_bus_id,write_device_bus_id,data_device_bus_id > /sys/bus/ccwgroup/drivers/qeth/groupecho read_device_bus_id,write_device_bus_id,data_device_bus_id > /sys/bus/ccwgroup/drivers/qeth/groupecho read_device_bus_id,write_device_bus_id,data_device_bus_id > /sys/bus/ccwgroup/drivers/qeth/groupecho read_device_bus_id,write_device_bus_id,data_device_bus_id > /sys/bus/ccwgroup/drivers/qeth/group
Copy to Clipboard Copied! Toggle word wrap Toggle overflow For example:echo 0.0.f500,0.0.f501,0.0.f502 > /sys/bus/ccwgroup/drivers/qeth/group
# echo 0.0.f500,0.0.f501,0.0.f502 > /sys/bus/ccwgroup/drivers/qeth/group
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Next, verify that the
qeth
group device was created properly by looking for the read channel:ls /sys/bus/ccwgroup/drivers/qeth/0.0.f500
# ls /sys/bus/ccwgroup/drivers/qeth/0.0.f500
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can optionally set additional parameters and features, depending on the way you are setting up your system and the features you require, such as:portno
layer2
portname
- Bring the device online by writing
1
to the onlinesysfs
attribute:echo 1 > /sys/bus/ccwgroup/drivers/qeth/0.0.f500/online
# echo 1 > /sys/bus/ccwgroup/drivers/qeth/0.0.f500/online
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Then verify the state of the device:
cat /sys/bus/ccwgroup/drivers/qeth/0.0.f500/online 1
# cat /sys/bus/ccwgroup/drivers/qeth/0.0.f500/online 1
Copy to Clipboard Copied! Toggle word wrap Toggle overflow A return value of1
indicates that the device is online, while a return value0
indicates that the device is offline. - Find the interface name that was assigned to the device:
cat /sys/bus/ccwgroup/drivers/qeth/0.0.f500/if_name enccw0.0.f500
# cat /sys/bus/ccwgroup/drivers/qeth/0.0.f500/if_name enccw0.0.f500
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Now you can continue to configure theenccw0.0.f500
network interface.The following command from the s390utils package shows the most important settings of yourqeth
device:Copy to Clipboard Copied! Toggle word wrap Toggle overflow
20.3.1.2. Dynamically Removing a qeth Device Copy linkLink copied to clipboard!
Copy linkLink copied to clipboard!
To remove a
qeth
device, use the znetconf utility. For example:
- Use the
znetconf
utility to show you all configured network devices:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Select the network device to be removed and run
znetconf
to set the device offline and ungroup theccw
> group device.znetconf -r f500
# znetconf -r f500 Remove network device 0.0.f500 (0.0.f500,0.0.f501,0.0.f502)? Warning: this may affect network connectivity! Do you want to continue (y/n)?y Successfully removed device 0.0.f500 (enccw0.0.f500)
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Verify the success of the removal:
znetconf -c
# znetconf -c Device IDs Type Card Type CHPID Drv. Name State -------------------------------------------------------------------------------------- 0.0.8036,0.0.8037,0.0.8038 1731/05 HiperSockets FB qeth hsi1 online 0.0.f5f0,0.0.f5f1,0.0.f5f2 1731/01 OSD_1000 76 qeth enccw0.0.09a0 online
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
20.3.1.3. Persistently Adding a qeth Device Copy linkLink copied to clipboard!
Copy linkLink copied to clipboard!
To make your new
qeth
device persistent, you need to create the configuration file for your new interface. The network interface configuration files are placed in the /etc/sysconfig/network-scripts/
directory.
The network configuration files use the naming convention
ifcfg-device
, where device is the value found in the if_name
file in the qeth
group device that was created earlier, for example enccw0.0.09a0
. The cio_ignore
commands are handled transparently for persistent device configurations and you do not need to free devices from the ignore list manually.
If a configuration file for another device of the same type already exists, the simplest way is to copy it to the new name and then edit it:
cd /etc/sysconfig/network-scripts cp ifcfg-enccw0.0.09a0 ifcfg-enccw0.0.0600
# cd /etc/sysconfig/network-scripts
# cp ifcfg-enccw0.0.09a0 ifcfg-enccw0.0.0600
To learn IDs of your network devices, use the lsqeth utility:
lsqeth -p
# lsqeth -p
devices CHPID interface cardtype port chksum prio-q'ing rtr4 rtr6 lay'2 cnt
-------------------------- ----- ---------------- -------------- ---- ------ ---------- ---- ---- ----- -----
0.0.09a0/0.0.09a1/0.0.09a2 x00 enccw0.0.09a0 Virt.NIC QDIO 0 sw always_q_2 n/a n/a 1 64
0.0.0600/0.0.0601/0.0.0602 x00 enccw0.0.0600 Virt.NIC QDIO 0 sw always_q_2 n/a n/a 1 64
If you do not have a similar device defined, you must create a new file. Use this example of
/etc/sysconfig/network-scripts/ifcfg-0.0.09a0
as a template:
Edit the new
ifcfg-0.0.0600
file as follows:
- Modify the
DEVICE
statement to reflect the contents of theif_name
file from yourccw
group. - Modify the
IPADDR
statement to reflect the IP address of your new interface. - Modify the
NETMASK
statement as needed. - If the new interface is to be activated at boot time, then make sure
ONBOOT
is set toyes
. - Make sure the
SUBCHANNELS
statement matches the hardware addresses for your qeth device. Please, note that the IDs must be specified in lowercase. - Modify the
PORTNAME
statement or leave it out if it is not necessary in your environment. - You can add any valid
sysfs
attribute and its value to theOPTIONS
parameter. The Red Hat Enterprise Linux installation program currently uses this to configure the layer mode (layer2
) and the relative port number (portno
) ofqeth
devices.Theqeth
device driver default for OSA devices is now layer 2 mode. To continue using oldifcfg
definitions that rely on the previous default of layer 3 mode, addlayer2=0
to theOPTIONS
parameter.
/etc/sysconfig/network-scripts/ifcfg-0.0.0600
Changes to an
ifcfg
file only become effective after rebooting the system or after the dynamic addition of new network device channels by changing the system's I/O configuration (for example, attaching under z/VM). Alternatively, you can trigger the activation of a ifcfg
file for network channels which were previously not active yet, by executing the following commands:
- Use the
cio_ignore
utility to remove the network channels from the list of ignored devices and make them visible to Linux:cio_ignore -r read_device_bus_id,write_device_bus_id,data_device_bus_id
# cio_ignore -r read_device_bus_id,write_device_bus_id,data_device_bus_idcio_ignore -r read_device_bus_id,write_device_bus_id,data_device_bus_idcio_ignore -r read_device_bus_id,write_device_bus_id,data_device_bus_idcio_ignore -r read_device_bus_id,write_device_bus_id,data_device_bus_idcio_ignore -r read_device_bus_id,write_device_bus_id,data_device_bus_idcio_ignore -r read_device_bus_id,write_device_bus_id,data_device_bus_id
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace read_device_bus_id,write_device_bus_id,data_device_bus_id with the three device bus IDs representing a network device. For example, if the read_device_bus_id is0.0.0600
, the write_device_bus_id is0.0.0601
, and the data_device_bus_id is0.0.0602
:cio_ignore -r 0.0.0600,0.0.0601,0.0.0602
# cio_ignore -r 0.0.0600,0.0.0601,0.0.0602
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - To trigger the uevent that activates the change, issue:
echo add > /sys/bus/ccw/devices/read-channel/uevent
# echo add > /sys/bus/ccw/devices/read-channel/ueventecho add > /sys/bus/ccw/devices/read-channel/ueventecho add > /sys/bus/ccw/devices/read-channel/uevent
Copy to Clipboard Copied! Toggle word wrap Toggle overflow For example:echo add > /sys/bus/ccw/devices/0.0.0600/uevent
# echo add > /sys/bus/ccw/devices/0.0.0600/uevent
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Check the status of the network device:
lsqeth
# lsqeth
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Now start the new interface:
ifup enccw0.0.0600
# ifup enccw0.0.0600
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Check the status of the interface:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Check the routing for the new interface:
ip route
# ip route default via 10.85.1.245 dev enccw0.0.0600 proto static metric 1024 12.34.4.95/24 dev enp0s25 proto kernel scope link src 12.34.4.201 12.38.4.128 via 12.38.19.254 dev enp0s25 proto dhcp metric 1 192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Verify your changes by using the
ping
utility to ping the gateway or another host on the subnet of the new device:ping -c 1 192.168.70.8
# ping -c 1 192.168.70.8 PING 192.168.70.8 (192.168.70.8) 56(84) bytes of data. 64 bytes from 192.168.70.8: icmp_seq=0 ttl=63 time=8.07 ms
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - If the default route information has changed, you must also update
/etc/sysconfig/network
accordingly.
20.3.2. Adding an LCS Device Copy linkLink copied to clipboard!
Copy linkLink copied to clipboard!
The LAN channel station (LCS) device driver supports 1000Base-T Ethernet on the OSA-Express2 and OSA-Express 3 features.
The
LCS
device driver assigns the following interface name for OSA-Express Fast Ethernet and Gigabit Ethernet devices: enccw
bus_ID. The bus ID is composed of the channel subsystem ID, subchannel set ID, and device number, for example enccw0.0.0a00
.
20.3.2.1. Dynamically Adding an LCS Device Copy linkLink copied to clipboard!
Copy linkLink copied to clipboard!
- Load the device driver:
modprobe lcs
# modprobe lcs
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Use the
cio_ignore
utility to remove the network channels from the list of ignored devices and make them visible to Linux:cio_ignore -r read_device_bus_id,write_device_bus_id
# cio_ignore -r read_device_bus_id,write_device_bus_idcio_ignore -r read_device_bus_id,write_device_bus_idcio_ignore -r read_device_bus_id,write_device_bus_idcio_ignore -r read_device_bus_id,write_device_bus_id
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace read_device_bus_id and write_device_bus_id with the two device bus IDs representing a network device. For example:cio_ignore -r 0.0.09a0,0.0.09a1
# cio_ignore -r 0.0.09a0,0.0.09a1
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Create the group device:
echo read_device_bus_id,write_device_bus_id > /sys/bus/ccwgroup/drivers/lcs/group
# echo read_device_bus_id,write_device_bus_id > /sys/bus/ccwgroup/drivers/lcs/groupecho read_device_bus_id,write_device_bus_id > /sys/bus/ccwgroup/drivers/lcs/groupecho read_device_bus_id,write_device_bus_id > /sys/bus/ccwgroup/drivers/lcs/groupecho read_device_bus_id,write_device_bus_id > /sys/bus/ccwgroup/drivers/lcs/groupecho read_device_bus_id,write_device_bus_id > /sys/bus/ccwgroup/drivers/lcs/group
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Configure the device. OSA cards can provide up to 16 ports for a single CHPID. By default, the LCS group device uses port
0
. To use a different port, issue a command similar to the following:echo portno > /sys/bus/ccwgroup/drivers/lcs/device_bus_id/portno
# echo portno > /sys/bus/ccwgroup/drivers/lcs/device_bus_id/portnoecho portno > /sys/bus/ccwgroup/drivers/lcs/device_bus_id/portnoecho portno > /sys/bus/ccwgroup/drivers/lcs/device_bus_id/portno
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace portno with the port number you want to use. - Set the device online:
echo 1 > /sys/bus/ccwgroup/drivers/lcs/read_device_bus_id/online
# echo 1 > /sys/bus/ccwgroup/drivers/lcs/read_device_bus_id/online
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - To find out what network device name has been assigned, enter the command:
ls -l /sys/bus/ccwgroup/drivers/lcs/read_device_bus_ID/net/
# ls -l /sys/bus/ccwgroup/drivers/lcs/read_device_bus_ID/net/ls -l /sys/bus/ccwgroup/drivers/lcs/read_device_bus_ID/net/ls -l /sys/bus/ccwgroup/drivers/lcs/read_device_bus_ID/net/ drwxr-xr-x 4 root root 0 2010-04-22 16:54 enccw0.0.0600
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
20.3.2.2. Persistently Adding an LCS Device Copy linkLink copied to clipboard!
Copy linkLink copied to clipboard!
The
cio_ignore
commands are handled transparently for persistent device configurations and you do not need to free devices from the ignore list manually.
To add an LCS device persistently, follow these steps:
- Create a configuration script as file in
/etc/sysconfig/network-scripts/
with a name likeifcfg-device
, where device is the value found in theif_name
file in theqeth
group device that was created earlier, for exampleenccw0.0.09a0
. The file should look similar to the following:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Modify the value of
PORTNAME
to reflect the LCS port number (portno
) you would like to use. You can add any valid lcs sysfs attribute and its value to the optionalOPTIONS
parameter. See Section 20.3.1.3, “Persistently Adding a qeth Device” for the syntax. - Set the
DEVICE
parameter as follows:DEVICE=enccwbus_ID
DEVICE=enccwbus_ID
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Issue an
ifup
command to activate the device:ifup enccwbus_ID
# ifup enccwbus_IDifup enccwbus_ID
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Changes to an
ifcfg
file only become effective after rebooting the system. You can trigger the activation of a ifcfg
file for network channels by executing the following commands:
- Use the
cio_ignore
utility to remove the LCS device adapter from the list of ignored devices and make it visible to Linux:cio_ignore -r read_device_bus_id,write_device_bus_id
# cio_ignore -r read_device_bus_id,write_device_bus_idcio_ignore -r read_device_bus_id,write_device_bus_idcio_ignore -r read_device_bus_id,write_device_bus_idcio_ignore -r read_device_bus_id,write_device_bus_id
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace read_device_bus_id and write_device_bus_id with the device bus IDs of the LCS device. For example:cio_ignore -r 0.0.09a0,0.0.09a1
# cio_ignore -r 0.0.09a0,0.0.09a1
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - To trigger the uevent that activates the change, issue:
echo add > /sys/bus/ccw/devices/read-channel/uevent
# echo add > /sys/bus/ccw/devices/read-channel/ueventecho add > /sys/bus/ccw/devices/read-channel/ueventecho add > /sys/bus/ccw/devices/read-channel/uevent
Copy to Clipboard Copied! Toggle word wrap Toggle overflow For example:echo add > /sys/bus/ccw/devices/0.0.09a0/uevent
# echo add > /sys/bus/ccw/devices/0.0.09a0/uevent
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
20.3.3. Configuring a IBM Z Network Device for Network Root File System Copy linkLink copied to clipboard!
Copy linkLink copied to clipboard!
To add a network device that is required to access the root file system, you only have to change the boot options. The boot options can be in a parameter file (see Chapter 21, Parameter and Configuration Files on IBM Z) or part of a
zipl.conf
on a DASD or FCP-attached SCSI LUN prepared with the zipl boot loader. There is no need to recreate the initramfs.
Dracut, the mkinitrd successor that provides the functionality in the initramfs that in turn replaces initrd, provides a boot parameter to activate network devices on IBM Z early in the boot process:
rd.znet=
.
As input, this parameter takes a comma-separated list of the
NETTYPE
(qeth, lcs, ctc), two (lcs, ctc) or three (qeth) device bus IDs, and optional additional parameters consisting of key-value pairs corresponding to network device sysfs attributes. This parameter configures and activates the IBM Z network hardware. The configuration of IP addresses and other network specifics works the same as for other platforms. See the dracut documentation for more details.
The cio_ignore commands for the network channels are handled transparently on boot.
Example boot options for a root file system accessed over the network through NFS:
root=10.16.105.196:/nfs/nfs_root cio_ignore=all,!condev rd.znet=qeth,0.0.0a00,0.0.0a01,0.0.0a02,layer2=1,portno=0,portname=OSAPORT ip=10.16.105.197:10.16.105.196:10.16.111.254:255.255.248.0:nfs‑server.subdomain.domain:enccw0.0.09a0:none rd_NO_LUKS rd_NO_LVM rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYTABLE=us
root=10.16.105.196:/nfs/nfs_root cio_ignore=all,!condev rd.znet=qeth,0.0.0a00,0.0.0a01,0.0.0a02,layer2=1,portno=0,portname=OSAPORT ip=10.16.105.197:10.16.105.196:10.16.111.254:255.255.248.0:nfs‑server.subdomain.domain:enccw0.0.09a0:none rd_NO_LUKS rd_NO_LVM rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYTABLE=us