Linux Installation on Intel® iSCSI Boot

This document explains how to install and boot Linux on an iSCSI target LU using the opensource linux-iSCSI initiator.

The example shows how to perform this process for SUSE* Linux Enterprise Server 9-SP3. If you have SLES 10 installed,
read the iSCSI Boot instructions in SLES 10 RELEASE NOTES.

The Red Hat* Enterprise Linux 4 process can be found here.

Prerequisites:

  1. A host system to install Linux on.
  2. A driver disk compatible with the host system and the diskless target (either USB key or floppy disk).
  3. The SUSE Linux Enterprise Server 9-SP3 install CDs.
  4. linux-iscsi files (this requirement will be satisfied by Section I)
    1. iscsi.ko
    2. iscsid
    3. initiatorname.iscsi
    4. iscsi.conf
NOTE: Some newer network device drivers are not included in either SLES9 or RHEL4. To enable these devices, the user will need to get the correct driver and load the driver before discovery is attempted when following the preinstall instructions.

Section I : Creating the Linux-iSCSI driver

This is the recommended method for creating the driver disk.

  1. On the system running SUSE Linux Enterprise Server 9-SP3, insert and mount the media to be used as the driver disk.
  2. YaST -> Add/Remove Software.
  3. Search "kernel"
  4. Select "kernel-default" and install the package.
  5. Search "iscsi"
  6. Select "linux-iscsi" and install the package.
  7. Copy the following files to your mounted media
    1. /lib/modules/2.6.5-7.244-default/extra/iscsi.ko
    2. /sbin/iscsid
    3. /etc/initiatorname.iscsi
    4. /etc/iscsi.conf
NOTES: Configure the initiatorname.iscsi and iscsi.conf files according to your target configuration. To do so you should read the README provided by the linux-iscsi source code. You must at least configure the DiscoveryAddress in the iscsi.conf.

If your target supports advanced features such as CHAP, you will need to make sure that the iscsi.conf and initiatorname.iscsi match your target configuration at this point. These files will be used throughout the install and boot process.

Alternatively, you may create the driver disk using a different host machine other than SUSE Linux Enterprise Server 9-SP3. You will need to compile the iscsi.ko and iscsid using the 2.6.5-7.244-default kernel. You can learn to do this by reading the README provided with the linux-iscsi source code.

Section II : Installing to the iSCSI target system

NOTE: You need an Intel® iSCSI Boot capable adapter to connect to your target and boot the OS from the iSCSI Target Disk.

Requirements:

  1. Diskless host compatible with the driver disk.
  2. The driver disk created in Section I.
  3. The full set of CDs for both SUSE Linux Enterprise Server 9 and SUSE Linux Enterprise Server 9-SP3.

Procedure:

  1. Boot from SUSE Linux Enterprise Server 9-SP3 installation CD1.
    1. Select "installation", but before pressing "enter", please enter the following boot option.

      DiscoveryAddress=<IP of target> InitiatorName=<initiator iqn> netsetup=1

  2. When you are prompted for CD1 insert SUSE Linux Enterprise Server 9 CD1.
  3. When you are prompted to accept the license agreement, press Ctrl-Alt-F2 to switch to the console
    1. mount the driver disk
    2. copy the following files to /etc
      1. initiatorname.iscsi
      2. iscsi.conf
    3. copy the following files to /tmp
      1. iscsi.ko
      2. iscsid
    4. unmount your driver disk and remove the driver disk
    5. cd /tmp
    6. insmod iscsi.ko
    7. execute iscsid
  4. Verify that you are connected to the target

Ctrl-Alt-F4 will display something such as:

"iSCSI target bus 0 target 0 = iqn.XYZ"

"Attached scsi disk ..."

  1. Press Alt-F7 or Ctrl+Alt+F7 to go back and continue installation. Accept the license agreement and continue until you reach the "Installation Settings" Dialog.
    NOTE: When partitioning the iSCSI LUs, ensure that your boot partition is not distributed across physical partitions on different iSCSI LUs. The Intel iSCSI Option ROM will only expose one iSCSI LU and if /boot is distributed across multiple LUs it will not be visible during the boot process.
  2. Select the Software packages to install
    1. Click "Software".
    2. Choose the software you wish to install.
    3. Select "Detailed selection". Change the value of the "Filter" drop-down box to "search".
    4. Search for "iSCSI".
    5. Make sure the "linux-iscsi" checkbox is selected.
    6. Make sure the "Autocheck" checkbox is selected.
    7. Click "accept" and "continue" if prompted.
  3. Change some of the Booting options
    1. Click "Booting"
    2. Select "Disk Order"
    3. Ensure that the disk that you are installing the OS on is the top disk in the list.
  4. When the installation completes, the user is warned that a reboot will occur, accompanied by a 10 second countdown. Press "Stop" to stop the system reboot.
  5. Press Ctrl-Alt-F2 to go to console
    1. Run "chroot /mnt"
    2. Edit line 10 of the file /etc/sysconfig/kernel. Append "e1000" & "iscsi" driver to INITRD_MODULES variable.

      Example: INITRD_MODULES="... e1000 iscsi".

    3. If your target requires CHAP authentication add the follow to /sbin/mkinitrd at line 1901

      echo Username=<CHAP username> >> /etc/iscsi.conf

      echo Password=<CHAP password> >> /etc/iscsi.conf

    4. mkinitrd -D ethX
    5. Press Ctrl-D
  6. Press Alt-F7 (Alt-F1 if using text mode installation) to return to installation screen.
    1. Select "OK" to reboot.
    2. Remove the install CD.
  7. After the reboot, YaST will step you through various configurations.
NOTE: After this first reboot, YaST may ask you to configure your network devices. Skip this step as it will reset the network interface that the iSCSI traffic is on.
  1. When you have booted to your iSCSI LU, edit /etc/rc.d/network. Add the following line to the script to prevent it from trying to reset the iscsi-ethernet interface. Please add carefully

    test "$a" = ethX && continue;

    immediately after line number 169 where ethX is the ethernet interface used for iSCSI communication.

    NOTE: After the system has booted, iscsid will be running. The user may want to add something like "kill `pidof iscsid`" to an init script. iscsid is only needed for discovery and can be killed in this way without problems.

Red Hat Enterprise Linux 4 Installation:

This process has two phases.

  1. Create a Driver Update Disk. This step is necessary to create a diskette with the initiator modules and its dependencies.
  2. Install to the iSCSI target LU. This stage basically consists of two parts; pre-install and post-install.

Prerequisites

  1. A computer with Red Hat Enterprise Linux 4 U3 installed on it as described in the following section. This system will be referred to as the build machine. This machine's architecture must match that of the install media as well as the architcture of the machine used to connect to the iSCSI Target LU.
    1. The system will be used to create a Driver Update Disk (DUD) and will need to have a 3.5' floppy drive or a USB 3.5' floppy drive.
  2. iSCSI initiator source. Available on Red Hat Enterprise Linux 4 U3 Source media.
  3. Source code for any dependency modules. iscsi initiator may depend on any other kernel modules.
  4. The iscsiboot.tar from the Intel provided media.
  5. A blank 3.5' disk.
NOTE: The system used to connect to the iSCSI target during the installation process must not have any local disks installed. It should also not have any USB storage devices attached when the install starts. The intention is to get anaconda (the Red Hat installer) to see no disks in the system and stop. This will allow us to install the initiator and discover the iSCSI LUs. If there is any media installed anaconda will continue and attempt to install to the locally connected device.

Phase I - Creating Driver Update Diskette

  1. Install Red Hat Enterprise Linux 4 U3 on your build machine. To avoid dependency problems "Customize software packages to be installed" and select "Everything" at the bottom of the "Package Group Selection".
  2. Install the dkms-2.0.13-1.noarch.rpm (http://linux.dell.com/dkms/) on the build machine, if it did not come with your distribution.
  3. Install the following packages.
    1. Use the following table to install required files. Execute the "COMMAND" on the specified file at the "LOCATION" and confirm that the "OUTPUT" was created correctly.

       

      COMMAND                     LOCATION OUTPUT
      rpm -ivh iscsi-initiator-utils-4.0.3.0-3.src.rpm Red Hat Enterprise Linux 4 U3 Source CD3 /usr/src/redhat/SPECS/iscsi-initiator-utils.spec
      rpm -ivh kernel-2.6.9-34.EL.src.rpm Red Hat Enterprise Linux 4 U3 Source CD4 /usr/src/redhat/SPECS/kernel-2.6.spec
      rpm -ivh kernel-devel-2.6.9-34.EL.i686.rpm Red Hat Enterprise Linux 4 U3 CD3 /usr/src/kernels/2.6.9-34.EL-i686/
      rpm -ivh kernel-smp-devel-2.6.9-34.EL.i686.rpm Red Hat Enterprise Linux 4 U3 CD3 /usr/src/kernels/2.6.9-34.EL-smp-i686/
      rpm -ivh kernel-hugemem-devel-2.6.9-34.EL.i686.rpm Red Hat Enterprise Linux 4 U3 CD3 /usr/src/kernels/2.6.9-34.EL-hugemem-i686
      rpmbuild -bp --target `uname -m` kernel-2.6.spec /usr/src/redhat/SPECS /usr/src/redhat/BUILD/kernel-2.6.9/
      rpmbuild -bp --target `uname -m` iscsi-initiator-utils.spec /usr/src/redhat/SPECS /usr/src/redhat/BUILD/iscsi-initiator-utils-4.0.3.0
  1. Copy and untar iscsiboot.tar.
  1. cp iscsiboot.tar.gz /home
  2. cd /home
  3. tar -xvf iscsiboot.tar.gz
  1. Insert the formatted blank diskette into the build machine.
  2. Run the make_disk_image script (run without any flags for usage).
    1. cd /home/iscsiboot/
    2. ./make_disk_image -k /usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9 -i

/usr/src/redhat/BUILD/iscsi-initiator-utils-4.0.3.0/linux-iscsi-4.0.3.0

-v 2.6.9-34.EL

This process uses dd to write the Driver Update Disk. You may specify the device node with the -t option or the script will prompt you. Keep in mind that you must use the device node and not a mounted partition.

Phase II - Installation to iSCSI disk

NOTE: This assumes you have flashed and configured your iSCSI firmware to connect to your target and iscsi firmware was able to connect to the iscsi target.
  1. Begin installation. Remember, no local storage devices can be connected at this point, including USB floppy devices.
  2. Install iSCSI Driver
    1. If the system that you're using to install Red Hat Enterprise Linux 4 U3 is diskless, you will be asked if you'd like to configure the disks.
    2. On the "Devices" dialog select "Add Device".
    3. If your driver disk uses the USB interface you will need to choose the "USB Mass Storage driver for Linux (usb-storage)".
    4. Insert your driver disk.
    5. Return to the "Select Device Driver to Load", press F2 and follow the dialogs to install the Cisco Driver.
    6. The "Select Device Driver to Load" dialog should reappear. Select "Cisco Systems iSCSI initiator (iscsi_sfnet)" and press "OK".
    7. DO NOT PRESS "DONE".
  3. Discover the target disks
    1. Press Ctrl+Alt+f2 to switch to command line mode.
    2. Make a mount point directory and mount the driver disk. (The mount point can be anything but /mnt)

Floppy

mkdir /mnt/floppy

mount /dev/fd0 /mnt/floppy.

USB Floppy Drive

dmesg

Examine the output of dmesg and determine what /dev node is reserved for your USB drive (i.e. /dev/sdX)

mkdir /mnt/usb

mknod /dev/sdX b 8 0

mount /dev/sdX /mnt/usb

  1. Copy files to a temporary location

mkdir /tmp/iscsi

cp -fr /mnt/usb/* /tmp/iscsi/

umount /mnt/usb

  1. Run the "preinstall/iscsipreinstall" script from the mount point.

/tmp/iscsi/preinstall/iscsipreinstall

  1. Type "dmesg" to verify that discovery occurred correctly. It should show that drives were recently discovered.
  2. Press Ctrl+Alt+F1 and select "Done".
  3. Continue with the installation.
NOTE: When partitioning the iSCSI LUs, ensure that your boot partition is not distributed across physical partitions on different iSCSI LUs. The Intel iSCSI Option ROM will only expose one iSCSI LU and if /boot is distributed across multiple LUs it will not be visible during the boot process.
  1. Configure the boot order
    1. On the "Boot Loader Configuration" select "Configure advanced boot loader options"
    2. Change the drive order so that your target disk is on the top of the list.
    3. Continue with remainder of installation.
    4. When the install is complete DO NOT press "Reboot".
  2. Make some final post install changes
    1. Instead of pressing "Reboot" press Ctrl-Alt-F2 to switch back to the console.
    2. Run the post install script and unmount your driver disk.

/tmp/iscsi/postinstall/iscsipostinstall

  1. Ensure the network is not shutdown during halt
  1. Edit the /mnt/sysimage/etc/init.d/halt and scroll to the bottom.
  2. Change the line stating HALTARGS="-i -d" to HALTARGS="-d"
  1. Return to the GUI and reboot
  1. Press Ctl+Alt+F7 and then select "Reboot".
  1. On your first boot, if Kudzu asks you if you'd like to configure your adapters, select to "ignore" the device.
NOTES: After the system has booted, iscsid will be running. The user may want to add something like "kill `pidof iscsid`" to an init script. iscsid is only needed for discovery and can be killed in this way without problems.

Though rare, during system reboot if kernel panics (while mounting root file system), here are the steps need to be followed:

  1. Uncompress initrd
  2. Open init script, go to line "sleep <value>" which is after line "/bin/iscsid"
  3. Increase value
  4. Install initrd, reboot
  5. If you still see the kernel panic, repeat steps 1 through 4.