Root-on-LVM-on-RAID with debian mkinitrd HOWTO

From: Charles Steinkuehler (cstein_at_newtek.com)
Date: 02/28/04

  • Next message: MacNean Tyrrell: "Sound and Linux Kernel 2.6" 
    Date: Sat, 28 Feb 2004 14:56:02 -0600
To: debian-boot@lists.debian.org, debian-user@lists.debian.org, linux-lvm-approval@msede.com

    Thanks to the work of others:
    http://lists.debian.org/debian-boot/2003/debian-boot-200301/msg00115.html
    http://www.midhgard.it/docs/index_en.html

    I have a Debian system running with root on LVM on RAID. Kernel
    updates, however, require either a manual update of an initrd file, or a
    custom kernel compile to include all required modules.

    I really wanted to use apt-get/dpkg to install kernel upgrades straight
    from the official Debian binaries, so I spent some time and got Debian's
    mkinitrd script working with root on LVM on RAID.

    NOTE: This procedure does *NOT* require any modifications to the
    existing mkinitrd script, it simply makes use of the hooks provided
    already to support LVM (RAID is already supported), so there's little
    danger of totally breaking something with a debian upgrade, unless the
    initrd structure changes drastically.

    To get mkinitrd working with LVM, it first has to be prevented from
    probing the root device, as it doesn't understand LVM devices. Just set
    root=/dev/md1 (or the appropriate raid device) in
    /etc/mkinitrd/mkinitrd.conf. With no probing, you also need to
    explicitly list the modules required to boot in /etc/mkinitrd/modules.

    To start mount root on an LVM partition, it's also necessary to add a
    few programs (/etc/mkinitrd/exe), device files (/etc/mkinitrd/files) and
    a script to the initial ramdisk. The acutal LVM modules will likely
    already be on your initial ramdisk, if you use the default
    "MODULES=most" setting in mkinitrd.conf.

    The following settings work for me (x86 arch), and should work well for
    other architectures if they use the same initial ramdisk scripts from
    Debian. I hope to try this with an Alpha system soon, but have not
    tried anything but x86 so far.

    -----------------------
    $ cat /etc/mkinitrd/exe
    -----------------------
    /bin/cp
    /sbin/lvmiopversion
    /sbin/vgscan
    /sbin/vgchange
    /lib/lvm-10/vgscan
    /lib/lvm-10/vgchange
    /lib/lvm-default/vgscan
    /lib/lvm-default/vgchange

    -------------------------
    $ cat /etc/mkinitrd/files
    -------------------------
    /dev/lvm

    ---------------------------------
    $ cat /etc/mkinitrd/mkinitrd.conf
    ---------------------------------
    # /etc/mkinitrd/mkinitrd.conf:
    # Configuration file for mkinitrd(8). See mkinitrd.conf(5).
    #
    # This file is meant to be parsed as a shell script.

    # What modules to install.
    MODULES=most

    # The length (in seconds) of the startup delay during which linuxrc may be
    # interrupted.
    DELAY=0

    # If this is set to probe mkinitrd will try to figure out what's needed to
    # mount the root file system. This is equivalent to the old PROBE=on
    setting.
    ROOT=/dev/md1

    # This controls the permission of the resulting initrd image.
    UMASK=022

    # Command to generate the initrd image.
    MKIMAGE='mkcramfs %s %s > /dev/null'

    -------------------------------------
    $ cat /etc/mkinitrd/scripts/00rootlvm
    -------------------------------------
    #!/bin/sh

    exec >> $INITRDDIR/script

    cat << "EOF"
    mount_lvm_root() {
             unset flags fstype rootdev
             set -f
             set +f $(cat /proc/cmdline)
             for i; do
                     case "$i" in
                     rootflags=*)
                             flags=${i#rootflags=}
                             ;;
                     rootfstype=*)
                             fstype=${i#rootfstype=}
                             ;;
                     root=*)
                             rootdev=${i#root=}
                             ;;
                     esac
             done
             if [ -n "$fstype" ]; then
                     mount -nrt "$fstype" ${flags:+-o "$flags"} $rootdev /mnt
                     return
             fi
             IFS=,
             set -f
             set +f -- $FSTYPES
             unset IFS
             for i; do
                     mount -nrt "$i" ${flags:+-o "$flags"} $rootdev /mnt &&
                             break
             done
    }

    cd /
    mount -nt proc proc proc
    mount -nt tmpfs tmpfs etc
    mount -nt tmpfs tmpfs dev2
    cp -a dev/* dev2/
    mount --bind dev2 dev
    vgscan
    vgchange -ay
    mount_lvm_root

    # Uncomment sh command below to allow manual debugging at boot-time.
    # If you can mount root on /mnt, exit the shell and the system will boot
    # normally, allowing you to fix whatever was wrong with this script.
    # If you can't mount root, boot a working kernel/initrd and try again.
    # sh

    cd /
    echo 256 > proc/sys/kernel/real-root-dev
    umount dev
    umount dev2
    umount etc
    umount proc
    cd mnt
    [ $DEVFS ] && mount -nt devfs devfs dev
    pivot_root . initrd
    EOF

    -------------------------
    Notes on 00rootlvm script
    -------------------------

    The 000rootlvm script appends some commans to the /script file in the
    initial ramdisk created by mkinitrd. By the time the code above is run,
    the default debian mkinitrd scripts have mounted the required kernel
    modules and activiated the raid partition /root resides on. All that is
    left is to start LVM and mount the LVM root partiton.

    In order for the LVM scritps to work happily, /dev and /etc need to be
    read-write, but the default debian mkinitrd is cramfs (read-only). To
    get around this, writable partitions are mounted on top of the existing
    cramfs directories. The /etc directory can be empty (it will be
    populated with lvmtab and lvmtab.d by vgscan), but the /dev directory
    needs at least the lvm and null devices (possibly others). I initially
    mount a tmpfs filesystem on dev2 and copy the existing devices to avoid
    hard-coding any device knowledge in the script. This way, you can
    simply add any special devices required for your system to
    /etc/mkinitrd/files and they'll automatically be avaialble when the
    script above runs. Once the dev2 filesystem is popluated, it's
    remounted on top of /dev with mount --bind.

    Finally, root is mounted using the root= portion of the kernel command
    line for the device (should be /dev/<volumegroup>/<logicalvolume>), and
    any supplied kernel command line parameters for mount flags or filesystem.

    If you want to give yourself a chance to look around (and maybe fix
    anything broken) before just blindly trying to boot, uncomment the 'sh'
    command in the script above. When booting, the script will drop to a
    shell after (hopefully) mounting root. As long as you exit the shell
    with root properly mounted in /mnt, the system should boot normally.

    --------------------------
    Additional important notes
    --------------------------

    - Obviously, you need to setup your boot loader to support initial
    ramdisks. If you're lucky, there will be an update-* command for your
    bootloader that will automatically create/remove boot entries when you
    add/remove a kernel. My /etc/kernel-img.conf contains:
       do_symlinks = No
       do_initrd = Yes
       do_bootloader = No
       postinst_hook = /sbin/update-grub
       postrm_hook = /sbin/update-grub

    - You'll need an appropriate setting for root in the kernel command line
    provided by your boot-loader. My /boot/grub/menu.lst file contains the
    following setting (inside the debian AUTOMAGIC KERNELS LIST markers):
       # kopt=ro root=/dev/SystemCore/root console=ttyS0,38400n8

    - My system has 2 raid devices, md0 (/boot) and md1 (the big RAID with
    logical volumes). Tweak the root= device in mkinitrd.conf to reflect
    the RAID device your root filesystem is on.

    - I had to add the following to prevent LVM from spitting out a bunch of
    (harmless) warnings at boot when the LVM code looks for logical volumes
    on *ALL* available block devices. This is likely specific to x86 arch,
    and of course you should leave this device properly enabled if you
    actually *HAVE* a BIOS RAID. I'm mentioning it here to hopefully save
    you a heart-attack, or at least a google search :-) :
       # Disable IDE BIOS RAID devices to prevent (harmless) LVM errors
       alias block-major-114 off

    Please CC: me directly on any comments or requests for more info or
    clarification, as I'm currently only subsribed to these lists through
    archive searches (which have been very helpful!). :) 

    -- 
Charles Steinkuehler
cstein@newtek.com
-- 
To UNSUBSCRIBE, email to debian-user-request@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
  • Next message: MacNean Tyrrell: "Sound and Linux Kernel 2.6"

    Relevant Pages