Moving to a new disk on Debian with root filesystem on LVM over encryption mini-HOWTO

Moving to a new disk on Debian system with root Filesystem on LVM over encryption mini-HOWTO
August 2007,

Thanks to the Debian team for all your hard work & great software.

Current setup
I used the standard Etch installer to setup and encrypted disk for my laptop. However, I now want to move to a bigger hard disk, without having to reinstall the system. In this mini howto I will show you how I did it. I will try and make it as generic as possible. However, following this recipe exactly will not work unless you have a similar set up. You will need to examine your own system and make modifications as applicable.
This technique could even be used to change the passphrase of an encrypted disk as it makes a straight copy. It could also create a bootable system on a USB external drive (provided your system can boot off external media), although there are other HOWTO's floating around that will probably be more helpful.
The basis of my method is to copy data to & modify the new disk only, leaving the original installed system on the old disk pristine. This is conservative, giving us both backup & reversion capability. Having the root filesystem on LVM over encryption complicates the the move as it requires an initramfs image to boot. However one of the limitations of LVM is it prevents us using the same volume-group name twice (which is logical). Unfortunately the update-initramfs script hard codes the LVM root filesystem volume group into the initramfs image, but we will need to change LVM volume groups for the new system in the intramfs image for it to be bootable. We will have to manually unpack, modify the hard coded root LVM volume group & repack our initramfs image so we can boot the new disk. I developed this method using a virtualbox debian virtual machine, installing a system that replicated my laptop's. This allowed me to practice everything without leaving my real system unbootable. I would highly recommend a run through on a virtual machine as practice, as this is a complex procedure. Any mistakes along the way can be fatal to your system.
I would also recommend reading the Hard disk upgrade mini HOWTO. If you don't understand the LVM commands take a close look at the LVM HOWTO.

This HOWTO is debian centric, however it should provide enough hints for other ditro's. I am just a user, documenting the steps I took to move my data. I'm sure there are easier methods & any suggestions or improvements would be welcomed. If it is useful to people, perhaps it could be included in the debian documentation. Please feel free to copy, modify & use it as you see fit.

Disclaimer: Use of the following information may destroy your data, hard drive, cause spontaneous combustion of your computer & possibly impregnate your dog. Use at your own risk, I take absolutely no responsibility for any loss or damage.
Having said all that, it worked for me :)

In brief, the step to moving the data to a new disk are:
1) Working out your disk layout
2) preparing a new disk
3) Copy your data
4) Rebuild initramfs image
5) Modify fstab, initramfs-tools & crypttab config
6) Install grub on your new disk MBR
7) Swap disks
8) Post move clean-up
9) Enjoy the new roominess!

1) Disk Layout
The Etch installer put the original system on two partitions, the first being a small unencrypted boot partition (hda1), while everything else is on a larger LVM over encrypted partition (hda5).
Working out exactly how your encrypted setup is configured is the one of the hardest parts of the procedure. dmsetup can help us here:
# dmsetup status
vg01-home: 0 25165824 linear
vg01-tmp: 0 1638400 linear
vg01-swap: 0 3072000 linear
vg01-usr: 0 13631488 linear
vg01-root: 0 1024000 linear
vg01-var: 0 4194304 linear
hda5_crypt: 0 58209881 crypt

Together with fstab (or mount) we can work out how things fit together. Here is an extract from my /etc/fstab
/dev/mapper/vg01-root / ext3 defaults,errors=remount-ro 0 1
/dev/hda1 /boot ext3 defaults 0 2
/dev/mapper/vg01-home /home ext3 defaults 0 2
/dev/mapper/vg01-tmp /tmp ext3 defaults 0 2
/dev/mapper/vg01-usr /usr ext3 defaults 0 2
/dev/mapper/vg01-var /var ext3 defaults 0 2
/dev/mapper/vg01-swap none swap sw 0 0

So, in my case, things are arranged as
/ logical volume vg01-root over encrypted disk hda5_crypt
/boot hda1
/home logical volume vg01-home over encrypted disk hda5_crypt
/tmp logical volume vg01-tmp over encrypted disk hda5_crypt
/usr logical volume vg01-usr over encrypted disk hda5_crypt
/var logical volume vg01-var over encrypted disk hda5_crypt
swap logical volume vg01-swap over encrypted disk hda5_crypt

This is how I visualize my disk being arranged:

| / |/home | /usr | /var |/tmp | swap | <--- Filesystem Layer
| vg01 LVM layer |
| /boot | hda5_crypt LUKS encrypted layer | --------|------------------------------------------------------|
| hda1 | hda5 Physical disk | ----------------------------------------------------------------

As I said, this is one of the hardest parts, working out how things are arranged. Stop here until you understand how your disk is arranged. You cannot complete the move until you know your layout.
From here on in, I will be using reference to *MY* old volume group vg01 & *MY* new volume group vg02. You will need to change this as appropriate for your old & new targets.

2) Preparing the new disk.
We are going to rebuild the new disk in layers, starting at the bottom, with the encryption, then the LVM & finally the filesystem.
I have an IBM T41, which allows me to install a second hard drive via the ultrabay, and it is seen as hdc. If you don't have access to an ultrabay, or cannot install a second drive, you could use an external USB drive case. You will need to change the target disk to suit your setup.
From here it is critical that you get your target devices correct, otherwise you will destroy your existing data. Type carefully, stop, think and execute.
I will be keeping the exact layout on the new disk. The new disk is partitioned with cfdisk
# cfdisk /dev/hdc
adding a small partition (250M) hdc1 for the boot, and the rest of the disk as hdc5 for the encrypted partition.
Optionally, many suggest over-writing the disk with random data. Goggle is your friend here.

We will now encrypt the larger partition.
# cryptsetup luksFormat /dev/hdc5
you will be prompted for a luks passphrase, twice. This is your chance to change your passphrase!

Now we can open the disk.
# cryptsetup luksOpen /dev/hdc5 hdc5_crypt
you will be prompted for your luks passphrase, enter it.

Now we prepare the LVM side of things, this is straight out of the LVM howto.
Make a LVM physical volume over the encrypted disk
# pvcreate /dev/mapper/hdc5_crypt

Create a new volume group. Gotcha, this *CANNOT* be the same as an existing volume group, this leads to complications later with the initramfs image. We will call the new volume group vg02.
# vgcreate vg02 /dev/mapper/hdc5_crypt

Activate the volume group
# vgchange -a y vg02

Now we can create the logical volumes for our data, feel free to choose you size as appropriate:
root 700Mb, home 20Gb, usr 7Gb, tmp 2Gb & swap 1.5Gb
# lvcreate -L700 -nroot vg02
# lvcreate -l20G -nhome vg02
# lvcreate -L7G -nusr vg02
# lvcreate -L2G -ntmp vg02
# lvcreate -L2G -nvar vg02
# lvcreate -L1.5G -nswap vg02

Make our filesystems, checking for bad blocks:
# mkfs.ext3 -c /dev/hdc1
# mkfs.ext3 -c /dev/vg02/root
# mkfs.ext3 -c /dev/vg02/home
# mkfs.ext3 -c /dev/vg02/usr
# mkfs.ext3 -c /dev/vg02/tmp
# mkfs.ext3 -c /dev/vg02/var

Finally make the swap
# mkswap /dev/vg02/swap

Now we can use the hard disk upgrade mini-howto

Mount the partitions
# mkdir /new-disk
# mount /dev/vg02/root /new-disk

# mkdir /new-disk/home
# mount /dev/vg02/home /new-disk/home

# mkdir /new-disk/usr
# mount /dev/vg02/usr /new-disk/usr

# mkdir /new-disk/tmp
# mount /dev/vg02/tmp /new-disk/tmp

# mkdir /new-disk/var
# mount /dev/vg02/var /new-disk/var

# mkdir /new-disk/boot
# mount /dev/hdc1 /new-disk/boot

let all users access tmp
# chmod 1777 /new-disk/tmp

3) Copy your Data
Unmount any network or external drives that you don't want to copy data from.
Drop into single user mode so the filesystem won't change.
# telinit 1

Copy the files (again straight from the upgrade howto) This will take quite some time.
*NOTE* that's the numeral one, not the letter l for -1Ab
# cd / && echo cp -a `/bin/ls -1Ab | egrep -v "^new-disk$|^proc$"` /new-disk | sh

Make the /proc directory:
# mkdir /new-disk/proc

4) Rebuild initramfs image
Because update-initramfs script only looks at the current running LVM volume group for the root filesystem, we need to manually update it to use our new volume group.

Make somewhere safe to manipulate the initramfs image.
# mkdir /new-disk/tmp/initramfs
# cd /new-disk/tmp/initramfs

Find current running kernel
# uname -a
Linux laptop 2.6.21-1-686 #1 SMP Sat May 26 16:14:59 UTC 2007 i686 GNU/Linux <---- yours may be different

Copy the current running kernel initramfs image
# cp /new-disk/boot/initrd.img-2.6.21-1-686 /new-disk/tmp/initramfs/ <-- use your running kernel from previous step

Unpack the initramfs image:
# zcat /new/disk/tmp/initramfs/initrd.img-2.6.21-1-686 | cpio -i

Remove the original iniramfs image from the current directory:
# rm /new-disk/tmp/initramfs/initrd.img-2.6.21-1-686

Find any instances of vg02 in the extracted files:
# grep -r vg01 /new-disk/tmp/initramfs/*

Change all references to vg01 to vg02. I had to modify the following files:
I also removed any backup files left by my editor.

Now we need to repack the initramfs image:
# find . | cpio --quiet -o -H newc | gzip -9 > /new-disk/tmp/initramfs/initrd.img-2.6.21-1-686

Move old image out of the way:
# mv /new-disk/boot/initrd.img-2.6.21-1-686 /new-disk/boot/initrd.img-2.6.21-1-686.OLD

copy the modified image back to the boot directory:
# cp /new-disk/tmp/initramfs/initrd.img-2.6.21-1-686 /new-disk/boot/

5) Modify fstab, initramfs-tools & crypttab config.
Now we need to modify fstab, once again changing any reference of vg01 to vg02 in /new-disk/etc/fstab

Initramfs-tools scripts
Find all references to vg01 & change to vg02 for initramfs-tools configs
# grep -r vg01 /new-disk/etc/initramfs-tools/*

Edit all the files that contain reference to vg01 and change to vg02. For me this is:
(I also removed the temporary files created by the editor joe ending with ~)

Crypttab only needs to be modified if you have changed the layout of your disk. In my case I *DID NOT* change anything, because hdc will become hda when swapped, and I kept the the encrypted partition as the first logical partition on that disk. For completeness, my crypttab:
# cat /new-disk/etc/crypttab
hda5_crypt /dev/hda5 none luks

6) Install grub on your new disk MBR
Change all references to vg01 to vg02 in the /new-disk/boot/grub/menu.1st file.
Make sure you modify the following line, even though it is commented out, grub-update uses this & *MUST* be updated to your new volume group.
# kopt=root=/dev/mapper/vg01-root ro

Install grub on to the MBR of the new disk (make sure you get your disk correct, or your current system may become unbootable)
# grub-install --root-directory=/new/disk --recheck /dev/hdc
(this takes a long time, about 10 minutes on my machine)

Final check
Optional check for any missed references to the original volume group vg01 on the new drive
# grep -r vg01 /new-disk/boot/*
# grep -r vg01 /new-disk/etc/*
The only files should be contain reference to vg01 are /etc/lvm/cache/.cache & /etc/lvm/.cache, these don't need to be changed (you may however also find backup files from you editor, Joe leaves behind files ending in ~).

7) Swap disks
Shutdown, physically swap disks. Boot into the kernel of the modified the initramfs image (2.6.21-1-686 for me). Hopefully you should see grub run the initrd image runs & be prompted for you luks password and a normal boot. If things don't work, you can swap disk back again, carefully re-read these instructions & try to work out what went wrong.

8) Post move cleanup
Remove the old image file we moved aside
# rm /boot/initrd.img-2.6.21-1-686.OLD

Now we need to do is rebuild all the initramfs images for all installed kernels. Have a look in your /boot directory
# ls | grep initrd

We can get each kernel package to rebuild its initrd image file with:
# dpkg-reconfigure linux-image-2.6.21-1-686
Repeat the process with all of your installed kernels. It will also re-run grub each time.

Finally, being a laptop I had to update uswsusp package config:
Change vg01 references to vg02 in /etc/uswsusp.conf & reconfigure the package.
# dpkg--reconfigure uswsusp


To UNSUBSCRIBE, email to debian-user-REQUEST@xxxxxxxxxxxxxxxx with a subject of "unsubscribe". Trouble? Contact listmaster@xxxxxxxxxxxxxxxx

Relevant Pages

  • Re: undo LVM
    ... I have a 1TB Data disk that is encrypted with luks and I ... On the partition screen I accidentally added the encrypted partition ... Now I have the problem how to revert the LVM part that I can decrypt ... And I don't know anything about encryption. ...
  • Re: lvm2 on top of dm-crypt - how to add new hard disk
    ... encryption scheme to the new hard disk using standard lvm tools and the ... and this arch wiki saying I should better have the reverse: luks on lvm. ... you could do what you're after if you had luks on lvm. ...
  • Re: undo LVM
    ... As my main problem is the LVM and not the encryption, ... As its full disk encryption, you can think of just a partition with ... normally use this encrypted partition again. ...
  • Re: Missing Disk Space or Partition
    ... Those will tell you what LVM thinks that it has. ... but it appears that your disk is ... Using overlapping partitions could cause problems, ... cylinders that might make sense, but I have never seen it before. ...
  • Re: lvm
    ... but ext4 got stable late in 2008 ... disk because you can expand it and with a little more work ... so in my wold LVM is useless, the big data are on sevrers ... snapshots and at least having a virtualization layer supporting ...