Setting Up Full Disk Encrypted LVM on RAID-1 for Ubuntu

For several years now, I have run ZFS on Linux on my home Ubuntu desktop. My friend, Matt, and I have even thrown together some pretty handy scripts for some simple Linux tasks with ZFS, such as deleting sets of snapshots and simplifying backups. It was always a lot of fun to play with, and I always felt a deep sense of loyalty to ZFS, having proudly worked at Sun Microsystems. But, sadly, the romance is over. I found myself regularly running into stability and performance issues with ZFS on Linux. Combined with a deep desire for full disk encryption (which was never open sourced by Oracle, unlike the rest of ZFS), I finally decided to pull the plug.

So what to replace it with? Well, for my personal machine, what I really wanted is reliability, flexibility, and encryption. The first, I figured, would be covered by a mirrored RAID array, although perhaps it's a bit of a stretch to call 2 disk RAID 1 an "array." The second would be covered by LVM, and for the third I'd use the standard Ubuntu LUKS setup.

I quickly learned the good news: the default Ubuntu 13.10 installer installation is a LUKS fully disk encryption setup on LVM! So I started digging around for the RAID option so I could get going. Sadly, it was nowhere to be found, and the Internet confirmed it. Everyone suggested using the Ubuntu server install to get a RAID setup. However, I really don't like messing around to find every package to get the exact right desktop setup; if at all possible, I really wanted to use the standard installer.

So I dug, and dug, and dug, and I couldn't find anything with a proper explanation of how to do a standard installation with RAID. The closest I found was this EncryptedFilesystemLVMHowTo guide. So I spent long time trying to get it all right, and I figured I ought to share the knowledge in case anyone is looking to do the same.

1. Start by booting the Live CD or USB to the "Try Ubuntu" Desktop. The straight installer is not going to do you any good.
2. Open a terminal and sudo -i. You'll need this root shell throughout.
3. In the terminal, run apt-get install mdadm. This is the standard Linux software RAID manager.
4. Open gparted and create your partitions. I ran into a bit of trouble because my disks are 3TB each. This is a problem because the standard partition table can't handle disks larger than 2TB, so I had to create a GPT partition table.
5. In my case, because I was using GPT, I needed to create a 1Mb parition at the start of both disks and turn on the "bios_grub" flag.
6. I then made two more partitions on each disk: a 256Mb partition which will be the boot partition, and the rest of the disk which will be the raid array. Thus, in my setup, the final partitioned devices were:
   /dev/sda1 /dev/sda2 /dev/sda3/dev/sdb1 /dev/sdb2 /dev/sdb3
   The devices that end in 2, I formatted as ext4. The other devices I left "unformatted," but you can format them however you like as they'll be overwritten shortly anyway.
7. Next I created the RAID array my running in the shell:mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda3 /dev/sdb3
   This should start up a new device, /dev/md0, automatically.
8. Next, I created the encrypted LUKS device. In this particular case, the Ubuntu GUI works quite well, so I used the "Disks" utility provided by ubuntu, selected the md0 device, selected "format," and chose the "encrypted + ext4" option. Enter in your password of choice, and click "OK."
9. Now you should have a new encrypted device whose unencrypted version is mapped inside /dev/mapper. It will probably be a long, complicated ID, but let's call it /dev/mapper/luks-dev for this tutorial.
10. Now to create the LVM partitions. I wanted 3 LVM partitions: 1 for swap, 1 for my root install, and 1 for my home directory. Part of the beauty of LVM, though, is that you can change these up later. I set them up with the following:
1. pvcreate /dev/mapper/luks-dev
2. vgcreate ubuntu /dev/mapper/luks-dev
3. lvcreate -L 20G swap ubuntu
4. lvcreate -L 300G root ubuntu
5. lvcreate -L 2.5T home ubuntu
11. After the previous step, you should now have the /dev/mapper/ubuntu-swap, /dev/mapper/ubuntu-root, and /dev/mapper/ubuntu-home devices. Now start the Ubuntu installer, following the onscreen instructions until you get to the partition screen.
12. Select "other" for the partitioning method. You will have to tell the installer where you want everything to go.
13. Select your swap, root, and home partitions, telling the installer to format swap as a swap partition and root and home as ext4, setting their mount points to / and /home, respectively. Additionally, select /dev/sda2, setting the mount point to /boot.
14. Finally, click "install" and follow the on screen instructions until it has completely finished.
15. Almost there, but not quite. Unfortunately, the installer didn't know that you were installing on top of RAID, LUKS, or LVM, so you're going to have to manually update some of the installation yourself. First things first, you need to mount your new installation and chroot into it. Run the following to get that setup:
1. mount /dev/mapper/ubuntu-root /mnt
2. mount /dev/mapper/ubuntu-home /mnt/home
3. mount /dev/sda2 /mnt/boot
4. mount --bind /dev /mnt/dev
5. mount --bind /sys /mnt/sys
6. mount --bind /proc /mnt/proc
7. mount --bind /etc/resolv.conf /mnt/etc/resolv.conf
8. chroot /mnt
16. You should now be in a chroot of your new install. Run apt-get install mdadm initramfs-tools to make sure you have RAID and initramfs setup tools installed on your system.
17. Next, edit /etc/crypttab in your text editor of choice. It may not exist yet, but that's okay. Add the following line:
    luks-dev   /dev/md0   none   luks,retry=1,lvm=ubuntu
    where luks-dev is the name of your encrypted device that we're calling /dev/mapper/luks-dev.
18. Next, you need to update your initramfs so the system knows how to boot, then install it on your grub device so the system can find it. Do this by running the following:
1. update-initramfs -k all -c
2. update-grub
3. grub-install /dev/sda
19. Finally, you probably want to copy your grub partition to your second RAID device so you can theoretically boot from either disk (in practice, these will get out of sync, so if your main boot device fails, you'll probably have to boot into a Live CD/USB and reinstall grub to the second disk). To do this copy, you'll need to first load gparted to get the starting sector of /dev/sda3 (let's pretend that number is 50000), and subtracting 1. Then run dd if=/dev/sda of=/dev/sdb count=49999.
20. Now reboot, and you should be good to go!