Monday 17 April 2017

Make Grub root your boot: Full Disk Encryption edition

You try to boot up your Linux machine, and hit an unfriendly Grub recovery console.

Oh no! Grub tends to fall over for a variety of reasons, including:
  • You recently updated your kernel (apt-get upgrade listed some linux-headers-* packages), and something didn't complete properly
  • You recently updated your kernel, and your proprietary nvidia/amd graphics drivers aren't playing nicely with it
  • /boot/grub/grub.conf got corrupted somehow
  • Your partitions/hdd's have changed
  • Grub thought you weren't sufficiently respecting it
In any case, there are generally two approaches to resolution:

Manually Boot

grub.cfg is basically just a "stable" set of boot commands, which generally boil down to
root='(hd0,1)'
linux /vmlinuz-3.19.0-32-generic root=/dev/sda2 ro quiet
initrd /initrd.img-3.19.0-32-generic
boot
root= sets what the current grub cli considers the boot root partition to be.
ls
Should list out all your devices. Try setting root='(a listed device)', and then try
ls /
If you have a separate /boot/ partition (ie. if your main root is encrypted), you should see a list of vmlinuz's and initrd.img's.
If your /boot/ is on your main root partition, you should see /bin, /sbin, /boot, /var, etc. There'll be vmlinuz's in /boot/*.
If you're seeing neither, try another device.


To find the right vmlinuz (Linux Kernel executable), you'll probably want the newest version you can see in ls / or ls /boot/ .

To find the right linux root=/path , you'll want to figure out which /dev/ device points at your typical / (with /bin, /sbin, /var, etc).
lsblk may help.
In a typical setup, this might be /dev/sda1
If your main root is encrypted, this might be something like /dev/mapper/nvme0n1p6_crypt . If you don't know and proceed with the wrong one, you'll probably be prompted to unlock your encrypted drive and then fallback to a busybox shell - at which point ls /dev/mapper/ should be able to list them.

For the initrd path (initial ramdisk), make sure to use the same version as your vmlinuz (kernel)!

Once you've booted, be sure to run
sudo update-grub
which should self-fix issues. If you notice in the resulting list of vmlinuz's and initrd's that the most recent one isn't a pair (you have a linuz-3.19.0-36 but no initrd.img-3.19.0-36), you'll want to reinstall that linux-image, ie
sudo aptitude reinstall linux-image-3.19.0-32-generic

Fix Grub using a Live USB

Boot up using any modernish version of Linux's live (ramdisk) mode, and try
sudo grub-install /dev/sda2
lsblk may help determine which drive to use. If you'd like to look around,
mkdir /mnt/somedrive && sudo mount /dev/sda2 /mnt/somedrive
may allow you to ls -l and such to confirm you have the right drive.
If your drive is encrypted, you may have to decrypt it first, and then it'll be named something like /dev/mapper/nvme0n1p6_crypt . I haven't had to do this yet, so I don't have further suggestions.