# Installing a cloud server with full disk encryption For hosting purposes, I am installing a new server on [Kimsufi](kimsufi.com/). This is a French public cloud provider (property of [OVH](ovh.com/)). It is well known that, at [Deuxfleurs](https://deuxfleurs.fr), we do not fancy hosting in datacenters: because of the lack on control on our machines, and potential privacy threats. In essence, people at Kimsufi could easily -- and without me knowing -- read my hard drive, and thus any sensitive user data stored inside. For that reason, I want to make it harder for their staff to access my data. A good first line of defense for that purpose is to encrypt the whole disk (*aka* Full Disk Encryption). Note that motivated attackers with physical access to my server would easily overcome the encryption. **If you really care about who accesses your data, keep your hard drives closest to you, e.g. at your home.** --- Long story short: this guide presents a step-by-step installation of Debian buster on Kimsufi, using the [`dm-crypt`](https://wiki.archlinux.org/index.php/Dm-crypt) Linux kernel subsystem to encrypt the `/` and the `swap` partitions. Because I don't have physical access to the server, I need a way to *remotely enter the disk decryption password*, that is `dropbear` (a lightweight SSH server) inside `initramfs` (a minimal operating system that runs from RAM, before Linux boots). If you plan on doing the same without Kimsufi, know that it makes no difference: I am basically using a "rescue" console on another computer to configure `/dev/sda` (my server's system drive) for booting Linux with encrypted partitions. I must credit several useful guides that helped me succeed: * [OpsBlog guide on installing Linux with full-disk encryption and DropBear SSH on Kimsufi](https://opsblog.net/posts/full-disk-encrypted-ubuntu-kimsufi-sever/) DropBear allows SSH connection before boot, to remotely decrypt the main partition. * [Debian guide on installing Debian from a Linux system](https://www.debian.org/releases/wheezy/mipsel/apds03.html.en) ## Overview of steps * Boot the Kimsufi server in rescue mode (gives you a remote shell with access to your system's hard drive). * Wipe filesystem, format partitions. I basically want: * A "cleartext" `/boot` partition * An encrypted `/` partition * An encrypted swap partition Docs on the encrypted swap: * [ArchLinux wiki: dm-crypt - Swap encryption](https://wiki.archlinux.org/index.php/Dm-crypt/Swap_encryption) * [cryptsetup FAQ](https://gitlab.com/cryptsetup/cryptsetup/-/wikis/FrequentlyAskedQuestions#2-setup): see "2.3 How do I set up encrypted swap?" * Install and configure Debian (including disk cryptography and decryption through SSH). * Boot and log into the server. * Setup automated decryption from a remote (in case of reboot). ## Detailed process ### Boot the server in rescue mode * Go to the Kimsufi admin panel, click "NetBoot", select "Rescue", pick "rescue64-pro", clock "Next", "Confirm", and then click the "Reboot" button on the admin panel. * Kimsufi will mail you the root password for you to log into the server in rescue mode. * Will likely cause an error from your SSH client, since the fingerprint of the server differs from the usual one. You should end up with a shell that has access to `/dev/sda`, your server's drive. ### Wipe filesystem & format partitions Target partition table: ``` +-----------+--------------------+-----------+--------------------+ Name: | /dev/sda1 | /dev/sda2 | /dev/sda3 | /dev/sda4 | | | | | | | | | | | | | +------------------+ | +------------------+ | | | /dev/mapper/swap | | | /dev/mapper/main | Format: | none | | swap | ext4 | | ext4 | Size: | 1MiB | | 8GiB | 512MiB | | remaining space | Flags: | bios_grub | | | | | | Mount: | | | swap | /boot | | / | +-----------+-+------------------+-----------+-+------------------+ ``` We use GPT partition table layout (without UEFI), which demands a small `bios_grub` partition at the beginning of the drive (stores GRUB's `core.img`). Hence `/dev/sda1`. See: * [Wikipedia: BIOS boot partition](https://en.wikipedia.org/wiki/BIOS_boot_partition) * [GNU: BIOS installation](https://www.gnu.org/software/grub/manual/grub/html_node/BIOS-installation.html#BIOS-installation) * [ArchLinux wiki: Partitioning](https://wiki.archlinux.org/index.php/partitioning#Partition_table) * [Gentoo wiki: Installation - Disks](https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Disks#Partition_tables) * Wipe all information about previous filesystem: DANGER!!! DANGER!!! WIPES EVERYTHING FROM YOUR DISK!!! ```bash wipefs -a /dev/sda ``` * Create GPT disk layout ```bash # Create GPT layout parted -a optimal /dev/sda mklabel gpt # Create first 1MiB bios_grub partition parted /dev/sda -a optimal mkpart bios_grub 0% 1MiB parted /dev/sda set 1 bios_grub on # Create third 8GiB swap partition parted /dev/sda -a optimal mkpart swap 1MiB 8001MiB # Create second 512MiB /boot partition parted /dev/sda -a optimal mkpart boot 8001MiB 8513MiB # Create last / partition using all remaining space parted /dev/sda -a optimal mkpart main 8513MiB 100% # Set first partition as bootable ``` * Format partitions * The ext4 boot partition Format it like so: ```bash mkfs.ext4 /dev/sda3 ``` * The encrypted swap partition All configuration of the encrypted swap partition comes after the OS installation. * The encrypted main partition ```bash # Set it up using cryptsetup (I searched for good parameters): cryptsetup --type luks2 --cipher aes-xts-plain64 --hash sha256 --iter-time 2000 --key-size 512 luksFormat /dev/sda4 # Get the UUID, save it for later: cryptsetup luksDump /dev/sda4 | grep UUID | awk '{print $2}' # Example output: 09762476-ba7c-4732-8856-44a716c23339 # Decrypt the partition: cryptsetup open /dev/sda4 main # Format it to ext4: mkfs.ext4 /dev/mapper/main ``` ### Install and configure Debian * Mount the partitions ```bash # Decrypt the main partition (if not already done): cryptsetup open /dev/sda4 main # mount it: mount /dev/mapper/main /mnt # Mount the boot partition: mkdir /mnt/boot mount /dev/sda3 /mnt/boot ``` * Bootstrap latest stable Debian into mounted partitions ```bash debootstrap --arch amd64 stable /mnt http://ftp.fr.debian.org/debian/ ``` * Mount system partitions ```bash mount -o bind /dev /mnt/dev mount -t proc proc /mnt/proc mount -t sysfs sys /mnt/sys mount -t devpts devpts /mnt/dev/pts ``` * Chroot chroot /mnt /bin/bash * Fill out `crypttab` and `fstab` (`UUID` is the output of above `cryptsetup luksDump`) `/etc/crypttab` ```bash cat << EOF > /etc/crypttab # main UUID=09762476-ba7c-4732-8856-44a716c23339 none luks swap /dev/sda2 /dev/urandom swap,noearly,cipher=aes-xts-plain64,size=512 EOF ``` `/etc/fstab` ```bash cat << EOF > /etc/fstab # /dev/mapper/main / ext4 errors=remount-ro 0 1 /dev/sda3 /boot ext4 defaults 0 2 /dev/mapper/swap none swap sw 0 0 EOF ``` * Do the `/proc/mounts` -> `/etc/mtab` symlink ```bash ln -s /proc/mounts /etc/mtab ``` Why? [Linux From Stratch](http://archive.linuxfromscratch.org/lfs-museum/3.3/LFS-BOOK-3.3-HTML/chapter06/mtablink.html) and [our guide on OpsBlog](https://opsblog.net/posts/full-disk-encrypted-ubuntu-kimsufi-sever/) both say it's needed. * Choose a hostname & DNS domain hammerhead & hammerhead.luxeylab.net * Fill in network-related files `/etc/sysctl.d/10-ipv6` Disable IPv6 auto-configuration ```bash cat << EOF > /etc/sysctl.d/10-ipv6 net.ipv6.conf.all.autoconf=0 net.ipv6.conf.all.accept_ra=0 EOF ``` `/etc/network/interfaces` Beware of your ethernet interface name! It's only after failing with `eth0` that the boot logs informed me that the iface name was `eno1`. ```bash cat << EOF > /etc/network/interfaces ###################################################################### # /etc/network/interfaces -- configuration file for ifup(8), ifdown(8) # See the interfaces(5) manpage for information on what options are # available. ###################################################################### # loopback interface auto lo iface lo inet loopback # eno1 auto eno1 allow-hotplug eno1 iface eno1 inet static address 5.135.179.11 netmask 255.255.255.0 gateway 5.135.179.254 iface eno1 inet6 static address 2001:41d0:8:ba0b::1 netmask 128 post-up /sbin/ip -f inet6 route add 2001:41d0:8:baff:ff:ff:ff:ff dev eno1 post-up /sbin/ip -f inet6 route add default via 2001:41d0:8:baff:ff:ff:ff:ff pre-down /sbin/ip -f inet6 route del 2001:41d0:8:baff:ff:ff:ff:ff dev eno1 pre-down /sbin/ip -f inet6 route del default via 2001:41d0:8:baff:ff:ff:ff:ff EOF ``` [This OVH IPv6 guide](https://docs.ovh.com/gb/en/dedicated/network-ipv6/) might help you out for the IPv6 part. `/etc/resolv.conf` ```bash cat << EOF > /etc/resolv.conf # OVH public DNS nameserver 213.186.33.99 # https://servers.opennicproject.org/edit.php?srv=ns2.he.de.dns.opennic.glue nameserver 172.104.136.243 EOF ``` `/etc/hosts` ```bash cat << EOF > /etc/hosts 127.0.0.1 localhost 127.0.1.1 hammerhead hammerhead.luxeylab.net # The following lines are desirable for IPv6 capable hosts ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters ff02::3 ip6-allhosts 213.186.33.116 ws.ovh.com 198.245.48.4 ws.ovh.ca 5.135.179.11 hammerhead hammerhead.luxeylab.net EOF ``` `/etc/hostname` ```bash echo "hammerhead" > /etc/hostname ``` `/etc/timezone` ```bash dpkg-reconfigure tzdata ``` * Update APT repositories in `/etc/apt/sources.list` ```bash cat << EOF > /etc/apt/sources.list deb http://ftp.fr.debian.org/debian buster main deb-src http://ftp.fr.debian.org/debian buster main deb http://ftp.fr.debian.org/debian-security/ buster/updates main deb-src http://ftp.fr.debian.org/debian-security/ buster/updates main deb http://ftp.fr.debian.org/debian buster-updates main deb-src http://ftp.fr.debian.org/debian buster-updates main EOF ``` * Configure locales & keyboard ```bash # Refresh packages list apt update # Install & configure locales apt install locales && dpkg-reconfigure locales # Install & configure keyboard apt install console-setup console-data && dpkg-reconfigure keyboard-configuration ``` * Pick up a kernel & install it ```bash apt-cache search linux-image # I selected the meta-package `linux-image-amd64` which is the latest apt install linux-image-amd64 linux-headers-amd64 ``` * Install additional necessary software ```bash apt install cryptsetup dropbear grub-pc man-db ssh ``` Because we use a GPT table layout but no UEFI, we install `grub-pc` and not `grub-efi-amd64`. Do think about installing `bash-completion` and `ufw` someday. * Set up SSH keys and the rest (where `` is your client SSH public key) ```bash mkdir /root/.ssh && chmod 600 /root/.ssh echo "" > /root/.ssh/authorized_keys echo "" > /etc/dropbear-initramfs/authorized_keys ``` * Configure dropbear-initramfs (SSH on boot) ```bash sed -i 's/#DROPBEAR_OPTIONS=/DROPBEAR_OPTIONS=\"-p 2222 -c \/bin\/cryptroot-unlock\"/' /etc/dropbear-initramfs/config ``` This changes the SSH listen post to 2222, and enforces that the only command run from `dropbear-initramfs` is `cryptroot-unlock`. * Configure sshd custom listen port (SSH after boot) ```bash sed -i 's/#Port 22/Port 2223/' /etc/ssh/sshd_config ``` * [Optional] Authorize root password login ```bash # Remove -s (disable password login) from dropbear-initramfs sed -i 's/local flags=\"Fs\"/local flags=\"F\"/' /usr/share/initramfs-tools/scripts/init-premount/dropbear ``` * Update GRUB and initramfs ```bash update-grub && update-initramfs -u ``` * Clean up before leaving ```bash exit umount /mnt/{boot,dev/pts,dev,proc,sys} umount /mnt cryptsetup luksClose main ``` ### Boot and log into the server * I configured my laptop's `~/.ssh/config` like so: ``` Host hammerhead-decrypt User root Hostname Port 2222 IdentityFile # Prevents signatures mismatch UserKnownHostsFile ~/.ssh/known_hosts_hammerhead-decrypt Host hammerhead User root Hostname Port 2223 IdentityFile ``` * And here is how I connect to the server: ```bash laptop$ ssh hammerhead-decrypt Please unlock disk main: # [...] Connection to closed. laptop$ ssh hammerhead # [...] root@hamerhead:~# echo A winner is you! A winner is you! ``` ### Setup automated decryption from a remote It is desirable to have a daemon running on a remote server, to automatically decrypt the drive when the encrypted server reboots without warning. The remote server is called a *key escrow*. One must be particularly careful about the escrow's security, since it holds the decryption keys for our server. Trinity recommends [Tang](https://github.com/latchset/tang) and [Clevis](https://github.com/latchset/clevis).