nixcfg/doc/nixos-install.md

## Preparation

Download NixOS 21.11 ISO. Burn to USB.

## Booting into install environment

Boot the ISO on PC to install.

Become root with `sudo su`

```bash
loadkeys fr
setfont sun12x22
```

Do network config if necessary, see [install guide](https://nixos.org/manual/nixos/stable/index.html#sec-installation-booting-networking)

## Make partitions

```bash
cgdisk /dev/sda
```

Recommended layout:

```
/dev/sda1 	512M 	ef00 	EFI System partition
/dev/sda2 	100% 	8309 	Linux LUKS 
```

## Setup cryptography 

```bash
cryptsetup luksFormat /dev/sda2
cryptsetup open /dev/sda2 cryptlvm
```

## Create PV, VG and LVs

```bash
pvcreate /dev/mapper/cryptlvm
vgcreate NixosVG /dev/mapper/cryptlvm
lvcreate -L 8G NixosVG -n swap
lvcreate -l 100%FREE NixosVG -n root
```

## Format partitions

```bash
mkfs.fat -F 32 -n boot /dev/sda1
mkswap /dev/NixosVG/swap
mkfs.ext4 /dev/NixosVG/root
```

## Mount partitions

```bash
swapon /dev/NixosVG/swap
mount /dev/NixosVG/root /mnt
mkdir /mnt/boot
mount /dev/sda1 /mnt/boot
```

## Generate base NixOS configuration

```bash
nixos-generate-config --root /mnt
```

## Update `hardware-configuration.nix`

This section is needed:

```nix
  boot.initrd.luks.devices."cryptlvm" = {
    device = "/dev/disk/by-uuid/<uuid of sda2>";
    allowDiscards = true;
  };
```

And for the root filesystem, remember to add the `relatime` and `discard` options so that it looks like this:

```nix
  fileSystems."/" =
    { device = "/dev/disk/by-uuid/<...>";
      fsType = "ext4";
      options = [ "relatime" "discard" ];
    };
```

## Update `configuration.nix`

Just enough so that basic tasks can be done from keyboard and remotely:

- timezone
- keyboard layout
- font `sun12x22`
- vim
- non-root user
- ssh
- tcp port 22 in firewall

## Do the installation

```bash
nixos-install
```

## First boot

Reboot machine. Login as `root`

```bash
passwd <nonroot user>
```

If necessary, assign static IP. E.g. `ip addr add 192.168.1.40/24 dev eno1` or sth (replace ip and device appropriately)

Remotely: `ssh-copy-id <user>@<ip>`. Check SSH access is good.

## Deploy from this repo

It's time!

**Files in this repo to create/change:**

- create node `.nix` file and symlink for node `.site.nix` (create site and
  cluster `.nix` files if necessary; use existing files of e.g.  the staging
  cluster as examples/templates)
- make sure values are filled in correctly
- add node to `ssh_config` with it's LAN IP, we don't have VPN at this stage

**Configuration steps on the node:**

```bash
# On node being installed
mkdir -p /var/lib/deuxfleurs/remote-unlock
cd /var/lib/deuxfleurs/remote-unlock
ssh-keygen -t ed25519 -N "" -f ./ssh_host_ed25519_key
```

**Try to deploy:**

```bash
# In nixcfg repository from your PC
./deploy.sh <cluster> <nodename>
```

Reboot.

Check remote unlocking works: `ssh -p 222 root@<ip>`

## Configure wireguard

```bash
# On node being installed
mkdir -p /var/lib/deuxfleurs/wireguard-keys
cd /var/lib/deuxfleurs/wireguard-keys
wg genkey | tee private | wg pubkey > public
```

Get the public key, make sure it is in `cluster.nix` so that nodes know one
another.  Also put it anywhere else like in your local wireguard config for
instance so that you can access the node from your PC by its wireguard address
and not only its LAN address.

Redo a deploy (`./deploy.sh <cluster> <nodename>`)

Check VPN works. Change IP in `ssh_config` to use VPN IP instead of LAN IP (required for deploy when away from home).

## Commit changes to `nixcfg` repo

This is a good point to commit your new/modified `.nix` files.

## Configure Nomad and Consul TLS

If you are bootstraping a new cluster, you need to `./genpki.sh <cluster>` to
make a TLS PKI for the Nomad+Consul cluster to work. Then redo a deploy.