write some minimal documentation to get nodes up and running

This commit is contained in:
Alex 2022-02-25 21:54:53 +01:00
parent 07b2e93014
commit f5f0927b9e
Signed by untrusted user: lx
GPG key ID: 0E496D15096376BE

176
doc/nixos-install.md Normal file
View file

@ -0,0 +1,176 @@
## Preparation
Download NixOS 21.11 ISO. Burn to USB.
## Booting into install environment
Boot the ISO on PC to install.
Become root with `sudo su`
```bash
loadkeys fr
setfont sun12x22
```
Do network config if necessary, see [install guide](https://nixos.org/manual/nixos/stable/index.html#sec-installation-booting-networking)
## Make partitions
```bash
cgdisk /dev/sda
```
Recommended layout:
```
/dev/sda1 512M ef00 EFI System partition
/dev/sda2 100% 8309 Linux LUKS
```
## Setup cryptography
```bash
cryptsetup luksFormat /dev/sda2
cryptsetup open /dev/sda2 cryptlvm
```
## Create PV, VG and LVs
```bash
pvcreate /dev/mapper/cryptlvm
vgcreate NixosVG /dev/mapper/cryptlvm
lvcreate -L 8G NixosVG -n swap
lvcreate -l 100%FREE NixosVG -n root
```
## Format partitions
```bash
mkfs.fat -F 32 -n boot /dev/sda1
mkswap /dev/NixosVG/swap
mkfs.ext4 /dev/NixosVG/root
```
## Mount partitions
```bash
swapon /dev/NixosVG/swap
mount /dev/NixosVG/root /mnt
mkdir /mnt/boot
mount /dev/sda1 /mnt/boot
```
## Generate base NixOS configuration
```bash
nixos-generate-config --root /mnt
```
## Update `hardware-configuration.nix`
This section is needed:
```nix
boot.initrd.luks.devices."cryptlvm" = {
device = "/dev/disk/by-uuid/<uuid of sda2>";
allowDiscards = true;
};
```
And for the root filesystem, remember to add the `relatime` and `discard` options so that it looks like this:
```nix
fileSystems."/" =
{ device = "/dev/disk/by-uuid/<...>";
fsType = "ext4";
options = [ "relatime" "discard" ];
};
```
## Update `configuration.nix`
Just enough so that basic tasks can be done from keyboard and remotely:
- timezone
- keyboard layout
- font `sun12x22`
- vim
- user
- ssh
- ssh port in firewall
## Do the installation
```bash
nixos-install
```
## First boot
Reboot machine. Login as `root`
```bash
passwd <user>
```
If necessary, assign static IP: `ip addr add 192.168.1.40/24 dev eno1` or sth (replace ip and device appropriately)
Remotely: `ssh-copy-id <user>@<ip>`. Check SSH access is good.
## Deploy from this repo
It's time!
**Changes in this repo:**
- create node `.nix` file, site `.nix` file if neccessary, and symlink for node `.site.nix`
(create site and cluster files if necessary; use existing files of e.g.
the staging cluster as examples/templates)
- make sure values are filled in correctly
- add node to `ssh_config` with it's LAN IP, we don't have VPN at this stage
**Configuration steps on the node:**
```bash
# On node being installed
mkdir -p /var/lib/deuxfleurs/remote-unlock
cd /var/lib/deuxfleurs/remote-unlock
ssh-keygen -t ed25519 -N "" -f ./ssh_host_ed25519_key
```
**Try to deploy:**
```bash
# In nixcfg repository from your PC
./deploy.sh <cluster> <nodename>
```
Reboot.
Check remote unlocking works: `ssh -p 222 root@<ip>`
## Configure wireguard
**Create wireguard keys:**
On the node:
```bash
# On node being installed
mkdir -p /var/lib/deuxfleurs/wireguard-keys
cd /var/lib/deuxfleurs/wireguard-keys
wg genkey | tee private | wg pubkey > public
```
Get the public key, make sure it is in `cluster.nix` so that nodes know one
another. Also put it anywhere else like in your local wireguard config for
instance so that you can access the node from your PC by its wireguard address
and not only its LAN address.
Redo a deploy (`./deploy.sh <cluster> <nodename>`)
## Configure Nomad and Consul TLS
If you are bootstraping a new cluster, you need to `./genpki.sh <cluster>` to
make a TLS PKI for the Nomad+Consul cluster to work. Then redo a deploy.