reorganize some things

This commit is contained in:
Alex 2022-12-24 22:59:37 +01:00
parent 4b527c4db8
commit 8d17a07c9b
No known key found for this signature in database
GPG key ID: 09EC5284AA804D3C
30 changed files with 43 additions and 34 deletions

View file

@ -2,11 +2,35 @@
This repository contains code to run Deuxfleur's infrastructure on NixOS.
It sets up the following:
## Our abstraction stack
- A Wireguard mesh between all nodes
- Consul, with TLS
- Nomad, with TLS
We try to build a generic abstraction stack between our different resources (CPU, RAM, disk, etc.) and our services (Chat, Storage, etc.), we develop our own tools when needed.
Our first abstraction level is the NixOS level, which installs a bunch of standard components:
* **Wireguard:** provides encrypted communication between remote nodes
* **Nomad:** schedule containers and handle their lifecycle
* **Consul:** distributed key value store + lock + service discovery
* **Docker:** package, distribute and isolate applications
Then, inside our Nomad+Consul orchestrator, we deploy a number of base services:
* **[Garage](https://git.deuxfleurs.fr/Deuxfleurs/garage/):** S3-compatible lightweight object store for self-hosted geo-distributed deployments (we also have a legacy glusterfs cluster)
* **[DiploNAT](https://git.deuxfleurs.fr/Deuxfleurs/diplonat):** network automation (firewalling, upnp igd)
* **[Bottin](https://git.deuxfleurs.fr/Deuxfleurs/bottin):** authentication and authorization (LDAP protocol, consul backend)
* **[Guichet](https://git.deuxfleurs.fr/Deuxfleurs/guichet):** a dashboard for our users and administrators
* **Stolon + PostgreSQL:** distributed relational database
* **Prometheus + Grafana:** monitoring
Some services we provide based on this abstraction:
* **Websites:** Garage (static) + fediverse blog (Plume)
* **Chat:** Synapse + Element Web (Matrix protocol)
* **Email:** Postfix SMTP + Dovecot IMAP + opendkim DKIM + Sogo webmail | Alps webmail (experimental)
* **Visioconference:** Jitsi
* **Collaboration:** CryptPad
As a generic abstraction is provided, deploying new services should be easy.
## How to use this?
@ -16,11 +40,7 @@ See the following documentation topics:
- [How to add new nodes to a cluster (rapid overview)](doc/adding-nodes.md)
- [Architecture of this repo, how the scripts work](doc/architecture.md)
- [List of TCP and UDP ports used by services](doc/ports)
Additionnal documentation topics:
- [Succint guide for NixOS installation with LUKX full disk encryption](doc/nixos-install-luks.md) (we don't do that in practice on our servers)
- [Example `hardware-config.nix` for a full disk encryption scenario](doc/example-hardware-configuration.nix)
- [Why not Ansible?](doc/why-not-ansible.md)

1
cluster/prod/app/secretmgr Symbolic link
View file

@ -0,0 +1 @@
../../../secretmgr/secretmgr

View file

@ -1 +0,0 @@
../../../secretmgr/secretmgr.py

View file

@ -1 +0,0 @@
../../../secretmgr/shell.nix

View file

@ -0,0 +1 @@
../../../secretmgr/secretmgr

View file

@ -1 +0,0 @@
../../../secretmgr/secretmgr.py

View file

@ -1 +0,0 @@
../../../secretmgr/shell.nix

View file

@ -1,6 +1,6 @@
## Preparation
Download NixOS 21.11 ISO. Burn to USB.
Download NixOS. Burn to USB.
## Booting into install environment
@ -120,7 +120,7 @@ Remotely: `ssh-copy-id <user>@<ip>`. Check SSH access is good.
## Deploy from this repo
See [this documentation](quick-start.md).
See the documentation in `/doc` in this repo. The old procedure described here is partly obsolete.
## Old guide
@ -154,7 +154,7 @@ Reboot.
Check remote unlocking works: `ssh -p 222 root@<ip>`
## Configure wireguard
### Configure wireguard
```bash
# On node being installed
@ -172,11 +172,11 @@ Redo a deploy (`./deploy.sh <cluster> <nodename>`)
Check VPN works. Change IP in `ssh_config` to use VPN IP instead of LAN IP (required for deploy when away from home).
## Commit changes to `nixcfg` repo
### Commit changes to `nixcfg` repo
This is a good point to commit your new/modified `.nix` files.
## Configure Nomad and Consul TLS
### Configure Nomad and Consul TLS
If you are bootstraping a new cluster, you need to `./genpki.sh <cluster>` to
make a TLS PKI for the Nomad+Consul cluster to work. Then redo a deploy.

View file

@ -1,4 +1,5 @@
#!/usr/bin/env python3
#!/usr/bin/env nix-shell
#!nix-shell -i python3 -p "python3.withPackages(ps: [ ps.pip ps.consul ps.ldap ps.passlib ps.requests ps.six ])"
# DEPENDENCY: python-consul
import consul

View file

@ -1,15 +0,0 @@
{
pkgs ? import <nixpkgs> {}
}:
with pkgs; mkShell {
nativeBuildInputs = [
nomad
docker-compose
python39Packages.pip
python39Packages.ldap
python39Packages.consul
python39Packages.passlib
];
}

View file

@ -1,6 +1,11 @@
#!/usr/bin/env bash
CMDFILE="$1"
if [ -z "$CMDFILE" ] || [ ! -f "$CMDFILE" ]; then
echo "sshtool is not meant to be called on its own."
echo "See scripts that use it (e.g. deploy_nixos) for usage examples."
exit 1
fi
shift 1
cd $(dirname $CMDFILE)