reorganize some things

This commit is contained in:
Alex 2022-12-24 22:59:37 +01:00
parent 4b527c4db8
commit 8d17a07c9b
No known key found for this signature in database
GPG key ID: 09EC5284AA804D3C
30 changed files with 43 additions and 34 deletions

View file

@ -2,11 +2,35 @@
This repository contains code to run Deuxfleur's infrastructure on NixOS. This repository contains code to run Deuxfleur's infrastructure on NixOS.
It sets up the following: ## Our abstraction stack
- A Wireguard mesh between all nodes We try to build a generic abstraction stack between our different resources (CPU, RAM, disk, etc.) and our services (Chat, Storage, etc.), we develop our own tools when needed.
- Consul, with TLS
- Nomad, with TLS Our first abstraction level is the NixOS level, which installs a bunch of standard components:
* **Wireguard:** provides encrypted communication between remote nodes
* **Nomad:** schedule containers and handle their lifecycle
* **Consul:** distributed key value store + lock + service discovery
* **Docker:** package, distribute and isolate applications
Then, inside our Nomad+Consul orchestrator, we deploy a number of base services:
* **[Garage](https://git.deuxfleurs.fr/Deuxfleurs/garage/):** S3-compatible lightweight object store for self-hosted geo-distributed deployments (we also have a legacy glusterfs cluster)
* **[DiploNAT](https://git.deuxfleurs.fr/Deuxfleurs/diplonat):** network automation (firewalling, upnp igd)
* **[Bottin](https://git.deuxfleurs.fr/Deuxfleurs/bottin):** authentication and authorization (LDAP protocol, consul backend)
* **[Guichet](https://git.deuxfleurs.fr/Deuxfleurs/guichet):** a dashboard for our users and administrators
* **Stolon + PostgreSQL:** distributed relational database
* **Prometheus + Grafana:** monitoring
Some services we provide based on this abstraction:
* **Websites:** Garage (static) + fediverse blog (Plume)
* **Chat:** Synapse + Element Web (Matrix protocol)
* **Email:** Postfix SMTP + Dovecot IMAP + opendkim DKIM + Sogo webmail | Alps webmail (experimental)
* **Visioconference:** Jitsi
* **Collaboration:** CryptPad
As a generic abstraction is provided, deploying new services should be easy.
## How to use this? ## How to use this?
@ -16,11 +40,7 @@ See the following documentation topics:
- [How to add new nodes to a cluster (rapid overview)](doc/adding-nodes.md) - [How to add new nodes to a cluster (rapid overview)](doc/adding-nodes.md)
- [Architecture of this repo, how the scripts work](doc/architecture.md) - [Architecture of this repo, how the scripts work](doc/architecture.md)
- [List of TCP and UDP ports used by services](doc/ports) - [List of TCP and UDP ports used by services](doc/ports)
Additionnal documentation topics:
- [Succint guide for NixOS installation with LUKX full disk encryption](doc/nixos-install-luks.md) (we don't do that in practice on our servers)
- [Example `hardware-config.nix` for a full disk encryption scenario](doc/example-hardware-configuration.nix)
- [Why not Ansible?](doc/why-not-ansible.md) - [Why not Ansible?](doc/why-not-ansible.md)

1
cluster/prod/app/secretmgr Symbolic link
View file

@ -0,0 +1 @@
../../../secretmgr/secretmgr

View file

@ -1 +0,0 @@
../../../secretmgr/secretmgr.py

View file

@ -1 +0,0 @@
../../../secretmgr/shell.nix

View file

@ -0,0 +1 @@
../../../secretmgr/secretmgr

View file

@ -1 +0,0 @@
../../../secretmgr/secretmgr.py

View file

@ -1 +0,0 @@
../../../secretmgr/shell.nix

View file

@ -1,6 +1,6 @@
## Preparation ## Preparation
Download NixOS 21.11 ISO. Burn to USB. Download NixOS. Burn to USB.
## Booting into install environment ## Booting into install environment
@ -120,7 +120,7 @@ Remotely: `ssh-copy-id <user>@<ip>`. Check SSH access is good.
## Deploy from this repo ## Deploy from this repo
See [this documentation](quick-start.md). See the documentation in `/doc` in this repo. The old procedure described here is partly obsolete.
## Old guide ## Old guide
@ -154,7 +154,7 @@ Reboot.
Check remote unlocking works: `ssh -p 222 root@<ip>` Check remote unlocking works: `ssh -p 222 root@<ip>`
## Configure wireguard ### Configure wireguard
```bash ```bash
# On node being installed # On node being installed
@ -172,11 +172,11 @@ Redo a deploy (`./deploy.sh <cluster> <nodename>`)
Check VPN works. Change IP in `ssh_config` to use VPN IP instead of LAN IP (required for deploy when away from home). Check VPN works. Change IP in `ssh_config` to use VPN IP instead of LAN IP (required for deploy when away from home).
## Commit changes to `nixcfg` repo ### Commit changes to `nixcfg` repo
This is a good point to commit your new/modified `.nix` files. This is a good point to commit your new/modified `.nix` files.
## Configure Nomad and Consul TLS ### Configure Nomad and Consul TLS
If you are bootstraping a new cluster, you need to `./genpki.sh <cluster>` to If you are bootstraping a new cluster, you need to `./genpki.sh <cluster>` to
make a TLS PKI for the Nomad+Consul cluster to work. Then redo a deploy. make a TLS PKI for the Nomad+Consul cluster to work. Then redo a deploy.

View file

@ -1,4 +1,5 @@
#!/usr/bin/env python3 #!/usr/bin/env nix-shell
#!nix-shell -i python3 -p "python3.withPackages(ps: [ ps.pip ps.consul ps.ldap ps.passlib ps.requests ps.six ])"
# DEPENDENCY: python-consul # DEPENDENCY: python-consul
import consul import consul

View file

@ -1,15 +0,0 @@
{
pkgs ? import <nixpkgs> {}
}:
with pkgs; mkShell {
nativeBuildInputs = [
nomad
docker-compose
python39Packages.pip
python39Packages.ldap
python39Packages.consul
python39Packages.passlib
];
}

View file

@ -1,6 +1,11 @@
#!/usr/bin/env bash #!/usr/bin/env bash
CMDFILE="$1" CMDFILE="$1"
if [ -z "$CMDFILE" ] || [ ! -f "$CMDFILE" ]; then
echo "sshtool is not meant to be called on its own."
echo "See scripts that use it (e.g. deploy_nixos) for usage examples."
exit 1
fi
shift 1 shift 1
cd $(dirname $CMDFILE) cd $(dirname $CMDFILE)