deuxfleurs.fr ============= **OBSOLETION NOTICE:** We are progressively migrating our stack to NixOS, to replace Ansible. Most of the files present in this repository are outdated or obsolete, the current code for our infrastructure is at: . ## Our abstraction stack We try to build a generic abstraction stack between our different resources (CPU, RAM, disk, etc.) and our services (Chat, Storage, etc.), we develop our own tools when needed: * **[garage](https://git.deuxfleurs.fr/Deuxfleurs/garage/):** S3-compatible lightweight object store for self-hosted geo-distributed deployments (we also have a legacy glusterfs cluster) * **[diplonat](https://git.deuxfleurs.fr/Deuxfleurs/diplonat):** network automation (firewalling, upnp igd) * **[bottin](https://git.deuxfleurs.fr/Deuxfleurs/bottin):** authentication and authorization (LDAP protocol, consul backend) * **[guichet](https://git.deuxfleurs.fr/Deuxfleurs/guichet):** a dashboard for our users and administrators * **ansible:** physical node configuration * **nomad:** schedule containers and handle their lifecycle * **consul:** distributed key value store + lock + service discovery * **stolon + postgresql:** distributed relational database * **docker:** package, distribute and isolate applications Some services we provide: * **Websites:** garage (static) + fediverse blog (plume) * **Chat:** Synapse + Element Web (Matrix protocol) * **Email:** Postfix SMTP + Dovecot IMAP + opendkim DKIM + Sogo webmail (legacy) | Alps webmail (experimental) * **Storage:** Seafile (legacy) | Nextcloud (experimental) * **Visio:** Jitsi As a generic abstraction is provided, deploying new services should be easy. ## I am lost, how this repo works? To ease the development, we make the choice of a fully integrated environment 1. `os` the base os for the cluster 1. `build`: where you will build our OS image based on Debian that you will install on your server 2. `config`: our Ansible recipes to configure and update your freshly installed server 2. `apps` apps we deploy on the cluster 1. `build`: our Docker files to build immutable images of our applications 2. `integration`: Our Docker compose files to test locally how our built images interact together 3. `config`: Files containing application configurations to be deployed on Consul Key Value Store 4. `deployment`: Files containing application definitions to be deployed on Nomad Scheduler 3. `op_guide`: Guides to explain you operations you can do cluster wide (like configuring postgres) ## Start hacking ### Deploying/Updating new services is done from your machine *The following instructions are provided for ops that already have access to the servers (meaning: their SSH public key is known by the cluster).* Deploy Nomad on your machine: ```bash export NOMAD_VER=1.0.1 wget https://releases.hashicorp.com/nomad/${NOMAD_VER}/nomad_${NOMAD_VER}_linux_amd64.zip unzip nomad_${NOMAD_VER}_linux_amd64.zip sudo mv nomad /usr/local/bin rm nomad_${NOMAD_VER}_linux_amd64.zip ``` Deploy Consul on your machine: ```bash export CONSUL_VER=1.9.0 wget https://releases.hashicorp.com/consul/${CONSUL_VER}/consul_${CONSUL_VER}_linux_amd64.zip unzip consul_${CONSUL_VER}_linux_amd64.zip sudo mv consul /usr/local/bin rm consul_${CONSUL_VER}_linux_amd64.zip ``` Create an alias (and put it in your `.bashrc`) to bind APIs on your machine: ``` alias bind_df="ssh \ -p110 \ -N \ -L 1389:bottin2.service.2.cluster.deuxfleurs.fr:389 \ -L 4646:127.0.0.1:4646 \ -L 5432:psql-proxy.service.2.cluster.deuxfleurs.fr:5432 \ -L 8082:traefik-admin.service.2.cluster.deuxfleurs.fr:8082 \ -L 8500:127.0.0.1:8500 \ " ``` and run: bind_df Adrien uses `.ssh/config` configuration instead. I works basically the same. Here it goes: ``` # in ~/.ssh/config Host deuxfleurs User adrien Hostname deuxfleurs.fr # If you don't use the default ~/.ssh/id_rsa to connect to Deuxfleurs IdentityFile PubKeyAuthentication yes ForwardAgent No LocalForward 1389 bottin2.service.2.cluster.deuxfleurs.fr:389 LocalForward 4646 127.0.0.1:4646 LocalForward 5432 psql-proxy.service.2.cluster.deuxfleurs.fr:5432 LocalForward 8082 traefik-admin.service.2.cluster.deuxfleurs.fr:8082 LocalForward 8500 127.0.0.1:8500 ``` Now, to connect, do the following: ssh deuxfleurs -N ## Test cluster Configured machines available for testing are listed in the **[`test_cluster` Ansible inventory](./os/config/test_cluster.inventory.yml)**.