prod: update nomad to 1.6

Reviewed-on: Deuxfleurs/nixcfg#23

README.md

@@ -1,160 +1,55 @@
 # Deuxfleurs on NixOS!
 
-This repository contains code to run Deuxfleur's infrastructure on NixOS.
+This repository contains code to run Deuxfleurs' infrastructure on NixOS.
 
-It sets up the following:
+## Our abstraction stack
 
-- A Wireguard mesh between all nodes
-- Consul, with TLS
-- Nomad, with TLS
+We try to build a generic abstraction stack between our different resources (CPU, RAM, disk, etc.) and our services (Chat, Storage, etc.), we develop our own tools when needed.
 
-## Configuring the OS
+Our first abstraction level is the NixOS level, which installs a bunch of standard components:
 
-This repo contains a bunch of scripts to configure NixOS on all cluster nodes.
-Most scripts are invoked with the following syntax:
+  * **Wireguard:** provides encrypted communication between remote nodes
+  * **Nomad:** schedule containers and handle their lifecycle
+  * **Consul:** distributed key value store + lock + service discovery
+  * **Docker:** package, distribute and isolate applications
   
-- for scripts that generate secrets: `./gen_<something> <cluster_name>` to generate the secrets to be used on cluster `<cluster_name>`
-- for deployment scripts:
-  - `./deploy_<something> <cluster_name>` to run the deployment script on all nodes of the cluster `<cluster_name>`
-  - `./deploy_<something> <cluster_name> <node1> <node2> ...` to run the deployment script only on nodes `node1, node2, ...` of cluster `<cluster_name>`.
+Then, inside our Nomad+Consul orchestrator, we deploy a number of base services:
 
-All deployment scripts can use the following parameters passed as environment variables:
+  * Data management
+    * **[Garage](https://git.deuxfleurs.fr/Deuxfleurs/garage/):** S3-compatible lightweight object store for self-hosted geo-distributed deployments
+    * **Stolon + PostgreSQL:** distributed relational database
+  * Network Control Plane
+    * **[DiploNAT](https://git.deuxfleurs.fr/Deuxfleurs/diplonat):** - network automation (firewalling, upnp igd)
+    * **[D53](https://git.deuxfleurs.fr/lx/d53)** - update DNS entries (A and AAAA) dynamically based on Nomad service scheduling and local node info
+    * **[Tricot](https://git.deuxfleurs.fr/Deuxfleurs/tricot)** - a dynamic reverse proxy for nomad+consul inspired by traefik
+    * **[wgautomesh](https://git.deuxfleurs.fr/Deuxfleurs/wgautomesh)** - a dynamic wireguard mesh configurator
+  * User Management
+    * **[Bottin](https://git.deuxfleurs.fr/Deuxfleurs/bottin):** authentication and authorization (LDAP protocol, consul backend)
+    * **[Guichet](https://git.deuxfleurs.fr/Deuxfleurs/guichet):** a dashboard for our users and administrators7
+  * Observability
+    * **Prometheus + Grafana:** monitoring
 
-- `SUDO_PASS`: optionnally, the password for `sudo` on cluster nodes. If not set, it will be asked at the begninning.
-- `SSH_USER`: optionnally, the user to try to login using SSH. If not set, the username from your local machine will be used.
+Some services we provide based on this abstraction:
 
-### Assumptions (how to setup your environment)
+  * **Websites:** Garage (static) + fediverse blog (Plume)
+  * **Chat:** Synapse + Element Web (Matrix protocol)
+  * **Email:** Postfix SMTP + Dovecot IMAP + opendkim DKIM + Sogo webmail  | Alps webmail (experimental)
+    - **[Aerogramme](https://git.deuxfleurs.fr/Deuxfleurs/aerogramme/):** an encrypted IMAP server
+  * **Visioconference:** Jitsi
+  * **Collaboration:** CryptPad
 
-- you have an SSH access to all of your cluster nodes (listed in `cluster/<cluster_name>/ssh_config`)
+As a generic abstraction is provided, deploying new services should be easy.
 
-- your account is in group `wheel` and you know its password (you need it to become root using `sudo`);
-  the password is the same on all cluster nodes (see below for password management tools)
+## How to use this?
 
-- you have a clone of the secrets repository in your `pass` password store, for instance at `~/.password-store/deuxfleurs`
-  (scripts in this repo will read and write all secrets in `pass` under `deuxfleurs/cluster/<cluster_name>/`)
+See the following documentation topics:
 
-### Deploying the NixOS configuration
+- [Quick start and onboarding for new administrators](doc/onboarding.md)
+- [How to add new nodes to a cluster (rapid overview)](doc/adding-nodes.md)
+- [Architecture of this repo, how the scripts work](doc/architecture.md)
+- [List of TCP and UDP ports used by services](doc/ports)
+- [Why not Ansible?](doc/why-not-ansible.md)
 
-The NixOS configuration makes use of a certain number of files:
+## Got personal services in addition to Deuxfleurs at home?
 
-- files in `nix/` that are the same for all deployments on all clusters
-- the file `cluster/<cluster_name>/cluster.nix`, a Nix configuration file that is specific to the cluster but is copied the same on all cluster nodes
-- files in `cluster/<cluster_name>/site/`, which are specific to the various sites on which Nix nodes are deployed
-- files in `cluster/<cluster_name>/node/` which are specific to each node
-
-To deploy the NixOS configuration on the cluster, simply do:
-
-```
-./deploy_nixos <cluster_name>
-```
-
-or to deploy only on a single node:
-
-```
-./deploy_nixos <cluster_name> <node_name>
-```
-
-To upgrade NixOS, use the `./upgrade_nixos` script instead (it has the same syntax).
-
-**When adding a node to the cluster:** just do `./deploy_nixos <cluster_name> <name_of_new_node>`
-
-### Deploying Wesher
-
-We use Wesher to provide an encrypted overlay network between nodes in the cluster.
-This is usefull in particular for securing services that are not able to do mTLS,
-but as a security-in-depth measure, we make all traffic go through Wesher even when
-TLS is done correctly. It is thus mandatory to have a working Wesher installation
-in the cluster for it to run correctly.
-
-First, if no Wesher shared secret key has been generated for this cluster yet,
-generate it with:
-
-```
-./gen_wesher_key <cluster_name>
-```
-
-This key will be stored in `pass`, so you must have a working `pass` installation
-for this script to run correctly.
-
-Then, deploy the key on all nodes with:
-
-```
-./deploy_wesher_key <cluster_name>
-```
-
-This should be done after `./deploy_nixos` has run successfully on all nodes.
-You should now have a working Wesher network between all your nodes!
-
-**When adding a node to the cluster:** just do `./deploy_wesher_key <cluster_name> <name_of_new_node>`
-
-### Generating and deploying a PKI for Consul and Nomad
-
-This is very similar to how we do for Wesher.
-
-First, if the PKI has not yet been created, create it with:
-
-```
-./gen_pki <cluster_name>
-```
-
-Then, deploy the PKI on all nodes with:
-
-```
-./deploy_pki <cluster_name>
-```
-
-**When adding a node to the cluster:** just do `./deploy_pki <cluster_name> <name_of_new_node>`
-
-### Adding administrators and password management
-
-Adminstrators are defined in the `cluster.nix` file for each cluster (they could also be defined in the site-specific Nix files if necessary).
-This is where their public SSH keys for remote access are put.
-
-Administrators will also need passwords to administrate the cluster, as we are not using passwordless sudo.
-To set the password for a new administrator, they must have a working `pass` installation as specified above.
-They must then run:
-
-```
-./passwd <cluster_name> <user_name>
-```
-
-to set their password in the `pass` database (the password is hashed, so other administrators cannot learn their password even if they have access to the `pass` db).
-
-Then, an administrator that already has root access must run the following (after syncing the `pass` db) to set the password correctly on all cluster nodes:
-
-```
-./deploy_passwords <cluster_name>
-```
-
-## Deploying stuff on Nomad
-
-### Connecting to Nomad
-
-Connect using SSH to one of the cluster nodes, forwarding port 14646 to port 4646 on localhost, and port 8501 to port 8501 on localhost.
-
-You can for instance use an entry in your `~/.ssh/config` that looks like this:
-
-```
-Host caribou
-	HostName 2a01:e0a:c:a720::23
-	LocalForward 14646 127.0.0.1:4646
-	LocalForward 8501 127.0.0.1:8501
-```
-
-Then, in a separate window, launch `./tlsproxy <cluster_name>`: this will
-launch `socat` proxies that strip the TLS layer and allow you to simply access
-Nomad and Consul on the regular, unencrypted URLs: `http://localhost:4646` for
-Nomad and `http://localhost:8500` for Consul.  Keep this terminal window for as
-long as you need to access Nomad and Consul on the cluster.
-
-### Launching services
-
-Stuff should be started in this order:
-
-- `app/core`
-- `app/frontend`
-- `app/garage-staging`
-
-At this point, we are able to have a systemd service called `mountgarage` that mounts Garage buckets in `/mnt/garage-staging`. This is used by the following services that can be launched afterwards:
-
-- `app/im`
+Go check [`cluster/prod/register_external_services.sh`](./cluster/prod/register_external_services.sh). In bash, we register a redirect from Tricot to your own services or your personal reverse proxy.

app/dummy/deploy/.gitignore

@@ -1 +0,0 @@
-dummy-volume.hcl

app/dummy/deploy/dummy-nginx.hcl

@@ -1,35 +0,0 @@
-job "dummy-nginx" {
-	datacenters = ["neptune"]
-	type = "service"
-	
-	group "nginx" {
-		count = 1
-
-		network {
-			port "http" {
-				to = 80
-			}
-		}
-
-		task "nginx" {
-			driver = "docker"
-			config {
-				image = "nginx"
-				ports = [ "http" ]
-			}
-		}
-
-		service {
-			port = "http"
-			tags = [
-				"tricot home.adnab.me 100",
-			]
-			check {
-				type = "http"
-				path = "/"
-				interval = "10s"
-				timeout = "2s"
-			}
-		}
-	}
-}

app/frontend/deploy/frontend-tricot.hcl

@@ -1,82 +0,0 @@
-job "frontend" {
-  datacenters = ["neptune"]
-  type = "service"
-  priority = 90
-
-  group "tricot" {
-    network {
-      port "http_port" { static = 80 }
-      port "https_port" { static = 443 }
-    }
-
-    task "server" {
-      driver = "docker"
-
-      config {
-        image = "lxpz/amd64_tricot:36"
-        network_mode = "host"
-        readonly_rootfs = true
-        ports = [ "http_port", "https_port" ]
-        volumes = [
-          "secrets:/etc/tricot",
-        ]
-      }
-
-      resources {
-        cpu = 2000
-        memory = 200
-      }
-
-      restart {
-        interval = "30m"
-        attempts = 2
-        delay    = "15s"
-        mode     = "delay"
-      }
-
-      template {
-        data = "{{ key \"secrets/consul/consul-ca.crt\" }}"
-        destination = "secrets/consul-ca.crt"
-      }
-
-      template {
-        data = "{{ key \"secrets/consul/consul-client.crt\" }}"
-        destination = "secrets/consul-client.crt"
-      }
-
-      template {
-        data = "{{ key \"secrets/consul/consul-client.key\" }}"
-        destination = "secrets/consul-client.key"
-      }
-
-      template {
-        data = <<EOH
-TRICOT_NODE_NAME={{ env "attr.unique.consul.name" }}
-TRICOT_LETSENCRYPT_EMAIL=alex@adnab.me
-TRICOT_ENABLE_COMPRESSION=true
-TRICOT_CONSUL_HOST=https://localhost:8501
-TRICOT_CONSUL_CA_CERT=/etc/tricot/consul-ca.crt
-TRICOT_CONSUL_CLIENT_CERT=/etc/tricot/consul-client.crt
-TRICOT_CONSUL_CLIENT_KEY=/etc/tricot/consul-client.key
-RUST_LOG=tricot=debug
-EOH
-        destination = "secrets/env"
-        env = true
-      }
-
-      service {
-        name = "tricot-http"
-        port = "http_port"
-        tags = [ "(diplonat (tcp_port 80))" ]
-        address_mode = "host"
-      }
-
-      service {
-        name = "tricot-https"
-        port = "https_port"
-        tags = [ "(diplonat (tcp_port 443))" ]
-        address_mode = "host"
-      }
-    }
-  }
-}

app/garage-staging/config/garage.toml

@@ -1,27 +0,0 @@
-block_size = 1048576
-
-metadata_dir = "/meta"
-data_dir = "/data"
-
-replication_mode = "3"
-
-rpc_bind_addr = "0.0.0.0:3991"
-rpc_secret = "{{ key "secrets/garage-staging/rpc_secret" | trimSpace }}"
-
-consul_host = "localhost:8500"
-consul_service_name = "garage-staging-rpc-self-advertised"
-
-bootstrap_peers = []
-
-[s3_api]
-s3_region = "garage-staging"
-api_bind_addr = "0.0.0.0:3990"
-
-[s3_web]
-bind_addr = "0.0.0.0:3992"
-root_domain = ".garage-staging-web.home.adnab.me"
-index = "index.html"
-
-[admin]
-api_bind_addr = "0.0.0.0:3909"
-trace_sink = "http://{{ env "attr.unique.network.ip-address" }}:4317"

app/garage-staging/deploy/garage.hcl

@@ -1,139 +0,0 @@
-job "garage-staging" {
-  type = "system"
-  #datacenters = [ "neptune", "pluton" ]
-  datacenters = [ "neptune" ]
-
-  priority = 80
-
-  constraint {
-    attribute = "${attr.cpu.arch}"
-    value     = "amd64"
-  }
-
-  group "garage-staging" {
-    network {
-      port "s3" { static = 3990 }
-      port "rpc" { static = 3991 }
-      port "web" { static = 3992 }
-      port "admin" { static = 3909 }
-    }
-
-    update {
-      max_parallel = 1
-      min_healthy_time = "30s"
-      healthy_deadline = "5m"
-    }
-
-    task "server" {
-      driver = "docker"
-
-      config {
-        image = "dxflrs/amd64_garage:v0.7.0"
-        command = "/garage"
-        args = [ "server" ]
-        network_mode = "host"
-        volumes = [
-          "/mnt/storage/garage-staging/data:/data",
-          "/mnt/ssd/garage-staging/meta:/meta",
-          "secrets/garage.toml:/etc/garage.toml",
-        ]
-      }
-
-      template {
-        data = file("../config/garage.toml")
-        destination = "secrets/garage.toml"
-      }
-
-      resources {
-        memory = 1000
-        cpu = 1000
-      }
-
-      kill_signal = "SIGINT"
-      kill_timeout = "20s"
-
-      service {
-        tags = [
-          "garage-staging-api",
-          "tricot garage-staging.home.adnab.me",
-          "tricot-add-header Access-Control-Allow-Origin *",
-        ]
-        port = 3990
-        address_mode = "driver"
-        name = "garage-staging-api"
-        check {
-          type = "tcp"
-          port = 3990
-          address_mode = "driver"
-          interval = "60s"
-          timeout = "5s"
-          check_restart {
-            limit = 3
-            grace = "90s"
-            ignore_warnings = false
-          }
-        }
-      }
-
-      service {
-        tags = ["garage-staging-rpc"]
-        port = 3991
-        address_mode = "driver"
-        name = "garage-staging-rpc"
-        check {
-          type = "tcp"
-          port = 3991
-          address_mode = "driver"
-          interval = "60s"
-          timeout = "5s"
-          check_restart {
-            limit = 3
-            grace = "90s"
-            ignore_warnings = false
-          }
-        }
-      }
-
-      service {
-        tags = [
-            "garage-staging-web",
-            "tricot *.garage-staging-web.home.adnab.me",
-            "tricot matrix.home.adnab.me/.well-known/matrix/server",
-            "tricot rust-docs",
-            "tricot-add-header Access-Control-Allow-Origin *",
-        ]
-        port = 3992
-        address_mode = "driver"
-        name = "garage-staging-web"
-        check {
-          type = "tcp"
-          port = 3992
-          address_mode = "driver"
-          interval = "60s"
-          timeout = "5s"
-          check_restart {
-            limit = 3
-            grace = "90s"
-            ignore_warnings = false
-          }
-        }
-      }
-
-      service {
-        tags = [
-          "garage-staging-admin",
-        ]
-        port = 3909
-        address_mode = "driver"
-        name = "garage-staging-admin"
-      }
-
-      restart { 
-        interval = "30m"  
-        attempts = 10  
-        delay    = "15s"  
-        mode     = "delay"
-      }
-    }
-  }
-}

app/garage-staging/secrets/garage-staging/rpc_secret

@@ -1 +0,0 @@
-CMD_ONCE openssl rand -hex 32

app/im/build/matrix-synapse/matrix-s3-async-sqlite

@@ -1,13 +0,0 @@
-#!/bin/bash
-
-cat > database.yaml <<EOF
-sqlite:
-  database: $SYNAPSE_SQLITE_DB
-EOF
-
-while true; do
-  /root/matrix-env/bin/s3_media_upload update-db 0d
-  /root/matrix-env/bin/s3_media_upload --no-progress check-deleted $SYNAPSE_MEDIA_STORE
-  /root/matrix-env/bin/s3_media_upload --no-progress upload $SYNAPSE_MEDIA_STORE $SYNAPSE_MEDIA_S3_BUCKET --delete --endpoint-url $S3_ENDPOINT
-  sleep 600
-done

app/im/secrets/synapse/form_secret

@@ -1 +0,0 @@
-USER Synapse's `form_secret` configuration parameter

app/im/secrets/synapse/macaroon_secret_key

@@ -1 +0,0 @@
-USER Synapse's `macaroon_secret_key` parameter

app/im/secrets/synapse/registration_shared_secret

@@ -1 +0,0 @@
-USER Synapse's `registration_shared_secret` parameter

app/im/secrets/synapse/s3_access_key

@@ -1 +0,0 @@
-USER S3 access key ID for database storage

app/im/secrets/synapse/s3_secret_key

@@ -1 +0,0 @@
-USER S3 secret key for database storage

app/im/secrets/synapse/signing_key

@@ -1 +0,0 @@
-USER Signing key for messages

app/secretmgr.py

@@ -1 +0,0 @@
-../../infrastructure/app/secretmgr.py

cluster/prod/app/backup/build/backup-consul/Dockerfile

@@ -0,0 +1,28 @@
+FROM golang:buster as builder
+
+WORKDIR /root
+RUN git clone https://filippo.io/age && cd age/cmd/age && go build -o age .
+
+FROM amd64/debian:buster
+
+COPY --from=builder /root/age/cmd/age/age /usr/local/bin/age
+
+RUN apt-get update && \
+	apt-get -qq -y full-upgrade && \
+	apt-get install -y rsync wget openssh-client unzip && \
+	apt-get clean && \
+	rm -f /var/lib/apt/lists/*_*
+
+RUN mkdir -p /root/.ssh
+WORKDIR /root
+
+RUN wget https://releases.hashicorp.com/consul/1.8.5/consul_1.8.5_linux_amd64.zip && \
+	unzip consul_1.8.5_linux_amd64.zip && \
+	chmod +x consul && \
+	mv consul /usr/local/bin && \
+	rm consul_1.8.5_linux_amd64.zip
+
+COPY do_backup.sh /root/do_backup.sh
+
+CMD "/root/do_backup.sh"
+

cluster/prod/app/backup/build/backup-consul/do_backup.sh

@@ -0,0 +1,20 @@
+#!/bin/sh
+
+set -x -e
+
+cd /root
+
+chmod 0600 .ssh/id_ed25519
+
+cat > .ssh/config <<EOF
+Host backuphost
+	HostName $TARGET_SSH_HOST
+	Port $TARGET_SSH_PORT
+	User $TARGET_SSH_USER
+EOF
+
+consul kv export | \
+	gzip | \
+	age -r "$(cat /root/.ssh/id_ed25519.pub)" | \
+	ssh backuphost "cat > $TARGET_SSH_DIR/consul/$(date --iso-8601=minute)_consul_kv_export.gz.age"
+

cluster/prod/app/backup/build/backup-garage/Dockerfile

@@ -0,0 +1,7 @@
+FROM alpine:3.17
+
+RUN apk add rclone curl bash jq
+
+COPY do-backup.sh /do-backup.sh
+
+CMD bash /do-backup.sh

cluster/prod/app/backup/build/backup-garage/do-backup.sh

@@ -0,0 +1,83 @@
+#!/usr/bin/env bash
+
+# DESCRIPTION:
+# 		Script to backup all buckets on a Garage cluster using rclone.
+#
+# REQUIREMENTS:
+# 		An access key for the backup script must be created in Garage beforehand.
+# 		This script will use the Garage administration API to grant read access
+# 		to this key on all buckets.
+#
+# 		A rclone configuration file is expected to be located at `/etc/secrets/rclone.conf`,
+# 		which contains credentials to the following two remotes:
+# 			garage:		the Garage server, for read access (using the backup access key)
+# 			backup:		the backup location
+#
+# DEPENDENCIES: (see Dockerfile)
+# 		curl
+# 		jq
+# 		rclone
+#
+# PARAMETERS (environmenet variables)
+#       $GARAGE_ADMIN_API_URL   => Garage administration API URL (e.g. http://localhost:3903)
+#       $GARAGE_ADMIN_TOKEN     => Garage administration access token
+#       $GARAGE_ACCESS_KEY      => Garage access key ID
+#       $TARGET_BACKUP_DIR      => Folder on the backup remote where to store buckets
+
+if [ -z "$GARAGE_ACCESS_KEY" -o -z "$GARAGE_ADMIN_TOKEN" -o -z "$GARAGE_ADMIN_API_URL" ]; then
+        echo "Missing parameters"
+fi
+
+# copy potentially immutable file to a mutable location,
+# otherwise rclone complains
+mkdir -p /root/.config/rclone
+cp /etc/secrets/rclone.conf /root/.config/rclone/rclone.conf
+
+function gcurl {
+        curl -s -H "Authorization: Bearer $GARAGE_ADMIN_TOKEN" $@
+}
+
+BUCKETS=$(gcurl "$GARAGE_ADMIN_API_URL/v0/bucket" | jq -r '.[].id')
+
+mkdir -p /tmp/buckets-info
+
+for BUCKET in $BUCKETS; do
+        echo "==== BUCKET $BUCKET ===="
+
+        gcurl "http://localhost:3903/v0/bucket?id=$BUCKET" > "/tmp/buckets-info/$BUCKET.json"
+		rclone copy "/tmp/buckets-info/$BUCKET.json" "backup:$TARGET_BACKUP_DIR/" 2>&1
+
+        ALIASES=$(jq -r '.globalAliases[]' < "/tmp/buckets-info/$BUCKET.json")
+        echo "(aka. $ALIASES)"
+
+        case $ALIASES in
+                *backup*)
+                        echo "Skipping $BUCKET (not doing backup of backup)"
+                        ;;
+                *cache*)
+                        echo "Skipping $BUCKET (not doing backup of cache)"
+                        ;;
+                *)
+                        echo "Backing up $BUCKET"
+
+                        gcurl -X POST -H "Content-Type: application/json"  --data @- "http://localhost:3903/v0/bucket/allow" >/dev/null <<EOF
+                                {
+                                        "bucketId": "$BUCKET",
+                                        "accessKeyId": "$GARAGE_ACCESS_KEY",
+                                        "permissions": {"read": true}
+                                }
+EOF
+
+                        rclone sync \
+                        		--transfers 32 \
+                                --fast-list \
+                                --stats-one-line \
+                                --stats 10s \
+                                --stats-log-level NOTICE \
+								"garage:$BUCKET" "backup:$TARGET_BACKUP_DIR/$BUCKET" 2>&1
+                        ;;
+        esac
+done
+
+echo "========= DONE SYNCHRONIZING =========="
+

cluster/prod/app/backup/build/backup-psql/.gitignore

@@ -0,0 +1 @@
+result

cluster/prod/app/backup/build/backup-psql/README.md

@@ -0,0 +1,8 @@
+## Build
+
+```bash
+docker load < $(nix-build docker.nix)
+docker push superboum/backup-psql:???
+```
+
+

cluster/prod/app/backup/build/backup-psql/backup-psql.py

@@ -0,0 +1,106 @@
+#!/usr/bin/env python3
+import shutil,sys,os,datetime,minio,subprocess
+
+working_directory = "."
+if 'CACHE_DIR' in os.environ: working_directory = os.environ['CACHE_DIR']
+required_space_in_bytes = 20 * 1024 * 1024 * 1024
+bucket = os.environ['AWS_BUCKET']
+key = os.environ['AWS_ACCESS_KEY_ID']
+secret = os.environ['AWS_SECRET_ACCESS_KEY']
+endpoint = os.environ['AWS_ENDPOINT']
+pubkey = os.environ['CRYPT_PUBLIC_KEY']
+psql_host = os.environ['PSQL_HOST']
+psql_user = os.environ['PSQL_USER']
+s3_prefix = str(datetime.datetime.now())
+files = [ "backup_manifest", "base.tar.gz", "pg_wal.tar.gz" ]
+clear_paths = [ os.path.join(working_directory, f) for f in files ]
+crypt_paths = [ os.path.join(working_directory, f) + ".age" for f in files ]
+s3_keys = [ s3_prefix + "/" + f for f in files ]
+
+def abort(msg):
+  for p in clear_paths + crypt_paths:
+    if os.path.exists(p):
+        print(f"Remove {p}")
+        os.remove(p)
+
+  if msg: sys.exit(msg)
+  else: print("success")
+
+# Check we have enough space on disk
+if shutil.disk_usage(working_directory).free < required_space_in_bytes:
+  abort(f"Not enough space on disk at path {working_directory} to perform a backup, aborting")
+
+# Check postgres password is set
+if 'PGPASSWORD' not in os.environ:
+  abort(f"You must pass postgres' password through the environment variable PGPASSWORD")
+
+# Check our working directory is empty
+if len(os.listdir(working_directory)) != 0:
+  abort(f"Working directory {working_directory} is not empty, aborting")
+
+# Check Minio
+client = minio.Minio(endpoint, key, secret)
+if not client.bucket_exists(bucket):
+  abort(f"Bucket {bucket} does not exist or its access is forbidden, aborting")
+
+# Perform the backup locally
+try:
+  ret = subprocess.run(["pg_basebackup",
+    f"--host={psql_host}",
+    f"--username={psql_user}",
+    f"--pgdata={working_directory}",
+    f"--format=tar",
+    "--wal-method=stream",
+    "--gzip",
+    "--compress=6",
+    "--progress",
+    "--max-rate=5M",
+  ])
+  if ret.returncode != 0:
+    abort(f"pg_basebackup exited, expected return code 0, got {ret.returncode}. aborting")
+except Exception as e:
+  abort(f"pg_basebackup raised exception {e}. aborting")
+
+# Check that the expected files are here
+for p in clear_paths:
+  print(f"Checking that {p} exists locally")
+  if not os.path.exists(p):
+    abort(f"File {p} expected but not found, aborting")
+
+# Cipher them
+for c, e in zip(clear_paths, crypt_paths):
+  print(f"Ciphering {c} to {e}")
+  try:
+    ret = subprocess.run(["age", "-r", pubkey, "-o", e, c])
+    if ret.returncode != 0:
+      abort(f"age exit code is {ret}, 0 expected. aborting")
+  except Exception as e:
+    abort(f"aged raised an exception. {e}. aborting")
+
+# Upload the backup to S3
+for p, k in zip(crypt_paths, s3_keys):
+  try:
+    print(f"Uploading {p} to {k}")
+    result = client.fput_object(bucket, k, p)
+    print(
+      "created {0} object; etag: {1}, version-id: {2}".format(
+        result.object_name, result.etag, result.version_id,
+      ),
+    )
+  except Exception as e:
+    abort(f"Exception {e} occured while upload {p}. aborting")
+
+# Check that the files have been uploaded
+for k in s3_keys:
+  try:
+    print(f"Checking that {k} exists remotely")
+    result = client.stat_object(bucket, k)
+    print(
+      "last-modified: {0}, size: {1}".format(
+        result.last_modified, result.size, 
+      ),
+    )
+  except Exception as e:
+    abort(f"{k} not found on S3. {e}. aborting")
+
+abort(None)

cluster/prod/app/backup/build/backup-psql/common.nix

@@ -0,0 +1,8 @@
+{
+  pkgsSrc = fetchTarball {
+    # Latest commit on https://github.com/NixOS/nixpkgs/tree/nixos-21.11
+    # As of 2022-04-15
+    url ="https://github.com/NixOS/nixpkgs/archive/2f06b87f64bc06229e05045853e0876666e1b023.tar.gz";
+    sha256 = "sha256:1d7zg96xw4qsqh7c89pgha9wkq3rbi9as3k3d88jlxy2z0ns0cy2";
+  };
+}

cluster/prod/app/backup/build/backup-psql/default.nix

@@ -0,0 +1,37 @@
+let
+  common = import ./common.nix;
+  pkgs = import common.pkgsSrc {};
+  python-with-my-packages = pkgs.python3.withPackages (p: with p; [
+    minio
+  ]);
+in
+  pkgs.stdenv.mkDerivation {
+    name = "backup-psql";
+    src = pkgs.lib.sourceFilesBySuffices ./. [ ".py" ];
+
+    buildInputs = [
+      python-with-my-packages
+      pkgs.age
+      pkgs.postgresql_14
+    ];
+
+    buildPhase = ''
+      cat > backup-psql <<EOF
+      #!${pkgs.bash}/bin/bash
+
+      export PYTHONPATH=${python-with-my-packages}/${python-with-my-packages.sitePackages}
+      export PATH=${python-with-my-packages}/bin:${pkgs.age}/bin:${pkgs.postgresql_14}/bin
+
+      ${python-with-my-packages}/bin/python3 $out/lib/backup-psql.py
+      EOF
+
+      chmod +x backup-psql
+    '';
+
+    installPhase = ''
+      mkdir -p $out/{bin,lib}
+      cp *.py $out/lib/backup-psql.py
+      cp backup-psql $out/bin/backup-psql
+    '';
+  }
+

cluster/prod/app/backup/build/backup-psql/docker.nix

@@ -0,0 +1,11 @@
+let
+  common = import ./common.nix;
+  app = import ./default.nix;
+  pkgs = import common.pkgsSrc {};
+in
+  pkgs.dockerTools.buildImage {
+    name = "superboum/backup-psql-docker";
+    config = {
+      Cmd = [ "${app}/bin/backup-psql" ];
+    };
+ }

cluster/prod/app/backup/deploy/backup-daily.hcl

@@ -0,0 +1,196 @@
+job "backup_daily" {
+  datacenters = ["neptune", "scorpio", "bespin"]
+  type = "batch"
+
+  priority = "60"
+
+  periodic {
+    cron = "@daily"
+    // Do not allow overlapping runs.
+    prohibit_overlap = true
+  }
+
+  group "backup-dovecot" {
+    constraint {
+      attribute = "${attr.unique.hostname}"
+      operator = "="
+      value = "ananas"
+    }
+
+    task "main" {
+      driver = "docker"
+
+      config {
+        image = "restic/restic:0.16.0"
+        entrypoint = [ "/bin/sh", "-c" ]
+        args = [ "restic backup /mail && restic forget --group-by paths --keep-within 1m1d --keep-within-weekly 3m --keep-within-monthly 1y && restic prune --max-unused 50% --max-repack-size 2G && restic check" ]
+        volumes = [
+          "/mnt/ssd/mail:/mail"
+        ]
+      }
+
+      template {
+        data = <<EOH
+AWS_ACCESS_KEY_ID={{ key "secrets/email/dovecot/backup_aws_access_key_id" }}
+AWS_SECRET_ACCESS_KEY={{ key "secrets/email/dovecot/backup_aws_secret_access_key" }}
+RESTIC_REPOSITORY={{ key "secrets/email/dovecot/backup_restic_repository" }}
+RESTIC_PASSWORD={{ key "secrets/email/dovecot/backup_restic_password" }}
+EOH
+
+         destination = "secrets/env_vars"
+         env = true
+      }
+
+      resources {
+        cpu = 500
+        memory = 100
+		memory_max = 1000
+      }
+
+      restart {
+        attempts = 2
+        interval = "30m"
+        delay = "15s"
+        mode = "fail"
+      }
+    }
+  }
+
+  group "backup-consul" {
+    task "consul-kv-export" {
+      driver = "docker"
+
+      lifecycle {      
+        hook = "prestart"      
+        sidecar = false    
+      }
+
+      config {
+        image = "consul:1.13.1"
+        network_mode = "host"
+        entrypoint = [ "/bin/sh", "-c" ]
+        args = [ "/bin/consul kv export > $NOMAD_ALLOC_DIR/consul.json" ]
+        volumes = [
+          "secrets:/etc/consul",
+        ]
+      }
+
+      env {
+        CONSUL_HTTP_ADDR = "https://consul.service.prod.consul:8501"
+	CONSUL_HTTP_SSL = "true"
+	CONSUL_CACERT = "/etc/consul/consul.crt"
+	CONSUL_CLIENT_CERT = "/etc/consul/consul-client.crt"
+	CONSUL_CLIENT_KEY = "/etc/consul/consul-client.key"
+      }
+
+      resources {
+        cpu = 200
+        memory = 200
+      }
+
+
+      template {
+        data = "{{ key \"secrets/consul/consul.crt\" }}"
+        destination = "secrets/consul.crt"
+      }
+
+      template {
+        data = "{{ key \"secrets/consul/consul-client.crt\" }}"
+        destination = "secrets/consul-client.crt"
+      }
+
+      template {
+        data = "{{ key \"secrets/consul/consul-client.key\" }}"
+        destination = "secrets/consul-client.key"
+      }
+
+      restart {
+        attempts = 2
+        interval = "30m"
+        delay = "15s"
+        mode = "fail"
+      }
+    }
+
+    task "restic-backup" {
+      driver = "docker"
+
+      config {
+        image = "restic/restic:0.16.0"
+        entrypoint = [ "/bin/sh", "-c" ]
+        args = [ "restic backup $NOMAD_ALLOC_DIR/consul.json && restic forget --group-by paths --keep-within 1m1d --keep-within-weekly 3m --keep-within-monthly 1y && restic prune --max-unused 50% --max-repack-size 2G && restic check" ]
+      }
+
+
+      template {
+        data = <<EOH
+AWS_ACCESS_KEY_ID={{ key "secrets/backup/consul/backup_aws_access_key_id" }}
+AWS_SECRET_ACCESS_KEY={{ key "secrets/backup/consul/backup_aws_secret_access_key" }}
+RESTIC_REPOSITORY={{ key "secrets/backup/consul/backup_restic_repository" }}
+RESTIC_PASSWORD={{ key "secrets/backup/consul/backup_restic_password" }}
+EOH
+
+         destination = "secrets/env_vars"
+         env = true
+      }
+
+      resources {
+        cpu = 200
+        memory = 200
+      }
+
+      restart {
+        attempts = 2
+        interval = "30m"
+        delay = "15s"
+        mode = "fail"
+      }
+    }
+  }
+
+  group "backup-cryptpad" {
+    constraint {
+      attribute = "${attr.unique.hostname}"
+      operator = "="
+      value = "concombre"
+    }
+
+    task "main" {
+      driver = "docker"
+
+      config {
+        image = "restic/restic:0.16.0"
+        entrypoint = [ "/bin/sh", "-c" ]
+        args = [ "restic backup /cryptpad && restic forget --group-by paths --keep-within 1m1d --keep-within-weekly 3m --keep-within-monthly 1y && restic prune --max-unused 50% --max-repack-size 2G && restic check" ]
+        volumes = [
+          "/mnt/ssd/cryptpad:/cryptpad"
+        ]
+      }
+
+      template {
+        data = <<EOH
+AWS_ACCESS_KEY_ID={{ key "secrets/backup/cryptpad/backup_aws_access_key_id" }}
+AWS_SECRET_ACCESS_KEY={{ key "secrets/backup/cryptpad/backup_aws_secret_access_key" }}
+RESTIC_REPOSITORY={{ key "secrets/backup/cryptpad/backup_restic_repository" }}
+RESTIC_PASSWORD={{ key "secrets/backup/cryptpad/backup_restic_password" }}
+EOH
+
+         destination = "secrets/env_vars"
+         env = true
+      }
+
+      resources {
+        cpu = 500
+        memory = 100
+		memory_max = 1000
+      }
+
+      restart {
+        attempts = 2
+        interval = "30m"
+        delay = "15s"
+        mode = "fail"
+      }
+    }
+  }
+}

cluster/prod/app/backup/deploy/backup-garage.hcl

@@ -0,0 +1,72 @@
+job "backup-garage" {
+  datacenters = ["neptune", "bespin", "scorpio"]
+  type = "batch"
+
+  priority = "60"
+
+  periodic {
+    cron = "@daily"
+    // Do not allow overlapping runs.
+    prohibit_overlap = true
+  }
+
+  group "backup-garage" {
+    task "main" {
+      driver = "docker"
+
+      config {
+        image = "lxpz/backup_garage:9"
+		network_mode = "host"
+        volumes = [
+		  "secrets/rclone.conf:/etc/secrets/rclone.conf"
+        ]
+      }
+
+      template {
+        data = <<EOH
+GARAGE_ADMIN_TOKEN={{ key "secrets/garage/admin_token" }}
+GARAGE_ADMIN_API_URL=http://localhost:3903
+GARAGE_ACCESS_KEY={{ key "secrets/backup/garage/s3_access_key_id" }}
+TARGET_BACKUP_DIR={{ key "secrets/backup/garage/target_sftp_directory" }}
+EOH
+         destination = "secrets/env_vars"
+         env = true
+      }
+
+      template {
+        data = <<EOH
+[garage]
+type = s3
+provider = Other
+env_auth = false
+access_key_id = {{ key "secrets/backup/garage/s3_access_key_id" }}
+secret_access_key = {{ key "secrets/backup/garage/s3_secret_access_key" }}
+endpoint = http://localhost:3900
+region = garage
+
+[backup]
+type = sftp
+host = {{ key "secrets/backup/garage/target_sftp_host" }}
+user = {{ key "secrets/backup/garage/target_sftp_user" }}
+port = {{ key "secrets/backup/garage/target_sftp_port" }}
+key_pem = {{ key "secrets/backup/garage/target_sftp_key_pem" | replaceAll "\n" "\\n" }}
+shell_type = unix
+EOH
+		destination = "secrets/rclone.conf"
+      }
+
+      resources {
+        cpu = 500
+        memory = 200
+		memory_max = 4000
+      }
+
+      restart {
+        attempts = 2
+        interval = "30m"
+        delay = "15s"
+        mode = "fail"
+      }
+    }
+  }
+}

cluster/prod/app/backup/deploy/backup-weekly.hcl

@@ -0,0 +1,55 @@
+job "backup_weekly" {
+  datacenters = ["scorpio", "neptune", "bespin"]
+  type = "batch"
+
+  priority = "60"
+
+  periodic {
+    cron = "@weekly"
+    // Do not allow overlapping runs.
+    prohibit_overlap = true
+  }
+
+  group "backup-psql" {
+    task "main" {
+      driver = "docker"
+
+      config {
+        image = "superboum/backup-psql-docker:gyr3aqgmhs0hxj0j9hkrdmm1m07i8za2"
+        volumes = [
+          // Mount a cache on the hard disk to avoid filling up the SSD
+          "/mnt/storage/tmp_bckp_psql:/mnt/cache"
+        ]
+      }
+
+      template {
+        data = <<EOH
+CACHE_DIR=/mnt/cache
+AWS_BUCKET=backups-pgbasebackup
+AWS_ENDPOINT=s3.deuxfleurs.shirokumo.net
+AWS_ACCESS_KEY_ID={{ key "secrets/postgres/backup/aws_access_key_id" }}
+AWS_SECRET_ACCESS_KEY={{ key "secrets/postgres/backup/aws_secret_access_key" }}
+CRYPT_PUBLIC_KEY={{ key "secrets/postgres/backup/crypt_public_key" }}
+PSQL_HOST={{ env "meta.site" }}.psql-proxy.service.prod.consul
+PSQL_USER={{ key "secrets/postgres/keeper/pg_repl_username" }}
+PGPASSWORD={{ key "secrets/postgres/keeper/pg_repl_pwd" }}
+EOH
+
+         destination = "secrets/env_vars"
+         env = true
+      }
+
+      resources {
+        cpu = 200
+        memory = 200
+      }
+
+      restart {
+        attempts = 2
+        interval = "30m"
+        delay = "15s"
+        mode = "fail"
+      }
+    }
+  }
+}

cluster/prod/app/backup/secrets.toml

@@ -0,0 +1,92 @@
+# Cryptpad backup
+
+[secrets."backup/cryptpad/backup_restic_password"]
+type = 'user'
+description = 'Restic password to encrypt backups'
+
+[secrets."backup/cryptpad/backup_aws_secret_access_key"]
+type = 'user'
+description = 'Backup AWS secret access key'
+
+[secrets."backup/cryptpad/backup_restic_repository"]
+type = 'user'
+description = 'Restic repository'
+example = 's3:https://s3.garage.tld'
+
+[secrets."backup/cryptpad/backup_aws_access_key_id"]
+type = 'user'
+description = 'Backup AWS access key ID'
+
+
+# Consul backup
+
+[secrets."backup/consul/backup_restic_password"]
+type = 'user'
+description = 'Restic password to encrypt backups'
+
+[secrets."backup/consul/backup_aws_secret_access_key"]
+type = 'user'
+description = 'Backup AWS secret access key'
+
+[secrets."backup/consul/backup_restic_repository"]
+type = 'user'
+description = 'Restic repository'
+example = 's3:https://s3.garage.tld'
+
+[secrets."backup/consul/backup_aws_access_key_id"]
+type = 'user'
+description = 'Backup AWS access key ID'
+
+
+# Postgresql backup
+
+[secrets."postgres/backup/aws_access_key_id"]
+type = 'user'
+description = 'Minio access key'
+
+[secrets."postgres/backup/aws_secret_access_key"]
+type = 'user'
+description = 'Minio secret key'
+
+[secrets."postgres/backup/crypt_public_key"]
+type = 'user'
+description = 'A public key to encypt backups with age'
+
+
+# Plume backup
+
+[secrets."plume/backup_restic_repository"]
+type = 'user'
+description = 'Restic repository'
+example = 's3:https://s3.garage.tld'
+
+[secrets."plume/backup_restic_password"]
+type = 'user'
+description = 'Restic password to encrypt backups'
+
+[secrets."plume/backup_aws_secret_access_key"]
+type = 'user'
+description = 'Backup AWS secret access key'
+
+[secrets."plume/backup_aws_access_key_id"]
+type = 'user'
+description = 'Backup AWS access key ID'
+
+
+# Dovecot backup
+
+[secrets."email/dovecot/backup_restic_password"]
+type = 'user'
+description = 'Restic backup password to encrypt data'
+
+[secrets."email/dovecot/backup_aws_secret_access_key"]
+type = 'user'
+description = 'AWS Secret Access key'
+
+[secrets."email/dovecot/backup_restic_repository"]
+type = 'user'
+description = 'Restic Repository URL, check op_guide/backup-minio to see the format'
+
+[secrets."email/dovecot/backup_aws_access_key_id"]
+type = 'user'
+description = 'AWS Acces Key ID'

cluster/prod/app/bagage/deploy/bagage.hcl

@@ -0,0 +1,88 @@
+job "bagage" {
+  datacenters = ["scorpio", "neptune"]
+  type = "service"
+  priority = 90
+
+  constraint {
+    attribute = "${attr.cpu.arch}"
+    value     = "amd64"
+  }
+
+  group "main" {
+    count = 1
+
+    network {
+      port "web_port" { 
+        static = 8080
+        to = 8080 
+      }
+      port "ssh_port" { 
+        static = 2222
+        to = 2222
+      }
+    }
+
+    task "server" {
+      driver = "docker"
+      config {
+        image = "lxpz/amd64_bagage:20231016-3"
+        readonly_rootfs = false
+        network_mode = "host"
+        volumes = [
+          "secrets/id_rsa:/id_rsa"
+        ]
+        ports = [ "web_port", "ssh_port" ]
+      }
+
+      env {
+        BAGAGE_LDAP_ENDPOINT = "bottin.service.prod.consul:389"
+      }
+
+      resources {
+        memory = 200
+        cpu = 100
+      }
+
+      template {
+        data = "{{ key \"secrets/bagage/id_rsa\" }}"
+        destination = "secrets/id_rsa"
+      }
+
+      service {
+        name = "bagage-ssh"
+        port = "ssh_port"
+        address_mode = "host"
+        tags = [
+          "bagage",
+          "(diplonat (tcp_port 2222))",
+          "d53-a sftp.deuxfleurs.fr",
+          "d53-aaaa sftp.deuxfleurs.fr",
+        ]
+      }
+
+      service {
+        name = "bagage-webdav"
+        tags = [
+          "bagage",
+          "tricot bagage.deuxfleurs.fr",
+          "d53-cname bagage.deuxfleurs.fr",
+        ]
+        port = "web_port"
+        address_mode = "host"
+        check {
+          type = "tcp"
+          port = "web_port"
+          address_mode = "host"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "90s"
+            ignore_warnings = false
+          }
+        }
+      }
+    }
+  }
+}
+

cluster/prod/app/bagage/secrets.toml

@@ -0,0 +1,4 @@
+[secrets."bagage/id_rsa"]
+type = 'command'
+rotate = true
+command = 'ssh-keygen -q -f >(cat) -N "" <<< y  2>/dev/null 1>&2 ; true'

cluster/prod/app/cms/config/teabag.env

@@ -0,0 +1,11 @@
+HOST=0.0.0.0
+PORT={{ env "NOMAD_PORT_web_port" }}
+SESSION_SECRET={{ key "secrets/cms/teabag/session" | trimSpace }}
+
+GITEA_KEY={{ key "secrets/cms/teabag/gitea_key" | trimSpace }}
+GITEA_SECRET={{ key "secrets/cms/teabag/gitea_secret" | trimSpace }}
+GITEA_BASE_URL=https://git.deuxfleurs.fr
+GITEA_AUTH_URI=login/oauth/authorize
+GITEA_TOKEN_URI=login/oauth/access_token
+GITEA_USER_URI=api/v1/user
+CALLBACK_URI=https://teabag.deuxfleurs.fr/callback

cluster/prod/app/cms/deploy/cms.hcl

@@ -0,0 +1,74 @@
+job "cms" {
+  datacenters = ["neptune", "scorpio"]
+  type = "service"
+
+  priority = 100
+
+  constraint {
+    attribute = "${attr.cpu.arch}"
+    value     = "amd64"
+  }
+
+  group "auth" {
+    count = 1
+
+    network {
+      port "web_port" { }
+    }
+
+    task "teabag" {
+      driver = "docker"
+      config {
+        # Using a digest to pin the container as no tag is provided
+        # https://github.com/denyskon/teabag/pkgs/container/teabag
+        image = "ghcr.io/denyskon/teabag@sha256:d5af7c6caf172727fbfa047c8ee82f9087ef904f0f3bffdeec656be04e9e0a14"
+        ports = [ "web_port" ]
+        volumes = [
+          "secrets/teabag.env:/etc/teabag/teabag.env",
+        ]
+      }
+
+      template {
+        data = file("../config/teabag.env")
+        destination = "secrets/teabag.env"
+      }
+
+      resources {
+        memory = 20
+        memory_max = 50
+        cpu = 50
+      }
+
+      service {
+        name = "teabag"
+        tags = [
+          "teabag",
+          "tricot teabag.deuxfleurs.fr",
+          "d53-cname teabag.deuxfleurs.fr",
+        ]
+        port = "web_port"
+        check {
+          type = "http"
+          protocol = "http"
+          port = "web_port"
+          path = "/"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "600s"
+            ignore_warnings = false
+          }
+        }
+      }
+
+      restart {
+        interval = "30m"
+        attempts = 20
+        delay    = "15s"
+        mode     = "delay"
+      }
+    }
+  }
+}
+

cluster/prod/app/cms/secrets.toml

@@ -0,0 +1,17 @@
+# HTTP Session Encryption Key
+[secrets."cms/teabag/session"]
+type = 'command'
+rotate = true
+command = 'openssl rand -base64 32'
+
+# Gitea Application Token
+[secrets."cms/teabag/gitea_key"]
+type = 'user'
+description = 'Gitea Application Key'
+example = '4fea0...'
+
+[secrets."cms/teabag/gitea_secret"]
+type = 'user'
+description = 'Gitea Secret Key'
+example = 'gto_bz6f...'
+

cluster/prod/app/core/config/bottin/config.json.tpl

@@ -0,0 +1,26 @@
+{
+  "suffix": "{{ key "secrets/directory/ldap_base_dn" }}",
+  "bind": "0.0.0.0:389",
+  "log_level": "debug",
+  "acl": [
+		"*,{{ key "secrets/directory/ldap_base_dn" }}::read:*:* !userpassword !user_secret !alternate_user_secrets !garage_s3_secret_key",
+		"*::read modify:SELF:*",
+		"ANONYMOUS::bind:*,ou=users,{{ key "secrets/directory/ldap_base_dn" }}:",
+		"ANONYMOUS::bind:cn=admin,{{ key "secrets/directory/ldap_base_dn" }}:",
+		"*,ou=services,ou=users,{{ key "secrets/directory/ldap_base_dn" }}::bind:*,ou=users,{{ key "secrets/directory/ldap_base_dn" }}:*",
+		"*,ou=services,ou=users,{{ key "secrets/directory/ldap_base_dn" }}::read:*:*",
+
+		"*:cn=asso_deuxfleurs,ou=groups,{{ key "secrets/directory/ldap_base_dn" }}:add:*,ou=invitations,{{ key "secrets/directory/ldap_base_dn" }}:*",
+		"ANONYMOUS::bind:*,ou=invitations,{{ key "secrets/directory/ldap_base_dn" }}:",
+		"*,ou=invitations,{{ key "secrets/directory/ldap_base_dn" }}::delete:SELF:*",
+
+		"*:cn=asso_deuxfleurs,ou=groups,{{ key "secrets/directory/ldap_base_dn" }}:add:*,ou=users,{{ key "secrets/directory/ldap_base_dn" }}:*",
+		"*,ou=invitations,{{ key "secrets/directory/ldap_base_dn" }}::add:*,ou=users,{{ key "secrets/directory/ldap_base_dn" }}:*",
+
+		"*:cn=asso_deuxfleurs,ou=groups,{{ key "secrets/directory/ldap_base_dn" }}:modifyAdd:cn=email,ou=groups,{{ key "secrets/directory/ldap_base_dn" }}:*",
+		"*,ou=invitations,{{ key "secrets/directory/ldap_base_dn" }}::modifyAdd:cn=email,ou=groups,{{ key "secrets/directory/ldap_base_dn" }}:*",
+
+		"cn=admin,{{ key "secrets/directory/ldap_base_dn" }}::read add modify delete:*:*",
+		"*:cn=admin,ou=groups,{{ key "secrets/directory/ldap_base_dn" }}:read add modify delete:*:*"
+  ]
+}

cluster/prod/app/core/deploy/bottin.hcl

@@ -0,0 +1,100 @@
+job "core-bottin" {
+  datacenters = ["neptune", "scorpio"]
+  type = "system"
+  priority = 90
+
+  update {
+    max_parallel = 1
+    stagger = "1m"
+  }
+
+  group "bottin" {
+    constraint {
+      distinct_property = "${meta.site}"
+      value = "1"
+    }
+
+    network {
+      port "ldap_port" {
+        static = 389
+        to = 389
+      }
+    }
+
+    task "bottin" {
+      driver = "docker"
+      config {
+        image = "dxflrs/bottin:7h18i30cckckaahv87d3c86pn4a7q41z"
+        network_mode = "host"
+        readonly_rootfs = true
+        ports = [ "ldap_port" ]
+        volumes = [
+          "secrets/config.json:/config.json",
+          "secrets:/etc/bottin",
+        ]
+      }
+
+      restart {
+        interval = "5m"
+        attempts = 10
+        delay    = "15s"
+        mode     = "delay"
+      }
+
+      resources {
+        memory = 100
+        memory_max = 200
+      }
+
+      template {
+        data = file("../config/bottin/config.json.tpl")
+        destination = "secrets/config.json"
+      }
+
+      template {
+        data = "{{ key \"secrets/consul/consul.crt\" }}"
+        destination = "secrets/consul.crt"
+      }
+
+      template {
+        data = "{{ key \"secrets/consul/consul-client.crt\" }}"
+        destination = "secrets/consul-client.crt"
+      }
+
+      template {
+        data = "{{ key \"secrets/consul/consul-client.key\" }}"
+        destination = "secrets/consul-client.key"
+      }
+
+      template {
+        data = <<EOH
+CONSUL_HTTP_ADDR=https://consul.service.prod.consul:8501
+CONSUL_HTTP_SSL=true
+CONSUL_CACERT=/etc/bottin/consul.crt
+CONSUL_CLIENT_CERT=/etc/bottin/consul-client.crt
+CONSUL_CLIENT_KEY=/etc/bottin/consul-client.key
+EOH
+        destination = "secrets/env"
+        env = true
+      }
+
+      service {
+        tags = [ "${meta.site}" ]
+        port = "ldap_port"
+        address_mode = "host"
+        name = "bottin"
+        check {
+          type = "tcp"
+          port = "ldap_port"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "90s"
+            ignore_warnings = false
+          }
+        }
+      }
+    }
+  }
+}

cluster/prod/app/core/deploy/d53.hcl

@@ -0,0 +1,102 @@
+job "core-d53" {
+  datacenters = ["neptune", "scorpio", "bespin"]
+  type = "service"
+  priority = 90
+
+  group "D53" {
+    count = 1
+
+    task "d53" {
+      driver = "docker"
+
+      config {
+        image = "lxpz/amd64_d53:4"
+        network_mode = "host"
+        readonly_rootfs = true
+        volumes = [
+          "secrets:/etc/d53",
+        ]
+      }
+
+      resources {
+        cpu = 100
+        memory = 100
+      }
+
+      restart {
+        interval = "3m"
+        attempts = 10
+        delay    = "15s"
+        mode     = "delay"
+      }
+
+      template {
+        data = "{{ key \"secrets/consul/consul-ca.crt\" }}"
+        destination = "secrets/consul-ca.crt"
+      }
+
+      template {
+        data = "{{ key \"secrets/consul/consul-client.crt\" }}"
+        destination = "secrets/consul-client.crt"
+      }
+
+      template {
+        data = "{{ key \"secrets/consul/consul-client.key\" }}"
+        destination = "secrets/consul-client.key"
+      }
+
+      template {
+        data = <<EOH
+D53_CONSUL_HOST=https://localhost:8501
+D53_CONSUL_CA_CERT=/etc/d53/consul-ca.crt
+D53_CONSUL_CLIENT_CERT=/etc/d53/consul-client.crt
+D53_CONSUL_CLIENT_KEY=/etc/d53/consul-client.key
+D53_PROVIDERS=deuxfleurs.fr:gandi
+D53_GANDI_API_KEY={{ key "secrets/d53/gandi_api_key" }}
+D53_ALLOWED_DOMAINS=deuxfleurs.fr
+RUST_LOG=d53=info
+EOH
+        destination = "secrets/env"
+        env = true
+      }
+    }
+  }
+
+  # Dummy task for Gitea (still on an external VM), runs on any bespin node
+  # and allows D53 to automatically update the A record for git.deuxfleurs.fr
+  # to the IPv4 address of the bespin site (that changes occasionnaly)
+  group "gitea-dummy" {
+    count = 1
+
+    network {
+      port "dummy" {
+        to = 999
+      }
+    }
+
+    task "main" {
+      driver = "docker"
+
+      constraint {
+        attribute = "${meta.site}"
+        operator = "="
+        value = "bespin"
+      }
+
+      config {
+        image = "alpine"
+        command = "sh"
+        args = ["-c", "while true; do echo x; sleep 60; done"]
+        ports = [ "dummy" ]
+      }
+
+      service {
+        name = "gitea-dummy"
+        port = "dummy"
+        tags = [
+          "d53-a git.deuxfleurs.fr",
+        ]
+      }
+    }
+  }
+}

cluster/prod/app/core/deploy/diplonat.hcl

@@ -1,41 +1,37 @@
-job "core" {
-  datacenters = ["dc1", "neptune"]
+job "core-diplonat" {
+  datacenters = ["neptune", "scorpio", "bespin"]
   type = "system"
   priority = 90
 
-  constraint {
-    attribute = "${attr.cpu.arch}"
-    value     = "amd64"
-  }
-
   update {
-    max_parallel     = 1
+    max_parallel = 2
     stagger = "1m"
   }
 
-  group "network" {
+  group "diplonat" {
     task "diplonat" {
       driver = "docker"
 
       config {
-        image = "lxpz/amd64_diplonat:3"
+        image = "lxpz/amd64_diplonat:7"
         network_mode = "host"
         readonly_rootfs = true
+        privileged = true
         volumes = [
           "secrets:/etc/diplonat",
         ]
       }
 
       restart {
-        interval = "30m"
-        attempts = 2
+        interval = "5m"
+        attempts = 10
         delay    = "15s"
         mode     = "delay"
       }
 
       template {
-        data = "{{ key \"secrets/consul/consul-ca.crt\" }}"
-        destination = "secrets/consul-ca.crt"
+        data = "{{ key \"secrets/consul/consul.crt\" }}"
+        destination = "secrets/consul.crt"
       }
 
       template {
@@ -53,8 +49,8 @@ job "core" {
 DIPLONAT_REFRESH_TIME=60
 DIPLONAT_EXPIRATION_TIME=300
 DIPLONAT_CONSUL_NODE_NAME={{ env "attr.unique.hostname" }}
-DIPLONAT_CONSUL_URL=https://localhost:8501
-DIPLONAT_CONSUL_CA_CERT=/etc/diplonat/consul-ca.crt
+DIPLONAT_CONSUL_URL=https://consul.service.prod.consul:8501
+DIPLONAT_CONSUL_TLS_SKIP_VERIFY=true
 DIPLONAT_CONSUL_CLIENT_CERT=/etc/diplonat/consul-client.crt
 DIPLONAT_CONSUL_CLIENT_KEY=/etc/diplonat/consul-client.key
 RUST_LOG=debug
@@ -64,7 +60,8 @@ EOH
       }
 
       resources {
-        memory = 40
+        memory = 100
+        memory_max = 200
       }
     }
   }

cluster/prod/app/core/deploy/tricot.hcl

@@ -0,0 +1,120 @@
+job "core-tricot" {
+  # bespin pas pour l'instant, on a des soucis de SSL avec gitea
+  # on pourra mettre bespin quand on aura migré gitea de la vm vers le cluster
+  # en attendant, les deux  ne sont pas capables de partager les certificats SSL
+  # donc on laisse la VM gitea gérer les certifs et prendre tout le trafic http(s)
+  datacenters = ["neptune", "scorpio"]
+  type = "system"
+  priority = 90
+
+  update {
+    max_parallel = 1
+    stagger = "5m"
+  }
+
+  group "tricot" {
+    constraint {
+      distinct_property = "${meta.site}"
+      value = "1"
+    }
+
+    network {
+      port "http_port" { static = 80 }
+      port "https_port" { static = 443 }
+      port "metrics_port" { static = 9334 }
+    }
+
+    task "server" {
+      driver = "docker"
+
+      config {
+        image = "superboum/amd64_tricot:54"
+        network_mode = "host"
+        readonly_rootfs = true
+        ports = [ "http_port", "https_port" ]
+        volumes = [
+          "secrets:/etc/tricot",
+        ]
+      }
+
+      resources {
+        cpu = 1000
+        memory = 200
+        memory_max = 500
+      }
+
+      restart {
+        interval = "5m"
+        attempts = 10
+        delay    = "15s"
+        mode     = "delay"
+      }
+
+      template {
+        data = "{{ key \"secrets/consul/consul-ca.crt\" }}"
+        destination = "secrets/consul-ca.crt"
+      }
+
+      template {
+        data = "{{ key \"secrets/consul/consul-client.crt\" }}"
+        destination = "secrets/consul-client.crt"
+      }
+
+      template {
+        data = "{{ key \"secrets/consul/consul-client.key\" }}"
+        destination = "secrets/consul-client.key"
+      }
+
+      template {
+        data = <<EOH
+TRICOT_NODE_NAME={{ env "attr.unique.hostname" }}
+TRICOT_LETSENCRYPT_EMAIL=prod-sysadmin@deuxfleurs.fr
+TRICOT_ENABLE_COMPRESSION=true
+TRICOT_CONSUL_HOST=https://consul.service.prod.consul:8501
+TRICOT_CONSUL_TLS_SKIP_VERIFY=true
+TRICOT_CONSUL_CLIENT_CERT=/etc/tricot/consul-client.crt
+TRICOT_CONSUL_CLIENT_KEY=/etc/tricot/consul-client.key
+TRICOT_HTTP_BIND_ADDR=[::]:80
+TRICOT_HTTPS_BIND_ADDR=[::]:443
+TRICOT_METRICS_BIND_ADDR=[::]:9334
+TRICOT_WARMUP_CERT_MEMORY_STORE=true
+RUST_LOG=tricot=debug
+EOH
+        destination = "secrets/env"
+        env = true
+      }
+
+      service {
+        name = "tricot-http"
+        port = "http_port"
+        tags = [
+          "(diplonat (tcp_port 80))",
+          "${meta.site}"
+         ]
+        address_mode = "host"
+      }
+
+      service {
+        name = "tricot-https"
+        port = "https_port"
+        tags = [
+          "(diplonat (tcp_port 443))",
+          "${meta.site}",
+          "d53-a global.site.deuxfleurs.fr",
+          "d53-aaaa global.site.deuxfleurs.fr",
+          "d53-a ${meta.site}.site.deuxfleurs.fr",
+          "d53-aaaa ${meta.site}.site.deuxfleurs.fr",
+          "d53-a v4.${meta.site}.site.deuxfleurs.fr",
+          "d53-aaaa v6.${meta.site}.site.deuxfleurs.fr",
+        ]
+        address_mode = "host"
+      }
+
+      service {
+        name = "tricot-metrics"
+        port = "metrics_port"
+        address_mode = "host"
+      }
+    }
+  }
+}

cluster/prod/app/core/secrets.toml

@@ -0,0 +1,5 @@
+[secrets."directory/ldap_base_dn"]
+type = 'user'
+description = 'LDAP base DN for everything'
+example = 'dc=example,dc=com'
+

cluster/prod/app/coturn/config/docker-entrypoint.sh

@@ -0,0 +1,15 @@
+#!/bin/sh
+
+turnserver \
+    -n \
+    --external-ip=$(detect-external-ip) \
+    --min-port=49160 \
+    --max-port=49169 \
+    --log-file=stdout \
+    --use-auth-secret \
+    --realm turn.deuxfleurs.fr \
+    --no-cli \
+    --no-tls \
+    --no-dtls \
+    --prometheus \
+    --static-auth-secret '{{ key "secrets/coturn/static-auth-secret" | trimSpace }}' 

cluster/prod/app/coturn/deploy/coturn.hcl

@@ -0,0 +1,87 @@
+job "coturn" {
+  datacenters = ["neptune", "scorpio"]
+  type = "service"
+
+  priority = 100
+
+  constraint {
+    attribute = "${attr.cpu.arch}"
+    value     = "amd64"
+  }
+
+  group "main" {
+    count = 1
+
+    network {
+      port "prometheus" { static = 9641 }
+      port "turn_ctrl" { static = 3478 }
+      port "turn_data0" { static = 49160 }
+      port "turn_data1" { static = 49161 }
+      port "turn_data2" { static = 49162 }
+      port "turn_data3" { static = 49163 }
+      port "turn_data4" { static = 49164 }
+      port "turn_data5" { static = 49165 }
+      port "turn_data6" { static = 49166 }
+      port "turn_data7" { static = 49167 }
+      port "turn_data8" { static = 49168 }
+      port "turn_data9" { static = 49169 }
+    }
+
+    task "turnserver" {
+      driver = "docker"
+      config {
+        image = "coturn/coturn:4.6.1-r2-alpine"
+        ports = [ "prometheus", "turn_ctrl", "turn_data0", "turn_data1", "turn_data2",
+                  "turn_data3", "turn_data4", "turn_data5", "turn_data6", "turn_data7",
+                  "turn_data8", "turn_data9" ]
+        network_mode = "host"
+        volumes = [
+          "secrets/docker-entrypoint.sh:/usr/local/bin/docker-entrypoint.sh",
+        ]
+      }
+
+      template {
+        data = file("../config/docker-entrypoint.sh")
+        destination = "secrets/docker-entrypoint.sh"
+        perms = 555
+      }
+
+      resources {
+        memory = 20
+        memory_max = 50
+        cpu = 50
+      }
+
+      service {
+        name = "coturn"
+        tags = [
+          "coturn",
+          "d53-cname turn.deuxfleurs.fr",
+          "(diplonat (tcp_port 3478) (udp_port 3478 49160 49161 49162 49163 49164 49165 49166 49167 49168 49169))",
+        ]
+        port = "turn_ctrl"
+        check {
+          type = "http"
+          protocol = "http"
+          port = "prometheus"
+          path = "/"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "600s"
+            ignore_warnings = false
+          }
+        }
+      }
+
+      restart {
+        interval = "30m"
+        attempts = 20
+        delay    = "15s"
+        mode     = "delay"
+      }
+    }
+  }
+}
+

cluster/prod/app/coturn/integration/cmd.sh

@@ -0,0 +1,7 @@
+docker run \
+    --name coturn \
+    --rm \
+    -it \
+    -v `pwd`/docker-entrypoint.sh:/usr/local/bin/docker-entrypoint.sh \
+    --network=host \
+    coturn/coturn:4.6.1-r2-alpine

cluster/prod/app/coturn/readme.md

@@ -0,0 +1,6 @@
+stun+turn
+tcp: 3478
+udp: 49160-49169
+
+prometheus:
+tcp: 9641

cluster/prod/app/coturn/secrets.toml

@@ -0,0 +1,5 @@
+# coturn 
+[secrets."coturn/static-auth-secret"]
+type = 'command'
+rotate = true
+command = "openssl rand -base64 64|tr -d '\n'"

cluster/prod/app/cryptpad/build/README.md

@@ -0,0 +1,56 @@
+# CryptPad for NixOS with Deuxfleurs flavour
+
+## Building
+
+The `default.nix` file follows the nixpkgs `callPackage` convention for fetching dependencies, so you need to either:
+
+- Run `nix-build --expr '{ ... }@args: (import <nixpkgs> {}).callPackage ./default.nix args'`
+- Do the `callPackage from a higher-level directory importing your package`
+
+### Docker
+
+The `docker.nix` derives into a Docker image you can load simply by running:
+
+```shell
+docker load -i $(nix-build docker.nix)
+```
+
+You can then test the built Docker image using the provided `docker-compose.yml` and `config.js` files, which are
+configured to render the instance accessible at `http://localhost:3000` with data stored into the `_data` folder.
+
+
+### Deuxfleurs flavour
+The `deuxfleurs.nix` file derives into two derivations: The CryptPad derivation itself and a Docker image,
+which can be choose by passing the `-A [name]` flags to `nix-build`
+
+For example, to build and load the Deuxfleurs-flavoured CryptPad Docker image, you run:
+
+```shell
+docker load -i $(nix-build deuxfleurs.nix -A docker)
+```
+
+## OnlyOffice integration
+Apart for `deuxfleurs.nix`, both `default.nix` and `docker.nix` files build CryptPad with a copy of OnlyOffice pre-built and
+used by CryptPad, which can result to large Docker image (~2.6GiB)
+
+This behaviour is configurable by passing the `--arg withOnlyOffice false` flag to `nix-build` when building them.
+
+## Updating the Deuxfleurs pinned nixpkgs
+The pinned sources files are generated with the [niv](https://github.com/nmattia/niv) tool.
+
+To update the pinned nixpkgs, you simply run the following command:
+
+```shell
+niv update
+```
+
+To modify the pinned nixpkgs, you can use the `niv modify` command, for example, to move to nixpkgs-unstable:
+
+```shell
+niv modify nixpkgs -b nixos-unstable
+```
+
+## Quirks
+
+- The CryptPad `package-lock.json` is included here because the upstream-provided one appeared to be desync'ed, so a
+    manual lockfile generation was needed

cluster/prod/app/cryptpad/build/default.nix

@@ -0,0 +1,118 @@
+{ lib
+, stdenvNoCC
+
+, buildNpmPackage
+, fetchFromGitHub
+
+, nodejs
+
+, withOnlyOffice ? true
+}: let
+  onlyOfficeVersions = {
+    v1 = {
+      rev = "4f370bebe96e3a0d4054df87412ee5b2c6ed8aaa";
+      hash = "sha256-TE/99qOx4wT2s0op9wi+SHwqTPYq/H+a9Uus9Zj4iSY=";
+    };
+    v2b = {
+      rev = "d9da72fda95daf93b90ffa345757c47eb5b919dd";
+      hash = "sha256-SiRDRc2vnLwCVnvtk+C8PKw7IeuSzHBaJmZHogRe3hQ=";
+    };
+    v4 = {
+      rev =  "6ebc6938b6841440ffad2efc1e23f1dc1ceda964";
+      hash = "sha256-eto1+8Tk/s3kbUCpbUh8qCS8EOq700FYG1/KiHyynaA=";
+    };
+    v5 = {
+      rev = "88a356f08ded2f0f4620bda66951caf1d7f02c21";
+      hash = "sha256-8j1rlAyHlKx6oAs2pIhjPKcGhJFj6ZzahOcgenyeOCc=";
+    };
+    v6 = {
+      rev = "abd8a309f6dd37289f950cd8cea40df4492d8a15";
+      hash = "sha256-BZdExj2q/bqUD3k9uluOot2dlrWKA+vpad49EdgXKww=";
+    };
+    v7 = {
+      rev = "9d8b914a81f0f9e5d0bc3f0fc631adf4b6d480e7";
+      hash = "sha256-M+rPJ/Xo2olhqB5ViynGRaesMLLfG/1ltUoLnepMPnM=";
+    };
+  };
+  mkOnlyOffice = {
+    pname, version
+  }: stdenvNoCC.mkDerivation (final: {
+    pname = "${pname}-onlyoffice";
+    inherit version;
+
+    srcs = lib.mapAttrsToList (version: { rev, hash ? lib.fakeHash }: fetchFromGitHub {
+      name = "${final.pname}-${version}-source";
+      owner = "cryptpad";
+      repo = "onlyoffice-builds";
+      inherit rev hash;
+    }) onlyOfficeVersions;
+
+    dontBuild = true;
+
+    sourceRoot = ".";
+
+    installPhase = ''
+      mkdir -p $out
+      ${lib.concatLines (map
+        (version: "cp -Tr ${final.pname}-${version}-source $out/${version}")
+        (builtins.attrNames onlyOfficeVersions)
+      )}
+    '';
+  });
+in buildNpmPackage rec {
+  pname = "cryptpad";
+  version = "2024.3.0";
+
+  src = fetchFromGitHub {
+    owner = "cryptpad";
+    repo = "cryptpad";
+    rev = version;
+    hash = "sha256-VUW6KvoSatk1/hlzklMQYlSNVH/tdbH+yU4ONUQ0JSQ=";
+  };
+
+  npmDepsHash = "sha256-tvTkoxxioPuNoe8KIuXSP7QQbvcpxMnygsMmzKBQIY0=";
+
+  inherit nodejs;
+
+  onlyOffice = lib.optional withOnlyOffice (mkOnlyOffice {
+    inherit pname version;
+  });
+
+  makeCacheWritable = true;
+  dontFixup = true;
+
+  postPatch = ''
+    cp -T ${./package-lock.json} package-lock.json
+  '';
+
+  preBuild = ''
+    npm run install:components
+  '' + lib.optionalString withOnlyOffice ''
+    ln -s $onlyOffice www/common/onlyoffice/dist
+  '';
+
+  postBuild = ''
+    rm -rf customize
+  '';
+
+  installPhase = ''
+    runHook preInstall
+
+    mkdir -p $out
+    cp -R . $out/
+
+    substituteInPlace $out/lib/workers/index.js \
+      --replace-warn "lib/workers/db-worker" "$out/lib/workers/db-worker"
+
+    makeWrapper ${lib.getExe nodejs} $out/bin/cryptpad-server \
+      --chdir $out \
+      --add-flags server.js
+
+    runHook postInstall
+  '';
+
+  meta = {
+    homepage = "https://cryptpad.org";
+    mainProgram = "cryptpad-server";
+  };
+}

cluster/prod/app/cryptpad/build/deuxfleurs.nix

@@ -0,0 +1,12 @@
+{ name ? "deuxfleurs/cryptpad"
+, tag ? "nix-latest"
+}: let
+  sources = import ./nix/sources.nix;
+  pkgs = import sources.nixpkgs {};
+in rec {
+  cryptpad = pkgs.callPackage ./default.nix {};
+  docker = pkgs.callPackage ./docker.nix {
+    inherit name tag;
+    inherit cryptpad;
+  };
+}

cluster/prod/app/cryptpad/build/docker.nix

@@ -0,0 +1,27 @@
+{ pkgs ? import <nixpkgs> {}
+
+, name ? "cryptpad"
+, tag ? "nix-latest"
+
+, withOnlyOffice ? true
+
+, cryptpad ? pkgs.callPackage ./default.nix { inherit withOnlyOffice; }
+}: let
+  cryptpad' = cryptpad.overrideAttrs {
+    postInstall = ''
+      ln -sf /cryptpad/customize $out/customize
+    '';
+  };
+in pkgs.dockerTools.buildImage {
+  inherit name tag;
+
+  config = {
+    Cmd = [
+      (pkgs.lib.getExe cryptpad')
+    ];
+
+    Volumes = {
+      "/cryptpad/customize" = {};
+    };
+  };
+}

cluster/prod/app/cryptpad/build/nix/sources.json

@@ -0,0 +1,14 @@
+{
+    "nixpkgs": {
+        "branch": "nixos-23.11",
+        "description": "Nix Packages collection",
+        "homepage": null,
+        "owner": "NixOS",
+        "repo": "nixpkgs",
+        "rev": "53a2c32bc66f5ae41a28d7a9a49d321172af621e",
+        "sha256": "0yqbwqbripb1bbhlwjfbqmg9qb0lai2fc0k1vfh674d6rrc8igwv",
+        "type": "tarball",
+        "url": "https://github.com/NixOS/nixpkgs/archive/53a2c32bc66f5ae41a28d7a9a49d321172af621e.tar.gz",
+        "url_template": "https://github.com/<owner>/<repo>/archive/<rev>.tar.gz"
+    }
+}

cluster/prod/app/cryptpad/build/nix/sources.nix

@@ -0,0 +1,198 @@
+# This file has been generated by Niv.
+
+let
+
+  #
+  # The fetchers. fetch_<type> fetches specs of type <type>.
+  #
+
+  fetch_file = pkgs: name: spec:
+    let
+      name' = sanitizeName name + "-src";
+    in
+    if spec.builtin or true then
+      builtins_fetchurl { inherit (spec) url sha256; name = name'; }
+    else
+      pkgs.fetchurl { inherit (spec) url sha256; name = name'; };
+
+  fetch_tarball = pkgs: name: spec:
+    let
+      name' = sanitizeName name + "-src";
+    in
+    if spec.builtin or true then
+      builtins_fetchTarball { name = name'; inherit (spec) url sha256; }
+    else
+      pkgs.fetchzip { name = name'; inherit (spec) url sha256; };
+
+  fetch_git = name: spec:
+    let
+      ref =
+        spec.ref or (
+          if spec ? branch then "refs/heads/${spec.branch}" else
+          if spec ? tag then "refs/tags/${spec.tag}" else
+          abort "In git source '${name}': Please specify `ref`, `tag` or `branch`!"
+        );
+      submodules = spec.submodules or false;
+      submoduleArg =
+        let
+          nixSupportsSubmodules = builtins.compareVersions builtins.nixVersion "2.4" >= 0;
+          emptyArgWithWarning =
+            if submodules
+            then
+              builtins.trace
+                (
+                  "The niv input \"${name}\" uses submodules "
+                  + "but your nix's (${builtins.nixVersion}) builtins.fetchGit "
+                  + "does not support them"
+                )
+                { }
+            else { };
+        in
+        if nixSupportsSubmodules
+        then { inherit submodules; }
+        else emptyArgWithWarning;
+    in
+    builtins.fetchGit
+      ({ url = spec.repo; inherit (spec) rev; inherit ref; } // submoduleArg);
+
+  fetch_local = spec: spec.path;
+
+  fetch_builtin-tarball = name: throw
+    ''[${name}] The niv type "builtin-tarball" is deprecated. You should instead use `builtin = true`.
+        $ niv modify ${name} -a type=tarball -a builtin=true'';
+
+  fetch_builtin-url = name: throw
+    ''[${name}] The niv type "builtin-url" will soon be deprecated. You should instead use `builtin = true`.
+        $ niv modify ${name} -a type=file -a builtin=true'';
+
+  #
+  # Various helpers
+  #
+
+  # https://github.com/NixOS/nixpkgs/pull/83241/files#diff-c6f540a4f3bfa4b0e8b6bafd4cd54e8bR695
+  sanitizeName = name:
+    (
+      concatMapStrings (s: if builtins.isList s then "-" else s)
+        (
+          builtins.split "[^[:alnum:]+._?=-]+"
+            ((x: builtins.elemAt (builtins.match "\\.*(.*)" x) 0) name)
+        )
+    );
+
+  # The set of packages used when specs are fetched using non-builtins.
+  mkPkgs = sources: system:
+    let
+      sourcesNixpkgs =
+        import (builtins_fetchTarball { inherit (sources.nixpkgs) url sha256; }) { inherit system; };
+      hasNixpkgsPath = builtins.any (x: x.prefix == "nixpkgs") builtins.nixPath;
+      hasThisAsNixpkgsPath = <nixpkgs> == ./.;
+    in
+    if builtins.hasAttr "nixpkgs" sources
+    then sourcesNixpkgs
+    else if hasNixpkgsPath && ! hasThisAsNixpkgsPath then
+      import <nixpkgs> { }
+    else
+      abort
+        ''
+          Please specify either <nixpkgs> (through -I or NIX_PATH=nixpkgs=...) or
+          add a package called "nixpkgs" to your sources.json.
+        '';
+
+  # The actual fetching function.
+  fetch = pkgs: name: spec:
+
+    if ! builtins.hasAttr "type" spec then
+      abort "ERROR: niv spec ${name} does not have a 'type' attribute"
+    else if spec.type == "file" then fetch_file pkgs name spec
+    else if spec.type == "tarball" then fetch_tarball pkgs name spec
+    else if spec.type == "git" then fetch_git name spec
+    else if spec.type == "local" then fetch_local spec
+    else if spec.type == "builtin-tarball" then fetch_builtin-tarball name
+    else if spec.type == "builtin-url" then fetch_builtin-url name
+    else
+      abort "ERROR: niv spec ${name} has unknown type ${builtins.toJSON spec.type}";
+
+  # If the environment variable NIV_OVERRIDE_${name} is set, then use
+  # the path directly as opposed to the fetched source.
+  replace = name: drv:
+    let
+      saneName = stringAsChars (c: if (builtins.match "[a-zA-Z0-9]" c) == null then "_" else c) name;
+      ersatz = builtins.getEnv "NIV_OVERRIDE_${saneName}";
+    in
+    if ersatz == "" then drv else
+      # this turns the string into an actual Nix path (for both absolute and
+      # relative paths)
+    if builtins.substring 0 1 ersatz == "/" then /. + ersatz else /. + builtins.getEnv "PWD" + "/${ersatz}";
+
+  # Ports of functions for older nix versions
+
+  # a Nix version of mapAttrs if the built-in doesn't exist
+  mapAttrs = builtins.mapAttrs or (
+    f: set: with builtins;
+    listToAttrs (map (attr: { name = attr; value = f attr set.${attr}; }) (attrNames set))
+  );
+
+  # https://github.com/NixOS/nixpkgs/blob/0258808f5744ca980b9a1f24fe0b1e6f0fecee9c/lib/lists.nix#L295
+  range = first: last: if first > last then [ ] else builtins.genList (n: first + n) (last - first + 1);
+
+  # https://github.com/NixOS/nixpkgs/blob/0258808f5744ca980b9a1f24fe0b1e6f0fecee9c/lib/strings.nix#L257
+  stringToCharacters = s: map (p: builtins.substring p 1 s) (range 0 (builtins.stringLength s - 1));
+
+  # https://github.com/NixOS/nixpkgs/blob/0258808f5744ca980b9a1f24fe0b1e6f0fecee9c/lib/strings.nix#L269
+  stringAsChars = f: s: concatStrings (map f (stringToCharacters s));
+  concatMapStrings = f: list: concatStrings (map f list);
+  concatStrings = builtins.concatStringsSep "";
+
+  # https://github.com/NixOS/nixpkgs/blob/8a9f58a375c401b96da862d969f66429def1d118/lib/attrsets.nix#L331
+  optionalAttrs = cond: as: if cond then as else { };
+
+  # fetchTarball version that is compatible between all the versions of Nix
+  builtins_fetchTarball = { url, name ? null, sha256 }@attrs:
+    let
+      inherit (builtins) lessThan nixVersion fetchTarball;
+    in
+    if lessThan nixVersion "1.12" then
+      fetchTarball ({ inherit url; } // (optionalAttrs (name != null) { inherit name; }))
+    else
+      fetchTarball attrs;
+
+  # fetchurl version that is compatible between all the versions of Nix
+  builtins_fetchurl = { url, name ? null, sha256 }@attrs:
+    let
+      inherit (builtins) lessThan nixVersion fetchurl;
+    in
+    if lessThan nixVersion "1.12" then
+      fetchurl ({ inherit url; } // (optionalAttrs (name != null) { inherit name; }))
+    else
+      fetchurl attrs;
+
+  # Create the final "sources" from the config
+  mkSources = config:
+    mapAttrs
+      (
+        name: spec:
+          if builtins.hasAttr "outPath" spec
+          then
+            abort
+              "The values in sources.json should not have an 'outPath' attribute"
+          else
+            spec // { outPath = replace name (fetch config.pkgs name spec); }
+      )
+      config.sources;
+
+  # The "config" used by the fetchers
+  mkConfig =
+    { sourcesFile ? if builtins.pathExists ./sources.json then ./sources.json else null
+    , sources ? if sourcesFile == null then { } else builtins.fromJSON (builtins.readFile sourcesFile)
+    , system ? builtins.currentSystem
+    , pkgs ? mkPkgs sources system
+    }: {
+      # The sources, i.e. the attribute set of spec name to spec
+      inherit sources;
+
+      # The "pkgs" (evaluated nixpkgs) to use for e.g. non-builtin fetchers
+      inherit pkgs;
+    };
+
+in
+mkSources (mkConfig { }) // { __functor = _: settings: mkSources (mkConfig settings); }

cluster/prod/app/cryptpad/config/application_config.js

@@ -0,0 +1,40 @@
+/*
+ * You can override the configurable values from this file.
+ * The recommended method is to make a copy of this file (/customize.dist/application_config.js)
+   in a 'customize' directory (/customize/application_config.js).
+ * If you want to check all the configurable values, you can open the internal configuration file
+   but you should not change it directly (/common/application_config_internal.js)
+*/
+define(['/common/application_config_internal.js'], function (AppConfig) {
+    // To inform users of the support ticket panel which languages your admins speak:
+    AppConfig.supportLanguages = [ 'en', 'fr' ];
+
+    /* Select the buttons displayed on the main page to create new collaborative sessions.
+     * Removing apps from the list will prevent users from accessing them. They will instead be
+     * redirected to the drive.
+     * You should never remove the drive from this list.
+     */
+    AppConfig.availablePadTypes = ['drive', 'teams', 'doc', 'presentation', 'pad', 'kanban', 'code', 'form', 'poll', 'whiteboard',
+                                'file', 'contacts', 'slide', 'convert'];
+    // disabled: sheet
+
+    /* You can display a link to your own privacy policy in the static pages footer.
+     * Since this is different for each individual or organization there is no default value.
+     * See the comments above for a description of possible configurations.
+     */
+    AppConfig.privacy = {
+      "default": "https://deuxfleurs.fr/CGU.html",
+    };
+
+    /* You can display a link to your instances's terms of service in the static pages footer.
+     * A default is included for backwards compatibility, but we recommend replacing this
+     * with your own terms.
+     *
+     * See the comments above for a description of possible configurations.
+     */
+    AppConfig.terms = {
+      "default": "https://deuxfleurs.fr/CGU.html",
+    };
+
+    return AppConfig;
+});

cluster/prod/app/cryptpad/config/config.js

@@ -0,0 +1,285 @@
+/* globals module */
+
+/*  DISCLAIMER:
+
+    There are two recommended methods of running a CryptPad instance:
+
+    1. Using a standalone nodejs server without HTTPS (suitable for local development)
+    2. Using NGINX to serve static assets and to handle HTTPS for API server's websocket traffic
+
+    We do not officially recommend or support Apache, Docker, Kubernetes, Traefik, or any other configuration.
+    Support requests for such setups should be directed to their authors.
+
+    If you're having difficulty difficulty configuring your instance
+    we suggest that you join the project's IRC/Matrix channel.
+
+    If you don't have any difficulty configuring your instance and you'd like to
+    support us for the work that went into making it pain-free we are quite happy
+    to accept donations via our opencollective page: https://opencollective.com/cryptpad
+
+*/
+module.exports = {
+/*  CryptPad is designed to serve its content over two domains.
+ *  Account passwords and cryptographic content is handled on the 'main' domain,
+ *  while the user interface is loaded on a 'sandbox' domain
+ *  which can only access information which the main domain willingly shares.
+ *
+ *  In the event of an XSS vulnerability in the UI (that's bad)
+ *  this system prevents attackers from gaining access to your account (that's good).
+ *
+ *  Most problems with new instances are related to this system blocking access
+ *  because of incorrectly configured sandboxes. If you only see a white screen
+ *  when you try to load CryptPad, this is probably the cause.
+ *
+ *  PLEASE READ THE FOLLOWING COMMENTS CAREFULLY.
+ *
+ */
+
+/*  httpUnsafeOrigin is the URL that clients will enter to load your instance.
+ *  Any other URL that somehow points to your instance is supposed to be blocked.
+ *  The default provided below assumes you are loading CryptPad from a server
+ *  which is running on the same machine, using port 3000.
+ *
+ *  In a production instance this should be available ONLY over HTTPS
+ *  using the default port for HTTPS (443) ie. https://cryptpad.fr
+ *  In such a case this should be also handled by NGINX, as documented in
+ *  cryptpad/docs/example.nginx.conf (see the $main_domain variable)
+ *
+ */
+    httpUnsafeOrigin: 'https://pad.deuxfleurs.fr',
+
+/*  httpSafeOrigin is the URL that is used for the 'sandbox' described above.
+ *  If you're testing or developing with CryptPad on your local machine then
+ *  it is appropriate to leave this blank. The default behaviour is to serve
+ *  the main domain over port 3000 and to serve the sandbox content over port 3001.
+ *
+ *  This is not appropriate in a production environment where invasive networks
+ *  may filter traffic going over abnormal ports.
+ *  To correctly configure your production instance you must provide a URL
+ *  with a different domain (a subdomain is sufficient).
+ *  It will be used to load the UI in our 'sandbox' system.
+ *
+ *  This value corresponds to the $sandbox_domain variable
+ *  in the example nginx file.
+ *
+ *  Note that in order for the sandboxing system to be effective
+ *  httpSafeOrigin must be different from httpUnsafeOrigin.
+ *
+ *  CUSTOMIZE AND UNCOMMENT THIS FOR PRODUCTION INSTALLATIONS.
+ */
+    httpSafeOrigin: "https://pad-sandbox.deuxfleurs.fr",
+
+/*  httpAddress specifies the address on which the nodejs server
+ *  should be accessible. By default it will listen on 127.0.0.1
+ *  (IPv4 localhost on most systems). If you want it to listen on
+ *  all addresses, including IPv6, set this to '::'.
+ *
+ */
+    httpAddress: '::',
+
+/*  httpPort specifies on which port the nodejs server should listen.
+ *  By default it will serve content over port 3000, which is suitable
+ *  for both local development and for use with the provided nginx example,
+ *  which will proxy websocket traffic to your node server.
+ *
+ */
+    httpPort: 3000,
+
+/*  httpSafePort allows you to specify an alternative port from which
+ *  the node process should serve sandboxed assets. The default value is
+ *  that of your httpPort + 1. You probably don't need to change this.
+ *
+ */
+    // httpSafePort: 3001,
+
+/*  CryptPad will launch a child process for every core available
+ *  in order to perform CPU-intensive tasks in parallel.
+ *  Some host environments may have a very large number of cores available
+ *  or you may want to limit how much computing power CryptPad can take.
+ *  If so, set 'maxWorkers' to a positive integer.
+ */
+    // maxWorkers: 4,
+
+    /* =====================
+     *         Admin
+     * ===================== */
+
+    /*
+     *  CryptPad contains an administration panel. Its access is restricted to specific
+     *  users using the following list.
+     *  To give access to the admin panel to a user account, just add their public signing
+     *  key, which can be found on the settings page for registered users.
+     *  Entries should be strings separated by a comma.
+     */
+    adminKeys: [
+      "[quentin@pad.deuxfleurs.fr/EWtzm-CiqJnM9RZL9mj-YyTgAtX-Zh76sru1K5bFpN8=]",
+      "[adrn@pad.deuxfleurs.fr/PxDpkPwd-jDJWkfWdAzFX7wtnLpnPlBeYZ4MmoEYS6E=]",
+      "[lx@pad.deuxfleurs.fr/FwQzcXywx1FIb83z6COB7c3sHnz8rNSDX1xhjPuH3Fg=]",
+      "[trinity-1686a@pad.deuxfleurs.fr/Pu6Ef03jEsAGBbZI6IOdKd6+5pORD5N51QIYt4-Ys1c=]",
+      "[Jill@pad.deuxfleurs.fr/tLW7W8EVNB2KYETXEaOYR+HmNiBQtZj7u+SOxS3hGmg=]"
+    ],
+
+    /* =====================
+     *        STORAGE
+     * ===================== */
+
+    /*  Pads that are not 'pinned' by any registered user can be set to expire
+     *  after a configurable number of days of inactivity (default 90 days).
+     *  The value can be changed or set to false to remove expiration.
+     *  Expired pads can then be removed using a cron job calling the
+     *  `evict-inactive.js` script with node
+     *
+     *  defaults to 90 days if nothing is provided
+     */
+    //inactiveTime: 90, // days
+
+    /*  CryptPad archives some data instead of deleting it outright.
+     *  This archived data still takes up space and so you'll probably still want to
+     *  remove these files after a brief period.
+     *
+     *  cryptpad/scripts/evict-inactive.js is intended to be run daily
+     *  from a crontab or similar scheduling service.
+     *
+     *  The intent with this feature is to provide a safety net in case of accidental
+     *  deletion. Set this value to the number of days you'd like to retain
+     *  archived data before it's removed permanently.
+     *
+     *  defaults to 15 days if nothing is provided
+     */
+    //archiveRetentionTime: 15,
+
+    /*  It's possible to configure your instance to remove data
+     *  stored on behalf of inactive accounts. Set 'accountRetentionTime'
+     *  to the number of days an account can remain idle before its
+     *  documents and other account data is removed.
+     *
+     *  Leave this value commented out to preserve all data stored
+     *  by user accounts regardless of inactivity.
+     */
+     //accountRetentionTime: 365,
+
+    /*  Starting with CryptPad 3.23.0, the server automatically runs
+     *  the script responsible for removing inactive data according to
+     *  your configured definition of inactivity. Set this value to `true`
+     *  if you prefer not to remove inactive data, or if you prefer to
+     *  do so manually using `scripts/evict-inactive.js`.
+     */
+    //disableIntegratedEviction: true,
+
+
+    /*  Max Upload Size (bytes)
+     *  this sets the maximum size of any one file uploaded to the server.
+     *  anything larger than this size will be rejected
+     *  defaults to 20MB if no value is provided
+     */
+    //maxUploadSize: 20 * 1024 * 1024,
+
+    /*  Users with premium accounts (those with a plan included in their customLimit)
+     *  can benefit from an increased upload size limit. By default they are restricted to the same
+     *  upload size as any other registered user.
+     *
+     */
+    //premiumUploadSize: 100 * 1024 * 1024,
+
+    /* =====================
+     *   DATABASE VOLUMES
+     * ===================== */
+
+    /*
+     *  CryptPad stores each document in an individual file on your hard drive.
+     *  Specify a directory where files should be stored.
+     *  It will be created automatically if it does not already exist.
+     */
+    filePath: '/mnt/datastore/',
+
+    /*  CryptPad offers the ability to archive data for a configurable period
+     *  before deleting it, allowing a means of recovering data in the event
+     *  that it was deleted accidentally.
+     *
+     *  To set the location of this archive directory to a custom value, change
+     *  the path below:
+     */
+    archivePath: '/mnt/data/archive',
+
+    /*  CryptPad allows logged in users to request that particular documents be
+     *  stored by the server indefinitely. This is called 'pinning'.
+     *  Pin requests are stored in a pin-store. The location of this store is
+     *  defined here.
+     */
+    pinPath: '/mnt/data/pins',
+
+    /*  if you would like the list of scheduled tasks to be stored in
+        a custom location, change the path below:
+    */
+    taskPath: '/mnt/data/tasks',
+
+    /*  if you would like users' authenticated blocks to be stored in
+        a custom location, change the path below:
+    */
+    blockPath: '/mnt/block',
+
+    /*  CryptPad allows logged in users to upload encrypted files. Files/blobs
+     *  are stored in a 'blob-store'. Set its location here.
+     */
+    blobPath: '/mnt/blob',
+
+    /*  CryptPad stores incomplete blobs in a 'staging' area until they are
+     *  fully uploaded. Set its location here.
+     */
+    blobStagingPath: '/mnt/data/blobstage',
+
+    decreePath: '/mnt/data/decrees',
+
+    /* CryptPad supports logging events directly to the disk in a 'logs' directory
+     * Set its location here, or set it to false (or nothing) if you'd rather not log
+     */
+    logPath: false,
+
+    /* =====================
+     *       Debugging
+     * ===================== */
+
+    /*  CryptPad can log activity to stdout
+     *  This may be useful for debugging
+     */
+    logToStdout: true,
+
+    /* CryptPad can be configured to log more or less
+     * the various settings are listed below by order of importance
+     *
+     * silly, verbose, debug, feedback, info, warn, error
+     *
+     * Choose the least important level of logging you wish to see.
+     * For example, a 'silly' logLevel will display everything,
+     * while 'info' will display 'info', 'warn', and 'error' logs
+     *
+     * This will affect both logging to the console and the disk.
+     */
+    logLevel: 'silly',
+
+    /*  clients can use the /settings/ app to opt out of usage feedback
+     *  which informs the server of things like how much each app is being
+     *  used, and whether certain clientside features are supported by
+     *  the client's browser. The intent is to provide feedback to the admin
+     *  such that the service can be improved. Enable this with `true`
+     *  and ignore feedback with `false` or by commenting the attribute
+     *
+     *  You will need to set your logLevel to include 'feedback'. Set this
+     *  to false if you'd like to exclude feedback from your logs.
+     */
+    logFeedback: false,
+
+    /*  CryptPad supports verbose logging
+     *  (false by default)
+     */
+    verbose: true,
+
+    /*  Surplus information:
+     *
+     *  'installMethod' is included in server telemetry to voluntarily
+     *  indicate how many instances are using unofficial installation methods
+     *  such as Docker.
+     *
+     */
+    installMethod: 'deuxfleurs.fr',
+};

cluster/prod/app/cryptpad/deploy/cryptpad.hcl

@@ -0,0 +1,78 @@
+job "cryptpad" {
+  datacenters = ["scorpio"]
+  type = "service"
+
+  group "cryptpad" {
+    count = 1
+
+    network {
+      port "http" {
+        to = 3000
+      }
+    }
+
+    restart {
+      attempts = 10
+      delay    = "30s"
+    }
+
+    task "main" {
+      driver = "docker"
+
+      constraint {
+        attribute = "${attr.unique.hostname}"
+        operator = "="
+        value = "abricot"
+      }
+
+      config {
+        image = "kokakiwi/cryptpad:2024.3.0"
+        ports = [ "http" ]
+
+        volumes = [
+          "/mnt/ssd/cryptpad:/mnt",
+          "secrets/config.js:/cryptpad/config.js",
+        ]
+      }
+      env {
+        CRYPTPAD_CONFIG = "/cryptpad/config.js"
+      }
+
+      template {
+        data = file("../config/config.js")
+        destination = "secrets/config.js"
+      }
+
+      /* Disabled because it requires modifications to the docker image and I do not want to invest the time yet
+      template {
+        data = file("../config/application_config.js")
+        destination = "secrets/config.js"
+      }
+      */
+
+      resources {
+        memory = 1000
+        cpu = 500
+      }
+
+      service {
+        name = "cryptpad"
+        port = "http"
+        tags = [
+          "tricot pad.deuxfleurs.fr",
+          "tricot pad-sandbox.deuxfleurs.fr",
+          "tricot-add-header Cross-Origin-Resource-Policy cross-origin",
+          "tricot-add-header Cross-Origin-Embedder-Policy require-corp",
+          "d53-cname pad.deuxfleurs.fr",
+          "d53-cname pad-sandbox.deuxfleurs.fr",
+        ]
+        check {
+          type = "http"
+          path = "/"
+          interval = "10s"
+          timeout = "2s"
+        }
+      }
+    }
+  }
+}

cluster/prod/app/email/build/alps/Dockerfile

@@ -0,0 +1,20 @@
+FROM golang:1.19.3-buster as builder
+
+ARG VERSION
+
+ENV CGO_ENABLED=0 GOOS=linux GOARCH=amd64
+WORKDIR /tmp/alps
+
+RUN git init && \
+    git remote add origin https://git.deuxfleurs.fr/Deuxfleurs/alps.git && \
+    git fetch --depth 1 origin ${VERSION} && \
+    git checkout FETCH_HEAD
+
+RUN go build -a -o /usr/local/bin/alps ./cmd/alps
+
+FROM scratch
+COPY --from=builder /usr/local/bin/alps /alps
+COPY --from=builder /tmp/alps/themes /themes
+COPY --from=builder /tmp/alps/plugins /plugins
+COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
+ENTRYPOINT ["/alps"]

cluster/prod/app/email/build/docker-compose.yml

@@ -0,0 +1,36 @@
+version: '3.4'
+services:
+
+  # Email
+  sogo:
+    build:
+      context: ./sogo
+      args:
+        # fake for now
+        VERSION: 5.0.0
+    image: superboum/amd64_sogo:v7
+
+  alps:
+    build:
+      context: ./alps
+      args:
+        VERSION: bf9ccc6ed17e8b50a230e9f5809d820e9de8562f
+    image: lxpz/amd64_alps:v4
+
+  dovecot:
+    build:
+      context: ./dovecot
+    image: superboum/amd64_dovecot:v6
+
+  postfix:
+    build:
+      context: ./postfix
+      args:
+        # https://packages.debian.org/fr/trixie/postfix
+        VERSION: 3.8.4-1
+    image: superboum/amd64_postfix:v4
+
+  opendkim:
+    build:
+      context: ./opendkim
+    image: superboum/amd64_opendkim:v6

cluster/prod/app/email/build/dovecot/.gitignore

@@ -0,0 +1 @@
+dovecot-ldap.conf

cluster/prod/app/email/build/dovecot/Dockerfile

@@ -0,0 +1,16 @@
+FROM amd64/debian:bullseye
+
+RUN apt-get update && \
+    apt-get install -y \
+      dovecot-antispam \
+      dovecot-core \
+      dovecot-imapd \
+      dovecot-ldap \
+      dovecot-managesieved \
+      dovecot-sieve \
+      dovecot-lmtpd && \
+    rm -rf /etc/dovecot/*
+RUN useradd mailstore
+COPY entrypoint.sh /usr/local/bin/entrypoint
+
+ENTRYPOINT ["/usr/local/bin/entrypoint"]

cluster/prod/app/email/build/dovecot/README.md

@@ -0,0 +1,18 @@
+```
+sudo docker build -t superboum/amd64_dovecot:v2 .
+```
+
+
+```
+sudo docker run -t -i \
+  -e TLSINFO="/C=FR/ST=Bretagne/L=Rennes/O=Deuxfleurs/CN=www.deuxfleurs.fr" \
+  -p 993:993 \
+  -p 143:143 \
+  -p 24:24 \
+  -p 1337:1337 \
+  -v /mnt/glusterfs/email/ssl:/etc/ssl/ \
+  -v /mnt/glusterfs/email/mail:/var/mail \
+  -v `pwd`/dovecot-ldap.conf:/etc/dovecot/dovecot-ldap.conf \
+  superboum/amd64_dovecot:v1 \
+  dovecot -F
+```

cluster/prod/app/email/build/dovecot/entrypoint.sh

@@ -0,0 +1,27 @@
+#!/bin/bash
+
+if [[ ! -f /etc/ssl/certs/dovecot.crt || ! -f /etc/ssl/private/dovecot.key ]]; then
+  cd /root
+  openssl req \
+    -new \
+    -newkey rsa:4096 \
+    -days 3650 \
+    -nodes \
+    -x509 \
+    -subj ${TLSINFO} \
+    -keyout dovecot.key \
+    -out dovecot.crt
+  
+  mkdir -p /etc/ssl/{certs,private}/
+
+  cp dovecot.crt /etc/ssl/certs/dovecot.crt 
+  cp dovecot.key /etc/ssl/private/dovecot.key 
+  chmod 400 /etc/ssl/certs/dovecot.crt 
+  chmod 400 /etc/ssl/private/dovecot.key
+fi
+
+if [[ $(stat -c '%U' /var/mail/) != "mailstore" ]]; then
+  chown -R mailstore /var/mail
+fi
+
+exec "$@"

cluster/prod/app/email/build/dovecot/legacy/all_before.sieve

@@ -0,0 +1,5 @@
+require ["fileinto", "mailbox"];
+if header :contains "X-Spam-Flag" "YES" {
+    fileinto :create "Junk";
+}
+

cluster/prod/app/email/build/dovecot/legacy/dovecot-ldap.sample.conf

@@ -0,0 +1,8 @@
+hosts = ldap.example.com
+dn = cn=admin,dc=example,dc=com
+dnpass = s3cr3t
+base = dc=example,dc=com
+scope = subtree
+user_filter = (&(mail=%u)(&(objectClass=inetOrgPerson)(memberOf=cn=email,ou=groups,dc=example,dc=com)))
+pass_filter = (&(mail=%u)(&(objectClass=inetOrgPerson)(memberOf=cn=email,ou=groups,dc=example,dc=com)))
+user_attrs = mail=/var/mail/%{ldap:mail}

cluster/prod/app/email/build/dovecot/legacy/report-ham.sieve

@@ -0,0 +1,17 @@
+require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables", "vnd.dovecot.debug"];
+
+if environment :matches "imap.mailbox" "*" {
+  set "mailbox" "${1}";
+}
+
+if string "${mailbox}" "Trash" {
+  stop;
+}
+
+if environment :matches "imap.user" "*" {
+  set "username" "${1}";
+}
+
+pipe :copy "sa-learn" [ "--ham", "-u", "debian-spamd" ];
+debug_log "ham reported by ${username}";
+

cluster/prod/app/email/build/dovecot/legacy/report-spam.sieve

@@ -0,0 +1,9 @@
+require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables", "vnd.dovecot.debug"];
+
+if environment :matches "imap.user" "*" {
+  set "username" "${1}";
+}
+
+pipe :copy "sa-learn" [ "--spam", "-u", "debian-spamd"];
+debug_log "spam reported by ${username}";
+

cluster/prod/app/email/build/opendkim/Dockerfile

@@ -0,0 +1,9 @@
+FROM amd64/debian:bullseye
+
+RUN apt-get update && \
+    apt-get dist-upgrade -y && \
+    apt-get install -y opendkim opendkim-tools
+
+COPY ./opendkim.conf /etc/opendkim.conf
+COPY ./entrypoint /entrypoint
+CMD ["/entrypoint"] 

cluster/prod/app/email/build/opendkim/README.md

@@ -0,0 +1,12 @@
+```
+sudo docker build -t superboum/amd64_opendkim:v1 .
+```
+
+```
+sudo docker run -t -i \
+  -v `pwd`/conf:/etc/dkim \
+  -v /dev/log:/dev/log \
+  -p 8999:8999
+  superboum/amd64_opendkim:v1
+  opendkim -f -v -x /etc/opendkim.conf
+```

cluster/prod/app/email/build/opendkim/entrypoint

@@ -0,0 +1,8 @@
+#!/bin/bash
+
+chown 0:0 /etc/dkim/*
+chown 0:0 /etc/dkim
+chmod 400 /etc/dkim/*
+chmod 700 /etc/dkim
+
+opendkim -f -v -x /etc/opendkim.conf

cluster/prod/app/email/build/opendkim/opendkim.conf

@@ -0,0 +1,12 @@
+Syslog			yes
+SyslogSuccess		yes
+LogWhy			yes
+UMask			007
+Mode			sv
+OversignHeaders		From
+TrustAnchorFile       /usr/share/dns/root.key
+KeyTable              refile:/etc/dkim/keytable
+SigningTable          refile:/etc/dkim/signingtable
+ExternalIgnoreList    refile:/etc/dkim/trusted
+InternalHosts	      refile:/etc/dkim/trusted
+Socket                inet:8999

cluster/prod/app/email/build/postfix/Dockerfile

@@ -0,0 +1,13 @@
+FROM amd64/debian:trixie
+
+ARG VERSION
+
+RUN apt-get update && \
+    apt-get install -y \
+      postfix=$VERSION \
+      postfix-ldap
+
+COPY entrypoint.sh /usr/local/bin/entrypoint
+
+ENTRYPOINT ["/usr/local/bin/entrypoint"]
+CMD ["postfix", "start-fg"]

cluster/prod/app/email/build/postfix/README.md

@@ -0,0 +1,18 @@
+```
+sudo docker build -t superboum/amd64_postfix:v1 .
+```
+
+```
+sudo docker run -t -i \
+  -e TLSINFO="/C=FR/ST=Bretagne/L=Rennes/O=Deuxfleurs/CN=smtp.deuxfleurs.fr" \
+  -e MAILNAME="smtp.deuxfleurs.fr" \
+  -p 25:25 \
+  -p 465:465 \
+  -p 587:587 \
+  -v `pwd`/../../ansible/roles/container_conf/files/email/postfix-conf:/etc/postfix-conf \
+  -v /mnt/glusterfs/email/postfix-ssl/private:/etc/ssl/private \
+  -v /mnt/glusterfs/email/postfix-ssl/certs:/etc/ssl/certs \
+  superboum/amd64_postfix:v1 \
+  bash
+```
+

cluster/prod/app/email/build/postfix/entrypoint.sh

@@ -0,0 +1,31 @@
+#!/bin/bash
+
+if [[ ! -f /etc/ssl/certs/postfix.crt || ! -f /etc/ssl/private/postfix.key ]]; then
+  cd /root
+  openssl req \
+    -new \
+    -newkey rsa:4096 \
+    -days 3650 \
+    -nodes \
+    -x509 \
+    -subj ${TLSINFO} \
+    -keyout postfix.key \
+    -out postfix.crt
+  
+  mkdir -p /etc/ssl/{certs,private}/
+
+  cp postfix.crt /etc/ssl/certs/postfix.crt 
+  cp postfix.key /etc/ssl/private/postfix.key 
+  chmod 400 /etc/ssl/certs/postfix.crt 
+  chmod 400 /etc/ssl/private/postfix.key
+fi
+
+# A way to map files inside the postfix folder :s
+for file in $(ls /etc/postfix-conf); do
+  cp /etc/postfix-conf/${file} /etc/postfix/${file}
+done
+
+echo ${MAILNAME} > /etc/mailname
+postmap /etc/postfix/transport
+
+exec "$@"

cluster/prod/app/email/build/sogo/Dockerfile

@@ -0,0 +1,17 @@
+#FROM amd64/debian:stretch as builder
+
+FROM amd64/debian:buster
+
+RUN mkdir ~/.gnupg && echo "disable-ipv6" >> ~/.gnupg/dirmngr.conf
+
+RUN apt-get update && \
+    apt-get install -y apt-transport-https gnupg2 sudo nginx && \
+    rm -rf /etc/nginx/sites-enabled/* && \
+    apt-key adv --keyserver keys.gnupg.net --recv-key 0x810273C4 && \
+    echo "deb http://packages.inverse.ca/SOGo/nightly/5/debian/ buster buster" > /etc/apt/sources.list.d/sogo.list && \
+    apt-get update && \
+    apt-get install -y sogo sogo-activesync sope4.9-gdl1-postgresql postgresql-client
+
+COPY sogo.nginx.conf /etc/nginx/sites-enabled/sogo.conf
+COPY entrypoint /usr/sbin/entrypoint
+ENTRYPOINT ["/usr/sbin/entrypoint"]

cluster/prod/app/email/build/sogo/README.md

@@ -0,0 +1,20 @@
+```
+docker build -t superboum/amd64_sogo:v6 .
+
+# privileged is only for debug
+docker run --rm -ti \
+  --privileged \
+  -p 8080:8080 \
+  -v /tmp/sogo/log:/var/log/sogo \
+  -v /tmp/sogo/run:/var/run/sogo \
+  -v /tmp/sogo/spool:/var/spool/sogo \
+  -v /tmp/sogo/tmp:/tmp \
+  -v `pwd`/sogo:/etc/sogo:ro \
+  superboum/amd64_sogo:v1
+```
+
+Password must be url encoded in sogo.conf for postgres
+Will need a nginx instance: http://wiki.sogo.nu/nginxSettings
+
+Might (or might not) be needed:
+traefik.frontend.headers.customRequestHeaders=x-webobjects-server-port:443||x-webobjects-server-name=sogo.deuxfleurs.fr||x-webobjects-server-url:https://sogo.deuxfleurs.fr

cluster/prod/app/email/build/sogo/entrypoint

@@ -0,0 +1,13 @@
+#!/bin/bash
+mkdir -p /var/log/sogo
+mkdir -p /var/run/sogo
+mkdir -p /var/spool/sogo
+chown sogo /var/log/sogo
+chown sogo /var/run/sogo
+chown sogo /var/spool/sogo
+
+nginx -g 'daemon on; master_process on;'
+sudo -u sogo memcached -d
+sudo -u sogo sogod
+sleep 10
+tail -n200 -f /var/log/sogo/sogo.log

cluster/prod/app/email/build/sogo/sogo.nginx.conf

@@ -0,0 +1,83 @@
+server {
+  listen 8080;
+  server_name default_server; 
+  root /usr/lib/GNUstep/SOGo/WebServerResources/; 
+
+  ## requirement to create new calendars in Thunderbird ##
+  proxy_http_version 1.1;
+
+  # Message size limit
+  client_max_body_size 50m;
+  client_body_buffer_size 128k;
+
+  location = / {
+    rewrite ^ '/SOGo'; 
+    allow all; 
+  }
+
+  location = /principals/ {
+    rewrite ^ '/SOGo/dav'; 
+    allow all; 
+  }
+
+  location ^~/SOGo {
+    proxy_pass 'http://127.0.0.1:20000'; 
+    proxy_redirect 'http://127.0.0.1:20000' default; 
+    # forward user's IP address 
+    proxy_set_header X-Real-IP $remote_addr; 
+    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 
+    proxy_set_header Host $host; 
+    proxy_set_header x-webobjects-server-protocol HTTP/1.0; 
+    proxy_set_header x-webobjects-remote-host 127.0.0.1; 
+    proxy_set_header x-webobjects-server-name $server_name; 
+    proxy_set_header x-webobjects-server-url $scheme://$host; 
+    proxy_set_header x-webobjects-server-port $server_port; 
+    proxy_connect_timeout 90;
+    proxy_send_timeout 90;
+    proxy_read_timeout 90;
+    proxy_buffer_size 4k;
+    proxy_buffers 4 32k;
+    proxy_busy_buffers_size 64k;
+    proxy_temp_file_write_size 64k;
+    break;
+  }
+
+  location /SOGo.woa/WebServerResources/ {
+    alias /usr/lib/GNUstep/SOGo/WebServerResources/;
+    allow all;
+    expires max;
+  }
+
+  location /SOGo/WebServerResources/ {
+    alias /usr/lib/GNUstep/SOGo/WebServerResources/; 
+    allow all;
+    expires max;
+  }
+
+  location (^/SOGo/so/ControlPanel/Products/([^/]*)/Resources/(.*)$) {
+    alias /usr/lib/GNUstep/SOGo/$1.SOGo/Resources/$2;
+    expires max;
+  }
+
+  location (^/SOGo/so/ControlPanel/Products/[^/]*UI/Resources/.*\.(jpg|png|gif|css|js)$) {
+    alias /usr/lib/GNUstep/SOGo/$1.SOGo/Resources/$2;
+    expires max;
+  }
+
+  location ^~ /Microsoft-Server-ActiveSync {
+    access_log /var/log/nginx/activesync.log;
+    error_log  /var/log/nginx/activesync-error.log;
+      
+    proxy_connect_timeout 75;
+    proxy_send_timeout 3600;
+    proxy_read_timeout 3600;
+    proxy_buffers 64 256k;
+      
+    proxy_set_header Host $host;
+    proxy_set_header X-Real-IP $remote_addr;
+    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+      
+    proxy_pass http://127.0.0.1:20000/SOGo/Microsoft-Server-ActiveSync;
+    proxy_redirect http://127.0.0.1:20000/SOGo/Microsoft-Server-ActiveSync /;
+  }
+}

cluster/prod/app/email/config/dkim/keytable

@@ -0,0 +1 @@
+smtp._domainkey.deuxfleurs.fr deuxfleurs.fr:smtp:/etc/dkim/smtp.private 

cluster/prod/app/email/config/dkim/signingtable

@@ -0,0 +1,9 @@
+*@deuxfleurs.fr smtp._domainkey.deuxfleurs.fr
+*@dufour.io smtp._domainkey.deuxfleurs.fr
+*@luxeylab.net smtp._domainkey.deuxfleurs.fr
+*@estherbouquet.com smtp._domainkey.deuxfleurs.fr
+*@pointecouteau.com smtp._domainkey.deuxfleurs.fr
+*@maycausesideeffects.com smtp._domainkey.deuxfleurs.fr
+*@e-x-t-r-a-c-t.me smtp._domainkey.deuxfleurs.fr
+*@courderec.re smtp._domainkey.deuxfleurs.fr
+*@trinity.fr.eu.org smtp._domainkey.deuxfleurs.fr

cluster/prod/app/email/config/dkim/trusted

@@ -0,0 +1,4 @@
+127.0.0.1
+localhost
+192.168.1.0/24
+172.16.0.0/12

cluster/prod/app/email/config/dovecot/dovecot-ldap.conf.tpl

@@ -0,0 +1,12 @@
+hosts = {{ env "meta.site" }}.bottin.service.prod.consul
+dn = {{ key "secrets/email/dovecot/ldap_binddn" | trimSpace }}
+dnpass = {{ key "secrets/email/dovecot/ldap_bindpwd" | trimSpace }}
+base = dc=deuxfleurs,dc=fr
+scope = subtree
+user_filter = (&(mail=%u)(&(objectClass=inetOrgPerson)(memberOf=cn=email,ou=groups,dc=deuxfleurs,dc=fr)))
+pass_filter = (&(mail=%u)(&(objectClass=inetOrgPerson)(memberOf=cn=email,ou=groups,dc=deuxfleurs,dc=fr)))
+user_attrs = \
+  =user=%{ldap:cn}, \
+  =mail=maildir:/var/mail/%{ldap:cn}, \
+  =uid=1000, \
+  =gid=1000 

cluster/prod/app/email/config/dovecot/dovecot.conf

@@ -0,0 +1,87 @@
+auth_mechanisms = plain login
+auth_username_format = %u
+log_timestamp = "%Y-%m-%d %H:%M:%S "
+mail_location = maildir:/var/mail/%u
+mail_privileged_group = mail
+
+log_path = /dev/stderr
+info_log_path = /dev/stdout
+debug_log_path = /dev/stdout
+
+protocols = imap sieve lmtp
+
+ssl_cert = < /etc/ssl/certs/dovecot.crt
+ssl_key = < /etc/ssl/private/dovecot.key
+
+service auth {
+ inet_listener {
+   port = 1337
+ }
+}
+
+
+
+service lmtp {
+  inet_listener lmtp {
+    address = 0.0.0.0
+    port = 24
+  }
+}
+
+# https://doc.dovecot.org/configuration_manual/authentication/ldap_authentication/
+passdb {
+  args = /etc/dovecot/dovecot-ldap.conf
+  driver = ldap
+}
+userdb {
+  driver = prefetch
+}
+userdb {
+  args = /etc/dovecot/dovecot-ldap.conf
+  driver = ldap
+}
+
+
+service imap-login {
+  service_count = 0 # performance mode. set to 1 for secure mode
+  process_min_avail = 1
+  inet_listener imap {
+    port = 143
+  }
+  inet_listener imaps {
+    port = 993
+  }
+}
+
+protocol imap {
+  mail_plugins = $mail_plugins imap_sieve
+}
+
+protocol lda {
+  auth_socket_path = /var/run/dovecot/auth-master
+  info_log_path = /var/log/dovecot-deliver.log
+  log_path = /var/log/dovecot-deliver-errors.log
+  postmaster_address = postmaster@deuxfleurs.fr
+  mail_plugins = $mail_plugins sieve
+}
+
+plugin {
+  sieve = file:~/sieve;active=~/dovecot.sieve
+  sieve_before = /etc/dovecot/all_before.sieve
+
+  # antispam learn
+  sieve_plugins = sieve_imapsieve sieve_extprograms
+  sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment +vnd.dovecot.debug
+  sieve_pipe_bin_dir = /usr/bin
+
+  imapsieve_mailbox1_name = Junk
+  imapsieve_mailbox1_causes = COPY FLAG APPEND
+  imapsieve_mailbox1_before = file:/etc/dovecot/report-spam.sieve
+
+  imapsieve_mailbox2_name = *
+  imapsieve_mailbox2_from = Spam
+  imapsieve_mailbox2_causes = COPY APPEND
+  imapsieve_mailbox2_before = file:/etc/dovecot/report-ham.sieve
+
+}
+

cluster/prod/app/email/config/postfix/dynamicmaps.cf

@@ -0,0 +1,9 @@
+# Postfix dynamic maps configuration file.
+#
+# The first match found is the one that is used.  Wildcards are not supported
+# as of postfix 2.0.2
+#
+#type	location of .so file			open function	(mkmap func)
+#====	================================	=============	============
+ldap	postfix-ldap.so	dict_ldap_open	
+sqlite	postfix-sqlite.so	dict_sqlite_open	

cluster/prod/app/email/config/postfix/header_checks

@@ -0,0 +1,3 @@
+/^Received:/              IGNORE
+/^X-Originating-IP:/    IGNORE
+/^X-Mailer:/            IGNORE

cluster/prod/app/email/config/postfix/ldap-account.cf.tpl

@@ -0,0 +1,12 @@
+bind = yes
+bind_dn = {{ key "secrets/email/postfix/ldap_binddn" | trimSpace }}
+bind_pw = {{ key "secrets/email/postfix/ldap_bindpwd" | trimSpace }}
+version = 3
+timeout = 20
+start_tls = no
+tls_require_cert = no
+server_host = ldap://{{ env "meta.site" }}.bottin.service.prod.consul
+scope = sub
+search_base = ou=users,dc=deuxfleurs,dc=fr
+query_filter = mail=%s
+result_attribute = mail

cluster/prod/app/email/config/postfix/ldap-alias.cf.tpl

@@ -0,0 +1,9 @@
+server_host = {{ env "meta.site" }}.bottin.service.prod.consul
+server_port = 389
+search_base = dc=deuxfleurs,dc=fr
+query_filter = (&(objectClass=inetOrgPerson)(memberOf=cn=%s,ou=mailing_lists,ou=groups,dc=deuxfleurs,dc=fr))
+result_attribute = mail
+bind = yes
+bind_dn = {{ key "secrets/email/postfix/ldap_binddn" | trimSpace }}
+bind_pw = {{ key "secrets/email/postfix/ldap_bindpwd" | trimSpace }}
+version = 3

cluster/prod/app/email/config/postfix/ldap-virtual-domains.cf.tpl

@@ -0,0 +1,12 @@
+bind = yes
+bind_dn = {{ key "secrets/email/postfix/ldap_binddn" | trimSpace }}
+bind_pw = {{ key "secrets/email/postfix/ldap_bindpwd" | trimSpace }}
+version = 3
+timeout = 20
+start_tls = no
+tls_require_cert = no
+server_host = ldap://{{ env "meta.site" }}.bottin.service.prod.consul
+scope = sub
+search_base = ou=domains,ou=groups,dc=deuxfleurs,dc=fr
+query_filter = (&(objectclass=dNSDomain)(domain=%s))
+result_attribute = domain

cluster/prod/app/email/config/postfix/main.cf

@@ -0,0 +1,110 @@
+#===
+# Base configuration
+#===
+myhostname = smtp.deuxfleurs.fr
+alias_maps = hash:/etc/aliases
+alias_database = hash:/etc/aliases
+myorigin = /etc/mailname
+mydestination = smtp.deuxfleurs.fr
+mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 192.168.1.0/24
+mailbox_size_limit = 0
+recipient_delimiter = +
+inet_protocols = all
+inet_interfaces = all
+message_size_limit = 204800000
+smtpd_banner = $myhostname
+biff = no
+append_dot_mydomain = no
+readme_directory = no
+compatibility_level = 2
+
+#===
+# TLS parameters
+#===
+smtpd_tls_cert_file=/etc/ssl/postfix.crt
+smtpd_tls_key_file=/etc/ssl/postfix.key
+smtpd_tls_dh1024_param_file=auto
+smtpd_use_tls=yes
+smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
+smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
+#smtp_tls_policy_maps = hash:/etc/postfix/tls_policy
+smtp_tls_security_level = may
+
+#===
+# Remove privacy related content from emails
+#===
+mime_header_checks = regexp:/etc/postfix/header_checks
+header_checks = regexp:/etc/postfix/header_checks
+
+#===
+# Handle user authentication (handled by dovecot)
+#===
+smtpd_sasl_auth_enable = yes
+smtpd_sasl_path = inet:dovecot-auth.service.prod.consul:1337
+smtpd_sasl_type = dovecot
+
+#===
+# Restrictions / Checks
+#===
+# -- Inspired by: http://www.postfix.org/SMTPD_ACCESS_README.html#lists
+
+# Require a valid HELO
+smtpd_helo_required = yes
+# As we use the same postfix to send and receive,
+# we can't enforce a valid HELO hostname...
+#smtpd_helo_restrictions = 
+#  reject_unknown_helo_hostname
+
+# Require that sender email has a valid domain
+smtpd_sender_restrictions = 
+  reject_unknown_sender_domain 
+
+# Delivering email policy
+# MyNetwork is required by sogo
+smtpd_recipient_restrictions =
+  permit_sasl_authenticated
+  permit_mynetworks
+  reject_unauth_destination
+  reject_rbl_client zen.spamhaus.org
+  reject_rhsbl_reverse_client dbl.spamhaus.org
+  reject_rhsbl_helo dbl.spamhaus.org
+  reject_rhsbl_sender dbl.spamhaus.org
+
+# Sending email policy
+# MyNetwork is required by sogo
+smtpd_relay_restrictions =
+  permit_sasl_authenticated
+  permit_mynetworks
+  reject_unauth_destination
+
+# Disable SMTP smuggling attacks
+# https://www.postfix.org/smtp-smuggling.html
+smtpd_forbid_unauth_pipelining = yes
+smtpd_discard_ehlo_keywords = chunking
+smtpd_forbid_bare_newline = yes
+
+smtpd_client_connection_rate_limit = 2
+
+#===
+# Rate limiting
+#===
+slow_destination_recipient_limit = 20
+slow_destination_concurrency_limit = 2
+
+#====
+# Transport configuration
+#====
+default_transport = smtp-ipv4
+transport_maps = hash:/etc/postfix/transport
+virtual_mailbox_domains = ldap:/etc/postfix/ldap-virtual-domains.cf
+virtual_mailbox_maps = ldap:/etc/postfix/ldap-account.cf
+virtual_alias_maps = ldap:/etc/postfix/ldap-alias.cf
+virtual_transport = lmtp:dovecot-lmtp.service.prod.consul:24
+
+#===
+# Mail filters
+#===
+milter_default_action = accept
+milter_protocol = 6
+smtpd_milters = inet:opendkim.service.prod.consul:8999
+non_smtpd_milters = inet:opendkim.service.prod.consul:8999

cluster/prod/app/email/config/postfix/master.cf

@@ -0,0 +1,117 @@
+#
+# Postfix master process configuration file.  For details on the format
+# of the file, see the master(5) manual page (command: "man 5 master").
+#
+# Do not forget to execute "postfix reload" after editing this file.
+#
+# ==========================================================================
+# service type  private unpriv  chroot  wakeup  maxproc command + args
+#               (yes)   (yes)   (yes)   (never) (100)
+# ==========================================================================
+smtp       inet  n       -       n       -       -       smtpd
+submission inet  n       -       n       -       -       smtpd
+    -o smtpd_tls_security_level=encrypt
+    -o smtpd_sasl_auth_enable=yes
+    -o smtpd_client_restrictions=permit_sasl_authenticated,reject
+    -o milter_macro_daemon_name=ORIGINATING
+smtps      inet  n       -       n       -       -       smtpd
+    -o smtpd_tls_wrappermode=yes
+    -o smtpd_sasl_auth_enable=yes
+    -o smtpd_client_restrictions=permit_sasl_authenticated,reject
+    -o milter_macro_daemon_name=ORIGINATING
+
+#628       inet  n       -       -       -       -       qmqpd
+pickup     fifo  n       -       n       60      1       pickup
+cleanup    unix  n       -       n       -       0       cleanup
+qmgr      fifo  n       -       n       300     1       qmgr
+#qmgr     fifo  n       -       -       300     1       oqmgr
+tlsmgr     unix  -       -       n       1000?   1       tlsmgr
+rewrite    unix  -       -       n       -       -       trivial-rewrite
+bounce     unix  -       -       n       -       0       bounce
+defer      unix  -       -       n       -       0       bounce
+trace      unix  -       -       n       -       0       bounce
+verify     unix  -       -       n       -       1       verify
+flush      unix  n       -       n       1000?   0       flush
+proxymap  unix  -       -       n       -       -       proxymap
+proxywrite unix -       -       n       -       1       proxymap
+# When relaying mail as backup MX, disable fallback_relay to avoid MX loops
+smtp       unix  -       -       n       -       -       smtp
+#       -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
+
+smtp-ipv4  unix  -       -       n       -       -       smtp
+    -o syslog_name=postfix-ipv4
+    -o inet_protocols=ipv4 
+slow  unix       -       -       n       -       5       smtp
+    -o syslog_name=postfix-slow
+    -o smtp_destination_concurrency_limit=3
+    -o slow_destination_rate_delay=1
+relay      unix  -       -       n       -       -       smtp
+    -o smtp_fallback_relay=
+showq      unix  n       -       n       -       -       showq
+error      unix  -       -       n       -       -       error
+retry      unix  -       -       n       -       -       error
+discard    unix  -       -       n       -       -       discard
+local     unix  -       n       n       -       -       local
+virtual   unix  -       n       n       -       -       virtual
+lmtp       unix  -       -       n       -       -       lmtp
+anvil      unix  -       -       n       -       1       anvil
+#
+# ====================================================================
+# Interfaces to non-Postfix software. Be sure to examine the manual
+# pages of the non-Postfix software to find out what options it wants.
+#
+# Many of the following services use the Postfix pipe(8) delivery
+# agent.  See the pipe(8) man page for information about ${recipient}
+# and other message envelope options.
+# ====================================================================
+#
+# maildrop. See the Postfix MAILDROP_README file for details.
+# Also specify in main.cf: maildrop_destination_recipient_limit=1
+#
+scache     unix  -       -       n       -       1       scache
+maildrop  unix  -       n       n       -       -       pipe
+  flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
+#
+# ====================================================================
+#
+# Recent Cyrus versions can use the existing "lmtp" master.cf entry.
+#
+# Specify in cyrus.conf:
+#   lmtp    cmd="lmtpd -a" listen="localhost:lmtp" proto=tcp4
+#
+# Specify in main.cf one or more of the following:
+#  mailbox_transport = lmtp:inet:localhost
+#  virtual_transport = lmtp:inet:localhost
+#
+# ====================================================================
+#
+# Cyrus 2.1.5 (Amos Gouaux)
+# Also specify in main.cf: cyrus_destination_recipient_limit=1
+#
+#cyrus     unix  -       n       n       -       -       pipe
+#  user=cyrus argv=/cyrus/bin/deliver -e -r ${sender} -m ${extension} ${user}
+#
+# ====================================================================
+# Old example of delivery via Cyrus.
+#
+#old-cyrus unix  -       n       n       -       -       pipe
+#  flags=R user=cyrus argv=/cyrus/bin/deliver -e -m ${extension} ${user}
+#
+# ====================================================================
+#
+# See the Postfix UUCP_README file for configuration details.
+#
+uucp      unix  -       n       n       -       -       pipe
+  flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
+#
+# Other external delivery methods.
+#
+ifmail    unix  -       n       n       -       -       pipe
+  flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
+bsmtp     unix  -       n       n       -       -       pipe
+  flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
+scalemail-backend unix	-	n	n	-	2	pipe
+  flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension}
+mailman   unix  -       n       n       -       -       pipe
+  flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
+  ${nexthop} ${user}

cluster/prod/app/email/config/postfix/transport

@@ -0,0 +1,6 @@
+#wanadoo.com slow:
+#wanadoo.fr slow:
+#orange.com slow:
+#orange.fr slow:
+#smtp.orange.fr slow:
+gmail.com smtp-ipv4: 

cluster/prod/app/email/config/sogo/sogo.conf.tpl

@@ -0,0 +1,76 @@
+{
+  WONoDetach = NO;
+  WOWorkersCount = 3;
+  SxVMemLimit = 600;
+  WOPort = "127.0.0.1:20000";
+  SOGoProfileURL = "postgresql://{{ key "secrets/email/sogo/postgre_auth" | trimSpace }}@{{ env "meta.site" }}.psql-proxy.service.prod.consul:5432/sogo/sogo_user_profile";
+  OCSFolderInfoURL = "postgresql://{{ key "secrets/email/sogo/postgre_auth" | trimSpace }}@{{ env "meta.site" }}.psql-proxy.service.prod.consul:5432/sogo/sogo_folder_info";
+  OCSSessionsFolderURL = "postgresql://{{ key "secrets/email/sogo/postgre_auth" | trimSpace }}@{{ env "meta.site" }}.psql-proxy.service.prod.consul:5432/sogo/sogo_sessions_folder";
+  OCSEMailAlarmsFolderURL = "postgresql://{{ key "secrets/email/sogo/postgre_auth" | trimSpace }}@{{ env "meta.site" }}.psql-proxy.service.prod.consul:5432/sogo/sogo_alarms_folder";
+  OCSStoreURL = "postgresql://{{ key "secrets/email/sogo/postgre_auth" | trimSpace }}@{{ env "meta.site" }}.psql-proxy.service.prod.consul:5432/sogo/sogo_store";
+  OCSAclURL = "postgresql://{{ key "secrets/email/sogo/postgre_auth" | trimSpace }}@{{ env "meta.site" }}.psql-proxy.service.prod.consul:5432/sogo/sogo_acl";
+  OCSCacheFolderURL = "postgresql://{{ key "secrets/email/sogo/postgre_auth" | trimSpace }}@{{ env "meta.site" }}.psql-proxy.service.prod.consul:5432/sogo/sogo_cache_folder";
+  SOGoTimeZone = "Europe/Paris";
+  SOGoMailDomain = "deuxfleurs.fr";
+  SOGoLanguage = French;
+  SOGoAppointmentSendEMailNotifications = YES;
+  SOGoEnablePublicAccess = YES;
+  SOGoMailingMechanism = smtp;
+  SOGoSMTPServer = postfix-smtp.service.prod.consul;
+  SOGoSMTPAuthenticationType = PLAIN;
+  SOGoForceExternalLoginWithEmail = YES;
+  SOGoIMAPAclConformsToIMAPExt = YES;
+  SOGoTimeZone = UTC;
+  SOGoSentFolderName = Sent;
+  SOGoTrashFolderName = Trash;
+  SOGoDraftsFolderName = Drafts;
+  SOGoIMAPServer = "imaps://dovecot-imaps.service.prod.consul:993/?tlsVerifyMode=none";
+  SOGoSieveServer = "sieve://sieve.service.prod.consul:4190/?tls=YES";
+  SOGoIMAPAclConformsToIMAPExt = YES;
+  SOGoVacationEnabled = NO;
+  SOGoForwardEnabled = NO;
+  SOGoSieveScriptsEnabled = NO;
+  SOGoFirstDayOfWeek = 1;
+  SOGoRefreshViewCheck = every_5_minutes;
+  SOGoMailAuxiliaryUserAccountsEnabled = NO;
+  SOGoPasswordChangeEnabled = YES;
+  SOGoPageTitle = "deuxfleurs.fr";
+  SOGoLoginModule = Mail;
+  SOGoMailAddOutgoingAddresses = YES;
+  SOGoSelectedAddressBook = autobook;
+  SOGoMailAuxiliaryUserAccountsEnabled = YES;
+  SOGoCalendarEventsDefaultClassification = PRIVATE;
+  SOGoMailReplyPlacement = above;
+  SOGoMailSignaturePlacement = above;
+  SOGoMailComposeMessageType = html;
+
+  SOGoLDAPContactInfoAttribute = "displayname";
+
+  SOGoDebugRequests = YES;
+  //SOGoEASDebugEnabled = YES;
+  //ImapDebugEnabled = YES;
+  LDAPDebugEnabled = YES;
+  //MySQL4DebugEnabled = YES;
+  PGDebugEnabled = YES;
+
+  SOGoUserSources = (
+    {
+        type = ldap;
+        CNFieldName = displayname;
+        IDFieldName = cn;
+        UIDFieldName = cn;
+        MailFieldNames = (mail, mailForwardingAddress);
+        SearchFieldNames = (displayname, cn, sn, mail, telephoneNumber);
+        IMAPLoginFieldName = mail;
+        baseDN = "ou=users,dc=deuxfleurs,dc=fr";
+        bindDN = "{{ key "secrets/email/sogo/ldap_binddn" | trimSpace }}";
+        bindPassword = "{{ key "secrets/email/sogo/ldap_bindpw" | trimSpace}}";
+        bindFields = (cn, mail);
+        canAuthenticate = YES;
+        displayName = "Bottin";
+        hostname = "ldap://{{ env "meta.site" }}.bottin.service.prod.consul:389";
+        id = bottin;
+        isAddressBook = NO;
+    }
+  );
+}

cluster/prod/app/email/deploy/email-android7.hcl

@@ -0,0 +1,126 @@
+job "email-android7" {
+  datacenters = ["neptune", "bespin"]
+  type = "service"
+  priority = 100
+
+  group "rsa-ecc-proxy" {
+    network {
+      port "smtps" {
+        static = 465
+        to = 465
+      }
+      port "imaps" {
+        static = 993
+        to = 993
+      }
+    } 
+    task "imaps-proxy" {
+      driver = "docker"
+      config {
+        image = "alpine/socat:1.7.4.4"
+        readonly_rootfs = true
+        ports = [ "imaps" ]
+        network_mode = "host"
+        args = [
+          "openssl-listen:993,reuseaddr,fork,verify=0,bind=0.0.0.0,cert=/var/secrets/rsa.crt,key=/var/secrets/rsa.key",
+          "openssl:imap.deuxfleurs.fr:993,verify=0",
+        ]
+        volumes = [
+          "secrets/certs:/var/secrets"
+        ]
+      }
+
+      template {
+        data = "{{ key \"secrets/email/tls-tls-proxy/rsa.crt\" }}"
+        destination = "secrets/certs/rsa.crt"
+      }
+      template {
+        data = "{{ key \"secrets/email/tls-tls-proxy/rsa.key\" }}"
+        destination = "secrets/certs/rsa.key"
+      }
+
+      resources {
+        cpu = 50
+        memory = 50
+      }
+
+      service {
+        name = "imap-android7"
+        port = "imaps"
+        address_mode = "host"
+        tags = [
+          "rsa-ecc-proxy",
+          "(diplonat (tcp_port 993))",
+          "d53-a imap-android7.deuxfleurs.fr",
+          # ipv6 is commented for now as socat does not listen on ipv6 now
+          # "d53-aaaa imap-android7.deuxfleurs.fr"
+        ]
+        check {
+          type = "tcp"
+          port = "imaps"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "90s"
+            ignore_warnings = false
+          }
+        }
+      }
+    }
+    task "smtps-proxy" {
+      driver = "docker"
+      config {
+        image = "alpine/socat:1.7.4.4"
+        readonly_rootfs = true
+        network_mode = "host"
+        ports = [ "smtps" ]
+        args = [
+          "openssl-listen:465,reuseaddr,fork,verify=0,bind=0.0.0.0,cert=/var/secrets/rsa.crt,key=/var/secrets/rsa.key",
+          "openssl:smtp.deuxfleurs.fr:465,verify=0",
+        ]
+        volumes = [
+          "secrets/certs:/var/secrets"
+        ]
+      }
+
+      template {
+        data = "{{ key \"secrets/email/tls-tls-proxy/rsa.crt\" }}"
+        destination = "secrets/certs/rsa.crt"
+      }
+      template {
+        data = "{{ key \"secrets/email/tls-tls-proxy/rsa.key\" }}"
+        destination = "secrets/certs/rsa.key"
+      }
+
+      resources {
+        cpu = 50
+        memory = 50
+      }
+
+      service {
+        name = "smtp-android7"
+        port = "smtps"
+        address_mode = "host"
+        tags = [
+          "rsa-ecc-proxy",
+          "(diplonat (tcp_port 465))",
+          "d53-a smtp-android7.deuxfleurs.fr",
+          # ipv6 is commented for now as socat does not listen on ipv6 now
+          # "d53-aaaa smtp-android7.deuxfleurs.fr"
+        ]
+        check {
+          type = "tcp"
+          port = "smtps"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "90s"
+            ignore_warnings = false
+          }
+        }
+      }
+    }
+  }
+}

cluster/prod/app/email/deploy/email.hcl

@@ -0,0 +1,505 @@
+job "email" {
+  datacenters = ["scorpio"]
+  type = "service"
+  priority = 65
+
+  group "dovecot" {
+    count = 1
+
+    network {
+      port "zauthentication_port" {
+        static = 1337
+        to = 1337
+      }
+      port "imaps_port" {
+        static = 993
+        to = 993
+      }
+      port "imap_port" {
+        static = 143
+        to = 143
+      }
+      port "lmtp_port" {
+        static = 24
+        to = 24
+      }
+    }
+
+    task "server" {
+      driver = "docker"
+
+      constraint {
+        attribute = "${attr.unique.hostname}"
+        operator = "="
+        value = "ananas"
+      }
+
+      config {
+        image = "superboum/amd64_dovecot:v6"
+        readonly_rootfs = false
+        network_mode = "host"
+        ports = [ "zauthentication_port", "imaps_port", "imap_port", "lmtp_port" ]
+        command = "dovecot"
+        args = [ "-F" ]
+        volumes = [
+          "secrets/ssl/certs:/etc/ssl/certs",
+          "secrets/ssl/private:/etc/ssl/private",
+          "secrets/conf/:/etc/dovecot/",
+          "/mnt/ssd/mail:/var/mail/",
+        ]
+      }
+
+      env {
+        TLSINFO = "/C=FR/ST=Bretagne/L=Rennes/O=Deuxfleurs/CN=imap.deuxfleurs.fr"
+      }
+
+      resources {
+        cpu = 100
+        memory = 200
+      }
+
+      service {
+        name = "dovecot-imap"
+        port = "imap_port"
+        tags = [
+          "dovecot",
+        ]
+        check {
+          type = "tcp"
+          port = "imap_port"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "90s"
+            ignore_warnings = false
+          }
+        }
+      }
+
+      service {
+        name = "dovecot-imaps"
+        port = "imaps_port"
+        tags = [
+          "dovecot",
+          "(diplonat (tcp_port 993))",
+          "d53-a imap.deuxfleurs.fr",
+          "d53-aaaa imap.deuxfleurs.fr",
+        ]
+
+        check {
+          type = "tcp"
+          port = "imaps_port"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "90s"
+            ignore_warnings = false
+          }
+        }
+      }
+
+      service {
+        name = "dovecot-lmtp"
+        port = "lmtp_port"
+        tags = [
+          "dovecot",
+        ]
+
+        check {
+          type = "tcp"
+          port = "lmtp_port"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "90s"
+            ignore_warnings = false
+          }
+        }
+      }
+
+      service {
+        name = "dovecot-auth"
+        port = "zauthentication_port"
+        tags = [
+          "dovecot",
+        ]
+        check {
+          type = "tcp"
+          port = "zauthentication_port"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "90s"
+            ignore_warnings = false
+          }
+        }
+      }
+
+      template {
+        data = file("../config/dovecot/dovecot-ldap.conf.tpl")
+        destination = "secrets/conf/dovecot-ldap.conf"
+        perms = "400"
+      }
+      template {
+        data = file("../config/dovecot/dovecot.conf")
+        destination = "secrets/conf/dovecot.conf"
+        perms = "400"
+      }
+
+      # ----- secrets ------
+      template {
+        data = "{{ with $d := key \"tricot/certs/imap.deuxfleurs.fr\" | parseJSON }}{{ $d.cert_pem }}{{ end }}"
+        destination = "secrets/ssl/certs/dovecot.crt"
+        perms = "400"
+      }
+      template {
+        data = "{{ with $d := key \"tricot/certs/imap.deuxfleurs.fr\" | parseJSON }}{{ $d.key_pem }}{{ end }}"
+        destination = "secrets/ssl/private/dovecot.key"
+        perms = "400"
+      }
+    }
+  }
+
+  group "opendkim" {
+    count = 1
+
+    network {
+      port "dkim_port" {
+        static = 8999
+        to = 8999
+      }
+    }
+
+    task "server" {
+      driver = "docker"
+      config {
+        image = "superboum/amd64_opendkim:v6"
+        readonly_rootfs = false
+        ports = [ "dkim_port" ]
+        volumes = [
+          "/dev/log:/dev/log",
+          "secrets/dkim:/etc/dkim",
+        ]
+      }
+
+      resources {
+        cpu = 100
+        memory = 50
+      }
+
+      service {
+        name = "opendkim"
+        port = "dkim_port"
+        address_mode = "host"
+        tags = [
+          "opendkim",
+        ]
+        check {
+          type = "tcp"
+          port = "dkim_port"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "90s"
+            ignore_warnings = false
+          }
+        }
+      }
+
+      template {
+        data = file("../config/dkim/keytable")
+        destination = "secrets/dkim/keytable"
+      }
+      template {
+        data = file("../config/dkim/signingtable")
+        destination = "secrets/dkim/signingtable"
+      }
+      template {
+        data = file("../config/dkim/trusted")
+        destination = "secrets/dkim/trusted"
+      }
+
+      # --- secrets ---
+      template {
+        data = "{{ key \"secrets/email/dkim/smtp.private\" }}"
+        destination = "secrets/dkim/smtp.private"
+      }
+    }
+  }
+
+  group "postfix" {
+    count = 1
+
+    network {
+      port "smtp_port" {
+        static = 25
+        to = 25
+      }
+      port "smtps_port" {
+        static = 465
+        to = 465
+      }
+      port "submission_port" {
+        static = 587
+        to = 587
+      }
+    }
+
+    task "server" {
+      driver = "docker"
+      config {
+        image = "superboum/amd64_postfix:v4"
+        readonly_rootfs = false
+        network_mode = "host"
+        ports = [ "smtp_port", "smtps_port", "submission_port" ]
+        command = "postfix"
+        args = [ "start-fg" ]
+        volumes = [
+          "secrets/ssl:/etc/ssl",
+          "secrets/postfix:/etc/postfix-conf",
+          "/dev/log:/dev/log"
+        ]
+      }
+
+      env {
+        TLSINFO = "/C=FR/ST=Bretagne/L=Rennes/O=Deuxfleurs/CN=smtp.deuxfleurs.fr"
+        MAILNAME = "smtp.deuxfleurs.fr"
+      }
+
+      resources {
+        cpu = 100
+        memory = 200
+      }
+
+      service {
+        name = "postfix-smtp"
+        port = "smtp_port"
+        address_mode = "host"
+        tags = [
+          "postfix",
+          "(diplonat (tcp_port 25 465 587))",
+          "d53-a smtp.deuxfleurs.fr",
+          "d53-aaaa smtp.deuxfleurs.fr"
+        ]
+        check {
+          type = "tcp"
+          port = "smtp_port"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "90s"
+            ignore_warnings = false
+          }
+        }
+      }
+
+      service {
+        name = "postfix-smtps"
+        port = "smtps_port"
+        address_mode = "host"
+        tags = [
+          "postfix",
+        ]
+
+        check {
+          type = "tcp"
+          port = "smtps_port"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "90s"
+            ignore_warnings = false
+          }
+        }
+      }
+
+      service {
+        name = "postfix-submission"
+        port = "submission_port"
+        address_mode = "host"
+        tags = [
+          "postfix",
+        ]
+
+        check {
+          type = "tcp"
+          port = "submission_port"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "90s"
+            ignore_warnings = false
+          }
+        }
+      }
+
+      template {
+        data = file("../config/postfix/ldap-account.cf.tpl")
+        destination = "secrets/postfix/ldap-account.cf"
+      }
+
+      template {
+        data = file("../config/postfix/ldap-alias.cf.tpl")
+        destination = "secrets/postfix/ldap-alias.cf"
+      }
+
+      template {
+        data = file("../config/postfix/ldap-virtual-domains.cf.tpl")
+        destination = "secrets/postfix/ldap-virtual-domains.cf"
+      }
+ 
+      template {
+        data = file("../config/postfix/dynamicmaps.cf")
+        destination = "secrets/postfix/dynamicmaps.cf"
+      }
+
+      template {
+        data = file("../config/postfix/header_checks")
+        destination = "secrets/postfix/header_checks"
+      }
+
+      template {
+        data = file("../config/postfix/main.cf")
+        destination = "secrets/postfix/main.cf"
+      }
+
+      template {
+        data = file("../config/postfix/master.cf")
+        destination = "secrets/postfix/master.cf"
+      }
+
+      template {
+        data = file("../config/postfix/transport")
+        destination = "secrets/postfix/transport"
+      }
+
+      # --- secrets ---
+      template {
+        data = "{{ with $d := key \"tricot/certs/smtp.deuxfleurs.fr\" | parseJSON }}{{ $d.cert_pem }}{{ end }}"
+        destination = "secrets/ssl/postfix.crt"
+        perms = "400"
+      }
+
+      template {
+        data = "{{ with $d := key \"tricot/certs/smtp.deuxfleurs.fr\" | parseJSON }}{{ $d.key_pem }}{{ end }}"
+        destination = "secrets/ssl/postfix.key"
+        perms = "400"
+      }
+    }
+  }
+
+  group "alps" {
+    count = 1
+
+    network {
+      port "alps_web_port" { to = 1323 }
+    }
+
+    task "main" {
+      driver = "docker"
+      config {
+        image = "lxpz/amd64_alps:v4"
+        readonly_rootfs = true
+        ports = [ "alps_web_port" ]
+        args = [
+          "-skiptlsverification",
+          "-theme",
+          "alps",
+          "imaps://imap.deuxfleurs.fr:993",
+          "smtps://smtp.deuxfleurs.fr:465"
+        ]
+      }
+      
+      resources {
+        cpu = 100
+        memory = 100
+      }
+      
+      service {
+        name = "alps"
+        port = "alps_web_port"
+        address_mode = "host"
+        tags = [
+          "alps",
+          "tricot alps.deuxfleurs.fr",
+          "d53-cname alps.deuxfleurs.fr",
+        ]
+        check {
+          type = "tcp"
+          port = "alps_web_port"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "5m"
+            ignore_warnings = false
+          }
+        }
+      }
+    }
+  }
+
+
+  group "sogo" {
+    count = 1
+
+    network {
+      port "sogo_web_port" { to = 8080 }
+    }
+
+    task "bundle" {
+      driver = "docker"
+      config {
+        image = "superboum/amd64_sogo:v7"
+        readonly_rootfs = false
+        ports = [ "sogo_web_port" ]
+        volumes = [
+          "secrets/sogo.conf:/etc/sogo/sogo.conf",
+        ]
+      }
+
+      template {
+        data = file("../config/sogo/sogo.conf.tpl")
+        destination = "secrets/sogo.conf"
+      }
+
+      resources {
+        cpu = 400
+        memory = 1500
+        memory_max = 2000
+      }
+      
+      service {
+        name = "sogo"
+        port = "sogo_web_port"
+        address_mode = "host"
+        tags = [
+          "sogo",
+          "tricot www.sogo.deuxfleurs.fr",
+          "tricot sogo.deuxfleurs.fr",
+          "d53-cname sogo.deuxfleurs.fr",
+        ]
+        check {
+          type = "tcp"
+          port = "sogo_web_port"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "5m"
+            ignore_warnings = false
+          }
+        }
+      }
+      
+    }
+  }
+}

cluster/prod/app/email/integration/README.md

@@ -0,0 +1,23 @@
+# Email
+
+## TLS TLS Proxy
+
+Required for Android 7.0 that does not support elliptic curves.
+
+Generate a key:
+
+```bash
+openssl req -x509 -newkey rsa:4096 -sha256 -days 3650 -nodes -keyout rsa.key -out rsa.crt -subj "/CN=imap.deuxfleurs.fr" -addext "subjectAltName=DNS:smtp.deuxfleurs.fr"
+```
+
+Run the command:
+
+```bash
+./integration/proxy.sh imap.deuxfleurs.fr:993 1993
+```
+
+Test it:
+
+```bash
+openssl s_client localhost:1993
+```

cluster/prod/app/email/integration/tls-tls-proxy.sh

@@ -0,0 +1,13 @@
+#!/usr/bin/env bash
+UPSTREAM=$1
+PROXY_PORT=$2
+socat -dd \
+"openssl-listen:${PROXY_PORT},\
+reuseaddr,\
+fork,\
+cert=/tmp/tls-tls-proxy/rsa.crt,\
+key=/tmp/tls-tls-proxy/rsa.key,\
+verify=0,\
+bind=0.0.0.0" \
+"openssl:${UPSTREAM},\
+verify=0"

cluster/prod/app/email/secrets.toml

@@ -0,0 +1,32 @@
+# ---- POSTFIX ----
+
+[secrets."email/dkim/smtp.private"]
+type = 'RSA_PRIVATE_KEY'
+name = 'dkim'
+
+# ---- DOVECOT ----
+
+[service_users."dovecot"]
+dn_secret = "email/dovecot/ldap_binddn"
+password_secret = "email/dovecot/ldap_bindpwd"
+
+
+# ---- SOGO ----
+
+[service_users."sogo"]
+dn_secret = "email/sogo/ldap_binddn"
+password_secret = "email/sogo/ldap_bindpw"
+
+[secrets."email/sogo/postgre_auth"]
+type = 'user'
+description = 'SoGo postgres auth (format: sogo:<password>) (TODO: replace this with two separate files and change template)'
+
+# ---- TLS TLS PROXY ---
+
+[secrets."email/tls-tls-proxy/rsa.crt"]
+type="user"
+description="PEM encoded file containing the RSA certificate"
+
+[secrets."email/tls-tls-proxy/rsa.key"]
+type="user"
+description="PEM encoded file containing the RSA key"

cluster/prod/app/garage/config/garage.toml

@@ -0,0 +1,47 @@
+block_size = 1048576
+
+metadata_dir = "/meta"
+data_dir = "/data"
+db_engine = "lmdb"
+
+replication_mode = "3"
+
+metadata_auto_snapshot_interval = "24h"
+
+# IPv6 config using the ipv6 address statically defined in Nomad's node metadata
+# make sure to put back double { and } if re-enabling this
+#rpc_bind_addr = "[{ env "meta.public_ipv6" }]:3901"
+#rpc_public_addr = "[{ env "meta.public_ipv6" }]:3901"
+
+# IPv6 config using the ipv6 address dynamically detected from diplonat
+{{ with $a := env "attr.unique.hostname" | printf "diplonat/autodiscovery/ipv6/%s" | key | parseJSON }}
+rpc_bind_addr = "[{{ $a.address }}]:3901"
+rpc_public_addr = "[{{ $a.address }}]:3901"
+{{ end }}
+
+rpc_secret = "{{ key "secrets/garage/rpc_secret" | trimSpace }}"
+
+[consul_discovery]
+consul_http_addr = "https://consul.service.prod.consul:8501"
+service_name = "garage-prod-discovery"
+ca_cert = "/etc/garage/consul-ca.crt"
+client_cert = "/etc/garage/consul-client.crt"
+client_key = "/etc/garage/consul-client.key"
+tls_skip_verify = true
+
+[s3_api]
+s3_region = "garage"
+api_bind_addr = "[::]:3900"
+root_domain = ".garage.deuxfleurs.fr"
+
+[k2v_api]
+api_bind_addr = "[::]:3904"
+
+[s3_web]
+bind_addr = "[::]:3902"
+root_domain = ".web.deuxfleurs.fr"
+
+[admin]
+api_bind_addr = "[::]:3903"
+metrics_token = "{{ key "secrets/garage/metrics_token" | trimSpace }}"
+admin_token = "{{ key "secrets/garage/admin_token" | trimSpace }}"

cluster/prod/app/garage/deploy/garage.hcl

@@ -0,0 +1,221 @@
+job "garage" {
+  datacenters = [ "neptune", "bespin", "scorpio" ]
+  type = "system"
+  priority = 80
+
+  update {
+    max_parallel = 2
+    min_healthy_time  = "60s"
+  }
+
+  group "garage" {
+    network {
+      port "s3" { static = 3900 }
+      port "rpc" { static = 3901 }
+      port "web" { static = 3902 }
+      port "admin" { static = 3903 }
+      port "k2v" { static = 3904 }
+    }
+
+    update {
+      max_parallel = 10
+      min_healthy_time = "30s"
+      healthy_deadline = "5m"
+    }
+
+    task "server" {
+      driver = "docker"
+      config {
+        image = "dxflrs/garage:v1.0.0-rc1"
+        command = "/garage"
+        args = [ "server" ]
+        network_mode = "host"
+        volumes = [
+          "/mnt/storage/garage/data:/data",
+          "/mnt/ssd/garage/meta:/meta",
+          "secrets/garage.toml:/etc/garage.toml",
+          "secrets:/etc/garage",
+        ]
+        logging {
+          type = "journald"
+        }
+      }
+
+      template {
+        data = file("../config/garage.toml")
+        destination = "secrets/garage.toml"
+        #change_mode = "noop"
+      }
+
+      template {
+        data = "{{ key \"secrets/consul/consul-ca.crt\" }}"
+        destination = "secrets/consul-ca.crt"
+      }
+
+      template {
+        data = "{{ key \"secrets/consul/consul-client.crt\" }}"
+        destination = "secrets/consul-client.crt"
+      }
+
+      template {
+        data = "{{ key \"secrets/consul/consul-client.key\" }}"
+        destination = "secrets/consul-client.key"
+      }
+
+      resources {
+        memory = 1000
+        memory_max = 3000
+        cpu = 1000
+      }
+
+      kill_timeout = "20s"
+
+      restart {
+        interval = "30m"
+        attempts = 10
+        delay    = "15s"
+        mode     = "delay"
+      }
+
+      #### Configuration for service ports: admin port (internal use only)
+
+      service {
+        port = "admin"
+        address_mode = "host"
+        name = "garage-admin"
+        # Check that Garage is alive and answering TCP connections
+        check {
+          type = "tcp"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "90s"
+            ignore_warnings = false
+          }
+        }
+      }
+
+      #### Configuration for service ports: externally available ports (API, web)
+
+      service {
+        tags = [
+          "garage_api",
+          "tricot garage.deuxfleurs.fr",
+          "tricot *.garage.deuxfleurs.fr",
+          "tricot-site-lb",
+        ]
+        port = "s3"
+        address_mode = "host"
+        name = "garage-api"
+        # Check 1: Garage is alive and answering TCP connections
+        check {
+          name = "garage-api-live"
+          type = "tcp"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "90s"
+            ignore_warnings = false
+          }
+        }
+        # Check 2: Garage is in a healthy state and requests should be routed here
+        check {
+          name = "garage-api-healthy"
+          port = "admin"
+          type = "http"
+          path = "/health"
+          interval = "60s"
+          timeout = "5s"
+        }
+      }
+
+      service {
+        tags = [
+            "garage-web",
+            "tricot * 1",
+            "tricot-add-header Strict-Transport-Security max-age=63072000; includeSubDomains; preload",
+            "tricot-add-header X-Frame-Options SAMEORIGIN",
+            "tricot-add-header X-XSS-Protection 1; mode=block",
+            "tricot-add-header X-Content-Type-Options nosniff",
+            "tricot-on-demand-tls-ask http://garage-admin.service.prod.consul:3903/check",
+            "tricot-site-lb",
+        ]
+        port = "web"
+        address_mode = "host"
+        name = "garage-web"
+        # Check 1: Garage is alive and answering TCP connections
+        check {
+          name = "garage-web-live"
+          type = "tcp"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "90s"
+            ignore_warnings = false
+          }
+        }
+        # Check 2: Garage is in a healthy state and requests should be routed here
+        check {
+          name = "garage-web-healthy"
+          port = "admin"
+          type = "http"
+          path = "/health"
+          interval = "60s"
+          timeout = "5s"
+        }
+      }
+
+      service {
+        tags = [
+          "garage-redirect-dummy",
+          "tricot www.deuxfleurs.fr 2",
+          "tricot osuny.org 2",
+          "tricot www.degrowth.net 2",
+          "tricot-add-redirect www.deuxfleurs.fr deuxfleurs.fr 301",
+          "tricot-add-redirect osuny.org www.osuny.org 301",
+          "tricot-add-redirect www.degrowth.net degrowth.net 301",
+        ]
+        name = "garage-redirect-dummy"
+        address_mode = "host"
+        port = "web"
+        on_update = "ignore"
+      }
+          
+
+      service {
+        tags = [
+          "garage_k2v",
+          "tricot k2v.deuxfleurs.fr",
+          "tricot-site-lb",
+        ]
+        port = "k2v"
+        address_mode = "host"
+        name = "garage-k2v"
+        # Check 1: Garage is alive and answering TCP connections
+        check {
+          name = "garage-k2v-live"
+          type = "tcp"
+          interval = "60s"
+          timeout = "5s"
+          check_restart {
+            limit = 3
+            grace = "90s"
+            ignore_warnings = false
+          }
+        }
+        # Check 2: Garage is in a healthy state and requests should be routed here
+        check {
+          name = "garage-k2v-healthy"
+          port = "admin"
+          type = "http"
+          path = "/health"
+          interval = "60s"
+          timeout = "5s"
+        }
+      }
+    }
+  }
+}

cluster/prod/app/garage/secrets.toml

@@ -0,0 +1,14 @@
+[secrets."garage/rpc_secret"]
+type = 'command'
+command = 'openssl rand -hex 32'
+# can't auto-rotate, because we still have some nodes outside of Nomad
+
+[secrets."garage/admin_token"]
+type = 'command'
+command = 'openssl rand -hex 32'
+rotate = true
+
+[secrets."garage/metrics_token"]
+type = 'command'
+command = 'openssl rand -hex 32'
+rotate = true

cluster/prod/app/guichet/config/guichet/config.json.tpl

@@ -0,0 +1,40 @@
+{
+  "http_bind_addr": ":9991",
+  "ldap_server_addr": "ldap://{{ env "meta.site" }}.bottin.service.prod.consul:389",
+
+  "base_dn": "{{ key "secrets/directory/ldap_base_dn" }}",
+  "user_base_dn": "ou=users,{{ key "secrets/directory/ldap_base_dn" }}",
+  "user_name_attr": "cn",
+  "group_base_dn": "ou=groups,{{ key "secrets/directory/ldap_base_dn" }}",
+  "group_name_attr": "cn",
+  "mailing_list_base_dn": "ou=mailing_lists,ou=groups,{{ key "secrets/directory/ldap_base_dn" }}",
+  "mailing_list_name_attr": "cn",
+  "mailing_list_guest_user_base_dn": "ou=guests,ou=users,{{ key "secrets/directory/ldap_base_dn" }}",
+
+  "invitation_base_dn": "ou=invitations,{{ key "secrets/directory/ldap_base_dn" }}",
+  "invitation_name_attr": "cn",
+  "invited_mail_format": "{}@{{ key "secrets/directory/guichet/mail_domain" | trimSpace }}",
+  "invited_auto_groups": [
+    "cn=email,ou=groups,{{ key "secrets/directory/ldap_base_dn" }}"
+  ],
+
+  "web_address": "https://{{ key "secrets/directory/guichet/web_hostname" }}",
+  "mail_from": "{{ key "secrets/directory/guichet/mail_from" }}",
+  "smtp_server": "{{ key "secrets/directory/guichet/smtp_server" }}",
+  "smtp_username": "{{ key "secrets/directory/guichet/smtp_user" | trimSpace }}",
+  "smtp_password": "{{ key "secrets/directory/guichet/smtp_pass" | trimSpace }}",
+
+  "admin_account": "cn=admin,{{ key "secrets/directory/ldap_base_dn" }}",
+  "group_can_admin": "cn=admin,ou=groups,{{ key "secrets/directory/ldap_base_dn" }}",
+  "group_can_invite": "cn=asso_deuxfleurs,ou=groups,{{ key "secrets/directory/ldap_base_dn" }}",
+
+  "s3_admin_endpoint": "garage-admin.service.prod.consul:3903",
+  "s3_admin_token": "{{ key "secrets/garage/admin_token" | trimSpace }}",
+
+  "s3_endpoint": "{{ key "secrets/directory/guichet/s3_endpoint" }}",
+  "s3_access_key": "{{ key "secrets/directory/guichet/s3_access_key" | trimSpace }}",
+  "s3_secret_key": "{{ key "secrets/directory/guichet/s3_secret_key" | trimSpace }}",
+  "s3_region": "{{ key "secrets/directory/guichet/s3_region" }}",
+  "s3_bucket": "{{ key "secrets/directory/guichet/s3_bucket" }}"
+}
+