quickstart + wip cookbook

content/documentation/cookbook/_index.md

@@ -5,4 +5,23 @@ sort_by = "weight"
 template = "documentation.html"
 +++
 
-*Not ready for production, nothing to see here...*
+## Minimal deployment
+ - Static user management
+ - Single-node Garage storage
+ - TLS encryption
+ - Integration with a service manager
+ - SMTP server integration (MTA)
+
+## Standard deployment
+ - LDAP user management
+ - Garage cluster storage
+ - Integration in an orchestrator
+ - Auto-discovery
+
+## Lifecycle
+ - Updates
+ - Observability
+ - Backups
+
+## Hardened flavor
+ - Manual configuration

content/documentation/quick-start/_index.md

@@ -5,11 +5,30 @@ sort_by = "weight"
 template = "documentation.html"
 +++
 
-*This quick start guide is specifically written with docker and docker-compose to make it more tangible. But nothing prevent your from deploying Aerogramme in production without docker or on other OS like FreeBSD.*
+This quickstart guide will teach you how to run an in-memory instance of Aerogramme,
+because it is easier to run than a distributed real-world deployment. The goal here
+is that you get familiar with the software, you understand what it does and what it does not,
+so you can evaluate it. Once ready for a real deployment, you will find all the information you need in the cookbook.
 
-## Pull an image
+*Enjoy your reading!*
 
-Go to the download page, select a version and follow the instruction steps:
+## What is Aerogramme
+
+Aerogramme is a [Message Delivery Agent (MDA)](https://en.wikipedia.org/wiki/Message_delivery_agent) that speaks the [IMAP protocol](https://en.wikipedia.org/wiki/Internet_Message_Access_Protocol).
+It makes sure that your mailbox, and thus your emails, are stored safely and stay in sync across all your devices.
+<!--It manages a bit more than just your emails, as contacts and calendars support is planned.-->
+
+Aerogramme is not (yet) a [Message Transfer Agent (MTA)](https://en.wikipedia.org/wiki/Message_transfer_agent) as it does not speak the [SMTP protocol](https://fr.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol).
+It means you can't send email for example with Aerogramme alone, you need to plug it to a MTA, like [Postfix](https://www.postfix.org/) or [opensmtpd](https://www.opensmtpd.org/).
+The connection between Aerogramme and the MTA (Postix, opensmtpd, etc.) is done through a protocol named [LMTP](https://en.wikipedia.org/wiki/Local_Mail_Transfer_Protocol).
+
+## Get a binary
+
+Aerogramme is shipped as a static binary for Linux only on `amd64`, `aarch64` and `armv6` architectures.
+For convenience, a docker container is also published both on Docker Hub and on Deuxfleurs' registry.
+Also, Aerogramme is an open-source software released under the [EUPL licence](https://commission.europa.eu/content/european-union-public-licence_en), and you can compile it yourself from the source.
+
+To get the latest version of Aerogramme and the precise download information, both for binary and source releases, go to the dedicated download page:
 
 <a
 href="/download/"
@@ -19,192 +38,149 @@ class="group flex items-center justify-center space-x-1 font-semibold shadow hov
 <span class="hidden md:inline text-white">Download</span>
 </a>
 
-Do the same for [Garage](https://garagehq.deuxfleurs.fr/download/), as Aerogramme can't run without Garage.
+To check if the binary you got is working, just launch it without any parameter, and an help message should be displayed like this one:
 
-## Prepare your environment
+```text
+aerogramme 0.2.0
+Alex Auvolat <alex@adnab.me>, Quentin Dufour <quentin@dufour.io>
+A robust email server
 
-Create a deployment folder:
+USAGE:
+    aerogramme [OPTIONS] <SUBCOMMAND>
 
-```bash
+...
-mkdir /tmp/mailstack && cd /tmp/mailstack
 ```
 
-*`/tmp` is used as the quick start guide assume you want to evaluate Aerogramme, not deploy it in production. Follow the cookbook for a production deployment.*
+## Starting the daemon in development mode
 
-Add a `docker-compose.yml` file inside it:
+To start an in-memory daemon for evaluation purposes, you don't need any configuration file. Just run:
-
-```yml
-version: "3.9"
-services:
-  aerogramme:
-    image: registry.deuxfleurs.org/aerogramme:{{TAG}}
-    volumes: 
-      - ./aerogramme.toml:/etc/aerogramme.toml
-    ports:
-      - 1143:1143 # IMAP
-      - 2424:2424 # LMTP
-  garage:
-    image: docker.io/dxflrs/garage:{{TAG}}
-    volumes: 
-      - ./garage.toml:/etc/garage.toml
-```
-
-Create empty configuration files for now (they will be filled later but docker requires them now):
 
 ```bash
-touch aerogramme.toml garage.toml
+aerogramme --dev provider daemon
 ```
 
-*Do not forget to replace `{{TAG}}` by the tag you chose!*
+Now Aerogramme is listening for IMAP on `[::1]:1143` and for LMTP on port `[::1]:1025`. A test account has been created with username `alice` and password `hunter2`.
 
-## Setup Garage
+<!--
+## Checking that the service is listening as intended
+
+*If you don't have socat installed on your computer or are not comfortable with interacting
+with low-level tools, you can simply skip this section.*
+
+You can try to connect on both ports with `socat` and see a welcome message from the server (CTRL + D to exit):
+
+```
+$ socat - tcp:localhost:1143,crlf
+* OK [CAPABILITY MOVE UIDPLUS LIST-STATUS ENABLE CONDSTORE LITERAL+ IDLE IMAP4REV1 UNSELECT] Aerogramme
+
+$ socat - tcp:localhost:1025,crlf
+220 example.tld Service ready
+```
+
+If you got these outputs, well done, your daemon is correctly started and is working!
+-->
+
+## Connecting Thunderbird
+
+Now that we have Aerogramme acting as a Mail Delivery Agent, we want to connect our devices to it.
+For this quick start guide, we will use Thunderbird on desktop. On the create account page,
+start filling the fields.
+
+**Your full name:** `Alice`  
+**Email address:** `alice@example.tld`  
+**Password:** `hunter2`  
+
+Then click on **Configure Manually**. New fields will appear, fill them as follow.
+
+**Protocol:** `IMAP`  
+**Hostname:** `::1`  
+**Port:** `1143`  
+**Connection Security:** `None`  
+**Authentication Method:** `Normal password`  
+**Username:** `alice`  
+
+*Aerogramme is not a SMTP server but Thunderbird will not let you configure an account without a SMTP server.
+You can lie to Thunderbird by putting any existing SMTP server (like `smtp.gmail.com`) and choose "No Authentication" for the Authentication method field.*
+
+A screenshot for reference:
+
+![Thunderbird Account Creation Page filled with Aerogramme Quickstart configuration](/images/thunderbird-quickstart.png)
+
+Clic "Done" (or "Re-test" then "Done) and your account should be configured:
+
+![Overview of the mailbox list in Thunderbird](/images/thunderbird-quickstart-mbx.png)
+
+As you can see, Thunderbird detected multiple mailboxes but they are all empty.
+You can copy emails from your other mailbox to fill your Aerogramme one, test search, add/remove flags, and so on.
+Mail reception will be emulated in the following section.
+
+## Emulate the reception of an email
+
+Remember, Aerogramme is about managing your mailbox, not managing mail transfer.
+As this is an Aerogramme specific, the goal here is not to deploy Postfix, opensmtpd or any other MTA software.
+
+Instead we will emulate their action: forwarding the messages they received to Aerogramme through the LMTP protocol.
+Python natively supports the LMTP protocol, the following command this capability to deliver a fake email:
 
-Create a basic configuration file for Garage:
 
 ```bash
-cat > garage.toml <<EOF
+python - <<EOF
-metadata_dir = "/tmp/meta"
+from smtplib import LMTP
-data_dir = "/tmp/data"
+with LMTP("::1", port=1025) as lmtp:
-replication_mode = "none"
+    lmtp.set_debuglevel(2)
-rpc_bind_addr = "[::]:3901"
+    lmtp.sendmail(
-rpc_secret = "$(openssl rand -hex 32)"
+        "bob@example.tld",
-[s3_api]
+        "alice@example.tld",
-s3_region = "garage"
+        "From: bob@example.tld\r\nTo: alice@example.tld\r\nSubject: Hello\r\n\r\nWorld\r\n",
-api_bind_addr = "[::]:3900"
+    )
-[k2v_api]
-api_bind_addr = "[::]:3904"
 EOF
 ```
 
-Start Garage:
+After running three times this script and clicking on 2 of these 3 emails on Thunderbird,
+your mailbox should look like that:
 
-```bash
+![Overview of Thunderbird inbox](/images/thunderbird-quickstart-emails.png)
-docker compose up -d garage
-```
 
-Configure Garage:
+## That's all folks!
 
-```bash
+That's (already) the end of the quickstart section of Aerogramme.
-alias garage="docker compose exec garage -ti /garage"
-garage layout assign -z dc1 -c 1 -t aero -- $(garage node id -q)
-garage layout show
-garage layout apply --version 1 # version number can change
-garage key new --name aerogramme-global-key 
-# note the "Key ID" and the "Secret Key"
-garage bucket create aerogramme
-garage bucket allow --read --write --key aerogramme-global-key aerogramme
-```
 
-*If you are struggling with Garage configuration, maybe start with [Garage's own quick start](https://garagehq.deuxfleurs.fr/documentation/quick-start/).*
+You have learnt:
+ - What Aerogramme does
+ - How to download Aerogramme
+ - How to run Aerogramme
+ - How to connect Thunderbird to Aerogramme
+ - How MTA are transferring emails to Aerogramme
 
-## Setup Aerogramme
+If you are considering a real-world deployment now, you will
+have many topics to consider, like integration with existing MTA, integration with a service manager, TLS encryption,
+user management, auto-discovery, persistent storage, high-availability design, observability, backup, updates, etc.
+All these points will be covered in the [Cookbook](@/documentation/cookbook/_index.md), it should be your next stop!
 
-Aerogramme also has a configuration file:
+<!--
+## Aerogramme's little secret: per-user mailbox encryption
 
-```toml
+Aerogramme can't store plaintext data, instead all users data must be encrypted with a per-user key on the storage target.
-s3_endpoint = "http://garage:3900" # use docker addressing
+Of course, cryptography is always a tradeoff with other properties (usability, compatibility, features, etc.),
-k2v_endpoint = "http://garage:3904" # same
+so the way the key is derived and where the encryption/decryption can take place can be configured.
-aws_region = "garage"
+These different configurations are identified as flavors:
 
-[lmtp]
+**Transparent Flavor** - Users' data are decrypted in the server's RAM. The key is derived from the user's password upon IMAP login.
-bind_addr = "[::]:2424"
+It's completely transparent to end users. While an attacker having full access to the server can still compromise users' data, it reduces the attack surface. 
-hostname = "aerogramme.tld"
+In term of security, it's similar to [TREES](https://0xacab.org/liberate/trees) (from [RiseUp](https://riseup.net)) or [scrambler](https://github.com/posteo/scrambler-plugin) (from [Posteo](https://posteo.de/)).
-
-[imap]
-bind_addr = "[::]:1143"
-
-[login_static]
-global_bucket = "aerogramme" # bucket created earlier
-global_aws_access_key_id = "GK..." # shown when key "aerogramme-global-key" was created
-global_aws_secret_access_key = "..." # same
-
-[login_static.users]
-# empty currently
-```
-
-Start the server as follow:
-
-```bash
-docker compose up -d aerogramme
-```
 
 
-## Create a static user
+**Hardened Flavor** - Users' data are decrypted on the user's device. The private key is only stored on the user's device.
+The server knows only users' public keys. When an email is received by the server, it is directly encrypted and stored by the server.
+The user must run Aerogramme locally as a proxy that will manipulate the encrypted blobs stored in the server
+and decrypt them locally, exposing an IMAP proxy interface. An attacker having full access to the server at a point in time
+will not be able to compromise your already received data (but can intercept new emails). It's similar to [Proton Mail Bridge](https://proton.me/fr/mail/bridge),
+but keep in mind that Aerogramme does not support (yet) end-to-end email encryption like Proton Mail or Tutanota, *so Aerogramme is less secure*.
 
-Let's create an alias for aerogramme too:
+When run on server (both for the transparent and hardened flavor), Aerogramme must be started in the "provider mode", as in "email service provider".
+When run on the end-user device (only the hardened flavor require that), Aerogramme must be started in the "companion mode", as in "a companion process of your email client".
+These 2 words are materialized as 2 subcommands on the Aerogramme binary: `aerogramme provider` and `aerogramme companion`.
 
-```bash
+*Earlier, we started Aerogramme in the provider mode with a test account configured with the transparent flavor.*
-alias aerogramme="docker compose exec -ti aerogramme /aerogramme"
-aerogramme
-```
 
-You must start by creating a user profile in Garage. Run the following command after adjusting the parameters to your configuration:
+-->
-
-```bash
-aerogramme first-login --static alice
-```
-
-The program will interactively ask you some questions and finally generates for you a snippet of configuration:
-
-```
-Append the following section to your .toml configuration file:
-
-[login_static.users.alice]
-password = "$argon2id$v=19$m=4096,t=3,p=1$..."
-user_secret = "..."
-```
-
-*Note: user-secret is not the user's password. It is an additional secret used when deriving user's secret key from their password. The idea is that, even if user leaks their password, their encrypted data remain safe as long as this additional secret does not leak. You can generate it with openssl for example: `openssl rand -base64 30`. Read [Cryptography & key management](./crypt-key.md) for more details.*
-
-Restart the server to load the new configuration:
-
-```bash
-docker compose up -d --force-recreate aerogramme
-```
-
-## Validate
-
-Now that your Aerogramme server is running, 
-
-### Inject emails
-
-Inject test emails:
-
-```bash
-./test/inject_emails.sh '<me@aerogramme.tld>' dxflrs
-```
-
-*replace with netcat*
-
-### Check with netcat
-
-*Todo*
-
-### Check with mutt
-
-Now you can connect your mailbox with `mutt`.
-Start by creating a config file, for example we used the following `~/.muttrc` file:
-
-```ini
-set imap_user = quentin
-set imap_pass = p455w0rd
-set folder = imap://localhost:1993
-set spoolfile = +INBOX
-set ssl_starttls = no
-set ssl_force_tls = no
-mailboxes = +INBOX
-bind index G imap-fetch-mail
-```
-
-And then simply launch `mutt`.
-The first time nothing will happen as Aerogramme must
-process your incoming emails. Just ask `mutt` to refresh its
-view by pressing `G` (for *Get*).
-
-Now, you should see some emails:
-
-![Screenshot of mutt mailbox](/screenshots/mutt_mb.png)
-
-And you can read them:
-
-![Screenshot of mutt mail view](/screenshots/mutt_mail.png)