automation/deployer/README.md

# Deployer: deploy your shit and make it run 

So lame to have to configure nginx, MySQL, and your filesystem to install a stupid Wordpress instance.

**Deployer** does my config for me like the slave it is. 

All the configuration is defined in `group_vars/all/vars.yml`, go check.

Create a side `group_vars/all/vault.yml` for your secrets, and encrypt it with Ansible Vault:

```bash
ansible-vault encrypt group_vars/all/vault.yml
# other sub-commands: edit, decrypt...
```

I usually run the following command:
	
	ansible-playbook --ask-vault-pass sites.yml -i inventory -v

## Required packages on remote

Python modules:

* docker 
* docker-compose 
* pymysql 
* psycopg2

TODO: Ansible task to install that before the rest

## Features

* Creating Wordpress instances (yoohoo, da best)

	* That send mail!!11!1!
	* Supports existing and new installs

* Creating Drupal instances 

	* Only existing ones (no new installs)

* Create Gitea instances 

	* Nginx and docker-compose configurations
	* Most of the work is by hand, because there is quite a lot of interaction between the host and the container (for forwarding ssh). 

* Create Synapse instances 

	* Configured to access PostGreSQL on host. 

		* *Access through TCP*: You need to allow postgres to listen to your docker network, e.g. `172.27.0.0/16`. See `/etc/postgresql/x.y/main/pg_hba.conf`, and read the comments about changing `listen_addresses` too.
		* Access through Unix socket: Make a non-superuser role for root, and configure Synapse to use `/var/run/postgresql` as DB host.

### Does not support

* Setting up the host

* SSL certificate creation (bro, do it yourself!). That is:

	```bash
	# Make an nginx file for certbot 
	cat << EOF > /etc/nginx/sites-enabled/yoursite.com
	server {
		listen 80;
		server_name www.yoursite.com yoursite.com;
		include snippets/letsencrypt.conf;
	}
	EOF
	nginx -t # Is everything alright?
	# If so, restart nginx
	service nginx restart 
	# Create the certificate
	certbot certonly --webroot -w /var/www/letsencrypt -d yoursite.com -d www.yoursite.com
	# Remove the stupid file
	rm /etc/nginx/sites-enabled/yoursite.com
	service nginx restart
	``` 

## Misc 

### Creating and rotating backups using `logrotate` 

This is quite cool because `logrotate` manages rotation/deletion of 'log' files very well, so why not use it to rotate backup archives?

One can also add `prerotate`/`postrotate` scripts to a `logrotate` block, which allows to create the backups using `logrotate` too! (This way, there is only one utility taking care of the full backup creation/rotation/deletion process.)

A problem is that **`logrotate` blocks won't run if the block's file does not exist**. So, if you create a block like so:

	/path/to/backup/dir/db-backup.sql.gz {
	  prerotate
	  	# create the backup file
	  endscript
	  weekly
	  missingok
	  nocompress
	  nocreate
	}

This block will never run unless `/path/to/backup/dir/db-backup.sql.gz` exists. This is why `roles/build/tasks/backup.yml` creates an empty backup file while defining the logrotate entry.

### Synapse 

Someone advised me to install matrix-media-repo to enable animated thumbnails as people's avatar (https://github.com/turt2live/matrix-media-repo/blob/master/config.sample.yaml#L394), and to setup https://github.com/ma1uta/ma1sd which is a federated identity server.

#### TODO 

* Finer logging (remove useless federation warnings): https://github.com/matrix-org/synapse/issues/9170

### NextCloud

Steps to dockerization:

* Check the databases
	* Modify character set to utf8mb4 / collate utf8mb4_general_ci.

			ALTER DATABASE owncloud CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;

	* Change the default for the whole server while at it:

			SET character_set_server = 'utf8mb4';
			SET collation_server = 'utf8mb4_general_ci';

* Backup:

		# Database 
		mysqldump -u root -R owncloud > /vault/backups/owncloud.sql
		# Data (exclude './data' folder which is too big):
		tar --exclude='./data' -czvf /vault/backups/nextcloud.tar.gz /var/www/nextcloud

Apparently this is needed, but since I'm using a single MariaDB for every service, I won't bother changing the global config:

* “READ COMMITED” transaction isolation level (See: Database “READ COMMITTED” transaction isolation level)
* Disabled or BINLOG_FORMAT = ROW configured Binary Logging (See: https://dev.mysql.com/doc/refman/5.7/en/binary-log-formats.html)
* For Emoji (UTF8 4-byte) support see Enabling MySQL 4-byte support


### Ansible 

* You can create passwords/keys in templates using the following Jinja2 command:
	
		{{ lookup('password', '/dev/null length=20') }}

	See https://docs.ansible.com/ansible/latest/plugins/lookup/password.html and https://docs.ansible.com/ansible/latest/user_guide/playbooks_lookups.html
	

### Useful MySQL commands

```sql
select host, user, password from mysql.user order by user;
create user 'arvuhez'@'172.26.0.2' identified by 'kjhs';
grant all on arvuhez.* to 'arvuhez'@'172.26.0.2';
show grants for 'arvuhez'@'172.26.0.2';
```