automation/deployer

Deployer: deploy your shit and make it run

So lame to have to configure nginx, MySQL, and your filesystem to install a stupid Wordpress instance.

Deployer does my config for me like the slave it is.

All the configuration is defined in group_vars/all/vars.yml, go check.

Create a side group_vars/all/vault.yml for your secrets, and encrypt it with Ansible Vault:

ansible-vault encrypt group_vars/all/vault.yml
# other sub-commands: edit, decrypt...

I usually run the following command:

ansible-playbook --ask-vault-pass sites.yml -i inventory -v

Required packages on remote

Python modules:

• docker
• docker-compose
• pymysql
• psycopg2

TODO: Ansible task to install that before the rest

Features

• Creating Wordpress instances (yoohoo, da best)

  • That send mail!!11!1!
  • Supports existing and new installs

• Creating Drupal instances

  • Only existing ones (no new installs)

• Create Gitea instances

  • Nginx and docker-compose configurations
  • Most of the work is by hand, because there is quite a lot of interaction between the host and the container (for forwarding ssh).

• Create Synapse instances

  • Configured to access PostGreSQL on host.

    • Access through TCP: You need to allow postgres to listen to your docker network, e.g. 172.27.0.0/16. See /etc/postgresql/x.y/main/pg_hba.conf, and read the comments about changing listen_addresses too.
    • Access through Unix socket: Make a non-superuser role for root, and configure Synapse to use /var/run/postgresql as DB host.

Does not support

• Setting up the host

• SSL certificate creation (bro, do it yourself!). That is:

  # Make an nginx file for certbot 
  cat << EOF > /etc/nginx/sites-enabled/yoursite.com
  server {
  	listen 80;
  	server_name www.yoursite.com yoursite.com;
  	include snippets/letsencrypt.conf;
  }
  EOF
  nginx -t # Is everything alright?
  # If so, restart nginx
  service nginx restart 
  # Create the certificate
  certbot certonly --webroot -w /var/www/letsencrypt -d yoursite.com -d www.yoursite.com
  # Remove the stupid file
  rm /etc/nginx/sites-enabled/yoursite.com
  service nginx restart
  

Misc

Creating and rotating backups using logrotate

This is quite cool because logrotate manages rotation/deletion of 'log' files very well, so why not use it to rotate backup archives?

One can also add prerotate/postrotate scripts to a logrotate block, which allows to create the backups using logrotate too! (This way, there is only one utility taking care of the full backup creation/rotation/deletion process.)

A problem is that logrotate blocks won't run if the block's file does not exist. So, if you create a block like so:

/path/to/backup/dir/db-backup.sql.gz {
  prerotate
  	# create the backup file
  endscript
  weekly
  missingok
  nocompress
  nocreate
}

This block will never run unless /path/to/backup/dir/db-backup.sql.gz exists. This is why roles/build/tasks/backup.yml creates an empty backup file while defining the logrotate entry.

Synapse

Someone advised me to install matrix-media-repo to enable animated thumbnails as people's avatar (https://github.com/turt2live/matrix-media-repo/blob/master/config.sample.yaml#L394), and to setup https://github.com/ma1uta/ma1sd which is a federated identity server.

TODO

• Finer logging (remove useless federation warnings): https://github.com/matrix-org/synapse/issues/9170

NextCloud

Steps to dockerization:

• Check the databases

  • Modify character set to utf8mb4 / collate utf8mb4_general_ci.

      ALTER DATABASE owncloud CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
    

  • Change the default for the whole server while at it:

      SET character_set_server = 'utf8mb4';
      SET collation_server = 'utf8mb4_general_ci';
    

• Backup:

    # Database 
    mysqldump -u root -R owncloud > /vault/backups/owncloud.sql
    # Data (exclude './data' folder which is too big):
    tar --exclude='./data' -czvf /vault/backups/nextcloud.tar.gz /var/www/nextcloud
  

Apparently this is needed, but since I'm using a single MariaDB for every service, I won't bother changing the global config:

• “READ COMMITED” transaction isolation level (See: Database “READ COMMITTED” transaction isolation level)
• Disabled or BINLOG_FORMAT = ROW configured Binary Logging (See: https://dev.mysql.com/doc/refman/5.7/en/binary-log-formats.html)
• For Emoji (UTF8 4-byte) support see Enabling MySQL 4-byte support

Ansible

• You can create passwords/keys in templates using the following Jinja2 command:

    {{ lookup('password', '/dev/null length=20') }}
  

  See https://docs.ansible.com/ansible/latest/plugins/lookup/password.html and https://docs.ansible.com/ansible/latest/user_guide/playbooks_lookups.html

Useful MySQL commands

select host, user, password from mysql.user order by user;
create user 'arvuhez'@'172.26.0.2' identified by 'kjhs';
grant all on arvuhez.* to 'arvuhez'@'172.26.0.2';
show grants for 'arvuhez'@'172.26.0.2';