forked from Deuxfleurs/infrastructure
Merge branch 'master' of git.deuxfleurs.fr:Deuxfleurs/infrastructure
This commit is contained in:
commit
8eaa7914d0
29 changed files with 92 additions and 26 deletions
33
README.md
33
README.md
|
@ -46,7 +46,7 @@ To ease the development, we make the choice of a fully integrated environment
|
|||
|
||||
### Deploying/Updating new services is done from your machine
|
||||
|
||||
*The following instructions are provided for ops that already have access to the servers.*
|
||||
*The following instructions are provided for ops that already have access to the servers (meaning: their SSH public key is known by the cluster).*
|
||||
|
||||
Deploy Nomad on your machine:
|
||||
|
||||
|
@ -74,16 +74,37 @@ Create an alias (and put it in your `.bashrc`) to bind APIs on your machine:
|
|||
alias bind_df="ssh \
|
||||
-p110 \
|
||||
-N \
|
||||
-L 4646:127.0.0.1:4646 \
|
||||
-L 8500:127.0.0.1:8500 \
|
||||
-L 8082:traefik-admin.service.2.cluster.deuxfleurs.fr:8082 \
|
||||
-L 5432:psql-proxy.service.2.cluster.deuxfleurs.fr:5432 \
|
||||
-L 1389:bottin2.service.2.cluster.deuxfleurs.fr:389 \
|
||||
-L 4646:127.0.0.1:4646 \
|
||||
-L 5432:psql-proxy.service.2.cluster.deuxfleurs.fr:5432 \
|
||||
-L 8082:traefik-admin.service.2.cluster.deuxfleurs.fr:8082 \
|
||||
-L 8500:127.0.0.1:8500 \
|
||||
<a server from the cluster>"
|
||||
```
|
||||
|
||||
and run:
|
||||
|
||||
bind_df
|
||||
|
||||
Adrien uses `.ssh/config` configuration instead. I works basically the same. Here it goes:
|
||||
|
||||
```
|
||||
bind_df
|
||||
# in ~/.ssh/config
|
||||
|
||||
Host deuxfleurs
|
||||
User adrien
|
||||
Hostname deuxfleurs.fr
|
||||
# If you don't use the default ~/.ssh/id_rsa to connect to Deuxfleurs
|
||||
IdentityFile <some_key_path>
|
||||
PubKeyAuthentication yes
|
||||
ForwardAgent No
|
||||
LocalForward 1389 bottin2.service.2.cluster.deuxfleurs.fr:389
|
||||
LocalForward 4646 127.0.0.1:4646
|
||||
LocalForward 5432 psql-proxy.service.2.cluster.deuxfleurs.fr:5432
|
||||
LocalForward 8082 traefik-admin.service.2.cluster.deuxfleurs.fr:8082
|
||||
LocalForward 8500 127.0.0.1:8500
|
||||
```
|
||||
|
||||
Now, to connect, do the following:
|
||||
|
||||
ssh deuxfleurs -N
|
||||
|
|
1
app/.gitignore
vendored
1
app/.gitignore
vendored
|
@ -1 +1,2 @@
|
|||
env/
|
||||
__pycache__
|
||||
|
|
|
@ -1,6 +1,4 @@
|
|||
## Understand this folder hierarchy
|
||||
|
||||
This folder contains the following hierarchy:
|
||||
# Folder hierarchy
|
||||
|
||||
- `<module>/build/<image_name>/`: folders with dockerfiles and other necessary resources for building container images
|
||||
- `<module>/config/`: folder containing configuration files, referenced by deployment file
|
||||
|
@ -8,18 +6,34 @@ This folder contains the following hierarchy:
|
|||
- `<module>/deploy/`: folder containing the HCL file(s) necessary for deploying the module
|
||||
- `<module>/integration/`: folder containing files for integration testing using docker-compose
|
||||
|
||||
# Secret Manager `secretmgr.py`
|
||||
|
||||
The Secret Manager ensures that all secrets are present where they should in the cluster.
|
||||
|
||||
**You need access to the cluster** (SSH port forwarding) for it to find any secret on the cluster. Refer to the previous directory's [README](../README.md), at the bottom of the file.
|
||||
|
||||
## How to install `secretmgr.py` dependencies
|
||||
|
||||
How to install its dependencies:
|
||||
|
||||
```bash
|
||||
# on fedora:
|
||||
dnf install -y openldap-devel
|
||||
# on ubuntu:
|
||||
apt-get install -y libldap2-dev
|
||||
### Install system dependencies first:
|
||||
## On fedora
|
||||
|
||||
# for eveyrone:
|
||||
pip3 install --user --requirement requirements.txt
|
||||
dnf install -y openldap-devel cyrus-sasl-devel
|
||||
## On ubuntu
|
||||
apt-get install -y libldap2-dev libsasl2-dev
|
||||
|
||||
### Now install the Python dependencies from requirements.txt:
|
||||
|
||||
## Either using a virtual environment
|
||||
# (requires virtualenv python module)
|
||||
python3 -m virtualenv env
|
||||
# Must be done everytime you create a new terminal window in this folder:
|
||||
. env/bin/activate
|
||||
# Install the deps
|
||||
pip install -r requirements.txt
|
||||
|
||||
## Either by installing the dependencies for your system user:
|
||||
pip3 install --user -r requirements.txt
|
||||
```
|
||||
|
||||
## How to use `secretmgr.py`
|
||||
|
@ -42,7 +56,7 @@ Rotate secrets for app `dummy`, overwriting existing ones (be careful, this is d
|
|||
./secretmgr.py regen dummy
|
||||
```
|
||||
|
||||
## How to upgrade our packaged apps to a new version?
|
||||
# Upgrading one of our packaged apps to a new version
|
||||
|
||||
1. Edit `docker-compose.yml`
|
||||
2. Change the `VERSION` variable to the desired version
|
||||
|
|
1
app/backup/secrets/backup/id_ed25519
Normal file
1
app/backup/secrets/backup/id_ed25519
Normal file
|
@ -0,0 +1 @@
|
|||
USER_LONG Private ed25519 key of the container doing the backup
|
1
app/backup/secrets/backup/id_ed25519.pub
Normal file
1
app/backup/secrets/backup/id_ed25519.pub
Normal file
|
@ -0,0 +1 @@
|
|||
USER Public ed25519 key of the container doing the backup (this key must be in authorized_keys on the backup target host)
|
1
app/backup/secrets/backup/target_ssh_dir
Normal file
1
app/backup/secrets/backup/target_ssh_dir
Normal file
|
@ -0,0 +1 @@
|
|||
USER Directory where to store backups on target host
|
1
app/backup/secrets/backup/target_ssh_fingerprint
Normal file
1
app/backup/secrets/backup/target_ssh_fingerprint
Normal file
|
@ -0,0 +1 @@
|
|||
USER SSH fingerprint of the target machine (format: copy here the corresponding line from your known_hosts file)
|
1
app/backup/secrets/backup/target_ssh_host
Normal file
1
app/backup/secrets/backup/target_ssh_host
Normal file
|
@ -0,0 +1 @@
|
|||
USER Hostname of the backup target host
|
1
app/backup/secrets/backup/target_ssh_port
Normal file
1
app/backup/secrets/backup/target_ssh_port
Normal file
|
@ -0,0 +1 @@
|
|||
USER SSH port number to connect to the target host
|
1
app/backup/secrets/backup/target_ssh_user
Normal file
1
app/backup/secrets/backup/target_ssh_user
Normal file
|
@ -0,0 +1 @@
|
|||
USER SSH username to log in as on the target host
|
1
app/garage/secrets/garage/garage-ca.crt
Normal file
1
app/garage/secrets/garage/garage-ca.crt
Normal file
|
@ -0,0 +1 @@
|
|||
USER_LONG garage-ca.crt (generated with Garage's genkeys.sh script)
|
1
app/garage/secrets/garage/garage-ca.key
Normal file
1
app/garage/secrets/garage/garage-ca.key
Normal file
|
@ -0,0 +1 @@
|
|||
USER_LONG garage-ca.key (generated with Garage's genkeys.sh script)
|
1
app/garage/secrets/garage/garage.crt
Normal file
1
app/garage/secrets/garage/garage.crt
Normal file
|
@ -0,0 +1 @@
|
|||
USER_LONG garage.crt (generated with Garage's genkeys.sh script)
|
1
app/garage/secrets/garage/garage.key
Normal file
1
app/garage/secrets/garage/garage.key
Normal file
|
@ -0,0 +1 @@
|
|||
USER_LONG garage.key (generated with Garage's genkeys.sh script)
|
|
@ -1 +1 @@
|
|||
USER cotorn static-auth (what is this?)
|
||||
USER coturn static-auth (what is this?)
|
||||
|
|
1
app/im/secrets/chat/easybridge/as_token
Normal file
1
app/im/secrets/chat/easybridge/as_token
Normal file
|
@ -0,0 +1 @@
|
|||
CMD openssl rand -hex 32
|
1
app/im/secrets/chat/easybridge/db_pass
Normal file
1
app/im/secrets/chat/easybridge/db_pass
Normal file
|
@ -0,0 +1 @@
|
|||
SERVICE_PASSWORD easybridge
|
1
app/im/secrets/chat/easybridge/db_user
Normal file
1
app/im/secrets/chat/easybridge/db_user
Normal file
|
@ -0,0 +1 @@
|
|||
CONST easybridge
|
1
app/im/secrets/chat/easybridge/hs_token
Normal file
1
app/im/secrets/chat/easybridge/hs_token
Normal file
|
@ -0,0 +1 @@
|
|||
CMD openssl rand -hex 32
|
2
app/im/secrets/chat/easybridge/web_session_key
Normal file
2
app/im/secrets/chat/easybridge/web_session_key
Normal file
|
@ -0,0 +1,2 @@
|
|||
CMD openssl rand -hex 32
|
||||
|
|
@ -1 +1 @@
|
|||
USER fb2mx API server token
|
||||
CMD openssl rand -hex 32
|
||||
|
|
|
@ -1 +1 @@
|
|||
USER fb2mx homeserver token
|
||||
CMD openssl rand -hex 32
|
||||
|
|
1
app/im/secrets/chat/synapse/homeserver.signing.key
Normal file
1
app/im/secrets/chat/synapse/homeserver.signing.key
Normal file
|
@ -0,0 +1 @@
|
|||
USER Synapse homeserver ed25519 signing key
|
|
@ -1 +1 @@
|
|||
USER Shared secret for homeserver registrations (?)
|
||||
CMD head -c 32 /dev/urandom | base64
|
||||
|
|
|
@ -1 +1 @@
|
|||
CMD openssl rand -base64 32
|
||||
SERVICE_PASSWORD plume
|
||||
|
|
|
@ -43,6 +43,9 @@ USER_LONG <description>
|
|||
CMD <command>
|
||||
(a secret that is generated by running this command)
|
||||
|
||||
CMD_ONCE <command>
|
||||
(same, but value is not changed when doing a regen)
|
||||
|
||||
CONST <constant value>
|
||||
(the secret has a constant value set here)
|
||||
|
||||
|
@ -81,6 +84,7 @@ consul_server = consul.Consul()
|
|||
USER = "USER"
|
||||
USER_LONG = "USER_LONG"
|
||||
CMD = "CMD"
|
||||
CMD_ONCE = "CMD_ONCE"
|
||||
CONST = "CONST"
|
||||
CONST_LONG = "CONST_LONG"
|
||||
SERVICE_DN = "SERVICE_DN"
|
||||
|
@ -103,12 +107,15 @@ class bcolors:
|
|||
|
||||
def read_secret(key, file_path):
|
||||
lines = [l.strip() for l in open(file_path, "r")]
|
||||
if len(lines) == 0:
|
||||
print(bcolors.FAIL, "ERROR:", bcolors.ENDC, "Empty file in", file_path)
|
||||
sys.exit(-1)
|
||||
l0 = lines[0].split(" ")
|
||||
stype = l0[0]
|
||||
secret = {"type": stype, "key": key}
|
||||
if stype in [USER, USER_LONG]:
|
||||
secret["desc"] = " ".join(l0[1:])
|
||||
elif stype == CMD:
|
||||
elif stype in [CMD, CMD_ONCE]:
|
||||
secret["cmd"] = " ".join(l0[1:])
|
||||
elif stype == CONST:
|
||||
secret["value"] = " ".join(l0[1:])
|
||||
|
@ -151,6 +158,7 @@ def get_secrets_services(secrets):
|
|||
if svc not in services:
|
||||
services[svc] = {
|
||||
"dn": "cn=%s,%s"%(svc, SERVICE_DN_SUFFIX),
|
||||
"desc": "(not provided)",
|
||||
"pass": None,
|
||||
"dn_at": [],
|
||||
"pass_at": [],
|
||||
|
@ -275,7 +283,7 @@ def gen_secrets_base(secrets, regen):
|
|||
line = input().strip()
|
||||
if line == ".":
|
||||
break
|
||||
vals.append(line)
|
||||
lines.append(line)
|
||||
val = "\n".join(lines)
|
||||
consul_server.kv.put(key, val)
|
||||
print(bcolors.OKCYAN, "Value set.", bcolors.ENDC)
|
||||
|
@ -289,7 +297,7 @@ def gen_secrets_base(secrets, regen):
|
|||
consul_server.kv.put(key, secret["value"])
|
||||
print(bcolors.OKCYAN, "Value set.", bcolors.ENDC)
|
||||
|
||||
if secret["type"] == CMD:
|
||||
if secret["type"] == CMD or (secret["type"] == CMD_ONCE and data is None):
|
||||
print("----")
|
||||
print(key)
|
||||
print("Executing command:", secret["cmd"])
|
||||
|
|
|
@ -8,6 +8,8 @@ Go to guichet.deuxfleurs.fr
|
|||
4. Hash it with `slappasswd`
|
||||
5. Add a `userpassword` entry with the hash
|
||||
|
||||
This step can also be done using the automated tool `secretmgr.py` in the app folder.
|
||||
|
||||
## 2. Connect to postgres with the admin users
|
||||
|
||||
```bash
|
||||
|
|
|
@ -1,3 +1,5 @@
|
|||
## Creating a new Plume user
|
||||
|
||||
1. Bind nomad on your machine with SSH (check the README file at the root of this repo)
|
||||
2. Go to http://127.0.0.1:4646
|
||||
3. Select `plume` -> click `exec` button (top right)
|
||||
|
|
Loading…
Reference in a new issue