210 lines
6.8 KiB
Markdown
210 lines
6.8 KiB
Markdown
# Albatros
|
|
|
|
A lightweight and (quasi-)stateless CI built on top of Nomad.
|
|
Our main principle: offload as much work to Nomad as possible.
|
|
We don't want to build an abstraction on top of it and hide
|
|
Nomad internals, but instead expose them as directly and transparently
|
|
as possible, so that you can benefit from all the features of this software.
|
|
**Albatros is a specialized CI for Nomad**
|
|
|
|
When we can't offload our work directly to Nomad, we should consider
|
|
offloading it to Gitea, Consul, and the others. At a last resort,
|
|
we might do it in Albatros...
|
|
|
|
## Builds
|
|
|
|
```
|
|
go build -tags containers_image_docker_daemon_stub,containers_image_storage_stub,containers_image_openpgp bin/alba.go
|
|
```
|
|
|
|
## Deploy
|
|
|
|
Requirements: Nomad, Consul, Gitea
|
|
|
|
Prepare your nomad cluster:
|
|
|
|
```
|
|
nomad namespace apply -description "Continuous Integration" ci
|
|
nomad run hcl/builder.hcl
|
|
```
|
|
|
|
Run from the terminal directly:
|
|
|
|
```
|
|
export ALBATROS_URL="https://albatros.deuxfleurs.fr"
|
|
export NOMAD_ADDR=...
|
|
export NOMAD_CLIENT_CERT=...
|
|
export NOMAD_CLIENT_KEY=...
|
|
export NOMAD_CACERT=...
|
|
export CONSUL_HTTP_ADDR=...
|
|
export CONSUL_CLIENT_CERT=...
|
|
export CONSUL_CLIENT_KEY=...
|
|
export CONSUL_CACERT=...
|
|
./albatros
|
|
```
|
|
|
|
Run from docker:
|
|
|
|
```
|
|
docker run --rm -it dxflrs/albatros:xxx
|
|
```
|
|
|
|
where `xxx` is the commit sha you want.
|
|
|
|
## Bring Your Own Builder
|
|
|
|
One aspect that I don't like with traditional CI is that the way you can
|
|
configure builds is very constrained by the vendor implementation choices.
|
|
Like it or not, but Albatros has very few constraints in term of what build infrastructure should look like:
|
|
- It must be a Nomad parameterized job
|
|
- Your job must accept four mandatory meta parameters: `REPO_URL`, `COMMIT`, `BRANCH`, `FLAVOR`
|
|
- Your job may receive a payload containing the secrets stored in your job definition (subject to change)
|
|
- It must have have a task group named `runner` and, inside this group, a task named `executor`
|
|
|
|
And... that's basically all you need.
|
|
|
|
### A NixOS builder with local cache
|
|
|
|
In the `hcl/` folder, you will find the definition of a builder named `nixcache`.
|
|
This builder has the following features:
|
|
- It creates a persisted nix store at `/var/cache/albatros` on your host if it does not exist yet.
|
|
- The nix store is mounted read-only during the build
|
|
- A nix daemon is run in a separate container, you can interact with it though its unix socket (everything is already setup for you). Simply run `nix build`.
|
|
- All your builds are automatically added as a nix gcroots by the nix daemon, `keep_derivation` is activated so your build dependencies are kept locally.
|
|
- At the end, all gcroots older than 7 days are deleted, and then the `nix store gc` is called effectively wiping data that were previously attached to old gcroots.
|
|
|
|
## Register a build
|
|
|
|
Add to Consul a key in albatros hierarchy
|
|
named after your repo URL as base64. Example:
|
|
|
|
```
|
|
albatros/aHR0cHM6Ly9naXQuZGV1eGZsZXVycy5mci9xdWVudGluL2FsYmF0cm9zLmdpdA==
|
|
```
|
|
|
|
The key must contain a JSON file with your desired token, gitea info, trust conditions, and secrets:
|
|
|
|
```json
|
|
{
|
|
"hook": {
|
|
"token": "s3cr3t"
|
|
},
|
|
"gitea": {
|
|
"url": "https://git.deuxfleurs.fr",
|
|
"token": "c0ffee..."
|
|
},
|
|
"trusted": {
|
|
"senders": [ "quentin", "lx" ]
|
|
},
|
|
"inject": "SECRET1=xx\nSECRET2=yy"
|
|
}
|
|
```
|
|
|
|
*Register an access token if Gitea for your Albatros by creating an application token.
|
|
The URL is `/user/settings/applications`. Name your app `albatros` for example, the token will be displayed once in light blue.
|
|
Put it in the JSON file.*
|
|
|
|
Your secret will be injected in your build environment only
|
|
when trustig condition are matched. It wil be available in a dedicated
|
|
file. Its path is communicated through an environment variable (see below).
|
|
For now, we can only check that based on sender's login.
|
|
|
|
Then you can trigger a build as follow:
|
|
|
|
```
|
|
$ curl -d @example/albatros.json http://localhost:8080/hook?token=s3cr3t&flavor=default
|
|
builder/dispatch-1678866433-15aad86a
|
|
```
|
|
|
|
You need to pass your token, and you can optionally pass a flavor, that can
|
|
be used later by your build script.
|
|
|
|
As you can see, you now have an identifier representing your job, you
|
|
can use it to follow your logs (don't forget to urlencode it):
|
|
|
|
```
|
|
$ curl http://localhost:8080/build?job=builder%2Fdispatch-1678866433-15aad86a&log=stderr
|
|
+ go build
|
|
...
|
|
```
|
|
|
|
Of course, most of that will be handled by Gitea.
|
|
|
|
## Writing your build script
|
|
|
|
You must create an executable `.albatros` file at the root of your repository.
|
|
Then, you can use the interpreter you want to execute it, let use `bash`
|
|
as an example, but please use python, javascript, or anything else that has
|
|
proper error handling...
|
|
|
|
```bash
|
|
#!/usr/bin/env bash
|
|
echo "Building commit $COMMIT"
|
|
go build
|
|
```
|
|
|
|
During the build, the following environment variables are available:
|
|
|
|
```bash
|
|
REPO_URL=https://git.deuxfleurs.fr/quentin/albatros.git
|
|
COMMIT=3fff73597f8ca18ef04c0d9bf64132ba55aadcaa
|
|
BRANCH=main
|
|
FLAVOR=default
|
|
SECRET_PATH=/var/run/secrets/albatros/secret.txt
|
|
```
|
|
|
|
## Security model
|
|
|
|
Albatros only tries to protect your secrets.
|
|
To achieve that, we only inject them in the build script
|
|
if the job has been `sent` (triggered) by a trusted sender.
|
|
This decision is taken by looking at the webhook payload content.
|
|
|
|
To protect against sender impersonification, your webhook
|
|
must be called only by trusted code, and more generally, your token
|
|
must remain secret.
|
|
|
|
We assume otherwise that anyone can trigger the webhook
|
|
through Gitea while replacing the content of the build script
|
|
by a malicious software. Signing the build script is tempting
|
|
but it will not prevent someone to put its malicious code, for example
|
|
in a Rust project, in the `build.rs` file. So you must
|
|
assume that your CI will execute untrusted code.
|
|
|
|
To protect against undesired code execution,
|
|
you must harden your environment, for example by using
|
|
VMs instead of containers, timeouts, and restricting
|
|
some IO. All of that must be handled by Nomad. Also,
|
|
be careful to the local network in which your workload
|
|
will be executed.
|
|
|
|
## Roadmap
|
|
|
|
Don't get me wrong, the 1.0 version
|
|
is not ambitious at all and will have to
|
|
few features for many of you. Still,
|
|
I think it will be enough for us.
|
|
|
|
### v1.0
|
|
|
|
See this v1.0 as a MVP that will serve Deuxfleurs needs,
|
|
nothing more. Don't have any expectation in term
|
|
of code quality, abstraction or anything else.
|
|
|
|
- [X] Read Nomad+Consul config from environment variables
|
|
- [X] Inject secrets only when the sender is trusted
|
|
- [ ] Test PR behavior
|
|
|
|
### Ideas
|
|
|
|
Ideas are just... ideas. They are pure speculation,
|
|
and if they are here, it means that will and time
|
|
to implement them has not been found yet...
|
|
|
|
- [ ] Validate the gitea payload with hmac
|
|
- [ ] Refactor the code
|
|
- [ ] Register the builder programatically
|
|
- [ ] Allow users to define their own set of builders (ones with more CPU+RAM, etc.)
|
|
- [ ] Allow users to choose their image/rootfs
|
|
- [ ] Make Gitea optional
|
|
- [ ] Your idea
|