nixcfg/doc/telemetry.md

generate ca and tls certs for elasticsearch cluster

start a bash in an elasticsearch image, such as docker.elastic.co/elasticsearch/elasticsearch:8.2.0: docker run -ti docker.elastic.co/elasticsearch/elasticsearch:8.2.0 bash

generate a ca and node certs:

./bin/elasticsearch-certutil ca
./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12

write these files in Consul at secrets/telemetry/elasticsearch/elastic-certificates.p12 and secrets/telemetry/elasticsearch/elastic-stack-ca.p12

start nomad services

nomad run telemetry-system.hcl
nomad run telemetry.hcl

create elasticsearch passwords

in an elasticsearch container that was launched by Nomad:

./bin/elasticsearch-reset-password -u elastic
./bin/elasticsearch-reset-password -u kibana

save passwords in consul, at:

• secrets/telemetry/elastic_passwords/kibana_system for user kibana_system
• secrets/telemetry/elastic_passwords/elastic for user elastic

check kibana works, login to kibana with user elastic

grafana and apm-server will use the elastic user (the admin) to write data to elasticsearch