add support for kubernetes service discovery #262
No reviewers
Labels
No labels
action
check-aws
action
discussion-needed
action
for-external-contributors
action
for-newcomers
action
more-info-needed
action
need-funding
action
triage-required
kind
correctness
kind
ideas
kind
improvement
kind
performance
kind
testing
kind
usability
kind
wrong-behavior
prio
critical
prio
low
scope
admin-api
scope
background-healing
scope
build
scope
documentation
scope
k8s
scope
layout
scope
metadata
scope
ops
scope
rpc
scope
s3-api
scope
security
scope
telemetry
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: Deuxfleurs/garage#262
Loading…
Reference in a new issue
No description provided.
Delete branch "audron/garage:feature/kubernetes_service_discovery"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This commit adds support to discover garage instances running in
kubernetes.
Once enabled by setting
kubernetes_namespace
andkubernetes_service_name
garage will create a Custom Resourcesgaragenodes.deuxfleurs.fr
with nodes public key as the resource name.and IP and Port information as spec in the namespace configured by
kubernetes_namespace
.For discovering nodes the resources are filtered with the optionally set
kubernetes_service_name
which sets a labelgarage.deuxfleurs.fr/service
on the resources.This allows to separate multiple garage deployments in a single
namespace.
the
kubernetes_skip_crd
variable allows to disable the creation of theCRD by garage itself. The user must deploy this manually.
The new dependencies on
kube
andpnet
i had to pin on an older versionbecause you are currently using rust 1.55 for building, which is too old
to support rust edition 2021, and these projects got bumped to edition
2021 in newer versions. They can be updated once the rust version was updated.
Thanks for the great work! Remember to run
cargo fmt
to reformat your code according to standard style guidelines, and to check thatcargo clippy
does not return any warning. Will merge when this + the two little comments below are addressed.@ -298,0 +326,4 @@
.ips
.first()
.unwrap()
.ip();
This should be put in a separate utility function to be also used in the function that advertises to Consul, it would be really helpful!
@ -298,0 +328,4 @@
.unwrap()
.ip();
SocketAddr::new(default_ip, 3901)
The port isn't always 3901, it should be derived from the
.port()
of the RPC bind addr that is set in the configuration file.@ -33,0 +33,4 @@
# rust edition 2021
kube = { version = "0.62", features = ["runtime", "derive"] }
k8s-openapi = { version = "0.13", features = ["v1_22"] }
schemars = "0.8"
could you add the following line please?
Garage tries to be cross-compilation friendly, and by default kube seems to use system openssl, which poses problem. Adding this feature flag makes compilation a bit slower as it requires compiling openssl, but makes cross-compilation possible.
kube also has rustls support, but i couldn't make it work for whatever reason rn.
I added your suggestion.
Fixes #241
e8f9086fd1
to2732ec5bb2
2732ec5bb2
to9c43679798
9c43679798
to0d619176c2
fd5f26a504
to9d44127245
LGTM