support DNS resolution for bind_addr #859

New issue

Closed

opened 2024-08-22 18:17:10 +00:00 by lychee · 5 comments

lychee commented

2024-08-22 18:17:10 +00:00

This is my current approach

    services.garage = {
      package = pkgs.garage_1_0_0;
      environmentFile = "/var/lib/secrets/garage/garage-secrets";
      settings = {
         # config.services.tailscale.ip being a custom option set in each host-specific file
        rpc_bind_addr = "${config.services.tailscale.ip}:9000";
        rpc_public_addr = "${config.services.tailscale.ip}:9000";
        root_domain = "s3.wires.cafe";
        replication_factor = 3;
        consul_discovery = {
          consul_http_addr = "http://127.0.0.1:8500";
          service_name = "garage-s3";
          tags = ["garage"];
        };

        s3_api = {
          api_bind_addr = "${config.services.tailscale.ip}:9001";
          s3_region = "lychee";
        };
      };
    };

but it would be ideal if i could just do

    services.garage = {
      package = pkgs.garage_1_0_0;
      environmentFile = "/var/lib/secrets/garage/garage-secrets";
      settings = {
        rpc_bind_addr = "${config.networking.hostName}:9000";
        rpc_public_addr = "${config.networking.hostName}:9000";
        root_domain = "s3.wires.cafe";
        replication_factor = 3;
        consul_discovery = {
          consul_http_addr = "http://127.0.0.1:8500";
          service_name = "garage-s3";
          tags = ["garage"];
        };

        s3_api = {
          api_bind_addr = "${config.networking.hostName}:9001";
          s3_region = "lychee";
        };
      };
    };

with DNS enabled

I currently use NixOS with Garage and i was having issues with using my tailscale hostname as the address which I later resolved by hardcoding the IP address in each host but it would be amazing if there was some sort of mechanism to automatically resolve the IP address so I can dynamically add more hosts and simply reference them through that, easing the burden of needing to manually add each IP address. This is my current approach ```nix services.garage = { package = pkgs.garage_1_0_0; environmentFile = "/var/lib/secrets/garage/garage-secrets"; settings = { # config.services.tailscale.ip being a custom option set in each host-specific file rpc_bind_addr = "${config.services.tailscale.ip}:9000"; rpc_public_addr = "${config.services.tailscale.ip}:9000"; root_domain = "s3.wires.cafe"; replication_factor = 3; consul_discovery = { consul_http_addr = "http://127.0.0.1:8500"; service_name = "garage-s3"; tags = ["garage"]; }; s3_api = { api_bind_addr = "${config.services.tailscale.ip}:9001"; s3_region = "lychee"; }; }; }; ``` but it would be ideal if i could just do ```nix services.garage = { package = pkgs.garage_1_0_0; environmentFile = "/var/lib/secrets/garage/garage-secrets"; settings = { rpc_bind_addr = "${config.networking.hostName}:9000"; rpc_public_addr = "${config.networking.hostName}:9000"; root_domain = "s3.wires.cafe"; replication_factor = 3; consul_discovery = { consul_http_addr = "http://127.0.0.1:8500"; service_name = "garage-s3"; tags = ["garage"]; }; s3_api = { api_bind_addr = "${config.networking.hostName}:9001"; s3_region = "lychee"; }; }; }; ``` with DNS enabled

lx commented

2024-08-24 09:12:24 +00:00

Owner

But wouldn't the host name of the local machine also resolve to a bunch of other addresses, including 127.0.0.1 and ::1?

More generally, the resolution method used in Garage is the basic DNS resolution from Tokio (see here) which gives a list of addresses, but we need to select a single address to bind to. How do we chose it?

But wouldn't the host name of the local machine also resolve to a bunch of other addresses, including `127.0.0.1` and `::1`? More generally, the resolution method used in Garage is the basic DNS resolution from Tokio ([see here](https://docs.rs/tokio/latest/tokio/net/trait.ToSocketAddrs.html)) which gives a list of addresses, but we need to select a single address to bind to. How do we chose it?

lychee commented

2024-08-25 02:21:31 +00:00

Author

heck I didn't even think about that

lychee commented

2024-08-26 14:41:17 +00:00

Author

~~would it be possible to set a DNS resolver to keep it at one address?~~

doesn't seem like tokio provides such an option from a quick glance

~~would it be possible to set a DNS resolver to keep it at one address?~~ doesn't seem like tokio provides such an option from a quick glance

quentin added the

labels 2024-08-29 09:39:58 +00:00

quentin commented

2024-08-29 09:45:19 +00:00

Owner

You can set rpc_addr to [::] (or :: I don't remeber) and it will automatically listen on all interfaces on both IP versions. Then you can completely remove rpc_public_addr from your configuration. And finally put the IP address you want when doing garage node connect xxxx@xxx:3901. If it happens that you change the IP addresses of your node one by one, Garage will automatically discover and update the new IP addresses (as the node that has a new IP address will try to contact the other nodes that still have the same IP address, and upon contact, this new IP address will be learnt and broadcasted).

Could this set of feature satisfy your requirements?

You can set `rpc_addr` to `[::]` (or `::` I don't remeber) and it will automatically listen on all interfaces on both IP versions. Then you can completely remove `rpc_public_addr` from your configuration. And finally put the IP address you want when doing `garage node connect xxxx@xxx:3901`. If it happens that you change the IP addresses of your node one by one, Garage will automatically discover and update the new IP addresses (as the node that has a new IP address will try to contact the other nodes that still have the same IP address, and upon contact, this new IP address will be learnt and broadcasted). Could this set of feature satisfy your requirements?

lychee commented

2024-09-02 22:38:37 +00:00

Author

sounds good! thank u