Performance question #900

New issue

Open

opened 2024-11-14 10:17:26 +00:00 by ptempier · 1 comment

ptempier commented

2024-11-14 10:17:26 +00:00

hi
i installed garage in mono hosts, everything is async,
the hosts 250gb of memory 16 cores,
20 disks in raid 10, 10gb link.

Using s5cm i get about 100MB/s while copying to /tmp (memory fs),
using the same setup i get more like 800MB/s using minio

And when uploading its at 100MB/s and after sometime get down to 50MB/s
and seems limited by disk usage that seems very high (lot of small ios ?).

Not sure what's going on, i don't see a lot of space to tweak the settings,
and even when its at 100MB/s i can't see what the bottelneck is,
i thought maybe latency but i also tried in local and had the same results.

hi i installed garage in mono hosts, everything is async, the hosts 250gb of memory 16 cores, 20 disks in raid 10, 10gb link. Using s5cm i get about 100MB/s while copying to /tmp (memory fs), using the same setup i get more like 800MB/s using minio And when uploading its at 100MB/s and after sometime get down to 50MB/s and seems limited by disk usage that seems very high (lot of small ios ?). Not sure what's going on, i don't see a lot of space to tweak the settings, and even when its at 100MB/s i can't see what the bottelneck is, i thought maybe latency but i also tried in local and had the same results.

lx commented

2024-11-19 09:18:10 +00:00

Owner

Garage was not tuned for high performance on such high-end systems, so it's normal that it would perform poorly compared to minio. Garage is tuned for low request latency in geodistributed clusters.

It would be interesting to see what is the bottleneck in your setup, I suspect it's CPU ?

You can try changing the block_size configuration parameter to see if it changes something for you, for example set it to 10M or more.

Garage was not tuned for high performance on such high-end systems, so it's normal that it would perform poorly compared to minio. Garage is tuned for low request latency in geodistributed clusters. It would be interesting to see what is the bottleneck in your setup, I suspect it's CPU ? You can try changing the `block_size` configuration parameter to see if it changes something for you, for example set it to `10M` or more.

👍 1