I think the logs were over the upload limit and they disappeared into the ether.
I'll get those to you.
metadata_dir = "/var/lib/garage/meta"
data_dir = "/var/lib/garage/data"
replication_mode = "3"
compression_level = 0
rpc_bind_addr = "[::]:3901"
rpc_secret…It's worth pointing out that it took days for any serious issues to start happening (we had zero monitoring for this failure case), and in this case, it was containers not coming up that…
And how I guessed that it was causing corruption:
Jul 10 00:21:48 cameo garage[1036]: 2023-07-10T00:21:48.377886Z INFO garage_block::resync: Resync block 366876f935fa2911: fetching absent…This is what the garage logs look like during this time:
Jul 09 23:42:49 cameo garage[1036]: 2023-07-09T23:42:49.523702Z INFO garage_table::sync: (bucket_object_counter) Sending 1 items…garage block list-errors
Hash RC Errors Last error Next try
02d36a316fc4a845b66f446e1913a21e6a6d344785d2e07dd798e63dc01752…I've had to vanish off the face of the earth due to some family issues. I'm back and I'm hoping to get back into this. I just wanted to give an update now that I'm back on the web.
In your use case, do you envision a scenario when missing a webhook trigger causes a lot of issues in your system, or could you recover from such missed events in some way? More genrally, what do…
Hi @lx! Sorry I totally missed your reply. I've rebased on main and removed the 5MB limit.
I thought about a smoke test, but tricking the clients into doing a multi-part less than 5mb seems impossible. Should be ready for review.