83 commits

Author	SHA1	Message	Date
Alex	8bfc16ba7d	Merge branch 'main' into next-0.10	2024-03-28 15:01:05 +01:00
Alex	85f580cbde	[fix-buffering] change request sending strategy and fix priorities ... remove LAS, priorize new requests but otherwise just do standard queuing	2024-03-27 16:22:40 +01:00
Alex	0d3e285d13	[fix-buffering] implement block_ram_buffer_max to avoid excessive RAM usage	2024-03-27 16:22:40 +01:00
Alex	f7cd4eb600	Merge branch 'main' into next-0.10	2024-03-26 16:34:40 +01:00
Alex	ba33bb31f1	[check-data-dir] add marker files in data directories (fix #601)	2024-03-20 15:20:25 +01:00
Alex	3165ab926c	[block-ref-repair] rename rc's rc field to rc_table	2024-03-19 16:20:22 +01:00
Alex	dc0b78cdb8	[block-ref-repair] Block refcount recalculation and repair ... - We always recalculate the reference count of a block before deleting it locally, to make sure that it is indeed zero. - If we had to fetch a remote block but we were not able to get it, check that refcount is indeed > 0. - Repair procedure that checks everything	2024-03-19 16:20:22 +01:00
Alex	0038ca8a78	Merge branch 'main' into next-0.10	2024-03-18 20:19:30 +01:00
Alex	7c86ff6c37	[disable-scrub] implement a disable_scrub configuration option	2024-03-14 17:01:16 +01:00
Alex	05c92204ec	[rm-sled] Remove counted_tree_hack	2024-03-08 15:09:57 +01:00
Alex	57acc60082	[sse-c] Implement SSE-C encryption	2024-03-07 15:43:47 +01:00
Alex	4b978b7533	Merge branch 'main' into next-0.10	2024-02-26 18:55:24 +01:00
Alex	b76c0c102e	[refactor-put] add ordering tag to blocks being sent to storage nodes	2024-02-26 18:35:11 +01:00
Alex	916c67ccf4	Merge branch 'main' into next-0.10	2024-02-23 16:50:34 +01:00
Alex	6ee691e65f	[refactor-block] simplify some more	2024-02-23 12:50:10 +01:00
Alex	e9c42bca34	[refactor-block] add DataBlockStream type	2024-02-23 12:22:29 +01:00
Alex	cd1069c1d4	[refactor-block] refactor DataBlock and DataBlockPath	2024-02-23 12:15:52 +01:00
Alex	07c7895948	[refactor-block] simplify rpc_get_block	2024-02-23 11:54:40 +01:00
Alex	9b41f4ff20	[refactor-block] move read_stream_to_end to garage_net	2024-02-23 11:46:57 +01:00
Alex	93552b9275	[refactor-block] Remove redundant BlockStream type	2024-02-23 11:33:38 +01:00
Alex	eb4a6ce106	Merge branch 'main' into next-0.10	2024-02-15 14:06:34 +01:00
Alex	5ea24254a9	[import-netapp] import Netapp code into Garage codebase	2024-02-15 12:15:07 +01:00
Alex	cf2af186fc	Merge branch 'main' into next-0.10	2024-02-13 11:36:28 +01:00
Alex	22332e6c35	[dep-upgrade-202402] simplify/refactor GetObject	2024-02-05 20:26:33 +01:00
Alex	5dd200c015	layout: move block_read_nodes_of to rpc_helper to avoid double-locking ... (in theory, this could have caused a deadlock)	2023-12-08 12:02:24 +01:00
Alex	d6d239fc79	block manager: read_block using old layout versions if necessary	2023-11-27 11:52:57 +01:00
Alex	33c8a489b0	layou: implement ack locking	2023-11-15 15:40:44 +01:00
Alex	90e1619b1e	table: take into account multiple write sets in inserts	2023-11-14 15:40:46 +01:00
Alex	3b361d2959	layout: prepare for write sets	2023-11-14 14:28:16 +01:00
Alex	8a2b1dd422	wip: split out layout management from System into separate LayoutManager	2023-11-09 12:55:36 +01:00
Alex	7f9ba49c71	block manager: remove data_dir field	2023-09-11 11:57:36 +02:00
Alex	be91ef6294	block manager: fix bug where rebalance didn't delete old copies	2023-09-07 16:04:03 +02:00
Alex	6b008b5bd3	block manager: add rebalance operation to rebalance multi-hdd setups	2023-09-07 13:44:11 +02:00
Alex	99ed18350f	block manager: refactor and fix monitoring/statistics	2023-09-07 12:41:36 +02:00
Alex	f38a31b330	block manager: avoid incorrect data_dir configs and avoid losing files	2023-09-06 17:49:30 +02:00
Alex	a44f486931	block manager: refactoring & increase max worker count to 8	2023-09-06 16:35:28 +02:00
Alex	93114a9747	block manager: refactoring	2023-09-06 16:35:28 +02:00
Alex	1b8c265c14	block manager: get rid of check_block_status	2023-09-06 16:35:28 +02:00
Alex	a09f86729c	block manager: move blocks in write_block if necessary	2023-09-06 16:35:28 +02:00
Alex	887b3233f4	block manager: use data paths from layout	2023-09-06 16:35:28 +02:00
Alex	6c420c0880	block manager: multi-directory layout computation	2023-09-06 16:35:28 +02:00
Alex	71c0188055	block manager: skeleton for multi-hdd support	2023-09-06 16:35:28 +02:00
Alex	e7e164a280	Make fsync an option for meta and data	2023-06-09 16:23:21 +02:00
Jonathan Davies	9c788059e2	block/manager.rs: In is_block_compressed - check which compression_level ... is configured on a node and check for raw block first if compression is disabled (to help reduce syscalls during a scrub).	2023-05-09 10:28:19 +01:00
Jonathan Davies	d218f475cb	block/manager.rs: Set defaults for scrub_persister.	2023-03-09 17:08:47 +00:00
Jonathan Davies	148b66b843	block/manager.rs: Display scrub-next-run.	2023-03-06 13:43:09 +00:00
Jonathan Davies	4cfb469d2b	block/metrics.rs: Added compression_level metric.	2023-01-10 10:40:03 +00:00
Alex	02e8eb167e	Merge pull request 'PutObject: better cleanup when request is interrupted in the middle' (#462) from interrupted-cleanup into main ... Reviewed-on: #462	2023-01-04 14:43:45 +00:00
Alex	f3f27293df	Uniform framework for bg variable management	2023-01-04 13:07:13 +01:00
Alex	936b6cb563	When saving block, delete .tmp file if we could not complete	2023-01-03 17:34:26 +01:00