66 commits

Author	SHA1	Message	Date
Alex	85f580cbde	[fix-buffering] change request sending strategy and fix priorities ... remove LAS, priorize new requests but otherwise just do standard queuing	2024-03-27 16:22:40 +01:00
Alex	0d3e285d13	[fix-buffering] implement block_ram_buffer_max to avoid excessive RAM usage	2024-03-27 16:22:40 +01:00
Alex	ba33bb31f1	[check-data-dir] add marker files in data directories (fix #601)	2024-03-20 15:20:25 +01:00
Alex	7c86ff6c37	[disable-scrub] implement a disable_scrub configuration option	2024-03-14 17:01:16 +01:00
Alex	b76c0c102e	[refactor-put] add ordering tag to blocks being sent to storage nodes	2024-02-26 18:35:11 +01:00
Alex	6ee691e65f	[refactor-block] simplify some more	2024-02-23 12:50:10 +01:00
Alex	e9c42bca34	[refactor-block] add DataBlockStream type	2024-02-23 12:22:29 +01:00
Alex	cd1069c1d4	[refactor-block] refactor DataBlock and DataBlockPath	2024-02-23 12:15:52 +01:00
Alex	07c7895948	[refactor-block] simplify rpc_get_block	2024-02-23 11:54:40 +01:00
Alex	9b41f4ff20	[refactor-block] move read_stream_to_end to garage_net	2024-02-23 11:46:57 +01:00
Alex	93552b9275	[refactor-block] Remove redundant BlockStream type	2024-02-23 11:33:38 +01:00
Alex	5ea24254a9	[import-netapp] import Netapp code into Garage codebase	2024-02-15 12:15:07 +01:00
Alex	22332e6c35	[dep-upgrade-202402] simplify/refactor GetObject	2024-02-05 20:26:33 +01:00
Alex	7f9ba49c71	block manager: remove data_dir field	2023-09-11 11:57:36 +02:00
Alex	be91ef6294	block manager: fix bug where rebalance didn't delete old copies	2023-09-07 16:04:03 +02:00
Alex	6b008b5bd3	block manager: add rebalance operation to rebalance multi-hdd setups	2023-09-07 13:44:11 +02:00
Alex	99ed18350f	block manager: refactor and fix monitoring/statistics	2023-09-07 12:41:36 +02:00
Alex	f38a31b330	block manager: avoid incorrect data_dir configs and avoid losing files	2023-09-06 17:49:30 +02:00
Alex	a44f486931	block manager: refactoring & increase max worker count to 8	2023-09-06 16:35:28 +02:00
Alex	93114a9747	block manager: refactoring	2023-09-06 16:35:28 +02:00
Alex	1b8c265c14	block manager: get rid of check_block_status	2023-09-06 16:35:28 +02:00
Alex	a09f86729c	block manager: move blocks in write_block if necessary	2023-09-06 16:35:28 +02:00
Alex	887b3233f4	block manager: use data paths from layout	2023-09-06 16:35:28 +02:00
Alex	6c420c0880	block manager: multi-directory layout computation	2023-09-06 16:35:28 +02:00
Alex	71c0188055	block manager: skeleton for multi-hdd support	2023-09-06 16:35:28 +02:00
Alex	e7e164a280	Make fsync an option for meta and data	2023-06-09 16:23:21 +02:00
Jonathan Davies	9c788059e2	block/manager.rs: In is_block_compressed - check which compression_level ... is configured on a node and check for raw block first if compression is disabled (to help reduce syscalls during a scrub).	2023-05-09 10:28:19 +01:00
Jonathan Davies	d218f475cb	block/manager.rs: Set defaults for scrub_persister.	2023-03-09 17:08:47 +00:00
Jonathan Davies	148b66b843	block/manager.rs: Display scrub-next-run.	2023-03-06 13:43:09 +00:00
Jonathan Davies	4cfb469d2b	block/metrics.rs: Added compression_level metric.	2023-01-10 10:40:03 +00:00
Alex	02e8eb167e	Merge pull request 'PutObject: better cleanup when request is interrupted in the middle' (#462) from interrupted-cleanup into main ... Reviewed-on: #462	2023-01-04 14:43:45 +00:00
Alex	f3f27293df	Uniform framework for bg variable management	2023-01-04 13:07:13 +01:00
Alex	936b6cb563	When saving block, delete .tmp file if we could not complete	2023-01-03 17:34:26 +01:00
Alex	d56c472712	Refactor background runner and get rid of job worker	2022-12-14 12:51:42 +01:00
Alex	2183518edc	Spawn all background workers in a separate step	2022-12-14 12:28:07 +01:00
Alex	041b60ed1d	Add block.rc_size, table.size and table.merkle_tree_size metrics	2022-12-13 15:54:03 +01:00
Alex	d6040e32a6	cli: prettier table in garage stats	2022-12-13 15:43:22 +01:00
Alex	687660b27f	Implement block list-errors and block info	2022-12-13 14:23:45 +01:00
Alex	56592e1853	RPC performance changes ... - configurable ping timeout - single, much higher, configurable RPC timeout - no more concurrency semaphore	2022-09-19 20:31:00 +02:00
Alex	b823151a0b	improvements in block manager	2022-09-12 16:57:38 +02:00
Alex	6b958979bd	Merge branch 'lx-perf-improvements' into netapp-stream-body	2022-09-06 22:13:01 +02:00
Alex	c2cc08852b	Reenable node ordering	2022-09-06 19:31:42 +02:00
Alex	07e6bcde85	Merge branch 'main' into lx-perf-improvements	2022-09-05 12:40:17 +02:00
Alex	5d4b937a00	Ability to have up to 4 concurrently working resync workers	2022-09-02 17:18:13 +02:00
Alex	5e8baa433d	Make BlockManagerLocked fully private again	2022-09-02 16:52:22 +02:00
Alex	47be652a1f	block manager: refactor: split resync into separate file	2022-09-02 16:47:15 +02:00
Alex	943d76c583	Ability to dynamically set resync tranquility	2022-09-02 15:34:21 +02:00
Alex	99b532b85b	Apply PRIO_SECONDARY to block data transfers	2022-09-01 16:35:43 +02:00
Alex	df094bd807	Less strict timeouts	2022-09-01 16:30:44 +02:00
Alex	bc977f9a7a	Update to Netapp with OrderTag support and exploit OrderTags	2022-09-01 12:58:20 +02:00