ragazenta/garage
1
0
Fork 0
forked from Deuxfleurs/garage
Code Pull requests Activity
1756 commits 2 branches 74 tags 54 MiB
Commit graph

59 commits

Author SHA1 Message Date
Alex Auvolat
cd1069c1d4
[refactor-block] refactor DataBlock and DataBlockPath 2024-02-23 12:15:52 +01:00
Alex Auvolat
07c7895948
[refactor-block] simplify rpc_get_block 2024-02-23 11:54:40 +01:00
Alex Auvolat
9b41f4ff20
[refactor-block] move read_stream_to_end to garage_net 2024-02-23 11:46:57 +01:00
Alex Auvolat
93552b9275
[refactor-block] Remove redundant BlockStream type 2024-02-23 11:33:38 +01:00
Alex Auvolat
5ea24254a9
[import-netapp] import Netapp code into Garage codebase 2024-02-15 12:15:07 +01:00
Alex Auvolat
22332e6c35
[dep-upgrade-202402] simplify/refactor GetObject 2024-02-05 20:26:33 +01:00
Alex Auvolat
7f9ba49c71 block manager: remove data_dir field 2023-09-11 11:57:36 +02:00
Alex Auvolat
be91ef6294 block manager: fix bug where rebalance didn't delete old copies 2023-09-07 16:04:03 +02:00
Alex Auvolat
6b008b5bd3 block manager: add rebalance operation to rebalance multi-hdd setups 2023-09-07 13:44:11 +02:00
Alex Auvolat
99ed18350f block manager: refactor and fix monitoring/statistics 2023-09-07 12:41:36 +02:00
Alex Auvolat
f38a31b330 block manager: avoid incorrect data_dir configs and avoid losing files 2023-09-06 17:49:30 +02:00
Alex Auvolat
a44f486931 block manager: refactoring & increase max worker count to 8 2023-09-06 16:35:28 +02:00
Alex Auvolat
93114a9747 block manager: refactoring 2023-09-06 16:35:28 +02:00
Alex Auvolat
1b8c265c14 block manager: get rid of check_block_status 2023-09-06 16:35:28 +02:00
Alex Auvolat
a09f86729c block manager: move blocks in write_block if necessary 2023-09-06 16:35:28 +02:00
Alex Auvolat
887b3233f4 block manager: use data paths from layout 2023-09-06 16:35:28 +02:00
Alex Auvolat
6c420c0880 block manager: multi-directory layout computation 2023-09-06 16:35:28 +02:00
Alex Auvolat
71c0188055 block manager: skeleton for multi-hdd support 2023-09-06 16:35:28 +02:00
Alex Auvolat
e7e164a280 Make fsync an option for meta and data 2023-06-09 16:23:21 +02:00
Jonathan Davies
9c788059e2 block/manager.rs: In is_block_compressed - check which compression_level 
is configured on a node and check for raw block first if compression is
disabled (to help reduce syscalls during a scrub).
 2023-05-09 10:28:19 +01:00
Jonathan Davies
d218f475cb block/manager.rs: Set defaults for scrub_persister. 2023-03-09 17:08:47 +00:00
Jonathan Davies
148b66b843 block/manager.rs: Display scrub-next-run. 2023-03-06 13:43:09 +00:00
Jonathan Davies
4cfb469d2b block/metrics.rs: Added compression_level metric. 2023-01-10 10:40:03 +00:00
Alex
02e8eb167e Merge pull request 'PutObject: better cleanup when request is interrupted in the middle' (#462) from interrupted-cleanup into main 
Reviewed-on: Deuxfleurs/garage#462
 2023-01-04 14:43:45 +00:00
Alex Auvolat
f3f27293df
Uniform framework for bg variable management 2023-01-04 13:07:13 +01:00
Alex Auvolat
936b6cb563
When saving block, delete .tmp file if we could not complete 2023-01-03 17:34:26 +01:00
Alex Auvolat
d56c472712
Refactor background runner and get rid of job worker 2022-12-14 12:51:42 +01:00
Alex Auvolat
2183518edc
Spawn all background workers in a separate step 2022-12-14 12:28:07 +01:00
Alex Auvolat
041b60ed1d
Add block.rc_size, table.size and table.merkle_tree_size metrics 2022-12-13 15:54:03 +01:00
Alex Auvolat
d6040e32a6
cli: prettier table in garage stats 2022-12-13 15:43:22 +01:00
Alex Auvolat
687660b27f
Implement block list-errors and block info 2022-12-13 14:23:45 +01:00
Alex Auvolat
56592e1853
RPC performance changes 
- configurable ping timeout
- single, much higher, configurable RPC timeout
- no more concurrency semaphore
 2022-09-19 20:31:00 +02:00
Alex Auvolat
b823151a0b
improvements in block manager 2022-09-12 16:57:38 +02:00
Alex Auvolat
6b958979bd
Merge branch 'lx-perf-improvements' into netapp-stream-body 2022-09-06 22:13:01 +02:00
Alex Auvolat
c2cc08852b
Reenable node ordering 2022-09-06 19:31:42 +02:00
Alex Auvolat
07e6bcde85
Merge branch 'main' into lx-perf-improvements 2022-09-05 12:40:17 +02:00
Alex Auvolat
5d4b937a00
Ability to have up to 4 concurrently working resync workers 2022-09-02 17:18:13 +02:00
Alex Auvolat
5e8baa433d
Make BlockManagerLocked fully private again 2022-09-02 16:52:22 +02:00
Alex Auvolat
47be652a1f
block manager: refactor: split resync into separate file 2022-09-02 16:47:15 +02:00
Alex Auvolat
943d76c583
Ability to dynamically set resync tranquility 2022-09-02 15:34:21 +02:00
Alex Auvolat
99b532b85b
Apply PRIO_SECONDARY to block data transfers 2022-09-01 16:35:43 +02:00
Alex Auvolat
df094bd807
Less strict timeouts 2022-09-01 16:30:44 +02:00
Alex Auvolat
bc977f9a7a
Update to Netapp with OrderTag support and exploit OrderTags 2022-09-01 12:58:20 +02:00
Alex Auvolat
70231d68b2
Fix bytes_read counter 2022-08-31 19:44:27 +02:00
Alex Auvolat
e935861854
Factor out node request order selection logic & use in manager 2022-07-29 12:25:03 +02:00
Alex Auvolat
605a630333
Use streaming in block manager 2022-07-29 12:25:02 +02:00
Alex Auvolat
8e7e680afe
First adaptation to WIP netapp with streaming body 2022-07-29 12:25:02 +02:00
Alex Auvolat
2f111e6b3d
Performance improvements: 
- reduce contention on mutation_lock by having 256 of them
- better lmdb defaults
 2022-07-29 12:24:48 +02:00
Alex Auvolat
1b2e1296eb
Compute hashes on dedicated threads 2022-07-29 12:24:44 +02:00
Alex
4f38cadf6e Background task manager (#332) 
- [x] New background worker trait
- [x] Adapt all current workers to use new API
- [x] Command to list currently running workers, and whether they are active, idle, or dead
- [x] Error reporting
- Optimizations
  - [x] Merkle updater: several items per iteration
  - [ ] Use `tokio::task::spawn_blocking` where appropriate so that CPU-intensive tasks don't block other things going on
- scrub:
  - [x] have only one worker with a channel to start/pause/cancel
  - [x] automatic scrub
  - [x] ability to view and change tranquility from CLI
  - [x] persistence of a few info
- [ ] Testing

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: Deuxfleurs/garage#332
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
 2022-07-08 13:30:26 +02:00