Background task manager #332

Merged

lx merged 35 commits from background-task-manager into main 2022-07-08 11:30:32 +00:00

Conversation 0 Commits 35 Files Changed 27 +2050 -738

lx commented 2022-06-21 12:08:05 +00:00

Owner

Copy Link

• New background worker trait
• Adapt all current workers to use new API
• Command to list currently running workers, and whether they are active, idle, or dead
• Error reporting
• Optimizations
  • Merkle updater: several items per iteration
  • Use tokio::task::spawn_blocking where appropriate so that CPU-intensive tasks don't block other things going on
• scrub:
  • have only one worker with a channel to start/pause/cancel
  • automatic scrub
  • ability to view and change tranquility from CLI
  • persistence of a few info
• Testing

- [x] New background worker trait - [x] Adapt all current workers to use new API - [x] Command to list currently running workers, and whether they are active, idle, or dead - [x] Error reporting - Optimizations - [x] Merkle updater: several items per iteration - [ ] Use `tokio::task::spawn_blocking` where appropriate so that CPU-intensive tasks don't block other things going on - scrub: - [x] have only one worker with a channel to start/pause/cancel - [x] automatic scrub - [x] ability to view and change tranquility from CLI - [x] persistence of a few info - [ ] Testing

lx added 3 commits 2022-06-21 12:08:06 +00:00

e12bc3b595

First try on background worker manager

3119ea59b0

New worker semantics applied to garage_table

269f996fd0

Block manager with new worker

lx added 1 commit 2022-06-21 12:24:11 +00:00

b8338dea56

New worker for index counter propagator

lx added 1 commit 2022-06-21 13:28:12 +00:00

a855c54bdb

Online repair new workers, except blocks and scrub

lx added 1 commit 2022-06-21 14:00:20 +00:00

ba1ace6cf6

Block repair with new worker semantics

lx added 1 commit 2022-06-21 14:27:21 +00:00

14337d2a56

rename things

lx added 1 commit 2022-06-21 15:18:38 +00:00

10c886111e

Rename things, garage worker list cmd

lx added 1 commit 2022-06-21 15:22:26 +00:00

708dab6e0f

fix clippy

lx added 1 commit 2022-06-21 16:04:22 +00:00

8fb8569912

fix clippy

lx added 1 commit 2022-06-21 16:07:07 +00:00

f82cf164f5

Update Cargo.nix

lx added 2 commits 2022-06-24 08:31:39 +00:00

59b43914d4

(to test) error reporting and throttling at higher layer

95ffba343f

Error reporting

lx added 2 commits 2022-06-24 09:05:22 +00:00

0837b3dacd

Slightly improve blocking code, add info to resync worker

e7810e9cb3

Smaller batches for index counter propagation

lx added 1 commit 2022-06-27 09:58:36 +00:00

928394cc32

(makefile with run release)

lx added 1 commit 2022-06-27 10:01:36 +00:00

fdfe7dd60d

Fix clippy lint

lx added 1 commit 2022-06-27 10:06:29 +00:00

fc50724256

Update cargo.nix

lx added 1 commit 2022-06-27 14:53:12 +00:00

0e5175abee

Report progress of scrub and block repair

lx added 1 commit 2022-06-27 15:58:16 +00:00

247dbcd598

Only one scrub worker (wip)

lx added 1 commit 2022-06-28 14:09:45 +00:00

f1c972289d

update cargo.nix

lx added 1 commit 2022-06-28 14:59:53 +00:00

b053fc0518

Persist scrub worker thing in Persister

lx added 1 commit 2022-06-28 15:11:16 +00:00

b0a181e17e

Count corruptions in scrub worker

lx added 1 commit 2022-06-28 15:31:36 +00:00

fc2bc8b0ca

cargo fmt

lx added 1 commit 2022-06-28 15:43:07 +00:00

368414a261

Small things

lx added 1 commit 2022-06-28 15:44:04 +00:00

c4be56704d

remove useless info!()

lx added 1 commit 2022-07-04 10:46:51 +00:00

17d0254c7c

Merge branch 'main' into background-task-manager

lx added 1 commit 2022-07-04 10:53:52 +00:00

5ee9cb7768

add delays in k2v test_items_and_indices

lx changed title from WIP: Background task manager to Background task manager 2022-07-04 11:03:04 +00:00

lx added 1 commit 2022-07-08 07:55:41 +00:00

0200eae679

cleanup

lx added 1 commit 2022-07-08 08:10:42 +00:00

4cc9a648ab

Small refactoring

lx added 1 commit 2022-07-08 08:16:56 +00:00

4312623930

whoops and small refactoring

lx added 1 commit 2022-07-08 08:32:51 +00:00

0f660b086c

fix comments

lx added 1 commit 2022-07-08 08:40:01 +00:00

d1cf1a0fa6

Rename WorkerStatus to WorkerState ...

because it's a state in a state machine

lx added 4 commits 2022-07-08 08:43:29 +00:00

b6d59ec19a

Fix poll item when item didn't change

fe3fa83de7 Publish k2v-client crate to crates.io (#337) ...

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: #337
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>

aab34bfe54

add delays in k2v test_items_and_indices

18d014dcc5

Merge branch 'main' into background-task-manager

lx added 1 commit 2022-07-08 08:45:27 +00:00

08cd5f2f1d

cargo fmt

lx merged commit 4f38cadf6e into main 2022-07-08 11:30:32 +00:00

lx referenced this issue from a commit 2022-07-08 11:30:32 +00:00

Background task manager (#332)

lx referenced this pull request 2022-09-14 11:12:42 +00:00

Improved monitoring of repair operations #207

lx referenced this pull request 2022-09-14 11:15:27 +00:00

Automatic scrub regularly #255

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

AdminAPI

Issues concerning the administration API

  

Bug

  

Check AWS

The behaviour should be checked against AWS S3

  

CI

Issues with Nix, Drone, Rust Compiler, etc.

  

Correctness

Issues of theoretical correctness that may impact edge-case scenarios

  

Critical

  

Documentation

  

Ideas

For abstract ideas/proposal to make Garage better, to better communicate our not yet achieved aims

  

Improvement

  

Low priority

For issues that we don't want to spend time on until made absolutely necessary

  

Newcomer

A bug that should be newcomer friendly.

  

Performance

  

S3 Compatibility

  

Testing

  

Usability

Propositions to make Garage easier to operate, by improving error reporting, CLI, etc.

No Label

AdminAPI

Bug

Check AWS

CI

Correctness

Critical

Documentation

Ideas

Improvement

Low priority

Newcomer

Performance

S3 Compatibility

Testing

Usability

Milestone

Clear milestone

No items

No Milestone

Assignees

Clear assignees

No Assignees

1 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: Deuxfleurs/garage#332

No description provided.