Add tranquilizer mechanism to improve on token bucket mechanism #145

Merged

lx merged 1 commit from tranquility into main

2021-11-04 13:16:43 +00:00

lx commented

2021-11-03 17:29:33 +00:00

Owner

Introduce new object: a Tranquilizer (see doc comment in tranquilizer.rs)
Replace token bucket by tranquilizer in scrub operation
Add tranquilizer to block resync loop, with constant tranquility factor currently set to 2 (meaning that at most 1/3 of time will be spent actually doing block resync operations)
Have deep thoughts about adding a tranquilizer to repair_versions and repair_block_refs: it's probably not worth it because network latency already naturally limits the throughput of these ops (which are not very intensive)
Think of whether we want to add a tranquilizer to the table sync loop and GC loop: we probably don't need to do it if we are not I/O bound on the metadata directory (and we are generally not when it is on SSD)

Documentation of the Tranquilizer object:

A tranquilizer is a helper object that is used to make
background operations not take up too much time.

Background operations are done in a loop that does the following:

do one step of the background process

tranquilize, i.e. wait some time to not overload the system

The tranquilizer observes how long the steps take, and keeps
in memory a number of observations. The tranquilize operation
simply sleeps k * avg(observed step times), where k is
the tranquility factor. For instance with a tranquility of 2,
the tranquilizer will sleep on average 2 units of time for every
1 unit of time spent doing the background task.

- [x] Introduce new object: a `Tranquilizer` (see doc comment in `tranquilizer.rs`) - [x] Replace token bucket by tranquilizer in scrub operation - [x] Add tranquilizer to block resync loop, with constant tranquility factor currently set to 2 (meaning that at most 1/3 of time will be spent actually doing block resync operations) - [x] Have deep thoughts about adding a tranquilizer to repair_versions and repair_block_refs: it's probably not worth it because network latency already naturally limits the throughput of these ops (which are not very intensive) - [ ] Think of whether we want to add a tranquilizer to the table sync loop and GC loop: we probably don't need to do it if we are not I/O bound on the metadata directory (and we are generally not when it is on SSD) Documentation of the Tranquilizer object: > A tranquilizer is a helper object that is used to make > background operations not take up too much time. > > Background operations are done in a loop that does the following: > - do one step of the background process > - tranquilize, i.e. wait some time to not overload the system > > The tranquilizer observes how long the steps take, and keeps > in memory a number of observations. The tranquilize operation > simply sleeps k * avg(observed step times), where k is > the tranquility factor. For instance with a tranquility of 2, > the tranquilizer will sleep on average 2 units of time for every > 1 unit of time spent doing the background task.

lx force-pushed tranquility from 57cec914f9 to cad7b89310

2021-11-03 17:30:00 +00:00

Compare

lx force-pushed tranquility from cad7b89310 to cdb2f66106

2021-11-03 17:45:21 +00:00

Compare

lx force-pushed tranquility from cdb2f66106 to 0d85322432

2021-11-03 17:54:34 +00:00

Compare

lx force-pushed tranquility from 0d85322432 to a24725f71b

2021-11-04 08:30:53 +00:00

Compare

lx force-pushed tranquility from a24725f71b to e190cb1173

2021-11-04 12:12:10 +00:00

Compare

lx force-pushed tranquility from e190cb1173 to 2090a6187f

2021-11-04 12:27:08 +00:00

Compare

lx changed title from ~~WIP: Add tranquilizer mechanism to improve on token bucket mechanism~~ to Add tranquilizer mechanism to improve on token bucket mechanism

2021-11-04 13:16:27 +00:00

lx merged commit 2090a6187f into main

2021-11-04 13:16:43 +00:00

No reviewers

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Deuxfleurs/garage#145

No description provided.