Deuxfleurs/garage

Fork 73

WIP: Allow external HTTP client (Aerogramme testing) #731

Draft

quentin wants to merge 1 commit from k2v/shared_http_client into main

quentin commented

2024-02-23 16:13:19 +00:00

Owner

Excerpt of a blog post on Aerogramme to come:

Another concern is the RAM consumption with the IDLE feature. The real cause is that we retain in-memory the full user profile (mailbox data, IO connectors, etc.): we should instead keep only the minimum data to be waken-up. That's the ideal fix, the final solution, that would take lots of time to design and implement. This fix is not necessary now, instead we can simply try to optimize the size of a full user profile in memory. On this aspect, the aws-sdk-s3 crate has the following note:

Client construction is expensive due to connection thread pool initialization, and should be done once at application start-up.

Digging deeper in the crate dependencies, we learn from the aws-smithy-runtime crate, we can read:

[Constructing] a Hyper client with the default TLS implementation (rustls) [...] can be useful when you want to share a Hyper connector between multiple generated Smithy clients.

But I also need to share an HTTP Client for my K2V session if I want to correctly benchmark Aerogramme.
That's why I am extending K2V to share an HTTP Client.

Not sure it will work or will improve anything, so no need to merge for now.
Just testing...

Excerpt of a blog post on Aerogramme to come: --- Another concern is the RAM consumption with the IDLE feature. The real cause is that we retain in-memory the full user profile (mailbox data, IO connectors, etc.): we should instead keep only the minimum data to be waken-up. That's the ideal fix, the final solution, that would take lots of time to design and implement. This fix is not necessary now, instead we can simply try to optimize the size of a full user profile in memory. On this aspect, the aws-sdk-s3 crate has the following note: > Client construction is expensive due to connection thread pool initialization, and should be done once at application start-up. Digging deeper in the crate dependencies, we learn from the aws-smithy-runtime crate, we can read: > [Constructing] a Hyper client with the default TLS implementation (rustls) [...] can be useful when you want to share a Hyper connector between multiple generated Smithy clients. --- But I also need to share an HTTP Client for my K2V session if I want to correctly benchmark Aerogramme. That's why I am extending K2V to share an HTTP Client. Not sure it will work or will improve anything, so no need to merge for now. Just testing...

quentin added 1 commit 2024-02-23 16:13:19 +00:00

Allow external HTTP client

ci/woodpecker/push/debug Pipeline was successful

Details

ci/woodpecker/pr/debug Pipeline was successful

Details

8b35a946d9

maximilien approved these changes 2024-08-07 22:02:23 +00:00

maximilien added this to the v1.1 milestone 2024-08-07 22:02:30 +00:00

maximilien removed this from the v1.1 milestone 2024-08-07 22:02:55 +00:00

maximilien added the

action

more-info-needed

label 2024-08-07 22:03:24 +00:00

ci/woodpecker/push/debug Pipeline was successful

Details

ci/woodpecker/pr/debug Pipeline was successful

Required

Details

This pull request is marked as a work in progress.

This branch is out-of-date with the base branch

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin k2v/shared_http_client:k2v/shared_http_client

git checkout k2v/shared_http_client

No reviewers

No milestone

No project

No assignees

2 participants

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Deuxfleurs/garage#731

No description provided.