improve internal item counter mechanisms and implement bucket quotas #326

Merged

lx merged 23 commits from counters into main 2022-06-15 18:20:31 +00:00

Conversation 1 Commits 23 Files Changed 32 +962 -347

lx commented 2022-06-08 08:44:31 +00:00

Owner

Copy Link

• Refactoring of internal counting API
• Repair procedure for counters (it's an offline procedure!!!)
• New counter for objects in buckets
• Add quotas to buckets struct
• Add CLI to manage bucket quotas
• Add admin API to manage bucket quotas
• Apply quotas by adding checks on put operations
• Proof-read

- [x] Refactoring of internal counting API - [x] Repair procedure for counters (it's an offline procedure!!!) - [x] New counter for objects in buckets - [x] Add quotas to buckets struct - [x] Add CLI to manage bucket quotas - [x] Add admin API to manage bucket quotas - [x] Apply quotas by adding checks on put operations - [x] Proof-read

lx added 1 commit 2022-06-08 08:44:32 +00:00

425fe56be8

Some refactoring of the index counter API

lx added 3 commits 2022-06-08 15:50:23 +00:00

f8a6fff2b7

First implementation of counter repair procedure

ab374e7178

Add .clear() on db trees

03e811bbbf

Call counter tables 'v2' because schema changed since last time

lx added 1 commit 2022-06-08 18:03:46 +00:00

17e1111393

First iteration of bucket object counters

lx added 1 commit 2022-06-08 18:16:10 +00:00

5b38f4cd26

Show bucket sizes in garage bucket list

lx added 1 commit 2022-06-09 12:50:36 +00:00

99f6ca4388

Silence clippy warning

lx added 1 commit 2022-06-09 13:06:34 +00:00

ea1022f832

Update Cargo.nix

lx added 1 commit 2022-06-09 13:43:42 +00:00

c1baa10202

Add quotas to bucket table and show them in CLI

lx added 1 commit 2022-06-09 13:56:37 +00:00

fe5cf8a530

CLI operation bucket set-quotas

lx added 1 commit 2022-06-09 14:41:01 +00:00

1c2e5d9c36

Quotas are now applied on uploads

lx added 2 commits 2022-06-10 08:33:04 +00:00

138e13071b

Fix garage_db build on 32-bit systems

c6c4f007f0

Merge branch 'main' into counters

lx added 1 commit 2022-06-10 08:52:37 +00:00

ef7db797bd

New admin API endpoint: UpdateBucket, that unifies website config and quotas

lx added 1 commit 2022-06-10 09:02:13 +00:00

9695e747e6

whoops bad typo here

lx added 1 commit 2022-06-10 09:03:08 +00:00

8bbd0db43d

reformat a bit

lx added 2 commits 2022-06-10 09:08:33 +00:00

6ae8bbeb13

typo in error message

e6ce8702d4

change k2v-only code

lx added 1 commit 2022-06-10 09:13:25 +00:00

ffe05c3c7d

Change bucket list command

lx added 1 commit 2022-06-10 09:21:17 +00:00

bc29d77ed3

Change object counter table to use full replication

lx added 2 commits 2022-06-10 09:28:09 +00:00

c054de43dd

Revert "Change object counter table to use full replication" ...

This reverts commit bc29d77ed3.

180e7fef0a

Actually distribute counters over nodes

lx added 1 commit 2022-06-10 09:30:50 +00:00

92d912faef

Update json in admin api doc

lx added 1 commit 2022-06-10 09:39:24 +00:00

5de641700f

Big whoops

lx changed title from WIP: improve internal item counter mechanisms and implement bucket quotas to improve internal item counter mechanisms and implement bucket quotas 2022-06-10 09:47:00 +00:00

lx commented 2022-06-13 12:09:09 +00:00

Author

Owner

Copy Link

Fix #71

Fix #71

lx merged commit 77e3fd6db2 into main 2022-06-15 18:20:31 +00:00

lx referenced this issue from a commit 2022-06-15 18:20:31 +00:00

improve internal item counter mechanisms and implement bucket quotas (#326)

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

AdminAPI

Issues concerning the administration API

  

Bug

  

Check AWS

The behaviour should be checked against AWS S3

  

CI

Issues with Nix, Drone, Rust Compiler, etc.

  

Correctness

Issues of theoretical correctness that may impact edge-case scenarios

  

Critical

  

Documentation

  

Ideas

For abstract ideas/proposal to make Garage better, to better communicate our not yet achieved aims

  

Improvement

  

Low priority

For issues that we don't want to spend time on until made absolutely necessary

  

Newcomer

A bug that should be newcomer friendly.

  

Performance

  

S3 Compatibility

  

Testing

  

Usability

Propositions to make Garage easier to operate, by improving error reporting, CLI, etc.

No Label

AdminAPI

Bug

Check AWS

CI

Correctness

Critical

Documentation

Ideas

Improvement

Low priority

Newcomer

Performance

S3 Compatibility

Testing

Usability

Milestone

Clear milestone

No items

No Milestone

Assignees

Clear assignees

No Assignees

1 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: Deuxfleurs/garage#326

No description provided.