537 commits

Author	SHA1	Message	Date
Alex	99976e11f8	Merge branch 'main' into admin-api	2022-05-18 22:35:49 +02:00
trinity-1686a	64c193e3db	Add a K2V client library and CLI (#303) ... lib.rs could use getting split in modules, but I'm not sure how exactly Co-authored-by: trinity-1686a <trinity@deuxfleurs.fr> Reviewed-on: #303 Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>	2022-05-18 22:24:09 +02:00
Alex	5367f8adb2	Refactor bucket emptiness check and add k2v check	2022-05-18 10:09:51 +02:00
Alex	30e393b439	Fix fmt	2022-05-18 00:32:51 +02:00
Alex	926b3c0fad	Rename error varian for Clippy	2022-05-18 00:27:57 +02:00
Alex	dcfa408887	Implement ImportKey	2022-05-17 19:02:13 +02:00
Alex	70383b4363	Implement ConnectClusterNodes	2022-05-17 18:43:47 +02:00
Alex	440a577563	Prefix all APIs with v0/	2022-05-17 17:51:38 +02:00
Alex	5072dbd228	Add PutBucketWebsite and DeleteBucketWebsite to admin api	2022-05-17 17:44:00 +02:00
Alex	2ce3513c10	Specify and implement {Global,Local}{Alias,Unalias}Bucket	2022-05-17 17:16:29 +02:00
Alex	e92c52eb65	refactor	2022-05-17 17:02:38 +02:00
Alex	8b1338ef2f	Fix error code	2022-05-17 16:48:23 +02:00
Alex	8ff95f09c9	Return website config in GetBucketInfo, use serde(rename_all)	2022-05-17 16:42:13 +02:00
Alex	86a3fe8ec5	Merge branch 'main' into admin-api	2022-05-17 13:10:53 +02:00
Alex	c692f55d5c	K2V: Fix end parameter and add tests (fix #305)	2022-05-17 11:50:23 +02:00
Alex	7b474855e3	Make background runner terminate correctly	2022-05-17 11:38:31 +02:00
Alex	ec50ffac42	Remove useless string conversions	2022-05-13 19:49:04 +02:00
Alex	d7736cb614	Revert useless thing	2022-05-13 19:43:40 +02:00
Alex	8033bdb0b4	More precisions in errors & small refactoring	2022-05-13 19:36:17 +02:00
Alex	5a535788fc	Json body for custom errors	2022-05-13 19:28:23 +02:00
Alex	ea325d78d3	More error refactoring	2022-05-13 19:18:51 +02:00
Alex	ec16d166f9	Separate error types for k2v and signature	2022-05-13 15:43:44 +02:00
Alex	7a5d329e49	More error refactoring	2022-05-13 15:21:32 +02:00
Alex	f82b938033	Rename error::Error to s3::error::Error	2022-05-13 15:10:52 +02:00
Alex	96b11524d5	Error refactoring	2022-05-13 15:04:53 +02:00
Alex	c0fb9fd0fe	Common error type and admin error type that uses it	2022-05-13 14:30:30 +02:00
Alex	983037d965	Possibility of different error types for different APIs	2022-05-13 13:51:34 +02:00
Alex	e4e1f8f0d6	Fix clippy	2022-05-12 17:11:45 +02:00
Alex	e7ddba53e3	Slightly more detailed error reporting from helper	2022-05-12 17:10:25 +02:00
Alex	ed76893581	Simplify	2022-05-12 11:21:23 +02:00
Alex	fc2f73ddb5	BucketAllowKey and BucketDenyKey	2022-05-12 11:19:41 +02:00
Alex	fe399a3265	DeleteBucket	2022-05-12 11:02:36 +02:00
Alex	de1a5b87b6	CreateBucket	2022-05-12 10:45:09 +02:00
Alex	2b93a01d2b	ListBucket and GetBucketInfo	2022-05-12 10:20:34 +02:00
Alex	393b76ecba	Implement CreateKey, DeleteKey and rudimentary UpdateKey	2022-05-11 11:40:26 +02:00
Alex	5c00c9fb46	First key endpoints: ListKeys and GetKeyInfo	2022-05-11 11:10:28 +02:00
Alex	f97a7845e9	Add API access key admin endpoints	2022-05-11 10:27:40 +02:00
Alex	bb6ec9ebd9	Update Cargo.nix and improve log message	2022-05-10 13:36:35 +02:00
Alex	dd54d0b2b1	Refactor code for apply/revert, implement Update/Apply/RevertLayout	2022-05-10 13:25:10 +02:00
Alex	01c4876fb4	Specify remaining cluster-related endpoints	2022-05-10 13:25:10 +02:00
Alex	ec03e3d16c	Fmt & cleanup	2022-05-10 13:25:10 +02:00
Alex	7a19daafbd	Implement /status Admin endpoint	2022-05-10 13:25:10 +02:00
Alex	99fcfa3844	Make background runner terminate correctly	2022-05-10 13:25:10 +02:00
Alex	633958c7b1	Refactor admin API to be in api/admin and use common code	2022-05-10 13:25:06 +02:00
Alex	5768bf3622	First implementation of K2V (#293) ... **Specification:** View spec at [this URL](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/k2v/doc/drafts/k2v-spec.md) - [x] Specify the structure of K2V triples - [x] Specify the DVVS format used for causality detection - [x] Specify the K2V index (just a counter of number of values per partition key) - [x] Specify single-item endpoints: ReadItem, InsertItem, DeleteItem - [x] Specify index endpoint: ReadIndex - [x] Specify multi-item endpoints: InsertBatch, ReadBatch, DeleteBatch - [x] Move to JSON objects instead of tuples - [x] Specify endpoints for polling for updates on single values (PollItem) **Implementation:** - [x] Table for K2V items, causal contexts - [x] Indexing mechanism and table for K2V index - [x] Make API handlers a bit more generic - [x] K2V API endpoint - [x] K2V API router - [x] ReadItem - [x] InsertItem - [x] DeleteItem - [x] PollItem - [x] ReadIndex - [x] InsertBatch - [x] ReadBatch - [x] DeleteBatch **Testing:** - [x] Just a simple Python script that does some requests to check visually that things are going right (does not contain parsing of results or assertions on returned values) - [x] Actual tests: - [x] Adapt testing framework - [x] Simple test with InsertItem + ReadItem - [x] Test with several Insert/Read/DeleteItem + ReadIndex - [x] Test all combinations of return formats for ReadItem - [x] Test with ReadBatch, InsertBatch, DeleteBatch - [x] Test with PollItem - [x] Test error codes - [ ] Fix most broken stuff - [x] test PollItem broken randomly - [x] when invalid causality tokens are given, errors should be 4xx not 5xx **Improvements:** - [x] Descending range queries - [x] Specify - [x] Implement - [x] Add test - [x] Batch updates to index counter - [x] Put K2V behind `k2v` feature flag Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: #293 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>	2022-05-10 13:16:57 +02:00
Alex	def78c5e6f	Update netapp to 0.4.4, fix #300	2022-05-09 12:08:47 +02:00
Alex	277a20ec44	Fix layout show to not show changes when there are no changes (#297) ... fixes #295, partially Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: #297 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>	2022-05-09 11:14:55 +02:00
Alex	c9ef3e461b	fix clippy	2022-04-19 12:50:40 +02:00
Alex	c93008d333	Prettier code for defragmentation	2022-04-19 12:50:40 +02:00
Alex	e5341ca47b	Defragmentation in UploadPartCopy: first pass (not pretty but it compiles)	2022-04-19 12:50:36 +02:00