190 commits

Author	SHA1	Message	Date
Alex Auvolat	d7736cb614	Revert useless thing	2022-05-13 19:43:40 +02:00
Alex Auvolat	8033bdb0b4	More precisions in errors & small refactoring	2022-05-13 19:36:17 +02:00
Alex Auvolat	5a535788fc	Json body for custom errors	2022-05-13 19:28:23 +02:00
Alex Auvolat	ea325d78d3	More error refactoring	2022-05-13 19:18:51 +02:00
Alex Auvolat	ec16d166f9	Separate error types for k2v and signature	2022-05-13 15:43:44 +02:00
Alex Auvolat	7a5d329e49	More error refactoring	2022-05-13 15:21:32 +02:00
Alex Auvolat	f82b938033	Rename error::Error to s3::error::Error	2022-05-13 15:10:52 +02:00
Alex Auvolat	96b11524d5	Error refactoring	2022-05-13 15:04:53 +02:00
Alex Auvolat	c0fb9fd0fe	Common error type and admin error type that uses it	2022-05-13 14:30:30 +02:00
Alex Auvolat	983037d965	Possibility of different error types for different APIs	2022-05-13 13:51:34 +02:00
Alex Auvolat	e4e1f8f0d6	Fix clippy	2022-05-12 17:11:45 +02:00
Alex Auvolat	e7ddba53e3	Slightly more detailed error reporting from helper	2022-05-12 17:10:25 +02:00
Alex Auvolat	ed76893581	Simplify	2022-05-12 11:21:23 +02:00
Alex Auvolat	fc2f73ddb5	BucketAllowKey and BucketDenyKey	2022-05-12 11:19:41 +02:00
Alex Auvolat	fe399a3265	DeleteBucket	2022-05-12 11:02:36 +02:00
Alex Auvolat	de1a5b87b6	CreateBucket	2022-05-12 10:45:09 +02:00
Alex Auvolat	2b93a01d2b	ListBucket and GetBucketInfo	2022-05-12 10:20:34 +02:00
Alex Auvolat	393b76ecba	Implement CreateKey, DeleteKey and rudimentary UpdateKey	2022-05-11 11:40:26 +02:00
Alex Auvolat	5c00c9fb46	First key endpoints: ListKeys and GetKeyInfo	2022-05-11 11:10:28 +02:00
Alex Auvolat	f97a7845e9	Add API access key admin endpoints	2022-05-11 10:27:40 +02:00
Alex Auvolat	dd54d0b2b1	Refactor code for apply/revert, implement Update/Apply/RevertLayout	2022-05-10 13:25:10 +02:00
Alex Auvolat	01c4876fb4	Specify remaining cluster-related endpoints	2022-05-10 13:25:10 +02:00
Alex Auvolat	ec03e3d16c	Fmt & cleanup	2022-05-10 13:25:10 +02:00
Alex Auvolat	7a19daafbd	Implement /status Admin endpoint	2022-05-10 13:25:10 +02:00
Alex Auvolat	633958c7b1	Refactor admin API to be in api/admin and use common code	2022-05-10 13:25:06 +02:00
Alex	5768bf3622	First implementation of K2V (#293) ... **Specification:** View spec at [this URL](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/k2v/doc/drafts/k2v-spec.md) - [x] Specify the structure of K2V triples - [x] Specify the DVVS format used for causality detection - [x] Specify the K2V index (just a counter of number of values per partition key) - [x] Specify single-item endpoints: ReadItem, InsertItem, DeleteItem - [x] Specify index endpoint: ReadIndex - [x] Specify multi-item endpoints: InsertBatch, ReadBatch, DeleteBatch - [x] Move to JSON objects instead of tuples - [x] Specify endpoints for polling for updates on single values (PollItem) **Implementation:** - [x] Table for K2V items, causal contexts - [x] Indexing mechanism and table for K2V index - [x] Make API handlers a bit more generic - [x] K2V API endpoint - [x] K2V API router - [x] ReadItem - [x] InsertItem - [x] DeleteItem - [x] PollItem - [x] ReadIndex - [x] InsertBatch - [x] ReadBatch - [x] DeleteBatch **Testing:** - [x] Just a simple Python script that does some requests to check visually that things are going right (does not contain parsing of results or assertions on returned values) - [x] Actual tests: - [x] Adapt testing framework - [x] Simple test with InsertItem + ReadItem - [x] Test with several Insert/Read/DeleteItem + ReadIndex - [x] Test all combinations of return formats for ReadItem - [x] Test with ReadBatch, InsertBatch, DeleteBatch - [x] Test with PollItem - [x] Test error codes - [ ] Fix most broken stuff - [x] test PollItem broken randomly - [x] when invalid causality tokens are given, errors should be 4xx not 5xx **Improvements:** - [x] Descending range queries - [x] Specify - [x] Implement - [x] Add test - [x] Batch updates to index counter - [x] Put K2V behind `k2v` feature flag Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: #293 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>	2022-05-10 13:16:57 +02:00
Alex Auvolat	c9ef3e461b	fix clippy	2022-04-19 12:50:40 +02:00
Alex Auvolat	c93008d333	Prettier code for defragmentation	2022-04-19 12:50:40 +02:00
Alex Auvolat	e5341ca47b	Defragmentation in UploadPartCopy: first pass (not pretty but it compiles)	2022-04-19 12:50:36 +02:00
withinboredom	a4f9f19ac3	remove size limitation in UploadPartCopy (#280) ... This removes the >1mb s3_copy restriction. This restriction doesn't seem to be documented anywhere (I could be wrong). It also causes some software to fail (such as #248). Co-authored-by: Rob Landers <landers.robert@gmail.com> Reviewed-on: #280 Co-authored-by: withinboredom <landers.robert@gmail.com> Co-committed-by: withinboredom <landers.robert@gmail.com>	2022-04-19 12:49:43 +02:00
Alex Auvolat	8f9cf3a5d1	fix a clippy lint	2022-03-28 15:48:55 +02:00
Alex Auvolat	c1d9854d2c	Move block manager to separate module	2022-03-23 10:25:15 +01:00
trinity-1686a	1eb7fdb08f	add test framework for arbitraty S3 requests ... and implement some basic test with it	2022-03-23 10:22:36 +01:00
KokaKiwi	e934934f14	garage_api: Update streaming payload stream unit tests	2022-03-23 10:22:36 +01:00
KokaKiwi	98545a16dd	garage_api: Handle streaming payload early in request handling	2022-03-23 10:22:36 +01:00
Alex Auvolat	ba6b56ae68	Fix some new clippy lints	2022-03-14 12:27:49 +01:00
Alex Auvolat	203e8d2c34	Bump version to 0.7 because of incompatible Netapp	2022-03-14 10:54:24 +01:00
Alex Auvolat	d9a35359bf	Add metrics to web endpoint	2022-03-14 10:53:50 +01:00
Alex Auvolat	2a5609b292	Add metrics to API endpoint	2022-03-14 10:53:36 +01:00
Alex Auvolat	f0d0cd9a20	Remove strum crate dependency; add protobuf nix dependency	2022-03-14 10:53:00 +01:00
Alex Auvolat	bb04d94fa9	Update to Netapp 0.4 which supports distributed tracing	2022-03-14 10:52:30 +01:00
Alex Auvolat	8c2fb0c066	Add tracing integration with opentelemetry	2022-03-14 10:52:13 +01:00
Alex Auvolat	8a5bbc3b0b	More permissive OPTIONS on S3 API	2022-03-01 11:15:16 +01:00
Alex Auvolat	97f245f218	Add tracing output to signature calculation	2022-02-28 12:22:39 +01:00
Alex Auvolat	8129a98291	Process CORS earlier in pipeline	2022-02-28 12:22:39 +01:00
trinity-1686a	f6f8b7f1ad	Support for PostObject (#222) ... Add support for [PostObject](https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPOST.html) - [x] routing PostObject properly - [x] parsing multipart body - [x] validating signature - [x] validating policy - [x] validating content length - [x] actually saving data Co-authored-by: trinity-1686a <trinity@deuxfleurs.fr> Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com> Reviewed-on: #222 Reviewed-by: Alex <alex@adnab.me> Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>	2022-02-21 23:02:30 +01:00
Alex Auvolat	c99f55c420	Add restriction on part ordering in CompleteMultipartUpload	2022-01-25 12:45:00 +01:00
Alex Auvolat	acdf893362	Fix partnumber	2022-01-25 12:25:23 +01:00
Alex Auvolat	338b1b83ee	Implement part_number for GetObject	2022-01-24 21:04:42 +01:00
Alex Auvolat	6dab836f3a	Multipart improvements ... - support part_number for HeadObject - add checks in complete_multipart_upload	2022-01-24 21:04:40 +01:00