171 commits

Author	SHA1	Message	Date
Alex	135858d067	Implement DeleteBucket	2022-01-05 16:28:19 +01:00
Alex	8395030e48	Implement CreateBucket	2022-01-05 15:56:48 +01:00
Alex	677ab60cc1	Small changes in key model and refactoring	2022-01-04 18:59:17 +01:00
Alex	df35feba18	New buckets for 0.6.0: make bucket id a SK and not a HK, CLI updates	2022-01-04 12:53:14 +01:00
Alex	1bcd6fabbd	New buckets for 0.6.0: small changes ... - Fix bucket delete - fix merge of bucket creation date - Replace deletable with option in aliases Rationale: if two aliases point to conflicting bucket, resolving by making an arbitrary choice risks making data accessible when it shouldn't be. We'd rather resolve to deleting the alias until someone puts it back.	2022-01-04 12:52:47 +01:00
Alex	2140cd7205	Remove website redirects	2022-01-04 12:52:46 +01:00
Alex	beeef4758e	Some movement of helper code and refactoring of error handling	2022-01-04 12:52:46 +01:00
Alex	d8ab5bdc3e	New buckets for 0.6.0: fix model and migration	2022-01-04 12:47:28 +01:00
Alex	b76d0580a0	Fix forgotten flag	2022-01-04 12:47:28 +01:00
Alex	b1cfd16913	New buckets for 0.6.0: small fixes, including: ... - ensure bucket names are correct aws s3 names - when making aliases, ensure timestamps of links in both ways are the same - fix small remarks by trinity - don't have a separate website_access field	2022-01-04 12:46:41 +01:00
Alex	0bbb6673e7	Model changes	2022-01-04 12:45:52 +01:00
Alex	5b1117e582	New model for buckets	2022-01-04 12:45:46 +01:00
trinity-1686a	ca7b438f3f	less strict ListBuckets (#178) ... fix #175 Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com> Reviewed-on: #178 Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>	2021-12-15 15:05:36 +01:00
trinity-1686a	60d4459926	BucketWebsite (#174) ... fix #77 this does not store anything but a on/off switch for website, and does not implement GetBucketWebsite as it would require storing more. GetBucketWebsite should be pretty easy to implement once data is stored though. Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com> Reviewed-on: #174 Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>	2021-12-15 10:41:39 +01:00
trinity-1686a	c4ac8835d3	add proper request router for s3 api (#163) ... fix #161 Current request router was organically grown, and is getting messier and messier with each addition. This router cover exaustively existing API endpoints (with exceptions listed in [#161(comment)](#161 (comment)) either because new and old api endpoint can't feasabily be differentied, or it's more lambda than s3). Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com> Reviewed-on: #163 Reviewed-by: Alex <alex@adnab.me> Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>	2021-12-06 15:17:47 +01:00
trinity-1686a	7f26ed55cd	Improved handling of HTTP ranges ... - correct HTTP code when range syntax is invalid (fix #140) - when multiple ranges are given, simply ignore and send whole file Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com> Reviewed-on: #157 Reviewed-by: Alex <alex@adnab.me> Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>	2021-11-29 11:52:42 +01:00
Quentin	8811bb08e6	In ListBuckets, hide entry if no perms	2021-11-22 12:10:28 +01:00
Alex	c94406f428	Improve how node roles are assigned in Garage ... - change the terminology: the network configuration becomes the role table, the configuration of a nodes becomes a node's role - the modification of the role table takes place in two steps: first, changes are staged in a CRDT data structure. Then, once the user is happy with the changes, they can commit them all at once (or revert them). - update documentation - fix tests - implement smarter partition assignation algorithm This patch breaks the format of the network configuration: when migrating, the cluster will be in a state where no roles are assigned. All roles must be re-assigned and commited at once. This migration should not pose an issue.	2021-11-16 16:05:53 +01:00
Trinity Pointard	53888995bd	update doc and comments	2021-11-16 15:41:41 +01:00
Trinity Pointard	396fe4c702	clippy	2021-11-16 15:41:41 +01:00
Trinity Pointard	02158ee666	fix issue where list on vhost-bucket would list bucket instead of bucket content	2021-11-16 15:41:41 +01:00
Trinity Pointard	9c58ec28d3	add support for vhost-style s3 bucket	2021-11-16 15:41:41 +01:00
Alex	ada7899b24	Fix clippy lints (fix #121)	2021-10-26 10:20:05 +02:00
Alex	de4276202a	Improve CLI, adapt tests, update documentation	2021-10-25 14:21:48 +02:00
Alex	1b450c4b49	Improvements to CLI and various fixes for netapp version ... Discovery via consul, persist peer list to file	2021-10-22 16:55:24 +02:00
Alex	4067797d01	First port of Garage to Netapp	2021-10-22 15:55:18 +02:00
Alex	f6060b92aa	Fix HTTP return code for DeleteObject (fix #98)	2021-10-11 14:24:49 +02:00
Alex	f3a097abdf	WIP: try to fix #93, and improve S3 ListObjects (v1 and v2) API calls	2021-10-11 11:15:47 +02:00
Alex	b9127dd6f8	Prepare for v0.3.0 and add migration path from v0.2.1.x	2021-05-28 15:29:58 +02:00
Quentin	5fdabf3e75	Add basic support for the "Versioning" command	2021-05-14 22:33:26 +02:00
Alex	6ccffc3162	Improved XML serialization ... - Use quick_xml and serde for all XML response returned by the S3 API. - Include tests for all structs used to generate XML - Remove old manual XML escaping function which was unsafe	2021-05-06 22:37:15 +02:00
Trinity Pointard	e4b9e4e24d	rename types to CamelCase	2021-05-03 22:15:09 +02:00
Trinity Pointard	84856e84e5	fix clippy warnings on api	2021-05-03 22:11:41 +02:00
Quentin	631c36b3ff	S3 API: support ListBuckets	2021-05-03 21:55:30 +02:00
Alex	ee2a3d363b	Remove STREAMING-AWS4-HMAC-SHA256-PAYLOAD (see #64)	2021-05-03 17:30:40 +02:00
Alex	dcfc32cf85	Many S3 compatibility improvements: ... - return XML errors - implement AuthorizationHeaderMalformed error to redirect clients to correct location (used by minio client) - implement GetBucketLocation - fix DeleteObjects XML parsing and response	2021-04-28 01:05:40 +02:00
Alex	642186c530	Fix #59 (& issue)	2021-04-27 23:10:43 +02:00
Trinity Pointard	74373aebcf	make most requested changes	2021-04-27 16:47:08 +02:00
Trinity Pointard	1e3df189d0	document api crate	2021-04-27 16:37:10 +02:00
Alex	f859d15062	update to v0.2.1	2021-03-19 13:39:18 +01:00
Alex	4c26a0b9c1	Update Cargo.toml files with AGPL license info	2021-03-18 21:59:17 +01:00
Alex	6edbc65847	Add trinity's comment in the code	2021-03-18 19:46:43 +01:00
Alex	bfa0ff8f82	Merge pull request 'add support for caching headers' (#49) from trinity-1686a/garage:cache-headers into master ... Reviewed-on: #49	2021-03-18 19:45:02 +01:00
Alex	dead945c8f	Prepare for release 0.2	2021-03-18 19:33:15 +01:00
Trinity Pointard	b4c903371c	add support for caching headers	2021-03-18 15:46:33 +01:00
Alex	f4346cc5f4	Update dependencies	2021-03-16 15:58:40 +01:00
Alex	0cd5b2ae19	WIP migrate to tokio 1	2021-03-15 22:36:41 +01:00
Alex	667e4e72a8	Small fixes	2021-03-15 19:51:16 +01:00
Alex	3bf2df622a	Time and metadata improvements	2021-03-15 16:21:41 +01:00
Alex	097c339d98	Fix race condition	2021-03-15 15:26:29 +01:00
Alex	bdcbdd1cd8	Fix list API bug	2021-03-15 14:46:37 +01:00
Alex	831eb35763	cargo fmt	2021-03-12 21:52:19 +01:00
Alex	c475471e7a	Implement table gc, currently for block_ref and version only	2021-03-12 19:57:37 +01:00
Alex	7fdaf7aef0	Fix merkle updater not being notified; improved logging	2021-03-12 14:37:46 +01:00
Alex	3214dd52dd	Very minor changes	2021-03-10 21:50:09 +01:00
Alex	af7600f989	Correctly implement CompleteMultipartUpload with etag check of parts	2021-03-10 17:01:05 +01:00
Alex	0fd7df8fa0	Switch to blake2 sum for identifying blocks by their data	2021-03-10 16:33:31 +01:00
Alex	f319a7d374	Refactor model stuff, including cleaner CRDTs	2021-03-10 16:21:56 +01:00
Alex	40763fd749	Cargo fmt	2021-02-23 18:46:25 +01:00
Alex	b1b640ae8b	rename hash() to sha256sum(), we might want to change it at some places	2021-02-21 15:24:30 +01:00
Alex	3bcbbe1e31	More precise logging (warn only when returning a 500)	2021-02-20 00:30:39 +01:00
Alex	10b983b8e7	Add verification of part numbers in CompleteMultipartUpload (WIP #30)	2021-02-20 00:13:07 +01:00
Alex	1de96248e0	add application/xml header and missing xml escapes	2021-02-19 23:40:18 +01:00
Alex	5d1fa591d9	Add compatibility list	2021-02-19 19:10:23 +01:00
Alex	e64ecbdccd	S3 compatibility: return 404 instead of 400 on some multipart commands	2021-02-19 18:51:05 +01:00
Alex	55a2a636ca	Implement ListObjectsV2	2021-02-19 16:44:06 +01:00
Alex	02d512f3fd	Fix #28, extra headers being ignored (because of profound stupidity)	2021-02-19 12:38:22 +01:00
Alex	76390085ef	Small improvements in the S3 put workflow	2021-02-19 12:11:02 +01:00
Alex	36814be447	Fix S3 ListObjects result and replace println!s by debug!s	2021-01-16 16:05:54 +01:00
Alex	e818f51073	Forgot a bump	2021-01-15 18:36:51 +01:00
Alex	022b386a50	Improved compatibility on list API call	2020-12-06 15:39:03 +01:00
Alex	4a5bbbb810	Propose ETag fix	2020-12-05 19:23:46 +01:00
Alex	76b489f3d3	Reformulate patch	2020-12-05 19:20:07 +01:00
Quentin	bd7e3d1bd1	Fix Content-Length	2020-12-05 18:57:22 +01:00
Alex	9f46fb699a	Content-range fix	2020-12-05 16:37:59 +01:00
Alex	d54f15b2c6	Small optimisation	2020-11-29 17:07:14 +01:00
Alex	fed97f37e1	ETag patch	2020-11-29 16:38:01 +01:00
Alex	601ae25ad2	Small refactorings	2020-11-29 16:21:28 +01:00
Quentin	17dc610f8a	Also check hash for < 3KB files	2020-11-22 11:14:46 +01:00
Quentin	435d5f9205	Fix base64/hex checksum comparison	2020-11-22 11:04:33 +01:00
Alex	f8a04852a2	Convert bucket table to better CRDT representation	2020-11-20 23:53:54 +01:00
Alex	e02e9e035e	Begin improve model to use better CRDTs	2020-11-20 23:53:54 +01:00
Alex	e9fd265ce6	Slight refactoring to make things clearer with DeletedFilter	2020-11-20 20:11:04 +01:00
Alex	7d7b9e95a9	Simplify and_then(Some) as map() and remove move	2020-11-11 16:36:48 +01:00
Quentin	5a5592c176	Replace with option syntaxic sugar	2020-11-11 16:12:42 +01:00
Alex	a50fa70d45	Refactor error management in API part	2020-11-08 15:05:28 +01:00
Alex	6a8b972f3a	Modif parse_bucket_key to fail when bucket is ""	2020-11-08 13:39:44 +01:00
Quentin	e1415f5976	Add a second test	2020-11-07 15:34:53 +01:00
Quentin	9a50ce12a8	Fix formatting	2020-11-07 13:59:30 +01:00
Quentin	8f4ada1965	Add a test for parse_bucket_key	2020-11-07 13:53:32 +01:00
Alex	5faf069e33	trace test	2020-07-15 15:41:49 +02:00
Alex	27a0d0d859	pretty	2020-07-15 15:37:35 +02:00
Alex	1c70552f95	Validate content MD5 and SHA256 sums for PutObject and UploadPart	2020-07-15 15:31:13 +02:00
Alex	6c7f9704ea	Implement correct ETag for objects created with PutObject	2020-07-13 16:51:30 +02:00
Alex	9305e5e87f	More headers taken into account	2020-07-09 17:04:43 +02:00
Alex	44dba0e53c	cargo fmt	2020-07-08 17:34:37 +02:00
Alex	64a6eda0d2	Migrate S3 api to use new model	2020-07-08 17:33:24 +02:00
Alex	86bf4dedac	Add support for model migrations	2020-07-08 16:10:53 +02:00
Alex	86fb7bbba5	Apply cargo fmt; add trace output when request signature is bad	2020-07-08 13:33:02 +02:00
Alex	f22ecb60a8	Update to Hyper 0.13.6 that accepts non-Sync streams in wrap_stream. ... Simplifies code and makes it possible to publish on crates.io	2020-07-07 17:15:53 +02:00