535 commits

Author	SHA1	Message	Date
Trinity Pointard	e9c265e9dc	format garage status output better	2021-06-01 19:12:27 +02:00
Trinity Pointard	289521886b	make most changes suggested during install-party	2021-05-29 21:37:49 +02:00
Alex	b9127dd6f8	Prepare for v0.3.0 and add migration path from v0.2.1.x	2021-05-28 15:29:58 +02:00
Alex	ddb2b29bfd	Rename datacenters into zones (doc not yet updated)	2021-05-28 14:07:36 +02:00
Alex	b490ebc7f6	Many improvements on ring/replication and its configuration: ... - Explicit "replication_mode" configuration parameters that takes either "none", "2" or "3" as values, instead of letting user configure replication factor themselves. These are presets whose corresponding replication/quorum values can be found in replication/mode.rs - Explicit support for single-node and two-node deployments (number of nodes must be at least "replication_mode", with "none" we can have only one node) - Ring is now stored much more compactly with 256*8 + n*32 bytes, instead of 256*32 bytes - Support for gateway-only nodes that do not store data (these nodes still need a metadata_directory to store the list of bucket and keys since those are stored on all nodes; it also technically needs a data_directory to start but it will stay empty unless we have bugs)	2021-05-28 14:07:36 +02:00
Quentin	5fdabf3e75	Add basic support for the "Versioning" command	2021-05-14 22:33:26 +02:00
Alex	6ccffc3162	Improved XML serialization ... - Use quick_xml and serde for all XML response returned by the S3 API. - Include tests for all structs used to generate XML - Remove old manual XML escaping function which was unsafe	2021-05-06 22:37:15 +02:00
Trinity Pointard	e4b9e4e24d	rename types to CamelCase	2021-05-03 22:15:09 +02:00
Trinity Pointard	6644df6b96	fix clippy warnings on garage	2021-05-03 22:11:42 +02:00
Trinity Pointard	f8ae8fc4be	fix clippy warnings on web	2021-05-03 22:11:42 +02:00
Trinity Pointard	84856e84e5	fix clippy warnings on api	2021-05-03 22:11:41 +02:00
Trinity Pointard	4a1e079e8f	fix clippy warnings on model	2021-05-03 22:11:41 +02:00
Trinity Pointard	f5a0cf0414	fix clippy warnings on table	2021-05-03 22:11:41 +02:00
Trinity Pointard	f05bb111c2	fix clippy warnings on util and rpc	2021-05-03 22:11:41 +02:00
Quentin	631c36b3ff	S3 API: support ListBuckets	2021-05-03 21:55:30 +02:00
Alex	ee2a3d363b	Remove STREAMING-AWS4-HMAC-SHA256-PAYLOAD (see #64)	2021-05-03 17:30:40 +02:00
Alex	575726358c	Tune Sled configuration ... - Make sled cache size and flush interval configurable - Set less agressive default values: - cache size 128MB instead of 1GB - Flush interval 2 seconds instead of .5 seconds	2021-05-03 17:27:43 +02:00
Alex	dcfc32cf85	Many S3 compatibility improvements: ... - return XML errors - implement AuthorizationHeaderMalformed error to redirect clients to correct location (used by minio client) - implement GetBucketLocation - fix DeleteObjects XML parsing and response	2021-04-28 01:05:40 +02:00
Alex	642186c530	Fix #59 (& issue)	2021-04-27 23:10:43 +02:00
Alex	119217f9f6	change a few comments	2021-04-27 16:53:47 +02:00
Trinity Pointard	2812a027ea	change some more comments and revert changes on TableSchema	2021-04-27 16:49:07 +02:00
Trinity Pointard	74373aebcf	make most requested changes	2021-04-27 16:47:08 +02:00
Alex	16300bbd89	remove useless comment	2021-04-27 16:44:01 +02:00
Trinity Pointard	5c4b2cec3c	document garage crate	2021-04-27 16:37:10 +02:00
Trinity Pointard	1e3df189d0	document api crate	2021-04-27 16:37:10 +02:00
Trinity Pointard	a2e1617d84	document web crate	2021-04-27 16:37:10 +02:00
Trinity Pointard	67585a4ffa	attempt at documenting model crate	2021-04-27 16:37:10 +02:00
Trinity Pointard	b437610812	attempt at documenting table crate	2021-04-27 16:37:10 +02:00
Trinity Pointard	f871689571	run cargo fmt on util and make missing doc warning	2021-04-27 16:37:10 +02:00
Trinity Pointard	8e0524ae15	document rpc crate	2021-04-27 16:37:10 +02:00
Trinity Pointard	f9bd2d8fb7	document util crate	2021-04-27 16:37:10 +02:00
Alex	6b2b400292	small simplify	2021-04-27 16:37:09 +02:00
Alex	8c33d565d6	Merge discovery loop with consul	2021-04-27 16:37:09 +02:00
Alex	948e44a3f6	cargo fmt	2021-04-27 16:37:09 +02:00
Alex	3e2e38c830	Print stats	2021-04-27 16:37:09 +02:00
Alex	2e53e31cdd	Cargo fmt	2021-04-27 16:37:09 +02:00
Alex	64b91c2645	Keep old data	2021-04-27 16:37:09 +02:00
Alex	e16077f40a	Persist directly and not in background	2021-04-27 16:37:09 +02:00
Alex	9ced9f78dc	Improve bootstraping: do it regularly; persist peer list	2021-04-27 16:37:08 +02:00
Alex	f859d15062	update to v0.2.1	2021-03-19 13:39:18 +01:00
Alex	fd8f4caa81	Support old CPUs	2021-03-19 12:19:40 +01:00
Alex	4c26a0b9c1	Update Cargo.toml files with AGPL license info	2021-03-18 21:59:17 +01:00
Alex	a1014224d3	garage node configure --replace <old_node_id> <new_node_id>	2021-03-18 21:49:12 +01:00
Trinity Pointard	f17cb6c969	resolve domain to multiple addresses ... And warn instead of failling when a domain can't be resolved	2021-03-18 21:04:30 +01:00
Trinity Pointard	c8a7ce5cdf	remove domain resolution for *_bind_addr	2021-03-18 19:47:51 +01:00
Trinity Pointard	81e9db783f	simplify addresse deserialialiser and limit allocations	2021-03-18 19:47:51 +01:00
Trinity Pointard	ae3b7029a9	add support for using domain name in configuration	2021-03-18 19:47:51 +01:00
Alex	6edbc65847	Add trinity's comment in the code	2021-03-18 19:46:43 +01:00
Alex	bfa0ff8f82	Merge pull request 'add support for caching headers' (#49) from trinity-1686a/garage:cache-headers into master ... Reviewed-on: #49	2021-03-18 19:45:02 +01:00
Alex	dead945c8f	Prepare for release 0.2	2021-03-18 19:33:15 +01:00
Alex	4eb16e8863	Allow to import keys from previous Garage instance	2021-03-18 19:24:59 +01:00
Trinity Pointard	b4c903371c	add support for caching headers	2021-03-18 15:46:33 +01:00
Alex	390ab02f41	Todo make a test for the Merkle updater	2021-03-16 20:13:07 +01:00
Alex	7b10245dfb	Leader-based GC	2021-03-16 18:42:33 +01:00
Alex	08bcd51956	GC object table in a specific case	2021-03-16 16:51:15 +01:00
Alex	3fadc5cbbd	Small changes	2021-03-16 16:35:10 +01:00
Alex	f4346cc5f4	Update dependencies	2021-03-16 15:58:40 +01:00
Alex	2a41b82384	Simpler Merkle & sync	2021-03-16 12:18:03 +01:00
Alex	0aad2f2e06	some reordering	2021-03-16 11:47:39 +01:00
Alex	515029d026	Refactor code	2021-03-16 11:43:58 +01:00
Alex	1d9961e411	Simplify replication logic	2021-03-16 11:14:27 +01:00
Alex	6a8439fd13	Some improvements in background worker but we terminate late	2021-03-15 23:14:12 +01:00
Alex	0cd5b2ae19	WIP migrate to tokio 1	2021-03-15 22:36:41 +01:00
Alex	4d4117f2b4	Refactor block resync loop; make workers infaillible	2021-03-15 20:09:44 +01:00
Alex	667e4e72a8	Small fixes	2021-03-15 19:51:16 +01:00
Alex	642bed601f	Make it case-insensitive	2021-03-15 19:16:42 +01:00
Alex	5ee1d956b6	Allow manipulation of keys by their shorthand in the CLI	2021-03-15 19:14:26 +01:00
Alex	537f652fec	Tiny things	2021-03-15 18:40:27 +01:00
Alex	0290afe1f8	Make block rc code more understandable	2021-03-15 18:27:26 +01:00
Alex	3bf2df622a	Time and metadata improvements	2021-03-15 16:21:41 +01:00
Alex	097c339d98	Fix race condition	2021-03-15 15:26:29 +01:00
Alex	bdcbdd1cd8	Fix list API bug	2021-03-15 14:46:37 +01:00
Alex	9b118160a8	Optim & refactor	2021-03-12 22:06:56 +01:00
Alex	831eb35763	cargo fmt	2021-03-12 21:52:19 +01:00
Alex	c475471e7a	Implement table gc, currently for block_ref and version only	2021-03-12 19:57:37 +01:00
Alex	f4aad8fe6e	cargo fmt	2021-03-12 18:16:03 +01:00
Alex	5ab33fddac	Refactor CLI and prettify CLI outpu	2021-03-12 18:12:31 +01:00
Alex	a1442f072a	Implement garage stats to get info on node contents	2021-03-12 15:40:54 +01:00
Alex	cbe7e1a66a	Move table rpc client out of tableaux	2021-03-12 15:07:23 +01:00
Alex	8860aa19b8	Make syncer have its own rpc client/server	2021-03-12 15:05:26 +01:00
Alex	1fea257291	Don't sync at beginning	2021-03-12 14:51:17 +01:00
Alex	7fdaf7aef0	Fix merkle updater not being notified; improved logging	2021-03-12 14:37:46 +01:00
Alex	1ec49980ec	whoops	2021-03-11 19:30:24 +01:00
Alex	3f7a496355	More security: don't delete stuff too easily	2021-03-11 19:06:27 +01:00
Alex	f7c2cd1cd7	Add comment, and also whoops, this wasn't doing what we expected	2021-03-11 18:56:18 +01:00
Alex	fae5104a2c	Add a nice warning	2021-03-11 18:50:32 +01:00
Alex	db7a9d4948	Tiny changes	2021-03-11 18:45:26 +01:00
Alex	046b649bcc	(not well tested) use merkle tree for sync	2021-03-11 18:28:27 +01:00
Alex	94f3d28774	WIP big refactoring	2021-03-11 16:54:15 +01:00
Alex	8d63738cb0	Checkpoint: add merkle tree in data table	2021-03-11 13:47:21 +01:00
Alex	3214dd52dd	Very minor changes	2021-03-10 21:50:09 +01:00
Alex	af7600f989	Correctly implement CompleteMultipartUpload with etag check of parts	2021-03-10 17:01:05 +01:00
Alex	445912dc6a	Remove migration paths from 0.1 branch	2021-03-10 16:38:31 +01:00
Alex	0fd7df8fa0	Switch to blake2 sum for identifying blocks by their data	2021-03-10 16:33:31 +01:00
Alex	2afd2c81ba	Change hash function to blake2 for partition keys based on strings	2021-03-10 16:23:57 +01:00
Alex	f319a7d374	Refactor model stuff, including cleaner CRDTs	2021-03-10 16:21:56 +01:00
Alex	6a3dcf3974	Rename n_tokens into capacity	2021-03-10 14:52:03 +01:00
Alex	7cda917b6b	update condition	2021-03-05 17:08:03 +01:00
Alex	d7e005251d	Not fully tested: new multi-dc MagLev	2021-03-05 16:22:29 +01:00
Alex	3882d5ba36	Remove epidemic propagation for fully replicated stuff: write directly to all nodes	2021-03-05 15:09:18 +01:00
Alex	49c25a1509	Simulate stuff moving around	2021-02-25 10:53:33 +01:00
Alex	5fe95ebae7	fix tracing	2021-02-24 12:18:01 +01:00
Alex	13e2eda0c2	Arrange block manager	2021-02-24 11:58:03 +01:00
Alex	09fd6ea7f0	I was tired yesterday	2021-02-24 11:05:59 +01:00
Alex	a52ab69640	fix misuse of sled transactions	2021-02-23 22:45:36 +01:00
Alex	20e6e9fa20	Update sled & try to debug deadlock (but its in sled...)	2021-02-23 21:27:28 +01:00
Alex	bf25c95fe2	Make updated() be a sync function that doesn't fail	2021-02-23 20:25:15 +01:00
Alex	28bc967c83	Handle correctly deletion dues to offloading	2021-02-23 19:59:43 +01:00
Alex	55156cca9d	Several changes in table_sync: ... - separate path for case of offloading a partition we don't store - use sync::Mutex instead of tokio::Mutex, make less fn's async	2021-02-23 19:11:02 +01:00
Alex	40763fd749	Cargo fmt	2021-02-23 18:46:25 +01:00
Alex	6e6f7e8555	Replace some checksums where it makes sense	2021-02-23 18:14:37 +01:00
Alex	e8e4418ca7	Add blake2 and xxhash hash functions	2021-02-23 17:52:28 +01:00
Alex	b1b640ae8b	rename hash() to sha256sum(), we might want to change it at some places	2021-02-21 15:24:30 +01:00
Alex	80892df8cc	Some refactoring	2021-02-21 13:11:10 +01:00
Alex	3bcbbe1e31	More precise logging (warn only when returning a 500)	2021-02-20 00:30:39 +01:00
Alex	10b983b8e7	Add verification of part numbers in CompleteMultipartUpload (WIP #30)	2021-02-20 00:13:07 +01:00
Alex	1de96248e0	add application/xml header and missing xml escapes	2021-02-19 23:40:18 +01:00
Alex	5d1fa591d9	Add compatibility list	2021-02-19 19:10:23 +01:00
Alex	e64ecbdccd	S3 compatibility: return 404 instead of 400 on some multipart commands	2021-02-19 18:51:05 +01:00
Alex	55a2a636ca	Implement ListObjectsV2	2021-02-19 16:44:06 +01:00
Alex	02d512f3fd	Fix #28, extra headers being ignored (because of profound stupidity)	2021-02-19 12:38:22 +01:00
Alex	76390085ef	Small improvements in the S3 put workflow	2021-02-19 12:11:02 +01:00
Alex	36814be447	Fix S3 ListObjects result and replace println!s by debug!s	2021-01-16 16:05:54 +01:00
Alex	e818f51073	Forgot a bump	2021-01-15 18:36:51 +01:00
Alex	ceeb0732a2	Use 0.1.0b instead of 0.1.0 (for compatibility with new Error type)	2021-01-15 18:27:58 +01:00
Alex	1d1d497e2b	Bump everything to 0.1.1	2021-01-15 17:54:48 +01:00
Alex	851893a3f2	Do not accept domains such as [hello	2021-01-15 17:49:10 +01:00
Quentin	f8a40e8c4f	Explicitly set code path unreachable	2021-01-15 17:11:15 +01:00
Quentin	fad7bc405b	Behavior problem: do not panic anymore + add tests	2021-01-15 17:03:54 +01:00
Quentin	1e10c6a61c	Doc tests that do not compile/work must be tagged with ignore	2021-01-15 17:03:38 +01:00
Quentin	11a79a95dd	Simplify Error file	2021-01-15 16:25:44 +01:00
Quentin	c441a358cd	Remove unused dependencies	2021-01-15 16:16:32 +01:00
Quentin	f496e41ef4	Replace an already done check by unreachable!()	2021-01-15 15:44:44 +01:00
Quentin	2f4378a9c4	Fix formatting	2020-12-17 22:51:44 +01:00
Quentin	ccda9ab1ca	Merge branch 'master' into feature/website	2020-12-17 21:09:50 +01:00
Quentin	3132deca58	Web server access control	2020-12-17 20:43:14 +01:00
Quentin	011ff87b5f	Push update	2020-12-15 13:23:22 +01:00
Quentin	3bc4d57a0f	First implementation of the CLI	2020-12-15 12:48:24 +01:00
Quentin	a3566e49da	Start to implement Website CLI	2020-12-14 21:50:40 +01:00
Quentin	d0eb6a457f	Migrate RPC to new schema	2020-12-14 21:46:49 +01:00
Quentin	96388acf23	Implement migration	2020-12-12 21:35:29 +01:00
Alex	8956db2a81	Make less things public	2020-12-12 17:58:19 +01:00
Alex	5c6c067b0c	More documentation on CRDTs (we should probably extract this to a ... standalone crate!)	2020-12-12 17:06:40 +01:00
Quentin	e1ce2b228a	WIP table migration	2020-12-12 17:00:31 +01:00
Alex	0b3084ca5f	Merge branch 'master' into doc/model	2020-12-12 16:05:28 +01:00
Quentin	e8c12072ce	Merge branch 'master' into feature/website	2020-12-10 20:12:56 +01:00
Quentin	51d0c14e44	CLI structure	2020-12-10 18:13:32 +01:00
Alex	022b386a50	Improved compatibility on list API call	2020-12-06 15:39:03 +01:00
Quentin	986e15459a	Merge branch 'master' into feature/website	2020-12-06 15:21:09 +01:00
Alex	4a5bbbb810	Propose ETag fix	2020-12-05 19:23:46 +01:00
Alex	76b489f3d3	Reformulate patch	2020-12-05 19:20:07 +01:00
Quentin	bd7e3d1bd1	Fix Content-Length	2020-12-05 18:57:22 +01:00
Alex	9f46fb699a	Content-range fix	2020-12-05 16:37:59 +01:00
Quentin	15f409d404	Merge branch 'master' into feature/website	2020-11-29 17:19:55 +01:00
Alex	d54f15b2c6	Small optimisation	2020-11-29 17:07:14 +01:00
Alex	fed97f37e1	ETag patch	2020-11-29 16:38:01 +01:00
Alex	601ae25ad2	Small refactorings	2020-11-29 16:21:28 +01:00
Quentin	cbd10c1b0a	Add some doc on LWW	2020-11-23 18:17:48 +01:00
Quentin	8722e27600	CRDT doc	2020-11-23 17:49:21 +01:00
Quentin	aa320aa04a	Merge branch 'master' into feature/website	2020-11-22 19:54:47 +01:00
Quentin	17dc610f8a	Also check hash for < 3KB files	2020-11-22 11:14:46 +01:00
Quentin	435d5f9205	Fix base64/hex checksum comparison	2020-11-22 11:04:33 +01:00
Quentin	fb18f5e17a	Fix wrong http status code	2020-11-21 18:14:02 +01:00
Quentin	28efe341cb	Merge branch 'master' into feature/website	2020-11-21 18:01:50 +01:00
Quentin	b7a377308b	Handle HEAD	2020-11-21 17:58:14 +01:00
Quentin	a88fd49f71	Use handle_get	2020-11-21 17:50:19 +01:00
Quentin	0f33231ee6	We are able to serve a file	2020-11-21 15:15:25 +01:00
Quentin	d4c7f4e374	Fix host to key	2020-11-21 12:01:02 +01:00
Quentin	2f6eca4ef3	Merge remote-tracking branch 'origin/master' into feature/website	2020-11-21 10:52:27 +01:00
Alex	a8b3c8fd58	data hexdump in warning	2020-11-20 23:53:54 +01:00
Alex	f9be964c3f	Warning when cannot decode entry (data format incompatibilities)	2020-11-20 23:53:54 +01:00
Alex	4f7f1d1cb3	less type bounds	2020-11-20 23:53:54 +01:00
Alex	68be5072e5	simplify	2020-11-20 23:53:54 +01:00
Alex	173f0dbac9	oops	2020-11-20 23:53:54 +01:00
Alex	f8a04852a2	Convert bucket table to better CRDT representation	2020-11-20 23:53:54 +01:00
Alex	e02e9e035e	Begin improve model to use better CRDTs	2020-11-20 23:53:54 +01:00
Quentin	5b363626f4	Support punnycode	2020-11-20 21:23:32 +01:00
Alex	2dc9a48022	Fix reconciliation logic	2020-11-20 20:12:32 +01:00
Alex	e9fd265ce6	Slight refactoring to make things clearer with DeletedFilter	2020-11-20 20:11:04 +01:00
Quentin	04f455ff7f	Make it compile again	2020-11-19 14:56:00 +01:00
Quentin	fc427b0b66	Merge branch 'master' into feature/website	2020-11-19 14:39:30 +01:00
Quentin	6076d869b1	Build error	2020-11-11 21:17:34 +01:00
Quentin	2765291796	Build path correctly	2020-11-11 19:48:01 +01:00
Alex	7d7b9e95a9	Simplify and_then(Some) as map() and remove move	2020-11-11 16:36:48 +01:00
Quentin	5a5592c176	Replace with option syntaxic sugar	2020-11-11 16:12:42 +01:00
Quentin	d445c4ef9c	WIP fetch object	2020-11-11 15:24:25 +01:00
Quentin	3cb3994cd2	Add documentation to host_to_bucket	2020-11-10 17:05:10 +01:00
Quentin	cacf8ddf2d	Panic when it is a logical error	2020-11-10 15:52:20 +01:00
Quentin	d1b2fcc1e7	Rewrite for clarity	2020-11-10 15:48:40 +01:00
Quentin	ab62c59acb	Fix indent again	2020-11-10 15:40:33 +01:00
Quentin	8797eed0ab	Fixes due to integration tests	2020-11-10 15:32:04 +01:00
Quentin	1e52ee9f5b	Rewrite authority to host while staying on stack	2020-11-10 15:26:48 +01:00
Quentin	27795a390c	Fix formatting	2020-11-10 09:59:52 +01:00
Quentin	4093833ae8	Extract bucket	2020-11-10 09:57:07 +01:00
Quentin	09137fd6b5	Log host	2020-11-08 16:06:52 +01:00
Quentin	c78df603d7	Add some documentation	2020-11-08 16:02:16 +01:00
Quentin	71721f5bcf	Merge branch 'master' into feature/website	2020-11-08 15:53:33 +01:00
Quentin	0791e7164e	Parse host header	2020-11-08 15:47:25 +01:00
Alex	a50fa70d45	Refactor error management in API part	2020-11-08 15:05:28 +01:00
Alex	6a8b972f3a	Modif parse_bucket_key to fail when bucket is ""	2020-11-08 13:39:44 +01:00
Quentin	e1415f5976	Add a second test	2020-11-07 15:34:53 +01:00
Quentin	9a50ce12a8	Fix formatting	2020-11-07 13:59:30 +01:00
Quentin	8f4ada1965	Add a test for parse_bucket_key	2020-11-07 13:53:32 +01:00
Quentin	0d3bc169ee	It compiles!	2020-11-03 12:37:16 +01:00
Quentin	b3caa3628d	Fix description of the crate	2020-11-02 15:57:23 +01:00
Quentin	cea871d944	Skeleton to the new web API	2020-11-02 15:48:39 +01:00
Quentin	104e2ce0a2	Add "web" configuration entry	2020-10-31 17:28:56 +01:00
Alex	5faf069e33	trace test	2020-07-15 15:41:49 +02:00
Alex	27a0d0d859	pretty	2020-07-15 15:37:35 +02:00
Alex	1c70552f95	Validate content MD5 and SHA256 sums for PutObject and UploadPart	2020-07-15 15:31:13 +02:00
Alex	6c7f9704ea	Implement correct ETag for objects created with PutObject	2020-07-13 16:51:30 +02:00
Alex	9305e5e87f	More headers taken into account	2020-07-09 17:04:43 +02:00
Alex	44dba0e53c	cargo fmt	2020-07-08 17:34:37 +02:00
Alex	64a6eda0d2	Migrate S3 api to use new model	2020-07-08 17:33:24 +02:00
Alex	84bbbfaa7b	Add multiple headers to object model	2020-07-08 17:33:14 +02:00
Alex	a5fa2a136b	(WIP) New object table model, TODO: update API calls to use it	2020-07-08 16:46:47 +02:00
Alex	86bf4dedac	Add support for model migrations	2020-07-08 16:10:53 +02:00
Alex	86fb7bbba5	Apply cargo fmt; add trace output when request signature is bad	2020-07-08 13:33:02 +02:00
Alex	f22ecb60a8	Update to Hyper 0.13.6 that accepts non-Sync streams in wrap_stream. ... Simplifies code and makes it possible to publish on crates.io	2020-07-07 17:15:53 +02:00
Alex	3b0b11085e	Add versions to dependencies	2020-07-07 14:18:47 +02:00
Alex	cc65cdc0fe	Add license, description and repository to .toml files	2020-07-07 14:14:58 +02:00
Alex	bec26a1312	Rename garage_core to garage_model	2020-07-07 13:59:22 +02:00
Alex	fbe8fe81f2	Add automatic peer discovery from Consul	2020-06-30 18:33:14 +02:00
Alex	db6f1f35a8	Rename epidemic_factor to epidemic_fanout (that's what it is); complete conf example in readme	2020-06-30 15:03:34 +02:00
Alex	fa13cf6996	Repair: do not mark deleted when upstream object is not found ... With the previous behaviour, repairing could see some data as absent and decide that the object or version was deleted, thus going on to delete the version and blocks. In the case where read_quorum + write_quorum <= replication_factor however, entries may not yet be returned by the get, thus data would have been deleted that should hot have been. The new behavior is more cautious and just skips the entry when the warning is emitted.	2020-05-04 13:30:42 +00:00
Alex	b46a7788d1	Implement HTTP ranges in get	2020-05-04 13:09:23 +00:00
Alex	16fbb32fd3	Rate limit requests a bit more seriously ... droping the slot later (after reading the request response) means that we aren't freeing our quota slot, so the maximum number of simultaneous requests now also counts the response reading phase TODO next: quotas per rpc destination node, or maybe per datacenter (?)	2020-05-01 19:18:54 +00:00
Alex	d867bbcfb5	Implement DeleteObjects	2020-05-01 15:52:35 +00:00
Alex	3324971701	Slightly improved S3 compatibility ... - ListBucket does not require any of the parameters (delimiter, prefix, max-keys, etc) - URLs are properly percent_decoded - PutObject and DeleteObject calls now answer correctly (empty body, version id in the x-amz-version-id: header)	2020-05-01 14:30:50 +00:00
Alex	3686f100b7	Compatibility fixes	2020-04-28 10:35:04 +00:00
Alex	0957d0fdfa	Work on API	2020-04-28 10:18:14 +00:00
Alex	0877a5500c	Abort multipart upload	2020-04-26 20:46:33 +00:00
Alex	81ecc4999e	Implement multipart uploads	2020-04-26 20:39:32 +00:00
Alex	1999c0ae51	Update delete code	2020-04-26 19:11:19 +00:00
Alex	0a283e4e70	Fix deletion propagation	2020-04-26 18:59:17 +00:00
Alex	9cb870f950	Prepare for multipart uploads	2020-04-26 18:55:13 +00:00
Alex	ea7e4748ed	S3 compatibility: fix bucket listing and HEAD and PUT on bucket	2020-04-26 16:22:33 +00:00
Alex	0e49e0c8b5	Add key table to repair procedure	2020-04-26 16:22:22 +00:00
Alex	e3203f998b	Remove leading / in keys; better delimiter handling	2020-04-24 22:28:15 +02:00
Alex	be4831d768	Less verbosity	2020-04-24 19:27:27 +00:00
Alex	a52db67954	xml escape	2020-04-24 18:56:00 +00:00
Alex	91b2d1fcc1	Some basic S3 functionnality	2020-04-24 18:47:11 +00:00
Alex	f2e05986c4	Starting to be S3 compatible	2020-04-24 17:46:52 +00:00
Alex	d8f5e643bc	Split code for modular compilation	2020-04-24 10:10:01 +00:00
Alex	51fb3799a1	Key management admin commands	2020-04-23 20:25:45 +00:00
Alex	4ef84a0558	Move repair to separate file	2020-04-23 18:36:12 +00:00
Alex	44a1089d95	Make table objects slightly more fool-proof; add key table	2020-04-23 18:16:52 +00:00
Alex	c9c6b0dbd4	Reorganize code	2020-04-23 17:05:46 +00:00
Alex	01a8acdeec	Better error reporting	2020-04-23 16:23:06 +00:00
Alex	82f4cd8719	Continue pinging nodes when they are down ; overall better handling of down nodes	2020-04-23 16:06:11 +00:00