garage

Author	SHA1	Message	Date
Quentin Dufour	43ddc933f9	Update Ceph S3 endpoints compatibility	2022-05-25 15:20:08 +02:00
Alex Auvolat	9f303f6308	Shorter page title	2022-05-24 15:47:42 +02:00
Alex Auvolat	3be43f3372	Add lost content for Restic with Garage Suggested-by: Quentin <quentin@deuxfleurs.fr>	2022-05-24 15:32:42 +02:00
Alex Auvolat	2da448b43f	Add documentation for new Admin API and a few infos on K2V	2022-05-24 15:28:37 +02:00
Alex	b2a2d3859f	K2V client improvements (#307 ) - [x] Better distinguish error types - [x] Parse error messages received from server - [x] Remove `src/` folder layer, we don't have that for other crates Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: Deuxfleurs/garage#307 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>	2022-05-24 12:48:05 +02:00
Alex	382e74c798	First version of admin API (#298 ) Spec: - [x] Start writing - [x] Specify all layout endpoints - [x] Specify all endpoints for operations on keys - [x] Specify all endpoints for operations on key/bucket permissions - [x] Specify all endpoints for operations on buckets - [x] Specify all endpoints for operations on bucket aliases View rendered spec at <https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/admin-api/doc/drafts/admin-api.md> Code: - [x] Refactor code for admin api to use common api code that was created for K2V General endpoints: - [x] Metrics - [x] GetClusterStatus - [x] ConnectClusterNodes - [x] GetClusterLayout - [x] UpdateClusterLayout - [x] ApplyClusterLayout - [x] RevertClusterLayout Key-related endpoints: - [x] ListKeys - [x] CreateKey - [x] ImportKey - [x] GetKeyInfo - [x] UpdateKey - [x] DeleteKey Bucket-related endpoints: - [x] ListBuckets - [x] CreateBucket - [x] GetBucketInfo - [x] DeleteBucket - [x] PutBucketWebsite - [x] DeleteBucketWebsite Operations on key/bucket permissions: - [x] BucketAllowKey - [x] BucketDenyKey Operations on bucket aliases: - [x] GlobalAliasBucket - [x] GlobalUnaliasBucket - [x] LocalAliasBucket - [x] LocalUnaliasBucket And also: - [x] Separate error type for the admin API (this PR includes a quite big refactoring of error handling) - [x] Add management of website access - [ ] Check that nothing is missing wrt what can be done using the CLI - [ ] Improve formatting of the spec - [x] Make sure everyone is cool with the API design Fix #231 Fix #295 Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: Deuxfleurs/garage#298 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>	2022-05-24 12:16:39 +02:00
trinity-1686a	64c193e3db	Add a K2V client library and CLI (#303 ) lib.rs could use getting split in modules, but I'm not sure how exactly Co-authored-by: trinity-1686a <trinity@deuxfleurs.fr> Reviewed-on: Deuxfleurs/garage#303 Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>	2022-05-18 22:24:09 +02:00
Alex Auvolat	c692f55d5c	K2V: Fix `end` parameter and add tests (fix #305 )	2022-05-17 11:50:23 +02:00
Alex Auvolat	7b474855e3	Make background runner terminate correctly	2022-05-17 11:38:31 +02:00
Alex Auvolat	176715c5b2	Fix ReadIndex spec and add JSON5 remark to doc	2022-05-16 11:54:37 +02:00
Alex	5768bf3622	First implementation of K2V (#293 ) Specification: View spec at [this URL](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/k2v/doc/drafts/k2v-spec.md) - [x] Specify the structure of K2V triples - [x] Specify the DVVS format used for causality detection - [x] Specify the K2V index (just a counter of number of values per partition key) - [x] Specify single-item endpoints: ReadItem, InsertItem, DeleteItem - [x] Specify index endpoint: ReadIndex - [x] Specify multi-item endpoints: InsertBatch, ReadBatch, DeleteBatch - [x] Move to JSON objects instead of tuples - [x] Specify endpoints for polling for updates on single values (PollItem) Implementation: - [x] Table for K2V items, causal contexts - [x] Indexing mechanism and table for K2V index - [x] Make API handlers a bit more generic - [x] K2V API endpoint - [x] K2V API router - [x] ReadItem - [x] InsertItem - [x] DeleteItem - [x] PollItem - [x] ReadIndex - [x] InsertBatch - [x] ReadBatch - [x] DeleteBatch Testing: - [x] Just a simple Python script that does some requests to check visually that things are going right (does not contain parsing of results or assertions on returned values) - [x] Actual tests: - [x] Adapt testing framework - [x] Simple test with InsertItem + ReadItem - [x] Test with several Insert/Read/DeleteItem + ReadIndex - [x] Test all combinations of return formats for ReadItem - [x] Test with ReadBatch, InsertBatch, DeleteBatch - [x] Test with PollItem - [x] Test error codes - [ ] Fix most broken stuff - [x] test PollItem broken randomly - [x] when invalid causality tokens are given, errors should be 4xx not 5xx Improvements: - [x] Descending range queries - [x] Specify - [x] Implement - [x] Add test - [x] Batch updates to index counter - [x] Put K2V behind `k2v` feature flag Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: Deuxfleurs/garage#293 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>	2022-05-10 13:16:57 +02:00
Alex Auvolat	def78c5e6f	Update netapp to 0.4.4, fix #300	2022-05-09 12:08:47 +02:00
Alex	277a20ec44	Fix `layout show` to not show changes when there are no changes (#297 ) fixes #295, partially Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: Deuxfleurs/garage#297 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>	2022-05-09 11:14:55 +02:00
Alex Auvolat	617f28bfa4	Correct small formatting issue	2022-05-05 14:21:57 +02:00
Mendes	948ff93cf1	Corrected the warnings and errors issued by cargo clippy	2022-05-01 16:05:39 +02:00
Alex Auvolat	3ba2c5b424	updated cargo.lock	2022-05-01 10:11:43 +02:00
Alex Auvolat	2aeaddd5e2	Apply cargo fmt	2022-05-01 09:57:05 +02:00
Alex Auvolat	c1d1646c4d	Change the way new layout assignations are computed. The function now computes an optimal assignation (with respect to partition size) that minimizes the distance to the former assignation, using flow algorithms. This commit was written by Mendes Oulamara <mendes.oulamara@pm.me>	2022-05-01 09:54:19 +02:00
Alex Auvolat	c9ef3e461b	fix clippy	2022-04-19 12:50:40 +02:00
Alex Auvolat	c93008d333	Prettier code for defragmentation	2022-04-19 12:50:40 +02:00
Alex Auvolat	e5341ca47b	Defragmentation in UploadPartCopy: first pass (not pretty but it compiles)	2022-04-19 12:50:36 +02:00
withinboredom	a4f9f19ac3	remove size limitation in UploadPartCopy (#280 ) This removes the >1mb s3_copy restriction. This restriction doesn't seem to be documented anywhere (I could be wrong). It also causes some software to fail (such as #248). Co-authored-by: Rob Landers <landers.robert@gmail.com> Reviewed-on: Deuxfleurs/garage#280 Co-authored-by: withinboredom <landers.robert@gmail.com> Co-committed-by: withinboredom <landers.robert@gmail.com>	2022-04-19 12:49:43 +02:00
Baptiste Jonglez	47e57518ec	Add documentation on running Kopia with Garage	2022-04-10 13:04:07 +02:00
Alex Auvolat	dffcd9f4b1	update Cargo.nix	2022-04-08 14:35:09 +02:00
Alex Auvolat	5d404dcd54	Add missing opentelemetry features	2022-04-08 14:21:04 +02:00
Quentin Dufour	62f0715abe	Add/Fix OpenTelemetry	2022-04-07 16:12:35 +02:00
Quentin Dufour	7e1ac51b58	Add files to quickly test k8s	2022-04-07 16:12:35 +02:00
Alex Auvolat	94f1e48fff	Update to netapp 0.4.2 (a tiny fix)	2022-04-07 11:50:03 +02:00
Alex Auvolat	cb5836d53c	Bring maximum exponential backoff time down from 16h to 1h	2022-04-07 11:49:29 +02:00
Quentin Dufour	8e3ee82c3e	Be clearer on what upgrades are (not) supported	2022-04-06 21:45:59 +02:00
Quentin Dufour	a122a8cb46	Add an "upgrading" section, add a guide for 0.7	2022-04-05 10:08:31 +02:00
Quentin Dufour	9fd8ec1dee	Add documentation for winscp+sftpgo	2022-03-31 10:25:56 +02:00
Alex Auvolat	0091002ef2	New replication modes and their documentation	2022-03-28 16:26:04 +02:00
Alex Auvolat	8f9cf3a5d1	fix a clippy lint	2022-03-28 15:48:55 +02:00
Alex Auvolat	913f7754bb	Add blocks in errored state to `garage stats`	2022-03-28 15:47:23 +02:00
Alex Auvolat	42dde54126	Log admin GET requests at debug level instead of info to reduce noise in logs	2022-03-28 15:46:52 +02:00
Alex Auvolat	dca2ffdf91	document administrative options	2022-03-28 12:26:08 +02:00
Quentin Dufour	0cf4efac89	Compile kuberetes-discovery only when release=true	2022-03-24 16:57:43 +01:00
Alex Auvolat	9d0ed78887	Add feature flag for Kubernetes discovery	2022-03-24 16:57:43 +01:00
Alex Auvolat	509d256c58	Make layout optimization work in relative terms	2022-03-24 15:27:14 +01:00
Alex Auvolat	2814d41842	Allow `garage layout assign` to assign to several nodes at once	2022-03-24 15:27:13 +01:00
Alex Auvolat	7e0e2ffda2	Slight change and add comment to layout assignation algo	2022-03-24 15:27:13 +01:00
Alex Auvolat	413ab0eaed	Small change to partition assignation algorithm This change helps ensure that nodes for each partition are spread over all datacenters, a property that wasn't ensured previously when going from a 2 DC deployment to a 3 DC deployment	2022-03-24 15:27:10 +01:00
Alex Auvolat	43945234ae	Add missing src/block to toplevel cargo.toml	2022-03-23 10:26:10 +01:00
Alex Auvolat	3dc9214172	Add lots of comments on how the resync queue works (I don't really want to change/refactor that code though)	2022-03-23 10:25:39 +01:00
Alex Auvolat	077dd1cde9	Clippy	2022-03-23 10:25:39 +01:00
Alex Auvolat	2d13f0aa13	run cargo2nix	2022-03-23 10:25:37 +01:00
Alex Auvolat	e480aaf338	Make background tranquility a configurable parameter	2022-03-23 10:25:19 +01:00
Alex Auvolat	8fd6745745	Move block RC code to separate `rc.rs`	2022-03-23 10:25:19 +01:00
Alex Auvolat	c3982a90b6	Move DataBlock out of manager.rs	2022-03-23 10:25:19 +01:00

... 19 20 21 22 23 ...

1738 commits