KokaKiwi
/
garage
forked from Deuxfleurs/garage
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
701 Commits 13 Branches 1 Tag 80 MiB
Commit Graph

86 Commits

Author SHA1 Message Date
Alex 9d0ed78887 Add feature flag for Kubernetes discovery 2022-03-24 16:57:43 +01:00
Alex c1d9854d2c
Move block manager to separate module 2022-03-23 10:25:15 +01:00
trinity-1686a 1eb7fdb08f add test framework for arbitraty S3 requests 
and implement some basic test with it
 2022-03-23 10:22:36 +01:00
Alex db46cdef79
Update netapp to v0.4.1 2022-03-15 17:09:57 +01:00
Alex 203e8d2c34
Bump version to 0.7 because of incompatible Netapp 2022-03-14 10:54:24 +01:00
Alex 0cc31ee169
add missing netapp telemetry feature 2022-03-14 10:54:11 +01:00
Alex d9a35359bf
Add metrics to web endpoint 2022-03-14 10:53:50 +01:00
Alex 818daa5c78
Refactor how durations are measured 2022-03-14 10:53:35 +01:00
Alex f0d0cd9a20
Remove strum crate dependency; add protobuf nix dependency 2022-03-14 10:53:00 +01:00
Alex bb04d94fa9
Update to Netapp 0.4 which supports distributed tracing 2022-03-14 10:52:30 +01:00
Alex 8c2fb0c066
Add tracing integration with opentelemetry 2022-03-14 10:52:13 +01:00
Alex 2cab84b1fe
Add many metrics in table/ and rpc/ 2022-03-14 10:51:50 +01:00
Maximilien R 1e2cf26373
Implement basic metrics in table 2022-03-14 10:51:17 +01:00
mricher e349af13a7
Update dependencies and add admin module with metrics 
- Global dependencies updated in Cargo.lock
- New module created in src/admin to host:
  - the (future) admin REST API
  - the metric collection
- add configuration block

No metrics implemented yet
 2022-03-14 10:51:12 +01:00
Max Audron 9d44127245
add support for kubernetes service discovery 
This commit adds support to discover garage instances running in
kubernetes.

Once enabled by setting `kubernetes_namespace` and
`kubernetes_service_name` garage will create a Custom Resources
`garagenodes.deuxfleurs.fr` with nodes public key as the resource name.
and IP and Port information as spec in the namespace configured by
`kubernetes_namespace`.

For discovering nodes the resources are filtered with the optionally set
`kubernetes_service_name` which sets a label
`garage.deuxfleurs.fr/service` on the resources.

This allows to separate multiple garage deployments in a single
namespace.

the `kubernetes_skip_crd` variable allows to disable the creation of the
CRD by garage itself. The user must deploy this manually.
 2022-03-12 13:05:52 +01:00
Quentin 8df1e186de Functional tests for website endpoints 2022-03-07 17:32:07 +01:00
trinity-1686a f6f8b7f1ad Support for PostObject (#222) 
Add support for [PostObject](https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPOST.html)

- [x] routing PostObject properly
- [x] parsing multipart body
- [x] validating signature
- [x] validating policy
- [x] validating content length
- [x] actually saving data

Co-authored-by: trinity-1686a <trinity@deuxfleurs.fr>
Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com>
Reviewed-on: Deuxfleurs/garage#222
Reviewed-by: Alex <alex@adnab.me>
Co-authored-by: trinity-1686a <trinity.pointard@gmail.com>
Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
 2022-02-21 23:02:30 +01:00
Jill dd407e7041
tests: Add garage integration tests (base) 2022-02-10 17:55:49 +01:00
Jill b45dcc1925 Support STREAMING-AWS4-HMAC-SHA256-PAYLOAD (#64) (#156) 
Closes #64.

Reviewed-on: Deuxfleurs/garage#156
Co-authored-by: Jill <kokakiwi@deuxfleurs.fr>
Co-committed-by: Jill <kokakiwi@deuxfleurs.fr>
 2022-01-17 10:55:31 +01:00
Alex beeef4758e
Some movement of helper code and refactoring of error handling 2022-01-04 12:52:46 +01:00
Alex b1cfd16913
New buckets for 0.6.0: small fixes, including: 
- ensure bucket names are correct aws s3 names
- when making aliases, ensure timestamps of links in both ways are the
  same
- fix small remarks by trinity
- don't have a separate website_access field
 2022-01-04 12:46:41 +01:00
Alex 4d30e62db4
New buckets for 0.6.0: migration code and build files 2022-01-04 12:46:13 +01:00
Alex 0bbb6673e7
Model changes 2022-01-04 12:45:52 +01:00
Alex 5b1117e582
New model for buckets 2022-01-04 12:45:46 +01:00
trinity-1686a 1eb972b1ac Add compression using zstd (#173) 
fix #27

Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com>
Reviewed-on: Deuxfleurs/garage#173
Co-authored-by: trinity-1686a <trinity.pointard@gmail.com>
Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
 2021-12-15 11:26:43 +01:00
Alex c94406f428
Improve how node roles are assigned in Garage 
- change the terminology: the network configuration becomes the role
  table, the configuration of a nodes becomes a node's role
- the modification of the role table takes place in two steps: first,
  changes are staged in a CRDT data structure. Then, once the user is
  happy with the changes, they can commit them all at once (or revert
  them).
- update documentation
- fix tests
- implement smarter partition assignation algorithm

This patch breaks the format of the network configuration: when
migrating, the cluster will be in a state where no roles are assigned.
All roles must be re-assigned and commited at once. This migration
should not pose an issue.
 2021-11-16 16:05:53 +01:00
Trinity Pointard 9c58ec28d3 add support for vhost-style s3 bucket 2021-11-16 15:41:41 +01:00
Alex 43e13a501d
Use published netapp crate instead of git repo 2021-10-26 10:36:57 +02:00
Alex de4276202a
Improve CLI, adapt tests, update documentation 2021-10-25 14:21:48 +02:00
Alex 1b450c4b49
Improvements to CLI and various fixes for netapp version 
Discovery via consul, persist peer list to file
 2021-10-22 16:55:24 +02:00
Alex 4067797d01
First port of Garage to Netapp 2021-10-22 15:55:18 +02:00
Alex b9127dd6f8
Prepare for v0.3.0 and add migration path from v0.2.1.x 2021-05-28 15:29:58 +02:00
Quentin 631c36b3ff S3 API: support ListBuckets 2021-05-03 21:55:30 +02:00
Alex f859d15062 update to v0.2.1 2021-03-19 13:39:18 +01:00
Alex fd8f4caa81 Support old CPUs 2021-03-19 12:19:40 +01:00
Alex dead945c8f Prepare for release 0.2 2021-03-18 19:33:15 +01:00
Alex f4346cc5f4 Update dependencies 2021-03-16 15:58:40 +01:00
Alex 0cd5b2ae19 WIP migrate to tokio 1 2021-03-15 22:36:41 +01:00
Alex 3bf2df622a Time and metadata improvements 2021-03-15 16:21:41 +01:00
Alex a1442f072a Implement garage stats to get info on node contents 2021-03-12 15:40:54 +01:00
Alex 94f3d28774 WIP big refactoring 2021-03-11 16:54:15 +01:00
Alex 445912dc6a Remove migration paths from 0.1 branch 2021-03-10 16:38:31 +01:00
Alex 20e6e9fa20 Update sled & try to debug deadlock (but its in sled...) 2021-02-23 21:27:28 +01:00
Alex e8e4418ca7 Add blake2 and xxhash hash functions 2021-02-23 17:52:28 +01:00
Alex 6a5add3386 Fix build 2021-01-15 19:12:08 +01:00
Alex e818f51073 Forgot a bump 2021-01-15 18:36:51 +01:00
Alex 1d1d497e2b Bump everything to 0.1.1 2021-01-15 17:54:48 +01:00
Quentin c441a358cd Remove unused dependencies 2021-01-15 16:16:32 +01:00
Quentin e8c12072ce Merge branch 'master' into feature/website 2020-12-10 20:12:56 +01:00
Alex 4a5bbbb810 Propose ETag fix 2020-12-05 19:23:46 +01:00