withings/garage
1
0
Fork 0
forked from Deuxfleurs/garage
Code Pull requests Activity
671 commits 33 branches 73 tags 53 MiB
Commit graph

457 commits

Author SHA1 Message Date
Alex Auvolat
fe62d01b7e
Implement exponential backoff for resync retries 2022-03-14 11:41:20 +01:00
Alex Auvolat
9b2b531f4d
Make admin server optional 2022-03-14 10:54:25 +01:00
Alex Auvolat
2377a92f6b
Add wrapper over sled tree to count items (used for big queues) 2022-03-14 10:54:25 +01:00
Alex Auvolat
203e8d2c34
Bump version to 0.7 because of incompatible Netapp 2022-03-14 10:54:24 +01:00
Alex Auvolat
f869ca625d
Add spans to table calls, change span names in RPC 2022-03-14 10:54:12 +01:00
Alex Auvolat
0cc31ee169
add missing netapp telemetry feature 2022-03-14 10:54:11 +01:00
Alex Auvolat
dc8d0496cc
Refactoring: rename config files, make modifications less invasive 2022-03-14 10:53:51 +01:00
Alex Auvolat
d9a35359bf
Add metrics to web endpoint 2022-03-14 10:53:50 +01:00
Alex Auvolat
2a5609b292
Add metrics to API endpoint 2022-03-14 10:53:36 +01:00
Alex Auvolat
818daa5c78
Refactor how durations are measured 2022-03-14 10:53:35 +01:00
Alex Auvolat
f0d0cd9a20
Remove strum crate dependency; add protobuf nix dependency 2022-03-14 10:53:00 +01:00
Alex Auvolat
55d4471599
Remove ... at end of hex IDs 2022-03-14 10:52:31 +01:00
Alex Auvolat
bb04d94fa9
Update to Netapp 0.4 which supports distributed tracing 2022-03-14 10:52:30 +01:00
Alex Auvolat
8c2fb0c066
Add tracing integration with opentelemetry 2022-03-14 10:52:13 +01:00
Alex Auvolat
2cab84b1fe
Add many metrics in table/ and rpc/ 2022-03-14 10:51:50 +01:00
Maximilien R
1e2cf26373
Implement basic metrics in table 2022-03-14 10:51:17 +01:00
mricher
e349af13a7
Update dependencies and add admin module with metrics 
- Global dependencies updated in Cargo.lock
- New module created in src/admin to host:
  - the (future) admin REST API
  - the metric collection
- add configuration block

No metrics implemented yet
 2022-03-14 10:51:12 +01:00
Max Audron
9d44127245
add support for kubernetes service discovery 
This commit adds support to discover garage instances running in
kubernetes.

Once enabled by setting `kubernetes_namespace` and
`kubernetes_service_name` garage will create a Custom Resources
`garagenodes.deuxfleurs.fr` with nodes public key as the resource name.
and IP and Port information as spec in the namespace configured by
`kubernetes_namespace`.

For discovering nodes the resources are filtered with the optionally set
`kubernetes_service_name` which sets a label
`garage.deuxfleurs.fr/service` on the resources.

This allows to separate multiple garage deployments in a single
namespace.

the `kubernetes_skip_crd` variable allows to disable the creation of the
CRD by garage itself. The user must deploy this manually.
 2022-03-12 13:05:52 +01:00
Quentin Dufour
c00b2c9948 Functional tests for admin commands 2022-03-07 17:32:07 +01:00
Quentin Dufour
8df1e186de Functional tests for website endpoints 2022-03-07 17:32:07 +01:00
Quentin Dufour
2ef60b8417 Functional test for multipart endpoints 2022-03-07 17:32:07 +01:00
Quentin Dufour
1e639ec67c Functional test for ListMultipartUploads 2022-03-07 17:32:07 +01:00
Quentin Dufour
cfea1e0315 Functional tests for bucket endpoints 2022-03-07 17:32:02 +01:00
Quentin Dufour
05eb79929e Functional tests for object operations 2022-03-07 17:05:10 +01:00
Quentin Dufour
0f4e0e8bb9 Move ListObjects tests to Rust 2022-03-07 17:05:10 +01:00
Alex Auvolat
8a5bbc3b0b
More permissive OPTIONS on S3 API 2022-03-01 11:15:16 +01:00
Alex Auvolat
97f245f218
Add tracing output to signature calculation 2022-02-28 12:22:39 +01:00
Alex Auvolat
8129a98291
Process CORS earlier in pipeline 2022-02-28 12:22:39 +01:00
trinity-1686a
f6f8b7f1ad Support for PostObject (#222) 
Add support for [PostObject](https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPOST.html)

- [x] routing PostObject properly
- [x] parsing multipart body
- [x] validating signature
- [x] validating policy
- [x] validating content length
- [x] actually saving data

Co-authored-by: trinity-1686a <trinity@deuxfleurs.fr>
Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com>
Reviewed-on: Deuxfleurs/garage#222
Reviewed-by: Alex <alex@adnab.me>
Co-authored-by: trinity-1686a <trinity.pointard@gmail.com>
Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
 2022-02-21 23:02:30 +01:00
Jill
84613e66a2
garage(tests): Remove RNG stuff 2022-02-11 10:50:55 +01:00
Jill
c8b30ebc79
garage(tests): Remove superfluous test 2022-02-10 17:55:51 +01:00
Jill
d7decda3f4
garage(tests): Add random suffix to created buckets. 2022-02-10 17:55:50 +01:00
Jill
cd13ea461b
garage(tests): Add some unsafe-usage doc and tweaks 2022-02-10 17:55:50 +01:00
Quentin Dufour
5d19f3d2d7
Add integration tests to Drone 2022-02-10 17:55:50 +01:00
Jill
3baa841d6f
tests: Fix garage integration test 2022-02-10 17:55:49 +01:00
Jill
dd407e7041
tests: Add garage integration tests (base) 2022-02-10 17:55:49 +01:00
Alex Auvolat
af261e1789 Fix a bug when a migration is followed by a rebalance 
Nodes would stabilize on different encoding formats for the values,
some having the pre-migration format and some having the post-migration
format. This would be reflected in the Merkle trees never converging
and thus having an infinite resync loop.
 2022-02-10 17:38:27 +01:00
Alex Auvolat
ae2f32baf1
Hide deleted key in bucket info (fix #211) 2022-02-02 17:12:48 +01:00
Alex Auvolat
c99f55c420
Add restriction on part ordering in CompleteMultipartUpload 2022-01-25 12:45:00 +01:00
Alex Auvolat
acdf893362
Fix partnumber 2022-01-25 12:25:23 +01:00
Alex Auvolat
338b1b83ee
Implement part_number for GetObject 2022-01-24 21:04:42 +01:00
Alex Auvolat
6dab836f3a
Multipart improvements 
- support part_number for HeadObject
- add checks in complete_multipart_upload
 2022-01-24 21:04:40 +01:00
Alex Auvolat
513a6b15f9
Handle OPTIONS on website endpoint 2022-01-24 12:32:28 +01:00
Alex Auvolat
ea7fb901eb
Implement {Put,Get,Delete}BucketCors and CORS in general 
- OPTIONS request against API endpoint
- Returning corresponding CORS headers on API calls
- Returning corresponding CORS headers on website GET's
 2022-01-24 11:58:00 +01:00
Trinity Pointard
820924534a use clamp instead of min(max()) 2022-01-24 11:56:59 +01:00
Quentin Dufour
440374524b Implement ListParts 2022-01-21 10:42:30 +01:00
Quentin Dufour
fe003d6fbc Add ListPartsResult structure 2022-01-20 16:38:55 +01:00
trinity-1686a
e55fa38c99 Add date verification to presigned urls (#196) 
fix #96
fix #162 by returning Forbidden instead Bad Request

Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com>
Reviewed-on: Deuxfleurs/garage#196
Co-authored-by: trinity-1686a <trinity.pointard@gmail.com>
Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
 2022-01-18 12:22:31 +01:00
trinity-1686a
178e35f868 refactor s3_router and api_server to make unused Endpoint parameters more obvious 2022-01-17 15:50:24 +01:00
Jill
fdcddbe168 Fix Multipart Upload with WinSCP (#164) (#193) 
Closes #164.

Reviewed-on: Deuxfleurs/garage#193
Co-authored-by: Jill <kokakiwi@deuxfleurs.fr>
Co-committed-by: Jill <kokakiwi@deuxfleurs.fr>
 2022-01-17 11:18:40 +01:00