91b2d1fcc1
Some basic S3 functionnality
2020-04-24 18:47:11 +00:00
f2e05986c4
Starting to be S3 compatible
2020-04-24 17:46:52 +00:00
d8f5e643bc
Split code for modular compilation
2020-04-24 10:10:01 +00:00
51fb3799a1
Key management admin commands
2020-04-23 20:25:45 +00:00
4ef84a0558
Move repair to separate file
2020-04-23 18:36:12 +00:00
44a1089d95
Make table objects slightly more fool-proof; add key table
2020-04-23 18:16:52 +00:00
c9c6b0dbd4
Reorganize code
2020-04-23 17:05:46 +00:00
01a8acdeec
Better error reporting
2020-04-23 16:23:06 +00:00
82f4cd8719
Continue pinging nodes when they are down ; overall better handling of down nodes
2020-04-23 16:06:11 +00:00
2fe82be3bc
RPC to ourself do not pass through serialization + HTTPS
2020-04-23 14:40:59 +00:00
37f880bc09
RequestStrategy with possible interruption or not
2020-04-23 13:37:10 +00:00
73574ab43e
Fix in rpc_client (see comment in code)
2020-04-22 20:42:23 +00:00
897fafa8db
Improvements to block resync queue & worker
2020-04-22 20:32:58 +00:00
2556a1e383
I'm stupid though
2020-04-22 20:06:12 +00:00
231cb32955
Do not delete block if just a single replication error. Write TODO stuff.
2020-04-22 19:25:15 +00:00
8971f34c81
Well they still have to exit when we're exiting though
2020-04-22 17:04:33 +00:00
e8214cb180
Better concurrency:
...
Use Notify instead of stupid sleep in background worker
Use Semaphore to limit concurrent requests in rpc_client
Make more background tasks cancellable
2020-04-22 16:51:52 +00:00
c0335ac690
Remove a few features in dependencies
2020-04-21 20:37:02 +00:00
ec59e896c6
Make UUID & Hash Copy and remove some .clone() noise
2020-04-21 17:08:42 +00:00
8915224966
Return BadRequest codes for some admin_rpc failure cases
2020-04-21 16:45:32 +00:00
b1ddb933b0
Make the repair command accept subcommands to not do everything all the time
2020-04-21 16:40:17 +00:00
a04218047e
Do full sync on node (re)start
2020-04-21 16:15:32 +00:00
2a84d965ab
Improve table sync
2020-04-21 16:05:55 +00:00
0226561035
Do not insert deletion marker if there is no object to delete
2020-04-21 14:33:12 +00:00
be0a2bae81
Add node tags in configuration
2020-04-21 14:08:28 +00:00
cc4f2f1cfb
Pretty logging
2020-04-21 12:54:55 +00:00
53cf4d1baa
Log which workers are doing what
2020-04-19 21:33:38 +00:00
ec7f9f07e2
Implement repair object->version and version->block ref
2020-04-19 21:27:08 +00:00
04acaea231
Don't do version & block_ref updates in background on deletion
2020-04-19 20:52:20 +00:00
5ae32972ef
Implement repair command
2020-04-19 20:36:36 +00:00
a54f3158f1
Less output
2020-04-19 19:38:45 +00:00
ea75564851
More aggressive sync timings & improve other stuff
2020-04-19 17:59:59 +00:00
e325c7f47a
Add hostname to node info
2020-04-19 19:08:48 +02:00
a6129d8626
Begin implement bucket management & admin commands
2020-04-19 17:15:48 +02:00
302502f4c1
Add support for fully replicated tables with epidemic dissemination of updates
2020-04-19 15:14:23 +02:00
7131553c53
Refactor sharding logic; coming next: full replication with epidemic dissemination
2020-04-19 13:22:28 +02:00
4ba54ccfca
Reorder imports.
...
Trying to separate:
1. Stuff for handling the swarm of nodes and generic table data replication
2. Stuff for the object store core application: metadata tables and block management
3. Stuff for the S3 API
2020-04-18 19:39:57 +02:00
bd1618e78e
Remove proto.rs & move some definitions out of data.rs
2020-04-18 19:30:05 +02:00
f41583e1b7
Massive RPC refactoring
2020-04-18 19:21:34 +02:00
3f40ef149f
Fix sync: use max root checksum level
2020-04-17 21:59:07 +02:00
f62b54f1df
Fix add to resync on incref
2020-04-17 21:14:06 +02:00
ace07da7c1
Fix walk_ring_from
2020-04-17 21:08:43 +02:00
40c48e6a59
Several resync workers; add delay on retry resync
2020-04-17 20:58:10 +02:00
29a1e94f23
Implement missing handler for read_range
2020-04-17 19:38:47 +02:00
b4e96bdcf0
Fix paths :o
2020-04-17 19:20:17 +02:00
4abfb75509
Implement sending blocks to nodes that need them
2020-04-17 19:16:08 +02:00
db1c4222ce
Don't send items...
...
...if syncer doesn't need them because he's going to delete the partition anyway.
Also, fix block resync queue
2020-04-17 18:51:29 +02:00
4bacaaf53f
Resync block on read error
2020-04-17 18:38:11 +02:00
b780f6485d
Make sync send data both ways
2020-04-17 18:27:29 +02:00
69f1d8fef2
WIP
...
TODOs:
- ensure sync goes both way
- finish sending blocks to other nodes when they need them before deleting
2020-04-17 17:09:57 +02:00
