Deuxfleurs
/
mknet
16
1
Fork 2
Code Issues Pull Requests Projects Releases Wiki Activity
Create a test network using network namespaces and tc-netem
60 Commits 1 Branch 0 Tags 8.6 MiB
Python 62.2%
Go 22.5%
R 14.9%
Nix 0.4%
Go to file
Quentin db88b939c3 Merge pull request 'Fix venv sourcing' (#2) from baptiste/mknet:misc_fixes into main 
Reviewed-on: #2
 		2022-09-29 20:57:53 +02:00
analysis Scenario based benchmarks will be ready soon 2022-09-14 18:22:46 +02:00
artifacts Add result with 30M objects on Grid'5000 2022-09-29 20:27:25 +02:00
benchmarks Bench PutObject with many parallel small obj 2022-09-24 15:13:40 +02:00
scenarios Add sled no-sync 2022-09-27 11:50:08 +02:00
topo Add S3TTFB 2022-09-23 22:50:50 +02:00
.gitignore Create a prepare script 2022-09-16 15:43:58 +02:00
.gitmodules Scenario based benchmarks will be ready soon 2022-09-14 18:22:46 +02:00
LICENSE add license 2021-12-12 17:54:54 +01:00
README.md Document s3billion 2022-09-29 10:40:34 +02:00
mknet Add support for txqueuelen 2022-09-19 18:02:33 +02:00
net.py Add support for txqueuelen 2022-09-19 18:02:33 +02:00
prepare.py prepare: fix venv sourcing when /bin/sh is Dash (e.g. on Debian) 2022-09-29 10:39:45 +02:00
setup.py New way to deploy Garage 2022-09-14 17:25:16 +02:00
shell.nix New way to deploy Garage 2022-09-14 17:25:16 +02:00

README.md

mknet

mknet is a tool to simulate various network topologies locally thanks to network namespaces and traffic control (tc).

Prepare your environment

Get the repository:

git clone https://git.deuxfleurs.fr/Deuxfleurs/mknet.git
cd mknet

Run our configuration script:

./prepare.py

Now, you are ready to launch an experiment:

sudo -i
source .venv/bin/activate
./mknet scenario ./topo/50ms.yml ./scenarios/garage-s3lat garage-v0.8

If a script crash, you must manually destroy the topology:

./mknet destroy

Topologies

All topologies:

  • ./topo/dc.yml - A 3 node topology connected with a 1Gbit/s link and a 1ms latency
  • ./topo/with-vdsl.yml - A topology mixing datacenters with fast internal connectivity and an isolated VDSL node
  • ./topo/50ms.yml - An artifical topology simulating nodes with high bandwidth but with a fixed 50ms latency, useful to quantify the impact of latency on a distributed software
  • ./topo/multi-dc.yml - Simulate 3 DC interconnected with 50ms latency WAN network and close to zero latency inside the DC
  • ./topo/{1,5,10,50,100}mbps.yml - Simulate 3 nodes interconnected with a bandwidth bottleneck.

Feel free to write new topologies!

Scenarios

All scenarios:

  • ./scenarios/garage-s3lat [garage-v0.7|garage-v0.8] - Run s3lat on Garage
  • ./scenarios/garage-concurrent [garage-v0.7|garage-v0.8] - Run s3concurrent on Garage
  • ./scenarios/garage-warp [garage-v0.7|garage-v0.8] [default|fast] - Run warp on Garage. 2 flavors are available: fast and default.
  • ./scenarios/garage-s3billion [garage-v0.7|garage-v0.8] - Run s3billion on Garage

Scenarios take optional flavors as input that modulate their behavioir. Passing them is not mandatory, a default one will be selected for you.

How to run them:

./mknet scenario <topology> <scenario> [flavors...]
./mknet scenario ./topo/50ms.yml ./scenarios/garage-s3lat garage-v0.8

How to write good scenarios:

  • If a scenario can be run with multiple different parameters, write one scenario with multiple flavors
  • If the logic ran is different, write a new scenario
  • A scenario code must remain short and looks like a DSL, abstract the logic in the fragments/ module

Manual usage

./mknet create ./topo/with-vdsl.yml
./mknet run-all ./scenarios/garage-manual
./mknet run dc1:dc1s1 /tmp/mknet-bin/garage* -c /tmp/mknet-store/garage/dc1/dc1s1/garage.toml status
./mknet run-all ./scenarios/clean
./mknet destroy