Add some doc

README.md

@@ -22,8 +22,8 @@ We try to document our journey here:
   - [Flash OpenWRT on the WPJ428](./doc/flash.md)
   - [Manually connect a modem to the network](./doc/manual-modem.md)
   - [Our OpenWRT configuration](./doc/config.md)
-  - The UDP+Don't Fragment bug (soon)
-  - Tracking the bandwidth bottleneck (soon)
+  - [The UDP+Don't Fragment bug](./doc/udp-df.md)
+  - [Tracking the bandwidth bottleneck](./doc/bottleneck.md)
   - [Some logs/data/info about our hardware](./doc/logs.md)
   - [Bibliography](./doc/bib.md)
   - [Some mostly useless hacks](./doc/hacks.md)

doc/bottleneck.md

@@ -0,0 +1,3 @@
+# Bottleneck
+
+*Work in progress*

doc/udp-df.md

@@ -0,0 +1,80 @@
+# The UDP/DF bug
+
+I have observed that when communicating with UDP, if the Don't Fragment bit in the IP flags header is not set,
+the packet is not delivered.
+
+## Identifying / reproducing the problem
+
+Here is an example with DNS where the 1st request is done with dnsmasq that set the DF bit and the second request is done via dig that does not set the DF bit. Both requests are sent to Quad9 DNS to resolve the `lesterpig.com` domain:
+
+```
+# tcpdump -vv -i wwan0 'port 53 and (host 9.9.9.10 or host 149.112.112.10)'
+tcpdump: listening on wwan0, link-type RAW (Raw IP), capture size 262144 bytes
+17:30:39.832839 IP (tos 0x0, ttl 64, id 40453, offset 0, flags [DF], proto UDP (17), length 70)
+    192.0.0.2.61829 > dns10.quad9.net.53: [udp sum ok] 26437+ [1au] A? lesterpig.com. ar: . OPT UDPsize=512 (42)
+17:30:39.913513 IP (tos 0x0, ttl 57, id 0, offset 0, flags [DF], proto UDP (17), length 86)
+    dns10.quad9.net.53 > 192.0.0.2.61829: [udp sum ok] 26437 q: A? lesterpig.com. 1/0/1 lesterpig.com. A 89.89.231.11 ar: . OPT UDPsize=512 (58)
+[...]
+17:30:39.913513 IP (tos 0x0, ttl 57, id 0, offset 0, flags [DF], proto UDP (17), length 86)
+    dns10.quad9.net.53 > 192.0.0.2.61829: [udp sum ok] 26437 q: A? lesterpig.com. 1/0/1 lesterpig.com. A 89.89.231.11 ar: . OPT UDPsize=512 (58)
+[...]
+17:30:49.497921 IP (tos 0x0, ttl 63, id 35598, offset 0, flags [none], proto UDP (17), length 82)
+    192.0.0.2.42500 > dns10.quad9.net.53: [udp sum ok] 41045+ [1au] A? lesterpig.com. ar: . OPT UDPsize=4096 (54)
+[...]
+17:30:54.497440 IP (tos 0x0, ttl 63, id 36723, offset 0, flags [none], proto UDP (17), length 82)
+    192.0.0.2.42500 > dns10.quad9.net.53: [udp sum ok] 41045+ [1au] A? lesterpig.com. ar: . OPT UDPsize=4096 (54)
+[...]
+17:30:59.497555 IP (tos 0x0, ttl 63, id 40590, offset 0, flags [none], proto UDP (17), length 82)
+    192.0.0.2.42500 > dns10.quad9.net.53: [udp sum ok] 41045+ [1au] A? lesterpig.com. ar: . OPT UDPsize=4096 (54)
+
+46 packets captured
+46 packets received by filter
+0 packets dropped by kernel
+```
+
+I have observed this problem at least twice before:
+  - With Trackmania
+  - With Wireguard
+
+I also reproduced the bug with Scapy+netcat.
+First I use netcat to listen on UDP on a public server:
+
+```
+nc -ul 2372
+```
+
+Then I try to send a packet through scapy:
+
+```python
+# IP flags's bits
+# X.. Reserved
+# .X. Don't fragment
+# ..X More fragment
+send(IP(dst="212.47.253.12/30",flags=0b010)/UDP(dport=2372)/Raw(load='does ... work\n'))
+send(IP(dst="212.47.253.12/30",flags=0b000)/UDP(dport=2372)/Raw(load='does not work\n'))
+```
+
+I did not observed this bug when I was using a TP-Link+Huawei e3372h, so it probably does not come from my ISP.
+(But it *still* could come from it as we activated IPv6 + 5G).
+
+Still, for now a question is not yet answered: do the bit must be set in both direction or only one?
+Additional tests are thus required
+
+## Some possible workarounds
+
+If the problem is only one way:
+
+ - [Add a netfilter extension to rewrite DF bits](https://github.com/semverchenko/dontfragment)
+ - Use netfilter netqueue
+ - Write a XDP patch
+
+If the problem is 2 ways:
+ - Use a VPN (openvpn works as it probably sets the DF bit)
+
+## More definitive solutions
+
+  - Find the code involved and patch it
+    - I have sent an email to Simcom to ask them to patch the firmware of their modem as of 2021-04-17
+  - If it is a misconfiguration from Free, inform them
+
+