Finalize the UDP writeup

README.md

@@ -31,17 +31,17 @@ We try to document our journey here:
     - [Our OpenWRT configuration](./doc/config.md)
     - Developped scripts
   - Problems we encountered
-    - Dimensions for the board in the datasheet are wrong
+    - ✅ Dimensions for the board in the datasheet are wrong
-    - Ethernet ports are not configured at boot
+    - ✅ Ethernet ports are not configured at boot
-    - Signal is not optimal with an antenna under the roof
+    - ✅ [The UDP+Don't Fragment bug](./doc/udp-df.md)
-    - [The UDP+Don't Fragment bug](./doc/udp-df.md)
+    - ⌛ [Tracking the bandwidth bottleneck](./doc/bottleneck.md)
-    - [Tracking the bandwidth bottleneck](./doc/bottleneck.md)
+    - ⌛ Signal is not optimal with an antenna under the roof
-    - Our modem never connects on NR5G Non StandAlone (NSA) aka E-utran New radio Dual Connectivity (EN-DC)
+    - ⌛ Our modem never connects on NR5G Non StandAlone (NSA) aka E-utran New radio Dual Connectivity (EN-DC)
-    - [Some logs/data/info about our hardware](./doc/logs.md)
   - Experience Reports
     - [Measurements next to the eNodeB](./doc/next_antenna.md)
   - Misc
     - [Our prototype: the TP-Link TL-WDR 3600 + Huawei e3372h](./doc/proto.md)
+    - [Some logs/data/info about our hardware](./doc/logs.md)
     - [Bibliography](./doc/bib.md)
     - [Some mostly useless hacks](./doc/hacks.md)
 

doc/udp-df.md

@@ -3,6 +3,13 @@
 I have observed that when communicating with UDP, if the Don't Fragment bit in the IP flags header is not set,
 the packet is not delivered.
 
+Here are the following impacted applications:
+  - Multiplayer mode of Trackmania
+  - Discord
+  - Wireguard
+  - dig
+  - Rocket League
+
 ## Identifying / reproducing the problem
 
 Here is an example with DNS where the 1st request is done with dnsmasq that set the DF bit and the second request is done via dig that does not set the DF bit. Both requests are sent to Quad9 DNS to resolve the `lesterpig.com` domain:
@@ -117,10 +124,24 @@ If the problem is only one way:
 If the problem is 2 ways:
  - Use a VPN (openvpn works as it probably sets the DF bit)
 
+## Choosing XDP as the workaround
+
+While we successfully validated the OpenVPN solution, we do not want to add this complexity in our daily setup.
+Moreover, as only the egress traffic (TX) is impacted and not the ingress (RX), we can rewrite the packets locally.
+By being fast and secure, we selected XDP to rewrite the packets.
+In the journey, we mainly learnt/discovered 3 main things:
+  - XDP is INGRESS only, an EGRESS patch has been proposed but is not yet merged. It means we can not rewrite packets that are sent over an interface, only packets that are received on the interface. Hopefully, when my computer sends an UDP packet (to a DNS server for example), it is sent to its ethernet interface, received on the `br-lan` interface of the router then forwarded on its `wwan` interface. So, the only place we can do the rewrite is on `br-lan`. It has two drawbacks: local packets are rewritten while its useless and the router can not benefit from the XDP module for its own traffic. Still, this situation seemed acceptable to us.
+  - At first, we thought it would be simple as we only want to flip a bit! But IPv4 has checksums (while IPv6 not), so we need to recompute it. bpf-helpers has some tools for that but they often requires a `sk_buff`, the internal structure of the kernel. But we are such low level here that this structure has not been created yet. It stll might be possible to use some functions but without any example and due to the quite strange interface and documentation, none of our tests we successful. We finally reimplemented in BPF the original checksuming from [RFC 1071](https://tools.ietf.org/html/rfc1071), and it works! We know that as we are only modifying part of the packet, we could do a more efficient incremental update as decribed in [RFC 1624](https://tools.ietf.org/html/rfc1624). We did not put the effort to understand and implement this optimization as the simple solution is easier to understand/debug/implement for us while providing satisfying performances.
+  - Finally, it appears that OpenWRT (in its upstream version) supports XDP out of the box, but you need a loader. Because we do not need BPF maps, we can use the loader embedded in iproute2. However, by default on OpenWRT, iproute2 is not compiled with the support for XDP/eBPF. To embed the loader in iproute2, you need to install/select the `ip-full` from OpenWRT (in the `Network/Routing and Redirection` section). You can find [these details in the Makefile](https://github.com/openwrt/openwrt/blob/master/package/network/utils/iproute2/Makefile)  of the package.
+
+Linked resources:
+  - [The code + instructions to compile and load](../xdp)
+  - [The compiled file](../files/var/lib/xdp/xdp_udp.o)
+
 ## More definitive solutions
 
   - Find the code involved and patch it
     - I have sent an email to Simcom to ask them to patch the firmware of their modem as of 2021-04-17
-  - If it is a misconfiguration from Free, inform them
+  - If it is a misconfiguration from Free Mobile, inform them
-
 
+➡️  The manufacturer has been contacted but did not answer