Session

In this presentation we present “TCP Offload via AF_XDP sockets”. This is an innovative, elegant, and generic solution for TCP connection offload that addresses long standing issues associated with TCP offload and TCP Offload Engines (TOE). The key of this approach is to offload TCP connection processing and TCP state machine from a host CPU to a SmartNIC CPU running Linux. AF_XDP sockets are used to provide a generic and extensible offload messaging interface. The use of AF_XDP sockets for messaging is a novel approach to managing TCP Offload that has a big advantage that no kernel changes are required, and we can leverage eBPF and XDP to optimize data flow operations. The primary benefit of TCP connection offload via AF_XDP sockets is that it substantially reduces the network processing on the host CPU; this is relevant to cloud providers who want to free up host CPU cycles and sell them to their customers. A second advantage is that the TCP stack is tightly co-located with the hardware in the SmartNIC and that creates opportunities for optimizations and features that wouldn’t be possible otherwise.

In this talk we cover the overall design, the messaging API used over AF_XDP sockets, the client library, the TCP “proxy” running in the SmartNIC CPU, and example application code. The full lifetime of an offloaded connection is described including connection setup and tear down, data operations, flow control over the messaging interface. We’ll also discuss the hardware requirements (there’s not a lot!).