Session

Nanotube: Turning XDP Code into Custom Hardware Pipelines for FPGA SmartNICs

Speakers

Stephan Diestelhorst
Neil Turton
Kieran Mansley
Kimon Karras
Rip Sohan
Thomas Calvert
Steven Pope

Label

Moonshot

Session Type

Talk

Contents

Description

Network packet processing with EBPF XDP on the CPU reduces context switching, data copying, memory overheads and wasted software parsing of the packet data, greatly improving efficiency for some use cases. However, the hardware overheads of delivering packets from a NIC to the CPU are still present and in high bandwidth data centre environments can present significant costs. Many modern NICs have built-in packet processing capabilities that can effectively perform the type of use case that XDP is often used for. Being able to seamlessly offload the standard EBPF C code from the host to the NIC – under the control of the Linux networking subsystem and tooling – would increase server efficiency and performance without affecting the user experience.

We present Nanotube: an open-source compiler that consumes standard EBPF XDP C code and produces a tailor-made, application-specific dataflow pipeline suitable for execution on FPGA NIC platforms. To achieve that Nanotube unrolls the XDP programs spatially into a pipeline with packets flowing in fixed size blocks (packet words) every clock cycle. Nanotube processes the application source with multiple steps performing: memory / pointer conversion, control flow restructuring, API lowering, packet and map access merging, control plane logic addition, automatic application splitting into multiple stages and generation of HLS C++ code. It is written as a set of LLVM passes.

We have demonstrated its use on AMD / Xilinx FPGA-based NICs to offload XDP from the Linux kernel, expecting to show throughput at line rate (100 Gbps) on AMD NICs and provide results for applications such as Katran.

This talk outlines (i) the structure of the compiler; (ii) performance of typical workloads; and (iii) how map accesses by host software are supported using specially annotated packets over the normal host data path.

Finally, we outline areas for discussion around how an offloaded XDP program can be better integrated and supported by the Linux kernel, in particular the validator, map handling, and vendor-neutral support