Session

Large-scale AI and HPC jobs depend on RDMA to move data between nodes with low latency and high throughput. RDMA commonly relies on userspace verbs and NIC hardware offload, with DPU-based datapaths becoming increasingly common in clustered deployments. Since that datapath often bypasses the normal host networking stack, encryption that protects ordinary host networking, such as TLS, kTLS, or in-kernel IPsec, can leave RDMA flows outside the intended protection boundary. This  matters in multi-tenant clusters where model weights, inference data, parameters, and intermediate tensors cross the fabric as the industry increasingly looks to secure this layer.

MACsec (IEEE 802.1AE) operates at Layer 2, below the RDMA transport, and when offloaded to hardware, encrypts egress traffic inline below RoCE and other RDMA transports without host CPU crypto cost. The Linux kernel’s MACsec HW offload infrastructure (macsec_ops, NETIF_F_HW_MACSEC, ip macsec offload mac) already supports this, but cloud deployments also need a way to turn MACsec capability into a schedulable property, so workloads land on nodes where RDMA egress can actually be protected. DPU-based RDMA offloads add another split to the model. The host may see an RDMA-capable PCIe function, while the MACsec engine, egress port, and control plane live on the DPU side.

We build on this netdev contract by carrying DPU-side MACsec datapath state into the host-visible RDMA netdev, then showing how that state can be consumed by DRANet, a Kubernetes Dynamic Resource Allocation network driver. DRANet-Sec publishes MACsec capability alongside RDMA, SR-IOV, and NUMA locality in the node ResourceSlice. This enables a workload to request placement on nodes where RDMA egress is eligible for hardware MACsec protection without application changes. In effect, MACsec becomes a second hardware offload stacked beneath RDMA and invisible to the workload above it. On a DPU-based RDMA testbed, we show how the pieces fit together end to end, from MACsec datapath state to host netdev feature advertisement, DRANet-Sec resource publication, and scheduler-visible placement. The talk focuses on the kernel and netdev mechanics needed to keep the advertised capability aligned with the real protected datapath, a challenge that recurs for any DPU-resident offload exposed to workload scheduling