Session

Traits: Rich Packet Metadata

Speakers

Arthur Fabre
Jakub Sitnicki
Jesper Brouer

Label

Moonshot

Session Type

Talk

Description

Currently, the only way to attach information to a sk_buff that travels through the network stack is by using the mark field. This 32-bit field is highly versatile - it can be read in firewall rules, drive routing decisions, and be accessed by BPF programs. However, its limited capacity creates competition for bits, restricting its practical use.

At Linux Plumbers 2024 [1], we proposed an alternative to the mark field: a binary blob storage, which extends the concept of the XDP metadata area, accompanied by custom BPF-based data channels for passing metadata between the kernel and user space. The idea originated from Cloudflare’s operational needs, which include:

  1. Tracing packets through the network stack and across the kernel-user space boundary.
  2. Metadata-driven packet redirection, routing, and socket steering with early classification in XDP.
  3. Extracting information from encapsulation headers and sharing it with user space or vice versa.

Based on community feedback, we shifted to a more refined solution: a bespoke key-value store for metadata, optimized for use within the XDP code. User-space access to the store is provided natively through an extension of the Linux Sockets API. The feature is tentatively called “packet traits” to differentiate it from the already overloaded term “metadata.”

In our talk, we will present the API features and performance characteristics of the packet traits KV store, backed by the existing packet headroom. We’ll also explain how a shared key space, combined with a registration API, addresses the challenges of metadata merging in scenarios like GRO packet coalescing and IP fragment reassembly.

Another focus of the talk will be the user API for reading and writing packet traits. We’ll discuss the design of a networking uAPI extension that supports both connection-oriented (TCP) and connectionless (UDP) sockets. Additionally, we’ll show how these extensions align with established Linux networking API patterns, ensuring ease of adoption.

[1] https://lpc.events/event/18/contributions/1935/