Download A Survey on Data Plane Programming with P4 and more Summaries Programming Languages in PDF only on Docsity!
A Survey on Data Plane Programming with P4:
Fundamentals, Advances, and Applied Research
Frederik Hausera, Marco Häberlea, Daniel Merlinga, Steffen Lindnera, Vladimir Gurevichb, Florian Zeigerc, Reinhard Frankc, Michael Mentha aUniversity of Tuebingen, Department of Computer Science, Chair of Communication Networks, Tuebingen, Germany bIntel, Barefoot Division (BXD), United States of America cSiemens AG, Corporate Technology, Munich, Germany
Abstract Programmable data planes allow users to define their own data plane algorithms for network devices including appropriate data plane application programming interfaces (APIs) which may be leveraged by user-defined software-defined net- working (SDN) control. This offers great flexibility for network customization, be it for specialized, commercial appliances, e.g., in 5G or data center networks, or for rapid prototyping in industrial and academic research. Programming protocol-independent packet processors (P4) has emerged as the currently most widespread abstraction, programming language, and concept for data plane pro- gramming. It is developed and standardized by an open community, and it is supported by various software and hardware platforms. In the first part of this paper we give a tutorial of data plane programming models, the P4 programming language, architectures, compilers, targets, and data plane APIs. We also consider research efforts to advance P4 technology. In the second part, we categorize a large body of literature of P4-based applied research into different research domains, summarize the contributions of these papers, and extract prototypes, target platforms, and source code availability. For each research domain, we analyze how the reviewed works benefit from P4’s core features. Finally, we discuss potential next steps based on our findings.
Keywords: P4, SDN, programmable data planes
Email addresses: frederik.hauser@uni-tuebingen.de (Frederik Hauser), marco.haeberle@uni-tuebingen.de (Marco Häberle), daniel.merling@uni-tuebingen.de (Daniel Merling), steffen.lindner@uni-tuebingen.de (Steffen Lindner), vladimir.gurevich@intel.com (Vladimir Gurevich), florian.zeiger@siemens.com (Florian Zeiger), reinhard.frank@siemens.com (Reinhard Frank), menth@uni-tuebingen.de (Michael Menth)
Preprint submitted to JNCA August 5, 2021
arXiv:2101.10632v3 [cs.NI] 4 Aug 2021
- Introduction
Traditional networking devices such as routers and switches process packets using data and control plane algorithms. Users can configure control plane features and protocols, e.g., via CLIs, web interfaces, or management APIs, but the underlying algorithms can be changed only by the vendor. This limitation has been broken up by SDN and even more by data plane programming. SDN makes network devices programmable by introducing an API that al- lows users to bypass the built-in control plane algorithms and to replace them with self-defined algorithms. Those algorithms are expressed in software and typically run on an SDN controller with an overall view of the network. Thereby, complex control plane algorithms designed for distributed control can be re- placed by simpler algorithms designed for centralized control. This is beneficial for use cases that are demanding with regard to flexibility, efficiency and secu- rity, e.g., massive data centers or 5G networks. Programmable data planes enable users to implement their own data plane algorithms on forwarding devices. Users, e.g., programmers, practitioners, or operators, may define new protocol headers and forwarding behavior, which is without programmable data planes only possible for a vendor. They may also add data plane APIs for SDN control. Data plane programming changes the power of the users as they can build custom network equipment without any compromise in performance, scalabil- ity, speed, or power on appropriate platforms. There are different data plane programming models, each with many implementations and programming lan- guages. Examples are Click [1], VPP [2], NPL [3], and SDNet [4]. Programming protocol-independent packet processors (P4) is currently the most widespread abstraction, programming language, and concept for data plane programming. First published as a research paper in 2014 [5], it is now developed and standardized in the P4 Language Consortium, it is supported by various software- and hardware-based target platforms, and it is widely applied in academia and industry. In the following, we clarify the contribution of this survey, point out its novelty, explain its organization, and provide a table with acronyms frequently used in this work.
1.1. Contributions
This survey pursues two objectives. First, it provides a comprehensive intro- duction and overview of P4. Second, it surveys publications describing applied research based on P4 technology. Its main contributions are the following:
- We explain the evolution of data plane programming with P4, relate it to prior developments such as SDN, and compare it to other data plane programming models.
- We give an overview of data plane programming with P4. It comprises the P4 programming language, architectures, compilers, targets, and data
1.3. Paper Organization
Figure 1 depicts the structure of this paper which is divided into two main parts: an overview of P4 and a survey of research publications. In the first part, Section 2 gives an introduction to network programmabil- ity. We describe the development from traditional networking and SDN to data plane programming and present the two most common data plane programming models. In Section 3, we give a technology-oriented tutorial of P4 based on its latest version P4 16. We introduce the P4 programming language and describe how user-provided P4 programs are compiled and executed on P4 targets. Sec- tion 4 presents the concept of P4 architectures as intermediate layer between the P4 programs and the targets. We introduce the four most common archi- tectures in detail and describe P4 compilers. In Section 5, we categorize and present platforms that execute P4 programs, so-called P4 targets that are based on software, FPGAs, ASICs, or NPUs. Section 6 gives an introduction to data plane APIs. We describe their functions, present a characterization, introduce the four main P4 data plane APIs that serve as interfaces for SDN controllers, and point out controller use case patterns. In Section 7, we summarize research efforts that aim to improve P4 data plane programming. The second part of the paper surveys P4-based applied research in com- munication networks. In Section 8, we classify core features of P4 that make it attractive for the implementation of data plane algorithms. We use these properties in later sections to effectively reason about P4’s value for the im- plementation of various prototypes. We present an overview of the research domains and compile statistics about the included publications. The super- ordinate research domains are monitoring (Section 9), traffic management and congestion control (Section 10), routing and forwarding (Section 11), advanced networking (Section 12), network security (Section 13), and miscellaneous (Sec- tion 14) to cover additional, different topics. Each category includes a table to give a quick overview of the analyzed papers with regard to prototype imple- mentations, target platforms, and source code availability. At the end of each section, we analyze how the reviewed works benefit from P4’s core features. In Section 15 we discuss insights from this survey and give an outlook on potential next steps. Section 16 concludes this work.
1.4. List of Acronyms
The following acronyms are used in this paper.
ACL access control list
ALU arithmetic logic unit
API application programming interface
AQM active queue management
ASIC application-specific integrated circuit
AWW adjusting advertised windows
bmv2 Behavioral Model version 2
Part I: Overview of P
Network Programmability (Sect. II)
The P4 Programming Language (Sect. III)
Part II: Applied Research Domains
Discussion & Outlook (Sect. XV)
Monitoring (Sect. IX)
Introduction (Sect. I)
P4 Architectures & Compilers (Sect. IV)
P Targets (Sect. V)
P4 Data Plane APIs (Sect. VI)
Advances in P4 Data Plane Programming (Sect. VII)
Classification & Overview (Sect. VIII)
Advanced Networking (Sect. XII)
Network Security (Sect. XIII)
Routing and Forwarding (Sect. XI)
Miscellaneous Research Domains (Sect. XIV)
Traffic Management and Congestion Control (Sect. X)
Conclusion (Sect. XVI)
Figure 1: Organization of the paper.
BGP Border Gateway Protocol
BPF Berkeley Packet Filter
CLI command line interface
DAG directed acyclic graph
DDoS distributed denial of service
DPI deep packet inspection
DPDK Data Plane Development Kit
DSL domain-specific language
eBPF Extended Berkeley Packet Filter
ECN Explicit Congestion Notification
FPGA field programmable gate array
FSM finite state machine
GTP GPRS tunneling protocol
TNA Tofino Native Architecture
uBPF user-space BPF
VM virtual machine
VNF virtual network function
VPP Vector Packet Processors
WG working group
XDP eXpress Data Path
- Network Programmability
In this section, we first define the notion of network programmability and related terms. Then, we discuss control plane programmability and data plane programming, elaborate on data plane programming models, and point out the benefits of data plane programming.
2.1. Definition of Terms
We define programmability as the ability of the software or the hardware to execute an externally defined processing algorithm. This ability separates programmable entities from flexible (or configurable) ones; the latter only allow changing different parameters of the internally defined algorithm which stays the same. Thus, the term network programmability means the ability to define the pro- cessing algorithm executed in a network and specifically in individual processing nodes, such as switches, routers, load balancers, etc. It is usually assumed that no special processing happens in the links connecting network nodes. If nec- essary, such processing can be described as if it takes place on the nodes that are the endpoints of the links or by adding a "bump-in-the-wire" node with one input and one output. Traditionally, the algorithms, executed by telecommunication devices, are split into three distinct classes: the data plane, the control plane, and the man- agement plane. Out of these three classes, the management plane algorithms have the smallest effect on both the overall packet processing and network be- havior. Moreover, they have been programmable for decades, e.g., SNMPv1 was standardized in 1988 and created even earlier than that. Therefore, management plane algorithms will not be further discussed in this section. True network programmability implies the ability to specify and change both the control plane and data plane algorithms. In practice this means the ability of network operators (users) to define both data and control plane algorithms on their own, without the need to involve the original designers of the network equipment. For the network equipment vendors (who typically design their own control plane anyway), network programmability mostly means the ability to define data plane algorithms without the need to involve the original designers of the chosen packet processing application-specific integrated circuit (ASIC).
Network programmability is a powerful concept that allows both the network equipment vendors and the users to build networks ideally suited to their needs. In addition, they can do it much faster and often cheaper than ever before and without compromising the performance or quality of the equipment. For a variety of technical reasons, different layers became programmable at different point in time. While the management plane became programmable in the 1980s, control plane programmability was not achieved until late 2000s to early 2010s and a programmable switching ASICs did not appear till the end of
Thus, despite the focus on data plane programmability, we will start by dis- cussing control plane programmability and its most well-known embodiment, called software-defined networking (SDN). This discussion will also better pre- pare us to understand the significance of data plane programmability.
2.2. Control Plane Programmability and SDN Traditional networking devices such as routers or switches have complex data and control plane algorithms. They are built into them and generally cannot be replaced by the users. Thus, the functionality of a device is defined by its vendor who is the only one who can change it. In industry parlance, vendors are often called original equipment manufacturers (OEMs). Software-defined networking (SDN) was historically the first attempt to make the devices, and specifically their control plane, programmable. On se- lected systems, device manufacturers allowed users to bypass built-in control plane algorithms so that the users can introduce their own. These algorithms could then directly supply the necessary forwarding information to the data plane which was still non-replaceable and remained under the control of the device vendor or their chosen silicon provider. For a variety of technical reasons, it was decided to provide an APIs that could be called remotely and that is how SDN was born. Figure 2 depicts SDN in comparison to traditional networking. Not only the control plane became programmable, but it also became possible to implement network-wide control plane algorithms in a centralized controller. In several important use cases, such as tightly controlled, massive data centers, these centralized, network-wide algorithms proved to be a lot simpler and more efficient, than the traditional algorithms (e.g. Border Gateway Protocol (BGP)) designed for decentralized control of many autonomous networks. The effort to standardize this approach resulted in the development of Open- Flow (OF) [21]. The hope was that once OF standardized the messaging API to control the data plane functionality, SDN applications will be able to lever- age the functions offered by this API to implement network control. There is a huge body of literature giving an overview of OF [13, 14, 15] and SDN [6, 7, 8, 9, 11, 10, 12]. However, it soon became apparent that OF assumed a specific data plane functionality which was not formally specified. Moreover, the specific data plane, that served as the basis for OF, could not be changed. It executed the sole, although relatively flexible, algorithm defined by the OF specifications.
Control plane
Data plane
Vendor-based creation of network devices with data plane programming
API
Control plane
Data plane
Full network programability with data plane programming
API
Programability (by the user)
Programability (by the user)
Figure 3: Data plane programmability may be used by vendors for more efficient development or by users to provide own data and control plane algorithms.
2.4. Data Plane Programming Models
Data plane algorithms can and often are expressed using standard program- ming languages. However, they do not map very well onto specialized hardware such as high-speed ASICs. Therefore, several data plane models have been pro- posed as abstractions of the hardware. Data plane programming languages are tailored to those data plane models and provide ways to express algorithms for them in an abstract way. The resulting code is then compiled for execu- tion on a specific packet processing node supporting the respective data plane programming model. Data flow graph abstractions and the Protocol Independent Switching Ar- chitecture (PISA) are examples for data plane models. We give an overview of the first and elaborate in-depths on the second as PISA is the data plane programming model for P4.
2.4.1. Data Flow Graph Abstractions In these data plane programming models, packet processing is described by a directed graph. The nodes of the graph represent simple, reusable primitives that can be applied to packets, e.g., packet header modifications. The directed edges of the graph represent packet traversals where traversal decisions are per- formed in nodes on a per-packet basis. Figure 4 shows an exemplary graph for IPv4 and IPv6 packet forwarding. Examples for programming languages that implement this data plane pro- gramming model are Click [1], Vector Packet Processors (VPP) [2], and BESS [22].
2.4.2. Protocol-Independent Switching Architecture (PISA) Figure 5 depicts the PISA. It is based on the concept of a programmable match-action pipeline that well matches modern switching hardware. It is a gen-
Ethernet Packet in
IPv input
IPv input
IPv lookup
IPv lookup
IPv out
IPv local
IPv out
IPv local
Figure 4: Data flow graph abstraction: example graph for IPv4 and IPv6 forwarding.
eralization of reconfigurable match-action tables (RMTs) [23] and disaggregated reconfigurable match-action tables (dRMTs) [24].
Match logic (^) ...
Programmable Programmable match-action pipeline parser
Programmable deparser
Match-action unit
Action logic
Match logic
Action logic
Match logic
Action logic
Action logic
Action logic
Match logic
Action logic
Match logic
Action logic
Match logic
Action logic
Action logic
Action logic
Match-action unit
Metadata Metadata Metadata
Figure 5: Protocol-Independent Switch Architecture (PISA).
PISA consists of a programmable parser, a programmable deparser, and a programmable match-action pipeline in between consisting of multiple stages.
- The programmable parser allows programmers to declare arbitrary headers together with a finite state machine that defines the order of the head- ers within packets. It converts the serialized packet headers into a well- structured form.
- The programmable match-action pipeline consists of multiple match-action units. Each unit includes one or more match-action-tables (MATs) to match packets and perform match-specific actions with supplied action data. The bulk of a packet processing algorithm is defined in the form of
The fixed-function components communicate with the programmable ones by generating and/or consuming intrinsic metadata. For example, the ingress port block generates ingress metadata that represents the ingress port number that might be used within the match-action units. To output a packet, the match-action units generates intrinsic metadata that represents an egress port number; this intrinsic metadata is then consumed by the traffic manager and/or egress port block. Figure 6 depicts a typical switch architecture based on PISA. It comprises a programmable ingress and egress pipeline and three fixed-function components: an ingress block, an egress block, and a packet replication engine together with a traffic manager between ingress and egress pipeline.
Programmable ingress pipeline
Packet replication engine +
traffic manager
Ingress^ Egress
Programmable egress pipeline
Fixed-function components
Metadata Metadata Metadata Metadata Metadata Metadata
Figure 6: Exemplary switch architecture based on PISA.
P4 (Programming Protocol-Independent Packet Processors) [5] is the most widely used domain-specific programming language for describing data plane algorithms for PISA. Its initial idea and name were introduced in 2013 [25] and it was published as a research paper in 2014 [5]. Since then, P4 has been further developed and standardized by the P4 Language Consortium [26] that is part of the Open Networking Foundation (ONF) since 2019. The P4 Language Consortium is managed by a technical steering committee and hosts five working groups (WGs). P4 14 [27] was the first standardized version of the language. The current specification is P4 16 [28] which was first introduced in 2016. Other data plane programming languages for PISA are FAST [29], Open- State [30], Domino [31], FlowBlaze [32], Protocol-Oblivious Forwarding [33], and NetKAT [34]. In addition, Broadcom [3] and Xilinx [4] offer vendor-specific programmable data planes based on match-action tables.
2.5. Benefits Data plane programmability entails multiple benefits. In the following, we summarize key benefits.
Data plane programming introduces full flexibility to network packet pro- cessing, i.e., algorithms, protocols, features can be added, modified, or removed by the user. In addition, programmable data planes can be equipped with a user-defined API for control plane programmability and SDN. To keep com- plexity low, only components needed for a particular use case might be included in the code. This improves security and efficiency compared to multi-purpose appliances. In conjunction with suitable hardware platforms, data plane programming allows network equipment designers and even users to experiment with new protocols and design unique applications; both do no longer depend on vendors of specialized packet-processing ASICs to implement custom algorithms. Com- pared to long development circles of new silicon-based solutions, new algorithms can be programmed and deployed in a matter of days. Data plane programming is also beneficial for network equipment developers that can easily create differentiated products despite using the same packet processing ASIC. In addition, they can keep their know-how to themselves without the need to share the details with the ASIC vendor and potentially disclose it to their competitors that will use the same ASIC. So far, modern data plane programs and programming languages have not yet achieved the degree of portability attained by the general-purpose program- ming languages. However, expressing data plane algorithms in a high-level lan- guage has the potential to make telecommunication systems significantly more target-independent. Also, data plane programming does not require but encour- ages full transparency. If the source code is shared, all definitions for protocols and behaviors can be viewed, analyzed, and reasoned about, so that data plane programs benefit from community development and review. As a result, users could choose cost-efficient hardware that is well suited for their purposes and run their algorithms on top of it. This trend has been fueled by SDN and is commonly known as network disaggregation.
- The P4 Programming Language
We give an overview of the P4 programming language. We briefly recap its specification history and describe how P4 programs are deployed. We introduce the P4 processing pipeline and data types. We discuss parsers, match-action controls, and deparsers. Finally, we give an overview of tutorials and guides to P4.
3.1. Specification History
The P4 Language Design Working Group (LDWG) of the P4 Language Con- sortium has standardized so far two distinct standards of P4: P4 14 and P4 16. Table 1 depicts their specification history. The P4 14 programming language dialect allows the programmers to describe data plane algorithms using a combination of familiar, general-purpose imper- ative constructs and more specialized declarative ones that provide support for
tinued, although it is still supported on a number of targets. Therefore, we focus on P4 16 in the remainder of this paper where P4 implicitly stands for P4 16.
3.2. Development and Deployment Process
Figure 8 illustrates the development and deployment process of P4 programs. P4-programmable nodes, so-called P4 targets, are available as software or specialized hardware (see Section 5). They feature packet processing pipelines consisting of both P4-programmable and fixed-function components. The exact structure of these pipelines is target-specific and is described by a corresponding P4 architecture model (see Section 4) which is provided by the manufacturer of the target. P4 programs are supplied by the user and are implemented for a particular P4 architecture model. They define algorithms that will be executed by the P4-programmable components and their interaction with the ones implemented in the fixed-function logic. The composition of the P4 programs and the fixed- function logic constitutes the full data plane algorithm. P4 compilers (see Section 4) are also provided by the manufacturers. They translate P4 programs into target-specific code which is loaded and executed by the P4 target. The P4 compiler also generates a data plane API that can be used by a user-supplied control plane (see Section 6) to manage the runtime behavior of the P4 target.
P4 program (data plane)
Control plane
P4 architecture model
Supplied by the manufacturer^ P4 target
Supplied by the user
Data plane API Code P4 compiler
Figure 8: P4 deployment process according to [28].
3.3. Information Flow
P4 16 adopts PISA’s concept of packet metadata. Figure 9 illustrates the information flow in the P4 processing pipeline. It comprises different blocks, where packet metadata (be it headers, user-defined or intrinsic metadata) is used to pass the information between them, therefore representing a uniform interface. The parser splits up the received packet into individual headers and the remaining payload. Intrinsic metadata from the ingress block, e.g., the ingress port number or the ingress timestamp, is often provided by the hardware and can
be made available for further processing. Many targets allow the user metadata to be initialized in the parser as well. Then, the headers and metadata are passed to the match-action pipeline that consists of one or more match-action units. The remaining payload travels separately and cannot be directly affected by the match-action pipeline processing. While traversing the individual match-action pipeline units, the headers can be added, modified, or removed and additional metadata can be generated. The deparser assembles the packet back by emitting the specified headers followed by the original packet payload. Packet output is configured with in- trinsic metadata that includes information such as a drop flag, desired egress port, queue number, etc.
Parser Deparser
Intrinsic metad.
Headers
Payload
Intrinsic metad.
Intrinsic md.
Headers
P4 block w/ interface
Match- action unit
Match- action unit
P4 block w/ interface
P4 block w/ interface
P4 block w/ interface
Headers
User metad.
User metad.
User metad.
Figure 9: Information flow.
3.4. Data Types
P4 16 is a statically typed language that supports a rich set of data types for data plane programming.
3.4.1. Basic Data Types P4 16 includes common basic types such as Boolean (bool), signed (int), and unsigned (bit) integers which are also known as bit strings. Unlike many common programming languages, the size of these integers is specified at bit granularity, with a wide range of supported widths. For example, types such as bit<1>, int<3>, bit<128> and wider are allowed. In addition, P4 supports bit strings of variable width, represented by a spe- cial varbit type. For example, IPv4 options can be represented as varbit<320> since the size of IPv4 options ranges from zero to 10 32-bit words. P4 16 also supports enumeration types that can be serializable (with the actual representation specified as bit or int during the type definition) or non-serializable, where the type representation is chosen by the compiler and hidden from the user.
Start
Ethernet
MPLS
ethertype= 0x
TCP UDP
protocol=
IPv
ethertype= 0x
default
Custom States
Accept
Reject
default
protocol=
Figure 11: Example for the FSM of a P4 parser that parses packets with Ethernet, MPLS, IPv4, TCP, and UDP headers.
state depends on the value of the EtherType field of the extracted Ethernet header. Based on previously parsed header information, any number of further headers can be extracted from the packet. If the header order does not comply with the expected order, a packet can be discarded by switching to the Reject state. The parser can also implicitly transition into the Reject state in case of a parser exception, e.g., if a packet is too short.
3.6. Match-Action Controls
Match-action controls express the bulk of the packet processing algorithm and resemble traditional imperative programs. They are executed after success- ful parsing of a packet. In some architectures they are also called match-action pipeline units. In the following, we give an overview of control blocks, actions, and match-action tables.
3.6.1. Control Blocks Control blocks, or just controls, are similar to functions in general-purpose languages. They are called by an apply() method. They have parameters and can call also other control blocks. The body of a control block contains the definition of resources, such as tables, actions, and externs that will be used for processing. Furthermore, a single apply() method is defined that expresses the processing algorithm. P4 offers statements to express the program flow within a control block. Unlike common programming languages, P4 does not provide any statements that would allow the programmer to create loops. This ensures that all the
parser SampleParser ( packet_in p , out headers h ) {
state start { transition parse_ethernet ; } state parse_ethernet { p. extract ( h. ethernet ) ; transition select ( h. ethernet. etherType ) { 0 x 8 8 4 7 : parse_mpls ; 0 x 0 8 0 0 : parse_ipv 4 ; default : reject ; }; }
state parse_ipv 4 { p. extract ( h. ipv 4 ) ; transition select ( h. ipv 4. protocol ) { 6 : parse_tcp ; 1 7 : parse_udp ; default : accept ; } }
state parse_udp { p. extract ( h. udp ) ; transition accept ; } /* Other states follow */ }
Figure 12: Sample parser implementation of the FSM in Figure 11.
algorithms that can be coded in P4 can be expressed as directed acyclic graphs (DAGs) and thus are guaranteed to complete within a predictable time interval. Specific control statements include:
- a block statement {} that expresses sequential execution of instructions.
- an if() statement that expresses an execution predicated on a Boolean condition
- a switch() statement that expresses a choice from multiple alternatives
- an exit() statement that ends the control flow within a control block and passes the control to the end of the top-level control
Transformations are performed by several constructs, such as
- An assignment statement which evaluates the expression on its right-hand- side and assigns the result to a header or a metadata fields
