Networks, Edge
& SysTems Lab

The blistering pace of AI innovation demands a shift toward agile benchmarking methodologies, specifically targeting architectural simulators and emulators to guide early-stage hardware-software co-design. While workload standardization efforts like Chakra provide schemas for representing execution graphs and dependencies, they lack fine-grained memory-trace information. Such detailed traces are foundational for evaluating emerging distributed infrastructures, including GPU memory pooling and memory-semantic fabrics. Existing memory tracing tools are either too coarse-grained, highly invasive to workload behavior, dependent on source-code modification, or fail to scale. To bridge this gap, we introduce an execution trace generator that efficiently captures, represents, and replays the fine-grained memory operation behavior of large-scale AI/ML workloads. We demonstrate the temporal distribution and dominance of memory operation types, memory hierarchies, and stall reasons for nano-GPT-10M with data parallelism across two H100 GPUs, which can be used to generate memory benchmarks.

Telecom operators and military/defense applications (e.g., UAVs) prefer to offload compute-intensive cryptographic processing to FPGA-based cryptographic accelerators over fixed-function ASICs for reconfigurability and short deployment cycles. The state-of-the-art FPGA-based cryptographic accelerators are designed either for high throughput or power efficiency. We propose FlexMesh, a programmable, dynamically reconfigurable, and size-aware framework for 5G/6G cryptographic primitives. We design variable-sized FPGA-based cryptographic cores by varying the degree of parallelism, and demonstrate that if the cryptographic core is chosen wisely, we can achieve performance without trading resource or power efficiency for certain use cases. FlexMesh dynamically orchestrates multiple asymmetric cryptographic cores (i.e., variable-sized cores) based on the observed request size distribution. To efficiently utilize the deployed cryptographic cores, we design a workload-aware, dynamically configurable load balancer and a scheduler. We test our system over realistic 5G datasets, and our results demonstrate that asymmetric cryptographic cores reduce the request processing latency and improve resource and power efficiencies, compared to the baseline symmetric cryptographic cores.

The growing demand for large-scale GPU clusters to train large language models (LLMs) poses a significant challenge to innovation due to high costs and limited accessibility. While state-of-the-art simulators address this issue, they assume a uniform infrastructure. However, device heterogeneity is unavoidable in cloud environments due to resource sharing, frequent updates in device generations, and the inherent intra-chip interconnect heterogeneity. We propose a heterogeneity-aware simulator for distributed LLM training that takes into account the real-world compute and network heterogeneity. Our simulator allows for custom configurations and models the impact of hardware diversity on training time.

In-network traffic control systems built on programmable data planes enhance network performance. However, these systems also increase the attack surface and are vulnerable to attacks not seen before. We focus on a problem that stems from the fact that a programmable switch data plane trusts and processes the messages from upper layers in the switch software (OS, SDK, drivers) and from neighbor nodes in the network. Since these messages can update the state maintained in the data plane, which can influence traffic control decisions, it is important to protect such messages from adversaries aiming to degrade performance, compromise privacy, bypass security, or, worst case, network outage.In this paper, we present P4Auth, a key-based protection mechanism that ensures the authenticity and integrity of such messages in in-network systems making fast traffic control decisions. Our key idea is to move key-based security primitives to the switch data plane so that it reduces the trusted computing base and exposure to switch software vulnerabilities while enabling faster checks in the data plane. To realize this idea, we design and develop an authentication protocol, secure key exchange mechanism, and associated data plane primitives. We prototype P4Auth for Intel Tofino and understand the overheads of P4Auth. We also demonstrate how P4Auth protects two in-network systems from man-in-the-middle (MitM) adversaries.

Network management systems built on programmable data planes enhance network performance, security, and reliability. However, these systems also increase the attack surface and, hence, are vulnerable to attacks not seen before. We focus on a problem that stems from the fact that a switch data plane trusts the control messages (e.g., table update messages) from upper layers in the switch control plane (OS, SDK, drivers). Since these control messages define the packet forwarding behavior, unauthorized modification to the messages by an adversary at any of these layers can lead to poor performance, privacy compromise, security bypass, and network outage. In this paper, we present P4TVal, a P4 Table rule Validation system that detects unauthorized updates to table rules. Our key idea is early detection, where a table rule update is promptly followed by an authenticated rule validation. To realize this idea, we craft and send custom packets to hit specific table rules in the data plane and validate the table rules actually applied to them. We prototype P4TVal for Intel Tofino and BMV2 and demonstrate how P4TVal can detect unauthorized modifications to table rules. Our evaluation shows that P4TVal's early detection reduces the detection time by 10X compared to the state-of-the-art approach. Index Terms—programmable data plane security, securing table update, detecting forwarding table manipulation attack.

Small-scale public cloud providers cannot adopt serialization datacopy optimizations that leverage hardware acceleration or kernelbypass zero-copy paradigm (e.g., DPDK) because of (1) accelerator deployment costs, (2) securing kernel-bypass zero-copy serialization communication relies on encryption at the NIC hardware, and (3) kernel-bypass solutions require developers to rewrite the business logic. We propose a serialization library that leverages the kernel scatter-gather communication primitives to reduce serialization data copy costs without using custom hardware or a custom network stack.

On-premise edge cloud provides opportunities to enable ML-based latency-critical services to resource-constrained end devices. The edge services are deployed as loosely coupled microservices using cloud orchestrators like Kubernetes, and a load balancer distributes requests from an upstream microservice instance (client) across many downstream microservice instances (servers). However, in a shared environment, transient and sporadic delay events are common due to contention for host and network resources (e.g., high load on servers, high network queuing delays). To meet low latency requirements of edge services, the load balancer should quickly adapt to such delay events and adjust routing decisions (e.g., pick the best downstream instance among all). In this paper, we propose DL3, a distributed load balancer (LB) that quickly adapts to server load and network queuing delays by adjusting routing decisions so that the requests are forwarded to the best possible servers. The key idea is to enable LB with visibility into both servers’ load and transient delays on network paths toward the servers. We prototype DL3 on a Kubernetes-managed edge cloud cluster and evaluated its performance for a latency-sensitive ML-based object detection service. Our preliminary results show that DL3 improves tail response time by 33% compared to the state-of-the-art load balance mechanism.

Bufferbloat is a major issue in computer networks, caused by network devices such as switches and routers with large buffers, resulting in congested queues and high network delays. Active Queue Management (AQM) algorithms like RED, CoDel, and PIE have been developed to address Bufferbloat. Among these, CoDel is the most effective as it does not require parameter tuning. However, software-based AQM solutions cannot scale for high-speed networks, while traditional hardware switches are fixed-function. Programmable switch ASICs offer more flexibility but impose programming constraints to achieve line rates. Due to these limitations, previous research has been unsuccessful in implementing an RFC-compliant CoDel AQM on programmable switch ASICs. To solve this challenge, we have introduced CoDel-ACT, a redesign of CoDel that can handle the global state across switch pipeline stages, improves the accuracy of the math function, and fully implements CoDel within the data plane of an Intel Tofino switch. Our evaluations demonstrate that CoDel-ACT solves Bufferbloat, has negligible overheads, and reduces the average packet queue delay by 52% compared to the existing solution.

Microservice architectures [25] provide benefits such as independent scaling of bottleneck functions, code reusability, and ease of management. Inspired by these benefits, we propose to migrate from monolithic architecture to microservice security architecture. In this work, we propose to design and implement an in-network, microservice-based security framework for offloaded applications by leveraging FPGA NICs. Our goal is to decompose the TLS cipher suite and 4G/5G security algorithms into independent crypto functions (i.e., microservices). We aim to design a programmable system that supports near line-rate processing for signaling and data messages. It should judiciously utilize the FPGA hardware to maximally pack crypto algorithms and support the autoscaling of crypto microservices based on the incoming traffic characteristics.

Network management tasks heavily rely on network telemetry data. Programmable data planes provide novel ways to collect this telemetry data efficiently using probabilistic data structures like bloom filters and their variants. Despite the benefits of the data structures (and associated data plane primitives), their exposure increases the attack surface. That is, they are at risk of adversarial network inputs. In this work, we examine the effects of adversarial network inputs to bloom filters that are integral to data plane primitives. Bloom filters are probabilistic and inherently susceptible to pollution attacks which increase their false positive rates. To quantify the impact, we demonstrate the feasibility of pollution attacks on FlowRadar, a network monitoring and debugging system that employs a data plane primitive to collect traffic statistics. We observe that the adversary can corrupt traffic statistics with a few well-crafted malicious flows (tens of flows), leading to a 99% drop in the accuracy of the core functionality of the FlowRadar system.

The latest generation of 5G telecommunication networks are expected to provide high throughput and low latency while catering to diverse applications like mobile broadband, dense IoT, and self-driving cars. A high performance User Plane Function (UPF), the main element in the 5G user plane, is critical to achieving these performance goals. This paper presents AccelUPF, a 5G UPF that offloads functionality to programmable dataplane hardware for performance acceleration. While prior work has proposed accelerating the UPF by offloading its data forwarding functionality to programmable hardware, the Packet Forwarding Control Protocol (PFCP) messages from the control plane that configure the hardware data forwarding rules were still processed in software. We show that only offloading data forwarding and not PFCP message processing leads to suboptimal performance in the UPF for applications like IoT that have a much higher ratio of PFCP messages to data traffic, due to a bottleneck at the software control plane that configures the hardware packet forwarding rules. In contrast to prior work, AccelUPF offloads both PFCP message processing as well as data forwarding to programmable hardware. AccelUPF overcomes several technical challenges pertaining to the processing of the complex variable-sized PFCP messages within the memory and compute constraints of programmable hardware platforms. Our evaluation of AccelUPF implemented over a Netronome programmable NIC and an Intel Tofino programmable switch demonstrates performance gains over the state-of-the-art UPFs for real-world traffic scenarios.

Control and management plane applications such as serverless function orchestration and 4G/5G control plane functions are offloaded to smartNICs to reduce communication and processing latency. Such applications involve multiple inter-host interactions that were traditionally secured using SSL/TLS gRPC-based communication channels. Offloading the applications to smartNIC implies that we must also offload the security algorithms. Otherwise, we need to send the application messages to the host VM/container for crypto operations, negating offload benefits. We propose crypto externs for Netronome Agilio smartNICs that implement authentication and confidentiality (encryption/decryption) using the ChaCha stream cipher algorithm. AES and ChaCha are two popular cipher suites, but we chose ChaCha since none of the smartNICs have ChaCha-based crypto accelerators. However, smartNICs have restricted instruction set, and limited memory, making it difficult to implement security algorithms. This paper identifies and addresses several challenges to implement ChaCha crypto primitives successfully. Our evaluations show that our crypto extern implementation satisfies the scalability requirement of popular applications such as serverless management functions and host in-band network telemetry.

Emerging 5G applications require a dataplane that has a high forwarding throughput and low processing latency, in addition to low cost and power consumption. To meet these requirements, the state-of-the-art 5G User Plane Functions (UPFs) are built over high performance packet I/O mechanisms like the Data Plane Development Kit (DPDK), and further offload some functionality to programmable dataplane hardware. In this paper, we design and implement several standards-compliant UPF prototypes, beginning with a software-only DPDK-based UPF, progressing to designs which offload different functions to programmable hardware. We evaluate and compare the performance of these designs, to highlight the costs and benefits of these offloads. Our results show that offload techniques employed in prior work help improve performance in certain scenarios, but also have their limitations. Overcoming these limitations and fully realizing the power of programmable hardware requires offloading more complex functionality than is done today. Our work presents a preliminary implementation towards a comprehensive programmable dataplane-accelerated 5G UPF.

Recent architectures of the mobile packet core advocate the separation of the control and dataplane components, with all signaling messages being processed by the control plane entities. This paper presents the design, implementation, and evaluation of TurboEPC, a redesign of the mobile packet core that revisits the division of work between the control and data planes. In TurboEPC, the control plane offloads a small amount of user state to programmable dataplane switches, using which the switches can correctly process a subset of signaling messages within the dataplane itself. The messages that are offloaded to the dataplane in TurboEPC constitute a significant fraction of the total signaling traffic in the packet core, and handling these messages on dataplane switches closer to the end-user improves both control plane processing throughput and latency. We implemented the TurboEPC design using P4-based software and hardware switches. The TurboEPC hardware prototype shows throughput and latency improvements by up to 102x and 98% respectively when the switch hardware stores the state of 65K concurrent users, and 22× and 97% respectively when the switch CPU is busy forwarding dataplane traffic at linerate, over the traditional EPC.