Speed Records in Network Flow Measurement on FPGA

2010

Programmable analysis of network traffic is critical for wide range of higher level traffic engineering and anomaly detection applications. Such applications demand stateful and programmable network traffic measurements (NTM) at high throughputs. We exploit the features and requirements of NTM, and develop an application-specific FPGA based Partial Dynamic Reconfiguration (PDR) scheme that is tailored to NTM problem. PDR has traditionally been done through swapping of statically compiled FPGA configuration data. Besides being latency intensive, static compilation cannot take into account exact requirements as they appear during real-time in many applications like NTM. In this paper, we make novel use of fine-grained PDR, performing minute logic changes in real-time and demonstrate its effectiveness in a prototype solution for programmable and real-time NTM. We specifically make use of flexibility available through the application and present a number of novel tools and algorithms that enabled developing the BURAQ system. Our results show 4x area and 1.3x latency improvements of BURAQ from a comparative statically compiled recent solution.

2019 X Southern Conference on Programmable Logic (SPL), 2019

This paper explores the problem of flow metering in 100 GbE links, presenting a flow exporter architecture based on a FPGA acceleration card using only on-chip memory. Peak performance without packet sampling even at the maximum packet rate is assured and means to avoid data loss are provided, since a low level of aggregation is achieved. This is the first approach in a series of architectures that are built upon the previous one, where the resources of the custom hardware are gradually increased, improving the aggregation level, while the required commodity hardware resources for subsequent stages are consequently lowered. We consider that FPGA-fabric offers adequate flexibility and performance for this task and is capable of reducing overall system cost. A functional prototype of the system has been implemented on the Xilinx VCU118 development board configured to export TCP sessions records. This achievement represents a cornerstone of a 100 GbE FPGA flow exporter design, that aims for supporting in the order of tens of millions concurrent flows. Index Terms-networking, packet processing, TCP flows.

2010

The problem of Network Traffic Classification (NTC) has attracted significant amount of interest in the research community, offering a wide range of solutions at various levels. The core challenge is in addressing high amounts of traffic diversity found in today's networks. The problem becomes more challenging if a quick detection is required as in the case of identifying malicious network behavior or new applications like peer-to-peer traffic that have potential to quickly throttle the network bandwidth or cause significant damage. Recently, Traffic Dispersion Graphs (TDGs) have been introduced as a viable candidate for NTC. The TDGs work by forming a network wide communication graphs that embed characteristic patterns of underlying network applications. However, these patterns need to be quickly evaluated for mounting real-time response against them. This paper addresses these concerns and presents a novel solution for real-time analysis of Traffic Dispersion Metrics (TDMs) in the TDGs. We evaluate the dispersion metrics of interest and present a dedicated solution on an FPGA for their analysis. We also present analytical measures and empirically evaluate operating effectiveness of our design. The mapped design on Virtex-5 device can process 7.4 million packets/second for a TDG comprising of 10k flows at very high accuracies of over 96%.

2013 23rd International Conference on Field programmable Logic and Applications, 2013

In this paper we present an architecture to classify 10 Gbps network traffic based on FPGAs using the NetFPGA platform. Flow-based network monitoring is today the most spread technology employed by engineers and administrators to analyze and detect network issues. The main improvements of our design, compared to the state-of-the-art flow analyzer tools, come from: (i) the architecture allows to process full rate 10 Gbps links without sampling even with the shortest packets (14.88 Mpps-Million packets per second); (ii) it is possible to manage up to 786,432 concurrent flows; (iii) the project is developed in an open-source hardware platform and the code is opened to the community; (iv) it relieves the networks and its devices from the overload of monitoring process.

Lecture Notes in Computer Science, 2009

Counting the number of flows present in network traffic is not trivial, given that the naive approach of using a hash table to track the active flows is too slow for the current backbone network speeds. Several algorithms have been proposed in the recent literature that can calculate an approximate count using small amount of memory and few memory accesses per packet. Fewer works have addressed the more complex problem of counting flows over sliding windows, where the main challenge is to continuously expire old information. One of the existing proposals is a straightforward adaptation of the direct bitmaps technique to the sliding window model. We present an algorithm called Countdown Vector that also builds upon the direct bitmaps technique. Our algorithm, however, obtains significant cost reductions both in terms of memory and CPU, by introducing an extra approximation in the mechanism in charge of the expiration of old information.

It is increasingly difficult for network devices to keep pace with rapid developments in network data rate speeds. Many such devices are unable to match the OC-192 link speed. This paper describes the use of a combined hardware-software system as an application-specific solution to this problem. Our approach maps the performancecritical tasks of packet classification and flow monitoring from software into hardware using a field programmable gate array (FPGA), such that operations can run in parallel where desirable. A feature of our architecture is its capability to process multiple flows in parallel. We explore the scalability of our system showing that it can support flows at multi-gigabit rate, which is faster than most software-based systems where acceptable data rates are typically no more than 100 Mbps.

IEEE Transactions on Computers, 2000

Security of today's networks heavily rely on Network Intrusion Detection Systems (NIDSs). The ability to promptly update the supported rule sets and detect new emerging attacks makes Field Programmable Gate Arrays (FPGAs) a very appealing technology. An important issue is how to scale FPGA-based NIDS implementations to ever faster network links. Whereas a trivial approach is to balance traffic over multiple, but functionally equivalent, hardware blocks, each implementing the whole rule set (several thousands rules), the obvious cons is the linear increase in the resource occupation. In this work, we promote a different, traffic-aware, modular approach in the design of FPGA-based NIDS. Instead of purely splitting traffic across equivalent modules, we classify and group homogeneous traffic, and dispatch it to differently capable hardware blocks, each supporting a (smaller) rule set tailored to the specific traffic category. We implement and validate our approach using the rule set of the well known Snort NIDS, and we experimentally investigate the emerging trade-offs and advantages, showing resource savings up to 80% based on real world traffic statistics gathered from an operator's backbone.

2017 International Conference on ReConFigurable Computing and FPGAs (ReConFig), 2017

In network traffic monitoring, a very important analysis is to find heavy hitters. That is, finding those flows that use most resources in a given network link. This information can be very useful for security or traffic management purposes. Though this analysis might seem easy to implement, since it is essentially based on counting, the fact is that doing it at 100 Gbit/s rates is far from trivial. In 100 Gbit/s Ethernet (100 GbE), up to 148 million packets per second can be received, thus making it very difficult to parse packets and maintain counters at such rate. In this paper, we leverage the integrated 100G Ethernet Subsystem available in Xilinx UltraScale devices to implement a heavy hitter detector for 100 GbE in a VCU108 evaluation kit. Thanks to the integration of the Count Sketch algorithm with a priority list and a network packet parser, the proposed architecture is able to work at line rate for average packet sizes bigger than 215 bytes. The work presents a theoretical analysis of the error, as well as the technical details of the proposed solution. The implementation has been validated using real-world traces, obtaining an average error of 1.29%.

2014

Streaming network traffic measurement and analysis is critical for detecting and preventing any real-time anomalies in the network. The high speeds and complexity of today's networks, coupled with ever evolving threats, necessitate closing of the loop between measurements and their analysis in real time. The ensuing system demands high levels of programmability and processing where streaming measurements adapt to the changing network behavior in a goal-oriented manner. In this work, we exploit the features and requirements of the problem and develop an application-specific FPGA-based closed-loop measurement (CLM) system. We make novel use of fine-grained partial dynamic reconfiguration (PDR) as underlying reprogramming paradigm, performing low-latency just-in-time compiled logic changes in FPGA fabric corresponding to the dynamic measurement requirements. Our innovative dynamically reconfigurable socket offers 3Â logic savings over conventional static solutions, while offering much reduced reconfiguration latencies over conventional PDR mechanisms. We integrate multiple sockets in a highly parallel CLM framework and demonstrate its effectiveness in identifying heavy flows in streaming network traffic. The results using an FPGA prototype offer 100 percent detection accuracy while sustaining increasing link speeds.

Lecture Notes in Computer Science, 2004

Field Programmable Gate Arrays (FPGAs) can be used in Intrusion Prevention Systems (IPS) to inspect application data contained within network flows. An IPS operating on high-speed network traffic can be used to stop the propagation of Internet worms and to protect networks from Denial of Services (DoS) attacks. When used in the backbone of a core network, the device will be exposed to millions of active flows simultaneously. In order to protect the data in each connection, network devices will need to track the state of every flow. This must be done at multi-gigabit line rates without introducing significant delays. This paper describes a high performance TCP processing system called TCP-Processor which supports flow processing in high-speed networks utilizing multiple devices. This circuit provides stateful flow tracking, TCP stream reassembly, context storage, and flow manipulation services for applications which process TCP data streams. A simple client interface eases the complexities associated with processing TCP data streams. In addition, a set of encoding and decoding circuits has been developed which efficiently transports this interface between multiple FPGA devices. The circuit has been implemented in FPGA hardware and tested using live Internet traffic.