Range Matching

Searching of patterns in large data sets is need of the hour to extract knowledge from data warehouses. This paper presents a new hashing based algorithm for fast search of multiple variable length patterns in large data sets. It rules out traditional way of generation of shift table for each character present in pattern. It can also accommodate patterns which come up during search time, thus works well for both predetermined as well as dynamic pattern set. Furthermore, its speed enhances as the minimum pattern length P increases for data set of length n taking O(n/P) time during search. Experimental results for runtime behavior of presented algorithm with varying parameters like number of patterns to be searched and length of data set extended upto (but not limited to) 200,000 characters are produced.

We present range encoding with no expansion (RENÉ)-a novel encoding scheme for short ranges on Ternary content addressable memory (TCAM), which, unlike previous solutions, does not impose row expansion, and uses bits proportionally to the maximal range length. We provide theoretical analysis to show that our encoding is the closest to the lower bound of number of bits used. In addition, we show several applications of our technique in the field of packet classification, and also, how the same technique could be used to efficiently solve other hard problems, such as the nearest-neighbor search problem and its variants. We show that using TCAM, one could solve such problems in much higher rates than previously suggested solutions, and outperform known lower bounds in traditional memory models. We show by experiments that the translation process of RENÉ on switch hardware induces only a negligible 2.5% latency overhead. Our nearest neighbor implementation on a TCAM device provides search rates that are up to four orders of magnitude higher than previous best prior-art solutions.

Packet classification has been a critical data path function for many emerging networking applications. An interesting approach is the use of Ternary Content Addressable Memory (TCAM) to achieve deterministic, high-speed packet classification performance. However, apart from high cost and power consumption, due to slow growing clock rate for memory technology, in general, the traditional single TCAM-based solution has difficulty to keep up with fast growing line rates. Moreover, the TCAM storage efficiency is largely affected by the need to support rules with ranges or range matching. In this paper, a distributed TCAM scheme that exploits chip-level-parallelism is proposed to greatly improve the throughput performance. This scheme seamlessly integrates with a range encoding scheme which not only solves the range matching problem, but also ensures a balanced high throughput performance. A thorough theoretical worst-case analysis of throughput, processing delay, and power consumption, as well as the experimental results show that the proposed solution can achieve scalable throughput performance matching up to OC768 line rate or higher. The added TCAM storage overhead is found to be reasonably small for the five real-world classifiers studied.

Packet classification has been a critical data path function for many emerging networking applications. An interesting approach is the use of Ternary Content Addressable Memory (TCAM) to achieve deterministic, high-speed packet classification performance. However, apart from high cost and power consumption, due to slow growing clock rate for memory technology, in general, the traditional single TCAM-based solution has difficulty to keep up with fast growing line rates. Moreover, the TCAM storage efficiency is largely affected by the need to support rules with ranges or range matching. In this paper, a distributed TCAM scheme that exploits chip-level-parallelism is proposed to greatly improve the throughput performance. This scheme seamlessly integrates with a range encoding scheme which not only solves the range matching problem, but also ensures a balanced high throughput performance. A thorough theoretical worst-case analysis of throughput, processing delay, and power consumption, as well as the experimental results show that the proposed solution can achieve scalable throughput performance matching up to OC768 line rate or higher. The added TCAM storage overhead is found to be reasonably small for the five real-world classifiers studied.

One of the most critical resource management issues in the use of ternary content-addressable memory (TCAM) for packet classification/filtering is how to effectively support filtering rules with ranges, known as range matching. In this paper, the Dynamic Range Encoding Scheme (DRES) is proposed to significantly improve the TCAM storage efficiency for range matching. Unlike the existing range encoding schemes requiring additional hardware support, DRES uses the TCAM coprocessor itself to assist range encoding. Hence, DRES can be readily programmed in a network processor using a TCAM coprocessor for packet classification. A salient feature of DRES is its ability to allow a subset of ranges to be encoded and, hence, to have full control over the range code size. This advantage allows DRES to exploit the TCAM structure to maximize the TCAM storage efficiency. DRES is a comprehensive solution, including a dynamic range selection algorithm, a search key encoding scheme, a range encoding scheme, and a dynamic encoded range update algorithm. Although the dynamic range selection algorithm running in the software allows optimal selection of ranges to be encoded to fully utilize the TCAM storage, the dynamic encoded range update algorithm allows the TCAM database to be updated lock free without interrupting the TCAM database lookup process. DRES is evaluated based on real-world databases and the results show that DRES can reduce the TCAM storage expansion ratio from 6.20 to 1.23. The performance analysis of DRES based on a probabilistic model demonstrates that DRES significantly improves the TCAM storage efficiency for a wide spectrum of range distributions.

Last, but not least, I would like to thank my husband and partner Danny for his unending support. His endless love and encouragement make it possible for me to carry out this project. This work has received sponsorship from Marvell Semiconductor. Any opinions, findings, recommendations or conclusions expressed in this thesis are those of the author and do not necessarily reflect the views of the sponsors.

Using packet classification algorithms in network equipment increases packet processing speed in Internet of Things (IoT). In the hardware implementation of these algorithms, ternary content-addressable memories (TCAMs) are often preferred to other implementations. As a common approach, TCAMs are used for the parallel search to match packet header information with the rules of the classifier. In two-stage architectures of hardware-based packet classifiers, first the decision tree is created, and then the rules are distributed among its leaves. In the second step, depending on the corresponding leaves, the second part of the rules, which includes the range of source and destination ports is stored in different blocks of TCAM. Due to inappropriate storage of port range fields, the existing architectures face the problem of wasting memory and growing power consumption. This paper proposes an efficient algorithm to encode the port range. This algorithm consists of three general steps including layering, bit allocation, and encoding. A greedy algorithm in the first step places the ranges with higher weights in higher layers. Next, an auction-based algorithm allocates several bits to each layer depending on the number of the ranges in that layer. Finally, in each layer, depending on the weight order of the ranges, the bits are given values for the intended range. The evaluation results show that unlike previous methods of storing range fields, the proposed method not only increases the speed of the classification but also uses the capacity of TCAM in the second stage more efficiently.

Ternary Content-Addressable Memory (TCAM) is a powerful tool to represent network services with line-rate lookup time. There are various software-based approaches to represent multi-field packet classifiers. Unfortunately, all of them either require exponential memory or apply additional constraints on field representations (e.g, prefixes or exact values) to have line-rate lookup time. In this work, we propose alternatives to TCAM and introduce a novel approach to represent packet classifiers based on ternary bit strings (without constraining field representation) on commodity longest-prefix-match (LPM) infrastructures. These representations are built on a novel property, prefix reorderability, that defines how to transform an ordered set of ternary bit strings to prefixes with LPM priorities in linear memory. Our results are supported by evaluations on large-scale packet classifiers with real parameters from ClassBench; moreover, we have developed a prototype in P4 to support these types of transformations.

Using packet classification algorithms in network equipment increases packet processing speed in Internet of Things (IoT). In the hardware implementation of these algorithms, ternary content-addressable memories (TCAMs) are often preferred to other implementations. As a common approach, TCAMs are used for the parallel search to match packet header information with the rules of the classifier. In two-stage architectures of hardware-based packet classifiers, first the decision tree is created, and then the rules are distributed among its leaves. In the second step, depending on the corresponding leaves, the second part of the rules, which includes the range of source and destination ports is stored in different blocks of TCAM. Due to inappropriate storage of port range fields, the existing architectures face the problem of wasting memory and growing power consumption. This paper proposes an efficient algorithm to encode the port range. This algorithm consists of three general steps including layering, bit allocation, and encoding. A greedy algorithm in the first step places the ranges with higher weights in higher layers. Next, an auction-based algorithm allocates several bits to each layer depending on the number of the ranges in that layer. Finally, in each layer, depending on the weight order of the ranges, the bits are given values for the intended range. The evaluation results show that unlike previous methods of storing range fields, the proposed method not only increases the speed of the classification but also uses the capacity of TCAM in the second stage more efficiently.

Packet classification methods rely upon packet content/header matching against rules. Thus, throughput of matching operations is critical in many networking applications. Further, with the advent of Software Defined Networking (SDN), efficient implementation of software approaches to matching are critical for the overall system performance. This article presents 1 GenMatcher, a generic, software-only, arbitrary matching framework for fast, efficient searches. The key idea of our approach is to represent arbitrary rules with efficient prefix-based tries. To support arbitrary wildcards, we rearrange bits within the rules such that wildcards accumulate to one side of the bitstring. Since many non-contiguous wildcards often remain, we use multiple prefix-based tries. The main challenge in this context is to generate efficient trie groupings and expansions to support all arbitrary rules. Finding an optimal mix of grouping and expansion is an NP-complete problem. Our contribution includes a novel, clustering-based grouping algorithm to group rules based upon their bit-level similarities. Our algorithm generates near-optimal trie groupings with low configuration times and provides significantly higher match throughput compared to prior techniques. Experiments with synthetic traffic show that our method can achieve a 58.9X speedup compared to the baseline on a single core processor under a given memory constraint.

Packet classification has wide applications such as unauthorized access prevention in firewalls and Quality of Service supported in Internet routers. The classifier containing pre-defined rules is processed by the router for finding the best matching rule for each incoming packet and for taking appropriate actions. Although many software-based solutions had been proposed, high search speed required for Internet backbone routers is not easy to achieve. To accelerate the packet classification, the state-of-the-art ternary content-addressable memory (TCAM) is a promising solution. In this paper, we propose an efficient multi-field range encoding scheme to solve the problem of storing ranges in TCAM and to decrease TCAM usage. Existing range encoding schemes are usually single-field schemes that perform range encoding processes in the range fields independently. Our performance experiments on real-life classifiers show that the proposed multi-field range encoding scheme uses less TCAM memory than the existing single field schemes. Compared with existing notable single-field encoding schemes, the proposed scheme uses 12%-33% of TCAM memory needed in DRIPE or SRGE and 56%-86% of TCAM memory needed in PPC for the classifiers of up to 10k rules.

In traffic monitoring, accounting, and network anomaly detection, it is often important to be able to detect high-volume traffic clusters in near real-time. Such heavy-hitter traffic clusters are often hierarchical (i.e., they may occur at different aggregation levels like ranges of IP addresses) and possibly multidimensional (i.e., they may involve the combination of different IP header fields like IP addresses, port numbers, and protocol). Without prior knowledge about the precise structures of such traffic clusters, a naive approach would require the monitoring system to examine all possible combinations of aggregates in order to detect the heavy hitters, which can be prohibitive in terms of computation resources.

Ternary content-addressable memories (TCAMs) are increasingly used for high-speed packet classification. TCAMs compare packet headers against all rules in a classification database in parallel and thus provide high throughput unparalleled by software-based solutions. TCAMs are not well-suited, however, for representing rules that contain range fields. Such rules typically have to be represented (or encoded) by multiple TCAM entries. The resulting range expansion can dramatically reduce TCAM utilization.

Network Firewalls are considered to be one of the most important security components in today's IP network architectures. Performance of firewalls has significant impact on the overall network performance.  Firewalls should be able to sustain a very high throughput and ensure network services availability. In this paper, we propose an analytical dynamic multilevel early packet filtering mechanism to enhance firewall performance. The proposed mechanism uses statistical splay tree filters that utilize traffic characteristics to minimize packet filtering time. The statistical splay tree filters are reordered according to the network traffic divergence upon certain threshold qualification (Chi-Square Test). That is, the proposed mechanism is able to decide whether or not there is a need to update the dynamic splay tree filters’ order for filtering the next net-work traffic window and predict the best order pattern. Furthermore, the im-portance of optimizing packet rejection and acceptance is done through the mul-tilevel packet filtering process; where in each level, unwanted packets are re-jected as early as possible. The proposed mechanism can also be considered as a device protection mechanism against denial of service (DoS) attacks targeting the default filtering rule. Early packet acceptance is done using the splay tree data structure which adapts dynamically according to network traffic flows. Consequently, repeated packets will have less memory accesses and therefore reduce the overall packets filtering time as demonstrated in the evaluation section.

The coexistence of range based and prefix based fields within the filtering rules is one of the most important cause that makes the packet classification problem difficult to resolve and the proposed hybrid solutions hard to implement. How to effectively support such complex filtering rules is really a challenge. Most of the cases range-based fields need to be converted into a set of standard prefixes. Actually, there is a manifested need to have new expressive conversion techniques to process efficiently multiple type of conditions. In this context, the problem of limited scalability is encountered and must be resolved to avoid the range to prefix blowout. In this paper, we propose the NAF conversion technique (Non-adjacent form) which is able to expand an arbitrary range or multiple ranges into an optimal set of signed prefixes. The proposed technique is flexible enough and let us the possibility to reach a better compression ratio than the previous proposed solutions.

The coexistence of range-based and prefix-based fields within the filtering policy is one of the most important causes that make the packet filtering problem difficult to solve and the proposed hybrid solutions hard to implement. In general, a packet filter must support rule sets involving any conditions and it must be able to scale the number of rules, the number of fields, and the field sizes that it supports in order to avoid being outdated by future Internet developments. Since the prefix-based solutions are the most efficient in practice, we try to efficiently incorporate ranges in such data structures using the new concept of signed prefixes that helps to guarantee homogeneity when matching on multiple packet header fields of distinct types. The proposed two-staged prefix-based model is able to achieve good performance in a practical environment and it scales well as the filtering list size increases and contains a large variety of range specifications. The proposed packet filtering model gives a worst case time complexity of Oððlog 2 ðwÞÞ 2 Þ and a worst case space complexity of OðNwlog 2 ðwÞÞ in the case of performing a binary search on each stage with N the size of the filtering table and w the size of packet header field to be inspected.

The coexistence of range-based and prefix-based fields within the filtering policy is one of the most important causes that make the packet filtering problem difficult to solve and the proposed hybrid solutions hard to implement. In general, a packet filter must support rule sets involving any conditions and it must be able to scale the number of rules, the number of fields, and the field sizes that it supports in order to avoid being outdated by future Internet developments. Since the prefix-based solutions are the most efficient in practice, we try to efficiently incorporate ranges in such data structures using the new concept of signed prefixes that helps to guarantee homogeneity when matching on multiple packet header fields of distinct types. The proposed two-staged prefix-based model is able to achieve good performance in a practical environment and it scales well as the filtering list size increases and contains a large variety of range specifications. The proposed packet filtering model gives a worst case time complexity of Oððlog 2 ðwÞÞ 2 Þ and a worst case space complexity of OðNwlog 2 ðwÞÞ in the case of performing a binary search on each stage with N the size of the filtering table and w the size of packet header field to be inspected.

Many researches are about optimizing schemes for packet classification and matching filters to increase the performance of many network devices such as firewalls and QoS routers. Most of the proposed algorithms do not process dynamically the packets and give no specific interest in the skewness of the traffic. In this paper, we conceive a set of selfadjusting tree filters by combining the scheme of binary search on prefix length with the splay tree model. Hence, we have at most 2 hash accesses per filter for consecutive values. Our proposed filter is adapted to easily assure exact matching for protocol field, prefix matching for IP addresses, and range matching for port numbers. Also, we use the splaying technique to optimize the early rejection of unwanted flows, which is important for many filtering devices such as firewalls.