Academia.eduAcademia.edu

Figure 25 – uploaded by paul nick

See full PDFdownloadDownload figure
Fig. 3.16 Level rings of SkipNet and the routing table

Figure 3 16 Level rings of SkipNet and the routing table

Related Figures (102)

Fig. 1.1 Peer-to-peer computing. Peer A requests for some data that Peer B has. However, Peer A has to first locate Peer B through other peers in the P2P network. Once Peer B is located, Peer A deals directly with Peer B
Fig. 1.1 Peer-to-peer computing. Peer A requests for some data that Peer B has. However, Peer A has to first locate Peer B through other peers in the P2P network. Once Peer B is located, Peer A deals directly with Peer B
We begin by looking at a taxonomy of P2P systems. This taxonomy is derived from examining existing P2P systems. Figure 2.1 shows the taxonomy. In general, we can categorize the systems into two broad categories, centralized vs. decentralized, based on the availability of one or more servers, and to what extent the peers depend on the services provided by those servers. As expected, most of the research focuses on decentralized systems. There are essentially two main design issues to consider in decentralized systems: (a) the structure—flat (single tier) vs. hierarchical (multi- tier); and (b) the overlay topology—unstructured vs. structured. Besides these two main categories, there are also hybrid P2P systems that combine both centralized and decentralized architectures to leverage the advantages of both architectures. We shall examine each of these issues here. shall examine each of these issues here.
We begin by looking at a taxonomy of P2P systems. This taxonomy is derived from examining existing P2P systems. Figure 2.1 shows the taxonomy. In general, we can categorize the systems into two broad categories, centralized vs. decentralized, based on the availability of one or more servers, and to what extent the peers depend on the services provided by those servers. As expected, most of the research focuses on decentralized systems. There are essentially two main design issues to consider in decentralized systems: (a) the structure—flat (single tier) vs. hierarchical (multi- tier); and (b) the overlay topology—unstructured vs. structured. Besides these two main categories, there are also hybrid P2P systems that combine both centralized and decentralized architectures to leverage the advantages of both architectures. We shall examine each of these issues here. shall examine each of these issues here.
Bdetiy speaking, a centralized P2P system enjoys two merits: (a) It speeds up the process of resource location, and guarantees finding all possible nodes that maintain the desired files; (b) It is easy to maintain, organize, and administer the whole system through the central server. However, the central server may become the bottleneck of the system’s scalability. Even worse, it may result in a single point of failure. Therefore, improving the scalability, robustness, and security of these systems is an important research issue. In the next two subsections, we present Napster and
Bdetiy speaking, a centralized P2P system enjoys two merits: (a) It speeds up the process of resource location, and guarantees finding all possible nodes that maintain the desired files; (b) It is easy to maintain, organize, and administer the whole system through the central server. However, the central server may become the bottleneck of the system’s scalability. Even worse, it may result in a single point of failure. Therefore, improving the scalability, robustness, and security of these systems is an important research issue. In the next two subsections, we present Napster and
Se ee ee ee ES DS a ae a eo ee ee ee a oe The key idea behind of the Canonical approach is that it uses recursive rout- ng structure to construct a hierarchical DHT. For example, Fig. 2.5 shows a part of hierarchical structure of National University of Singapore. In the Canonical ap- oroach, all nodes in any domain are interconnected with each other to form a flat DHT routing structure. Notice that “nodes in domain X” refers to all nodes in the subtree rooted at X. However, different from any flat DHT structure, the Canonical upproach ensures the nodes located in different sub-domains, will be merged into 1 new, high-level DHT structure in the current domain by adding some links from -ach node in one domain to some nodes in other domains. For example, there are hree research groups in the SoC domain, i.e., DB, IS, and DS, and all computers of each sub-domain forms a Chord ring. Similarly, in SoC domain, all nodes of DB, [S, and DS domains form a new, high-level Chord ring. In this way, all participating 10des will form a hierarchical DHT according to the hierarchical structure of their Jomains.
Se ee ee ee ES DS a ae a eo ee ee ee a oe The key idea behind of the Canonical approach is that it uses recursive rout- ng structure to construct a hierarchical DHT. For example, Fig. 2.5 shows a part of hierarchical structure of National University of Singapore. In the Canonical ap- oroach, all nodes in any domain are interconnected with each other to form a flat DHT routing structure. Notice that “nodes in domain X” refers to all nodes in the subtree rooted at X. However, different from any flat DHT structure, the Canonical upproach ensures the nodes located in different sub-domains, will be merged into 1 new, high-level DHT structure in the current domain by adding some links from -ach node in one domain to some nodes in other domains. For example, there are hree research groups in the SoC domain, i.e., DB, IS, and DS, and all computers of each sub-domain forms a Chord ring. Similarly, in SoC domain, all nodes of DB, [S, and DS domains form a new, high-level Chord ring. In this way, all participating 10des will form a hierarchical DHT according to the hierarchical structure of their Jomains.
Fig. 2.6 Merging Chord rings of DB and IS into Chord ring of SoC
Fig. 2.6 Merging Chord rings of DB and IS into Chord ring of SoC
Fig. 2.7 A Skip List with 5 nodes. The arrow lines show the process of search node 41 from head, through node 26 and node 30, to node 41
Fig. 2.7 A Skip List with 5 nodes. The arrow lines show the process of search node 41 from head, through node 26 and node 30, to node 41
Fig. 2.8 A skip graph with 7 nodes, where level 0, 1, 2, and 3 contain | list, 2 lists, 4 lists, and 7 lists, respectively. For example, in level 1, there are 2 lists and their membership vectors are m(x) = 0 (node 7, node 23, node 30, and node 57) and m(x) = 1 (node 12, node 26, and node 41)
Fig. 2.8 A skip graph with 7 nodes, where level 0, 1, 2, and 3 contain | list, 2 lists, 4 lists, and 7 lists, respectively. For example, in level 1, there are 2 lists and their membership vectors are m(x) = 0 (node 7, node 23, node 30, and node 57) and m(x) = 1 (node 12, node 26, and node 41)
the query result, the information of the key k is also cached (stored) in the routing tables of Peer 1 and Peer 3. 3.2.2 Heuristic-Based Routing Strategies OI COAST EA EON TE LENA “SIRS TO EEN SS EIS SERS RE MEMETS te TAS SENN fue, Ree =| INAR Mee OGRE givea specific example | to explain the use of P and W during query processing. Suppose we have a policy of three iterations, P = 2,4,7 and W = 5 second: Under this policy, the source node S first sends a query message to the network vi BFS search of depth 2. The nodes that are within 2-hops away from the source nod would receive the message. These nodes then evaluate this query message locall and return the results to S. In the meantime, the query would be temporarily store on the nodes that are 2-hops away from S. After W seconds, S would terminat the query if the results satisfy the user. Otherwise, S issues another resend quer message with a BFS depth of 4, whose query ID is the same as the query ID o the previous query message. Upon receiving this resend query, the nodes that ar less than 2-hops away from the query node would do nothing but forward the quer to their neighbors. This process repeats until the resend query message reaches th
the query result, the information of the key k is also cached (stored) in the routing tables of Peer 1 and Peer 3. 3.2.2 Heuristic-Based Routing Strategies OI COAST EA EON TE LENA “SIRS TO EEN SS EIS SERS RE MEMETS te TAS SENN fue, Ree =| INAR Mee OGRE givea specific example | to explain the use of P and W during query processing. Suppose we have a policy of three iterations, P = 2,4,7 and W = 5 second: Under this policy, the source node S first sends a query message to the network vi BFS search of depth 2. The nodes that are within 2-hops away from the source nod would receive the message. These nodes then evaluate this query message locall and return the results to S. In the meantime, the query would be temporarily store on the nodes that are 2-hops away from S. After W seconds, S would terminat the query if the results satisfy the user. Otherwise, S issues another resend quer message with a BFS depth of 4, whose query ID is the same as the query ID o the previous query message. Upon receiving this resend query, the nodes that ar less than 2-hops away from the query node would do nothing but forward the quer to their neighbors. This process repeats until the resend query message reaches th
»f the performance of the networks in terms of query success rates [207]. A common onclusion of these works is that hamiltonicity (i.e., the existence of a hamiltonian -ycle) is an important property of the structures. This is mainly because it provides . “default” route in the network. [207] goes a bit further with the analysis of the yroperties of these networks and identifies the node bisection width (the number yf nodes that need to be removed to split the network in equal subnetworks) as a ignificant measure of quality for the underlying structures. Through extensive tests yver the different networks mentioned above in this section, [207] shows that the 1igher the node bisection width of the underlying graph, the higher the success rate yf the queries.
»f the performance of the networks in terms of query success rates [207]. A common onclusion of these works is that hamiltonicity (i.e., the existence of a hamiltonian -ycle) is an important property of the structures. This is mainly because it provides . “default” route in the network. [207] goes a bit further with the analysis of the yroperties of these networks and identifies the node bisection width (the number yf nodes that need to be removed to split the network in equal subnetworks) as a ignificant measure of quality for the underlying structures. Through extensive tests yver the different networks mentioned above in this section, [207] shows that the 1igher the node bisection width of the underlying graph, the higher the success rate yf the queries.
Chord orders identifiers of nodes and data items in an identifier circle modulo 2” i.e., the identifier space is a circle of numbers from 0 to 2” — 1. This is the reason why we call Chord a ring structure. The system assigns a key k to the first node n whose identifier is equal or follows the identifier of k in the circle space. In other words, k is assigned to the first node clockwise from k. In this case, n is called the successor node of k, denoted as n(k). Figure 3.5 shows a graph representing a Chord ring where two keys Ke and Kj are assigned to the same node identifier N30 obtained by hashing the IP address “202.120.224.102”; Key Ks56 obtained by hashing the word “Sailing” is assigned to node identifier N79; Key K100 is assigned to node identifier Nj95; Two nodes N42 and Njo9 store no data items. Simple Lookup Algorithm. In this algorithm, each node in the system only needs to know its immediate successor node to perform lookup operations. When a node = me
Chord orders identifiers of nodes and data items in an identifier circle modulo 2” i.e., the identifier space is a circle of numbers from 0 to 2” — 1. This is the reason why we call Chord a ring structure. The system assigns a key k to the first node n whose identifier is equal or follows the identifier of k in the circle space. In other words, k is assigned to the first node clockwise from k. In this case, n is called the successor node of k, denoted as n(k). Figure 3.5 shows a graph representing a Chord ring where two keys Ke and Kj are assigned to the same node identifier N30 obtained by hashing the IP address “202.120.224.102”; Key Ks56 obtained by hashing the word “Sailing” is assigned to node identifier N79; Key K100 is assigned to node identifier Nj95; Two nodes N42 and Njo9 store no data items. Simple Lookup Algorithm. In this algorithm, each node in the system only needs to know its immediate successor node to perform lookup operations. When a node = me
node, node N7 asks node Ng; to resolve the query. In turn, node Ng; determines node N15 as the farthest node in its finger table that precedes K 17 in the identi- fier space. As a result, Ng; forwards the query to Nios. Node Nios then discovers that its immediate successor, node Nj29, succeeds key K 117, and hence it continues to forward the query to Nj29. Finally, since Nj29 holds the queried key, it returns the result to node N7. It is shown that the expected number of routing hops in this search algorithm is O(log NV) steps, assuming accurate finger tables and no recent node failures.
node, node N7 asks node Ng; to resolve the query. In turn, node Ng; determines node N15 as the farthest node in its finger table that precedes K 17 in the identi- fier space. As a result, Ng; forwards the query to Nios. Node Nios then discovers that its immediate successor, node Nj29, succeeds key K 117, and hence it continues to forward the query to Nj29. Finally, since Nj29 holds the queried key, it returns the result to node N7. It is shown that the expected number of routing hops in this search algorithm is O(log NV) steps, assuming accurate finger tables and no recent node failures.
nate space into different zones, each of which is assigned to a node. Each node in the system stores all data items belonging to the zone it is responsible for. To insert a new data item, the system uses a uniform hash function to map the key value of the data item to a point p in the coordinate space and finds the node n whose zone covers p to store the new data item. To process a query, the system also uses the same hash function to map the query to a point p’ in the coordinate space and find the node n’ whose zone covers p’. If the query’s result exists, it should be stored at n'. To find a node whose zone covers a point, the system starts at the requester node and follows immediate neighbor nodes step by step until it reaches the destina- tion node. As a result, for routing purpose, each node in CAN needs to maintain the information of its neighbor nodes that are nodes holding adjacent coordinate nodes to the node’s zone. Information of a node’s neighbor nodes forms the routing table of that node. Figure 3.9 shows an example of a CAN system with 5 nodes in a two dimensional space.
nate space into different zones, each of which is assigned to a node. Each node in the system stores all data items belonging to the zone it is responsible for. To insert a new data item, the system uses a uniform hash function to map the key value of the data item to a point p in the coordinate space and finds the node n whose zone covers p to store the new data item. To process a query, the system also uses the same hash function to map the query to a point p’ in the coordinate space and find the node n’ whose zone covers p’. If the query’s result exists, it should be stored at n'. To find a node whose zone covers a point, the system starts at the requester node and follows immediate neighbor nodes step by step until it reaches the destina- tion node. As a result, for routing purpose, each node in CAN needs to maintain the information of its neighbor nodes that are nodes holding adjacent coordinate nodes to the node’s zone. Information of a node’s neighbor nodes forms the routing table of that node. Figure 3.9 shows an example of a CAN system with 5 nodes in a two dimensional space.
nS Re NE: REO Nears Me creer Oe eee RO OPN ie TR The Basic Idea. In PRR scheme, each node is assigned a unique t-bit identifier (nodeID) randomly from identifier space (below we denote it as ID-Space). We can divide the nodeID into / levels of w = t/I bits each; let w bits represent a digit and h = 2w be the basis of node identifier. Each node i has (b — 1) log N neighbors ! = log N levels of 2w neighbors each, such that each level k, 0 < k </, will have nodes with identifiers as follows: k common digits with i’s identifier, followed by all possible 2” values for the (kK + 1)st digit, and any of the 2” possible values for each of the remaining digits. These sets of neighbors form a routing table. An example of routing table is given in Fig. 3.10. In the figure, at level Lj, 3AF2 has one neighbor 3C57 (these two nodes share the common first digit), and at level L3, 3AF2’s two neighbors (3AF1 and 3AFC) share the same first three digits. For each node i in the [D-Space, we define closest(i) as the node whose nodelID is “closest” to i, where he definition of “closest” varies, but subject to the ID-Space. Routing in PRR Tree. It is essential to find closest(i) of each node i. In general, we can find closest(i) from node with identifier 7 by greedily forwarding the message o anode &k in the routing table of j that (1) has the longest matching prefix with } among neighbor nodes and (2) has the longer matching prefix compared to the a I a ee ee a a Oe a era eee
nS Re NE: REO Nears Me creer Oe eee RO OPN ie TR The Basic Idea. In PRR scheme, each node is assigned a unique t-bit identifier (nodeID) randomly from identifier space (below we denote it as ID-Space). We can divide the nodeID into / levels of w = t/I bits each; let w bits represent a digit and h = 2w be the basis of node identifier. Each node i has (b — 1) log N neighbors ! = log N levels of 2w neighbors each, such that each level k, 0 < k </, will have nodes with identifiers as follows: k common digits with i’s identifier, followed by all possible 2” values for the (kK + 1)st digit, and any of the 2” possible values for each of the remaining digits. These sets of neighbors form a routing table. An example of routing table is given in Fig. 3.10. In the figure, at level Lj, 3AF2 has one neighbor 3C57 (these two nodes share the common first digit), and at level L3, 3AF2’s two neighbors (3AF1 and 3AFC) share the same first three digits. For each node i in the [D-Space, we define closest(i) as the node whose nodelID is “closest” to i, where he definition of “closest” varies, but subject to the ID-Space. Routing in PRR Tree. It is essential to find closest(i) of each node i. In general, we can find closest(i) from node with identifier 7 by greedily forwarding the message o anode &k in the routing table of j that (1) has the longest matching prefix with } among neighbor nodes and (2) has the longer matching prefix compared to the a I a ee ee a a Oe a era eee
example, in Fig. 3.12, node FFFF is the host of object 0000, so the routing path is also the publication path. Node OFO0 is an intermediate node of the publication path, which has a reference to 0000’s host (node FFFF) indicated by the dashed line. If there are several replicas of object O on different nodes, each node publishes its own copy independently. Tapestry nodes can store references to object O according to network latency from their perspective. To locate object O in the Tapestry network, a client routes a message to object O’s root node R to get the reference to O. If there is a reference to object O in a node of the routing path, the message is redirected to node S.
example, in Fig. 3.12, node FFFF is the host of object 0000, so the routing path is also the publication path. Node OFO0 is an intermediate node of the publication path, which has a reference to 0000’s host (node FFFF) indicated by the dashed line. If there are several replicas of object O on different nodes, each node publishes its own copy independently. Tapestry nodes can store references to object O according to network latency from their perspective. To locate object O in the Tapestry network, a client routes a message to object O’s root node R to get the reference to O. If there is a reference to object O in a node of the routing path, the message is redirected to node S.
down-right link to node D. At last node D issues a “vicinity” search to the destina- tion node B using its level-ring link. Canon [126] is a kind of hierarchical DHT schemes for structured P2P overlay, and Crescendo is a canonical version of Chord. The members of Crescendo system form a conceptional hierarchy that is constituted by multiple domains in different levels. In each level of the hierarchy, all nodes in one domain maintain a Chord- like ring and add links to outside nodes. In addition, two separate Chord rings in low level can be merged into a large Crescendo ring of upper level according to the following conditions that, besides retaining all its original links each node n in one ring establishes a link to node m in the other ring iff: (a) m immediately succeeds n at least distance 2! away for some 0 <i < N (the space size is N-bit), and (b) m is closer to n than any node in n’s ring. —_ a4 1 1 . a a” + Gee We oF a°64 7 1 oe ae oS eg &
down-right link to node D. At last node D issues a “vicinity” search to the destina- tion node B using its level-ring link. Canon [126] is a kind of hierarchical DHT schemes for structured P2P overlay, and Crescendo is a canonical version of Chord. The members of Crescendo system form a conceptional hierarchy that is constituted by multiple domains in different levels. In each level of the hierarchy, all nodes in one domain maintain a Chord- like ring and add links to outside nodes. In addition, two separate Chord rings in low level can be merged into a large Crescendo ring of upper level according to the following conditions that, besides retaining all its original links each node n in one ring establishes a link to node m in the other ring iff: (a) m immediately succeeds n at least distance 2! away for some 0 <i < N (the space size is N-bit), and (b) m is closer to n than any node in n’s ring. —_ a4 1 1 . a a” + Gee We oF a°64 7 1 oe ae oS eg &
to node 6. Then node 6 switches routing to the merged ring of the next higher level of the hierarchy, and in turn routes to node 10 using the greedy clockwise algorithm.
to node 6. Then node 6 switches routing to the merged ring of the next higher level of the hierarchy, and in turn routes to node 10 using the greedy clockwise algorithm.
Searching in Skip Graph is based on the same principle as searching in Skip List except with a minor difference. Instead of sending a search query from a low level node to a high level node, in a Skip Graph, when a node issues a query, the search process always starts at the highest level of that node. At each step, if there is a neighbor node at the same level that keeps a closer value to the search key, the node forwards the query to that neighbor node. Otherwise, the node continues the search process at a lower lever. The destination node containing the result is found when the search process reaches the bottom level. The search algorithm of Skip Graph is shown in Algorithm 5. As in Skip List, it takes approximately O(log NV) steps to process a query in an N node system.
Searching in Skip Graph is based on the same principle as searching in Skip List except with a minor difference. Instead of sending a search query from a low level node to a high level node, in a Skip Graph, when a node issues a query, the search process always starts at the highest level of that node. At each step, if there is a neighbor node at the same level that keeps a closer value to the search key, the node forwards the query to that neighbor node. Otherwise, the node continues the search process at a lower lever. The destination node containing the result is found when the search process reaches the bottom level. The search algorithm of Skip Graph is shown in Algorithm 5. As in Skip List, it takes approximately O(log NV) steps to process a query in an N node system.
74 as in Fig. 3.20. Since the searched value is greater than H’s upper bound, H first sends the query to L, the farthest neighbor node nearer to but not overshooting the searched key. Similarly, L then forwards the query to M. Since there is no neighbor node of M whose upper bound is less then the searched key, M continues to forward the query to its right child W. Finally, following right adjacent link from W to C, the query is forwarded to the destination node.
74 as in Fig. 3.20. Since the searched value is greater than H’s upper bound, H first sends the query to L, the farthest neighbor node nearer to but not overshooting the searched key. Similarly, L then forwards the query to M. Since there is no neighbor node of M whose upper bound is less then the searched key, M continues to forward the query to its right child W. Finally, following right adjacent link from W to C, the query is forwarded to the destination node.
schema. As such, all organizations can use the standard query language RDF-QEL-i to find their desired resources and process raw materials for their further purposes. The core Edutella network is a HyperCuP [286] network that is composed of superpeers. All common peers will select one of superpeers as their access point for the whole network. In order to route queries on behalf of client peers, each superpeer is also responsible for indexing its client peers’ RDF schemes, which is defined as superpeer-peer routing indices. To route queries among superpeers, all superpeers then exchange these indexed RDF schemes to build a high level index, which is defined as superpeer-superpeer routing indices. oe ee ee
schema. As such, all organizations can use the standard query language RDF-QEL-i to find their desired resources and process raw materials for their further purposes. The core Edutella network is a HyperCuP [286] network that is composed of superpeers. All common peers will select one of superpeers as their access point for the whole network. In order to route queries on behalf of client peers, each superpeer is also responsible for indexing its client peers’ RDF schemes, which is defined as superpeer-peer routing indices. To route queries among superpeers, all superpeers then exchange these indexed RDF schemes to build a high level index, which is defined as superpeer-superpeer routing indices. oe ee ee
as ans In the ultrapeer network, all ultrapeers form an unstructured P2P network. To locate desired resources, a leaf peer should send a search query to its ultrapeer. The ultrapeer will propagate the query to all neighboring ultrapeers and each neighbor- ing ultrapeer then propagates the query to its neighboring ultrapeers and so on. Once receiving the query, the ultrapeer will search the query with its reflector indexing to locate which leaf peer has the desired resources. If a leaf peer is found, then the [P address of the leaf peer will be returned to the query peer. Here, we notice the concept of “reflector indexing”, which is a kind of index for meta data of shared
as ans In the ultrapeer network, all ultrapeers form an unstructured P2P network. To locate desired resources, a leaf peer should send a search query to its ultrapeer. The ultrapeer will propagate the query to all neighboring ultrapeers and each neighbor- ing ultrapeer then propagates the query to its neighboring ultrapeers and so on. Once receiving the query, the ultrapeer will search the query with its reflector indexing to locate which leaf peer has the desired resources. If a leaf peer is found, then the [P address of the leaf peer will be returned to the query peer. Here, we notice the concept of “reflector indexing”, which is a kind of index for meta data of shared
To locate the peer with a specific key, the query peer first requests its superpeer and the superpeer checks whether the key falls into its own range. If so, the query peer will use the Chord protocol to locate the peer. Otherwise, the lookup key will be forwarded to the superpeer whose range contains the key, by using the super- peer table. Last, the superpeer sends the lookup query to its successor and then the successor uses the Chord protocol to locate the desired peer.
To locate the peer with a specific key, the query peer first requests its superpeer and the superpeer checks whether the key falls into its own range. If so, the query peer will use the Chord protocol to locate the peer. Otherwise, the lookup key will be forwarded to the superpeer whose range contains the key, by using the super- peer table. Last, the superpeer sends the lookup query to its successor and then the successor uses the Chord protocol to locate the desired peer.
Table 3.1 Summary of existing routing methods in P2P systems
Table 3.1 Summary of existing routing methods in P2P systems
Table 3.1 (continued)
Table 3.1 (continued)
hat is in the other side of the tree rooted at a’. In this way, VBI-Tree can search all 10des whose region intersects with the searched region without causing bottle neck it the root or nodes near the root. Algorithm 9 displays the basic search algorithm of VBI-Tree. It is important to mention that (1) to avoid search loop, the system <eeps track the search path and (2) to avoid frequently update of upside links due to changes in covered regions at ancestor nodes, VBI-Tree proposes a new concept of liscrete data. Discrete data are data that are stored at internal nodes and do not fall nto any regions covered by children of internal nodes.
hat is in the other side of the tree rooted at a’. In this way, VBI-Tree can search all 10des whose region intersects with the searched region without causing bottle neck it the root or nodes near the root. Algorithm 9 displays the basic search algorithm of VBI-Tree. It is important to mention that (1) to avoid search loop, the system <eeps track the search path and (2) to avoid frequently update of upside links due to changes in covered regions at ancestor nodes, VBI-Tree proposes a new concept of liscrete data. Discrete data are data that are stored at internal nodes and do not fall nto any regions covered by children of internal nodes.
Fig. 4.2. Query processing in VBI-Tree
Fig. 4.2. Query processing in VBI-Tree
attribute produces a lot of results, the system performance degrades significantly. Therefore, it is important to choose the dominant attribute wisely. The disadvantage of Mercury is that it incurs a high cost in data insertion and deletion especially when the number of index attributes is big. It is because when a new data item is added to the system, it is inserted to all hubs corresponding to index attributes. Similarly, when an existing data item is deleted, all hubs are checked to delete the corresponding data item.
attribute produces a lot of results, the system performance degrades significantly. Therefore, it is important to choose the dominant attribute wisely. The disadvantage of Mercury is that it incurs a high cost in data insertion and deletion especially when the number of index attributes is big. It is because when a new data item is added to the system, it is inserted to all hubs corresponding to index attributes. Similarly, when an existing data item is deleted, all hubs are checked to delete the corresponding data item.
values in one-dimensional space. As an example, Fig. 4.5(b) shows a query corre- sponding to the shaded region in the Hilbert space. This query is converted to three different search segments in the one-dimensional index space. This is indeed a com- mon problem with the use of curves in data mapping. In particular, the higher the dimension is, the higher the possibility of having more search segments in query processing is and hence the higher the cost of query processing is.
values in one-dimensional space. As an example, Fig. 4.5(b) shows a query corre- sponding to the shaded region in the Hilbert space. This query is converted to three different search segments in the one-dimensional index space. This is indeed a com- mon problem with the use of curves in data mapping. In particular, the higher the dimension is, the higher the possibility of having more search segments in query processing is and hence the higher the cost of query processing is.
in a Skip Graph structure is determined by the position of its holding subspace in the Z-curve ordering. An example of ZNet structure is shown in Fig. 4.6, where the partition of the data space is displayed in Fig. 4.6(a) and the positions of ZNet nodes in a Skip Graph structure are displayed in Fig. 4.6(b). eames
in a Skip Graph structure is determined by the position of its holding subspace in the Z-curve ordering. An example of ZNet structure is shown in Fig. 4.6, where the partition of the data space is displayed in Fig. 4.6(a) and the positions of ZNet nodes in a Skip Graph structure are displayed in Fig. 4.6(b). eames
LSH Forest [39] is an index structure containing a set of / LSH Trees, each of whicl is a prefix tree. For indexing purposes, the system uses a family H of locality sen sitive hash functions. In this index structure, each inserted object O is indexed times to / LSH Trees. At each LSH Tree, O is assigned a variable length x-digi identifier, which is the concatenating result of {h1(O),h2(O),...,4x(O)}, wher h\(O),...,hx(O) are hashing functions chosen from 71. This identifier determine the position of O in the LSH Tree where O is a leaf node and each digit in the iden tifier of O represents a part of the path from the root to O’s position. An example o a LSH Tree containing 4 objects is shown in Fig. 4.9 where the identifiers of thes objects are 00, 01, 110, and 111. It is important to note that the length of objec identifiers should be long enough so that each object can have a distinct identifier and hence a distinct position in the LSH Tree. To process a similarity search query, LSH Forest first uses hash functions se
LSH Forest [39] is an index structure containing a set of / LSH Trees, each of whicl is a prefix tree. For indexing purposes, the system uses a family H of locality sen sitive hash functions. In this index structure, each inserted object O is indexed times to / LSH Trees. At each LSH Tree, O is assigned a variable length x-digi identifier, which is the concatenating result of {h1(O),h2(O),...,4x(O)}, wher h\(O),...,hx(O) are hashing functions chosen from 71. This identifier determine the position of O in the LSH Tree where O is a leaf node and each digit in the iden tifier of O represents a part of the path from the root to O’s position. An example o a LSH Tree containing 4 objects is shown in Fig. 4.9 where the identifiers of thes objects are 00, 01, 110, and 111. It is important to note that the length of objec identifiers should be long enough so that each object can have a distinct identifier and hence a distinct position in the LSH Tree. To process a similarity search query, LSH Forest first uses hash functions se
The tree structure and summary index structure are illustrated in Fig. 4.11. Based on this summary index structure, when a peer node issues a query, it first sends the query to its super peer. The super-peer then checks the super-peer summary level to find super-peers holding relevant indices to the query. These relevant super-peers are forwarded the query. When a super-peer receives a query from another super- peer, it checks the peer summary level to find suitable peers to forward the query. Finally, at peer nodes in the leaf-peer level, document summary level is referenced to process the query and the results are returned to the query issuer. Algorithm 12 shows the overall search algorithm. —* Tt °4 a? ree 1 ae 1 . 4 mn - . ae
The tree structure and summary index structure are illustrated in Fig. 4.11. Based on this summary index structure, when a peer node issues a query, it first sends the query to its super peer. The super-peer then checks the super-peer summary level to find super-peers holding relevant indices to the query. These relevant super-peers are forwarded the query. When a super-peer receives a query from another super- peer, it checks the peer summary level to find suitable peers to forward the query. Finally, at peer nodes in the leaf-peer level, document summary level is referenced to process the query and the results are returned to the query issuer. Algorithm 12 shows the overall search algorithm. —* Tt °4 a? ree 1 ae 1 . 4 mn - . ae
documents are indexed in nearby regions, the overlay network returns all document indices with in a radius r from the query point (r is a configurable parameter de- termined by either the similarity threshold or the number of wanted documents). A pSearch engine constructed on a 6 CAN nodes is illustrated in Fig. 4.12. Note that in pSearch global knowledge is created and maintained at each node in the overlay network. The way global knowledge is managed is independent with the way documents are indexed.
documents are indexed in nearby regions, the overlay network returns all document indices with in a radius r from the query point (r is a configurable parameter de- termined by either the similarity threshold or the number of wanted documents). A pSearch engine constructed on a 6 CAN nodes is illustrated in Fig. 4.12. Note that in pSearch global knowledge is created and maintained at each node in the overlay network. The way global knowledge is managed is independent with the way documents are indexed.
Fig. 4.13 A rotation of a 4-dimensional points to two 2-dimensional points
Fig. 4.13 A rotation of a 4-dimensional points to two 2-dimensional points
scans peers containing data in N, and fetches corresponding data in S for each tuple in N. FetchMatches has the disadvantage that selections on non-DHT attributes cannot be pushed into the DHT layer [159]. It is because if the selection functions are integrated into the DHT layer, they may dirtying the DHT APIs. To solve this problem, two improved algorithms, namely symmetric semi-join and Bloom join, are studied [159].
scans peers containing data in N, and fetches corresponding data in S for each tuple in N. FetchMatches has the disadvantage that selections on non-DHT attributes cannot be pushed into the DHT layer [159]. It is because if the selection functions are integrated into the DHT layer, they may dirtying the DHT APIs. To solve this problem, two improved algorithms, namely symmetric semi-join and Bloom join, are studied [159].
Fig. 4.17 Reformulation rule-goal tree
Fig. 4.17 Reformulation rule-goal tree
This is the most complex scheme involving many lightly loaded nodes and many heavily loaded nodes in the load balancing process. To serve the process of moving virtual nodes, this scheme creates a pool to keep virtual nodes temporarily while they are not assigned to any node. The scheme is executed in three phases:
This is the most complex scheme involving many lightly loaded nodes and many heavily loaded nodes in the load balancing process. To serve the process of moving virtual nodes, this scheme creates a pool to keep virtual nodes temporarily while they are not assigned to any node. The scheme is executed in three phases:
Fig. 5.3. Many-to-many load balancing scheme
Fig. 5.3. Many-to-many load balancing scheme
node. The problem now is how to direct a query, given that the routing table keeps only one entry for log N nodes. Yo deals with this problem by modifying the search algorithm a little: A node still tries to forward the search request to the farthest node in its routing table not overshooting the searched key and the message stops when it arrives at a node whose range of values covers the search key, as in Chord. However, that node may not actually keep the key because there are internal nodes between the virtual nodes that are actually maintained at the node. At this point, the succes- sor link of the virtual node, which is the nearest with respect to the searched key, is selected for forwarding the request. The next node receiving the request repeats the same process to forward the query through a successor link towards the destination node. In particular, if a virtual node keeps O(log N) successor links (the main pur- pose is fault tolerance!), the query processing only takes O(1) additional steps. The Yo protocol is illustrated as in Fig. 5.4.
node. The problem now is how to direct a query, given that the routing table keeps only one entry for log N nodes. Yo deals with this problem by modifying the search algorithm a little: A node still tries to forward the search request to the farthest node in its routing table not overshooting the searched key and the message stops when it arrives at a node whose range of values covers the search key, as in Chord. However, that node may not actually keep the key because there are internal nodes between the virtual nodes that are actually maintained at the node. At this point, the succes- sor link of the virtual node, which is the nearest with respect to the searched key, is selected for forwarding the request. The next node receiving the request repeats the same process to forward the query through a successor link towards the destination node. In particular, if a virtual node keeps O(log N) successor links (the main pur- pose is fault tolerance!), the query processing only takes O(1) additional steps. The Yo protocol is illustrated as in Fig. 5.4.
5.2.4 A Combination of Both Local and Random Probes In order to reduce the cost of random choices paradigm, some authors [182, 215] suggest that the node should ask for the load of not only random nodes (random probe) but also neighbor nodes surrounding them (local probe) to estimate the load distribution of the system. Since it is much cheaper to ask for the load of an adja- cent node of a contact node than to ask for the load of a contact node (1 message compared to O(log NV) messages), the node can reduce the cost of load querying by decreasing the number of contact random nodes and increasing the number of contact neighbor nodes in the vicinity of contact nodes while still achieve the same performance.
5.2.4 A Combination of Both Local and Random Probes In order to reduce the cost of random choices paradigm, some authors [182, 215] suggest that the node should ask for the load of not only random nodes (random probe) but also neighbor nodes surrounding them (local probe) to estimate the load distribution of the system. Since it is much cheaper to ask for the load of an adja- cent node of a contact node than to ask for the load of a contact node (1 message compared to O(log NV) messages), the node can reduce the cost of load querying by decreasing the number of contact random nodes and increasing the number of contact neighbor nodes in the vicinity of contact nodes while still achieve the same performance.
get local load distribution. The node also periodically samples a number of ran- dom distant nodes, to ask for the local load distribution of these nodes. The process of sampling distant nodes is based on random walk algorithm. In particular, the node first sends sampling requests with log N hops time-to-live to random neighbor nodes. After that, at each node along the sampling path, a random neighbor node is selected to forward the request. In this way, the process finally sends sampling requests uniformly to nodes in the system, with high probability. The node at the end of the sampling path returns to the requester node its local load distribution. Based on the received samples, a load distribution histogram is constructed at the node. The process of constructing and maintaining the histogram of a node is illus- trated as in Fig. 5.7. As discussed before the method to construct and maintain the histogram is based on the combination of local and random probes presented in the above section. In particular, in the figure of the example, values of random probe (r) and local probe (v) are respectively, 4 and 5.
get local load distribution. The node also periodically samples a number of ran- dom distant nodes, to ask for the local load distribution of these nodes. The process of sampling distant nodes is based on random walk algorithm. In particular, the node first sends sampling requests with log N hops time-to-live to random neighbor nodes. After that, at each node along the sampling path, a random neighbor node is selected to forward the request. In this way, the process finally sends sampling requests uniformly to nodes in the system, with high probability. The node at the end of the sampling path returns to the requester node its local load distribution. Based on the received samples, a load distribution histogram is constructed at the node. The process of constructing and maintaining the histogram of a node is illus- trated as in Fig. 5.7. As discussed before the method to construct and maintain the histogram is based on the combination of local and random probes presented in the above section. In particular, in the figure of the example, values of random probe (r) and local probe (v) are respectively, 4 and 5.
Even though maintaining such a Skip Graph is expensive, since any of operation from node join, node departure to data insertion or data deletion can make node in the Skip Graph change their positions, this method can achieve the maximun imbalance ratio load, which is the ratio between the heavily loaded node and thi lightly loaded node in the system, by 5°. Note that, depending on applications, 6 ca be set to a low or a high value. The lower value the 6 is, the better the load balanc: of the system is and certainly the higher the cost of load balancing is. in the Skip Graph change their positions, this method can achieve the maximum
Even though maintaining such a Skip Graph is expensive, since any of operation from node join, node departure to data insertion or data deletion can make node in the Skip Graph change their positions, this method can achieve the maximun imbalance ratio load, which is the ratio between the heavily loaded node and thi lightly loaded node in the system, by 5°. Note that, depending on applications, 6 ca be set to a low or a high value. The lower value the 6 is, the better the load balanc: of the system is and certainly the higher the cost of load balancing is. in the Skip Graph change their positions, this method can achieve the maximum
_ The first technique uses the successor replication scheme. Each block stored at a node is replicated to k following nodes in the Chord ring to increase avail- ability of the block. This technique is based on a property of Chord ring that if a node fails, the immediate successor node will take responsibility for all data belonging to the failed node. As a result, using this replication technique, blocks are always available to retrieve even thought the block owner fails. Furthermore, this technique can help clients to select the fastest download server from a group
_ The first technique uses the successor replication scheme. Each block stored at a node is replicated to k following nodes in the Chord ring to increase avail- ability of the block. This technique is based on a property of Chord ring that if a node fails, the immediate successor node will take responsibility for all data belonging to the failed node. As a result, using this replication technique, blocks are always available to retrieve even thought the block owner fails. Furthermore, this technique can help clients to select the fastest download server from a group
At the center of Beehive there is an analytical model, which helps to identify optimal replication levels for objects based on the query distribution. Let a be the parameter of the query distribution in which the i most popular object has the number of queries proportional to i~%; M and N be the number of objects and the number of nodes in the system; b be the fan-out of the search tree structure in the system; k = log, N be the maximum number of search steps; C be the desired lookup constant.
At the center of Beehive there is an analytical model, which helps to identify optimal replication levels for objects based on the query distribution. Let a be the parameter of the query distribution in which the i most popular object has the number of queries proportional to i~%; M and N be the number of objects and the number of nodes in the system; b be the fan-out of the search tree structure in the system; k = log, N be the maximum number of search steps; C be the desired lookup constant.
Fig. 6.4 Self-verification scheme version and another fragment has to be obtained. A solution to this problem is to build a Merkle hash tree for the data object and its fragment i.e., the data object and its fragments form the leaves of the tree (in a similar manner as that described in Sect. 6.4.1). Figure 6.4(a) shows an exam- ple of a Merkle hash tree for data object with 4 fragments. Here, fragment F; has hash value of Hj and the data object has hash value of Hy. The digest GUID is the unique identifier that can be used to identify and verify the object. Each fragment is made self-verifiable by storing in each fragment all necessary information to ver- ify the fragment’s hash value i.e., hash values of sibling nodes in the path from the leaf fragment to the root. Figure 6.4(b) shows the self-verifiable fragments’ content for the example in Fig. 6.4(a). When a node receives a fragment to reconstruct the object, the node verifies the fragment by first computing the hash value of the frag- ment, and then repeating the process of hashing the result computed the previous step with the correspondent hash value in the Merkle hash tree until the hash value of the root is obtained. If the final hash value matches the GUID, the fragment is a valid fragment and can be used for reconstruction; otherwise, it is a corrupted version and another fragment has to be obtained.
Fig. 6.4 Self-verification scheme version and another fragment has to be obtained. A solution to this problem is to build a Merkle hash tree for the data object and its fragment i.e., the data object and its fragments form the leaves of the tree (in a similar manner as that described in Sect. 6.4.1). Figure 6.4(a) shows an exam- ple of a Merkle hash tree for data object with 4 fragments. Here, fragment F; has hash value of Hj and the data object has hash value of Hy. The digest GUID is the unique identifier that can be used to identify and verify the object. Each fragment is made self-verifiable by storing in each fragment all necessary information to ver- ify the fragment’s hash value i.e., hash values of sibling nodes in the path from the leaf fragment to the root. Figure 6.4(b) shows the self-verifiable fragments’ content for the example in Fig. 6.4(a). When a node receives a fragment to reconstruct the object, the node verifies the fragment by first computing the hash value of the frag- ment, and then repeating the process of hashing the result computed the previous step with the correspondent hash value in the Merkle hash tree until the hash value of the root is obtained. If the final hash value matches the GUID, the fragment is a valid fragment and can be used for reconstruction; otherwise, it is a corrupted version and another fragment has to be obtained.
Fig. 6.5 Exchange-based scheme
Fig. 6.5 Exchange-based scheme
a “safe” programming language to uniformly describe policies, credentials, and trust relationships instead of treating them separately as other systems [162, 352]. In the system, each agent can take a decision in a flexible way: either locally or via a third party. Furthermore, PolicyMaker separates mechanism from policy, and hence makes applications independent from the infrastructure.
a “safe” programming language to uniformly describe policies, credentials, and trust relationships instead of treating them separately as other systems [162, 352]. In the system, each agent can take a decision in a flexible way: either locally or via a third party. Furthermore, PolicyMaker separates mechanism from policy, and hence makes applications independent from the infrastructure.
Fig. 7.3. Phases of a negotiation The negotiation process and its phases are described as in Fig. 7.3.
Fig. 7.3. Phases of a negotiation The negotiation process and its phases are described as in Fig. 7.3.
— Strongest path: in this strategy, a peer first finds the strongest path between itself and the destination node. The strongest path is determined as the path whose minimum valued edge along the path or the product of all edges along the path is the highest amongst all possible paths between two nodes. Once the strongest path is known, the trust value is the smallest weight among the edges in the path. For example, if we use the minimum valued edge function for finding the strongest path, the strongest path between A and F in Fig. 7.5 is A—> C > D— F. As aresult, the trust value of F inferred by A is 0.65.
— Strongest path: in this strategy, a peer first finds the strongest path between itself and the destination node. The strongest path is determined as the path whose minimum valued edge along the path or the product of all edges along the path is the highest amongst all possible paths between two nodes. Once the strongest path is known, the trust value is the smallest weight among the edges in the path. For example, if we use the minimum valued edge function for finding the strongest path, the strongest path between A and F in Fig. 7.5 is A—> C > D— F. As aresult, the trust value of F inferred by A is 0.65.
7.5 Trust Systems Based on Both Individual Reputation and Social Relationship
7.5 Trust Systems Based on Both Individual Reputation and Social Relationship
benefits from society reputation once they know the way societies are formed. Here, OR;— ;(k) = SR; ;(k) if k is an atomic feature; ws, represents some weights in the calculation, normalized such that )~ kechildren(s) Wsk = 1. The exis- tence of these weights is necessary because different agents may have different points of view on properties in the formula. Furthermore, these weights at each peer are not fixed. They may change over time, depending on necessities. Figure 7.7 shows an example of an ontology for the reputation of a hotel, consisting of price, service, and location.
benefits from society reputation once they know the way societies are formed. Here, OR;— ;(k) = SR; ;(k) if k is an atomic feature; ws, represents some weights in the calculation, normalized such that )~ kechildren(s) Wsk = 1. The exis- tence of these weights is necessary because different agents may have different points of view on properties in the formula. Furthermore, these weights at each peer are not fixed. They may change over time, depending on necessities. Figure 7.7 shows an example of an ontology for the reputation of a hotel, consisting of price, service, and location.
this problem is asking a peer to evaluate the reputation of referenced peers too. This may lead to a circular check for all peers in the system, which is costly, but in many cases, the system can decide the correctness of reputation values just after a few steps. This method can be applied in any structured Peer-to-Peer systems such as P-Grid [17], CAN [266], or CHORD [173]. aaa Fr a A a a ee fo In conclusion, the taxonomy of trust management methods can be displayed as in Fig. 7.9. It is interesting to realize that the taxonomy of trust management is similar to the taxonomy of Peer-to-Peer networks: the first model can be consid- ered as the unstructured, server based Peer-to-Peer model, the second model using gossiping can be considered as the unstructured, pure Peer-to-Peer model, and the last one as the structured Peer-to-Peer model. In the following parts of this sec- tion, we introduce three systems that are representatives of these models. They are XenoTrust [113], using server based trust management model, EigenRep [171], em- ploying gossiping based trust management model, and the work of Ge, Luo, and Xu [19], using P-Grid for trust management.
this problem is asking a peer to evaluate the reputation of referenced peers too. This may lead to a circular check for all peers in the system, which is costly, but in many cases, the system can decide the correctness of reputation values just after a few steps. This method can be applied in any structured Peer-to-Peer systems such as P-Grid [17], CAN [266], or CHORD [173]. aaa Fr a A a a ee fo In conclusion, the taxonomy of trust management methods can be displayed as in Fig. 7.9. It is interesting to realize that the taxonomy of trust management is similar to the taxonomy of Peer-to-Peer networks: the first model can be consid- ered as the unstructured, server based Peer-to-Peer model, the second model using gossiping can be considered as the unstructured, pure Peer-to-Peer model, and the last one as the structured Peer-to-Peer model. In the following parts of this sec- tion, we introduce three systems that are representatives of these models. They are XenoTrust [113], using server based trust management model, EigenRep [171], em- ploying gossiping based trust management model, and the work of Ge, Luo, and Xu [19], using P-Grid for trust management.
The framework of the XenoServer Open Platform can be seen in the top part of Fig. 7.10. Although the platform has a component, XenoCorp, that can provide a trusted payment service for its clients, it cannot let them know the reputation of each other (i.e., if they are good or bad servers/clients). Since servers and clients are autonomous, it is necessary to provide users (both servers and clients) a mechanism for evaluating the reputation of each other. XenoTrust is dedicated for this purpose. for evaluating the reputation of each other. XenoTrust is dedicated for this purpose.
The framework of the XenoServer Open Platform can be seen in the top part of Fig. 7.10. Although the platform has a component, XenoCorp, that can provide a trusted payment service for its clients, it cannot let them know the reputation of each other (i.e., if they are good or bad servers/clients). Since servers and clients are autonomous, it is necessary to provide users (both servers and clients) a mechanism for evaluating the reputation of each other. XenoTrust is dedicated for this purpose. for evaluating the reputation of each other. XenoTrust is dedicated for this purpose.
7.6.3 Trust Management with P-Grid
7.6.3 Trust Management with P-Grid
Table 7.1 Summary of representative P2P systems discussed through the chapter
Table 7.1 Summary of representative P2P systems discussed through the chapter
JXTA defines a set of concepts, at the core of which lies, naturally, that of a peer and of peer services. A peer may be any type of device, ranging from the smallest smart-card to a fully blown server. Function of its capabilities, a peer can provide more or less services for other peers. Figure 8.2 illustrates the hierarchy of services and specific denominations given to the peers implementing only subsets of these services. In this figure, it is worth pointing out the existence of super-peers which are peers that help other peers to communicate with the network by translating some particular communication protocol into the general TCP/IP stack, or act as meeting points between peers. They implement all or a subset of the Relay, Rendezvous and Proxy protocols.
JXTA defines a set of concepts, at the core of which lies, naturally, that of a peer and of peer services. A peer may be any type of device, ranging from the smallest smart-card to a fully blown server. Function of its capabilities, a peer can provide more or less services for other peers. Figure 8.2 illustrates the hierarchy of services and specific denominations given to the peers implementing only subsets of these services. In this figure, it is worth pointing out the existence of super-peers which are peers that help other peers to communicate with the network by translating some particular communication protocol into the general TCP/IP stack, or act as meeting points between peers. They implement all or a subset of the Relay, Rendezvous and Proxy protocols.
Fig. 8.5 Composition of a PNRP identifier
Fig. 8.5 Composition of a PNRP identifier
Fig. 9.2 Overview of pStore The pStore system [38] separates backup and underlying Peer-to-Peer functionality. pStore completely surrenders all Peer-to-Peer tasks to Chord [173], a distributed hash table (DHT) implementation, concentrating on the implementation of backup semantics on top of it. When creating a new backup, pStore splits each individual file of the backup set into a number of equal-size blocks (except the last block, which may be smaller) and creates a metadata record, which are then stored inside the hash table (Fig. 9.2). The mapping of data blocks to nodes inside the Peer-to-Peer overlay is managed by Chord.
Fig. 9.2 Overview of pStore The pStore system [38] separates backup and underlying Peer-to-Peer functionality. pStore completely surrenders all Peer-to-Peer tasks to Chord [173], a distributed hash table (DHT) implementation, concentrating on the implementation of backup semantics on top of it. When creating a new backup, pStore splits each individual file of the backup set into a number of equal-size blocks (except the last block, which may be smaller) and creates a metadata record, which are then stored inside the hash table (Fig. 9.2). The mapping of data blocks to nodes inside the Peer-to-Peer overlay is managed by Chord.
Fig. 9.3. Block creation process in pStore
Fig. 9.3. Block creation process in pStore
Fig. 9.6 Adding version to file in pStore By employing these techniques, significant savings on the global storage require- ments can be achieved. The authors of pStore claim a reduction of between 3 and 60 percent in total storage requirements. However, savings depend heavily on the data inserted into the backup system and scale directly with the degree of overlap between hosts. (A study conducted as part of the Farsite project [55] found up to 50% overlap between the computers of Microsoft employees.)
Fig. 9.6 Adding version to file in pStore By employing these techniques, significant savings on the global storage require- ments can be achieved. The authors of pStore claim a reduction of between 3 and 60 percent in total storage requirements. However, savings depend heavily on the data inserted into the backup system and scale directly with the degree of overlap between hosts. (A study conducted as part of the Farsite project [55] found up to 50% overlap between the computers of Microsoft employees.)
Fig. 9.5 Block matching in pStore
Fig. 9.5 Block matching in pStore
Fig. 9.7 Virtual disk using erasure code IER conse ess One ae nee oor a ae ee The main idea of this scheme is: Peers exchange disk space in a symmetric man- ner. When forming a trading partnership, both peers agree on terms like the ex- change quantity, the time-frame of the exchange and certain availability of each peer. Of course, it is rare for two peers to find exact matches in exchanging quanti- ties, so trade ratio might be used. Once both peers have agreed to the deal, they can begin storing data on each other’s hard disk.
Fig. 9.7 Virtual disk using erasure code IER conse ess One ae nee oor a ae ee The main idea of this scheme is: Peers exchange disk space in a symmetric man- ner. When forming a trading partnership, both peers agree on terms like the ex- change quantity, the time-frame of the exchange and certain availability of each peer. Of course, it is rare for two peers to find exact matches in exchanging quanti- ties, so trade ratio might be used. Once both peers have agreed to the deal, they can begin storing data on each other’s hard disk.
Fig. 9.8 Buddy searching process in pastiche Discovering Buddies During the process of finding the partner, one important crite- ria is the overlapping data in backup sets. The reason is that the higher the overlap between buddies the less data they need to exchange. Discovering matching peers is done in a distributed scheme with the help of a central server which is also used by Cooperative Internet Backup Scheme.
Fig. 9.8 Buddy searching process in pastiche Discovering Buddies During the process of finding the partner, one important crite- ria is the overlapping data in backup sets. The reason is that the higher the overlap between buddies the less data they need to exchange. Discovering matching peers is done in a distributed scheme with the help of a central server which is also used by Cooperative Internet Backup Scheme.
Fig. 9.9 Robustness of anchors during file modification on patterns to discover block boundaries, only directly affected blocks are changed during an update process of the file, while the rest blocks are undisturbed (Fig. 9.9).
Fig. 9.9 Robustness of anchors during file modification on patterns to discover block boundaries, only directly affected blocks are changed during an update process of the file, while the rest blocks are undisturbed (Fig. 9.9).
Fig. 9.10 Generation and forwarding of storage claims in samsara
Fig. 9.10 Generation and forwarding of storage claims in samsara
Fig. 9.11 Data migration in distributed hash table DHT is used to store data items indexed by keys, additional maintenance cost, and probably the more dominant cost, will rise: data migration cost. Whenever a pee! joins or leaves the network, it forces a reorganization of the key mapping in DHT which causes data items to become misplaced. As a result, a number of data items will be stored at a peer that is no longer responsible for them. In order to ensure correct retrieval of these misplaced data items, they need to be moved to the pee that is now responsible for them. As depicted in Fig. 9.11, without this process, more and more data items in the DHT would not be retrievable as queries for them will eventually reach the wrong responsible peer. Ts pies eh. ses thek. Sees A whe Aes serie ee Se Ree tH [Hc
Fig. 9.11 Data migration in distributed hash table DHT is used to store data items indexed by keys, additional maintenance cost, and probably the more dominant cost, will rise: data migration cost. Whenever a pee! joins or leaves the network, it forces a reorganization of the key mapping in DHT which causes data items to become misplaced. As a result, a number of data items will be stored at a peer that is no longer responsible for them. In order to ensure correct retrieval of these misplaced data items, they need to be moved to the pee that is now responsible for them. As depicted in Fig. 9.11, without this process, more and more data items in the DHT would not be retrievable as queries for them will eventually reach the wrong responsible peer. Ts pies eh. ses thek. Sees A whe Aes serie ee Se Ree tH [Hc
Manager modules. The namespace, resourceID, and instanceID are used to uniquely identify an object, and the namespace and resourceID are used to calculate the DHT key [159]. In implementation, namespaces are relation names, while resourceIDs are by default the values of primary keys, and instanceIDs are used to separate objects with the same namespace and resourceID. This CAN-based DHT layer is enhanced to provide following operators in Provider module:
Manager modules. The namespace, resourceID, and instanceID are used to uniquely identify an object, and the namespace and resourceID are used to calculate the DHT key [159]. In implementation, namespaces are relation names, while resourceIDs are by default the values of primary keys, and instanceIDs are used to separate objects with the same namespace and resourceID. This CAN-based DHT layer is enhanced to provide following operators in Provider module:
a Pidvlitdal FLivino. One of the scenarios that BestPeer v2.0 starts from, is that of a Supply Chain Management application, where different participants (i.e., peers) need to share data for mutual advantage (Fig. 9.13). Consequently, apart from the “traditional” PDMS areas of interest (query processing, index management), BestPeer v2.0 also addresses issues of access control and data synchronization (Fig. 9.14) in order to protect each participant’s private data stored, for instance, in an internal ERP* sys- tem. Enterprise Resource Planning.
a Pidvlitdal FLivino. One of the scenarios that BestPeer v2.0 starts from, is that of a Supply Chain Management application, where different participants (i.e., peers) need to share data for mutual advantage (Fig. 9.13). Consequently, apart from the “traditional” PDMS areas of interest (query processing, index management), BestPeer v2.0 also addresses issues of access control and data synchronization (Fig. 9.14) in order to protect each participant’s private data stored, for instance, in an internal ERP* sys- tem. Enterprise Resource Planning.
Table 9.4 Index types for BestPeer v2.0 in addition, it also comes with a native load balancing mechanism. To speed up query processing, be they range or otherwise, BestPeer v2.0 uses three types of indices: table index, column index, and data index. Table 9.4 shows their formats. The first two types (table and column) are rather straightforward: they maintain a list of peers that store that particular table or column. For the data index, the idea used by this system is to assign an ID to every column and to map this ID to a position in a bitmap. Then, when indexing a particular data value, the generated bitmap contains values of 1 only where the data appears as a value in the corresponding column.
Table 9.4 Index types for BestPeer v2.0 in addition, it also comes with a native load balancing mechanism. To speed up query processing, be they range or otherwise, BestPeer v2.0 uses three types of indices: table index, column index, and data index. Table 9.4 shows their formats. The first two types (table and column) are rather straightforward: they maintain a list of peers that store that particular table or column. For the data index, the idea used by this system is to assign an ID to every column and to map this ID to a position in a bitmap. Then, when indexing a particular data value, the generated bitmap contains values of 1 only where the data appears as a value in the corresponding column.
picts the architecture of an autonomous peer in Buddy Web. The web browser serves as the front-end interface to the user. Thanks to its transparency, users are not aware of any difference between the Buddy Web-enabled browser and the common one. In the Buddy Web enabled browser, there is a personal proxy that works with the local cache and a HTTP daemon to support HTTP re- quests. The cache, in collaboration with the BestPeer platform [234], is responsible for sharing cached data with other peers in the BuddyWeb network. The low level communication between peers is managed by the BestPeer platform. Whenever the web browser submits an URL query, the local proxy will receive and rewrite the query into the input format of the BestPeer platform. The query will then be passed to the BestPeer platform. Based on the requirement, BestPeer generates several mo- bile agents and dispatches them to the BuddyWeb network to search for matching documents. Upon receiving a match, BestPeer returns the information, i.e., docu- ment location, back to the personal proxy. In this way, the personal proxy will issue HTTP daemon directly to the peer that has the desired documents. The answer con- tributor, upon receiving the HTTP request, will process it by the HTTP daemon and send the requested documents to the requester.
picts the architecture of an autonomous peer in Buddy Web. The web browser serves as the front-end interface to the user. Thanks to its transparency, users are not aware of any difference between the Buddy Web-enabled browser and the common one. In the Buddy Web enabled browser, there is a personal proxy that works with the local cache and a HTTP daemon to support HTTP re- quests. The cache, in collaboration with the BestPeer platform [234], is responsible for sharing cached data with other peers in the BuddyWeb network. The low level communication between peers is managed by the BestPeer platform. Whenever the web browser submits an URL query, the local proxy will receive and rewrite the query into the input format of the BestPeer platform. The query will then be passed to the BestPeer platform. Based on the requirement, BestPeer generates several mo- bile agents and dispatches them to the BuddyWeb network to search for matching documents. Upon receiving a match, BestPeer returns the information, i.e., docu- ment location, back to the personal proxy. In this way, the personal proxy will issue HTTP daemon directly to the peer that has the desired documents. The answer con- tributor, upon receiving the HTTP request, will process it by the HTTP daemon and send the requested documents to the requester.
We envision an infrastructure for distributed information sharing among different parties based on Peer-to-Peer technology. Using distributed hash tables, the index- ing mechanism is both efficient and scalable, but challenges arise due to the hetero- geneous nature of the data as well as due to the potential security threats that are characteristic of peer-to-peer networks. We will address these shortly, but first let us consider why would we even want to use this approach.
We envision an infrastructure for distributed information sharing among different parties based on Peer-to-Peer technology. Using distributed hash tables, the index- ing mechanism is both efficient and scalable, but challenges arise due to the hetero- geneous nature of the data as well as due to the potential security threats that are characteristic of peer-to-peer networks. We will address these shortly, but first let us consider why would we even want to use this approach.
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved