Random Spanning Trees and the Prediction of Weighted Graphs

2012

The task of predicting the label of a network node, based on the labels of the remaining nodes, is an area of growing interest in machine learning, as various types of data are naturally represented as nodes in a graph. As an increasing number of methods and approaches are proposed to solve this task, the problem of comparing their performance becomes of key importance. In this paper we present an extensive experimental comparison of 15 different methods, on 15 different labelled-networks, as well as releasing all datasets and source code. In addition, we release a further set of networks that were not used in this study (as not all benchmarked methods could manage very large datasets). Besides the release of data, protocols and algorithms, the key contribution of this study is that in each of the 225 combinations we tested, the best performance—both in accuracy and running time—was achieved by the same algorithm: Online Majority Vote. This is also one of the simplest methods to imp...

2008

This short report analyses a simple and intuitive online learning algorithm -termed the graphtron -for learning a labeling over a fixed graph, given a sequence of labels. The contribution is twofold, (a) we give a theoretical characterization of the possible sequence of mistakes, and (b) we indicate the use for extremely large-scale problems due to sublinear space complexity and nearly linear time complexity.

The Annals of Applied Probability, 1999

For a complete graph K,, on n vertices with weighted edges, define the weight of a spanning tree (more generally, spanning forest) as the product of edge weights involved. Define the tree weight (forest weight) of Kn as the total weight of all spanning trees (forests). The uniform edge weight distribution is shown to maximize the tree weight, and an explicit bound on the tree weight is formulated in terms of the overall variance of edge weights as well as the variance of the sum of edge weights over nodes. An application to sparse random graphs leads to a bound for the relative risk of observing a spanning tree in a well-defined neighborhood of the uniform distribution. An analogous result shows that, for each positive integer k, the weight of all forests with k rooted trees is also maximized under the uniform distribution. A key ingredient for the latter result is a formula for the weight of forests of k rooted trees that generalizes Maxwell's rule for spanning trees. Our formulas also enable us to show that the number of trees in a random rooted forest is intrinsically divisible, that is, representable as a sum of n independent binary random variables ?j, with parameter (1 + A)-1, A's being the eigenvalues of the Kirchhoff matrix. This is directly analogous to the properties of the number of blocks in a random set partition (Harper), of the size of the random matching set (Godsil) and of the number of leaves in a random tree (Steele).

This paper introduces a novel cost sensitive weighted samples approach to a cascade of Graph Neural Networks for learning from imbalanced data in the graph structured input domain. This is shown to be very effective in addressing the effects of imbalanced data distribution on learning systems. The proposed idea is based on a weighting mechanism which forces the network to encode misclassified graphs (or nodes) more strongly. We evaluate the approach through an application to the well known Web spam detection problem, and demonstrate that the generalization performance is improved as a result. Indeed the results obtained reported in this paper are the best reported so far for both datasets.

Consider the setting of randomly weighted graphs, namely, graphs whose edge weights are chosen independently according to probability distributions with finite support over the nonnegative reals. Under this setting, properties of weighted graphs typically become random variables and we are interested in computing their statistical features. Unfortunately, this turns out to be computationally hard for some properties albeit the problem of computing them in the traditional setting of algorithmic graph theory is tractable. For example, there are well known efficient algorithms that compute the diameter of a given weighted graph, yet, computing the expected diameter of a given randomly weighted graph is #P-hard even if the edge weights are identically distributed.

2013

Large graphs abound in machine learning, data mining, and several related areas. A useful step towards analyzing such graphs is that of obtaining certain summary statistics - e.g., or the expected length of a shortest path between two nodes, or the expected weight of a minimum spanning tree of the graph, etc. These statistics provide insight into the structure of a graph, and they can help predict global properties of a graph. Motivated thus, we propose to study statistical properties of structured subgraphs (of a given graph), in particular, to estimate the expected objective function value of a combinatorial optimization problem over these subgraphs. The general task is very difficult, if not unsolvable; so for concreteness we describe a more specific statistical estimation problem based on spanning trees. We hope that our position paper encourages others to also study other types of graphical structures for which one can prove nontrivial statistical estimates.

Lecture Notes in Computer Science, 2009

Motivated by a problem of targeted advertising in social networks, we introduce and study a new model of online learning on labeled graphs where the graph is initially unknown, and the algorithm is free to choose the next vertex to predict. After observing that natural nonadaptive exploration/prediction strategies (like depth-first with majority vote) badly fail on simple binary labeled graphs, we introduce an adaptive strategy that performs well under the hypothesis that the vertices of the unknown graph (i.e., the members of the social network) can be partitioned into a few well-separated clusters within which labels are roughly constant (i.e., members in the same cluster tend to prefer the same products). Our algorithm is efficiently implementable and provably competitive against the best of these partitions.

Many social network applications face the following problem: given a network G = (V, E) with labels on a small subset O ⊂ V of nodes and an optional set of features on nodes and edges, predict the labels of the remaining nodes. Much research has gone into designing learning models and inference algorithms for accurate predictions in this setting. However, a core hurdle to any prediction effort is that for many nodes there is insufficient evidence for inferring a label. We propose that instead of focusing on the impossible task of providing high accuracy over all nodes, we should focus on selectively making the few node predictions which will be correct with high probability. Any selective prediction strategy will require that the scores attached to node predictions be well-calibrated. Our evaluations show that existing prediction algorithms are poorly calibrated. We propose a new method of training a graphical model using a conditional likelihood objective that provides better calibration than the existing joint likelihood objective. We augment it with a decoupled confidence model created using a novel unbiased training process. Empirical evaluation on two large social networks show that we are able to select a large number of predictions with accuracy as high as 95%, even when the best overall accuracy is only 40%.

Information Processing Letters, 2010

We present short proofs on the mistake bounds of the 1-nearest neighbor algorithm on an online prediction problem of path labels. The algorithm is one of key ingredients in the algorithm by Herbster, Lever, and Pontil for general graphs. Our proofs are combinatorial and naturally show that the algorithm works when the set of labels is not binary.

2010

Active learning algorithms for graph node classification select a subset L of nodes in a given graph. The goal is to minimize the mistakes made on the remaining nodes by a standard node classifier using L as training set. Bilmes and Guillory introduced a combinatorial quantity, Ψ * (L), and related it to the performance of the mincut classifier run on any given training set L. While no efficient algorithms for minimizing Ψ * are known, they show that simple heuristics for (approximately) minimizing it do not work well in practice. Building on previous theoretical results about active learning on trees, we show that exact minimization of Ψ * on suitable spanning trees of the graph yields an efficient active learner that compares well against standard baselines on real-world graphs.