General pedigrees can be encoded as Bayesian networks, where the common MPE query corresponds to ... more General pedigrees can be encoded as Bayesian networks, where the common MPE query corresponds to finding the most likely haplotype configuration. Based on this, a strategy for grid parallelization of a stateof-the-art Branch and Bound algorithm for MPE is introduced: independent worker nodes concurrently solve subproblems, managed by a Branch and Bound master node. The likelihood functions are used to predict subproblem complexity, enabling efficient automation of the parallelization process. Experimental evaluation on up to 20 parallel nodes yields very promising results and suggest the effectiveness of the scheme, solving several very hard problem instances. The system runs on loosely coupled commodity hardware, simplifying deployment on a larger scale in the future.
It is well known that computing relative approximations of weighted counting queries such as the ... more It is well known that computing relative approximations of weighted counting queries such as the probability of evidence in a Bayesian network, the partition function of a Markov network, and the number of solutions of a constraint satisfaction problem is NP-hard. In this paper, we settle therefore on an easier problem of computing highconfidence lower bounds and propose an algorithm based on importance sampling and Markov inequality for it. However, a straight-forward application of Markov inequality often yields poor lower bounds because it uses only one sample. We therefore propose several new schemes that extend it to multiple samples. Empirically, we show that our new schemes are quite powerful, often yielding substantially higher (better) lower bounds than state-of-the-art schemes.
One popular and efficient scheme for solving combinatorial optimization problems over graphical m... more One popular and efficient scheme for solving combinatorial optimization problems over graphical models exactly is depth-first Branch and Bound. However, when the algorithm exploits problem decomposition using AND/OR search spaces, its anytime behavior breaks down. This article 1) analyzes and demonstrates this inherent conflict between effective exploitation of problem decomposition (through AND/OR search spaces) and the anytime behavior of depthfirst search (DFS), 2) presents a new search scheme to address this issue while maintaining desirable DFS memory properties, and 3) analyzes and demonstrates its effectiveness through comprehensive empirical evaluation. Our work is applicable to any problem that can be cast as search over an AND/OR search space.
The paper presents a new look-ahead scheme for backtracking search for solving constraint satisfa... more The paper presents a new look-ahead scheme for backtracking search for solving constraint satisfaction problems. This look-ahead scheme computes a heuristic for value ordering and domain pruning. The heuristic is based on approximating the number of solutions extending each partial solution. In particular, we investigate a recent partitionbased approximation of tree-clustering algorithms, Iterative Join-Graph Propagation (IJGP), which belongs to the class of belief propagation algorithms that attracted substantial interest due to their success for probabilistic inference. Our empirical evaluation demonstrates that the counting-based heuristic approximated by IJGP yields a scalable, focused search.
Uncertainty in Artificial Intelligence, Jul 26, 2005
In this paper, we consider Hybrid Mixed Networks (HMN) which are Hybrid Bayesian Networks that al... more In this paper, we consider Hybrid Mixed Networks (HMN) which are Hybrid Bayesian Networks that allow discrete deterministic information to be modeled explicitly in the form of constraints. We present two approximate inference algorithms for HMNs that integrate and adjust well known algorithmic principles such as Generalized Belief Propagation, Rao-Blackwellised Importance Sampling and Constraint Propagation to address the complexity of modeling and reasoning in HMNs. We demonstrate the performance of our approximate inference algorithms on randomly generated HMNs.
Uncertainty in Artificial Intelligence, Aug 6, 2018
We present a new sampling scheme for approximating hard to compute queries over graphical models,... more We present a new sampling scheme for approximating hard to compute queries over graphical models, such as computing the partition function. The scheme builds upon exact algorithms that traverse a weighted directed state-space graph representing a global function over a graphical model (e.g., probability distribution). With the aid of an abstraction function and randomization, the state space can be compacted (or trimmed) to facilitate tractable computation, yielding a Monte Carlo Estimate that is unbiased. We present the general scheme and analyze its properties analytically and empirically, investigating two specific ideas for picking abstractions -targeting reduction of variance or search space size.
Computing the probability of evidence even with known error bounds is NP-hard. In this paper we a... more Computing the probability of evidence even with known error bounds is NP-hard. In this paper we address this hard problem by settling on an easier problem. We propose an approximation which provides high confidence lower bounds on probability of evidence but does not have any guarantees in terms of relative or absolute error. Our proposed approximation is a randomized importance sampling scheme that uses the Markov inequality. However, a straight-forward application of the Markov inequality may lead to poor lower bounds. We therefore propose several heuristic measures to improve its performance in practice. Empirical evaluation of our scheme with stateof-the-art lower bounding schemes reveals the promise of our approach.
European Conference on Artificial Intelligence, May 22, 2006
This paper extends previously proposed bound propagation algorithm for computing lower and upper ... more This paper extends previously proposed bound propagation algorithm for computing lower and upper bounds on posterior marginals in Bayesian networks. We improve the bound propagation scheme by taking advantage of the directionality in Bayesian networks and applying the notion of relevant subnetwork. We also propose an approximation scheme for the linear optimization subproblems. We demonstrate empirically that while the resulting bounds loose some precision, we achieve 10-100 times speedup compared to original bound propagation using a simplex solver.
National Conference on Artificial Intelligence, Aug 4, 1996
GSAT is a randomized greedy local repair procedure that was introduced for solving propositional ... more GSAT is a randomized greedy local repair procedure that was introduced for solving propositional satisfiability and constraint satisfaction problems. We present an improvement to GSAT that is sensitive to the problem's structure. When the problem has a tree structure the algorithm is guaranteed to find a solution in linear time. For non-tree networks, the algorithm designates a subset of nodes, called cutset, and executes a regular GSAT algorithm on this set of variables. On all the rest of the variables it executes a specialized local search algorithm for trees. This algorithm finds an assignment that, like GSAT, locally minimizes the sum of unsatisfied constraints and also globally minimizes the number of conflicts in every tree-like subnetwork. We will present results of experiments showing that this new algorithm outperforms regular GSAT on sparse networks whose cycle-cutset size is bounded by 3OYo of the nodes. 'For simplicity of exposition we restrict ourselves to binary constraint problems. However everything is applicable to the general CSP.
Computing the partition function is a key inference task in many graphical models. In this paper,... more Computing the partition function is a key inference task in many graphical models. In this paper, we propose a dynamic importance sampling scheme that provides anytime finite-sample bounds for the partition function. Our algorithm balances the advantages of the three major inference strategies, heuristic search, variational bounds, and Monte Carlo methods, blending sampling with search to refine a variationally defined proposal. Our algorithm combines and generalizes recent work on anytime search [16] and probabilistic bounds of the partition function. By using an intelligently chosen weighted average over the samples, we construct an unbiased estimator of the partition function with strong finite-sample confidence intervals that inherit both the rapid early improvement rate of sampling and the long-term benefits of an improved proposal from search. This gives significantly improved anytime behavior, and more flexible trade-offs between memory, time, and solution quality. We demonstrate the effectiveness of our approach empirically on real-world problem instances taken from recent UAI competitions.
This paper develops a measure for bounding the performance of AND/OR search algorithms for solvin... more This paper develops a measure for bounding the performance of AND/OR search algorithms for solving a variety of queries over graphical models. We show how drawing a connection to the recent notion of hypertree decompositions allows to exploit determinism in the problem specification and produce tighter bounds. We demonstrate on a variety of practical problem instances that we are often able to improve upon existing bounds by several orders of magnitude.
One popular and efficient scheme for solving exactly MPE/MAP and related problems over graphical ... more One popular and efficient scheme for solving exactly MPE/MAP and related problems over graphical models is depth-first Branch and Bound. However, when the algorithm exploits problem decomposition using AND/OR search spaces, its anytime behavior breaks down. This paper 1) analyzes and demonstrates this inherent conflict between effective exploitation of problem decomposition (through AND/OR search spaces) and the anytime behavior of depth-first search (DFS), 2) presents a first scheme to address this issue while maintaining desirable DFS memory properties, 3) analyzes and demonstrates its effectiveness. Our work is applicable to any problem that can be cast as search over an AND/OR search space.
The paper introduces AND/OR importance sampling for probabilistic graphical models. In contrast t... more The paper introduces AND/OR importance sampling for probabilistic graphical models. In contrast to importance sampling, AND/OR importance sampling caches samples in the AND/OR space and then extracts a new sample mean from the stored samples. We prove that AND/OR importance sampling may have lower variance than importance sampling; thereby providing a theoretical justification for preferring it over importance sampling. Our empirical evaluation demonstrates that AND/OR importance sampling is far more accurate than importance sampling in many cases.
International Joint Conference on Artificial Intelligence, Jul 31, 1999
The paper describes a branch and bound scheme that uses heuristics generated mechanically by the ... more The paper describes a branch and bound scheme that uses heuristics generated mechanically by the mini-bucket approximation. This scheme is presented and evaluated for optimization tasks such as finding the Most Probable Explanation (MPE ) in Bayesian networks. The mini-bucket scheme yields monotonic heuristics of varying strengths which cause different amounts of pruning, allowing a controlled tradeoff between preprocessing and search. The resulting Branch and Bound with Mini-Bucket heuristic (BBMB), is evaluated using random networks, probabilistic decoding and medical diagnosis networks. Results show that the BBMB scheme overcomes the memory explosion of bucket-elimination allowing a gradual tradeoff of space for time, and of time for accuracy.
The paper presents an iterative version of join-tree clustering that applies the message passing ... more The paper presents an iterative version of join-tree clustering that applies the message passing of join-tree clustering algorithm to join-graphs rather than to join-trees, itera tively. It is inspired by the success of Pearl's belief propagation algorithm (BP) as an it erative approximation scheme on one hand, and by a recently introduced mini-clustering (MC(i)) success as an anytime approximation method, on the other. The proposed Iterative Join-graph Propagation (IJGP) belongs to the class of generalized belief propagation meth ods, recently proposed using analogy with algorithms in statistical physics. Empirical evaluation of this approach on a number of problem classes demonstrates that even the most time-efficient variant is almost always superior to IBP and MC(i), and is sometimes more accurate by as much as several orders of magnitude.
We develop several algorithms taking advantage of two common approaches for bounding MPE queries ... more We develop several algorithms taking advantage of two common approaches for bounding MPE queries in graphical models: minibucket elimination and message-passing updates for linear programming relaxations. Both methods are quite similar, and offer useful perspectives for the other; our hybrid approaches attempt to balance the advantages of each. We demonstrate the power of our hybrid algorithms through extensive empirical evaluation. Most notably, a Branch and Bound search guided by the heuristic function calculated by one of our new algorithms has recently won first place in the PASCAL2 inference challenge.
This paper presents measures for upper and lower bounding the instancebased complexity of AND/OR ... more This paper presents measures for upper and lower bounding the instancebased complexity of AND/OR search algorithms for solution counting and related #P problems. This can be of utmost importance in selecting the right set of parameters for fitting an algorithm to a problem instance and in devising heuristics during execution. To this end we estimate the size of the search space, with special consideration given to the impact of determinism in a problem. The resulting schemes are evaluated empirically on a variety of problem instances; in many cases relatively tight bounds are obtained, far better than those implied by the tree width or hypertree width. Specific results are provided detailing how these measures can be useful for discriminating between variable orderings.
Proceedings of the AAAI Conference on Artificial Intelligence
The paper investigates the potential of look-ahead in the con-text of AND/OR search in graphical ... more The paper investigates the potential of look-ahead in the con-text of AND/OR search in graphical models using the Mini-Bucket heuristic for combinatorial optimization tasks (e.g., MAP/MPE or weighted CSPs). We present and analyze the complexity of computing the residual (a.k.a Bellman update) of the Mini-Bucket heuristic and show how this can be used to identify which parts of the search space are more likely to benefit from look-ahead and how to bound its overhead. We also rephrase the look-ahead computation as a graphical model, to facilitate structure exploiting inference schemes. We demonstrate empirically that augmenting Mini-Bucket heuristics by look-ahead is a cost-effective way of increasing the power of Branch-And-Bound search.
Best-first search can be regarded as an anytime scheme for producing lower bounds on the optimal ... more Best-first search can be regarded as an anytime scheme for producing lower bounds on the optimal solution, a characteristic that is mostly overlooked. In this paper we explore this topic in the context of AND/OR best-first search, guided by the mini-bucket heuristic, when solving graphical models. In that context, the impact of the secondary heuristic for subproblem ordering becomes apparent, especially when viewed in the anytime context. Specifically, we show how the concept of bucket errors, introduced recently, can yield effective subproblem orderings in AND/OR search and that it is often superior to the baseline approach which uses the same heuristic for node selection (OR nodes) and for subproblem orderings (AND nodes). Our experiments show an improvement in performance both for proving optimality and also in the anytime performance.
Annals of Mathematics and Artificial Intelligence, 2004
This paper presents new look-ahead schemes for backtracking search when solving constraint satisf... more This paper presents new look-ahead schemes for backtracking search when solving constraint satisfaction problems. The look-ahead schemes compute a heuristic for value ordering and domain pruning, which influences variable orderings at each node in the search space. As a basis for a heuristic, we investigate two tasks, both harder than the CSP task. The first is finding the solution with min-number of conflicts. The second is counting solutions. Clearly each of these tasks also finds a solution to the CSP problem, if one exists, or decides that the problem is inconsistent. Our plan is to use approximations of these more complex tasks as heuristics for guiding search for a solution of a CSP task. In particular, we investigate two recent partitionbased strategies that approximate variable elimination algorithms, Mini-Bucket-Tree Elimination and Iterative Join-Graph Propagation (ijgp). The latter belong to the class of belief propagation algorithm that attracted substantial interest due to their surprising success for probabilistic inference. Our preliminary empirical evaluation is very encouraging, demonstrating that the countingbased heuristic approximated by by IJGP yields a very focused search even for hard problems.
Uploads
Papers by Rina Dechter