Academia.eduAcademia.edu

Outline

Title
Abstract
FAQs
Figures
Introduction
Incomplete Data and Non-Random Sampling
Knowledge and Data Fusion for Mebn Structure
Summary and Discussion
References
All Topics
Mathematics
Mathematical Logic

Knowledge and data fusion in probabilistic networks

Profile image of Kathryn LaskeyKathryn Laskey

2003

Abstract

Probability theory provides the theoretical basis for a logically coherent process of combining prior knowledge with empirical data to draw plausible inferences and to refine theories as observations accrue. Increases in the expressive power of languages for expressing probabilistic theories have been accompanied by refinements and adaptations of Bayesian learning methods to handle the more expressive constructs. These innovations have established Bayesian learning as a unifying theoretical framework for learning in intelligent systems, and have given rise to practical techniques that are receiving wide application. This paper describes theory and methods for exact and approximate learning of probabilistic theories from a combination of background knowledge and observations. The concepts and methods can be adapted to any knowledge representation framework that can express probability distributions over interpretations of a first-order logic. We focus specifically on methods to learn theories that can be expressed in the Multi-Entity Bayesian Network (MEBN) probabilistic logic. MEBN logic is sufficiently general to represent a probability distribution over interpretations of any set of statements that can be expressed in firstorder predicate calculus. Bayesian inference provides both a proof theory for combining prior knowledge with observational evidence to derive plausible conclusions and a learning theory for refining a representation as observational evidence accrues. A formal specification is provided for the MEBN logic. A semantics is based on random variables provides a logically coherent foundation for open world reasoning. The paper describes modifications of standard Bayesian learning methods to handle the repeated structures that occur in MEBN theories. Methods are given for specifying domain knowledge as MEBN fragments with structure and parameter prior distributions.

FAQs

sparkles

AI

How does knowledge-data fusion enhance learning in probabilistic networks?add

The paper reveals that integrating expert knowledge with empirical data significantly improves the performance of intelligent agents by allowing them to adapt and refine their models effectively under diverse conditions.

What is the role of MEBN logic in probabilistic reasoning?add

MEBN logic extends Bayesian networks by allowing the representation of complex first-order theories, which enhances expressive power and modularity in learning probabilistic models.

How do Bayesian networks learn structure and parameters from data?add

The study discusses general methods that learn both the structural elements and parameters of Bayesian networks from observations, allowing for a coherent integration of expert guidance.

What methods facilitate knowledge integration in MEBN learning?add

The paper describes techniques such as using prior distributions over structures and parameters based on expert input to enhance the fusion of knowledge and data.

What challenges arise from incomplete data in MEBN logic?add

MEBN logic tackles the bias introduced by non-ignorable missing data through observability modeling and expert assessments to ensure accurate parameter estimates despite data limitations.

Figures (8)
Figure 1: Bayesian Network for Diagnostic Task
Figure 1: Bayesian Network for Diagnostic Task
Table 1: Correspondence Between MEBN and First-Order Logic Syntactic Elements Definition 2: Let V = {Vx}xgg be a symbol system with instance labels [], random variable labels
Table 1: Correspondence Between MEBN and First-Order Logic Syntactic Elements Definition 2: Let V = {Vx}xgg be a symbol system with instance labels [], random variable labels
Note that in the model of Figure 2, all the arguments of parent random variables are functionally related to the arguments of their children (e.g., the conditioning constraint r=MachineLocation(m) specifies a functional relationship between the argument of RoomTemp(r) and EngineStatus(m)). This means that each local distribution can be specified just as in an ordinary Bayesian network, by defining a conditional distribution on states of child nodes given Figure 2: Multi-Entity Bayesian Network for Diagnostic Task
Note that in the model of Figure 2, all the arguments of parent random variables are functionally related to the arguments of their children (e.g., the conditioning constraint r=MachineLocation(m) specifies a functional relationship between the argument of RoomTemp(r) and EngineStatus(m)). This means that each local distribution can be specified just as in an ordinary Bayesian network, by defining a conditional distribution on states of child nodes given Figure 2: Multi-Entity Bayesian Network for Diagnostic Task
Figure 3: MEBN Fragment Combining Influences from Multiple Parent Instance
Figure 3: MEBN Fragment Combining Influences from Multiple Parent Instance
For many commonly applied distributions, there is an additive decomposition of [](x,.c, into separate components for each element [}; of a parameter vector. When the prior hyperparameter exhibits the same additive decomposition, we say the problem exhibits local parameter independence. When there is local parameter independence, the learning problem can be decomposed into separate learning problems for each parameter of the local distribution. When the local distributions are exponential family distributions and the prior distributions are conjugate distributions that decompose additively, there is a natural approach to knowledge- data fusion in which we regard the domain expert as a source of data and represent his or her input as a collection of sufficient statistics, one for each of the local distributions to be specified. When data relevant to a parameter vector [}, are expected to be plentiful or when inferences of interest are relatively insensitive to [],, we can economize on expert effort by using a standard non-informative prior distribution, i.e., one in which the number of “virtual prior instances” represented by the hyperparameter [}, is small. When there is stronger prior information relative to the amount of data, informative prior distributions can be elicited. The effort invested in modeling and eliciting prior distributions for different parts of the model can be adjusted according to the sensitivity of inferences of interest to specification of the prior distribution. we ae ~ wy “aplag ‘i. a
For many commonly applied distributions, there is an additive decomposition of [](x,.c, into separate components for each element [}; of a parameter vector. When the prior hyperparameter exhibits the same additive decomposition, we say the problem exhibits local parameter independence. When there is local parameter independence, the learning problem can be decomposed into separate learning problems for each parameter of the local distribution. When the local distributions are exponential family distributions and the prior distributions are conjugate distributions that decompose additively, there is a natural approach to knowledge- data fusion in which we regard the domain expert as a source of data and represent his or her input as a collection of sufficient statistics, one for each of the local distributions to be specified. When data relevant to a parameter vector [}, are expected to be plentiful or when inferences of interest are relatively insensitive to [],, we can economize on expert effort by using a standard non-informative prior distribution, i.e., one in which the number of “virtual prior instances” represented by the hyperparameter [}, is small. When there is stronger prior information relative to the amount of data, informative prior distributions can be elicited. The effort invested in modeling and eliciting prior distributions for different parts of the model can be adjusted according to the sensitivity of inferences of interest to specification of the prior distribution. we ae ~ wy “aplag ‘i. a
= ~ — Identifying and adjusting for biases due to observability is an important role for expert knowledge in knowledge-data fusion. Sensitivity analysis (Laskey, 1995; Castillo, et al., 1997; Coupe and van der Gaag, 1999) can be used to determine bounds on the potential bias due to systematically missing observations. When the bias may be large, the expert can be asked to construct a model of the observability process and to provide judgmental assessments of the parameters of the observability model. These judgmental assessments could be used directly, or they could be used to plan additional data collection efforts designed to estimate the degree of bias. Modeling the observation process and collecting additional data to estimate parameters of the observability model can dramatically reduce the bias due to systematically missing observations.
= ~ — Identifying and adjusting for biases due to observability is an important role for expert knowledge in knowledge-data fusion. Sensitivity analysis (Laskey, 1995; Castillo, et al., 1997; Coupe and van der Gaag, 1999) can be used to determine bounds on the potential bias due to systematically missing observations. When the bias may be large, the expert can be asked to construct a model of the observability process and to provide judgmental assessments of the parameters of the observability model. These judgmental assessments could be used directly, or they could be used to plan additional data collection efforts designed to estimate the degree of bias. Modeling the observation process and collecting additional data to estimate parameters of the observability model can dramatically reduce the bias due to systematically missing observations.
A random variable for which there are no observations at all is called a hidden or Jatent variable. Latent variables may correspond to observable features for which no data are available Fre: ioe eal ARR Atma that se wae Sam balan
A random variable for which there are no observations at all is called a hidden or Jatent variable. Latent variables may correspond to observable features for which no data are available Fre: ioe eal ARR Atma that se wae Sam balan

Related papers

Probabilistic Logic Programming and Bayesian Networks
Peter Haddawy

1995

We present a probabilistic logic programming framework that allows the representation of conditional probabilities. While conditional probabilities are the most commonly used method for representing uncertainty in probabilistic expert systems, they have been largely neglected by work in quantitative logic programming. We define a fixpoint theory, declarative semantics, and proof procedure for the new class of probabilistic logic programs. Compared to other approaches to quantitative logic programming, we provide a true probabilistic framework with potential applications in probabilistic expert systems and decision support systems. We also discuss the relationship between such programs and Bayesian networks, thus moving toward a unification of two major approaches to automated reasoning.

View PDFchevron_right
Knowledge Representation with Possibilstic and Certain Bayesian Networks
Mohamed Nazih OMRI

2005

Possibilistic logic and Bayesian networks have provided advantageous methodologies and techniques for computerbased knowledge representation. This paper proposes a framework that combines these two disciplines to exploit their own advantages in uncertain and imprecise knowledge representation problems. The framework proposed is a possibilistic logic based one in which Bayesian nodes and their properties are represented by local necessity-valued knowledge base. Data in properties are interpreted as set of valuated formulas. In our contribution possibilistic Bayesian networks have a qualitative part and a quantitative part, represented by local knowledge bases. The general idea is to study how a fusion of these two formalisms would permit representing compact way to solve efficiently problems for knowledge representation. We show how to apply possibility and necessity measures to the problem of knowledge representation with large scale data.

View PDFchevron_right
Some Useful Properties of Probabilistic Knowledge Representations From the Point of View of Intelligent Systems
Marek Druzdzel

Although probabilistic knowledge representations and probabilistic reasoning have by now secured their position in intelligent systems research, it is not uncommon to encounter misunderstanding of their foundations and lack of appreciation for their strengths. This paper discusses ve issues related to intelligent systems research and shows how they are addressed by the probabilistic knowledge representations. Directed probabilistic graphs capture essential qualitative properties of a domain, along with its causal structure. Concepts such as relevance and con icting evidence have a natural, formally sound meaning in probabilistic models. Probabilistic schemes support sound reasoning at a variety of levels ranging from purely quantitative to purely qualitative levels. Probabilistic knowledge representations provide insight into the foundations of logic-based schemes for reasoning under uncertainty, showing their di culties in highly uncertain domains. Finally, probabilistic knowledge representations support automatic generation of understandable explanations of inference for the sake of user interfaces to intelligent systems.

View PDFchevron_right
Probabilistic Logics and Probabilistic Networks
Gregory Wheeler

2011

While in principle probabilistic logics might be applied to solve a range of problems, in practice they are rarely applied at present. This is perhaps because they seem disparate, complicated, and computationally intractable. However, we shall argue in this programmatic paper that several approaches to probabilistic logic fit into a simple unifying framework: logically complex evidence can be used to associate probability intervals or probabilities with sentences.

View PDFchevron_right
Multi-entity bayesian networks without multi-tears
Kathryn Laskey

Draft Version, 2005

An introduction is provided to Multi-Entity Bayesian Networks (MEBN), a logic system that integrates First Order Logic (FOL) with Bayesian probability theory. MEBN extends ordinary Bayesian networks to allow representation of graphical models with repeated sub-structures. Knowledge is encoded as a collection of Bayesian network fragments (MFrags) that can be instantiated and combined to form highly complex situation-specific Bayesian networks. A MEBN theory (MTheory) implicitly represents a joint probability distribution over possibly unbounded numbers of hypotheses, and uses Bayesian learning to refine a knowledge base as observations accrue. MEBN provides a logical foundation for the emerging collection of highly expressive probability-based languages. A running example illustrates the representation and reasoning power of the MEBN formalism.

View PDFchevron_right
Fusing multiple Bayesian knowledge sources
Eugene Santos

International Journal of Approximate Reasoning, 2011

We address the problem of information fusion in uncertain environments. Imagine there are multiple experts building probabilistic models of the same situation and we wish to aggregate the information they provide. There are several problems we may run into by naively merging the information from each. For example, the experts may disagree on the probability of a certain event or they may disagree on the direction of causality between two events (e.g., one thinks A causes B while another thinks B causes A). They may even disagree on the entire structure of dependencies among a set of variables in a probabilistic network. In our proposed solution to this problem, we represent the probabilistic models as Bayesian Knowledge Bases (BKBs) and propose an algorithm called Bayesian knowledge fusion that allows the fusion of multiple BKBs into a single BKB that retains the information from all input sources. This allows for easy aggregation and de-aggregation of information from multiple expert sources and facilitates multi-expert decision making by providing a framework in which all opinions can be preserved and reasoned over.

View PDFchevron_right
Five Useful Properties of Probabilistic Knowledge Representations From the Point of View of Intelligent Systems
Marek Druzdzel

Fundamenta Informaticae

Although probabilistic knowledge representations and probabilistic reasoning have by now secured their position in arti cial intelligence, it is not uncommon to encounter misunderstanding of their foundations and lack of appreciation for their strengths. This paper describes ve properties of probabilistic knowledge representations that are particularly useful in intelligent systems research. (1) Directed probabilistic graphs capture essential qualitative properties of a domain, along with its causal structure. (2) Concepts such as relevance and con icting evidence have a natural, formally sound meaning in probabilistic models. (3) Probabilistic schemes support sound reasoning at a variety of levels ranging from purely quantitative to purely qualitative levels. (4) The role of probability theory in reasoning under uncertainty can be compared to the role of rst order logic in reasoning under certainty. Probabilistic knowledge representations provide insight into the foundations of logic-based schemes, showing their di culties in highly uncertain domains. Finally, (5) probabilistic knowledge representations support automatic generation of understandable explanations of inference for the sake of user interfaces to intelligent systems.

View PDFchevron_right
Combining Subjective Probabilities and Data in Training Markov Logic Networks
Shalini Ghosh

Machine Learning and Knowledge Discovery in Databases, 2012

Markov logic is a rich language that allows one to specify a knowledge base as a set of weighted first-order logic formulas, and to define a probability distribution over truth assignments to ground atoms using this knowledge base. Usually, the weight of a formula cannot be related to the probability of the formula without taking into account the weights of the other formulas. In general, this is not an issue, since the weights are learned from training data. However, in many domains (e.g. healthcare, dependable systems, etc.), only little or no training data may be available, but one has access to a domain expert whose knowledge is available in the form of subjective probabilities. Within the framework of Bayesian statistics, we present a formalism for using a domain expert's knowledge for weight learning. Our approach defines priors that are different from and more general than previously used Gaussian priors over weights. We show how one can learn weights in an MLN by combining subjective probabilities and training data, without requiring that the domain expert provides consistent knowledge. Additionally, we also provide a formalism for capturing conditional subjective probabilities, which are often easier to obtain and more reliable than non-conditional probabilities. We demonstrate the effectiveness of our approach by extensive experiments in a domain that models failure dependencies in a cyber-physical system. Moreover, we demonstrate the advantages of using our proposed prior over that of using non-zero mean Gaussian priors in a commonly cited social network MLN testbed.

View PDFchevron_right
Logical Bayesian Networks and Their Relation to Other Probabilistic Logical Models
Jan Ramon

Inductive Logic …, 2005

View PDFchevron_right
First-order Bayesian logic
Kathryn Laskey

2005

Uncertainty is a fundamental and irreducible aspect of our knowledge about the world. Until recently, classical first-order logic has reigned as the de facto standard logical foundation for artificial intelligence. The lack of a built-in, semantically grounded capability for reasoning under uncertainty renders classical first-order logic inadequate for many important classes of problems. General-purpose languages are beginning to emerge for which the fundamental logical basis is probability. Increasingly expressive probabilistic languages demand a theoretical foundation that fully integrates classical first-order logic and probability. In first-order Bayesian logic (FOBL), probability distributions are defined over interpretations of classical first-order axiom systems. Predicates and functions of a classical first-order theory correspond to a random variables in the corresponding first-order Bayesian theory. This is a natural correspondence, given that random variables are formalized in mathematical statistics as measurable functions on a probability space. A formal system called Multi-Entity Bayesian Networks (MEBN) is presented for composing distributions on interpretations by instantiating and combining parameterized fragments of directed graphical models. A construction is given of a MEBN theory that assigns a non-zero probability to any satisfiable sentence in classical first-order logic. By conditioning this distribution on consistent sets of sentences, FOBL can represent a probability distribution over interpretations of any finitely axiomatizable first-order theory, as well as over interpretations of infinite axiom sets when a limiting distribution exists. FOBL is inherently open, having the ability to incorporate new axioms into existing theories, and to modify probabilities in the light of evidence. Bayesian inference provides both a proof theory for combining prior knowledge with observations, and a learning theory for refining a representation as evidence accrues. The results of this paper provide a logical foundation for the rapidly evolving literature on first-order Bayesian knowledge representation, and point the way toward Bayesian languages suitable for generalpurpose knowledge representation and computing. Because FOBL contains classical first-order logic as a deterministic subset, it is a natural candidate as a universal representation for integrating domain ontologies expressed in languages based on classical first-order logic or subsets thereof.

View PDFchevron_right

References (90)

  1. Fahiem Bacchus. Representing and Reasoning with Probabilistic Knowledge. MIT Press, Cambridge, Massachusetts, 1990.
  2. Fahiem Bacchus, Adam Grove, Joseph Y. Halpern and Daphne Koller. From statistical knowledge bases to degrees of belief. Artificial Intelligence 87: 75-143, 1997.
  3. Olav Bangsø and Pierre-Henri Wuillemin. Object Oriented Bayesian Networks: A Framework for Topdown Specification of Large Bayesian Networks and Repetitive Structures. Technical Report CIT-87.2-00-obphw1, Department of Computer Science, Aalborg University, 2000.
  4. Olav Bangsø, Helge Langseth, and Thomas Nielsen. Structural Learning in Object Oriented Domains. In Proceedings of the Thirteenth International Florida Artificial Intelligence Research Society Conference, pages 282-286. AAAI Press, Menlo Park, California, 2001.
  5. Matthew J. Beal and Zoubin Gharamani, The variational Bayesian EM algorithm for incomplete data: with application to scoring graphical model structures, Proceedings of the Seventh Valencia Conference on Bayesian Statistics, in press. Available electronically from http://www.gatsby.ucl.ac.uk/~zoubin/papers.html.
  6. Thomas Binford and Tod S. Levitt. Evidential reasoning for object recognition, I E E E Transactions Pattern Analysis and Machine Intelligence, in press.
  7. Xavier Boyen and Daphne Koller. Tractable inference for complex stochastic processes. In Proceedings of the Conference on Uncertainty in Artificial Intelligence, 1998.
  8. Michael P. Brown, Richard Hughey, Anders Krogh, I. Saira Mian, Kimmen Sjolander, and David Haussler. Using Dirichlet Mixture Priors to Derive Hidden Markov Models for Protein Families. In Proceedings of the First International Conference on Intelligent Systems for Molecular Biology, pages 47-55, AAAI/MIT Press, Menlo Park, California, 1993.
  9. Wray Buntine. Operations for Learning with Graphical Models. Journal of Artificial Intelligence Research, 2: 159-225, 1994.
  10. Enrique Castillo, José Manuel Gutiérrez, and Ali S. Hadi. Sensitivity Analysis in Discrete Bayesian Networks. IEEE Transactions on Systems, Man and Cybernetics, 27: 412-423, 1997. Eugene Charniak and Robert Goldman. A Bayesian Model of Plan Recognition. Artificial Intelligence, 64: 53-79, 1993.
  11. Ming-Hui Chen, Qui-Man Shao, and Joseph G. Ibrahim. Monte Carlo Methods in Bayesian Computation. Springer-Verlag, New York, New York, 2000.
  12. Gregory Cooper and Edward Herskovits. A Bayesian Method for the Induction of Probabilistic Networks from Data. Machine Learning, 9: 309-347, 1992.
  13. Veerle M. H. Coupe and Linda C. van der Gaag. Properties of Sensitivity Analysis of Bayesian Belief Networks. Universitet Utrecht UU-CS-1999-29, 1999.
  14. Robert G. Cowell, A. Phillip Dawid, and David J. Spiegelhalter. Sequential Model Criticism in Probabilistic Expert Systems. IEEE Transactions on Pattern Analysis and Machine Intelligence,15(3): 209-219, 1993.
  15. Robert G. Cowell, A. Phillip Dawid, Steffen L. Lauritzen, and David J. Spiegelhalter, Probabilistic Networks and Expert Systems. Springer-Verlag, New York, NY, 1999.
  16. Bruce d'Ambrosio. Local expression languages for probabilistic dependency. In Proceedings of the Seventh Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann Publishers, San Mateo, California, 1991.
  17. Bruce d'Ambrosio, Masami Takikawa, and Daniel Upper. Dynamic Situation Modeling. In DARPA Information Survivability Conference and Exposition, 2001. Available electronically via http://www.iet.com/Projects/jspi/.
  18. Ernest Davis. Representations of Commonsense Knowledge. Morgan Kaufmann Publishers, Inc., San Mateo, CA, 1990.
  19. A. Philip Dawid. Statistical Theory: the Prequential Approach. Journal of the Royal Statistical Society, A 147: 278-292, 1984.
  20. Morris DeGroot and Mark J. Schervish. Probability and Statistics (3 rd edition). Addison Wesley, Boston, Massachusetts, 2002.
  21. Arthur P. Dempster, Nan M. Laird, and Donald Rubin. Maximum Likelihood Estimation from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society, 39: 1-38, 1977.
  22. David Draper. Assessment and propagation of model uncertainty (with discussion). Journal of the Royal Statistical Society, B 57: 45-97, 1995.
  23. Marek J. Druzdzel and Fransisco Diaz. Criteria for Combining Knowledge from Different Sources in Probabilistic Models. In Notes from the Workshop on Fusion of Domain Knowledge with Data for Decision Support, Sixteenth Conference on Uncertainty in Artificial Intelligence, Stanford, California, 2000.
  24. Marek J. Druzdzel and Herbert A. Simon. Causality in Bayesian belief networks. In Proceedings of the Ninth Annual Conference on Uncertainty in Artificial Intelligence (UAI-93), pp. 3-11, Morgan Kaufmann Publishers, Inc., San Francisco, CA, 1993.
  25. Marek J. Druzdzel and Linda C. van der Gaag. Building probabilistic networks: "Where do the numbers come from?" IEEE Transactions on Knowledge and Data Engineering, 12: 481-486, 2000.
  26. Robert J. Elliott, Lakhdar Aggoun and John B. Moore. Hidden Markov Models: Estimation and Control. Springer-Verlag, New York, New York, 1995.
  27. Nir Friedman. The Bayesian Structural EM Algorithm. In Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann Publishers, San Mateo, California, 1998.
  28. Nir Friedman. Learning belief networks in the presence of missing values and hidden variables. In Proceedings of the Fourteenth International Conference on Machine Learning, Morgan Kaufmann Publishers, San Mateo, California, 1997.
  29. Nir Friedman and Moises Goldszmidt. Sequential update of Bayesian network structure. In Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann Publishers, San Mateo, California, 1997.
  30. Nir Friedman and Moises Goldszmidt. Learning Bayesian Networks with Local Structure. In Proceedings of the Twelfth Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann Publishers, San Mateo, California, 1996.
  31. Nir Friedman and Daphne Koller. Being Bayesian about Network Structure. In Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann Publishers, San Mateo, California, 2000.
  32. Daniel Geiger and David Heckerman. Advances in probabilistic reasoning. In Proceedings of the Seventh Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann Publishers, San Mateo, California, 1991.
  33. Andrew Gelman, John B. Carlin, Hal S. Stern, and Donald B. Rubin. Bayesian Data Analysis. Chapman and Hall, London, 1995.
  34. Michael R. Genesereth and Nils J. Nilsson. Logical Foundations of Artificial Intelligence. Morgan Kaufmann Publishers, San Mateo, California, 1987.
  35. Christian Genest and J. Zidek. Combining probability distributions: A critique and annotated bibliography. Statistical Science 1(1), 114-148, 1986.
  36. Lise Getoor, Nir Friedman, Daphne Koller, and Avi Pfeffer. Learning Probabilistic Relational Models. In Saso Dzeroski and Nada Lavrac, editors. Relational Data Mining, Springer- Verlag, New York, New York, 2001.
  37. Lise Getoor, Daphne Koller, Benjamin Taskar, and Nir Friedman. Learning Probabilistic Relational Models with Structural Uncertainty. In Proceedings of the ICML-2000 Workshop on Attribute-Value and Relational Learning:Crossing the Boundaries, Stanford, California, 2000.
  38. W. Gilks, A. Thomas, and David J. Spiegelhalter. A language and program for complex Bayesian modeling. The Statistician, 43: 169-178, 1994.
  39. W. Gilks, Sylvia Richardson, and David J. Spiegelhalter. Markov Chain Monte Carlo in Practice. Chapman and Hall, London, 1996.
  40. Ulf Grenander. Elements of Pattern Theory. Johns Hopkins University Press, Baltimore, MD, 1995.
  41. Liang Gu and Kenneth Rose, Sub-state tying in tied mixture hidden Markov models, Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, pp.1013-1016, Istanbul, Turkey, Jun. 2000.
  42. Robin Hanson. Combinatorial information market design. Information Systems Frontiers 5(1):105-119, 2003.
  43. David Heckerman. Probabilistic Similarity Networks. MIT Press, Cambridge, MA, 1991.
  44. David Heckerman. A Tutorial on Learning with Bayesian Networks. Microsoft, Redmond, Washington, 1996.
  45. David Heckerman, Daniel Geiger, and David Maxwell Chickering. Learning Bayesian Networks: The Combination of Knowledge and Statistical Data. Machine Learning, 20: 197-243, 1995.
  46. Jennifer Hoeting, David Madigan, Adrian Raftery, and Chris Volinsky. Bayesian Model Averaging. Statistical Science, 14(4): 382-417, 1999.
  47. William H. Jefferys and James O. Berger, Ockham's razor and Bayesian analy\sis, American Scientist 80: 64-72, 1992.
  48. Finn V. Jensen. Bayesian Networks and Decision Graphs. Springer-Verlag, New York, New York, 2001.
  49. Michael I. Jordan, editor. Learning in Graphical Models. MIT Press, Cambridge, Massachussets, 1999.
  50. Robert Kass and Adrian Raftery. Bayes Factors. Journal of the American Statistical Association, 90(430): 773-795, 1995.
  51. Daphne Koller and Avi Pfeffer. Object-Oriented Bayesian Networks. In Uncertainty in Artificial Intelligence: Proceedings of the Thirteenth Conference, pages 302-313, Morgan Kaufmann Publishers, San Francisco, California, 1997.
  52. Helge Langseth and Olav Bangsø, Parameter learning in object oriented Bayesian networks. Annals of Mathematics and Artificial Intelligence, 31(1/4): 221-243, 2001.
  53. Helge Langseth and Thomas Nielsen. Fusion of Domain Knowledge with Data for Structured Learning in Object-Oriented Domains. Journal of Machine Learning Research, 2003
  54. Kathryn B. Laskey. Learning Extensible Multi-Entity Directed Graphical Models. In Proceedings of the Workshop on Artificial Intelligence and Statistics, 1999.
  55. Kathryn B. Laskey. Sensitivity Analysis for Probability Assessments in Bayesian Networks. IEEE Transactions in Systems, Man and Cybernetics, 25(6): 901-909,1995.
  56. Kathryn B. Laskey, Bruce d'Ambrosio, Tod Levitt, and Suzanne M. Mahoney. Limited Rationality in Action: Decision Support for Military Situation Assessment. Minds and Machines, 10(1): 53-77, 2000.
  57. Kathryn B. Laskey and Suzanne M. Mahoney. Network Engineering for Agile Belief Network Models. IEEE Transactions in Knowledge and Data Engineering, 2000.
  58. Kathryn B. Laskey and Suzanne M. Mahoney. Network Fragments: Representing Knowledge for Constructing Probabilistic Models. In Uncertainty in Artificial Intelligence: Proceedings of the Thirteenth Conference, Morgan Kaufmann Publishers, San Mateo, California, 1997.
  59. Kathryn B. Laskey, Suzanne M. Mahoney, and Edward Wright. Hypothesis Management in Situation-Specific Network Construction. In Uncertainty in Artificial Intelligence: Proceedings of the Seventeenth Conference, Morgan Kaufmann Publishers, San Mateo, California, 2001.
  60. Kathryn B. Laskey and James M. Myers. Population Markov Chain Monte Carlo. Machine Learning, 50(1): 175-196, 2003.
  61. Stephan Lauritzen. Graphical Models. Oxford Science Publications, Oxford, 1996. Stephan Lauritzen. The EM algorithm for graphical association models with missing data. Computational Statistics & Data Analysis, 19: 191-201, 1995.
  62. Tod S. Levitt, C. Larrabee Winter, Charles J. Turner, Richard A. Chestek, Gil J. Ettinger, and Steve M. Sayre. Bayesian Inference-Based Fusion of Radar Imagery, Military Forces and Tactical Terrain Models in the Image Exploitation System/Balanced Technology Initiative. International Journal of Human-Computer Studies, 42, 1995.
  63. Roderick JA Little and Donald B. Rubin. Statistical Analysis with Missing Data. Wiley, New York, NY, 1987.
  64. David Madigan and Jeremy York. Bayesian Graphical Models for Discrete Data. International Statistical Review, 63: 215-232, 1995.
  65. Suzanne M. Mahoney. Network Fragments. PhD thesis, George Mason University, Fairfax, Virginia, 1999.
  66. Suzanne M. Mahoney and Kathryn B. Laskey. Representing and Combining Partially Specified Conditional Probability Tables. In Uncertainty in Artificial Intelligence: Proceedings of the Fifteenth Conference, Morgan Kaufmann Publishers, San Mateo, California, 1999.
  67. Suzanne M. Mahoney and Kathryn B. Laskey. Constructing Situation Specific Networks. In Uncertainty in Artificial Intelligence: Proceedings of the Fourteenth Conference, Morgan Kaufmann Publishers, San Mateo, California, 1998.
  68. Ron Musick. Minimal Assumption Distribution Propagation in Belief Networks. In Proceedings of the Ninth Conference on Uncertainty in Artificial Intelligence, pages 251-258, Morgan Kaufmann Publishers, San Mateo, California, 1993.
  69. Liem Ngo and Peter Haddawy. Answering Queries from Context-Sensitive Probabilistic Knowledge Bases. Theoretical Computer Science, 171:147, 1996.
  70. Richard E. Neapolitan and James R. Kenenvan. Investigation of Variances in Belief Networks. In Proceedings of the 7th Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann Publishers, San Mateo, California, 1991.
  71. Ann Nicholson, Tal Boneh, Tim Wilkin, Kaye Stacey, Liz Sonenberg, and Vicki Steinle. A Case Study in Knowledge Discovery and Elicitation in an Intelligent Tutoring Application. In Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann Publishers, San Mateo, California, 2001.
  72. Agnieszka Onisko, Marek J. Druzdzel, and Hanna Wasyluk. Learning Bayesian Network Parameters from Small Data Sets: Application of Noisy-OR gates. In Working Notes of the Workshop on Bayesian and Causal Networks: From Inference to Data Mining, 12th European Conference on Artificial Intelligence, Berlin, Germany. Available online at http://www.pitt.edu/~druzdzel/publ.html, 2000.
  73. Judea Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers, San Francisco, California, 1988.
  74. Judea Pearl. Causality. Cambridge University Press, Cambridge, UK, 2000.
  75. Kim-Leng Poh. Utility-based Categorization. PhD thesis, Stanford University, Stanford, California, 1993.
  76. Marco Ramoni and Paola Sebastiani. Robust learning with missing data. Machine Learning, 45: 147-170, 2001.
  77. Christian Robert. The Bayesian Choice (2 nd edition). Springer-Verlag, New York, New York, 2001. Christian Robert and George Casella. Monte Carlo Statistical Methods. Springer-Verlag, New York, New York, 1999.
  78. Stuart Russell. Rationality and Intelligence. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, 1995.
  79. Stuart Russell and Peter Norvig. Artificial Intelligence: A Modern Approach. Prentice-Hall, New York, New York, 1995.
  80. Stuart Russell and Devika Subramanian. Provably Bounded Optimal Agents. Journal of Artificial Intelligence Research, 2, 1995.
  81. David Schum. Evidential Foundations of Probabilistic Reasoning. Wiley, New York, New York, 1994.
  82. Dana S. Scott. A type-theoretic alternative to CUCH, ISWIM, OWHY. Manuscript, 1969. Later published in Theoretical Computer Science 121: 411-440, 1993.
  83. John F. Sowa. Knowledge Representation: Logical, Philosophical, and Computational Foundations. Pacific Grove, California, Brooks/Cole Thomson Learning, 2000.
  84. David J. Spiegelhalter and Adrian Smith. Bayes Factors and Choice Criteria for Linear Models. Journal of the Royal Statistical Society B, 42: 213-220, 1980.
  85. David J. Spiegelhalter, Andrew Thomas, and Nicky Best. Computation on Graphical Models. Bayesian Statistics, 5: 407-425,1996.
  86. Harald Steck and Tommi Jaakkola. On the Dirichlet prior and Bayesian regularization. Proceedings of the Neural Information Processing Society, 2002a.
  87. Harald Steck and Tommi Jaakola. Unsupervised active learning in large domains. Proceedings of the Seventeenth Annual Conference on Uncertainty in Artificial Intelligence, 2002b.
  88. Masami Takikawa, Bruce d'Ambrosio and Ed Wright, Real-time inference with large-scale temporal Bayes nets, Proceedings of the Conference on Uncertainty in Artificial Intelligence, 2001. Henri Thiel. Principles of Econometrics. Wiley, New York, New York, 1971.
  89. Manfred Warmuth and Olivier Bousquet. Tracking a small set of experts by mixing past posteriors, Journal of Machine Learning Research 3:363-296, 2002.
  90. Joe Whittaker.. Graphical Models in Applied Multivariate Statistics. Wiley, Chichester, 1990.

Related papers

MEBN A Language for First-Order Bayesian Knowledge Bases
Kathryn Laskey

Although classical first-order logic is the de facto standard logical foundation for artificial intelligence, the lack of a built-in, semantically grounded capability for reasoning under uncertainty renders it inadequate for many important classes of problems. Probability is the bestunderstood and most widely applied formalism for computational scientific reasoning under uncertainty. Increasingly expressive languages are emerging for which the fundamental logical basis is probability. This paper presents Multi-Entity Bayesian Networks (MEBN), a first-order language for specifying probabilistic knowledge bases as parameterized fragments of Bayesian networks. MEBN fragments (MFrags) can be instantiated and combined to form arbitrarily complex graphical probability models. An MFrag represents probabilistic relationships among a conceptually meaningful group of uncertain hypotheses. Thus, MEBN facilitates representation of knowledge at a natural level of granularity. The semantics of MEBN assigns a probability distribution over interpretations of an associated classical first-order theory on a finite or countably infinite domain. Bayesian inference provides both a proof theory for combining prior knowledge with observations, and a learning theory for refining a representation as evidence accrues. A proof is given that MEBN can represent a probability distribution on interpretations of any finitely axiomatizable first-order theory.

View PDFchevron_right
Probabilistic Logic for Belief Nets
John Hooker

1990

We describe how to combine probabilistic logic and Bayesian networks to obtain a new framework (\Bayesian logic") for dealing with uncertainty and causal relationships in an expert system. Probabilistic logic, invented by Boole, is a technique for drawing inferences from uncertain propositions for which there are no independence assumptions. A Bayesian network is a \belief net" that can represent complex conditional independence assumptions. We show how to solve inference problems in Bayesian logic by applying Benders decomposition to a nonlinear programming formulation. We also show that the number of constraints grows only linearly with the problem size for a large class of networks.

View PDFchevron_right
Representing and Reasoning With Probabilistic Knowledge: A Bayesian Approach
Marie desJardins

PAGODA (Probabilistic Autonomous Goal-Directed Agent) is a model for autonomous learning in probabilistic domains [desJardins, 1992] that incorporates innovative techniques for using the agent's existing knowledge to guide and constrain the learning process and for representing, reasoning with, and learning probabilistic knowledge. This paper describes the probabilistic representation and inference mechanism used in PAGODA. PAGODA forms theories about the effects of its actions and the world state on the environment over time. These theories are represented as conditional probability distributions. A restriction is imposed on the structure of the theories that allows the inference mechanism to find a unique predicted distribution for any action and world state description. These restricted theories are called uniquely predictive theories. The inference mechanism, Probability Combination using Independence (PCI), uses minimal independence assumptions to combine the probabilities ...

View PDFchevron_right
Bayesian Knowledge Fusion
Eugene Santos

We address the problem of information fusion in uncertain environments. Imagine there are multiple experts building probabilistic models of the same situation and we wish to aggregate the information they provide. There are several problems we may run into by naively merging the information from each. For example, the experts may disagree on the probability of a certain event or they may disagree on the direction of causility between two events (e.g., one thinks A causes B while another thinks B causes A). They may even disagree on the entire structure of dependencies among a set of variables in a probabilistic network. In our proposed solution to this problem, we represent the probabilistic models as Bayesian Knowledge Bases (BKBs) and propose an algorithm called Bayesian knowledge fusion that allows the fusion of multiple BKBs into a single BKB that retains the information from all input sources. This allows for easy aggregation and de-aggregation of information from multiple expert sources and facilitates multi-expert decision making by providing a framework in which all opinions can be preserved and reasoned over.

View PDFchevron_right
Knowledge Representation with Possibilistic and Certain Bayesian Networks
Adel M. Alimi

Possibilistic logic and Bayesian networks have provided advantageous methodologies and techniques for computerbased knowledge representation. This paper proposes a framework that combines these two disciplines to exploit their own advantages in uncertain and imprecise knowledge representation problems. The framework proposed is a possibilistic logic based one in which Bayesian nodes and their properties are represented by local necessity-valued knowledge base. Data in properties are interpreted as set of valuated formulas. In our contribution possibilistic Bayesian networks have a qualitative part and a quantitative part, represented by local knowledge bases. The general idea is to study how a fusion of these two formalisms would permit representing compact way to solve efficiently problems for knowledge representation. We show how to apply possibility and necessity measures to the problem of knowledge representation with large scale data.

View PDFchevron_right

Related topics

  • Mathematics
  • Probability Theory
  • Proof Theory
  • Theory and Method
  • Formal Specification
  • Bayesian Network
  • First Order Logic
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved