Representing Distributions in BN - Continuous child node with continuous parent node

-
In contrast with discrete variables, when a variable $X$ is continuous and $D_{1}, · · · , D_{n}$ are real values there is no representation that can depict all the possible densities. A very powerful way for representing these variables, which is widely used in literature, is the choice of multivariate continuous distributions, namely the use of Gaussian distributions. These variables can be represented in a BN by using linear Gaussian conditional densities. In this representation, the conditional density of the continuous variable given its parent is given by the following formula [1] [2]: \[ P(X|D_{1},...,D_{n})=N(\alpha_{o} + \sum_{i} \alpha_{i} \cdot u_{i}, \sigma^{2}) \] That means that $X$ is normally distributed and if all the variables in a network are represented as linear Gaussian distributions then the joint distribution is a multivariate Gaussian [3]. If the variables cannot be considered to follow a Gaussian mixture model, the process of discretisation is applied; in this case, each continuous variable is replaced by a set of discrete ones so as to fully cover the range of values that can be found in the original data and approximate the probability distribution of the data as close as possible. The main difficulty in this case is that the BN designer has to come up with a conditional probability table for each combination of the current node and its parent(s) values.

It is worth mentioning at this point that the existing BN software (such as AGENA and HUGIN ), use what they call expressions in order to automate the construction of conditional probability tables for discretised continuous nodes that follow a certain, standard distribution (such as a Normal, Binominal, Beta, Gamma, etc.). They also employ arithmetic operators, standard mathematical functions, and logical operators for enhancing their representational capabilities. However, even in this case, the BN designer assumes that the conditional probability distribution for a variable follows (at least approximately) a certain form of function or distribution — while still suffering from the loss of information that the discretisation process itself entails.

Moreover, it has been proved that any type of distribution that satisfies the axioms of the Shenoy-Shafer framework [4] can be used to represent the conditional distributions between continuous nodes. An application of the Shenoy-Shafer framework is the use of Mixture of Truncated Exponentials (MTEs) [5] [6], where the real probability distribution of a continuous random variable p(x) is approximated by \[ \tilde{p}(x)= \sum_{i=1}^{N}\alpha_{i}\cdot e^{{-\sideset{}{_{j=1}^{M}}\sum}b_{ij} x_{j}} \]
where N is the number of components used for the approximation.
In both cases, the desired conditional probability has to be approximated. In the first case using a single Gaussian and in the second case using a MTE.

Lastly, a novel method [7] for representing conditional distributions between continuous nodes is introduced by the author in this thesis. This method is based on combining the particle filter theory with the Shenoy-Shafer framework for evidential reasoning. The proposed method meets the requirements and fulfills the axioms of the Shenoy Shafer framework, and proves that sums of Gaussians can be used to represent continuous nodes.

[1] N. Friedman,M. Linial, I. Nachman, and D. Pe’er. Using Bayesian Networks to Analyse Expression Data. Journal of Computational Biology, 7(3–4):601–620, August 2000.
[2] S. L. Lauritzen and N. Wermuth. Graphical Models for Associations between Variables, some of which are Qualitative and some Quantitative. Annals of Statistics, 17(1):31–57, March 1989.
[3] S. L. Lauritzen and D. J. Spiegelhalter. Local Computations with Probabilities on Graphical Structures and their Applications to Expert Systems. Journal of the Royal Statistical Society. Series B (Methodological), 50(2):157–224, 1988.
[4] P. P. Shenoy and G. Shafer. Axioms for probability and belief-function proagation. In Proceedings of the 4th Annual Conference on Uncertainty in Artificial Intelligence (UAI 1988), pages 169–198. North-Holland Publishing Co., 1988. [5] B. R. Cobb and P. P. Shenoy. Inference in Hybrid Bayesian Networks with Mixtures of Truncated Exponentials. International Journal of Approximate Reasoning, 41(3):257–286, April 2006.
[6] B. R. Cobb, P. P. Shenoy, and R. Rum´ı. Approximating Probability Density Functions in Hybrid Bayesian Networks with Mixtures of Truncated Exponentials. Statistics and Computing, 16(3):293–308, September 2006.
[7] Athanasiou, M., Probabilistic reasoning for medical diagnosis : the Dimitra-PRO system for spinal injury care, PhD Thesis, University of Surrey, 2009

Back to: Representing Distributions in Bayesian Networks