Representing Distributions in BN - Discrete child node with continuous parent node

-
In this case, the most common method used is the discretisation of the continuous parent node; as a result, the BN designer faces all the difficulties in order to construct the conditional probability table. This is due to the fact that the BN theory so far does not provide us with a method that allows us to propagate evidence in such topologies. Still though, attempts to use the logistic function in the case of a discrete variable with continuous parent has been carried out by Murphy [1]. If we assume $X$ is a discrete variable with values $x_{1} . . . x_{n}$ and $Z = Z_{1} . . . Z_{k}$ are its continuous parents with values $z_{1} . . . z_{k}$, the logistic function for representing the conditional probability of the discrete child node given its parents is defined as: \[ P(X=x_{i}|Z)= \frac { e^{{g_{i}+{\sideset{}{_{n=1}^{k}}\sum}s}_{i,n}z_{n}}} {{\sideset{}{_{j=1}^{m}}\sum} e^{{g_{j}+{\sideset{}{_{n=1}^{k}}\sum}s}_{j,n}z_{n}}} \]
where $s_{i,n}$ determines the steepness of the curve as per parent node $i$ and possible discrete value $n$, and $g_{i}$ is its offset (as per the parent node). A large $s_{i,n}$ corresponds to a hard threshold (steep curve) and a small $w_{i,n}$ corresponds to soft threshold. Essentially, in this work, the logistic is initially converted into the closest-fitting Gaussian — which allows for exact inference to take place.

Then, by iteratively adjusting the parameters of the node potential, it is possible to achieve a good approximation for the inference problem. Therefore, inference can take place in such cases — which makes representing distributions with logistic functions also feasible.

The MTE based approximation is also applicable in this case. Based on the Shenoy-Shafer framework, this is done by performing local computations, as seen in [2]. While the use of local computations allows us to overcome the problem of inference presented in the baseline BN theory, it is still limited by the fact that the BN designer must perform the approximation of the continuous PDF to an MTE-based distribution by hand — thus, while this method does allow us to perform inference, it is difficult to implement in practice.

At this point, it is obvious that, in the general case, using continuous random variables within BNs requires considerable skill by the BN designer — both in setting up a network topology that makes reasoning using the classical BN theory possible, as well as selecting a good way of approximating the real continuous distribution (either by using Gaussians according to the standard BN theory, or other distributions where approximations have been devised).

Therefore, it is clear that a solution similar to that used within the Shenoy-Shafer framework is much more appropriate if we wish to extend BN-based reasoning to domains where complex relationships among random variables exist. Such a solution would have to be separated in two levels:

  • An apparatus which will enable unified handling of discrete and continuous expressions — where continuous expressions are assumed to fulfill the criteria set out by the Shenoy-Shafer framework, while the discrete expressions do so by definition. For discrete variables, the main issue lies in finding a formulation that allows them to be easily combined with continuous variables. This is formulation is presented in this thesis.
  • A method for representing continuous distributions of any kind in a simple, unified manner. We have seen that MTEs can be used to approximate continuous distributions; however, due to the nature of their formulation, their applicability is limited to known distributions with simple analytical formulations. In the thesis presented by the author, a novel method proposed for extracting representations of continuous distributions in a unified manner, which can achieve an arbitrary level of accuracy in approximating any continuous distribution.


[1] K. P. Murphy. A Variational Approximation for Bayesian Networks with Discrete and Continuous Latent Variables. In Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence (UAI’1999), pages 457–466. Morgan Kaufmann, July–August 1999.
[2] B. R. Cobb, P. P. Shenoy, and R. Rum´ı. Approximating Probability Density Functions in Hybrid Bayesian Networks with Mixtures of Truncated Exponentials. Statistics and Computing, 16(3):293–308, September 2006.

Back to: Representing Distributions in Bayesian Networks