Graphical Models - Artificial Intelligence - Lecture Slides, Slides of Artificial Intelligence

Download Graphical Models - Artificial Intelligence - Lecture Slides and more Slides Artificial Intelligence in PDF only on Docsity!

Introduction to Graphical Models

Part 2 of 2

Lecture 31 of 41

Graphical Models Overview [1]:

Bayesian Networks

P ( 20s , Female , Low, Non-Smoker , No-Cancer, Negative, Negative )

= P ( T ) · P ( F ) · P ( L | T ) · P ( N | T , F ) · P ( N | L, N ) · P ( N | N ) · P ( N | N )

• Conditional Independence
  • X is conditionally independent (CI) from Y given Z (sometimes written X  Y | Z ) iff

P ( X | Y , Z ) = P ( X | Z ) for all values of X, Y , and Z

• Example: P ( Thunder | Rain , Lightning ) = P ( Thunder | Lightning )  T  R | L
• Bayesian (Belief) Network
• Acyclic directed graph model B = ( V, E, ) representing CI assertions over 
• Vertices (nodes) V : denote events (each a random variable)
• Edges (arcs, links) E : denote conditional dependencies
• Markov Condition for BBNs (Chain Rule):
• Example BBN





n

i

P X 1 ,X 2 , ,Xn P Xi|parents Xi

1



X 1 X 3

X 4

X 5

Age

Exposure-To-Toxins

Smoking

Cancer

X 6

Serum Calcium

Gender^ X 2 X 7

Lung Tumor     

Non  Descendants



Parents

 

Descendants

Bayesian Learning

• Framework: Interpretations of Probability [Cheeseman, 1985]
  • Bayesian subjectivist view
    • A measure of an agent’s belief in a proposition
    • Proposition denoted by random variable (sample space: range)
    • e.g., Pr ( Outlook = Sunny ) = 0.
  • Frequentist view: probability is the frequency of observations of an event
  • Logicist view: probability is inferential evidence in favor of a proposition
• Typical Applications
  • HCI: learning natural language; intelligent displays; decision support
  • Approaches: prediction; sensor and data fusion (e.g., bioinformatics)
• Prediction: Examples
  • Measure relevant parameters : temperature, barometric pressure, wind speed
  • Make statement of the form Pr ( Tomorrow’s-Weather = Rain ) = 0.
  • College admissions: Pr ( Acceptance )  p
    • Plain beliefs: unconditional acceptance ( p = 1) or categorical rejection ( p = 0)
    • Conditional beliefs: depends on reviewer (use probabilistic model)

Choosing Hypotheses

arg max  f  x 

x  Ω

• Bayes’s Theorem
• MAP Hypothesis
  • Generally want most probable hypothesis given the training data
  • Define:  the value of x in the sample space  with the highest f ( x )
  • Maximum a posteriori hypothesis, hMAP
• ML Hypothesis
  • Assume that p ( hi ) = p ( hj ) for all pairs i , j (uniform priors, i.e., PH ~ Uniform)
  • Can further simplify and choose the maximum likelihood hypothesis, hML

argmaxP  D |h  P  h 

P D

PD|hP h argmax

h argmaxP h|D

h H

hH

hH MAP













P  D 

P h D

P D

P D|hP h P h|D

  

 i 

h H

hML argmaxPD|h i 



Inference by Clustering [1]: Graph Operations

(Moralization, Triangulation, Maximal Cliques)

A

D

B E G

C

H

F

Bayesian Network

(Acyclic Digraph)

A

D

B E G

C

H

F

Moralize

A 1

D 8

B 2

E 3

G 5 C 4 H 7

F 6

Triangulate

Clq

D

C

G

H

C

Clq

G

F

E

Clq

E3^ G

C Clq

A

B

Clq

E

C

B

Clq

Find Maximal Cliques

Inference by Clustering [2]:

Junction Tree – Lauritzen & Spiegelhalter (1988)

Input: list of cliques of triangulated, moralized graph G u

Output:

Tree of cliques

Separators nodes Si,

Residual nodes R i and potential probability (Clq i ) for all cliques

Algorithm:

1. Si = Clqi (Clq 1  Clq 2 … Clqi-1)
2. Ri = Clqi - Si
3. If i >1 then identify a j < i such that Clq j is a parent of Clq i
4. Assign each node v to a unique clique Clq i that^ v^ ^ c( v )^ ^ Clq i
5. Compute (Clqi) = f(v) Clqi = P( v | c ( v )) {1 if no v is assigned to Clq i }
6. Store Clq i , Ri , Si, and (Clqi) at each vertex in the tree of cliques

Inference by Loop Cutset Conditioning

Split vertex in

undirected cycle;

condition upon each

of its state values

Number of network

instantiations:

Product of arity of

nodes in minimal loop

cutset

Posterior: marginal

conditioned upon

cutset variable values

X 3

X 4

X 5

Exposure-To-

Toxins

Smoking

Cancer (^) X 6

Serum Calcium

X 2

Gender

X 7

Lung Tumor

X1,

Age = [0, 10)

X1,

Age = [10, 20)

X1,

Age = [100, )

• Deciding Optimal Cutset: NP -hard
• Current Open Problems
  • Bounded cutset conditioning: ordering heuristics
  • Finding randomized algorithms for loop cutset optimization

Inference by Variable Elimination [1]:

Intuition

http://aima.cs.berkeley.edu/

Inference by Variable Elimination [3]:

Example

A

B C

F

G

Season

Sprinkler

Rain

Wet

Slippery

D

Manual

Watering

P(A|G=1) =?

d = < A, C, B, F, D, G >

G D F B C A

λG(f) = ΣG=1 P(G|F)

P(A), P(B|A), P(C|A), P(D|B,A), P(F|B,C), P(G|F)

P(G|F)

P(D|B,A)

P(F|B,C)

P(B|A)

P(C|A)

P(A)

G=