Quadratic Forms.
Recall that, Y′AY is called a quadratic form of Ywhen Yis a random vector.
Result. If X∼Np(µ, Σ), Σ >0, then (X−µ)′Σ−1(X−µ)∼χ2
p.
Proof. Z= Σ−1/2(X−µ)∼Np(0, Ip). i.e., Z1, Z2,...,Zpare i.i.d. N(0,1).
Therefore Z′Z=Pp
i=1 Z2
i∼χ2
p. Note that (X−µ)′Σ−1(X−µ) = Z′Z.
Result. If X1, X2,...,Xnis a random sample from N(µ, σ2), then ¯
Xand
S2=Pn
i=1(Xi−¯
X)2are independent, ¯
X∼N(µ, σ2/n) and S2/σ2∼χ2
n−1.
Proof. First note that X= (X1, X2,...,Xn)′∼Nn(µ1, σ2In). Now con-
sider an orthogonal matrix An×n= ((aij )) with the first row being a′
1=
(1
√n,1
√n,..., 1
√n) = 1
√n1′. (Simply consider a basis for Rnwith a1as the
first vector, orthogonalize the rest.) Now let Y=AX. i.e., Yi=a′
iX,i=
1,2,...,n. Since X∼Nn(µ1, σ2In), we have that Y∼Nn(µA1, σ2AA′) =
Nn(µA1, σ2In). Therfeore, Yiare independent normal with variance σ2. Fur-
ther, E(Yi) = E(a′
iX) = µa′
i1. Thus, E(Y1) = µa′
11=µ1
√n1′1=√nµ.
For i > 1, E(Yi) = µa′
i1=µ√na′
ia1= 0. i.e., Y2,...,Ynare i.i.d N(0, σ2).
Therefore, Pn
i=2 Y2
i∼χ2
n−1. Further, Y1=a′
1X=1
√nPn
i=1 Xi=√n¯
X∼
N(√nµ, σ2) and is independent of (Y2,...,Yn). Also, S2=Pn
i=1(Xi−¯
X)2=
Pn
i=1 X2
i−n¯
X2=X′X−Y2
1=Y′Y−Y2
1=Pn
i=2 Y2
i∼χ2
n−1which is inde-
pendent of Y1, and therefore of ¯
X.
If X∼Np(0, I), then X′X=Pp
i=1 X2
i∼χ2
p. i.e., X′IX ∼χ2
p. Also, note
X′(1
√p11
√p1′)X=p¯
X2∼χ2
1and X′(I−1
p11′)X∼χ2
p−1.
What is the distribution of X′AX for any arbitrary Awhich is p.s.d.? With-
out loss of generality we can assume that Ais symmetric since
X′AX =X′(1
2(A+A′))X=X′BX, where B=1
2(A+A′) is always symmetric.
Since Ais symmetric p.s.d., A= ΓDλΓ′, so X′AX =X′ΓDλΓ′X=Y′DλY,
where Y= Γ′X∼Np(0,Γ′Γ = I). Therefore X′AX =Pp
i=1 diY2
i, where
diare eigen values of Aand Yiare i.i.d N(0,1). Therefore X′AX has the
χ2distribution if di= 1 or 0. Equivalently, X′AX ∼χ2if A2=Aor
Ais symmetric idempotent or Ais an orthogonal projection matrix. The
equivalence may be seen as follows. If d1≥d2≥... ≥dp≥0 are such that
1
