CS 2750 Machine
Learning

Averaging decreases variance

**Example
**

Assume we measure a random variable x with a *N*(m,s2)
distribution

If
only one measurement *x**1* is done,

The
expected mean of the measurement is m

Variance
is Var(x1)=s2

If
random variable *x* is measured *K* times (x1,x2,
xk)
and the value is estimated as: (x1+x2+
+xk)/K,

Mean
of the estimate is still m

But,
variance is smaller:

[Var(x1)+
Var(xk)]/K2=Ks2 / K2 = s2/K

Observe:
**Bagging is a kind of averaging!**