The Normal Distribution

On this page we will be discussing an important probability distribution, called the Normal Distribution. Before we introduce this distribution, we must present a fact that will be proven in a later mathematics class since it requires a sleight-of -hand with double integrals to verify. We will use this fact is studying the properties of the Gaussian or Normal distribution

Normal Distribution in Standard Form

The fact we need is that

$\int_{-\infty}^{\infty}~e^{-\frac{x^2}{2}}~dx = \sqrt{2 \pi}$

We will define the probability distribution

$p(x) = C~e^{-\frac{x^2}{2}}$

As the probability distribution in normal form. The shape of this function is shown below:

It can be seen that this is a symmetric curve, with a peak at , and with tails that go into the positive and negative x quadrants. The exponential function is always positive, so that, this distribution, unlike others we have already seen, lives on the whole real axis $- \infty < x < \infty$ In order to ensure that this is a normalized distribution, we must have

$\int_{-\infty}^{\infty}~p(x)~dx = 1 \] \[ C~ \int_{-\infty}^{\infty}~e^{-\frac{x^2}{2}}~dx = 1 \] \[ C \sqrt{2 \pi} = 1 \] \[ C= \frac{1}{\sqrt{2 \pi}}$
Thus the Normal Distribution in Standard Form is defined to be the function

$p(x) = \frac{1}{\sqrt{2 \pi}} ~e^{-\frac{x^2}{2}}$

However in order to explore the properties of this function, we will have to understand integrals in which both endpoints are infinite. This will bring us, shortly, to the notion of improper integrals.

For now we will simply state the results (which we will calculate later that the following properties describe this distribution:

For the Normal probability distribution in standard form,

$p(x) = \frac{1}{\sqrt{2 \pi}} ~e^{-\frac{x^2}{2}}$
$\mu ~ ({\rm mean}) = \frac{1}{\sqrt{2 \pi}}~\int_{-\infty}^{\infty}~ x~e^{-\frac{x^2}{2}}~dx = 0 \] \[ V~ ({\rm variance}) = \frac{1}{\sqrt{2 \pi}}~\int_{-\infty}^{\infty}~x^2~e^{-\frac{x^2}{2}}~dx = 1 \] \[ \sigma~({\rm standard ~deviation}) = \sqrt{V} = 1$

We will calculate these explicitly once we understand how to handle improper integrals.

For your consideration:

(1) Use differentiation to show that the maximum value of occurs at

(2) Explain the integrals given in the above box. In particular, why is the Variance the same as the second moment in this case?

(3) Why is the mean at zero ? Can you use symmetry arguments to establish this fact?

(4) Where would you expect the median to be? Explain your answer.

Normal Distribution (Gaussian Distribution) in General Form

A more general type of Gaussian distribution is given by the function

$p(x) = \frac{1}{\sqrt{2 \pi} \sigma} ~e^{-\frac{(x-\mu)^2}{2 \sigma^2}}$
The constant in front of the distribution is chosen for the purpose of normalization again. The shape of this function is shown in the picture below. You will note that it could be wider and shorter, and that while this is still a symmetric curve, its peak may have shifted.

We can relate properties of this distribution to properties of the Normal Distribution in Standard form. Indeed, we can show that

For the general form of the probability distribution,
$p(x) = \frac{1}{\sqrt{2 \pi} \sigma} ~e^{-\frac{(x-\mu)^2}{2 \sigma^2}}$
${\rm mean} = \mu \] \[ {\rm variance} = \sigma^2 \] \[ {\rm standard ~deviation} = \sigma$

We will show how one of these facts can be verified by relating the general form of the distribution to its standard form.

The Mean of the Normal Distribution

We calculate that
${\rm mean} = \int_{-\infty}^{\infty}~ x ~p(x)~dx = \frac{1}{\sqrt{2 \pi} \sigma}~ \int_{-\infty}^{\infty}~ x ~e^{-\frac{(x-\mu)^2}{2 \sigma^2}} dx$
We now make the following substitution:
$z=\frac{(x-\mu)}{ \sigma} \] \[ dz = \frac{dx}{ \sigma} \] \[ x = \sigma z + \mu$
Then, plugging into the integral for the mean, we find that
${\rm mean} = \frac{1}{\sqrt{2 \pi} \sigma}~ \int_{-\infty}^{\infty}~(\sigma z + \mu) ~e^{-\frac{z^2}{2}} \sigma dz$
This can be simplified as follows:
${\rm mean} = \frac{1}{\sqrt{2 \pi} }~(~ \int_{-\infty}^{\infty}~\sigma z ~e^{-\frac{z^2}{2}} dz + \int_{-\infty}^{\infty} \mu ~e^{-\frac{z^2}{2}} dz) \] \[ ~~= \frac{1}{\sqrt{2 \pi} }~(\sigma \int_{-\infty}^{\infty}~ z ~e^{-\frac{z^2}{2}} dz + \mu \int_{-\infty}^{\infty} ~e^{-\frac{z^2}{2}} dz)$
We recognize the integrals above as properties of the original normal distribution in standard form, so that we can simplify this cumbersome expression to:
${\rm mean} = \frac{1}{\sqrt{2 \pi} }~(\sigma ({\rm mean ~ of~ standard ~ normal}) + \mu \sqrt{2 \pi}) = \mu$
(we get this using the fact that the mean of the standard form normal distribution is just zero. We have thus verified that the new distribution has mean $\mu$ .

For your consideration:

(1) Show that the new distribution has the property that
$\int_{-\infty}^{\infty}~p(x)~dx =1$

(2) Find the Variance of the Gaussian Distribution in this general form.

(3) Where would you expect the median of the distribution to be?

In the demo below, you can explore what happens to the Normal distribution when its peak value is increased or decreased, and when its mean is moved. You will note that to preserve the area under the curve, the peak gets narrower if it gets taller, and fatter if it gets shorter. The line segment underneath the graph represents the standard deviation of the distribution.
<