Welcome to BLOCK 5 of
Probability and Statistics my name is Lidia Montero
and this video contains
two important results related to statistics.
The law of large number and the central limit theorem.
And we will see some common radom variables.
With a companion video
you will see some example and problems solved using the concepts
that we are discussing in this video.
So. Law of large numbers.
We have seen in the case of the two dice experiment
that the empirical probability is the probability of the event
divided into the total number of events
in the sample space, in the event space.
So by experimentation, if we toss a fair
coin, or we toss a fair die,
then what we see is the probability of one
is one divided into six (1/6).
Okay, the law of large numbers would say
that after
repeating several times the experiment,
then the empirical probability of one, for example,
approaches to
the theoretical probability and this is very important
because this can lead us to some cases
as the one shown in this slide.
Let's assume that this is tossing a fair die
and that we have 6 possible values,
and we are interested in the mean.
So, technically, we know that the theoretical mean is 3.5.
That, as we repeat once and again,
the experiment of tossing the die,
then we see that the mean of the results that we are obtaining approaches
to the theoretical probability.
So the red line that is the empirical probability
is approaching to the mean value, the true mean value
according to the theoretical probability
with a fair die.
Okay, so this is important
and this is connected to the central limit theorem.
But in fact, before the central limit theorem,
we have to discuss a little bit the distribution
of a very important random variable that is the normal distribution.
We have seen so far that an experiment
of tossing 2 dice,
what we have done is to define the probability function.
Fine, there is all the paraphernalia, the event space
and the definition of the probabilities and the values
for the sum of two outcomes, fine.
But this is not practical in many real situations.
So, what we would like is to have a random variable
that is a common random variable.
For example, for modeling the height of boys
in the 11 year class that could be
this is roughly normally distributed.
To model the number of cars
arriving at a toll station
in the weekday from 7:30 to 9:30.
This is a Poisson distribution.
The interarrival times
to a toll station in the morning,
the first period that can be model according
to an exponential distribution
or the number of heads
after ten trials of the experiment
of tossing a fair coin,
this would be a binomial distribution.
Okay.
Now normal distribution.
Normal distribution is a continuous distribution.
The shape can be seen in the diagram
and is a symmetric shape around the mean.
So, this is the shape of the density function.
The density function
formula is included in the slide and we will discuss this further,
but if we are interested in the probability of a normal variable
that has two parameters, 'mu' expectation
and sigma, that is the standard deviation.
So, the squared root of the variance,
then what we have to do
is to calculate the integral,
the area under the curve,
in the set of values or in the interval
that we are interested in, from a
to b, this could be area under the curve.
But, the problem with normal distributions is that the primitive
of the probability density function
is not analytical. So,
we have to
use some numeric simulation tables
and we will be using a trick.
So, let me highlight that
the expectation of a normal variable is 'mu'
according to the first parameter and the variance is sigma squared.
So, the parameterization is sigma or sigma squared,
it depends on the books, and we will see if we add two normal variables.
Will the outcome be normally distributed?
We will see.
The normal distribution can be generalized
to the multivariate normal distribution.
I will not be insisting on this topic
because I would like to highlight that the density function for
a univariate normal distribution as we so we have seen before,
has a term that involves 'mu' and 'sigma'.
So, expectation and standard deviation
and we are using two constants 'e',
the Euler constant, 2.71
and 'pi'.
Once this is said, then the shape of the density
function depends on 'mu' and 'sigma'.
Okay, let's see.
It can be shown that no matter
the values of 'mu' and 'sigma',
so, the expectation and the standard deviation
(a non-negative value),
then it can be proved that
the area under the curve
that is from -1 'sigma' to the left to 1 'sigma' to the right
with respect to the expectation has a value of 0.68 (68%),
the red part in the chart.
If we take twice
sigma, twice the standard deviation from the mean.
then the green area, under
the curve is 0.95
and three times
from the expectation the area under the curve,
so blue, from left to right
is 99.7%.
This regularity stands for
any normal variable.
Okay, let's assume the SAT math score
roughly follows a normal distribution
in the US population of college students.
Okay, so it ranges between 200 and 800.
So, the average is 500 and the standard deviation is 50.
So, since
one sigma from the mean, the expectation,
this is 500-50 to
500+50.
Then, we have 68% of the students in the SAT score.
So, from 450 to 550.
With the same point of view,
95% of the results lie between
400 and 600 and 99%,
something more, lie between 350 and 650.
So, let's stress
the trick that we will be using for the calculation
of cumulative probabilities
based on a normal distribution.
And this is focusing on the standard normal distribution.
The standard normal distribution is the normal distribution
with expectation equal zero and variance equal one.
Okay.
So, the standard deviation also equal one,
as you see in the graphic.
Then, this is a particular case.
So, we have the formula.
But in fact, what is much more interesting
is that any normal distribution can be
scaled, so
transformed into a standard
normal distribution, just by subtracting the expectation, 'mu',
and dividing into the standard deviation.
So, the calculations
of probabilities based on
any normal distribution
can be done using this trick
and translated to calculations on the standard normal distribution.
And we have tabulated values for the cumulative
probability of a standard normal distribution.
This is one of the tables that they have taken from the Internet.
So, in this case, an example, let's see,
I'm interested in calculating
the probability for a standard
normal distribution to be less than 1.51.
So, taking into account the table one,
I have to select
the row, taking into account
the one point something so 1.5.
Then I have to select the column
according to the second digit.
In this case, 0.01.
And this takes me to a probability
that is 0.93 something
and this probability corresponds to the probability
for a standard normal distribution to be less than
1.51.
Okay.
So, the area under the curve is
93.45%, the same argument,
So, now
the second important result, the central limit theorem.
The central limit theorem states that,
once we know the existence of normal distributions,
if we have a set of an independent
normal variates,
all sharing the same parameters.
So, independent and identically distributed,
then it's well known that
the mean of these independent and identically distributed
normal variables is also a normal variable.
This is the first line.
Okay.
But what is more,
if I have
random variables
regardless the distribution,
but all of them identically distributed with the same mean, 'mu',
and the same variance sigma squared.
Then, regardless the original distribution,
the mean of these variables,
as the number of variables
increases, tends to be normally distributed.
So, means are normally distributed,
means of common variables that are independent.
So this is great and very useful. Okay,
once seen this result,
let's start with the description
of common discrete random variables.
The best known of the well-known
and basic discrete distribution is the Bernoulli
distribution, it is the experiment
where you have two possible outcomes: positive or negative.
Okay, then this is, for example, tossing a coin.
Two possible results, positive outcome head.
Okay.
And we will say that binomial is
a situation where we repeat the Bernoulli experiment
several times, the independent Bernoulli experiments,
and as the number of repetitions is n
then we count the number of positive outcomes
then, in this case, the number of positive outcomes
follows a binomial distribution with two parameters
where n is the number of repetitions of the Bernoulli experiment,
the simple 0/1 experiment, and p
is the probability of the positive outcome.
That, of course, is the same for all the repetitions of the experiment.
In this case, the possible values range between
0 and n, the total number of repetitions.
Okay.
In the
Bernoulli process, the repetition of the Bernoulli process,
we find two additional distributions that are commonly used.
They show geometric distribution and the negative binomial distribution.
So, in the geometric is a particular case of the negative binomial
and in fact the negative binomial would be the number of trials
that are needed to get 'r' positive outcomes
where 'r' is an integer number, non negative,
and in the particular case of r=1,
the negative binomial becomes a geometric distribution.
Okay, possible values are 'r', 'r+1',
'r+2' to infinite.
And, then we have to start
this list, first with the Poisson distribution.
This is a very natural model for a counting process.
So, the number of diseases in a given area,
or number of telephone calls is in a given period.
The possible values in this counting process are actually 0, 1, 2 to infinite.
Okay,
We have here
the probability function for a Bernoulli law.
It has 2 values, positive outcome
that this is 1, it is probability p and negative outcome
that is modeling 0 is 1 - p.
It's very easy to state in a table and to see that the density,
sorry the probability function, is the one shown in the slide.
We can see that the expectation is p and the variance is p(1-p).
So, this is the pattern,
the model to be used, for example, for the outcome of tossing a coin
with head and tail, two possible outcomes.
Let's focus on repeating this basic Bernoulli experiment.
Okay.
This takes to the binomial distribution.
In the binomial distribution, the probability functions.
So, the probability of obtaining
k positive outcomes after
n repetitions of the basic Bernoulli
trial can be seen to be the combinatorial number n
over k
by the probability of the positive outcome
power at k and by (1-p)
power at n-k. Okay
the distribution function is defined as the cumulative
probabilities of the individual terms.
The expectation is np and the variance is
np(1-p).
We have here,
also in the next slide, some details about the probability
function, the distribution function and the expectation
and for the geometric distribution
and the same for the negative binomial.
So, number of repetitions of the Bernoulli
trial up to
and including r positive successes, so positive outcomes.
Okay.
So, here we have the formulas. These are not so used.
So okay, very common.
But, in fact
I prefer to focus on the Poisson variable that is extremely important and linked
to the very important continuous distribution that is the exponential distribution.
So, in the Poisson variable, we have a parameter,
the parameter in this case is
lambda. Lambda states
for the average number of events
occurring in the time interval, 0 to T,
or the number of events in a given area.
So, it's a counting process and lambda stands
for the meaning of the mean,
the expectation of the number of events.
So, in this case we see here the formulas for the probability
and the distribution function, the expectation, as I have said
before, it has just one parameter
and variance is equal to the expectation.
And in this case, this is a very popular model
for the modeling of a counting process.
Okay.
Possible values, of course, 0,1,2 to infinite.
The representation of these formulas.
The probability function for this discrete
variable leads us to a shape that, of course, depends on lambda.
We have here the chart
above that is the probability function and the chart below
that corresponds to the distribution function.
The distribution function is not continuous because it has some stairs,
because the cumulative issue is given at the integer values.
Okay.
Anyway, this distribution
can be approximated by a normal distribution.
Using the rule of thumb, lambda has to be greater than five.
Because it is a little hard to use these
formulas for the
the calculus of the probabilities in problems.
Okay, so we use approximations.
The same happens to the binomial distribution.
I'm not going to enter in details now,
but in the companion video
we will see some examples and we will see that
in binomial distribution we can use an approximation, rule of thumb,
when the expectation is greater than five,
then we can use for the calculus of probabilities,
the approximation based on the standard normal distribution.
Okay,
so, just to finish,
we have to say something about the continuous distributions.
Continuous distributions is a little bit harder to deal with
based on mathematics because we have to use integration
and the integration of the density function from minus
infinite to plus infinite is equal to one.
Okay, the point probability is zero,
but, we have to deal with ranges.
Intervals. Okay.
The most common and well known distribution
is the exponential distribution and exponential distribution
is a model for interarrival times,
so it's a model for non-negative
values and real values.
In this case, the probability
density function can be seen on the slide.
It depends on one parameter that it is lambda.
Yes, this lambda is the rate.
Okay, so is the same rate that exists,
the connection to the Poisson process.
Then we have a cumulative distribution
that now is a continuous
and derivable function.
Okay, the expectation is 1 divided into lambda
and variance is 1 divided into lambda squared.
Here, we have the shape of density function depending on lambda,
above and now the cumulative
probability function depending on lambda
can be seen in the chart below.
Okay.
And with this, just highlight that
in order to calculate probabilities,
what we have to do is to integrate
the density function and to use some.
Okay.
Yes, primitive and basic, but we'll have to use
some mathematics on integration.
Okay.
So, this
is the last slide today.
Thank you very much for your attention
and see you in the companion video!