Welcome to BLOCK 5:
Correlation and Regression analysis.
My name is Lídia Montero and I will be the instructor.
We will see what is bivariate data analysis.
We will analyze scatter plots,
scattergrams and the correlation coefficient and we will see how to state
and how to estimate simple linear regression models.
This is all for these for this video with a companion that will contain
some practical interpretation and graphics.
So, terminology Bivariate Data Analysis
stands for the study of the association
or relationship between two numeric variables.
So, we are taking two variables simultaneously.
Okay, this is useful for determining
how the change in one exploratory variable affects
to the result of the target variable.
So, in this case the terminology explanatory variable
and the target variable, that it is response variable
or sometimes call, but I don't like the terminology, dependent variable.
Regression models are
useful to determine
the optimal operating conditions
when the explanatory variables are control variables.
And also they are extremely useful for predicting,
forecasting the value of the response
variable in a new situation.
So this is, for example,
the main purpose of the regression
models in the Master of Urban Mobility and in particular,
in the data analysis and demand modeling.
So, we have some examples.
Explanatory variable, average speed of a car and
response variable is fuel consumption.
Fuel efficiency.
The goal could be to determine the speed
that optimizes the efficiency of the car.
A second example, temperature
and in a chemical reaction,
the yield of the reaction.
So ,again, we have the yield
and the explanatory variable is temperature,
Or, for example, the amount of fertilizer
and the yields for a given plant.
Okay.
Or just the thickness of a stack of paper
and the response could be the number of sheets.
So these are the examples.
We have several examples as the time of study
and the GPA outcome.
So we're dealing with two numeric variables.
So what we have, first of all,
let's take a look at the chart, the basic charts that are useful
for taking a look and making
a rough assessment of the relationship between the two variables.
The basic plot is called
the scattergram, the scatterplot,
we have also a bivariate chart.
It has several terminologies, but basically what we have
is the explanatory variable goes in the horizontal axis.
The X-axis and the response variable
is depicted in the Y-axis.
So, the vertical axis. And in this case, we have the basic plot
with some sort of customization to the left,
with the base library, and to the right using the ggplot.
In fact, the ggplot could be improved a lot, but this is not done.
So, here you have some of the parameters that are useful
for customizing the base library in R.
Let's see an example
about age and average fatness.
So, this is a sample
of around 20 observations.
And the question is, what is the relationship
between age and fatness?
So if we scatterplot this data,
what we see is a clear trend,
that increasing age one, the X-axis, this is the explanatory
variable means increasing the percentage of fat.
So, this is the response variable.
So I have increasing trend
a direct relationship between both variables
and it looks like a linear relation.
For example, Advertising and Sales.
We have again as much
as we spend in advertising,
Sales are increasing, this is fine, we have a direct relation.
We have two approaches to study this
relationship between the two numeric variables. We have,
first, a summary that is based on the correlation coefficient by default.
Pearson's correlation coefficient and this correlation
coefficient is an statistic that is r adimensional in the sense
that it ranges between -1 to 1
and it indicates the direct
or inverse relation between the two variables
and also the strength of the relation between the two variables.
We will see some examples.
So in fact the correlation coefficient
can be obtained by
using what is named the sample
covariance between x and y.
This is s_x,y.
You have the formula there and on the slides and divided
into the product of the standard deviations
of the two variables.
So this is Pearson's correlation coefficient.
So, the alternate way to analyze
the relation is just to adjust
a linear regression to the scatterplot.
So, let's see in the scatterplot,
we have to describe the pattern; i.e.
upwards or downwards, direct or inverse,
curved or not pattern. Strength of the relation.
Unusual observations.
This is maybe highlighting some clouds of outliers. Here, we have some examples.
In the first case, we have a direct relation,
it is a strong relation and the correlation
coefficient would be positive.
In the second case, I have a weaker relation
and the coefficient of correlation would be negative,
because as increasing x means decreasing y,
and the data is spread around this imaginary line,
That could be the linear regression line.
In the
fourth example, I have a curvilinear shape.
So, the linear model is not suitable and in the last plot, what we have
is just a scattered data with no
known pattern or clear pattern between X and Y.
So this means that I would have no correlation and
no relationship. And no correlation because the correlation would be zero.
Okay, so
the table shows height and weight
for some college football players.
If you use the calculator and you apply the formulas that I showed you before, then
you can see that the correlation between height and weight
is 0.82 is positive.
Okay, so increasing height means increasing weight
and the strength is also high.
So it means that
there exists a strong connection.
So, this would be the scatterplot in MINITAB, for example.
Okay, so
in this case, it's a strong correlation.
We see one observation that it is not exactly properly aligned.
This maybe could be an outlier of the regression model.
If we interpret the correlation coefficient,
we see that it ranges between -1 and 1.
Okay, so it's adimensional.
Then, in the case of zero, it means
no relationship between X and Y.
So nothing, in the case of r
approaching in absolute value
to 1,
then, it means that we have a strong relation, either positive, or negative.
And, in the case of r
coefficient of correlation equals to 1 or -1,
it indicates a perfect fit,
a perfect linear relation between X and Y.
This is not common in data because, in data, we have variability.
So, we denote
X as the exploratory variable, Y as the response variable.
And, let's move
forward to the formulation of the simple linear regression.
We would like to find
how Y depends on X
and how to predict Y (fat)
according to the values of X (age).
Okay.
So, this is to fit a straight line to the cloud of points.
A straight line consists of an intercept, that it's very clear, because
is the value in the Y axis corresponding
to a zero on the X axis, and the slope
that is the angle with respect to the horizontal axis.
In fact, in a practical point of view, we have several factors
that influence the relation between age and fatness.
So, we cannot see in real data
a perfect straight line, a perfect linear relation.
Okay, so we have to model, to establish a probabilistic model.
And we define the target as a deterministic model
based on the explanatory variable plus a random error.
Okay,
so the random
error represents the fluctuation, the extra issues,
many, many variables that contribute to give some sort of variability
to the relationship between X and Y.
So, the probabilistic model is assumed to be valid
to stand for the population, on the right;
Y = alpha + beta X + epsilon.
So, this would be the model that stands for the population.
Okay, so we are using Greek letters.
This is very common, but in particular in particular, we will see that
the estimation of the model parameters
alpha and beta from a sample means
that we have some alpha hat and beta hat.
So, or maybe using the
the Latin alphabet, we will see how to do this.
So, once we have the model, we have some sort of
condition, the statement of the model about the random deviations.
So, this random error.
Okay, so we will see
the hypotheses for the linear model.
Basically, we have to understand, that we have some points.
We adjust a line, we have alpha,
that is the intercept.
We have seen this before.
And graphically,
the model means to estimate the straight line.
But the points, as the one that we that we see in red,
is the observation that does
not follow on the line.
And in this case, we have this random error.
Okay?
The random error in a particular sample is called the residual.
And in the case of this new dot, it's a point
that has a negative deviation because it is under the straight line.
Okay, so this is
to fit the model to the cloud of points, but in fact, we need some assumptions.
The first assumption is that the random error
has a mean of zero at any X.
Okay.
And the standard deviation for this
random error is the same at any point of X,
and this standard deviation is denoted
by s,
then the distribution of this random
error is normal,
so conditional to any X, the distribution
of the random error is normal and random errors are independent.
And taking into account
the rest of the assumptions written above,
random errors are independent and identically distributed
normal values, normal variates,
Interpretation of the terms.
Here, we have the straight line, the slope b,
I'm using the Latin alphabet here, because slope b
is the estimate of the parameter, the population parameter.
Okay.
And I have the intercept that this
is the estimate of alpha,
that it's the point where X is equal to zero,
the value that stands for Y.
Okay.
So, we will see that in this case,
we have a plot, on the left, with small sigma,
small sigma means small deviation.
The standard deviation of the random error is quite small.
And on the right, I see
the standard to be greater.
So the residual term is
is more important than in the left side plot.
Okay.
So what it is important is that this established
the relationship between X and Y; and increasing by one X
and applying the straight line,
and I obtain the effect on Y.
So this is the meaning of the slope, it's extremely important.
As I mentioned before, it's required
that the random term to be normally distributed for any X.
This is the basic condition.
Okay,
let's use an example.
We have described so far the probabilistic model,
the theory, i.i.d., independent and identically distributed random errors.
We have the three population parameters,
alpha, beta and sigma.
Okay.
And then, we have a cloud of points.
Okay.
So the probabilistic model allows us to estimate alpha and beta.
This can be notated, again, as alpha, beta hat, or just the Latin letters.
So, a and b, a for intercept, b for slope.
Okay. And the
mean of
the residual terms is zero.
This should stand.
So, the method of the least
squares consists on minimizing
the squared difference between the observations and the predicted values
according to the straight line
that fits the cloud of points.
Okay, so we have the best fitted line.
We have the term that is the residual sum of squares (RSS)
and we have to minimize the squared differences
between observations and fitted values.
Okay, So fitted values use the notation 'y hat'
and in fact, it's the value according
to the straight line that we are trying to fit.
So, these can be seen to minimize
when some conditions are satisfied.
We are going not going to see the the first order conditions,
but, in fact, the beta, the estimate of beta,
the slope, it's just the covariance between the two variables.
divided into the variance of the Xs
and alpha hat is just the estimate
for the difference in the relation
to the mean of X and the mean of Y.
Once applied the estimate for the slope.
Let's see.
Let's see how it works.
We have here this data set. It's the set just in terms of a table
I am using the table, to add X squared and the product of X and Y.
This is useful for the case of the covariance
between both variables.
Then I have these numbers.
Then, the next step is to apply the formulas
that we have seen before, to this particular case.
So, I'm not going to detail how
this is done, but it's just substitution
and this takes us to the estimate of beta,
the slope, that it's 0.55 and then applying
the straight line to the means of both variables
I have an intercept that it is estimated to 3.22.
So, here it's the fitted line.
So, the straight line that matches, that fits, the cloud of points.
Okay, so 3.22 plus
0.55 by x, this is my model.
We have to explain a
little bit about the coefficient of determination. The goodness of fit
in an exercise of simple linear regression
has a measure of the quality.
This is usually the goodness of fit.
This is the terminology that we use in Statistics, and in this case
is based on the coefficient of determination.
The coefficient of determination is
the proportion of the total sum of the squares
explained by the straight line, divided into the total
sum of the squares that is related to the target variability,
or just taking into account the properties, the orthogonal properties
between the sum of the squares, we can see that
this coefficient of determination is 1 minus
the residual sum of squares, the definition has been seen before,
divided into the total sum of the squares.
So, it's the fraction of the target
variability that it's explained by the current model,
or one minus the fraction
that it's not explained by the current model because the residual
sum of squares corresponds to the part of the error term
that it's not in the systematic part of the model.
Okay.
So this R-squared is the coefficient
of determination, and it's also
the square of the correlation coefficient (Pearson's correlation
coefficient) that we have seen before, in the case of simple linear regression.
Okay, so it's a number between zero and one
and gives the proportion of variation
between X and Y.
This is the part of the target.
The Y variability that can be explained by the relation between X and Y.
Okay.
So high value close to one means
an excellent model and close to zero.
it means that the model does not fulfill the requirements and it is not useful
for modeling the relationship between X and Y.
So, finally we have to say something
about how to estimate the variability of the random term.
And, this it's based on the residual sum of the squares
that we have seen before divided into n-2,
because we have two parameters in the model.
This is usually, in the general case, what we call the mean
square error and the square root of the mean
square error is the estimate of the standard deviation of the model
that it's usually
the residual standard error. This is the terminology
used in R and RStudio.
Okay, So we have the estimate of the variance,
the estimate of the standard deviation, s.
Be careful, because we have here an example that we will detail during
the course in statistics and
adjusting/fitting a straight line
to any of this set can be done
and we can see that for any case
the straight line is 3 + 0.5 X
and the coefficient of determination is 0.6667
So, the straight line explains
two thirds of the target variability.
But, in fact, we can see that
the sort of display of the scatterplot it's not the same.
Okay.
So in the case of B with a curvilinear
relation, it is not suitable to be modeled using a linear model.
And in the case of C, for example, we have an outlier,
because it does not follow the same pattern.
This is a task for model validation.
Okay, so any model can be fitted, but we have to assess
how suitable the model is our data.
This is model validation. Always needed.
Thank you very much for your attention.