Overall test and Individual tests

In this blog, i'm going to talk about overall test and individual tests.

 

1. Overall test : This is also referred to as the overall F-test of a multiple regression model. An overall test helps us decide whether predictors are related to the response in the population.

 1-1) hypotheses : we also specify the null hypothesis when we conduct overall test. If there's no relation between the predictors and the response variable, this means that neither predictors helps to predict the reponse variable. In other words, the regression coefficient for both predictors will be 0. This is how we visualize it.

On the other hand, the alternative hypothesis states that at least one of the predictors is related to the response variable. The plane will no longer be flat. In other words, at least one, several or all of the regression coefficients will differ from 0.

** If there is a relation between the set of predictors and the response variable, we still don't know which predictors contribute. To find out which predictors contribute, we'll follow-up with individual tests of the regression coefficient.

** The overall test is associated with a number of assumptions that need to be met in order for the test to give valid results. 

 

 1-2) How to perform : To compute the test statistic F, we take the regression and error sums of squares(the residual sum of squares). We turn these sum of squares into variance and we divide them.

** the word 'mean square' is the same as variance. The regression mean square means the variance in the response variable captured by our model. The residual(error) mean square represents the variance of the residuals that we failed to capture with our model. So the F-test statistic is the explained variance divided by the error variance.

** The F test statistic also needs the degrees of freedom.

The first degree of freedom is often referred to as the numberator, or regression degree of freedom. And the second is often called the denominator or error degree of freedom.

 

2. Individual test : The obvious follow-up question after calculating overall F statistic is, which of the predictors are reponsible for the significanct overall effect? To answer this question, we perfomr individual follow-up t-tests for each predictor to assess its relationship with the response variable while controlling for the other predictors. The assumptions that need to be met in order for the t-test to give valid results are the same as for the overall test. But, in this situation, we assume all the assumptions are met.

 2-1) hypotheses : the null hypothesis states that for a particular predictor, the regression coefficient equals 0 while controlling for the other predictors. The alternative hypothesis says that the regression coefficient does not equal to zero. Directional alternative hypotheses are possible.

 2-2) The test statistic : We use the same t test statistic as we did in simple linear regression. Here the formula is.

** Notice that in this case, we use different the degrees of freedom, the number of observations minus k. K is the number of predictors plus 1 for the response. In simple linear regression, we used different the degrees of freedom. It was the total number of cases minus 2 that is for the intercept and the regression coefficient.

 2-3) The confidence interval : we can also calculate confidence intervals for the regression coefficients. The formula is the same as in simple regression.