Simple Linear Regression

Simple Linear Regression - SLR

Basic idea: How well does the variation in one variable explain the variation in another?

Simple Linear Regression Model:

[pic 1]

[pic 2]

Y is the dependent variable, X is the independent variable, beta0 is the constant, while beta1 is the coefficient on X. Epsilon() is the error – or what we do not explain in the model. [pic 3]

Model to be estimated:  [pic 4]

The residuals are then: [pic 5]

Assumptions of SLR are focused on these errors (e). We want them to exhibit “white noise”, or more specifically, we want them to be normally distributed with zero mean and constant variance and we want them to be independent.

Starting with a “scatterplot” may help us gain an idea of the statistical relationship between the two variables (X and Y) – the causal relationship cannot come from statistics – it must come from theory or common understanding.

3 levels of interpretation (2-3-1)
level 2) overall model test (F-test)
level 3) Slope and coefficient / individual component testing (t-tests)
level 1) Explanatory power (r2)

Residual Analysis (graphical check of assumptions)
1) normal probability plot of residuals
2) plot residuals by predicted or x values
[3) plot residual by observation (IF time element or other dependency in Y and/or X)]

Interpret the model based on your estimations
- How much of the variation in Y is explained by the model (i.e. variation in X)?
- What is the constant and how can it be interpreted?
- What is the coefficient on X and how do you explain it?