Regression Theory and Analysis

Syllabus Details

• Unit I:(Simple and Multiple Linear Regression) Simple linear regression model, least-squares estimation, hypothesis testing on the slope and intercept, interval estimation, prediction of new observations, coefficient of determination, regression through the origin, maximum likelihood estimation of parameters, case with regressor x as random; multiple regression models: estimation, hypothesis testing, confidence intervals, prediction of new observations, hidden extrapolation in multiple regression, standardized regression coefficients, multi-collinearity.
• Unit II:(Model Adequacy, Diagnostics and Polynomial Regression) Model adequacy checking: residual analysis, PRESS statistic, detection and treatment of outliers, lack of fit of the regression model; transformations and weighting to correct model inadequacies: variance-stabilizing transformations, transformations to linearize the model, analytical methods for selecting a transformation, generalized and weighted least squares, regression models with random effect; diagnostics for leverage and influence: influential observations, leverage, Cook’s D, DFFITS and DFBETAS, a measure of model performance, groups of influential observations, treatment of influential observations; polynomial regression models: models in one variable, nonparametric regression, models in two or more variables, orthogonal polynomials.
• Unit III:(Indicator Variables, Multicollinearity and Model Building) Indicator variables: concept, use of indicator variables, regression approach to analysis of variance; multicollinearity: sources, effects, diagnostics, methods for dealing with multicollinearity; variable selection and model building: techniques for variable selection, strategy for variable selection and model building; validation of regression models: techniques.
• Unit IV:(Nonlinear Regression and Other Topics) Nonlinear regression: models, origins of nonlinear models, nonlinear least squares; other topics: robust regression, measurement errors, calibration problem, bootstrapping in regression.

Course Learning Objective

1. Discuss the different types of regression and when these are applicable.
2. Visualize simple linear regression model fits.
3. Discuss the role of residuals.
4. Discuss the concept of multiple linear regression models and how these can be constructed.
5. Discuss and apply different regression diagnostics.
6. Discuss and apply alternative regression models.
7. Apply the methods in a software and interpret the output from the same software.

Recommended Books

1. Montgomery, D. C., Peck, E. A. and Vining, G. G. Introduction to Linear Regression Analysis. 3rd ed. John Wiley, 2003
2. Draper, N. R. and Smith, H. Applied Regression Analysis. 3rd ed. John Wiley & Sons, NY, 2005
3. McCullagh, P. and Nelder, J. A. Generalized Linear Models. 2nd ed. CRC Books, London, 2000
4. Seber, G. E. F. and Wild, C. J. Nonlinear Regression. John Wiley, NY, 1989
5. Neter, J., Kutner, M., Wasserman, W. and Nachtsheim, C. Applied Linear Statistical Models. 4th ed. McGraw-Hill/Irwin, 1996