Fundamentals of Statistics contains material of various lectures and courses of H. Lohninger on statistics, data analysis and here for more.

Linear vs.
Nonlinear Models

Most people have difficulties in determining whether a model is linear or non-linear. Before discussing the issues of linear vs. non-linear systems, let's have a short look at some examples, displaying several types of discrimination lines between two classes:

Have you already guessed the difference between linear and non-linear models ? Here's the answer: linear models are linear in the parameters which have to be estimated, but not necessarily in the independent variables. This explains why the middle of the three figures above shows a linear discrimination line between the two classes, although the line is not linear in the sense of a straight line.

Another example of a linear model is shown in the figure below. It displays a parabolic regression line, which of course has a curvature, but is a linear model:

It's not the independent variable, x, which counts for linearity, but the parameters of the model (in our parabolic example a, b, and c). From this simple insight it follows that multiple linear regression can be used to estimate the parameters of "curved" models.

Last Update: 2012-10-08