Curvilinear Regression

Simple linear regression has been developed to fit straight lines to data points. However, sometimes the relationship between two variables may be represented by a curve instead of a straight line. Such "non-linear" relationships need not be non-linear in a mathematical sense. For example, a parabolic relationship may be well-modeled by a (modified) linear regression, since a parabola is a linear equation, as far as its parameters are concerned. Sometimes, such relationships are called "curvilinear".
 

Please note that the term "non-linear" has a double meaning: first, people use the term when they think of curves which are not straight lines, and secondly, a non-linear relationship in its mathematical sense is a function which relates the x and y variable(s) by one or more non-linear functions (such as a cosine). More details can be found here.

There are several ways to fit a curve other than a line (or, generally speaking, an n-dimensional hyperplane) to the data:
 

• deriving the proper regression formula, which may be cumbersome, and requires some calculus knowledge,
• using linear regression after transforming the data into a linear problem,
• using optimization algorithms to minimize the error surface, and
• using non-linear modeling techniques, such as neural networks.

The first two approaches require the type of functional relationship to be known. In many standard cases, the second approach may be appropriate: