Fundamentals of Statistics contains material of various lectures and courses of H. Lohninger on statistics, data analysis and chemometrics......click here for more. 
Home Multivariate Data Modeling Validation of Models Validation of Models  
See also: PRESS, chance correlation, crossvalidation  
Validation of ModelsCreating new models from a finite amount of data always includes a (small) chance that the model will not reflect the underlying relationship, but has been caused by random effects. The chance of invalid models increases with a decreasing number of measurements and an increasing number of variables. This has led to the rule of thumb (which is often too loose, especially with nonlinear methods) that the number of measurements has to be at least three times the number of variables in the model. Some (linear) multivariate methods provide theoretical foundation on the estimation of the reliability of such a model. When it comes to more sophisticated methods, or to nonlinear methods, the resulting models have to be validated by a heuristic approach. In principle, there are several methods to perform this, certain ones often being tailored to a specific model.
One approach for validation, however, always performs quite well.
This approach is called crossvalidation,
also known as the "leaveoneout" method. Crossvalidation permits the determination of a measure for the prediction
error called PRESS (prediction error sum of
squares). Another little used procedure for the validation of models is the addition of noise and checking the reaction of the model.


Home Multivariate Data Modeling Validation of Models Validation of Models 