The analysis of variance (ANOVA) is a tool to find
those factors in a multidimensional model which influence the model most.
This can be primarily reduced to the question whether the means of several
samples are the same. The samples are, in general, not independent of each
other and are often obtained from a designed
experiment (factorial design).
In order to compare several means one could possibly use a two-sample
t-test for each pair of samples. While this seems reasonable at first,
closer inspection reveals at least four drawbacks to this approach:
the number of pairs is n*(n-1)/2 which results in a large number
the level of significance is automatically increased by performing
multiple t-tests. If we, for example, define a level of significance of
= 0.01 for the individual tests, the probability of avoiding a type I error
is 0.99. If we have to perform k independent tests, the overall probability
of avoiding a type I error on all tests is (1-α)k.
This means that the propability of making a type I error (which is the
level of significance for all tests) is 1-(1-α)k.
For α = 0.01 and 10 means which have to be compared
(resulting in 10*(10-1)/2 = 45 t-tests) this would create an overall level
of significance of 0.364, which is rather poor.
the individual tests are not independent of each other. Suppose
we have three samples and we have to compare three means. If we know the
differences between two pairs of means, we immediately know the difference
between the means of the third pair, i.e. only two of the differences are
independent. Again this increases the probability of making a type I error
(or, equally, increasing the level of significance).
the individual tests may produce contradictory results. In the case
of an n-sample problem only one of the t-tests may be significant. This
means that two of the means are not equal, while all other pairs of means
are equal. However, this is a contradictory result, since the result of
that particular t-test could be calculated from the results of all other
t-tests - and those have been found to be equal.
In order to avoid these problems, R.A. Fisher introduced a method
which is commonly called "analysis of variance". The idea behind the ANOVA
is that any differences among population means should be reflected
in the variance among the samples obtained from these populations.