Fundamentals of Statistics contains material of various lectures and courses of H. Lohninger on statistics, data analysis and here for more.

Optimization - Introduction

Optimization problems are arising in nearly all fields of science and technology. Therefore the range of problems is huge and a large number of possible approaches for solutions are available. Here are some examples of optimization problems:

    A company producing electronic devices needs to optimize the quality assurance procedure: more elaborate testing increases the overall production costs, while an excessive amount of low quality products will result in complaints and a degradation in the image of the company.

    A mathematician has to optimize the bus schedule of a city. The timing of the buses has to be optimized considering several constraints: a minimum number of buses per hour; optimum alignment of the departure times to the higher order railway and subway system; unusual traffic densities at certain locations which may slow down the buses; ....

    An analytical chemist has to set up a mass spectrometer for high resolution measurements. One of the prerequisites is that the voltages at the deflection lenses of the ion accelerator have to be set in a way that the ion beam shows a rectangular cross-section with uniform ion density.

These examples give a quick impression of the diversity of optimization problems. In more general terms, the optimization process can be defined as finding the minimum (or the maximum) of a response function f (sometimes called an objective function or quality function). The response function may be rather complicated and of high dimensionality. From a computational standpoint, any optimizing has to be done quickly and cheaply, which usually means that the response function f should be evaluated as few times as possible during the optimization process.