Probability
Introduction

Probability is often used synonymously with "chance", "risk" or "odds", etc. The probability tells us how likely an event is. What are our chances of winning the lottery? What are the odds that we are hit by a hurricane ? What is the risk of this investment? Assessing and estimating the probability of an event is often very difficult. But in some very well-defined and simple situations, the probabilities can be assigned in a straightforward manner.

When all outcomes of an experiment are of equal likelihood and when we know the sample space, then we can assign to each sample point a probability equal to 1/N, with N being the number of sample points. We can conclude this, since only one of the N outcomes is possible for each experiment.

Example 1:

When we toss a coin, we know that there are two possible outcomes (the sample space), therefore we conclude that we have a 50:50 chance of seeing a head, unless we suspect that the coin is not well-balanced.

For complex experiments, the number of sample points is often too large to be listed explicitly. We therefore need more sophisticated 'counting rules' for determining the number of sample points.
 
 

Example 2:

What is the probability of throwing exactly 2 heads with 3 coins?
The sample space is:

HHH     TTH
THH     THT
HTH     TTH
HHT     TTT

So the probability is 3/8 to get exactly 2 heads with 3 coins.

Example 3:

What is the probability of throwing exactly 3 heads with 5 coins?
The possible combinations of getting 3 heads are:

TTHHH           HTTHH           HHTTH           HHHTT
THTHH           HTHTH           HHTHT
THHTH           HTHHT           
THHHT.

In order to get the probability, we must know all the other possible outcomes. There are 32 of them. You can arrive at this number through the following reasoning:
 

number of heads	number of possible outcomes with the specified number of heads	comment
0	1	there is only one way
1	5	the head can be any of the five positions
2	10	if we exchange H and T in the list above
3	10	we have listed them above
4	5	the same as one tail
5	1	there is only one way
Sum	32

When we cannot rely on the assumption that all sample points are equally likely, we have to determine the probability of an event experimentally. We perform a large number of experiments N and count how often each of the sample points is obtained. The ratio of the number of occurrences of a certain sample point to the total number of experiments is called the relative frequency.

The probability is then assigned the relative frequency of the occurrence of a sample point in this long series of repetitions of the experiment. This is based on the axiom, called the "law of large numbers", which says that the relative frequency approaches the true (theoretical) probability of the outcome if the experiment is repeated over and over again.

Where n(E) is the number of times, the event E took place out of a total of N experiments. From this definition we can see that the probability is a number between 0 and 1. When the probability is 1, then we know that a particular outcome is certain.
 
 

Definition:

The probability of a sample point measures the likelihood that a specific outcome will occur when the experiment is performed. It is a number between 0 and 1. Requirements: All probabilities have to be 0 £P(E) £ 1. The probabilities of all the sample points have to add up to one. P(S) = 1.