The purpose of this programming assignment is for you to get a hands-on experience
with confidence intervals. This will be useful for you to understand how reliable
is the predicting of the performance of a classifier.
Program the following experiment in a programming language of your choice.
- Simulate a biased coin toss with probability of success p (use p as a parameter).
In mathematical terms this is a Bernoulli trial with probability p.
- Now simultate tossing N such biased coins and count the proportion of successes that you had.
- Repeat the above experiment 10000 times and plot the distribution of successes by using a histogram with 100 intervals.
Do the experiments for p=0.75, 0.80, 0.90, and for each case try N=100 and N=1000.
- For each case, use your statistical knowledge to compute 80%, 90%, and 99%
confidence intervals for p (as if it was unknown to you). Do the experimental results match the theory?