Probability — Beginner

Probability Spaces and Discrete Distributions (First- to Second-Year University Level)

Overview of the Beginner level

The Beginner level covers the foundations of probability theory taught in the first year of university. It rebuilds the intuitive understanding of probability from the Introduction into a mathematically rigorous form.

Learning goals

  • Understand the definition and structure of a probability space
  • Handle conditional probability and independence rigorously
  • Apply Bayes' theorem
  • Compute discrete random variables and their expected values
  • Understand the main discrete distributions (binomial, Poisson, and so on)

Contents

  1. Introduction

    Goals and structure of the Beginner level

  2. Probability Space

    Sample space $\Omega$, events, probability measure $P$, the axioms of probability

  3. Conditional Probability and Independence

    Definition of conditional probability, independent events, independent trials

  4. Bayes' Theorem

    The law of total probability, Bayes' theorem and its applications

  5. Random Variables

    Definition of a random variable, probability mass function, cumulative distribution function

  6. Representative Distributions

    Bernoulli, binomial, geometric, and Poisson distributions

  7. Properties of Expected Value

    Expected value $E[X]$, linearity, variance $V[X]$, covariance

Topic pages

  1. Probability Space

    Rigorous definition of a probability space, $\sigma$-algebra, probability measure

  2. Bayes' Theorem

    Derivation of Bayes' theorem, prior and posterior probabilities, worked examples

  3. Variance

    Definition and properties of variance, standard deviation, Chebyshev's inequality

  4. Uniform Distribution

    PDF and CDF of the continuous and discrete uniform distributions, mean and variance, inverse transform method, order statistics, maximum entropy

  5. Bernoulli Distribution

    Definition, mean and variance, MGF, relation to the binomial distribution, exponential family, examples

  6. Binomial Distribution

    Bernoulli trials, mean and variance, normal approximation, Poisson approximation

  7. Hypergeometric Distribution

    Sampling without replacement, finite population correction, binomial approximation, Fisher's exact test, acceptance sampling, capture-recapture

  8. Multinomial Distribution

    Multinomial coefficients, probability mass function, mean, variance, covariance, marginal and conditional distributions, applications

  9. Normal Distribution

    Definition of the normal distribution, standard normal distribution, central limit theorem

  10. Poisson Distribution

    Definition, derivation from the binomial distribution (law of rare events), mean and variance, moment generating function, Poisson process, reproductive property, normal approximation

  11. Geometric Distribution

    Definition, the two conventions, mean and variance, memorylessness, moment generating function, relation to the negative binomial distribution, examples

  12. Negative Binomial Distribution

    Definition, probability mass function, mean and variance, relation to the geometric distribution, moment generating function, reproductive property, Poisson-gamma mixture, examples

  13. Exponential Distribution

    Definition, mean and variance, memorylessness, relation to the Poisson process, moment generating function, hazard function, examples

  14. Chebyshev's Inequality

    Markov's inequality, proof of Chebyshev's inequality, one-sided Chebyshev (Cantelli's inequality), equality conditions, the weak law of large numbers, comparison with other probability inequalities

  • Triangular Distribution — A triangular density on the interval [a,b] peaking at the mode c...
  • Discrete Uniform Distribution — Definition, probability mass function, mean and variance, dice...
  • Categorical Distribution — Definition, probability mass function, mean and variance, relation to the multinomial and Dirichlet distributions...
  • Distribution Function — Definition and properties of the cumulative distribution function (CDF), and its relation to probability density and mass.
  • Indicator Function — The indicator function that represents an event as 0/1, and its relation to expectation and probability.
  • Formal Definition of a Probability Space — A rigorous formulation of the probability space (Ω, F, P) via σ-algebras and a probability measure (Kolmogorov's axioms).

Prerequisites