Statistics

Statistics

About This Series

Statistics is the discipline of extracting information from data and making decisions under uncertainty. This series provides a step-by-step learning path, starting from the fundamentals of descriptive statistics through inferential statistics, maximum likelihood estimation, and Bayesian statistics.

Statistics is essential knowledge in every field that deals with data, including scientific research, business analysis, and machine learning.

Learning by Level

Introduction

Accessible with high-school-level mathematics

  • Data organization
  • Measures of central tendency (mean, median)
  • Variance and standard deviation
  • Scatter plots and correlation

Basics

University years 1-2 level

  • Probability distributions
  • Central limit theorem
  • Point and interval estimation
  • Hypothesis testing

Intermediate

University years 3-4 level

  • Maximum likelihood estimation
  • Regression analysis
  • Analysis of variance
  • Multivariate analysis

Advanced

Graduate level

  • Bayesian statistics
  • Asymptotic theory
  • EM algorithm
  • Model selection

Learning Flow

A flow diagram showing the four-stage learning path from introduction to advanced, and the branching into frequentist statistics and Bayesian statistics at the intermediate level and beyond. Introduction High School Basics University 1-2 Intermediate University 3-4 Advanced Graduate Intro: Data organization, central tendency, variance, correlation Basics: Probability distributions, CLT, estimation, testing Intermediate: MLE, regression, ANOVA, multivariate Advanced: Bayesian statistics, EM algorithm, model selection Two Major Approaches to Statistical Inference Frequentist Statistics Parameters are fixed but unknown Repeated sampling Confidence intervals, p-values Bayesian Statistics Uncertainty about parameters is a distribution Prior → Posterior distribution Credible intervals, posterior probability

Key Topics

Descriptive Statistics

Techniques for summarizing data, including data organization, measures of central tendency, measures of dispersion, and correlation.

Inferential Statistics

Inferring population characteristics from samples through interval estimation and hypothesis testing.

Regression Analysis

Methods for modeling relationships between variables and applying them to prediction.

Bayesian Statistics

The Bayesian approach to inference, combining prior knowledge with data.

Individual Topics

What is GMM (Gaussian Mixture Model)? Complete Derivation of the EM Algorithm

Explains the definition, mechanism, and applications of GMM (Gaussian Mixture Model), and provides a complete derivation of the estimation of mean, variance-covariance matrix, and mixing coefficients via the EM algorithm, without omitting any intermediate calculations.

Prerequisites

  • The introduction is accessible with high-school-level mathematics (reading formulas, reading basic graphs)
  • From the basics level onward, a grounding in calculus is desirable (probability density functions and expectations of continuous distributions are handled through integrals)
  • From the intermediate level onward, linear algebra is used (for multivariate analysis)
  • At the advanced level, a grounding in probability theory and analysis makes the material easier to follow

Frequently Asked Questions

What prerequisite knowledge is needed to study statistics?

The introductory level is accessible with high-school-level mathematics (reading formulas, reading basic graphs). From the basics level onward, a grounding in calculus is desirable, since probability density functions and expectations of continuous distributions are handled through integrals. From the intermediate level onward, linear algebra is used as well, for multivariate analysis.

What is the difference between frequentist statistics and Bayesian statistics?

Frequentist statistics treats a parameter as a fixed but unknown value and carries out estimation and testing under the idea of drawing samples repeatedly (confidence intervals, p-values). Bayesian statistics represents the uncertainty about an unknown parameter with a probability distribution, updating a prior with data to obtain a posterior (credible intervals, posterior probabilities). The difference lies in how each treats probability and unknown parameters; it is not a contrast between an objective and a subjective approach. Frequentist analyses also involve the analyst's judgement in choosing models, test procedures and significance levels, and Bayesian statistics has objective-Bayes and weakly informative priors.

In what order should statistics be studied?

It is effective to progress in this order: Introduction (descriptive statistics, data visualization) → Basics (probability distributions, normal distribution, central limit theorem, point and interval estimation, hypothesis testing) → Intermediate (maximum likelihood estimation, regression analysis, analysis of variance, multivariate analysis) → Advanced (Bayesian statistics, asymptotic theory, the EM algorithm, model selection). Each level builds upon the knowledge from the previous level.