Syllabus

Foundations:

  1. Computational Complexity
  2. Introduction to R / basic programming.
  3. Introduction to Algorithms and basic concepts (e.g. sorting)
  4. Divide-and-Conquer Strategy.
  5. Linear Systems (Gaussian Elimination, Cholesky Decomposition etc.)

Sampling:

  1. Accept-Reject Scheme
  2. Rejection sampling and Envelope.
  3. Advanced Sampling Techniques:
    • Monte Carlo Integration
    • Importance Sampling, SIR, Adaptive Rejection Sampling.
  4. Markov Chain Monte Carlo.
    • Basic Theory.
    • Metropolis & Metropolis-Hastings.
    • Gibbs sampler.
  5. Sampling strategies for Bayesian Inference.

Statistical Learning:

  1. Unsupervised and Supervised Learning.
  2. Examples.
  3. Bias-Variance Decomposition.
  4. Accuracy vs. Model Interpretability.

Unsupervised

  1. K-nearest neighbour
  2. PCA.

Supervised

  1. Modern Regression: (large \(p\), small \(n\): wide data)
  • Basics - Geometry of Regression.
  • Subset selection: forward selection/backward selection/Best subset - AIC.
  • Shrinkage and Selection: Ridge, LASSO, Elastic Net.
  • Cross-validation (general)
  • Bayesian Lasso and Horseshoe.
  • Principal Components Regression.
  1. Classification:
  • Logistic Regression (with \(\ell_1\) penalty).
  • LDA and QDA.
  • Comparison.
  1. Decision Tree
  2. Bagging, Boosting.
  3. Bootstrap (general)

Random Forest.

Support Vector Machines.

Multidimensional Scaling.

In-class examples

All the R examples shown in class for Stat 5443 (Computational Statistics) are given below as a list. Some of the codes are my own and the rest are either derived or taken from the R codes are taken from various resources such as matrix examples in R tutorial by Prof. Giovanni Petris, MCMC examples by Prof. David Draper and the R codes accompanying the ISLR book.

The book website also has a link to the MOOC for Statistical Learning by Prof.s Hastie and Tibshriani which has a large interesection with the second half of the class : starting from modern regression to random forest. It’s a great lecture series.

  1. Matrix Example in R
  2. Random Variable Generation
  3. Lab Example: Quantile Transform
  4. Importance Sampling
  5. PCA
  6. Modern Regression 1
  7. Modern Regression 2
  8. Lab on GLMNET
  9. Lab on GLMNET & Solution
  10. Modern Regression 3
  11. Logistic Demo
  12. Final Project Data
  13. Decision Tree, Random Forest
  14. Multidimensional Scaling
  15. Support Vector Machine