Statistical Learning: Algorithmic and Nonparametric Approaches

• The class outline. For each section, you can obtain the class notes in pdf and the R code used to generate the analyses and graphs.
• Links for homework : data needed, assignments sheets in pdf, and the latex files.
• Books often referenced
• Computing resource.
• Class general information.

1. Review of probability, the central limit theorem, and inference [PDF]
2. Introduction to Regression and Prediction [PDF,R]
3. Overview of Supervised Learning [PDF,R]
4. Linear Methods for Regression [PDF,R]
5. Linear Methods for Classification [PDF,R]
6. Splines, Wavelets, and Friends [PDF,R, S-Plus]
7. Kernel Methods [PDF,R]
8. Model Assessment and Selection [PDF,R]
9. Model Inference and Averaging [PDF]
   • Resampling
   • Bootstrap
   • Bayesian Inference
   • EM and MCMC
10. Additive Models, Trees, and Related Methods [PDF]
    • Boosting and Additive Trees
    • Neural Nets
    • Support Vector Machines
    • Clustering
11. Special lecture on CART, Boosting, Bagging, and Random Forrests [PDF]
12. Special lecture on EM [PDF]

    ---

    Homework:

    • Homework 1 [Due 2/17]: Look through the top journal in your field for a paper in which a regression analysis was performed, many covariates were available, and p-values were reported.

      If your field is mathematical (statistics, biostatistics, engineering,etc..) then look through the top journal of your favorite public health application. If you don't have one then use American Journal of Epidemiology (there should be plenty of regrssion analyses in this journal).

      • Discuss how the model was motivated. Deductively, empirically, both or neither?
      • Give me your thoughts on their model choice? Could they have done something differently? Are the results described model driven?
      • Where does the p in p-value come from? i.e. Where does the randomness come from? Random sample, randomization, or nature...? If nature, then write a paragraph explain how.
    • Homework 2 [Due 3/3]
      • Use this training data to predict the outcomes for this data. You should give the 100 predictions and an estimate of the number of mistakes you've made. Please include a description of what you did. Whomever predicts best wins first prize. Whomever best estimates the number of mistakes they make comes in second.
      • Derive the discrimination function for LDA (third equation on page 65.. but notice there is a mistake) and show it is linear.
      • Show that LDA and regression are equivalent when the outcomes are binary.
    • Homework 3 [Due 3/10]
      • Dowload the Strontium Data [text file] and fit a polynomial of degree 1,2,3,4,6,12, a spline (you pick the knots) and smoothing splines. Make plots of the data and the fitted curves.
      • Write a paragraph describing your project.

    ---

    Data-sets:

    • Prostate Cancer Data [Description, R image, CSV file] ]
    • Vowel Training Data [Description, Train, Test, R Image]
    • Strontium Data [text file]

    ---

    Recommended Books
    • T. Hastie, R. Tibshirani, and J. H. Fried. (2001) The Elements of Statistical Learning. Springer-Verlag: New York. [ Web Page]
    • Venables, W.N. and Ripley, B.D. (2002) Modern Applied Statistics with S-Plus. Springer-Verlag: New York.
    • Brian D. Ripley. (1996) Pattern Recognition and Neural Networks. Cambridge University Press.

    ---

    Resources
    • R

    ---

    Class General Info
    
    • Course title: Statistical Learning: Algorithmic and Nonparametric Approaches (140.649)
    • Lab Hour:
    • Instructor: Rafael Irizarry
    • Department of Biostatistics
    • Phone 410-614-5157, email: rafa@jhu.edu
    • I assume you know: Linear algebra and statistical principles at a 651--654 level.
    • It will be useful to learn one of the following programming languages: R (recommended), S-Plus, or MATLAB.
    • Grading: 3 homeworks 60%, 2 quizzes 20%, 1 project 20%
    • Course description: Teaches public health students to use modern, computationally-based methods for exploring and drawing inferences from data. After a brief review of probability, the central limit theorem, and inference, the course covers resampling methods, non-parametric regression, prediction, and dimension reduction and clustering. Specifically covers: Monte Carlo simulation, bootstrap cross-validation, splines, local weighted regression, CART, random forests, neural networks, support vector machines, and hierarchical clustering.

    ---

    Last updated: 1/18/2004
    
    
    

    You are visitor number