News

2014.11.18: Will's office hour on THURS, NOV 20 is cancelled. Office hours will be held as usual on Tuesdays at 2:00 pm and Thursdays at 3:00 pm from Nov 25 through Dec 4 (our final exam is on Dec 6).

2014.10.15: Professor Salibian has set up a STAT 447B website for the second half of the course

2014.10.14: Materials from today's class posted under Course Materials; class schedule updated to the end of the first half of the course

2014.10.09: Assignment 2 (due Oct 23) posted under Assignments and Quizzes; it will shortly be available at connect.ubc.ca too

2014.09.18: Quiz 2 will be on November 4

2014.09.10: Office hours for David and Will can be found under Office Hours below

2014.09.02: WELCOME TO STAT 447B! News items will be posted here.

Teaching Team

Instructors: Professor Will Welch (will [at] stat [dot] ubc [dot] ca) will teach the first half of the course (up to and including Tuesday, October 14). Professor Matias Salibian will take over for the second half.

Teaching Assistant: David Lee (david [dot] lee [at] stat [dot] ubc [dot] ca)

About the course

The course is an introduction to modern methods of regression and classification in statistical learning (machine learning). It builds upon the more classical regression and classification models you have seen in STAT 306 or equivalent. There will be a strong emphasis on applications and interpretation. We will use computer software intensively, even during lectures. Students will be encouraged to bring a laptop to class and work on the examples discussed in the lectures alongside the instructor. Participation through short in-class activities, discussion, and clicker responses is expected.

Office Hours

David Fridays 1:00-2:00 in ESB 1043 (Department of Statistics Learning Resource Centre, ESB first floor)
Will Tuesdays 2:00-3:00 and Thursdays 3:00-4:00 in ESB 3132

Course Texts

You will need a CLICKER, available at the bookstore. See How to Register Your Clicker to connect your clicker to the course.

"An Introduction to Statistical Learning", James, G., Witten, D., Hastie, T., and Tibshirani, R., 2013, Springer, New York
is the text you will probably find most useful. It is available online at the UBC library.

A list of reference books is provided in the course outline.

How to Register Your Clicker

1. Login to connect.ubc.ca with your CWL and click on the course
2. Choose STAT 447B (it should be available if you are registered for the course)
3. Click on the link "i>clicker registration" under "Course content"
4. Enter the information requested to register the code on the back of your clicker.

R Software

How to install the R software on your own PC

Basic R commands (Lab 0 from David Lee's website) Handout R code

An Introduction to R (a fairly lengthy tutorial)

R web site

Labs

David Lee has set up a lab website for current and past labs

Course Materials

Note that all files for R code have a ".R" extension. They are plain text files and can be opened in any plain-text editor.

Course Outline
Schedule of topics by class cross-referenced to the James et al textbook, revised October 14 for the first half of the course

September 4 class: Slides Clicker questions (Answers: Q4,A)

September 9 class: Slides Clicker questions (Answers: Q1,B; Q2,D; Q3,C or A; Q4,B)
R code for the Lidar data: lidar_linear_nolinear.R (linear versus nonlinear models) lidar_poly.R (polynomial models) lidar_linear_splines.R (linear splines)
R code for the trade-union data: trade-union_age_nonlinear.R (linear versus nonlinear models)

September 11 class: Slides Clicker questions (Answers: Q1,E; Q2,B; Q3,E; Q4,E; Q5,A)
R code for the Lidar data: lidar_linear_splines5v10.R (linear splines, 5 versus 10 knots; includes spline.basis, an R function to compute the X matrix for linear, quadratic, etc. splines)
The function spline.basis was revised 2013.09.13; the previous version worked for all examples to date but needed revision for future examples.)

September 16 class: Slides Clicker questions (Answers: Q1,D; Q2,B; Q3,E; Q4,A; Q5,D (not polled); Q6,D)
R code for cross validation applied to the Lidar data: lidar_splines_cv.R (some variables have been renamed for better readability of the code, as discussed in class)

September 18 class: Slides Clicker questions (Answers: Q1,A; Q2,D)
ethanol_penalized.R (R code for penalized splines and GAM applied to ethanol data)

September 23 class: Slides (no clicker questions; see Sep 25 class for R code)

September 25 class: Slides Clicker questions (Answers: Q1,A; Q2,D; Q3,B; Q4,E; Q5,C; Q6,B (not polled); Q7,C)
vaso_logistic.R (R code for logistic regression via glm applied to vaso data)

September 30 class: Slides Clicker questions (Answers: Q1,B; Q2,E; Q3,C; Q4 to be called again next class)
Digit recognition example: Data digits_logistic_cv.R (R code for logistic regression and cv)
R code for Vaso constriction data: vaso_logistic_plot_2d.R (2-d plot of logistic fit) vaso_gam.R (gam and cv) vaso_lda_plot_2d.R (2-d plot of lda fit)

October 2 class: Slides Clicker questions (Answers: Q1,A; Q2,D; Q3,B)
Admissions Data example: Data admissions_lda.R (R code for 3-class LDA) admissions_logistic.R (R code for 3-class logistic fit)

October 7 class: Slides Clicker questions (Answers: Q1,D; Q2,C; Q3,D; Q4,A; Q5,D)
Digits recognition example (all 10 classes, same data as Sep 30): digits_knn.R (R code for k-NN)
Checkerboard simulated data: checktrain.txt (training data) checker_rpart.R (R code for tree using function rpart) checker_tree.R (R code for tree using function tree)

October 9 class: Slides (some trees redrawn after class for better legibility) Clicker questions (Answers: Q1,C; Q2,C; Q3,E; Q4,B)
Admissions Data example: Data admissions_tree.R (R code for 3-class trees)

October 14 class: Slides Clicker questions (Answers: Q1,E; Q2,C; Q3,B; Q4,C)
Prostate data

Assignments and Quizzes

Please submit your solution electronically to connect.ubc.ca.

Questions calling for R work will have solutions consisting of R code, results from R, and short text explanations or comments. All this can be arranged as plain text.

Other questions require small amounts of mathematical notation. Some of you might use Microsoft Word, and some might even know Latex. Then you will be able to upload the material directly. This is not absolutely necessary, and you may scan a hand-written solution for this part of your solution if you prefer.

Assignment 1 (due Friday, Sep 19) Assignment 1 Solution Assignment 1 R code

Assignment 2 (due Thursday, Oct 23)
As mentioned in class, you should choose the data set 1-6 at the web site according to your last (family) name: A-C 1; D-K 2; L 3; M-S 5; T-Z 6. e.g., David Lee would use data set 3 and Will Welch would use data set 6.

Quiz 1 Solution

Useful Links

Department of Statistics

Set of videos on how to study by Stephen L. Chew, a cognitive psychologist