Statistical Learning and Visualization

In eight weeks we will dive into statistical learning and visualization in R. We will focus on supervised learning techniques and data wrangling in the context of data analysis and inference, as well as the connection to research philosophy. During every lecture we will treat a different theoretical aspect. Following each lecture there will be a computer lab exercise that connects the statistical theory to practice, as well as a Q&A meeting (Wednesdays @ 11am - Dalton 500 8.27) wherein you can pose any and all questions that remain unanswered about the course materials, theory, practice and your practical assignments.

Assignment and Grading

The final grade is computed as follows

To develop the necessary skills for completing the assignments and the presentations, R exercises must be made and submitted via the course Blackboard page. These exercises are not graded, but students must fulfill them to pass the course.

In order to pass the course, the final grade must be 5.5 or higher, your contribution to the course should be sufficient, all R exercises should be handed in and all assignments and the final exam must have a passing grade. Otherwise, additional work is required concerning the assignments and/or exercises you have failed.

Schedule

Course description

Supervised learning is such an integral part of contemporary data science, that you will most likely use it dozens of times a day, without knowing it. In this class you will learn about the most effective supervised learning techniques and you will acquire the skills to implement them to work for you.We will not only discuss the theoretical underpinnings of supervised learning, but focus also on the skills and experience to rapidly apply these techniques to new problems.

During this course, participants will actively learn how to apply the main statistical methods in data analysis and how to use machine learning algorithms and visualizing techniques. The course has a strongly practical, hands-on focus: rather than focusing on the mathematics and background of the discussed techniques, you will gain hands-on experience in using them on real data during the course and interpreting the results. This course provides a broad introduction to supervised learning and visualization. Topics include:

• Data manipulation and data wrangling with R
• Data visualization
• Exploratory data analysis
• Regression and classification
• Non-linear modeling
• Bagging, boosting and ensemble learning

Students will learn to adapt these techniques in their way of thinking about analyses problems. We will consider statistical learning techniques in the context of estimation, testing and prediction. Students will learn to adapt these techniques in their way of thinking about statistical inference, which will help students to quantify the uncertainty and measure the accuracy of statistical estimates. Students will develop fundamental R programming skills and will gain experience with tidyverse, visualize data with ggplot2 and perform basic data wrangling techniques with dplyr. This course makes students better equipped for a further career (e.g. junior researcher or research assistant) or education in research, such as a (research) Master program, or a PhD.

Assignments

Students will form groups to choose work on two assignments. Students will need to perform calculations and program code for these assignments. All work needs to be combined in an easy understandable, self-contained and insightful RStudio project and must be submitted to the course Blackboard page. Each assignment will be graded.

Grading

Students will be evaluated on the following aspects:

1. apply and interpret the theories, principles, methods and techniques related to contemporary data science, and understand and explain different approaches to data analysis:

• apply data wrangling and preprocessing techniques to tidy data sets
• apply, implement, understand and explain methods and techniques that are associated with statistical learning, including regression, trees, clustering, classification techniques and learning ensembles in R
• evaluate the performance of these techniques with appropriate performance measures.
• select appropriate techniques to solve specific data science problems
• motivate and explain the choice for techniques to investigate data problems
• implement and understand generic data science tools, such as bootstrapping, cross validation, bagging, boosting and error evaluation in R
• interpret and evaluate the results of analyses and explain these techniques in simple terminology to a broad audience
• understand and explain the basic principles of data visualization and the grammar of graphics.
• construct appropriate visualizations for each data analysis technique in R

Relation between assessment and objective

In this course, skills and knowledge are evaluated on these separate occasions:

1. With the exam and the assignments the knowledge from methodological and statistical concepts is evaluated, as well as the application of these concepts to research scenarios. During the exam students need to interpret, evaluate and explain statistical software output and results.
2. With the practical lab and the assignments it is tested if the student has sufficient skills to solve analysis problems and execute the relevant methodology on real-life data sets.

After taking this course students can understand innovations in statistical markup, statistical simulation and reproducible research. Students are also able to approach challenges from different professional viewpoints. They have gained experience in marking up a professional manuscript and designing a state-of-the-art statistical archive in an open source repository.

Preparing your machine for the course

Dear all,

This semester you will participate in the Supervised Learning & Visualization course at Utrecht University. To realize a steeper learning curve, we will use some functionality that is not part of the base installation for R. Many of you are already familiar with R. The below guide serves as a point of departure for those who are not. The following steps guide you through installing both R as well as some of the necessary packages.

We look forward to seeing you all,

David Goretzko, Maarten Cruyff, and Erik-Jan van Kesteren

System requirements

Bring a laptop computer to the course and make sure that you have full write access and administrator rights to the machine. We will explore programming and compiling in this course. This means that you need full access to your machine. Some corporate laptops come with limited access for their users, I therefore advice you to bring a personal laptop computer to the workgroup meetings.

1. Install R

R can be obtained here. We won’t use R directly in the course, but rather call R through RStudio. Therefore it needs to be installed.

2. Install RStudio Desktop

Rstudio is an Integrated Development Environment (IDE). It can be obtained as stand-alone software here. The free and open source RStudio Desktop version is sufficient.

3. Start RStudio and install the following packages.

Execute the following lines of code in the console window:

install.packages(c("ggplot2", "tidyverse", "magrittr", "micemd", "jomo", "pan", 
                 "lme4", "knitr", "rmarkdown", "plotly", "ggplot2", "shiny", 
                 "devtools", "boot", "class", "car", "MASS", "ggplot2movies", 
                 "ISLR", "DAAG", "mice", "mlr3verse"), 
                 dependencies = TRUE)

If you are not sure where to execute code, use the following figure to identify the console:

Just copy and paste the installation command and press the return key. When asked

Do you want to install from sources the package which needs 
compilation? (Yes/no/cancel)

type Yes in the console and press the return key.

What if the steps to the left do not work for me?

If all fails and you have insufficient rights to your machine, the following web-based service will offer a solution.

1. Open a free account on rstudio.cloud. You can run your own cloud-based RStudio environment there.
2. Use Utrecht University’s MyWorkPlace. You would have access to R and RStudio there. You may need to install packages for new sessions during the course.

Naturally, you will need internet access for these services to be accessed.

Assignment 1

You can find this year’s assignment 1 here

Due date is Oct 1 @ 23.59pm.

Assignment 2

You can find this year’s assignment 2 here

Due date is Nov 11 @ 23.59pm.

Rubric for Grading

You can find the rubric that we will use for grading here

Version Control

If you want to use version control for your group work, you might want to check out GitHub. It is not mandatory to use it, but if you want to give it a try, you can study this GitHub tutorial. You can also use GitHub Desktop to handle your commits and merge requests. Don’t make your workflow too complicated, though!

Lecture (Monday 9am - Dalton 500 8.27)

Please find the Lecture Slides here

Practical

This week’s practical is split into two parts - practical 1 on data wrangling and basics in R and practical 2 on data visualization with ggplot2. The answers are also given to you. Use these answers to help yourself when you’re stuck.

• Practical 1a: Data wrangling

• Practical 1a: Data wrangling with answers

• googleplaystore.csv

• students.xlsx

• SLV_practical1 - or create your own project

• Practical 1b: Data visualization

• Practical 1b: Data visualization with code solutions

Hand in your answers to both practicals separately via blackboard.

A must watch

The following lecture by Dewey Dunnington is quite informative. Also for those that already (think they) know the ggplot2 package.

Required reading

Study the following materials

• ISLR V2 Chapter 1
• R4DS Chapters 1 -3 and 9 - 16

Useful links

The following links may be useful

• Gerko’s quick R scripting course with video walkthroughs
• ISLR’s web page
• ggplot’s reference page

Lecture (Monday 9am - Dalton 500 8.27)

This week’s slides:

• Lecture Slides part 1
• Lecture Slides part 2
• The titanic set from part 2. I don’t remember where I got it from, originally. There is also a copy of the titanic data in R, but I found this data set and its labeling to be more informative.
• A Best Practices Guide for Data Visualization published by the Royal Statistical Society for self-study.

Practical

This week’s practical is on exploratory data analysis. Code solutions are also given to you. Use these answers to help yourself when you’re stuck.

• Practical 2: EDA
• Practical 2: EDA with code solutions

Hand in via blackboard.

Required reading

Study the following materials

• R4DS Section on Explore (Chapters 2-8)
• FIMD Chapter 1 and Chapter 2.2, 2.3 and 2.6

Useful links

The following links may be useful

• A wonderful post about misusing the y-axis.
• ggplot’s reference page
• A tool to pick color palettes including palettes that are color-blind accessible.
• A color palette tutorial for seaborn, vega-altair, and ggplot2

Must watch

The following lecture by Edward Tufte is quite nice.

Lecture (Monday 9am - Dalton 500 8.27)

This week’s slides:

• Lecture 3

Practical

This week’s practical is on regression. Code solutions are also given to you. Use these answers to help yourself when you’re stuck.

• Practical 3: Regression
• Practical 3: Regression with code solutions

Hand in via blackboard.

Required reading

Study the following materials

-ISLR Chapter 1, 2 and 3

Lecture (Monday 9am - Dalton 500 8.27)

This week’s slides:

• Lecture 4

Practical

This week’s practical is on classification. Code solutions are also given to you. Use these answers to help yourself when you’re stuck.

• Practical 4: Classification
• Practical 4: Classification answers
• data

Hand in via blackboard.

NOT HAND IN, BUT STILL USEFUL: I also give you the following practical from last year. Please view and study the practical up to and including Exercise 10. No need to hand anything in. All the regularization (glmnet) exercises 11-19 are not specifically covered in this course, but you may of course use the methods in your Assignment 2 if you wish. Let us know if you have any questions.

• Practical: Best subset selection and regularization
• the generate_formulas() code

Required reading

Study the following materials

• ISLR Sections 4 and 8.1

Lecture (Monday 9am - Dalton 500 8.27)

This week’s slides:

• Lecture 5

Practical

This week’s practical is on classification. Code solutions are also given to you. Use these answers to help yourself when you’re stuck.

• Practical 5: Classification evaluation
• Practical 5: Classification evaluation answers
• data

Hand in via blackboard.

Required reading

Study the following materials

• ISLR Sections 5.1, 5.2, 8.2.1, 8.2.2, 8.2.3

Lecture (Monday 9am - Dalton 500 8.27)

This week’s slides:

• Lecture 6

Practical

This week’s practical is on nonlinear regression. Code solutions are also given to you. Use these answers to help yourself when you’re stuck.

• Practical 6: Nonlinear regression
• Practical 6: Nonlinear regression answers

Hand in via blackboard.

Required reading

Study the following materials

• ISLR Chapter 7

Lecture (Monday 9am - Dalton 500 8.27)

This week’s slides:

• Lecture 7

Practical

This week’s practical is on nonlinear regression. Code solutions are also given to you. Use these answers to help yourself when you’re stuck.

• Practical 7: Tree-based methods and SVM
• Practical 7: Tree-based methods and SVM answers
• Data

Hand in via blackboard.

Required reading

Study the following materials

• ISLR Chapters 8.1, 8.2 and 9

Lecture (Monday 9am - Dalton 500 8.27)

This week’s slides:

• Lecture Slides

Practical

This week’s practical is about conducting ML benchmarks with mlr3. Code solutions are also given to you. Use these answers to help yourself when you’re stuck.

• Practical 8: ML Benchmark
• Practical 8: ML Benchmark answers
• Data

Hand in via blackboard.

Required reading

Recap: ISLR Chapters 4,5,8 and 9

Useful links

The following links may be useful:

• mlr3-book

especially:

• mlr3-book Chapter 3

• mlr3-book Chapter 4