Week 7
Machine learning in practice
Practical machine learning for the predictors introduced in week 6. The week covers how to measure a classifier honestly, how to combine heterogeneous evidence such as sequence, structure, expression, and interaction data, and how regression, the bias–variance tradeoff, and regularization explain why models overfit or underfit. It ends with diagnosing a model that underperforms and choosing what to change. There's no assigned reading; Goodfellow ch. 5 from week 6 covers the theory.
Objectives
- Compute accuracy, precision, recall (sensitivity), specificity, F1, and Matthews correlation from a confusion matrix, and explain why accuracy misleads when classes are imbalanced.
- Plot ROC and precision–recall curves by sweeping a decision threshold, compute and interpret the area under each, and choose between them for imbalanced data.
- Set up k-fold cross-validation with a separate held-out test set, and prevent leakage such as homologous sequences landing on both sides of a split.
- Integrate heterogeneous data sources (sequence, structure, expression, interactions) by concatenating features, combining separate classifiers, or combining kernels, and weigh the trade-offs of each.
- Fit linear regression by least squares, both with the normal equations and with gradient descent, and extend it with polynomial features.
- Decompose expected squared error into bias², variance, and irreducible noise, and relate each term to model complexity.
- Apply L2 (ridge) and L1 (lasso) regularization, explain how each shrinks weights and why L1 gives sparse solutions, and tune the penalty strength on validation data.
- Diagnose high bias versus high variance from training and validation error and from learning curves, then choose a fix: more data, fewer or more features, stronger or weaker regularization, or a different model.