Week 12
3D structure from sequence: AlphaFold 2 and ESMFold
Predicting 3D structure end to end from sequence. A protein is text (sequence), a graph (residue pairs), and a point cloud (atoms in 3D), and modern predictors reason over all three. The tools are graph neural networks and networks equivariant to rotation and translation. AlphaFold 2 combines MSA and pair representations in the Evoformer with an equivariant structure module; ESMFold replaces the MSA with a week-11 language model. The same graph tools apply to protein function and protein–protein interaction networks.
Objectives
- Represent a protein as a sequence, a residue graph (nodes and edges with pair features), and a 3D point cloud, and say which tasks and architectures fit each.
- Write one message-passing layer (compute messages from neighbors, aggregate with a permutation-invariant function, update node states) and explain why the output is permutation equivariant.
- Define invariance and equivariance under 3D rotations and translations, check whether a given layer has them, and explain why models that output coordinates need equivariance.
- Trace AlphaFold 2's MSA and pair representations through an Evoformer block: row attention biased by the pair representation, column attention, the outer-product mean, and triangle multiplicative updates and triangle attention motivated by the triangle inequality.
- Explain how AlphaFold 2's structure module treats each residue as a rigid frame, updates the frames with invariant point attention, predicts side-chain torsions, and trains with the frame-aligned point error (FAPE) loss.
- Describe recycling, the masked-MSA auxiliary loss, and self-distillation in AlphaFold 2 training, and interpret pLDDT and predicted aligned error as confidence measures.
- Explain how ESMFold feeds ESM-2 embeddings into a folding trunk and structure module without an MSA, and weigh its speed (up to 60× faster) against its accuracy relative to AlphaFold 2.
- Relate language-model scale and perplexity to structure prediction accuracy, using ESM-2's scaling from 8M to 15B parameters.
- Frame protein function prediction or protein–protein interaction prediction as node classification or link prediction on a graph, and set up a GNN for it.
Papers
J Jumper et al. “Highly accurate protein structure prediction with AlphaFold”. Nature 596, 2021. PDF
AlphaFold 2: can structure be predicted at atomic accuracy even without a similar known structure? A network built around 48 Evoformer blocks (attention over MSA and pair representations with triangle updates) and a structure module using invariant point attention on residue frames, trained end to end with FAPE, recycling, and self-distillation. At CASP14 its median backbone accuracy was 0.96 Å r.m.s.d.95 versus 2.8 Å for the next best method, with a well-calibrated pLDDT. It is the week's main case study of graph-like, equivariant, multimodal deep learning.
Z Lin et al. “Evolutionary-scale prediction of atomic-level protein structure with a language model”. Science 379(6637), 2023. PDF
Can a language model alone carry enough evolutionary information to predict structure from a single sequence? The authors train ESM-2 from 8M to 15B parameters; contact and structure accuracy rise as perplexity falls. ESMFold puts a folding trunk and an AlphaFold-style structure module on ESM-2 and needs no MSA; it is up to 60× faster, near RoseTTAFold on CAMEO, though below AlphaFold 2 with MSAs. The speed enabled the ESM Metagenomic Atlas of over 617 million predicted structures. It ties week 11's pLMs to week 12's structure prediction.