fields/structural-biology/model families
Dimensionality reductions in structural biology and protein science
How structural biology and protein science squeeze large representations of proteins into smaller ones: fewer numbers per residue, per window or per protein, while keeping what predicts behaviour.
How it is used
Structural biology and protein science ask what shape a protein takes and what that shape lets it do. The raw material is large: a chain of thousands of amino acids, a three-dimensional model of every atom, or a vector of hundreds of numbers attached to each residue by a protein language model (a network trained on many sequences, which turns each residue into numbers that reflect its context). Dimensionality reduction cuts that down. Averaging the numbers across a sliding window or across a whole protein turns a long chain of vectors into one short vector. Reading a predicted structure as a handful of per-residue measurements, such as how buried a residue is or how many contacts it makes, does the same.
The reduction can also be built into the model rather than applied afterwards. Networks are run in narrower configurations, with smaller hidden layers, so each residue is carried in fewer numbers than the original design used. Labels are reshaped too: a numerical property measured for many sequences can be transformed so its spread is easier for a regression head to fit. At the data level, near-duplicate sequences are removed and long chains trimmed at their ends, which shrinks the problem in a different direction. In the examples here, these steps sit between a large pre-trained model or a simulation and a small predictor that has to be trained on a limited set of labelled sites.
Why researchers use it
The reason is usually mismatch of scale. Labelled data in this field is scarce: experimentally confirmed modification sites, or properties computed by simulation, number in the thousands at best, while an embedding from a language model may carry hundreds of numbers for every residue of every protein. A predictor given all of it would have far more inputs than examples, and would fit noise. Reducing the representation to a window average, a whole-protein average or a short list of structural measurements gives the predictor something it can learn from.
It also buys interpretability and reuse. A few named quantities, such as burial or contact count, can be inspected and compared across proteins in a way that a raw embedding cannot. And pooled vectors are cheap to store and feed to simple models, so a large pre-trained network can be run once, frozen, and its output reused for several tasks. Physicochemical parameters averaged over a window serve the same purpose from a different direction: they compress known chemistry into a form a small model can take alongside the learned numbers.
In the record
Limits
Pooling discards position. A window average cannot say which residue in the window mattered, and a whole-protein average flattens a chain whose behaviour may turn on a single site or a single contact. Reduced structural measurements inherit any error in the predicted structure they were read from, and a narrowed network may simply lack the capacity its full version had. Because the choice of window, layer or summary is made by the researcher, a predictor can look accurate while the reduction has quietly removed the signal that would explain why.
Written by AIxSci: the background from general knowledge, the examples from checked records.
Related
Every article
aix-00181
Protein language model embeddings used to predict succinylation sites in proteins
aix-00186
Language model trained on protein sequences predicts shape and behaviour of disordered proteins
aix-00081
Language models trained on protein sequences used to predict sugar-attachment sites
aix-00001