learn/model families
Gradient-boosted trees
Gradient-boosted trees are a prediction method that adds up many simple decision trees, each new tree fitted to correct the errors left over by the ones before it.
What it is
A decision tree is a chain of yes-or-no questions about a row of numbers. Is this quantity above some threshold? Then go left, and ask again. At the end of the chain the tree gives a number or a label. One small tree is a crude predictor. Boosting builds many of them in sequence. The first tree makes a rough guess for every row in the training data. The leftover error, the gap between the guess and the true answer, becomes the target for the second tree. The third tree attacks what is still wrong, and so on. The final prediction is the sum of all the trees.
The trees are deliberately kept shallow, and each one's contribution is shrunk by a small factor, so the method creeps towards the answer rather than leaping at it. The gradient in the name refers to how the leftover error is measured: the method follows the direction in which the chosen error measure falls fastest. The main settings a researcher chooses are how many trees to add, how deep each may grow and how much of each to keep. Several widely used software packages implement this, including the one named XGBoost.
Why scientists use it
Most scientific data arrives as a table: one row per sample, one column per measured or computed quantity. Gradient-boosted trees are good at tables. They do not care that one column is a temperature and the next a dimensionless ratio, since each tree only compares values with thresholds. They pick up effects that depend on combinations of columns without being told which combinations to look for. They work with modest amounts of data, train quickly on an ordinary computer, and often match or beat more elaborate methods on this kind of input.
What they stand in for is a formula nobody has written down. A scientist may suspect that an outcome depends on a set of properties, without knowing the relationship. Fitting trees to examples supplies a usable stand-in: feed in the properties, get an estimate. That estimate can then guide the next experiment or calculation, in chemistry, in biology and in astronomy alike.
In the record
Limits
Trees only compare values with thresholds learned from the training set, so outside the range of that set the prediction simply flattens out rather than continuing sensibly. Predictions also change in steps, which is wrong if the real relationship is smooth. If the training table is skewed, say towards materials that happened to be published, the model inherits that skew. Add too many trees and it memorises noise. It offers no mechanism, and the rankings of which columns mattered can mislead when columns are correlated. Raw spectra, images and waveforms usually need other methods.
Written by AIxSci: the background from general knowledge, the examples from checked records.
By field
- Gradient-boosted trees in structural biology and protein science
- Gradient-boosted trees in materials and computational chemistry
- Gradient-boosted trees in astronomy and astrophysics
- Gradient-boosted trees in weather, climate and oceans
Named models
Every article
aix-00286
AI fills gaps in river temperature records to track overlapping heatwaves
aix-00288
Machine learning turns satellite cloud images into hourly rainfall maps for central Vietnam
aix-00289
Machine learning points to filter loading as main driver of levoglucosan desorption temperature
aix-00281
Bacteria-derived aerosol matter made to produce oxidants, then modelled from its optical fingerprints
aix-00293
Machine learning used to predict fine particle emissions at a Greek regional airport
aix-00279
Machine learning predicts where dopant atoms sit in hexaferrite crystals before refinement
aix-00273
Deep learning model flags bacterial proteins likely to drive infection
aix-00271
Machine learning predicts galaxy 'relicness' from survey measurements, cutting spectroscopy demand
aix-00238
Classifiers label individual TESS brightness readings as planet transits or not
aix-00210
Language models suggest ingredients and firing temperatures for inorganic materials recipes
aix-00206
Neural network designs five hard metallic glasses, all confirmed by experiment
aix-00163
Peptides designed by simulation and machine learning sit at condensate surfaces
aix-00170
Machine learning predicts strength of PLA plastic filled with boron nitride flakes
aix-00133
Regression models predict how graphene loading changes aluminium's electron emission
aix-00181
Protein language model embeddings used to predict succinylation sites in proteins
aix-00184
Machine learning scores short protein stretches that mark proteins for destruction
aix-00188
Machine learning sorts antibody heavy chains by the germ they target
aix-00124
Software predictors compare protein floppiness across retinal disease protein sets
aix-00121
Machine learning sorts mitochondrial proteins into three compartments from sequence alone
aix-00117
Deep learning model predicts polymer heat-softening across 48,208 designed candidates
aix-00107
Machine learning sorts plant protein sequences into disease-resistance proteins or not
aix-00090
Machine learning reads planet-forming disk masses from archival ALMA observations
aix-00081
Language models trained on protein sequences used to predict sugar-attachment sites
aix-00071
Simulations map how surface atoms rearrange in alloys when molecules attach
aix-00068
Machine learning predicts marine steel corrosion from seawater conditions and alloy make-up
aix-00066
Machine learning sifts 25,000 known materials for quantum-technology host candidates
aix-00051
Ultrasound-treated clay bleaches cooking oil, with machine learning picking the settings
aix-00050
Machine learning tool flags protein helices likely to bind DNA or RNA
aix-00049
Eight machine learning methods sorted supernova gravitational wave signals by nuclear physics model
aix-00040
Statistical model of 5,441 perovskite solar cells predicts efficiency and suggests recipes
aix-00035
GPT-4 reads band gap values from paper sentences to train better predictors
aix-00025
Machine learning picks a new metal mixture, which casting and X-rays confirm
aix-00010
Robotic lab with machine learning proposes and runs 353 inorganic synthesis experiments
aix-00002