LightGBM · AIxSci

What it is

A decision tree is a chain of yes-or-no questions about the data that ends in a guess. One tree alone is crude. Gradient boosting builds trees in sequence: the first makes a rough guess, the next is trained on what the first got wrong, and so on, with the final answer being the sum of all of them. LightGBM is one library, or ready-made bundle of code, that does this. Its particular design choices are about speed. It groups continuous values into buckets rather than testing every possible split point, and it grows each tree by extending whichever branch promises the biggest improvement, instead of filling out every level evenly.

The input is a table: one row per thing being studied, one column per measured quantity. In research, that table is often made first. Sequences, images or chemical formulas are turned into lists of numbers, and those numbers become the columns. LightGBM then learns the relationship between the columns and the answer. It has settings, such as how many trees to build and how deep they may go, which are usually chosen by trying many combinations and keeping whichever does best on data the model has not seen.

Why scientists use it

Much scientific data arrives as a table of mixed, hand-crafted measurements rather than as pictures or text. For that shape of data, boosted trees tend to work well without much coaxing. They cope with columns on wildly different scales, with columns that matter only in combination, and with relationships that bend rather than run straight. They also report which columns influenced the predictions most, which researchers use both as a rough guide to what matters and as a way to throw out columns that do not.

It is also commonly used as one candidate among several. A study will fit a handful of different methods to the same table, compare them fairly, and keep the best. Or it will combine them, letting each make a prediction and training a further simple model to weigh those predictions up. LightGBM often appears in such line-ups because it trains quickly enough that the whole comparison stays affordable.

In the record

structural-biologyMachine learning sorts plant protein sequences into disease-resistance proteins or notPredicted whether a plant protein sequence is a resistance protein, from numeric features derived from a pairwise energy matrix treated as a two-dimensional signal.materials-chemistryMachine learning predicts marine steel corrosion from seawater conditions and alloy make-upFitted alongside four other regressors to predict marine corrosion, with settings searched by a genetic algorithm and the best tuned model carried forward.materials-chemistryRegression models predict how graphene loading changes aluminium's electron emissionPredicted field emission behaviour of graphene-aluminium composites from a merged table of electric field, current density, current and time.structural-biologyMachine learning sorts antibody heavy chains by the germ they targetClassified antibody specificity from an 81-dimensional vector of sequence descriptors built from heavy chain amino acid sequences.

Limits

It learns patterns in the table it was given, not causes, so a column that merely accompanies the answer will be used as happily as one that produces it. Its importance scores say which columns the trees leaned on, which is not the same as which quantities matter in nature. With few rows it can fit noise and look better than it is. It cannot predict outside the range of values it has seen, and because the answer is a sum of many trees, no single readable rule explains any one prediction.

Written by AIxSci: the background from general knowledge, the examples from checked records.