Semi-supervised

What it is

Most machine learning is taught by example. A model is shown inputs paired with the right answer, called a label, and adjusts itself until its guesses match. Labels usually come from people, or from slow and expensive experiments or simulations, so they are scarce. Raw, unlabelled data is often abundant. Semi-supervised learning uses both. The model first learns from the small labelled set. It then makes predictions on the unlabelled data, keeps the predictions it is most confident about as if they were real labels, and trains again on the enlarged set. This cycle can be repeated.

Other versions of the idea never invent labels at all. Instead the unlabelled data is used to work out which items resemble each other, and that structure is fed to the model as a starting assumption: things that look alike should probably get the same answer. Either way, the unlabelled data shapes the model even though nobody has said what any of it means.

Why scientists use it

Scientists reach for it when answers are costly but measurements are cheap. A structure solved in a laboratory, an outcome computed by a long physics simulation, a sample judged by an expert: each one takes time, so a study may have only a handful. Meanwhile instruments and public databases pour out unlabelled material. Semi-supervised learning is a way of spending the cheap data to make the expensive data go further.

It stands in for a larger labelled collection that does not exist and could not be afforded. It also stands in for prior knowledge. Patterns found in unlabelled data often encode something real about the system being studied, and handing those patterns to the model saves it from having to rediscover them from a few examples.

In the record

astronomyMachine learning predicts whether a dying massive star explodes, from its density profileExplosion outcomes from one hundred two-dimensional simulations supplied the labels, while density profiles of 1512 one-dimensional progenitor models were re-binned and normalised as inputs.materials-chemistryA learning algorithm ran a synchrotron beamline to find a phase-change materialRaw spectra gathered before the run became similarity weights on a composition graph, giving a prior for sorting the diffraction patterns measured later into phase regions.structural-biologyNeural network predicts protein atom positions from amino acid sequence aloneLarge sequence databases were searched to build alignments for each protein, and a structure database searched for templates; both, with the sequence, fed the network.

Limits

The method can teach a model its own mistakes. Labels it invents carry its early biases, and retraining on them makes those biases look confirmed rather than correcting them, so confidence rises while accuracy does not. It also assumes the unlabelled data comes from the same kind of source as the labelled data; if it does not, the extra material pulls the model the wrong way. And a pattern found in unlabelled data may reflect how an instrument behaves rather than anything about nature.

Written by AIxSci: the background from general knowledge, the examples from checked records.