Self-supervised

What it is

Most machine learning needs labelled examples: a picture with the word "cat" attached, a protein sequence with its known shape. Labels are slow and costly to make, and in science they often do not exist. Self-supervised learning avoids them. It invents a task whose answer is already inside the raw data. A common version hides, or masks, part of each example and asks the model to fill the gap: a missing word in a sentence, a missing patch of an image, a missing stretch of a genetic sequence. The answer is simply the part that was hidden, so the data supplies its own correction with nobody writing anything down.

The invented task is called a pretext task, because it is not what anyone actually wants to know. It is a pretext for making the model learn structure. Another family of pretext tasks shows the model two altered versions of the same example, perhaps cropped or shifted, and asks it to recognise that they belong together while telling them apart from other examples. Either way, the model ends up turning each input into a compact list of numbers, a representation, that captures what matters. That representation is then reused for the real question, usually by training a small extra piece on whatever few labels do exist.

Why scientists use it

Scientists use it because unlabelled measurements are abundant and labels are scarce. Telescopes, microscopes, sequencers and simulations all produce far more raw output than anyone can annotate by hand, and annotating often needs the very expert judgement that is in short supply. Self-supervised learning puts that unused bulk to work. It stands in for a large labelled dataset: the model first learns the general shape of the data from everything available, then needs only a handful of labelled cases to be pointed at a specific question.

In the record

Limits

The pretext task is a guess about what matters, and a model can get very good at filling gaps while learning little that helps the real question. What it learns reflects whatever the raw data happens to contain, including its biases and its instrumental quirks, and nothing in the method reveals them. Because no labels are involved, there is no direct measure of success during this stage, so quality only becomes visible later. Some labelled data is almost always still needed, both to aim the model and to check it honestly.

Written by AIxSci: the background from general knowledge, the examples from checked records.

Every article