Inference

What it is

A model is a piece of software that has been tuned, or trained, on examples until it can map an input to an answer. Training is the slow, expensive part, and it happens once. Inference is what comes afterwards: you hand the finished model something it has never seen, and it returns an answer. In practice that answer is a prediction, a category or a number expressing confidence. The model's internal settings do not change during inference. The same model can be run over one item or over millions, and each run is independent of the others, which is why inference can be pushed through fast hardware in bulk or placed inside an instrument that is taking measurements as they happen.

Inference is worth separating from generation. A generative model invents something that did not exist before, such as a sentence, an image or a candidate molecule. Inference judges something that already exists. A detector takes a photograph and says what is in it; a classifier takes a measurement and says which of several known classes it belongs to; a scorer takes an object and returns a number saying how strongly a property applies. The output is a verdict on the input, not a new object. Many research pipelines use both, with generation proposing and inference sorting.

Why scientists use it

Science produces more data than people can look at. Telescopes, sequencing machines, microscopes and diffraction detectors all emit records far faster than any team can inspect them. Inference stands in for the trained eye of an expert applied one record at a time. Where a specialist would once have judged each spectrum, each image or each variant by hand, a model trained on past expert judgements now does the same job at the speed the instrument runs. That changes what is possible: whole catalogues can be scored rather than sampled, and the interesting cases can be surfaced for a human to examine.

Speed matters in another way too. Some decisions must be made while the experiment is still running. If a sample is changing, or a passing event in the sky will fade within days, a judgement that arrives next month is useless. Because inference is cheap once training is done, it can sit in the loop: score the latest measurement, decide what to measure next, and keep going. This turns a model from an analysis tool applied afterwards into part of the apparatus itself.

In the record

Limits

A model at inference time can only apply what it absorbed during training. Shown data unlike its training examples, it still returns an answer, often a confident one, with no signal that it is out of its depth. Scores look like probabilities but need not be well calibrated, so a high number is not proof. Inference also inherits every bias in the examples it learned from, and repeats that bias uniformly across millions of items. It tells you what resembles what was seen before, not what is true.

Written by AIxSci: the background from general knowledge, the examples from checked records.

Every article