Optimisation

What it is

Many scientific questions have the same shape. There is a large set of possible answers, and some way of scoring how good any one answer is. Optimisation is the method of finding a high-scoring answer without checking every option, because checking every option would take far too long. It works as a loop. Pick a candidate, score it, then use what the score revealed to pick the next candidate. Good candidates suggest where to look again; poor ones rule out whole regions. Over many rounds the search drifts towards better answers.

The set of possible answers is often called the search space, and it can be enormous: every shape a protein might fold into, every recipe for making a material, every setting of a model's internal dials. Optimisation methods differ in how they use the score. Some follow the slope, moving a little in whatever direction improves the score fastest. Others keep a running estimate of which untried regions look promising and deliberately test there. Machine learning adds a way to guess the score cheaply, so that expensive real measurements are spent only where the guess says they are worth making.

Why scientists use it

Scientists reach for optimisation when the answer cannot be written down directly but can be recognised once seen. Nobody can calculate the correct fold of a protein from first principles, but a proposed fold can be checked against measurements. Nobody can derive the right furnace recipe, but a batch can be made and examined. In these cases the loop stands in for understanding. It substitutes many cheap judgements for one piece of reasoning that is not available.

It also stands in for human patience. A loop that proposes, measures and revises can run overnight, or across a telescope's nightly flood of data, or over thousands of variations of a model's settings. The same pattern appears in laboratories mixing powders, in astronomy deciding which objects to observe next, and in fitting the structure of a computer model itself. What varies is only the search space and the score; the loop is the same.

In the record

Limits

A loop can only find what its score rewards. If the score is a poor stand-in for what matters, the search will faithfully deliver the wrong thing. Loops also settle into local optima: a spot better than everything nearby, but far from the best overall, with nothing in the score to signal the difference. And each round costs something, so the search space must be narrowed by hand before the loop starts, which quietly builds in whatever assumptions that narrowing carried.

Written by AIxSci: the background from general knowledge, the examples from checked records.

Every article