Reinforcement

What it is

Most machine learning is taught with worked examples: here is an input, here is the right answer, copy the pattern. Reinforcement learning has no answer key. Instead there is an environment the program can act on, and a reward, which is just a number saying how well things are going. The program, usually called an agent, picks an action, sees what the environment does, and collects its reward. Over many rounds it adjusts itself to favour actions that tend to pay off. Because the reward may arrive long after the action that earned it, the hard part is working out which earlier choices deserve the credit.

The environment can be anything that responds to a choice and can be scored: a simulation, a piece of laboratory equipment, a set of equations. The agent must also balance two urges. It can exploit what has worked so far, or explore something untried that might work better. Too much of the first and it settles for a mediocre habit; too much of the second and it never settles at all. Related methods share the same trial-and-feedback shape without being reinforcement learning proper. An evolutionary or genetic search, for instance, keeps a population of candidate answers, scores them, and breeds the better ones, rather than training a single agent.

Why scientists use it

Scientists reach for reinforcement learning when they know how to judge an outcome but not how to produce it. Tuning a model so its output matches a measurement, choosing the next experiment, or steering a control system all have that shape: the goal is easy to score and tedious to solve by hand. The reward therefore stands in for the expert who would otherwise sit there adjusting dials and deciding whether the result looks right.

It also suits problems where the choices are too numerous to try one by one. Each decision changes what the next decision faces, so the number of possible routes grows beyond counting. A method that learns from its own attempts can find a workable route without anyone enumerating them, which is why the same idea turns up in chemistry, engineering and astronomy alike.

In the record

Limits

Reinforcement learning is only as sensible as its reward. If the score rewards the wrong thing, the agent will pursue that wrong thing diligently, sometimes by exploiting a flaw in the simulation rather than solving the problem. It needs very many attempts, so it is usually confined to simulations rather than slow or costly real experiments; and skill learned in a simulation may not survive contact with the real system. What it learns is a habit, not an explanation, so it rarely says why its choices work.

Written by AIxSci: the background from general knowledge, the examples from checked records.

Every article