learn/basics
Training, testing and using a model
A model is not built by hand but fitted to examples, then checked on examples it has never met, before being set loose on cases nobody knows the answer to.
A model is a rule that turns an input into an answer: a picture into a diagnosis, a protein sequence into a shape, a set of weather readings into tomorrow's rainfall. Nobody writes the rule out. Instead it starts off useless, with its internal settings at arbitrary values, and is shown examples where the right answer is already known. Each time it answers, the error is measured, and the settings are nudged so the error shrinks. Repeated over many examples, this is called training, and the examples are the training data. What comes out is a rule that fits those examples well.
Fitting the training examples well is not the point, though, so the examples are split up before training begins. One part, the training set, is used to nudge the settings. Another part, held back and never shown, is the test set. The model's score on that held-back part is the honest estimate of how it will do on cases it has not seen. Once a scientist is satisfied, the model is run on real unknowns, which is called inference, or simply using the model. At that stage no right answers exist to compare against.
Why it matters for science
Scientists use this arrangement because a model can succeed for the wrong reason. A rule with many adjustable settings can memorise its training examples, including their quirks and noise, and still have learnt nothing general. This is called overfitting. The only way to tell memorising apart from learning is to ask about something the model has never seen. So the held-back test acts as a stand-in for the future: for the next patient, the next galaxy, the next compound.
This matters for reading research. When a paper reports how well a model performs, the number means something quite different depending on what the model was tested on. A score on data the model trained on says almost nothing. A score on held-back data from the same source says the model works in those conditions. A score on data gathered elsewhere, by other people, says much more. Papers that describe this split carefully are making a stronger claim than ones that do not.
Limits
A good test score describes past data, not any particular future case. If the held-back examples came from the same laboratory, instrument or population as the training ones, the test will miss failures that appear elsewhere. Leakage is a common trap: if the same specimen, patient or near-duplicate record sits on both sides of the split, the test quietly becomes a memory exercise and flatters the model. And once a test set has been consulted many times while tweaking a model, it has in effect become training data too.