~/aixsci
weekly updates

learn/model families

k-nearest neighbours

A way of predicting something by finding the examples in your records that most resemble the case in hand, and answering from them, with nothing learned or fitted in between.

What it is

Each example is described by a set of measurements, so it can be thought of as a point in a space with one direction for each measurement. To predict something about a new case, the method measures the distance from that case to every stored example, picks the handful that lie closest, and reads the answer off them. If the task is to put the case in a category, the closest neighbours vote. If the task is to estimate a quantity, their values are averaged. The number of neighbours consulted is chosen by the researcher, and is the k in the name.

Nothing is built during training: the records simply sit there until a question arrives. Because distance does the work, measurements on large numerical scales would otherwise dominate, so values are usually rescaled first to comparable ranges.

Why scientists use it

Scientists use it because it makes almost no assumption about the shape of the pattern it is tracking. A method that fits an equation to the data commits in advance to a form that pattern must take. Looking up neighbours commits to nothing beyond the idea that similar cases behave similarly, so it can follow patterns that bend and fold in ways no simple formula captures.

It also serves as a yardstick. Because it is easy to describe and has little to tune, researchers often run it alongside more elaborate methods to see whether the extra machinery earns its keep. The same lookup idea is used to repair records too: a missing measurement can be filled in from the values of the most similar entries.

In the record

Limits

Every prediction needs the whole archive to hand, and a comparison against all of it, so answering grows slow as records pile up. It offers no account of why a case was placed where it was, beyond pointing at its neighbours. It can only repeat patterns already recorded, so a case unlike anything stored will still be assigned its nearest neighbours, however distant they are. When measurements are many, almost everything is far from everything else, and closeness stops meaning much.

Written by AIxSci: the background from general knowledge, the examples from checked records.

By field

Every article

8articles
4structural-biology
2materials-chemistry
2astronomy
0climate-weather