Experimental design in astronomy and astrophysics

How it is used

Modern sky surveys photograph the same patches of sky again and again, and software compares each new image with earlier ones. Anything that has changed brightness or position produces an alert. Millions arrive each night. Only a handful are worth pointing a second, larger telescope at, and the chance is often gone within hours. Experimental design here means choosing those few: which object to observe next, with which instrument, and when. The examples show machine learning doing that choosing. Models sift alerts, judge whether each is a genuine astronomical change or an artefact of the camera or processing, guess what kind of object it is from a few early measurements, and pass the survivors to a telescope.

Two kinds of model appear. One is the neural network, a layered arrangement of simple numerical units trained on many labelled examples until it maps raw pixel patches and alert properties onto a score. The other is the random forest, a large collection of simple yes-or-no decision trees whose votes are averaged into a probability. Both are trained on past alerts that humans have already sorted, including, in one case, the candidates that human scanners looked at and declined to keep. One example trains a forest on a small set of labelled light curves, records of how brightness changes over time, and lets it ask for the labels it would most benefit from.

Why researchers use it

The binding constraint in this kind of astronomy is not data but attention. A survey telescope is cheap per observation and generous with alerts; a spectrograph, which splits an object's light into its colours and so reveals what it is made of and how fast it is moving, is scarce, expensive and slow. Worse, the most informative moment is the earliest one, when a star has only just begun to explode and its light still carries traces of the material around it. A human cannot read a night's alerts before that moment passes. The models stand in for the human scanner: they apply the same judgement, learned from that scanner's past decisions, at the speed the sky demands.

Because the decision is a choice rather than a final answer, the models are allowed to be imperfect in a controlled way. A score threshold can be moved to trade missed discoveries against wasted telescope time, and in one example it is raised to suppress spurious alerts near bright galaxy centres. The models also sit inside filters that are otherwise made of hand-written rules, so human reasoning still sets the boundaries and learning fills in the judgement that rules express badly.

In the record

astronomyNeural network picks bright exploding stars from sky survey alerts and books telescope timeA deep learning model was trained on labelled survey alerts covering confirmed bright transients, variable and active galactic sources, dim transient-like sources, and candidates human scanners had declined.astronomyTelescope targets chosen nightly by a classifier learning from its own spectraA random forest of a thousand trees scored each alert for being an early Type Ia supernova, using fitted light-curve shape and noise features, starting from thirty labelled light curves.astronomySoftware spotted a nearby supernova and booked a telescope within minutesA learned real-or-spurious model scored incoming alerts inside a rule-based filter, with its pass threshold raised to cut false alerts over bright galactic nuclei.

Limits

These models learn from past human choices, so they inherit the preferences and blind spots of those choices, and they are weakest on the rare or unfamiliar object, which is often the one most worth observing. A score is not a classification: a high probability early in an event can still be wrong, and the telescope time is then spent. Thresholds make the trade-off explicit but cannot remove it. Performance also drifts when the survey, the camera or the processing changes, because the model's training no longer matches the alerts it sees.

Written by AIxSci: the background from general knowledge, the examples from checked records.