ECCV · 2020 · Springer Lecture Notes7 min read

Active learning for connectomics, two streams are better than one when the annotator is a radiologist.

Annotating electron-microscopy brain volumes is agonising — dense, tiny structures, experts paid by the hour. Active learning tries to pick the most informative unlabeled samples for the human to label next. We proposed a two-stream scheme that combines what the classifier already knows with what the data itself looks like.
HUGE UNLABELLED EM VOLUME (~100k patches)STREAM A · task uncertaintywhat the classifieris uncertain aboutsupervised embedderSTREAM B · data structurewhat the data lookslike in the rawunsupervised embedderSMALL LABELLED POOL (+k patches)annotated by radiologistStream A says "the model is wrong about this one." Stream B says "and it looks nothing like anything you've labeled."
Fig. 1 — Two independent signals decide what to label next. The combination is better than either alone.

Connectomics — the attempt to trace every neuron and every synapse in a tissue block — is a labor-intensive field. A cubic millimetre of mouse cortex is roughly a billion electron-microscopy voxels, and meaningful segmentation requires expert-level annotation on enough of them to train a model. Nobody has enough annotator-hours. Active learning is one of the few viable escape valves: instead of labelling randomly, let the model tell you which examples to label next.

Why one-stream active learning falls short

Most active-learning methods use uncertainty from the downstream task model — the most uncertain patch is queried first. This is a reasonable heuristic, and it fails in a specific mode: if the unlabelled pool has outliers that look nothing like your labeled set, the task model will be highly uncertain about them for the wrong reason (no prior), and you will burn expensive annotation budget on patches that turn out to be noise or artifacts.

The fix is to complement task uncertainty with a second signal that is computed from unlabelled data alone — something that tells you where the unlabeled example sits in the underlying data distribution. If a patch is highly uncertain and lives in a dense region of the unlabeled data (i.e. it's typical, not exotic), it's worth labelling. If it's highly uncertain and lives in an outlier region, it's probably a waste.

Interactive · allocate your labelling budget
SEG. ACCURACY AFTER K QUERIES01000 labelsaccrandomtask-uncertainty onlytwo-stream (ours)
Early in the budget, two-stream blows past the single-stream baseline. Late budgets converge — once you have enough labels, every method is fine.
Simulated · matches Fig. 5 qualitatively

A detail I like

The unsupervised encoder is trained once, on the full unlabelled volume, using a standard contrastive objective. It is not retrained as more labels arrive. This means the "data-structure" stream is a fixed reference against which the evolving task stream can be compared. We tried retraining it jointly; it gave up a little accuracy and a lot of simplicity. Fixed wins.

Active learning fails in the wild not because the uncertainty signal is wrong but because it is one-dimensional. Two dimensions is nearly always enough; what matters is that the second dimension is independent of the first.
−40%
Labels to reach same accuracy
ECCV 2020
Published
bio-CV
Domain

My role

I was a minor collaborator on this one, at the junior-PhD tail of a Harvard connectomics team. Zudi Lin drove the method; I contributed on the experiment design and ran some of the ablations. The project is the reason I believe active-learning research is more about decomposing the uncertainty signal into complementary parts than about inventing new acquisition functions. Most new acquisition functions are rearrangements of the same one signal.

← all fun papers Next: 20 Interactive 360° →