Mathias Wilms, Assistant Professor of Radiology, Medical School
Watch RecordingMathias Wilms arrived at the University of Michigan with a project that sounded, at first, like a straightforward success story: take MRI scans, train a convolutional neural network, classify Parkinson’s disease. Binary problem. Eighty percent accuracy. Clinicians signed off on the saliency maps. Done.
Except it wasn’t done — and the gap between the clean result and the messy reality is where Wilms’s most important work began.
The clinical motivation was genuine. Parkinson’s disease affects more than 10 million people globally, has no cure, and is typically diagnosed through a neurologist’s observation of motor symptoms. Early diagnosis matters for managing those symptoms, and clinicians in Calgary had approached Wilms with a direct question: could an AI classifier do the job? The team assembled what they could — 72 labeled MRI scans of Parkinson’s patients from a local study setting, supplemented eventually by more than 1,800 scans from 40 centers around the world after a painstaking effort to reach collaborators across Canada and internationally. Even that number, Wilms noted, is small for a supervised deep learning problem.
The model performed reasonably well overall. When Wilms showed the clinicians saliency maps — visualizations of which brain regions the model weighted most heavily — they were satisfied. The highlighted regions matched known Parkinson’s pathology. “We trust this system,” they said. But then the team held back one site entirely from training and tested the model on it, and saw a sudden drop in accuracy. Something was being learned that wasn’t Parkinson’s disease.
To find out what, they ran a diagnostic that was elegant in its simplicity: freeze the model’s feature extraction layers and try to predict variables that had nothing to do with Parkinson’s — scanner manufacturer, imaging site, field strength. If those features could be predicted from the same representations the model used for disease classification, the model had learned the scanner, not the disease. The results were stark. Accuracy for predicting scanner type was on par with — sometimes higher than — accuracy for predicting Parkinson’s itself. Saliency maps computed separately for each scanner configuration showed that each one induced subtly different characteristics in the images, invisible to the human eye, but legible to the network. The multi-site data that had been essential to training the model in the first place was also the source of its most fundamental flaw.
This is what Wilms called the central paradox of multi-site neuroimaging research: “multi-site data is a curse and a blessing.” Detecting the bias required not just a computer scientist but an MRI physicist who could explain how different scanner configurations induce systematic artifacts, and clinicians who could confirm what should — and shouldn’t — be driving the model’s attention. The team that thought it had a solved classification problem discovered it actually had a data harmonization problem, a shortcut learning problem, and a model interpretability problem, all at once.
From that discovery, two new technical projects emerged. The first was a causal harmonization approach using generative AI: a model that maps an incoming brain MRI into a latent space, intervenes on the site variable to remove scanner effects, and reconstructs the image with biological signal intact. The second was SIMBA — a synthetic data framework that generates brain MRI images with known, controllable shortcut features, allowing systematic evaluation of whether debiasing methods actually fix the problem or merely obscure it. Both grew directly from what the baseline analysis revealed: the engineering was downstream of the diagnosis, not the other way around.
Wilms closed with a lesson that applies well beyond neuroimaging: “What initially seemed like a super simple problem — just images in, classification result out — got super complicated.” The simplicity was a surface feature of the problem statement. The complexity was always there in the data.