“I had a dream my paintbrush split while I was picking up brain slices. Nightmare scenario, right? Then I woke up and thought: I really miss the lab,” a neuroscientist friend of mine recently lamented over a virtual happy hour.
She’s not alone. As the pandemic spread across the US, neuroscience labs shuttered their doors, leaving behind half-finished experiments, aging lab animals, and precious brain specimens that—if unprocessed—could mean losing months of hard work. Like many other fields, the pandemic has brought hands-on, “wet lab” brain research to a screeching halt. Solving the brain’s mysteries will just have to wait.
Or will it?
In mid-March, a trifecta of academic, non-profit, and commercial neurotech companies united to launch a neuroscientific challenge that embodied a more collaborative future for solving the brain, while physically apart and working from home. (Full disclosure: I was fortunate to be involved in the very initial organization of the project.) The challenge embraced a one-two formula for accelerating scientific progress, most recently encapsulated by research into Covid-19. One, large open databases enabling AI-based tools for analysis; and two, global interest and unity.
These sentiments aren’t exactly new to neuroscience. Thanks to big data and machine learning, large-scale collaborations for mapping the brain have already been made possible through data-sharing, such as the Human Brain Project or the BRAIN Initiative. However, applying these collaborative ideas to clinical datasets has been difficult, in part due to the complexity and messiness of the data, as well as privacy concerns.
The NeurekaTM challenge is much smaller in scale, but it embodies the same principles. The project employed an open database of electroencephalogram (EEG) brain recordings from people with epilepsy, curated by Dr. Joseph Picone and colleagues at the Temple University Hospital (TUH), and challenged a global community of neurotech enthusiasts—while away from their labs and staying at home—to deploy machine learning and other AI tools to better decipher those neural data and predict seizures before they occur.
In just six weeks, the challenge received over a dozen submissions from Australia, Belgium, China, Israel, India, Russia, and other countries, with several innovative solutions to capture seizure warnings using the brain’s electrical signals alone.
“These challenges can only be successful when there’re sufficient data resources to support the development of the technology and a common scoring methodology that allows direct comparison of results,” said Dr. Picone.
To Yannick Roy, co-founder and Executive Director of NeuroTechX, a non-profit organization that supports education and development of a global community of neurotechnology professionals and enthusiasts, the challenge highlights the promise of AI in deciphering neural signals—as long as the dataset is widely available.
“Large clinical datasets will be key to unlocking AI in the coming years in fields like epilepsy. Dr. Picone has helped greatly, not just in terms of the dataset but also the tools, documentation, and support to understand and use the data,” said Roy, who led the organization of the challenge.
The results aren’t just a scientific community’s self-pat on the back. Results from the challenge, when further explored, could potentially revolutionize epilepsy monitoring for patients at home. This is particularly critical as patients with chronic disorders are hesitant to visit the clinic in fear of Covid-19.
“Telehealth is especially important today when we’re staying home to help flatten the Covid curve,” said Ray Iskander, CEO of Novela Neurotech, which helped conceptualize and sponsor the challenge. “But more importantly, the Neureka competition shows us what can be done with an open dataset, even during lockdown, to move neuroscience research forward. Our goal is to support similar data-driven approaches to personalize neurological solutions in telehealth.”
The Science Paradigm Shift
I’ve talked about this a bunch: neuroscience has had a paradigm shift towards a highly collaborative model for the past decade. The rapid rise of deep learning and other machine learning tools have made it especially intriguing to build brain recording databases and mine for hypotheses and insights.
The Temple University Hospital’s Seizure Detection Corpus (TUSZ), led by Picone, was a recent ambitious attempt. Part of the Neural Engineering Data Consortium, founded in 2012 to standardize methodologies for bioengineering applications, the Seizure Corpus is a subset of the “world’s largest open-source collection of EEG data, with over 50,000 clinical recordings known as the Temple University Hospital EE Corpus,” said Picone.
The seizure corpus encapsulates EEG recordings from people with epilepsy, which is the gold standard for monitoring seizures. Although these datasets are valuable, the raw data are also massive in size, difficult and time-consuming to manually decipher. The seizure corpus helped “label” much of the data—somewhat similar to labeling data in supervised machine learning—paving the road for machines to better “read” human electrical brain recordings. This, in turn, allows scientists to explore ways to automatically detect seizures.
That’s huge. Being able to predict a seizure episode, especially several minutes before they happen, could provide patients with an early alert system that helps reduce injuries and even save lives. The six-week challenge doesn’t take us there. But it’s a powerful proof-of-concept of what’s possible during lockdown for accelerating neuroscience research.
The NeurekaTM Challenge
Picone offered his support to the challenge, building several sub-databases from the Seizure Corpus for various teams to develop their software algorithms on, and to test for accuracy and efficiency. Altogether, the dataset provided nearly 1,100 hours of EEG recordings from almost 700 patients, which encapsulated more than 3,500 seizure events.
The main goal was to use machine learning or other AI methods to detect seizures as early as possible, while utilizing the bare minimum of EEG channels. Normally, the international standard headset has 21 channels (including reference and ground) embedded in a swimcap-like contraption. It’s data-intensive, bulky, and uncomfortable for everyday use.
“By lowering the number of channels needed to detect seizures, we could potentially build a more comfortable wearable to help people with epilepsy monitor their seizures at home,” said Iskander.
To guide participants along, Roy at NeuroTechX and Picone at Temple University hosted an online demo session on data structure, organization, and analysis, while the challenge maintained a lively Slack channel to support immediate questions.
“The competition showed us that large clinical datasets are not ‘textbook problems’ that you see in school when you first get started. The data’s messy, comes from different hardware equipment, and is used differently by multiple practitioners,” said Roy. “This variability is challenging to train high-performing models on. But it’s also necessary for the AI model to generalize well from the training dataset.”
Altogether, in just 6 weeks’ time including recruitment, data model training, and evaluation, the challenge received 15 submissions, with several solutions outperforming current models of seizure detection.
“The highest performing systems this year were based on deep learning. We found two important findings: one, that preprocessing the data can result in significant improvements in performance, and two, that intelligently reducing the number of EEG channels, if done properly, can provide great performance with significantly reduced complexity,” said Picone.
In the spirit of open science, the winner’s code will be released online for others to continue to build upon. The results of the competition are set to be featured at the IEEE Signal Processing in Medicine and Biology Symposium at Temple University in Philadelphia, PA this December.
The challenge dataset remains open today for others to analyze. To the tri-organizational team, however, the challenge was an unequivocal win.
“After this successful challenge, we’re already looking forward to the next one!” said Roy.
“While we’re still a ways from clinical acceptance, the results from the competition demonstrate that performance has significantly improved in the past year. It’s only a matter of time before clinically-acceptable performance is achieved,” said Picone.
To Iskander, the challenge is a proof-of-concept of the power of adopting an open-data, collaborative approach for solving difficult neurological disorders—even if, or especially during, the pandemic where scientists are cut off from their labs.
“We’re blown away by the creative approaches explored in this competition. The challenge isn’t the end-all. With papers in the pipeline and the results in the open, we’re hoping that the challenge will help expedite remote monitoring of people with epilepsy and increase their well-being both during and after the pandemic,” he said.
Image Credit: Gerd Altmann from Pixabay