MTNeuro: A Benchmark for Evaluating Representations of Brain Structure Across Multiple Levels of Abstraction![Download PDF](/images/pdf_icon_blue.svg)
Keywords: self-supervised learning, visual reasoning, neuroanatomy, semantic segmentation, multi-scale context aggregation
TL;DR: A multi-task, multi-scale benchmark for evaluating representations of volumetric brain data
Abstract: There are multiple scales of abstraction from which we can describe the same image, depending on whether we are focusing on fine-grained details or a more global attribute of the image. In brain mapping, learning to automatically parse images to build representations of both small-scale features (e.g., the presence of cells or blood vessels) and global properties of an image (e.g., which brain region the image comes from) is a crucial and open challenge. However, most existing datasets and benchmarks for neuroanatomy consider only a single downstream task at a time. To bridge this gap, we introduce a new dataset, annotations, and multiple downstream tasks that provide diverse ways to readout information about brain structure and architecture from the same image. Our multi-task neuroimaging benchmark (MTNeuro) is built on volumetric, micrometer-resolution X-ray microtomography images spanning a large thalamocortical section of mouse brain, encompassing multiple cortical and subcortical regions. We generated a number of different prediction challenges and evaluated several supervised and self-supervised models for brain-region prediction and pixel-level semantic segmentation of microstructures. Our experiments not only highlight the rich heterogeneity of this dataset, but also provide insights into how self-supervised approaches can be used to learn representations that capture multiple attributes of a single image and perform well on a variety of downstream tasks. Datasets, code, and pre-trained baseline models are provided at: https://mtneuro.github.io/.
Supplementary Material: pdf
URL: https://mtneuro.github.io/
Dataset Url: https://mtneuro.github.io/
License: The dataset is released under the CC by 4.0 license (https://creativecommons.org/licenses/by/4.0/)
The code is released under the MIT license (https://opensource.org/licenses/MIT)
Author Statement: Yes
Contribution Process Agreement: Yes
In Person Attendance: Yes
Community Implementations: [![CatalyzeX](/images/catalyzex_icon.svg) 4 code implementations](https://www.catalyzex.com/paper/mtneuro-a-benchmark-for-evaluating/code)
21 Replies
Loading