The Role of a Mask - Understanding the Performance of Deep Neural Networks to Detect, Segment, and Extract Cellular Nuclei from Microscopy Images

Cell segmentation is the task of identifying cell nuclei instances in fluorescence microscopy images. It plays a key role in biomedical analysis tasks like cell characterization, cancer cell identification, and gene expression measurement. There has been a recent proliferation of deep neural network-based object detection techniques for cell segmentation. However, previous literature does not suitably address the proper understanding of the different methods and the merits of the various neural architectures proposed. One key design point relates to the learning of segmentation masks and how neural methods go about detecting, segmenting, and extracting nuclei from unseen images. The goal of this paper is to benchmark the performance of representative deep learning techniques for cell nuclei segmentation using standard datasets and common evaluation criteria. We investigate whether for cell nuclei segmentation, learning the nuclei masks in parallel with boundary detection features of the image provides superior performance and significantly cleaner separation of nuclei from images. We further explore the trade-off between increased accuracy, achieved through more complex deep learning models, and the heavy requirements imposed on both computational resources and training times. We believe this paper establishes an important baseline for cell nuclei segmentation, enabling researchers to continually refine and deploy neural models for real-world clinical applications.