How to improve image analyses of cancer cells

Florian Kromp, PhD, and Sabine Taschner-Mandl, PhD, use automated imaging to characterize pediatric tumor cells more precisely.

(Vienna, 15.04.2021) New publication on artificial intelligence: Scientists at St. Anna Children‘s Cancer Research Institute present novel insights on automated image analyses of cancer cells. Based on their results the researchers recommend the creation of artificial images to train deep neural networks. Furthermore, they propose methods to effectively segment complex microscopy images, such as tumor tissue.  

When studying solid pediatric tumors, biomedical image analysis used to characterize  cells in tissue sections or bone marrow aspirates play an important role in diagnostics- as well as research settings. This characterization of cells enable to  understand pathophysiological features of tumors for risk assessment and therapy choice in pediatric cancer. Complex algorithms, such as "Deep learning architectures" allow for the detection of even subtle biological changes, while benefitting from the statistical power of analyzing thousands of cells. A study led by Florian Kromp, PhD, from Sabine Taschner-Mandl’s Tumor Biology Group, for the first time systematically evaluates multiple deep learning architectures on accurate detection and separation of cell nuclei in complex fluorescent images, as frequently found in tumor tissue.

Challenge of complex images

Deep neural networks can be trained to detect cells accurately if expert-annotated datasets are available. Unfortunately, available datasets are limited in size and complexity.

“Inaccurate image segmentation can lead to wrong biological conclusions and is frequently caused by tightly aggregated nuclei (see figure) that cannot be separated by the segmentation algorithm. Separating and labeling each ‘nuclear instance’ (=termed instance-aware segmentation) is the key challenge in nuclear image segmentation”, explains Florian Kromp.

To evaluate state-of-the art segmentation methods on complex images, the scientists use their recently published dataset of expert-annotated images (Kromp et al., BioStudies Database 2020)

Examples of nuclear morphologies in various tissue preparations. (a)
Neuroblastoma bone marrow cytospin presenting varying nuclear intensity and size. (b) Annotated mask of (a). (c) Ganglioneuroma tissue cryosection presenting overlapping/aggregated nuclei with varying morphology and intensity. (d) Annotated mask of (c). © Kromp et al., IEEE Transactions on Medical Imaging 2021,,

Let machines learn better

To increase the training material for deep learning architectures, the scientists propose to create artificial images focusing on overlapping nuclei, thereby simulating complex nuclear images. These artificially generated images should be used in combination with instance-aware segmentation algorithms such as Mask R-CNN.

Based on their results the scientists further recommend “silver-standard” image annotation (see figure) to train deep-learning algorithms. Silver-standard training sets contain partially inaccurate annotation masks. However, they have the advantage that they can be generated by trained under-graduate students, whereas gold-standard training sets have to be carefully generated and curated by biology and pathology experts. “We could prove that in combination with artificial images and simple annotations, deep-learning architectures deliver an excellent detection and separation of each cell in a complex tissue. This is crucial for the exact measurements of biological changes in the tumor”, says Co-Senior study author, Sabine Taschner-Mandl, PhD.

Examples of all types of preparations/specimen including a comparison between annotations from undergraduates (silver-standard) and from experts (gold-standard). Green arrows indicate differences between silver-standard and gold-standard annotations. © Kromp et al., IEEE Transactions on Medical Imaging 2021,,

Wide range of application

Sabine Taschner-Mandl comments: “By using automated imaging, we aim to characterize tumor cells more precisely, which could help us on the long run to stratify patients for more personalized cancer treatment.”

“Our recent study has the potential to improve accuracy and enable broad applications of microscopy based image analysis workflows on complex images of various samples”, summarizes Florian Kromp.

Publication: Florian Kromp et al., "Evaluation of Deep Learning architectures for complex immunofluorescence nuclear image segmentation," in IEEE Transactions on Medical Imaging. doi: 10.1109/TMI.2021.3069558