Deep Learning in Fluorescence Microscopy

Fluorescence microscopy is a technique that uses fluorescent light to examine complex tissues at a level of detail that could not be achieved with conventional microscopes.

Fluorescence Microscopy

Image Credit: Vshivkova/Shutterstock.com

Using fluorescence microscopy, researchers can acquire high-sensitivity, high-resolution images of cellular structures and molecular species, for example, in samples that have been labeled with fluorophores.

On absorbing high-intensity light, the samples then emit a fluorescent light and scientists can magnify images of the resulting fluorescing material.

Deep learning is a branch of artificial intelligence that uses algorithms to model large data sets in a way that enables automated learning that resembles human brain function.

Deep learning is increasingly capturing the interest of researchers in the field of fluorescence microscopy for its current and potential applications in super-resolution and 3D image construction.

Turning low-resolution images into super-resolved ones

A recent study by Ozcan et. al. (2019) described how a deep learning-based approach provided super-resolution across a range of fluorescent microscopy modalities.

The method, which does not require numerical models of the imaging, is based entirely on a machine learning system called the generative adversarial network (GAN). The system is made up of algorithms that use two neural networks (one degenerative, one discriminative) that statistically learn and compete to provide higher-level predictions.

The team used GAN to change low-resolution input images from optical systems that had been diffraction-limited into super-resolution images.

The team demonstrated that the wide-field images acquired when objectives with low-numerical aperture are used could be super-resolved to match the resolution achieved when objectives with a high numerical aperture are used.

The team also showed that this super-resolution could be achieved across other modalities. They also showed that GAN can be used to change confocal microscopy images into the super-resolved fluorescent images that can be obtained using a stimulated emission depletion (STED) microscope.

Furthermore, the team demonstrated that images of cellular structures that can be obtained with total internal reflection fluorescence (TIRF) microscopy could also be super-resolved by GAN to match the level obtained with a TIRF-based structured illumination microscope.

In addition, the researchers say the GAN rapidly provides these super-resolved images, without requiring any search for iterations of parameters and could even work when it was applied to sample types that it had not been trained to super-resolve.

Refocusing 2D images into 3D ones

Another study by Ozcan et. al. (2019) showed that another deep learning neural network called Deep-Z learned to refocus 2D fluorescence microscopy images into stacks of 3D image slices of sample surfaces that enabled activity to be observed within the sample.

The team reports that the 3D images obtained with Deep-Z enabled them to observe neuronal activity within the Caenorhabditis elegans worm.

The 3D images were obtained using a sequence of fluorescent microscope images that were captured at a single focal plane, enhancing the field depth by 20 times, without requiring axial scanning, any extra hardware or having to compromise resolution or acquisition speed.

The method could also be used to connect other imaging modalities. The method 3D-focused a wide-field fluorescence microscopy image to achieve the images that can be obtained at various sample planes with confocal microscopy.

Another research group involving Tahir et. al (2019) applied an end-to-end Protein Localization Convolutional Neural Network (PLCNN) to accurately and reliably image the distribution of proteins in cells.

The researchers say that although many advances have been made in the field of fluorescent microscopy over recent decades, the resulting large number of images has presented challenges in terms of processing and analyzing.

Furthermore, the subjective examination of the images may not accurately distinguish the proteins, which may look different despite being from the same class or look the same despite belonging to different classes.

Although such challenges can be overcome using conventional machine learning, the preprocessing and extraction approaches involved before classification can be unreliable. This has meant researchers have increasingly relied on automated computational techniques over the last 20 years, which has improved the accuracy of protein subcellular localization based on fluorescent microscopy.

Tahir et. al. (2019) have applied the PLCNN deep learning approach to five publicly available subcellular localization datasets, namely the HeLa dataset, the CHO dataset, two LOCATE datasets and the Yeast dataset.

The researchers say that PLCNN consistently outperformed current state-of-the-art machine learning and deep learning algorithms across a wide range of protein localization datasets.

Analysis of the network showed that it captures protein features of interest, without capturing irrelevant details. The team adds that the generalizability of the approach is demonstrated by the consistent performance across all datasets used.

Sources:

  • Belthangady C and Royer L. Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction. Nature Methods 2019;16, p215–1225 https://doi.org/10.1038/s41592-019-0458-z
  • Ozcan A, et al. Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nature Methods 2019;16, 103–110 (2019). https://doi.org/10.1038/s41592-018-0239-0
  • Ozcan A, et al. Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning. Nature Methods 2019;16, p 323–1331 https://doi.org/10.1038/s41592-019-0622-5
  • Tahir M et al. Deep localization of protein structures in fluorescence microscopy images. Deep AI 2019. Available at: deepai.org/.../deep-localization-of-protein-structures-in-fluorescence-microscopy-images

Further Reading

Last Updated: Mar 13, 2020

Sally Robertson

Written by

Sally Robertson

Sally first developed an interest in medical communications when she took on the role of Journal Development Editor for BioMed Central (BMC), after having graduated with a degree in biomedical science from Greenwich University.

Citations

Please use one of the following formats to cite this article in your essay, paper or report:

  • APA

    Robertson, Sally. (2020, March 13). Deep Learning in Fluorescence Microscopy. AZoLifeSciences. Retrieved on October 17, 2021 from https://www.azolifesciences.com/article/Deep-Learning-in-Fluorescence-Microscopy.aspx.

  • MLA

    Robertson, Sally. "Deep Learning in Fluorescence Microscopy". AZoLifeSciences. 17 October 2021. <https://www.azolifesciences.com/article/Deep-Learning-in-Fluorescence-Microscopy.aspx>.

  • Chicago

    Robertson, Sally. "Deep Learning in Fluorescence Microscopy". AZoLifeSciences. https://www.azolifesciences.com/article/Deep-Learning-in-Fluorescence-Microscopy.aspx. (accessed October 17, 2021).

  • Harvard

    Robertson, Sally. 2020. Deep Learning in Fluorescence Microscopy. AZoLifeSciences, viewed 17 October 2021, https://www.azolifesciences.com/article/Deep-Learning-in-Fluorescence-Microscopy.aspx.

Comments

The opinions expressed here are the views of the writer and do not necessarily reflect the views and opinions of AZoLifeSciences.
You might also like... ×
Advanced video microscopy reveals molecular details of malaria invasion