Artificial Intelligence for 3D Visualization and Analysis Software

Accelerate your imaging data processing, segmentation, and classification using Machine and Deep Learning in Amira, Avizo, and PerGeos Software

Artificial intelligence (AI) methods, such as machine learning and deep learning, have proven to be powerful approaches for automating image segmentation and improving image quality.

The use of AI-based tools in Thermo Scientific Amira-Avizo 2D Software, Amira-Avizo Software, and PerGeos Software is a major leap forward and enriches processing capabilities by allowing the ability to mix both traditional and AI-based algorithms.

Deep Learning in Amira-Avizo Software and PerGeos Software

Deep-learned neural networks have proven to be invaluable tools for many research and industrial purposes in recent years. Using deep learning for processing images allows researchers to go beyond traditional image processing for greatly improved results.

Amira-Avizo Software and PerGeos Software provide ideal environments for deep learning.

A rich image pre/post-processing toolbox supplementing user-friendly manual and supervised segmentation allows enhanced data annotation and preparation for the training phase and the prediction phase. It also leverages the actual model building, training, and prediction steps from experienced deep learning frameworks, such as TensorFlow and Keras. The workflow for learning from a manually segmented subset and performing the prediction on a complete data set is as simple as applying two modules.

Click image to enlarge

Deep learning model trained on a subvolume for detecting membranes and mitochondria, prediction applied on the full volume. Data courtesy of Cardona A, et al. 2010. An Integrated Micro- and Macroarchitectural Analysis of the Drosophila Brain by Computer-Assisted Serial Section Electron Microscopy. PLoS Biol 8(10): e1000502. doi:10.1371/journal.pbio.1000502.

Download the above example

Deep Learning off-the-shelf

The Deep Learning Prediction module allows for trained models to be used on data. It can be incorporated within any workflow or recipe to automate your image processing, segmentation, or analysis tasks.

The deep learning training modules feature a highly configurable tool for training models using state-of-the-art architectures, such as Unet with Resnet or VGG backbones, data augmentation, a selection of loss, and metric functions. The training can occur from scratch (random weights) or from pre-trained weights.

The training is monitored in real time using TensorBoard to track metrics, such as loss and accuracy, or to visualize the model’s architecture.

Watch accuracy and loss on training and validation sets.

Customizable solution

Amira-Avizo Software and PerGeos Software’s latest versions provide a default set of deep learning Python packages and modules based on Keras, a high-level neural networks API running on top of TensorFlow. You can go further by customizing your Python environment or creating multiple self-contained Python environments with their own sets of packages and use them within your own Python Script Modules (pyscro).

Advanced users and Python programmers can customize both the Deep Learning Training and Prediction modules. A plugin system allows for the definition of custom model architectures, loss, or metric functions, which can be readily available from the GUI Deep Learning Training module.

Prediction can also be customized from pre- to post-prediction processing, to enable full control on the input and outputs and to optimize memory usage.

Applications and use cases

Image segmentation of mitochondria blobs

Mitochondria are difficult to segment using traditional approaches because they have connections with the outer endoplasmic reticulum and an internal membrane-like structure.

The model trained with Amira-Avizo Software’s deep learning tool allows the automatic extraction of mitochondria from a FIB-SEM stack. The training was done using only a few slices, which were segmented manually with Amira-Avizo Software’s segmentation editor. It was then possible to automatically segment the rest of the stack, saving hours of manual work.

(Left) Manual segmentation using Amira-Avizo Software's segmentation editor, and (right) 3D visualization of the mitochondria from the automatic segmentation of the full stack with deep learning. Data courtesy of Advanced Imaging Res. Center, Kurume Univ. Sch. Med.

Image denoising of SEM

For 3D serial sectioning and 2D tiling applications, time to data versus image quality must be carefully balanced. Usually, the data is heavily down-sampled to process it. Following acquisition, conventional algorithms, such as gaussian-smoothing and non-local-means filtering, leave artifacts. Alternatively, deep learning algorithms can be tuned in such a way that they do not induce artifacts. Processing can be done relatively quickly when a deep learning model is available. Below, we highlight a model that can quickly restore SEM images.

Download the above example

Deep Learning segmentation of pores in solid oxide fuel cells in FIB-SEM

FIB-SEM 3D images of non-impregnated porous media suffered from a so-called pore-back effect, in which the back of a pore can be easily confused with a solid material lying on the section. In this case, we trained a deep learning model to recognize pore-backs by segmenting a number of image patches using traditional supervised techniques.

Illustration of the training of a deep learning model based on patches, including prediction on the full dataset. Solid Oxide Fuel Cell dataset courtesy of Sabanci University, Turkey.

Facies Classification with Supervised Machine Learning

Rock type (facies) identification plays a key role in the exploration and development of oil and gas reservoirs. Traditional core-based facies identification is costly, time consuming, and subjective. Machine Learning allows you to address the challenges in a fully automated and reproducible way.

Learn more about Automatic Facies Classification

Click image to enlarge

Color Auto Classification based on machine learning

Based on machine learning, the Color Auto Classification tool automatically segments a color image into labels. A supervised random forest method is used.

Automatic segmentation of an optical image of a thin section using Color Auto Classification. Data courtesy of Stratum Reservoir.

Texture Supervised Classification based on machine learning

Texture classification is a machine learning technique that relies on learning texture patterns from markers defined by the user and then classifying each pixel of the image according to its similarity to the learned patterns.

Products

Style Sheet for Instrument Cards Origin