Machine Learning and artificial intelligence algorithms for smartphone-based malaria screening and diagnostics

Data scientists with machine learning and artificial intelligence expertise at NLM have developed new intelligent methods to screen for malaria. Particularly, scientists used deep learning methods to detect and count infected red blood cells and parasites. Deep learning is a family of machine learning methods based on artificial neural networks. The scientists adapted these methods to take account of the specific features of blood smear images, in particular the relatively small size of parasites. Algorithms were developed for both thin and thick smears, which are the two types of bloods smears used by experts in the field to diagnose malaria. The networks can detect blood cells in thin smears and classify them into infected and uninfected cells. They can also detect parasites in thick smears, thus covering the full screening spectrum in practice. Proven algorithms were then transformed for use on on Android smartphones. This required research into designing new and smaller network structures that can cope with the lower hardware specifications of smartphones in terms of processing units and memory. The resulting system is the first smartphone application for malaria screening that can process both thin and thick smears using deep learning. The application has an integrated image upload function to upload images to a cloud storage for further processing or archiving, and supports multiple languages. The software is publicly available in the Google Play Store (NLM MalariaScreener), where researchers can download it for testing or contributing training data. Future efforts plan to make the modularized source code publicly available to promote greater worldwide use. Field testing continues with collaborations with several sites worldwide: viz., Thailand, Bangladesh, Kenya, Uganda, Pakistan, Thailand, and Mali. The goal is to test the stability of the app in different environments, under different imaging parameters, and to verify the high correlation of the algorithmic output with expert counts, as in the practical experiments published. Researchers are also in contact with FIND (www.finddx.org), a non-profit organization driving innovation in the development and delivery of diagnostics, to test the app in the field and make improvements. Several suggested advances have been already implemented.. To train the mobile smartphone application for thin and thick smears, scientists used annotated images sets. For thick smears, they acquired about 3000 images from 200 patients at a hospital in Chittagong, Bangladesh, including 85,000 malaria parasites, which an expert all annotated manually. They collected a similarly-sized dataset for thin smears. Both thin and thick smear images were acquired for Plasmodium falciparum, which is the deadliest parasite species causing malaria in humans. In this fiscal year, researchers acquired similar data with Plasmodium vivax, which is another common malaria parasite species, in both thin and thick smears. Using this data, researchers have started training of deep learning networks for detecting and discriminating parasite species, which is an important clinical decision in practice. CEB will make all acquired data publicly available over time so that other researchers can train and test their algorithm. As a first step, researchers made more than 1800 thick smear images from 150 patients with Plasmodium falciparum infections available in conjunction with a publication. In terms of methods, NLM researchers developed a custom-made segmentation technique to extract red blood cells from a blood smear image for classification, arguing that the relatively small size of red blood cells compared to the image dimensions can be a problem for other techniques. Researchers also implemented a multi-class classifier than can detect infected and uninfected red blood cells as well as white blood cells, in an attempt to avoid detecting white blood cells in a separate preprocessing step. Other methods developed include an augmentation method to add new instances of objects, such as infected cells, into an image to generate larger, and balanced training data. This method can be used as a pre-processing step for segmentation and object detection networks (for example, Faster R-CNN, Mask R-CNN, or YOLO).

NIH Project details

Diagnostics
Digital Tools
Surveillance

Jan 2017 — Jan 2020

$2.64M

United States