Search

Pathological diagnosis relies on the visual inspection of histologically stained thin tissue specimens, where different types of stains are applied to bring contrast to and highlight various desired histological features. However, the destructive histochemical staining procedures are usually irreversible, making it very difficult to obtain multiple stains on the same tissue section. Here, we demonstrate a virtual stain transfer framework via a cascaded deep neural network (C-DNN) to digitally transform hematoxylin and eosin (H&E) stained tissue images into other types of histological stains. Unlike a single neural network structure which only takes one stain type as input to digitally output images of another stain type, C-DNN first uses virtual staining to transform autofluorescence microscopy images into H&E and then performs stain transfer from H&E to the domain of the other stain in a cascaded manner. This cascaded structure in the training phase allows the model to directly exploit histochemically stained image data on both H&E and the target special stain of interest. This advantage alleviates the challenge of paired data acquisition and improves the image quality and color accuracy of the virtual stain transfer from H&E to another stain. We validated the superior performance of this C-DNN approach using kidney needle core biopsy tissue sections and successfully transferred the H&E-stained tissue images into virtual PAS (periodic acid-Schiff) stain. This method provides high-quality virtual images of special stains using existing, histochemically stained slides and creates new opportunities in digital pathology by performing highly accurate stain-to-stain transformations., 14 Pages, 4 Figures, 1 Table

Early detection and identification of pathogenic bacteria such as Escherichia coli (E. coli) is an essential task for public health. The conventional culture-based methods for bacterial colony detection usually take >24 hours to get the final read-out. Here, we demonstrate a bacterial colony-forming-unit (CFU) detection system exploiting a thin-film-transistor (TFT)-based image sensor array that saves ~12 hours compared to the Environmental Protection Agency (EPA)-approved methods. To demonstrate the efficacy of this CFU detection system, a lensfree imaging modality was built using the TFT image sensor with a sample field-of-view of ~10 cm^2. Time-lapse images of bacterial colonies cultured on chromogenic agar plates were automatically collected at 5-minute intervals. Two deep neural networks were used to detect and count the growing colonies and identify their species. When blindly tested with 265 colonies of E. coli and other coliform bacteria (i.e., Citrobacter and Klebsiella pneumoniae), our system reached an average CFU detection rate of 97.3% at 9 hours of incubation and an average recovery rate of 91.6% at ~12 hours. This TFT-based sensor can be applied to various microbiological detection methods. Due to the large scalability, ultra-large field-of-view, and low cost of the TFT-based image sensors, this platform can be integrated with each agar plate to be tested and disposed of after the automated CFU count. The imaging field-of-view of this platform can be cost-effectively increased to >100 cm^2 to provide a massive throughput for CFU detection using, e.g., roll-to-roll manufacturing of TFTs as used in the flexible display industry., 18 Pages, 6 Figures

The application of deep learning techniques has greatly enhanced holographic imaging capabilities, leading to improved phase recovery and image reconstruction. Here, we introduce a deep neural network termed enhanced Fourier Imager Network (eFIN) as a highly generalizable framework for hologram reconstruction with pixel super-resolution and image autofocusing. Through holographic microscopy experiments involving lung, prostate and salivary gland tissue sections and Papanicolau (Pap) smears, we demonstrate that eFIN has a superior image reconstruction quality and exhibits external generalization to new types of samples never seen during the training phase. This network achieves a wide autofocusing axial range of 0.35 mm, with the capability to accurately predict the hologram axial distances by physics-informed learning. eFIN enables 3x pixel super-resolution imaging and increases the space-bandwidth product of the reconstructed images by 9-fold with almost no performance loss, which allows for significant time savings in holographic imaging and data processing steps. Our results showcase the advancements of eFIN in pushing the boundaries of holographic imaging for various applications in e.g., quantitative phase imaging and label-free microscopy., 10 Pages, 4 Figures

A well-known problem with distance-based formation control is the existence of multiple equilibrium points not associated with the desired formation. This problem can be potentially mitigated by introducing an additional controlled variable. In this article, we generalize the distance + angle-based scheme for 2-D formations of single-integrator agents by using directed graphs and triangulation of the n-agent formation. We show that under certain conditions on the control gains and desired formation shape, our controller ensures the asymptotic stability of the correct formation for almost all initial agent positions.

Human multiple myeloma is an incurability of haematological cancer with recurrence rates above 90%. For the successful use of chemotherapy in MM, the critical limiting factor is the severe adverse effects of this drug. The anticancer efficiency of Losmapimod encapsulated in poly(lactic-co-glycolic acid) nanoparticles (LOS@PLGA-NPs) in vitro using multiple myeloma cells has been established. Furthermore, the morphological examination, hydrodynamic size and zeta potential of the nanoparticle were examined by the high-resolution transmission electron microscopy and dynamic light scattering method. The Cell Counting Kit-8 analysis has shown a significant inhibitory impact of LOS@PLGA-NPs on MM cancer cells (U266 and IM9). Furthermore, an investigation of the mechanism showed that apoptosis in cancer cells has performed by anticancer activity. We also demonstrated that such drugs showed promise antimetastatic efficacy and inhibited cancer cell invasiveness. Importantly, LOS@PLGA-NPs have confirmed to indicate that the dramatically better animal safety profile of the LOS is highlighted as prospective candidates for MM cancer treatment using these LOS@PLGA-NPs and deserve further evaluation.

Exposure to bio-aerosols such as mold spores and pollen can lead to adverse health effects. There is a need for a portable and cost-effective device for long-term monitoring and quantification of various bio-aerosols. To address this need, we present a mobile and cost-effective label-free bio-aerosol sensor that takes holographic images of flowing particulate matter concentrated by a virtual impactor, which selectively slows down and guides particles larger than ~6 microns to fly through an imaging window. The flowing particles are illuminated by a pulsed laser diode, casting their inline holograms on a CMOS image sensor in a lens-free mobile imaging device. The illumination contains three short pulses with a negligible shift of the flowing particle within one pulse, and triplicate holograms of the same particle are recorded at a single frame before it exits the imaging field-of-view, revealing different perspectives of each particle. The particles within the virtual impactor are localized through a differential detection scheme, and a deep neural network classifies the aerosol type in a label-free manner, based on the acquired holographic images. We demonstrated the success of this mobile bio-aerosol detector with a virtual impactor using different types of pollen (i.e., bermuda, elm, oak, pine, sycamore, and wheat) and achieved a blind classification accuracy of 92.91%. This mobile and cost-effective device weighs ~700 g and can be used for label-free sensing and quantification of various bio-aerosols over extended periods since it is based on a cartridge-free virtual impactor that does not capture or immobilize particulate matter., 23 Pages, 5 Figures, 1 Table

Digital holography is one of the most widely used label-free microscopy techniques in biomedical imaging. Recovery of the missing phase information of a hologram is an important step in holographic image reconstruction. Here we demonstrate a convolutional recurrent neural network (RNN) based phase recovery approach that uses multiple holograms, captured at different sample-to-sensor distances to rapidly reconstruct the phase and amplitude information of a sample, while also performing autofocusing through the same network. We demonstrated the success of this deep learning-enabled holography method by imaging microscopic features of human tissue samples and Papanicolaou (Pap) smears. These results constitute the first demonstration of the use of recurrent neural networks for holographic imaging and phase recovery, and compared with existing methods, the presented approach improves the reconstructed image quality, while also increasing the depth-of-field and inference speed., Comment: 18 Pages, 7 Figures, 1 Table

In this brief, we present solutions to the flocking and target interception problems of multiple nonholonomic unicycle-type robots using the distance-based framework. The control laws are designed at the kinematic level and are based on the rigidity properties of the graph modeling the sensing/communication interactions among the robots. An input transformation is used to facilitate the control design by converting the nonholonomic model into the single-integrator-like equation. We assume only a subset of the robots knows the desired, time-varying flocking velocity, or the target’s motion. The resulting control schemes include distributed, variable structure observers to estimate the unknown signals. Our stability analyses prove convergence to the desired formation while tracking the flocking velocity or the target motion. The results are supported by experiments.

We present a deep learning-based image reconstruction method in swept-source optical coherent tomography (OCT) using undersampled spectral data. This method can improve the imaging speed without the need for any hardware modifications.

We report in vivo virtual histology of skin without a biopsy, where deep learning is used to virtually stain tissue and generate hematoxylin and eosin (H&E)-like microscopic images of skin using a reflectance confocal microscope.

We demonstrate a recurrent neural network (RNN) enabled holographic imaging method that simultaneously performs autofocusing and phase recovery, achieving faster reconstruction speed and extended depth-of-field.

We introduce an end-to-end neural network called Fourier Imager Network (FIN) for rapid phase recovery and hologram reconstruction that achieves superior generalization to unseen sample types over the existing deep learning methods.

We demonstrate few-shot generalization of an RNN-based holographic image reconstruction network to small datasets of new sample/tissue types never seen in training, which achieved faster convergence and improved reconstruction quality with less trainable parameters.

We present a deep learning-based technique to computationally transform H&E-stained tissue sections into different special stains. We also demonstrate that this stain-to-stain transformation framework improves diagnostic accuracy over the use of H&E only.

We present a deep learning-based framework to virtually transfer brightfield images of H&E-stained tissue slides to other types of stains using cascaded networks, providing high-quality images of special stains from existing H&E stained tissue images.

We report a rapid, automated, portable and high-throughput bacterial colony detection and classification system using a thin-film-transistor array and deep learning, achieving >12 hours of time savings compared to US Environmental Protection Agency-approved culture-based methods.

An invasive biopsy followed by histological staining is the benchmark for pathological diagnosis of skin tumors. The process is cumbersome and time-consuming, often leading to unnecessary biopsies and scars. Emerging noninvasive optical technologies such as reflectance confocal microscopy (RCM) can provide label-free, cellular-level resolution, in vivo images of skin without performing a biopsy. Although RCM is a useful diagnostic tool, it requires specialized training because the acquired images are grayscale, lack nuclear features, and are difficult to correlate with tissue pathology. Here, we present a deep learning-based framework that uses a convolutional neural network to rapidly transform in vivo RCM images of unstained skin into virtually-stained hematoxylin and eosin-like images with microscopic resolution, enabling visualization of the epidermis, dermal-epidermal junction, and superficial dermis layers. The network was trained under an adversarial learning scheme, which takes ex vivo RCM images of excised unstained/label-free tissue as inputs and uses the microscopic images of the same tissue labeled with acetic acid nuclear contrast staining as the ground truth. We show that this trained neural network can be used to rapidly perform virtual histology of in vivo, label-free RCM images of normal skin structure, basal cell carcinoma, and melanocytic nevi with pigmented melanocytes, demonstrating similar histological features to traditional histology from the same excised tissue. This application of deep learning-based virtual staining to noninvasive imaging technologies may permit more rapid diagnoses of malignant skin neoplasms and reduce invasive skin biopsies.

Pathology is practiced by visual inspection of histochemically stained tissue slides. While the hematoxylin and eosin (H&E) stain is most commonly used, special stains can provide additional contrast to different tissue components. Here, we demonstrate the utility of supervised learning-based computational stain transformation from H&E to special stains (Masson’s Trichrome, periodic acid-Schiff and Jones silver stain) using kidney needle core biopsy tissue sections. Based on the evaluation by three renal pathologists, followed by adjudication by a fourth pathologist, we show that the generation of virtual special stains from existing H&E images improves the diagnosis of several non-neoplastic kidney diseases, sampled from 58 unique subjects (P = 0.0095). A second study found that the quality of the computationally generated special stains was statistically equivalent to those which were histochemically stained. This stain-to-stain transformation framework can improve preliminary diagnoses when additional special stains are needed, also providing significant savings in time and cost., Performing multiple histological stains on a biopsy can be costly and time consuming. Here the authors present a method for the digital transformation of H&E stained tissue into special stains (e.g., PAS, Masson’s Trichrome and Jones silver stain), and demonstrate that it improves diagnoses over the use of H&E only.

A method for dealing with the problem of convergence to incorrect equilibrium points of distance-based formation controllers was recently proposed by introducing an additional controlled variable, ...

We present a deep learning framework based on a generative adversarial network (GAN) to perform super-resolution in coherent imaging systems. We demonstrate that this framework can enhance the resolution of both pixel size-limited and diffraction-limited coherent imaging systems. We experimentally validated the capabilities of this deep learning-based coherent imaging approach by super-resolving complex images acquired using a lensfree on-chip holographic microscope, the resolution of which was pixel size-limited. Using the same GAN-based approach, we also improved the resolution of a lens-based holographic imaging system that was limited in resolution by the numerical aperture of its objective lens. This deep learning-based super-resolution framework can be broadly applied to enhance the space-bandwidth product of coherent imaging systems using image data and convolutional neural networks, and provides a rapid, non-iterative method for solving inverse image reconstruction or enhancement problems in optics., Comment: 18 pages, 9 figures, 3 tables

The efficiency of drug development defined as a number of successfully launched new pharmaceuticals normalized by financial investments has significantly declined. Nonetheless, recent advances in high-throughput experimental techniques and computational modeling promise reductions in the costs and development times required to bring new drugs to market. The prediction of toxicity of drug candidates is one of the important components of modern drug discovery. In this work, we describe eToxPred, a new approach to reliably estimate the toxicity and synthetic accessibility of small organic compounds. eToxPred employs machine learning algorithms trained on molecular fingerprints to evaluate drug candidates. The performance is assessed against multiple datasets containing known drugs, potentially hazardous chemicals, natural products, and synthetic bioactive compounds. Encouragingly, eToxPred predicts the synthetic accessibility with the mean square error of only 4% and the toxicity with the accuracy of as high as 72%. eToxPred can be incorporated into protocols to construct custom libraries for virtual screening in order to filter out those drug candidates that are potentially toxic or would be difficult to synthesize. It is freely available as a stand-alone software at https://github.com/pulimeng/etoxpred .

Parasitic infections constitute a major global public health issue. Existing screening methods that are based on manual microscopic examination often struggle to provide sufficient volumetric throughput and sensitivity to facilitate early diagnosis. Here, we demonstrate a motility-based label-free computational imaging platform to rapidly detect motile parasites in optically dense bodily fluids by utilizing the locomotion of the parasites as a specific biomarker and endogenous contrast mechanism. Based on this principle, a cost-effective and mobile instrument, which rapidly screens ~3.2 mL of fluid sample in three dimensions, was built to automatically detect and count motile microorganisms using their holographic time-lapse speckle patterns. We demonstrate the capabilities of our platform by detecting trypanosomes, which are motile protozoan parasites, with various species that cause deadly diseases affecting millions of people worldwide. Using a holographic speckle analysis algorithm combined with deep learning-based classification, we demonstrate sensitive and label-free detection of trypanosomes within spiked whole blood and artificial cerebrospinal fluid (CSF) samples, achieving a limit of detection of ten trypanosomes per mL of whole blood (~five-fold better than the current state-of-the-art parasitological method) and three trypanosomes per mL of CSF. We further demonstrate that this platform can be applied to detect other motile parasites by imaging Trichomonas vaginalis, the causative agent of trichomoniasis, which affects 275 million people worldwide. With its cost-effective, portable design and rapid screening time, this unique platform has the potential to be applied for sensitive and timely diagnosis of neglected tropical diseases caused by motile parasites and other parasitic infections in resource-limited regions., Imaging: Detecting parasites with holograms and AI Scientists have developed a cost-effective and portable technology that uses three-dimensional holograms to detect parasites in human bodily fluids, providing a more sensitive technique for diagnosing diseases, without the need for chemical labeling. A major global health problem, parasitic infections affect billions of people around the world and can lead to severe illnesses and death. Some of the current screening techniques, however, use manual microscopic examinations that struggle to provide the sensitivity required for early diagnoses. Now, Aydogan Ozcan and colleagues from the University of California Los Angeles in the United States have developed a cost-effective and portable optical device that uses lensless time-resolved holographic speckle imaging combined with artificial intelligence to automatically detect rare parasites in bodily fluids. The device has the potential to perform rapid and early diagnoses of diseases caused by parasitic infections and could be used in developing countries.

We present a deep learning-based high-throughput cytometer to detect rare cells in whole blood using a cost-effective and light-weight design. This system uses magnetic-particles to label and enrich the target cells. Then, a periodically-alternating magnetic-field creates time-modulated diffraction patterns of the target cells that are recorded using a lensless microscope. Finally, a custom-designed convolutional network is used to detect and classify the target cells based on their modulated spatio-temporal patterns. This cytometer was tested with cancer cells spiked in whole blood to achieve a limit-of-detection of 10 cells/mL. This compact, cost-effective and high-throughput cytometer might serve diagnostics needs in resource-limited-settings.