Search

Head and neck squamous cell carcinoma (HNSCC) has a high mortality rate. In this study, we developed a Stokes-vector-derived polarized hyperspectral imaging (PHSI) system for H&E-stained pathological slides with HNSCC and built a dataset to develop a deep learning classification method based on convolutional neural networks (CNN). We use our polarized hyperspectral microscope to collect the four Stokes parameter hypercubes (S0, S1, S2, and S3) from 56 patients and synthesize pseudo-RGB images using a transformation function that approximates the human eye's spectral response to visual stimuli. Each image is divided into patches. Data augmentation is applied using rotations and flipping. We create a four-branch model architecture where each branch is trained on one Stokes parameter individually, then we freeze the branches and fine-tune the top layers of our model to generate final predictions. Our results show high accuracy, sensitivity, and specificity, indicating that our model performed well on our dataset. Future works can improve upon these results by training on more varied data, classifying tumors based on their grade, and introducing more recent architectural techniques.

Zebrafish is a well-established animal model for developmental and disease studies. Its optical transparency at early developmental stages is ideal for tissue visualization. Interaction of light with zebrafish tissues provides information on their structure and properties. In this study, we developed a microscopic imaging system for improving the visualization of unstained zebrafish tissues on tissue slides, with two different setups: polarized light imaging and polarized hyperspectral imaging. Based on the polarized light imaging setup, we collected the RGB images of Stokes vector parameters (S0, S1, S2, and S3), and calculated the Stokes vector derived parameters: the degree of polarization (DOP), the degree of linear polarization (DOLP)). We also calculated Stokes vector data based on the polarized hyperspectral imaging setup. The preliminary results demonstrate that Stokes vector data in two imaging setups (polarized light imaging and polarized hyperspectral imaging) are capable of improving the visualization of different types of zebrafish tissues (brain, muscle, skin cells, blood vessels, and yolk). Using the images collected from larval zebrafish samples by polarized light imaging, we found that DOP and DOLP could show clearer structural information of the brain and of skin cells, muscle and blood vessels in the tail. Furthermore, DOP and DOLP parameters derived from images collected by polarized hyperspectral imaging could show clearer structural information of skin cells developing around yolk as well as the surrounding blood vessel network. In addition, polarized hyperspectral imaging could provide complementary spectral information to the spatial information on Stokes vector data of zebrafish tissues. The polarized light imaging & polarized hyperspectral imaging systems provide a better insight into the microstructures of zebrafish tissues.

White blood cells, also called leukocytes, are hematopoietic cells of the immune system that are involved in protecting the body against both infectious diseases and foreign materials. The abnormal development and uncontrolled proliferation of these cells can lead to devastating cancers. Their timely recognition in the peripheral blood is critical to diagnosis and treatment. In this study, we developed a microscopic imaging system for improving the visualization of white blood cells on Wright's stained blood smear slides, with two different setups: polarized light imaging and polarized hyperspectral imaging. Based on the polarized light imaging setup, we collected the RGB images of Stokes vector parameters (S0, S1, S2, and S3) of five types of white blood cells (neutrophil, eosinophil, basophil, lymphocyte, and monocyte), and calculated the Stokes vector derived parameters: the degree of polarization (DOP), the degree of linear polarization (DOLP), and the degree of circular polarization (DOCP)). We also calculated Stokes vector data based on the polarized hyperspectral imaging setup. The preliminary results demonstrate that Stokes vector derived parameters (DOP, DOLP, and DOCP) could improve the visualization of granules in granulocytes (neutrophils, eosinophils, and basophils). Furthermore, Stokes vector derived parameters (DOP, DOLP, and DOCP) could improve the visualization of surface structures (protein patterns) of lymphocytes enabling subclassification of lymphocyte subpopulations. Finally, S2, S3, and DOCP could enhance the morphologic visualization of monocyte nucleus. We also demonstrated that the polarized hyperspectral imaging setup could provide complementary spectral information to the spatial information on different Stokes vector parameters of white blood cells. This work demonstrates that polarized light imaging & polarized hyperspectral imaging has the potential to become a strong imaging tool in the diagnosis of disorders arising from white blood cells.

We developed a polarized hyperspectral microscope to collect four types of Stokes vector data cubes (S0, S1, S2, and S3) of the pathologic slides with head and neck squamous cell carcinoma (HNSCC). Our system consists of an optical light microscope with a movable stage, two polarizers, two liquid crystal variable retarders (LCVRs), and a SnapScan hyperspectral camera. The polarizers and LCVRs work in tandem with the hyperspectral camera to acquire polarized hyperspectral images. Synthetic pseudo-RGB images are generated from the four Stokes vector data cubes based on a transformation function similar to the spectral response of human eye for the visualization of hyperspectral images. Collagen is the most abundant extracellular matrix (ECM) protein in the human body. A major focus of studying the ECM in tumor microenvironment is the role of collagen in both normal and abnormal function. Collagen tends to accumulate in and around tumors during cancer development and growth. In this study, we acquired images from normal regions containing normal cells and collagen fibers and from tumor regions containing cancerous squamous cells and collagen fibers on HNSCC pathologic slides. The preliminary results demonstrated that our customized polarized hyperspectral microscope is able to improve the visualization of collagen on HNSCC pathologic slides under different situations, including thick fibers of normal stroma, thin fibers of normal stroma, fibers of normal muscle cells, fibers accumulated in tumors, fibers accumulated around tumors. Our preliminary results also demonstrated that the customized polarized hyperspectral microscope is capable of extracting the spectral signatures of collagen based on Stokes vector parameters and can have various applications in pathology and oncology.

Polarized light interactions with biological tissues can reveal information regarding tissue structure, while spectral characteristics are closely related to tissue composition. An integration of both modalities in a compact system could better assist tissue assessment. This study aims to develop a polarized hyperspectral imaging (PHSI) system that fulfills both linearly and circularly polarized hyperspectral imaging for in vivo and ex vivo applications. The system is comprised of a white LED, two linear polarizers, two liquid crystal variable retarders (LCVRs), and a hyperspectral snapshot camera. The system was calibrated to compute the full Stokes polarimetry. For tissue differentiation, fresh ex vivo mouse tissue specimens from kidney, liver, spleen, muscle, lung, and salivary gland of mice were imaged. The spectra of three features, named degree of polarization (DOP), degree of linear polarization (DOLP), and degree of circular polarization (DOCP), were generated. A k-nearest neighbor (k-NN) classifier was trained with multi-class spectra and 5-fold cross validation. It was found that DOP better differentiates tissue with an average accuracy of 0.87. Additionally, support vector machine (SVM) classifiers were trained to differentiate between each two of the organs, and it was determined that DOLP better identified kidney, liver, and spleen, whereas DOCP better identified muscle and lung tissues. Then, the setup was employed to image in vivo human fingers with and without a blood occlusion to qualitatively estimate oxygen saturation. Preliminary results demonstrate that both DOLP and DOCP reveal a distinction of oxygen saturation states. These results demonstrate the feasibility of the PHSI system for distinguishing between optical properties of tissues, which has the potential to reveal disease-related information for diverse medical applications.

The study is to incorporate polarized hyperspectral imaging (PHSI) with deep learning for automatic detection of head and neck squamous cell carcinoma (SCC) on hematoxylin and eosin (H&E) stained tissue slides. A polarized hyperspectral imaging microscope had been developed in our group. In this paper, we firstly collected the Stokes vector data cubes (S0, S1, S2, and S3) of histologic slides from 17 patients with SCC by the PHSI microscope, under the wavelength range from 467 nm to 750 nm. Secondly, we generated the synthetic RGB images from the original Stokes vector data cubes. Thirdly, we cropped the synthetic RGB images into image patches at the image size of 96×96 pixels, and then set up a ResNet50-based convolutional neural network (CNN) to classify the image patches of the four Stokes vector parameters (S0, S1, S2, and S3) by application of transfer learning. To test the performances of the model, each time we trained the model based on the image patches (S0, S1, S2, and S3) of 16 patients out of 17 patients, and used the trained model to calculate the testing accuracy based on the image patches of the rest 1 patient (S0, S1, S2, and S3). We repeated the process for 6 times and obtained 24 testing accuracies (S0, S1, S2, and S3) from 6 different patients out of the 17 patients. The preliminary results showed that the average testing accuracy (84.2%) on S3 outperformed the average testing accuracy (83.5%) on S0. Furthermore, 4 of 6 testing accuracies of S3 (96.0%, 87.3%, 82.8%, and 86.7%) outperformed the testing accuracies of S0 (93.3%, 85.2%, 80.2%, and 79.0%). The study demonstrated the potential of using polarized hyperspectral imaging and deep learning for automatic detection of head and neck SCC on pathologic slides.