Search

Abstract Hyperspectral (HS) imaging (HSI) technology combines the main features of two existing technologies: imaging and spectroscopy. This allows to analyse simultaneously the morphological and chemical attributes of the objects captured by a HS camera. In recent years, the use of HSI provides valuable insights into the interaction between light and biological tissues, and makes it possible to detect patterns, cells, or biomarkers, thus, being able to identify diseases. This work presents the HistologyHSI-GB dataset, which contains 469 HS images from 13 patients diagnosed with brain tumours, specifically glioblastoma. The slides were stained with haematoxylin and eosin (H&E) and captured using a microscope at 20× power magnification. Skilled histopathologists diagnosed the slides and provided image-level annotations. The dataset was acquired using custom HSI instrumentation, consisting of a microscope equipped with an HS camera covering the spectral range from 400 to 1000 nm.

Abstract Brain surgery is one of the most common and effective treatments for brain tumour. However, neurosurgeons face the challenge of determining the boundaries of the tumour to achieve maximum resection, while avoiding damage to normal tissue that may cause neurological sequelae to patients. Hyperspectral (HS) imaging (HSI) has shown remarkable results as a diagnostic tool for tumour detection in different medical applications. In this work, we demonstrate, with a robust k-fold cross-validation approach, that HSI combined with the proposed processing framework is a promising intraoperative tool for in-vivo identification and delineation of brain tumours, including both primary (high-grade and low-grade) and secondary tumours. Analysis of the in-vivo brain database, consisting of 61 HS images from 34 different patients, achieve a highest median macro F1-Score result of 70.2 ± 7.9% on the test set using both spectral and spatial information. Here, we provide a benchmark based on machine learning for further developments in the field of in-vivo brain tumour detection and delineation using hyperspectral imaging to be used as a real-time decision support tool during neurosurgical workflows.

Data scarcity is a common and challenging issue when working with Artificial Intelligence solutions, especially those including Deep Learning (DL) models for tasks such as image classification. This is particularly relevant in healthcare scenarios, in which data collection requires a long-lasting process, involving specific control protocols. The performance of DL models is usually quantified by different classification metrics, which may provide biased results, due to the lack of sufficient data. In this paper, an innovative approach is proposed to evaluate the performance of DL models when labeled data is scarce. This approach, which aims to detect the poor performance provided by DL models, in spite of traditional assessing metrics indicating otherwise, is based on information theoretic concepts and motivated by the Information Bottleneck framework. This methodology has been evaluated by implementing several experimental configurations to classify samples from a plantar thermogram dataset, focused on early stage detection of diabetic foot ulcers, as a case study. The proposed network architectures exhibited high results in terms of classification metrics. However, as our approach shows, only two of those models are indeed consistent to generalize the data properly. In conclusion, a new methodology was introduced and tested to identify promising DL models for image classification over small datasets without relying exclusively on the widely employed classification metrics. Example code and supplementary material using a state-of-the-art DL model are available at https://github.com/mt4sd/PerformanceEvaluationScarceDataset.

Abstract Currently, intraoperative guidance tools used for brain tumor resection assistance during surgery have several limitations. Hyperspectral (HS) imaging is arising as a novel imaging technique that could offer new capabilities to delineate brain tumor tissue in surgical-time. However, the HS acquisition systems have some limitations regarding spatial and spectral resolution depending on the spectral range to be captured. Image fusion techniques combine information from different sensors to obtain an HS cube with improved spatial and spectral resolution. This paper describes the contributions to HS image fusion using two push-broom HS cameras, covering the visual and near-infrared (VNIR) [400–1000 nm] and near-infrared (NIR) [900–1700 nm] spectral ranges, which are integrated into an intraoperative HS acquisition system developed to delineate brain tumor tissue during neurosurgical procedures. Both HS images were registered using intensity-based and feature-based techniques with different geometric transformations to perform the HS image fusion, obtaining an HS cube with wide spectral range [435–1638 nm]. Four HS datasets were captured to verify the image registration and the fusion process. Moreover, segmentation and classification methods were evaluated to compare the performance results between the use of the VNIR and NIR data, independently, with respect to the fused data. The results reveal that the proposed methodology for fusing VNIR–NIR data improves the classification results up to 21% of accuracy with respect to the use of each data modality independently, depending on the targeted classification problem.

Due to the optimistic outcomes of the research on high-entropy alloys, new designs of these alloys are being encouraged. We studied the high-entropy CoCrFeMoNi alloy and the CoCrFeMoNi alloy doped with Zr. In order to choose the best electrical equivalent circuit for the prediction of the behavior of these high-entropy alloys at various potentials in artificial seawater, electrochemical impedance spectroscopy (EIS) measurements were conducted on samples with and without Zr-doped CoCrFeMoNi. At various potential levels, the impedance spectra were measured between −1.0 and +0.8 V vs. SCE. The study consists of a preliminary section with microstructure by metallography, open-circuit potential, and linear polarization curves by direct-current tests followed by visual analysis of the impedance spectra, and, finally, the selection of an equivalent electrical circuit model to fit the experimental data. By leveraging the advantages of EIS analysis, the information is essential for materials development, corrosion-mitigation strategies, and the successful implementation of these alloys in practical applications. It is important to note that selecting an equivalent circuit is often an iterative and subjective process, as it involves a balance between model complexity and the ability to accurately represent the system’s behavior.

Several causes make brain cancer identification a challenging task for neurosurgeons during the surgical procedure. The surgeons' naked eye sometimes is not enough to accurately delineate the brain tumor location and extension due to its diffuse nature that infiltrates in the surrounding healthy tissue. For this reason, a support system that provides accurate cancer delimitation is essential in order to improve the surgery outcomes and hence the patient's quality of life. The brain cancer detection system developed as part of the “HypErspectraL Imaging Cancer Detection” (HELICoiD) European project meets this requirement exploiting a non-invasive technique suitable for medical diagnosis: the hyperspectral imaging (HSI). A crucial constraint that this system has to satisfy is providing a real-time response in order to not prolong the surgery. The large amount of data that characterizes the hyperspectral images, and the complex elaborations performed by the classification system make the High Performance Computing (HPC) systems essential to provide real-time processing. The most efficient implementation developed in this work, which exploits the Graphic Processing Unit (GPU) technology, is able to classify the biggest image of the database (worst case) in less than three seconds, largely satisfying the real-time constraint set to 1 minute for surgical procedures, becoming a potential solution to implement hyperspectral video processing in the near future.

Diabetes mellitus presents a high prevalence around the world. A common and long-term derived complication is diabetic foot ulcers (DFUs), which have a global prevalence of roughly 6.3%, and a lifetime incidence of up to 34%. Infrared thermograms, covering the entire plantar aspect of both feet, can be employed to monitor the risk of developing a foot ulcer, because diabetic patients exhibit an abnormal pattern that may indicate a foot disorder. In this study, the publicly available INAOE dataset composed of thermogram images of healthy and diabetic subjects was employed to extract relevant features aiming to establish a set of state-of-the-art features that efficiently classify DFU. This database was extended and balanced by fusing it with private local thermograms from healthy volunteers and generating synthetic data via synthetic minority oversampling technique (SMOTE). State-of-the-art features were extracted using two classical approaches, LASSO and random forest, as well as two variational deep learning (DL)-based ones: concrete and variational dropout. Then, the most relevant features were detected and ranked. Subsequently, the extracted features were employed to classify subjects at risk of developing an ulcer using as reference a support vector machine (SVM) classifier with a fixed hyperparameter configuration to evaluate the robustness of the selected features. The new set of features extracted considerably differed from those currently considered state-of-the-art but provided a fair performance. Among the implemented extraction approaches, the variational DL ones, particularly the concrete dropout, performed the best, reporting an F1 score of 90% using the aforementioned SVM classifier. In comparison with features previously considered as the state-of-the-art, approximately 15% better performance was achieved for classification.

Currently, one of the most common causes of death worldwide is cancer. The development of innovative methods to support the early and accurate detection of cancers is required to increase the recovery rate of patients. Several studies have shown that medical Hyperspectral Imaging (HSI) combined with artificial intelligence algorithms is a powerful tool for cancer detection. Various preprocessing methods are commonly applied to hyperspectral data to improve the performance of the algorithms. However, there is currently no standard for these methods, and no studies have compared them so far in the medical field. In this work, we evaluated different combinations of preprocessing steps, including spatial and spectral smoothing, Min-Max scaling, Standard Normal Variate normalization, and a median spatial smoothing technique, with the goal of improving tumor detection in three different HSI databases concerning colorectal, esophagogastric, and brain cancers. Two machine learning and deep learning models were used to perform the pixel-wise classification. The results showed that the choice of preprocessing method affects the performance of tumor identification. The method that showed slightly better results with respect to identifing colorectal tumors was Median Filter preprocessing (0.94 of area under the curve). On the other hand, esophagogastric and brain tumors were more accurately identified using Min-Max scaling preprocessing (0.93 and 0.92 of area under the curve, respectively). However, it is observed that the Median Filter method smooths sharp spectral features, resulting in high variability in the classification performance. Therefore, based on these results, obtained with different databases acquired by different HSI instrumentation, the most relevant preprocessing technique identified in this work is Min-Max scaling.

Cancer originates from the uncontrolled growth of healthy cells into a mass. Chromophores, such as hemoglobin and melanin, characterize skin spectral properties, allowing the classification of lesions into different etiologies. Hyperspectral imaging systems gather skin-reflected and transmitted light into several wavelength ranges of the electromagnetic spectrum, enabling potential skin-lesion differentiation through machine learning algorithms. Challenged by data availability and tiny inter and intra-tumoral variability, here we introduce a pipeline based on deep neural networks to diagnose hyperspectral skin cancer images, targeting a handheld device equipped with a low-power graphical processing unit for routine clinical testing. Enhanced by data augmentation, transfer learning, and hyperparameter tuning, the proposed architectures aim to meet and improve the well-known dermatologist-level detection performances concerning both benign-malignant and multiclass classification tasks, being able to diagnose hyperspectral data considering real-time constraints. Experiments show 87% sensitivity and 88% specificity for benign-malignant classification and specificity above 80% for the multiclass scenario. AUC measurements suggest classification performance improvement above 90% with adequate thresholding. Concerning binary segmentation, we measured skin DICE and IOU higher than 90%. We estimated 1.21 s, at most, consuming 5 Watts to segment the epidermal lesions with the U-Net++ architecture, meeting the imposed time limit. Hence, we can diagnose hyperspectral epidermal data assuming real-time constraints.

Currently, the use of hyperspectral imaging (HSI) for the inspection of microscopic samples is an emerging trend in different fields. The use of push-broom hyperspectral (HS) cameras against other HSI technologies is motivated by their high spectral resolution and their capabilities to exploit spectral ranges beyond 1000 nm. Nevertheless, using push-broom cameras in miscroscopes imposes to perform an accurate spatial scanning of the sample to collect the HS data. In this manuscript, we present a methodology to correctly set-up a push-broom HS microscope to acquire high-quality HS images. Firstly, we describe a custom 3D printed mechanical system developed to perform the spatial scanning by producing a precise linear movement of the microscope stage. Then, we discuss how the dynamic range maximisation, the focusing, the alignment and the adequate speed determination affect the overall quality of the images. Finally, we present some examples of HS data showing the most common defects that usually appear when capturing HS images using a push-broom camera, and also a set of images acquired from real microscopic samples.

The use of hyperspectral imaging for medical applications is becoming more common in recent years. One of the main obstacles that researchers find when developing hyperspectral algorithms for medical applications is the lack of specific, publicly available, and hyperspectral medical data. The work described in this paper was developed within the framework of the European project HELICoiD (HypErspectraL Imaging Cancer Detection), which had as a main goal the application of hyperspectral imaging to the delineation of brain tumors in real-time during neurosurgical operations. In this paper, the methodology followed to generate the first hyperspectral database of in-vivo human brain tissues is presented. Data was acquired employing a customized hyperspectral acquisition system capable of capturing information in the Visual and Near InfraRed (VNIR) range from 400 to 1000 nm. Repeatability was assessed for the cases where two images of the same scene were captured consecutively. The analysis reveals that the system works more efficiently in the spectral range between 450 and 900 nm. A total of 36 hyperspectral images from 22 different patients were obtained. From these data, more than 300 000 spectral signatures were labeled employing a semi-automatic methodology based on the spectral angle mapper algorithm. Four different classes were defined: normal tissue, tumor tissue, blood vessel, and background elements. All the hyperspectral data has been made available in a public repository.

Hyperspectral (HS) imaging presents itself as a non-contact, non-ionizing and non-invasive technique, proven to be suitable for medical diagnosis. However, the volume of information contained in these images makes difficult providing the surgeon with information about the boundaries in real-time. To that end, High-Performance-Computing (HPC) platforms become necessary. This paper presents a comparison between the performances provided by five different HPC platforms while processing a spatial-spectral approach to classify HS images, assessing their main benefits and drawbacks. To provide a complete study, two different medical applications, with two different requirements, have been analyzed. The first application consists of HS images taken from neurosurgical operations; the second one presents HS images taken from dermatological interventions. While the main constraint for neurosurgical applications is the processing time, in other environments, as the dermatological one, other requirements can be considered. In that sense, energy efficiency is becoming a major challenge, since this kind of applications are usually developed as hand-held devices, thus depending on the battery capacity. These requirements have been considered to choose the target platforms: on the one hand, three of the most powerful Graphic Processing Units (GPUs) available in the market; and, on the other hand, a low-power GPU and a manycore architecture, both specifically thought for being used in battery-dependent environments.

In recent years, researchers designed several artificial intelligence solutions for healthcare applications, which usually evolved into functional solutions for clinical practice. Furthermore, deep learning (DL) methods are well-suited to process the broad amounts of data acquired by wearable devices, smartphones, and other sensors employed in different medical domains. Conceived to serve the role of diagnostic tool and surgical guidance, hyperspectral images emerged as a non-contact, non-ionizing, and label-free technology. However, the lack of large datasets to efficiently train the models limits DL applications in the medical field. Hence, its usage with hyperspectral images is still at an early stage. We propose a deep convolutional generative adversarial network to generate synthetic hyperspectral images of epidermal lesions, targeting skin cancer diagnosis, and overcome small-sized datasets challenges to train DL architectures. Experimental results show the effectiveness of the proposed framework, capable of generating synthetic data to train DL classifiers.

Hyperspectral Imaging (HSI) techniques have demonstrated potential to provide useful information in a broad set of applications in different domains, from precision agriculture to environmental science. A first step in the preparation of the algorithms to be employed outdoors starts at a laboratory level, capturing a high amount of samples to be analysed and processed in order to extract the necessary information about the spectral characteristics of the studied samples in the most precise way. In this article, a custom-made scanning system for hyperspectral image acquisition is described. Commercially available components have been carefully selected in order to be integrated into a flexible infrastructure able to obtain data from any Generic Interface for Cameras (GenICam) compliant devices using the gigabyte Ethernet interface. The entire setup has been tested using the Specim FX hyperspectral series (FX10 and FX17) and a Graphical User Interface (GUI) has been developed in order to control the individual components and visualise data. Morphological analysis, spectral response and optical aberration of these pushbroom-type hyperspectral cameras have been evaluated prior to the validation of the whole system with different plastic samples for which spectral signatures are extracted and compared with well-known spectral libraries.

This paper shows new contributions in the detection of skin cancer, where we present the use of a customized hyperspectral system that captures images in the spectral range from 450 to 950 nm. By choosing a 7 × 7 sub-image of each channel in the hyperspectral image (HSI) and then taking the mean and standard deviation of these sub-images, we were able to make fits of the resulting curves. These fitted curves had certain characteristics, which then served as a basis of classification. The most distinct fit was for the melanoma pigmented skin lesions (PSLs), which is also the most aggressive malignant cancer. Furthermore, we were able to classify the other PSLs in malignant and benign classes. This gives us a rather complete classification method for PSLs with a novel perspective of the classification procedure by exploiting the variability of each channel in the HSI.

Hyperspectral imaging is a multidimensional optical technique with the potential of providing fast and accurate tissue classification. The main challenge is the adequate processing of the multidimensional information usually linked to long processing times and significant computational costs, which require expensive hardware. In this study, we address the problem of tissue classification for intraoperative hyperspectral images of in vivo brain tissue. For this goal, two methodologies are introduced that rely on a blind linear unmixing (BLU) scheme for practical tissue classification. Both methodologies identify the characteristic end-members related to the studied tissue classes by BLU from a training dataset and classify the pixels by a minimum distance approach. The proposed methodologies are compared with a machine learning method based on a supervised support vector machine (SVM) classifier. The methodologies based on BLU achieve speedup factors of ~459× and ~429× compared to the SVM scheme, while keeping constant and even slightly improving the classification performance.

Given an object of interest that evolves in time, one often wants to detect possible changes in its properties. The first changes may be small and occur in different scales and it may be crucial to detect them as early as possible. Examples include identification of potentially malignant changes in skin moles or the gradual onset of food quality deterioration. Statistical scale-space methodologies can be very useful in such situations since exploring the measurements in multiple resolutions can help identify even subtle changes. We extend a recently proposed scale-space methodology to a technique that successfully detects such small changes and at the same time keeps false alarms at a very low level. The potential of the novel methodology is first demonstrated with hyperspectral skin mole data artificially distorted to include a very small change. Our real data application considers hyperspectral images used for food quality detection. In these experiments the performance of the proposed method is either superior or on par with a standard approach such as principal component analysis.

The main goal of brain cancer surgery is to perform an accurate resection of the tumor, preserving as much normal brain tissue as possible for the patient. The development of a non-contact and label-free method to provide reliable support for tumor resection in real-time during neurosurgical procedures is a current clinical need. Hyperspectral imaging is a non-contact, non-ionizing, and label-free imaging modality that can assist surgeons during this challenging task without using any contrast agent. In this work, we present a deep learning-based framework for processing hyperspectral images of in vivo human brain tissue. The proposed framework was evaluated by our human image database, which includes 26 in vivo hyperspectral cubes from 16 different patients, among which 258,810 pixels were labeled. The proposed framework is able to generate a thematic map where the parenchymal area of the brain is delineated and the location of the tumor is identified, providing guidance to the operating surgeon for a successful and precise tumor resection. The deep learning pipeline achieves an overall accuracy of 80% for multiclass classification, improving the results obtained with traditional support vector machine (SVM)-based approaches. In addition, an aid visualization system is presented, where the final thematic map can be adjusted by the operating surgeon to find the optimal classification threshold for the current situation during the surgical procedure.

The work presented in this paper is focused on the use of spectroscopy to identify the type of tissue of human brain samples employing support vector machine classifiers. Two different spectrometers were used to acquire infrared spectroscopic signatures in the wavenumber range between 1200⁻3500 cm−1. An extensive analysis was performed to find the optimal configuration for a support vector machine classifier and determine the most relevant regions of the spectra for this particular application. The results demonstrate that the developed algorithm is robust enough to classify the infrared spectroscopic data of human brain tissue at three different discrimination levels.

Super-resolution is a smart process capable of generating images with a higher resolution than the resolution of the sensor used to acquire the images. Due to this reason, it has acquired a significant relevance within the medical community during the last years, especially for those specialties closely related with the medical imaging field. However, the super-resolution algorithms used in this field are normally extremely complex and thus, they tend to be slow and difficult to be implemented in hardware. This paper proposes a new super-resolution algorithm for video sequences that, while maintaining excellent levels in the objective and subjective visual quality of the processed images, presents a reduced computational cost due to its non-iterative nature and the use of fast motion estimation techniques. Additionally, the algorithm has been successfully implemented in a low-cost hardware platform, which guarantees the viability of the proposed solution for real-time biomedical systems-on-chip.