Search

In this contribution it is coined and presented a new concept: "Precision Archaeology'" (PA). PA is a concept based on observing, measuring and responding to inter and intra-field variability in land. PA aims at defining a decision support system for archaeology management for optimizing returns on inputs while preserving resources.

The aim of our research is to artificially generate fake historical manuscripts using GAN. At this stage, these experiments pursue two objectives: evaluating feasibility of generating realistic fake manuscripts under certain conditions of layout, script, or date, and creating artificial data to augment HTR training. Examples are taken from Classical Armenian and Old French.

Artificial Intelligence (AI) technologies such as Deep Learning and Transfer Learning are rapidly advancing text and image classification in many disciplines (e.g., Lucas et al. 2022, Huang et al. 2022, Kumar et al. 2020). With the progressing digitisation of cultural heritage, AI increasingly finds applications in arts and humanities disciplines as well (Cetinic et al. 2019, Saleh / Elgammal 2016). It is evident that AI technologies open new possibilities for processing and analysing large, heterogeneous historical data corpora in a semi-automated way (Im et al. 2022). But do they have the potential to “overturn intellectual legacies” and to revolutionise the field? In the project Ottoman Nature in Travelogues (ONiT 2023) we strive to leverage the application of AI technologies to historical research by developing an interdisciplinary methodological framework for the semi-automated, AI-driven analysis of text–image relations. Our object of study is a large multilingual corpus of travelogues printed 1501–1850 (Rörden et al. 2020) that contains representations of Ottoman “nature” (i.e., flora, fauna, landscapes) both in text and image. This short presentation focuses on our first results achieved for improving image classification of historical prints (i.e., mostly woodcuts, engravings, and etchings). We present our approach to robustly identify and classify key image types in our corpus and discuss challenges and lessons learned during this process. &nbsp

Traditionally, two approaches have developed in dialectology that focus on researching lexical variation: on the one hand, dialect dictionaries, whose task is to document dialect vocabulary; on the other hand, dialect atlases, whose focus is on linguistic-geographical variation in dialect vocabulary. The Wörterbuch der bairischen Mundarten in Österreich (WBÖ), a long-term project of the Austrian Academy of Sciences (ÖAW), is an undertaking of the first type. Until 2015, the first five volumes (A–Ezzes) were published as printed works; since 2018, the Lexikalisches Informationssystem Österreich/Lexical Information System Austria (LIÖ) has served as the publication platform for the articles starting with the letter F. LIÖ is a cooperation project between the Austrian Centre for Digital Humanities and Cultural Heritage (ACDH-CH) of the ÖAW and the FWF Special Research Programme German in Austria (DiÖ) of the University of Vienna. At the moment, the information system only contains content related to the WBÖ project itself. As a lexically oriented platform, however, its content is to be expanded in the coming years, i.e. lexical material from other corpora is also to be made accessible via it. A first expansion of LIÖ will take place in October 2022 with the pilot study DWA Austria, a cooperation project between the Research Center Deutscher Sprachatlas (DSA) of the Philipps University of Marburg and the Department of Linguistics of the ACDH-CH. Within the framework of this cooperation, the entire Austrian surveys of the Deutscher Wortatlas/German Word Atlas (DWA) are to be digitally processed for the first time. Last but not least, the project will serve to expand the paradigm for researching lexical variation described above to include a dialect-geographical component. In this collaboration, the Marburg team will provide the high-resolution scans of the DWA surveys. The team in Vienna is building a model for automatic transliteration on this basis with the help of the Transkribus software, which in turn can be used by the DSA team for the German DWA sheets. The surveys for the DWA were conducted indirectly between 1939 and 1942 and are still among the most comprehensive surveys of the 20th century. Questionnaires were sent to a total of about 50,000 places, 3,700 of which are in the territory of the Republic of Austria. Since previous automatic text recognition methods (i.e. OCR) have only provided insufficient results, the questionnaires, which were mostly handwritten, had to be laboriously transliterated manually. The recent use of artificial intelligence (AI) has been showing promising results for several years. With the help of the Transkribus platform, the Austrian DWA questionnaires are now being captured and made usable as part of a pilot study. Unlike conventional OCR products, Transkribus uses artificial intelligence (AI) to convert the written content of digital records into searchable text. The scans of the DWA sheets were made by the DSA and made available to the ACDH-CH. There, in a first step, they manually transliterate a set of scanned sheets. This step can be supported by already existing models that are, for example, tailored to German Kurrent script. Based on these correct transliterations and the corresponding scans, a model can now be built with the help of Deep Learning that is tailored to the document type. The layout of the text is also taken into account. First models on the minimum amount of training material still showed a rather high error rate (CER Val. 9.4) and have been continuously improved since then. The pilot study DWA Austria shows how data sets that could previously only be used to a limited extent due to time-consuming and costly efforts can now be opened up by AI-supported methods to an extent that would not have been possible with conventional methods. In particular, the example shows how the respective expertise of the individual project partners can bring about a significant increase in efficiency and thus once again illustrates the potential for synergies that result from cooperation. Bibliography: DiÖ = SFB German in Austria. URL: https://www.dioe.at/en/ [2023-05-01] DSA = Forschungszentrum Deutscher Sprachatlas. URL: https://www.uni-marburg.de/en/fb09/dsa [2023-05-01] DWA = Mitzka, Walther & Ludwig Erich Schmitt. 1951 – 1980. Deutscher Wortatlas. Gießen: Schmitz LiÖ = Lexikalisches Informationssystem Österreich. URL: https://lioe.dioe.at/ [2023-05-01] Transkribus. AI powered Handwritten Text Recognition. URL: https://readcoop.eu/transkribus/ [2023-05-01] WBÖ = Österreichischen Akademie der Wissenschaften. 1970 – 2015. Wörterbuch der bairischen Mundarten in Österreich. Wien: Verlag der Österreichischen Akademie der Wissenschaften.

This poster presents the preliminary results from the first author's PhD research project, consisting of the early steps towards customising Computer Vision (CV) to work with features characterising historical photographs from Ireland.

The use of Artificial Intelligence techniques for the automatic recognition of architectural elements from point clouds can provide valuable support to facilitate preservation operations, and support many other activities related to Digital Cultural Heritage domain.

Background We examined whether a context and process-sensitive ‘intelligent' checklist increases compliance with best practice compared with a paper checklist during intensive care ward rounds. Methods We conducted a single-centre prospective before-and-after mixed-method trial in a 35 bed medical and surgical ICU. Daily ICU ward rounds were observed during two periods of 8 weeks. We compared paper checklists (control) with a dynamic (digital) clinical checklist (DCC, intervention). The primary outcome was compliance with best clinical practice, measured as the percentages of checked items and unchecked critical items. Secondary outcomes included ICU stay and the usability of digital checklists. Data are presented as median (interquartile range). Results Clinical characteristics and severity of critical illness were similar during both control and intervention periods of study. A total of 36 clinicians visited 197 patients during 352 ward rounds using the paper checklist, compared with 211 patients during 366 ward rounds using the DCC. Per ICU round, a median of 100% of items (94.4–100.0) were completed by DCC, compared with 75.1% (66.7–86.4) by paper checklist (P=0.03). No critical items remained unchecked by the DCC, compared with 15.4% (8.3–27.3) by the paper checklist (P=0.01). The DCC was associated with reduced ICU stay (1 day [1–3]), compared with the paper checklist (2 days [1–4]; P=0.05). Usability of the DCC was judged by clinicians to require further improvement. Conclusions A digital checklist improved compliance with best clinical practice, compared with a paper checklist, during ward rounds on a mixed ICU. Clinical trial registration NCT 03599856.

In this study, we aim to compare the performance of systems and forensic facial comparison experts in terms of likelihood ratio computation to assess the potential of the machine to support the human expert in the courtroom. In forensics, transparency in the methods is essential. Consequently, state‐of‐the‐art free software was preferred over commercial software. Three different open‐source automated systems chosen for their availability and clarity were as follows: OpenFace, SeetaFace, and FaceNet; all three based on convolutional neural networks that return a distance (OpenFace, FaceNet) or similarity (SeetaFace). The returned distance or similarity is converted to a likelihood ratio using three different distribution fits: parametric fit Weibull distribution, nonparametric fit kernel density estimation, and isotonic regression with pool adjacent violators algorithm. The results show that with low‐quality frontal images, automated systems have better performance to detect nonmatches than investigators: 100% of precision and specificity in confusion matrix against 89% and 86% obtained by investigators, but with good quality images forensic experts have better results. The rank correlation between investigators and software is around 80%. We conclude that the software can assist in reporting officers as it can do faster and more reliable comparisons with full‐frontal images, which can help the forensic expert in casework.

Significance: Hyperspectral imaging (HSI) has emerged as a promising optical technique. Besides optical properties of a sample, other sample physical properties also affect the recorded images. They are significantly affected by the sample curvature and sample surface to camera distance. A correction method to reduce the artifacts is necessary to reliably extract sample properties. Aim: Our aim is to correct hyperspectral images using the three-dimensional (3D) surface data and assess how the correction affects the extracted sample properties. Approach: We propose the combination of HSI and 3D profilometry to correct the images using the Lambert cosine law. The feasibility of the correction method is presented first on hemispherical tissue phantoms and next on human hands before, during, and after the vascular occlusion test (VOT). Results: Seven different phantoms with known optical properties were created and imaged with a hyperspectral system. The correction method worked up to 60 deg inclination angle, whereas for uncorrected images the maximum angles were 20 deg. Imaging hands before, during, and after VOT shows good agreement between the expected and extracted skin physiological parameters. Conclusions: The correction method was successfully applied on the images of tissue phantoms of known optical properties and geometry and VOT. The proposed method could be applied to any reflectance optical imaging technique and should be used whenever the sample parameters need to be extracted from a curved surface sample.

Significance: Polyp size is important for selecting the surveillance interval or treatment policy. Nevertheless, it is challenging to accurately estimate the polyp size during endoscopy. An easy and cost-effective function to assist in polyp size estimation is required. Aim: To propose a virtual scale function for endoscopy and evaluate its performance and expected accuracy. Approach: An adaptive virtual scale behavior was demonstrated. The measurement error of the virtual scale along the distance between the tip of the endoscope and the object plane was evaluated using graph paper. The accuracy of polyp size estimation by an expert endoscopist was compared with the accuracy of the biopsy forceps method using phantom images. Results: The measurement errors of the virtual scale were ≤ 0.7 mm when the distance to the graph paper, which faced the tip of the endoscope, varied from 4 to 30 mm. The accuracy with the virtual scale was significantly higher than that obtained with biopsy forceps (5.3 ± 5.5 % versus 11.9 ± 9.4 % , P

Significance: Screening and early detection of oral potentially malignant lesions (OPMLs) are of great significance in reducing the mortality rates associated with head and neck malignancies. Intra-oral multispectral optical imaging of tissues in conjunction with cloud-based machine learning (CBML) can be used to detect oral precancers at the point-of-care (POC) and guide the clinician to the most malignant site for biopsy. Aim: Develop a bimodal multispectral imaging system (BMIS) combining tissue autofluorescence and diffuse reflectance (DR) for mapping changes in oxygenated hemoglobin (HbO2) absorption in the oral mucosa, quantifying tissue abnormalities, and guiding biopsies. Approach: The hand-held widefield BMIS consisting of LEDs emitting at 405, 545, 575, and 610 nm, 5MPx monochrome camera, and proprietary Windows-based software was developed for image capture, processing, and analytics. The DR image ratio (R610/R545) was compared with pathologic classification to develop a CBML algorithm for real-time assessment of tissue status at the POC. Results: Sensitivity of 97.5% and specificity of 92.5% were achieved for discrimination of OPML from patient normal in 40 sites, whereas 82% sensitivity and 96.6% specificity were obtained for discrimination of abnormal (OPML + SCC) in 89 sites. Site-specific algorithms derived for buccal mucosa (27 sites) showed improved sensitivity and specificity of 96.3% for discrimination of OPML from normal. Conclusions: Assessment of oral cancer risk is possible by mapping of HbO2 absorption in tissues, and the BMIS system developed appears to be suitable for biopsy guidance and early detection of oral cancers.

Significance: Subretinal injection is an effective way of delivering transplant genes and cells to treat many degenerative retinal diseases. However, the technique requires high-dexterity and microscale precision of experienced surgeons, who have to overcome the physiological hand tremor and limited visualization of the subretinal space. Aim: To automatically guide the axial motion of microsurgical tools (i.e., a subretinal injector) with microscale precision in real time using a fiber-optic common-path swept-source optical coherence tomography distal sensor. Approach: We propose, implement, and study real-time retinal boundary tracking of A-scan optical coherence tomography (OCT) images using a convolutional neural network (CNN) for automatic depth targeting of a selected retinal boundary for accurate subretinal injection guidance. A simplified 1D U-net is used for the retinal layer segmentation on A-scan OCT images. A Kalman filter, combining retinal boundary position measurement by CNN and velocity measurement by cross correlation between consecutive A-scan images, is applied to optimally estimate the retinal boundary position. Unwanted axial motions of the surgical tools are compensated by a piezoelectric linear motor based on the retinal boundary tracking. Results: CNN-based segmentation on A-scan OCT images achieves the mean unsigned error (MUE) of ∼3 pixels (8.1 μm) using an ex vivo bovine retina model. GPU parallel computing allows real-time inference (∼2 ms) and thus real-time retinal boundary tracking. Involuntary tremors, which include low-frequency draft in hundreds of micrometers and physiological tremors in tens of micrometers, are compensated effectively. The standard deviations of photoreceptor (PR) and choroid (CH) boundary positions get as low as 10.8 μm when the depth targeting is activated. Conclusions: A CNN-based common-path OCT distal sensor successfully tracks retinal boundaries, especially the PR/CH boundary for subretinal injection, and automatically guides the tooltip’s axial position in real time. The microscale depth targeting accuracy of our system shows its promising possibility for clinical application.

Significance: Designing optode layouts is an essential step for functional near-infrared spectroscopy (fNIRS) experiments as the quality of the measured signal and the sensitivity to cortical regions-of-interest depend on how optodes are arranged on the scalp. This becomes particularly relevant for fNIRS-based brain–computer interfaces (BCIs), where developing robust systems with few optodes is crucial for clinical applications. Aim: Available resources often dictate the approach researchers use for optode-layout design. We investigated whether guiding optode layout design using different amounts of subject-specific magnetic resonance imaging (MRI) data affects the fNIRS signal quality and sensitivity to brain activation when healthy participants perform mental-imagery tasks typically used in fNIRS-BCI experiments. Approach: We compared four approaches that incrementally incorporated subject-specific MRI information while participants performed mental-calculation, mental-rotation, and inner-speech tasks. The literature-based approach (LIT) used a literature review to guide the optode layout design. The probabilistic approach (PROB) employed individual anatomical data and probabilistic maps of functional MRI (fMRI)-activation from an independent dataset. The individual fMRI (iFMRI) approach used individual anatomical and fMRI data, and the fourth approach used individual anatomical, functional, and vascular information of the same subject (fVASC). Results: The four approaches resulted in different optode layouts and the more informed approaches outperformed the minimally informed approach (LIT) in terms of signal quality and sensitivity. Further, PROB, iFMRI, and fVASC approaches resulted in a similar outcome. Conclusions: We conclude that additional individual MRI data lead to a better outcome, but that not all the modalities tested here are required to achieve a robust setup. Finally, we give preliminary advice to efficiently using resources for developing robust optode layouts for BCI and neurofeedback applications.

Purpose: Coronavirus disease 2019 (COVID-19) is a new infection that has spread worldwide and with no automatic model to reliably detect its presence from images. We aim to investigate the potential of deep transfer learning to predict COVID-19 infection using chest computed tomography (CT) and x-ray images. Approach: Regions of interest (ROI) corresponding to ground-glass opacities (GGO), consolidations, and pleural effusions were labeled in 100 axial lung CT images from 60 COVID-19-infected subjects. These segmented regions were then employed as an additional input to six deep convolutional neural network (CNN) architectures (AlexNet, DenseNet, GoogleNet, NASNet-Mobile, ResNet18, and DarkNet), pretrained on natural images, to differentiate between COVID-19 and normal CT images. We also explored the model’s ability to classify x-ray images as COVID-19, non-COVID-19 pneumonia, or normal. Performance on test images was measured with global accuracy and area under the receiver operating characteristic curve (AUC). Results: When using raw CT images as input to the tested models, the highest accuracy of 82% and AUC of 88.16% is achieved. Incorporating the three ROIs as an additional model inputs further boosts performance to an accuracy of 82.30% and an AUC of 90.10% (DarkNet). For x-ray images, we obtained an outstanding AUC of 97% for classifying COVID-19 versus normal versus other. Combing chest CT and x-ray images, DarkNet architecture achieves the highest accuracy of 99.09% and AUC of 99.89% in classifying COVID-19 from non-COVID-19. Our results confirm the ability of deep CNNs with transfer learning to predict COVID-19 in both chest CT and x-ray images. Conclusions: The proposed method could help radiologists increase the accuracy of their diagnosis and increase efficiency in COVID-19 management.

Significance: Digital holographic microscopy is widely used to get the quantitative phase information of transparent cells. Aim: However, the sample phase is superimposed with aberrations. To quantify the phase information, aberrations need to be fully compensated. Approach: We propose a technique to obtain aberration-free phase imaging, using the derivative-based principal component analysis (dPCA). Results: With dPCA, almost all aberrations can be extracted and compensated without requirements on background segmentation, making it efficient and convenient. Conclusions: It solves the problem that the conventional principal component analysis (PCA) algorithm cannot compensate the common but intricate higher order cross-term aberrations, such as astigmatism and coma. Moreover, the dPCA strategy proposed here is not only suitable for aberration compensation but also applicable for other cases where there exist cross-terms that cannot be analyzed with the PCA algorithm.

Significance: Physiological parameters extracted from diffuse reflectance spectroscopy (DRS) provide clinicians quantitative information about tissue that helps aid in diagnosis. There is a great need for an accurate and cost-effective method for extracting parameters from DRS measurements. Aim: The aim is to explore the accuracy and speed of physiological parameter extraction using machine learning models compared to that of the widely used Monte Carlo lookup table (MCLUT) inverse model. Approach: Diffuse reflectance spectra were simulated using a light transport model based on Monte Carlo simulations and weighted to six wavelengths. Deep learning (DL), random forest (RF), gradient boosting machine (GBM), and generalized linear model (GLM) machine learning models were built using a training set of 10,000 spectra from the simulated data. The MCLUT and machine learning models were used to predict physiological parameters from a separate test set of 30,000 simulated spectra. Mean absolute errors were calculated to evaluate the accuracy and compare it among MCLUT and machine learning models. In addition, the computational time to predict parameters from the test set was recorded to compare the speed among MCLUT and machine learning models. Results: The DL, RF, GBM, and GLM models all had significantly lower errors than the MCLUT inverse method for six wavelengths. The DL model proved to have the lowest errors, with all absolute percent errors under 10%. The DL model had much faster runtimes than the MCLUT. Conclusions: Machine learning is promising for extracting physiological parameters from six-wavelength DRS data, with both lower errors and a faster runtime than the widely used MCLUT model.

We introduce and explore an approach for constructing force fields for small molecules, which combines intuitive low body order empirical force field terms with the concepts of data driven statistical fits of recent machine learned potentials. We bring these two key ideas together to bridge the gap between established empirical force fields that have a high degree of transferability on the one hand, and the machine learned potentials that are systematically improvable and can converge to very high accuracy, on the other. Our framework extends the atomic permutationally invariant polynomials (aPIP) developed for elemental materials in (2019 Mach. Learn.: Sci. Technol. 1 015004) to molecular systems. The body order decomposition allows us to keep the dimensionality of each term low, while the use of an iterative fitting scheme as well as regularisation procedures improve the extrapolation outside the training set. We investigate aPIP force fields with up to generalised 4-body terms, and examine the performance on a set of small organic molecules. We achieve a high level of accuracy when fitting individual molecules, comparable to those of the many-body machine learned force fields. Fitted to a combined training set of short linear alkanes, the accuracy of the aPIP force field still significantly exceeds what can be expected from classical empirical force fields, while retaining reasonable transferability to both configurations far from the training set and to new molecules.

Significance: Non-contact, camera-based heart rate variability estimation is desirable in numerous applications, including medical, automotive, and entertainment. Unfortunately, camera-based HRV accuracy and reliability suffer due to two challenges: (a) darker skin tones result in lower SNR and (b) relative motion induces measurement artifacts. Aim: We propose an algorithm HRVCam that provides sufficient robustness to low SNR and motion-induced artifacts commonly present in imaging photoplethysmography (iPPG) signals. Approach: HRVCam computes camera-based HRV from the instantaneous frequency of the iPPG signal. HRVCam uses automatic adaptive bandwidth filtering along with discrete energy separation to estimate the instantaneous frequency. The parameters of HRVCam use the observed characteristics of HRV and iPPG signals. Results: We capture a new dataset containing 16 participants with diverse skin tones. We demonstrate that HRVCam reduces the error in camera-based HRV metrics significantly (more than 50% reduction) for videos with dark skin and face motion. Conclusion: HRVCam can be used on top of iPPG estimation algorithms to provide robust HRV measurements making camera-based HRV practical.

Significance: With the increasing popularity of functional near-infrared spectroscopy (fNIRS), the need to determine localization of the source and nature of the signals has grown. Aim: We compare strategies for removal of non-neural signals for a finger-thumb tapping task, which shows responses in contralateral motor cortex and a visual checkerboard viewing task that produces activity within the occipital lobe. Approach: We compare temporal regression strategies using short-channel separation to a spatial principal component (PC) filter that removes global signals present in all channels. For short-channel temporal regression, we compare non-neural signal removal using first and combined first and second PCs from a broad distribution of short channels to limited distribution on the forehead. Results: Temporal regression of non-neural information from broadly distributed short channels did not differ from forehead-only distribution. Spatial PC filtering provides results similar to short-channel separation using the temporal domain. Utilizing both first and second PCs from short channels removes additional non-neural information. Conclusions: We conclude that short-channel information in the temporal domain and spatial domain regression filtering methods remove similar non-neural components represented in scalp hemodynamics from fNIRS signals and that either technique is sufficient to remove non-neural components.

Summarizing video content is an important task in many applications. This task can be defined as the computation of the ordered list of actions present in a video. Such a list could be extracted using action detection algorithms. However, it is not necessary to determine the temporal boundaries of actions to know their existence. Moreover, localizing precise boundaries usually requires dense video analysis to be effective. In this work, we propose to directly compute this ordered list by sparsely browsing the video and selecting one frame per action instance, task known as action spotting in literature. To do this, we propose ActionSpotter, a spotting algorithm that takes advantage of Deep Reinforcement Learning to efficiently spot actions while adapting its video browsing speed, without additional supervision. Experiments performed on datasets THUMOS14 and ActivityNet show that our framework outperforms state of the art detection methods. In particular, the spotting mean Average Precision on THUMOS14 is significantly improved from 59.7% to 65.6% while skipping 23% of video.

Purpose: Image registration is an important aspect of medical image analysis and a key component in many analysis concepts. Applications include fusion of multimodal images, multi-atlas segmentation, and whole-body analysis. Deformable image registration is often computationally expensive, and the need for efficient registration methods is highlighted by the emergence of large-scale image databases, e.g., the UK Biobank, providing imaging from 100,000 participants. Approach: We present a heterogeneous computing approach, utilizing both the CPU and the graphics processing unit (GPU), to accelerate a previously proposed image registration method. The parallelizable task of computing the matching criterion is offloaded to the GPU, where it can be computed efficiently, while the more complex optimization task is performed on the CPU. To lessen the impact of data synchronization between the CPU and GPU, we propose a pipeline model, effectively overlapping computational tasks with data synchronization. The performance is evaluated on a brain labeling task and compared with a CPU implementation of the same method and the popular advanced normalization tools (ANTs) software. Results: The proposed method presents a speed-up by factors of 4 and 8 against the CPU implementation and the ANTs software, respectively. A significant improvement in labeling quality was also observed, with measured mean Dice overlaps of 0.712 and 0.701 for our method and ANTs, respectively. Conclusions: We showed that the proposed method compares favorably to the ANTs software yielding both a significant speed-up and an improvement in labeling quality. The registration method together with the proposed parallelization strategy is implemented as an open-source software package, deform.

Significance: Functional near-infrared spectroscopy (fNIRS) has been widely used to probe human brain function during task state and resting state. However, the existing analysis toolboxes mainly focus on task activation analysis, few software packages can assist resting-state fNIRS studies. Aim: We aimed to provide a versatile and easy-to-use toolbox to perform analysis for both resting state and task fNIRS. Approach: We developed a MATLAB toolbox called NIRS-KIT that works for both resting-state analysis and task activation detection. Results: NIRS-KIT implements common and necessary processing steps for performing fNIRS data analysis, including data preparation, quality control, preprocessing, individual-level analysis, group-level statistics with several popular statistical models, and multiple comparison correction methods, and finally results visualization. For resting-state fNIRS analysis, functional connectivity analysis, graph theory-based network analysis, and amplitude of low-frequency fluctuations analysis are provided. Additionally, NIRS-KIT also supports activation analysis for task fNIRS. Conclusions: NIRS-KIT offers an open source tool for researchers to analyze resting-state and/or task fNIRS data in one suite. It contains several key features: (1) good compatibility, supporting multiple fNIRS recording systems, data formats of NIRS-SPM and Homer2, and the shared near-infrared spectroscopy format data format recommended by the fNIRS society; (2) flexibility, supporting customized preprocessing scripts; (3) ease-to-use, allowing processing fNIRS signals in batch manner with user-friendly graphical user interfaces; and (4) feature-packed data viewing and result visualization. We anticipate that this NIRS-KIT will facilitate the development of the fNIRS field.

Automatic segmentation of bones in computed tomography (CT) images is used for instance in beam hardening correction algorithms where it improves the accuracy of resulting CT numbers. Of special interest are pelvic bones, which—because of their strong attenuation—affect the accuracy of brachytherapy in this region. This work evaluated the performance of the JJ2016 algorithm with the performance of MK2014v2 and JS2018 algorithms; all these algorithms were developed by authors. Visual comparison, and, in the latter case, also Dice similarity coefficients derived from the ground truth were used. It was found that the 3D-based JJ2016 performed better than the 2D-based MK2014v2, mainly because of the more accurate hole filling that benefitted from information in adjacent slices. The neural network-based JS2018 outperformed both traditional algorithms. It was, however, limited to the resolution of 1283 owing to the limited amount of memory in the graphical processing unit (GPU). Funding: VetenskapsradetSwedish Research Council [VR-NT 2016-05033]

Photoacoustic tomography (PAT) is an imaging technique that enables us to visualize vasculature networks at high resolution at depths much greater than optical microscopy.1,2 PAT can also resolve tissue chromophores such as oxyhemoglobin and deoxyhemoglobin, which allows estimation of tissue oxygentation levels.1,3 Accurate estimation of tissue oxygentation is very important in the fields of neuroimaging, cancer diagnostics, and critical care medicine.4,5 PAT involves collection of broadband acoustic waves generated by thermoelastic expansion of tissue upon absorption of light (illuminated at near-infrared wavelengths).6,7 PAT image reconstruction involves recovery of the acoustic source distribution from the detected broadband acoustic waves, which can be performed using analytical or model-based methods.8 Analytical methods are computationally less expensive compared to model-based image reconstruction.8,9 In contrary, model-based schemes are known to generate accurate images compared to analytical schemes, specifically in scenarios where irregular acquisition geometry is involved or limited data is acquired.9,10 Even with the model-based schemes, PAT inversion becomes difficult when the number of ultrasound detector positions is limited, and the situation gets aggravated due to the presence of noise in the data. More recently deep learning-based image postprocessing schemes have been developed for artifact removal and for limited data situations in PAT/microscopy;11–13 however, these schemes were trained on simulated data and generating large experimental data is very difficult. Segmentation approaches were proposed to delineate skin lining in mesoscopic photoacoustic images.14 Active contours-based image segmentation was used to obtain the PAT region of interest (ROI), the obtained ROI was then used to perform fluence correction to make PAT more quantitative.15,16 Different computational techniques have been reported in the literature to obtain vascular network in photoacoustic imaging. Vesselness filter has been proposed to enable accurate recovery of vasculature in PAT;17–19 however, questions have been raised on the accuracy of using the vesselness filter (due to generation of artificial vasculature).20 Probabilistic approach involving different steps such as smoothing and filtering the data, clustering, vessel-segmentation using clusters, and morphological filling was developed for segmenting vessels in photoacoustic images.21 More recently convolution neural network-based approaches were also developed for jointly performing the image reconstruction and segmentation.22 Further, deep learning-based approach was proposed for segmenting animal boundary using multi-modal photoacoustic and ultrasound data.23 Further, deep learning-based method has also been developed for animal brain imaging.24 Notably segmenting vasculature is very important in the context of diagnosis and surgical guidance, specifically for cardiovascular diseases.25,26 In this work, we developed a binary tomography approach wherein we assume the reconstruction distribution contains only two unknowns namely the background and the absorbers. The major contributions of this work include: (a) application of binary tomographic methods for photoacoustic tomographic imaging involving spherical wave propagation as opposed to straight line approximation applied in x-ray computed tomography; (b) showing the utility of binary tomographic methods for improving photoacoustic tomographic imaging of vasculature; and (c) establishing the binary tomographic method superiority in photoacoustic tomographic imaging for providing approximate priors to improve model-based image reconstruction. Binary tomography problem is solved based on a dual optimization approach with an auxiliary variable posed as a constrained optimization problem. We compared the binary tomography algorithm against standard backprojection approach and model-based reconstruction schemes such as Tikhonov regularization and sparsity-based regularization.27 The performance of these different algorithms was validated with in-silico data, physical phantom, and in-vivo brain vasculature imaging data.

Significance: Label-free quantitative phase imaging is a promising technique for the automatic detection of abnormal red blood cells (RBCs) in real time. Although deep-learning techniques can accurately detect abnormal RBCs from quantitative phase images efficiently, their applications in diagnostic testing are limited by the lack of transparency. More interpretable results such as morphological and biochemical characteristics of individual RBCs are highly desirable. Aim: An end-to-end deep-learning model was developed to efficiently discriminate thalassemic RBCs (tRBCs) from healthy RBCs (hRBCs) in quantitative phase images and segment RBCs for single-cell characterization. Approach: Two-dimensional quantitative phase images of hRBCs and tRBCs were acquired using digital holographic microscopy. A mask region-based convolutional neural network (Mask R-CNN) model was trained to discriminate tRBCs and segment individual RBCs. Characterization of tRBCs was achieved utilizing SHapley Additive exPlanation analysis and canonical correlation analysis on automatically segmented RBC phase images. Results: The implemented model achieved 97.8% accuracy in detecting tRBCs. Phase-shift statistics showed the highest influence on the correct classification of tRBCs. Associations between the phase-shift features and three-dimensional morphological features were revealed. Conclusions: The implemented Mask R-CNN model accurately identified tRBCs and segmented RBCs to provide single-RBC characterization, which has the potential to aid clinical decision-making.

Reducing the bit-depth is an effective approach to lower the cost of optical coherence tomography (OCT) systems and increase the transmission efficiency in data acquisition and telemedicine. However, a low bit-depth will lead to the degeneration of the detection sensitivity thus reduce the signal-to-noise ratio (SNR) of OCT images. In this paper, we propose to use deep learning for the reconstruction of the high SNR OCT images from the low bit-depth acquisition. Its feasibility was preliminarily evaluated by applying the proposed method to the quantized $3\sim8$-bit data from native 12-bit interference fringes. We employed a pixel-to-pixel generative adversarial network architecture in the low to high bit-depth OCT image transition. Retinal OCT data of a healthy subject from a homemade spectral-domain OCT system was used in the study. Extensively qualitative and quantitative results show this deep-learning-based approach could significantly improve the SNR of the low bit-depth OCT images especially at the choroidal region. Superior similarity and SNR between the reconstructed images and the original 12-bit OCT images could be derived when the bit-depth $\geq 5$. This work demonstrates the proper integration of OCT and deep learning could benefit the development of healthcare in low-resource settings., Comment: submitted

Purpose: Hippocampus contouring for radiotherapy planning is performed on MR image data due to poor anatomical visibility on computed tomography (CT) data. Deep learning methods for direct CT hippocampus auto-segmentation exist, but use MR-based training contours. We investigate if these can be replaced by CT-based contours without loss in segmentation performance. This would remove the MR not only from inference but also from training. Approach: The hippocampus was contoured by medical experts on MR and CT data of 45 patients. Convolutional neural networks (CNNs) for hippocampus segmentation on CT were trained on CT-based or propagated MR-based contours. In both cases, their predictions were evaluated against the MR-based contours considered as the ground truth. Performance was measured using several metrics, including Dice score, surface distances, and contour Dice score. Bayesian dropout was used to estimate model uncertainty. Results: CNNs trained on propagated MR contours (median Dice 0.67) significantly outperform those trained on CT contours (0.59) and also experts contouring manually on CT (0.59). Differences between the latter two are not significant. Training on MR contours results in lower model uncertainty than training on CT contours. All contouring methods (manual or CNN) on CT perform significantly worse than a CNN segmenting the hippocampus directly on MR (median Dice 0.76). Additional data augmentation by rigid transformations improves the quantitative results but the difference remains significant. Conclusions: CT-based training contours for CT hippocampus segmentation cannot replace propagated MR-based contours without significant loss in performance. However, if MR-based contours are used, the resulting segmentations outperform experts in contouring the hippocampus on CT.

Purpose: The goal of this research is to develop innovative methods of acquiring simultaneous multidimensional molecular images of several different physiological random processes (PRPs) that might all be active in a particular disease such as COVID-19. Approach: Our study is part of an ongoing effort at the University of Arizona to derive biologically accurate yet mathematically tractable models of the objects of interest in molecular imaging and of the images they produce. In both cases, the models are fully stochastic, in the sense that they provide ways to estimate any estimable property of the object or image. The mathematical tool we use for images is the characteristic function, which can be calculated if the multivariate probability density function for the image data is known. For objects, which are functions of continuous variables rather than discrete pixels or voxels, the characteristic function becomes infinite dimensional, and we refer to it as the characteristic functional. Results: Several innovative mathematical results are derived, in particular for simultaneous imaging of multiple PRPs. Then the application of these methods to cancers that disrupt the mammalian target of rapamycin signaling pathway and to COVID-19 are discussed qualitatively. One reason for choosing these two problems is that they both involve lipid rafts. Conclusions: We found that it was necessary to employ a new algorithm for energy estimation to do simultaneous single-photon emission computerized tomography imaging of a large number of different tracers. With this caveat, however, we expect to be able to acquire and analyze an unprecedented amount of molecular imaging data for an individual COVID patient.

Significance: Melanoma is a deadly cancer that physicians struggle to diagnose early because they lack the knowledge to differentiate benign from malignant lesions. Deep machine learning approaches to image analysis offer promise but lack the transparency to be widely adopted as stand-alone diagnostics. Aim: We aimed to create a transparent machine learning technology (i.e., not deep learning) to discriminate melanomas from nevi in dermoscopy images and an interface for sensory cue integration. Approach: Imaging biomarker cues (IBCs) fed ensemble machine learning classifier (Eclass) training while raw images fed deep learning classifier training. We compared the areas under the diagnostic receiver operator curves. Results: Our interpretable machine learning algorithm outperformed the leading deep-learning approach 75% of the time. The user interface displayed only the diagnostic imaging biomarkers as IBCs. Conclusions: From a translational perspective, Eclass is better than convolutional machine learning diagnosis in that physicians can embrace it faster than black box outputs. Imaging biomarkers cues may be used during sensory cue integration in clinical screening. Our method may be applied to other image-based diagnostic analyses, including pathology and radiology.

A previous paper published in this journal proposed a model for evaluating the location of fingermarks on two-dimensional items (de Ronde, van Aken, de Puit and de Poot (2019)). In this paper, we apply the proposed model to a dataset consisting of letters to test whether the activity of writing a letter can be distinguished from the alternative activity of reading a letter based on the location of the fingermarks on the letters. An experiment was conducted in which participants were asked to read a letter and write a letter as separate activities on A4- and A5-sized papers. The fingermarks on the letters were visualized, and the resulting images were transformed into grid representations. A binary classification model was used to classify the letters into the activities of reading and writing based on the location of the fingermarks in the grid representations. Furthermore, the limitations of the model were studied by testing the influence of the length of the letter, the right- or left-handedness of the donor and the size of the paper with an additional activity of folding the paper. The results show that the model can predict the activities of reading or writing a letter based on the fingermark locations on A4-sized letters of right-handed donors with 98 % accuracy. Additionally, the length of the written letter and the handedness of the donor did not influence the performance of the classification model. Changing the size of the letters and adding an activity of folding the paper after writing on it decreased the model's accuracy. Expanding the training set with part of this new set had a positive influence on the model's accuracy. The results demonstrate that the model proposed by de Ronde, van Aken, de Puit and de Poot (2019) can indeed be applied to other two-dimensional items on which the disputed activities would be expected to lead to different fingermark locations. Moreover, we show that the location of fingermarks on letters provides valuable information about the activity that is carried out.

Significance: Despite recent developments in microscopy, temporal aliasing can arise when imaging dynamic samples. Modern sampling frameworks, such as generalized sampling, mitigate aliasing but require measurement of temporally overlapping and potentially negative-valued inner products. Conventional cameras cannot collect these directly as they operate sequentially and are only sensitive to light intensity., Aim: We aim to mitigate aliasing in microscopy of dynamic monochrome samples by implementing generalized sampling via the use of a color camera and modulated color illumination., Approach: We solve the overlap problem by spectrally multiplexing the acquisitions and using (positive) B-spline segments as projection kernels. Reconstruction involves spectral unmixing and inverse filtering. We implemented this method using a color LED illuminator. We evaluated its performance by imaging a rotating grid and its applicability by imaging the beating zebrafish embryo heart in transmission and light-sheet microscopes., Results: Compared to stroboscopic imaging, our method mitigates aliasing with performance improving as the projection order increases. The approach can be implemented in conventional microscopes but is limited by the number of available LED colors and camera channels., Conclusions: Generalized sampling can be implemented via color modulation in microscopy to mitigate temporal aliasing. The simple hardware requirements could make it applicable to other optical imaging modalities. (C) The Authors.

Significance: Detection and characterization of coronary atherosclerotic plaques often need reviews of a large number of optical coherence tomography (OCT) imaging slices to make a clinical decision. However, it is a challenge to manually review all the slices and consider the interrelationship between adjacent slices. Approach: Inspired by the recent success of deep convolutional network on the classification of medical images, we proposed a ResNet-3D network for classification of coronary plaque calcification in OCT pullbacks. The ResNet-3D network was initialized with a trained ResNet-50 network and a three-dimensional convolution filter filled with zeros padding and non-zeros padding with a convolutional filter. To retrain ResNet-50, we used a dataset of ∼4860 OCT images, derived by 18 entire pullbacks from different patients. In addition, we investigated a two-phase training method to address the data imbalance. For an improved performance, we evaluated different input sizes for the ResNet-3D network, such as 3, 5, and 7 OCT slices. Furthermore, we integrated all ResNet-3D results by majority voting. Results: A comparative analysis proved the effectiveness of the proposed ResNet-3D networks against ResNet-2D network in the OCT dataset. The classification performance (F1-scores=94% for non-zeros padding and F1-score=96% for zeros padding) demonstrated the potential of convolutional neural networks (CNNs) in classifying plaque calcification. Conclusions: This work may provide a foundation for further work in extending the CNN to voxel segmentation, which may lead to a supportive diagnostic tool for assessment of coronary plaque vulnerability.

Significance: Functional near-infrared spectroscopy (fNIRS) uses surface-placed light sources and detectors to record underlying changes in the brain due to fluctuations in hemoglobin levels and oxygenation. Since these measurements are recorded from the surface of the scalp, the mapping from underlying regions-of-interest (ROIs) in the brain space to the fNIRS channel space measurements depends on the registration of the sensors, the anatomy of the head/brain, and the sensitivity of these diffuse measurements through the tissue. However, small displacements in the probe position can change the distribution of recorded brain activity across the fNIRS measurements. Aim: We propose an approach using either individual or atlas-based brain-space anatomical information to define ROI-based statistical hypotheses to test the null involvement of specific regions, which allows us to test the analogous ROI across subjects while adjusting for fNIRS probe placement and sensitivity differences due to head size variations without a localizer task. Approach: We use the optical forward model to project the underlying brain-space ROI into a tapered contrast vector, which defines the relative weighting of the fNIRS channels contributing to the ROI and allows us to test the null hypothesis of no brain activity in this region during a functional task. We demonstrate this method through simulation and compare the sensitivity-specificity of this approach to other conventional methods. Results: We examine the performance of this method in the scenario where head size and probe registration are both an accurately known parameters and where this is subject to unknown experimental errors. This method is compared with the performance of the conventional method using 364 different simulation parameter combinations. Conclusion: The proposed method is always recommended in ROI-based analysis, since it significantly improves the analysis performance without a localizer task, wherever the fNIRS probe registration is known or unknown.

Significance: The reliability of functional near-infrared spectroscopy (fNIRS) measurements is reduced by systemic physiology. Short-channel regression algorithms aim at removing systemic “noise” by subtracting the signal measured at a short source–detector separation (mainly scalp hemodynamics) from the one of a long separation (brain and scalp hemodynamics). In literature, incongruent approaches on the selection of the optimal regressor signal are reported based on different assumptions on scalp hemodynamics properties. Aim: We investigated the spatial and temporal distribution of scalp hemodynamics over the sensorimotor cortex and evaluated its influence on the effectiveness of short-channel regressions. Approach: We performed hand-grasping and resting-state experiments with five subjects, measuring with 16 optodes over sensorimotor areas, including eight 8-mm channels. We performed detailed correlation analyses of scalp hemodynamics and evaluated 180 hand-grasping and 270 simulated (overlaid on resting-state measurements) trials. Five short-channel regressor combinations were implemented with general linear models. Three were chosen according to literature, and two were proposed based on additional physiological assumptions [considering multiple short channels and their Mayer wave (MW) oscillations]. Results: We found heterogeneous hemodynamics in the scalp, coming on top of a global close-to-homogeneous behavior (correlation 0.69 to 0.92). The results further demonstrate that short-channel regression always improves brain activity estimates but that better results are obtained when heterogeneity is assumed. In particular, we highlight that short-channel regression is more effective when combining multiple scalp regressors and when MWs are additionally included. Conclusion: We shed light on the selection of optimal regressor signals for improving the removal of systemic physiological artifacts in fNIRS. We conclude that short-channel regression is most effective when assuming heterogeneous hemodynamics, in particular when combining spatial- and frequency-specific information. A better understanding of scalp hemodynamics and more effective short-channel regression will promote more accurate assessments of functional brain activity in clinical and research settings., Neurophotonics, 7 (03)

Significance: The rates of melanoma and nonmelanoma skin cancer are rising across the globe. Due to a shortage of board-certified dermatologists, the burden of dermal lesion screening and erythema monitoring has fallen to primary care physicians (PCPs). An adjunctive device for lesion screening and erythema monitoring would be beneficial because PCPs are not typically extensively trained in dermatological care. Aim: We aim to examine the feasibility of using a smartphone-camera-based dermascope and a USB-camera-based dermascope utilizing polarized white-light imaging (PWLI) and polarized multispectral imaging (PMSI) to map dermal chromophores and erythema. Approach: Two dermascopes integrating LED-based PWLI and PMSI with both a smartphone-based camera and a USB-connected camera were developed to capture images of dermal lesions and erythema. Image processing algorithms were implemented to provide chromophore concentrations and redness measures. Results: PWLI images were successfully converted to an alternate colorspace for erythema measures, and the spectral bandwidth of the PMSI LED illumination was sufficient for mapping of deoxyhemoglobin, oxyhemoglobin, and melanin chromophores. Both types of dermascopes were able to achieve similar relative concentration results. Conclusion: Chromophore mapping and erythema monitoring are feasible with PWLI and PMSI using LED illumination and smartphone-based cameras. These systems can provide a simpler, more portable geometry and reduce device costs compared with interference-filter-based or spectrometer-based clinical-grade systems. Future research should include a rigorous clinical trial to collect longitudinal data and a large enough dataset to train and implement a machine learning-based image classifier.

Significance: The human brain is a highly complex system with nonlinear, dynamic behavior. A majority of brain imaging studies employing functional near-infrared spectroscopy (fNIRS), however, have considered only the spatial domain and have ignored the temporal properties of fNIRS recordings. Methods capable of revealing nonlinearities in fNIRS recordings can provide new insights about how the brain functions. Aim: The temporal characteristics of fNIRS signals are explored by comprehensively investigating their fractal properties. Approach: Fractality of fNIRS signals is analyzed using scaled windowed variance (SWV), as well as using visibility graph (VG), a method which converts a given time series into a graph. Additionally, the fractality of fNIRS signals obtained under resting-state and task-based conditions is compared, and the application of fractality in differentiating brain states is demonstrated for the first time via various classification approaches. Results: Results from SWV analysis show the existence of high fractality in fNIRS recordings. It is shown that differences in the temporal characteristics of fNIRS signals related to task-based and resting-state conditions can be revealed via the VGs constructed for each case. Conclusions: fNIRS recordings, regardless of the experimental conditions, exhibit high fractality. Furthermore, VG-based metrics can be employed to differentiate rest and task-execution brain states.

Significance: Accurate and objective identification of Alzheimer’s disease (AD) and dementia with Lewy bodies (DLB) is of major clinical importance due to the current lack of low-cost and noninvasive diagnostic tools to differentiate between the two. Developing an approach for such identification can have a great impact in the field of dementia diseases as it would offer physicians a routine objective test to support their diagnoses. The problem is especially acute because these two dementias have some common symptoms and characteristics, which can lead to misdiagnosis of DLB as AD and vice versa, mainly at their early stages. Aim: The aim is to evaluate the potential of mid-infrared (IR) spectroscopy in tandem with machine learning algorithms as a sensitive method to detect minor changes in the biochemical structures that accompany the development of AD and DLB based on a simple peripheral blood test, thus improving the diagnostic accuracy of differentiation between DLB and AD. Approach: IR microspectroscopy was used to examine white blood cells and plasma isolated from 56 individuals: 26 controls, 20 AD patients, and 10 DLB patients. The measured spectra were analyzed via machine learning. Results: Our encouraging results show that it is possible to differentiate between dementia (AD and DLB) and controls with an ∼86% success rate and between DLB and AD patients with a success rate of better than 93%. Conclusions: The success of this method makes it possible to suggest a new, simple, and powerful tool for the mental health professional, with the potential to improve the reliability and objectivity of diagnoses of both AD and DLB.

Significance: We introduce an application of machine learning trained on optical phase features of epithelial and mesenchymal cells to grade cancer cells’ morphologies, relevant to evaluation of cancer phenotype in screening assays and clinical biopsies. Aim: Our objective was to determine quantitative epithelial and mesenchymal qualities of breast cancer cells through an unbiased, generalizable, and linear score covering the range of observed morphologies. Approach: Digital holographic microscopy was used to generate phase height maps of noncancerous epithelial (Gie-No3B11) and fibroblast (human gingival) cell lines, as well as MDA-MB-231 and MCF-7 breast cancer cell lines. Several machine learning algorithms were evaluated as binary classifiers of the noncancerous cells that graded the cancer cells by transfer learning. Results: Epithelial and mesenchymal cells were classified with 96% to 100% accuracy. Breast cancer cells had scores in between the noncancer scores, indicating both epithelial and mesenchymal morphological qualities. The MCF-7 cells skewed toward epithelial scores, while MDA-MB-231 cells skewed toward mesenchymal scores. Linear support vector machines (SVMs) produced the most distinct score distributions for each cell line. Conclusions: The proposed epithelial–mesenchymal score, derived from linear SVM learning, is a sensitive and quantitative approach for detecting epithelial and mesenchymal characteristics of unknown cells based on well-characterized cell lines. We establish a framework for rapid and accurate morphological evaluation of single cells and subtle phenotypic shifts in imaged cell populations.

Noninvasive in vivo imaging of microcirculatory blood flow or perfusion (average speed times concentration of moving red blood cells) is of interest in several clinical applications, including monitoring of burn wounds1,2 and investigation of peripheral arterial disease,3 as well as several others.4 One technique that has gained a lot of focus in the last decades is laser speckle contrast imaging [LSCI; sometimes laser speckle contrast analysis (LASCA)] in which tissue is illuminated with a laser and the resulting speckle pattern is detected with a camera. Light that scatters when interacting with moving particles in the tissue will obtain a Doppler frequency shift depending on the velocity of the particles. The interference of light with different frequencies will give rise to fluctuations in the speckle pattern formed on the imaging sensor. The movement of the speckles on the sensor will cause an image blur that increases with exposure time. The local spatial statistics of this blur can be related to the movement of particles in the tissue, as is done in LSCI.5 LSCI has been increasingly used over laser Doppler imaging mainly due to its simple setup and fast acquisition and data processing. However, conventional LSCI has drawbacks such as a nonlinear response to perfusion, dependency to static scattering contrast, and a high variability in the presence of noise.5,6 Recent work has gone into calibrating and correcting for these issues to make LSCI a more accurate technique.7 Despite this, while it is generally accepted that LSCI is related to microcirculatory perfusion, the relationship is complex and a direct mapping is still not known.4,8 For the older laser Doppler flowmetry (LDF) technique, the relation to microcircular perfusion is better understood, where it is, for example, possible to theoretically show that the perfusion estimate is linearly related to the flow speed.9,10 To address the nonlinearity of LSCI to perfusion, Parthasarathy et al.11 proposed a new setup using multiple exposure times. The technique, called multi-exposure laser speckle contrast imaging (MELSCI, sometimes MESI), obtains information about the speckle motion blur at various exposures, enabling more advanced models to be used when estimating perfusion from the contrast.5,12 Several approaches to capture multi-exposure images have been proposed. The initial approach by Parthasarathy et al.11 was to use a time-modulated laser to achieve multi-exposure images. By capturing all images with a fixed exposure time but only illuminating the tissue during parts of that time, multi-exposures can be generated. However, a problem with this approach is that images with different exposure times are separated in time since they are captured sequentially. Dragojevic et al.13 suggested another approach that addressed this drawback. This was based on a high-speed camera, only capturing images at the shortest exposure time required, and in postprocessing adding successive images together to create longer exposure times. These synthetic exposure times are valid if the interframe delay in the camera is negligible. This method is faster and more accurate than previous methods due to all exposure times essentially being captured simultaneously, but it produces immense amounts of data that must be transferred and processed. At the time, this was, therefore, limited to an offline technique. Hence, MELSCI has so far been held back by technical limitations in both the imaging setup and the computation time of the models. To address this problem of a data and processing bottleneck, we have previously presented a system for synthetic MELSCI,14 based on a high-speed 1000-frames per second (fps) 1-megapixel camera directly interfaced to a field programmable gate array (FPGA) performing the multi-exposure contrast algorithm outlined by Dragojevic13 in real-time. Processed contrast images were sent to a computer, massively reducing the data throughput requirements. This system could produce multi-exposure contrast images at 15.6 fps, using all available frames from the 1000-fps camera, without any loss of data. However, due to insufficient processing power, the system was not able to transfer the processed images to the computer fast enough, and thus could not provide a real-time video-rate output. Furthermore, while the calculation of multi-exposure contrast images was fast enough, at the time there was no multi-exposure perfusion algorithm fast enough to keep up with the continuous stream of contrast images. The model proposed by Parthasarathy et al.11 and later refined by Kazmi et al.15 requires nonlinear fitting to extract perfusion-related parameters from the multi-exposure contrast. This process must be performed individually in each pixel for full field perfusion images, which is not feasible in real-time with satisfactory frame rates.6 We have previously presented a method using machine learning and simulated data from thousands of Monte Carlo tissue models from which we train an artificial neural network (ANN) to translate multi-exposure contrast to laser Doppler perfusion. We showed that this technique not only achieves a model-inference time of

In order to work and travel safely during the outbreak of COVID-19, a method of security detection based on deep learning is proposed by using machine vision instead of manual monitoring. To detect the illegal behaviors of workers without masks in workplaces and densely populated areas, an improved convolutional neural network VGG-19 algorithm is proposed under the framework of tensorflow, and more than 3000 images are collected for model training and testing. Using VGG-19 network model, three FC layers are optimized into one flat layer and two FC layers with reduced parameters. The softmax classification layer of the original model is replaced by a 2-label softmax classifier. The experimental results show that the precision of the model is 97.62% and the recall is 96.31%. The precision of identifying the workers without masks is 96.82%, the recall is 94.07%, and the data set provided has a high precision. For the future social health and safety to provide favorable test data.

Significance: Machine learning is increasingly being applied to the classification of microscopic data. In order to detect some complex and dynamic cellular processes, time-resolved live-cell imaging might be necessary. Incorporating the temporal information into the classification process may allow for a better and more specific classification. Aim: We propose a methodology for cell classification based on the time-lapse quantitative phase images (QPIs) gained by digital holographic microscopy (DHM) with the goal of increasing performance of classification of dynamic cellular processes. Approach: The methodology was demonstrated by studying epithelial-mesenchymal transition (EMT) which entails major and distinct time-dependent morphological changes. The time-lapse QPIs of EMT were obtained over a 48-h period and specific novel features representing the dynamic cell behavior were extracted. The two distinct end-state phenotypes were classified by several supervised machine learning algorithms and the results were compared with the classification performed on single-time-point images. Results: In comparison to the single-time-point approach, our data suggest the incorporation of temporal information into the classification of cell phenotypes during EMT improves performance by nearly 9% in terms of accuracy, and further indicate the potential of DHM to monitor cellular morphological changes. Conclusions: Proposed approach based on the time-lapse images gained by DHM could improve the monitoring of live cell behavior in an automated fashion and could be further developed into a tool for high-throughput automated analysis of unique cell behavior. (C) The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License.

International audience; Intraoperative optical imaging is a localization technique for the functional areas of the human brain cortex during neurosurgical procedures. However, it still lacks robustness to be used as a clinical standard. In particular, new biomarkers of brain functionality with improved sensitivity and specificity are needed. We present a method for the computation of hemodynamics-based functional brain maps using an RGB camera and a white light source. We measure the quantitative oxy and deoxyhemoglobin concentration changes in the human brain cortex with the modified Beer-Lambert law and Monte Carlo simulations. A functional model has been implemented to evaluate the functional brain areas following neuronal activation by physiological stimuli. The results show a good correlation between the computed quantitative functional maps and the brain areas localized by electrical brain stimulation (EBS). We demonstrate that an RGB camera combined with a quantitative modeling of brain hemodynamics biomarkers can evaluate in a robust way the functional areas during neurosurgery and serve as a tool of choice to complement EBS. © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Spontaneous resting-state neural activity or hemodynamics has been used to reveal functional connectivity in the brain. However, most of the commonly used clustering algorithms for functional parcellation are time-consuming, especially for high-resolution imaging data. We propose a density center-based fast clustering (DCBFC) method that can rapidly perform the functional parcellation of isocortex. DCBFC was validated using both simulation data and the spontaneous calcium signals from widefield fluorescence imaging of excitatory neuron-expressing transgenic mice (Vglut2-GCaMP6s). Compared to commonly used clustering methods such as k-means, hierarchical, and spectral, DCBFC showed a higher adjusted Rand index when the signal-to-noise ratio was greater than −8 dB for simulated data and higher silhouette coefficient for in vivo mouse data. The resting-state functional connectivity (RSFC) patterns obtained by DCBFC were compared with the anatomic axonal projection density (PDs) maps derived from the voxel-scale model. The results showed a high spatial correlation between RSFC patterns and PDs.

Non-contact vital sign monitoring enables the estimation of vital signs, such as heart rate, respiratory rate and oxygen saturation (SpO2), by measuring subtle color changes on the skin surface using a video camera. For patients in a hospital ward, the main challenges in the development of continuous and robust non-contact monitoring techniques are the identification of time periods and the segmentation of skin regions of interest (ROIs) from which vital signs can be estimated. We propose a deep learning framework to tackle these challenges. Approach: This paper presents two convolutional neural network (CNN) models. The first network was designed for detecting the presence of a patient and segmenting the patient’s skin area. The second network combined the output from the first network with optical flow for identifying time periods of clinical intervention so that these periods can be excluded from the estimation of vital signs. Both networks were trained using video recordings from a clinical study involving 15 pre-term infants conducted in the high dependency area of the neonatal intensive care unit (NICU) of the John Radcliffe Hospital in Oxford, UK. Main results: Our proposed methods achieved an accuracy of 98.8% for patient detection, a mean intersection-over-union (IOU) score of 88.6% for skin segmentation and an accuracy of 94.5% for clinical intervention detection using two-fold cross validation. Our deep learning models produced accurate results and were robust to different skin tones, changes in light conditions, pose variations and different clinical interventions by medical staff and family visitors. Significance: Our approach allows cardio-respiratory signals to be continuously derived from the patient’s skin during which the patient is present and no clinical intervention is undertaken.

Fluorescence microscopy plays a vital role in understanding the subcellular structures of living cells. However, it requires considerable effort in sample preparation related to chemical fixation, staining, cost, and time. To reduce those factors, we present a virtual fluorescence staining method based on deep neural networks (VirFluoNet) to transform fluorescence images of molecular labels into other molecular fluorescence labels in the same field-of-view. To achieve this goal, we develop and train a conditional generative adversarial network (cGAN) to perform digital fluorescence imaging demonstrated on human osteosarcoma U2OS cell fluorescence images captured under Cell Painting staining protocol. A detailed comparative analysis is also conducted on the performance of the cGAN network between predicting fluorescence channels based on phase contrast or based on another fluorescence channel using human breast cancer MDA-MB-231 cell line as a test case. In addition, we implement a deep learning model to perform autofocusing on another human U2OS fluorescence dataset as a preprocessing step to defocus an out-focus channel in U2OS dataset. A quantitative index of image prediction error is introduced based on signal pixel-wise spatial and intensity differences with ground truth to evaluate the performance of prediction to high-complex and throughput fluorescence. This index provides a rational way to perform image segmentation on error signals and to understand the likelihood of mis-interpreting biology from the predicted image. In total, these findings contribute to the utility of deep learning image regression for fluorescence microscopy datasets of biological cells, balanced against savings of cost, time, and experimental effort. Furthermore, the approach introduced here holds promise for modeling the internal relationships between organelles and biomolecules within living cells., 20 pages, 9 figures