Search

Expressing the visual content of an image in natural language form has gained relevance due to technological and algorithmic advances together with improved computational processing capacity. Many smartphone applications for image captioning have been developed recently as built-in cameras provide advantages of easy-operation and portability, resulting in capturing an image whenever or wherever needed. Here, an encoder-decoder framework based new image captioning approach with a multi-layer gated recurrent unit is proposed. The Inception-v3 convolutional neural network is employed in the encoder due to its capability of more feature extraction from small regions. The proposed recurrent neural network-based decoder utilizes these features in the multi-layer gated recurrent unit to produce a natural language expression word-by-word. Experimental evaluations on the MSCOCO dataset demonstrate that our proposed approach has the advantage over existing approaches consistently across different evaluation metrics. With the integration of the proposed approach to our custom-designed Android application, named “VirtualEye+”, it has great potential to implement image captioning in daily routine.

Diabetes is a group of metabolic conditions resulting in high blood sugar levels over prolonged periods that affects hundreds of millions of patients worldwide. Measuring glucose concentration enables patient-specific insulin therapy, and is essential to reduce the severity of the disease, potential complications, and related mortalities. Recent advances and developments in smartphone-based colorimetric glucose detection systems are discussed in this review. The importance of glucose monitoring, data collection, transfer, and analysis, via non-invasive/invasive methods is highlighted. The review also presents various approaches using 3D-printed materials, screen-printed electrodes, polymer templates, designs allowing multiple glucose analysis, bioanalytes and/or nanostructures for glucose detection. The positive effects of advances in improving the performance of smartphone-based platforms are introduced along with future directions and trends in the application of emerging technologies in smartphone-based diagnostics.

In this study, a microfluidic paper-based analytical device (mu PAD) was integrated with a smartphone app capable of offline (without internet access) image processing and analysis for the rapid colorimetric detection of glucose. A self-inking stamp was used to form hydrophobic channels on a piece of paper-towel due to its superior water absorption efficiency. As demonstrated, the developed sensor was employed for the colorimetric detection of glucose in artificial saliva in the linear scope of 0 - 1 mM with a calculated detection limit of 29.65 mu M. The experimental results show that the quantitative analysis of glucose with the proposed smartphone platform could be completed in less than one minute. The app developed for the smartphone platform is capable of extracting the color-changing area with an embedded image processing tool which could address the problem of color uniformity in the detection zones of mu PAD. The integrated platform has great potential to be used for non-invasive measurements of glucose in body fluids, like tears, sweat and saliva.

Sequential Monte Carlo probability hypothesis density (SMC-PHD) filtering is a popular method used recently for audio-visual (AV) multi-speaker tracking. However, due to the weight degeneracy problem, the posterior distribution can be represented poorly by the estimated probability, when only a few particles are present around the peak of the likelihood density function. To address this issue, we propose a new framework where particle flow (PF) is used to migrate particles smoothly from the prior to the posterior probability density. We consider both zero and non-zero diffusion particle flows (ZPF/NPF), and developed two new algorithms, AV-ZPF-SMC-PHD and AV-NPF-SMC-PHD, where the speaker states from the previous frames are also considered for particle relocation. The proposed algorithms are compared systematically with several baseline tracking methods using the AV16.3, AVDIAR and CLEAR datasets, and are shown to offer improved tracking accuracy and average effective sample size (ESS).

The objective of this study was to develop novel colorimetric films for food freshness monitoring. UV light irradiation (365 nm) and carbon dots (CDs) were tested as the potential crosslinkers in the fabrication of anthocyanins doped fish gelatin (FG) films. The effect of crosslinkers on the optical, surface, structural, barrier and mechanical properties of FG films was investigated. The incorporation of CDs under UV irradiation improved the tested properties of FG films. The kinetic colorimetric responses of FG films against ammonia vapor were studied to simulate the food spoilage and determine the ammonia sensitivity of the films. Among the tested films, UV-treated FG films containing 100 mg/l (FG-UV-CD100) indicated the best properties. Later, the color difference of FG-UV-CD100 films was observed to correlate well with microbial growth and TVB-N release in skinless chicken breast samples. At the same time, a custom-designed smartphone application (SmartFood) was developed to be used with the FG-UV-CD100 film for quantitative estimation of food freshness in real-time. The proposed food freshness monitoring platform reveals a great potential to minimize global food waste and the outbreak of foodborne illness.

In the present study, iodide-mediated 3,3′,5,5′-tetramethylbenzidine (TMB)-H2O2 reaction system was applied to a microfluidic paper-based analytical device (μPAD) for non-enzymatic colorimetric determination of H2O2. The proposed system is portable and incorporates a μPAD with a machine learning-based smartphone app. A smartphone app called “Hi-perox Sens” capable of image capture, cropping and processing was developed to make the system simple and user-friendly. Briefly, circular μPADs were designed and tested with varying concentrations of H2O2. Following the color change, the images of the μPADs were taken with four different smartphones under seven different illumination conditions. In order to make the system more robust and adaptive against illumination variation and camera optics, the images were first processed for feature extraction and then used to train machine learning classifiers. According to the results, TMB + KI showed the highest classification accuracy (97.8%) with inter-phone repeatability at t = 30 s under versatile illumination and maintained its accuracy for 10 minutes. In addition, the performance of the system was also comparable to two different commercially available H2O2 kits in real samples.

Background Tabanus bromius (Diptera: Tabanidae) is one of the most notable Tabanidae species of veterinary and medical importance distributed throughout the Palearctic region. In this study, we investigate the genetic diversity and the phylogeographic structure of T. bromius sampled from Turkey, Croatia, and Iran. Methods For this purpose, a 686-base-pair (bp) fragment of mitochondrial DNA cytochrome oxidase I gene (COI) and 1339 bp of the nuclear DNA internal transcribed spacer (ITS) were sequenced from 247 individuals representing 15 populations. Results The sequences generated 169 COI haplotypes and 90 ITS alleles. A higher haplotype/allele diversity (h = 0.9909 for the COI gene and Ad = 0.8193 for the ITS region) compared to a low nucleotide diversity (π = 0.020605 for COI gene and π = 0.013667 for the ITS region), present for a high number of singleton and private haplotypes/alleles imply population expansion in the past. The results of phylogenetic analysis led to the uncovering of geographically significant groupings of lineages with regard to the entrance of the species into Anatolia and the location of major geographic barriers. According to current data, the species appears to have entered Turkey from Caucasia and Iran. A molecular clock applied to the COI data suggests that T. bromius diverged from the outgroup species nearly 8.83 million years ago, around the end of the Miocene era. Conclusions The results of this study indicate remarkable genetic diversity across the studied range of the species. High haplotype/allele versus low nucleotide diversity and demographic analyses implied that the T. bromius populations have undergone a series of expansions and retreats in the past. Our current findings suggest that T. bromius split from outgroups around the Late Miocene. Subsequent diversification events during the climatic and environmental fluctuation times of the Late Pliocene and Early Pleistocene periods also significantly influenced the species, resulting in the formation of some major genetic lineages. The phylogenetic analyses indicate that T. bromius most likely entered Turkey from the Caucasus region and Iran. Graphical Abstract

Conventional enzyme-based glucose quantification approaches are not feasible due to their high cost, specific working temperatures, short shelf life, and poor stability. Therefore, a portable platform, which offers rapid response, cost-efficiency, and high sensitivity, is indispensable for the healthcare of diabetes. In this study, we proposed a portable platform incorporating gold (Au) and silver (Ag) nanoparticles (NPs) with a smartphone application based on machine learning for non-enzymatic glucose quantification. The color change obtained from the reaction of small and large Au/Ag NPs with glucose was captured using a smartphone camera to create a dataset for the training of machine-learning classifiers. Our custom-designed user-friendly smartphone application called "GlucoQuantifier" uses a cloud system to communicate with a remote server running a machine-learning classifier. Among the tested classifiers, linear discriminant analysis exhibits the best classification performance (93.63%) with small Au/Ag NPs and it demonstrates that incorporating Au/Ag NPs with machine learning under a smartphone application can be used for non-enzymatic glucose quantification.

Pharyngitis is defined as inflammation in the back wall of the nose and mouth cavity. Mobile technologies have received an increasing amount of attention in the recent global epidemic due to their advantage in pre-diagnosis of diseases that show respiratory symptoms such as pharyngitis. In this study, we propose a custom-designed Android application that offers pharyngitis detection based on artificial intelligence using throat images. Deep learning, a subset of artificial intelligence, allows being embedded in Android applications which leads to be giving fast and highly accurate results without an internet connection. Popular deep learning architectures including Inception-v3, MobileNet-v2, Xception, VGG16, VGG19 and ResNet50, have been trained to evaluate their performance in pharyngitis detection. Detection of pharyngitis for the images could be performed after they were verified as inner of the mouth. Therefore, two sequential classifiers were designed. The first classifiers were trained with the MSCOCO dataset, while the second-ranked classifiers were trained with the dataset, including 131 pharyngitis and 208 non-pharyngitis throat images augmented with specific methods. Among the above architectures, ResNet50 showed the highest performance with 96.20% accuracy. By embedding the ResNet50 architecture into our custom-designed Android application named ‘Farenjit Tanimlama’, users will be able to pre-diagnose in a practical way, thus contributing to reducing the burden on the health system caused by the epidemic.

Yeni koronavirüs salgını (COVID-19), ilk olarak Aralık 2019'da Çin'in Wuhan şehrinde ortaya çıktığından beri hızla yayılmakta ve dünya çapında pandemiye neden olmaktadır. Pozitif vakaların erken teşhisi, COVID-19 tespiti için hızlı ve doğru yanıtlar veren tanı yöntemlerinin gelişmesine yol açan salgının daha da yayılmasını önlemede önemli bir rol oynar. Önceki çalışmalar, göğüs bilgisayarlı tomografisinin (BT) COVID-19 vakalarının erken taranması ve teşhisi için gerekli bir araç olduğunu doğrulamıştır. İncelemeler sonucu COVID-19 hastalığına yakalanmış bir insanın akciğer BT taramalarında buzlu cam opasitesi adı verilen, renk ve doku değişikliğine sebep olan bir radyolojik bulgu bulunmuştur. Uzun saatler çalışan radyologların dikkatsizliği ve bulguların farklı hastalıklarla karıştırılmasıyla sonuçlanan yanlış tanı nedeniyle radyologlara yardımcı olan otomatik bir sisteme ihtiyaç doğmaktadır. Bu çalışmada, 3D BT kesitleri kullanılarak COVID-19 tespiti için bulanık sınıflandırmaya dayalı yeni bir yaklaşım sunuyoruz. Önerilen yaklaşımda COVID ve Normal olmak üzere iki sınıftan oluşan hastaların 3D BT görüntülerinin çarpıklık, basıklık ve ortalama istatistiksel özellikleri hesaplanır ve her iki sınıf için değer aralıkları belirlenir. Hesaplanan üç istatistiksel özellik ve değer aralıkları geliştirilen bulanık mantık sınıflandırıcısında üyelik fonksiyonu olarak kullanılır. Uygun üyelik fonksiyonları ve kural tabanıyla ile geliştirilen bulanık sınıflandırıcı 3D BT taramalarını sınıflandırarak çıkışa verir. Bu bulanık sınıflandırıcı sistem geliştirilen arayüz ile birleştirerek radyologlara yardımcı bir sistem tasarlanmıştır. Önerilen yaklaşım, kullanıcı dostu bu arayüz altında COVID ve Normal (ikili sınıflandırma) açısından hızlı ve doğru teşhis sağlamaktadır. Deneysel değerlendirmeler, yaklaşımımızın radyologların ilk taramalarını doğrulaması ve erken tanı, izolasyon ve tedaviyi iyileştirmesi için büyük bir potansiyele sahip olduğunu ve bu da enfeksiyonun önlenmesine ve salgının kontrolüne katkıda bulunduğunu göstermektedir., The novel coronavirus 2019 (COVID-19) is still spreading rapidly since it first appeared in Wuhan city of China in December 2019, resulting in a worldwide pandemic. Early detection of positive cases plays a key role in preventing the further spread of the epidemic which leads to the development of diagnostic methods that give rapid and accurate responses for the detection of COVID-19. Previous studies confirmed that chest computed tomography (CT) is an indispensable tool for early screening and diagnosing of COVID-19 cases. As a result of examinations on CT scans, a radiological finding that is called ground-glass opacity, causing color, and texture change, was found in the lung of a person with COVID-19. Due to the carelessness of radiologists who work long hours and the misdiagnosis resulting in confusion of the findings with different diseases, an automatic system that helps radiologists is needed. In this paper, we present a new approach based on fuzzy classification for the detection of COVID-19 using 3D CT volumes. In the proposed approach, the skewness, kurtosis, and average statistical features of 3D CT images of patients consisting of two classes, COVID and Normal, are calculated and the value ranges are determined for both classes. Three statistical features and value ranges are used as membership functions in the development of fuzzy logic classifier. The proposed approach provides rapid and accurate diagnostics in terms of COVID vs. Normal (binary classification) under a user-friendly interface. Experimental evaluations demonstrate that our approach has great potential for radiologists to validate their initial screening and improve early diagnosis, isolation, and treatment, which contributes to infection prevention and control of the epidemic.

Image captioning is the description of an image with natural language expressions using computer vision and natural language processing fields. Recent advances in hardware and processing power in smartphones lead the development of many image captioning applications. In this study, a novel automatic image captioning system based on the encoder-decoder approach that can be applied in smartphones is proposed. While high-level visual information is extracted with the ResNet152V2 convolutional neural network in the encoder part, the proposed decoder transforms the extracted visual information into natural expressions of the images. The proposed decoder with the multilayer gated recurrent unit structure allows generating more meaningful captions using the most relevant visual information. The proposed system has been evaluated using different performance metrics on the MSCOCO dataset and it outperforms the state-ofthe-art approaches. The proposed system is also integrated with our custom-designed Android application, named IMECA, which generates captions in offline mode unlike similar applications. Thus, image captioning is intended to be practical for more people.

The need for real-time and high-precision localization in indoor or outdoor areas has increased considerably recently. Real-time location systems offer solutions for this need with different technologies. One of the solutions is to determine the location using the time difference of arrival of the signal between anchor and tag. In this study, a new algorithm is proposed for indoor localization of wearable smart devices. The algorithm, which is based on the time difference of arrival of the signal from the tag to the anchor, allows the localization with submeter accuracy. Thus, it will be possible to track the location of patients, personnel or objects in military, health and commercial areas using real-time, accurate and precise location information via wearable smart devices.

The demands for the improvement of production cost and quality have been increased with the widespread application of marble. The problem of detecting fractures and cracks on marble has attracted an increasing amount of attention recently. In this study, an artificial intelligence-based fracture and crack detection system with high accuracy is proposed. In the proposed system, images in the dataset were first trained with the convolutional neural network-based semantic segmentation model, DeepLabv3+, in order to detect fractures and cracks. The dataset has been augmented by adding different ground images containing fracture and crack surfaces as well as marble images due to the positive contribution of the size of the dataset to the accuracy. After training with 5 different convolutional neural networks and 3 different optimization algorithms, the mIoU and weighted mIoU values were achieved as 0.672 and 0.967, respectively, with the RMSprop algorithm and ResNet-50 architecture. Unlike similar studies, the proposed system has been integrated with our custom-designed interface. Thus, it is aimed to make fast and efficient crack detection on the production line.

Diabetes is a chronic disease that requires lifelong treatment to keep blood sugar at a normal level. Hyperglycemia (high blood sugar) and hypoglycemia (low blood sugar) are critical blood glucose levels that should be monitored during the treatment. Alerting the patient when the blood glucose is at critical levels may minimize possible complications that may occur. Therefore, it was aimed to classify critical blood glucose levels with machine learning algorithms in this study. The performance of the classifiers has been tested with synthetic and real data. Synthetic data were created by adding noise to the sinusoidal wave while real data were obtained from diabetic patients. Features were extracted using the time series analysis method as the data is time-dependent. Machine learning algorithms were trained with these extracted features and blood glucose was classified in 5 levels (hypoglycemia, pre-hypoglycemia, normal, pre-hyperglycemia and hyperglycemia) with 95.12% accuracy.

The recent advances in smartphone technology have led to the emergence of simple, low-cost and portable platforms that aim reliable qualitative and quantitative analysis of colored solutions. In this paper, we present a new approach based on fuzzy classification for quantitative detection of peroxide relying on color change due to concentration level. The proposed fuzzy classification algorithm was integrated with a smartphone via a cloud system. Our custom-designed Android application was developed with a user-friendly interface to allow the users to capture the image of peroxide strips and crop the region of interest on the image for quantification. Average RGB (Red-Green-Blue) values in the cropped region were calculated to transfer these values to our proposed fuzzy logic classifier via a cloud system to quantify the peroxide content. The proposed fuzzy classifier evaluates the concentration value based on RGB values and sends back to the Android application via the cloud system. Experimental investigations show the advantage of the proposed system for the quantification of peroxide.

Generating captions and text descriptions of images will enable visually impaired extended accessibility to the real world, thus reducing their social isolation, improving their well-being, employability and education experience. In this paper, we propose a captioning system based on a smartphone, which offers ultra-low-cost, portable and user-friendly platform for visually impaired. Image captioning aims to understand and describe a visual scene in terms of natural language expressions, which requires an advanced level of image understanding. In that sense, it brings computer vision and natural language process (NLP) fields together to generate meaningful sentences. In the computer vision side, more distinguished visual attributes need to be extracted from the images to feed the NLP for caption generation. The proposed captioning system based on a neural encoder-decoder framework that uses inception-v3 deep learning architecture as the encoder to extract the visual attributes. On the other hand, the decoder is based on the NLP architecture, which consists of multi-layer recurrent neural networks, to generate captions. Our captioning system is run on the remote server integrated to our custom-designed Android application via a cloud system after trained on MSCOCO 2017 captioning dataset. The user can choose an image from the gallery or capture a new image to transfer the image to the remote server. The caption is generated by the proposed captioning system and transferred back to the Android application to be displayed on the screen. The caption can be read out loudly using the narrator option. Experimental results show that the proposed system offers great advantages to the visually impaired in terms of portability, simple operation and rapid response.

Paper-based sensors have great potential for use in a variety of areas, from environmental monitoring to clinical and pointof-care testings. Here, a microfluidic paper-based analytical device (µPAD) was integrated with a smartphone app capable of offline (without internet access) image processing and analysis for rapid colorimetric detection of glucose. A self-inking stamp was used to form hydrophobic channels on a piece of paper-towel due to its superior water absorption efficiency. As demonstrated, the developed sensor was employed for colorimetric detection of glucose in artificial saliva in the linear scope of 0-1 mM with a calculated detection limit of 29.65 µM. In addition, experimental results show that quantitative analysis of glucose with the proposed smartphone platform could be completed in less than one minute. The app developed for the smartphone platform is capable of extracting the color changing area with an embedded image processing tool which could address the problem of color uniformity in detection zones of µPAD. The total cost of a µPAD is less than $0.2. By integrating µPAD with a smartphone and user-friendly app together, the proposed smartphone-based platform could be used for the quantitative analysis of glucose with advantages such as portability, simple operation, rapid response, ultra-low cost, field-deployable, selectivity and sensitivity. The results show that the integrated platform has great potential to be used for non-invasive measurement of glucose in body fluids like a tear, sweat and saliva.

In this paper, we present a smartphone platform for colorimetric water quality detection based on the use of built-in camera for capturing a single-use reference image. A custom-developed app processes this image for training and creates a reference model to be used later in real experimental conditions to calculate the concentration of the unknown solution. This platform has been tested on four different water quality colorimetric assays with various concentration levels, and results show that the presented platform provides approximately 100% accuracy for colorimetric assays with noticeable color difference. This portable, cost-effective, and user-friendly platform is promising for application in water quality monitoring.

A smartphone application based on machine learning classifier algorithms was developed for quantifying peroxide content on colorimetric test strips. The strip images were taken from five different Android based smartphones under seven different illumination conditions to train binary and multi-class classifiers and to extract the learning model. A custom app, “ChemTrainer”, was designed to capture, crop, and process the active region of the strip, and then to communicate with a remote server that contains the learning model through a Cloud hosted service. The application was able to detect the color change in peroxide strips with over 90% success rate for primary colors with inter-phone repeatability under versatile illumination. The utilization of a grey-world color constancy image processing algorithm positively affected the classification accuracy for binary classifiers. The developed app with a Cloud based learning model paves the way for better colorimetric detection for paper-based chemical assays.

Automatic caption generation in natural language to describe the visual content of an image has attracted an increasing amount of attention in the last decade due to its potential applications. It is a challenging task to generate captions with proper linguistics properties as it requires an advanced level of image understanding that goes far beyond image classification and object detection. In this paper, we propose to use the Stanford CoreNLP model to generate a caption after images are trained using VGG16 deep learning architecture. The visual attributes of images are extracted with the VGG16, which conveys richer content, and then they are fed into the Stanford model for caption generation. Experimental results on the MSCOCO dataset show that the proposed model significantly outperforms the state-of-the-art approaches consistently across different evaluation metrics.

Visually and hearing impaired people face troubles due to inaccessible infrastructure and social challenges in daily life. To increase the life quality of those people, we report a portable and user-friendly smartphone-based platform capable of generating captions and text descriptions, including the option of a narrator, using image obtained from a smartphone camera. Image captioning is to generate a sentence to describe the visual content of an image in natural language and has attracted an increasing amount of attention in the fields of computer vision and natural language processing due to its potential applications. Generating image captions with proper linguistic properties is a challenging task as it needs to combine advanced level of image understanding algorithms with natural language processing methods. In this study, we propose to use Long Short-Term Memory (LSTM) model to generate a caption after images are trained using VGG16 deep learning architecture. The visual attributes of images are extracted with the VGG16, which conveys richer content, and then they are fed into the LSTM model for caption generation. This system is integrated with our custom- designed Android application, named as "Eye of Horus" which transfers the images from smartphone to the remote server via a cloud system, and displays the captions after the images are processed with the proposed captioning approach. The results show that the integrated platform has great potential to be used for image captioning by visually and hearing impaired people with advantages such as portability, simple operation and rapid response.

Paper-based microfluidic analytical devices (¼PAD) have great potential for use in a variety of areas, from environmental monitoring to clinical and bedside testing. Such devices are generally designed to be disposable, cost effective, practical and user-friendly. In this study, a smartphone based ¼PAD was developed which can be used in high sensitive quantitative analysis of H2O2 in transparent liquids such as water. Solid ink technology is used in the production of ¼PADs. The H2O2 concentration dependent color change (colorimetric) used in the measurement was catalyzed by horseradish peroxidase (HRP). In addition, the effect of chitosan on the measurement sensitivity was investigated. The resulting color changes were recorded in a dark environment with a special apparatus integrated smartphone camera and the color changes were digitized by an algorithm developed in MATLAB. The results showed that the produced ¼PADs can successfully be used to quantitatively analyze H2O2 between X and Y, and the chitosan significantly increases the sensitivity.

Paper-based sensors have great potential for use in many different fields so far. In this study, a paper-based microfluidic analytical device (¼PAD) was integrated with a smartphone application capable of offline (no internet access) image processing and analysis for rapid colorimetric glucose detection. A paper towel was preferred due to its better water absorption efficiency than other papers. A stamp containing a 3D printed mold was used to form hydrophobic channels on a paper towel for the production of ADPAD. A hydrophobic by nature and light polymerizable resin was used as the ink. In order to increase the accuracy of the analysis, an image processing algorithm has been developed for the extraction of the region of interest (ROI) on the uPAD. The developed integrated platform gave a linear response in the range of 0.1 to 1 mM glucose and the limit of detection was calculated as 66.2 ¼M. The whole analysis was completed in less than one minute. Finally, a smart mobile phone application capable of offline image processing was developed to make the system user friendly. The integrated sensor system is portable, fast, user-friendly, ultra-low cost, usable by everyone, simple and sensitive.