Search

A light intensity-modulated optical fiber sensor for continuous level measurement is proposed and demonstrated. The sensor is constructed by vertically arranging groups of fibers on the oblique surface which reflects all the incident light when the fibers are in the air but a small proportion of light when the fibers are immersed in the liquid. In order to implement easy modulation and detection, the angle of oblique surface is specially designed to get the maximum intensity modulation according to Fresnel reflection. The fiber groups consist of emitting and receiving fibers, which are tightly fixed to make it easy to detect reflected light based on side coupling effect. The calculation model of the oblique structure and the light path simulation by finite element analysis are used to optimize the oblique angle and prove the feasibility of the sensor. Gradient level experiments and stability tests in the laboratory indicate that the sensor has high sensitivity of 0.21mW/cm, superior resolution and good stability. © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Cross-modal medical imaging techniques are predominantly being used in the clinical suite. The ensemble learning methods using cross-modal medical imaging adds reliability to several medical image analysis tasks. Motivated by the performance of deep learning in several medical imaging tasks, a deep learning-based denoising method Cross-Modality Guided Denoising Network CMGDNet for removing Rician noise in T1-weighted (T1-w) Magnetic Resonance Images (MRI) is proposed in this paper. CMGDNet uses a guidance image, which is a cross-modal (T2-w) image of better perceptual quality to guide the model in denoising its noisy T1-w counterpart. This cross-modal combination allows the network to exploit complementary information existing in both images and therefore improve the learning capability of the model. The proposed framework consists of two components: Paired Hierarchical Learning (PHL) module and Cross-Modal Assisted Reconstruction (CMAR) module. PHL module uses Siamese network to extract hierarchical features from dual images, which are then combined in a densely connected manner in the CMAR module to finally reconstruct the image. The impact of using registered guidance data is investigated in removing noise as well as retaining structural similarity with the original image. Several experiments were conducted on two publicly available brain imaging datasets available on the IXI database. The quantitative assessment using Peak Signal to noise ratio (PSNR), Structural Similarity Index (SSIM), and Feature Similarity Index (FSIM) demonstrates that the proposed method exhibits 4.7% and 2.3% gain (average), respectively, in SSIM and FSIM values compared to other state-of-the-art denoising methods that do not integrate cross-modal image information in removing various levels of noise.

In recent years, there has been a growing interest in building smart cities based on the Internet of Things (IoT) technology. However, selecting a low-cost IoT wireless technology that enables low-power connectivity remains one of the key challenges in integrating IoT to smart cities. In this context, the IQRF technology offers promising opportunities to provide cost-effective solutions. Yet, in the literature, there are limited studies on utilizing IQRF technology for smart city applications. Therefore, this study is aimed at increasing the awareness about the use of IQRF technology in IoT-based smart city development. For this purpose, a review of smart city architectures along with challenges/requirements in adopting IoT for smart cities is provided. Then, some of the common cost-effective IoT wireless technologies that enable low-power consumption are briefly presented. Next, the benefits of IQRF technology over other technologies are discussed by making theoretical comparisons based on technical documentations and reports. Moreover, the research efforts currently being undertaken by the authors as a part of ongoing project on the development of IoT-based smart city system in Gjøvik Municipality, Norway, are conceptually introduced. Finally, the future research directions are addressed.

Deep Learning-based chest Computed Tomography (CT) analysis has been proven to be effective and efficient for COVID-19 diagnosis. Existing deep learning approaches heavily rely on large labeled data sets, which are difficult to acquire in this pandemic situation. Therefore, semi-supervised approaches are in demand. In this paper, we propose an end-to-end semi-supervised COVID-19 detection approach, ResNext+, that only requires volume level data labels and can provide slice level prediction. The proposed approach incorporates a lung segmentation mask as well as spatial and channel attention to extract spatial features. Besides, Long Short Term Memory (LSTM) is utilized to acquire the axial dependency of the slices. Moreover, a slice attention module is applied before the final fully connected layer to generate the slice level prediction without additional supervision. An ablation study is conducted to show the efficiency of the attention blocks and the segmentation mask block. Experimental results, obtained from publicly available datasets, show a precision of 81.9% and F1 score of 81.4%. The closest state-of-the-art gives 76.7% precision and 78.8% F1 score. The 5% improvement in precision and 3% in the F1 score demonstrate the effectiveness of the proposed method. It is worth noticing that, applying image enhancement approaches do not boost the performance of the proposed method, sometimes even harm the scores, although the enhanced images illustrate better perceived visual image quality. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/

Graphs frequently appear as a type of data structure that efficiently models a set of interrelated objects as nodes and their relations as edges between them. Geometric deep learning emerged as a promising field for modeling non-Euclidean geometric data as graphs, Riemannian manifolds and meshes. In this paper, we review geometric deep learning based state-of-the-art approaches for multi-object tracking. We illustrate the main concept, the model architecture, loss functions, and optimization strategies of the main algorithms. Moreover, algorithms are quantitatively evaluated on standard performance metrics using the standard dataset. The potential of geometric deep learning in tracking is discussed and future directions are proposed.

Contrast enhancement is an important processing task needed to improve the overall perceptual quality of an image. Ideally, a contrast enhancement method should neither affect the naturalness of images nor introduce other unwanted artifacts when improving perceptual quality. While humans are able to detect any improvement or such unwanted side-effects relatively easily, designing an objective metric that does the same is still a challenge. Most existing contrast enhancement evaluation (CEE) metrics do not take into consideration all these important criteria when evaluating quality. In this work, we propose a new measure for CEE that combines some of the carefully selected existing metrics effectively. Each of the selected metrics tackles only one of the important criteria of an effective contrast enhancement method such as structure preservation or contrast improvement. For this novel fusion-based metric, we have further exploited the full potential of the selected metrics by applying them on decomposed image using cartoon-texture decomposition. The results on two CEE databases show the effectiveness of our proposed metric.

This paper describes an innovative e-learning project which is the development of a digital workbook that helps teaching handwriting at school. In this work, we propose a new qualitative and quantitative analysis process of cursive handwriting. This process detects automatically mistakes, gives a real-time feedback and helps teachers evaluate children’s writing skills. The main aim of this digital workbook is to help children learn how to write correctly. The proposed process is composed of five main criteria: shape,direction, stroke order, position respect to the reference lines and kinematics of the trace. It analyzes the handwriting quality and gives automatically feedback based on the Beta-Elliptic Model using similarity detection (SD) and dissimilarity distance (DD) measure. Our work apprehends dynamic and visual representation of the acquired traces and selects efficient features adapted to various handwriting styles and multiple script languages such as Arabic, Latin, digits, and symbols drawing. It demonstrates that beta-elliptic is not only a model for segmentation and recognition but also a tool to evaluate handwriting. Our application offers two interactive interfaces respectively dedicated to learners, and experts or teachers who can adapt it easily to the specificity of each child. The validation of the proposed system is done on a database collected in Tunisia primary schools with 400 children. Experimental results show that the efficiency and robustness of our suggested framework that do help teachers and children by offering positive feedback throughout the handwriting learning process using tactile digital devices.

The development of novel techniques for social modeling in the context of surveillance applications has significantly reduced manual processing of large and continuous video data. These techniques for social modeling widely cover crowd motion analysis since the impact of social modeling on crowd is significant. However, existing crowd motion analysis methods face a number of problems including limited availability of crowd data representing a specific behavior and weaknesses of proposed models to explore the underlying patterns of crowd behavior. To cope with these problems, we propose a novel method based on energy modeling and social interaction of individual particles in crowd to detect unusual behavior. Our method describes collective dissipative interactions among particles in a crowd scene. We reveal the changing patterns about the crowd behavior states, to support the conversion between different social behaviors during evolution. To further improve the performance of our method, virtual reality can be considered to consolidate the acquisition of data associated with a particular behavior. Therefore, we provide theoretical background of immersive implication considering virtual reality that can expose individuals to virtual crowds and acquire useful data on human motion and behaviors in crowds. The experimental evaluation of our energy and social interaction driven method shows convincing results.

Tumor segmentation in Computed Tomography (CT) images is a crucial step in image-guided surgery. However, low-contrast CT images impede the performance of subsequent segmentation tasks. Contrast enhancement is then used as a preprocessing step to highlight the relevant structures, thus facilitating not only medical diagnosis but also image segmentation with higher accuracy. In this paper, we propose a goal-oriented contrast enhancement method to improve tumor segmentation performance. The proposed method is based on two concepts, namely guided image enhancement and image quality control through an optimization scheme. The proposed OPTimized Guided Contrast Enhancement (OPTGCE) scheme exploits both contextual information from the guidance image and structural information from the input image in a two-step process. The first step consists of applying a two-dimensional histogram specification exploiting contextual information in the corresponding guidance image, i.e. Magnetic Resonance Image (MRI). In the second step, an optimization scheme using a structural similarity measure to preserve the structural information of the original image is performed. To the best of our knowledge, this kind of contrast enhancement optimization scheme using cross-modal guidance is proposed for the first time in the medical imaging context. The experimental results obtained on real data demonstrate the effectiveness of the method in terms of enhancement and segmentation quality in comparison to some state-of-the-art methods based on the histogram.

The industrial demands of immersive videos for virtual reality/augmented reality applications are crescendo, where the video stream provides a choice to the user viewing object of interest with the illusion of “being there”. However, in industry 4.0, streaming of such huge-sized video over the network consumes a tremendous amount of bandwidth, where the users are only interested in specific regions of the immersive videos. Furthermore, for delivering full excitement videos and minimizing the bandwidth consumption, the automatic selection of the user’s Region of Interest in a 360∘ video is very challenging because of subjectivity and difference in contentment. To tackle these challenges, we employ two efficient convolutional neural networks for salient object detection and memorability computation in a unified framework to find the most prominent portion of a 360∘ video. The proposed system is four-fold: preprocessing, intelligent visual interest predictor, final viewport selection, and virtual camera steerer. Firstly, an input 360∘ video frame is split into three Field of Views (FoVs), each with a viewing angle of 120∘. Next, each FoV is passed to object detection and memorability prediction model for visual interestingness computation. Further, the FoV is supplied as a viewport, containing the most salient and memorable objects. Finally, a virtual camera steerer is designed using enriched salient features from YOLO and LSTM that are forwarded to the dense optical flow to follow the salient object inside the immersive video. Performance evaluation of the proposed system over our own collected data from various websites as well as on public datasets indicates the effectiveness for diverse categories of 360∘ videos and helps in the minimization of the bandwidth usage, making it suitable for industry 4.0 applications.

Laparoscopic images and videos are often affected by different types of distortion like noise, smoke, blur and nonuniform illumination. Automatic detection of these distortions, followed generally by application of appropriate image quality enhancement methods, is critical to avoid errors during surgery. In this context, a crucial step involves an objective assessment of the image quality, which is a two-fold problem requiring both the classification of the distortion type affecting the image and the estimation of the severity level of that distortion. Unlike existing image quality measures which focus mainly on estimating a quality score, we propose in this paper to formulate the image quality assessment task as a multi-label classification problem taking into account both the type as well as the severity level (or rank) of distortions. Here, this problem is then solved by resorting to a deep neural networks based approach. The obtained results on a laparoscopic image dataset show the efficiency of the proposed approach., Comment: 5 Pages, ICIP 2020

From the beginning of civilizations, transportation has been one of the most important requirements for humans. Over the years, it has been evolved to modern transportation systems such as road, train, and air transportation. With the development of technology, intelligent transportation systems have been enriched with Information and Communications Technology (ICT). Nowadays, smart city concept that integrates ICT and Internet-of-Things (IoT) have been appeared to optimize the efficiency of city operations and services. Recently, several IoT-based smart applications for smart cities have been developed. Among these applications, smart services for transportation are highly required to ease the issues especially regarding to road safety. In this context, this study presents a literature review that elaborates the existing IoT-based smart transportation systems especially in terms of road safety. In this way, the current state of IoT-based smart transportation systems for safer roads are provided. Then, the current research efforts undertaken by the authors to provide an IoT-based safe smart traffic system are briefly introduced. It is emphasized that road safety can be improved using Vehicle-to-Infrastructure (V2I) communication technologies via the cloud (Infrastructure-to-Cloud – I2C). Therefore, it is believed that this study offers useful information to researchers for developing safer roads in smart cities.

Among various physiological signal acquisition methods for the study of the human brain, EEG (Electroencephalography) is more effective. EEG provides a convenient, non-intrusive, and accurate way of capturing brain signals in multiple channels at fine temporal resolution. We propose an ensemble learning algorithm for automatically computing the most discriminative subset of EEG channels for internal emotion recognition. Our method describes an EEG channel using kernel-based representations computed from the training EEG recordings. For ensemble learning, we formulate a graph embedding linear discriminant objective function using the kernel representations. The objective function is efficiently solved via sparse non-negative principal component analysis and the final classifier is learned using the sparse projection coefficients. Our algorithm is useful in reducing the amount of data while improving computational efficiency and classification accuracy at the same time. The experiments on publicly available EEG dataset demonstrate the superiority of the proposed algorithm over the compared methods. Open Access

The success of minimally invasive interventions and the remarkable technological and medical progress have made endoscopic image enhancement a very active research field. Due to the intrinsic endoscopic domain characteristics and the surgical exercise, stereo endoscopic images may suffer from different degradations which affect its quality. Therefore, in order to provide the surgeons with a better visual feedback and improve the outcomes of possible subsequent processing steps, namely, a 3-D organ reconstruction/registration, it would be interesting to improve the stereo endoscopic image quality. To this end, we propose, in this paper, two joint enhancement methods which operate in the wavelet transform domain. More precisely, by resorting to a joint wavelet decomposition, the wavelet subbands of the right and left views are simultaneously processed to exploit the binocular vision properties. While the first proposed technique combines only the approximation subbands of both views, the second method combines all the wavelet subbands yielding an inter-view processing fully adapted to the local features of the stereo endoscopic images. Experimental results, carried out on various stereo endoscopic datasets, have demonstrated the efficiency of the proposed enhancement methods in terms of perceived visual image quality.

The widespread use of 3D acquisition and display technologies has increased the interest of stereo image dataset in various application fields. As a result, it becomes necessary to have an efficient 3D quality assessment method to measure the human perception of stereoscopic images. While most of the state-of-the-art methods belong to the class of full-reference methods which require the original stereo images to be able to assess the quality, we propose in this paper a no-reference quality metric which does not require any information of the original stereo images. The proposed method operates in the wavelet transform domain and adopts a statistical framework to predict the quality of stereo images. More precisely, a joint wavelet decomposition is first performed on the stereo images to exploit simultaneously the intra and inter-views redundancies. A wavelet transform is also applied to their associated estimated disparity maps. Then, relevant features are extracted from the resulting wavelet subbands by resorting to appropriate statistical models. Simulations, carried out on the standard Live 3D image quality database, show that our proposed design model achieves significant improvement compared to the state-of-the-art 3D quality assessment methods. We present a novel no-reference method for stereo image quality assessment.The developed method operates in the wavelet transform domain.A joint multiscale decomposition is applied to the left and right views.Statistical features of both views as well as the disparity map are extracted.

Background and objective: Medical image segmentation plays a vital role in medical image analysis. There are many algorithms developed for medical image segmentation which are based on edge or region characteristics. These are dependent on the quality of the image. The contrast of a CT or MRI image plays an important role in identifying region of interest i.e. lesion(s). In order to enhance the contrast of image, clinicians generally use manual histogram adjustment technique which is based on 1D histogram specification. This is time consuming and results in poor distribution of pixels over the image. Cross modality based contrast enhancement is 2D histogram specification technique. This is robust and provides a more uniform distribution of pixels over CT image by exploiting the inner structure information from MRI image. This helps in increasing the sensitivity and accuracy of lesion segmentation from enhanced CT image. The sequential implementation of cross modality based contrast enhancement is slow. Hence we propose GPU acceleration of cross modality based contrast enhancement for tumor segmentation. Methods: The aim of this study is fast parallel cross modality based contrast enhancement for CT liver images. This includes pairwise 2D histogram, histogram equalization and histogram matching. The sequential implementation of the cross modality based contrast enhancement is computationally expensive and hence time consuming. We propose persistence and grid-stride loop based fast parallel contrast enhancement for CT liver images. We use enhanced CT liver image for the lesion or tumor segmentation. We implement the fast parallel gradient based dynamic seeded region growing for lesion segmentation. Results: The proposed parallel approach is 104.416 ( ± 5.166) times faster compared to the sequential implementation and increases the sensitivity and specificity of tumor segmentation. Conclusion: The cross modality approach is inspired by 2D histogram specification which incorporates spatial information existing in both guidance and input images for remapping the input image intensity values. The cross modality based liver contrast enhancement improves the quality of tumor segmentation.

Animal behavior analysis is a crucial task for the industrial farming. In an indoor farm setting, extracting Key joints of animals is essential for tracking the animal for a longer period of time. In this paper, we proposed a deep network that exploits transfer learning to train the network for the pig skeleton extraction in an end to end fashion. The backbone of the architecture is based on an hourglass stacked dense-net. In order to train the network, keyframes are selected from the test data using K-mean sampler. In total, 9 Keypoints are annotated that gives a brief detailed behavior analysis in the farm setting. Extensive experiments are conducted and the quantitative results show that the network has the potential of increasing the tracking performance by a substantial margin.

An effective contrast enhancement method should not only improve the perceptual quality of an image but should also avoid adding any artifacts or affecting naturalness of images. This makes Contrast Enhancement Evaluation (CEE) a challenging task in the sense that both the improvement in image quality and unwanted side-effects need to be checked for. Currently, there is no single CEE metric that works well for all kinds of enhancement criteria. In this paper, we propose a new Multi-Criteria CEE (MCCEE) measure which combines different metrics effectively to give a single quality score. In order to fully exploit the potential of these metrics, we have further proposed to apply them on the decomposed image using wavelet transform. This new metric has been tested on two natural image contrast enhancement databases as well as on medical Computed Tomography (CT) images. The results show a substantial improvement as compared to the existing evaluation metrics. The code for the metric is available at: https://github.com/zakopz/MCCEE-Contrast-Enhancement-Metric

Head detection in real-world videos is a classical research problem in computer vision. Head detection in videos is challenging than in a single image due to many nuisances that are commonly observed in natural videos, including arbitrary poses, appearances, and scales. Generally, head detection is treated as a particular case of object detection in a single image. However, the performance of object detectors deteriorates in unconstrained videos. In this paper, we propose a temporal consistency model (TCM) to enhance the performance of a generic object detector by integrating spatial-temporal information that exists among subsequent frames of a particular video. Generally, our model takes detection from a generic detector as input and improves mean average precision (mAP) by recovering missed detection and suppressing false positives. We compare and evaluate the proposed framework on four challenging datasets, i.e., HollywoodHeads, Casablanca, BOSS, and PAMELA. Experimental evaluation shows that the performance is improved by employing the proposed TCM model. We demonstrate both qualitatively and quantitatively that our proposed framework obtains significant improvements over other methods. Copyright © 2020 Sultan Daud Khan et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

We propose a novel method for crowd video classification, based on a two-stream convolutional architecture which incorporates spatial and temporal networks. Our proposed method cope with the key challenge of capturing the complementary information on appearance from still frames and motion between frames. In our proposed method, a motion flow field is obtained from the video through dense optical flow. We demonstrate that the proposed method trained on information including dense optical flow achieves significant improvement in performance. We train and evaluate our proposed method on a benchmark crowd video dataset. The experimental results of our method show that it outperforms the reference methods. Refereed/Peer-reviewed

Detecting heads with full variations in camera view points, human poses, appearances, and scales is a key problem for many computer vision applications. Region convolutional neural networks (RCNN) achieved considerable success in handling the variances in poses and appearances. However, RCNN are inefficient in handling human heads of diverse scales. In this work, we focus on detecting human heads in complex scenes. Starting with traditional RCNN model, we extend it by leveraging person-scene relations and propose a dimension invariant convolutional neural network (DCNN) that coarsely predicts locations and scales of heads directly from the full image. We evaluate and compare our method with famous methods by using two benchmarks datasets. Experimental results show that our method outperforms these methods.

Sensor-based human activity recognition is an essential task for automatic behavior analysis for sports player, senior citizens, and IoT applications. The traditional approaches are based on hand-crafted features which use fixed mathematical rules to extract the features from the input data and are not capable of incremental learning. In this paper, we proposed a stacked long Short-term memory (LSTM) network for recognizing six human behaviors from the smartphone data. The network consists of a five LSTM cell that is trained end-to-end on the sensor data. The network is preceded by a single layer neural network that pre-processes the data for the stacked LSTM network. An L 2 regularizer is used in the cost function which helps the network in generalization. The network is evaluated on public domain UCI dataset and quantitative results are compared against six state-of-the-art methods. The performance is calculated in terms of precision-recall and the average accuracy. The proposed network improves the average accuracy by 0.93% as compared to the closest state-of-the-art method without any manual feature engineering.

People counting in sports venues is emerging as a new domain in the field of video surveillance. People counting in these venues faces many key challenges, such as severe occlusions, few pixels per head, and significant variations in person's head sizes due to wide sport areas. We propose a deep model based method, which works as a head detector and takes into consideration the scale variations of heads in videos. Our method is based on the notion that head is the most visible part in the sports venues where large number of people are gathered. To cope with the problem of different scales, we generate scale aware head proposals based on scale map. Scale aware proposals are then fed to the Convolutional Neural Network (CNN) and it provides a response matrix containing the presence probabilities of people observed across scene scales. We then use non-maximal suppression to get the accurate head positions. For the performance evaluation, we carry out extensive experiments on two standard datasets and compare the results with state-of-the-art (SoA) methods. The results in terms of Average Precision (AvP), Average Recall (AvR), and Average F1-Score (AvF-Score) show that our method is better than SoA methods.

The object sizes in images are diverse, therefore, capturing multiple scale context information is essential for semantic segmentation. Existing context aggregation methods such as pyramid pooling module (PPM) and atrous spatial pyramid pooling (ASPP) employ different pooling size or atrous rate, such that multiple scale information is captured. However, the pooling sizes and atrous rates are chosen empirically. Rethinking of ASPP leads to our observation that learnable sampling locations of the convolution operation can endow the network learnable fieldof- view, thus the ability of capturing object context information adaptively. Following this observation, in this paper, we propose an adaptive context encoding (ACE) module based on deformable convolution operation where sampling locations of the convolution operation are learnable. Our ACE module can be embedded into other Convolutional Neural Networks (CNNs) easily for context aggregation. The effectiveness of the proposed module is demonstrated on Pascal-Context and ADE20K datasets. Although our proposed ACE only consists of three deformable convolution blocks, it outperforms PPM and ASPP in terms of mean Intersection of Union (mIoU) on both datasets. All the experimental studies confirm that our proposed module is effective compared to the state-of-the-art methods.

International audience; In this paper, we propose a saliency-weighted stereoscopic JND (SSJND) model constructed based on psychophysical experiments, accounting for binocular disparity and spatial masking effects of the human visual system (HVS). Specifically, a disparity-aware binocular JND model is first developed using psychophysical data, and then is employed to estimate the JND threshold for non-occluded pixel (NOP). In addition, to derive a reliable 3D-JND prediction, we determine the visibility threshold for occluded pixel (OP) by including a robust 2D-JND model. Finally, SSJND thresholds of one view are obtained by weighting the resulting JND for NOP and OP with their visual saliency. Based on subjective experiments, we demonstrate that the proposed model outperforms the other 3D-JND models in terms of perceptual quality at the same noise level.

Computer-assisted systems for planning and navigation of liver resection procedures rely on the use of patient-specific 3D geometric models obtained from computed tomography. In this work, we propose the application of Poisson surface reconstruction (PSR) to obtain 3D models of the liver surface with applications to planning and navigation of liver surgery. In order to apply PSR, the introduction of an efficient transformation of the segmentation data, based on computation of gradient fields, is proposed. One of the advantages of PSR is that it requires only one control parameter, allowing the process to be fully automatic once the optimal value is estimated. Validation of our results is performed via comparison with 3D models obtained by state-of-art Marching Cubes incorporating Laplacian smoothing and decimation (MCSD). Our results show that PSR provides smooth liver models with better accuracy/complexity trade-off than those obtained by MCSD. After estimating the optimal parameter, automatic reconstruction of liver surfaces using PSR is achieved keeping similar processing time as MCSD. Models from this automatic approach show an average reduction of 79.59% of the polygons compared to the MCSD models presenting similar smoothness properties. Concerning visual quality, on one hand, and despite this reduction in polygons, clinicians perceive the quality of automatic PSR models to be the same as complex MCSD models. On the other hand, clinicians perceive a significant improvement on visual quality for automatic PSR models compared to optimal (obtained in terms of accuracy/complexity) MCSD models. The median reconstruction error using automatic PSR was as low as 1.03±0.23mm, which makes the method suitable for clinical applications. Automatic PSR is currently employed at Oslo University Hospital to obtain patient-specific liver models in selected patients undergoing laparoscopic liver resection.

In minimally invasive surgery, smoke generated by such as electrocautery and laser ablation deteriorates image quality severely. This creates discomfortable view for the surgeon which may increase surgical risk and degrade the performance of computer assisted surgery algorithms such as segmentation, reconstruction, tracking, etc. Therefore, real-time smoke removal is required to keep a clear field of view. In this paper, we propose a real-time smoke removal approach based on Convolutional Neural Network (CNN). An encoder-decoder architecture with Laplacian image pyramid decomposition input strategy is proposed. This is an end-to-end network which takes the smoke image and its Laplacian image pyramid decomposition as inputs, and outputs a smoke free image directly without relying on any physical models or estimation of intermediate parameters. This design can be further embedded to deep learning based follow-up image guided surgery processes such as segmentation and tracking tasks easily. A dataset with synthetic smoke images generated from Blender and Adobe Photoshop is employed for training the network. The result is evaluated quantitatively on synthetic images and qualitatively on a laparoscopic dataset degraded with real smoke. Our proposed method can eliminate smoke effectively while preserving the original colors and reaches 26 fps for a video of size 512 × 512 on our training machine. The obtained results not only demonstrate the efficiency and effectiveness of the proposed CNN structure, but also prove the potency of training the network on synthetic dataset. © 2019 Society of Photo Optical Instrumentation Engineers. One print or electronic copy may be made for personal use only. Systematic reproduction and distribution, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper are prohibited.

Human brain undergoes state changes during sleep which produces distinctive signal patterns when recorded by electroencephalography (EEG). Automatic identification of these stages is crucial to diagnosing and treating sleep related disorders. We propose an image processing based technique for automatic identification of sleep stages from EEG signals. We generate two dimensional image representations from the high dynamic range Fourier transform features of the one dimensional EEG signals. Using these representations, we learn a deep and dense convolutional neural network (CNN) model for prediction. The key advantage of the proposed method is its seamless use of the existing well studied and powerful deep CNN models designed for computer vision problems. Experiments on the popular Sleep-EDF database show that the proposed method significantly outperforms the compared methods for automatic sleep stage identification Refereed/Peer-reviewed

People counting in high density crowds is emerging as a new frontier in crowd video surveillance. Crowd counting in high density crowds encounters many challenges, such as severe occlusions, few pixels per head, and large variations in person's head sizes. In this paper, we propose a novel Density Independent and Scale Aware model (DISAM), which works as a head detector and takes into account the scale variations of heads in images. Our model is based on the intuition that head is the only visible part in high density crowds. In order to deal with different scales, unlike off-the-shelf Convolutional Neural Network (CNN) based object detectors which use general object proposals as inputs to CNN, we generate scale aware head proposals based on scale map. Scale aware proposals are then fed to the CNN and it renders a response matrix consisting of probabilities of heads. We then explore non-maximal suppression to get the accurate head positions. We conduct comprehensive experiments on two benchmark datasets and compare the performance with other state-of-theart methods. Our experiments show that the proposed DISAM outperforms the compared methods in both frame-level and pixel-level comparisons. © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Low contrast Computed Tomographic (CT) images often hamper the diagnosis of critical tumors found in various human organs. Contrast enhancement schemes play significant role in improving the visualization of these structures. To achieve this objective, Crossmodality Guided Enhancement (CMGE) method is proposed in this paper. The idea is to exploit the diversity of the information extracted from one modality to enhance the important structures including vessels and tumors in another modality. Our method employs information from liver Magnetic Resonance Image (MRI) to generate an enhanced CT image. It entails applying two dimensional histogram specification to map 2D histogram of CT to that of MRI followed by application of top and bottom hat transformations. These morphological operations highlight areas brighter than their surroundings and suppress darker areas. The final image is obtained by combining the results of these operations. Our method is compared with other state of the art contrast enhancement methods both visually and in terms of quality assessment metrics IEM and EME. The results show that our method performs better than these methods. CMGE technique yields improved contrast in low contrast CT images of the human liver and highlights tumors and vessels. © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Just noticeable difference (JND) for stereoscopic 3D content reflects the maximum tolerable distortion; it corresponds to the visibility threshold of the asymmetric distortions in the left and right contents. The 3D-JND models can be used to improve the efficiency of the 3D compression or the 3D quality assessment. Compared to 2D-JND models, the 3D-JND models appeared recently and the related literature is rather limited. In this paper, we give a deep and comprehensive study of the pixel-based 3D-JND models. To our best knowledge, this is the first review on 3D-JND models. Each model is briefly described by giving its rationale and main components in addition to providing exhaustive information about the targeted application, the pros, and cons. Moreover, we present the characteristics of the human visual system presented in these models. In addition, we analyze and compare the 3D-JND models thoroughly using qualitative and quantitative performance evaluation based on Middlebury stereo datasets. Besides, we measure the JND thresholds of the asymmetric distortion based on psychophysical experiments and compare these experimental results to the estimates from the 3D-JND models in order to evaluate the accuracy of each model. Copyright the Author(s). Open Access under a CC BY 4.0 license. Users are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles, or use them for any other lawful purpose, as long as proper attribution is given.

The recent advances in 3D acquisition and display technologies have led to the use of stereoscopy for a wide range of applications. The quality assessment of such stereo data becomes of great interest especially when the reference image is not available. For this reason, we propose in this paper a no-reference 3D image quality assessment algorithm based on joint statistical modeling of the wavelet subband coefficients of the stereo pairs. More precisely, we resort to bivariate and multivariate statistical modeling of the texture images to build efficient statistical features. These features are then combined with the depth ones and used to predict the quality score based on machine learning tools. The proposed methods are evaluated on LIVE 3D database and the obtained results show the good performance of joint statistical modeling based approaches.

International audience; No-reference (NR) stereoscopic 3D (S3D) image quality assessment (SIQA) is still challenging due to the poor understanding of how the human visual system (HVS) judges image quality based on binocular vision. In this paper, we propose an efficient opinion-aware NR Stereoscopic Quality predictor based on local contrast statistics combination (SQSC). Specifically, for left and right views, we first extract statistical features of the gradient magnitude (GM) and Laplacian of Gaussian (LoG) responses, describing the image local structures from different perspectives. The HVS is insensitive to low-order statistical redundancies that can be removed by LoG filtering. Hence, the monocular statistical features are then fused to derive the binocular features based on a linear combination model using LoG responses-based weightings. These weightings can efficiently simulate the binocular rivalry (BR) phenomenon. Finally, the binocular features and the subjective scores were jointly employed to construct a learned regression model obtained by the support vector regression (SVR) algorithm. Experimental results on three widely used 3D IQA databases demonstrate the high prediction performance of the proposed method when compared to recent well performing SIQA methods.

During video-guided minimally invasive surgery, quality of frames may be degraded severely by cauterization-induced smoke and condensation of vapor. This degradation of quality creates discomfort for the operating surgeon, and causes serious problems for automatic follow-up processes such as registration, segmentation and tracking. This paper proposes a novel deep neural network based smoke removal solution that is able to enhance the quality of surgery video frames in real-time. It employs synthetically generated training dataset including smoke embedded and clean reference versions. Results calculated on the test set indicate that our network outperforms previous defogging methods in terms of quantitative and qualitative measures. While eliminating apparent smoke, it also successfully preserves the natural appearance of tissue surface. To the best of our knowledge, the presented method is the first deep neural network based approach for the surgical field smoke removal problem. © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

We propose a deep discriminative appearance model (DDAM) based on convolutional neural network (CNN) for pedestrians. The training stage of our supervised D-DAM model does not depend on a large amount of data. In our model, we introduce a progressive batch refinement technique to fine tune the CNN for modeling the appearance of the pedestrian. After fine-tuning, the model achieves 98% accuracy for pedestrian and non-pedestrian classification. Moreover, we also introduce a novel discrimination index (DI) for evaluating the spatio-temporal discrimination effectiveness of both hand-crafted and deep features. We perform experiments on pre-trained CNN model, our D-DAM model, and 3 baseline hand-crafted features including HoG, LBP, and Color histogram. The results show that our D-DAM model achieves higher classification accuracy and better spatio-temporal discrimination ability compared to all the hand-crafted features.

We propose a Directed Sparse Graphical Model (DSGM) for multi-target tracking. In the category of global optimization for multi-target tracking, traditional approaches have two main drawbacks. First, a cost function is defined in terms of the linear combination of the spatial and appearance constraints of the targets which results a highly non-convex function. And second, a very dense graph is constructed to capture the global attribute of the targets. In such a graph, It is impossible to find reliable tracks in polynomial time unless some relaxation and heuristics are used. To address these limitations, we proposed DSGM which finds a set of reliable tracks for the targets without any heuristics or relaxation and keeps the computational complexity very low through the design of the graph. Irrespective of traditional approaches where spatial and appearance constraints are added up linearly with a given weight factor, we incorporated these constraints in a cascaded fashion. First, we exploited a Hidden Markov Model (HMM) for the spatial constraints of the target and obtain most probable locations of the targets in a segment of video. Afterwards, a deep feature based appearance model is used to generate the sparse graph. The track for each target is found through dynamic programming. Experiments are performed on 3 challenging sports datasets (football, basketball and sprint) and promising results are achieved.

In this paper, we present efficient bit allocation methods for stereo image coding purpose. Since the common idea behind most of the existing stereo compression schemes consists of encoding a reference and residual images as well as a disparity map, we mainly focus on the bit allocation issue between the reference and residual images. Generally, this problem is solved in an empirical manner by looking for the optimal rates leading to the minimum distortion value. Thanks to recent approximations of the entropy and distortion functions, we propose accurate and fast bit allocation schemes appropriate for the open-loop- and closed-loop-based stereo coding structures. Experimental results show the benefits which can be drawn from the proposed bit allocation methods.

Laparoscopic surgery has a limited field of view. Laser ablation in a laproscopic surgery causes smoke, which inevitably influences the surgeon's visibility. Therefore, it is of vital importance to remove the smoke, such that a clear visualization is possible. In order to employ a desmoking technique, one needs to know beforehand if the image contains smoke or not, to this date, there exists no accurate method that could classify the smoke/non-smoke images completely. In this work, we propose a new enhancement method which enhances the informative details in the RGB images for discrimination of smoke/non-smoke images. Our proposed method utilizes weighted least squares optimization framework~(WLS). For feature extraction, we use statistical features based on bivariate histogram distribution of gradient magnitude~(GM) and Laplacian of Gaussian~(LoG). We then train a SVM classifier with binary smoke/non-smoke classification task. We demonstrate the effectiveness of our method on Cholec80 dataset. Experiments using our proposed enhancement method show promising results with improvements of 4\% in accuracy and 4\% in F1-Score over the baseline performance of RGB images. In addition, our approach improves over the saturation histogram based classification methodologies Saturation Analysis~(SAN) and Saturation Peak Analysis~(SPA) by 1/5\% and 1/6\% in accuracy/F1-Score metrics., Comment: In proceedings of IST, Color and Imaging Conference (CIC 26). Congcong Wang and Vivek Sharma contributed equally to this work and listed in alphabetical order

Articulation modeling, feature extraction, and classification are the important components of pedestrian segmentation. Usually, these components are modeled independently from each other and then combined in a sequential way. However, this approach is prone to poor segmentation if any individual component is weakly designed. To cope with this problem, we proposed a spatio-temporal convolutional neural network named PedNet which exploits temporal information for spatial segmentation. The backbone of the PedNet consists of an encoder&ndash, decoder network for downsampling and upsampling the feature maps, respectively. The input to the network is a set of three frames and the output is a binary mask of the segmented regions in the middle frame. Irrespective of classical deep models where the convolution layers are followed by a fully connected layer for classification, PedNet is a Fully Convolutional Network (FCN). It is trained end-to-end and the segmentation is achieved without the need of any pre- or post-processing. The main characteristic of PedNet is its unique design where it performs segmentation on a frame-by-frame basis but it uses the temporal information from the previous and the future frame for segmenting the pedestrian in the current frame. Moreover, to combine the low-level features with the high-level semantic information learned by the deeper layers, we used long-skip connections from the encoder to decoder network and concatenate the output of low-level layers with the higher level layers. This approach helps to get segmentation map with sharp boundaries. To show the potential benefits of temporal information, we also visualized different layers of the network. The visualization showed that the network learned different information from the consecutive frames and then combined the information optimally to segment the middle frame. We evaluated our approach on eight challenging datasets where humans are involved in different activities with severe articulation (football, road crossing, surveillance). The most common CamVid dataset which is used for calculating the performance of the segmentation algorithm is evaluated against seven state-of-the-art methods. The performance is shown on precision/recall, F 1 , F 2 , and mIoU. The qualitative and quantitative results show that PedNet achieves promising results against state-of-the-art methods with substantial improvement in terms of all the performance metrics.

Background In laparoscopic surgery, image quality can be severely degraded by surgical smoke, which not only introduces errors for the image processing algorithms (used in image guided surgery), but also reduces the visibility of the observed organs and tissues. To overcome these drawbacks, this work aims to remove smoke in laparoscopic images using an image preprocessing method based on a variational approach. Methods In this paper, we present the physical smoke model where the degraded image is separated into two parts: direct attenuation and smoke veil and propose an efficient variational-based desmoking method for laparoscopic images. To estimate the smoke veil, the proposed method relies on the observation that smoke veil has low contrast and low inter-channel differences. A cost function is defined based on this prior knowledge and is solved using an augmented Lagrangian method. The obtained smoke veil is then subtracted from the original degraded image, resulting in the direct attenuation part. Finally, the smoke free image is computed using a linear intensity transformation of the direct attenuation part. Results The performance of the proposed method is evaluated quantitatively and qualitatively using three datasets: two public real smoked laparoscopic datasets and one generated synthetic dataset. No-reference and reduced-reference image quality assessment metrics are used with the two real datasets, and show that the proposed method outperforms the state-of-the-art ones. Besides, standard full-reference ones are employed with the synthetic dataset, and indicate also the good performance of the proposed method. Furthermore, the qualitative visual inspection of the results shows that our method removes smoke effectively from the laparoscopic images. Conclusion All the obtained results show that the proposed approach reduces the smoke effectively while preserving the important perceptual information of the image. This allows to provide a better visualization of the operation field for surgeons and improve the image guided laparoscopic surgery procedure.

In this paper we propose a novel Computational Attention Models (CAM) that fuses bottom-up, top-down and salient motion visual cues to compute visual salience in surveillance videos. When dealing with a number of visual features/cues in a system, it is always challenging to combine or fuse them. As there is no commonly agreed natural way of combining different conspicuity maps obtained from different features: face and motion for example, the challenge is thus to find the right mix of visual cues to get a salience map that is the closest to a corresponding gaze map? In the literature many CAMs have used fixed weights for combining different visual cues. This is computationally attractive but is a very crude way of combining the different cues. Furthermore, the weights are typically set in an ad hoc fashion. Therefore in this paper, we propose a machine learning approach, using an Artificial Neural Network (ANN) to estimate these weights. The ANN is trained using gaze maps, obtained by eye tracking in psycho-physical experiments. These weights are then used to combine the conspicuities of the different visual cues in our CAM, which is later applied to surveillance videos. The proposed model is designed in a way to consider important visual cues typically present in surveillance videos, and to combine their conspicuities via ANN. The obtained results are encouraging and show a clear improvement over state-of-the-art CAMs.

We propose a novel Hierarchical Feature Model (HFM) for multi-target tracking. The traditional tracking algorithms use handcrafted features that cannot track targets accurately when the target model changes due to articulation, illumination intensity variation or perspective distortions. Our HFM explore deep features to model the appearance of targets. Then, we use an unsupervised dimensionality reduction for sparse representation of the feature vectors to cope with the time-critical nature of the tracking problem. Subsequently, a Bayesian filter is adopted as the tracker and a discrete combinatorial optimization is considered for target association. We compare our proposed HFM against 4 state-of-the-art trackers using 4 benchmark datasets. The experimental results show that our HFM outperforms all the state-of-the-art methods in terms of both Multi Object Tracking Accuracy (MOTA) and Multi Object Tracking Precision (MOTP).

One of the most challenging issues in stereoscopic image quality assessment (SIQA) is how to effectively model the binocular behavior of the human visual system (HVS). The latter has a great impact on the perceptual 3D quality. In this paper, we propose a SIQA metric accounting for binocular combination properties and disparity information. Instead of computing the quality of the left and the right views separately, the proposed metric predicts the quality of a cyclopean image so as to have a good consistency with 3D human perception. The cyclopean image is synthesized based on the local entropy and the visual saliency of each view with the aim to simulate the phenomena of binocular fusion/rivalry. A 2D IQA metric is employed to assess the quality of both the cyclopean image and the disparity map. The obtained scores are used to derive the 3D quality score thanks to a pooling stage. Experimental results on three public 3D IQA databases show that the proposed method outperforms many other state-of-the-art SIQA methods, and achieves high prediction accuracy on these databases.

One of the most challenging issues in stereoscopic image quality assessment (IQA) is how to effectively model the binocular behaviors of the human visual system (HVS). The latter has a great impact on the perceptual stereoscopic 3D (S3D) quality. This paper presents a stereoscopic IQA metric based on the properties of the HVS. Instead of measuring the quality of the left and the right views separately, the proposed method predicts the quality of a cyclopean image to ensure that the overall S3D quality is as close as possible to the binocular vision. The cyclopean image is synthesized based on the local entropy of each view with the aim to simulate the phenomena of the binocular rivalry/suppression. A 2D IQA metric is employed to assess the quality of both the cyclopean image and the disparity map. Additionally, the quality of the cyclopean image is modulated according to the visual importance of each pixel defined by the just noticeable difference (JND). Finally, the 3D quality score is derived by combining the quality estimates of the cyclopean image and disparity map. Experimental results show that the proposed method outperforms many other state-of-the-art SIQA methods in terms of prediction accuracy and computational efficiency.