Search

Deep learning-based methods, in particular, convolutional neural networks and fully convolutional networks are now widely used in the medical image analysis domain. The scope of this review focuses on the analysis using deep learning of focal liver lesions, with a special interest in hepatocellular carcinoma and metastatic cancer; and structures like the parenchyma or the vascular system. Here, we address several neural network architectures used for analyzing the anatomical structures and lesions in the liver from various imaging modalities such as computed tomography, magnetic resonance imaging and ultrasound. Image analysis tasks like segmentation, object detection and classification for the liver, liver vessels and liver lesions are discussed. Based on the qualitative search, 91 papers were filtered out for the survey, including journal publications and conference proceedings. The papers reviewed in this work are grouped into eight categories based on the methodologies used. By comparing the evaluation metrics, hybrid models performed better for both the liver and the lesion segmentation tasks, ensemble classifiers performed better for the vessel segmentation tasks and combined approach performed better for both the lesion classification and detection tasks. The performance was measured based on the Dice score for the segmentation, and accuracy for the classification and detection tasks, which are the most commonly used metrics.

Cross-modal medical imaging techniques are predominantly being used in the clinical suite. The ensemble learning methods using cross-modal medical imaging adds reliability to several medical image analysis tasks. Motivated by the performance of deep learning in several medical imaging tasks, a deep learning-based denoising method Cross-Modality Guided Denoising Network CMGDNet for removing Rician noise in T1-weighted (T1-w) Magnetic Resonance Images (MRI) is proposed in this paper. CMGDNet uses a guidance image, which is a cross-modal (T2-w) image of better perceptual quality to guide the model in denoising its noisy T1-w counterpart. This cross-modal combination allows the network to exploit complementary information existing in both images and therefore improve the learning capability of the model. The proposed framework consists of two components: Paired Hierarchical Learning (PHL) module and Cross-Modal Assisted Reconstruction (CMAR) module. PHL module uses Siamese network to extract hierarchical features from dual images, which are then combined in a densely connected manner in the CMAR module to finally reconstruct the image. The impact of using registered guidance data is investigated in removing noise as well as retaining structural similarity with the original image. Several experiments were conducted on two publicly available brain imaging datasets available on the IXI database. The quantitative assessment using Peak Signal to noise ratio (PSNR), Structural Similarity Index (SSIM), and Feature Similarity Index (FSIM) demonstrates that the proposed method exhibits 4.7% and 2.3% gain (average), respectively, in SSIM and FSIM values compared to other state-of-the-art denoising methods that do not integrate cross-modal image information in removing various levels of noise.

Deep Learning-based chest Computed Tomography (CT) analysis has been proven to be effective and efficient for COVID-19 diagnosis. Existing deep learning approaches heavily rely on large labeled data sets, which are difficult to acquire in this pandemic situation. Therefore, semi-supervised approaches are in demand. In this paper, we propose an end-to-end semi-supervised COVID-19 detection approach, ResNext+, that only requires volume level data labels and can provide slice level prediction. The proposed approach incorporates a lung segmentation mask as well as spatial and channel attention to extract spatial features. Besides, Long Short Term Memory (LSTM) is utilized to acquire the axial dependency of the slices. Moreover, a slice attention module is applied before the final fully connected layer to generate the slice level prediction without additional supervision. An ablation study is conducted to show the efficiency of the attention blocks and the segmentation mask block. Experimental results, obtained from publicly available datasets, show a precision of 81.9% and F1 score of 81.4%. The closest state-of-the-art gives 76.7% precision and 78.8% F1 score. The 5% improvement in precision and 3% in the F1 score demonstrate the effectiveness of the proposed method. It is worth noticing that, applying image enhancement approaches do not boost the performance of the proposed method, sometimes even harm the scores, although the enhanced images illustrate better perceived visual image quality. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/

Owing to its efficiency in the Internet of Things (IoT) applications in terms of low-power connectivity, IQRF (Intelligent Connectivity using Radio Frequency) technology appears to be one of the most reasonable IoT technologies in the commercial market. To realize emerging smart building applications using IQRF, it is necessary to study the propagation characteristics of IQRF technology in indoor environments. In this study, preliminary propagation measurements are conducted using IQRF transceivers that operate on the 868 MHz band in a peer-to-peer (P2P) configured system. The measurements are conducted both in a single corridor of a building in a Line-of-Sight (LoS) link and two perpendicular corridors in a Non-Line-of-Sight (NLoS) with one single knife-edge link. Moreover, the measured path loss values are compared with the predicted path loss values in order to comparatively assess the prediction accuracy of the well-known empirical models, such as log-distance, ITU, and WINNER II. According to the results, it is concluded that the ITU-1 path loss model agrees well with the measurements and could be used in the planning of an IQRF network deployment in a typical LoS corridor environment. For NLoS corridors, both ITU-3 and WINNERII-2 models could be used due to their higher prediction accuracy. We expect that the initial results achieved in this study could open new perspectives for future research on the development of smart building applications.

Graphs frequently appear as a type of data structure that efficiently models a set of interrelated objects as nodes and their relations as edges between them. Geometric deep learning emerged as a promising field for modeling non-Euclidean geometric data as graphs, Riemannian manifolds and meshes. In this paper, we review geometric deep learning based state-of-the-art approaches for multi-object tracking. We illustrate the main concept, the model architecture, loss functions, and optimization strategies of the main algorithms. Moreover, algorithms are quantitatively evaluated on standard performance metrics using the standard dataset. The potential of geometric deep learning in tracking is discussed and future directions are proposed.

An innovative e-learning project is presented in this paper, which is a mobile workbook that teaches handwriting at school. This mobile application proposes a new qualitative and quantitative analysis process of online cursive handwriting. It gives a real-time feedback, detects mistakes and helps teachers evaluate children’s writing skills. The main aim of this notebook is to aid kids learn how to write correctly. We analyze handwriting according to major criteria like shape, kinematics of the trace, position respect to the reference lines,stroke order and direction.

This paper describes an innovative e-learning project which is the development of a digital workbook that helps teaching handwriting at school. In this work, we propose a new qualitative and quantitative analysis process of cursive handwriting. This process detects automatically mistakes, gives a real-time feedback and helps teachers evaluate children’s writing skills. The main aim of this digital workbook is to help children learn how to write correctly. The proposed process is composed of five main criteria: shape,direction, stroke order, position respect to the reference lines and kinematics of the trace. It analyzes the handwriting quality and gives automatically feedback based on the Beta-Elliptic Model using similarity detection (SD) and dissimilarity distance (DD) measure. Our work apprehends dynamic and visual representation of the acquired traces and selects efficient features adapted to various handwriting styles and multiple script languages such as Arabic, Latin, digits, and symbols drawing. It demonstrates that beta-elliptic is not only a model for segmentation and recognition but also a tool to evaluate handwriting. Our application offers two interactive interfaces respectively dedicated to learners, and experts or teachers who can adapt it easily to the specificity of each child. The validation of the proposed system is done on a database collected in Tunisia primary schools with 400 children. Experimental results show that the efficiency and robustness of our suggested framework that do help teachers and children by offering positive feedback throughout the handwriting learning process using tactile digital devices.

Tumor segmentation in Computed Tomography (CT) images is a crucial step in image-guided surgery. However, low-contrast CT images impede the performance of subsequent segmentation tasks. Contrast enhancement is then used as a preprocessing step to highlight the relevant structures, thus facilitating not only medical diagnosis but also image segmentation with higher accuracy. In this paper, we propose a goal-oriented contrast enhancement method to improve tumor segmentation performance. The proposed method is based on two concepts, namely guided image enhancement and image quality control through an optimization scheme. The proposed OPTimized Guided Contrast Enhancement (OPTGCE) scheme exploits both contextual information from the guidance image and structural information from the input image in a two-step process. The first step consists of applying a two-dimensional histogram specification exploiting contextual information in the corresponding guidance image, i.e. Magnetic Resonance Image (MRI). In the second step, an optimization scheme using a structural similarity measure to preserve the structural information of the original image is performed. To the best of our knowledge, this kind of contrast enhancement optimization scheme using cross-modal guidance is proposed for the first time in the medical imaging context. The experimental results obtained on real data demonstrate the effectiveness of the method in terms of enhancement and segmentation quality in comparison to some state-of-the-art methods based on the histogram.

The industrial demands of immersive videos for virtual reality/augmented reality applications are crescendo, where the video stream provides a choice to the user viewing object of interest with the illusion of “being there”. However, in industry 4.0, streaming of such huge-sized video over the network consumes a tremendous amount of bandwidth, where the users are only interested in specific regions of the immersive videos. Furthermore, for delivering full excitement videos and minimizing the bandwidth consumption, the automatic selection of the user’s Region of Interest in a 360∘ video is very challenging because of subjectivity and difference in contentment. To tackle these challenges, we employ two efficient convolutional neural networks for salient object detection and memorability computation in a unified framework to find the most prominent portion of a 360∘ video. The proposed system is four-fold: preprocessing, intelligent visual interest predictor, final viewport selection, and virtual camera steerer. Firstly, an input 360∘ video frame is split into three Field of Views (FoVs), each with a viewing angle of 120∘. Next, each FoV is passed to object detection and memorability prediction model for visual interestingness computation. Further, the FoV is supplied as a viewport, containing the most salient and memorable objects. Finally, a virtual camera steerer is designed using enriched salient features from YOLO and LSTM that are forwarded to the dense optical flow to follow the salient object inside the immersive video. Performance evaluation of the proposed system over our own collected data from various websites as well as on public datasets indicates the effectiveness for diverse categories of 360∘ videos and helps in the minimization of the bandwidth usage, making it suitable for industry 4.0 applications.

Laparoscopic images and videos are often affected by different types of distortion like noise, smoke, blur and nonuniform illumination. Automatic detection of these distortions, followed generally by application of appropriate image quality enhancement methods, is critical to avoid errors during surgery. In this context, a crucial step involves an objective assessment of the image quality, which is a two-fold problem requiring both the classification of the distortion type affecting the image and the estimation of the severity level of that distortion. Unlike existing image quality measures which focus mainly on estimating a quality score, we propose in this paper to formulate the image quality assessment task as a multi-label classification problem taking into account both the type as well as the severity level (or rank) of distortions. Here, this problem is then solved by resorting to a deep neural networks based approach. The obtained results on a laparoscopic image dataset show the efficiency of the proposed approach., Comment: 5 Pages, ICIP 2020

Security and monitoring systems are more and more demanding in terms of quality, reliability and flexibility especially those dedicated to video surveillance. The quality of the acquired video signal strongly affects the performance of the high level tasks such as visual tracking, face detection and recognition. The design of a video quality assessment metric dedicated to this particular application requires a preliminary study on the common distortions encountered in video surveillance. To this end, we present in this paper a dataset dedicated to video quality assessment in the context of video surveillance. This database consists of a set of common distortions at different levels of annoyance. The subjective tests are performed using a classical pair comparison protocol with some new configurations. The subjective results obtained through the psycho-visual tests are analyzed and compared to some objective video quality assessment metrics. The preliminary results are encouraging and open a new framework for building smart video surveillance based security systems. Database link: https://data.mendeley.com/datasets/prf8p9rwhd/draft?a=d9646d2f-5154-4d49-8eb3-e7989187a24c

In tracking-by-detection paradigm for multi-target tracking, target association is modeled as an optimization problem that is usually solved through network flow formulation. In this paper, we proposed combinatorial optimization formulation and used a bipartite graph matching for associating the targets in the consecutive frames. Usually, the target of interest is represented in a bounding box and track the whole box as a single entity. However, in the case of humans, the body goes through complex articulation and occlusion that severely deteriorate the tracking performance. To partially tackle the problem of occlusion, we argue that tracking the rigid body organ could lead to better tracking performance compared to the whole body tracking. Based on this assumption, we generated the target hypothesis of only the spatial locations of person’s heads in every frame. After the localization of head location, a constant velocity motion model is used for the temporal evolution of the targets in the visual scene. Qualitative results are evaluated on four challenging video surveillance dataset and promising results has been achieved.

Among various physiological signal acquisition methods for the study of the human brain, EEG (Electroencephalography) is more effective. EEG provides a convenient, non-intrusive, and accurate way of capturing brain signals in multiple channels at fine temporal resolution. We propose an ensemble learning algorithm for automatically computing the most discriminative subset of EEG channels for internal emotion recognition. Our method describes an EEG channel using kernel-based representations computed from the training EEG recordings. For ensemble learning, we formulate a graph embedding linear discriminant objective function using the kernel representations. The objective function is efficiently solved via sparse non-negative principal component analysis and the final classifier is learned using the sparse projection coefficients. Our algorithm is useful in reducing the amount of data while improving computational efficiency and classification accuracy at the same time. The experiments on publicly available EEG dataset demonstrate the superiority of the proposed algorithm over the compared methods. Open Access

The success of minimally invasive interventions and the remarkable technological and medical progress have made endoscopic image enhancement a very active research field. Due to the intrinsic endoscopic domain characteristics and the surgical exercise, stereo endoscopic images may suffer from different degradations which affect its quality. Therefore, in order to provide the surgeons with a better visual feedback and improve the outcomes of possible subsequent processing steps, namely, a 3-D organ reconstruction/registration, it would be interesting to improve the stereo endoscopic image quality. To this end, we propose, in this paper, two joint enhancement methods which operate in the wavelet transform domain. More precisely, by resorting to a joint wavelet decomposition, the wavelet subbands of the right and left views are simultaneously processed to exploit the binocular vision properties. While the first proposed technique combines only the approximation subbands of both views, the second method combines all the wavelet subbands yielding an inter-view processing fully adapted to the local features of the stereo endoscopic images. Experimental results, carried out on various stereo endoscopic datasets, have demonstrated the efficiency of the proposed enhancement methods in terms of perceived visual image quality.

Laparoscopic videos can be affected by different distortions which may impact the performance of surgery and introduce surgical errors. In this work, we propose a framework for automatically detecting and identifying such distortions and their severity using video quality assessment. There are three major contributions presented in this work (i) a proposal for a novel video enhancement framework for laparoscopic surgery; (ii) a publicly available database for quality assessment of laparoscopic videos evaluated by expert as well as non-expert observers and (iii) objective video quality assessment of laparoscopic videos including their correlations with expert and non-expert scores., SPIE Medical Imaging 2020 (Draft version)

An effective contrast enhancement method should not only improve the perceptual quality of an image but should also avoid adding any artifacts or affecting naturalness of images. This makes Contrast Enhancement Evaluation (CEE) a challenging task in the sense that both the improvement in image quality and unwanted side-effects need to be checked for. Currently, there is no single CEE metric that works well for all kinds of enhancement criteria. In this paper, we propose a new Multi-Criteria CEE (MCCEE) measure which combines different metrics effectively to give a single quality score. In order to fully exploit the potential of these metrics, we have further proposed to apply them on the decomposed image using wavelet transform. This new metric has been tested on two natural image contrast enhancement databases as well as on medical Computed Tomography (CT) images. The results show a substantial improvement as compared to the existing evaluation metrics. The code for the metric is available at: https://github.com/zakopz/MCCEE-Contrast-Enhancement-Metric

Head detection in real-world videos is a classical research problem in computer vision. Head detection in videos is challenging than in a single image due to many nuisances that are commonly observed in natural videos, including arbitrary poses, appearances, and scales. Generally, head detection is treated as a particular case of object detection in a single image. However, the performance of object detectors deteriorates in unconstrained videos. In this paper, we propose a temporal consistency model (TCM) to enhance the performance of a generic object detector by integrating spatial-temporal information that exists among subsequent frames of a particular video. Generally, our model takes detection from a generic detector as input and improves mean average precision (mAP) by recovering missed detection and suppressing false positives. We compare and evaluate the proposed framework on four challenging datasets, i.e., HollywoodHeads, Casablanca, BOSS, and PAMELA. Experimental evaluation shows that the performance is improved by employing the proposed TCM model. We demonstrate both qualitatively and quantitatively that our proposed framework obtains significant improvements over other methods. Copyright © 2020 Sultan Daud Khan et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Detecting heads with full variations in camera view points, human poses, appearances, and scales is a key problem for many computer vision applications. Region convolutional neural networks (RCNN) achieved considerable success in handling the variances in poses and appearances. However, RCNN are inefficient in handling human heads of diverse scales. In this work, we focus on detecting human heads in complex scenes. Starting with traditional RCNN model, we extend it by leveraging person-scene relations and propose a dimension invariant convolutional neural network (DCNN) that coarsely predicts locations and scales of heads directly from the full image. We evaluate and compare our method with famous methods by using two benchmarks datasets. Experimental results show that our method outperforms these methods.

Sensor-based human activity recognition is an essential task for automatic behavior analysis for sports player, senior citizens, and IoT applications. The traditional approaches are based on hand-crafted features which use fixed mathematical rules to extract the features from the input data and are not capable of incremental learning. In this paper, we proposed a stacked long Short-term memory (LSTM) network for recognizing six human behaviors from the smartphone data. The network consists of a five LSTM cell that is trained end-to-end on the sensor data. The network is preceded by a single layer neural network that pre-processes the data for the stacked LSTM network. An L 2 regularizer is used in the cost function which helps the network in generalization. The network is evaluated on public domain UCI dataset and quantitative results are compared against six state-of-the-art methods. The performance is calculated in terms of precision-recall and the average accuracy. The proposed network improves the average accuracy by 0.93% as compared to the closest state-of-the-art method without any manual feature engineering.

International audience; In this paper, we present a no-reference (NR) quality predictor for stereoscopic/3D images based on statistics aggregation of monocular and binocular local contrast features. In particular, for left and right views, we first extract statistical features of the image gradient magnitude (GM) and the Laplacian of Gaussian (LoG), describing the image local structures from different perspectives. The monocular statistical features are then combined to derive the binocular features based on a linear summation model using weightings based on LoGresponse and image local-entropy, independently. These weights can effectively simulate the strength of the views dominance on binocular rivalry (BR) behavior of the human visual system. Subsequently, we further compute the GM features of the difference map between left and right views reflecting the distortion on disparity/depth information. Finally, the BR-inspired combined monocular and disparityrelated binocular features associated with subjective quality scores are jointly used to construct a learned regression model relying on support vector machine regressor. Experimental results on three 3D-IQA benchmark databases demonstrate that our method achieves high quality prediction accuracy and competitive performance compared to state-of-the-art methods.

People counting in sports venues is emerging as a new domain in the field of video surveillance. People counting in these venues faces many key challenges, such as severe occlusions, few pixels per head, and significant variations in person's head sizes due to wide sport areas. We propose a deep model based method, which works as a head detector and takes into consideration the scale variations of heads in videos. Our method is based on the notion that head is the most visible part in the sports venues where large number of people are gathered. To cope with the problem of different scales, we generate scale aware head proposals based on scale map. Scale aware proposals are then fed to the Convolutional Neural Network (CNN) and it provides a response matrix containing the presence probabilities of people observed across scene scales. We then use non-maximal suppression to get the accurate head positions. For the performance evaluation, we carry out extensive experiments on two standard datasets and compare the results with state-of-the-art (SoA) methods. The results in terms of Average Precision (AvP), Average Recall (AvR), and Average F1-Score (AvF-Score) show that our method is better than SoA methods.

The object sizes in images are diverse, therefore, capturing multiple scale context information is essential for semantic segmentation. Existing context aggregation methods such as pyramid pooling module (PPM) and atrous spatial pyramid pooling (ASPP) employ different pooling size or atrous rate, such that multiple scale information is captured. However, the pooling sizes and atrous rates are chosen empirically. Rethinking of ASPP leads to our observation that learnable sampling locations of the convolution operation can endow the network learnable fieldof- view, thus the ability of capturing object context information adaptively. Following this observation, in this paper, we propose an adaptive context encoding (ACE) module based on deformable convolution operation where sampling locations of the convolution operation are learnable. Our ACE module can be embedded into other Convolutional Neural Networks (CNNs) easily for context aggregation. The effectiveness of the proposed module is demonstrated on Pascal-Context and ADE20K datasets. Although our proposed ACE only consists of three deformable convolution blocks, it outperforms PPM and ASPP in terms of mean Intersection of Union (mIoU) on both datasets. All the experimental studies confirm that our proposed module is effective compared to the state-of-the-art methods.

International audience; In this paper, we propose a saliency-weighted stereoscopic JND (SSJND) model constructed based on psychophysical experiments, accounting for binocular disparity and spatial masking effects of the human visual system (HVS). Specifically, a disparity-aware binocular JND model is first developed using psychophysical data, and then is employed to estimate the JND threshold for non-occluded pixel (NOP). In addition, to derive a reliable 3D-JND prediction, we determine the visibility threshold for occluded pixel (OP) by including a robust 2D-JND model. Finally, SSJND thresholds of one view are obtained by weighting the resulting JND for NOP and OP with their visual saliency. Based on subjective experiments, we demonstrate that the proposed model outperforms the other 3D-JND models in terms of perceptual quality at the same noise level.

Today’s eLearning websites are heavily loaded with multimedia contents, which are often unstructured, unedited, unsynchronized, and lack inter-links among different multimedia components. Hyperlinking different media modality may provide a solution for quick navigation and easy retrieval of pedagogical content in media driven eLearning websites. In addition, finding meta-data information to describe and annotate media content in eLearning platforms is challenging, laborious, prone to errors, and time-consuming task. Thus annotations for multimedia especially of lecture videos became an important part of video learning objects. To address this issue, this paper proposes three major contributions namely, automated video annotation, the 3-Dimensional (3D) tag clouds, and the hyper interactive presenter (HIP) eLearning platform. Combining existing state-of-the-art SIFT together with tag cloud, a novel approach for automatic lecture video annotation for the HIP is proposed. New video annotations are implemented automatically providing the needed random access in lecture videos within the platform, and a 3D tag cloud is proposed as a new way of user interaction mechanism. A preliminary study of the usefulness of the system has been carried out, and the initial results suggest that 70% of the students opted for using HIP as their preferred eLearning platform at Gjøvik University College (GUC). This work is licensed under a Creative Commons Attribution 4.0 License.CC-BY

In minimally invasive surgery, smoke generated by such as electrocautery and laser ablation deteriorates image quality severely. This creates discomfortable view for the surgeon which may increase surgical risk and degrade the performance of computer assisted surgery algorithms such as segmentation, reconstruction, tracking, etc. Therefore, real-time smoke removal is required to keep a clear field of view. In this paper, we propose a real-time smoke removal approach based on Convolutional Neural Network (CNN). An encoder-decoder architecture with Laplacian image pyramid decomposition input strategy is proposed. This is an end-to-end network which takes the smoke image and its Laplacian image pyramid decomposition as inputs, and outputs a smoke free image directly without relying on any physical models or estimation of intermediate parameters. This design can be further embedded to deep learning based follow-up image guided surgery processes such as segmentation and tracking tasks easily. A dataset with synthetic smoke images generated from Blender and Adobe Photoshop is employed for training the network. The result is evaluated quantitatively on synthetic images and qualitatively on a laparoscopic dataset degraded with real smoke. Our proposed method can eliminate smoke effectively while preserving the original colors and reaches 26 fps for a video of size 512 × 512 on our training machine. The obtained results not only demonstrate the efficiency and effectiveness of the proposed CNN structure, but also prove the potency of training the network on synthetic dataset. © 2019 Society of Photo Optical Instrumentation Engineers. One print or electronic copy may be made for personal use only. Systematic reproduction and distribution, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper are prohibited.

Human brain undergoes state changes during sleep which produces distinctive signal patterns when recorded by electroencephalography (EEG). Automatic identification of these stages is crucial to diagnosing and treating sleep related disorders. We propose an image processing based technique for automatic identification of sleep stages from EEG signals. We generate two dimensional image representations from the high dynamic range Fourier transform features of the one dimensional EEG signals. Using these representations, we learn a deep and dense convolutional neural network (CNN) model for prediction. The key advantage of the proposed method is its seamless use of the existing well studied and powerful deep CNN models designed for computer vision problems. Experiments on the popular Sleep-EDF database show that the proposed method significantly outperforms the compared methods for automatic sleep stage identification Refereed/Peer-reviewed

Low contrast Computed Tomographic (CT) images often hamper the diagnosis of critical tumors found in various human organs. Contrast enhancement schemes play significant role in improving the visualization of these structures. To achieve this objective, Crossmodality Guided Enhancement (CMGE) method is proposed in this paper. The idea is to exploit the diversity of the information extracted from one modality to enhance the important structures including vessels and tumors in another modality. Our method employs information from liver Magnetic Resonance Image (MRI) to generate an enhanced CT image. It entails applying two dimensional histogram specification to map 2D histogram of CT to that of MRI followed by application of top and bottom hat transformations. These morphological operations highlight areas brighter than their surroundings and suppress darker areas. The final image is obtained by combining the results of these operations. Our method is compared with other state of the art contrast enhancement methods both visually and in terms of quality assessment metrics IEM and EME. The results show that our method performs better than these methods. CMGE technique yields improved contrast in low contrast CT images of the human liver and highlights tumors and vessels. © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Just noticeable difference (JND) for stereoscopic 3D content reflects the maximum tolerable distortion; it corresponds to the visibility threshold of the asymmetric distortions in the left and right contents. The 3D-JND models can be used to improve the efficiency of the 3D compression or the 3D quality assessment. Compared to 2D-JND models, the 3D-JND models appeared recently and the related literature is rather limited. In this paper, we give a deep and comprehensive study of the pixel-based 3D-JND models. To our best knowledge, this is the first review on 3D-JND models. Each model is briefly described by giving its rationale and main components in addition to providing exhaustive information about the targeted application, the pros, and cons. Moreover, we present the characteristics of the human visual system presented in these models. In addition, we analyze and compare the 3D-JND models thoroughly using qualitative and quantitative performance evaluation based on Middlebury stereo datasets. Besides, we measure the JND thresholds of the asymmetric distortion based on psychophysical experiments and compare these experimental results to the estimates from the 3D-JND models in order to evaluate the accuracy of each model. Copyright the Author(s). Open Access under a CC BY 4.0 license. Users are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles, or use them for any other lawful purpose, as long as proper attribution is given.

International audience; No-reference (NR) stereoscopic 3D (S3D) image quality assessment (SIQA) is still challenging due to the poor understanding of how the human visual system (HVS) judges image quality based on binocular vision. In this paper, we propose an efficient opinion-aware NR Stereoscopic Quality predictor based on local contrast statistics combination (SQSC). Specifically, for left and right views, we first extract statistical features of the gradient magnitude (GM) and Laplacian of Gaussian (LoG) responses, describing the image local structures from different perspectives. The HVS is insensitive to low-order statistical redundancies that can be removed by LoG filtering. Hence, the monocular statistical features are then fused to derive the binocular features based on a linear combination model using LoG responses-based weightings. These weightings can efficiently simulate the binocular rivalry (BR) phenomenon. Finally, the binocular features and the subjective scores were jointly employed to construct a learned regression model obtained by the support vector regression (SVR) algorithm. Experimental results on three widely used 3D IQA databases demonstrate the high prediction performance of the proposed method when compared to recent well performing SIQA methods.

During video-guided minimally invasive surgery, quality of frames may be degraded severely by cauterization-induced smoke and condensation of vapor. This degradation of quality creates discomfort for the operating surgeon, and causes serious problems for automatic follow-up processes such as registration, segmentation and tracking. This paper proposes a novel deep neural network based smoke removal solution that is able to enhance the quality of surgery video frames in real-time. It employs synthetically generated training dataset including smoke embedded and clean reference versions. Results calculated on the test set indicate that our network outperforms previous defogging methods in terms of quantitative and qualitative measures. While eliminating apparent smoke, it also successfully preserves the natural appearance of tissue surface. To the best of our knowledge, the presented method is the first deep neural network based approach for the surgical field smoke removal problem. © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

For the first time, a database dedicated to contrast enhancement performance evaluation is proposed. This database has recently been developed by our group. In this paper, a detailed description of this database and the methodology we used to build it are discussed. We show that the perceptual quality of contrasted images is related to some specific distortions and artifacts that should be taken into account when building such a database. To this end, we provide some guidelines on how to build a database suitable for testing contrast enhancement quality evaluation metrics, and how to use a psychophysical experiment to obtain the quality judgments for this database. Some open problems and new ideas for using and extending this database are also discussed. The Contrast Enhancement Evaluation Database (CEED), is made publicly accessible through http://dx.doi.org/10.17632/3hfzp6vwkm.3.

We propose a Directed Sparse Graphical Model (DSGM) for multi-target tracking. In the category of global optimization for multi-target tracking, traditional approaches have two main drawbacks. First, a cost function is defined in terms of the linear combination of the spatial and appearance constraints of the targets which results a highly non-convex function. And second, a very dense graph is constructed to capture the global attribute of the targets. In such a graph, It is impossible to find reliable tracks in polynomial time unless some relaxation and heuristics are used. To address these limitations, we proposed DSGM which finds a set of reliable tracks for the targets without any heuristics or relaxation and keeps the computational complexity very low through the design of the graph. Irrespective of traditional approaches where spatial and appearance constraints are added up linearly with a given weight factor, we incorporated these constraints in a cascaded fashion. First, we exploited a Hidden Markov Model (HMM) for the spatial constraints of the target and obtain most probable locations of the targets in a segment of video. Afterwards, a deep feature based appearance model is used to generate the sparse graph. The track for each target is found through dynamic programming. Experiments are performed on 3 challenging sports datasets (football, basketball and sprint) and promising results are achieved.

In this paper, we present efficient bit allocation methods for stereo image coding purpose. Since the common idea behind most of the existing stereo compression schemes consists of encoding a reference and residual images as well as a disparity map, we mainly focus on the bit allocation issue between the reference and residual images. Generally, this problem is solved in an empirical manner by looking for the optimal rates leading to the minimum distortion value. Thanks to recent approximations of the entropy and distortion functions, we propose accurate and fast bit allocation schemes appropriate for the open-loop- and closed-loop-based stereo coding structures. Experimental results show the benefits which can be drawn from the proposed bit allocation methods.

Laparoscopic surgery has a limited field of view. Laser ablation in a laproscopic surgery causes smoke, which inevitably influences the surgeon's visibility. Therefore, it is of vital importance to remove the smoke, such that a clear visualization is possible. In order to employ a desmoking technique, one needs to know beforehand if the image contains smoke or not, to this date, there exists no accurate method that could classify the smoke/non-smoke images completely. In this work, we propose a new enhancement method which enhances the informative details in the RGB images for discrimination of smoke/non-smoke images. Our proposed method utilizes weighted least squares optimization framework~(WLS). For feature extraction, we use statistical features based on bivariate histogram distribution of gradient magnitude~(GM) and Laplacian of Gaussian~(LoG). We then train a SVM classifier with binary smoke/non-smoke classification task. We demonstrate the effectiveness of our method on Cholec80 dataset. Experiments using our proposed enhancement method show promising results with improvements of 4\% in accuracy and 4\% in F1-Score over the baseline performance of RGB images. In addition, our approach improves over the saturation histogram based classification methodologies Saturation Analysis~(SAN) and Saturation Peak Analysis~(SPA) by 1/5\% and 1/6\% in accuracy/F1-Score metrics., Comment: In proceedings of IST, Color and Imaging Conference (CIC 26). Congcong Wang and Vivek Sharma contributed equally to this work and listed in alphabetical order

Articulation modeling, feature extraction, and classification are the important components of pedestrian segmentation. Usually, these components are modeled independently from each other and then combined in a sequential way. However, this approach is prone to poor segmentation if any individual component is weakly designed. To cope with this problem, we proposed a spatio-temporal convolutional neural network named PedNet which exploits temporal information for spatial segmentation. The backbone of the PedNet consists of an encoder&ndash, decoder network for downsampling and upsampling the feature maps, respectively. The input to the network is a set of three frames and the output is a binary mask of the segmented regions in the middle frame. Irrespective of classical deep models where the convolution layers are followed by a fully connected layer for classification, PedNet is a Fully Convolutional Network (FCN). It is trained end-to-end and the segmentation is achieved without the need of any pre- or post-processing. The main characteristic of PedNet is its unique design where it performs segmentation on a frame-by-frame basis but it uses the temporal information from the previous and the future frame for segmenting the pedestrian in the current frame. Moreover, to combine the low-level features with the high-level semantic information learned by the deeper layers, we used long-skip connections from the encoder to decoder network and concatenate the output of low-level layers with the higher level layers. This approach helps to get segmentation map with sharp boundaries. To show the potential benefits of temporal information, we also visualized different layers of the network. The visualization showed that the network learned different information from the consecutive frames and then combined the information optimally to segment the middle frame. We evaluated our approach on eight challenging datasets where humans are involved in different activities with severe articulation (football, road crossing, surveillance). The most common CamVid dataset which is used for calculating the performance of the segmentation algorithm is evaluated against seven state-of-the-art methods. The performance is shown on precision/recall, F 1 , F 2 , and mIoU. The qualitative and quantitative results show that PedNet achieves promising results against state-of-the-art methods with substantial improvement in terms of all the performance metrics.