Search

Video coding using dynamic background frame achieves better compression compared to the traditional techniques by encoding background and foreground separately. This process reduces coding bits for the overall frame significantly; however, encoding background still requires many bits that can be compressed further for achieving better coding efficiency. The cuboid coding framework has been proven to be one of the most effective methods of image compression which exploits homogeneous pixel correlation within a frame and has better alignment with object boundary compared to traditional block-based coding. In a video sequence, the cuboid-based frame partitioning varies with the changes of the foreground. However, since the background remains static for a group of pictures, the cuboid coding exploits better spatial pixel homogeneity. In this work, the impact of cuboid coding on the background frame for high-resolution videos (Ultra-High-Definition (UHD) and 360-degree videos) is investigated using the multilayer framework of SHVC. After the cuboid partitioning, the method of coarse frame generation has been improved with a novel idea by keeping human-visual sensitive information. Unlike the traditional SHVC scheme, in the proposed method, cuboid coded background and the foreground are encoded in separate layers in an implicit manner. Simulation results show that the proposed video coding method achieves an average BD-Rate reduction of 26.69% and BD-PSNR gain of 1.51 dB against SHVC with significant encoding time reduction for both UHD and 360 videos. It also achieves an average of 13.88% BD-Rate reduction and 0.78 dB BD-PSNR gain compared to the existing relevant method proposed by X. Hoang Van.

Detecting COVID-19 early may help in devising an appropriate treatment plan and disease containment decisions. In this study, we demonstrate how transfer learning from deep learning models can be used to perform COVID-19 detection using images from three most commonly used medical imaging modes X-Ray, Ultrasound, and CT scan. The aim is to provide over-stressed medical professionals a second pair of eyes through intelligent deep learning image classification models. We identify a suitable Convolutional Neural Network (CNN) model through initial comparative study of several popular CNN models. We then optimize the selected VGG19 model for the image modalities to show how the models can be used for the highly scarce and challenging COVID-19 datasets. We highlight the challenges (including dataset size and quality) in utilizing current publicly available COVID-19 datasets for developing useful deep learning models and how it adversely impacts the trainability of complex models. We also propose an image pre-processing stage to create a trustworthy image dataset for developing and testing the deep learning models. The new approach is aimed to reduce unwanted noise from the images so that deep learning models can focus on detecting diseases with specific features from them. Our results indicate that Ultrasound images provide superior detection accuracy compared to X-Ray and CT scans. The experimental results highlight that with limited data, most of the deeper networks struggle to train well and provides less consistency over the three imaging modes we are using. The selected VGG19 model, which is then extensively tuned with appropriate parameters, performs in considerable levels of COVID-19 detection against pneumonia or normal for all three lung image modes with the precision of up to 86% for X-Ray, 100% for Ultrasound and 84% for CT scans.

Lung cancer is the leading cause of cancer death and morbidity worldwide. Many studies have shown machine learning models to be effective in detecting lung nodules from chest X-ray images. However, these techniques have yet to be embraced by the medical community due to several practical, ethical, and regulatory constraints stemming from the “black-box” nature of deep learning models. Additionally, most lung nodules visible on chest X-rays are benign, therefore, the narrow task of computer vision-based lung nodule detection cannot be equated to automated lung cancer detection. Addressing both concerns, this study introduces a novel hybrid deep learning and decision tree-based computer vision model, which presents lung cancer malignancy predictions as interpretable decision trees. The deep learning component of this process is trained using a large publicly available dataset on pathological biomarkers associated with lung cancer. These models are then used to inference biomarker scores for chest X-ray images from two independent data sets, for which malignancy metadata is available. Next, multi-variate predictive models were mined by fitting shallow decision trees to the malignancy stratified datasets and interrogating a range of metrics to determine the best model. The best decision tree model achieved sensitivity and specificity of 86.7% and 80.0%, respectively, with a positive predictive value of 92.9%. Decision trees mined using this method may be considered as a starting point for refinement into clinically useful multi-variate lung cancer malignancy models for implementation as a workflow augmentation tool to improve the efficiency of human radiologists.

Detecting animals to estimate abundance can be difficult, particularly when the habitat is dense or the target animals are fossorial. The recent surge in the use of thermal imagers in ecology and their use in animal detections can increase the accuracy of population estimates and improve the subsequent implementation of management programs. However, the use of thermal imagers results in many hours of captured flight videos which require manual review for confirmation of species detection and identification. Therefore, the perceived cost and efficiency trade-off often restricts the use of these systems. Additionally, for many off-the-shelf systems, the exported imagery can be quite low resolution (

In this paper, a deep convolutional neural network (CNN) is proposed for accurate segmentation of retinal blood vessels. This method plays a significant role in observing many eye diseases. A strided-CNN model is proposed for accurate segmentation of retinal vessels, especially the tiny vessels. The model is a fully convolutional model consisting of an encoder part and a decoder part where the pooling layers are replaced with strided convolutional layers. The strided convolutional layer approach was chosen over the pooling layers approach as the former can be trained. The morphological mappings along with the Principal Component Analysis (PCA)- based pre-processing steps are used to generate contrast images for training dataset. Skip connections are implemented to concatenate features from the encoder part and the decoder part to enhance the vessels segmentation especially the tiny vessels and to make the vessel’s edges sharper. We used a class balancing loss function to train and optimize the proposed model to improve vessel image quality. The impact of the proposed segmentation method is evaluated on four databases namely DRIVE, STARE, CHASE-DB1 and HRF. Overall model performance, particularly with respect to tiny vessels, is primarily influenced by sensitivity and accuracy metrics. We demonstrate that our model outperforms other models with a sensitivity of 0.87, 0.808, 0.886 and 0.829 on DRIVE, STARE, CHASE_DB1 and HRF respectively, along with respective accuracies of 0.956, 0.954, 0.976 and 0.962.

Video summarization is the process to extract the most significant contents of a video and to represent it in a concise form. The existing methods for video summarization could not achieve a satisfactory result for a video with camera movement and significant illumination changes. To solve these problems, in this paper, a new framework for video summarization is proposed based on eye tracker data, as human eyes can track moving object accurately in these cases. The smooth pursuit is the state of eye movement when a user follows a moving object in a video. This motivates us to implement a new method to distinguish smooth pursuit from other type of gaze points, such as fixation and saccade. The smooth pursuit provides only the location of moving objects in a video frame; however, it does not indicate whether the located moving objects are very attractive (i.e., salient regions) to viewers or not, as well as the amount of motion of the moving objects. The amount of salient regions and object motions are the two important features to measure the viewer’s attention level for determining the key frames for video summarization. To find the most attractive objects, a new spatial saliency prediction method is also proposed by constructing a saliency map around each smooth pursuit gaze point based on human visual field, such as fovea, parafoveal, and perifovea regions. To identify the amount of object motions, the total distances between the current and the previous gaze points of viewers during smooth pursuit are measured as a motion saliency score. The motivation is that the movement of eye gaze is related to the motion of the objects during smooth pursuit. Finally, both spatial and motion saliency maps are combined to obtain an aggregated saliency score for each frame and a set of key frames are selected based on user selected or system default skimming ratio. The proposed method is implemented on Office video data set that contains videos with camera movements and illumination changes. Experimental results confirm the superior performance of the proposed spatial and motion saliency prediction method compared with the state-of-the-art methods.

As the human vision is a definitive assessor of video quality, the expanded interest for no-reference subjective quality assessment (SQA) is focusing on a definitive goal of coordinating with human observation. However, the widely used subjective estimator-mean opinion score (MOS) is often biased by the testing environment, viewers mode, expertise, domain knowledge, and other factors which may influence on actual assessment. In this paper, a no-reference SQA metric is devised by simply exploiting the nature of human eye browsing on videos and analyzing the associated quality correlation features. The high efficiency video coding (HEVC) reference test model is first employed to produce different forms of coded video quality which then displayed to a number of partakers. Their eye-tracker recorded spatiotemporal gaze-data indicate more concentrated eye-traversing approach for relatively better quality. Thus, we calculate the quality assessment related to assorted features such as length pursuit, angle deflection, pupil deviation, and gaze interlude from recorded gaze trajectory. The content and resolution invariant operations are carried out prior to synthesizing them using an adaptive weighted function to develop a new quality metric-eye maneuver (EMAN). Tested results reveal that the quality evaluation carried out by the EMAN is comparatively better than MOS and structural similarity (SSIM) in terms of assessing different aspects of coded video quality for a wide range of single view video contents. For the free viewpoint video (FVV), where the reference frame is not available, the EMAN could also better distinguish different qualities compared to the MOS and SSIM.

This paper presents a comprehensive review of the principle and application of deep learning in retinal image analysis. Many eye diseases often lead to blindness in the absence of proper clinical diagnosis and medical treatment. For example, diabetic retinopathy (DR) is one such disease in which the retinal blood vessels of human eyes are damaged. The ophthalmologists diagnose DR based on their professional knowledge, that is labor intensive. With the advances in image processing and artificial intelligence, computer vision-based techniques have been applied rapidly and widely in the field of medical images analysis and are becoming a better way to advance ophthalmology in practice. Such approaches utilize accurate visual analysis to identify the abnormality of blood vessels with improved performance over manual procedures. More recently, machine learning, in particular, deep learning, has been successfully implemented in this area. In this paper, we focus on recent advances in deep learning methods for retinal image analysis. We review the related publications since 1982, which include more than 80 papers for retinal vessels detections in the research scope spanning from segmentation to classification. Although deep learning has been successfully implemented in other areas, we found only 17 papers so far focus on retinal blood vessel segmentation. This paper characterizes each deep learning based segmentation method as described in the literature. Analyzing along with the limitations and advantages of each method. In the end, we offer some recommendations for future improvement for retinal image analysis.

Immersive media representation format based on point clouds has underpinned significant opportunities for extended reality applications. Point cloud in its uncompressed format require very high data rate for storage and transmission. The video based point cloud compression (V-PCC) technique projects a dynamic point cloud into geometry and texture video sequences. The projected texture video is then coded using modern video coding standard like HEVC. Since the properties of projected texture video frames are different from traditional video frames, HEVC-based commonality modeling can be inefficient. An improved commonality modeling technique is proposed that employs edge position difference oriented motion model. Experimental results show that the proposed commonality modeling technique can yield savings in bit rate of up to 3.15% over the V-PCC HEVC reference encoder.

The median survival time of patients with high grade glioma, a form of brain tumour, is 1-3 years. The current best practice adopts Convolutional Neural Network (CNN) for image classification and tumour detection. This method provides a significant improvement in brain tumour segmentation of Magnetic Resonance Imaging (MRI) images in comparison to other frameworks, but it is nonetheless slow and lacks precision. We sought to build upon the current best practice model by utilising a Deep Neural Network (DNN) model, which entailed modification of the segmentation and feature-extraction stages in order to improve the accuracy of those stages and the resulting segmentation. We contrasted the accuracy and efficiency of our model to the current best practice model using 10 brain tumour patient MRI datasets. First, the segmentation accuracy of our proposed model (M= 90%) outperformed that of the current best practice (M=78%). Second, the tumour detection processing time of our proposed model (M=34 ms) also outperformed that of the current best practice (M=73 ms). We, therefore, replicated previous studies by showing that automatic segmentation can aid in brain tumour detection. Importantly, we extended previous studies by proposing a model that classifies a brain tumour with greater accuracy and within lower processing times. Validation of the model with a larger dataset is recommended.

In conventional video compression systems, motion model is used to approximate the geometry of moving object boundaries. It is possible to relieve motion model from describing discontinuities in the underlying motion field, by incorporating motion hint that can predict the spatial structure of future frames using the structure of reference frames. However, formation of highly accurate motion hint is computationally demanding, in particular for high resolution video sequences. Cuboids, rectangular regions derived using statistical features, attempt to separate out different objects present in the scene; they are computationally efficient and have sparse representation. Leveraging on the advantages of cuboids, in this paper, we propose to discover homogeneous motion regions and their associated motion based on cuboids. Afterwards, the estimated motion models and their domains are applied to form a prediction of the current frame. Experimental results show that a savings in bit rate of 3.96% is achievable over standalone HEVC reference, if this predicted frame is used as an additional reference frame for the current frame.

The COVID-19 pandemic has triggered an urgent need to contribute to the fight against an immense threat to the human population. Computer Vision, as a subfield of Artificial Intelligence, has enjoyed recent success in solving various complex problems in health care and has the potential to contribute to the fight of controlling COVID-19. In response to this call, computer vision researchers are putting their knowledge base at work to devise effective ways to counter COVID-19 challenge and serve the global community. New contributions are being shared with every passing day. It motivated us to review the recent work, collect information about available research resources and an indication of future research directions. We want to make it available to computer vision researchers to save precious time. This survey paper is intended to provide a preliminary review of the available literature on the computer vision efforts against COVID-19 pandemic., 24 Pages, 9 Figures

© Springer Nature Switzerland AG 2020. Digital images are used to transfer most critical data in areas like medical, research, business, military, etc. The images transfer takes place over an unsecured Internet network. Therefore, there is a need for reliable security and protection for these sensitive images. Medical images play an important role in the field of Telemedicine and Tele surgery. Thus, before making any diagnostic decisions and treatments, the authenticity and the integrity of the received medical images need to be verified to avoid misdiagnosis. This paper proposes a block-wise and blind fragile watermarking mechanism for medical image authentication and recovery. By eliminating embedded insignificant data and considering different content complexity for each block during feature extraction and recovery, the capacity of data embedding without loss of quality is increased. This new embedding watermark method can embed a copy of the compressed image inside itself as a watermark to increase the recovered image quality. In our proposed hybrid scheme, the block features are utilized to improve the efficiency of data concealing for authentication and reduce tampering. Therefore, the scheme can achieve better results in terms of the recovered image quality and greater tampering protection, compared with the current schemes.

The COVID-19 pandemic has triggered an urgent call to contribute to the fight against an immense threat to the human population. Computer Vision, as a subfield of artificial intelligence, has enjoyed recent success in solving various complex problems in health care and has the potential to contribute to the fight of controlling COVID-19. In response to this call, computer vision researchers are putting their knowledge base at test to devise effective ways to counter COVID-19 challenge and serve the global community. New contributions are being shared with every passing day. It motivated us to review the recent work, collect information about available research resources, and an indication of future research directions. We want to make it possible for computer vision researchers to find existing and future research directions. This survey article presents a preliminary review of the literature on research community efforts against COVID-19 pandemic., IEEE Access, 8, ISSN:2169-3536

The main challenge in a rain removal algorithm is to differentiate rain streak from moving objects. This paper addresses this problem using the spatiotemporal appearance technique (STA). Although the STA-based technique can significantly remove rain from video, in some cases it cannot properly retain all the moving object regions. The photometric feature of rain streak was used to solve this issue. In this paper, a new algorithm combining STA and the photometric correlation between rain streak and background is proposed. Rain streak and moving objects were successfully detected and separated by combining both techniques, then fused to obtain well-recovered moving objects with rain-free video. The experimental results reveal that the proposed algorithm significantly outperforms the state-of-the-art methods for both real and synthetic rain streak.

Mixed reality in surgical telepresence has not been practically implemented as yet and research done, until now, is only theoretical. Aiming to implement a proposed new solution by merging virtual and augmented realities and bringing about improvements by removing occlusion and the noise caused by hand motions during surgery. The proposed system consists of an extended Camshift algorithm for the detection of the remote expert surgeon’s hand during surgery and enhancement of an image synthesization algorithm to remove the occlusion by including two Red-Green-Blue-Depth (RGBD) cameras. The experimental results from 10 breast samples reduce the average accuracy error of the video frame image overlay from 1.44 mm to 1.28 mm with average processing time enhanced to 76 sec compared to the state-of-the-art method. The proposed system provides a satisfactory outcome in terms of accuracy and operating time, which allows surgeons in local and remote sites to collaborate more effectively.

Augmented reality-based constructive jaw surgery has been facing various limitations such as noise in real-time images, the navigational error of implants and jaw, image overlay error, and occlusion handling which have limited the implementation of augmented reality (AR) in corrective jaw surgery. This research aimed to improve the navigational accuracy, through noise and occlusion removal, during positioning of an implant in relation to the jaw bone to be cut or drilled. The proposed system consists of a weighting-based de-noising filter and depth mapping-based occlusion removal for removing any occluded object such as surgical tools, the surgeon’s body parts, and blood. The maxillary (upper jaw) and mandibular (lower jaw) jaw bone sample results show that the proposed method can achieve the image overlay error (video accuracy) of 0.23~0.35 mm and processing time of 8–12 frames per second compared to 0.35~0.45 mm and 6–11 frames per second by the existing best system. The proposed system concentrates on removing the noise from the real-time video frame and the occlusion. Thus, the acceptable range of accuracy and the processing time are provided by this study for surgeons for carrying out a smooth surgical flow.

Capturing a scene using multiple cameras from different angles is expected to provide the necessary interactivity in the 3-D space to satisfy end-users’ demands for observing objects and actions from different angles and depths. Existing multiview video coding (MVC) technologies face tradeoff among rate-distortion performance, random access frame delay, i.e., interactivity, and computational time. To address above mentioned tradeoffs, a novel cuboid MVC strategy is proposed with 3-D frame referencing structure to improve interactivity and computational time, an additional reference frame to improve rate-distortion performance for occluded areas, and visual attention-based bit allocation to provide better perceptual video quality. The experimental results reveal that the proposed scheme provides better interactivity, reduced computational time, and better perceptual quality compared to the 3D-HEVC implementation, HTM 15.0.

An automatic retinal vessel segmentation method is proposed for quick and accurate segmentation of eye vessels. Such a method would play a vital role in analyzing many eye diseases. A retinal fundus image contains varying low contrasts, which undermine the performance of the segmentation process. Independent component analysis (ICA) is largely used for noise removal and consists of two architectures, designated as ICA1 and ICA2. We have validated both ICA architectures on retinal color fundus images and selected the one that provides improved contrast values. For retinal fundus, ICA2 architecture performed better than ICA1 by virtue of being more effective in compensating the low contrast values. Experiments conducted here validated the improvements over previously reported state-of-the-art methods. The impact of proposed segmentation model was assessed on publicly available databases like DRIVE and STARE. In case of the DRIVE database, the sensitivity increased 3% by (from 72% to 75%) while maintaining a segmentation accuracy of around 96%.

The depth image-based rendering paves the path to success of 3-D video. However, one issue still remained in 3-D video is how to fill the disocclusion areas. To this end, Gaussian mixture model (GMM) is commonly employed to generate the background, and then to fill the holes. Nevertheless, GMM usually has poor performance for sequences with big foreground reciprocation. In this paper, we aim to enhance the synthesis performance. Firstly, we propose an expectation maximization based GMM background generation method, in which the pixel mixture distribution is derived. Secondly, we propose a refined foreground depth correlation approach, which recovers the background frame-by-frame based on depth information. Finally, we adaptively choose the background pixels from these two methods for filling. Experimental results show that the proposed method outperforms existing non-deep learning based hole filling methods by around 1.1 dB, and significantly surpasses deep learning based alternative in terms of subjective quality.

The latest High Efficiency Video Coding (HEVC) standard has greatly improved the coding efficiency compared to its predecessor H.264. An important share of which is the adoption of hierarchical block partitioning structures and an extended number of modes. The structure of existing inter-modes is appropriate mainly to handle the rectangular and square aligned motion patterns. However, they could not be suitable for the block partitioning of depth objects having partial foreground motion with irregular edges and background. In such cases, the HEVC reference test model (HM) normally explores finer level block partitioning that requires more bits and encoding time to compensate large residuals. Since motion detection is the underlying criteria for mode selection, in this work, we use the energy concentration ratio feature of phase correlation to capture different types of motion in depth object. For better motion modeling focusing at depth edges, the proposed technique also uses an extra pattern mode comprising a group of templates with various rectangular and non-rectangular object shapes and edges. As the pattern mode could save bits by encoding only the foreground areas and beat all other inter-modes in a block once selected, the proposed technique could improve the rate-distortion performance. It could also reduce encoding time by skipping further branching using the pattern mode and selecting a subset of modes using innovative pre-processing criteria. Experimentally it could save 29% average encoding time and improve 0.10 dB Bjontegaard Delta peak signal-to-noise ratio compared to the HM.

Eye-related disease such as diabetic retinopathy (DR) is a medical ailment in which the retina of the human eye is smashed because of damage to the tiny retinal blood vessels in the retina. Ophthalmologists identify DR based on various features such as the blood vessels, textures and pathologies. With the rapid development of methods of analysis of biomedical images and advanced computing techniques, image processing-based software for the detection of eye disease has been widely used as an important tool by ophthalmologists. In particular, computer vision-based methods are growing rapidly in the field of medical images analysis and are appropriate to advance ophthalmology. These tools depend entirely on visual analysis to identify abnormalities in Retinal Fundus images. During the past two decades, exciting improvement in the development of DR detection computerised systems has been observed. This paper reviews the development of analysing retinal images for the detection of DR in three aspects: automatic algorithms (classification or pixel to pixel methods), detection methods of pathologies from retinal fundus images, and extraction of blood vessels of retinal fundus image algorithms for the detection of DR. The paper presents a detailed explanation of each problem with respect to retinal images. The current techniques that are used to analyse retinal images and DR detection issues are also discussed in detail and recommendations are made for some future directions.

Retinal vessel segmentation plays a major role in the detection of many eye diseases, and it is required to implement an automated algorithm for analyzing the progress of eye diseases. A variety of automated segmentation methods have been presented but almost all studies to date showed weakness in their low sensitivity toward narrow low-contrast vessels. A new segmentation method is proposed to address the issue of low sensitivity, by including modules such as principal component analysis-based color-to-gray conversion, scale normalization factors for improved narrow vessel detection, anisotropic diffusion filtering with an adequate stopping rule, and edge pixel-based hysteresis threshold. The impact of these additional steps is assessed on publicly available databases like DRIVE and STARE. For the case of DRIVE database, the sensitivity is raised from 73 to 75%, while maintaining the accuracy of 96.5%, and found to provide evidence of improved sensitivity.

Capturing images in the challenging atmosphere for example during rain and snow critically degrades the quality of the images. These external phenomena create low contrast, blur and thereby reduce the visibility of the images. Many computer vision applications like visual traffic surveillance, intelligent vehicles, and entertainments are affected by reduced visibility. Rain streaks removal in a video (RSRV) has significant importance in the outdoor vision systems like surveillance video and has recently been studied comprehensively. In the last few years, many approaches have been made towards solving this problem including matrix decomposition, convolutional neural network, streak orientation modelling, Gaussian mixture modelling (GMM), etc. In this paper, we propose a concise model for RSRV based on the spatiotemporal appearance (STA) of rain streak in a video to exploit the rain streak appearance property. Here we apply Gaussian mixture modelling (GMM) to separate the background and foreground. Then, we use STA to separate the rain streak from moving foreground. The experimental results establish that the proposed method outperforms the state-of-the-art methods significantly for both real and synthetic rains.

Due to the powerful image editing tools, image integrity protection is becoming important, for instance, in a court of law, news reports, insurance claims or telemedicine. Digital image watermarking is a technique to detect, localize and recover tampered image but the accuracy of performance by the existing methods is still an issue. This paper proposes a new watermarking embedding mechanism for effective image authentication with a recovery scheme. In the method, a host image is divided into blocks and the feature of each block is embedded in the block itself and another block as watermark. For detecting tampering, the extracted watermarked data is checked with source and destination block leading to more accuracy. The recovery data is extracted from the destination block which is intact. The proposed scheme achieves better results in terms of tamper detection accuracy and the quality of recovered image compared with the state-of-art schemes.

Due to the powerful image editing tools, image integrity protection is becoming important, for instance, in a court of law, news reports, insurance claims or telemedicine. Digital image watermarking is a technique for detecting, localizing and recovering of tampered image but capability and accuracy of performance by the existing methods are still issues especially when tampering rate is relatively high. This paper proposes a novel blind fragile watermarking mechanism for effective image authentication and recovery. In the proposed scheme, not only the probability of detection and localization of tampering is increased because of embedding appropriate authentication code in the proper position but also there is an efficient post-processing of recovery in a way that any pixel has treated differently leading to higher quality of recovered image. Furthermore, embedding more fragile authentication code with a smaller number of bits can help to provide a second opportunity for content recovery while the quality of watermarked image is preserved. Thus, better results have been achieved in terms of accuracy of detection and the quality of recovered image after higher tampering rate compared with some of the state-of-the-art schemes.

Parameter fine tuning is a transfer learning approach whereby learned parameters from pre-trained source network are transferred to the target network followed by fine-tuning. Prior research has shown that this approach is capable of improving task performance. However, the impact of the ImageNet pre-trained classification layer in parameter fine-tuning is mostly unexplored in the literature. In this paper, we propose a fine-tuning approach with the pre-trained classification layer. We employ layer-wise fine-tuning to determine which layers should be frozen for optimal performance. Our empirical analysis demonstrates that the proposed fine-tuning performs better than traditional fine-tuning. This finding indicates that the pre-trained classification layer holds less category-specific or more global information than believed earlier. Thus, we hypothesize that the presence of this layer is crucial for growing network depth to adapt better to a new task. Our study manifests that careful normalization and scaling are essential for creating harmony between the pre-trained and new layers for target domain adaptation. We evaluate the proposed depth augmented networks for fine-tuning on several challenging benchmark datasets and show that they can achieve higher classification accuracy than contemporary transfer learning approaches.

Three dimensional (3D) i.e., multi-view extension of High Efficiency Video Coding (HEVC) standard provides a better compression compared to the simulcast coding (i.e., HEVC) technique by exploiting inter- and intra-view redundancy for multi-view video coding. However, this technique imposes a random access frame delay (RAFD) problem as well as requires huge computational time. In this paper three novel techniques are proposed to overcome the problems mentioned above. Firstly, a simulcast video coding technique is proposed where each view is encoded individually using hierarchical bi-predictive structure with extra virtual reference frame generated by dynamic background modeling (popularly known as McFIS -?the most common frame in a scene) of the corresponding view. Secondly a novel technique is proposed using 3D motion estimation (3DME) where a 3D frame is formed using the same temporal frames of all views and motion estimation is carried out for the current 3D frame using dual 3D reference frames. Finally, a modification of 3DME dual reference frame technique (3DME-McFIS) is proposed where 3D McFIS is used as the second reference frame. Experimental results confirm that the proposed three techniques reduce the overall computational time and reduce the RAFD problem with comparable or better rate-distortion performance compared to the HEVC-3D extension. Specifically the proposed 3DME-McFIS technique outperforms the HEVC-3D coding standard by improving 0.90dB PSNR on average, by reducing computational time by 50%, and by reducing RAFD problem compared to the existing HEVC-3D coding standard.

High Efficiency Video Coding (HEVC) standard adopts several new approaches to achieve higher coding efficiency (approximately 50% bit-rate reduction) compared to its predecessor H.264/AVC with same perceptual image quality. Huge computational time has also increased due to the algorithmic complexity of HEVC compared to H.264/AVC. However, it is really a demanding task to reduce the encoding time while preserving the similar quality of the video sequences. In this paper, we propose a novel efficient intermode selection technique and incorporate into HEVC framework to predict motion estimation and motion compensation modes between current and reference blocks and perform faster inter mode selection based on three dissimilar motion types in divergent video sequences. Instead of exploring and traversing all the modes exhaustively, we merely select a subset of candidate modes and the final mode from the selected subset is determined based on their lowest Lagrangian cost function. The experimental results reveal that average encoding time can be downscaled by 40% with similar rate-distortion performance compared to the exhaustive mode selection strategy in HEVC.

With the advancement of displaying technologies, virtual viewpoint video needs to be synthesized from adjacent viewpoints to provide immersive perceptual viewing experience of a scene. View synthesized techniques suffer poor rendering quality due to holes created by occlusion in the warping process. Currently, spatial and temporal correlation techniques are used to improve the quality of the synthesized view. However, spatial correlation e. g. inpainting and inverse mapping (IM) techniques cannot fill holes efficiently due to low spatial correlation in the edge between foreground and background pixels. On the other hand, the temporal correlation among already synthesized frames through learning by Gaussian mixture modelling (GMM) may fill occluded areas efficiently. However, there are no frames for GMM learning when the user switches view instantly. To address the aforementioned issues, in the proposed view synthesis technique, we apply GMM on the adjacent viewpoint videos. Then, we utilize the number of GMM models to refine pixel intensities of the synthesized view by using a weighting factor between pixel intensities in GMM models and warped images. This technique provides a better pixel correspondence, which improves 0.47~0.58dB PSNR compared to the IM technique.

Accurate segmentation of vessels is an arduous task in the analysis of medical images, particularly the extraction of vessels from colored retinal fundus images. Many image processing tactics have been implemented for accurate detection of vessels, but many vessels have been dropped. In this paper, we propose a deep learning method based on the convolutional neural network (CNN) with dice loss function for retinal vessel segmentation. To our knowledge, we are the first to form the CNN on the basis of the dice loss function for the extraction of vessels from a colored retinal image. The pre-processing steps are used to eliminate uneven illumination to make the training process more efficient. We implement the CNN model based on a variational auto-encoder (VAE), which is a modified version of U-Net. Our main contribution to the implementation of CNN is to replace all pooling layers with progressive convolution and deeper layers. It takes the retinal image as input and generates the image of segmented output vessels with the same resolution as the input image. The proposed segmentation method showed better performance than the existing methods on the most used databases, namely: DRIVE and STARE. In addition, it gives a sensitivity of 0.739 on the DRIVE database with an accuracy of 0.948 and a sensitivity of 0.748 on the STARE database with an accuracy of 0.947.

Virtual viewpoint video needs to be synthesised from adjacent reference viewpoints to provide immersive perceptual 3D viewing experience of a scene. View synthesised techniques suffer poor rendering quality due to holes created by occlusion in the warping process. Currently, spatial and temporal correlation of texture images and depth maps are exploited to improve the quality of the final synthesised view. Due to the low spatial correlation at the edge between foreground and background pixels, spatial correlation e.g. inpainting and inverse mapping (IM) techniques cannot fill holes effectively. Conversely, a temporal correlation among already synthesised frames through learning by Gaussian mixture modelling (GMM) fill missing pixels in occluded areas efficiently. In this process, there are no frames for GMM learning when the user switches view instantly. To address the above issues, in the proposed view synthesis technique, we apply GMM on the adjacent reference viewpoint texture images and depth maps to generate a most common frame in a scene (McFIS). Then, texture McFIS is warped into the target viewpoint by using depth McFIS and both warped McFISes are merged. Then, we utilize the number of GMM models to refine pixel intensities of the synthesised view by using a weighting factor between the pixel intensities of the merged McFIS and the warped images. This technique provides a better pixel correspondence and improves 0.58∼0.70dB PSNR compared to the IM technique.

The free viewpoint video (FVV) allows users to interactively control the viewpoint and generate new views of a dynamic scene from any 3D position for better 3D visual experience with depth perception. Multiview video coding exploits both texture and depth video information from various angles to encode a number of views to facilitate FVV. The usual practice for the single view or multiview quality assessment is characterized by evolving the objective quality assessment metrics due to their simplicity and real time applications such as the peak signal-to-noise ratio (PSNR) or the structural similarity index (SSIM). However, the PSNR or SSIM requires reference image for quality evaluation and could not be successfully employed in FVV as the new view in FVV does not have any reference view to compare with. Conversely, the widely used subjective estimator- mean opinion score (MOS) is often biased by the testing environment, viewers mode, domain knowledge, and many other factors that may actively influence on actual assessment. To address this limitation, in this work, we devise a no-reference subjective quality assessment metric by simply exploiting the pattern of human eye browsing on FVV. Over different quality contents of FVV, the participants eye-tracker recorded spatio-temporal gaze-data indicate more concentrated eye-traversing approach for relatively better quality. Thus, we calculate the Length, Angle, Pupil-size, and Gaze-duration features from the recorded gaze trajectory. The content and resolution invariant operation is carried out prior to synthesizing them using an adaptive weighted function to develop a new quality metric using eye traversal (QMET). Tested results reveal that the proposed QMET performs better than the SSIM and MOS in terms of assessing different aspects of coded video quality for a wide range of FVV contents.

Electroencephalogram (EEG) has a great potential for diagnosis and treatment of brain disorders like epileptic seizure. Feature extraction and classification of EEG signals is the crucial task to detect the stages of ictal and interictal signals for treatment and precaution of epileptic patients. However, existing seizure and non-seizure feature extraction techniques are not good enough for the classification of ictal and interictal EEG signals considering the non-abruptness phenomena and inconsistency in different brain locations. In this study, the authors present a new approach for feature extraction and classification by exploiting temporal correlation within EEG signals for better seizure detection as any abruptness in the temporal correlation within a signal represents the transition of a phenomenon. In the proposed methods, they divide an EEG signal into a number of epochs and arrange them into two-dimensional matrix and then apply different transformation/decomposition to extract a number of statistical features. These features are then used as an input into LS-SVM to classify them. Experimental results show that the proposed methods outperform the existing state-of-the-art method for better classification in terms of sensitivity, specificity and accuracy of ictal and interictal period of epilepsy for benchmark datasets and different brain locations.

High quality virtual views need to be synthesized from adjacent available views for free viewpoint video and multiview video coding (MVC) to provide users with a more realistic 3D viewing experience of a scene. View synthesis techniques suffer from poor rendering quality due to holes created by occlusion and rounding integer error through warping. To remove the holes in the virtual view, the existing techniques use spatial and temporal correlation in intra/inter-view images and depth maps. However, they still suffer quality degradation in the boundary region of foreground and background areas due to the low spatial correlation in texture images and low correspondence in inter-view depth maps. To overcome the above-mentioned limitations, we use a number of models in the Gaussian mixture modeling (GMM) to separate background and foreground pixels in our proposed technique. Here, the missing pixels introduced from the warping process are recovered by the adaptive weighted average of the pixel intensities from the corresponding GMM model(s) and warped image. The weights vary with time to accommodate the changes due to a dynamic background and the motions of the moving objects for view synthesis. We also introduce an adaptive strategy to reset the GMM modeling if the contributions of the pixel intensities drop significantly. Our experimental results indicate that the proposed approach provides 5.40–6.60-dB PSNR improvement compared with the relevant methods. To verify the effectiveness of the proposed view synthesis technique, we use it as an extra reference frame in the motion estimation for MVC. The experimental results confirm that the proposed view synthesis is able to improve PSNR by 3.15–5.13 dB compared with the conventional three reference frames.

Video Summarization is a computer-based technique to generate a shorter version of the original long video for memory management and information retrieval. The existing method for video summarization applies affective (the state of excitement, interestingness, and panic) level of a viewer captured by Electroencephalography (EEG) while watching a video. The traditional methods for extracting features from EEG signals employ Discrete Fourier Transform (DFT). However, DFT is mainly suitable for stationary signals. As EEG signals are non-linear and non-stationary in nature, DFT is not appropriate for EEG signals. In addition to this, the high-frequency components of EEG signals usually contain the distinct properties than the other frequency components. Empirical Mode Decomposition (EMD) technique extracts the different frequency components from high to low from non-linear and non-stationary signals, such EEG. Therefore, we propose a new video summarization method applying EMD Decomposed EEG Signals (EDES) to extract the high-frequency components. The proposed approach calculates Power Spectral Density (PSD) from the high-frequency components and generates a neuronal attention curve for a video. Finally, a video summary is produced by selecting the affective video events from the neuronal attention curve. The experimental results reveal that the proposed approach performs better than the existing state- of-the-art method.

The popularly used subjective estimator- mean opinion score (MOS) is often biased by the testing environment, viewers mode, domain expertise, and many other factors that may actively influence on actual assessment. We therefore, devise a no- reference subjective quality assessment metric by exploiting the nature of human eye browsing on videos. The participants' eye-tracker recorded gaze-data indicate more concentrated eye- traversing approach for relatively better quality. We calculate the Length, Angle, Pupil-size, and Gaze-duration features from the recorded gaze trajectory. The content and resolution invariant operation is carried out prior to synthesizing them using an adaptive weighted function to develop a new quality metric using eye traversal (QMET). Tested results reveal that the quality evaluation carried out by QMET demonstrates a strong correlation with the most widely used peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and the MOS.

Video summarization is a process to extract objects and their activities from a video and represent them in a condensed form. Existing methods for video summarization fail to detect moving (dynamic) objects in the low color contrast area of a video frame due to the pixel intensities of objects and non-objects are almost similar. However, edges of objects are prominent in the low contrast regions. Moreover, to represent objects, geometric primitives (such as lines, arcs) are distinguishable and high level shape descriptors than edges. In this paper, a novel method is proposed for video summarization using geometric primitives such as conic parts, line segments and angles. Using these features, objects are extracted from each video frame. A cost function is applied to measure the dissimilarity of locations of geometric primitives to detect the movement of objects between consecutive frames. The total distance of object movement is calculated and each video frame is assigned a probability score. Finally, a set of key frames is selected based on the probability scores as per user provided skimming ratio or system default skimming ratio. The proposed approach is evaluated using three benchmark datasets-BL-7F, Office, and Lobby. The experimental results show that our approach outperforms the state-of-the-art method in terms of accuracy.

Accurate vessel segmentation is a tough task for various medical images applications especially the segmentation of retinal images vessels. A computerised algorithm is required for analysing the progress of eye diseases. A variety of computerised retinal segmentation methods have been proposed but almost all methods to date show low sensitivity for narrowly low contrast vessels. We propose a new retinal vessel segmentation algorithm to address the issue of low sensitivity. The proposed method introduces a deep learning model along with pre-processing and post-processing. The pre-processing is used to handle the issue of uneven illuminations. We design a fully Convolutional Neural Network (CNN) and train it to get fine vessels observation.The post-processing step is used to remove the background noise pixels to achieve well-segmented vessels. The proposed segmentation method gives good segmented images especially for detecting tiny vessels. We evaluate our method on the commonly used publicly available databases: DRIVE and STARE databases. The higher sensitivity of 75% leads to proper detection of tiny vessels with an accuracy of 94.7%.

With the development of displaying techniques, free viewpoint video (FVV) system shows its potential to provide immersive perceptual feeling by changing viewpoints. To provide this luxury, a large number of high quality views have to be synthesised from limited number of viewpoints. However, in this process, a portion of the background is occluded by the foreground object in the generated synthesised videos. Recent techniques, i.e. view synthesized prediction using Gaussian model (VSPGM) and adaptive weighting between warped and learned foregrounds indicate that learning techniques may fill occluded areas almost correctly. However, these techniques use temporal correlation by assuming that original texture of the target viewpoint are already available to fill up occluded areas which is not a practical solution. Moreover, if a pixel position experiences foreground once during learning, the existing techniques considered it as foreground throughout the process. However, the actual fact is that after experiencing a foreground a pixel position can be background again. To address the aforementioned issues, in the proposed view synthesise technique, we apply Gaussian mixture modelling (GMM) on the output images of inverse mapping (IM) technique for further improving the quality of the synthesised videos. In this technique, the foreground and background pixel intensities are refined from adaptive weights of the output of inverse mapping and the pixel intensities from the corresponding model(s) of the GMM. This technique provides a better pixel correspondence, which improves 0.10~0.46dB PSNR compared to the IM technique.

The eye disease such as Diabetic Retinopathy(DR) can be analysed through segmentation of retinal blood vessels. In the last five years, many methods for retinal blood vessels segmentation were proposed. These methods give arise to the improved accuracy, however the sensitivity of low contrast vessels is often ignored. The performance of diagnosis in terms of segmentation of vessels can be degraded due to missing tiny vessels. In this study, we propose a novel algorithm aiming at improving the performance of segmenting small vessels. The proposed approach adopts a morphological and filtering method to handle the background noise and uneven illumination and uses anisotropic diffusion filtering to coherent the vessels and give initial detection of vessels, followed by a double threshold based region growing method.

In a free viewpoint video (FVV) system, a user has a freedom to choose a view angle arbitrarily. To facilitate view synthesis, we need to interpolate a number of adjacent views by exploiting spatial and temporal redundancy among them. Existing synthesizing techniques may concern poor rendering quality due to the occluded background areas and rounding integer error through warping. Currently, inpainting and background update techniques are used to address these issues. Inpainting techniques normally suffer quality degradation due to the low spatial correlation in the foreground/background boundary areas. Background update technique improves the quality, however, still suffers quality degradation due to the dependency on background warping and spatial correlation. To address the problem, in this paper we learn the foreground and background in each pixel using Gaussian mixture model (GMM) on the previous images in the interpolated view and then use the number of Gaussian models to identify the foreground and background pixels. The background pixel of the synthesized view is determined using the background model and the foreground pixel is determined using the weighted contribution of the foreground model and the warping image between two adjacent views. An adaptive weighting is proposed in this paper based on the amount of multiple models. The experimental results reveal that the proposed approach provides 0.60∼3.15dB PSNR improvement of the synthesized view compared with the three standard state-of-the-art methods.

Predicting upcoming bands of hyperspectral images is an important task in modern image compression algorithms. This paper proposes a new algorithm to predict the band-wise correlation of hyperspectral images based on a generalized regression neural network (GRNN). The proposed algorithm uses the intensity values of the previous bands to train the GRNN and approximates the correlation between them. The next band is then predicted using the trained network and the immediately previous band. This algorithm works on a pixel-by-pixel basis and does not involve any mathematical modeling or any previous knowledge of the images. The performance of the proposed algorithm is evaluated by applying it to several Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) reflectance datasets. Simulation results show that the proposed algorithm provides substantial accuracy in the prediction of upcoming bands.

Existing methods for video summarization fails to achieve a satisfactory result for a video with camera movement, low contrast, and significant illumination changes. To solve these problems, we propose a novel framework for video summarization based on the smooth pursuit which is the state of eye movement when a user follows a moving object in a video. First, we propose a new method to distinguish smooth pursuit from another type of gaze points, such as fixation and saccade. Later, we assign a probability score to each frame based on the smooth pursuit information. Finally, we select a set of key frames based on the probability score. To evaluate the proposed method, we implement it on Office video dataset that contains videos with camera movement/shaking and illumination changes. Experimental results show the superior performance compared to the single view results of the state-of-the-art GMM based method.