Search

Video coding using dynamic background frame achieves better compression compared to the traditional techniques by encoding background and foreground separately. This process reduces coding bits for the overall frame significantly; however, encoding background still requires many bits that can be compressed further for achieving better coding efficiency. The cuboid coding framework has been proven to be one of the most effective methods of image compression which exploits homogeneous pixel correlation within a frame and has better alignment with object boundary compared to traditional block-based coding. In a video sequence, the cuboid-based frame partitioning varies with the changes of the foreground. However, since the background remains static for a group of pictures, the cuboid coding exploits better spatial pixel homogeneity. In this work, the impact of cuboid coding on the background frame for high-resolution videos (Ultra-High-Definition (UHD) and 360-degree videos) is investigated using the multilayer framework of SHVC. After the cuboid partitioning, the method of coarse frame generation has been improved with a novel idea by keeping human-visual sensitive information. Unlike the traditional SHVC scheme, in the proposed method, cuboid coded background and the foreground are encoded in separate layers in an implicit manner. Simulation results show that the proposed video coding method achieves an average BD-Rate reduction of 26.69% and BD-PSNR gain of 1.51 dB against SHVC with significant encoding time reduction for both UHD and 360 videos. It also achieves an average of 13.88% BD-Rate reduction and 0.78 dB BD-PSNR gain compared to the existing relevant method proposed by X. Hoang Van.

Immersive media representation format based on point clouds has underpinned significant opportunities for extended reality applications. Point cloud in its uncompressed format require very high data rate for storage and transmission. The video based point cloud compression (V-PCC) technique projects a dynamic point cloud into geometry and texture video sequences. The projected texture video is then coded using modern video coding standard like HEVC. Since the properties of projected texture video frames are different from traditional video frames, HEVC-based commonality modeling can be inefficient. An improved commonality modeling technique is proposed that employs edge position difference oriented motion model. Experimental results show that the proposed commonality modeling technique can yield savings in bit rate of up to 3.15% over the V-PCC HEVC reference encoder.

The median survival time of patients with high grade glioma, a form of brain tumour, is 1-3 years. The current best practice adopts Convolutional Neural Network (CNN) for image classification and tumour detection. This method provides a significant improvement in brain tumour segmentation of Magnetic Resonance Imaging (MRI) images in comparison to other frameworks, but it is nonetheless slow and lacks precision. We sought to build upon the current best practice model by utilising a Deep Neural Network (DNN) model, which entailed modification of the segmentation and feature-extraction stages in order to improve the accuracy of those stages and the resulting segmentation. We contrasted the accuracy and efficiency of our model to the current best practice model using 10 brain tumour patient MRI datasets. First, the segmentation accuracy of our proposed model (M= 90%) outperformed that of the current best practice (M=78%). Second, the tumour detection processing time of our proposed model (M=34 ms) also outperformed that of the current best practice (M=73 ms). We, therefore, replicated previous studies by showing that automatic segmentation can aid in brain tumour detection. Importantly, we extended previous studies by proposing a model that classifies a brain tumour with greater accuracy and within lower processing times. Validation of the model with a larger dataset is recommended.

With the growth of video technologies, super-resolution videos, including 360-degree immersive video has become a reality due to exciting applications such as augmented/virtual/mixed reality for better interaction and a wide-angle user-view experience of a scene compared to traditional video with narrow-focused viewing angle. The new generation video contents are bandwidth-intensive in nature due to high resolution and demand high bit rate as well as low latency delivery requirements that pose challenges in solving the bottleneck of transmission and storage burdens. There is limited optimisation space in traditional video coding schemes for improving video coding efficiency in intra-frame due to the fixed size of processing block. This paper presents a new approach for improving intra-frame coding especially at low bit rate video transmission for 360-degree video for lossy mode of HEVC. Prior to using traditional HEVC intra-prediction, this approach exploits the global redundancy of entire frame by extracting common important information using multi-level discrete wavelet transformation. This paper demonstrates that the proposed method considering only low frequency information of a frame and encoding this can outperform the HEVC standard at low bit rates. The experimental results indicate that the proposed intra-frame coding strategy achieves an average of 54.07% BD-rate reduction and 2.84 dB BD-PSNR gain for low bit rate scenario compared to the HEVC. It also achieves a significant improvement in encoding time reduction of about 66.84% on an average. Moreover, this finding also demonstrates that the existing HEVC block partitioning can be applied in the transform domain for better exploitation of information concentration as we applied HEVC on wavelet frequency domain.

Video coding algorithms attempt to minimize the significant commonality that exists within a video sequence. Each new video coding standard contains tools that can perform this task more efficiently compared to its predecessors. In this work, we form a coarse representation of the current frame by minimizing commonality within that frame while preserving important structural properties of the frame. The building blocks of this coarse representation are rectangular regions called cuboids, which are computationally simple and has a compact description. Then we propose to employ the coarse frame as an additional source for predictive coding of the current frame. Experimental results show an improvement in bit rate savings over a reference codec for HEVC, with minor increase in the codec computational complexity.

In conventional video compression systems, motion model is used to approximate the geometry of moving object boundaries. It is possible to relieve motion model from describing discontinuities in the underlying motion field, by incorporating motion hint that can predict the spatial structure of future frames using the structure of reference frames. However, formation of highly accurate motion hint is computationally demanding, in particular for high resolution video sequences. Cuboids, rectangular regions derived using statistical features, attempt to separate out different objects present in the scene; they are computationally efficient and have sparse representation. Leveraging on the advantages of cuboids, in this paper, we propose to discover homogeneous motion regions and their associated motion based on cuboids. Afterwards, the estimated motion models and their domains are applied to form a prediction of the current frame. Experimental results show that a savings in bit rate of 3.96% is achievable over standalone HEVC reference, if this predicted frame is used as an additional reference frame for the current frame.

The latest high efficiency video coding (HEVC) standard introduces a large number of inter-mode block partitioning modes. The HEVC reference test model (HM) uses partially exhaustive tree-structured mode selection, which still explores a large number of prediction unit (PU) modes for a coding unit (CU). This impacts on encoding time rise which deprives a number of electronic devices having limited processing resources to use various features of HEVC. By analyzing the homogeneity, residual, and different statistical correlation among modes, many researchers speed-up the encoding process through the number of PU mode reduction. However, these approaches could not demonstrate the similar rate-distortion (RD) performance with the HM due to their dependency on existing Lagrangian cost function (LCF) within the HEVC framework. In this paper, to avoid the complete dependency on LCF in the initial phase, we exploit visual sensitive foreground motion and spatial salient metric (FMSSM) in a block. To capture its motion and saliency features, we use the dynamic background and visual saliency modeling, respectively. According to the FMSSM values, a subset of PU modes is then explored for encoding the CU. This preprocessing phase is independent from the existing LCF. As the proposed coding technique further reduces the number of PU modes using two simple criteria (i.e., motion and saliency), it outperforms the HM in terms of encoding time reduction. As it also encodes the uncovered and static background areas using the dynamic background frame as a substituted reference frame, it does not sacrifice quality. Tested results reveal that the proposed method achieves 32% average encoding time reduction of the HM without any quality loss for a wide range of videos.

Capturing images in the challenging atmosphere for example during rain and snow critically degrades the quality of the images. These external phenomena create low contrast, blur and thereby reduce the visibility of the images. Many computer vision applications like visual traffic surveillance, intelligent vehicles, and entertainments are affected by reduced visibility. Rain streaks removal in a video (RSRV) has significant importance in the outdoor vision systems like surveillance video and has recently been studied comprehensively. In the last few years, many approaches have been made towards solving this problem including matrix decomposition, convolutional neural network, streak orientation modelling, Gaussian mixture modelling (GMM), etc. In this paper, we propose a concise model for RSRV based on the spatiotemporal appearance (STA) of rain streak in a video to exploit the rain streak appearance property. Here we apply Gaussian mixture modelling (GMM) to separate the background and foreground. Then, we use STA to separate the rain streak from moving foreground. The experimental results establish that the proposed method outperforms the state-of-the-art methods significantly for both real and synthetic rains.

Motion modeling plays a central role in video compression. This role is even more critical in fisheye video sequences since the wide-angle fisheye imagery has special characteristics as in exhibiting radial distortion. While the translational motion model employed by modern video coding standards, such as HEVC, is sufficient in most cases, using higher order models is beneficial; for this reason, the upcoming video coding standard, VVC, employs a 4-parameter affine model. Discrete cosine basis has the ability to efficiently model complex motion fields. In this work, we investigate the motion modeling behaviour of the discrete cosine basis equipped with higher frequency cosine vectors. In particular, the developed discrete cosine basis is used as a single high-order model to describe a fisheye frame's motion; we employ this motion to produce an extra prediction reference, which is added to the HEVC list of references. Experimental results show an increase in delta bit rate, over conventional HEVC, when higher frequency cosine vectors are added in the motion modeling process. Then leveraging on this modified discrete cosine basis, we propose to employ it for predicting the motion in 360 degree video frames because of their resemblance with fisheye images. In this case, a delta bit rate of 2% is achieved, over conventional HEVC.

Visual features are generally used in video summarization including sports videos. However, audio features are also important in summarization as they characterize the exciting events of a sport, especially when external thing e.g. raining effects the visual feature. The commentator's voice, referee's whistle, and audience-voice are strongly correlated to the exciting events e.g. goal, penalty shootout, red card, etc. in a soccer game. The existing methods using original audio signals could not provide expected summary due to the various noises from the audience such as horn and sound from various musical instruments. In this paper, we propose a novel approach to extract audio features using Empirical Mode Decomposition (EMD) that can filter out noises. We have conducted a set of experiments on four real soccer videos. The experimental results demonstrate that the proposed method can detect goal events with 96% accuracy which outperforms the existing state-of-the-art method by 42%.

Due to the powerful image editing tools, image integrity protection is becoming important, for instance, in a court of law, news reports, insurance claims or telemedicine. Digital image watermarking is a technique to detect, localize and recover tampered image but the accuracy of performance by the existing methods is still an issue. This paper proposes a new watermarking embedding mechanism for effective image authentication with a recovery scheme. In the method, a host image is divided into blocks and the feature of each block is embedded in the block itself and another block as watermark. For detecting tampering, the extracted watermarked data is checked with source and destination block leading to more accuracy. The recovery data is extracted from the destination block which is intact. The proposed scheme achieves better results in terms of tamper detection accuracy and the quality of recovered image compared with the state-of-art schemes.

With the advancement of displaying technologies, virtual viewpoint video needs to be synthesized from adjacent viewpoints to provide immersive perceptual viewing experience of a scene. View synthesized techniques suffer poor rendering quality due to holes created by occlusion in the warping process. Currently, spatial and temporal correlation techniques are used to improve the quality of the synthesized view. However, spatial correlation e. g. inpainting and inverse mapping (IM) techniques cannot fill holes efficiently due to low spatial correlation in the edge between foreground and background pixels. On the other hand, the temporal correlation among already synthesized frames through learning by Gaussian mixture modelling (GMM) may fill occluded areas efficiently. However, there are no frames for GMM learning when the user switches view instantly. To address the aforementioned issues, in the proposed view synthesis technique, we apply GMM on the adjacent viewpoint videos. Then, we utilize the number of GMM models to refine pixel intensities of the synthesized view by using a weighting factor between pixel intensities in GMM models and warped images. This technique provides a better pixel correspondence, which improves 0.47~0.58dB PSNR compared to the IM technique.

Accurate segmentation of vessels is an arduous task in the analysis of medical images, particularly the extraction of vessels from colored retinal fundus images. Many image processing tactics have been implemented for accurate detection of vessels, but many vessels have been dropped. In this paper, we propose a deep learning method based on the convolutional neural network (CNN) with dice loss function for retinal vessel segmentation. To our knowledge, we are the first to form the CNN on the basis of the dice loss function for the extraction of vessels from a colored retinal image. The pre-processing steps are used to eliminate uneven illumination to make the training process more efficient. We implement the CNN model based on a variational auto-encoder (VAE), which is a modified version of U-Net. Our main contribution to the implementation of CNN is to replace all pooling layers with progressive convolution and deeper layers. It takes the retinal image as input and generates the image of segmented output vessels with the same resolution as the input image. The proposed segmentation method showed better performance than the existing methods on the most used databases, namely: DRIVE and STARE. In addition, it gives a sensitivity of 0.739 on the DRIVE database with an accuracy of 0.948 and a sensitivity of 0.748 on the STARE database with an accuracy of 0.947.

Virtual viewpoint video needs to be synthesised from adjacent reference viewpoints to provide immersive perceptual 3D viewing experience of a scene. View synthesised techniques suffer poor rendering quality due to holes created by occlusion in the warping process. Currently, spatial and temporal correlation of texture images and depth maps are exploited to improve the quality of the final synthesised view. Due to the low spatial correlation at the edge between foreground and background pixels, spatial correlation e.g. inpainting and inverse mapping (IM) techniques cannot fill holes effectively. Conversely, a temporal correlation among already synthesised frames through learning by Gaussian mixture modelling (GMM) fill missing pixels in occluded areas efficiently. In this process, there are no frames for GMM learning when the user switches view instantly. To address the above issues, in the proposed view synthesis technique, we apply GMM on the adjacent reference viewpoint texture images and depth maps to generate a most common frame in a scene (McFIS). Then, texture McFIS is warped into the target viewpoint by using depth McFIS and both warped McFISes are merged. Then, we utilize the number of GMM models to refine pixel intensities of the synthesised view by using a weighting factor between the pixel intensities of the merged McFIS and the warped images. This technique provides a better pixel correspondence and improves 0.58∼0.70dB PSNR compared to the IM technique.

Proprioception plays an important role in neuromuscular control and stability. Taping the knee or ankle with strapping tape (ST) is a common means to increase stability and limit unwanted joint motion. Kinesiology tape (KT) is an alternative tape has been proposed to enhance proprioceptive information from the skin muscles and joints. Comparisons of muscle responses associated with different taping methods has not been investigated using a joint reposition test (JRT). The current study investigated lower limb muscle activity during a blind folded JRT in a group of college students with no known injuries. Thirty nine healthy college students between 18–35 years of age were recruited using convenience sampling. Electromyographical (EMG) data was recorded from lower limb muscles and 3D video recordings tracked knee joint angle accuracy for the JRT. Participants were blindfolded and guided to 40 degrees of knee flexion by the experimenter, and were asked to repeat this joint position five times unaided. Higuchi fractal dimension and sample entropy were used to determine the nonlinear dynamic properties of the muscle responses. Statistical analysis was with repeated measures ANOVA and Tukey post hoc test. Significance was set at p

Video Summarization is a computer-based technique to generate a shorter version of the original long video for memory management and information retrieval. The existing method for video summarization applies affective (the state of excitement, interestingness, and panic) level of a viewer captured by Electroencephalography (EEG) while watching a video. The traditional methods for extracting features from EEG signals employ Discrete Fourier Transform (DFT). However, DFT is mainly suitable for stationary signals. As EEG signals are non-linear and non-stationary in nature, DFT is not appropriate for EEG signals. In addition to this, the high-frequency components of EEG signals usually contain the distinct properties than the other frequency components. Empirical Mode Decomposition (EMD) technique extracts the different frequency components from high to low from non-linear and non-stationary signals, such EEG. Therefore, we propose a new video summarization method applying EMD Decomposed EEG Signals (EDES) to extract the high-frequency components. The proposed approach calculates Power Spectral Density (PSD) from the high-frequency components and generates a neuronal attention curve for a video. Finally, a video summary is produced by selecting the affective video events from the neuronal attention curve. The experimental results reveal that the proposed approach performs better than the existing state- of-the-art method.

The popularly used subjective estimator- mean opinion score (MOS) is often biased by the testing environment, viewers mode, domain expertise, and many other factors that may actively influence on actual assessment. We therefore, devise a no- reference subjective quality assessment metric by exploiting the nature of human eye browsing on videos. The participants' eye-tracker recorded gaze-data indicate more concentrated eye- traversing approach for relatively better quality. We calculate the Length, Angle, Pupil-size, and Gaze-duration features from the recorded gaze trajectory. The content and resolution invariant operation is carried out prior to synthesizing them using an adaptive weighted function to develop a new quality metric using eye traversal (QMET). Tested results reveal that the quality evaluation carried out by QMET demonstrates a strong correlation with the most widely used peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and the MOS.

Accurate vessel segmentation is a tough task for various medical images applications especially the segmentation of retinal images vessels. A computerised algorithm is required for analysing the progress of eye diseases. A variety of computerised retinal segmentation methods have been proposed but almost all methods to date show low sensitivity for narrowly low contrast vessels. We propose a new retinal vessel segmentation algorithm to address the issue of low sensitivity. The proposed method introduces a deep learning model along with pre-processing and post-processing. The pre-processing is used to handle the issue of uneven illuminations. We design a fully Convolutional Neural Network (CNN) and train it to get fine vessels observation.The post-processing step is used to remove the background noise pixels to achieve well-segmented vessels. The proposed segmentation method gives good segmented images especially for detecting tiny vessels. We evaluate our method on the commonly used publicly available databases: DRIVE and STARE databases. The higher sensitivity of 75% leads to proper detection of tiny vessels with an accuracy of 94.7%.

With the development of displaying techniques, free viewpoint video (FVV) system shows its potential to provide immersive perceptual feeling by changing viewpoints. To provide this luxury, a large number of high quality views have to be synthesised from limited number of viewpoints. However, in this process, a portion of the background is occluded by the foreground object in the generated synthesised videos. Recent techniques, i.e. view synthesized prediction using Gaussian model (VSPGM) and adaptive weighting between warped and learned foregrounds indicate that learning techniques may fill occluded areas almost correctly. However, these techniques use temporal correlation by assuming that original texture of the target viewpoint are already available to fill up occluded areas which is not a practical solution. Moreover, if a pixel position experiences foreground once during learning, the existing techniques considered it as foreground throughout the process. However, the actual fact is that after experiencing a foreground a pixel position can be background again. To address the aforementioned issues, in the proposed view synthesise technique, we apply Gaussian mixture modelling (GMM) on the output images of inverse mapping (IM) technique for further improving the quality of the synthesised videos. In this technique, the foreground and background pixel intensities are refined from adaptive weights of the output of inverse mapping and the pixel intensities from the corresponding model(s) of the GMM. This technique provides a better pixel correspondence, which improves 0.10~0.46dB PSNR compared to the IM technique.

This paper addresses the problem of error-resilient source coding for 3-D video transmission over packet-loss networks. The proposed approach jointly optimizes the texture coding mode and the depth coding mode for each macroblock in the reference views. Firstly, a distortion model is developed to capture the effect of the texture distortion and depth distortion on the synthesized view. Then, joint optimization of texture and depth coding modes is derived based upon an operational rate-distortion framework using Lagrange multiplier method. In particular, a dual trellis-based algorithm is introduced in order to overcome the macroblock interdependencies of texture and depth map in the optimization procedure. Simulation results demonstrate that significant and consistent gains can be achieved over currently used techniques.

High Efficiency Video Coding (HEVC) is the latest video encoding standard and has approximately 50% bit-rate saving compared to its predecessor. However, the motion estimation (ME) is considerably complicated by the incorporation of varieties of partitioning modes and a quad-tree based coding structure, and also by increasing the basic coding unit size by a factor of 16. Motion estimation is the most complex task in the video encoding process, consuming 60–80% of overall encoding time. This paper proposes a new algorithm, angle-restricted test zone (ARTZ) for motion estimation which is based on a test zone (TZ) search, exploiting directional probabilities of motion vector search. In our experiments, this proposal achieves a time saving in motion estimation of about 20% to 50% compared to a TZ search in the HEVC test model (HM) implementation for UHD videos without significant degradation of PSNR.

The eye disease such as Diabetic Retinopathy(DR) can be analysed through segmentation of retinal blood vessels. In the last five years, many methods for retinal blood vessels segmentation were proposed. These methods give arise to the improved accuracy, however the sensitivity of low contrast vessels is often ignored. The performance of diagnosis in terms of segmentation of vessels can be degraded due to missing tiny vessels. In this study, we propose a novel algorithm aiming at improving the performance of segmenting small vessels. The proposed approach adopts a morphological and filtering method to handle the background noise and uneven illumination and uses anisotropic diffusion filtering to coherent the vessels and give initial detection of vessels, followed by a double threshold based region growing method.

In a free viewpoint video (FVV) system, a user has a freedom to choose a view angle arbitrarily. To facilitate view synthesis, we need to interpolate a number of adjacent views by exploiting spatial and temporal redundancy among them. Existing synthesizing techniques may concern poor rendering quality due to the occluded background areas and rounding integer error through warping. Currently, inpainting and background update techniques are used to address these issues. Inpainting techniques normally suffer quality degradation due to the low spatial correlation in the foreground/background boundary areas. Background update technique improves the quality, however, still suffers quality degradation due to the dependency on background warping and spatial correlation. To address the problem, in this paper we learn the foreground and background in each pixel using Gaussian mixture model (GMM) on the previous images in the interpolated view and then use the number of Gaussian models to identify the foreground and background pixels. The background pixel of the synthesized view is determined using the background model and the foreground pixel is determined using the weighted contribution of the foreground model and the warping image between two adjacent views. An adaptive weighting is proposed in this paper based on the amount of multiple models. The experimental results reveal that the proposed approach provides 0.60∼3.15dB PSNR improvement of the synthesized view compared with the three standard state-of-the-art methods.