Search

Watermarking is an important measure for protecting proprietary digital multimedia data. This paper presents a novel approach to achieving robust and imperceptible blind speech watermarking on a frame-by-frame basis. The proposed method employs two modules operating in the fast Fourier transform (FFT) domain. The first module is referred to as downward progressive quantization index modulation. It modulates the vector norms drawn from FFT coefficients according to a guideline deduced from human auditory masking properties. The second module is referred to as boundary-constrained iterative adjustment. It provides a smooth transition across frames in the resulting speech waveform. Experiment results confirm the imperceptibility of the proposed modulation scheme in terms of the mean opinion score – listening quality objective (MOS–LQO) based on the perceptual evaluation of speech quality (PESQ) metric. The proposed watermarking method matched and exceeded the performance of five state-of-the-art methods in terms of robustness against common speech processing attacks.

This paper examines influences of the network latency on cooperative work in a networked virtual environment with haptics by QoE (Quality of Experience) subjective and objective assessment. As the work, two users cooperatively carry an object in a networked maze system by using haptic interface devices. Assessment results demonstrate that as the network latency becomes longer, MOS (Mean Opinion Score) as a subjective measure becomes higher. We also clarify the relations between MOS and several objective measures by regression analysis.

This paper highlights a practical approach for management and orchestration (MANO) of Network Function Virtualization (NFV) based on Open Source MANO (OSM). We have proposed a testbed that uses the Open Source Orchestration Platform for OSM to manage and orchestrate Voice over Internet Protocol (VoIP) Our testbed has two Virtual Infrastructure Managers (VIMs, OpenStack) logically located in a separate area, which places us in a Wide Area Network (WAN) context. The concept of Software-Defined Networking (SDN) has been implemented through an Open Network Operating System (ONOS) controller managing the traffic between the different VIMs. Our experiments show that the solution is feasible, with latency, jitter, packet loss and Mean Opinion Score all within the required range.

Conventional singing voice conversion (SVC) methods often suffer from operating in high-resolution audio owing to a high dimensionality of data. In this paper, we propose a hierarchical representation learning that enables the learning of disentangled representations with multiple resolutions independently. With the learned disentangled representations, the proposed method progressively performs SVC from low to high resolutions. Experimental results show that the proposed method outperforms baselines that operate with a single resolution in terms of mean opinion score (MOS), similarity score, and pitch accuracy., accepted at IJCNN 2021

Unlike traditional 2D image quality assessment, the structural relationship among sub-aperture images (SAIs) is an essential factor affecting the quality evaluation of light field (LF) images, where the labeled datasets are also not sufficient for improving learning performances. To solve these problems, we present a novel deep neural network-based approach to accurately predict the quality of compressed LF images without pristine images. Two modules dubbed SAI-Fusion and Global Context Perception (GCP) are proposed to obtain the relationship among SAIs. For effective training, we compress LF images from EPFL and HCI datasets and propose a ranking-based method to generate pseudo-labels as equivalents of Mean Opinion Score (MOS), i.e., Ranking-MOS. Therefore, we can pre-train our quality assessment network on compressed LF images with Ranking-MOS, and then fine-tune the model at small-scale datasets with real labels. Experiments demonstrate that the proposed method achieves state-of-the-art performance on compressed LF images of Win5-LID dataset.

Point cloud (PC) quality assessment is of fundamental importance to enable the efficient processing, coding and transmission of 3D data for virtual/augmented reality, autonomous driving, cultural heritage, etc. The quality metrics proposed so far aim at quantifying the distortion in the PC geometry and/or attributes with respect to a reference pristine point cloud, using simple features extracted by the points. In this work, we target instead a blind (no-reference) scenario in which the original point cloud is not available. In addition, we learn features from data using deep neural networks. Given the limited availability of subjectively annotated datasets of corrupted point clouds, and the consequent difficulty to learn in an end-to-end fashion PC quality features, in this work we use instead a two-step procedure. First, we extract from 10-cal patches three relevant low-level features which have been commonly used in other PC quality metrics, i.e., geometric distance, local curvature and luminance values. Afterwards, we employ a deep neural network to learn, from these low-level features, a mapping to the PC ground truth mean opinion score. Our results on two state-of-the-art PC quality datasets show the potential of the proposed approach. The code is available here: https://git;hub.com/achet;ouani/NR-CNN-3D-PC

Text to speech (TTS) has been evolving rapidly these years. Researchers have successfully converted English text into speech which sounds like natural speaker, proposing numerous models from RNN to non-autoregressive network. However, the migration of these models to Chinese TTS is still an issue because of its prosodic phrasing problems and large character set, not to mention the disappointing outcomes of those successfully-migrated models, most of which are autoregressive. In this paper, we successfully migrate FastSpeech2 to the field of Chinese TTS with generative adversarial network (GAN) as its discriminator for training to enhance the outcome. Postnet of Tactron2 is also applied to fine-tune the mel-spectrogram. We also use x-vector-based voiceprint extraction model to extract voiceprint to achieve voice cloning. The experiment is operated on both models which offers results of 3.83 mean opinion score (MOS) in terms of naturalness and 3.82 MOS in terms of similarity.

Packet loss may affect a wide range of applications that use voice over IP (VoIP), e.g. video conferencing. In this paper, we investigate a time-domain convolutional recurrent network (CRN) for online packet loss concealment. The CRN comprises a convolutional encoder-decoder structure and long short-term memory (LSTM) layers, which have been shown to be suitable for real-time speech enhancement applications. Moreover, we propose lookahead and masked training to further improve the performance of the CRN framework. Experimental results show that the proposed system outperforms a baseline system using only LSTM layers in terms of two objective metrics – perceptual evaluation of speech quality (PESQ) and short-term objective intelligibility (STOI); it also reduces the word error rate (WER) more than the baseline when used as a frontend for speech recognition. The advantage of the proposed system is also verified in a subjective evaluation by the mean opinion score (MOS).

Syntactic structure of a sentence text is correlated with the prosodic structure of the speech that is crucial for improving the prosody and naturalness of a text-to-speech (TTS) system. Nowadays TTS systems usually try to incorporate syntactic structure information with manually designed features based on expert knowledge. In this paper, we propose a syntactic representation learning method based on syntactic parse tree traversal to automatically utilize the syntactic structure information. Two constituent label sequences are linearized through left-first and right-first traversals from constituent parse tree. Syntactic representations are then extracted at word level from each constituent label sequence by a corresponding uni-directional gated recurrent unit (GRU) network. Meanwhile, nuclear-norm maximization loss is introduced to enhance the discriminability and diversity of the embeddings of constituent labels. Upsampled syntactic representations and phoneme embeddings are concatenated to serve as the encoder input of Tacotron2. Experimental results demonstrate the effectiveness of our proposed approach, with mean opinion score (MOS) increasing from 3.70 to 3.82 and ABX preference exceeding by 17% compared with the baseline. In addition, for sentences with multiple syntactic parse trees, prosodic differences can be clearly perceived from the synthesized speeches., Comment: This paper was submitted to ICASSP2021

This paper applies the dual-signal transformation LSTM network (DTLN) to the task of real-time acoustic echo cancellation (AEC). The DTLN combines a short-time Fourier transform and a learned feature representation in a stacked network approach, which enables robust information processing in the time-frequency and in the time domain, which also includes phase information. The model is only trained on 60 h of real and synthetic echo scenarios. The training setup includes multi-lingual speech, data augmentation, additional noise and reverberation to create a model that should generalize well to a large variety of real-world conditions. The DTLN approach produces state-of-the-art performance on clean and noisy echo conditions reducing acoustic echo and additional noise robustly. The method outperforms the AEC-Challenge baseline by 0.30 in terms of Mean Opinion Score (MOS).

Research on single channel speech enhancement (SE) has a long tradition, but two main practical problems still remain unsolved. Firstly, it’s hard to balance between enhancement quality and computational efficiency, and low-latency always brings loss of quality. Secondly, enhancement in specific scenarios, such as singing and emotional speech, is also an intricate problem of conventional methods. In this paper, we propose a computationally efficient real-time speech enhancement network with densely connected multi-stage structures, which progressively enhances the channel-wise subband speech. The enhanced speech from earlier stage is used to guide the processing of deeper stage in order to obtain coarse to fine estimations. Besides, supervision is applied to all intermediate results in order to stabilize training and accelerate convergence. Moreover, an adaptive fine-tune step is utilized with some small datasets of specific scenarios, which achieves superb improvement under corresponding scenes. As a result, the proposed method achieves promising performance improvements in terms of speech quality and demonstrates robustness in complex scenarios. We submitt the proposed method to the deep noise suppression (DNS) challenge 2021, real-time denoising track, which was held by Microsoft. In the subjective evaluation, our system outperforms DNS-Challenge baseline by 0.14 points in terms of mean opinion score (MOS).

Cross-lingual voice conversion (VC) is an important and challenging problem due to significant mismatches of the phonetic set and the speech prosody of different languages. In this paper, we build upon the neural text-to-speech (TTS) model, i.e., FastSpeech, and LPCNet neural vocoder to design a new cross-lingual VC framework named FastSpeech-VC. We address the mismatches of the phonetic set and the speech prosody by applying Phonetic PosteriorGrams (PPGs), which have been proved to bridge across speaker and language boundaries. Moreover, we add normalized logarithm-scale fundamental frequency (Log-F0) to further compensate for the prosodic mismatches and significantly improve naturalness. Our experiments on English and Mandarin languages demonstrate that with only mono-lingual corpus, the proposed FastSpeech-VC can achieve high quality converted speech with mean opinion score (MOS) close to the professional records while maintaining good speaker similarity. Compared to the baselines using Tacotron2 and Transformer TTS models, the FastSpeech-VC can achieve controllable converted speech rate and much faster inference speed. More importantly, the FastSpeech-VC can easily be adapted to a speaker with limited training utterances., 5 pages, 2 figures, 4 tables, accepted by ICASSP 2021

State-of-the-art neural text-to-speech (TTS) networks are trained with a large amount of speech data, which significantly improves the quality of synthetic speech compared with traditional approaches. However, the prosody and controllability of the generated speech is still insufficient, especially in tonal languages. Moreover, the generated prosody is solely defined by the input text, which does not allow for different styles for the same sentence or words. In this study, we extended Tacotron2 with a pitch prediction task to capture discrete pitch-related representations. Specifically, the learned pitch-related suprasegmental information is fed simultaneously with traditional character features into the decoder to generate final Mel spectrogram. Experiments show that the proposed method can improve the quality of the generated speech (mean opinion score of 4.37 vs. 4.22). Moreover, we demonstrated that we can easily achieve word-level pitch control during generation by changing local pitch-related representations before passing them to the decoder network.

Recently, deep convolutional neural networks (DCNN) that leverage the adversarial training framework for image restoration and enhancement have significantly improved the processed images' sharpness. Surprisingly, although these DCNNs produced crispier images than other methods visually, they may get a lower quality score when popular measures are employed for evaluating them. Therefore it is necessary to develop a quantitative metric to reflect their performances, which is well-aligned with the perceived quality of an image. Famous quantitative metrics such as Peak signal-to-noise ratio (PSNR), The structural similarity index measure (SSIM), and Perceptual Index (PI) are not well-correlated with the mean opinion score (MOS) for an image, especially for the neural networks trained with adversarial loss functions. This paper has proposed a convolutional neural network using an extension architecture of the traditional Siamese network so-called Siamese-Difference neural network. We have equipped this architecture with the spatial and channel-wise attention mechanism to increase our method's performance. Finally, we employed an auxiliary loss function to train our model. The suggested additional cost function surrogates ranking loss to increase Spearman's rank correlation coefficient while it is differentiable concerning the neural network parameters. Our method achieved superior performance in \textbf{\textit{NTIRE 2021 Perceptual Image Quality Assessment}} Challenge. The implementations of our proposed method are publicly available.

As video traffic volume increases, video streaming providers are struggling to improve the quality of experience (QoE). On the other hand, video viewers often prefer a lower traffic volume over a QoE that is too high since they contract for data-capped or pay-per-use communication plans. Thus, traffic volume should be reduced as much as possible while achieving the required QoE, which is the minimum QoE that will satisfy users. However, the required QoE depends on the context (e.g., the type of content and the preference of the user), and it is unrealistic and costly for users to set the required quality separately for each possible context. In this paper, we propose a context-aware adaptive bitrate streaming system that reduces the traffic volume while achieving the required QoE. Instead of requiring the required QoE to be configured by users, the proposed system uses the viewing time as implicit feedback on the QoE. By using this feedback, the proposed system automatically controls the QoE with a two-stage approach: it estimates the required QoE and calculates the bitrate to reduce the traffic volume while maintaining a QoE above the required QoE. To determine the required QoE based on few views, the proposed system searches for the required QoE in the mean opinion score space and uses Bayesian optimization. The results of trace-based simulations show that the proposed system can control the QoE close to the context-dependent required QoE based on fewer views than baseline algorithms.

This paper examines a practical approach for Network Functions Virtualization (NFV) management and orchestration (MANO) based on the Open Source MANO (OSM). The proposed testbed uses the OSM Open Source orchestration platform, in order to handle and orchestrate the NFV Voice over Internet Protocol (VoIP) in OpenStack used as Virtual Infrastructure Manager (VIM). On top of this, we implemented Software-Defined Networking (SDN) by means of the Open Network Operating System (ONOS) controller and Open vSwitch. Our experiments show that the solution is feasible, with the latency, jitter, packet-loss and Mean Opinion Score (MOS) all within the range.

Speech is a fundamental way of expressing ideas and thoughts. The ability to produce synthetic speech has always been a great interest to mankind. Text-to speech (TTS) or speech synthesis is an approach of producing an artificial speech in contrast to a given input text. Text-to-speech systems have immensely increased and improved in recent years. These advances in speech synthesis are contributed by deep learning techniques. The aim of this paper is to give an understanding regarding dynamics of research in this field and gives a brief introduction of traditional methods used in speech synthesis. Further, the use of various data sets for training deep learning TTS or Speech Synthesis Systems have also been discussed. Moreover, this paper emphasizes on deep learning based end-to-end speech synthesis models which have achieved spectacular achievement in terms of mean opinion score (MOS).

In literatures, many works on the area of massive MIMO (Multi Input Multi Output) presented the improvement of signal quality by using precoding in accordance with partial structure. However, only signal quality is not good enough to represent the satisfaction of users. In this paper, we proposed the precoding for partially-structured massive MIMO by utilizing user satisfaction in term of Quality of Experience (QoE). The level of Mean Opinion Score (MOS) is adopted to express QoE of web browsing service. In the first step, we optimize the baseband precoding and RF precoding by proposing the alternating minimization methods, then optimize the QoS parameters for the best QoE. The results show that the optimized QoS parameters lead to the satisfaction of user and the achieving of the required level of QoE.

Compared with traditional speech synthesis systems, end-to-end speech synthesis systems based on deep learning (such as DeepVoice3, Tacotron2) not only reduce the requirements for linguistic knowledge, but the synthesis effect is almost close to the level of human pronunciation. However, the end-to-end speech synthesis system based on deep learning has disadvantages such as missing words, repeated pronunciation, and slow synthesis speed. In view of the local information preference of the Tacotron2 model in the decoder, this paper proposes to maximize the interactive information between the text and the predicted acoustic features and use the WaveGlow synthesizer to reduce the local information preference and the problem of slow synthesis speed, pronunciation in the Tacotron2 model. Experimental results show that the improved model subjective evaluation MOS (Mean Opinion Score) score is 3.94, and the synthesis speed is significantly improved.

This paper provides a subjective analysis of streaming video to understand human visual Quality of Experience (QoE). The emergence of the Corona pandemic has prompted a rise in demand for internet services, particularly for video streaming. A limited number of companies in Europe choose to reduce video transmission quality to escape Internet strain. The QoE markers are defined as either satisfactory if the video loads in less than 10 seconds and plays out efficiently or undesirable if the loading time is above 10 seconds or the playout suffers from stands. Online media will display various kinds of high motion sports and music videos. Such metrics would enable internet providers to assess the degree to which the QoE of video streaming customers can decline under bad network conditions. Video quality is influenced by consumer behavior and content and network and device efficiency. A more significant number of content types would help us to study human viewer behavior better. Network providers typically do not see this kind of traffic. QoE is now believed to be an essential aspect of the reactive traffic control mechanism within network operators. The researcher looked for the best to create a database that contains as many subject forms as possible. The mean opinion score (MOS) has an exponential relationship with the video playout's number of stalls. Today's Global Adaptive Streaming over HTTP (DASH) definition to distribute the content-based streaming services employ differential bitrate techniques. QoE tracking often considers a variety of video QoE influence factors (IFs) and contableuration schemes. Multimedia information QoE can be assessed using subjective or objective methods.

The learning process requires specific methods to achieve effective and efficient learning goals. Learning methodology is a way of carrying out activities between educators and students when interacting in the learning process. One of the learning methodology methods is the demonstration method, which using objects or other teaching materials at the time of teaching. Students expected can understand the correlation between microprocessors and memory practically is one of the main goals in the microprocessor lectures. This research aims to help the learning process in the microprocessors lecture by designing and implementing the teaching aid of the “Microprocessor Interconnection with Memory” module. This aid demonstrates an input and processes it into output in the form of lit LED using the 80C88 Microprocessor. The test results of this aid obtain an average of 75.34% by using the Mean Opinion Score (MOS) method.

The voice quality evaluation of communication systems is necessary for technical and commercial reasons for the expansion of digital networks, mobile or VoIP and speech synthesis systems. Voice quality can be evaluated using two types of methods: subjective MOS (Mean Opinion Score) and objective PESQ (Perceptual Evaluation Speech Quality). In this work, we propose a study of voice quality evaluation methods; in which a subjective and objective evaluation platform of voice quality is provided. We applied different tests used in the assessment of the speech signal quality to speech synthesis signal for the Arabic Language (GArabic: Generic Arabic). The listeners recognized the majority of changes of consonants and vowels in words and sentences with respective percentages of 98.26%, 92.97%. The comparison between MOS and PESQ tests gave a good correlation coefficient of 0.922.

Single Image Super-Resolution (SISR) has proven itself as a highly challenging and ill-posed problem. Multiple methods have been applied to this problem in the past, with varying degrees of success. Recently, methods using deep learning such as Generative Adversarial Networks (GAN) and Variational Auto-Encoders (VAE) in particular have proven to be extremely effective. However, most of the present methods either create a blurry output, lacking fine details, or use extremely heavy models to achieve better results. We introduce a novel, lightweight GAN architecture for 4× super-resolution of images, which builds on previous methods, showing high quality of features both qualitatively and quantitatively. To achieve this, we use dilated convolutions in our generator architecture, a Markovian discriminator, a modified loss function and a training process more typical of a conditional GAN (cGAN). For testing our results qualitatively, we use Mean Opinion Score (MOS). The obtained MOS show the effectiveness of our model at generating visually superior images. Our code is available at https://www.github.com/kushalchordiya216/Super-Resolution.

The emergence of Audio-based Augmented Reality has been calling for increasing data-rates for audio signals, with significant reduction in power to enable extremely energy-constrained sensor nodes. Typically, the communication power dominates sensing and computing power in a node [1]. For highly energy constrained scenarios, compressive sensing (CS) have been demonstrated (Fig. 1), where samples are first compressed at the sensor to contain the same information in a smaller number of samples, before transmitting to a receiver, where the signal is reconstructed. Previous CS works [2]–[5] have focused entirely on “sparse” physiological signals, operating in low speed regime. This work illustrates the first CS design, enabled with a discrete wavelet transform (DWT) sparsifier for catering to non-sparse signals such as high definition audio. Audio recording and playback are quite sensitive to quality, thereby requiring audio codecs, such as. aac, for efficient compression and decompression of audio streams, which usually consume power in the order of mW [6]. Audio inferencing operated in intelligent assistants are more tolerant to input quality, functioning effectively when the Perceptual Evaluation of Audio Quality Mean Opinion Score (PAEQ MOS) [7], an ITU-R standard objective metric for characterizing perceived audio quality, exceeds 1.5. CS presents an opportunity to achieve >10X reduction in transmitted audio data with orders of magnitude lower power, as compared to codecs. The design is implemented in 65 nm CMOS and consumes 238 uW power at 0.65 V and 15 Mbps.

With the development of wireless communication systems, it is particularly essential to maximize the quality of experience (QoE) of machine-to-machine (M2M) communication. In this paper, we propose a new QoE-oriented uplink rate control and resource allocation scheme for the Internet of Things (IoT) network, by introducing an evaluation model based on mean opinion score (MOS) for different machine-type communication (MTC) devices. The existing works are only dedicated to solving the short-term resource allocation problems by considering the current transmission time slots, which cannot handle long-standing problems. To this end, based on the recently developed Lyapunov optimization, we convert the original long-term optimization problem into the admission rate control subproblem and the resource allocation subproblem in each time slot. To solve the joint power optimization and sub-channel selection subproblems, Gale-Shapley algorithm is utilized to formulate it as a two-dimensional matching problem, and the preference lists are established by the transmission rate and signal to interference plus noise ratio (SINR). In the proposed algorithms, a priority mechanism is employed to ensure fairness. The simulation results demonstrate that without prior knowledge of the data arrivals and sub-channel statistics, the proposed algorithms can significantly improve the overall perceived quality from the users' perspective.

Text-to-speech (TTS) synthesis is a rapidly growing field of research. Deep learning has shown impressive results in speech synthesis and outperformed the older concatenative and parametric methods. In this paper, speech synthesis using deep learning architectures is explored and two models are utilized in an end-to-end Arabic TTS system. The results of the two systems are compared to concatenative TTS system using the Mean Opinion Score (MOS) of the synthesized speech and indicates that deep learning based systems have outperformed the concatenative system when it comes to naturalness and intelligibility; moreover, it reduces system complexity.

This paper aims to present a new model called SRMIGAN that performs super-resolution for MRI and CT medical images to help doctors reach a better diagnosis. This model, SRMIGAN adopts deep learning by applying generative adversarial networks technique. It is developed using MSE loss and by exploiting different optimization techniques. This model is compared to other adopted models by using both objective and subjective metrics. Hence PSNR, SSIM, and mean opinion score results are included. The results show that our model beats the other examined models.

This paper presents application of a speech quality measurement method to evaluate the naturalness of synthesized speech from Text-To-Speech synthesis (TTS) which is an important issue for synthesized speech quality. In this study, the Thai language synthesized speech from Apple's Siri and Google Translate have been evaluated by 48 Thai users. Surprisingly, it has been found that the Mean Opinion Score (MOS) associated with the naturalness of Thai synthesized speech from Apple's Siri and Google Translate are 3.19 and 3.54 respectively. After the analysis, it is confirmed that Google Translate provides better naturalness of Thai synthesized speech than Siri.

In this paper, we present a Pali (Thai) speech synthesis system using the parametric statistical approach. To develop the system, we recorded 40 Pali chants. Data were extracted and represented by the Mel frequency cepstral coefficients and fundamental frequency (F0), and labeled by force alignment. These parameters were modeled using the hidden Markov model (HMM). To generate synthesized speech, the input text was converted into context-dependent phonemes and generated speech parameters from the trained HMM model. The resulting parameters were used for synthesizing speech using a speech vocoder. In the study, we modeled two speech synthesized models: the first model represents tone in syllable levels (tone-syllable) and the second model represents tone in phoneme levels (tone-phoneme). To evaluate the naturalness of the proposed system, we asked 13 users to participate in listening tests comparing the two synthesized speech models (tone-syllable and tone-phoneme models) and original speech. The results, expressing naturalness in mean opinion score (MOS), were 4.21, 3.25, and 3.32 (from 5) for the original, tone-syllable, and tone-phoneme synthesized speeches, respectively. We also conducted an objective test in which we calculated the cepstral distance between the cepstral coefficients of the original speeches and synthesized speeches. The average distances were 3.67 and 3.60 for the tone-syllable and the tone-phoneme models, respectively.

Experiencing images with suitable music can greatly enrich the overall user experience. The proposed image analysis method treats an artwork image differently from a photograph image. Automatic image classification is performed using deep-learning based models. An illustrative analysis showcasing the ability of our deep-models to inherently learn and utilize perceptually relevant features when classifying artworks is also presented. The Mean Opinion Score (MOS) obtained from subjective assessments of the respective image and recommended music pairs supports the effectiveness of our approach., IEEE International Conference on Consumer Electronics (IEEE ICCE 2021)

Video Summarization is a technique that reduces the viewing duration of a video by extracting important parts from a large video. As most of the cricket matches are long-lasting, the audience preferably needs to have the summarized format of the large content. In this paper, we intend to speed up large-scale video processing by providing a concise synopsis that still conveys the story of a cricket match. For addressing this challenge, we develop a Deep Cricket Summarization Network (DCSN) that automatically extracts key-shots from an input video. Due to the limitations of the few available datasets in this domain, we introduce a new dataset named CricSum. The quality of the generated summary highly relies on users’ subjective understandings. So we demonstrate the efficiency of the proposed summarization system using the Mean Opinion Score (MOS) technique. The automatically generated summarized videos collectively obtained MOS value 4 out of 5.

With the development of converged optical-wireless networks, the mobile users can access a very high bandwidth, any time and anywhere, for the upcoming real-time applications. In the modern digital world, quality-of-experience (QoE) has become the central area of concern to ensure end-to-end user satisfaction. In this paper, we analyze the quality-of-service (QoS) for different traffic classes for a radio-over-fiber (RoF) based attocell network. For this, we use a simple polling algorithm with dynamic weighted queuing for grant sizing and maintains a delay within the maximum bounds for high and medium priority traffic classes with high channel utilization. Further, we propose a model to map QoE with the QoS parameters, like delay, jitter, and packet loss rate, and quantify QoE by representing the degree of user satisfaction by a mean opinion score.

Super resolution techniques are used to reconstruct the detail of high-resolution image from low-resolution lossy image. The introduction of a deep learning approach in super resolution has drawn a lot of research interest in recent years due to its learning capability and noise immunity. The applications of deep learning super resolution can mainly be found in image recovery, medical imaging, and microscopy. In this paper, the deep learning super resolutions are explored in detail based on its models and architecture. They can be classified into three neural network (NN) models, i.e. Convolutional NN-based models, Recursive NN-based models, and Adversarial Network-based models. CNN-based models apply convolution operation to embed the latent feature and subsequently decode it with deconvolution operation to achieve a higher dimension. RNN-based models integrate the recursive depth model to enhance recursive learning with past memory. On the other hand, adversarial network-based models apply the generative manner to learn the probability of the input pattern to forecast the possible high dimension of output information. The details of each unsupervised model are discussed in this paper to highlight its advantages and limitations. The measurement metrics such as Peak Signal-to-Noise Ratio (PSNR), structural similarity (SSIM) and Mean opinion score (MOS) are highlighted for performance evaluation of each super resolution models. The significance of this study provide a compact review of the current development and trend in super resolution using various deep learning models.

Recent researches on multispeaker text-to-speech synthesis allow cloning a voice unseen during training without retraining the model with new speech samples. Multispeaker text-to-speech synthesis consists of three blocks, namely speaker encoder, synthesizer and a vocoder. An embedding from the short utterance of a single speaker is derived using speaker encoder in the front-end. Synthesizer generates a spectrogram from text through the speaker embedding deduced from the encoder, and the vocoder generates an audio file from the mel-spectrogram output of synthesizer. The choice of the vocoder is an essential factor in determining the quality of the generated audio. In this paper, we study the performance of two vocoders, namely, WaveNet and WavGlow using subjective and objective analysis. A perception test is conducted to analyse the mean opinion score (MOS). Structural similarity index (SSI) between mel-spectrograms of synthesizer output and generated audio samples is also computed. The study shows that WaveNet encoder outperforms WavGlow in voice quality analysis in multispeaker text-to-speech synthesis

In the recent years, HTTP Adaptive Bit Rate (ABR) streaming including Dynamic Adaptive Streaming over HTTP (DASH) has become the most popular technology for video streaming over the Internet. The client device requests segments of content using HTTP, with an ABR algorithm selecting the quality at which to request each segment to trade-off video quality with the avoidance of stalling. This introduces high latency compared to traditional broadcast methods, mostly in the client buffer which needs to hold enough data to absorb any changes in network conditions. Clients employ an ABR algorithm which monitors network conditions and adjusts the quality at which segments are requested to maximise the user's Quality of Experience. The size of the client buffer depends on the ABR algorithm's capability to respond to changes in network conditions in a timely manner, hence, low latency live streaming requires an ABR algorithm that can perform well with a small client buffer. In this paper, we present Llama - a new ABR algorithm specifically designed to operate in such scenarios. Our new ABR algorithm employs the novel idea of using two independent throughput measurements made over different timescales. We have evaluated Llama by comparing it against four popular ABR algorithms in terms of multiple QoE metrics, across multiple client settings, and in various network scenarios based on CDN logs of a commercial live TV service. Llama outperforms other ABR algorithms, improving the P.1203 Mean Opinion Score (MOS) as well as reducing rebuffering by 33% when using DASH, and 68% with CMAF in the lowest latency scenario.

Speech synthesis systems are now getting smarter and more natural thanks to the power of deep neural networks. However, each language has a different phonological and contextual characteristics, we have conducted experiments, statistics, and applied Vietnamese phonetics to improve speech synthesis systems based on Tacotron2 neural networks. Our methods achieve the accuracy of 97% in text normalization task, and the synthesized speeches with a MOS score of 3.97, asymptotic to 4.43 of the voices that are directly recorded. We also provide a library for standardizing Vietnamese text called Vinorm and a package that converts text into a phonetic format called Viphoneme, which is used as an input for end-to-end neural networks, make the synthesis process faster, more intelligent and natural than using character inputs.

Text to Speech (TTS) is a form of speech synthesis where the text is converted into a spoken human-like voice output. The state of the art methods for TTS employs a neural network based approach. This work aims to look at some of the issues and limitations present in the current works, specifically Tacotron-2, and attempts to further improve its performance by modifying its architecture. The modified model uses Transformer network as a Spectrogram Prediction Network (SPN) and WaveGlow as an Audio Generation Network (AGN). For the modified model, performance improvements are seen in terms of the speech output generated for corresponding texts, the inference time taken for audio generation, and a Mean Opinion Score (MOS) of 4.10 (out of 5) is obtained.

End-to-end text-to-speech (TTS) systems has proved its great success in the presence of a large amount of high-quality training data recorded in anechoic room with high-quality microphone. Another approach is to use available source of found data like radio broadcast news. We aim to optimize the naturalness of TTS system on the found data using a novel data processing method. The data processing method includes 1) utterance selection and 2) prosodic punctuation insertion to prepare training data which can optimize the naturalness of TTS systems. We showed that using the processing data method, an end-to-end TTS achieved a mean opinion score (MOS) of 4.1 compared to 4.3 of natural speech. We showed that the punctuation insertion contributed the most to the result. To facilitate the research and development of TTS systems, we distributed the processed data of one speaker at https://forms.gle/6Hk5YkqgDxAaC2BU6., Comment: 8 pages, 2 figures, submit to Oriental Cocosda

The low light performance of mobile phone cameras has gained much improvement due to the rapid development of digital imaging technologies. Recently, many phone manufactures have advertised their remarkable night mode for taking photos in low light environment. At the same time, more and more mobile video applications are enhancing their low light performance. This work investigates image quality measurement for mobile video calling applications in low light environment. At first, a video database is constructed by capturing many videos under different lighting conditions. Then a no-reference quality measurement model, which contains spatial information, noise and motion blur metrics, is proposed and is fitted to human observers' mean opinion score (MOS). The experiment results show that this measurement method gives high correlation with subjective MOS. Using the proposed model, we can analyze how camera sensor parameters influence to image quality, which helps the development of more accurate exposure algorithms for video calling applications in low light environment.

Visual comfortable is certainly one of the most critical issue in the research field of stereoscopic video. This is because that some viewers feel visual fatigue and discomfort when they see a wonderful stereoscopic video for a moment. Therefore, the problem of how to use related technologies to solve the series of problems exist in 3D comfortable prediction becomes urgent that we should settle. In this paper, we present a new three-dimensional (3D) video database (3DVID) for developing a new visual comfortable prediction technique of stereoscopic video. Specifically, this database contains 68 stereoscopic video pairs, where all of them are 1920×1080 resolution. The single stimulus continuous quality evaluation (SSCQE) method is adopted in our subjective tests, and the Mean Opinion Score (MOS) can be computed accordingly. In addition, we also show that the relationship between disparity, texture, motion and subjective MOS for each stereopair. Experimental results show that the proposed 3D video database contains a wide variety of scene variations and comfort levels, which is a very good benchmarking tool for 3D visual comfortable prediction algorithm design.

Myanmar is one of the developing countries situated in South-East Asia, and there are still many areas that have been under-developed with respect to advanced natural language processing technologies, where text-to-speech is one of them. The main motivation of this paper is to improve the naturalness of Myanmar text-to-speech system that is able to generate human-like speech. In this paper, we apply the neural network architecture based on Tacotron-2 that generates a mel spectrogram for speech synthesis directly from the sequence of text. Our proposed method is composed of three steps. In the first step, we create a speech corpus of 5k sentences of text and audio pair of Myanmar text from a large set of news articles, novel books, daily usages and travel-related expressions. We segment the Myanmar text into a sequence of characters by using a syllable segmenter and text normalizer. In the second step, we utilize the recurrent sequence-to-sequence feature prediction network that maps character embedding to mel-scale spectrograms. In the final step, we use Griffin-Lim algorithm to convert the corresponding text into generate Myanmar speech output. We compare our proposed method with an end-to-end generative model based on Tacotron. Furthermore, we investigate the subjective evaluation for both methods in speech synthesis by using mean opinion score (MOS). The experimental results show that our proposed method obtains an improvement over Tacotron based speech synthesis in terms of naturalness and intelligibility.

The main motivation of this paper is to improve the naturalness of Myanmar text-to-speech system using an end-to-end generative model called Tacotron. We introduce the open-source implementation for Myanmar text-to-speech system with very high natural-sounding. In this paper, there are four main parts: speech corpus creation, data pre-processing, applying end-to-end generative model, and speech synthesis. Firstly, we develop a speech corpus of 8k sentences from a large set of news articles, novel books, daily usages and travel-related expressions for corpus creation. Secondly, we use a syllable segmenter and text normalizer for data pre-processing. Thirdly, we apply end-to-end generative model called Tacotron that synthesizes speech directly from the sequence of text characters. Finally, we use Griffin-Lim algorithm to convert the corresponding text into the output speech. For the subjective evaluation, we compare our synthesized speech output with the original recording speech in both intelligibility and naturalness by using mean opinion score (MOS). The experimental results show that we can obtain the synthesized speech comparable to the similar state-of-the-art synthsizers for other languages.

This research develops end-to-end deep learning-based text-to-speech (TTS) in Indonesian, Javanese, and Sundanese. While end-to-end neural TTS, such as Tacotron-2, has made remarkable progress recently, it still suffers from a data scarcity problem for low-resource languages such as Javanese and Sundanese. Our preliminary study shows that Tacotron-2-based TTS needs a large amount of training data; a minimum of 10 hours of training data is required for the model to be able to synthesize acceptable quality and intelligible speech. To solve this low-resource problem, our work proposes a hierarchical transfer learning to train TTS for Javanese and Sundanese, by taking advantage of a dissimilar high-resource language of English domain and a similar intermediate-resource language of Indonesian domain. We report that the evaluation of synthesized speech using the mean opinion score (MOS) reaches 4.27 for Indonesian, and 4.08 for Javanese, and 3.92 for Sundanese. The word accuracy (WAcc) evaluation on semantically unpredicted sentences (SUS) reaches 98.26% for Indonesian, 95.02% for Javanese, and 95.43% for Sundanese. The subjective evaluations of the synthetic speech quality demonstrate that our transfer learning scheme is successfully applied to TTS model for low-resource target domain. Using less than one hour of training data, 38 minutes for Indonesian, 16 minutes for Javanese, and 19 minutes for Sundanese, TTS models can learn fast and achieve adequate performance.

Sign Language Synthesis (SLS) is a domain-specific problem where multiple sign language words are stitched to generate a whole sentence in video, which serves to facilitate communications between the hearing-impaired people and healthy population. This paper presents a Variable Motion Frame Interpolation (VMFI) method for highly fluent SLS in scattered videos. Existing approaches for SLS mainly focus on mechanical virtual human technology, lacking high flexibility and natural effect. Also, the representative solutions to interpolate frames usually assume that the motion object moves at a constant speed which is not suitable for predicting the complex hand motion in frames of scattered sign language videos. To address the above issues, the proposed VMFI adopts acceleration to predict more accurate interpolated frames based on an end-to-end convolutional neural network. The framework of VMFI consists of variable optical flow estimation network and high-quality frame synthesis network that can approximate and fuse the intermediate optical flow to generate interpolated frames for synthesis. Experimental results on our realistic collected Chinese sign language dataset demonstrate that the proposed VMFI model achieves efficiency by performing better in PSNR (Peak Signal to Noise Ratio), SSIM (Structural Similarity) and MA (Motion Activity) and gets higher score in MOS (Mean Opinion Score) than other two representative methods.