Search

Most of the existing studies on mapping of Quality of Service (QoS) to Quality of Experience (QoE) ignores any correlation between the QoS metrics. Network emulators such as NetEm that are used to study various network situations, have similar limitations. NetEm's inability to emulate realistic long-tail packet latency and correlation between QoS metrics may have already resulted in doubtful QoS to QoE functions. In this paper, we analysed real-time cellular QoS metrics data, and observed that there is significant correlation between several QoS parameters. We used them to create bespoke distributions for NetEm using different statistical methods such as maximum likelihood estimation and cumulative distribution functions. These distributions were used to modify NetEm to address the lack of representative packet waiting-time distributions. In addition, the bespoke distributions were also used to provide a way to emulate correlated packet delay and loss in NetEm. This bespoke emulator was then used to observe the impact of correlated delay-loss metric on end user QoE of video streaming applications using PSNR, SSIM and VMAF objective evaluation techniques. The emulation results showed that bespoke emulator gives better QoE compared to default NetEm configuration for same Packet Loss Ratio (PLR). In this paper, we analysed real-time cellular QoS metrics data, and observed that there is significant correlation between several QoS parameters. We used them to create bespoke distributions for NetEm using different statistical methods such as maximum likelihood estimation and cumulative distribution functions. These distributions were used to modify NetEm to address the lack of representative packet waiting-time distributions. In addition, the bespoke distributions were also used to provide a way to emulate correlated packet delay and loss in NetEm. This bespoke emulator was then used to observe the impact of correlated delay-loss metric on end user QoE of video streaming applications using PSNR, SSIM and VMAF objective evaluation techniques. The emulation results showed that bespoke emulator gives better QoE compared to default NetEm configuration for same Packet Loss Ratio (PLR).

The increasing popularity of the Video on Demand (VoD) applications like Netflix and YouTube has caused a large increase in video streaming over the network. To increase subscriptions and avoid customer churn, service providers are keen to evaluate the end-user experience more effectively. In this paper, we present an approach that can be used to evaluate the distribution of the Quality of Experience (QoE) metric evaluated from the measured Quality of Service (QoS) distributions. By using the network QoS measurements of delay and jitter, we find the probability density functions (PDF) for these parameters. Focussing on VoD applications employing improved VP9 codec (as used by Netflix) we use subjective testing and regression to map QoE onto jitter. The resulting QoE model and theory of direct propagation of probability density function was used to evaluate the distribution of QoE. We found that the empirical delay and jitter measurements follow a lognormal distribution whose characteristic parameters, sigma ( σ) and mu ( μ), vary for different network providers in the UK. Our results indicate that, even for slight variations of the underlying jitter distribution, the QoE distribution shows significant variations. These variations were recorded and presented as confidence intervals for four leading UK network providers.

The rise in popularity of the streaming platforms for gaming videos like Twitch.tv results in more users watching and streaming gaming videos passively. These streaming videos show great dependence on Quality of Service (QoS) parameters like latency and packet loss. This motivates us to evaluate the Quality of Experience (QoE) of three popular streaming game videos for different Packet Loss Ratios (PLR). In this paper, using streaming gaming videos, we demonstrated the limitations of using Mean Opinion Score (MOS) to evaluate QoE. Using a novel method of data representation, we presented our results as distributions of user-rating at different PLR configuration and compared them with other subjective QoE evaluation methods like MOS and standard deviation of Opinion score (SOS). Our results show that evaluating QoE using MOS loses information about user diversity. Our findings also include SOS retaining user diversity but posing challenges to quantifying the QoE at varying PLR values. For streaming video games we show that QoE distributions evaluate user perception accurately. Our work in this paper also studies how different games with different content complexity, genre and pace are affected by PLR. Our results show that slow-paced games like FIFA 20 get affected more significantly by packet loss.