Search

Identifying dependency between two random variables is a fundamental problem. Clear interpretability and ability of a procedure to provide information on the form of the possible dependence is particularly important in evaluating dependencies. We introduce a new estimator of the quantile dependence function and pertinent local acceptance regions. This leads to insightful visualization and evaluation of underlying dependence structure. We also propose a test of independence of two random variables, pertinent to this new estimator. Our procedures are based on ranks and we derive a finite-sample theory that guarantees the inferential validity of our solutions at any given sample size. The procedures are simple to implement and computationally efficient. Large-sample consistency of the proposed test is also proved. We show that, in terms of power, new test is one of the best statistics for independence testing when considering a wide range of alternative models. Finally, we demonstrate use of our approach to visualize dependence structure and to detect local departures from independence through analyzing some datasets.

In this paper we reproduce experimental results presented in our earlier work titled "Describing Subjective Experiment Consistency by $p$-Value P-P Plot" that was presented in the course of the 28th ACM International Conference on Multimedia. The paper aims at verifying the soundness of our prior results and helping others understand our software framework. We present artifacts that help reproduce tables, figures and all the data derived from raw subjective responses that were included in our earlier work. Using the artifacts we show that our results are reproducible. We invite everyone to use our software framework for subjective responses analyses going beyond reproducibility efforts., Comment: Please refer to the original publication: https://dl.acm.org/doi/10.1145/3474085.3477935 Related paper: https://dl.acm.org/doi/abs/10.1145/3394171.3413749 or arXiv:2009.13372

A class of discrete probability distributions contains distributions with limited support. A typical example is some variant of a Likert scale, with response mapped to either the $\{1, 2, \ldots, 5\}$ or $\{-3, -2, \ldots, 2, 3\}$ set. An interesting subclass of discrete distributions with finite support are distributions limited to two parameters and having no more than one change in probability monotonicity. The main contribution of this paper is to propose a family of distributions fitting the above description, which we call the Generalised Score Distribution (GSD) class. The proposed GSD class covers the whole set of possible mean and variances, for any fixed and finite support. Furthermore, the GSD class can be treated as an underdispersed continuation of a reparametrized beta-binomial distribution. The GSD class parameters are intuitive and can be easily estimated by the method of moments. We also offer a Maximum Likelihood Estimation (MLE) algorithm for the GSD class and evidence that the class properly describes response distributions coming from 24 Multimedia Quality Assessment experiments. At last, we show that the GSD class can be represented as a sum of dichotomous zero-one random variables, which points to an interesting interpretation of the class., Comment: All authors contributed equally. 21 pages, 15 figures

Subjective responses from Multimedia Quality Assessment (MQA) experiments are conventionally analysed with methods not suitable for the data type these responses represent. Furthermore, obtaining subjective responses is resource intensive. A method allowing reuse of existing responses would be thus beneficial. Applying improper data analysis methods leads to difficult to interpret results. This encourages drawing erroneous conclusions. Building upon existing subjective responses is resource friendly and helps develop machine learning (ML) based visual quality predictors. We show that using a discrete model for analysis of responses from MQA subjective experiments is feasible. We indicate that our proposed Generalised Score Distribution (GSD) properly describes response distributions observed in typical MQA experiments. We highlight interpretability of GSD parameters and indicate that the GSD outperforms the approach based on sample empirical distribution when it comes to bootstrapping. We evidence that the GSD outcompetes the state-of-the-art model both in terms of goodness-of-fit and bootstrapping capabilities. To do all of that we analyse more than one million subjective responses from more than 30 subjective experiments. Furthermore, we make the code implementing the GSD model and related analyses available through our GitHub repository: https://github.com/Qub3k/subjective-exp-consistency-check, Comment: 15 pages, 6 figures. Under review in IEEE Transactions on Multimedia

There are phenomena that cannot be measured without subjective testing. However, subjective testing is a complex issue with many influencing factors. These interplay to yield either precise or incorrect results. Researchers require a tool to classify results of subjective experiment as either consistent or inconsistent. This is necessary in order to decide whether to treat the gathered scores as quality ground truth data. Knowing if subjective scores can be trusted is key to drawing valid conclusions and building functional tools based on those scores (e.g., algorithms assessing the perceived quality of multimedia materials). We provide a tool to classify subjective experiment (and all its results) as either consistent or inconsistent. Additionally, the tool identifies stimuli having irregular score distribution. The approach is based on treating subjective scores as a random variable coming from the discrete Generalized Score Distribution (GSD). The GSD, in combination with a bootstrapped G-test of goodness-of-fit, allows to construct $p$-value P-P plot that visualizes experiment's consistency. The tool safeguards researchers from using inconsistent subjective data. In this way, it makes sure that conclusions they draw and tools they build are more precise and trustworthy. The proposed approach works in line with expectations drawn solely on experiment design descriptions of 21 real-life multimedia quality subjective experiments., Comment: 11 pages, 3 figures. Accepted to 28th ACM International Conference on Multimedia (MM '20). For associated data sets, source codes and documentation, see https://github.com/Qub3k/subjective-exp-consistency-check

A class of discrete probability distributions contains distributions with limited support, i.e. possible argument values are limited to a set of numbers (typically consecutive). Examples of such data are results from subjective experiments utilizing the Absolute Category Rating (ACR) technique, where possible answers (argument values) are $\{1, 2, \cdots, 5\}$ or typical Likert scale $\{-3, -2, \cdots, 3\}$. An interesting subclass of those distributions are distributions limited to two parameters: describing the mean value and the spread of the answers, and having no more than one change in the probability monotonicity. In this paper we propose a general distribution passing those limitations called Generalized Score Distribution (GSD). The proposed GSD covers all spreads of the answers, from very small, given by the Bernoulli distribution, to the maximum given by a Beta Binomial distribution. We also show that GSD correctly describes subjective experiments scores from video quality evaluations with probability of 99.7\%. A Google Collaboratory website with implementation of the GSD estimation, simulation, and visualization is provided., Comment: 13 pages, 14 Figures Submitted to Journal of Survey Statistics and Methodology

This paper compares the Anderson-Darling and some Eicker-Jaeschke statistics to the classical unweighted Kolmogorov-Smirnov statistic. The goal is to provide a quantitative comparison of such tests and to study real possibilities of using them to detect departures from the hypothesized distribution that occur in the tails. This contribution covers the case when under the alternative a moderately large portion of probability mass is allocated towards the tails. It is demonstrated that the approach allows for tractable, analytic comparison between the given test and the benchmark, and for reliable quantitative evaluation of weighted statistics. Finite sample results illustrate the proposed approach and confirm the theoretical findings. In the course of the investigation we also prove that a slight and natural modification of the solution proposed by Borovkov and Sycheva (1968) leads to a statistic which is a member of Eicker-Jaeschke class and can be considered an attractive competitor of the very popular supremum-type Anderson-Darling statistic.

With a simple generic approach, we develop a classification that encodes and measures the strength of completeness (or compactness) properties in various types of spaces and ordered structures. The approach also allows us to encode notions of functions being contractive in these spaces and structures. As a sample of possible applications we discuss metric spaces, ultrametric spaces, ordered groups and fields, topological spaces, partially ordered sets, and lattices. We describe several notions of completeness in these spaces and structures and determine their respective strengths. In order to illustrate some consequences of the levels of strength, we give examples of generic fixed point theorems which then can be specialized to theorems in various applications which work with contracting functions and some completeness property of the underlying space. Ball spaces are nonempty sets of nonempty subsets of a given set. They are called spherically complete if every chain of balls has a nonempty intersection. This is all that is needed for the encoding of completeness notions. We discuss operations on the sets of balls to determine when they lead to larger sets of balls; if so, then the properties of the so obtained new ball spaces are determined. The operations can lead to increased level of strength, or to ball spaces of newly constructed structures, such as products. Further, the general framework makes it possible to transfer concepts and approaches from one application to the other; as examples we discuss theorems analogous to the Knaster--Tarski Fixed Point Theorem for lattices and theorems analogous to the Tychonoff Theorem for topological spaces.

Recognizing, quantifying and visualizing associations between two variables is increasingly important. This paper investigates how a new function-valued measure of dependence, the quantile dependence function, can be used to construct tests for independence and to provide an easily interpretable diagnostic plot of existing departures from the null model. The dependence function is designed to detect general dependence structure between variables in quantiles of the joint distribution. It gives an insight into how the dependence structures changes in different parts of the joint distribution. We define new estimators of the dependence function, discuss some of their properties, and apply them to construct new tests of independence. Numerical evidence is given on the test's benefits against three recognized independence tests introduced in the previous years. In real-data analysis, we illustrate the use of our tests and the graphical presentation of the underlying dependence structure., Comment: 40 pages, 3 figures, 1 table

Based on the theory of ball spaces introduced by Kuhlmann and Kuhlmann we introduce and study Caristi-Kirk and Oettli-Th\'era ball spaces. We show that if the underlying metric space is complete, then these have a very strong property: every ball contains a singleton ball. This fact provides quick proofs for several results which are equivalent to the Caristi-Kirk Fixed Point Theorem, namely Ekeland's Variational Principles, the Oettli-Th\'era Theorem, Takahashi's Theorem and the Flower Petal Theorem., Comment: 15 pages, submitted to Journal of Fixed Point Theory and Applications

The problem of testing hypothesis that a density function has no more than $\mu$ derivatives versus it has more than $\mu$ derivatives is considered. For a solution, the $L^2$ norms of wavelet orthogonal projections on some orthogonal "differences" of spaces from a multiresolution analysis is used. For the construction of the smoothness test an asymptotic distribution of a smoothness estimator is used. To analyze that asymptotic distribution, a new technique of enrichment procedure is proposed. The finite sample behaviour of the smoothness test is demonstrated in a numerical experiment in case of determination if a density function is continues or discontinues.

We construct an adaptive estimator of a density function on $d$ dimensional unit sphere $S^d$ ($d \geq 2 $), using a new type of spherical frames. The frames, or as we call them, stereografic wavelets are obtained by transforming a wavelet system, namely Daubechies, using some stereographic operators. We prove that our estimator achieves an optimal rate of convergence on some Besov type class of functions by adapting to unknown smoothness. Our new construction of stereografic wavelet system gives us a multiresolution approximation of $L^2(S^d)$ which can be used in many approximation and estimation problems. In this paper we also demonstrate how to implement the density estimator in $S^2$ and we present a finite sample behavior of that estimator in a numerical experiment.

The basic motivation and primary goal of this paper is a qualitative evaluation of the performance of a new weighted statistic for a nonparametric test for stochastic dominance based on two samples, which was introduced in Ledwina and Wy{\l}upek (2012). For this purpose, we elaborate a useful variant of Kallenberg's notion of intermediate efficiency. This variant is general enough to be applicable to other nonparametric problems. We provide a formal defnition of the proposed variant of intermediate efficiency, describe the technical tools used in its calculation, and provide proofs of related asymptotic results. Next, we apply this approach to calculating the intermediate efficiency of the new test with respect to the classical one-sided Kolmogorov-Smirnov test, which is a recognized standard for this problem. It turns out that for a very large class of convergent alternatives the new test is more efficient than the classical one. We also report the results of an extensive simulation study on the powers of the tests considered, which shows that the new variant of intermediate efficiency reflects the exact behavior of the power well., Comment: 41 pages, 4 figures