Search

This study compares response time and accuracy in a binary data labeling task on two platforms: a desktop computer and a mobile phone. Three methods were used on the computer: a keyboard, a mouse, and a mouse with perceptual variability that was designed to combat vigilance decrement. Three additional methods were used on the mobile phone: a tap, a swipe touch-gesture, and a tap with perceptual variability. Results of the study show that the fastest was the keyboard, which may explain its popularity in labeling tasks. The second fastest was the tap interface, suggesting the unexploited potential of mobile devices for "labeling while waiting for the bus." In terms of accuracy, we found clear evidence for a speed–accuracy tradeoff, and the advantage of the perceptual variability methods to be more accurate than the matching methods without perceptual variability. The results suggest that different user input methods should be employed depending on whether response time or accuracy is more valued. [ABSTRACT FROM AUTHOR]

Spatial predictions, like other supervised learning tasks, require some criterion for a predictor’s quality. Typical data-splitting schemes, such as holdouts and k-fold cross-validation, ignore the fact that the training data are usually not available where predictions are being made. The common data-splitting schemes are thus biased estimates of a predictor’s performance, which in turn may lead to choosing suboptimal predictors. In this contribution, we borrow ideas from the domain adaptation machine-learning literature, to suggest the importance-weighted source risk (IWSR). IWSR is a principled approach for weighting the prediction risk, which allows the practitioner to explicitly state the target locations for prediction. IWSR essentially consists of down-weighting training locations and up-weighting target locations. We show that, unlike the usual (unweighted) empirical risk, IWSR is an unbiased estimator of the prediction error. Equipped with this risk estimator, we use it to learn a model in the empirical risk minimization framework and to evaluate the existing predictors. We show the superiority of this weighted risk, using both simulated data and an empirical control: air-temperature prediction in France. [ABSTRACT FROM AUTHOR]

The estimated accuracy of a classifier is a random quantity with variability. A common practice in supervised machine learning, is thus to test if the estimated accuracy is significantly better than chance level. This method of signal detection is particularly popular in neuroimaging and genetics. We provide evidence that using a classifier's accuracy as a test statistic can be an underpowered strategy for finding differences between populations, compared to a bona fide statistical test. It is also computationally more demanding than a statistical test. Via simulation, we compare test statistics that are based on classification accuracy, to others based on multivariate test statistics. We find that the probability of detecting differences between two distributions is lower for accuracy-based statistics. We examine several candidate causes for the low power of accuracy-tests. These causes include: the discrete nature of the accuracy-test statistic, the type of signal accuracy-tests are designed to detect, their inefficient use of the data, and their suboptimal regularization. When the purpose of the analysis is the evaluation of a particular classifier, not signal detection, we suggest several improvements to increase power. In particular, to replace V-fold cross-validation with the Leave-One-Out Bootstrap. [ABSTRACT FROM AUTHOR]

A common approach to statistical learning with Big-data is to randomly split it among m machines and learn the parameter of interest by averaging the m individual estimates. In this paper, focusing on empirical risk minimization or equivalently M-estimation, we study the statistical error incurred by this strategy. We consider two large-sample settings: first, a classical setting where the number of parameters p is fixed, and the number of samples per machine n → ∞. Second, a high-dimensional regime where both p, n → ∞ with p/n → (0, 1). For both regimes and under suitable assumptions, we present asymptotically exact expressions for this estimation error. In the fixed-p setting, we prove that to leading order averaging is as accurate as the centralized solution. We also derive the second-order error terms, and show that these can be non-negligible, notably for nonlinear models. The high-dimensional setting, in contrast, exhibits a qualitatively different behavior: data splitting incurs a first-order accuracy loss, which increases linearly with the number of machines. The dependence of our error approximations on the number of machines traces an interesting accuracy-complexity tradeoff, allowing the practitioner an informed choice on the number of machines to deploy. Finally, we confirm our theoretical analysis with several simulations. [ABSTRACT FROM AUTHOR]

While driving, we make numerous conscious decisions such as route and turn direction selection. Although drivers are held responsible, the neural processes that govern such decisions are not clear. We recorded intracranial EEG signals from six patients engaged in a computer-based driving simulator. Patients decided which way to turn (left/right) and subsequently reported the time of the decision. We show that power modulations of gamma band oscillations (30-100 Hz) preceding the reported time of decision (up to 5.5 sec) allow prediction of decision content with high accuracy (up to 82.4%) on a trial-by-trial basis, irrespective of subsequent motor output. Moreover, these modulations exhibited a spatiotemporal gradient, differentiating left/right decisions earliest in premotor cortices and later in more anterior and lateral regions. Our results suggest a preconscious role for the premotor cortices in early stages of decision-making, which permits foreseeing and perhaps modifying the content of real-life human choices before they are consciously made. [ABSTRACT FROM AUTHOR]

Human perception thresholds can improve through learning. Here we report findings challenging the fundamental 'practice makes perfect' basis of procedural learning theory, showing that brief reactivations of encoded visual memories are sufficient to improve perceptual discrimination thresholds. Learning was comparable to standard practice-induced learning and was not due to short training per se, nor to an epiphenomenon of primed retrieval enhancement. The results demonstrate that basic perceptual functions can be substantially improved by memory reactivation, supporting a new account of perceptual learning dynamics.