1. How to embrace variation and accept uncertainty in linguistic and psycholinguistic data analysis
- Author
-
Shravan Vasishth and andrew gelman
- Subjects
Linguistics and Language ,Computer science ,PsyArXiv|Social and Behavioral Sciences|Linguistics|Semantics and Pragmatics ,PsyArXiv|Social and Behavioral Sciences|Cognitive Psychology|Memory ,PsyArXiv|Social and Behavioral Sciences|Linguistics|Syntax ,Language and Linguistics ,Linguistics ,bepress|Social and Behavioral Sciences|Psychology|Cognitive Psychology ,PsyArXiv|Social and Behavioral Sciences|Cognitive Psychology|Language ,PsyArXiv|Social and Behavioral Sciences ,Variation (linguistics) ,bepress|Social and Behavioral Sciences|Linguistics|Syntax ,PsyArXiv|Social and Behavioral Sciences|Linguistics|Psycholinguistics and Neurolinguistics ,bepress|Social and Behavioral Sciences|Linguistics|Semantics and Pragmatics ,bepress|Social and Behavioral Sciences ,PsyArXiv|Social and Behavioral Sciences|Cognitive Psychology ,Statistical inference ,bepress|Social and Behavioral Sciences|Linguistics|Psycholinguistics and Neurolinguistics ,Uncertainty quantification ,bepress|Social and Behavioral Sciences|Linguistics ,PsyArXiv|Social and Behavioral Sciences|Linguistics - Abstract
The use of statistical inference in linguistics and related areas like psychology typically involves a binary decision: either reject or accept some null hypothesis using statistical significance testing. When statistical power is low, this frequentist data-analytic approach breaks down: null results are uninformative, and effect size estimates associated with significant results are overestimated. Using an example from psycholinguistics, several alternative approaches are demonstrated for reporting inconsistencies between the data and a theoretical prediction. The key here is to focus on committing to a falsifiable prediction, on quantifying uncertainty statistically, and learning to accept the fact that – in almost all practical data analysis situations – we can only draw uncertain conclusions from data, regardless of whether we manage to obtain statistical significance or not. A focus on uncertainty quantification is likely to lead to fewer excessively bold claims that, on closer investigation, may turn out to be not supported by the data.
- Published
- 2021