Back to Search
Start Over
Estimating minimum effect with outlier selection
- Source :
- Annals of Statistics, Annals of Statistics, Institute of Mathematical Statistics, 2021, 49 (1), pp.272-294. ⟨10.1214/20-AOS1956⟩, Ann. Statist. 49, no. 1 (2021), 272-294
- Publication Year :
- 2018
- Publisher :
- arXiv, 2018.
-
Abstract
- We introduce one-sided versions of Huber's contamination model, in which corrupted samples tend to take larger values than uncorrupted ones. Two intertwined problems are addressed: estimation of the mean of uncorrupted samples (minimum effect) and selection of corrupted samples (outliers). Regarding the minimum effect estimation, we derive the minimax risks and introduce adaptive estimators to the unknown number of contaminations. Interestingly, the optimal convergence rate highly differs from that in classical Huber's contamination model. Also, our analysis uncovers the effect of particular structural assumptions on the distribution of the contaminated samples. As for the problem of selecting the outliers, we formulate the problem in a multiple testing framework for which the location/scaling of the null hypotheses are unknown. We rigorously prove how estimating the null hypothesis is possible while maintaining a theoretical guarantee on the amount of the falsely selected outliers, both through false discovery rate (FDR) or post hoc bounds. As a by-product, we address a long-standing open issue on FDR control under equi-correlation, which reinforces the interest of removing dependency when making multiple testing.<br />Comment: 70 pages; 7 figures
- Subjects :
- Statistics and Probability
False discovery rate
FOS: Computer and information sciences
minimax rate
selective inference
equicorrelation
moment matching
Mathematics - Statistics Theory
Statistics Theory (math.ST)
01 natural sciences
Methodology (stat.ME)
010104 statistics & probability
Contamination
[MATH.MATH-ST]Mathematics [math]/Statistics [math.ST]
0502 economics and business
Convergence (routing)
Statistics
FOS: Mathematics
[MATH]Mathematics [math]
0101 mathematics
[MATH.MATH-ST] Mathematics [math]/Statistics [math.ST]
Selection (genetic algorithm)
Statistics - Methodology
050205 econometrics
Mathematics
Hermite polynomials
62C20
multiple testing
sparsity
05 social sciences
Estimator
Minimax
[MATH.MATH-PR]Mathematics [math]/Probability [math.PR]
Multiple comparisons problem
Outlier
false discovery rate
Statistics, Probability and Uncertainty
Null hypothesis
post hoc
62G10
Subjects
Details
- ISSN :
- 00905364 and 21688966
- Database :
- OpenAIRE
- Journal :
- Annals of Statistics, Annals of Statistics, Institute of Mathematical Statistics, 2021, 49 (1), pp.272-294. ⟨10.1214/20-AOS1956⟩, Ann. Statist. 49, no. 1 (2021), 272-294
- Accession number :
- edsair.doi.dedup.....b0d711d355a0ba32f7716aae51a20466
- Full Text :
- https://doi.org/10.48550/arxiv.1809.08330