The influence of study characteristics on coordinate-based fMRI meta-analyses

1 Introduction The reproducibility of fMRI studies suffers from small sample sizes, an overly stringent focus on minimizing type I errors (at the cost of power) and topological instability (Durnez et al., 2014; Roels et al., 2015). It is therefore increasingly recognized that further progress in understanding brain function will require integration of data across studies using meta-analyses (Wager et al., 2007). Methods for coordinate-based meta-analyses are designed to determine locations of activity based on studies that only report peak voxels whose evidence survive a selected threshold. In this contribution, we study the impact of analysis choices made at the individual study level on the coordinate-based meta-analysis methods. More specifically, we focus on pooling of subjects in which we consider (1) fixed effects pooling (no between-subjects variability), (2) ordinary least squares or OLS (homogeneous within-subjects variances) and (3) mixed effects pooling (estimating both). We use a large database to set up a simulation study and implement 3 coordinate- based meta-analysis techniques: activation likelihood estimation (ALE, Turkeltaub et al., 2002) and two weighted average meta-analyses. ALE calculates the convergence of activation at a specific voxel. The weighted average meta-analysis is either implemented as a fixed effects meta-analysis (assumes no between study variability) or a random effects meta-analysis (seed based d-mapping, SBdM, Radua et al. (2012)). 2 Method The design of the study is depicted in figure 1. We use data of 1400 subjects from the processed math > language contrast of the IMAGEN project (Schumann et al., 2010). In one iteration, 200 subjects are sampled that go into a ‘ground truth’ for which activation is assessed through mixed effects pooling with the False Discovery Rate (FDR) controlled at level 0.001. The other 200 subjects go into a test condition and are then sampled as 10 separate studies. Within each study, we apply the three different pooling methods and we obtain peak locations (FDR control at level 0.05). These study results are then combined in meta-analyses. For ALE, images are either uncorrected for multiple testing or corrected through FDR control (‘pID’ assumes independent or positive de- pendent test statistics among voxels, ‘pID’ makes no assumption on the joint distribution of test statistics). All meta-analyses are thresholded to control the type I error rate or FDR at level 0.05. The meta-analytic results are compared with the ground truth (200 different subjects) and a benchmark (group analysis as in ground truth but using the same 200 subjects as in the meta-analyses). We calculate the overlap-of-activation, power and false positive rate, FPR (Maitra, 2010). In total, 30 iterations are performed. 3 Results Results are given in figure 2. At study level, the OLS pooling method shows consistently lower levels of overlap, power and FPR while the mixed effects pooling method has the highest levels of overlap, power and FPR. For the meta-analysis methods, the fixed and random effects analyses show higher levels of overlap and power compared to ALE though with a slight increase in FPR. 4 Conclusions With this study, we show how analytical choices at the individual study level impact results on the meta-analytical level. More specifically, OLS leads to more conservative results while fixed/mixed effects pooling is more liberal. The different performance of the meta-analysis methods requires further research. One possible explanation is the implemented kernel sizes to model local maxima that differ between the methods.