510 results on '"Juan Botella"'
Search Results
2. Heterogeneity estimation in meta-analysis of standardized mean differences when the distribution of random effects departs from normal: A Monte Carlo simulation study
- Author
-
Desirée Blázquez-Rincón, Julio Sánchez-Meca, Juan Botella, and Manuel Suero
- Subjects
Heterogeneity ,Non-normality ,Random effects ,Meta-analysis ,Between-study variance ,Simulation study ,Medicine (General) ,R5-920 - Abstract
Abstract Background Advantages of meta-analysis depend on the assumptions underlying the statistical procedures used being met. One of the main assumptions that is usually taken for granted is the normality underlying the population of true effects in a random-effects model, even though the available evidence suggests that this assumption is often not met. This paper examines how 21 frequentist and 24 Bayesian methods, including several novel procedures, for computing a point estimate of the heterogeneity parameter ( $${\tau }^{2}$$ τ 2 ) perform when the distribution of random effects departs from normality compared to normal scenarios in meta-analysis of standardized mean differences. Methods A Monte Carlo simulation was carried out using the R software, generating data for meta-analyses using the standardized mean difference. The simulation factors were the number and average sample size of primary studies, the amount of heterogeneity, as well as the shape of the random-effects distribution. The point estimators were compared in terms of absolute bias and variance, although results regarding mean squared error were also discussed. Results Although not all the estimators were affected to the same extent, there was a general tendency to obtain lower and more variable $${\tau }^{2}$$ τ 2 estimates as the random-effects distribution departed from normality. However, the estimators ranking in terms of their absolute bias and variance did not change: Those estimators that obtained lower bias also showed greater variance. Finally, a large number and sample size of primary studies acted as a bias-protective factor against a lack of normality for several procedures, whereas only a high number of studies was a variance-protective factor for most of the estimators analyzed. Conclusions Although the estimation and inference of the combined effect have proven to be sufficiently robust, our work highlights the role that the deviation from normality may be playing in the meta-analytic conclusions from the simulation results and the numerical examples included in this work. With the aim to exercise caution in the interpretation of the results obtained from random-effects models, the tau2() R function is made available for obtaining the range of $${\tau }^{2}$$ τ 2 values computed from the 45 estimators analyzed in this work, as well as to assess how the pooled effect, its confidence and prediction intervals vary according to the estimator chosen.
- Published
- 2023
- Full Text
- View/download PDF
3. Factors Associated with Revictimization in Intimate Partner Violence: A Systematic Review and Meta-Analysis
- Author
-
Ana Bellot, Marina J. Muñoz-Rivas, Juan Botella, and Ignacio Montorio
- Subjects
meta-analysis ,women ,revictimization ,intimate partner violence ,Psychology ,BF1-990 - Abstract
This study conducted a meta-analysis to identify the primary risk and protective factors associated with the revictimization in intimate partner violence against women (IPVAW). Out of 2382 studies initially identified in eight databases, 22 studies met the inclusion criteria and provided the necessary data for calculating pooled effect sizes. The analysis focused on non-manipulative quantitative studies examining revictimization in heterosexual women of legal age. Separate statistical analyses were performed for prospective and retrospective studies, resulting in findings related to 14 variables. The Metafor package in RStudio was used with a random-effects model. The meta-analysis revealed that childhood abuse was the most strongly associated risk factor for revictimization, while belonging to a white ethnicity was the most prominent protective factor. Other significant risk factors included alcohol and drug use, recent physical violence, severity of violence, and PTSD symptomatology. The study also found that older age was a protective factor in prospective studies. The consistency of results across different study designs and sensitivity analyses further supported the robustness of the findings. It is important to note that the existing literature on revictimization in women facing intimate partner violence is limited and exhibits significant heterogeneity in terms of methodology and conceptual frameworks.
- Published
- 2024
- Full Text
- View/download PDF
4. Behavioral Lifestyles and Survival: A Meta-Analysis
- Author
-
Rocío Fernández-Ballesteros, Elizabeth Valeriano-Lorenzo, Macarena Sánchez-Izquierdo, and Juan Botella
- Subjects
meta-analysis ,mortality ,sleep ,healthy diet ,sleeping ,weight control ,Psychology ,BF1-990 - Abstract
The aim of the study is to determine the association between Behavioral Lifestyles (regular physical activity, healthy diet, sleeping, and weight control) and longevity in the elderly. A search strategy was conducted in the PsycInfo, Medline, PubMed, Web of Science (WoS), and Scopus databases. The primary outcome was mortality/survival. Four variables (mean of participant's age at the baseline of the study, follow-up years of the study, gender, and year of publication) were analyzed to evaluate the role of potential moderators. Ninety-three articles, totaling more than 2,800,000 people, were included in the meta-analysis. We found that the lifestyles analyzed predict greater survival. Specifically, doing regular physical activity, engaging in leisure activities, sleeping 7–8 h a day, and staying outside the BMI ranges considered as underweight or obesity are habits that each separately has a greater probability associated with survival after a period of several years.
- Published
- 2022
- Full Text
- View/download PDF
5. Commentary: The Extent and Consequences of P-Hacking in Science
- Author
-
Juan Botella and Manuel Suero
- Subjects
questionable practice ,p-hacking ,p-values distribution ,cross disciplinary ,scientific hypotheses ,Psychology ,BF1-990 - Published
- 2020
- Full Text
- View/download PDF
6. Validity of Self-Reports Provided by People With Autism Spectrum Disorder Without Intellectual Disability: A Meta-Analysis
- Author
-
Marta Danés, Juan Botella, and Mercedes Belinchón
- Subjects
General Psychology - Abstract
Background: Literature collects that people with Autism Spectrum Disorder without Intellectual Disability (ASD-noID) frequently suffer co-occurring clinical problems that predict poorer adult outcome and reduce quality of life (QoL). Previous studies pose doubts about their capacity to recognize and communicate these problems, but that is an issue that, although relevant, has not yet been systematically revised. Method: A meta-analysis on 32 studies (1422 pairs of participants) was carried out to assess to what extent self-reports provided by people with ASD-noID match those provided by their parents. The articles were categorized according to the domains assessed and method. Results: Significant differences between self- and hetero-reports were found in the domains of Depression, QoL and Social skills (d= 0.406; -0.399 and -0.683, respectively), but not in Anxiety and Social anxiety. The average degree of agreement between both groups of reports was r = .40. Conclusions: Self- and hetero-reports do not provide interchangeable results on the problems of people with ASD-noID. However, a general inability to validate self-reporting cannot be attributed to them. Deeper analysis is necessary to allow to optimize the use of self-reports in this population with both clinical and research purposes. Antecedentes: La literatura recoge que en las personas con Trastorno del Espectro del Autismo sin discapacidad intelectual (TEA-noDI) concurren frecuentemente problemas clínicos que predicen peor ajuste en su vida adulta y reducen su calidad de vida (CdV). Estudios previos plantean dudas sobre la capacidad de estas personas para reconocer y comunicar estos problemas, pero esta cuestión, aunque relevante, no se ha revisado sistemáticamente. Método: Se realizó un meta-análisis sobre 32 estudios (1422 pares de participantes) para evaluar hasta qué punto los autoinformes aportados por personas con TEA-noDI coinciden con los aportados por sus padres. Los artículos se categorizaron en función de los dominios evaluados y el método. Resultados: Se encontraron diferencias significativas entre auto- y hetero-informes en los dominios de Depresión, CdV y Habilidades sociales (d = 0.406; -0.399 y -0.683, respectivamente), pero no en Ansiedad y Ansiedad social. El grado de acuerdo medio entre ambos grupos de evaluaciones fue r = .40. Conclusiones: Los autoinformes y los hetero-informes no proporcionan resultados intercambiables, pero no se puede atribuir a estas personas una incapacidad general para aportar autoinformes válidos. Se necesitan análisis más profundos que permitan optimizar el uso de autoinformes en esta población tanto con fines clínicos como de investigación.
- Published
- 2023
- Full Text
- View/download PDF
7. EL META-ANÁLISIS: UNA METODOLOGÍA PARA LA INVESTIGACIÓN EN EDUCACIÓN
- Author
-
Juan Botella and Ángela Zamora
- Subjects
Education - Abstract
Se presenta la metodolog a del meta-an lisis como herramienta para la investigaci n educativa. Un meta-an lisis implica una s ntesis cuantitativa de la evidencia acumulada sobre una pregunta de investigaci n previa - mente de nida. La respuesta se basar en la informaci n contenida en los estudios previamente publicados (estudios primarios). Las principales ca - racter sticas del meta-an lisis son la precisi n, la objetividad y la replica - bilidad. Su aplicaci n permite obtener una estimaci n combinada del ta - ma o del efecto. Tambi n permite evaluar la heterogeneidad observada en un campo de estudio. Ello permite a su vez formular nuevas hip tesis que incorporan el papel de variables que no se hab an tenido en cuenta hasta el momento. Un meta-an lisis comienza con la formulaci n del problema. A continuaci n se lleva a cabo la localizaci n de estudios relacionados con ella. En la fase de codi caci n se caracterizan los trabajos recogidos en la fase anterior. Finalmente, un an lisis estad stico conduce a los resultados, que se presentar n en una publicaci n que asegure la replicabilidad del estudio. Entre las herramientas gr cas asociadas al meta-an lisis des - taca el llamado Forest Plot , que permite presentar de forma conjunta las estimaciones puntuales y de intervalo de los estudios seleccionados para el meta-an lisis. Aunque el potencial del meta-an lisis es evidente, tam - bi n presenta algunas debilidades derivadas de la falta de acotaci n de los constructos estudiados o de la existencia de un sesgo en la publicaci n de resultados, sobre todo de los estad sticamente signi cativos. La valoraci n general que se hace del papel del meta-an lisis en la investigaci n en edu - caci n es positiva aunque tambi n se se alan reas de mejora. Se indican algunos recursos, tanto bibliogr cos como de software , que permiten a los investigadores interesados adentrarse en esta herramienta tan til en el campo de la investigaci n educativa.
- Published
- 2017
8. Research in Moral Education: The Contribution of P4C to the Moral Growth of Students
- Author
-
Félix García-Moriyón, Jara González-Lamas, Juan Botella, Javier González Vela, Tomás Miranda-Alonso, Antonio Palacios, and Rafael Robles-Loro
- Subjects
philosophy for children ,moral growth ,character ,virtues ,ethical education ,good person ,Education - Abstract
Moral education and moral growth are very important topics, and have been so as much in the fields of moral psychology and moral education as in the policies of governments and international institutions over the past decades. These two topics are also central themes within the educational proposal of Philosophy for Children (P4C), as seen in theoretical reflection and in educational research. It is necessary to start from a more global approach to moral growth, focused on the development of capacities. Such abilities are to be understood as virtues or personality traits that enable us to achieve a full life, that is, to become good people. The transformation of classrooms into communities of philosophical inquiry, following the educational guidelines of P4C, can contribute to the achievement of this objective. Here we present the psychological and methodological underpinnings of an educational research project that we are applying to a small sample that includes two groups—experimental and control—in a typical classroom environment. We are administering tests at the beginning and the end of the application of a moral education model according to the basic principles of Philosophy for Children. The objective is to verify that our research design could be used to evaluate the contribution of this educational model to the students’ moral growth, understood as the consolidation of the students’ moral habits and competences.
- Published
- 2020
- Full Text
- View/download PDF
9. Assessing Individual Change Without Knowing the Test Properties: Item Bootstrapping
- Author
-
Juan Botella, Desirée Blázquez, Manuel Suero, and James F. Juola
- Subjects
bootstrap ,individual change ,reliable change ,significant change ,psychometric properties ,meta-analysis ,Psychology ,BF1-990 - Abstract
Assessing significant change (or reliable change) in a person often involve comparing the responses of that person in two administrations of a test or scale. Several procedures have been proposed to determine if a difference between two observed scores is statistically significant or rather is within the range of mere random fluctuations due to measurement error. Application of those procedures involve some knowledge of the test properties. But sometimes those procedures cannot be employed because the properties are unknown or are not trustworthy. In this paper we propose the bootstrap of items procedure to create confidence intervals of the individual's scores without using any known psychometric properties of the test. Six databases containing the responses of several groups to one or more subscales have been analyzed using two methods: bootstrap of items and a classical procedure based on confidence intervals to estimate the true score. The rates of significant change obtained were very similar, suggesting that item bootstrapping is a promising solution when other methods cannot be applied.
- Published
- 2018
- Full Text
- View/download PDF
10. Intelligence and life expectancy in late adulthood: A meta-analysis
- Author
-
Macarena Sánchez-Izquierdo, Rocío Fernández-Ballesteros, Elizabeth Lucía Valeriano-Lorenzo, and Juan Botella
- Subjects
Arts and Humanities (miscellaneous) ,Developmental and Educational Psychology ,Experimental and Cognitive Psychology - Published
- 2023
- Full Text
- View/download PDF
11. Concurrent memory load can make RSVP search more efficient
- Author
-
Beatriz Gil-Gómez de Liaño and Juan Botella
- Subjects
Philology. Linguistics ,P1-1091 ,Psychology ,BF1-990 ,Consciousness. Cognition ,BF309-499 - Abstract
El efecto perjudicial de la carga de memoria en la atención selectiva ha sido ampliamente estudiado en muy diversas situaciones. Sin embargo, en situaciones de búsqueda visual en el tiempo utilizando Presentaciones Rápidas de Series Visuales (PRSV), no está claro cómo la carga de memoria puede afectar los procesos atencionales involucrados en este tipo de tareas; una ausencia de efecto, así como efectos tanto beneficiosos como perjudiciales de la carga de memoria se han encontrado utilizando este tipo de tareas. El principal objetivo del presente trabajo es aportar más evidencia sobre el papel que la memoria de trabajo juega en situaciones de búsqueda visual en el tiempo. Utilizando un paradigma de PRSV, manipulamos el número de distractores (experimento 1) y la congruencia en una tarea tipo-Stroop (experimento 2), encontrando que bajo situaciones de alta carga de memoria se puede incrementar la eficiencia en la búsqueda visual en el tiempo. Nuestros datos apoyan la existencia de similitudes entre la atención en el espacio y en el tiempo, planteando la posibilidad de que exista un sistema atencional general independiente de la dimensión estimular. Sin embargo, también encontramos importantes diferencias, por lo que se discuten las implicaciones teóricas que puedan explicar los resultados encontrados.
- Published
- 2011
12. The hard but necessary task of gathering order-one effect size indices in meta-analysis
- Author
-
Juan Botella and Carmen Ortego
- Subjects
Philology. Linguistics ,P1-1091 ,Psychology ,BF1-990 ,Consciousness. Cognition ,BF309-499 - Abstract
El meta-análisis de estudios primarios con diseño de dos grupos y medidas en dos momentos debe emplear índices de tamaño de efecto de orden uno. Especialmente si la asignación no es aleatoria, con diseños de grupo control no equivalente, las conclusiones alcanzadas pueden estar fuertemente sesgadas si sólo se incluyen las medidas post tratamiento. 109 estudios primarios incluidos en 4 meta-análisis fueron recopilados y se contactó con sus autores para pedirles los datos originales con fin de estimar los índices de orden uno. De este total sólo se consiguió 13 estudios primarios. Los resultados obtenidos con los datos originales fueron comparados con los estimados con los índices de orden cero y uno. A pesar de las dificultades para conseguir los datos, el pequeño grupo de datos analizados mostró que si el meta-análisis se realizaba con índices de orden cero las conclusiones eran erróneas.
- Published
- 2010
13. Event-related potentials and illusory conjunctions in the time domain
- Author
-
Beatriz Gil-Gómez de Liaño, Carmen Rodríguez, Ma Eugenia Rubio, Fernando Valle-Inclán, and Juan Botella
- Subjects
Philology. Linguistics ,P1-1091 ,Psychology ,BF1-990 ,Consciousness. Cognition ,BF309-499 - Abstract
(Rapid Serial Visual Presentation, RSVP) can migrate forming a wrong combination or illusory conjunction. Several serial and parallel models have been proposed to explain the generation of this type of errors. The behavioral results fit better the two-stage parallel model than other serial and parallel models. However, they have not been studied the psychophysiological correlates that distinguish successful bindings from Illusory Conjunctions. The goal here is to collect electrophysiological records during this task to determine the degree to which they converge with the evidence from behavioral results. One RSVP task required to identify the only uppercase word in a stream of lowercase words at a rate of 12 items/sec. As in previous experiments, more intrusions from post-target items than from pre-target items were observed. The results from eventrelated potentials are also more supportive for the two-stage parallel model than for the serial or other parallel models, as reflected in the differential waves associated to correct and wrong combinations
- Published
- 2008
14. Methods for estimating the sampling variance of the standardized mean difference
- Author
-
Manuel Suero, Juan Botella, and Juan I. Durán
- Subjects
Meta-analysis ,Standardized Mean Difference ,Variance of g ,Effect Size Variance ,Sampling Variance of d ,Psychology (miscellaneous) - Abstract
One of the most widely used effect size indices for meta-analysis in psychology is the standardized mean difference (SMD). The most common way to synthesize a set of estimates of the SMD is to weight them by the inverse of their variances. For this, it is necessary to estimate the corresponding sampling variances. Meta-analysts have a formula for obtaining unbiased estimates of sampling variances, but they often use a variety of alternative, simpler methods. The bias and efficiency of five different methods that have been proposed and that are implemented in different computerized calculation tools are compared and assessed. The data from a set of published meta-analyses are also reanalyzed, calculating the combined estimates and their confidence intervals, as well as estimates of the specific, between-studies variance, using the five estimation methods. This test of sensitivity shows that the results of a meta-analysis can change noticeably depending on the method used to estimate the sampling variance of SMD values, especially under a random-effects model. Some practical recommendations are made about how to choose and implement the methods in calculation resources Ministerio de Ciencia e Innovación 2021-22
- Published
- 2021
- Full Text
- View/download PDF
15. Doing and reporting a meta-analysis
- Author
-
Juan Botella and Hilda Gambara
- Subjects
Psychology ,BF1-990 - Abstract
El meta-análisis, una metodología para revisar, integrar y analizar resultados de investigación, se ha extendido por todos los campos de la Psicología. Sin embargo, debido a su relativa juventud y a la relativa heterogeneidad de las prácticas aparecidas bajo este epígrafe, muchos investigadores no han adquirido una buena formación básica sobre ella. En el presente estudio teórico se describe el proceso completo, comenzando por las preguntas preliminares y las motivaciones que lo justifican; en segundo lugar, se detalla cómo afrontar cada una de las tareas, ordenadas como fases, que integran un meta-análisis; por último, se expone la forma como se debe informar de los resultados de un meta-análisis en una revista científica como International Journal of Clinical and Health Psychology. Además de destacar algunos problemas que surgen con frecuencia durante el proceso, se presentan también algunos consejos dirigidos a los novatos. Para terminar, se propone una guía de aspectos relevantes que se deben tener en cuenta para la elaboración de un informe meta-analítico. Esta guía está dirigida tanto a los autores como a los revisores de un meta-análisis.
- Published
- 2006
16. Autoestima e inicio de actividad sexual en la adolescencia: un estudio meta-analítico
- Author
-
Elvia Vargas-Trujillo, Hilda Gambara, and Juan Botella
- Subjects
Psychology ,BF1-990 - Abstract
RESUMEN. Se presentan los resultados de un estudio meta-analítico sobre la relación de la autoestima con el inicio de actividad sexual en la adolescencia. Se ubicaron 38 informes de investigación realizados entre 1975 y 2003 con muestras provenientes de Australia, Bolivia, Estados Unidos, Malí, Noruega, Nueva Zelanda y Perú. Se obtuvieron 55 tamaños del efecto independientes que recogían 68.703 datos de adolescentes. Se definió como índice del tamaño del efecto la diferencia media tipificada de autoestima entre el grupo de adolescentes activos o que inician actividad sexual antes de los 16 años (grupo de alto riesgo) y el grupo de adolescentes no activos o que inician más tarde (grupo de bajo riesgo). Los resultados mostraron una diferencia media tipificada pequeña pero estadísticamente significativa en el nivel de autoestima que presentan los adolescentes que conforman el grupo de alto riesgo y los que hacen parte del grupo de bajo riesgo (d = -0,0334). El tamaño del efecto sugiere que los adolescentes no activos o que inician más tarde tienden a informar niveles de autoestima más altos que sus pares activos o que inician más temprano. El tamaño del efecto varía en función de algunas características metodológicas de las investigaciones, del grupo étnico y el país de origen de la muestra.
- Published
- 2006
17. Comparación entre índices de tamaño del efecto para variables dicotomizadas en Meta-análisis
- Author
-
Mónica Teresa González Ramírez and Juan Botella
- Subjects
Philology. Linguistics ,P1-1091 ,Psychology ,BF1-990 ,Consciousness. Cognition ,BF309-499 - Abstract
El objetivo principal del presente trabajo es comparar, con datos provenientes de un meta-análisis real, el comportamiento de 7 índices para calcular el tamaño del efecto en variables dicotomizadas: dp, dφ, dasine, dHH, dCox, dprobit, dbis. Estos índices fueron estudiados por Sánchez-Meca, Marín-Martínez y Chacón-Moscoso (2003) a través de una simulación Monte Carlo. Considerando los resultados, se concluye que los 7 índices para calcular TE no son equivalentes; asimismo, se pone de manifiesto que el uso de uno u otro índice para calcular el tamaño del efecto puede tener consecuencias importantes en un meta-análisis, conduciendo incluso a conclusiones opuestas. Es por esto que se recomienda que para cálculos del tamaño del efecto para variables dependientes dicotomizadas se utilicen los índices dCox, dprobit y dHH para poder comparar sus resultados e incrementar la validez de las conclusiones de un meta-análisis. Estas conclusiones coinciden con las del estudio de simulación de Sánchez-Meca et al. (2003).
- Published
- 2006
18. Aspectos controvertidos de la imagen corporal en los trastornos de la conducta alimentaria
- Author
-
ANA R. SEPÚLVEDA, JOSÉ A. LEÓN, and JUAN BOTELLA
- Subjects
Psychology ,BF1-990 - Abstract
Este trabajo revisa los resultados de 31 estudios que evalúan de alguna forma la "alteración de la imagen corporal" en anorexia nerviosa y bulimia nerviosa a través de los métodos de evaluación perceptiva o los métodos de evaluación actitudinal. Los estudios presentan en la mayoría de los casos resultados contradictorios entre ellos. Los autores ofrecen una serie de explicaciones metodológicas y meta-analíticas para tratar de interpretar los resultados.
- Published
- 2004
19. Testing Categorical Moderators in Mixed-Effects Meta-analysis in the Presence of Heteroscedasticity
- Author
-
Juan Botella, María Rubio-Aparicio, José A López-López, Julio Sánchez-Meca, Fulgencio Marín-Martínez, Wolfgang Viechtbauer, Psychiatrie & Neuropsychologie, RS: MHeNs - R2 - Mental Health, Universidad de Alicante. Departamento de Psicología de la Salud, and Psicología Aplicada a la Salud y Comportamiento Humano (PSYBHE)
- Subjects
Mixed model ,Heteroscedasticity ,VARIANCE ESTIMATORS ,Residual ,Education ,Mixed-effects model ,subgroup analyses ,Statistical significance ,Statistics ,Developmental and Educational Psychology ,HETEROGENEITY ,Association (psychology) ,Categorical variable ,EFFECTS META-REGRESSION ,Mathematics ,ANOVA ,Residual between-studies variance ,fungi ,food and beverages ,Moderation ,Meta-analysis ,Subgroup analyses ,Personalidad, Evaluación y Tratamiento Psicológico ,residual between-studies variance ,mixed-effects model - Abstract
Mixed-effects models can be used to examine the association between a categorical moderator and the magnitude of the effect size. Two approaches are available to estimate the residual between-studies variance, s 2res —namely, separate estimation within each category of the moderator versus pooled estimation across all categories. We examine, by means of a Monte Carlo simulation study, both approaches for s 2res estimation in combination with two methods, the Wald-type X 2 and F tests, to test the statistical significance of the moderator. Results suggest that the F test using a pooled estimate of s 2res across categories is the best option in most conditions, although the F test using separate estimates of s 2res is preferable if the residual heterogeneity variances are heteroscedastic.
- Published
- 2020
- Full Text
- View/download PDF
20. Familiarity, recollection, and receiver-operating characteristic (ROC) curves in recognition memory
- Author
-
Juan Botella, Alexandra Caballero-Sanz, James F. Juola, Manuel Suero, and Adrián R. Muñoz-García
- Subjects
Adult ,Male ,Adolescent ,Experimental and Cognitive Psychology ,Models, Psychological ,050105 experimental psychology ,Young Adult ,03 medical and health sciences ,0302 clinical medicine ,Arts and Humanities (miscellaneous) ,Humans ,0501 psychology and cognitive sciences ,Detection theory ,Sensitivity (control systems) ,Recognition memory ,Interpretation (logic) ,Recall ,Receiver operating characteristic ,business.industry ,05 social sciences ,Recognition, Psychology ,Pattern recognition ,Data set ,Neuropsychology and Physiological Psychology ,ROC Curve ,Mental Recall ,Female ,Artificial intelligence ,Construct (philosophy) ,Psychology ,business ,030217 neurology & neurosurgery - Abstract
The Atkinson-Shiffrin theory describes and explains some of the processes involved in storing and retrieving information in human memory. Here we examine predictions of related models for search and decision processes in recognizing information in long-term memory. In some models, recognition is presumably based on a test item's familiarity judgment, and subsequent decisions follow from the sensitivity and decision parameters of signal detection theory. Other models dispense with the continuous notion of familiarity and base recognition on discrete internal states such as relative certainty that an item has or has not been previously studied, with an intermediate state of uncertainty that produces guesses. Still others are hybrid models with two criteria located along a familiarity continuum defining areas for rapid decisions based on high or low familiarities. For intermediate familiarity values, the decision can be delayed pending the results of search for, and occasional recollection of, relevant episodic information. Here we present the results from a study of human recognition memory for lists of words using both response time and error data to construct receiver-operating characteristic (ROC) curves derived from three standard methods based on the same data set. Models are evaluated against, and parameters estimated from, group as well as individual subjects' behavior. We report substantially different ROC curves when they are based on variations in target-word frequency, confidence judgments, and response latencies. The results indicate that individual versus group data must be used with caution in determining the appropriate theoretical interpretation of recognition memory performance.
- Published
- 2019
- Full Text
- View/download PDF
21. The Fear of COVID-19 Scale: A Reliability Generalization Meta-Analysis
- Author
-
Juan Botella, Desirée Blázquez-Rincón, and Juan I Durán
- Subjects
reliability generalization ,FCV-19S ,Rasch model ,Scale (ratio) ,Psychometrics ,Generalization ,SARS-CoV-2 ,COVID-19 ,Reproducibility of Results ,Fear ,Standard deviation ,Classical test theory ,meta-analysis ,Clinical Psychology ,Cronbach's alpha ,Meta-analysis ,Statistics ,classical test theory ,fear ,Humans ,Rasch measurement model ,Psychology ,Pandemics ,Applied Psychology ,Reliability (statistics) - Abstract
A reliability generalization meta-analysis was carried out to estimate the average reliability of the seven-item, 5-point Likert-type Fear of COVID-19 Scale (FCV-19S), one of the most widespread scales developed around the COVID-19 pandemic. Different reliability coefficients from classical test theory and the Rasch Measurement Model were meta-analyzed, heterogeneity among the most reported reliability estimates was examined by searching for moderators, and a predictive model to estimate the expected reliability was proposed. At least one reliability estimate was available for a total of 44 independent samples out of 42 studies, being that Cronbach’s alpha was most frequently reported. The coefficients exhibited pooled estimates ranging from .85 to .90. The moderator analyses led to a predictive model in which the standard deviation of scores explained 36.7% of the total variability among alpha coefficients. The FCV-19S has been shown to be consistently reliable regardless of the moderator variables examined.
- Published
- 2021
22. El pequeño impacto del haqueo de resultados marginalmente significativos sobre la estimación meta-analítica del tamaño del efecto
- Author
-
Desirée Blázquez, Manuel Suero, Juan I Durán, and Juan Botella
- Subjects
meta-analysis ,Meta-análisis ,Philosophy ,effect size ,1 - Filosofía y psicología::159.9 - Psicología [CDU] ,p-hacking ,Psychology ,P-hacking ,Humanities ,General Psychology ,Tamaño del efecto ,BF1-990 - Abstract
La etiqueta p-hacking (pH) se refiere a un conjunto de prácticas oportunistas destinadas a hacer que sean significativos algunos valores p que deberían ser no significativos. Algunos han argumentado que debemos prevenir y luchar contra el pH por varias razones, especialmente debido a sus posibles efectos nocivos en la evaluación de los resultados de la investigación primaria y su síntesis meta-analítica. Nos focalizamos aquí en el efecto de un tipo específico de pH, centrado en estudios marginalmente significativos, en la estimación combinada del tamaño del efecto en el meta-análisis. Queremos saber cuánto deberíamos preocuparnos por su efecto de sesgo al evaluar los resultados de un meta-análisis. Hemos calculado el sesgo en una variedad de situaciones que parecen realistas en términos de prevalencia y de la definición operativa del pH. Los resultados muestran que en la mayoría de las situaciones analizadas el sesgo es inferior a una centésima (± 0.01), en términos de d o r. Para alcanzar un nivel de sesgo de cinco centésimas (± 0.05), tendría que haber una presencia masiva de este tipo de pH, lo que parece poco realista. Hay muchas buenas razones para luchar contra el pH, pero nuestra conclusión principal es que entre esas razones no se incluye que tenga un gran impacto en la estimación meta-analítica del tamaño del efecto. The label p-hacking (pH) refers to a set of opportunistic practices aimed at making statistically significant p values that should be non-significant. Some have argued that we should prevent and fight against pH for several reasons, especially because of its potential harmful effects on the assessment of both primary research results and their meta-analytical synthesis. We focus here on the effect of a specific type of pH, focused on marginally significant studies, on the combined estimation of effect size in meta-analysis. We want to know how much we should be concerned with its biasing effect when assessing the results of a meta-analysis. We have calculated the bias in a range of situations that seem realistic in terms of the prevalence and the operational definition of pH. The results show that in most of the situations analyzed the bias is less than one hundredth (± 0.01), in terms of d or r. To reach a level of bias of five-hundredths (± 0.05), there would have to be a massive presence of this type of pH, which seems rather unrealistic. There are many good reasons for fighting against pH, but our main conclusion is that among them is not that it has a big impact on the meta-analytical estimation of effect size.
- Published
- 2021
23. Recovering the variance of d' from hit and false alarm statistics
- Author
-
Juan Botella and Manuel Suero
- Subjects
Models, Statistical ,Homogeneity (statistics) ,Decision Making ,05 social sciences ,Recognition, Psychology ,Experimental and Cognitive Psychology ,Multiple-criteria decision analysis ,050105 experimental psychology ,Confidence interval ,03 medical and health sciences ,0302 clinical medicine ,Bias ,Arts and Humanities (miscellaneous) ,Meta-analysis ,Statistics ,Developmental and Educational Psychology ,Humans ,0501 psychology and cognitive sciences ,Detection theory ,Sample variance ,Psychology (miscellaneous) ,False alarm ,030217 neurology & neurosurgery ,General Psychology ,Mathematics - Abstract
Sometimes the reports of primary studies that are potentially analyzable within the signal detection theory framework do not report sample statistics for its main indexes, especially the sample variance of d'. We describe a procedure for estimating the variance of d' from other sample statistics (specifically, the mean and variance of the observed rates of hit and false alarm). The procedure acknowledges that individuals can be heterogeneous in their sensitivity and/or decision criteria, and it does not adopt unjustifiable or needlessly complex assumptions. In two simulation studies reported here, we show that the procedure produces certain biases, but, when used in meta-analysis, it produces very reasonable results. Specifically, the weighted estimate of the mean sensitivity is very accurate, and the coverage of the confidence interval is very close to the nominal confidence level. We applied the procedure to 20 experimental groups or conditions from seven articles (employing recognition memory or attention tasks) that reported statistics for both the hit and false alarm rates, as well as for d'. In most of these studies the assumption of homogeneity was untenable. The variances estimated by our method, based on the hit and false alarm rates, approximate reasonably to the variances in d' reported in those articles. The method is useful for estimating unreported variances of d', so that the associated studies can be retained for meta-analyses.
- Published
- 2019
- Full Text
- View/download PDF
24. Estimating the expected value and variance of SDT indexes with heterogeneous individuals
- Author
-
Jesús Privado, Juan Botella, and Manuel Suero
- Subjects
Applied Mathematics ,05 social sciences ,050109 social psychology ,Sample (statistics) ,Variance (accounting) ,Expected value ,050105 experimental psychology ,Homogeneous ,Statistics ,0501 psychology and cognitive sciences ,Detection theory ,Sensitivity (control systems) ,General Psychology ,Parametric statistics ,Mathematics - Abstract
Several methods of calculating the variance of the more popular parametric indexes of signal detection theory (SDT) have been proposed. Miller’s exact method and Gourevitch and Galanter’s approximate method have proved appropriate for most practical situations using yes–no detection paradigms. Those methods, specifically developed for single-point estimates, are also useful for a sample of estimates that share their parametric values of sensitivity and bias, δ and C , respectively. However, sometimes it has been observed that the empirical variances of the estimates, σ ˆ d ′ 2 and σ ˆ c 2 , exceeds by several factors the expected values calculated by the above methods. These discrepancies can be due to a number of reasons, perhaps the most likely one being that the sample of observers or participants is not actually homogeneous. We present further developments of those two methods for calculating the variance of the indexes, d ′ and c , when the samples are heterogeneous in sensitivity and/or bias. Simulation studies show that our formulas provide reasonably accurate estimates of the variance of d ′ and c .
- Published
- 2018
- Full Text
- View/download PDF
25. What is the Psychosocial Impact of Disasters? A Meta-Analysis
- Author
-
Juan Botella, Marcelo Leiva-Bianchi, Andrea Araneda, and Francisco Ahumada
- Subjects
Health Services Needs and Demand ,Mental Disorders ,MEDLINE ,Witness ,030227 psychiatry ,Disasters ,03 medical and health sciences ,0302 clinical medicine ,Work (electrical) ,Meta-analysis ,Humans ,030212 general & internal medicine ,Meaning (existential) ,Pshychiatric Mental Health ,Psychology ,Psychosocial ,Social psychology - Abstract
The psychosocial impact concept improves the understanding of the effects of disasters on people and communities. However, its definition is not clear. This work explores consistencies between studies (k = 21) that refer to the psychosocial impact of disasters by way of a meta-analytic synthesis. This synthesis indicates that people are more prone to illness when they are exposed to a disaster, and less when they are protected (OR = 2.737). Please check the change conveys the intended meaning or amend Nevertheless, there are no differences in healthy responses, regardless of how protected or exposed they are (OR = 1.053). Finally, a model is proposed to explain four types of psychosocial impact: resilient, traumatic, sensitive, witness.
- Published
- 2018
- Full Text
- View/download PDF
26. Compartir datos: hacia una investigación más sostenible
- Author
-
Ausina, Juan Botella and Maté, Carmen Ortego
- Published
- 2010
27. Memory bias for threatening information related to anxiety: an updated meta-analytic review
- Author
-
Ignacio Montorio, Isabel Cabrera, Sara Herrera, and Juan Botella
- Subjects
050103 clinical psychology ,Memory errors ,Recall ,05 social sciences ,Experimental and Cognitive Psychology ,050105 experimental psychology ,Cognitive bias ,Free recall ,Encoding (memory) ,Meta-analysis ,Explicit memory ,medicine ,Anxiety ,0501 psychology and cognitive sciences ,medicine.symptom ,Psychology ,Cognitive psychology - Abstract
The evidence for an anxiety-related memory bias is contradictory. We compiled 171 articles published until October 2016 including a group with clinical or subclinical anxiety and a control group in tasks involving implicit or explicit memory using threatening stimuli. There was an anxiety-related memory bias in free recall tasks, but it was not observed in another memory task. The between-groups differences showed that the anxious group recalled more threatening stimuli than the control group (d = 0.321). When we compared the group differences (anxious vs. control participants) in the within-groups effect (threatening vs. neutral stimuli), a moderate effect size emerged (dbw = 0.714). This anxiety-related memory bias was observed with shallow processing, that is consistent with attentional biases related to anxiety. There was also evidence that high-anxious persons recall fewer positive stimuli. Future research is needed to investigate whether this result is a memory or encoding bias and explore o...
- Published
- 2017
- Full Text
- View/download PDF
28. On the Reality of Illusory Conjunctions
- Author
-
Manuel Suero, Juan Botella, and Juan I Durán
- Subjects
Male ,Adolescent ,Alternative hypothesis ,media_common.quotation_subject ,Experimental and Cognitive Psychology ,Space (commercial competition) ,050105 experimental psychology ,Gender Studies ,Young Adult ,03 medical and health sciences ,0302 clinical medicine ,Arts and Humanities (miscellaneous) ,Phenomenon ,Perception ,Humans ,Attention ,0501 psychology and cognitive sciences ,media_common ,Communication ,Spacetime ,business.industry ,05 social sciences ,Models, Theoretical ,Illusions ,Conjunction (grammar) ,Feature (linguistics) ,Visual Perception ,Illusory conjunctions ,Female ,Psychology ,business ,030217 neurology & neurosurgery ,Cognitive psychology - Abstract
The reality of illusory conjunctions in perception has been sometimes questioned, arguing that they can be explained by other mechanisms. Most relevant experiments are based on migrations along the space dimension. But the low rate of illusory conjunctions along space can easily hide them among other types of errors. As migrations over time are a more frequent phenomenon, illusory conjunctions can be disentangled from other errors. We report an experiment in which series of colored letters were presented in several spatial locations, allowing for migrations over both space and time. The distribution of frequencies were fit by several multinomial tree models based on alternative hypothesis about illusory conjunctions and the potential sources of free-floating features. The best-fit model acknowledges that most illusory conjunctions are migrations in the time domain. Migrations in space are probably present, but the rate is very low. Other conjunction errors, as those produced by guessing or miscategorizations of the to-be-reported feature, are also present in the experiment. The main conclusion is that illusory conjunctions do exist.
- Published
- 2017
- Full Text
- View/download PDF
29. Measurement of executive functioning and high intellectual ability in childhood: A comparative meta-analysis
- Author
-
Sylvia Sastre-Riba, Lourdes Viana-Sáenz, Juan Botella, Maria Luz Urraca-Martínez, and UAM. Departamento de Psicología Social y Metodología
- Subjects
validity ,Geography, Planning and Development ,TJ807-830 ,Validity ,Management, Monitoring, Policy and Law ,TD194-195 ,Renewable energy sources ,050105 experimental psychology ,Developmental psychology ,Executive functions ,03 medical and health sciences ,0302 clinical medicine ,Keywords: high intellectual ability ,GE1-350 ,0501 psychology and cognitive sciences ,Reliability (statistics) ,Measurement ,reliability ,Environmental effects of industries and plants ,Renewable Energy, Sustainability and the Environment ,Working memory ,Average intelligence ,05 social sciences ,Intellectual ability ,Flexibility (personality) ,executive functions ,Reliability ,Psicología ,meta-analysis ,Environmental sciences ,Meta-analysis ,High intellectual ability ,measurement ,Psychology ,030217 neurology & neurosurgery - Abstract
From a neuroconstructivist approach and a developmental model of high intellectual ability (HIA), it is argued that the management of intellectual resources through executive functioning (EF) is one of the factors influencing the expression of high potential. The main objective is to determine the effectiveness of measures of executive functioning used comparing schoolchildren with HIA and those of average intelligence. A meta-analysis was carried out on a selection of 17 studies for a total sample of 1518 children with either HIA or an average level of intelligence. Pooled estimates of effect size revealed a significant difference favoring the HIA individuals in the two components of EF related with WM verbal (d = 1.015), and WM visual-spatial (d = 0.709). Other components did not show significant differences: inhibition (d = &minus, 0.014), flexibility (d = 0.068), and planification (d = &minus, 0.038). The empirical heterogeneity was very high. It is concluded that these instruments show a degree of measurement impurity, which condition their validity and reliability, and that schoolchildren with HIA display better executive functioning in the components of verbal and visual-spatial working memory.
- Published
- 2020
30. Impairment of cognitive memory inhibition in individuals with intellectual disability: A meta-analysis
- Author
-
Elena, Palomino, José M, López-Frutos, Juan, Botella, and María, Sotillo
- Subjects
Executive Function ,Inhibition, Psychological ,Memory Disorders ,Memory, Short-Term ,Intellectual Disability ,Age Factors ,Humans ,Selection Bias - Abstract
Cognitive inhibition impairment is intimately related to the forgetfulness of relevant information. This meta-analysis aims to synthesise the evidence of impaired function of cognitive inhibition processes over memory in individuals with intellectual disability (ID).Eleven studies were selected and analysed and included a total of 683 participants. The studies were categorised according to variables such as the task used, the processes involved, the sensory modalities and the method.Despite the small sample of studies, the results revealed significant difficulties with cognitive memory inhibition (CMI) tasks in individuals with ID compared with typical development (TD) individuals (d = 0.62). CMI problems were found in all life stages except the 19-45-year-old stage. In this stage, there was a smaller amount of evidence even though it included the 31-40-year-old range, during which premature aging has been observed in ID.An impairment of CMI in people with ID was observed. More studies are needed to more reliably assess the potential moderating role of age and other factors.
- Published
- 2019
31. A meta-analytical answer to the crisis of confidence in psychology
- Author
-
Juan I Durán, Juan Botella, and UAM. Departamento de Psicología Social y Metodología
- Subjects
Questionable practices ,questionable practices ,Meta-análisis ,Process (engineering) ,Psychological research ,Perspective (graphical) ,Publication bias ,Crisis of confidence ,Effect size ,Object (philosophy) ,Psicología ,159.9 ,Replication (computing) ,Prácticas cuestionables ,BF1-990 ,Epistemology ,Unit (housing) ,Meta-analysis ,Crisis de confianza ,Psychology ,Natural (music) ,Tamaño del efecto ,General Psychology - Abstract
El meta-análisis es una metodología firmemente establecida y una parte integral del proceso de generación de conocimiento en las ciencias empíricas. El meta-análisis también se ha centrado en la metodología y se ha convertido en uno de los principales críticos de las deficiencias metodológicas. Destacamos varios puntos problemáticos sobre cómo investigamos en psicología: exceso de heterogeneidad en los resultados y dificultades para la replicación, sesgo de publicación, calidad metodológica subóptima y prácticas cuestionables de los investigadores. Estos y otros problemas condujeron a una "crisis de confianza" en psicología. Discutimos cómo la perspectiva meta-analítica y sus procedimientos pueden ayudar a afrontar la crisis. Una perspectiva más cooperativa, en lugar de competitiva, puede ayudar a cambiar para que consideremos la replicación como una contribución más valiosa. El conocimiento no puede basarse en estudios aislados. Dada la naturaleza del objeto de estudio de la psicología la unidad natural para generar conocimiento debe ser la distribución estimada del tamaño del efecto, no la decisión dicotómica sobre la significación estadística en estudios específicos. Se ofrecen algunas sugerencias sobre cómo re-dirigir la investigación y las prácticas de los investigadores, de modo que sus intereses personales y los de la ciencia en sí estén más alineados, Meta-analysis is a firmly established methodology and an inte-gral part of the process of generating knowledge across the empirical sci-ences. Meta-analysis has also focused on methodology and has become a dominant critic of methodological shortcomings. We highlight several problematic issues on how we research in psychology: excess of heteroge-neity in the results and difficulties for replication, publication bias, subop-timal methodological quality, and questionable practices of the researchers. These and other problems led to a “crisis of confidence” in psychology. We discuss how the meta-analytical perspective and its procedures can help to overcome the crisis. A more cooperative perspective, instead of a competitive one, can shift to consider replication as a more valuable con-tribution. Knowledge cannot be based in isolated studies. Given the nature of the object of study of psychology, the natural unit to generate knowledge must be the estimated distribution of the effect sizes, not the dichotomous decision on statistical significance in specific studies. We make some suggestions on how to redirect the research and the research-ers' practices, so that their personal interests and those of science as such are better aligned, This research was supported by the Ministerio de Economía y Competitividad of Spain (project reference: PSI2017-82490-P)
- Published
- 2019
32. Temporal interactions between target and distractor processing: Positive and negative priming effects
- Author
-
María Isabel Barriopedro, James F. Joula, and Juan Botella
- Subjects
Philology. Linguistics ,P1-1091 ,Psychology ,BF1-990 ,Consciousness. Cognition ,BF309-499 - Abstract
Interacciones Temporales entre el Procesamiento del Blanco y los Distractores: Efectos de Priming Positivo y Negativo. El paradigma de los flancos y el paradigma con presentaciones de preparación/prueba (prime/probe) para el estudio del priming positivo y negativo se basan en la compatibilidad entre información relevante e irrelevante presente en los mismos estímulos o en estímulos que son temporal o espacialmente contiguos. En el paradigma de los flancos se presentan distractores a la vez que el blanco que pueden mejorar el rendimiento sin son flancos compatibles y empeorarlo si son incompatibles. En el paradigma de priming los distractores pueden facilitar o interferir con respuestas a blancos compatibles que se presentan posteriormente. En los experimentos que se describen aquí hemos conseguido una transición gradual entre estos dos paradigmas, mediante el uso del procedimiento de Presentación Rápida de Series Visuales (PRSV), manipulando la compatibilidad del distractor y el desfase temporal entre los distractores y el blanco. Con SOAs cortos los distractores compatibles facilitan y los incompatibles interfieren; pero con SOAs en torno a 400 mseg. el rendimiento es peor con distractores compatibles que con incompatibles. Se han obtenido resultados similares tanto con paradigmas en los que los participantes deben responder al estímulo que produce el efecto (es un blanco) como con paradigmas en los que no tienen que responder a él (es un distractor). Los presentes resultados implican importantes limitaciones en las explicaciones teóricas tanto del efecto de compatibilidad de los flancos como de la dinámica temporal del priming positivo y negativo.
- Published
- 2002
33. Can personality traits be measured analyzing written language? A meta-analytic study on computational methods
- Author
-
Ricardo Olmos, Juan Botella, José Á. Martínez-Huertas, Guillermo Jorge-Botana, J. D. Moreno, UAM. Departamento de Psicología Básica, and UAM. Departamento de Psicología Social y Metodología
- Subjects
Agreeableness ,media_common.quotation_subject ,Computational models of language ,05 social sciences ,050109 social psychology ,Neuroticism ,Psicología ,050105 experimental psychology ,Meta-analysis ,Openness to experience ,Big five ,Personality ,0501 psychology and cognitive sciences ,Written language ,Big Five personality traits ,Empirical evidence ,Psychology ,Relevant information ,General Psychology ,Language ,Cognitive psychology ,media_common - Abstract
In the last two decades, empirical evidence has shown that personality traits could be related to the characteristics of written language. This study describes a meta-analysis that synthesizes 23 independent estimates of the correlations between the Big Five major personality traits, and some computationally obtained indicators from written language. The results show significant combined estimates of the correlations, albeit small to moderate according to Cohen's conventions to interpret effect sizes, for the five traits (between r = 0.26 for agreeableness and neuroticism, and 0.30 for openness). These estimates are moderated by the type of information in the texts, the use of prediction mechanisms, and the source of publication of the primary studies. Generally, the same effective moderators operate for the five traits. It is concluded that written language analyzed through computational methods could be used to extract relevant information of personality. But further research is still needed to consider it as predictive or explanatory tool for individual differences.
- Published
- 2021
- Full Text
- View/download PDF
34. Educación XX1 : revista de la Facultad de Educación
- Author
-
Juan Botella and Ángela Zamora
- Subjects
Statement (computer science) ,estado de la cuestión ,Point (typography) ,Computer science ,Educación ,05 social sciences ,050401 social sciences methods ,050301 education ,Context (language use) ,investigación educativa ,investigación teórica ,Data science ,evaluación ,Field (geography) ,Education ,Educational research ,0504 sociology ,Forest plot ,Objectivity (science) ,lcsh:L ,0503 education ,Research question ,lcsh:Education - Abstract
Se presenta la metodolog a del meta-an lisis como herramienta para la investigaci n educativa. Un meta-an lisis implica una s ntesis cuantitativa de la evidencia acumulada sobre una pregunta de investigaci n previa - mente de nida. La respuesta se basar en la informaci n contenida en los estudios previamente publicados (estudios primarios). Las principales ca - racter sticas del meta-an lisis son la precisi n, la objetividad y la replica - bilidad. Su aplicaci n permite obtener una estimaci n combinada del ta - ma o del efecto. Tambi n permite evaluar la heterogeneidad observada en un campo de estudio. Ello permite a su vez formular nuevas hip tesis que incorporan el papel de variables que no se hab an tenido en cuenta hasta el momento. Un meta-an lisis comienza con la formulaci n del problema. A continuaci n se lleva a cabo la localizaci n de estudios relacionados con ella. En la fase de codi caci n se caracterizan los trabajos recogidos en la fase anterior. Finalmente, un an lisis estad stico conduce a los resultados, que se presentar n en una publicaci n que asegure la replicabilidad del estudio. Entre las herramientas gr cas asociadas al meta-an lisis des - taca el llamado Forest Plot , que permite presentar de forma conjunta las estimaciones puntuales y de intervalo de los estudios seleccionados para el meta-an lisis. Aunque el potencial del meta-an lisis es evidente, tam - bi n presenta algunas debilidades derivadas de la falta de acotaci n de los constructos estudiados o de la existencia de un sesgo en la publicaci n de resultados, sobre todo de los estad sticamente signi cativos. La valoraci n general que se hace del papel del meta-an lisis en la investigaci n en edu - caci n es positiva aunque tambi n se se alan reas de mejora. Se indican algunos recursos, tanto bibliogr cos como de software , que permiten a los investigadores interesados adentrarse en esta herramienta tan til en el campo de la investigaci n educativa.
- Published
- 2017
35. Group analyses can hide heterogeneity effects when searching for a general model: Evidence based on a conflict monitoring task
- Author
-
Jesús Privado, Juan Botella, Roberto Colom, Manuel Suero, and James F. Juola
- Subjects
Adult ,Male ,Evidence-based practice ,Adolescent ,Experimental psychology ,Differential psychology ,Experimental and Cognitive Psychology ,Models, Psychological ,Neuropsychological Tests ,050105 experimental psychology ,Task (project management) ,Conflict, Psychological ,03 medical and health sciences ,Executive Function ,Young Adult ,0302 clinical medicine ,Cognition ,Arts and Humanities (miscellaneous) ,Developmental and Educational Psychology ,Reaction Time ,Humans ,0501 psychology and cognitive sciences ,Simon effect ,Psychological research ,05 social sciences ,Reproducibility of Results ,General Medicine ,Middle Aged ,Female ,Psychology ,Priming (psychology) ,030217 neurology & neurosurgery ,Cognitive psychology - Abstract
In experimental psychology, a unique model of general processing is often sought to represent the behaviors of all individuals. We address the question of whether seeking this objective - a unique model - is the most fruitful scientific strategy by studying a specific case example. In order to approach an answer to such a question, we compared the conventional approach in experimental psychology with analyses at the individual level by applying a specific mathematical modeling approach. A sample of 1159 individuals completed an experimental task based on managing conflict (a type of Simon task). Key findings revealed that at least four models are required to properly account for individuals' performance. Interestingly, four out of ten participants failed to show stimulus-response congruency effects in the experimental task, whereas the remaining 60% followed distinguishable theoretical models (consistent with conflict-monitoring theory and/or priming and episodic memory effects). The reported findings suggest that individuals' psychological characteristics might help to explain some of the reproducibility issues that are currently of great concern in psychology. These findings, along with further recent research, support the view that general and differential psychological approaches work better together for addressing relevant theoretical issues in psychological research.
- Published
- 2018
36. Analysis of Categorical Moderators in Mixed-effects Meta-analysis:Consequences of Using Pooled vs. Separate Estimates of the Residual Between-studies Variances
- Author
-
Juan Botella, María Rubio-Aparicio, Fulgencio Marín-Martínez, José A López-López, and Julio Sánchez-Meca
- Subjects
Statistics and Probability ,Mixed model ,Network Meta-Analysis ,Subgroup analysis ,Biostatistics ,Residual ,01 natural sciences ,010104 statistics & probability ,Arts and Humanities (miscellaneous) ,Statistical conclusion validity ,Statistics ,Econometrics ,Humans ,Psychology ,Computer Simulation ,0101 mathematics ,subgroup analysis ,between-studies variance ,Categorical variable ,General Psychology ,Mathematics ,Analysis of Variance ,Models, Statistical ,05 social sciences ,050301 education ,General Medicine ,Variance (accounting) ,Moderation ,meta-analysis ,Pooled variance ,Data Interpretation, Statistical ,0503 education ,mixed-effects model - Abstract
Subgroup analyses allow to examining the influence of a categorical moderator on the effect magnitude in meta-analysis. We conducted a simulation study using a dichotomous moderator, and compared the impact of pooled versus separate estimates of the residual between-studies variance on the statistical performance of the QB(P) and QB(S) tests for subgroup analyses assuming a mixed-effects model. Our results suggested that a similar performance can be expected as long as there are at least 20 studies and these are approximately balanced across categories. Conversely, when subgroups were unbalanced, the practical consequences of having heterogeneous residual between-studies variances were more evident, with both tests leading to the wrong statistical conclusion more often than in the conditions with balanced subgroups. A pooled estimate should be preferred for most scenarios, unless the residual between-studies variances are clearly different and there are enough studies in each category to get precise separate estimates.
- Published
- 2017
- Full Text
- View/download PDF
37. Self-assessment, self-regulated learning and self-efficacy : a meta-analysis
- Author
-
Jönsson, Anders, Panadero, Ernesto, Juan, Botella, Jönsson, Anders, Panadero, Ernesto, and Juan, Botella
- Abstract
Research about student self-assessment (SSA) has shown that academic performancetends to increase for students trained in SSA, but also that SSA training may increasethe use of self-regulated learning (SRL) strategies. Consequently, there is widespreadadvocacy for SSA, not least through the “assessment for learning” reform agenda. The educational gains from SSA are suggested to be related to the enhancement ofownership of learning and use of self-regulatory strategies. In addition, SSA has beensuggested to support students’ self-efficacy (SE). These relationships between SSA andSRL/SE, have been claimed theoretically, but without proper empirical support. Thisstudy therefore uses meta-analytic methodology to explore the relationships betweenSSA and SRL/SE. The 21 studies included in the analysis were found by a databasesearch and meta-analyses were conducted using a random-effect model. Inter-studyvariance was estimated by the maximum likelihood method. The findings from thisstudy confirms the theoretical connection between SSA and SRL/SE. By training inSSA, students’ use of self-regulating strategies for learning increase, while their use ofperformance/avoidance strategies decrease. SSA interventions also support students’ SE(.73), meaning that they have a more adequate perception of their own capabilities. Thefindings also confirm what has been noted in some of the individual studies on theeffects on SRL/SE from SSA interventions that girls tend to benefit more from suchinterventions, at least in relation to SE.
- Published
- 2018
38. Alternatives for Mixed-Effects Meta-Regression Models in the Reliability Generalization Approach
- Author
-
Fulgencio Marín-Martínez, Julio Sánchez-Meca, Juan Botella, and José A López-López
- Subjects
Mathematical optimization ,Cronbach's alpha ,Generalization ,Mixed effects ,Regression analysis ,Statistical analysis ,Meta-regression ,Moderation ,Social Sciences (miscellaneous) ,Reliability (statistics) ,Education ,Mathematics - Abstract
Since heterogeneity between reliability coefficients is usually found in reliability generalization studies, moderator analyses constitute a crucial step for that meta-analytic approach. In this study, different procedures for conducting mixed-effects meta-regression analyses were compared. Specifically, four transformation methods for the reliability coefficients, two estimators of the residual between-studies variance, and two methods for testing regression coefficients significance were combined in a Monte Carlo simulation study. The different methods were compared in terms of bias and mean square error (MSE) of the slope estimates, and Type I error and statistical power rates for the slope statistical tests. The results of the simulation study did not vary as a function of the residual variance estimator. All transformation methods provided negatively biased estimates, but both bias and MSE were reasonably small in all cases. In contrast, important differences were found regarding statistical tests, with the method proposed by Knapp and Hartung showing a better adjustment to the nominal significance level and higher power rates than the standard method.
- Published
- 2013
- Full Text
- View/download PDF
39. Our moral choices are foreign to us
- Author
-
Boaz Keysar, Juan Botella, Alice Foucart, Melina Aparici, Joanna D. Corey, Albert Costa, and Sayuri Hayakawa
- Subjects
Adult ,Male ,Linguistics and Language ,Bilingualism ,First language ,media_common.quotation_subject ,Foreign language ,050109 social psychology ,Experimental and Cognitive Psychology ,Morals ,Choice Behavior ,Vocabulary ,050105 experimental psychology ,Language and Linguistics ,Moral psychology ,Judgment ,Humans ,0501 psychology and cognitive sciences ,Interpersonal Relations ,media_common ,Moral disengagement ,Language ,05 social sciences ,16. Peace & justice ,Morality ,Social cognitive theory of morality ,Moral development ,Construal level theory ,Female ,Psychology ,Decision making ,Social psychology ,Decision-making ,Cognitive psychology - Abstract
Though moral intuitions and choices seem fundamental to our core being, there is surprising new evidence that people resolve moral dilemmas differently when they consider them in a foreign language (Cipolletti et al., 2016; Costa et al., 2014a; Geipel et al., 2015): People are more willing to sacrifice 1 person to save 5 when they use a foreign language compared with when they use their native tongue. Our findings show that the phenomenon is robust across various contexts and that multiple factors affect it, such as the severity of the negative consequences associated with saving the larger group. This has also allowed us to better describe the phenomenon and investigate potential explanations. Together, our results suggest that the foreign language effect is most likely attributable to an increase in psychological distance and a reduction in emotional response. This research was partially funded by grants from the Spanish Government (PSI2011-23033, Consolider Ingenio 2010 CSD2007-00048, and Ministry of Economy and Competitiveness PSI2014-52181-P), from the Catalan Government (SGR 2009-1521), from the European Research Council under the European Community's Seventh Framework (FP7/2007-2013 Cooperation grant agreement 613465-AThEME), from the John Templeton Foundation and the National Science Foundation #1520074 to the University of Chicago. Joanna D. Corey was supported by a grant from the Catalan Government (FI-DGR).
- Published
- 2017
40. Effects of self-assessment on self-regulated learning and self-efficacy: Four meta-analyses
- Author
-
Ernesto Panadero, Anders Jonsson, Juan Botella, UAM. Departamento de Psicología Evolutiva y de la Educación, and UAM. Departamento de Psicología Social y Metodología
- Subjects
Self-assessment ,Educación ,education ,Psychological intervention ,Self-regulated learning measurement ,Education ,Developmental psychology ,Self-regulated learning ,0502 economics and business ,Self-evaluation ,Self-efficacy ,Motivation ,05 social sciences ,050301 education ,Emotional regulation ,Gender educational differences ,Psicología ,Learning strategies ,Self evaluation ,Self-regulation ,Psychology ,0503 education ,050203 business & management - Abstract
This meta-analytic review explores the effects of self-assessment on students' selfregulated learning (SRL) and self-efficacy. A total of 19 studies were included in the four different meta-analyses conducted with a total sample of 2305 students. The effects sizes from the three meta-analyses addressing effects on different measures of SRL were 0.23, 0.65, and 0.43. The effect size from the meta-analysis on self-efficacy was 0.73. In addition, it was found that gender (with girls benefiting more) and certain self-assessment components (such as self-monitoring) were significant moderators of the effects on selfefficacy. These results point to the importance of self-assessment interventions to promote students’ use of learning strategies and its effects on motivational variables such as self-efficacy, First author funded by the Ministerio de Economía y Competitividad via Spanish Ramon y Cajal programme (Referencia RYC-2013-13469)
- Published
- 2017
41. Re-paying attention to visitor behavior: a re-analysis using meta-analytic techniques
- Author
-
Mikel Asensio, Yone Castro, Juan Botella, UAM. Departamento de Psicología Básica, and UAM. Departamento de Psicología Social y Metodología
- Subjects
Linguistics and Language ,Museology ,Walking ,01 natural sciences ,Language and Linguistics ,Field (computer science) ,Exhibition ,010104 statistics & probability ,0504 sociology ,Feature (machine learning) ,Humans ,0101 mathematics ,General Psychology ,Consumer behaviour ,museum studies ,Information retrieval ,visitor behavior ,Visitor pattern ,Museums ,05 social sciences ,050401 social sciences methods ,Consumer Behavior ,Psicología ,Preferred walking speed ,meta-analysis ,Meta-analysis ,Psychology ,Social psychology - Abstract
This article has been published in a revised form in The Spanish Journal of Psychology http://dx.doi.org/10.1017/sjp.2016.39. This version is free to view and download for private research and study only. Not for re-distribution, re-sale or use in derivative works, The present study describes a meta-analytic review of museum visitors' behavior. Although there is a large number of visitor studies available, their cumulative importance has not been determined due to the lack of rigorous methods to determine common causes of visitors' behaviors. We analyzed Serrell's (1998) database of 110 studies, defining a number of variables that measure visitors' behaviors in exhibition spaces which exceeded the most typical and obvious ones. We defined four indexes of effect size and obtained their combined estimates: average time per feature [ATF● = 0.43 (0.49; 0.37)], percentage of diligent visitors [dv● = 30% (0.39; 0.23)], inverse of velocity [Iv● = 4.07 min/100m2 (4.55; 3.59)], and stops per feature [SF● = 0.35 (0.38; 0.33)], and we analyzed the role of relevant moderating variables. Key findings indicate, for example, that the visiting time for each display element relates to the size of the exhibition and its newness, and visitor walking speed is higher in large exhibit areas. The indexes obtained in this study can be understood as references to be used for comparison with new evaluations. They may help to predict people's behavior and appreciation of new exhibitions, identifying important problems in museum designs, and providing new research tools for this field
- Published
- 2016
42. Managing Heterogeneity of Variance in Studies of Reliability Generalization With Alpha Coefficients
- Author
-
Manuel Suero and Juan Botella
- Subjects
Scale (ratio) ,Cronbach's alpha ,Generalization ,Statistics ,Econometrics ,Range (statistics) ,General Social Sciences ,Sampling (statistics) ,Variance (accounting) ,General Psychology ,Reliability (statistics) ,Mathematics ,Sampling bias - Abstract
In Reliability Generalization (RG) meta-analyses, the importance of bearing in mind the problems of range restriction or biased sampling and their influence on reliability estimation has often been highlighted. Nevertheless, the presence of heterogeneous variances in the included studies has been diagnosed in a subjective way and has not been taken into account in later analyses. Procedures to detect the presence of a variety of sampling schemes and to manage them in the analyses are proposed. The procedures are further explained with an example, by applying them to 25 estimates of Cronbach’s alpha coefficient in the Hamilton Scale for Depression.
- Published
- 2012
- Full Text
- View/download PDF
43. The types of stimuli loaded in memory can modulate its effects on visual search
- Author
-
Juan Botella, David Pascual-Ezama, and Beatriz Gil-Gómez de Liaño
- Subjects
Visual search ,Working memory ,Similarity (psychology) ,Experimental and Cognitive Psychology ,Context (language use) ,Visual short-term memory ,Iconic memory ,Memory load ,Psychology ,Cognitive psychology ,Task (project management) - Abstract
The effects of memory load in visual search (VS) have shown a diversity of results from the absence through beneficial and detrimental effects of a concurrent memory load in VS performance. One of the hypotheses intended to explain the heterogeneity of results follows the idea proposed by certain models in the context of VS that the contents of working memory (WM) can modulate the attentional processes involved in VS (Desimone & Duncan, 1995; Duncan & Humphreys, 1989). In four experiments, we manipulated the similarity of information maintained in WM and those materials playing the role of target and distractors in the VS task. The results showed a beneficial effect in the first two experiments, where the materials in WM matched the target in VS. However, when they matched the distractors in the attentional task there is no effect in the slope of the search function. Present results strengthen those theories supporting that visual working memory is fractionated to allow for maintenance of items not essent...
- Published
- 2011
- Full Text
- View/download PDF
44. Factors Associated with Emotional Well-being in Primary and Secondary Caregivers of Patients with Eating Disorders
- Author
-
Gonzalo Morandé, Dimitra Anastasiadou, Montserrat Graell, José Antonio Carrobles, Ana R. Sepúlveda, Enrique Berbel, and Juan Botella
- Subjects
medicine.medical_specialty ,Cross-sectional study ,medicine.disease ,Hospital Anxiety and Depression Scale ,Emotional well-being ,Psychiatry and Mental health ,Clinical Psychology ,Interpersonal relationship ,Eating disorders ,Quality of life (healthcare) ,medicine ,Anxiety ,medicine.symptom ,Psychiatry ,Psychology ,Depression (differential diagnoses) ,Clinical psychology - Abstract
The aim of this study was to examine and identify the psychological, clinical and sociodemographic factors associated with emotional well-being by type of caregiver. A cross-sectional study was conducted among 185 primary caregivers and 92 secondary caregivers of patients with an eating disorder using the Hospital Anxiety and Depression Scale, the Experience of Caregiving Inventory and the Eating Disorders Symptom Impact Scale. According to a multiple regression analysis, a total of four models were obtained accounting for 42%-47% of the variance in emotional well-being. The variable that accounted for most of the variance of emotional well-being in three of the models was the impact of nutrition. Improving aspects of the relationships with the patients reduced anxiety and depression levels in primary caregivers. Similarly, a positive personal experience reduced depression in secondary caregivers. A higher education level was associated with decreased anxiety levels in both types of caregivers. Specific family interventions including both types of caregivers may be useful for providing emotional and adaptive personal coping skills.
- Published
- 2011
- Full Text
- View/download PDF
45. Psychometric inferences from a meta-analysis of reliability and internal consistency coefficients
- Author
-
Manuel Suero, Hilda Gambara, and Juan Botella
- Subjects
Selection bias ,Models, Statistical ,Personality Inventory ,Psychometrics ,Generalization ,media_common.quotation_subject ,Reproducibility of Results ,Inference ,Moderation ,Meta-Analysis as Topic ,Research Design ,Meta-analysis ,Statistics ,Econometrics ,Range (statistics) ,Humans ,Psychology ,Psychology (miscellaneous) ,Set (psychology) ,Selection Bias ,media_common - Abstract
A meta-analysis of the reliability of the scores from a specific test, also called reliability generalization, allows the quantitative synthesis of its properties from a set of studies. It is usually assumed that part of the variation in the reliability coefficients is due to some unknown and implicit mechanism that restricts and biases the selection of participants in the studies' samples. Sometimes this variation has been reduced by adjusting the coefficients by a formula associated with range restrictions. We propose a framework in which that variation is included (instead of adjusted) in the models intended to explain the variability and in which parallel analyses of the studies' means and variances are performed. Furthermore, the analysis of the residuals enables inferences to be made about the nature of the variability accounted for by moderator variables. The meta-analysis of the 3 studies' statistics-reliability coefficient, mean, and variance--allows psychometric inferences about the test scores. A numerical example illustrates the proposed framework.
- Published
- 2010
- Full Text
- View/download PDF
46. Do Inferences about Age Differences in Emotional Experience Depend on the Parameters Analyzed?
- Author
-
Juan Botella, Rocío Fernández-Ballesteros, Virginia Fernández, Laura Cobo, and Giovanna Caprara
- Subjects
Age differences ,media_common.quotation_subject ,Happiness ,Emotional expression ,Positive psychology ,Psychology ,Self report ,Social Sciences (miscellaneous) ,media_common ,Balance (ability) ,Developmental psychology ,Quality of Life Research - Abstract
Age differences in emotional experience are assessed through self-report scales or questionnaires referring to a specific period of varying length, and examining different parameters of emotional response. A recent meta-analysis suggests that the type of instrument and parameter used could account for some of the inconsistencies in the results. The present study shows comparisons between emotional experience in samples of younger (N = 120, aged 20-27) and older (N = 103, aged 55-75) participants. An Emotional Self-Monitoring record was administered every day for a week. The results show that emotional expression was highly stable over time. However, they also show that some theoretical assumptions about individual age differences in emotional experience and age depend on the parameters on which the analysis is based (occurrence, frequency and intensity, and emotional balance).
- Published
- 2009
- Full Text
- View/download PDF
47. Priming and intrusion errors in RSVP streams with two response dimensions
- Author
-
Daniel Loach, Jesús Privado, Juan Botella, and John K. Tsotsos
- Subjects
medicine.medical_specialty ,genetic structures ,media_common.quotation_subject ,Experimental and Cognitive Psychology ,Stimulus (physiology) ,Audiology ,Intrusion ,Arts and Humanities (miscellaneous) ,Perception ,Reaction Time ,Developmental and Educational Psychology ,medicine ,Humans ,Attention ,Limited evidence ,media_common ,Communication ,business.industry ,Cognition ,General Medicine ,Rapid serial visual presentation ,Close relationship ,Visual Perception ,Psychology ,business ,Priming (psychology) - Abstract
Loach and Marí-Beffa (Vis Cogn, 10:513-526, 2003) observed that a distractor stimulus, presented immediately after a behaviorally relevant target stimulus, negatively primed a related probe stimulus indicating that the distractor had been inhibited. They argued that "post-target inhibition" may be a mechanism for preventing interference from temporally proximal stimuli; interference that could potentially result in a binding/intrusion error. In order to test this hypothesis, the authors carried out two rapid serial visual presentation (RSVP) experiments in which participants had to report either the identity (Experiment 1) or color (Experiment 2) of a target letter surrounded by distractor letters. In Experiment 1, a close relationship between priming and errors was observed. When a distractor stimulus showed evidence of being inhibited the participant was less likely to commit a binding error. The opposite was true when a distractor stimulus showed evidence of being facilitated. The results of Experiment 2 showed limited evidence of the same relationship.
- Published
- 2007
- Full Text
- View/download PDF
48. Optimization of sample size in controlled experiments: The CLAST rule
- Author
-
Javier Revuelta, Manuel Suero, Juan Botella, and Carmen Ximénez
- Subjects
Contingency table ,Sequential estimation ,Stopping rule ,Experimental and Cognitive Psychology ,Context (language use) ,Decision Support Techniques ,Test (assessment) ,Arts and Humanities (miscellaneous) ,Sample size determination ,Sample Size ,Statistics ,Sequential probability ratio test ,Developmental and Educational Psychology ,Humans ,Computer Simulation ,Psychology (miscellaneous) ,Psychology ,Algorithm ,General Psychology ,Behavioral Research ,Statistical hypothesis testing - Abstract
Sequential rules are explored in the context of null hypothesis significance testing. Several studies have demonstrated that the fixed-sample stopping rule, in which the sample size used by researchers is determined in advance, is less practical and less efficient than sequential stopping rules. It is proposed that a sequential stopping rule called CLAST (composite limited adaptive sequential test) is a superior variant of COAST (composite open adaptive sequential test), a sequential rule proposed by Frick (1998). Simulation studies are conducted to test the efficiency of the proposed rule in terms of sample size and power. Two statistical tests are used: the one-tailed t test of mean differences with two matched samples, and the chi-square independence test for twofold contingency tables. The results show that the CLAST rule is more efficient than the COAST rule and reflects more realistically the practice of experimental psychology researchers.
- Published
- 2006
- Full Text
- View/download PDF
49. Assessing heterogeneity in meta-analysis: Q statistic or I² index?
- Author
-
Tania B. Huedo-Medina, Juan Botella, Julio Sánchez-Meca, and Fulgencio Marín-Martínez
- Subjects
Index (economics) ,Meta-analysis ,Q-statistic ,Monte Carlo method ,Statistics ,Econometrics ,Nonparametric statistics ,Psychology (miscellaneous) ,Psychology ,Confidence interval ,Statistical hypothesis testing ,Complement (set theory) - Abstract
In meta-analysis, the usual way of assessing whether a set of single studies is homogeneous is by means of the Q test. However, the Q test only informs meta-analysts about the presence versus the absence of heterogeneity, but it does not report on the extent of such heterogeneity. Recently, the I(2) index has been proposed to quantify the degree of heterogeneity in a meta-analysis. In this article, the performances of the Q test and the confidence interval around the I(2) index are compared by means of a Monte Carlo simulation. The results show the utility of the I(2) index as a complement to the Q test, although it has the same problems of power with a small number of studies.
- Published
- 2006
- Full Text
- View/download PDF
50. El papel del Feedback y la experiencia en la apreciación del 'Fuera de Juego' en fútbol
- Author
-
Maite G\u00F3mez and Juan Botella
- Subjects
lcsh:Psychology ,lcsh:BF1-990 - Abstract
En la aplicación de la regla del "fuera de juego" en fútbol los árbitros y asistentes cometen con cierta frecuencia errores de apreciación. Recientemente Botella y Palacios (2002) han propuesto una explicación de estos errores basada en la psicología de la atención. Desde esta perspectiva la tarea se analiza como una tarea de Juicios de Orden Temporal, en la que el observador debe decidir acerca de cuál de dos eventos se produjo antes. Profundizamos en esta línea analizando los efectos de dos factores hasta ahora inexplorados: la experiencia del observador y el feedback después de cada ensayo. Para el estudio de la experiencia se seleccionaron dos grupos, uno de novatos (estudiantes) y otro de expertos (árbitros y árbitros asistentes de primera y segunda división). Para el estudio del feedback se empleó un diseño ABA de retirada en el que se analiza en la fase B el efecto de la introducción del feedback y en la segunda fase A la permanencia de estos efectos tras retirar el feedback. Los resultados muestran que en la primera fase la tasa de aciertos es igual en los dos grupos, pero la distribución de errores es diferente; mientras los novatos muestran una relación de 3:1 entre falsas alarmas y omisiones, en los expertos estos dos tipos de errores se distribuyen aproximadamente igual. Al introducir el feedback los novatos pasan a comportarse como los expertos, mientras que éstos permanecen estables. Al retirar el feedback el rendimiento de ambos grupos permanece como en la fase B. Este patrón de resultados se interpreta en el sentido de que en los novatos hay una especie de ángulo ciego que sesga el tipo de errores. La introducción del feedback hace visible este ángulo pero su retirada no afecta al rendimiento, que se mantiene como en la fase B.
- Published
- 2005
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.