Search

We examined how Learning Analytics literature represents participants from diverse societies by comparing the studies published with samples from WEIRD (Western, Industrialized, Rich, Democratic) nations versus non-WEIRD nations. By analyzing the Learning Analytics studies published during 2015-2019 (N = 360), we found that most of the studies were on WEIRD samples, with at least 58 percent of the total studies on WEIRD samples. Through keyword analysis, we found that the studies on WEIRD samples' research topics focused on self-regulated learning and feedback received in learning environments. The studies on non-WEIRD samples focused on the collaborative and social nature of learning. Our investigation of the analysis tools used for the studies suggested the limitations of some software in analyzing languages in diverse countries. Our analysis of theoretical frameworks revealed that most studies on both WEIRD and non-WEIRD samples did not identify a theoretical framework. The studies on WEIRD and non-WEIRD samples convey the similarities of Learning Analytics and Educational Data Mining. We conclude by discussing the importance of integrating multifaceted elements of the participant samples, including cultural values, societal values, and geographic areas, and present recommendations on ways to promote inclusion and diversity in Learning Analytics research.

A computer science camp for pre-collegiate students was operated during the summers of 2022 and 2023. The effect the camp had on attitudes was quantitatively assessed using a survey instrument. However, enrollment at the summer camp was small, which meant the well-known Pearson's Chi-Squared to measure the significance of results was not applied. Thus, a quantitative analysis method using a multinomial probability distribution as a model of a multilevel Likert scale survey was used. Exact calculations of a multinomial probability model with likelihood ratio were performed to quantitatively analyze the results of questionnaires administered to participants in two cohort groups (combined N=17). Probabilities per Likert categories were determined from the data itself using Bayes theorem with a Dirichlet prior. Each cohort functioned as part of a homogenous sample, thus allowing cohorts to be pooled. Post-test revealed significant changes in participants' attitudes after camp completion. Using this technique has implications for studies with small sample sizes. Using exact calculation of the multinomial probability model with the use of likelihood ratio as a statistical test of evidence has advantages: a) it is an exact value that can be used on any size sample, although it offers a quantitative analysis option for small sample size studies; b) depends only on what was observed during a study; c) does not require advanced calculation; d) modern spreadsheet and statistical package programs can calculate the analysis; and e) likelihood ratio employed in Bayes theorem can update prior beliefs according to evidence. Utilizing small sample size quantitative analysis can strengthen insights into data trends and showcase the importance of this quantitative technique.

This research aims to analyze the results of studies conducted in Turkey on reasoning skills in mathematics teaching and to reveal what kind of trend there is in this field. Within the scope of this study, databases were searched with the keywords "reasoning"(muhakeme, akil yürütme) and "reasoning skill" (Muhakeme becerisi, akil yürütme becerisi), and the results were examined in accordance with the inclusion criteria regarding mathematics teaching. One hundred sixty-three studies were included. Each of the studies included in the meta-synthesis study was analyzed descriptively according to type, year, method, sample type and size, data collection tools, statistical analysis, learning field, keywords, reasoning type and purpose. In addition, the studies' results were content analyzed and tabulated by coding the differences and similarities between them with a holistic approach. It has been determined that studies on the learning fields in the mathematics curriculum are mostly carried out in the field of learning numbers and operations in the secondary school mathematics curriculum. When evaluated in terms of reasoning types, almost half of the studies were conducted on mathematical reasoning. It has been observed that after mathematical reasoning, the most focus is on proportional reasoning. When the aims of the studies included in the research were examined, it was determined that the most focused ones were "examining the factors affecting reasoning skills", "measurement of reasoning skills" and "the effect of teaching practices on reasoning skills". In the studies examined, it was seen that there were 33 teaching practices whose effects on reasoning skills were examined.

Objective: To determine if treatment of co-occurring adult ADHD and Cannabis Use Disorder (CUD) with extended-release mixed amphetamine salts (MAS-ER) would be effective at improving ADHD symptoms and promoting abstinence. Method: A 12-week randomized, double-blind, two-arm pilot feasibility trial of adults with comorbid ADHD and CUD (n = 28) comparing MAS-ER (80 mg) to placebo. Main outcomes: ADHD: [greater than or equal to] 30% symptom reduction, measured by the Adult ADHD Investigator Symptom Rating Scale (AISRS). CUD: Abstinence during last 2 observed weeks of maintenance phase. Results: Overall, medication was well-tolerated. There was no significant difference in ADHD symptom reduction (MAS-ER: 83.3%; placebo: 71.4%; p = 0.65) or cannabis abstinence (MAS-ER: 15.4%; placebo: 0%; p = 0.27). MAS-ER group showed a significant decrease in weekly cannabis use days over time compared to placebo (p < 0.0001). Conclusions: MAS-ER was generally well-tolerated. The small sample size precluded a determination of MAS-ER's superiority reducing ADHD symptoms or promoting abstinence. Notably, MAS-ER significantly reduced weekly days of use over time.

Cluster randomized trials (CRTs) are commonly used to evaluate the causal effects of educational interventions, where the entire clusters (e.g., schools) are randomly assigned to treatment or control conditions. This study introduces statistical methods for designing and analyzing two-level (e.g., students nested within schools) and three-level (e.g., students nested within classrooms nested within schools) CRTs. Specifically, we utilize hierarchical linear models (HLMs) to account for the dependency of the intervention participants within the same clusters, estimating the average treatment effects (ATEs) of educational interventions and other effects of interest (e.g., moderator and mediator effects). We demonstrate methods and tools for sample size planning and statistical power analysis. Additionally, we discuss common challenges and potential solutions in the design and analysis phases, including the effects of omitting one level of clustering, non-compliance, heterogeneous variance, blocking, threats to external validity, and cost-effectiveness of the intervention. We conclude with some practical suggestions for CRT design and analysis, along with recommendations for further readings.

The network scale-up method (NSUM) is a survey-based method for estimating the number of individuals in a hidden or hard-to-reach subgroup of a general population. In NSUM surveys, sampled individuals report how many others they know in the subpopulation of interest (e.g. "How many sex workers do you know?") and how many others they know in subpopulations of the general population (e.g. "How many bus drivers do you know?"). NSUM is widely used to estimate the size of important sociological and epidemiological risk groups, including men who have sex with men, sex workers, HIV+ individuals, and drug users. Unlike several other methods for population size estimation, NSUM requires only a single random sample and the estimator has a conveniently simple form. Despite its popularity, there are no published guidelines for the minimum sample size calculation to achieve a desired statistical precision. Here, we provide a sample size formula that can be employed in any NSUM survey. We show analytically and by simulation that the sample size controls error at the nominal rate and is robust to some forms of network model mis-specification. We apply this methodology to study the minimum sample size and relative error properties of several published NSUM surveys.

Since the learning disability (LD) population comprises the largest group receiving special education services, there is a need for research to examine the self-concept of this population at a global level. This systematic literature review synthesized 20 years of quantitative research (k = 16) about the self-concept of students with LD. The overarching theme was that the diagnosis of LD relies on divergent criteria among the studies reviewed. While the academic self-concept was the center of the research, regardless of its domains, results indicated that students with LD had a lower level of self-concept. To have a deeper understanding of the phenomenon, an instrument specifically designed for assessing self-concept of students with LD is needed. Limitations of the study and implications for research and practice are discussed.

Marginal maximum likelihood, a common estimation method for item response theory models, is not inherently a Bayesian procedure. However, due to estimation difficulties, Bayesian priors are often applied to the likelihood when estimating 3PL models, especially with small samples. Little focus has been placed on choosing the priors for marginal maximum estimation. In this study, using sample sizes of 1,000 or smaller, not using priors often led to extreme, implausible parameter estimates. Applying prior distributions to the c-parameters alleviated the estimation problems with samples of 500 or more; for the samples of 100, priors on both the a-parameters and c-parameters were needed. Estimates were biased when the mode of the prior did not match the true parameter value, but the degree of the bias did not depend on the strength of the prior unless it was extremely informative. The root mean squared error (RMSE) of the a-parameters and b-parameters did not depend greatly on either the mode or the strength of the prior unless it was extremely informative. The RMSE of the c-parameters, like the bias, depended on the mode of the prior for c.

Contribution: This longitudinal study modeled student leadership growth in a course sequence supporting long-term, large-scale, multidisciplinary projects embedded in faculty research. Students (half from computer science, computational media, electrical engineering, and computer engineering) participated for 1-4 semesters. Background: Project- based learning (PBL) is used widely in higher education. It is used in industry for leadership development, but leadership development in project-based learning (PBL) has not been explored in higher education. A preliminary analysis implied leadership growth through the third semester of participation, but the design did not control for attrition. Research Questions: At the student level, how do leadership role ratings change over multiple semesters of participation? Do first (and second) semester ratings differ by number of semesters students eventually participate? Methodology: The study involved two peer evaluation questions on 1) the degree to which students coordinated the team's work and 2) served as technical/content area leaders. Analysis employed analysis of variance to examine attrition by initial ratings (N = 1045) and multilevel growth modeling to study change over time (N = 585). A strength of using peer evaluations is the large sample size, but a weakness is that the tool was developed for student assessment and not educational research. The study did not control for participation in leadership programs outside the course. Findings: On average, individual leadership role ratings increased each semester through the third semester of participation. Ratings of students who left the program after 1 or 2 semesters did not differ from ratings for those who participated longer.

Cognitive diagnosis models (CDMs) are the assessment tools that provide valuable formative feedback about skill mastery at both the individual and population level. Recent work has explored the performance of CDMs with small sample sizes but has focused solely on the estimates of individual profiles. The current research focuses on obtaining accurate estimates of skill mastery at the population level. We introduce a novel algorithm (bagging algorithm for deterministic inputs noisy "and" gate) that is inspired by ensemble learning methods in the machine learning literature and produces more stable and accurate estimates of the population skill mastery profile distribution for small sample sizes. Using both simulated data and real data from the Examination for the Certificate of Proficiency in English, we demonstrate that the proposed method outperforms other methods on several metrics in a wide variety of scenarios.

Short scales are time-efficient for participants and cost-effective in research. However, researchers often mistakenly expect short scales to have the same reliability as long ones without considering the effect of scale length. We argue that applying a universal benchmark for alpha is problematic as the impact of low-quality items is greater on shorter scales. In this study, we proposed simple guidelines for item reduction using the "alpha-if-item-deleted" procedure in scale construction. An item can be removed if alpha increases or decreases by less than 0.02, especially for short scales. Conversely, an item should be retained if alpha decreases by more than 0.04 upon its removal. For reliability benchmarks, 0.80 is relatively safe in most conditions, but higher benchmarks are recommended for longer scales and smaller sample sizes. Supplementary analyses, including item content, face validity, and content coverage, are critical to ensure scale quality.

A Monte Carlo simulation study was conducted to compare fit indices used for detecting the correct latent class in three dichotomous mixture item response theory (IRT) models. Ten indices were considered: Akaike's information criterion (AIC), the corrected AIC (AICc), Bayesian information criterion (BIC), consistent AIC (CAIC), Draper's information criterion (DIC), sample size adjusted BIC (SABIC), relative entropy, the integrated classification likelihood criterion (ICL-BIC), the adjusted Lo-Mendell-Rubin (LMR), and Vuong-Lo-Mendell-Rubin (VLMR). The accuracy of the fit indices was assessed for correct detection of the number of latent classes for different simulation conditions including sample size (2,500 and 5,000), test length (15, 30, and 45), mixture proportions (equal and unequal), number of latent classes (2, 3, and 4), and latent class separation (no-separation and small separation). Simulation study results indicated that as the number of examinees or number of items increased, correct identification rates also increased for most of the indices. Correct identification rates by the different fit indices, however, decreased as the number of estimated latent classes or parameters (i.e., model complexity) increased. Results were good for BIC, CAIC, DIC, SABIC, ICL-BIC, LMR, and VLMR, and the relative entropy index tended to select correct models most of the time. Consistent with previous studies, AIC and AICc showed poor performance. Most of these indices had limited utility for three-class and four-class mixture 3PL model conditions.

Although parallel analysis has been found to be an accurate method for determining the number of factors in many conditions with complete data, its application under missing data is limited. The existing literature recommends that, after using an appropriate multiple imputation method, researchers either apply parallel analysis to every imputed data set and use the number of factors suggested by most of the data copies or average the correlation matrices across all data copies, followed by applying the parallel analysis to the average correlation matrix. Both approaches for pooling the results provide a single suggested number without reflecting the uncertainty introduced by missing values. The present study proposes the use of an alternative approach, which calculates the proportion of imputed data sets that result in k (k = 1, 2, 3 . . .) factors. This approach will inform applied researchers of the degree of uncertainty due to the missingness. Results from a simulation experiment show that the proposed method can more likely suggest the correct number of factors when missingness contributes to a large amount of uncertainty.

Differentiated instruction (DI) is a beneficial approach to addressing students' diverse learning needs, abilities, and interests to ensure that each student has the opportunity to make academic progress. To answer the question of how teachers utilize DI in K-12 classrooms, this systematic review was based on 61 empirical studies on DI published between 2000 and 2022. It examined the current status and trends of implementing DI in K-12 education and integrated various factors involved in the process of DI, including educational levels, subjects, student difference analysis, instructional methods, content, tools, assessment methods, and instructional effectiveness. The findings indicated that (1) DI was most commonly used in primary school mathematics and language classrooms, with the majority of studies having sample sizes exceeding 100 and lasting for more than 6 months; (2) The most frequently employed form of DI was ability grouping, often grouped based on academic achievement; (3) Information technology tools and resources can empower differentiated instruction; (4) Most studies utilized standardized tests, questionnaires, and scales as evaluation tools, with a focus on the impact of DI on students' academic achievement and skills; and (5) The effectiveness of DI was controversial and influenced by multiple factors, such as may be associated with the instructional methods. In response to these findings, the study introduces a comprehensive DI model. This model, rooted in the perspective of instructional design, elucidates the interconnected factors of DI. It serves as a valuable reference for the future design and implementation of DI, offering a practical guide for educators aiming to create inclusive and effective learning environments.

Research on working memory (WM) has followed two largely independent traditions: One concerned with memory for sequentially presented lists of discrete items, and the other with short-term maintenance of simultaneously presented arrays of objects with simple, continuously varying features. Here we present a formal model of WM, the interference model (IM), that explains benchmark findings from both traditions: The shape of the error distribution from continuous reproduction of visual features, and how it is affected by memory set size; the effects of serial position for sequentially presented items, the effect of output position, and the intrusion of nontargets as a function of their distance from the target in space and in time. We apply the model to two experiments combining features of popular paradigms from both traditions: Lists of colors (Experiment 1) or of nonwords (Experiment 2) are presented sequentially and tested through selection of the target from a set of candidates, ordered by their similarity. The core assumptions of the IM are: Contents are encoded into WM through temporary bindings to contexts that serve as retrieval cues to access the contents. Bindings have limited precision on the context and the content dimension. A subset of the memory set--usually one item and its context--is maintained in a focus of attention with high precision. Successive events in an episode are encoded with decreasing strength, generating a primacy gradient. With each encoded event, automatic updating of WM reduces the strength of preceding memories, creating a recency gradient and output interference.

Recent reviews of behavior analytic journals suggest that participant demographics are inadequately described. These reviews have been limited to brief periods across several journals, emphasized specific variables (e.g., socioeconomic status), or only included specific populations. The current scoping review included all published articles in "The Analysis of Verbal Behavior" from 1982-2020. Six demographic variables were coded for 1888 participants across 226 articles. Despite small sample sizes (i.e., fewer than six participants in 62.3% of studies), only age (85.4%) and gender identity (71.6%) were reported for the majority of participants. Socioeconomic status, race/ethnicity, and primary language were reported for fewer than 20% of participants. Over time, the number of demographic variables reported showed a slight increasing trend, although considerable variability was observed across years. These findings suggest that editors and reviewers must consider what constitutes acceptable participant characterization. Researchers might also be emboldened to extend their work to populations currently underrepresented in the journal.

The Doctor of Education (EdD) degree is vital in preparing individuals for leadership positions in educational settings. A key element of the EdD program is the faculty's proficiency in their specific fields of expertise. Assessing the faculty's mastery of subject matter is crucial for guaranteeing the quality of education and offering students a comprehensive and fulfilling learning experience. This case analysis examined the evaluation of EdD faculty's mastery of subject matter from the student's standpoint. In conclusion, students' ratings of their EdD professors' depth of knowledge and expertise varied, with some praising their professors for exceptional expertise and others expressing mixed opinions or dissatisfaction. However, it is important to consider that these ratings are subjective and may not provide a complete assessment of a professor's expertise. Factors such as sample size and specific context can influence students' perceptions. A more comprehensive evaluation that includes multiple students' opinions, objective measures of expertise, and overall learning outcomes would offer a more accurate assessment.

As the theme of the 2022 annual meeting of the American Education Research Association, cultivating equitable education systems has gained renewed attention amid an increasingly diverse society. However, systemic inequalities persist for traditionally underserved student populations. As a way to better address diverse students' needs, it is of critical importance to understand different subgroups' performances. In the educational measurement field, evaluating the differences among multiple groups is an important consideration in addressing fairness issues for diverse groups of students. This article offers one technique to do so. It demonstrates how commonly-used multiple regression analysis can be applied to evaluate the equivalence of predictive structure across multiple groups in place of the factor analytic approach that requires a relatively large sample size per subgroup and strong assumptions. The technique is utilized in examining the relationship between English language proficiency and academic performance of English learners in one state when the subgroups are categorized by home language. The results showed statistically significant group differences between the reference group (Spanish-speaking ELs) and other focal groups (different home-language ELs) in various levels of comparisons (model fit, model structure, and individual predictor weights). The strengths and limitations of a proposed multiple group regression (MGR) approach are discussed in the educational research context.

We demonstrate that all meta-analyses of partial correlations are biased, and yet hundreds of meta-analyses of partial correlation coefficients (PCCs) are conducted each year widely across economics, business, education, psychology, and medical research. To address these biases, we offer a new weighted average, UWLS[subscript +3]. UWLS[subscript +3] is the unrestricted weighted least squares weighted average that makes an adjustment to the degrees of freedom that are used to calculate partial correlations and, by doing so, renders trivial any remaining meta-analysis bias. Our simulations also reveal that these meta-analysis biases are small-sample biases (n < 200), and a simple correction factor of (n - 2)/(n - 1) greatly reduces these small-sample biases along with Fisher's z. In many applications where primary studies typically have hundreds or more observations, partial correlations can be meta-analyzed in standard ways with only negligible bias. However, in other fields in the social and the medical sciences that are dominated by small samples, these meta-analysis biases are easily avoidable by our proposed methods.

With the study reported on here we aimed to synthesise recent quantitative research to specify the effect of inclusive practices on attitudes via meta-analysis. Since attitudes have an integral role in the performance of inclusion programmes, within the scope of the study, the cumulative findings of experimental studies conducted on attitudes towards inclusive practices were reinterpreted. To this end, studies with a pretest-posttest control group carried out in 2000 and 2021 were scanned from databases according to the inclusion criteria. After the search process, 23 studies that met the inclusion criteria were selected from 54 studies. The overall sample size of the studies included 2,016 participants. The mean effect size calculations, heterogeneity test, moderator analyses and publication bias analyses were conducted through a comprehensive meta-analysis programme (CMA 3.0). The findings that were discussed in accordance with the random effects model (REM) suggest that inclusive practices have a positive effect on attitudes and this effect is at a large level (g = 1.328) with respect to Cohen's classification. This result indicates that inclusive practices have been strongly influenced by positive attitudes to yield favourable results. According to the moderator analyses, the highest effect sizes were found in the teachers' group (g = 1,880) according to group level and in primary education (g = 1,374) according to school grades. The attitudes towards inclusion have been strongly influenced by teachers' beliefs about the power of their teaching. More empirical studies on inclusive practices are recommended.

Randomized A/B tests within online learning platforms represent an exciting direction in learning sciences. With minimal assumptions, they allow causal effect estimation without confounding bias and exact statistical inference even in small samples. However, often experimental samples and/or treatment effects are small, A/B tests are underpowered, and effect estimates are overly imprecise. Recent methodological advances have shown that power and statistical precision can be substantially boosted by coupling design-based causal estimation to machine-learning models of rich log data from historical users who were not in the experiment. Estimates using these techniques remain unbiased and inference remains exact without any additional assumptions. This paper reviews those methods and applies them to a new dataset including over 250 randomized A/B comparisons conducted within ASSISTments, an online learning platform. We compare results across experiments using four novel deep-learning models of auxiliary data and show that incorporating auxiliary data into causal estimates is roughly equivalent to increasing the sample size by 20% on average, or as much as 50-80% in some cases, relative to t-tests, and by about 10% on average, or as much as 30-50%, compared to cutting-edge machine learning unbiased estimates that use only data from the experiments. We show that the gains can be even larger for estimating subgroup effects, hold even when the remnant is unrepresentative of the A/B test sample, and extend to post-stratification population effects estimators.

Likert-type item is the most popular response format for collecting data in social, educational, and psychological studies through scales or questionnaires. However, there is no consensus on whether parametric or non-parametric tests should be preferred when analyzing Likert-type data. This study examined the statistical power of parametric and non-parametric tests when each Likert-type item was analyzed independently in survey studies. The main purpose of the study is to examine the statistical power of Wilcoxon-Mann-Whitney, Welch's t, and Student's t tests for Likert-type data, which are pairwise comparison tests. For this purpose, a Monte Carlo simulation study was conducted. The statistical significance of the selected tests was examined under the conditions of sample size, group size ratio, and effect size. The results showed that the Wilcoxon-Mann-Whitney test was superior to its counterparts, especially for small samples and unequal group sizes. However, the Student's t-test for Likert-type data had similar statistical power to the Wilcoxon-Mann-Whitney test under conditions of equal group sizes when the sample size was 200 or more. Consistent with the empirical results, practical recommendations were provided for researchers on what to consider when collecting and analyzing Likert-type data.

To ensure the validity of the tests is to check that all items have similar results across different groups of individuals. However, differential item functioning (DIF) occurs when the results of individuals with equal ability levels from different groups differ from each other on the same test item. Based on Item Response Theory and Classic Test Theory, there are some methods, with different advantages and limitations to identify items that show DIF. This study aims to compare the performances of five methods for detecting DIF. The efficacies of Mantel-Haenszel (MH), Logistic Regression (LR), Crossing simultaneous item bias test (CSIBTEST), Lord's chi-square (LORD), and Raju's area measure (RAJU) methods are examined considering conditions of the sample size, DIF ratio, and test length. In this study, to compare the detection methods, power and Type I error rates are evaluated using a simulation study with 100 replications conducted for each condition. Results show that LR and MH have the lowest Type I error and the highest power rate in detecting uniform DIF. In addition, CSIBTEST has a similar power rate to MH and LR. Under DIF conditions, sample size, DIF ratio, test length and their interactions affect Type I error and power rates.

The after-school program is a crucial initiative for implementing the Double Reduction policy; however, prior research has not provided conclusive evidence on whether extended school hours contribute to students' cognitive and non-cognitive development or on which types of after-school services are more beneficial for student development. This study analyzed 37 after-school programs from 18 publications using meta-analytic techniques, and the results indicated that participation in after-school programs had positive effects on student cognitive and non-cognitive development despite the small effect size (d = 0.327, p = 0.000). The decomposition of the effects of after-school programs revealed that they had modestly positive effects on academic achievement (d = 0.369) and social-emotional competence (d = 0.220). In addition, the analysis of moderating variables revealed that socioeconomic status, educational phase, number of after-school service days per week, sample size, and testing instrument all influenced the after-school program effects. This study concludes, based on the results of the meta-analysis, that there should be a balanced consideration of the development of student cognitive and non-cognitive abilities in planning after-school service, a substantial variety of activities in afterschool programs, a flexible adoption of diverse after-school programs, and a reasonable participation frequency in after-school service.

In the context of Artificial Intelligence, Generative Adversarial Nets (GANs) allow the creation and reproduction of artificial data from real datasets. The aims of this work are to seek to verify the equivalence of synthetic data with real data and to verify the possibilities of GAN in educational research. The research methodology begins with the creation of a survey that collects data related to the self-perceptions of university teachers regarding their digital competence and technological-pedagogical knowledge of the content (TPACK model). Once the original dataset is generated, twenty-nine different synthetic samples are created (with an increasing N) using the COPULA-GAN procedure. Finally, a two-stage cluster analysis is applied to verify the interchangeability of the synthetic samples with the original, in addition to extracting descriptive data of the distribution characteristics, thereby checking the similarity of the qualitative results. In the results, qualitatively very similar cluster structures have been obtained in the 150 tests carried out, with a clear tendency to identify three types of teaching profiles, based on their level of technical-pedagogical knowledge of the content. It is concluded that the use of synthetic samples is an interesting way of improving data quality, both for security and anonymization and for increasing sample sizes.

The aim of the study was to examine postgraduate theses on epistemological beliefs in science education in Türkiye. Data collected included the publication years, researcher genders, universities, disciplines, aims, methods, sample/study groups, time allocated to the research, and data collection tools. The thematic content analysis method was used in the study. The data were obtained from the doctoral and master's thesis published until 2022 (including 2022) inclusive held at the CoHE National Thesis Centre. Access was gained to 149 theses dealing with the subject of epistemological beliefs in science education. The theses in the study were classified with reference to the matrix prepared by Ormanci, Çepni, Deveci and Aydin. The data obtained were analysed using content and descriptive analysis methods. The majority of the theses aimed to investigate the effect of a certain learning-teaching method on epistemological beliefs and the relationship between epistemological beliefs and some variables. It was determined that scales and questionnaires were mostly used as data collection tools in the evaluation of epistemological beliefs. There is a need for studies on the effect of current science learning-teaching methods on the development of epistemological beliefs or the relationship between epistemological beliefs and 21st-century skills.

In this study, it was aimed to examine the effect of missing data in different patterns and sizes on test equating methods under the NEAT design for different factors. For this purpose, as part of this study, factors such as sample size, average difficulty level difference between the test forms, difference between the ability distribution, missing data rate, and missing data mechanisms were manipulated. The effects of these factors on the equating error of test equating methods (chained-equipercentile equating, Tucker, frequency estimation equating, and Braun-Holland) were investigated. In the study, two separate sets of 10,000 dichotomous data were generated consistent with a 2-parameter logistic model. While generating data, the MCAR and MAR missing data mechanisms were used. All analyses were conducted by R 4.2.2. As a result of the study, it was seen that the RMSE of the equating methods increased significantly as the missing data rate increased. The results indicate that the RMSE of the equating methods with imputed missing data are reduced compared to equating without imputed missing data. Furthermore, the percentage of missing data, along with the difference between ability levels and the average difficulty difference between forms, was found to significantly affect equating errors in the presence of missing data. Although increasing sample size did not have a significant effect on equating error in the presence of missing data, it did lead to more accurate equating when there was no missing data present.

Among all, Cronbach's Alpha and McDonald's Omega are commonly used for reliability estimations. The alpha uses inter-item correlations while omega is based on a factor analysis result. This study uses simulated ordinal data sets to test whether the alpha and omega produce different estimates. Their performances were compared according to the sample size, number of items, and deviance from tau equivalence. Based on the result, the alpha and omega had similar results, except for the small sample size, the smaller number of items, and the low factor loading values. When there were 5 or more items in the scale and factor analysis which the omega was calculated from showed fit to the data set, using omega over alpha could be preferred. Also, as the number of items exceeds 5, the alpha and omega differences disappear. Since calculating the alpha is easier compared to the omega (omega requires fitting a factor model first) using alpha over omega can also be suggested. However, when the number of items and the correlations among the items were small, omega performed worse than alpha. Therefore, alpha should be used for the reliability estimations.

The main purpose of this study is to examine the Type I error and statistical power ratios of Differential Item Functioning (DIF) techniques based on different theories under different conditions. For this purpose, a simulation study was conducted by using Mantel-Haenszel (MH), Logistic Regression (LR), Lord's [chi-squared], and Raju's Areas Measures techniques. In the simulation-based research model, the two-parameter item response model, group's ability distribution, and DIF type were the fixed conditions while sample size (1800, 3000), rates of sample size (0.50, 1), test length (20, 80) and DIF-containing item rate (0, 0.05, 0.10) were manipulated conditions. The total number of conditions is 24 (2x2x2x3), and statistical analysis was performed in the R software. The current study found that the Type I error rates in all conditions were higher than the nominal error level. It was also demonstrated that MH had the highest error rate while Raju's Areas Measures had the lowest error rate. Also, MH produced the highest statistical power rates. The analysis of the findings of Type 1 error and statistical power rates illustrated that techniques based on both of the theories performed better in the 1800 sample size. Furthermore, the increase in the sample size affected techniques based on CTT rather than IRT. Also, the findings demonstrated that the techniques' Type 1 error rates were lower while their statistical power rates were higher under conditions where the test length was 80, and the sample sizes were not equal.

It is desirable for an educational assessment to be constructed of items that can differentiate different performance levels of test takers, and thus it is important to estimate accurately the item discrimination parameters in either classical test theory or item response theory. It is particularly challenging to do so when the sample sizes are small. The current study reexamined the relationship between the biserial correlation coefficient and the discrimination parameter to investigate whether the biserial correlation coefficient estimator could be modified and whether biserial-based estimators could be used as alternate estimates of the item discrimination indices. Results show that the modified and alternative approaches work slightly better under certain circumstances (e.g., for small sample sizes or shorter tests), assuming normality of the latent ability distribution. Applications of these alternative estimators are presented in item scaling and weighted differential item functioning analyses. Recommendations and limitations are discussed for practical use of these proposed methods.

The aim of the study was to examine the common items in the mixed format (e.g., multiple-choices and essay items) contain parameter drifts in the test equating processes performed with the common item nonequivalent groups design. In this study, which was carried out using Monte Carlo simulation with a fully crossed design, the factors of test length (30 and 50), sample size (1000 and 3000), common item ratio (30 and 40%), ratio of items with item parameter drift (IPD) in common items (20 and 30%), location of common items in tests (at the beginning, randomly distributed, and at the end) and IPD size in multiple-choice items (low [0.2] and high [1.0]) were studied. Four test forms were created, and two test forms do not contain parameter drifts. After the parameter drift was performed on the first of the other two test forms, the parameter drift was again performed on the second test form. Test equating results were compared using the root mean squared error (RMSE) value. As a result of the research, ratio of items with IPD in common items, IPD size in multiple-choice items, common item ratio, sample size and test length on equating errors were found to be significant.

In statistics classes, the central limit theorem has been demonstrated using simulation-based illustrations. Known population distributions such as a uniform or exponential distribution are often used to consider the behavior of the sample mean in simulated samples. Unlike such simulations, a number of real-data-based simulations are here implemented in which the populations are empirical distributions of data selected from Japanese firms. The dataset chosen contains 38 variables familiar to business students, such as sales and assets. The maximum population size of the variables is 2243. One thousand samples with replacement are selected for specific variable--sample size combinations. Hypothesis testing results indicate that the normality hypothesis for the sample mean is rejected for 31 variables at the 0.1% level even with a sample size of 500. It is emphasized that the data for these variables indicate that this should not be a surprise, and emphasize the importance of looking at data.

Multisite cluster randomized trials (MCRTs), in which, the intermediate-level clusters (e.g., classrooms) are randomly assigned to the treatment or control condition within each site (e.g., school), are among the most commonly used experimental designs across a broad range of disciplines. MCRTs often align with the theory that programs are delivered at a cluster-level (e.g., teacher professional development) and provide opportunities to explore treatment effect heterogeneity across sites. In designing experimental studies, a critical step is the statistical power analysis and sample size determination. However, the statistical tools for power analysis of moderator effects in three-level MCRTs are not available. In this study, we derived formulas for calculating the statistical power and the minimum detectable effect size difference (MDESD) with confidence intervals for investigating the effects of various moderators in three-level MCRTs. We considered the levels of the moderators (level-1, -2, and -3), the scales of the moderators (binary and continuous), and random and nonrandomly varying slopes of the (moderated) treatment effects. We validated our formulas through Monte Carlo simulations. Finally, we conclude with directions for future work.

Purpose: The lack of a definite standard for determining the sample size in qualitative research leaves the research process to the initiative of the researcher, and this situation overshadows the scientificity of the research. The primary purpose of this research is to propose a model by questioning the problem of determining the sample size, which is one of the essential issues in qualitative research. The fuzzy logic model is proposed to determine the sample size in qualitative research. Design/methodology/approach: Considering the structure of the problem in the present study, the proposed fuzzy logic model will benefit and contribute to the literature and practical applications. In this context, ten variables, namely scope of research, data quality, participant genuineness, duration of the interview, number of interviews, homogeneity, information strength, drilling ability, triangulation and research design, are used as inputs. A total of 20 different scenarios were created to demonstrate the applicability of the model proposed in the research and how the model works. Findings: The authors reflected the results of each scenario in the table and showed the values for the sample size in qualitative studies in Table 4. The research results show that the proposed model's results are of a quality that will support the literature. The research findings show that it is possible to develop a model using the laws of fuzzy logic to determine the sample size in qualitative research. Originality/value: The model developed in this research can contribute to the literature, and in any case, it can be argued that determining the sample volume is a much more effective and functional model than leaving it to the initiative of the researcher.

Estimating confidence intervals (CIs) for the correlation has been a challenge because the correlation sampling distribution changes depending on the correlation magnitude. The Fisher z-transformation was one of the first attempts at estimating correlation CIs but has historically shown to not have acceptable coverage probability if data were non-normal. Bootstrap methods for correlation CIs were then proposed but have been met with mixed results. A recent study consolidating some of the previous literature found that the bootstrap correlation CIs had acceptable coverage probability for non-normal data if sample size was large enough. Additionally, it was found that pairing a normal distribution with any other distribution resulted in better coverage probability for all correlation CIs investigated. Notably, the Fisher z-transformation CI had consistent acceptable coverage probability in this case. Therefore, the purpose of the current study was to investigate if this normality effect extends to other distributions that are symmetric. It was found that the Fisher z-transformation, percentile bootstrap (PB), and bias-corrected and accelerated (BCa) CIs all benefited from having data that came from a symmetric distribution. However, the PB and BCa CIs did not have acceptable coverage probability when a uniform distribution was paired with a chi-square (df = 1).

Simple matrix sampling planned missing (SMS PD) design, introduce missing data patterns that lead to covariances between variables that are not jointly observed, and create difficulties for analyses other than mean and variance estimations. Based on prior research, we adopted a new multigroup confirmatory factor analysis (CFA) approach to handle missing data in such designs, in comparison to a regular CFA with full information maximum likelihood estimator. In Study 1, we tested the two approaches in 36 scenarios (4 sample sizes x 3 inter-item correlations x 3 numbers of x-set items) given a total of 20 items. We found that, the multigroup CFA approach performed with acceptable convergence rates, power to recover population values, acceptable standard errors and model fit in certain scenarios by larger sample size, higher bivariate correlation, and more items in the x-set. We found a few scenarios where regular CFA with FIML performed well. These findings suggested that the approaches can be implemented to handle the special missing data introduced by the SMS PM designs, and, thereby, enhance the utility of SMS PM data. In Study 2, we applied the multigroup CFA approach in real-world data to demonstrate the feasibility and analytic value of this approach.

As in other subfields of psychology, developmental science faces a long-standing problem of limited diversity in research participants. This issue especially raises concerns when researchers make unwarranted broad claims about their results, such as using generic language that implies that a finding is unvarying and applies across participants and contexts. Variation in editorial practices may contribute to the extent to which generic language is used or how much information is provided about participant demographics. The present study expands on an existing corpus of articles published in developmental psychology journals and examines the extent to which generic language differed by journal and sample characteristics (such as racial/ethnic diversity or sample size). We observed widespread use of generic language: 73% of articles included at least one generic statement in article components that were coded (i.e. titles, highlights, and abstracts). However, differences across journals were observed: Articles published in "Developmental Science" and "JECP" (which require authors to include short research highlights) tended to have higher rates of generic language than articles published in "Child Development" and "Developmental Psychology" (which did not require highlights). Articles that fully reported sample race and ethnicity also included fewer generic statements to describe their results than articles with more incomplete reporting. We conclude by highlighting the importance of editorial policies in shaping scholarship, as well as challenges and opportunities for researchers to consider when reporting demographic information in a global field and clearly and accurately communicating the importance of research findings.

The purpose of this instructional piece was to provide a nontechnical synthesis of common internal consistency reliability estimates used in professional counseling and in related fields. The article begins with an overview of coefficients alpha, omega, omega hierarchical, and H, with guidelines for their selection. Next, I provide recommendations for interpretive cutoff scores for higher and lower stakes testing followed by commentary on the limitations of relying too heavily on such guidelines. I discuss the importance of reporting confidence intervals (CIs) for reliability estimates to enhance reliability generalizations. When evaluating internal consistency reliability estimates of scores based on sample data counselors are advised to (a) determine the intended use of test scores in terms of higher or lower stakes testing, (b) take a multifaceted approach and consider the requirements of each reliability index, (c) include CIs, and (d) refer to interpretive cutoff scores as tentative general guidelines, not absolute standards.

Confirmatory factor analyses (CFA) are often used in psychological research when developing measurement models for psychological constructs. Evaluating CFA model fit can be quite challenging, as tests for exact model fit may focus on negligible deviances, while fit indices cannot be interpreted absolutely without specifying thresholds or cutoffs. In this study, we review how model fit in CFA is evaluated in psychological research using fit indices and compare the reported values with established cutoff rules. For this, we collected data on all CFA models in "Psychological Assessment" from the years 2015 to 2020 (N[subscript Studies]=221). In addition, we reevaluate model fit with newly developed methods that derive fit index cutoffs that are tailored to the respective measurement model and the data characteristics at hand. The results of our review indicate that the model fit in many studies has to be seen critically, especially with regard to the usually imposed independent clusters constraints. In addition, many studies do not fully report all results that are necessary to re-evaluate model fit. We discuss these findings against new developments in model fit evaluation and methods for specification search.

Multiple imputation (MI) is one of the recommended techniques for handling missing data in ordinal factor analysis models. However, methods for computing MI-based fit indices under ordinal factor analysis models have yet to be developed. In this short note, we introduced the methods of using the standardized root mean squared residual (SRMR) and the root mean square error of approximation (RMSEA) to assess the fit of ordinal factor analysis models with multiply imputed data. Specifically, we described the procedure for computing the MI-based sample estimates and constructing the confidence intervals. Simulation results showed that the proposed methods could yield sufficiently accurate point and interval estimates for both SRMR and RMSEA, especially in conditions with larger sample sizes, less missing data, more response categories, and higher degrees of misfit. Based on the findings, implications and recommendations were discussed.

Previous research extensively studied reasons for and ways to avoid low response rates, but it largely ignored the primary research issue of the degree to which response rates matter, which we address. Methodological survey research on response rates has been concerned with how to increase responsiveness and with the effects of response rates on variables' means and frequencies. More important to theory-oriented research, however, are effects of response rates on effect sizes (strength of relationships between variables). We examined recent survey research (446 correlations in 252 studies 2000-2020) on commonly studied variables in the example domain of organizational behavior, where surveys are especially common. We found a nonsignificant relationship (r = 0.01) between response rates and effect sizes. The relationship between sample sizes (the numerator in response rates) and effect sizes was weak but significant (r = 0.13, p = 0.01). Furthermore, there was no evidence for variability across studies and no curvilinear effects. Response rates on surveys have no noticeable effect on correlations nor therefore on theory development and testing.

Conventional random-effects models in meta-analysis rely on large sample approximations instead of exact small sample results. While random-effects methods produce efficient estimates and confidence intervals for the summary effect have correct coverage when the number of studies is sufficiently large, we demonstrate that conventional methods result in confidence intervals that are not wide enough when the number of studies is small, depending on the configuration of sample sizes across studies, the degree of true heterogeneity and number of studies. We introduce two alternative variance estimators with better small sample properties, investigate degrees of freedom adjustments for computing confidence intervals, and study their effectiveness via simulation studies.

A recent paper proposed an alternative weighting scheme when performing matching-adjusted indirect comparisons. This alternative approach follows the conventional one in matching the covariate means across two studies but differs in that it maximizes the effective sample size when doing so. The appendix of this paper showed, assuming there is one covariate and negative weights are permitted, that the resulting weights are linear in the covariates. This explains how the alternative method achieves a larger effective sample size and results in a metric that quantifies the difficulty of matching on particular covariates. We explain how these key results generalize to the case where there are multiple covariates, giving rise to a new metric that can be used to quantify the impact of matching on multiple covariates.

Applications of polytomous IRT models in applied fields (e.g., health, education, psychology) are abound. However, little is known about the impact of the number of categories and sample size requirements for precise parameter recovery. In a simulation study, we investigated the impact of the number of response categories and required sample size for successful parameter recovery and also examined model fit. Our results suggested that sample size requirements of approximately 250 or larger are typically sufficient for reasonable recovery of either person or item parameters (in particular discrimination). Further, we found that in many cases, the conditions of 10 items yielded more favorable results than those conditions when only 5 items were considered. With respect to the number of response categories, results were inconsistent, suggesting more complex recommendations in selecting the number of categories when developing scales. We discuss limitations of the current study, identify future directions, and provide recommendations for applied researchers who engage in scale development.

In this study a standardized effect size was created for use with the SIBTEST procedure. Using this standardized effect size, a single set of heuristics was developed that are appropriate for data fitting different item response models (e.g., 2-parameter logistic, 3-parameter logistic). The standardized effect size rescales the raw beta-uni value using a pooled variation that incorporates the beta-uni inclusion factor. Although the heuristics for the standardized and unstandardized effect sizes provide similar true-positive and false-positive rates in most conditions, the standardized effect size provides higher true-positive rates for conditions where item response variability is smaller in proportion to raw score differences. Inflated false-positive rates were solely impacted by smaller sample sizes, whereas larger sample sizes improved true-positive rates. An empirical application is provided to demonstrate how the standardized effect size provides for a more consistent comparison across items with varying response distributions. This study lays the foundation for the utilization of a standardized effect size for both dichotomous and polytomous item response models using the suite of SIBTEST procedures.

Comprehensive evaluation of treatment effects is aided by considerations for moderated effects. In educational research, the combination of natural hierarchical structures and prevalence of group-administered or shared facilitator treatments often produces three-level partially nested data structures. Literature details planning strategies for a variety of experimental designs when moderation effects are of interest but has yet to establish power formulas for detecting moderation effects in three-level partially nested designs. To address this gap, we derive and assess the accuracy of power formulas for detecting the different types of moderation effects possible in these designs. Using Monte Carlo simulation studies, we probe power rates and adequate sample sizes for detecting the different moderation effects while varying common influential factors including variance in the outcome explained by covariates, magnitude of the moderation effect, and sample sizes. The power formulas developed improve the planning of experimental studies with partial nesting and encourage the inclusion of moderator variables to capture for whom and under what conditions a treatment is effective. Educational researchers also have some initial guidance regarding adequate sample sizes and the factors that influence detecting moderation effects in three-level partially nested designs.

The use of brief dyslexia rating scales is increasing given current dyslexia legislation efforts across the United States. The purpose of this article is to provide an overview of the historical context of the use of brief dyslexia rating scales, strengths, and limitations of using these measures, criteria for selecting these measures, and a review of three commonly used brief dyslexia rating scales in the United States. A discussion of how brief dyslexia rating scales can be used in conjunction with other forms of assessment is also provided. Implications for practice on using brief dyslexia rating scales are addressed in the context of both assessment and intervention.