Search

The move toward evidence-based practice within counseling intervention research continues to grow. However, there are many practical barriers to implementing research designs that can support causal claims regarding counseling intervention efficacy. Overcoming these hurdles serves a larger goal of improving client care and further legitimizing the counseling profession. This article outlines an underused research design called regression discontinuity (RD), one of the strongest quasi-experimental designs available. RD allows researchers to use a cutoff score to determine treatment group assignment in situations where a randomized control trial is not feasible or ethical, such as when assigning care based on need consistent with the social justice values of the field. Historically, RD is rarely employed in counseling research. Therefore, to facilitate use, we provide a primer on key RD design concepts along with a concrete demonstration of the design, data analysis, and interpretation. We review the causal logic of the basic RD design followed by two data outcomes for a hypothetical scenario. Data and SPSS syntax are provided to allow readers to follow along and conduct their own analyses. [ABSTRACT FROM AUTHOR]

The authors empirically examined whether the validity of a residualized dependent variable after covariance adjustment is comparable to that of the original variable of interest. When variance of a dependent variable is removed as a result of one or more covariates, the residual variance may not reflect the same meaning. Using the pretest–posttest design as a general framework, the authors compared the nomological validity network for the (a) original dependent variable scores and (b) residualized dependent variable scores after having covaried-out variance explainable by a pretest. Heuristic and empirical examples are provided that demonstrate potential variation in construct validity of residualized dependent variables is a function of correlations among dependent, covariate, and validity variables. [ABSTRACT FROM PUBLISHER]

In the face of multicollinearity, researchers face challenges interpreting canonical correlation analysis (CCA) results. Although standardized function and structure coefficients provide insight into the canonical variates produced, they fall short when researchers want to fully report canonical effects. This article revisits the interpretation of CCA results, providing a tutorial and demonstrating canonical commonalty analysis. Commonality analysis fully explains the canonical effects produced by using the variables in a given canonical set to partition the variance of canonical variates produced from the other canonical set. Conducting canonical commonality analysis without the aid of software is laborious and may be untenable, depending on the number of noteworthy canonical functions and variables in either canonical set. Commonality analysis software is identified for the canonical correlation case and we demonstrate its use in facilitating model interpretation. Data from Holzinger and Swineford (1939) are employed to test a hypothetical theory that problem-solving skills are predicted by fundamental math ability. [ABSTRACT FROM AUTHOR]

The purpose of the present study was to explore the effect of both short- and long term Child-Centered Play Therapy on teacher–student relationship stress. Teachers identified 58 students exhibiting emotional and behavioral difficulties who were randomly assigned to one of two treatment groups. Students in the short-term intensive play therapy group participated in 16 sessions of play therapy over 8 weeks, and students in the long-term play therapy group participated in 16 sessions over 16 weeks. Results indicated that both intervention groups demonstrated significant improvement in teacher–student relationship stress from pre- to posttest. Post hoc analyses indicated that the short-term intensive intervention demonstrated statistical significance and larger effect sizes in overall total stress, teacher characteristics, and student characteristics. © 2008 Wiley Periodicals, Inc. [ABSTRACT FROM AUTHOR]

Relatively little attention has been devoted to the social validation of potentially effective autism interventions. Thus, it is often difficult to identify and implement evidence-based practices, and programming is often inadequate. The authors identified autism intervention components with reported effectiveness for school settings. The results of a social validation survey completed by parents, teachers, and administrators indicate strong, consistent support for program components falling within five functional areas: (a) individualized programming, (b) data collection, (c) the use of empirically-based strategies, (d) active collaboration, and (e) a focus on long-term outcomes. These socially validated interventions can be used to evaluate existing autism curricula and develop training for professionals, parents, and students in order to improve public school autism programs. [ABSTRACT FROM AUTHOR]

Given the proliferation of factor analysis applications in the literature, the present article examines the use of factor analysis in current published research across four psychological journals. Notwithstanding ease of analysis due to computers, the appropriate use of factor analysis requires a series of thoughtful researcher judgments. These judgments directly affect results and interpretations. The authors examine across studies (a) the decisions made while conducting exploratory factor analyses (N = 60) and (b) the information reported from the analyses. In doing so, they present a review of the current status of factor analytic practice, including comment on common errors in use and reporting. Recommendations are proffered for future practice as regards analytic decisions and reporting in empirical research. [ABSTRACT FROM AUTHOR]

The use of reliability generalization methodology promises to, among other things, inform researchers about the importance of reporting reliability coefficients and their use in result interpretation. This study presents results from a reliability generalization study of the Self-Description Questionnaire (SDQ). The average score reliabilities produced by the instrument as well as sample and instrument characteristics that may affect those reliabilities are presented. Results varied somewhat across the three SDQ instruments and their various subscales, but internal consistency estimates were generally strong for the SDQ. [ABSTRACT FROM AUTHOR]

The purpose of this article is to reduce potential statistical barriers and open doors to canonical correlation analysis (CCA) for applied behavioral scientists and personality researchers. CCA was selected for discussion, as it represents the highest level of the general linear model (GLM) and can be rather easily conceptualized as a method closely linked with the more widely understood Pearson r correlation coefficient. An understanding of CCA can lead to a more global appreciation of other univariate and multivariate methods in the GLM. We attempt to demonstrate CCA with basic language, using technical terminology only when necessary for understanding and use of the method. We present an entire example of a CCA analysis using SPSS (Version 11.0) with personality data. [ABSTRACT FROM AUTHOR]

The present study focused on the personality types of emergency certification teachers as predictors of classroom management and self-efficacy beliefs. Results indicated limited relationship between personality and management and efficacy beliefs. For the small effect observed, however, extraversion v. introversion was the salient personality variable and relationships were in expected directions. Marginal score reliabilities may have attenuated the observed effects. [ABSTRACT FROM AUTHOR]

Presents a study which examined the perception of college students about extreme or action sports. Methods; Results; Discussion.

Founded in social cognitive theory, teachers' self-efficacy beliefs have been repeatedly associated with positive teaching behaviors and student outcomes. However, teacher efficacy has developed a storied history regarding construct validity and measurement integrity. Study of teacher efficacy now stands on the verge of maturity, but such developmental growth will likely be contingent on development of strong theoretical models and effective instrumentation to assess theoretical constructs. The purpose of this article is to (a) briefly review the theoretical foundation of teacher efficacy and critically evaluate historical attempts to measure teacher efficacy, (b) discuss important substantive implications stemming from efficacy research that may advance the field, (c) present recent measurement advances, and (d) highlight several methodologies that have been underutilized in development of teacher efficacy instruments. [ABSTRACT FROM AUTHOR]

Reliability generalization (RG) is a measurement meta-analytic method used to explore the variability in score reliability estimates and to characterize the possible sources of this variance. This article briefly summarizes some RG considerations. Included is a description of how reliability confidence intervals might be portrayed graphically. The article includes tabulations across various RG studies, including how frequently authors (a) report score reliabilities for their own data, (b) conduct reliability induction, or (c) do not even mention reliability. [ABSTRACT FROM AUTHOR]

The Learning Style Inventory (LSI) is a commonly employed measure of learning styles based on Kolb's Experiential Learning Model. Nevertheless, the psychometric soundness of LSI scores has historically been critiqued. The present article extends this critique by conducting a reliability generalization study across studies and versions of the test. Results indicated that internal consistency and test-retest reliabilities for LSI scores fluctuate considerably and contribute to deleterious cumulative measurement error. Reliability variation was predictable by test version and several study features. [ABSTRACT FROM AUTHOR]

Presents recommended guidelines in conducting reliability generalization (RG) to characterize the measurement errors in scores across studies in counseling. Selection of a test for RG study; Development of coding sheet; Identification and use of meta-analysis dependent variables.

Some authors debate whether effect sizes should be reported (a) for all null hypothesis tests, even non-statistically significant ones, or (b) only after a finding is first determined to be statistically significant. The decision to report and interpret small effects may partially depend on the amount of bias in the effect size measure used. Based on the recognitions that variance-accounted-for effect statistics are positively biased and that standardized difference effect sizes such as Cohen's d can be converted into r[sup 2] metrics and vice versa, the authors considered that d also may be biased. The authors therefore explored the amount of bias in Cohen's d across a series of simulated study conditions. Results from their simulations indicated relatively no bias (close to zero) in Cohen's d across all study conditions. [ABSTRACT FROM AUTHOR]

Focuses on the internal consistency reliability estimates in the United States. Assessment of individual differences on a variety of constructs; Impact of reliability on substantive research; Correlation between score consistency and score reliability.

Teacher efficacy has proven to be an important variable in teacher effectiveness. It is consistently related to positive teaching behaviors and student outcomes. However, the measurement of this construct is the subject of current debate, which includes critical examination of predominant instruments used to assess teacher efficacy. The present study extends this critical evaluation and examines sources of measurement error variance in the Teacher Efficacy Scale (TES), historically the most frequently used instrument in the area. Reliability generalization was used to characterize the typical score reliability for the TES and potential sources of measurement error variance across studies. Other related instruments were also examined as regards measurement integrity. [ABSTRACT FROM AUTHOR]

This study compared three approaches for handling a fourth level of nesting when analyzing cluster-randomized trial (CRT) data. Although CRT data analyses may include repeated measures, individual, and cluster levels, there may be an additional fourth level that is typically ignored. This study examined the impact of ignoring this fourth level, accounting for it using a model-based approach, and accounting for it using a design-based approach on parameter and standard error (SE) estimates. Several fixed effect and random effect variance parameters and SEs were biased across all three models. Results suggest if a meaningful fourth level exists, researchers should acknowledge it using a design-based approach. If the fourth level is not practically important, researchers may ignore it altogether, resulting in more accurate parameter and SE estimates. [ABSTRACT FROM AUTHOR]

Methods of reliability generalization characterize the reliability of a scale's scores across studies. Charter in 2003 presented useful formulae for computing combined estimates of reliability. The present article illustrates use of Charter's formulae and application of confidence intervals for reliability coefficients, including the combined estimate. Researchers may find these methods for reliability generalization useful. [ABSTRACT FROM AUTHOR]

Given the potential negative effects of vicarious racism, we sought to examine the impact of vicarious racism via the media on college students of color. Using a sample of 217 college students of color, we analyzed positive and negative affect and craving for alcohol and marijuana before and after exposure to media stimuli. Split‐plot analysis of variance results revealed a statistically significant interaction effect between time and group for negative affect, but not cravings for substances. [ABSTRACT FROM AUTHOR]

Multivariate analysis of variance (MANOVA) is a statistical method used to examine group differences on multiple outcomes. This article reports results of a review of MANOVA in gifted education journals between 2011 and 2017 (N = 56). Findings suggest a number of conceptual and procedural misunderstandings about the nature of MANOVA and its application, including pervasive use of univariate post hoc tests to interpret MANOVA results. Accordingly, this article aims to make MANOVA more accessible to gifted education scholars by clarifying its purpose and introducing descriptive discriminant analysis as a more appropriate post hoc technique. A heuristic data set is used to demonstrate the procedures for running a descriptive discriminant analysis, both in place of a one-way MANOVA and as a post hoc analysis to a factorial design. SPSS and R syntax are provided. [ABSTRACT FROM AUTHOR]

The rise in consumer and corporate interest in action sports, also known as extreme sports, has been phenomenal. The apparent popularity of action sports, when combined with the sponsorships, endorsements, and advertising dollars they have quickly garnered, lends itself to scientific inquiry regarding the level and nature of public interest. The purpose of this study was to examine Generation Y's perceptions of action sports, with a specific focus on the expressed popularity of action sports and the relationship between action sports interest and use of the media. The 39-item Action Sports Questionnaire (ESQ) was constructed to examine Generation Y perceptions of action sports, sports related viewing preferences, and sports related media usage among middle and high school aged students. The present findings suggested that these members of the Generation Y (n = 367) niche market preferred action sports over the traditional sports of basketball and baseball. Respondents also indicated stronger preference for soccer, but would prefer to watch the X-Games over the World Cup. There is an indication that soccer and action sports are more popular among the younger generation than some traditional team sports. Males were slightly more supportive that action sports would become more popular in the future, and the male respondents were likewise more familiar with action sports. More members of Generation Y watch action sports than their predecessors, and they likewise tend to be optimistic about the future of action sports if they watch events on television. [ABSTRACT FROM AUTHOR]

A split-sample cross-validation confirmatory factor analysis (N = 1,210) confirmed the 4-factor structure of the Adlerian Personality Priority Assessment (APPA) with 2 distinct subsamples. Discriminant validity, prevalence rates of priorities, and preliminary group norms are reported. [ABSTRACT FROM AUTHOR]

A common practice in general linear model (GLM) analyses is to interpret regression coefficients (e.g., standardizedβweights) as indicators of variable importance. However, focusing solely on standardized beta weights may provide limited or erroneous information. For example,βweights become increasingly unreliable when predictor variables are correlated, which is often the case in the social sciences. To address this issue, structure coefficients, which are simply the bivariate correlation between a predictor and the syntheticŶvariable, should also be interpreted. By examiningβweights and structure coefficients in conjunction, the predictive worth of each independent variable can be more accurately judged. Despite this benefit, researchers in the field of sport and exercise science have rarely reported structure coefficients when conducting multiple regression analysis. Thus, the purpose of the present article is to discuss problems associated with the sole interpretation ofβweights and to demonstrate how structure coefficients can be incorporated to improve accuracy of interpretation. Additionally, a content analysis was conducted to examine current trends in reporting multiple regression results within sport and exercise science research. [ABSTRACT FROM AUTHOR]

Aim/Purpose The purpose of the current study is to explore student perceptions of their own doctoral-level education and quantitative proficiency. Background The challenges of preparing doctoral students in education have been discussed in the literature, but largely from the perspective of university faculty and program administrators. The current study directly explores the student voice on this issue. Methodology Utilizing a sequential explanatory mixed-methods research design, the present study seeks to better understand doctoral-level education students’ perceptions of their quantitative methods training at a large public university in the southwestern United States. Findings Results from both phases present the need for more application and consistency in doctoral-level quantitative courses. Additionally, there was a consistent theme of internal motivation in the responses, suggesting students perceive their quantitative training to be valuable beyond their personal interest in the topic. Recommendations for Practitioners Quantitative methods instructors should emphasize practice in their quantitative courses and consider providing additional support for students through the inclusion of lab sections, tutoring, and/or differentiation. Pre-testing statistical ability at the start of a course is also suggested to better meet student needs. Impact on Society The ultimate goal of quantitative methods in doctoral education is to produce high-quality educational researchers who are prepared to apply their knowledge to problems and research in education. Results of the present study can inform faculty and administrator decisions in doctoral education to best support this goal. Future Research Using the student perspectives presented in the present study, future researchers should continue to explore effective instructional strategies and curriculum design within education doctoral programs. The inclusion of student voice can strengthen and guide future work in this area. [ABSTRACT FROM AUTHOR]

There has been a recent call to improve data reporting in kinesiology journals, including the appropriate use of univariate and multivariate analysis techniques. For example, a multivariate analysis of variance (MANOVA) with univariate post hocs and a Bonferroni correction is frequently used to investigate group differences on multiple dependent variables. However, this univariate approach decreases power, increases the risk for Type 1 error, and contradicts the rationale for conducting multivariate tests in the first place. Purpose: The purpose of this study was to provide a user-friendly primer on conducting descriptive discriminant analysis (DDA), which is a post-hoc strategy to MANOVA that takes into account the complex relationships among multiple dependent variables. Method: A real-world example using the Statistical Package for the Social Sciences syntax and data from 1,095 middle school students on their body composition and body image are provided to explain and interpret the results from DDA. Results: While univariate post hocs increased the risk for Type 1 error to 76%, the DDA identified which dependent variables contributed to group differences and which groups were different from each other. For example, students in the very lean and Healthy Fitness Zone categories for body mass index experienced less pressure to lose weight, more satisfaction with their body, and higher physical self-concept than the Needs Improvement Zone groups. However, perceived pressure to gain weight did not contribute to group differences because it was a suppressor variable. Conclusion: Researchers are encouraged to use DDA when investigating group differences on multiple correlated dependent variables to determine which variables contributed to group differences. [ABSTRACT FROM AUTHOR]

It is not uncommon to use unidimensional item response theory models to estimate ability in multidimensional data with computerized adaptive testing (CAT). The current Monte Carlo study investigated the penalty of this model misspecification in CAT implementations using different item selection methods and exposure control strategies. Three item selection methods—maximum information (MAXI), a-stratification (STRA), and a-stratification with b-blocking (STRB) with and without Sympson–Hetter (SH) exposure control strategy—were investigated. Calibrating multidimensional items as unidimensional items resulted in inaccurate item parameter estimates. Therefore, MAXI performed better than STRA and STRB in estimating the ability parameters. However, all three methods had relatively large standard errors. SH exposure control had no impact on the number of overexposed items. Existing unidimensional CAT implementations might consider using MAXI only if recalibration as multidimensional model is too expensive. Otherwise, building a CAT pool by calibrating multidimensional data as unidimensional is not recommended. [ABSTRACT FROM PUBLISHER]

The current study was designed to test specific hypotheses associated with W. J. Lyddon and A. Sherry's (2001) attachment theory model of developmental personality styles. More specifically, 4 adult attachment dimensions were correlated with 10 personality scales on the Millon Clinical Multiaxial Inventory-III (T. Millon, R. Davis, & C. Millon, 1997) with a sample of 273 men and women. Findings indicated that the adult attachment dimensions were able to predict 7 of the 10 personality styles. [ABSTRACT FROM AUTHOR]

A reliability generalization (RG) study was conducted on the Coopersmith Self-Esteem Inventory (CSEI) to (a) examine the variability of reliability estimates across studies and (b) identify study characteristics that may predict this variability. Typical RG methodology was extended by estimating internal consistency reliability with KR-21 in cases when authors did not report reliability for their obtained scores. Accordingly, the present study investigated the psychometric properties of the CSEI and serves as a model to extend typical RG methods in some cases when examining dichotomously scored instruments. [ABSTRACT FROM AUTHOR]

The Mathematics Anxiety Rating Scale (MARS) was submitted to a reliability generalization analysis (RG) to characterize the variability of measurement error in MARS scores across administrations and identify possible study characteristics that are predictive of score reliability variations. In general, the MARS and its variants yielded scores with strong internal consistency and test-retest reliability estimates, although variation was observed. Adult samples were related to lower score reliability compared to other age groupings. Inclusion of total score standard deviation in the regression models resulted in roughly 25% increases in R[sup 2] effects. [ABSTRACT FROM AUTHOR]

Sexual addiction is prevalent among college students and warrants continued examination. Using a descriptive discriminant analysis, the authors assessed differences in attachment style and religious coping between students in the clinical range of sexual addiction ( n = 56) and those in the nonclinical range ( n = 270). Results indicated significant differences between the 2 groups, with a large effect size. Specifically, attachment anxiety and negative religious coping were most responsible for the explained variance. [ABSTRACT FROM AUTHOR]