Search

Background An abundance of rapidly accumulating scientific evidence presents novel opportunities for researchers and practitioners alike, yet such advantages are often overshadowed by resource demands associated with finding and aggregating a continually expanding body of scientific information. Data extraction activities associated with evidence synthesis have been described as time-consuming to the point of critically limiting the usefulness of research. Across social science disciplines, the use of automation technologies for timely and accurate knowledge synthesis can enhance research translation value, better inform key policy development, and expand the current understanding of human interactions, organizations, and systems. Ongoing developments surrounding automation are highly concentrated in research for evidence-based medicine with limited evidence surrounding tools and techniques applied outside of the clinical research community. The goal of the present study is to extend the automation knowledge base by synthesizing current trends in the application of extraction technologies of key data elements of interest for social scientists. Methods We report the baseline results of a living systematic review of automated data extraction techniques supporting systematic reviews and meta-analyses in the social sciences. This review follows PRISMA standards for reporting systematic reviews. Results The baseline review of social science research yielded 23 relevant studies. Conclusions When considering the process of automating systematic review and meta-analysis information extraction, social science research falls short as compared to clinical research that focuses on automatic processing of information related to the PICO framework. With a few exceptions, most tools were either in the infancy stage and not accessible to applied researchers, were domain specific, or required substantial manual coding of articles before automation could occur. Additionally, few solutions considered extraction of data from tables which is where key data elements reside that social and behavioral scientists analyze.

Background An abundance of rapidly accumulating scientific evidence presents novel opportunities for researchers and practitioners alike, yet such advantages are often overshadowed by resource demands associated with finding and aggregating a continually expanding body of scientific information. Data extraction activities associated with evidence synthesis have been described as time-consuming to the point of critically limiting the usefulness of research. Across social science disciplines, the use of automation technologies for timely and accurate knowledge synthesis can enhance research translation value, better inform key policy development, and expand the current understanding of human interactions, organizations, and systems. Ongoing developments surrounding automation are highly concentrated in research for evidence-based medicine with limited evidence surrounding tools and techniques applied outside of the clinical research community. The goal of the present study is to extend the automation knowledge base by synthesizing current trends in the application of extraction technologies of key data elements of interest for social scientists. Methods We report the baseline results of a living systematic review of automated data extraction techniques supporting systematic reviews and meta-analyses in the social sciences. This review follows PRISMA standards for reporting systematic reviews. Results The baseline review of social science research yielded 23 relevant studies. Conclusions When considering the process of automating systematic review and meta-analysis information extraction, social science research falls short as compared to clinical research that focuses on automatic processing of information related to the PICO framework. With a few exceptions, most tools were either in the infancy stage and not accessible to applied researchers, were domain specific, or required substantial manual coding of articles before automation could occur. Additionally, few solutions considered extraction of data from tables which is where key data elements reside that social and behavioral scientists analyze.

Managerial coaching remains a widespread and popular organizational development intervention applied across numerous industries to enhance critical workplace outcomes and employee attitudes, yet no studies to date have evaluated the temporal precedence within these relationships. This study sought to assess the predictive validity of the widely used Employee Perceptions of Supervisor/Line Manager Coaching Behavior Measure managerial coaching scale (CBI), employing a longitudinal design and following the testing of the causal hypothesized relationship framework. Three hypotheses were evaluated using three commonly associated variables with managerial coaching (role clarity, job satisfaction, and organization commitment), using longitudinal data collected over two waves from full-time US employees (n = 313). The study followed a two-wave design, collecting data over two time points to test for longitudinal measurement invariance and three reciprocal cross-lagged models. Results detected statistically significant cross-lagged and reciprocal cross-lagged effects in the role clarity and organization commitment models, highlighting a reciprocal relationship between managerial coaching behaviors and the two variables. However, only the reciprocal cross-lagged effect was statistically significant in the job satisfaction model. Findings suggest the predictive validity of the CBI scale for role clarity and organization commitment. Moreover, results indicate employee attitudes influenced managerial coaching behaviors over time across all three models, emphasizing the potential impact of employee attitudes on leadership effectiveness. This study highlights the complex relationships between managerial coaching and workplace outcomes, offering nuanced insights for improved understanding.

This is a review of a range of empirical studies that use digital text algorithms to predict and model response patterns from humans to Likert-scale items, using texts only as inputs. The studies show that statistics used in construct validation is predictable on sample and individual levels, that this happens across languages and cultures, and that the relationship between variables are often semantic instead of empirical. That is, the relationships among variables are given a priori and evidently computable as such. We explain this by replacing the idea of “nomological networks” with “semantic networks” to designate computable relationships between abstract concepts. Understanding constructs as nodes in semantic networks makes it clear why psychological research has produced constant average explained variance at 42% since 1956. Together, these findings shed new light on the formidable capability of human minds to operate with fast and intersubjectively similar semantic processing. Our review identifies a categorical error present in much psychological research, measuring representations instead of the purportedly represented. We discuss how this has grave consequences for the empirical truth in research using traditional psychometric methods.

The recent surge in artificial intelligence (AI) has significantly transformed work dynamics, particularly in human resource development (HRD) and related domains. Scholars, recognizing the significant potential of AI in HRD functions and processes, have contributed to the growing body of literature reviews on AI in HRD and related domains. Despite the valuable insights provided by these individual reviews, the challenge of collectively interpreting them within the HRD domain remains unresolved. This protocol outlines the methodology for an umbrella review aiming to systematically synthesize existing reviews on AI in HRD. The review seeks to address key research questions regarding AI's contributions to HRD functions and processes, as well as the opportunities and threats associated with its implementation by employing a technology-aided systematic approach. The coding framework will be used to synthesize the contents of the selected systematic reviews such as their search strategies, data synthesis approaches, and HRD-related findings. The results of this umbrella review are expected to provide insights for HRD scholars and practitioners, promoting continuous improvement in AI-driven HRD initiatives. This protocol is preregistered on the Open Science Framework (https://doi.org/10.17605/OSF.IO/Z8NM6) on May 27, 2024.

Background: An abundance of rapidly accumulating scientific evidence presents novel opportunities for researchers and practitioners alike, yet such advantages are often overshadowed by resource demands associated with finding and aggregating a continually expanding body of scientific information. Across social science disciplines, the use of automation technologies for timely and accurate knowledge synthesis can enhance research translation value, better inform key policy development, and expand the current understanding of human interactions, organizations, and systems. Ongoing developments surrounding automation are highly concentrated in research for evidence-based medicine with limited evidence surrounding tools and techniques applied outside of the clinical research community. Our objective is to conduct a living systematic review of automated data extraction techniques supporting systematic reviews and meta-analyses in the social sciences. The aim of this study is to extend the automation knowledge base by synthesizing current trends in the application of extraction technologies of key data elements of interest for social scientists. Methods: The proposed study is a living systematic review employing a partial replication framework based on extant literature surrounding automation of data extraction for systematic reviews and meta-analyses. Protocol development, base review, and updates follow PRISMA standards for reporting systematic reviews. This protocol is preregistered in OSF: (Semi)Automated Approaches to Data Extraction for Systematic Reviews and Meta-Analyses in Social Sciences: A Living Review Protocol on August 14, 2022. Conclusions: Anticipated outcomes of this study include: (a) generate insights supporting advancement in transferring existing reliable methods to social science research; (b) provide a foundation for protocol development leading to enhancement of comparability and benchmarking standards across disciplines; and (c) uncover exigencies that spur continued value-adding innovation and interdisciplinary collaboration for the benefit of the collective systematic review community.

This study uses latent semantic analysis (LSA) to explore how prevalent measures of motivation are interpreted across very diverse job types. Building on the Semantic Theory of Survey Response (STSR), we calculate “semantic compliance” as the degree to which an individual’s responses follow a semantically predictable pattern. This allows us to examine how context, in the form of job type, influences respondent interpretations of items. In total, 399 respondents from 18 widely different job types (from CEOs through lawyers, priests and artists to sex workers and professional soldiers) self-rated their work motivation on eight commonly applied scales from research on motivation. A second sample served as an external evaluation panel (n = 30) and rated the 18 job types across eight job characteristics. Independent measures of the job types’ salary levels were obtained from national statistics. The findings indicate that while job type predicts motivational score levels significantly, semantic compliance as moderated by job type job also predicts motivational score levels usually at a lesser but significant magnitude. Combined, semantic compliance and job type explained up to 41% of the differences in motional score levels. The variation in semantic compliance was also significantly related to job characteristics as rated by an external panel, and to national income levels. Our findings indicate that people in different contexts interpret items differently to a degree that substantially affects their score levels. We discuss how future measurements of motivation may improve by taking semantic compliance and the STSR perspective into consideration.

A reliability generalization meta-analysis was performed to explore the relationship between study factors and levels of alpha reliability for the 15 subscales of the Motivated Strategies for Learning Questionnaire (MSLQ). The MSLQ has been widely adapted over the past 25 years to investigate the role of motivation and strategies in learning, primarily at the postsecondary level. A literature search from the years 1991 to 2015 yielded 295 peer-reviewed journal articles and 1,369 alpha reliability coefficients. Articles were coded for six potential moderator study variables. A novel varying coefficient (VC) model was adopted to determine average reliabilities across studies for each subscale and to perform multiple regression analyses to identify study variables that may moderate alpha reliability estimates. Commonality analyses were used to aid in interpretation of regression results. Meta-analyzed alpha reliabilities were lower than values published in the test manual for all but three of the subscales. Ability of specific moderators to predict score reliability varied across subscales; however, studies in North America, in English, or using a 7-point response scale generally corresponded to increased reliability estimates. Knowledge of expected levels of score reliability under varying sample and study conditions may provide useful information for researchers planning future use of the MSLQ.

The Online Student Connectedness Survey (OSCS) was introduced to the academic community in 2012 as an instrument designed to measure feelings of connectedness between students participating in online degree and certification programs. The purpose of this study was to examine data from the instrument for initial evidence of validity and reliability and to establish a nomological network between the OSCS, the Classroom Connectedness Survey (CCS), and the Community of Inquiry Survey (COI), which are similar instruments in the field. Results provided evidence of factor validity and reliability. Additionally, statistically and practically significant correlations were demonstrated between factors contained in the OSCS and established instruments measuring factors related to student connectedness. These results indicate that for the sample used in this study, the OSCS provides data that are valid and reliable for assessing feelings of connection between participants in online courses at institutions of higher learning.

Background An abundance of rapidly accumulating scientific evidence presents novel opportunities for researchers and practitioners alike, yet such advantages are often overshadowed by resource demands associated with finding and aggregating a continually expanding body of scientific information. Data extraction activities associated with evidence synthesis have been described as time-consuming to the point of critically limiting the usefulness of research. Across social science disciplines, the use of automation technologies for timely and accurate knowledge synthesis can enhance research translation value, better inform key policy development, and expand the current understanding of human interactions, organizations, and systems. Ongoing developments surrounding automation are highly concentrated in research for evidence-based medicine with limited evidence surrounding tools and techniques applied outside of the clinical research community. The goal of the present study is to extend the automation knowledge base by synthesizing current trends in the application of extraction technologies of key data elements of interest for social scientists. Methods We report the baseline results of a living systematic review of automated data extraction techniques supporting systematic reviews and meta-analyses in the social sciences. This review follows PRISMA standards for reporting systematic reviews. Results The baseline review of social science research yielded 23 relevant studies. Conclusions When considering the process of automating systematic review and meta-analysis information extraction, social science research falls short as compared to clinical research that focuses on automatic processing of information related to the PICO framework. With a few exceptions, most tools were either in the infancy stage and not accessible to applied researchers, were domain specific, or required substantial manual coding of articles before automation could occur. Additionally, few solutions considered extraction of data from tables which is where key data elements reside that social and behavioral scientists analyze.

Background An abundance of rapidly accumulating scientific evidence presents novel opportunities for researchers and practitioners alike, yet such advantages are often overshadowed by resource demands associated with finding and aggregating a continually expanding body of scientific information. Data extraction activities associated with evidence synthesis have been described as time-consuming to the point of critically limiting the usefulness of research. Across social science disciplines, the use of automation technologies for timely and accurate knowledge synthesis can enhance research translation value, better inform key policy development, and expand the current understanding of human interactions, organizations, and systems. Ongoing developments surrounding automation are highly concentrated in research for evidence-based medicine with limited evidence surrounding tools and techniques applied outside of the clinical research community. The goal of the present study is to extend the automation knowledge base by synthesizing current trends in the application of extraction technologies of key data elements of interest for social scientists. Methods We report the baseline results of a living systematic review of automated data extraction techniques supporting systematic reviews and meta-analyses in the social sciences. This review follows PRISMA standards for reporting systematic reviews. Results The baseline review of social science research yielded 23 relevant studies. Conclusions When considering the process of automating systematic review and meta-analysis information extraction, social science research falls short as compared to clinical research that focuses on automatic processing of information related to the PICO framework. With a few exceptions, most tools were either in the infancy stage and not accessible to applied researchers, were domain specific, or required substantial manual coding of articles before automation could occur. Additionally, few solutions considered extraction of data from tables which is where key data elements reside that social and behavioral scientists analyze.

Background: An abundance of rapidly accumulating scientific evidence presents novel opportunities for researchers and practitioners alike, yet such advantages are often overshadowed by resource demands associated with finding and aggregating a continually expanding body of scientific information. Across social science disciplines, the use of automation technologies for timely and accurate knowledge synthesis can enhance research translation value, better inform key policy development, and expand the current understanding of human interactions, organizations, and systems. Ongoing developments surrounding automation are highly concentrated in research for evidence-based medicine with limited evidence surrounding tools and techniques applied outside of the clinical research community. Our objective is to conduct a living systematic review of automated data extraction techniques supporting systematic reviews and meta-analyses in the social sciences. The aim of this study is to extend the automation knowledge base by synthesizing current trends in the application of extraction technologies of key data elements of interest for social scientists. Methods: The proposed study is a living systematic review employing a partial replication framework based on extant literature surrounding automation of data extraction for systematic reviews and meta-analyses. Protocol development, base review, and updates follow PRISMA standards for reporting systematic reviews. This protocol is preregistered in OSF: (Semi)Automated Approaches to Data Extraction for Systematic Reviews and Meta-Analyses in Social Sciences: A Living Review Protocol on August 14, 2022. Conclusions: Anticipated outcomes of this study include: (a) generate insights supporting advancement in transferring existing reliable methods to social science research; (b) provide a foundation for protocol development leading to enhancement of comparability and benchmarking standards across disciplines; and (c) uncover exigencies that spur continued value-adding innovation and interdisciplinary collaboration for the benefit of the collective systematic review community.

Background: An abundance of rapidly accumulating scientific evidence presents novel opportunities for researchers and practitioners alike, yet such advantages are often overshadowed by resource demands associated with finding and aggregating a continually expanding body of scientific information. Across social science disciplines, the use of automation technologies for timely and accurate knowledge synthesis can enhance research translation value, better inform key policy development, and expand the current understanding of human interactions, organizations, and systems. Ongoing developments surrounding automation are highly concentrated in research for evidence-based medicine with limited evidence surrounding tools and techniques applied outside of the clinical research community. Our objective is to conduct a living systematic review of automated data extraction techniques supporting systematic reviews and meta-analyses in the social sciences. The aim of this study is to extend the automation knowledge base by synthesizing current trends in the application of extraction technologies of key data elements of interest for social scientists. Methods: The proposed study is a living systematic review employing a partial replication framework based on extant literature surrounding automation of data extraction for systematic reviews and meta-analyses. Protocol development, base review, and updates follow PRISMA standards for reporting systematic reviews. This protocol is preregistered in OSF: (Semi)Automated Approaches to Data Extraction for Systematic Reviews and Meta-Analyses in Social Sciences: A Living Review Protocol on August 14, 2022. Conclusions: Anticipated outcomes of this study include: (a) generate insights supporting advancement in transferring existing reliable methods to social science research; (b) provide a foundation for protocol development leading to enhancement of comparability and benchmarking standards across disciplines; and (c) uncover exigencies that spur continued value-adding innovation and interdisciplinary collaboration for the benefit of the collective systematic review community.

Managerial coaching remains a widespread and popular organizational development intervention applied across numerous industries to enhance critical workplace outcomes and employee attitudes, yet no studies to date have evaluated the temporal precedence within these relationships. This study sought to assess the predictive validity of the widely used Employee Perceptions of Supervisor/Line Manager Coaching Behavior Measure managerial coaching scale (CBI), employing a longitudinal design and following the testing of the causal hypothesized relationship framework. Three hypotheses were evaluated using three commonly associated variables with managerial coaching (role clarity, job satisfaction, and organization commitment), using longitudinal data collected over two waves from full-time US employees (n  =  313). The study followed a two-wave design, collecting data over two time points to test for longitudinal measurement invariance and three reciprocal cross-lagged models. Results detected statistically significant cross-lagged and reciprocal cross-lagged effects in the role clarity and organization commitment models, highlighting a reciprocal relationship between managerial coaching behaviors and the two variables. However, only the reciprocal cross-lagged effect was statistically significant in the job satisfaction model. Findings suggest the predictive validity of the CBI scale for role clarity and organization commitment. Moreover, results indicate employee attitudes influenced managerial coaching behaviors over time across all three models, emphasizing the potential impact of employee attitudes on leadership effectiveness. This study highlights the complex relationships between managerial coaching and workplace outcomes, offering nuanced insights for improved understanding. [ABSTRACT FROM AUTHOR]