Search

This study aims to explore the effects of pre-service teachers' use of rubric in self-assessment with instructor feedback on academic achievement and self-regulated learning. Their perceptions and experiences of the self-assessment intervention were also investigated. A total of 79 pre-service teachers participated in the study. A mixed methods approach was used as a blend of experimental and qualitative design. The quasi-experimental research model with pretest/posttest control group design was employed in the quantitative phase of the study. The pre-service teachers (N = 79) were assigned to either use of rubric in self-assessment involving tutor feedback condition or a non-self-assessment condition for their essay assignments. Besides, the pre-service teachers' perceptions and experiences of using self-assessment with instructor feedback were explored in the qualitative phase of the study. Data were collected using a rubric, an achievement test, a self-regulation in learning subscale and reflective journals. The results indicated that the rubric used in self-assessment with instructor feedback group had higher achievement and use of self-regulated learning strategies than the no-intervention group. The reflective journals also revealed that most of the pre-service teachers found the self-assessment a useful learning tool. They felt that it helped them to improve their learning by guiding them to set their own goals, monitor their progress and reflect on their learning through their own tasks. The implications for educational research and practice are discussed. [ABSTRACT FROM AUTHOR]

This meta-analysis explores the effect of using scoring rubrics on self-efficacy and self-regulation in K-16 formal learning settings and its potential moderators. From the literature, we identified 14 relevant experimental or quasi-experimental primary studies conducted with a total of 2793 students. We retrieved 17 effect sizes for self-efficacy and 18 effect sizes for self-regulation outcomes from the primary studies. Rubric use has a statistically significant moderate to large positive effect on students’ self-efficacy (Hedges’ g = 0.39) and self-regulation (Hedges’ g = 1.00). Large within- and -between study variability of effect sizes is common: self-efficacy (Hedges’ g: −.06; 2.47) and self-regulation (Hedges’ g: −1.17; 3.30). We found no significant moderation of the effect of rubric use by students’ level of education, providing feedback, or instruction using the rubric, whereas there is evidence of an effect of rubrics on self-efficacy and self-regulation, variability of theoretical approaches, measures, and implementation quality raise questions about best practices for rubric development and use. [ABSTRACT FROM AUTHOR]

A qualitative study was conducted in a secondary school to evaluate student learning processes and outcomes by examining their inquiry questions, journals, and maker artifacts in a curriculum-based maker learning environment supported by the Guided Inquiry Design (GID). Thirteen 8th-grade students in a suburban middle school in the southwest of the United States participated in the study. Inquiry questions, maker artifacts, and inquiry journals were collected and analyzed with rubrics that were developed based on a critical review of literature drawn from different bodies of literature. The objectives of this study were three-fold: (1) evaluate the quality of students' inquiry questions and maker artifacts, (2) assess students' internal knowledge representation (IKR) and external knowledge representation (EKR), and (3) develop a robust and valid assessment framework for maker learning considering students' expertise development over time. The findings revealed that the students progressed at different developmental levels; therefore, expertise development should be incorporated into the assessment framework for maker learning. This study also implied that scaffolding should be tailored to meet special needs of each student in a maker learning environment. The originality of this research is that the assessment framework takes into account individuals' development and progress towards expertise over time, instead of focusing on their learning outcomes at a specific time point. An additional value of this study is that the tools and supporting materials that serve as instructional scaffolds also serve as tools to collect evidence about student learning performance, processes, and outcomes. [ABSTRACT FROM AUTHOR]

Rubrics are widely used as instructional and learning instrument. Though they have been claimed to have positive effects on students’ learning, these effects have not been meta-analyzed. Our aim was to synthesize the effects of rubrics on academic performance, self-regulated learning, and self-efficacy. The moderator effect of the following variables was also investigated: year of publication, gender, mean age, educational level, type of educational level (compulsory vs. higher education), number of sessions, number of assessment criteria, number of performance levels, use of self and peer assessment, research design, and empirical quality of the study. Standardized mean differences (for the three outcomes) and standardized mean changes (SMC; for academic performance) were calculated from the retrieved studies. After correcting for publication bias, a moderate and positive effect was found in favor of rubrics on academic performance (g = 0.45, k = 21, m = 54, 95% CI [0.312, 0.831]; SMC = 0.38, 95% CI [0.02, 0.75], k = 12, m = 30), whereas a small pooled effect was observed for self-regulated learning (g = 0.23, k = 5, m = 17, 95% CI [-0.15, 0.60]) and for self-efficacy (g = 0.18, k = 3, m = 5, 95% CI [-0.81, 0.91]). Most of the moderator variables were not significant. Importantly, to improve the quality of future reports on the effects of rubrics, we provide an instrument to be filled out for rubric scholars in forthcoming studies. [ABSTRACT FROM AUTHOR]

This article details how and why we have developed a flexible and responsive process-based rubric exemplar for teaching, learning, and assessing critical and creative thinking. We hope to contribute to global discussions of and efforts toward instrumentalising the challenge of assessing, but not standardising, creativity in compulsory education. Here, we respond to the key ideas of the four interrelated elements in the critical and creative thinking general capability in the Australian Curriculum learning continuum: inquiring; generating ideas, possibilities, actions; reflecting on thinking processes; and analysing, synthesising and evaluating reasoning and procedures. The rubrics, radical because they privilege process over outcome, have been designed to be used alongside the current NAPLAN tests in Years 5, 7 and 9 to build an Australian-based national creativity measure. We do so to argue the need for local and global measures of creativity in education as the first round of testing and results of the PISA Assessment of Creative Thinking approach and to contribute to the recognition of creative thinking (and doing) as a core twenty-first century literacy alongside literacy and numeracy. [ABSTRACT FROM AUTHOR]

Rubrics for writing courses have become increasingly popular yet there have been few studies published from the GCC. This study investigated the effectiveness of rubrics in business writing classes at a Kuwait university. The participants were 104 students who were given a survey that evaluated their opinions of the rubrics. In addition, several instructors from these writing courses were interviewed. It found that a robust rubric may help ensure that grading is consistent and fair, and that it was highly useful as a guide for students when writing the assignments, and assisted students in crafting the essay. A proportion of students suggested that additional feedback beyond the rubric was desirable. The writing instructors were generally supportive of the value of rubrics, especially as a means to indicate to students what is required in classroom assignments. Overall, this study contributes to the body of literature demonstrating the value of rubrics for writing classes. [ABSTRACT FROM AUTHOR]

A significant number of instructors, researchers and students have claimed that chemistry is a challenging subject to teach and learn at all education levels. Its main learning difficulties are in line with certain sights of its phenomena that are abstract, and some chemistry teachers do not specify what to be learned and assessed in chemistry lesson. The current work investigates the use of formative assessment rubrics for supporting secondary school students' progressive learning in organic chemistry through a technology-based learning project approach in Rwanda. The investigators used a convergent parallel research design, and quantitative data were gathered by distributing questionnaires to the students, and the answers were statistically analyzed. Qualitative data were obtained through observation and interview and were narratively analyzed. The results from this study showed that rubric-based formative assessment supported students' learning of organic chemistry via technology-based learning approach. This is accredited to the fact that students were motivated while doing their assessment and they were able to do self-assessment by applying the provided rubrics via technology. The students understood instructors' expectations, encouraged their learning, sharpened their technology skills, and their knowledge retention was also increased. The instructors were able to grade the students' tasks fast with the help of an analytic rubric and good formative feedback was availed to students on time. The instructors were also able to diagnose the strengths and weaknesses of the learners and give them quick formative feedback. [ABSTRACT FROM AUTHOR]

Ethical capability is one of a suite of broad competencies that are recent inclusions in national and state curricula in Australia. Problems arise in how to support teachers to assess student progress in complex skill areas. Using methodology derived from the work of Griffin, this study investigated whether a criterion-referenced assessment framework, in the form of rubrics and a learning progression, could be developed for the construct of Ethical Decision-Making (EDM). Firstly, the domain of ethical competency was critically interrogated by examining curriculum documents, educational models, and empirical research into moral development. A construct of EDM and a derived skills framework were then presented to an expert panel of teachers. Over the course of two workshops, the panel critically reviewed the framework and co-constructed a set of rubrics. In a final iteration, the rubrics formed the basis for developing a hypothetical learning progression. Further research would be required to determine whether these assessment materials are useful to support teacher judgement about student progress and curriculum planning. [ABSTRACT FROM AUTHOR]

Models and modeling are central to both scientific literacy and practices as demonstrated by the Next Generation Science Standards. Through a design-based research framework, we developed a model-based assessment (MBA) and associated rubric as tools for teachers to understand and support students in their conceptualization of the flow of energy and matter within ecosystems. The MBA was piloted with four middle school students (n = 4) and implemented in two sixth grade student cohorts (n = 89 & n = 98). The MBA and rubrics went through several design iterations in order to best capture student understanding of complex systems. The design of the MBA allows students to express conceptual understanding while also capturing the transformation of their understanding as they are exposed to new information and experiences within the curricular content. [ABSTRACT FROM AUTHOR]

Rubrics, which are designed to inform learners about assessment criteria and quality levels concerning a task assignment, are a widely used means to support learners in self-assessing their task performance. Usually, rubrics enhance task performance. Surprisingly, however, relatively little is known about the underlying mechanisms via which rubrics exert their beneficial effects. Although it is frequently suggested that the effects of rubrics on task performance are due to the fact that they support learners in accurately self-assessing their level of performance, which, in turn, paves the way for effective regulation, the empirical evidence that supports this notion is relatively scarce. Tightly controlled experimental studies, which allow isolating the effects of rubrics on the accuracy of self-assessments, are scarce. The present study was designed to close this gap. Specifically, in an experiment with N = 93 high school students we tested whether providing learners with a rubric in self-assessing the quality of self-written scientific abstracts would enhance judgment accuracy. As main results, we found that the rubric group showed higher absolute accuracy and lower bias. Furthermore, the rubric reduced cognitive load in self-assessing. These findings support the notion that one mechanism via which rubrics unfold their efficacy is by enhancing judgment accuracy. [ABSTRACT FROM AUTHOR]

This article investigates the feasibility of using automated scoring methods to evaluate the quality of student-written essays. In 2012, Kaggle hosted an Automated Student Assessment Prize contest to find effective solutions to automated testing and grading. This article: a) analyzes the datasets from the contest – which contained hand-graded essays – to measure their suitability for developing competent automated grading tools; b) evaluates the potential for deep learning in automated essay scoring (AES) to produce sophisticated testing and grading algorithms; c) advocates for thorough and transparent performance reports on AES research, which will facilitate fairer comparisons among various AES systems and permit study replication; d) uses both deep neural networks and state-of-the-art NLP tools to predict finer-grained rubric scores, to illustrate how rubric scores are determined from a linguistic perspective, and to uncover important features of an effective rubric scoring model. This study's findings first highlight the level of agreement that exists between two human raters for each rubric as captured in the investigated essay dataset, that is, 0.60 on average as measured by the quadratic weighted kappa (QWK). Only one related study has been found in the literature which also performed rubric score predictions through models trained on the same dataset. At best, the predictive models had an average agreement level (QWK) of 0.53 with the human raters, below the level of agreement among human raters. In contrast, this research's findings report an average agreement level per rubric with the two human raters' resolved scores of 0.72 (QWK), well beyond the agreement level between the two human raters. Further, the AES system proposed in this article predicts holistic essay scores through its predicted rubric scores and produces a QWK of 0.78, a competitive performance according to recent literature where cutting-edge AES tools generate agreement levels between 0.77 and 0.81, results computed as per the same procedure as in this article. This study's AES system goes one step further toward interpretability and the provision of high-level explanations to justify the predicted holistic and rubric scores. It contends that predicting rubric scores is essential to automated essay scoring, because it reveals the reasoning behind AIED-based AES systems. Will building AIED accountability improve the trustworthiness of the formative feedback generated by AES? Will AIED-empowered AES systems thoroughly mimic, or even outperform, a competent human rater? Will such machine-grading systems be subjected to verification by human raters, thus paving the way for a human-in-the-loop assessment mechanism? Will trust in new generations of AES systems be improved with the addition of models that explain the inner workings of a deep learning black box? This study seeks to expand these horizons of AES to make the technique practical, explainable, and trustable. [ABSTRACT FROM AUTHOR]

When determining the score given to candidates in multiple mini-interview (MMI) stations, raters have to translate a narrative judgment to an ordinal rating scale. When adding individual scores to calculate final ranking, it is generally presumed that the values of possible scores on the evaluation grid are separated by constant intervals, following a linear function, although this assumption is seldom validated with raters themselves. Inaccurate interval values could lead to systemic bias that could potentially distort candidates' final cumulative scores. The aim of this study was to establish rating scale values based on rater's intent, to validate these with an independent quantitative method, to explore their impact on final score, and to appraise their meaning according to experienced MMI interviewers. A 4-round consensus-group exercise was independently conducted with 42 MMI interviewers who were asked to determine relative values for the 6-point rating scale (from A to F) used in the Canadian integrated French MMI (IFMMI). In parallel, relative values were also calculated for each option of the scale by comparing the average scores concurrently given to the same individual in other stations every time that option was selected during three consecutive IFMMI years. Data from the same three cohorts was used to simulate the impact of using new score values on final rankings. Comments from the consensus group exercise were reviewed independently by two authors to explore raters' rationale for choosing specific values. Relative to the maximum (A = 100%) and minimum (F = 0%), experienced raters concluded to values of 86.7% (95% CI 86.3–87.1), 69.5% (68.9–70.1), 51.2% (50.6–51.8), and 29.3% (28.1–30.5), for scores of B, C, D and E respectively. The concurrent score approach was based on 43,412 IFMMI stations performed by 4345 medical school applicants. It provided quasi-identical values of 87.1% (82.4–91.5), 70.4% (66.1–74.7), 51.2% (47.1–55.3) and 31.8% (27.9–35.7), respectively. Qualitative analysis explained that while high scores are usually based on minor details of relatively low importance, low scores are usually attributed for more serious offenses and were assumed by the raters to carry more weight in the final score. Individual drop or increase in final MMI ranking with the use of new scale values ranged from − 21 to + 5 percentiles, with the average candidate changing by ± 1.4 percentiles. Consulting with experienced interviewers is a simple and effective approach to establish rating scale values that truly reflects raters' intent in MMI, thus improving the accuracy of the instrument and contributing to the general fairness of the process. [ABSTRACT FROM AUTHOR]

Background: Process skills such as critical thinking and information processing are commonly stated outcomes for STEM undergraduate degree programs, but instructors often do not explicitly assess these skills in their courses. Students are more likely to develop these crucial skills if there is constructive alignment between an instructor's intended learning outcomes, the tasks that the instructor and students perform, and the assessment tools that the instructor uses. Rubrics for each process skill can enhance this alignment by creating a shared understanding of process skills between instructors and students. Rubrics can also enable instructors to reflect on their teaching practices with regard to developing their students' process skills and facilitating feedback to students to identify areas for improvement. Results: Here, we provide rubrics that can be used to assess critical thinking and information processing in STEM undergraduate classrooms and to provide students with formative feedback. As part of the Enhancing Learning by Improving Process Skills in STEM (ELIPSS) Project, rubrics were developed to assess these two skills in STEM undergraduate students' written work. The rubrics were implemented in multiple STEM disciplines, class sizes, course levels, and institution types to ensure they were practical for everyday classroom use. Instructors reported via surveys that the rubrics supported assessment of students' written work in multiple STEM learning environments. Graduate teaching assistants also indicated that they could effectively use the rubrics to assess student work and that the rubrics clarified the instructor's expectations for how they should assess students. Students reported that they understood the content of the rubrics and could use the feedback provided by the rubric to change their future performance. Conclusion: The ELIPSS rubrics allowed instructors to explicitly assess the critical thinking and information processing skills that they wanted their students to develop in their courses. The instructors were able to clarify their expectations for both their teaching assistants and students and provide consistent feedback to students about their performance. Supporting the adoption of active-learning pedagogies should also include changes to assessment strategies to measure the skills that are developed as students engage in more meaningful learning experiences. Tools such as the ELIPSS rubrics provide a resource for instructors to better align assessments with intended learning outcomes. [ABSTRACT FROM AUTHOR]

For learners, it can be difficult to imagine how to perform a complex skill based on textual information from solely a text-based analytic rubric. Rubrics lack (1) the contextual information needed to convey real-world attributes (2) the dynamic information (such as gesturing in the complex skill of presenting) (3) the procedural information required to support the automation of constituent skills. We propose to address the text-based rubric's deficiencies by adding video-modeling examples, self-explanation prompts, an intertwined educational- and instructional narrative, natural segmentation, and a non-verbal script. With the resulting Video Enhanced Rubric, we aim to improve the formative assessment of complex skills by fostering learner's mental model development, feedback quality, and complex skill mastery. Designing multimedia to support the formative assessment of complex skills using can cause dilemmas for instructional designers. For example, is learner control needed to foster intrinsic motivation or is it deemed to create extraneous cognitive load? Is it wise to use a video modeling example of peer-aged learners when the model does not display perfect performance? We found seven dilemmas around proven complex skill development, motivational design, and multimedia design guidelines. This paper presents a theoretical contribution to instructional design by introducing a framework to address dilemmas regarding such design dilemmas. As a practical contribution, we support educational researchers and practitioners by presenting six practical guidelines for designing a Video Enhanced Rubric. A use case of the Viewbrics-project provides insight into the practical application of the framework within the context of Dutch pre-university education. [ABSTRACT FROM AUTHOR]

The recent literature has shown the importance of preservice elementary science teachers (PESTs) having a deep understanding of argumentation, as this factor may affect the nature of the class activities that are taught and what students learn. A lack of understanding of this factor may represent an obstacle in the development of science education programmes in line with the development of scientific competences. This paper presents the results of the design and implementation of a training programme of 6 sessions (12 h of class participation plus 8 h of personal homework) on argumentation. The programme was carried out by 57 Spanish PESTs from Malaga, Spain. The training programme incorporates the innovative use of certain strategies to improve competence in argumentation, such as teaching PESTs to identify the elements of arguments in order to design assessment rubrics or by including peer assessment during evaluation with and without rubrics. The results obtained on implementing the training programme were evaluated based on the development of PESTs’ argumentation competence using Toulmin’s argumentative model. Data collection methods involved two tasks carried out at the beginning and the end of the programme, i.e., pre- and post-test, respectively. The conclusion of the study is that students made significant progress in their argumentation competence on completing the course. In addition, PESTs who followed the training programme achieved statistically better results at the end than those in the control group (n = 41), who followed a traditional teaching programme. A 6-month transfer task showed a slight improvement for the PESTs of the experimental group in relation to the control group in their ability to transfer argumentation to practice. [ABSTRACT FROM AUTHOR]

Rubrics are assessment tools that help students gain complex competencies. Our quasi-experimental study aimed to evaluate whether rubrics help teachers teach and assess mathematical reasoning in primary school and whether such an instrument might support student learning. In two Swiss cantons, 762 students in 44 5th- and 6th-grade primary classes worked on their reasoning competencies, and half of them additionally employed our standards-based rubric. All of the teachers received a 1-day training and participated in the final project evaluation. To standardise and support the teachers during the implementation phase, they received a detailed curriculum. An achievement test and questionnaires for students and teachers were administered before and at the end of the intervention. The results of our quantitative longitudinal analyses indicate that the rubric fosters the teachers' perceived diagnostic skills but only indirectly impacts their use of formative feedback. Based on the students' perceptions, however, we observed a direct effect of the rubric on formative feedback and student self-assessment. Effects on students' outcomes could not be observed, but there are indications of effects mediated by self-regulation and self-efficacy. [ABSTRACT FROM AUTHOR]

An increased need for self-paced, online professional development opportunities in higher education has emerged from a variety of factors including dispersed geographic locations of faculty, full teaching loads, and institutional evaluation requirements. This article is a report of the examination of the design and evaluation of a self-paced online faculty development course about rubrics. A mixed-methods research design was implemented to collect quantitative and qualitative data from faculty members upon completion. Data obtained from an evaluation questionnaire, rubric artifact, self-reflections, and teaching improvement plan revealed that faculty viewed the course as a positive and useful experience. Specific design recommendations for instructional designers and faculty developers of self-paced online courses are provided. These design recommendations emphasize adult learning principles and transformative learning through self-reflection. [ABSTRACT FROM AUTHOR]

With the Standards of Training, Certification and Watchkeeping Convention 1995 (STCW'95) moving seafarer training towards outcome-based education (OBE), emphasis has shifted to assessment practices that will allow seafarer students to demonstrate their ability to perform workplace tasks at standards described in the STCW Code. This paper argues that authentic assessment comprising of performance-based tasks applied in real-world and meaningful contexts can provide a holistic approach to competence assessment for seafarers. But, authentic assessment can capture essential aspects of workplace tasks and result in consistency of student performance in different contexts only if they are valid and reliable. Rubrics as assessment tools are known to increase validity and reliability of assessments; however, it can do so only if different aspects of its own validity and reliability have been addressed. A literature review undertaken for this paper has uncovered an absence of academic investigation and empirical study on the different aspects of validity and reliability of authentic assessment through assessment rubrics. Moreover, there exists an even greater absence of global research on authentic assessment in the area of seafarer training. Through an investigation of authentic assessment, this research has uncovered the importance of using valid and reliable rubrics in order to improve not only the assessment process but also the tools and methods used to support the valid, reliable, and authentic assessment of outcomes achieved in the learning process. Future research aims to offer insights into improving the validity and reliability of rubrics and to empirically investigate how they can be used in authentic assessment within the confines of the STCW Code, in particular, to improve seafarer training practices, student engagement, resulting learning outcomes, and employer and regulator satisfaction with the attainment of the standards stipulated in the STCW Code to produce an evidence of competence. [ABSTRACT FROM AUTHOR]

Recently-adopted standards call for more emphasis on writing in early elementary grades. Rubrics may assist students in attending to important characteristics of effective writing, but research data on their use in early childhood is lacking. This study explores the effects of rubric use on writing instruction of opinion paragraphs for 20 first grade students (9 female, 11 male; 11 Black, 4 White, 4 Hispanic, 1 Asian; 8 English Language Learners) and 12 second grade students (7 male, 5 female; 6 Black, 2 Hispanic, 2 Middle Eastern, 1 Asian, 1 White; 0 English Language Learners) from a low socio-economic Midwestern United States elementary school that was designated a School in Need of Assistance. The stratified-grouped, counterbalanced pretest-midpoint test-posttest study determined if providing a rubric and instructing students on how to use it, in addition to teaching writing mini-lessons regarding specific areas of the rubric, would improve student writing scores compared to the mini lessons alone. Also, data were collected on student attitudes throughout the study. The results indicated that providing instruction on the skills outlined by the rubric, paired with student access to the rubric, improved student writing scores with a large effect size (Cohen's d = 0.92). Although student attitude ratings did not show a significant difference between conditions, student-supplied reasons for those ratings indicated that initial use of a rubric promoted later positive attitudes toward writing and higher self-efficacy while using the rubric. The authors recommend rubric use in writing to assist students in remembering and self-monitoring important components of writing. [ABSTRACT FROM AUTHOR]

Internationally, attempts at developing explicit descriptions of academic achievement standards have been steadily intensifying. The aim has been to capture the essence of the standards in words, symbols or diagrams (collectively referred to as codifications) so that standards can be: set and maintained at appropriate levels; made broadly comparable in different specified contexts; and generally shared and understood better by assessors, academic program directors, students, employers, quality assurance agencies and the public at large. The scale of this practice ranges from rubrics for single assessment tasks to national standards statements used as academic performance benchmarks for graduates from academic programs. A critical analysis shows that the underlying assumptions of this process are fundamentally flawed. Codifications are inherently incapable of meeting the requirements because key terms lack the necessary attributes. A fundamentally different material form of representation is therefore necessary if the original intentions are to be realised. [ABSTRACT FROM AUTHOR]

The success of research in integrated environmental and natural resource management relies on the participation and involvement of different disciplines and stakeholders. This can be difficult to achieve in practice because many initiatives fail to address the underlying social processes required for successful engagement and social learning. We used an action research approach to support a research-based group with a range of disciplinary and stakeholder expertise to critically reflect on their engagement practice and identify lessons around how to collaborate more effectively. This approach is provided here as a guide that can be used to support reflective research practice for engagement in other integration-based initiatives. This paper is set in the context of an integrated wildlife management research case study in New Zealand. We illustrate how multi-, inter- and trans-disciplinary approaches can provide a framework for considering the different conversations that need to occur in an integrated research program. We then outline rubrics that list the criteria required in inter- and trans-disciplinary collaborations, along with examples of effective engagement processes that directly support integration through such efforts. Finally, we discuss the implications of these experiences for other researchers and managers seeking to improve engagement and collaboration in integrated science, management and policy initiatives. Our experiences reaffirm the need for those involved in integrative initiatives to attend to the processes of engagement in both formal and informal settings, to provide opportunities for critical reflective practice, and to look for measures of success that acknowledge the importance of effective social process. [ABSTRACT FROM AUTHOR]