Search

The process of developing a publication guide titled “How to write your first academic paper” was presented in this paper. A detailed qualitative approach was followed to extract key elements from transaction and conference papers that appeared in the IEEE Transactions of Information Theory during the past decade. The approach taken was to learn from the experts, in this case IEEE Fellows and distill it into a step-by-step guide, conforming to a set of design principles. The publication guide was implemented at the Faculty of Engineering and many positive results achieved.

Peer review expert selection is an important link of fund project review; it can not only improve the matching accuracy of domain experts and projects, but also improve the appraisal quality in fund project review. This paper mainly studies the peer review expert selection method in fund project review. With the actual experience of fund project management, based on the author's knowledge set theory, paper analyzes deeply knowledge set representation of expert knowledge, improves the similarity calculation of expert and project knowledge, builds the peer review expert selection method, and gives the actual example.

In this paper we present a study where a peer reviewing system targeted for conference paper evaluation has been used for enhancing the learning of programming though peer review. For peer reviewing purposes two open source systems were evaluated and MyReview was selected for evaluation. The peer review system was used in one programming focused course and the benefits of peer reviewing were analyzed from the reviewer and reviewee point of view. The results show that a peer review system targeted for conference paper evaluations is applicable for reviewing programs although the system should be much simpler. Students appreciated the documented comments from other students but were more interested in lecturer or other expert opinion.

The review cycle for papers takes way too long in many disciplines. The problem is that while authors want to have their own papers reviewed fast, that are often unwilling to review the papers of others in a timely manner. This paper explores what would be required to incentivize fast reviews using a public reputation/scoring system that exploits the fact that the referees are drawn from the same pool as paper authors. The challenge in maintaining a public reputation system is to ensure that the identity of referees remain as anonymous as possible. A model is proposed in this work, wherein authors have an incentive to commit to reviewing papers and are rewarded for meeting this commitment in a manner that prioritizes their own papers for reviews. This ensures stability (bounded reviewing delays) for all fair contributors while freeloaders face a potentially unstable system. A naive implementation of the scoring system, however, leaks information that would allow authors to infer the likely identities of their referees. A distortion to the observed public score process is then studied, which is shown to enhance anonymity while preserving the incentives for timely refereeing.

This paper proposes that experience knowledge would be beneficial for iterative software development. In this paper, experience knowledge- based artifacts have been linked to Extreme Programming via peer reviews. Experience knowledge can be defined as human experiences documented with the object of exploiting them in the organization. Extreme Programming has been selected to represent iterative software development processes. Systematically gathering experiences would support building evidence for improving and teaching iterative software development processes. This paper presents one possible solution as to how to put experience knowledge in practice in the context of iterative software development processes. It also reports results from projects where experience knowledge has been studied. The main findings of this paper are two ways in which experience knowledge- based artifacts can be used with iterative software development, and that peer reviews are a good way to collect and treat experience knowledge-based artifacts.

Peer assessment is widely used in educational settings as an alternative evaluation approach. It brings various benefits to the students, increasing engagement and interactivity and fostering critical thinking and reflection. Several platforms for managing the peer review process have been proposed in the literature, but most of them are confined to a particular domain or course and have various limitations related to reliability, reviewer allocation, reputation and training of reviewers or instructor support. In an attempt to address these challenges, we propose an innovative general-purpose peer assessment platform, called LearnEval. In this paper we focus on the student module part of the system, describing its functionalities, pedagogical rationale and implementation details.

Work in Progress: This paper reports the study on how peer review instruments based on existing e-learning tools can affect the quality of peer review process and how this type of tools can benefit the reviewers. The study starts from a pilot study on constructing peer assessment instruments for an assignment in a Master course by using PeerMark and Moodle workshop. A comparison of these two tools has been given. Moodle workshop is more robotic, provide more options, but not very easy to use. Turnitin PeerMark is easy to use, more effective, but provides less options. Moodle workshop seems to have the potentials to facilitate guidance to the reviewers, feedback to the reviewers, peer interaction and knowledge construction better. A combination of the features of both tools should make a better peer assessment tool. Further investigation is required on how these tools can be improved or combined to organize a complex peer assessment marking criteria properly and to benefit the reviewers better. A plan for the next step research has been given, including further evaluation, software implementation and formal evaluation framework development.

This paper attempted to have students assess the presentation component of “real world” information systems case studies by fellow students and present the results of the peer assessment. The results could be of help to those who consider using peer assessment on presentations of “real world” information systems case studies by students.

Peer code review has been adopted as an effective quality improvement practice by many Open Source Software (OSS) communities. In addition to increasing software quality, there is anecdotal evidence that peer code review has other benefits, including: sharing knowledge, sharing expertise, sharing development techniques, and most importantly building accurate peer impressions between the code review participants. To further investigate the presence of these benefits, we surveyed members of popular OSS communities who were involved with peer code review. We used established scales from Psychology, Information science, and Organizational Behavior to create survey questions. We also enforced multiple reliability and validity measures to ensure higher confidence in the survey results. In this paper, we present a subset of the surveys results focused on better understanding four aspects of peer impression formation: trust, reliability, perception of expertise, and friendship. The results indicate that there is indeed a high level of trust, reliability, perception of expertise, and friendship between OSS peers who have participated in code review for a period of time. Because code review involves examining someone else's code, unsurprisingly, peer code review helped most in building a perception of expertise between code review partners.

Since 2002, students in a first-year engineering course at Purdue University have participated in several Model-Eliciting Activities (MEAs). MEAs are realistic, open-ended, client-driven engineering problems designed to foster students' mathematical modeling abilities. The primary artifact produced by each team is a memo to the client describing a procedure for solving the engineering problem. Since 2007, three cycles of feedback and revision have been used in the course, with students receiving reviews from peers and teaching assistants (TAs) throughout the process. Prior to 2009, students received the first round of feedback from TAs, and the second from peers. After 2009, this order was reversed, and students received the first round of feedback from peers, and the second from TAs. This paper investigates whether the amount of change differed based on whether reviews were provided by TAs or by peers. Using the Levenshtein distance [1], the amount of change between consecutive drafts was calculated for four offerings of two different MEAs. Each MEA was offered with both the pre- and post-2009 sequencing, allowing a clear comparison of sequencing methods. Results from this study indicate that on the first revision, the amount of change resulting from TA reviews was indistinguishable from that resulting from peer reviews. However, on the second revision, the amount of change resulting from TA reviews was significantly different (greater) than the change resulting from peer reviews. Overall, the amount of change resulting from feedback was greater when peer reviews were provided first, and TA reviews were provided second. These results suggest that review expertise is more critical in the later stages of the process of solving an MEA.

Open source is an important model of collaborative knowledge work and virtual organizations. One of its work practices, peer review, is considered critical to its success, as Linus's law highlights. Thus, understanding open source peer review, particular effective review practices, will improve the understanding of how to support collaborative work in new ways. Therefore, we conduct case studies in two open source communities that are well recognized as effective and successful, Mozilla and Python. In this paper, we present the preliminary results of our analysis on data from the bug tracking systems of those two organizations. We identify four common activities critical to open source software peer review, submission, identification, resolution and evaluation. Differences between communities indicate factors, such as reporter expertise, product type and structure, and organization size, affect review activities. We also discuss features of open source software peer review distinct from traditional review, as well as reconsiderations of Linus's law.

Teacher works as evaluator in his day by day, and very often, this job is based on exams correction. Traditional exams have been fundamental, but present a problem: students are not connected with evaluation process. Otherwise, peer to peer evaluation is based on the role of students as evaluators: they have to know and assume evaluation criteria and applying them on other exams. With this, students are able to understand better the contents, and develop skills to qualify his partners. Then, teacher could define evaluation grades: numeric, byelements presence, by rubrics... And at last, only has to define a schedule for leaving works, evaluating and to generate the final qualification. This paper shows experiences using peer to peer evaluation across Moodie platform on two academic courses.

With the advent of the web, more and more information systems are being built web-centric, and more recently, these systems are leveraging the advantages of online community practice as well. This paper describes a lightweight community-based software code review web solution that allows a user to easily upload source code, review, and vote on peer reviews and reviewers.

Post-catastrophic recovery is gaining more importance to human societies. This paper proposes use of a distributed autonomous economic medium to support recovery of communities after catastrophic events. It proposes a model in which everyone in the world can help each other as peers, instead of creating one's dependencies on others. The model applies peer-to-peer technologies on the Internet to achieve an environment for mutual help in the planetary scale

North Dakota State University has recently created the peer review of teaching (PRT) program which seeks to promote student-centered learning through the use of cooperative peer review teams to promote enhanced teaching methods, techniques, and strategies. The PRT project is a faculty-driven initiative intended to offer individual faculty added feedback related to instruction. Faculty members work together to set goals and to interpret student reactions to instruction strategies. The process is intentionally limited to formative assessment, and the peer assessment data is owned by the individual faculty members. Project participants are required to observe the teaching materials and teaching activity of a peer for at least one class per semester, provide meaningful feedback to his or her peer related to his or her syllabus and teaching strategies, and provide a measure of evaluation and assessment related to enhanced student learning. The contents of this paper document: 1) the background and development of the PRT program, 2) the selection and coordination of the peer teams, 3) the development of the peer philosophy and project deliverables, and 4) an evaluation and assessment of the PRT program.