Search

PurposePeer reviewer evaluations of academic papers are known to be variable in content and overall judgements but are important academic publishing safeguards. This article introduces a sentiment analysis program, PeerJudge, to detect praise and criticism in peer evaluations. It is designed to support editorial management decisions and reviewers in the scholarly publishing process and for grant funding decision workflows. The initial version of PeerJudge is tailored for reviews from F1000Research's open peer review publishing platform.Design/methodology/approachPeerJudge uses a lexical sentiment analysis approach with a human-coded initial sentiment lexicon and machine learning adjustments and additions. It was built with an F1000Research development corpus and evaluated on a different F1000Research test corpus using reviewer ratings.FindingsPeerJudge can predict F1000Research judgements from negative evaluations in reviewers' comments more accurately than baseline approaches, although not from positive reviewer comments, which seem to be largely unrelated to reviewer decisions. Within the F1000Research mode of post-publication peer review, the absence of any detected negative comments is a reliable indicator that an article will be ‘approved’, but the presence of moderately negative comments could lead to either an approved or approved with reservations decision.Originality/valuePeerJudge is the first transparent AI approach to peer review sentiment detection. It may be used to identify anomalous reviews with text potentially not matching judgements for individual checks or systematic bias assessments.

Purpose Online reviews are increasingly available for a wide range of products and services in e-commerce. Most consumers rely heavily on online reviews when making purchase decisions, so an important topic is that of understanding what makes some online reviews helpful in the eyes of consumers. Researchers have demonstrated the benefits of the presence of customer reviews to an online retailer, however, few studies have investigated how images in review content and the facial expressions of reviewers’ avatars influence the judgment of online review helpfulness. This study draws on self-construal theory, attribution theory and affect-as-information theory to empirically test a model of the interaction effects of images in review content and the facial expressions of reviewers’ avatars on online review helpfulness. Furthermore, the purpose of this paper is to identify an underlying mechanism of causal attribution toward store performance on the above effects. Design/methodology/approach This study conducted two online experiments. Study 1 is a 2 (images in review content: one person with a product vs a group of people with a product) ×2 (facial expression of the reviewer’s avatar: happy vs angry) between-subjects design. Study 2 is a 3 (image: product alone vs one person with a product vs a group of people with a product) ×2 (facial expression of the reviewer’s avatar: happy vs angry) ×3 (valence of the review: positive vs negative vs neutral) between-subjects design. Findings The results indicate that when consumers were exposed to a happy-looking avatar, they were likely to express higher perceptions of online review helpfulness in response to an image showing a group of people in a restaurant than they would for an image of one person in the same situation. In contrast, when consumers were exposed to an angry-looking avatar, their perceptions of online review helpfulness did not differ in response to images of either a group of people or of one person. Furthermore, cause attribution toward store performance mediated the interaction between images in content of reviews and the facial expression of a reviewer’s avatar on the perceptions of online review helpfulness. Practical implications The authors provide insights into how to develop guidelines on how online reviews should be written so that readers perceive them to be helpful, and how to design effective reward mechanisms for customer feedback. Originality/value Compared with previous studies, this study provides further contributions in three ways. First, it contributes to the literature on review content by showing which images in reviews are deemed to be helpful. Second, it extends previous findings from the literature relating to online peer reviews by demonstrating the importance of facial expressions in reviewers’ avatars (i.e. happy vs angry) when explaining helpfulness, rather than the strength of purchase intent. Third, this study contributes by further highlighting a novel mechanism which shows that a causal attribution toward store performance motivates the perceptions of online review helpfulness.