Exploring users’ appropriateness as a proxy for experts when screening new product/service ideas

Despite the popularity and extensive use of engaging users in crowdvoting, very little research has been conducted into the appropriateness of users as substitutes for experts when judging ideas. Accordingly, the purpose of this paper is to investigate the appropriateness of using users as a proxy for professional experts during the initial idea screening of new product/service ideas. In other words, are users' assessments during idea screening conformant with professional experts' assessments and are they reliable as a proxy for experts during idea screening? In a comparative study, two different approaches to outsourcing the screening of wireless ideas to users are examined, including assessment by two different user panels: (1) technically skilled users and (2) technically naïve users. These two approaches were compared with the assessments made by professional experts. The results showed no conformance between users and experts when comparing their absolute scores. However, during a relative comparison (the ranking of ideas), both user panels were conformant with the professional experts. A test of the user panels' ability to select the same top ideas as the professional experts was successful, indicating good conformance between the user panels and the professional experts. This paper's contribution is knowledge of how conformant external users are compared with professional expert judges during idea screening. The results indicate that companies can employ users during the initial screening process using criteria assessment to select the best ideas for further elaboration, something that would significantly reduce the number of ideas. The paper suggests an alternative design to crowdvoting, whereby the users assess the relevant criteria.