1. Issues and Red Herrings in Evaluating Natural Language Interfaces
- Author
-
Ralph Weischedel
- Subjects
Point (typography) ,business.industry ,Computer science ,Analytic hierarchy process ,computer.software_genre ,Data science ,Expert system ,Broad spectrum ,Computer software ,Position paper ,Artificial intelligence ,Computational linguistics ,business ,computer ,Natural language - Abstract
Due to growing interest in using natural language interfaces, it is appropriate to begin discussing their evaluation. This position paper presents the point of view that natural language interfaces offer special problems for evaluation, compared to problems encountered with other computer software. If ignored, such problems will make evaluation ineffectual. First, the paper describes the purposes and dimensions of problems in such an evaluation; differing purposes dictate differing approaches to evaluating natural language interfaces. Then, the paper points out some issues that turn out to be red herrings, and suggests a modest proposal that covers much of the broad spectrum of difficult issues involved. Since some of the issues and problems are common to both expert systems and natural language interfaces, comparisons of issues in evaluation of expert systems is also provided.
- Published
- 1987