A Situation-Aware and Interactive System for Assisting People Fill out Paper Forms

People often need help when filling out paper forms, because they do not fully understand the meaning of form fields. Commonly adopted solutions are referring to the form filling instructions or consulting other people. However, they are either inefficient or inconvenient. In this paper, we propose a situation-aware and interactive system, named Interact Form, to help people fill out paper forms. First, it provides an easy-to-use tool to build the knowledge offline about any given form. The knowledge includes the instruction and examples of each form field, and the constraints between fields. Then, when people are filling out a paper form, with the equipped video camera, the system can determine the field-grained position of the pen and provide text or audio instructions based on the user's form filling situations. When a pre-defined constraint is possibly going to be violated, the provided pen will send out an alert by vibration, and an explanation of the alert will also be given at the same time. We evaluate Interact Form with 60 paper form filling activities on 10 real-world paper forms, and the results show the usability of the system.