Search

The electrovibration tactile display could render the tactile feeling of different textured surfaces by generating the frictional force through voltage modulation. When a user is sliding his/her finger on the display surface, he/she can feel the frictional texture. However, it is not trivial to prepare and fine-tune the appropriate frictional signals for haptic design and texture simulation. In this paper, we present a deep-learning-based framework to generate the frictional signals from the textured images of fabric materials. The generated frictional signal can be used for the tactile rendering on the electrovibration tactile display. Leveraging GANs (Generative Adversarial Networks), our system could generate the displacement-based data of frictional coefficients for the tactile display to simulate the tactile feedback of different fabric materials. Our experimental results show that the proposed generative model could generate the frictional-coefficient signals visually and statistically close to the ground-truth signals. The following user studies on fabric-texture simulation show that users could not discriminate the generated and the ground-truth frictional signals being rendered on the electrovibration tactile display, suggesting the effectiveness of our deep-frictional-signal-generation model. [Display omitted] • The deep-learning-based image-to-friction generation framework for tactile simulation of fabric material. • The augmented visual-to-frictional database based on HapTex for image-to-friction generation. • The technical experiment of frictional-coefficient signal generation evidencing the performance of the proposed generative model. • The user-perception experiment validating the effectiveness of generated signals for tactile simulation of fabrics on the electrovibration tactile display. [ABSTRACT FROM AUTHOR]

• We construct a lexical circuit-event database for modeling novice users' natural-language behaviors on physical-computing system design and development. • For proof of concept, we develop FrtizBot, a BiLSTM-CRF-based system leveraging natural-language interaction for designing and creating Arduino based physical-computing systems. • We conduct a user study showing the effectiveness of FritzBot on sup1 porting novice users' physical-computing task through natural-language interaction. Creating physical-computing systems, especially selecting correct electronic components, assembling the circuit, and implementing the program, can be challenging for novice users. In this paper, we present FritzBot, a data-driven conversational agent supporting novice users on creating physical-computing systems through natural-language interaction. FritzBot is built upon the structure of a BiLSTM-CRF (bi-directional Long Short-term Memory Network and Conditional Random Field) neural network, as a plug-in for Fritzing. The neural network is trained on a lexical circuit-event database derived from 152 students' reports on their physical-computing course projects. By processing the user's textual description on his/her physical-computing idea, FritzBot can extract the causal relationships between the input and the output events, identify the corresponding electronic components, and generate the Arduino-based circuit and the code along with the step-by-step construction guidelines. Our user study shows that compared to the original Arduino software and the circuit-autocompletion software available in the commercial market, FritzBot significantly shortens the time spent, reduces the perceived workload, and enhances the satisfaction/joy for inexperienced users on designing and prototyping physical-computing systems. [ABSTRACT FROM AUTHOR]