Search

Many technologies have been developed in recent years to present audiovisual information in new ways, but developing an information presentation interface to convey tactile information is still a challenge. We propose a tactile device using wearable technology that is an all-around pressure presentation system using pneumatic actuators. Specifically, we develop a system in which a choker equipped with a pneumatic actuator is worn around the neck, that applies pressure in any direction to indicate to the user the direction in which to walk and also when to start and stop walking. In this paper, we describe the construction of the device, evaluation experiments, our assessment of the prototype, and future plans for the device., Comment: Proceedings of AH 2021. 4 pages, 7 figures

Shape-changing interfaces are promising for users to change the physical properties of common objects. However, prevailing approaches of actuation devices require either professional equipment or materials that are not commonly accessible to non-professional users. In this work, we focus on the controllable soft actuators with inflatable structures because they are soft thus safe for human computer interaction. We propose a soft actuator design, called xBalloon, that is workable, inexpensive and easy-to-fabricate. It consists of daily materials including balloons and plastics and can realize bending actuation very effectively. For characterization, we fabricated xBalloon samples with different geometrical parameters and tested them regarding the bending performance and found the analytical model describing the relationship between the shape and the bending width. We then used xBalloons to animate a series of common objects and all can work satisfactorily. We further verified the user experience about the the fabrication and found that even those with no prior robotic knowledge can fabricate xBalloons with ease and confidence. Given all these advantages, we believe that xBalloon is an ideal platform for interaction design and entertainment applications., Comment: Proceedings of AH 2021.6 pages, 10 figures

Due to the influence of the new coronavirus disease (COVID-19), teleworking has been expanding rapidly. Although existing interactive remote working systems are convenient, they do not allow users to adjust their spatial distance to team members at will, %"Arbitrarily" is probably not the best word here. It means without apparent reason. A better expression might be "at will." and they ignore the discomfort caused by different levels of intimacy. To solve this issue, we propose a telework support system using spatial augmented reality technology. This system calibrates the space in which videos are projected with real space and adjusts the spatial distance between users by changing the position of projections. Users can switch the projection position of the video using hand-wave gestures. We also synchronize audio according to distance to further emphasize the sense of space within the remote interaction: the distance between projection position and user is inversely proportional to the audio volume. We conducted a telework experiment and a questionnaire survey to evaluate our system. The results show that the system enables users to adjust distance according to intimacy and thus improve the users' comfort., Comment: Proceedings of HCII 2021. 12 pages, 6 figures

In this work, we propose an interactive general instruction framework SketchMeHow to guidance the common users to complete the daily tasks in real-time. In contrast to the conventional augmented reality-based instruction systems, the proposed framework utilizes the user sketches as system inputs to acquire the users' production intentions from the drawing interfaces. Given the user sketches, the designated task instruction can be analyzed based on the sub-task division and spatial localization for each task. The projector-camera system is adopted in the projection guidance to the end-users with the spatial augmented reality technology. To verify the proposed framework, we conducted two case studies of domino arrangement and bento production. From our user studies, the proposed systems can help novice users complete the tasks efficiently with user satisfaction. We believe the proposed SketchMeHow can broaden the research topics in sketch-based real-world applications in human-computer interaction., Comment: Proceedings of HCII2021. 16 pages, 11 figures

This work proposes a compact robotic limb, AugLimb, that can augment our body functions and support the daily activities. AugLimb adopts the double-layer scissor unit for the extendable mechanism which can achieve 2.5 times longer than the forearm length. The proposed device can be mounted on the user's upper arm, and transform into compact state without obstruction to wearers. The proposed device is lightweight with low burden exerted on the wearer. We developed the prototype of AugLimb to demonstrate the proposed mechanisms. We believe that the design methodology of AugLimb can facilitate human augmentation research for practical use. see http://www.jaist.ac.jp/~xie/auglimb.html, Comment: 2 pages, 3 figures

Constructing stroke correspondences between keyframes is one of the most important processes in the production pipeline of hand-drawn inbetweening frames. This process requires time-consuming manual work imposing a tremendous burden on the animators. We propose a method to estimate stroke correspondences between raster character images (keyframes) without vectorization processes. First, the proposed system separates the closed areas in each keyframe and estimates the correspondences between closed areas by using the characteristics of shape, depth, and closed area connection. Second, the proposed system estimates stroke correspondences from the estimated closed area correspondences. We demonstrate the effectiveness of our method by performing a user study and comparing the proposed system with conventional approaches., Comment: Proceedings of NICOGRAPH International 2021. 9 pages, 11 figures

Along the rapid development of deep learning techniques in generative models, it is becoming an urgent issue to combine machine intelligence with human intelligence to solve the practical applications. Motivated by this methodology, this work aims to adjust the machine generated character fonts with the effort of human workers in the perception study. Although numerous fonts are available online for public usage, it is difficult and challenging to generate and explore a font to meet the preferences for common users. To solve the specific issue, we propose the perceptual manifold of fonts to visualize the perceptual adjustment in the latent space of a generative model of fonts. In our framework, we adopt the variational autoencoder network for the font generation. Then, we conduct a perceptual study on the generated fonts from the multi-dimensional latent space of the generative model. After we obtained the distribution data of specific preferences, we utilize manifold learning approach to visualize the font distribution. In contrast to the conventional user interface in our user study, the proposed font-exploring user interface is efficient and helpful in the designated user preference., Comment: 9 pages, 16 figures

We present an interactive core training system for core training using a monocular camera image as input in this paper. It is commonly expensive to capture human pose using depth cameras or multiple cameras with conventional approaches. To solve this issue, we employ the skinned multi-person linear model of human shape to recover the 3D human pose from 2D images using pose estimation and human mesh recovery approaches. In order to support the user in maintaining the correct postures from target poses in the training, we adopt 3D human shape estimation for both the target image and input camera video. We propose CoreUI, a user interface for providing visual guidance showing the differences among the estimated targets and current human shapes in core training, which are visualized by markers at ten body parts with color changes. From our user studies, the proposed core training system is effective and convenient compared with the conventional guidance of 2D skeletons., Comment: 9 pages, 12 figures

We present a novel direction-aware feature-level frequency decomposition network for single image deraining. Compared with existing solutions, the proposed network has three compelling characteristics. First, unlike previous algorithms, we propose to perform frequency decomposition at feature-level instead of image-level, allowing both low-frequency maps containing structures and high-frequency maps containing details to be continuously refined during the training procedure. Second, we further establish communication channels between low-frequency maps and high-frequency maps to interactively capture structures from high-frequency maps and add them back to low-frequency maps and, simultaneously, extract details from low-frequency maps and send them back to high-frequency maps, thereby removing rain streaks while preserving more delicate features in the input image. Third, different from existing algorithms using convolutional filters consistent in all directions, we propose a direction-aware filter to capture the direction of rain streaks in order to more effectively and thoroughly purge the input images of rain streaks. We extensively evaluate the proposed approach in three representative datasets and experimental results corroborate our approach consistently outperforms state-of-the-art deraining algorithms.

The free-form deformation model can represent a wide range of non-rigid deformations by manipulating a control point lattice over the image. However, due to a large number of parameters, it is challenging to fit the free-form deformation model directly to the deformed image for deformation estimation because of the complexity of the fitness landscape. In this paper, we cast the registration task as a multi-objective optimization problem (MOP) according to the fact that regions affected by each control point overlap with each other. Specifically, by partitioning the template image into several regions and measuring the similarity of each region independently, multiple objectives are built and deformation estimation can thus be realized by solving the MOP with off-the-shelf multi-objective evolutionary algorithms (MOEAs). In addition, a coarse-to-fine strategy is realized by image pyramid combined with control point mesh subdivision. Specifically, the optimized candidate solutions of the current image level are inherited by the next level, which increases the ability to deal with large deformation. Also, a post-processing procedure is proposed to generate a single output utilizing the Pareto optimal solutions. Comparative experiments on both synthetic and real-world images show the effectiveness and usefulness of our deformation estimation method.

In this paper, we propose dualFace, a portrait drawing interface to assist users with different levels of drawing skills to complete recognizable and authentic face sketches. dualFace consists of two-stage drawing assistance to provide global and local visual guidance: global guidance, which helps users draw contour lines of portraits (i.e., geometric structure), and local guidance, which helps users draws details of facial parts (which conform to user-drawn contour lines), inspired by traditional artist workflows in portrait drawing. In the stage of global guidance, the user draws several contour lines, and dualFace then searches several relevant images from an internal database and displays the suggested face contour lines over the background of the canvas. In the stage of local guidance, we synthesize detailed portrait images with a deep generative model from user-drawn contour lines, but use the synthesized results as detailed drawing guidance. We conducted a user study to verify the effectiveness of dualFace, and we confirmed that dualFace significantly helps achieve a detailed portrait sketch. see http://www.jaist.ac.jp/~xie/dualface.html, Comment: Accepted in the Journal of Computational Visual Media Conference 2021. 13 pages, 12 figures

Normal map is an important and efficient way to represent complex 3D models. A designer may benefit from the auto-generation of high quality and accurate normal maps from freehand sketches in 3D content creation. This paper proposes a deep generative model for generating normal maps from users sketch with geometric sampling. Our generative model is based on Conditional Generative Adversarial Network with the curvature-sensitive points sampling of conditional masks. This sampling process can help eliminate the ambiguity of generation results as network input. In addition, we adopted a U-Net structure discriminator to help the generator be better trained. It is verified that the proposed framework can generate more accurate normal maps., Comment: accepted in International Workshop on Advanced Image Technology 2021, 5 pages, 2 figures

To increase the awareness and impact, soft robotics needs to go beyond the lab environment and should be readily accessible to those even with no robotic expertise. However, most prevailing manufacturing methodologies require either professional equipment or materials that are not usually available to common people, thereby constraining the accessibility of soft robotics. In this communication, we propose a lightweight and low-cost soft bending actuator, called BPActuator, that can be easily fabricated with plastics and balloons. We fabricated a range of actuators with various morphology for characterization in terms of deformation and load-bearing capacity, and demonstrated that they can bend up to 35 degrees and exert force at the tip around 0.070$\pm$0.015N, which is over 5 times higher than their average gravity. We further implemented a gripper with three fingers using the proposed actuators, and found that the gripper can realize human-like grasp of a range of daily objects. The gripper can lift objects at least 8 times heavier than its own weight. Furthermore, the BPActuator is cost effective and each costs about 0.22 USD. Given these advantages, the BPActuators are expected to significantly improve the accessibility of soft robotics to a wider group without robotic expertise., Comment: Accepted to the 4th IEEE International Conference on Soft Robotics (RoboSoft), IEEE copyright

As one of the prevalent topic mining tools, neural topic modeling has attracted a lot of interests for the advantages of high efficiency in training and strong generalisation abilities. However, due to the lack of context in each short text, the existing neural topic models may suffer from feature sparsity on such documents. To alleviate this issue, we propose a Context Reinforced Neural Topic Model (CRNTM), whose characteristics can be summarized as follows. Firstly, by assuming that each short text covers only a few salient topics, CRNTM infers the topic for each word in a narrow range. Secondly, our model exploits pre-trained word embeddings by treating topics as multivariate Gaussian distributions or Gaussian mixture distributions in the embedding space. Extensive experiments on two benchmark datasets validate the effectiveness of the proposed model on both topic discovery and text classification.

Topic modeling has been one of the most active research areas in machine learning in recent years. Hierarchical latent tree analysis (HLTA) has been recently proposed for hierarchical topic modeling and has shown superior performance over state-of-the-art methods. However, the models used in HLTA have a tree structure and cannot represent the different meanings of multiword expressions sharing the same word appropriately. Therefore, we propose a method for extracting and selecting collocations as a preprocessing step for HLTA. The selected collocations are replaced with single tokens in the bag-of-words model before running HLTA. Our empirical evaluation shows that the proposed method led to better performance of HLTA on three of the four data sets tested.