Search

This paper introduces Sketched Reality, an approach that combines AR sketching and actuated tangible user interfaces (TUI) for bidirectional sketching interaction. Bi-directional sketching enables virtual sketches and physical objects to "affect" each other through physical actuation and digital computation. In the existing AR sketching, the relationship between virtual and physical worlds is only one-directional -- while physical interaction can affect virtual sketches, virtual sketches have no return effect on the physical objects or environment. In contrast, bi-directional sketching interaction allows the seamless coupling between sketches and actuated TUIs. In this paper, we employ tabletop-size small robots (Sony Toio) and an iPad-based AR sketching tool to demonstrate the concept. In our system, virtual sketches drawn and simulated on an iPad (e.g., lines, walls, pendulums, and springs) can move, actuate, collide, and constrain physical Toio robots, as if virtual sketches and the physical objects exist in the same space through seamless coupling between AR and robot motion. This paper contributes a set of novel interactions and a design space of bi-directional AR sketching. We demonstrate a series of potential applications, such as tangible physics education, explorable mechanism, tangible gaming for children, and in-situ robot programming via sketching., Comment: UIST 2022

In the field of HCI, methods for creating morphing food have required the process of making sheets and cutting out the shapes by hand. In order to simplify the fabrication process and further expand the customization and design of shape-changing foods, we have developed magashi, a new method which enables users to design food in different shapes with intricate patterns by utilizing extrusion-based food printing and baking. With our technique, the printed food with mesh patterns can form into curved structures when baked. In order to realize this method, the material's ingredients, printing patterns, and thickness were optimized to control the drying process of the material, consequently inducing it to bend upward or downward during baking. We also propose several applications for creating shape-changing food and cutlery in different shapes, patterns and curvatures.

Soft sensors made of deformable materials, that are capable of sensing touches or gestures, have attracted considerable attention for use in tangible interfaces or soft robotics. However, to achieve multimodal gesture detection with soft sensors, prior studies have combined multiple sensors or utilized complex configurations with multiple wires. To achieve multimodal gesture sensing with a simpler configuration, a novel soft sensor consisting of a conductive foam with a single wire, was proposed in this study. This sensor was named as foamin and utilizes an impedance measurement technique at multiple frequencies called foamin. Additionally, a surface-shielding method was designed for improving the detection performance of the sensor. Several patterns of foamin were implemented to investigate the detection accuracy and three application scenarios based on the sensor were proposed.

The development and widespread use of digital fabrication techniques has accelerated the fabrication of three-dimensional (3D) objects, enabling not only to design their shapes but also to customize their internal structures and materials to realize various functions. In this research we propose a novel design method to embed internal pumps and liquid paths inside 3D objects. We utilized a pump driven by electrohydrodynamics (EHD) of fluid migrating in response to an applied electric field. To arrange the EHD pumps and fluid paths in appropriate positions inside 3D objects, we stacked layered acrylic plates cut with digital fabrication tools such as laser cutters and cutting plotters. Using these embedded pumps, we can control the flow of fluid to change color, shape, and motion of the objects. In this paper, we detailed our fabrication techniques named LayerPump, and demonstrated its design space and application scenarios.

Despite the recent growth in popularity of personal mobility devices (e.g., e-scooters and e-skateboards), they still suffer from limited safety and narrow design form factors, due to their rigid structures. On the other hand, inflatable interfaces studied in human-computer interaction can achieve large volume change by simple inflation/deflation. Inflatable structure also offers soft and safe interaction owing to material compliance and diverse fabrication methods that lead to a wide range of forms and aesthetics. In this paper, we propose poimo, a new family of POrtable and Inflatable MObility devices, which consists of inflatable frames, inflatable wheels, and inflatable steering mechanisms made of a mass-manufacturable material called drop-stitch fabric. First, we defined the basic material properties of a drop-stitch inflatable structure that is sufficiently strong to carry a person while simultaneously allowing soft deformation and deflation for storage and portability. We then implemented an interactive design system that can scan the user's desired riding posture to generate a customized personal mobility device and can add the user's shape and color preferences. To demonstrate the custom-design capability and mobility, we designed several 3D models using our system and built physical samples for two basic templates: a motorcycle and a wheelchair. Finally, we conducted an online user study to examine the usability of the design system and share lessons learned for further improvements in the design and fabrication of poimo.

In this study, we propose a novel fabrication method to create large-scale 3D objects by bending and piling up an inflated single polyethylene tube. A unique point of our method is that we can redo the design and reconfigure it using the same tube after evacuating the air from the object. To provide detachable constraints on the tube for bending, we developed a novel technique with a hook-and-loop fastener attached on the tube. We also designed software to convert 3D targeted object shape to the bending pattern of the tube. Furthermore, we implemented a prototype of a machine for the automation of the attaching process of the fastener on the tube. In this paper, we describe the details of the design, implementation, results and future works.

Large-scale shape-changing interfaces have great potential, but creating such systems requires substantial time, cost, space, and efforts, which hinders the research community to explore interactions beyond the scale of human hands. We introduce modular inflatable actuators as building blocks for prototyping room-scale shape-changing interfaces. Each actuator can change its height from 15cm to 150cm, actuated and controlled by air pressure. Each unit is low-cost (8 USD), lightweight (10 kg), compact (15 cm), and robust, making it well-suited for prototyping room-scale shape transformations. Moreover, our modular and reconfigurable design allows researchers and designers to quickly construct different geometries and to explore various applications. This paper contributes to the design and implementation of highly extendable inflatable actuators, and demonstrates a range of scenarios that can leverage this modular building block., Comment: TEI 2020

ExpandFab is a fabrication method for creating expanding objects using foam materials. The printed objects change their shape and volume, which is advantageous for reducing the printing time and transportation costs. For the fabrication of expanding objects, we investigated a basic principle of the expansion rate and developed materials by mixing a foam powder and elastic adhesive. Furthermore, we developed a fabrication method using the foam materials. A user can design expanded objects using our design software and sets the expansion areas on the surface. The software simulates and exports the 3d model into a three-dimensional (3D) printer. The 3D printer prints the expandable object by curing with ultraviolet light. Finally, the user heats the printed objects, and the objects expand to maximum approximately 2.7 times of their original size. ExpandFab allows users to prototype products that expand and morph into various shapes, such as objects changing from one shape to various shapes, and functional prototype with electronic components. In this paper, we describe the basic principle of this technique, implementation of the software and hardware, application examples, limitations and discussions, and future works.

This paper introduces LiftTiles, a modular and reconfigurable room-scale shape display. LiftTiles consist of an array of retractable and inflatable actuator that is compact (e.g., 15cm tall) and light (e.g., 1.8kg), while extending up to 1.5m to allow for large-scale shape transformation. Inflatable actuation also provides a robust structure that can support heavy objects (e.g., 10 kg weight). This paper describes the design and implementation of LiftTiles and explores the application space for reconfigurable room-scale shape displays.

We introduce MorphIO, entirely soft sensing and actuation modules for programming by demonstration of soft robots and shape-changing interfaces.MorphIO's hardware consists of a soft pneumatic actuator containing a conductive sponge sensor.This allows both input and output of three-dimensional deformation of a soft material.Leveraging this capability, MorphIO enables a user to record and later playback physical motion of programmable shape-changing materials.In addition, the modular design of MorphIO's unit allows the user to construct various shapes and topologies through magnetic connection.We demonstrate several application scenarios, including tangible character animation, locomotion experiment of a soft robot, and prototyping tools for animated soft objects.Our user study with six participants confirms the benefits of MorphIO, as compared to the existing programming paradigm.

We present FoldTronics, a 2D-cutting based fabrication technique to integrate electronics into 3D folded objects. The key idea is to cut and perforate a 2D sheet to make it foldable into a honeycomb structure using a cutting plotter; before folding the sheet into a 3D structure, users place the electronic components and circuitry onto the sheet. The fabrication process only takes a few minutes allowing to rapidly prototype functional interactive devices. The resulting objects are lightweight and rigid, thus allowing for weight-sensitive and force-sensitive applications. Finally, due to the nature of the honeycomb structure, the objects can be folded flat along one axis and thus can be efficiently transported in this compact form factor. We describe the structure of the foldable sheet, and present a design tool that enables users to quickly prototype the desired objects. We showcase a range of examples made with our design tool, including objects with integrated sensors and display elements.

In this paper, we present an approach to realize the levitation of a small object with an embedded electronic circuit. Luciola is a light-emitting particle with a diameter of 3.5mm and a weight of 16.2mg moving in mid-air. The novelty of this paper is the ultrasonically levitated electronic object powered by resonant inductive coupling. To enable the levitation of a particle, a custom IC chip is essential in reducing the size and weight of the particle. Ths custom IC chip is designed to achieve an intermittent lighting of the LED, which increases the maximal distance between the transmitter and the receiver coils. Luciola is applied to a self-luminous pixel in a 3-dimensional (3D) mid-air display and the drawing of characters in mid-air is also demonstrated.

In this paper, we propose a method to create 3D inflatable objects by laminating plastic layers. AccordionFab is a fabrication method in which the user can prototype multi-layered inflatable structures rapidly with a common laser cutter. Our key finding is that it is possible to selectively weld the two uppermost plastic sheets out of the stacked sheets by defocusing the laser and inserting the heat-resistant paper below the desired welding layer. As the contribution of our research, we investigated the optimal distance between the lens and the workpiece for cutting and welding and developed an attachment which supports welding process. Next, we developed a mechanism of changing the thickness and bending angle of multi-layered objects and created a simulation software. Using these techniques, the user can create various prototypes such as personal furniture that fits user's body and packing containers that fit the contents.

This paper introduces Dynamic 3D Printing, a fast and reconstructable shape formation system. Dynamic 3D Printing can assemble an arbitrary three-dimensional shape from a large number of small physical elements. Also, it can disassemble the shape back to elements and reconstruct a new shape. Dynamic 3D Printing combines the capabilities of 3D printers and shape displays: Like conventional 3D printing, it can generate arbitrary and graspable three-dimensional shapes, while allowing shapes to be rapidly formed and reformed as in a shape display. To demonstrate the idea, we describe the design and implementation of Dynablock, a working prototype of a dynamic 3D printer. Dynablock can form a three-dimensional shape in seconds by assembling 3,000 9 mm blocks, leveraging a 24 x 16 pin-based shape display as a parallel assembler. Dynamic 3D printing is a step toward achieving our long-term vision in which 3D printing becomes an interactive medium, rather than the means for fabrication that it is today. In this paper, we explore possibilities for this vision by illustrating application scenarios that are difficult to achieve with conventional 3D printing or shape display systems.

In this research, we propose a system which makes paper through additive manufacturing process by using a dispenser mounted on XY plotter. By using our system, graphic designers can design and output paper itself which is hard in an existing paper production process. This time, we designed and implemented a machine for fabricating paper and created several output examples. In SIGGRAPH, we will provide a workshop for participants to design their original paper using our machines.

In this paper, we propose a new system called xSlate, a stiffness controlled surface for shape changing interfaces. It is enabled by a deformable frame structure that consists of linear actuators, and an elastic skin surface that can configure its stiffness by pneumatic jamming. We describe the implementation methods of xSlate and how it can be used for future applications.

We propose a new method for shape- and color-changing display called COLORISE. Our 'COLORISE' system has inflatable shape changing pixels that can change their colors without using any light-emitting type devices. The array of the modules enables various color patterns, making full use of the characteristics of the material. Each pixel also has a touch sensing function that enables users to interact with it intuitively. This paper describes the design and mechanism of our system, explores interactions with users, and presents technical evaluations of the proposed pixel modules.

We are increasingly offloading a lot of our mental and subjective tasks to machines. Over the last few years, with advances in machine learning, especially in deep learning, machines are becoming increasingly capable of modeling and being more intimately involved in our everyday tasks. Though a lot of the tasks we do today are related with manipulating virtual data, our bodies still belong in a physical space and make up a large part of our activities. Artificial intelligence (AI) interactions are generally limited to on-screen or audio format, and therefore lack support in our physical tasks where our bodies play a major role. Also, being behind the screen and virtual in nature, we often do not acknowledge the extent to which AI plays a role in a lot of our subjective tasks. We present (author)rise, a physical human-machine intelligence based handwriting system, where machines generate handwritten text in continuation to human handwriting and move the human hand on the paper to write it out, thereby tightly coupling human and machine intelligence in the same physical input-output space.

In this paper, we propose a new method of communication with a remote partner. The system, which we call "Paralogue" (parasitic + dialogue), utilizes the user's arm and a hand as an avatar that represents the remote conversation partner. This method is realized by controlling the movement of the user's arm with electrical muscle stimulations(EMS) and making the user's arm behave as if the remote conversation partner were speaking to the user. Therefore, it is possible to add the physical presence of the remote conversation partner to the situation. This can also be a telepresence system using real human arms. However, unlike many telepresence systems, external actuators and large-sized devices are not required for this system and can be completed by only utilizing physical interaction. In this paper, we present its design and implementation.