Search

3D modelling and printing are becoming increasingly popular. However, beginners often face high barriers of entry when trying to use existing 3D modelling tools, even for creating simple objects. This is further complicated on mobile devices by the lack of direct manipulation in the Z dimension. In this paper, we explore the possibility of using foldable mobile devices for modelling simple objects by constructing a 2.5D display and interaction space with folded touch screens. We present a set of novel interaction techniques - AngleCAD, which allows users to view and navigate a 3D space through folded screens, and to modify the 3D object using the physical support of touchscreens and folding angles. The design of these techniques was inspired by woodworking practices to support surface-based operations that allow users to cut, snap and taper objects directly with the touch screen, and extrude and drill them according to the physical fold angle. A preliminary study identified the benefits of this approach and the key design factors that affect the user experience.

International audience; Damping is usually associated with irreversibility. Here, we present a counterintuitive concept to achieve time reversal of waves propagating in a lossless medium using a transitory dissipation pulse. Applying a sudden and strong damping in a limited time generates a time-reversed wave. In the limit of a high damping shock, this amounts to "freezing" the initial wave by maintaining the wave amplitude while canceling its time derivative. The initial wave then splits in two counterpropagating waves with half of its amplitude and time evolutions in opposite directions. We implement this damping-based time reversal using phonon waves propagating in a lattice of interacting magnets placed on an air cushion. We show with computer simulations that this concept also applies to broadband time reversal in complex disordered systems.

In this paper we introduce novel Virtual Reality (VR) and Augmented Reality (AR) treatments to improve the psychological well being of patients in palliative care, based on interviews with a clinical psychologist who has successfully implemented VR assisted interventions on palliative care patients in the Hong Kong hospital system. Our VR and AR assisted interventions are adaptations of traditional palliative care therapies which simultaneously facilitate patients communication with family and friends while isolated in hospital due to physical weakness and COVID-19 related restrictions. The first system we propose is a networked, metaverse platform for palliative care patients to create customized virtual environments with therapists, family and friends which function as immersive and collaborative versions of 'life review' and 'reminiscence therapy'. The second proposed system will investigate the use of Mixed Reality telepresence and haptic touch in an AR environment, which will allow palliative care patients to physically feel friends and family in a virtual space, adding to the sense of presence and immersion in that environment.

A remarkable result from integral geometry is Cauchy's formula, which relates the mean path length of ballistic trajectories randomly crossing a convex 2D domain, to the ratio between the region area and its perimeter. This theorem has been generalized for non-convex domains and extended to the case of Brownian motion to find many applications in various fields including biological locomotion and wave physics. Here, we generalize the theorem to arbitrary closed trajectories exploring arbitrary domains. We demonstrate that, regardless of the complexity of the trajectory, the mean arc length still satisfies Cauchy's formula provided that no trajectory is entirely contained in the domain. Below this threshold, the mean arc length decreases with the size of the trajectory. In this case, an approximate analytical formula can still be given for convex trajectories intersecting convex domains provided they are small in comparison. To validate our analysis, we performed numerical simulations of different types of trajectories exploring arbitrary 2D domains. Our results could be applied to retrieve geometric information of bounded domains from the mean first entrance-exit length., 14 pages, 5 figures

Visual stimuli can modulate the temperature at which people perceive heat pain. However, very little research exists on the potential use of Augmented Reality (AR) to modulate the heat pain threshold (HPT). In this paper, we investigate whether participants’ HPTs can be modulated observing virtual flames on their hands through a video see-through head-mounted display (VST-HMD). In a pilot study (n = 7), we found that rendering virtual flames had a significant effect (p < 0.05) on the HPT. The virtual flames on the participant’s hand led to a decrease in the temperature at which they would perceive pain related to heat. These results indicate that AR-induced stimuli may be an effective way to achieve top-down modulation of the experience of pain.

Some of the biggest challenges in applying Augmented Reality (AR) technologies to the industry floor are in the form factor, and safety requirements of the head worn display. This includes alleviating issues such as peripheral view occlusion, and adaptation to personal protective equipment. In this work we present the design of Project Ariel, an Open Source 3D printable display specifically designed for use in industrial environments. It is our hope that with this technology, the average tradesman can utilize the powerful visualizations AR has to offer, significantly improving their daily work flow.

Recent years have seen an explosion in Extended Reality (XR) experiences. Until now, researchers and developers were limited to relatively expensive, closed-source, and consumer-level hardware when creating high-fidelity XR experiences, as these were the only devices that can create a high-fidelity XR experience out of the box with minimal setup times. In this paper, we present Project Esky, a complete open-source modular software framework that enables the rapid creation of high-fidelity XR experiences on any combination of display and tracker hardware. Our framework includes several components that handle 6DOF head tracking with head pose prediction to minimize latency, hand interactions with virtual content, and network components to facilitate between-user co-located experiences. Our framework also includes an asynchronous rendering pipeline that simplifies the representation of optical distortions, as well as a simplified planar-based temporal re-projection technique to minimize rendering latency. It is our hope that the techniques described in this paper, and the open source software implementations that support it, will be a stepping stone towards bringing high-fidelity XR content to researchers and hobbyist users alike.

This demonstration showcases the improvements made to Project Esky[2], an open source Extended Reality (XR) software framework capable of high fidelity natural hand-interactions with virtual content. In combination with showing our temporal re-projection rendering technique in realtime, we also showcase several live desktop demonstrations, such as a virtually controlled powerpoint presentation, an e-learning example with an animated car engine, a fire simulation and a model painting application. It is our hope that the demonstrations presented will inspire others to work with Project Esky to build their own experiences, and will be a stepping stone towards bringing high-fidelity XR content to researchers and hobbyists.

In this paper, we report on our experiment to investigate the extent to which the human body reacts to virtual thermal stimuli presented in Augmented Reality (AR). Our AR experience enables users to see and hear their own hands burning while looking through a Video SeeThrough Head-Mounted Display (VST-HMD). We hypothesized that this AR experience can lead to a change in Skin Blood Flow (SkBF). Our results show that the SkBF in the affected hand did change significantly on some participants. They experienced a change of blood flow similarly to real thermal stimulation.

"Serial paintings" are a series of works highlighting the role of the canvas - or more generally the support of the painting - in the construction of the final form. "Back and Forth - Pneumatic Anadrome" is the first work in the series. In it, a string of coloured beads is pushed back and forth between two spiralling "canvases" using compressed air. Without rearranging the order of the beads, the string is forced to coil alternatively into one or the other spiral: this folding and unfolding reveals in turn images or text with opposing meanings or connotations. Each image has to be destroyed in order to create the other; this cyclic process of creation and destruction is purposefully revealed and triggered by the curiosity of the public.

We present the design and implementation of a ”Laser Graphics Processing Unit” (LGPU) featuring a proposed re-configurable graphics pipeline capable of minimal latency interactive feedback, without the need of computer communication. This is a novel approach for creating interactive graphics where a simple program describes the interaction on a vertex. Similar in design to a geometry or fragment shader on a GPU, these programs are uploaded on initialisation and do not require input from any external micro-controller while running. The interaction shader takes input from a light sensor and updates the vertex and fragment shader, an operation that can be parallelised. Once loaded onto our prototype LGPU the pipeline can create laser graphics that react within 4 ms of interaction and can run without input from a computer. The pipeline achieves this low latency by having the interaction shader communicate with the geometry and vertex shaders that are also running on the LGPU. This enables the creation of low latency displays such as car counters, musical instrument interfaces, and non-touch projected widgets or buttons. From our testing we were able to achieve a reaction time of 4 ms and from a range of up to 15 m.

This demonstration showcases a complete Open-Source Augmented Reality (AR) modular platform capable of high fidelity natural hand-interactions with virtual content, high field of view, and spatial mapping for environment interactions. We do this via several live desktop demonstrations. Finally, included in this demonstration is a completed open source schematic, allowing anyone interested in utilizing our proposed platform to engage with high fidelity AR. It is our hope that the work described in this demo will be a stepping stone towards bringing high-fidelity AR content to researchers and commodity users alike.

Globally, it is estimated that up to 1 billion children aged 2-17 years, have experienced physical, sexual, or emotional violence in the past year [1], and 30% of the abused child is likely to develop Post-traumatic stress disorder (PTSD) [2]; 354 million adult war survivors are suffering from PTSD [3]; At where the natural disaster occurred, 70.7% of the survivor will suffer from acute PTSD [4]. PTSD has not only high prevalence but also high lethality, which is accompanied by multiple physical and mental comorbidities as well as strong suicidal tendencies [5]-[7]. This doctoral research aims to contribute to the development of PTSD treatment by investigating the possibility of adopting Augmented Reality (AR) narrative in treating PTSD. This four-year research project consists of three steps. In the first stage of research, we will conduct a comparative study between AR and VR narratives with healthy participants to verify whether AR narratives work better in eliciting the emotional engagement of the participants than VR narratives. In the second stage, we will create a system that integrates AR narratives with prolonged exposure (PE) treatment and experiment it with PTSD patients to verify its treatment efficacy. In the final stage, a semi-automatic and patient-authored AR system is expected to be achieved, through which the patients can design their own exposure environment via voice input. This project will provide valuable experimental samples and scientific analysis for the research of psychotherapy, narrative studies, and AR application.

Visually disabled people increasingly use computers in everyday life, thanks to novel assistive technologies better tailored to their cognitive functioning. Like sighted people, many are interested in computer games - videogames and audio-games. Tactile-games are beginning to emerge. The Tactile Radar is a device through which a visually disabled person is able to detect distal obstacles. In this study, it is connected to a computer running a tactile-game. The game consists in finding and collecting randomly arranged coins in a virtual room.The study was conducted with nine congenital blind people including both sexes, aged 20-64 years old. Complementary methods of first and third person were used: the debriefing interview and the quasi-experimental design.The results indicate that the Tactile Radar is suitable for the creation of computer games specifically tailored for visually disabled people.Furthermore, the device seems capable of eliciting a powerful immersive experience. Methodologically speaking, this research contributes to the consolidation and development of first and third person complementary methods, particularly useful in disabled people research field, including the evaluation by users of the Tactile Radar effectiveness in a virtual reality context. Implications for rehabilitation Despite the growing interest in virtual games for visually disabled people, they still find barriers to access such games. Through the development of assistive technologies such as the Tactile Radar, applied in virtual games, we can create new opportunities for leisure, socialization and education for visually disabled people. The results of our study indicate that the Tactile Radar is adapted to the creation of video games for visually disabled people, providing a playful interaction with the players.

Displacements of large objects induced by vibration are a common occurrence, but the motion is generally unpredictable. Think for instance of an unbalanced front-loading washing machine. For controlled movement, wheels or legs are usually preferred. In this work, we explore a strategy for moving everyday objects by introducing or harvesting random external vibration rather than using a mechanical system with wheels. We designed vibration modules that can be easily attached to furniture and objects. After learning how several random bursts of vibration affect its pose, an optimization algorithm discovers the optimal sequence of vibration patterns required to (slowly but surely) move the object to a specified position. This principle may be useful for displacing large or small objects in situations where attaching wheels or complete lifting is impossible assuming the speed of the process is not a concern. Embedding vibration modules as part of mass-produced objects may provide a low-cost way to make almost anything mobile.

Latest hardware improvements on transceivers supporting Low Power Wide Area Networks (LPWAN) make it feasible to connect small battery powered devices hundred of meters or even km away. In this paper, we propose a physical computing paradigm fully exploiting this novel technology. Spatial physical computing (SPC) leverages not only natural manipulation typically used on TUI and construction kits but also integrates the necessary deambulation (around a building or a city) in the process of creating, testing and tuning a distributed smart sensor/actuator network. The overall system is a compound of (an unlimited) set of independent data-processing nodes supporting an event-driven data-flow programming scheme. We demonstrate in a few examples how such networks can be deployed Spatially programmed - through intuitive physical actions, and discuss the unique qualities and challenges of Spatial Physical Computing.

BACKGROUND: Individuals lacking or having impaired vision face serious difficulties during autonomous locomotion. Sensory substitution devices can contribute to alleviate such difficulties, significantly (and measurably) reducing anxiety. OBJECTIVE: The present paper evaluates a device – the Tactile Radar (TR) – that can detect obstacles at a certain distance from the user and generate meaningful and unobtrusive tactile stimuli. METHODS: We evaluate the impact of its use on the degree of anxiety that autonomous locomotion usually trigger on people who are blind. RESULTS:Decreased anxiety aswell as increased sense of safety and independence was observed on the tested subjects, through subjective (semi-structured interviews) and objective assessments (STAI inventories). CONCLUSIONS: This device seems promising. More experimentation is needed to evaluate the capacity of the TR to enhance indoor localization and body placement with respect to walls and obstacles, as well as evaluation of the device in real life situations including outdoors. Last but not least, we need to consider ways of moving from a prototyping to a real production phase – of an affordable yet reliable device that can reach as soon as possible the interested population.

An Egyptian statue on display at the Manchester Museum mysteriously spins on its axis every day; it is eventually discovered that this is due to anisotropic friction forces, and that the motile power comes from imperceptible mechanical waves caused by visitors' footsteps and nearby traffic. This phenomena involves microscopic ratchets, and is pervasive in the microscopic world - this is basically how muscles contract. It was the source of inspiration to think about everyday objects that move by harvesting external vibration rather than using mechanical traction and steering wheels. We propose here a strategy for displacing objects by attaching relatively small vibration sources. After learning how several random bursts of vibration affect its pose, an optimization algorithm discovers the optimal sequence of vibration patterns required to (slowly but surely) move the object to a very different specified position. We describe and demonstrate two application scenarios, namely assisted transportation of heavy objects with little effort on the part of the human and self arranging furniture, useful for instance to clean classrooms or restaurants during vacant hours.

This article reports on two art-science collaborations, A-me: Augmented Memories and BrainCloud, that interrogate the central role of localization in neuroscience—including the use of technologies that augment sociability using localization as a central reference point. The two projects result from a series of interactions where a science/technology development fostered art, which in turn led to a science application, which potentially may lead to further artistic activity. A-me is an art installation that repurposes navigation and visualization tools normally reserved for medical clinicians and scientists, inviting reflection on the ongoing endeavor of neuroscience to explain and map cognitive functions such as memory. BrainCloud is a software prototype that provides neuroscientists with an interface for interacting with existing data and knowledge about the brain. Organized visually as a brain atlas, it forms a social network that allows neuroscientists to connect and share their ongoing research and ideas. © 2016 Leonardo/ International Society for the Arts, Sciences and Technology (ISAST)

We describe and demonstrate a pair of diopter-adjustable eyeglasses aimed to correct presbyopia; the glasses provide a tunable optical power in the whole surface of the lens cell, eliminating the optical distortion typical of bifocal/trifocal or progressive glasses. The wearer can actively control the optical power by a simple sliding gesture on the bridge of the glasses, so that presbyopic vision can be interactively corrected. Results from a preliminary experiment showed that a presbyopia sufferer could clearly observe near and far objects under the assistant accommodation of the glasses. Designing a truly wearable system poses some challenges – none of them theoretical – so the system should be feasible in the near future.

A-me is a fictitious memory-evoking apparatus at the intersection of science, art and technology. The system enables users to experience other people's memories as well as store their own by interacting with a volumetric representation (MR) of a human brain. The user retrieves or stores memories (audio traces) by pointing and clicking at precise voxels locations. Triggered by their exploratory action, a story is slowly revealed and recomposed in the form of whispering voices revealing intimate stories. A-me it's a public receptacle for private memories, thus exploring the possibility of a collective physical brain.The installation introduces an original optical see-through AR setup for neuronavigation capable of overlaying a volume rendered MR scan onto a physical dummy head. Implementing such a system also forced us to address technical questions on quality assessment of AR systems for brain visualization.

This workshop provides a forum for discussing emerging trends in interactive surfaces that leverage alternative display types and form factors to enable more expressive interaction with information. The goal of the workshop is to push the current discussion forward towards a synthesis of emerging visualization and interaction concepts in the area of improvised, minimal, curved and malleable interactive surfaces. By doing so, we aim to generate an agenda for future research and development in interactive surfaces.

Mobile and ubiquitous systems create opportunities for new kinds of interactivity in public spaces. Examples of human-technology interactions in public spaces include interactive displays on different scales; mobile systems enabling projection in public environments; smart interactive and reactive objects; tangible interfaces; and public media arts. Human-system and mediated human-human interactions become public and visible to the people around the same space. This creates many possibilities and challenges for designing the user experience that arise primarily from the social and physical context. This workshop will bring together researchers, designers, practitioners and media artists to discuss elements and viewpoints of such new forms of experiences. The results of the workshop will be an "experience design space" and a research agenda for experiences with interactive systems used in public spaces.

Recent developments in computer vision hardware have popularized the use of (free hand) gestures as well as full body posture as a form of input control in commercial gaming applications. However, the computer screen remains the place where the eyes must be placed at all times. Freeing graphic output from that rectangular cage is a hot topic in Spatial Augmented Reality (SAR). Using static or dynamic projection mapping and 'smart projectors', it is possible to recruit any surface in the surrounding for displaying the game's graphics. The present work introduces an original interaction paradigm building on kinetic interfaces and SAR: in 'Skin Games' the body acts simultaneously as the controller and as the (wildly deformable) projection surface on which to display the game's output.

"scoreLight" and "scoreBots" are two experimental platforms for performative sound design and manipulation. Both are essentially synesthetic interfaces - synesthetic musical instruments - capable of translating free-hand drawings into a sonic language of beats and pitches, all in real time. While scoreLight uses a modified "smart" laser scanner to track the figure's relevant features (in particular contours), scoreBots rely on one or more tiny line-follower robots to do the same. We present here some of our latest experimentations in an informal way.

We propose a wearable device capable of translating bio-sensed data into cartoon-like graphics projected in the physical surrounding. Such 'expressive Laser Aura' (LA) may serve for biofeed-back purposes; but more interestingly, as the display extends past the wearer's personal space it could complement non-verbal social communication by giving others an instant cue about a person's real inner state. In this preliminary work, we explore a proof-of-principle scenario: the possibility of enhancing empathic behavior at the working place, where people have little or no time to explicitly communicate a need for help, or on the contrary a need for isolation. The subject is sitting at the office desk; the LA (presently non-wearable) is projected on a nearby wall or on the floor. It takes the shape of an halo that changes its behavior as a function of the subject stress level. (The LA is also capable of displaying icons and text messages in an autonomous way, or as prompted by the user.) To measure stress, we first tried the commercial 'Mind Flex' head-worn device (combining simple EEG and EMG data). This proved to be too much dependent on test subjects, so we opted for a simpler strategy consisting on measuring physical restlessness by fitting the chair with an accelerometer. The custom laser projector unit measures about 20x20x10cm. The LA graphic output is inspired from cartoon conventions, paranormal folklore and synesthetic considerations. When the subject is relaxed, the halo is nearly circular, uniform and smooth; restlessness disturbs the aura by introducing dynamic spikes. This mapping seems very intuitive - in our preliminary experiment, casual observers never failed to recognize the intended meaning at first glance. When projected on the floor (either from the back of the chair or, in the future, using a laser projection belt [Wilde et al. 2010]), the LA may represent the limits of the personal space. By enlarging it, the subject can claim personal space and signify a need for isolation; on the contrary, while shrinking (or fading or changing its color) the aura may indicate the user readiness to engage in closer social interaction.

Recently the miniaturization of mobile devices has progressed and such devices are difficult to have input interface that has wide operation area on their surface. Conventional input interface on a cell phone, such as a touch panel or keypad, has limited operation area. There has been many approaches to handle this problem, but they require users to wear some physical devices[Harrison and Hudson 2009] or to use in some specific environments[Roeber et al. 2003].

The 'Smart Laser Projector' (SLP) is a modified laser-based projector capable of displaying while simultaneously using the laser beam (at the same or different wavelength or polarization) as a LIDAR probe gathering information about the projection surface (its borders, 3d shape, relative position and orientation, as well as fine texture and spectral reflectance). This information can then be used to correct perspective warp, perform per-pixel contrast compensation, or even reroute the scanning/projecting path altogether (for tracking, feature discovery or barcode reading for instance). We demonstrate here raster-scan and vector graphics applications on two different prototypes. The first relies on a pair of galvanomirrors, and is used for demonstrating simultaneous tracking and display on the palm of the hand, depth-discriminating active contours (for spatially augmented reality surveying), and interactive games. The other relies on a single 2-axis MEMS mirror working in resonant mode, and is used to demonstrate edge enhancement of printed material and 'artificial fluorescence' - all with perfect projection-to-real-world registration by construction.

The Volume Slicing Display enables interactive exploration of volumetric data (for example, medical images) using a piece of plexiglass (or paper) that functions both as a control interface and a passive, untethered projection screen.

We present here a first prototype of the Virtual Haptic Radar (VHR), a wearable device helping actors become aware of the presence of invisible virtual objects in their path when evolving in a virtual studio (such as a "bluescreen" filming stage [Figure 1]). The VHR is a natural extension of the Haptic Radar (HR) and its principle [Cassinelli et al. 2006] in the realm of virtual reality: while each module of the HR had a small vibrator and a rangefinder to measure distance to real obstacles, the VHR module lacks the rangefinder but accommodates instead a (cheap) ultrasound-based indoor positioning system that gives it the ability to know exactly where it is situated relatively to an external frame of reference.

This saccade-based display is a device capable of presenting two-dimensional images using a unique bar of addressable light sources (a column of LEDs). In a dimly lit environment, each time the saccadic eye movement of the observer is detected, the flashing pattern of the column light expands, and ghostly images appear in midair.

We propose a variant of the multi-touch display technology that introduces an original way of manipulating three-dimensional data. The underlying metaphor is that of a deformable screen that acts as a boundary surface between the real and the virtual worlds. By doing so, the interface can create the illusion of continuity between the userpsilas real space and the virtual three-dimensional space. The prototype system presented here enables this by employing three key technologies: a tangible and deformable projection screen, a real-time three-dimensional sensing mechanism, and an algorithm for dynamic compensation for anamorphic projection. This paper introduces the concept of the deformable tangible workspace, and describes the required technologies for implementing it. Also, several applications developed on a prototype system are detailed and demonstrated.

Meta-perception is both an interaction design concept and the theme of a research group at the University of Tokyo. As a design concept, meta-perception is used to describe experience of novel phenomena made possible by devices that extend the human precepts. As a research group, our goal is to develop methods for capturing and manipulating information that is normally inaccessible to humans and machines. In this paper we describe various displays and devices that exemplify meta-perception. These include: several displays with which the human bodily interacts and wearable haptic devices that act as an extended skin. We reflect upon a design approach which borrows from elements of philosophy and media art to describe a different relationship between humans and technology.

We are developing a modular electronic device to allow users to perceive and respond simultaneously to multiple spatial information sources using haptic stimulus. Each module of this wearable "haptic radar" acts as an artificial hair capable of sensing obstacles, measuring their range and transducing this information as a vibro-tactile cue on the skin directly beneath the module. Our first prototype (a headband) provides the wearer with 360 degrees of spatial awareness thanks to invisible, insect-like antennas. During a proof-of-principle experiment, a significant proportion (87%, p = 1.26 * 10-5) of participants moved to avoid an unseen object approaching from behind without any previous training. Participants reported the system as more of a help, easy, and intuitive. Among the numerous potential applications of this interface are electronic travel aids and visual prosthetics for the blind, augmentation of spatial awareness in hazardous working environments, as well as enhanced obstacle awareness for motorcycle or car drivers (in this case the sensors may cover the surface of the car).

We present a device whose goal is to allow wearers to perceive and respond to range information from multiple sensors using haptic cues. It uses an array of modules, each of which senses range information and transduces it as vibro-tactile cues on the skin directly beneath the module. Moreover, this modular interface can cover precise skin regions, be distributed in a discrete manner over the skin surface, or span the entire body surface (and then function as a sort of double skin). Among the numerous applications of this interface are visual prosthetics for the blind, augmentation of spatial awareness in hazardous working environments, as well as enhanced obstacle awareness for car drivers. In an experiment, a significant proportion (87%, p=1.26 * 10-5) of subjects moved to avoid an unseen object. On a questionnaire, subjects reported the system as more of a help, easy, and intuitive.

The problem of tracking hands and fingers on natural scenes has received much attention using passive acquisition vision systems and computationally intense image processing. We are currently studying a simple active tracking system using a laser diode, steering mirrors, and a single non-imaging detector, which is capable of acquiring three dimensional coordinates in real time without the need of any image processing at all. Essentially, it is a smart rangefinder scanner that instead of continuously scanning over the full field of view restricts its scanning area, on the basis of a real-time analysis of the backscattered signal, to a very narrow window precisely the size of the target. The complexity of the whole setup is equivalent to that of a portable laser-based barcode reader, making the system compatible with wearable computers.

Demo Hour highlights new prototypes and projects that exemplify innovation and novel forms of interaction. Audrey Desjardins, Editor