Search

Augmenting people with wearable technology can enhance their natural sensing, actuation, and communication capabilities. Interaction with smart devices can become easier and less explicit when combining multiple wearables instead of using device-specific apps on a single smartphone. We demonstrate a prototype for smart device control by combining quasi real-time visual device recognition on a smartphone and EMG-based gesture recognition on a Myo armband.

When compared to image recognition, object detection is a much more challenging task because it requires the accurate real-time localization of an object in the target image. In interaction scenarios, this pipeline can be simplified by incorporating the users’ point of regard. Wearable eye trackers can estimate the gaze direction, but lack own processing capabilities. We enable mobile gaze-aware applications by developing an open-source platform which supports mobile eye tracking based on the Pupil headset and a smartphone running Android OS. Through our platform, we offer researchers and developers a rapid prototyping environment for gaze-enabled applications. We describe the concept, our current progress, and research implications.

Pictures and videos taken with smartphone cameras often suffer from motion blur due to handshake during the exposure time. Recovering a sharp frame from a blurry one is an ill-posed problem but in smartphone applications additional cues can aid the solution. We propose a blur removal algorithm that exploits information from subsequent camera frames and the built-in inertial sensors of an unmodified smartphone. We extend the fast non-blind uniform blur removal algorithm of Krishnan and Fergus to non-uniform blur and to multiple input frames. We estimate piecewise uniform blur kernels from the gyroscope measurements of the smartphone and we adaptively steer our multiframe deconvolution framework towards the sharpest input patches. We show in qualitative experiments that our algorithm can remove synthetic and real blur from individual frames of a degraded image sequence within a few seconds.

We present a tool for visualizing device interactions in smart environments as a magic lens by augmenting the live camera view of a tablet with relevant connections between recognized devices in the camera's field of view.

Supporting human users when interacting with smart devices is important to drive the successful adoption of the Internet of Things in people's homes and at their workplaces. In this poster contribution, we present a system that helps users control Web-enabled smart things in their environment. Our approach involves a handheld interaction device that recognizes smart things in its view using state-of-the-art visual object recognition techniques. It then augments the camera feed with appropriate interaction primitives such as knobs or buttons for control, and can also display measured values, for instance, when recognizing a sensor. The interaction primitives are generated from user interface descriptions that are embedded in the Web representations of the smart things. Our prototype implementation achieves frame rates that allow for interactive use of the system by human users, and indeed proved to facilitate the interaction with smart things in a demonstration testbed in our research group.

Supporting human users when interacting with smart devices is important to drive the successful adoption of the Internet of Things in people’s homes and at their workplaces. In this poster contribution we present a system that helps users control Web enabled smart things in their environment. Our approach involves a handheld interaction device that recognizes smart things in its view using state of the art visual object recognition techniques. It then augments the camera feed with appropriate interaction primitives such as knobs or buttons for control and can also display measured values for instance when recognizing a sensor. The interaction primitives are generated from user interface descriptions that are embedded in the Web representations of the smart things. Our prototype implementation achieves frame rates that allow for interactive use of the system by human users and indeed proved to facilitate the interaction with smart things in a demonstration testbed in our research group.

As the Internet of Things finds its way into private households, more and more everyday objects communicate with services that are running inside the home and on the Internet. For individuals to trust their smart homes, they should be aware of possibly privacy-sensitive data flows and control commands. In this demo paper, we present a system that combines a real time network analysis tool with an augmented reality user interface to visualize data streams within the home network and to remote services. Our system requires no modifications to a typical home network infrastructure, as it operates by merely observing packets sent over the network.

As the Internet of Things finds its way into private households more and more everyday objects communicate with services that are running inside the home and on the Internet. For individuals to trust their smart homes they should be aware of possibly privacy sensitive data flows and control commands. In this demo paper we present a system that combines a real time network analysis tool with an augmented reality user interface to visualize data streams within the home network and to remote services. Our system requires no modifications to a typical home network infrastructure as it operates by merely observing packets sent over the network.