Search

We propose an approach for the generation of topology-optimized structures with text-guided appearance stylization. This methodology aims to enrich the concurrent design of a structure’s physical functionality and aesthetic appearance. Users can effortlessly input descriptive text to govern the style of the structure. Our system employs a hash-encoded neural network as the implicit structure representation backbone, which serves as the foundation for the co-optimization of structural mechanical performance, style, and connectivity, to ensure full-color, high-quality 3D-printable solutions. We substantiate the effectiveness of our system through extensive comparisons, demonstrations, and a 3D-printing test. [ABSTRACT FROM AUTHOR]

We present a focal surface projection to solve the narrow depth‐of‐field problem in projection mapping applications. We apply a phase‐only spatial light modulator to realize nonuniform focusing distances, whereby the projected contents appear focused on a surface with considerable depth variations. The feasibility of the proposed technique was validated through a physical experiment. [ABSTRACT FROM AUTHOR]

Nature evolves structures like honeycombs at optimized performance with limited material. These efficient structures can be artificially created with the collaboration of structural topology optimization and additive manufacturing. However, the extensive computation cost of topology optimization causes low mesh resolution, long solving time, and rough boundaries that fail to match the requirements for meeting the growing personal fabrication demands and printing capability. Therefore, we propose the neural synthesizing topology optimization that leverages a self‐supervised coordinate‐based network to optimize structures with significantly shorter computation time, where the network encodes the structural material layout as an implicit function of coordinates. Continuous solution space is further generated from optimization tasks under varying boundary conditions or constraints for users' instant inference of novel solutions. We demonstrate the system's efficacy for a broad usage scenario through numerical experiments and 3D printing. [ABSTRACT FROM AUTHOR]

A small pufferfish, Torquigener albomaculosus, is known to construct an elaborate geometric circular structure, which has been referred to as a "mystery circle," with a diameter of ~2 m in the sand of the seabed. We reconstructed a 3D model of this structure for the first time using a "structure from motion" (SfM) algorithm. The mystery circle constructed by the pufferfish may have potential applications for biomimetics similar to the structures constructed by termites and prairie dogs. To support the significance of its structural characteristics, it was observed that the water passing through the valley upstream always gathers in the center of the structure, regardless of the direction of water flow. Furthermore, it has the function of extracting fine-grained sand particles from the valleys and directing these to the center. Computational fluid analysis can be performed immediately using the quantified 3D data, and the structural features of the mystery circle is expected to be applied in a wide range of fields, such as architecture and engineering, via biomimetics. Measurement(s) 3D structure Technology Type(s) SfM Factor Type(s) 3D model Sample Characteristic - Organism Tetraodontidae • Torquigener albomaculosus • pisces Sample Characteristic - Environment sandy seabed • underwater • subtropical sea Sample Characteristic - Location Amami-Oshima Island • Japan • Pacific Ocean [ABSTRACT FROM AUTHOR]

Projector deblurring is an important technology for dynamic projection mapping (PM), where the distance between a projector and a projection surface changes in time. However, conventional projector deblurring techniques do not support dynamic PM because they need to project calibration patterns to estimate the amount of defocus blur each time the surface moves. We present a deep neural network that can compensate for defocus blur in dynamic PM. The primary contribution of this paper is a unique network structure that consists of an extractor and a generator. The extractor explicitly estimates a defocus blur map and a luminance attenuation map. These maps are then injected into the middle layers of the generator network that computes the compensation image. We also propose a pseudo-projection technique for synthesizing physically plausible training data, considering the geometric misregistration that potentially happens in actual PM systems. We conducted simulation and actual PM experiments and confirmed that: (1) the proposed network structure is more suitable than a simple, more general structure for projector deblurring; (2) the network trained with the proposed pseudo-projection technique can compensate projection images for defocus blur artifacts in dynamic PM; and (3) the network supports the translation speed of the surface movement within a certain range that covers normal human motions. [ABSTRACT FROM AUTHOR]

Stereoscopic projection mapping (PM) allows a user to see a three-dimensional (3D) computer-generated (CG) object floating over physical surfaces of arbitrary shapes around us using projected imagery. However, the current stereoscopic PM technology only satisfies binocular cues and is not capable of providing correct focus cues, which causes a vergence-accommodation conflict (VAC). Therefore, we propose a multifocal approach to mitigate VAC in stereoscopic PM. Our primary technical contribution is to attach electrically focus-tunable lenses (ETLs) to active shutter glasses to control both vergence and accommodation. Specifically, we apply fast and periodical focal sweeps to the ETLs, which causes the “virtual image” (as an optical term) of a scene observed through the ETLs to move back and forth during each sweep period. A 3D CG object is projected from a synchronized high-speed projector only when the virtual image of the projected imagery is located at a desired distance. This provides an observer with the correct focus cues required. In this study, we solve three technical issues that are unique to stereoscopic PM: The 3D CG object is displayed on non-planar and even moving surfaces; the physical surfaces need to be shown without the focus modulation; the shutter glasses additionally need to be synchronized with the ETLs and the projector. We also develop a novel compensation technique to deal with the “lens breathing” artifact that varies the retinal size of the virtual image through focal length modulation. Further, using a proof-of-concept prototype, we demonstrate that our technique can present the virtual image of a target 3D CG object at the correct depth. Finally, we validate the advantage provided by our technique by comparing it with conventional stereoscopic PM using a user study on a depth-matching task. [ABSTRACT FROM AUTHOR]

Intrinsic projector calibration is essential in projection mapping (PM) applications, especially in dynamic PM. However, due to the shallow depth-of-field (DOF) of a projector, more work is needed to ensure accurate calibration. We aim to estimate the intrinsic parameters of a projector while avoiding the limitation of shallow DOF. As the core of our technique, we present a practical calibration device that requires a minimal working volume directly in front of the projector lens regardless of the projector's focusing distance and aperture size. The device consists of a flat-bed scanner and pinhole-array masks. For calibration, a projector projects a series of structured light patterns in the device. The pinholes directionally decompose the structured light, and only the projected rays that pass through the pinholes hit the scanner plane. For each pinhole, we extract a ray passing through the optical center of the projector. Consequently, we regard the projector as a pinhole projector that projects the extracted rays only, and we calibrate the projector by applying the standard camera calibration technique, which assumes a pinhole camera model. Using a proof-of-concept prototype, we demonstrate that our technique can calibrate projectors with different focusing distances and aperture sizes at the same accuracy as a conventional method. Finally, we confirm that our technique can provide intrinsic parameters accurate enough for a dynamic PM application, even when a projector is placed too far from a projection target for a conventional method to calibrate the projector using a fiducial object of reasonable size. [ABSTRACT FROM AUTHOR]

We present computational phase-modulated eyeglasses, a see-through optical system that modulates the view of the user using phase-only spatial light modulators (PSLM). A PSLM is a programmable reflective device that can selectively retardate, or delay, the incoming light rays. As a result, a PSLM works as a computational dynamic lens device. We demonstrate our computational phase-modulated eyeglasses with either a single PSLM or dual PSLMs and show that the concept can realize various optical operations including focus correction, bi-focus, image shift, and field of view manipulation, namely optical zoom. Compared to other programmable optics, computational phase-modulated eyeglasses have the advantage in terms of its versatility. In addition, we also presents some prototypical focus-loop applications where the lens is dynamically optimized based on distances of objects observed by a scene camera. We further discuss the implementation, applications but also discuss limitations of the current prototypes and remaining issues that need to be addressed in future research. [ABSTRACT FROM AUTHOR]

Background: Dementia is one the major problems of aging societies, and, novel and effective non-drug therapies are required as interventions in the oldest-old to prevent cognitive decline. Objective: This study aims to examine the efficacy and safety of reminiscence using immersive virtual reality (iVR reminiscence) focusing on anxiety that often appears with cognitive decline. The secondary objective is to reveal the preference for VR image types for reminiscence: live-action (LA) or computer graphics (CG). Methods: This was a pilot, open-label, and randomized crossover study which was conducted on January 2020 at a single nursing home. The subjects were randomly divided into two groups (A or B) in equal numbers, and they alternately viewed two types of VR images (LA and CG) themed on the mid- to late Showa era (A.D. 1955–1980) in Japan. In group A, the CG images were viewed first, and then the LA images were viewed (CG→ LA). In group B, the images were viewed in the opposite order (LA→ CG). Before VR viewing, subjects responded to Mini-Mental State Examination (MMSE) Japanese version and State-Trait Anxiety Inventory (STAI) Japanese version. After viewing the first and second VR, subjects responded to STAI and the numerical rating scale (NRS) for satisfaction and side effects (nausea, dizziness, headache, and tiredness). Results: Ten subjects participated in this study. The values of analyses are presented in the mean (SD). The age was 87.1 years (4.2), and the MMSE was 28.5 (1.8). The total STAI score before VR viewing was 36.1 (7.2), but it significantly decreased to 26.8 (4.9) after the first VR viewing (P = 0.0010), and further decreased to 23.4 (2.8) after the second VR viewing (P < 0.001). The NRS score for satisfaction tended to be higher after viewing LA in group A (CG→ LA) (CG vs. LA; 7.0 (2.3) vs. 8.6 (1.5), P = 0.0993), while in group B (LA→ CG), the score after CG was slightly lower than that after LA. There were no serious side effects. Conclusions: This study suggests that iVR reminiscence can reduce anxiety in the oldest-old without causing serious side effects. Furthermore, the impacts might be better with LA images. [ABSTRACT FROM AUTHOR]

Aiming at realizing novel vision augmentation experiences, this paper proposes the IlluminatedFocus technique, which spatially defocuses real-world appearances regardless of the distance from the user's eyes to observed real objects. With the proposed technique, a part of a real object in an image appears blurred, while the fine details of the other part at the same distance remain visible. We apply Electrically Focus-Tunable Lenses (ETL) as eyeglasses and a synchronized high-speed projector as illumination for a real scene. We periodically modulate the focal lengths of the glasses (focal sweep) at more than 60 Hz so that a wearer cannot perceive the modulation. A part of the scene to appear focused is illuminated by the projector when it is in focus of the user's eyes, while another part to appear blurred is illuminated when it is out of the focus. As the basis of our spatial focus control, we build mathematical models to predict the range of distance from the ETL within which real objects become blurred on the retina of a user. Based on the blur range, we discuss a design guideline for effective illumination timing and focal sweep range. We also model the apparent size of a real scene altered by the focal length modulation. This leads to an undesirable visible seam between focused and blurred areas. We solve this unique problem by gradually blending the two areas. Finally, we demonstrate the feasibility of our proposal by implementing various vision augmentation applications. [ABSTRACT FROM AUTHOR]

This paper presents a novel active marker for dynamic projection mapping (PM) that emits a temporal blinking pattern of infrared (IR) light representing its ID. We used a multi-material three dimensional (3D) printer to fabricate a projection object with optical fibers that can guide IR light from LEDs attached on the bottom of the object. The aperture of an optical fiber is typically very small; thus, it is unnoticeable to human observers under projection and can be placed on a strongly curved part of a projection surface. In addition, the working range of our system can be larger than previous marker-based methods as the blinking patterns can theoretically be recognized by a camera placed at a wide range of distances from markers. We propose an automatic marker placement algorithm to spread multiple active markers over the surface of a projection object such that its pose can be robustly estimated using captured images from arbitrary directions. We also propose an optimization framework for determining the routes of the optical fibers in such a way that collisions of the fibers can be avoided while minimizing the loss of light intensity in the fibers. Through experiments conducted using three fabricated objects containing strongly curved surfaces, we confirmed that the proposed method can achieve accurate dynamic PMs in a significantly wide working range. [ABSTRACT FROM AUTHOR]

The appearance of fabric motion is suggested to affect the human perception of bending stiffness. This study presents a novel spatial augmented reality, or projection mapping, approach that can visually manipulate the perceived bending stiffness of a fabric. Particularly, we proposed a flow enhancement method that can change the apparent fabric motion by using a simple optical flow analysis technique rather than complex physical simulations for interactive applications. Through a psychophysical experiment, we investigated the relationship between the magnification factor of our flow enhancement and the perceived bending stiffness of a fabric. Furthermore, we constructed a prototype application system that allows users to control the stiffness of a fabric without changing the actual physical fabric. By evaluating the prototype, we confirmed that the proposed technique can manipulate the perceived stiffness of various materials (i.e., cotton, polyester, and mixed cotton and linen) at an average accuracy of 90.3 percent. [ABSTRACT FROM AUTHOR]

This paper proposes a learning-based quality evaluation framework for inpainted results that does not require any subjectively annotated training data. Image inpainting, which removes and restores unwanted regions in images, is widely acknowledged as a task whose results are quite difficult to evaluate objectively. Thus, existing learning-based image quality assessment (IQA) methods for inpainting require subjectively annotated data for training. However, subjective annotation requires huge cost and subjects' judgment occasionally differs from person to person in accordance with the judgment criteria. To overcome these difficulties, the proposed framework generates and uses simulated failure results of inpainted images whose subjective qualities are controlled as the training data. We also propose a masking method for generating training data towards fully automated training data generation. These approaches make it possible to successfully estimate better inpainted images, even though the task is quite subjective. To demonstrate the effectiveness of our approach, we test our algorithm with various datasets and show it outperforms existing IQA methods for inpainting. [ABSTRACT FROM AUTHOR]

This paper presents an approach for non-contact haptic augmentation with spatial augmented reality (SAR). We construct a thermo-visual projection system by combining a standard RGB projector and a fabricated infrared (IR) projector. The primary contribution of this paper is that we conduct thorough psychophysical experiments to investigate a design guideline for spatiotemporal projection patterns for both RGB and IR projectors to render a warm object with high presence. We develop application systems to evaluate the validity of the proposed system and design guideline. The evaluation results demonstrate that the proposed system can render warm objects with significantly higher presence than a standard SAR system. [ABSTRACT FROM AUTHOR]

This paper introduces a novel photometric compensation technique for inter-projector luminance and chrominance variations. Although it sounds as a classical technical issue, to the best of our knowledge there is no existing solution to alleviate the spatial non-uniformity among strongly heterogeneous projectors at perceptually acceptable quality. Primary goal of our method is increasing the perceived seamlessness of the projection system by automatically generating an improved and consistent visual quality. It builds upon the existing research of multi-projection systems, but instead of working with perceptually non-uniform color spaces such as CIEXYZ, the overall computation is carried out using the RLab color appearance model which models the color processing in an adaptive, perceptual manner. Besides, we propose an adaptive color gamut acquisition, spatially varying gamut mapping, and optimization framework for edge blending. The paper describes the overall workflow and detailed algorithm of each component, followed by an evaluation validating the proposed method. The experimental results both qualitatively and quantitatively show the proposed method significant improved the visual quality of projected results of a multi-projection display with projectors with severely heterogeneous color processing. [ABSTRACT FROM AUTHOR]

Spatial augmented reality (SAR) pursues realism in rendering materials and objects. To advance this goal, we propose a hybrid SAR (HySAR) that combines a projector with optical see-through head-mounted displays (OST-HMD). In an ordinary SAR scenario with co-located viewers, the viewers perceive the same virtual material on physical surfaces. In general, the material consists of two components: a view-independent (VI) component such as diffuse reflection, and a view-dependent (VD) component such as specular reflection. The VI component is static over viewpoints, whereas the VD should change for each viewpoint even if a projector can simulate only one viewpoint at one time. In HySAR, a projector only renders the static VI components. In addition, the OST-HMD renders the dynamic VD components according to the viewer's current viewpoint. Unlike conventional SAR, the HySAR concept theoretically allows an unlimited number of co-located viewers to see the correct material over different viewpoints. Furthermore, the combination enhances the total dynamic range, the maximum intensity, and the resolution of perceived materials. With proof-of-concept systems, we demonstrate HySAR both qualitatively and quantitatively with real objects. First, we demonstrate HySAR by rendering synthetic material properties on a real object from different viewpoints. Our quantitative evaluation shows that our system increases the dynamic range by 2.24 times and the maximum intensity by 2.12 times compared to an ordinary SAR system. Second, we replicate the material properties of a real object by SAR and HySAR, and show that HySAR outperforms SAR in rendering VD specular components. [ABSTRACT FROM PUBLISHER]

We propose a visual marker embedding method for the pose estimation of a projection surface to correctly map projected images onto the surface. Assuming that the surface is fabricated by a full-color or multi-material three-dimensional (3D) printer, we propose to automatically embed visual markers on the surface with mechanical accuracy. The appearance of the marker is designed such that the marker is detected by infrared cameras even when printed on a non-planar surface while its appearance can be diminished by the projection to be as imperceptible as possible to human observers. The marker placement is optimized using a genetic algorithm to maximize the number of valid viewpoints from which the pose of the object can be estimated correctly using a stereo camera system. We also propose a radiometric compensation technique to quickly diminish the marker appearance. Experimental results confirm that the pose of projection objects are correctly estimated while the appearance of the markers was diminished to an imperceptible level. At the same time, we confirmed the limitations of the current method; only one object can be handled, and pose estimation is not performed at interactive frame rates. Finally, we demonstrate the proposed technique to show that it works successfully for various surface shapes and target textures. [ABSTRACT FROM AUTHOR]

This paper presents a novel projected pixel localization principle for online geometric registration in dynamic projection mapping applications. We propose applying a time measurement of a laser projector raster-scanning beam using a photosensor to estimate its position while the projector displays meaningful visual information to human observers. Based on this principle, we develop two types of position estimation techniques. One estimates the position of a projected beam when it directly illuminates a photosensor. The other localizes a beam by measuring the reflection from a retro-reflective marker with the photosensor placed in the optical path of the projector. We conduct system evaluations using prototypes to validate this method as well as to confirm the applicability of our principle. In addition, we discuss the technical limitations of the prototypes based on the evaluation results. Finally, we build several dynamic projection mapping applications to demonstrate the feasibility of our principle. [ABSTRACT FROM AUTHOR]

This paper presents the extended work on LazyNav, a head-free, eyes-free and hands-free mid-air ground navigation control model presented at the IEEE 3D User Interfaces (3DUI) 2015, in particular with a new application to the head-mounted display (HMD). Our mid-air interaction metaphor makes use of only a single pair of the remaining tracked body elements to tailor the navigation. Therefore, the user can navigate in the scene while still being able to perform other interactions with her hands and head, e.g., carrying a bag, grasping a cup of coffee, or observing the content by moving her eyes and locally rotating her head. We design several body motions for navigation by considering the use of non-critical body parts and develop assumptions about ground navigation techniques. Through the user studies, we investigate the motions that are easy to discover, easy to control, socially acceptable, accurate and not tiring. Finally, we evaluate the desired ground navigation features with a prototype application in both a large display (LD) and a HMD navigation scenarios. We highlight several recommendations for designing a particular mid-air ground navigation technique for a LD and a HMD. [ABSTRACT FROM AUTHOR]

We propose the first system for live dynamic augmentation of human faces. Using projector-based illumination, we alter the appearance of human performers during novel performances. The key challenge of live augmentation is latency - an image is generated according to a specific pose, but is displayed on a different facial configuration by the time it is projected. Therefore, our system aims at reducing latency during every step of the process, from capture, through processing, to projection. Using infrared illumination, an optically and computationally aligned high-speed camera detects facial orientation as well as expression. The estimated expression blendshapes are mapped onto a lower dimensional space, and the facial motion and non-rigid deformation are estimated, smoothed and predicted through adaptive Kalman filtering. Finally, the desired appearance is generated interpolating precomputed offset textures according to time, global position, and expression. We have evaluated our system through an optimized CPU and GPU prototype, and demonstrated successful low latency augmentation for different performers and performances with varying facial play and motion speed. In contrast to existing methods, the presented system is the first method which fully supports dynamic facial projection mapping without the requirement of any physical tracking markers and incorporates facial expressions. [ABSTRACT FROM AUTHOR]

Animated animatronic figures are a unique way to give physical presence to a character. However, their movement and expressions are often limited due to mechanical constraints. In this paper, we propose a complete process for augmenting physical avatars using projector-based illumination, significantly increasing their expressiveness. Given an input animation, the system decomposes the motion into low-frequency motion that can be physically reproduced by the animatronic head and high-frequency details that are added using projected shading. At the core is a spatio-temporal optimization process that compresses the motion in gradient space, ensuring faithful motion replay while respecting the physical limitations of the system. We also propose a complete multi-camera and projection system, including a novel defocused projection and subsurface scattering compensation scheme. The result of our system is a highly expressive physical avatar that features facial details and motion otherwise unattainable due to physical constraints. [ABSTRACT FROM AUTHOR]

We propose a method to calibrate viewpoint-dependent, channel-wise image blur of near-eye displays, especially of Optical See-Through Head-Mounted Displays (OST-HMDs). Imperfections in HMD optics cause channel-wise image shift and blur that degrade the image quality of the display at a user's viewpoint. If we can estimate such characteristics perfectly, we could mitigate the effect by applying correction techniques from the computational photography in computer vision as analogous to cameras. Unfortunately, directly applying existing calibration techniques of cameras to OST-HMDs is not a straightforward task. Unlike ordinary imaging systems, image blur in OST-HMDs is viewpoint-dependent, i.e., the optical characteristic of a display dynamically changes depending on the current viewpoint of the user. This constraint makes the problem challenging since we must measure image blur of an HMD, ideally, over the entire 3D eyebox in which a user can see an image. To overcome this problem, we model the viewpoint-dependent blur as a Gaussian Light Field (GLF) that stores spatial information of the display screen as a (4D) light field with depth information and the blur as point-spread functions in the form of Gaussian kernels, respectively. We first describe both our GLF model and a calibration procedure to learn a GLF for a given OST-HMD. We then apply our calibration method to two HMDs that use different optics: a cubic prism or holographic gratings. The results show that our method achieves significantly better accuracy in Point-Spread Function (PSF) estimations with an accuracy about 2 to 7 dB in Peak SNR. [ABSTRACT FROM PUBLISHER]

We propose a novel inter-reflection compensation technique for immersive projection displays wherein we spatially modulate the reflectance pattern on the screen to improve the compensation performance of conventional methods. As the luminance of light reflected on a projection surface is mathematically represented as the multiplication of the illuminance of incident light and the surface reflectance, we can reduce undesirable intensity elevation because of inter-reflections by decreasing surface reflectance. Based on this principle, we improve conventional inter-reflection compensation techniques by applying reflectance pattern modulation. We realize spatial reflectance modulation of a projection screen by painting it with a photochromic compound, which changes its color (i.e., the reflectance of the screen) when ultraviolet (UV) light is applied and by controlling UV irradiation with a UV LED array placed behind the screen. The main contribution of this paper is a computational model to optimize a reflectance pattern for the accurate reproduction of a target appearance by decreasing the intensity elevation caused by inter-reflection while maintaining the maximum intensity of the target appearance. Through simulation and physical experiments, we demonstrate the feasibility of the proposed model and confirm its advantage over conventional methods. [ABSTRACT FROM AUTHOR]

In this special issue of IEEE Transactions on Visualization and Computer Graphics (TVCG), we are pleased to present the journal papers from the 21st IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2022), which will be held as a hybrid conference between October 17 and 21, 2022 in Singapore. ISMAR continues the over twenty year long tradition of IWAR, ISMR, and ISAR, and is the premier conference for Mixed and Augmented Reality in the world. [ABSTRACT FROM AUTHOR]

We present SoftAR, a novel spatial augmented reality (AR) technique based on a pseudo-haptics mechanism that visually manipulates the sense of softness perceived by a user pushing a soft physical object. Considering the limitations of projection-based approaches that change only the surface appearance of a physical object, we propose two projection visual effects, i.e., surface deformation effect (SDE) and body appearance effect (BAE), on the basis of the observations of humans pushing physical objects. The SDE visualizes a two-dimensional deformation of the object surface with a controlled softness parameter, and BAE changes the color of the pushing hand. Through psychophysical experiments, we confirm that the SDE can manipulate softness perception such that the participant perceives significantly greater softness than the actual softness. Furthermore, fBAE, in which BAE is applied only for the finger area, significantly enhances manipulation of the perception of softness. We create a computational model that estimates perceived softness when SDE+fBAE is applied. We construct a prototype SoftAR system in which two application frameworks are implemented. The softness adjustment allows a user to adjust the softness parameter of a physical object, and the softness transfer allows the user to replace the softness with that of another object. [ABSTRACT FROM AUTHOR]

This paper proposes a novel radiometric compensation technique for cooperative projection system based-on distributed optimization. To achieve high scalability and robustness, we assume cooperative projection environments such that 1. each projector does not have information about other projectors as well as target images, 2. the camera does not have information about the projectors either, while having the target images, and 3. only a broadcast communication from the camera to the projectors is allowed to suppress the data transfer bandwidth. To this end, we first investigate a distributed optimization based feedback mechanism that is suitable for the required decentralized information processing environment. Next, we show that this mechanism works well for still image projection, however not necessary for moving images due to the lack of dynamic responsiveness. To overcome this issue, we propose to implement an additional feedforward mechanism. Such a 2 Degree Of Freedom (2-DOF) control structure is well-known in control engineering community as a typical method to enhance not only disturbance rejection but also reference tracking capability, simultaneously. We theoretically guarantee and experimentally demonstrate that this 2-DOF structure yields the moving image projection accuracy that is overwhelming the best achievable performance only by the distributed optimization mechanisms. [ABSTRACT FROM AUTHOR]

A simple and cost-efficient method for extending a projector's depth-of-field (DOF) is proposed. By leveraging liquid lens technology, we can periodically modulate the focal length of a projector at a frequency that is higher than the critical flicker fusion (CFF) frequency. Fast periodic focal length modulation results in forward and backward sweeping of focusing distance. Fast focal sweep projection makes the point spread function (PSF) of each projected pixel integrated over a sweep period (IPSF; integrated PSF) nearly invariant to the distance from the projector to the projection surface as long as it is positioned within sweep range. This modulation is not perceivable by human observers. Once we compensate projection images for the IPSF, the projected results can be focused at any point within the range. Consequently, the proposed method requires only a single offline PSF measurement; thus, it is an open-loop process. We have proved the approximate invariance of the projector's IPSF both numerically and experimentally. Through experiments using a prototype system, we have confirmed that the image quality of the proposed method is superior to that of normal projection with fixed focal length. In addition, we demonstrate that a structured light pattern projection technique using the proposed method can measure the shape of an object with large depth variances more accurately than normal projection techniques. [ABSTRACT FROM AUTHOR]

SUMMARY More and more recent studies of emotion estimation have taken the approach of ambient sensing, where users are monitored without constraining their bodies and without being aware of the sensing system. Few of them, however, focus on audience emotions. We attempted to examine audience emotions, which are difficult to estimate when using only individual data, and which come to be estimated more easily when using ambient sensing of synchrony among multiple users. We developed an ambient sensing chair, which employs wireless sensors to determine the center of force of the seated user, and designed an experiment for an audience watching a soccer game on TV or a technical presentation in order to examine the relationship between audience body sway and emotional states in a natural situation. We found that synchrony could be well correlated with audience excitement and boredom, something not observable in the individual data. These results indicate the possibility of development of service marketing systems that evaluate TV content, and feedback systems that give notice of the emotional state of an audience to a presenter. [ABSTRACT FROM AUTHOR]

This paper presents a novel technique to reduce the motion blur artifacts of foveal projection in a dynamic focus-plus-context (DF+C) display. The DF+C display is generally configured with multiple projectors and provides a nonuniform spatial resolution that consists of high-resolution (hi-res) regions (foveal projection) and low-resolution regions (peripheral projection). A serious problem of the DF+C display is motion blur, which inevitably occurs when a foveal projection is moved by a pan-tilt mirror or gantry. We propose a solution that reduces the motion blur artifacts, and evaluate how this solution improves the image quality using both qualitative and quantitative experiments. Our proposed method defines an error function to assess the displayed image quality as the difference between an original hi-res image and the displayed image by considering the nonuniform spatial property of human visual acuity. Then, it decides the set of positions and moving techniques of foveal projections so that the sum of errors during a video sequence is minimized. Through experiment, we confirmed that the proposed method can provide a better image quality and significantly improve the motion blur artifacts when compared with a conventional DF+C display. [ABSTRACT FROM AUTHOR]

In this paper, we propose a projection-based mixed reality system that visualizes the tangential deformation of a nonrigid surface by superimposing graphics directly onto the surface by projected imagery. The superimposed graphics are deformed according to the surface deformation. To achieve this goal, we develop a computer vision technique that estimates the tangential deformation by measuring the frame-by-frame movement of an infrared (IR) texture on the surface. IR ink, which can be captured by an IR camera under IR light, but is invisible to the human eye, is used to provide the surface texture. Consequently, the texture does not degrade the image quality of the augmented graphics. The proposed technique measures individually the surface motion between two successive frames. Therefore, it does not suffer from occlusions caused by interactions and allows touching, pushing, pulling, and pinching, etc. The moving least squares technique interpolates the measured result to estimate denser surface deformation. The proposed method relies only on the apparent motion measurement; thus, it is not limited to a specific deformation characteristic, but is flexible for multiple deformable materials, such as viscoelastic and elastic materials. Experiments confirm that, with the proposed method, we can visualize the surface deformation of various materials by projected illumination, even when the user's hand occludes the surface from the camera. [ABSTRACT FROM AUTHOR]

SUMMARY This paper presents a technique for writing-state classification by sedentary behavior sensing with a sensor-equipped chair, which wirelessly monitors the user's states such as behaviors or physiological and psychological conditions, for estimation of the user's subjective difficulty with a studying problem on a desktop. Avoiding user discomfort, four load cells are mounted behind the seat plate of a regular office chair. The system simply measures the weight and the center of pressure of the seat. These data are divided into time segments, which are labeled into four sedentary-body sway primitives by the decision tree algorithm, and then the behaviors are derived from the number and pitch of those primitives in a certain interval. In system-evaluation experiments, it achieved more than 80% labeling accuracy for untrained users. We conducted a user experiment showing its potential for desk work applications. The results of the experiment showed that our proposed system could classify user states into the writing state and other states such as reading a document and watching a movie at a classification rate of 87%. [ABSTRACT FROM AUTHOR]

This paper presents a shadow removal technique for a multiple overlapping projection system. In particular, this paper deals with situations where cameras cannot be placed between the occluder and projection surface. We apply a synthetic aperture capturing technique to estimate the appearance of the projection surface, and a visual hull reconstruction technique to measure the shape of the occluder. Once the shape is acquired, shadow regions on the surface can be estimated. The proposed shadow removal technique allows users to balance between the following two criteria: the likelihood of new shadow emergence and the spatial resolution of the projected results. Through a real projection experiment, we evaluate the proposed shadow removal technique [ABSTRACT FROM AUTHOR]

We introduce a shadow-based interface for interactive tabletops. The proposed interface allows a user to browse graphical information by casting the shadow of his/her body, such as a hand, on a tabletop surface. Central to our technique is a new optical design that utilizes polarization in addition to the additive nature of light so that the desired graphical information is displayed only in a shadow area on a tabletop surface. In other words, our technique conceals the graphical information on surfaces other than the shadow area, such as the surface of the occluder and non-shadow areas on the tabletop surface. We combine the proposed shadow-based interface with a multi-touch detection technique to realize a novel interaction technique for interactive tabletops. We implemented a prototype system and conducted proof-of-concept experiments along with a quantitative evaluation to assess the feasibility of the proposed optical design. Finally, we showed implemented application systems of the proposed shadow-based interface. [ABSTRACT FROM PUBLISHER]