Search

In Redirected Walking (RDW), resets are an overt method that explicitly interrupts users, and they should be avoided to provide a quality user experience. The number of resets depends on the configuration of the physical environment; thus, inappropriate object placement can lead to frequent resets, causing motion sickness and degrading presence. However, estimating the number of resets based on the physical layout is challenging. It is difficult to measure reset frequency with real users repeatedly testing different layouts, and virtual simulations offer limited real-time verification. As a result, while rearranging objects can reduce resets, users have not been able to fully take advantage of this opportunity, highlighting the need for rapid assessment of object placement. To address this, in Study 1, we collected simulation data and analyzed the average number of resets for various object placements. In study 2, we developed a system that allows users to evaluate reset frequency using a real-time placement interface powered by the first learning-based reset prediction model. Our model predicts resets from top-down views of the physical space, leveraging a Vision Transformer architecture. The model achieved a root mean square error (RMSE) of $23.88$. We visualized the model's attention scores using heatmaps to analyze the regions of focus during prediction. Through the interface, users can reorganize furniture while instantly observing the change in the predicted number of resets, thus improving their interior for a better RDW experience with fewer resets.

Large language models (LLMs) have been shown to be effective on tabular prediction tasks in the low-data regime, leveraging their internal knowledge and ability to learn from instructions and examples. However, LLMs can fail to generate predictions that satisfy group fairness, that is, produce equitable outcomes across groups. Critically, conventional debiasing approaches for natural language tasks do not directly translate to mitigating group unfairness in tabular settings. In this work, we systematically investigate four empirical approaches to improve group fairness of LLM predictions on tabular datasets, including fair prompt optimization, soft prompt tuning, strategic selection of few-shot examples, and self-refining predictions via chain-of-thought reasoning. Through experiments on four tabular datasets using both open-source and proprietary LLMs, we show the effectiveness of these methods in enhancing demographic parity while maintaining high overall performance. Our analysis provides actionable insights for practitioners in selecting the most suitable approach based on their specific requirements and constraints.

Hand-face interactions play a key role in many everyday tasks, providing insights into user habits, behaviors, intentions, and expressions. However, existing wearable sensing systems often struggle to track these interactions in daily settings due to their reliance on multiple sensors or privacy-sensitive, vision-based approaches. To address these challenges, we propose WristSonic, a wrist-worn active acoustic sensing system that uses speakers and microphones to capture ultrasonic reflections from hand, arm, and face movements, enabling fine-grained detection of hand-face interactions with minimal intrusion. By transmitting and analyzing ultrasonic waves, WristSonic distinguishes a wide range of gestures, such as tapping the temple, brushing teeth, and nodding, using a Transformer-based neural network architecture. This approach achieves robust recognition of 21 distinct actions with a single, low-power, privacy-conscious wearable. Through two user studies with 15 participants in controlled and semi-in-the-wild settings, WristSonic demonstrates high efficacy, achieving macro F1-scores of 93.08% and 82.65%, respectively., Comment: 21 Pages, 10 Figures, Under Review

Eye gaze is considered a promising interaction modality in extende reality (XR) environments. However, determining selection intention from gaze data often requires additional manual selection techniques. We present a Bayesian-based machine learning (ML) model to predict user selection intention in real-time using only gaze data. Our model uses a Bayesian approach to transform gaze data into selection probabilities, which are then fed into an ML model to discriminate selection intentions. In Study 1, our model achieved real-time inference with an accuracy of 0.97 and an F1 score of 0.96. In Study 2, we found that the selection intention inferred by our model enables more comfortable and accurate interactions compared to traditional techniques., Comment: 18 pages, 6 figures, 2 Tables

Burst image super-resolution has been a topic of active research in recent years due to its ability to obtain a high-resolution image by using complementary information between multiple frames in the burst. In this work, we explore using burst shots with non-uniform exposures to confront real-world practical scenarios by introducing a new benchmark dataset, dubbed Non-uniformly Exposed Burst Image (NEBI), that includes the burst frames at varying exposure times to obtain a broader range of irradiance and motion characteristics within a scene. As burst shots with non-uniform exposures exhibit varying levels of degradation, fusing information of the burst shots into the first frame as a base frame may not result in optimal image quality. To address this limitation, we propose a Frame Selection Network (FSN) for non-uniform scenarios. This network seamlessly integrates into existing super-resolution methods in a plug-and-play manner with low computational costs. The comparative analysis reveals the effectiveness of the nonuniform setting for the practical scenario and our FSN on synthetic-/real- NEBI datasets., Comment: CVPR2024W NTIRE accepted

Seams are areas of overlapping fabric formed by stitching two or more pieces of fabric together in the cut-and-sew apparel manufacturing process. In SeamPose, we repurposed seams as capacitive sensors in a shirt for continuous upper-body pose estimation. Compared to previous all-textile motion-capturing garments that place the electrodes on the clothing surface, our solution leverages existing seams inside of a shirt by machine-sewing insulated conductive threads over the seams. The unique invisibilities and placements of the seams afford the sensing shirt to look and wear similarly as a conventional shirt while providing exciting pose-tracking capabilities. To validate this approach, we implemented a proof-of-concept untethered shirt with 8 capacitive sensing seams. With a 12-participant user study, our customized deep-learning pipeline accurately estimates the relative (to the pelvis) upper-body 3D joint positions with a mean per joint position error (MPJPE) of 6.0 cm. SeamPose represents a step towards unobtrusive integration of smart clothing for everyday pose estimation.

We present Ring-a-Pose, a single untethered ring that tracks continuous 3D hand poses. Located in the center of the hand, the ring emits an inaudible acoustic signal that each hand pose reflects differently. Ring-a-Pose imposes minimal obtrusions on the hand, unlike multi-ring or glove systems. It is not affected by the choice of clothing that may cover wrist-worn systems. In a series of three user studies with a total of 30 participants, we evaluate Ring-a-Pose's performance on pose tracking and micro-finger gesture recognition. Without collecting any training data from a user, Ring-a-Pose tracks continuous hand poses with a joint error of 14.1mm. The joint error decreases to 10.3mm for fine-tuned user-dependent models. Ring-a-Pose recognizes 7-class micro-gestures with a 90.60% and 99.27% accuracy for user-independent and user-dependent models, respectively. Furthermore, the ring exhibits promising performance when worn on any finger. Ring-a-Pose enables the future of smart rings to track and recognize hand poses using relatively low-power acoustic sensing.

Our hands serve as a fundamental means of interaction with the world around us. Therefore, understanding hand poses and interaction context is critical for human-computer interaction. We present EchoWrist, a low-power wristband that continuously estimates 3D hand pose and recognizes hand-object interactions using active acoustic sensing. EchoWrist is equipped with two speakers emitting inaudible sound waves toward the hand. These sound waves interact with the hand and its surroundings through reflections and diffractions, carrying rich information about the hand's shape and the objects it interacts with. The information captured by the two microphones goes through a deep learning inference system that recovers hand poses and identifies various everyday hand activities. Results from the two 12-participant user studies show that EchoWrist is effective and efficient at tracking 3D hand poses and recognizing hand-object interactions. Operating at 57.9mW, EchoWrist is able to continuously reconstruct 20 3D hand joints with MJEDE of 4.81mm and recognize 12 naturalistic hand-object interactions with 97.6% accuracy.

Optical spectroscopy the measurement of electromagnetic spectra is fundamental to various scientific domains and serves as the building block of numerous technologies. Computational spectrometry is an emerging field that employs an array of photodetectors with different spectral responses or a single photodetector device with tunable spectral response, in conjunction with numerical algorithms, for spectroscopic measurements. Compact single photodetectors made from layered materials are particularly attractive, since they eliminate the need for bulky mechanical and optical components used in traditional spectrometers and can easily be engineered as heterostructures to optimize device performance. However, compact tunable photodetectors are typically nonlinear devices and this adds complexity to extracting optical spectra from the device response. Here, we report on the training of an artificial neural network (ANN) to recover the full nonlinear spectral photoresponse of a nonlinear problem of high dimensionality of a single GeSe-InSe p-n heterojunction device. We demonstrate the functionality of a calibrated spectrometer in the spectral range of 400-1100 nm, with a small device footprint of ~25X25 micrometers, and we achieve a mean reconstruction error of 0.0002 for the power-spectrum at a spectral resolution of 0.35 nm. Using our device, we demonstrate a solution to metamerism, an apparent matching of colors with different power spectral distributions, which is a fundamental problem in optical imaging., Comment: 15 pages, 5 figures