Search

Arbitrary varying channels (AVC) are used to model communication settings in which a channel state may vary arbitrarily over time. Their primary objective is to circumvent statistical assumptions on channel variation. Traditional studies on AVCs optimize rate subject to the worst-case state sequence. While this approach is resilient to channel variations, it may result in low rates for state sequences that are associated with relatively good channels. This paper addresses the analysis of AVCs through the lens of competitive analysis, where solution quality is measured with respect to the optimal solution had the state sequence been known in advance. Our main result demonstrates that codes constructed by a single input distribution do not achieve optimal competitive performance over AVCs. This stands in contrast to the single-letter capacity formulae for AVCs, and it indicates, in our setting, that even though the encoder cannot predict the subsequent channel states, it benefits from varying its input distribution as time proceeds.

We consider the problem of remotely stabilizing a linear dynamical system. In this setting, a sensor co-located with the system communicates the system's state to a controller over a noisy communication channel with feedback. The objective of the controller (decoder) is to use the channel outputs to estimate the vector state with finite zero-delay mean squared error (MSE) at the infinite horizon. It has been shown in [1] that for a vector Gauss-Markov source and either a single-input multiple-output (SIMO) or a multiple-input single-output (MISO) channel, linear codes require the minimum capacity to achieve finite MSE. This paper considers the more general problem of linear zero-delay joint-source channel coding (JSCC) of a vector-valued source over a multiple-input multiple-output (MIMO) Gaussian channel with feedback. We study sufficient and necessary conditions for linear codes to achieve finite MSE. For sufficiency, we introduce a coding scheme where each unstable source mode is allocated to a single channel for estimation. Our proof for the necessity of this scheme relies on a matrix-algebraic conjecture that we prove to be true if either the source or channel is scalar. We show that linear codes achieve finite MSE for a scalar source over a MIMO channel if and only if the best scalar sub-channel can achieve finite MSE. Finally, we provide a new counter-example demonstrating that linear codes are generally sub-optimal for coding over MIMO channels., Comment: Accepted for presentation at the 2024 IEEE International Symposium on Information Theory

In lead halide perovskites (APbX3), the effect of the A-site cation on optical and electronic properties has initially been thought to be marginal. Yet, evidence of beneficial effects on solar cell performance and light emission is accumulating. Here, we report that the A-cation in soft APbBr3 colloidal quantum dots (QDs) controls the phonon-induced localization of the exciton wavefunction. Insights from ab initio molecular dynamics and single-particle fluorescence spectroscopy demonstrate that anharmonic lattice vibrations and the resulting polymorphism act as an additional confinement potential. Avoiding the trade-off between single-photon purity and optical stability faced by downsizing conventional QDs into the strong confinement regime, dynamical phonon-induced confinement in large organic-inorganic perovskite QDs enables bright (10^6 photons/s), stable (> 1h), and pure (> 95%) single-photon emission in a widely tuneable spectral range (495-745 nm). Strong electron-phonon interaction in soft perovskite QDs provides an unconventional route toward the development of scalable room-temperature quantum light sources.

We present super-resolved coherent anti-Stokes Raman scattering (CARS) microscopy by implementing phase-resolved image scanning microscopy (ISM), achieving up to two-fold resolution increase as compared with a conventional CARS microscope. Phase-sensitivity is required for the standard pixel-reassignment procedure since the scattered field is coherent, thus the point-spread function (PSF) is well-defined only for the field amplitude. We resolve the complex field by a simple add-on to the CARS setup enabling inline interferometry. Phase-sensitivity offers additional contrast which informs the spatial distribution of both resonant and nonresonant scatterers. As compared with alternative super-resolution schemes in coherent nonlinear microscopy, the proposed method is simple, requires only low-intensity excitation, and is compatible with any conventional forward-detected CARS imaging setup.

Personal devices have adopted diverse authentication methods, including biometric recognition and passcodes. In contrast, headphones have limited input mechanisms, depending solely on the authentication of connected devices. We present Moonwalk, a novel method for passive user recognition utilizing the built-in headphone accelerometer. Our approach centers on gait recognition; enabling users to establish their identity simply by walking for a brief interval, despite the sensor's placement away from the feet. We employ self-supervised metric learning to train a model that yields a highly discriminative representation of a user's 3D acceleration, with no retraining required. We tested our method in a study involving 50 participants, achieving an average F1 score of 92.9% and equal error rate of 2.3%. We extend our evaluation by assessing performance under various conditions (e.g. shoe types and surfaces). We discuss the opportunities and challenges these variations introduce and propose new directions for advancing passive authentication for wearable devices.

Hand gesture recognition is becoming a more prevalent mode of human-computer interaction, especially as cameras proliferate across everyday devices. Despite continued progress in this field, gesture customization is often underexplored. Customization is crucial since it enables users to define and demonstrate gestures that are more natural, memorable, and accessible. However, customization requires efficient usage of user-provided data. We introduce a method that enables users to easily design bespoke gestures with a monocular camera from one demonstration. We employ transformers and meta-learning techniques to address few-shot learning challenges. Unlike prior work, our method supports any combination of one-handed, two-handed, static, and dynamic gestures, including different viewpoints, and the ability to handle irrelevant hand movements. We implement three real-world applications using our customization method, conduct a user study, and achieve up to 94% average recognition accuracy from one demonstration. Our work provides a viable path for vision-based gesture customization, laying the foundation for future advancements in this domain., Comment: 2024 (UIST' 24). USA, 14 pages

Streamlining content discovery within media archives requires integrating advanced data representations and effective visualization techniques for clear communication of video topics to users. The proposed system addresses the challenge of efficiently navigating large video collections by exploiting a fusion of visual, audio, and textual features to accurately index and categorize video content through a text-based method. Additionally, semantic embeddings are employed to provide contextually relevant information and recommendations to users, resulting in an intuitive and engaging exploratory experience over our topics ontology map using OpenAI GPT-4.

This paper investigates the capacity of finite-state channels (FSCs) with feedback. We derive an upper bound on the feedback capacity of FSCs by extending the duality upper bound method from mutual information to the case of directed information. The upper bound is expressed as a multi-letter expression that depends on a test distribution on the sequence of channel outputs. For any FSC, we show that if the test distribution is structured on a $Q$-graph, the upper bound can be formulated as a Markov decision process (MDP) whose state being a belief on the channel state. In the case of FSCs and states that are either unifilar or have a finite memory, the MDP state simplifies to take values in a finite set. Consequently, the MDP consists of a finite number of states, actions, and disturbances. This finite nature of the MDP is of significant importance, as it ensures that dynamic programming algorithms can solve the associated Bellman equation to establish analytical upper bounds, even for channels with large alphabets. We demonstrate the simplicity of computing bounds by establishing the capacity of a broad family of Noisy Output is the State (NOST) channels as a simple closed-form analytical expression. Furthermore, we introduce novel, nearly optimal analytical upper bounds on the capacity of the Noisy Ising channel.

Instruction-based image editing holds immense potential for a variety of applications, as it enables users to perform any editing operation using a natural language instruction. However, current models in this domain often struggle with accurately executing user instructions. We present Emu Edit, a multi-task image editing model which sets state-of-the-art results in instruction-based image editing. To develop Emu Edit we train it to multi-task across an unprecedented range of tasks, such as region-based editing, free-form editing, and Computer Vision tasks, all of which are formulated as generative tasks. Additionally, to enhance Emu Edit's multi-task learning abilities, we provide it with learned task embeddings which guide the generation process towards the correct edit type. Both these elements are essential for Emu Edit's outstanding performance. Furthermore, we show that Emu Edit can generalize to new tasks, such as image inpainting, super-resolution, and compositions of editing tasks, with just a few labeled examples. This capability offers a significant advantage in scenarios where high-quality samples are scarce. Lastly, to facilitate a more rigorous and informed assessment of instructable image editing models, we release a new challenging and versatile benchmark that includes seven different image editing tasks.

This paper studies online solutions for regret-optimal control in partially observable systems over an infinite-horizon. Regret-optimal control aims to minimize the difference in LQR cost between causal and non-causal controllers while considering the worst-case regret across all $\ell_2$-norm-bounded disturbance and measurement sequences. Building on ideas from Sabag et al., 2023, on the the full-information setting, our work extends the framework to the scenario of partial observability (measurement-feedback). We derive an explicit state-space solution when the non-causal solution is the one that minimizes the $\mathcal H_2$ criterion, and demonstrate its practical utility on several practical examples. These results underscore the framework's significant relevance and applicability in real-world systems., Comment: Submitted to ACC 2024

Much of our progress in understanding microscale biology has been powered by advances in microscopy. For instance, super-resolution microscopes allow the observation of biological structures at near-atomic-scale resolution, while multi-photon microscopes allow imaging deep into tissue. However, biological structures and dynamics still often remain out of reach of existing microscopes, with further advances in signal-to-noise, resolution and speed needed to access them. In many cases, the performance of microscopes is now limited by quantum effects -- such as noise due to the quantisation of light into photons or, for multi-photon microscopes, the low cross-section of multi-photon scattering. These limitations can be overcome by exploiting features of quantum mechanics such as entanglement. Quantum effects can also provide new ways to enhance the performance of microscopes, such as new super-resolution techniques and new techniques to image at difficult to reach wavelengths. This review provides an overview of these various ways in which quantum techniques can improve microscopy, including recent experimental progress. It seeks to provide a realistic picture of what is possible, and what the constraints and opportunities are.

This paper investigates the problem of zero-delay joint source-channel coding of a vector Gauss-Markov source over a multiple-input multiple-output (MIMO) additive white Gaussian noise (AWGN) channel with feedback. In contrast to the classical problem of causal estimation using noisy observations, we examine a system where the source can be encoded before transmission. An encoder, equipped with feedback of past channel outputs, observes the source state and encodes the information in a causal manner as inputs to the channel while adhering to a power constraint. The objective of the code is to estimate the source state with minimum mean square error at the infinite horizon. This work shows a fundamental theorem for two scenarios: for the transmission of an unstable vector Gauss-Markov source over either a multiple-input single-output (MISO) or a single-input multiple-output (SIMO) AWGN channel, finite estimation error is achievable if and only if the sum of logs of the unstable eigenvalues of the state gain matrix is less than the Shannon channel capacity. We prove these results by showing an optimal linear innovations encoder that can be applied to sources and channels of any dimension and analyzing it together with the corresponding Kalman filter decoder., Comment: Presented at 59th Allerton Conference on Communication, Control, and Computing

We present CM3Leon (pronounced "Chameleon"), a retrieval-augmented, token-based, decoder-only multi-modal language model capable of generating and infilling both text and images. CM3Leon uses the CM3 multi-modal architecture but additionally shows the extreme benefits of scaling up and tuning on more diverse instruction-style data. It is the first multi-modal model trained with a recipe adapted from text-only language models, including a large-scale retrieval-augmented pre-training stage and a second multi-task supervised fine-tuning (SFT) stage. It is also a general-purpose model that can do both text-to-image and image-to-text generation, allowing us to introduce self-contained contrastive decoding methods that produce high-quality outputs. Extensive experiments demonstrate that this recipe is highly effective for multi-modal models. CM3Leon achieves state-of-the-art performance in text-to-image generation with 5x less training compute than comparable methods (zero-shot MS-COCO FID of 4.88). After SFT, CM3Leon can also demonstrate unprecedented levels of controllability in tasks ranging from language-guided image editing to image-controlled generation and segmentation.

We consider communication over channels whose statistics are not known in full, but can be parameterized as a finite family of memoryless channels. A typical approach to address channel uncertainty is to design codes for the worst channel in the family, resulting in the well-known compound channel capacity. Although this approach is robust, it may suffer a significant loss of performance if the capacity-achieving distribution of the worst channel attains low rates over other channels. In this work, we cope with channel uncertainty through the lens of {\em competitive analysis}. The main idea is to optimize a relative metric that compares the performance of the designed code and a clairvoyant code that has access to the true channel. To allow communication rates that adapt to the channel at use, we consider rateless codes with a fixed number of message bits and random decoding times. We propose two competitive metrics: the competitive ratio between the expected rates of the two codes, and a regret defined as the difference between the expected rates. The competitive ratio, for instance, provides a percentage guarantee on the expected rate of the designed code when compared to the rate of the clairvoyant code that knows the channel at hand. Our main results are single-letter expressions for the optimal {\em competitive-ratio} and {\em regret}, expressed as a max-min or min-max optimization. Several examples illustrate the benefits of the competitive analysis approach to code design compared to the compound channel.

Coupled colloidal quantum dot molecules are an emerging class of nanomaterials, introducing new degrees of freedom for designing quantum dot-based technologies. The properties of multiply excited states in these materials are crucial to their performance as quantum light emitters but cannot be fully resolved by existing spectroscopic techniques. Here we study the characteristics of biexcitonic species, which represent a rich landscape of different configurations, such as segregated and localized biexciton states. To this end, we introduce an extension of Heralded Spectroscopy to resolve different biexciton species in the prototypical CdSe/CdS coupled quantum dot dimer system. We uncover the coexistence and interplay of two distinct biexciton species: A fast-decaying, strongly-interacting biexciton species, analogous to biexcitons in single quantum dots, and a long-lived, weakly-interacting species corresponding to two nearly-independent excitons separated to the two sides of the coupled quantum dot pair. The two biexciton types are consistent with numerical simulations, assigning the strongly-interacting species to two excitons localized at one side of the quantum dot molecule and the weakly-interacting species to excitons segregated to the two quantum dot molecule sides. This deeper understanding of multiply excited states in coupled quantum dot molecules can support the rational design of tunable single- or multiple-photon quantum emitters., Comment: 13 pages, 5 figures

Medical clowns help hospitalized children in reducing pain and anxiety symptoms and increase the level of satisfaction in children's wards. Unfortunately, there is a shortage of medical clowns around the world. Furthermore, isolated children can not enjoy this service. This study explored the concept of a Robotic Medical Clown (RMC) and its role. We used mixed methods of elicitation to create a design space model for future robotic medical clowns. We investigated the needs, perceptions, and preferences of children and teenagers using four methods: interviewing medical clowns to learn how they perceive their role and the potential role of an RMC, conducting focus groups with teenagers, a one-on-one experience of children with a robot, and an online questionnaire. The concept of RMCs was acceptable to children, teenagers, and medical clowns. We found that the RMC's appearance affects the perception of its characters and role. Future work should investigate the interaction in hospitals., Comment: Working paper based on the poster presented at ICRA 2023

Socially assistive robots (SARs) are becoming more prevalent in everyday life, emphasizing the need to make them socially acceptable and aligned with users' expectations. Robots' appearance impacts users' behaviors and attitudes towards them. Therefore, product designers choose visual qualities to give the robot a character and to imply its functionality and personality. In this work, we sought to investigate the effect of cultural differences on Israeli and German designers' perceptions and preferences regarding the suitable visual qualities of SARs in four different contexts: a service robot for an assisted living/retirement residence facility, a medical assistant robot for a hospital environment, a COVID-19 officer robot, and a personal assistant robot for domestic use. Our results indicate that Israeli and German designers share similar perceptions of visual qualities and most of the robotics roles. However, we found differences in the perception of the COVID-19 officer robot's role and, by that, its most suitable visual design. This work indicates that context and culture play a role in users' perceptions and expectations; therefore, they should be taken into account when designing new SARs for diverse contexts., Comment: Submitted to ACM Transactions on Human-Robot Interaction (THRI)

In this paper, we investigate the capacity of finite-state channels (FSCs) in presence of delayed feedback. We show that the capacity of a FSC with delayed feedback can be computed as that of a new FSC with instantaneous feedback and an extended state. Consequently, graph-based methods to obtain computable upper and lower bounds on the delayed feedback capacity of unifilar FSCs are proposed. Based on these methods, we establish that the capacity of the trapdoor channel with delayed feedback of two time instances is given by $\log_2(3/2)$. In addition, we derive an analytical upper bound on the delayed feedback capacity of the binary symmetric channel with a no consecutive ones input constraint. This bound also serves as a novel upper bound on its non-feedback capacity, which outperforms all previously known bounds. Lastly, we demonstrate that feedback does improve the capacity of the dicode erasure channel.

We present MAV3D (Make-A-Video3D), a method for generating three-dimensional dynamic scenes from text descriptions. Our approach uses a 4D dynamic Neural Radiance Field (NeRF), which is optimized for scene appearance, density, and motion consistency by querying a Text-to-Video (T2V) diffusion-based model. The dynamic video output generated from the provided text can be viewed from any camera location and angle, and can be composited into any 3D environment. MAV3D does not require any 3D or 4D data and the T2V model is trained only on Text-Image pairs and unlabeled videos. We demonstrate the effectiveness of our approach using comprehensive quantitative and qualitative experiments and show an improvement over previously established internal baselines. To the best of our knowledge, our method is the first to generate 3D dynamic scenes given a text description.

Semiconductor nanocrystal emission polarization is a crucial probe of nanocrystal physics and an essential factor for nanocrystal-based technologies. While the transition dipole moment of the lowest excited state to ground state transition is well characterized, the dipole moment of higher multiexcitonic transitions is inaccessible via most spectroscopy techniques. Here, we realize direct characterization of the doubly-excited state relaxation transition dipole by heralded defocused imaging. Defocused imaging maps the dipole emission pattern onto a fast single-photon avalanche diode detector array, allowing the post-selection of photon pairs emitted from the biexciton-exciton emission cascade and resolving the differences in transition dipole moments. Type-I1/2 seeded nanorods exhibit higher anisotropy of the biexciton-to-exciton transition compared to the exciton-to-ground state transition. In contrast, type-II seeded nanorods display a reduction of biexciton emission anisotropy. These findings are rationalized in terms of an interplay between transient dynamics of the refractive index and the excitonic fine structure.

We consider finite state channels (FSCs) with feedback and state information known causally at the encoder. This setting is quite general and includes: a memoryless channel with i.i.d. state (the Shannon strategy), Markovian states that include look-ahead (LA) access to the state and energy harvesting. We characterize the feedback capacity of the general setting as the directed information between auxiliary random variables with memory to the channel outputs. We also propose two methods for computing the feedback capacity: (i) formulating an infinite-horizon average-reward dynamic program; and (ii) a single-letter lower bound based on auxiliary directed graphs called $Q$-graphs. We demonstrate our computation methods on several examples. In the first example, we introduce a channel with LA and derive a closed-form, analytic lower bound on its feedback capacity. Furthermore, we show that the mentioned methods achieve the feedback capacity of known unifilar FSCs such as the trapdoor channel, the Ising channel and the input-constrained erasure channel. Finally, we analyze the feedback capacity of a channel whose state is stochastically dependent on the input., Comment: 39 pages, 10 figures. The material in this paper was presented in part at the 56th Annual Allerton Conference on Communication, Control, and Computing, Monticello, IL, USA, October 2018, and at the IEEE International Symposium on Information Theory, Los Angeles, CA, USA, June 2020

Robots' visual qualities (VQs) impact people's perception of their characteristics and affect users' behaviors and attitudes toward the robot. Recent years point toward a growing need for Socially Assistive Robots (SARs) in various contexts and functions, interacting with various users. Since SAR types have functional differences, the user experience must vary by the context of use, functionality, user characteristics, and environmental conditions. Still, SAR manufacturers often design and deploy the same robotic embodiment for diverse contexts. We argue that the visual design of SARs requires a more scientific approach considering their multiple evolving roles in future society. In this work, we define four contextual layers: the domain in which the SAR exists, the physical environment, its intended users, and the robot's role. Via an online questionnaire, we collected potential users' expectations regarding the desired characteristics and visual qualities of four different SARs: a service robot for an assisted living/retirement residence facility, a medical assistant robot for a hospital environment, a COVID-19 officer robot, and a personal assistant robot for domestic use. Results indicated that users' expectations differ regarding the robot's desired characteristics and the anticipated visual qualities for each context and use case., Comment: Accepted to the 15th International Workshop on Human-Friendly Robotics