Search

Estimating 3D full-body avatars from AR/VR devices is essential for creating immersive experiences in AR/VR applications. This task is challenging due to the limited input from Head Mounted Devices, which capture only sparse observations from the head and hands. Predicting the full-body avatars, particularly the lower body, from these sparse observations presents significant difficulties. In this paper, we are inspired by the inherent property of the kinematic tree defined in the Skinned Multi-Person Linear (SMPL) model, where the upper body and lower body share only one common ancestor node, bringing the potential of decoupled reconstruction. We propose a stratified approach to decouple the conventional full-body avatar reconstruction pipeline into two stages, with the reconstruction of the upper body first and a subsequent reconstruction of the lower body conditioned on the previous stage. To implement this straightforward idea, we leverage the latent diffusion model as a powerful probabilistic generator, and train it to follow the latent distribution of decoupled motions explored by a VQ-VAE encoder-decoder model. Extensive experiments on AMASS mocap dataset demonstrate our state-of-the-art performance in the reconstruction of full-body motions., Comment: Accepted by CVPR 2024 (Oral)

Creating 4D fields of Gaussian Splatting from images or videos is a challenging task due to its under-constrained nature. While the optimization can draw photometric reference from the input videos or be regulated by generative models, directly supervising Gaussian motions remains underexplored. In this paper, we introduce a novel concept, Gaussian flow, which connects the dynamics of 3D Gaussians and pixel velocities between consecutive frames. The Gaussian flow can be efficiently obtained by splatting Gaussian dynamics into the image space. This differentiable process enables direct dynamic supervision from optical flow. Our method significantly benefits 4D dynamic content generation and 4D novel view synthesis with Gaussian Splatting, especially for contents with rich motions that are hard to be handled by existing methods. The common color drifting issue that happens in 4D generation is also resolved with improved Guassian dynamics. Superior visual quality on extensive experiments demonstrates our method's effectiveness. Quantitative and qualitative evaluations show that our method achieves state-of-the-art results on both tasks of 4D generation and 4D novel view synthesis. Project page: https://zerg-overmind.github.io/GaussianFlow.github.io/

This study aimed to examine an assumption that generative artificial intelligence (GAI) tools can overcome the cognitive intensity that humans suffer when solving problems. We compared the performance of ChatGPT and GPT-4 on 2019 NAEP science assessments with students by cognitive demands of the items. Fifty-four tasks were coded by experts using a two-dimensional cognitive load framework, including task cognitive complexity and dimensionality. ChatGPT and GPT-4 responses were scored using the scoring keys of NAEP. The analysis of the available data was based on the average student ability scores for students who answered each item correctly and the percentage of students who responded to individual items. Results showed that both ChatGPT and GPT-4 consistently outperformed most students who answered the NAEP science assessments. As the cognitive demand for NAEP tasks increases, statistically higher average student ability scores are required to correctly address the questions. This pattern was observed for students in grades 4, 8, and 12, respectively. However, ChatGPT and GPT-4 were not statistically sensitive to the increase in cognitive demands of the tasks, except for Grade 4. As the first study focusing on comparing GAI and K-12 students in problem-solving in science, this finding implies the need for changes to educational objectives to prepare students with competence to work with GAI tools in the future. Education ought to emphasize the cultivation of advanced cognitive skills rather than depending solely on tasks that demand cognitive intensity. This approach would foster critical thinking, analytical skills, and the application of knowledge in novel contexts. Findings also suggest the need for innovative assessment practices by moving away from cognitive intensity tasks toward creativity and analytical skills to avoid the negative effects of GAI on testing more efficiently.

One of the most intriguing advantage of Floquet engineering is to generate new phases with large topological invariants. In this work, we experimentally simulate a periodically quenched generalized Haldane model on an NV center in diamond, and observe its Floquet Chern insulator phases with Chern numbers $C=1,2,4$ by imaging the static and dynamic spin textures in momentum space. Our work reveals the power of Floquet driving in transforming system's topology and generating large Chern number phases. It further establishes a unique experimental scheme to detect Floquet topological phases in two and higher spatial dimensions., Comment: 11 pages, figure 1-4, figure s1-s2

This paper studies the problem of holistic 3D wireframe perception (HoW-3D), a new task of perceiving both the visible 3D wireframes and the invisible ones from single-view 2D images. As the non-front surfaces of an object cannot be directly observed in a single view, estimating the non-line-of-sight (NLOS) geometries in HoW-3D is a fundamentally challenging problem and remains open in computer vision. We study the problem of HoW-3D by proposing an ABC-HoW benchmark, which is created on top of CAD models sourced from the ABC-dataset with 12k single-view images and the corresponding holistic 3D wireframe models. With our large-scale ABC-HoW benchmark available, we present a novel Deep Spatial Gestalt (DSG) model to learn the visible junctions and line segments as the basis and then infer the NLOS 3D structures from the visible cues by following the Gestalt principles of human vision systems. In our experiments, we demonstrate that our DSG model performs very well in inferring the holistic 3D wireframes from single-view images. Compared with the strong baseline methods, our DSG model outperforms the previous wireframe detectors in detecting the invisible line geometry in single-view images and is even very competitive with prior arts that take high-fidelity PointCloud as inputs on reconstructing 3D wireframes., Comment: To appear in IEEE 3DV 2022. Code and Dataset are available at https://github.com/Wenchao-M/HoW-3D

Meta-optics have rapidly become a major research field within the optics and photonics community, strongly driven by the seemingly limitless opportunities made possible by controlling optical wavefronts through interaction with arrays of sub-wavelength scatterers. As more and more modalities are explored, the design strategies to achieve desired functionalities become increasingly demanding, necessitating more advanced design techniques. Herein, the inverse-design approach is utilized to create a set of single-layer meta-optics that simultaneously focus light and shape the spectra of focused light without using any filters. Thus, both spatial and spectral properties of the meta-optics are optimized, resulting in spectra that mimic the color matching functions of the CIE 1931 XYZ color space, which links the distributions of wavelengths in light and the color perception of a human eye. Experimental demonstrations of these meta-optics show qualitative agreement with the theoretical predictions and help elucidate the focusing mechanism of these devices.

Precision measurement plays a crucial role in all fields of science. The use of entangled sensors in quantum metrology improves the precision limit from the standard quantum limit (SQL) to the Heisenberg limit (HL). To date, most experiments beating the SQL are performed on the sensors which are well isolated under extreme conditions. However, it has not been realized in solid-state spin systems at ambient conditions, owing to its intrinsic complexity for the preparation and survival of pure and entangled quantum states. Here we show a full interferometer sequence beating the SQL by employing a hybrid multi-spin system, namely the nitrogen-vacancy (NV) defect in diamond. The interferometer sequence starts from a deterministic and joint initialization, undergoes entanglement and disentanglement of multiple spins, and ends up with projective measurement. In particular, the deterministic and joint initialization of NV negative state, NV electron spin, and two nuclear spins is realized at room temperature for the first time. By means of optimal control, non-local gates are implemented with an estimated fidelity above the threshold for fault-tolerant quantum computation. With these techniques combined, we achieve two-spin interference with a phase sensitivity of 1.79 \pm 0.06 dB beyond the SQL and three-spin 2.77 \pm 0.10 dB. Moreover, the deviations from the HL induced by experimental imperfections are completely accountable. The techniques used here are of fundamental importance for quantum sensing and computing, and naturally applicable to other solid-state spin systems., Comment: 44 pages, 14 figures

Quantum coherence is the most fundamental of all quantum quantifiers, underlying other well-known quantities such as entanglement, quantum discord, and Bell correlations. It can be distributed in a multipartite system in various ways -- for example, in a bipartite system it can exist within subsystems (local coherence) or collectively between the subsystems (global coherence) and exhibits a trade-off relation. In quantum systems with more than two subsystems, there are more trade-off relations, due to the various decomposition ways of the coherence. In this paper, we experimentally verify these coherence trade-off relations in adiabatically evolved quantum systems using a spin system by changing the state from a product state to a tripartite entangled state. We study the full set of coherence trade-off relations between the original state, the bipartite product state, the tripartite product state, and the decohered product state. We also experimentally verify the monogamy inequality and show that both the quantum systems are polygamous except for the initial product state. We find that despite the different types of states involved, the properties of the state in terms of coherence and monogamy are equivalent. This illustrates the utility of using coherence as a characterization tool for quantum states., Comment: 12 pages, 7 figures

Traditional phase-shifting interferometry technique cannot be used to measure time-varying phase distributions. But single shot techniques could resolve the problem. Many efforts have been made on the phase retrieval methods from a single shot interferogram. In the paper, a simple and effective method is presented without complex computation. The interference fringe is transferred to a phase distribution with a look-up-table. And then it is divided into different regions according to the parity of every pixel. The pixels in the same region have the same parity, which determines the wrapped phase. Additionally, the light spot displacement of a local wavefront is obtained to solve the global sign ambiguity. The theoretical simulation results indicate that the PV of wavefront error is 0.00054(lambda) and the rms is 0.000125(lambda), which is much better than the results from the Fast Fourier Transformation method. We also use it in the experimentally measured interferogram. Our algorithm has the advantages of simplicity, high precision and effective for both open and closed interferometer fringes, which will be valuable for real time monitoring the optical elements shape during their processing., Comment: 12 pages, 8 figures

Magnetic resonance imaging (MRI) is a non-invasive and label-free technique widely used in medical diagnosis and life science research, and its success has benefited greatly from continuing efforts on enhancing contrast and resolution. Here we reported nanoscale MRI in a single cell using an atomic-size quantum sensor. With nitrogen-vacancy center in diamond, the intracellular protein ferritin has been imaged with a spatial resolution of ~ 10 nanometers, and ferritin-containing organelles were co-localized by correlative MRI and electron microscopy. Comparing to the current micrometer resolution in current state-of-art conventional MRI, our approach represents a 100-fold enhancement, and paves the way for MRI of intracellular proteins., Comment: 10 pages, 4 figures

Entanglement is a counterintuitive feature of quantum physics that is at the heart of quantum technology. High-dimensional quantum states offer unique advantages in various quantum information tasks. Integrated photonic chips have recently emerged as a leading platform for the generation, manipulation and detection of entangled photons. Here, we report a silicon photonic chip that uses novel interferometric resonance-enhanced photon-pair sources, spectral demultiplexers and high-dimensional reconfigurable circuitries to generate, manipulate and analyse path-entangled three-dimensional qutrit states. By minimizing on-chip electrical and thermal cross-talk, we obtain high-quality quantum interference with visibilities above 96.5% and a maximumly entangled qutrit state with a fidelity of 95.5%. We further explore the fundamental properties of entangled qutrits to test quantum nonlocality and contextuality, and to implement quantum simulations of graphs and high-precision optical phase measurements. Our work paves the path for the development of multiphoton high-dimensional quantum technologies., Comment: 36 pages, 14 figures