Search

Object detection, a quintessential task in the realm of perceptual computing, can be tackled using a generative methodology. In the present study, we introduce a novel framework designed to articulate object detection as a denoising diffusion process, which operates on the perturbed bounding boxes of annotated entities. This framework, termed ConsistencyDet, leverages an innovative denoising concept known as the Consistency Model. The hallmark of this model is its self-consistency feature, which empowers the model to map distorted information from any temporal stage back to its pristine state, thereby realizing a "one-step denoising" mechanism. Such an attribute markedly elevates the operational efficiency of the model, setting it apart from the conventional Diffusion Model. Throughout the training phase, ConsistencyDet initiates the diffusion sequence with noise-infused boxes derived from the ground-truth annotations and conditions the model to perform the denoising task. Subsequently, in the inference stage, the model employs a denoising sampling strategy that commences with bounding boxes randomly sampled from a normal distribution. Through iterative refinement, the model transforms an assortment of arbitrarily generated boxes into definitive detections. Comprehensive evaluations employing standard benchmarks, such as MS-COCO and LVIS, corroborate that ConsistencyDet surpasses other leading-edge detectors in performance metrics. Our code is available at https://github.com/Tankowa/ConsistencyDet.

The burgeoning interest in developing Large Language Models (LLMs) with up to trillion parameters has been met with concerns regarding resource efficiency and practical expense, particularly given the immense cost of experimentation. This scenario underscores the importance of exploring the potential of Small Language Models (SLMs) as a resource-efficient alternative. In this context, we introduce MiniCPM, specifically the 1.2B and 2.4B non-embedding parameter variants, not only excel in their respective categories but also demonstrate capabilities on par with 7B-13B LLMs. While focusing on SLMs, our approach exhibits scalability in both model and data dimensions for future LLM research. Regarding model scaling, we employ extensive model wind tunnel experiments for stable and optimal scaling. For data scaling, we introduce a Warmup-Stable-Decay (WSD) learning rate scheduler (LRS), conducive to continuous training and domain adaptation. We present an in-depth analysis of the intriguing training dynamics that occurred in the WSD LRS. With WSD LRS, we are now able to efficiently study data-model scaling law without extensive retraining experiments on both axes of model and data, from which we derive the much higher compute optimal data-model ratio than Chinchilla Optimal. Additionally, we introduce MiniCPM family, including MiniCPM-DPO, MiniCPM-MoE and MiniCPM-128K, whose excellent performance further cementing MiniCPM's foundation in diverse SLM applications. MiniCPM models are available publicly at https://github.com/OpenBMB/MiniCPM ., Comment: revise according to peer review

We propose the piecemeal quantum telescope through bit-by-bit iteration using different baselines. It improves precision exponentially with number of baselines, and it works robustly under large observation errors such as statistical errors, channel noise, operational errors and so on. For example, under the noisy channel of random phase drifts, our method can provide an exponential precision provided that the detected data is not entirely noisy. Being fault tolerant to statistical error, it requests only a small number of incident single-photons in detecting the star angle with exponential precision. As a result, it requests to detect only a few hundreds of photons from the target star for a precision breaking classical limit by 4 to 5 magnitude orders. This demonstrates a super efficiency., Comment: 6 figures

This paper addresses the challenge of object-centric layout generation under spatial constraints, seen in multiple domains including floorplan design process. The design process typically involves specifying a set of spatial constraints that include object attributes like size and inter-object relations such as relative positioning. Existing works, which typically represent objects as single nodes, lack the granularity to accurately model complex interactions between objects. For instance, often only certain parts of an object, like a room's right wall, interact with adjacent objects. To address this gap, we introduce a factor graph based approach with four latent variable nodes for each room, and a factor node for each constraint. The factor nodes represent dependencies among the variables to which they are connected, effectively capturing constraints that are potentially of a higher order. We then develop message-passing on the bipartite graph, forming a factor graph neural network that is trained to produce a floorplan that aligns with the desired requirements. Our approach is simple and generates layouts faithful to the user requirements, demonstrated by a large improvement in IOU scores over existing methods. Additionally, our approach, being inferential and accurate, is well-suited to the practical human-in-the-loop design process where specifications evolve iteratively, offering a practical and powerful tool for AI-guided design., Comment: To be published at IEEE/CVF CVPR 2024

Squeezing light into subwavelength dimensions is vital for on-chip integration of photonic technologies. One approach to overcome the diffraction limit is coupling light to material excitations, leading to polariton states. Here, we showcase how low-loss mid-infrared surface phonon polaritons enable metasurfaces supporting quasi-bound states in the continuum (qBICs) with deeply subwavelength unit cells. Utilizing 100 nm thick free-standing silicon carbide membranes, we achieve highly confined qBIC states with a unit cell volume ~ 10^4 times smaller than the diffraction limit. This results in remarkable robustness of the platform against the incident angle that is unique among qBIC systems. We also demonstrate vibrational strong coupling with a thin layer of spin-coated molecules, leveraging the small mode volume. This work introduces phononic qBICs as a novel ultra-confined nanophotonic platform, paving a way for the miniaturization of mid-infrared devices for molecular sensing and thermal radiation engineering.

In this study, we propose two strategies to determine complex refractive indexes along armchair and zigzag axes for BP flakes, aiming in predicting angle-resolved polarized Raman (ARPR) intensity by explicitly considering birefringence, linear dichroism, and anisotropic cavity interference effects within multilayered structures. By leveraging this methodology, we have identified the intrinsic complex Raman tensors for phonon modes of BP flakes, independent of BP flake thickness (>20 nm). We also elucidated the flake thickness-dependent effective complex Raman tensor elements, allowing for precise prediction of the observed ARPR intensity profile for specific BP flake. This framework can be extended to other ALM flakes deposited on dielectric substrate to determine the Raman tensors for fully predicting their ARPR response., Comment: 6 pages, 4 figures

The precise measurement of the gravity of the earth plays a pivotal role in various fundamental research and application fields. Although a few gravimeters have been reported to achieve this goal, miniaturization of high-precision gravimetry remains a challenge. In this work, we have proposed and demonstrated a miniaturized gravimetry operating at room temperature based on a diamagnetic levitated micro-oscillator with a proof mass of only 215 mg. Compared with the latest reported miniaturized gravimeters based on Micro-Electro-Mechanical Systems, the performance of our gravimetry has substantial improvements in that an acceleration sensitivity of 15 $\mu Gal/\sqrt{Hz}$ and a drift as low as 61 $\mu Gal$ per day have been reached. Based on this diamagnetic levitation gravimetry, we observed the earth tides, and the correlation coefficient between the experimental data and theoretical data reached 0.97. Some moderate foreseeable improvements can develop this diamagnetic levitation gravimetry into chip size device, making it suitable for mobile platforms such as drones. Our advancement in gravimetry is expected to facilitate a multitude of applications, including underground density surveying and the forecasting of natural hazards., Comment: 5 pages, 3 figures

Prediction the conversion to early-stage dementia is critical for mitigating its progression but remains challenging due to subtle cognitive impairments and structural brain changes. Traditional T1-weighted magnetic resonance imaging (T1-MRI) research focus on identifying brain atrophy regions but often fails to address the intricate connectivity between them. This limitation underscores the necessity of focuing on inter-regional connectivity for a comprehensive understand of the brain's complex network. Moreover, there is a pressing demand for methods that adaptively preserve and extract critical information, particularly specialized subgraph mining techniques for brain networks. These are essential for developing high-quality feature representations that reveal critical spatial impacts of structural brain changes and its topology. In this paper, we propose Brain-SubGNN, a novel graph representation network to mine and enhance critical subgraphs based on T1-MRI. This network provides a subgraph-level interpretation, enhancing interpretability and insights for graph analysis. The process begins by extracting node features and a correlation matrix between nodes to construct a task-oriented brain network. Brain-SubGNN then adaptively identifies and enhances critical subgraphs, capturing both loop and neighbor subgraphs. This method reflects the loop topology and local changes, indicative of long-range connections, and maintains local and global brain attributes. Extensive experiments validate the effectiveness and advantages of Brain-SubGNN, demonstrating its potential as a powerful tool for understanding and diagnosing early-stage dementia. Source code is available at https://github.com/Leng-10/Brain-SubGNN., Comment: 20 pages

We present a complete classification of $\operatorname{SL}(n)$ contravariant, $C(\mathbb{R}^n\setminus\{o\})$-valued valuations on polytopes, without any additional assumptions.It extends the previous results of the second author [Int. Math. Res. Not. 2020] which have a good connection with the $L_p$ and Orlicz Brunn-Minkowski theory. Additionally, our results deduce a complete classification of $\operatorname{SL}(n)$ contravariant symmetric-tensor-valued valuations on polytopes.

Coherent dispersive wave emission, as an important phenomenon of soliton dynamics, manifests itself in multiple platforms of nonlinear optics from fibre waveguides to integrated photonics. Limited by its resonance nature, efficient generation of coherent dispersive wave with ultra-broad bandwidth has, however, proved difficult to realize. Here, we unveil a new regime of soliton dynamics in which the dispersive wave emission process strongly couples with the splitting dynamics of the driving pulse. High-order dispersion and self-steepening effects, accumulated over soliton self-compression, break the system symmetry, giving rise to high-efficiency generation of coherent dispersive wave in the ultraviolet region. Simultaneously, asymmetric soliton splitting results in the appearance of a temporally-delayed ultrashort pulse with high intensity, overlapping and copropagating with the dispersive wave pulse. Intense cross-phase modulations lead to octave-wide broadening of the dispersive wave spectrum, covering 200 to 400 nm wavelengths. The highly-coherent, octave-wide ultraviolet spectrum, generated from the simple capillary fibre set-up, is in great demand for time-resolved spectroscopy, ultrafast electron microscopy and frequency metrology applications, and the critical role of the secondary pulse in this process reveals some new opportunities for all-optical control of versatile soliton dynamics.

The Mott-Ioffe-Regel limit sets the lower bound of carrier mean free path for coherent quasiparticle transport. Metallicity beyond this limit is of great interest because it is often closely related to quantum criticality and unconventional superconductivity. Progress along this direction mainly focuses on the strange-metal behaviors originating from the evolution of quasiparticle scattering rate such as linear-in-temperature resistivity, while the quasiparticle coherence phenomena in this regime are much less explored due to the short mean free path at the diffusive bound. Here we report the observation of quantum oscillations from Landau quantization near the Mott-Ioffe-Regel limit in CaAs3. Despite the insulator-like temperature dependence of resistivity, CaAs3 presents giant magnetoresistance and prominent Shubnikov-de Haas oscillations from Fermi surfaces, indicating highly coherent band transport. In contrast, the quantum oscillation is absent in the magnetic torque. The quasiparticle effective mass increases systematically with magnetic fields, manifesting a much larger value than the expectation given by magneto-infrared spectroscopy. It suggests a strong many-body renormalization effect near Fermi surface. We find that these unconventional behaviors may be explained by the interplay between the mobility edge and the van Hove singularity, which results in the formation of coherent cyclotron orbits emerging at the diffusive bound. Our results call for further study on the electron correlation effect of the van Hove singularity., Comment: 18 pages, 5 figures

Urban Air Mobility (UAM) expands vehicles from the ground to the near-ground space, envisioned as a revolution for transportation systems. Comprehensive scene perception is the foundation for autonomous aerial driving. However, UAM encounters the intelligent perception challenge: high perception learning requirements conflict with the limited sensors and computing chips of flying cars. To overcome the challenge, federated learning (FL) and other collaborative learning have been proposed to enable resource-limited devices to conduct onboard deep learning (DL) collaboratively. But traditional collaborative learning like FL relies on a central integrator for DL model aggregation, which is difficult to deploy in dynamic environments. The fully decentralized learning schemes may be the intuitive solution while the convergence of distributed learning cannot be guaranteed. Accordingly, this paper explores reconfigurable intelligent surfaces (RIS) empowered distributed learning, taking account of topological attributes to facilitate the learning performance with convergence guarantee. We propose several FL topological criteria for optimizing the transmission delay and convergence rate by exploiting the Laplacian matrix eigenvalues of the communication network. Subsequently, we innovatively leverage the RIS link modification ability to remold the current network according to the proposed topological criteria. This paper rethinks the functions of RIS from the perspective of the network layer. Furthermore, a deep deterministic policy gradient-based RIS phase shift control algorithm is developed to construct or deconstruct the network links simultaneously to reshape the communication network. Simulation experiments are conducted over MobileNet-based multi-view learning to verify the efficiency of the distributed FL framework.

While recent large-scale text-to-speech (TTS) models have achieved significant progress, they still fall short in speech quality, similarity, and prosody. Considering speech intricately encompasses various attributes (e.g., content, prosody, timbre, and acoustic details) that pose significant challenges for generation, a natural idea is to factorize speech into individual subspaces representing different attributes and generate them individually. Motivated by it, we propose NaturalSpeech 3, a TTS system with novel factorized diffusion models to generate natural speech in a zero-shot way. Specifically, 1) we design a neural codec with factorized vector quantization (FVQ) to disentangle speech waveform into subspaces of content, prosody, timbre, and acoustic details; 2) we propose a factorized diffusion model to generate attributes in each subspace following its corresponding prompt. With this factorization design, NaturalSpeech 3 can effectively and efficiently model intricate speech with disentangled subspaces in a divide-and-conquer way. Experiments show that NaturalSpeech 3 outperforms the state-of-the-art TTS systems on quality, similarity, prosody, and intelligibility, and achieves on-par quality with human recordings. Furthermore, we achieve better performance by scaling to 1B parameters and 200K hours of training data., Comment: Achieving human-level quality and naturalness on multi-speaker datasets (e.g., LibriSpeech) in a zero-shot way

3D shape generation from text is a fundamental task in 3D representation learning. The text-shape pairs exhibit a hierarchical structure, where a general text like ``chair" covers all 3D shapes of the chair, while more detailed prompts refer to more specific shapes. Furthermore, both text and 3D shapes are inherently hierarchical structures. However, existing Text2Shape methods, such as SDFusion, do not exploit that. In this work, we propose HyperSDFusion, a dual-branch diffusion model that generates 3D shapes from a given text. Since hyperbolic space is suitable for handling hierarchical data, we propose to learn the hierarchical representations of text and 3D shapes in hyperbolic space. First, we introduce a hyperbolic text-image encoder to learn the sequential and multi-modal hierarchical features of text in hyperbolic space. In addition, we design a hyperbolic text-graph convolution module to learn the hierarchical features of text in hyperbolic space. In order to fully utilize these text features, we introduce a dual-branch structure to embed text features in 3D feature space. At last, to endow the generated 3D shapes with a hierarchical structure, we devise a hyperbolic hierarchical loss. Our method is the first to explore the hyperbolic hierarchical representation for text-to-shape generation. Experimental results on the existing text-to-shape paired dataset, Text2Shape, achieved state-of-the-art results. We release our implementation under HyperSDFusion.github.io.