Search

Conventional GAN-based models for talking head generation often suffer from limited quality and unstable training. Recent approaches based on diffusion models aimed to address these limitations and improve fidelity. However, they still face challenges, including extensive sampling times and difficulties in maintaining temporal consistency due to the high stochasticity of diffusion models. To overcome these challenges, we propose a novel motion-disentangled diffusion model for high-quality talking head generation, dubbed MoDiTalker. We introduce the two modules: audio-to-motion (AToM), designed to generate a synchronized lip motion from audio, and motion-to-video (MToV), designed to produce high-quality head video following the generated motion. AToM excels in capturing subtle lip movements by leveraging an audio attention mechanism. In addition, MToV enhances temporal consistency by leveraging an efficient tri-plane representation. Our experiments conducted on standard benchmarks demonstrate that our model achieves superior performance compared to existing models. We also provide comprehensive ablation studies and user study results.

Analogue test structures were fabricated using the Tower Partners Semiconductor Co. CMOS 65 nm ISC process. The purpose was to characterise and qualify this process and to optimise the sensor for the next generation of Monolithic Active Pixels Sensors for high-energy physics. The technology was explored in several variants which differed by: doping levels, pixel geometries and pixel pitches (10-25 $\mu$m). These variants have been tested following exposure to varying levels of irradiation up to 3 MGy and $10^{16}$ 1 MeV n$_\text{eq}$ cm$^{-2}$. Here the results from prototypes that feature direct analogue output of a 4$\times$4 pixel matrix are reported, allowing the systematic and detailed study of charge collection properties. Measurements were taken both using $^{55}$Fe X-ray sources and in beam tests using minimum ionizing particles. The results not only demonstrate the feasibility of using this technology for particle detection but also serve as a reference for future applications and optimisations.

Unsupervised semantic segmentation (USS) aims to discover and recognize meaningful categories without any labels. For a successful USS, two key abilities are required: 1) information compression and 2) clustering capability. Previous methods have relied on feature dimension reduction for information compression, however, this approach may hinder the process of clustering. In this paper, we propose a novel USS framework called Expand-and-Quantize Unsupervised Semantic Segmentation (EQUSS), which combines the benefits of high-dimensional spaces for better clustering and product quantization for effective information compression. Our extensive experiments demonstrate that EQUSS achieves state-of-the-art results on three standard benchmarks. In addition, we analyze the entropy of USS features, which is the first step towards understanding USS from the perspective of information theory., Comment: Accepted to AAAI 2024

A popular physical picture of the mechanism behind the four-dimensional chiral anomaly is provided by the massless Dirac equation in the presence of constant electric and magnetic background fields. The magnetic field creates highly degenerate Landau levels, the lowest of which is gapless. Any parallel component of the electric field drives a spectral flow in the gapless mode that causes particles to emerge from, or disappear into, the Dirac sea. Seeking a similar picture for the gravitational contribution to the chiral anomaly, we consider the massless Dirac equation in a background spacetime that creates {\it gravitational\} Landau levels. We find that in this case the resulting spectral flow, with its explicit particle production, accounts for only a small part of the anomalous creation of chiral charge. The balance is provided by the vacuum expectation charge arising from the spectral asymmetry.

Monocular depth estimation is very challenging because clues to the exact depth are incomplete in a single RGB image. To overcome the limitation, deep neural networks rely on various visual hints such as size, shade, and texture extracted from RGB information. However, we observe that if such hints are overly exploited, the network can be biased on RGB information without considering the comprehensive view. We propose a novel depth estimation model named RElative Depth Transformer (RED-T) that uses relative depth as guidance in self-attention. Specifically, the model assigns high attention weights to pixels of close depth and low attention weights to pixels of distant depth. As a result, the features of similar depth can become more likely to each other and thus less prone to misused visual hints. We show that the proposed model achieves competitive results in monocular depth estimation benchmarks and is less biased to RGB information. In addition, we propose a novel monocular depth estimation benchmark that limits the observable depth range during training in order to evaluate the robustness of the model for unseen depths., Comment: Accepted for IJCAI 2023

We review the construction of the adiabatic expansion for Bose and Fermi systems and show how it may be used to explore the chiral and parity anomalies for Dirac fermions without the need to compute Feynman diagrams., Comment: 50 pages, Latex

Beamforming technique realized by the multiple-input-multiple-output (MIMO) antenna arrays has been widely used to compensate for the severe path loss in the millimeter wave (mmWave) bands. In 5G NR system, the beam sweeping and beam refinement are employed to find out the best beam codeword aligned to the mobile. Due to the complicated handshaking and finite resolution of the codebook, today's 5G-based beam management strategy is ineffective in various scenarios in terms of the data rate, energy consumption, and also processing latency. An aim of this article is to introduce a new type of beam management framework based on the computer vision (CV) technique. In this framework referred to as computer vision-aided beam management (CVBM), a camera attached to the BS captures the image and then the deep learning-based object detector identifies the 3D location of the mobile. Since the base station can directly set the beam direction without codebook quantization and feedback delay, CVBM achieves the significant beamforming gain and latency reduction. Using the specially designed dataset called Vision Objects for Beam Management (VOBEM), we demonstrate that CVBM achieves more than 40% improvement in the beamforming gain and 40% reduction in the beam training overhead over the 5G NR beam management.

Ferroelectric hafnium zirconium oxide (HZO) thin films show significant promise for applications in ferroelectric random-access memory, ferroelectric field-effect transistors, and ferroelectric tunneling junctions. However, there are shortcomings in understanding ferroelectric switching, which is crucial in the operation of these devices. Here a computational model based on phase field method is developed to simulate the switching behavior of polycrystalline HZO thin films. Furthermore, we introduce a novel approach to optimize the effective Landau coefficients describing the free energy of HZO by combining the phase field model with a genetic algorithm. We validate the model by accurately simulating switching curves for HZO thin films with different ferroelectric phase fractions. The simulated domain dynamics during switching also shows amazing similarity to the available experimental observations. The present work also provides fundamental insights into enhancing the ferroelectricity in HZO thin films by controlling grain morphology and crystalline texture. It can potentially be extended to improve the ferroelectric properties of other hafnia based thin films., Comment: Supplementary information is available with the main manuscript