Search

In this paper, we construct two lower order mixed elements for the linear elasticity problem in the Hellinger-Reissner formulation, one for the 2D problem and one for the 3D problem, both on macro-element meshes. The discrete stress spaces enrich the analogous $P_k$ stress spaces in [J. Hu and S. Zhang, arxiv, 2014, J. Hu and S. Zhang, Sci. China Math., 2015] with simple macro-element bubble functions, and the discrete displacement spaces are discontinuous piecewise $P_{k-1}$ polynomial spaces, with $k=2,3$, respectively. Discrete stability and optimal convergence is proved by using the macro-element technique. As a byproduct, the discrete stability and optimal convergence of the $P_2-P_1$ mixed element in [L. Chen and X. Huang, SIAM J. Numer. Anal., 2022] in 3D is proved on another macro-element mesh. For the mixed element in 2D, an $H^2$-conforming composite element is constructed and an exact discrete elasticity sequence is presented. Numerical experiments confirm the theoretical results.

Photonic integrated switches that are both space and wavelength selective are a highly promising technology for data-intensive applications as they benefit from multi-dimensional manipulation of optical signals. However, scaling these switches normally poses stringent challenges such as increased fabrication complexity and control difficulties, due to the growing number of switching elements. In this work, we propose a novel dilated crosspoint topology, which efficiently handles both space and wavelength selective switching, while reducing the required switching element count by an order of magnitude compared to reported designs. To the best of our knowledge, our design requires the fewest switching elements for an equivalent routing paths number and it fully cancels the first-order in-band crosstalk. We demonstrate such an ultra-compact space-and-wavelength-selective switch (SWSS) at a scale of 4{\times}4{\times}4{\lambda} on the silicon-on-insulator (SOI) platform. Experimental results reveal that the switch achieves an insertion loss ranging from 2.3 dB to 8.6 dB and crosstalk levels in between -35.3 dB and -59.7 dB. The add-drop microring-resonators (MRRs) are equipped with micro-heaters, exhibiting a rise and fall time of 46 {\mu}s and 0.33 {\mu}s, respectively. These performance characteristics highlight the switch's ultra-low element count and crosstalk with low insertion loss, making it a promising candidate for advanced data center applications.

The 2023 Big ANN Challenge, held at NeurIPS 2023, focused on advancing the state-of-the-art in indexing data structures and search algorithms for practical variants of Approximate Nearest Neighbor (ANN) search that reflect the growing complexity and diversity of workloads. Unlike prior challenges that emphasized scaling up classical ANN search ~\cite{DBLP:conf/nips/SimhadriWADBBCH21}, this competition addressed filtered search, out-of-distribution data, sparse and streaming variants of ANNS. Participants developed and submitted innovative solutions that were evaluated on new standard datasets with constrained computational resources. The results showcased significant improvements in search accuracy and efficiency over industry-standard baselines, with notable contributions from both academic and industrial teams. This paper summarizes the competition tracks, datasets, evaluation metrics, and the innovative approaches of the top-performing submissions, providing insights into the current advancements and future directions in the field of approximate nearest neighbor search., Comment: Code: https://github.com/harsha-simhadri/big-ann-benchmarks/releases/tag/v0.3.0

Current Pose-Guided Person Image Synthesis (PGPIS) methods depend heavily on large amounts of labeled triplet data to train the generator in a supervised manner. However, they often falter when applied to in-the-wild samples, primarily due to the distribution gap between the training datasets and real-world test samples. While some researchers aim to enhance model generalizability through sophisticated training procedures, advanced architectures, or by creating more diverse datasets, we adopt the test-time fine-tuning paradigm to customize a pre-trained Text2Image (T2I) model. However, naively applying test-time tuning results in inconsistencies in facial identities and appearance attributes. To address this, we introduce a Visual Consistency Module (VCM), which enhances appearance consistency by combining the face, text, and image embedding. Our approach, named OnePoseTrans, requires only a single source image to generate high-quality pose transfer results, offering greater stability than state-of-the-art data-driven methods. For each test case, OnePoseTrans customizes a model in around 48 seconds with an NVIDIA V100 GPU.

This paper presents a new mixed finite element method for the Cahn-Hilliard equation. The well-posedness of the mixed formulation is established and the error estimates for its linearized fully discrete scheme are provided. Numerical experiments are given to validate the efficiency and accuracy of the theoretical results.

Integrated photonics provides an important platform for simulating physical models with high-performance chip-scale devices, where the lattice size and the time-dependence of a model are key ingredients for further enriching the functionality of a photonic chip. Here, we propose and demonstrate the construction of various time-dependent Hamiltonian models using a single microresonator on thin-film lithium niobate chip. Such an integrated microresonator holds high quality factor to 10^6, and supports the construction of the synthetic frequency lattice with effective lattice sites up to 152 under the electro-optic modulation. By further applying a bichromatic modulation composed of two radio-frequency signals oppositely detuned from the resonant frequency in the microresonator, we build different time-dependent Hamiltonians with the time-varying nearest-neighbor coupling strength in synthetic frequency lattice. We measure the temporal features from capturing the dynamic band structures of the lattice and demonstrate a variety of time-dependent synthetic lattice models by engineering the driven pattern of the modulation, highlighting great flexibility of the microresonator. Our work shows a photonic chip for simulating versatile time-dependent Hamiltonians, which pushes forward quantum simulations in integrated photonics with great experimental tunability and reconfigurability., Comment: 14 pages, 5 figures

This paper proposes a learning framework, RoSE-Opt, to achieve robust and efficient analog circuit parameter optimization. RoSE-Opt has two important features. First, it incorporates key domain knowledge of analog circuit design, such as circuit topology, couplings between circuit specifications, and variations of process, supply voltage, and temperature, into the learning loop. This strategy facilitates the training of an artificial agent capable of achieving design goals by identifying device parameters that are optimal and robust. Second, it exploits a two-level optimization method, that is, integrating Bayesian optimization (BO) with reinforcement learning (RL) to improve sample efficiency. In particular, BO is used for a coarse yet quick search of an initial starting point for optimization. This sets a solid foundation to efficiently train the RL agent with fewer samples. Experimental evaluations on benchmarking circuits show promising sample efficiency, extraordinary figure-of-merit in terms of design efficiency and design success rate, and Pareto optimality in circuit performance of our framework, compared to previous methods. Furthermore, this work thoroughly studies the performance of different RL optimization algorithms, such as Deep Deterministic Policy Gradients (DDPG) with an off-policy learning mechanism and Proximal Policy Optimization (PPO) with an on-policy learning mechanism. This investigation provides users with guidance on choosing the appropriate RL algorithms to optimize the device parameters of analog circuits. Finally, our study also demonstrates RoSE-Opt's promise in parasitic-aware device optimization for analog circuits. In summary, our work reports a knowledge-infused BO-RL design automation framework for reliable and efficient optimization of analog circuits' device parameters., Comment: 14 pages, 12 Figures. Accepted by IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems

In this paper, we address the problem of plausible object placement for the challenging task of realistic image composition. We propose DiffPop, the first framework that utilizes plausibility-guided denoising diffusion probabilistic model to learn the scale and spatial relations among multiple objects and the corresponding scene image. First, we train an unguided diffusion model to directly learn the object placement parameters in a self-supervised manner. Then, we develop a human-in-the-loop pipeline which exploits human labeling on the diffusion-generated composite images to provide the weak supervision for training a structural plausibility classifier. The classifier is further used to guide the diffusion sampling process towards generating the plausible object placement. Experimental results verify the superiority of our method for producing plausible and diverse composite images on the new Cityscapes-OP dataset and the public OPA dataset, as well as demonstrate its potential in applications such as data augmentation and multi-object placement tasks. Our dataset and code will be released.

Optical frequency combs (OFCs), composed of equally spaced frequency tones, have spurred advancements in communications, spectroscopy, precision measurement and fundamental physics research. A prevalent method for generating OFCs involves the electro-optic (EO) effect, i.e., EO comb, renowned for its rapid tunability via precise microwave field control. Recent advances in integrated lithium niobate (LN) photonics have greatly enhanced the efficiency of EO effect, enabling the generation of broadband combs with reduced microwave power. However, parasitic nonlinear effects, such as Raman scattering and four-wave mixing, often emerge in high quality nonlinear devices, impeding the expansion of comb bandwidth and the minimization of frequency noise. Here, we tame these nonlinear effects and present a novel type of OFC, i.e., the self-locked Raman-electro-optic (REO) microcomb by leveraging the collaboration of EO, Kerr and Raman scattering processes. The spectral width of the REO microcomb benefits from the Raman gain and Kerr effect, encompassing nearly 1400 comb lines spanning over 300 nm with a fine repetition rate of 26.03 GHz, much larger than the pure EO combs. Remarkably, the system can maintain a self-locked low-noise state in the presence of multiple nonlinearities without the need for external active feedback. Our approach points to a direction for improving the performance of microcombs and paves the way for exploring new nonlinear physics, such as new laser locking techniques, through the collaboration of inevitable multiple nonlinear effects in integrated photonics.

Forecasting future scenarios in dynamic environments is essential for intelligent decision-making and navigation, a challenge yet to be fully realized in computer vision and robotics. Traditional approaches like video prediction and novel-view synthesis either lack the ability to forecast from arbitrary viewpoints or to predict temporal dynamics. In this paper, we introduce GaussianPrediction, a novel framework that empowers 3D Gaussian representations with dynamic scene modeling and future scenario synthesis in dynamic environments. GaussianPrediction can forecast future states from any viewpoint, using video observations of dynamic scenes. To this end, we first propose a 3D Gaussian canonical space with deformation modeling to capture the appearance and geometry of dynamic scenes, and integrate the lifecycle property into Gaussians for irreversible deformations. To make the prediction feasible and efficient, a concentric motion distillation approach is developed by distilling the scene motion with key points. Finally, a Graph Convolutional Network is employed to predict the motions of key points, enabling the rendering of photorealistic images of future scenarios. Our framework shows outstanding performance on both synthetic and real-world datasets, demonstrating its efficacy in predicting and rendering future environments., Comment: Accepted to SIGGRAPH 2024 Conference. Project Page: https://zju3dv.github.io/gaussian-prediction/

3D sketches are widely used for visually representing the 3D shape and structure of objects or scenes. However, the creation of 3D sketch often requires users to possess professional artistic skills. Existing research efforts primarily focus on enhancing the ability of interactive sketch generation in 3D virtual systems. In this work, we propose Diff3DS, a novel differentiable rendering framework for generating view-consistent 3D sketch by optimizing 3D parametric curves under various supervisions. Specifically, we perform perspective projection to render the 3D rational B\'ezier curves into 2D curves, which are subsequently converted to a 2D raster image via our customized differentiable rasterizer. Our framework bridges the domains of 3D sketch and raster image, achieving end-toend optimization of 3D sketch through gradients computed in the 2D image domain. Our Diff3DS can enable a series of novel 3D sketch generation tasks, including textto-3D sketch and image-to-3D sketch, supported by the popular distillation-based supervision, such as Score Distillation Sampling (SDS). Extensive experiments have yielded promising results and demonstrated the potential of our framework., Comment: Project: https://yiboz2001.github.io/Diff3DS/

A neural network model, EFITNN, has been developed capable of real-time magnetic equilibrium reconstruction based on HL-3 tokamak magnetic measurement signals. The model processes inputs from 68 channels of magnetic measurement data gathered from 1159 HL-3 experimental discharges, including plasma current, loop voltage, and the poloidal magnetic fields measured by equilibrium probes. The outputs of the model feature eight key plasma parameters, alongside high-resolution ($129\times129$) reconstructions of the toroidal current density $J_{\text P}$ and poloidal magnetic flux profiles $\Psi_{rz}$. Moreover, the network's architecture employs a multi-task learning structure, which enables the sharing of weights and mutual correction among different outputs, and lead to increase the model's accuracy by up to 32%. The performance of EFITNN demonstrates remarkable consistency with the offline EFIT, achieving average $R^2 = 0.941, 0.997$ and $0.959$ for eight plasma parameters, $\Psi_{rz}$ and $J_{\text P}$, respectively. The model's robust generalization capabilities are particularly evident in its successful predictions of quasi-snowflake (QSF) divertor configurations and its adept handling of data from shot numbers or plasma current intervals not previously encountered during training. Compared to numerical methods, EFITNN significantly enhances computational efficiency with average computation time ranging from 0.08ms to 0.45ms, indicating its potential utility in real-time isoflux control and plasma profile management.

Kerr microcombs generated in optical microresonators provide broadband light sources bridging optical and microwave signals. Their translation to thin-film lithium niobate unlocks second-order nonlinear optical interfaces such as electro-optic modulation and frequency doubling for completing comb functionalities. However, the strong Raman response of LiNbO$_3$ has complicated the formation of Kerr microcombs. Until now, dark pulse microcombs, requiring a double balance between Kerr nonlinearity and normal group velocity dispersion as well as gain and loss, have remained elusive in LiNbO$_3$ microresonators. Here, by incorporating dissipation engineering, we demonstrate dark pulse microcombs with 25 GHz repetition frequency and 200 nm span in a high-$Q$ LiNbO$_3$ microresonator. Resonances near the Raman-active wavelengths are strongly damped by controlling phase-matching conditions of a specially designed pulley coupler. The coherence and tunability of the dark pulse microcombs are also investigated. Our work provides a solution to realize high-power microcombs operating at microwave rates on LiNbO$_3$ chips, promising new opportunities for the monolithic integration of applications spanning communication to microwave photonics.

Recent advancements in image inpainting, particularly through diffusion modeling, have yielded promising outcomes. However, when tested in scenarios involving the completion of images based on the foreground objects, current methods that aim to inpaint an image in an end-to-end manner encounter challenges such as "over-imagination", inconsistency between foreground and background, and limited diversity. In response, we introduce Anywhere, a pioneering multi-agent framework designed to address these issues. Anywhere utilizes a sophisticated pipeline framework comprising various agents such as Visual Language Model (VLM), Large Language Model (LLM), and image generation models. This framework consists of three principal components: the prompt generation module, the image generation module, and the outcome analyzer. The prompt generation module conducts a semantic analysis of the input foreground image, leveraging VLM to predict relevant language descriptions and LLM to recommend optimal language prompts. In the image generation module, we employ a text-guided canny-to-image generation model to create a template image based on the edge map of the foreground image and language prompts, and an image refiner to produce the outcome by blending the input foreground and the template image. The outcome analyzer employs VLM to evaluate image content rationality, aesthetic score, and foreground-background relevance, triggering prompt and image regeneration as needed. Extensive experiments demonstrate that our Anywhere framework excels in foreground-conditioned image inpainting, mitigating "over-imagination", resolving foreground-background discrepancies, and enhancing diversity. It successfully elevates foreground-conditioned image inpainting to produce more reliable and diverse results., Comment: 16 pages, 9 figures, project page: https://anywheremultiagent.github.io

Recent techniques on implicit geometry representation learning and neural rendering have shown promising results for 3D clothed human reconstruction from sparse video inputs. However, it is still challenging to reconstruct detailed surface geometry and even more difficult to synthesize photorealistic novel views with animated human poses. In this work, we introduce PGAHum, a prior-guided geometry and appearance learning framework for high-fidelity animatable human reconstruction. We thoroughly exploit 3D human priors in three key modules of PGAHum to achieve high-quality geometry reconstruction with intricate details and photorealistic view synthesis on unseen poses. First, a prior-based implicit geometry representation of 3D human, which contains a delta SDF predicted by a tri-plane network and a base SDF derived from the prior SMPL model, is proposed to model the surface details and the body shape in a disentangled manner. Second, we introduce a novel prior-guided sampling strategy that fully leverages the prior information of the human pose and body to sample the query points within or near the body surface. By avoiding unnecessary learning in the empty 3D space, the neural rendering can recover more appearance details. Last, we propose a novel iterative backward deformation strategy to progressively find the correspondence for the query point in observation space. A skinning weights prediction model is learned based on the prior provided by the SMPL model to achieve the iterative backward LBS deformation. Extensive quantitative and qualitative comparisons on various datasets are conducted and the results demonstrate the superiority of our framework. Ablation studies also verify the effectiveness of each scheme for geometry and appearance learning.

Creating large LiDAR datasets with pixel-level labeling poses significant challenges. While numerous data augmentation methods have been developed to reduce the reliance on manual labeling, these methods predominantly focus on static scenes and they overlook the importance of data augmentation for dynamic scenes, which is critical for autonomous driving. To address this issue, we propose D-Aug, a LiDAR data augmentation method tailored for augmenting dynamic scenes. D-Aug extracts objects and inserts them into dynamic scenes, considering the continuity of these objects across consecutive frames. For seamless insertion into dynamic scenes, we propose a reference-guided method that involves dynamic collision detection and rotation alignment. Additionally, we present a pixel-level road identification strategy to efficiently determine suitable insertion positions. We validated our method using the nuScenes dataset with various 3D detection and tracking methods. Comparative experiments demonstrate the superiority of D-Aug., Comment: 4pages, 4 figures

This paper proposes a mixed variational formulation for the problem of two coupled plates with a rigid {junction}. The proposed mixed {formulation} introduces {the union of} stresses and moments as {an auxiliary variable}, which {are} commonly of great interest in practical applications. The primary challenge lies in determining a suitable {space involving} both boundary and junction conditions of the auxiliary variable. The {theory} of densely defined operators in Hilbert spaces is employed to define {a nonstandard Sobolev space} without the use of trace operators. The well-posedness is established for the mixed formulation. Based on these conditions, this paper provides a framework {of} conforming {mixed} finite element methods. Numerical experiments are given to validate the theoretical results.

Personalized text-to-image (T2I) models not only produce lifelike and varied visuals but also allow users to tailor the images to fit their personal taste. These personalization techniques can grasp the essence of a concept through a collection of images, or adjust a pre-trained text-to-image model with a specific image input for subject-driven or attribute-aware guidance. Yet, accurately capturing the distinct visual attributes of an individual image poses a challenge for these methods. To address this issue, we introduce OSTAF, a novel parameter-efficient one-shot fine-tuning method which only utilizes one reference image for T2I personalization. A novel hypernetwork-powered attribute-focused fine-tuning mechanism is employed to achieve the precise learning of various attribute features (e.g., appearance, shape or drawing style) from the reference image. Comparing to existing image customization methods, our method shows significant superiority in attribute identification and application, as well as achieves a good balance between efficiency and output quality.

With the development of neural radiance fields and generative models, numerous methods have been proposed for learning 3D human generation from 2D images. These methods allow control over the pose of the generated 3D human and enable rendering from different viewpoints. However, none of these methods explore semantic disentanglement in human image synthesis, i.e., they can not disentangle the generation of different semantic parts, such as the body, tops, and bottoms. Furthermore, existing methods are limited to synthesize images at $512^2$ resolution due to the high computational cost of neural radiance fields. To address these limitations, we introduce SemanticHuman-HD, the first method to achieve semantic disentangled human image synthesis. Notably, SemanticHuman-HD is also the first method to achieve 3D-aware image synthesis at $1024^2$ resolution, benefiting from our proposed 3D-aware super-resolution module. By leveraging the depth maps and semantic masks as guidance for the 3D-aware super-resolution, we significantly reduce the number of sampling points during volume rendering, thereby reducing the computational cost. Our comparative experiments demonstrate the superiority of our method. The effectiveness of each proposed component is also verified through ablation studies. Moreover, our method opens up exciting possibilities for various applications, including 3D garment generation, semantic-aware image synthesis, controllable image synthesis, and out-of-domain image synthesis., Comment: 26 pages, 14 figures

Text-driven 3D scene generation techniques have made rapid progress in recent years. Their success is mainly attributed to using existing generative models to iteratively perform image warping and inpainting to generate 3D scenes. However, these methods heavily rely on the outputs of existing models, leading to error accumulation in geometry and appearance that prevent the models from being used in various scenarios (e.g., outdoor and unreal scenarios). To address this limitation, we generatively refine the newly generated local views by querying and aggregating global 3D information, and then progressively generate the 3D scene. Specifically, we employ a tri-plane features-based NeRF as a unified representation of the 3D scene to constrain global 3D consistency, and propose a generative refinement network to synthesize new contents with higher quality by exploiting the natural image prior from 2D diffusion model as well as the global 3D information of the current scene. Our extensive experiments demonstrate that, in comparison to previous methods, our approach supports wide variety of scene generation and arbitrary camera trajectories with improved visual quality and 3D consistency., Comment: 11 pages, 7 figures

Optical circuit switching is connection-oriented, being deterministic through the reservation of a complete wavelength channel or spatial path for a certain period. However, this comes at a trade-off against link dynamics, and overall capacity can thus be constrained by the time slot reservations, especially for switches with microsecond- to millisecond-scale reconfiguration times. For data-intensive applications, the communication patterns associated with random data sets typically yield short-lived flows. This situation calls for a new multi-dimensional switching paradigm that fully exploits not only the space and wavelength domains but also with nanosecond-scale reconfigurable capability in the time domain to enable ultra-dynamic links. In this work, we focus on the exploitation of micro-ring resonant phase shifters (RPSs) that are wavelength selective for optical switching in a single plane. By proposing an innovative analytical method with transmission circle chart, we fully unlock the power of RPS with nanosecond-scale reconfigurability and the capability to arbitrarily manipulate its phase and amplitude. Such a compact model offers fresh insights into designs with under and critically coupled RPSs beyond the commonly explored over-coupling condition. This creates not only versatile switch elements but also perfect absorbers for robust multi-wavelength operations. The proposed device can bring about a breakthrough in the optical switching capacity that potentially addresses the challenges faced by modern data center networks, as well as other photonic circuits for high-throughput signal processing., Comment: 10 pages

Coordinate-based neural implicit representation or implicit fields have been widely studied for 3D geometry representation or novel view synthesis. Recently, a series of efforts have been devoted to accelerating the speed and improving the quality of the coordinate-based implicit field learning. Instead of learning heavy MLPs to predict the neural implicit values for the query coordinates, neural voxels or grids combined with shallow MLPs have been proposed to achieve high-quality implicit field learning with reduced optimization time. On the other hand, lightweight field representations such as linear grid have been proposed to further improve the learning speed. In this paper, we aim for both fast and high-quality implicit field learning, and propose TaylorGrid, a novel implicit field representation which can be efficiently computed via direct Taylor expansion optimization on 2D or 3D grids. As a general representation, TaylorGrid can be adapted to different implicit fields learning tasks such as SDF learning or NeRF. From extensive quantitative and qualitative comparisons, TaylorGrid achieves a balance between the linear grid and neural voxels, showing its superiority in fast and high-quality implicit field learning.

Neural network compression techniques, such as knowledge distillation (KD) and network pruning, have received increasing attention. Recent work `Prune, then Distill' reveals that a pruned student-friendly teacher network can benefit the performance of KD. However, the conventional teacher-student pipeline, which entails cumbersome pre-training of the teacher and complicated compression steps, makes pruning with KD less efficient. In addition to compressing models, recent compression techniques also emphasize the aspect of efficiency. Early pruning demands significantly less computational cost in comparison to the conventional pruning methods as it does not require a large pre-trained model. Likewise, a special case of KD, known as self-distillation (SD), is more efficient since it requires no pre-training or student-teacher pair selection. This inspires us to collaborate early pruning with SD for efficient model compression. In this work, we propose the framework named Early Pruning with Self-Distillation (EPSD), which identifies and preserves distillable weights in early pruning for a given SD task. EPSD efficiently combines early pruning and self-distillation in a two-step process, maintaining the pruned network's trainability for compression. Instead of a simple combination of pruning and SD, EPSD enables the pruned network to favor SD by keeping more distillable weights before training to ensure better distillation of the pruned network. We demonstrated that EPSD improves the training of pruned networks, supported by visual and quantitative analyses. Our evaluation covered diverse benchmarks (CIFAR-10/100, Tiny-ImageNet, full ImageNet, CUB-200-2011, and Pascal VOC), with EPSD outperforming advanced pruning and SD techniques., Comment: The first two authors are with equal contributions. Paper accepted by AAAI 2024

Existing 3D-aware portrait synthesis methods can generate impressive high-quality images while preserving strong 3D consistency. However, most of them cannot support the fine-grained part-level control over synthesized images. Conversely, some GAN-based 2D portrait synthesis methods can achieve clear disentanglement of facial regions, but they cannot preserve view consistency due to a lack of 3D modeling abilities. To address these issues, we propose 3D-SSGAN, a novel framework for 3D-aware compositional portrait image synthesis. First, a simple yet effective depth-guided 2D-to-3D lifting module maps the generated 2D part features and semantics to 3D. Then, a volume renderer with a novel 3D-aware semantic mask renderer is utilized to produce the composed face features and corresponding masks. The whole framework is trained end-to-end by discriminating between real and synthesized 2D images and their semantic masks. Quantitative and qualitative evaluations demonstrate the superiority of 3D-SSGAN in controllable part-level synthesis while preserving 3D view consistency.

Integrating watermarks into generative images is a critical strategy for protecting intellectual property and enhancing artificial intelligence security. This paper proposes Plug-in Generative Watermarking (PiGW) as a general framework for integrating watermarks into generative images. More specifically, PiGW embeds watermark information into the initial noise using a learnable watermark embedding network and an adaptive frequency spectrum mask. Furthermore, it optimizes training costs by gradually increasing timesteps. Extensive experiments demonstrate that PiGW enables embedding watermarks into the generated image with negligible quality loss while achieving true invisibility and high resistance to noise attacks. Moreover, PiGW can serve as a plugin for various commonly used generative structures and multimodal generative content types. Finally, we demonstrate how PiGW can also be utilized for detecting generated images, contributing to the promotion of secure AI development. The project code will be made available on GitHub., Comment: Improve experimental content

Copyright law confers upon creators the exclusive rights to reproduce, distribute, and monetize their creative works. However, recent progress in text-to-image generation has introduced formidable challenges to copyright enforcement. These technologies enable the unauthorized learning and replication of copyrighted content, artistic creations, and likenesses, leading to the proliferation of unregulated content. Notably, models like stable diffusion, which excel in text-to-image synthesis, heighten the risk of copyright infringement and unauthorized distribution.Machine unlearning, which seeks to eradicate the influence of specific data or concepts from machine learning models, emerges as a promising solution by eliminating the \enquote{copyright memories} ingrained in diffusion models. Yet, the absence of comprehensive large-scale datasets and standardized benchmarks for evaluating the efficacy of unlearning techniques in the copyright protection scenarios impedes the development of more effective unlearning methods. To address this gap, we introduce a novel pipeline that harmonizes CLIP, ChatGPT, and diffusion models to curate a dataset. This dataset encompasses anchor images, associated prompts, and images synthesized by text-to-image models. Additionally, we have developed a mixed metric based on semantic and style information, validated through both human and artist assessments, to gauge the effectiveness of unlearning approaches. Our dataset, benchmark library, and evaluation metrics will be made publicly available to foster future research and practical applications (https://rmpku.github.io/CPDM-page/, website / http://149.104.22.83/unlearning.tar.gz, dataset)., Comment: 20 pages, 7 figures, 3 table

Inferring missing regions from severely occluded point clouds is highly challenging. Especially for 3D shapes with rich geometry and structure details, inherent ambiguities of the unknown parts are existing. Existing approaches either learn a one-to-one mapping in a supervised manner or train a generative model to synthesize the missing points for the completion of 3D point cloud shapes. These methods, however, lack the controllability for the completion process and the results are either deterministic or exhibiting uncontrolled diversity. Inspired by the prompt-driven data generation and editing, we propose a novel prompt-guided point cloud completion framework, coined P2M2-Net, to enable more controllable and more diverse shape completion. Given an input partial point cloud and a text prompt describing the part-aware information such as semantics and structure of the missing region, our Transformer-based completion network can efficiently fuse the multimodal features and generate diverse results following the prompt guidance. We train the P2M2-Net on a new large-scale PartNet-Prompt dataset and conduct extensive experiments on two challenging shape completion benchmarks. Quantitative and qualitative results show the efficacy of incorporating prompts for more controllable part-aware point cloud completion and generation. Code and data are available at https://github.com/JLU-ICL/P2M2-Net., Comment: Best Poster Award of CAD/Graphics 2023

Thin-film lithium niobate (TFLN) enables extremely high-efficiency second-order nonlinear optical effects due to large nonlinear coefficient d33 and strong optical field localization. Here, we first designed and fabricated a pulley-waveguide-coupled microring resonator with an intrinsic quality factor above 9.4 x10^5 on the reverse-polarized double-layer X-cut TFLN. In such a TFLN resonator without fine domain structures, second harmonic generation with an absolute (normalized) conversion efficiency of 30% (440,000% W-1), comparable to that in periodically poled lithium niobate (PPLN) microring resonators, was realized with a sub-microwatt continuous pump. This work reduces the dependence of high-efficiency nonlinear frequency conversion on PPLN microcavities that are difficult to prepare.

Due to the potential applications in biomedical engineering, it becomes more and more important to understand the process of engulfment and internalization of nanoparticles (NPs) by cell membranes. Despite the fact that the interaction between cell membranes and passive particles has been widely studied, the interaction between cell membranes and self-propelled nonspherical NPs remains to be elucidated. Here we present a theoretical model to systematically investigate the influence of the active force, aspect ratio of NPs, particle size and membrane properties (adhesion energy density and membrane tension) on the cellular uptake of a nonspherical nanoparticle. It is found that the active force generated by an NP can trigger a type of first-order wrapping transition from a small partial wrapping state to a large one. In addition, the phase diagram in the force-aspect ratio (particle size, adhesion energy density and membrane tension) space displays more complex behaviors compared with that for the passive wrapping mediated merely by adhesion. These results may provide a useful guidance to the study of activity-driven cellular entry of active particles into cells.

In the past decade, photonic integrated circuits (PICs) based on thin-film lithium niobate (TFLN) have advanced in various fields, including optical communication, nonlinear photonics, and quantum optics. A critical component is an efficient edge coupler connecting PICs to light sources or detectors. Here, we propose an innovative edge coupler design with a wedge-shaped TFLN waveguide and a silicon oxynitride (SiON) cladding. Experimental results show that the coupling loss between the TFLN PIC and a 3-{\mu}m mode field diameter (MFD) lensed fiber is low at 1.52 dB/facet, with the potential for improvement to 0.43 dB/facet theoretically. The coupling loss between the edge coupler and a UHNA7 fiber with an MFD of 3.2 {\mu}m is reduced to 0.92 dB/facet. This design maintains robust fabrication and alignment tolerance. Importantly, the minimum linewidth of the TFLN waveguide of the coupler (600 nm) can be easily achieved using foundry-available i-line stepper lithography. This work benefits the development of TFLN integrated platforms, such as on-chip electro-optic modulators, frequency comb generation, and quantum sensors.

Characterizing the orbital angular momentum (OAM) of a vortex beam is critically important for OAM-encoded data transfer. However, in typical OAM-based applications where vortex beams transmit through diffusers, the accompanying scattering effect tends to be either deliberately prevented, or characterized and then modulated actively based on complex wavefront shaping and interferometry techniques. Here, we aim to investigate the characteristics of blurred speckles obtained after a vortex beam transmits through a ground glass diffuser. It is theoretically and experimentally demonstrated that a cross-correlation annulus can be identified by implementing the cross-correlation operation between speckle patterns corresponding to vortex beams with different OAM values. Besides, it is worth noting that, the size of the cross-correlation annulus is determined by the absolute value of the topological charge difference between the two corresponding vortex beams. Based on this mechanism, the OAM modes can be easily sorted from the incoherently measured OAM-dependent speckles as well as their cross-correlation. Furthermore, to make full use of the orthogonal feature of the OAM-dependent speckles, demultiplexing of OAM-encoded data transfer is verified using a ground glass diffuser. Both 8-bit grayscale and 24-bit RGB OAM-encoded data transfers are carried out in experiments with superior error rates. We can conclude that the OAM-dependent speckles can be not only utilized as a competitive candidate for the OAM mode sorting function in a simple way but also provide an efficient method for the demultiplexing of OAM-encoded data transfer in a practical application.

Spinal curvature estimation is important to the diagnosis and treatment of the scoliosis. Existing methods face several issues such as the need of expensive annotations on the vertebral landmarks and being sensitive to the image quality. It is challenging to achieve robust estimation and obtain interpretable results, especially for low-quality images which are blurry and hazy. In this paper, we propose B-Spine, a novel deep learning pipeline to learn B-spline curve representation of the spine and estimate the Cobb angles for spinal curvature estimation from low-quality X-ray images. Given a low-quality input, a novel SegRefine network which employs the unpaired image-to-image translation is proposed to generate a high quality spine mask from the initial segmentation result. Next, a novel mask-based B-spline prediction model is proposed to predict the B-spline curve for the spine centerline. Finally, the Cobb angles are estimated by a hybrid approach which combines the curve slope analysis and a curve-based regression model. We conduct quantitative and qualitative comparisons with the representative and SOTA learning-based methods on the public AASCE2019 dataset and our new proposed CJUH-JLU dataset which contains more challenging low-quality images. The superior performance on both datasets shows our method can achieve both robustness and interpretability for spinal curvature estimation.

The Low-Energy X-ray Polarization Detector (LPD) is one of the payloads in the POLAR-2 experiment, designed as an external payload for the China Space Station (CSS) deployment in early 2024. LPD is specifically designed to observe the polarization of Gamma-Ray Bursts (GRBs) prompt emission in the energy range of 2-10 keV, with a wide field of view (FoV) of 90 degrees in preliminary design. This observation is achieved using an array of X-ray photoelectric polarimeters based on gas pixel detectors. Due to the wide FoV configuration, the in-orbit background count rate in the soft X-ray range is high, while GRBs themselves also exhibit a high flux in this energy band. In order to assess the contribution of various background components to the total count rate, we conducted detailed simulations using the GEANT4 C++ package. Our simulations encompassed the main interactions within the instrument materials and provided insights into various background components within the wide FoV scheme. The simulation results reveal that among the background components, the primary contributors are the cosmic X-ray background (CXB) and bright X-ray sources. The total background count rate of LPD, after applying the charged particle background rejection algorithm, is approximately 0.55 counts/cm^2/s on average, and it varies with the detector's orbit and pointing direction. Furthermore, we performed comprehensive simulations and comparative analyses of the CXB and X-ray bright sources under different FoVs and detector pointings. These analyses provide valuable insights into the background characteristic for soft X-ray polarimeter with wide FoV.

On-device training has become an increasingly popular approach to machine learning, enabling models to be trained directly on mobile and edge devices. However, a major challenge in this area is the limited memory available on these devices, which can severely restrict the size and complexity of the models that can be trained. In this systematic survey, we aim to explore the current state-of-the-art techniques for breaking on-device training memory walls, focusing on methods that can enable larger and more complex models to be trained on resource-constrained devices. Specifically, we first analyze the key factors that contribute to the phenomenon of memory walls encountered during on-device training. Then, we present a comprehensive literature review of on-device training, which addresses the issue of memory limitations. Finally, we summarize on-device training and highlight the open problems for future research. By providing a comprehensive overview of these techniques and their effectiveness in breaking memory walls, we hope to help researchers and practitioners in this field navigate the rapidly evolving landscape of on-device training., Comment: 8 pages, 3 figures