Search

Compositing an object into an image involves multiple non-trivial sub-tasks such as object placement and scaling, color/lighting harmonization, viewpoint/geometry adjustment, and shadow/reflection generation. Recent generative image compositing methods leverage diffusion models to handle multiple sub-tasks at once. However, existing models face limitations due to their reliance on masking the original object during training, which constrains their generation to the input mask. Furthermore, obtaining an accurate input mask specifying the location and scale of the object in a new image can be highly challenging. To overcome such limitations, we define a novel problem of unconstrained generative object compositing, i.e., the generation is not bounded by the mask, and train a diffusion-based model on a synthesized paired dataset. Our first-of-its-kind model is able to generate object effects such as shadows and reflections that go beyond the mask, enhancing image realism. Additionally, if an empty mask is provided, our model automatically places the object in diverse natural locations and scales, accelerating the compositing workflow. Our model outperforms existing object placement and compositing models in various quality metrics and user studies.

Audio generation has achieved remarkable progress with the advance of sophisticated generative models, such as diffusion models (DMs) and autoregressive (AR) models. However, due to the naturally significant sequence length of audio, the efficiency of audio generation remains an essential issue to be addressed, especially for AR models that are incorporated in large language models (LLMs). In this paper, we analyze the token length of audio tokenization and propose a novel \textbf{S}cale-level \textbf{A}udio \textbf{T}okenizer (SAT), with improved residual quantization. Based on SAT, a scale-level \textbf{A}coustic \textbf{A}uto\textbf{R}egressive (AAR) modeling framework is further proposed, which shifts the next-token AR prediction to next-scale AR prediction, significantly reducing the training cost and inference time. To validate the effectiveness of the proposed approach, we comprehensively analyze design choices and demonstrate the proposed AAR framework achieves a remarkable \textbf{35}$\times$ faster inference speed and +\textbf{1.33} Fr\'echet Audio Distance (FAD) against baselines on the AudioSet benchmark. Code: \url{https://github.com/qiuk2/AAR}., Comment: 7 pages, 6 figures, 7 tables

We present the results of the ALMA Large Program Early Planet Formation in Embedded disks observations of the Class 0 protostar GSS30 IRS3. Our observations included 1.3 mm continuum with a resolution of 0.''05 (7.8 au) and several molecular species including $^{12}$CO, $^{13}$CO, C$^{18}$O, H$_{2}$CO and c-C$_{3}$H$_{2}$. The dust continuum analysis unveiled a disk-shaped structure with a major axis size of $\sim$200 au. We observed an asymmetry in the minor axis of the continuum emission suggesting that the emission is optically thick and the disk is flared. On the other hand, we identified two prominent bumps along the major axis located at distances of 26 and 50 au from the central protostar. The origin of the bumps remains uncertain and might be due to an embedded substructure within the disk or the result of the temperature distribution instead of surface density due to optically thick continuum emission. The $^{12}$CO emission reveals a molecular outflow consisting of three distinct components: a collimated one, an intermediate velocity component exhibiting an hourglass shape, and a wider angle low-velocity component. We associate these components with the coexistence of a jet and a disk-wind. The C$^{18}$O emission traces both a Keplerian rotating circumstellar disk and the infall of the rotating envelope. We measured a stellar dynamical mass of 0.35$\pm$0.09 M$_{\odot}$., Comment: 25 pages, 19 figures. Accepted for publication in A&A

The HH 111 protostellar disk has recently been found to host a pair of spiral arms. Here we report the dust polarization results in the disk as well as the inner envelope around it, obtained with the Atacama Large Millimeter/submillimeter Array in continuum at lambda ~ 870 micron and ~ 0. 05" resolution. In the inner envelope, polarization is detected with a polarization degree of ~ 6% and an orientation almost everywhere parallel to the minor axis of the disk, and thus likely to be due to the dust grains magnetically aligned mainly by toroidal fields. In the disk, the polarization orientation is roughly azimuthal on the far side and becomes parallel to the minor axis on the near side, with a polarization gap in between on the far side near the central protostar. The disk polarization degree is ~ 2%. The polarized intensity is higher on the near side than the far side, showing a near-far side asymmetry. More importantly, the polarized intensity and thus polarization degree are lower in the spiral arms, but higher in between the arms, showing an anticorrelation of the polarized intensity with the spiral arms. Our modeling results indicate that this anticorrelation is useful for constraining the polarization mechanism and is consistent with the dust self-scattering by the grains that have grown to a size of ~ 150 micron. The interarms are sandwiched and illuminated by two brighter spiral arms and thus have higher polarized intensity. Our dust self-scattering model can also reproduce the observed polarization orientation parallel to the minor axis on the near side and the observed azimuthal polarization orientation at the two disk edges in the major axis., Comment: 23 pages, 5 figures

We present Atacama Large Millimeter/submillimeter Array (ALMA) observations of the binary Class 0 protostellar system BHR 71 IRS1 and IRS2 as part of the Early Planet Formation in Embedded Disks (eDisk) ALMA Large Program. We describe the $^{12}$CO ($J$=2--1), $^{13}$CO ($J$=2--1), C$^{18}$O ($J$=2--1), H$_2$CO ($J=3_{2,1}$--$2_{2,0}$), and SiO ($J$=5--4) molecular lines along with the 1.3 mm continuum at high spatial resolution ($\sim$0.08" or $\sim$5 au). Dust continuum emission is detected toward BHR 71 IRS1 and IRS2, with a central compact component and extended continuum emission. The compact components are smooth and show no sign of substructures such as spirals, rings or gaps. However, there is a brightness asymmetry along the minor axis of the presumed disk in IRS1, possibly indicative of an inclined geometrically and optically thick disk-like component. Using a position-velocity diagram analysis of the C$^{18}$O line, clear Keplerian motions were not detected toward either source. If Keplerian rotationally-supported disks are present, they are likely deeply embedded in their envelope. However, we can set upper limits of the central protostellar mass of 0.46 M$_\odot$ and 0.26 M$_\odot$ for BHR 71 IRS1 and BHR 71 IRS2, respectively. Outflows traced by $^{12}$CO and SiO are detected in both sources. The outflows can be divided into two components, a wide-angle outflow and a jet. In IRS1, the jet exhibits a double helical structure, reflecting the removal of angular momentum from the system. In IRS2, the jet is very collimated and shows a chain of knots, suggesting episodic accretion events., Comment: 38 pages, 29 figures, accepted in ApJ

Recently, the use of large language models (LLMs) for software code generation, e.g., C/C++ and Python, has proven a great success. However, LLMs still suffer from low syntactic and functional correctness when it comes to the generation of register-transfer level (RTL) code, such as Verilog. To address this issue, in this paper, we develop AutoVCoder, a systematic open-source framework that significantly improves the LLMs' correctness of generating Verilog code and enhances the quality of its output at the same time. Our framework integrates three novel techniques, including a high-quality hardware dataset generation approach, a two-round LLM fine-tuning method and a domain-specific retrieval-augmented generation (RAG) mechanism. Experimental results demonstrate that AutoVCoder outperforms both industrial and academic LLMs in Verilog code generation. Specifically, AutoVCoder shows a 0.5% and 2.2% improvement in functional correctness on the EvalMachine and EvalHuman benchmarks compared with BetterV, and also achieves a 3.4% increase in syntax correctness and a 3.4% increase in functional correctness on the RTLLM benchmark compared with RTLCoder.

Recent (sub)millimeter polarization observations of protoplanetary disks reveal toroidally aligned, effectively prolate dust grains large enough (at least ~100 $\mu$m) to efficiently scatter millimeter light. The alignment mechanism for these grains remains unclear. We explore the possibility that gas drag aligns grains through gas-dust relative motion when the grain's center of mass is offset from its geometric center, analogous to a badminton birdie's alignment in flight. A simple grain model of two non-identical spheres illustrates how a grain undergoes damped oscillations from flow-induced restoring torques which align its geometric center in the flow direction relative to its center of mass. Assuming specular reflection and subsonic flow, we derive an analytical equation of motion for spheroids where the center of mass can be shifted away from the spheroid's geometric center. We show that a prolate or an oblate grain can be aligned with the long axis parallel to the gas flow when the center of mass is shifted along that axis. Both scenarios can explain the required effectively prolate grains inferred from observations. Application to a simple disk model shows that the alignment timescales are shorter than or comparable to the orbital time. The grain alignment direction in a disk depends on the disk (sub-)structure and grain Stokes number (St) with azimuthal alignment for large St grains in sub-Keplerian smooth gas disks and for small St grains near the gas pressure extrema, such as rings and gaps., Comment: 21 pages, 12 figures, submitted to MNRAS

Text relevance or text matching of query and product is an essential technique for the e-commerce search system to ensure that the displayed products can match the intent of the query. Many studies focus on improving the performance of the relevance model in search system. Recently, pre-trained language models like BERT have achieved promising performance on the text relevance task. While these models perform well on the offline test dataset, there are still obstacles to deploy the pre-trained language model to the online system as their high latency. The two-tower model is extensively employed in industrial scenarios, owing to its ability to harmonize performance with computational efficiency. Regrettably, such models present an opaque ``black box'' nature, which prevents developers from making special optimizations. In this paper, we raise deep Bag-of-Words (DeepBoW) model, an efficient and interpretable relevance architecture for Chinese e-commerce. Our approach proposes to encode the query and the product into the sparse BoW representation, which is a set of word-weight pairs. The weight means the important or the relevant score between the corresponding word and the raw text. The relevance score is measured by the accumulation of the matched word between the sparse BoW representation of the query and the product. Compared to popular dense distributed representation that usually suffers from the drawback of black-box, the most advantage of the proposed representation model is highly explainable and interventionable, which is a superior advantage to the deployment and operation of online search engines. Moreover, the online efficiency of the proposed model is even better than the most efficient inner product form of dense representation ..., Comment: KDD'24 accepted paper

Traditional discrete learning methods involve discretizing continuous equations using difference schemes, necessitating considerations of stability and convergence. Integrable nonlinear lattice equations possess a profound mathematical structure that enables them to revert to continuous integrable equations in the continuous limit, particularly retaining integrable properties such as conservation laws, Hamiltonian structure, and multiple soliton solutions. The pseudo grid-based physics-informed convolutional-recurrent network (PG-PhyCRNet) is proposed to investigate the localized wave solutions of integrable lattice equations, which significantly enhances the model's extrapolation capability to lattice points beyond the temporal domain. We conduct a comparative analysis of PG-PhyCRNet with and without pseudo grid by investigating the multi-soliton solutions and rational solitons of the Toda lattice and self-dual network equation. The results indicate that the PG-PhyCRNet excels in capturing long-term evolution and enhances the model's extrapolation capability for solitons, particularly those with steep waveforms and high wave speeds. Finally, the robustness of the PG-PhyCRNet method and its effect on the prediction of solutions in different scenarios are confirmed through repeated experiments involving pseudo grid partitioning.

Conditional visual generation has witnessed remarkable progress with the advent of diffusion models (DMs), especially in tasks like control-to-image generation. However, challenges such as expensive computational cost, high inference latency, and difficulties of integration with large language models (LLMs) have necessitated exploring alternatives to DMs. This paper introduces ControlVAR, a novel framework that explores pixel-level controls in visual autoregressive (VAR) modeling for flexible and efficient conditional generation. In contrast to traditional conditional models that learn the conditional distribution, ControlVAR jointly models the distribution of image and pixel-level conditions during training and imposes conditional controls during testing. To enhance the joint modeling, we adopt the next-scale AR prediction paradigm and unify control and image representations. A teacher-forcing guidance strategy is proposed to further facilitate controllable generation with joint modeling. Extensive experiments demonstrate the superior efficacy and flexibility of ControlVAR across various conditional generation tasks against popular conditional DMs, \eg, ControlNet and T2I-Adaptor., Comment: 24 pages, 19 figures, 4 tables

Deoccluding the hidden portions of objects in a scene is a formidable task, particularly when addressing real-world scenes. In this paper, we present a new self-supervised PArallel visible-to-COmplete diffusion framework, named PACO, a foundation model for object-level scene deocclusion. Leveraging the rich prior of pre-trained models, we first design the parallel variational autoencoder, which produces a full-view feature map that simultaneously encodes multiple complete objects, and the visible-to-complete latent generator, which learns to implicitly predict the full-view feature map from partial-view feature map and text prompts extracted from the incomplete objects in the input image. To train PACO, we create a large-scale dataset with 500k samples to enable self-supervised learning, avoiding tedious annotations of the amodal masks and occluded regions. At inference, we devise a layer-wise deocclusion strategy to improve efficiency while maintaining the deocclusion quality. Extensive experiments on COCOA and various real-world scenes demonstrate the superior capability of PACO for scene deocclusion, surpassing the state of the arts by a large margin. Our method can also be extended to cross-domain scenes and novel categories that are not covered by the training set. Further, we demonstrate the deocclusion applicability of PACO in single-view 3D scene reconstruction and object recomposition., Comment: SIGGRAPH 2024. A foundation model for category-agnostic object deocclusion

In this paper, we use a new method to prove cut-elimination of weak intuitionistic tense logic. This method focuses on splitting the contraction rule and cut rules. Further general theories and applications of this method shall be developed in the future.

Diffusion Models (DMs) have exhibited superior performance in generating high-quality and diverse images. However, this exceptional performance comes at the cost of expensive architectural design, particularly due to the attention module heavily used in leading models. Existing works mainly adopt a retraining process to enhance DM efficiency. This is computationally expensive and not very scalable. To this end, we introduce the Attention-driven Training-free Efficient Diffusion Model (AT-EDM) framework that leverages attention maps to perform run-time pruning of redundant tokens, without the need for any retraining. Specifically, for single-denoising-step pruning, we develop a novel ranking algorithm, Generalized Weighted Page Rank (G-WPR), to identify redundant tokens, and a similarity-based recovery method to restore tokens for the convolution operation. In addition, we propose a Denoising-Steps-Aware Pruning (DSAP) approach to adjust the pruning budget across different denoising timesteps for better generation quality. Extensive evaluations show that AT-EDM performs favorably against prior art in terms of efficiency (e.g., 38.8% FLOPs saving and up to 1.53x speed-up over Stable Diffusion XL) while maintaining nearly the same FID and CLIP scores as the full model. Project webpage: https://atedm.github.io., Comment: Accepted to IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024

Millimeter and sub-millimeter observations of continuum linear dust polarization provide insight into dust grain growth in protoplanetary disks, which are the progenitors of planetary systems. We present the results of the first survey of dust polarization in protoplanetary disks at 870 $\mu$m and 3 mm. We find that protoplanetary disks in the same molecular cloud at similar evolutionary stages can exhibit different correlations between observing wavelength and polarization morphology and fraction. We explore possible origins for these differences in polarization, including differences in dust populations and protostar properties. For RY Tau and MWC 480, which are consistent with scattering at both wavelengths, we present models of the scattering polarization from several dust grain size distributions. These models aim to reproduce two features of the observational results for these disks: (1) both disks have an observable degree of polarization at both wavelengths and (2) the polarization fraction is higher at 3 mm than at 870 $\mu$m in the centers of the disks. For both disks, these features can be reproduced by a power-law distribution of spherical dust grains with a maximum radius of 200 $\mu$m and high optical depth. In MWC 480, we can also reproduce features (1) and (2) with a model containing large grains ($a_{max}$ = 490 $\mu$m ) near the disk midplane and small grains ($a_{max}$ = 140 $\mu$m) above and below the midplane., Comment: 22 pages, 12 figures

Young protostellar binary systems, with expected ages less than $\sim$10$^5$ years, are little modified since birth, providing key clues to binary formation and evolution. We present a first look at the young, Class 0 binary protostellar system R CrA IRAS 32 from the Early Planet Formation in Embedded Disks (eDisk) ALMA large program, which observed the system in the 1.3 mm continuum emission, $^{12}$CO (2-1), $^{13}$CO (2-1), C$^{18}$O (2-1), SO (6$_5$-5$_4$), and nine other molecular lines that trace disk, envelope, shocks, and outflows. With a continuum resolution of $\sim$0.03$^{\prime\prime}$ ($\sim$5 au, at a distance of 150 pc), we characterize the newly discovered binary system with a separation of 207 au, their circumstellar disks, and a circumbinary disk-like structure. The circumstellar disk radii are 26.9$\pm$0.3 and 22.8$\pm$0.3 au for sources A and B, respectively, and their circumstellar disk dust masses are estimated as 22.5$\pm$1.1 and 12.4$\pm$0.6 M$_{\Earth}$. The circumstellar disks and the circumbinary structure have well aligned position angles and inclinations, indicating formation in a smooth, ordered process such as disk fragmentation. In addition, the circumstellar disks have a near/far-side asymmetry in the continuum emission suggesting that the dust has yet to settle into a thin layer near the midplane. Spectral analysis of CO isotopologues reveals outflows that originate from both of the sources and possibly from the circumbinary disk-like structure. Furthermore, we detect Keplerian rotation in the $^{13}$CO isotopologues toward both circumstellar disks and likely Keplerian rotation in the circumbinary structure; the latter suggests that it is probably a circumbinary disk., Comment: 33 pages, 29 figures, 6 tables

Generative object compositing emerges as a promising new avenue for compositional image editing. However, the requirement of object identity preservation poses a significant challenge, limiting practical usage of most existing methods. In response, this paper introduces IMPRINT, a novel diffusion-based generative model trained with a two-stage learning framework that decouples learning of identity preservation from that of compositing. The first stage is targeted for context-agnostic, identity-preserving pretraining of the object encoder, enabling the encoder to learn an embedding that is both view-invariant and conducive to enhanced detail preservation. The subsequent stage leverages this representation to learn seamless harmonization of the object composited to the background. In addition, IMPRINT incorporates a shape-guidance mechanism offering user-directed control over the compositing process. Extensive experiments demonstrate that IMPRINT significantly outperforms existing methods and various baselines on identity preservation and composition quality.

We investigate the crescent-shaped dust trap in the transition disk, Oph IRS 48, using well-resolved (sub)millimeter polarimetric observations at ALMA Band 7 (870 $\mu$m). The dust polarization map reveals patterns consistent with dust scattering-induced polarization. There is a relative displacement between the polarized flux and the total flux, which holds the key to understanding the dust scale heights in this system. We model the polarization observations, focusing on the effects of dust scale heights. We find that the interplay between the inclination-induced polarization and the polarization arising from radiation anisotropy in the crescent determines the observed polarization; the anisotropy is controlled by the dust optical depth along the midplane, which is, in turn, determined by the dust scale height in the vertical direction. We find that the dust grains can neither be completely settled nor well mixed with the gas. The completely settled case produces little radial displacement between the total and polarized flux, while the well-mixed case produces an azimuthal pattern in the outer (radial) edge of the crescent that is not observed. Our best model has a gas-to-dust scale height ratio of 2, and can reproduce both the radial displacement and the azimuthal displacement between the total and polarized flux. We infer an effective turbulence $\alpha$ parameter of approximately $0.0001-0.005$. The scattering-induced polarization provides insight into a turbulent vortex with a moderate level of dust settling in the IRS 48 system, which is hard to achieve otherwise., Comment: 9 pages in the main text, 5 figures, accepted by ApJ

Human image editing includes tasks like changing a person's pose, their clothing, or editing the image according to a text prompt. However, prior work often tackles these tasks separately, overlooking the benefit of mutual reinforcement from learning them jointly. In this paper, we propose UniHuman, a unified model that addresses multiple facets of human image editing in real-world settings. To enhance the model's generation quality and generalization capacity, we leverage guidance from human visual encoders and introduce a lightweight pose-warping module that can exploit different pose representations, accommodating unseen textures and patterns. Furthermore, to bridge the disparity between existing human editing benchmarks with real-world data, we curated 400K high-quality human image-text pairs for training and collected 2K human images for out-of-domain testing, both encompassing diverse clothing styles, backgrounds, and age groups. Experiments on both in-domain and out-of-domain test sets demonstrate that UniHuman outperforms task-specific models by a significant margin. In user studies, UniHuman is preferred by the users in an average of 77% of cases. Our project is available at https://github.com/NannanLi999/UniHuman., Comment: Accepted to CVPR 2024

The field of generative image inpainting and object insertion has made significant progress with the recent advent of latent diffusion models. Utilizing a precise object mask can greatly enhance these applications. However, due to the challenges users encounter in creating high-fidelity masks, there is a tendency for these methods to rely on more coarse masks (e.g., bounding box) for these applications. This results in limited control and compromised background content preservation. To overcome these limitations, we introduce SmartMask, which allows any novice user to create detailed masks for precise object insertion. Combined with a ControlNet-Inpaint model, our experiments demonstrate that SmartMask achieves superior object insertion quality, preserving the background content more effectively than previous methods. Notably, unlike prior works the proposed approach can also be used even without user-mask guidance, which allows it to perform mask-free object insertion at diverse positions and scales. Furthermore, we find that when used iteratively with a novel instruction-tuning based planning model, SmartMask can be used to design detailed layouts from scratch. As compared with user-scribble based layout design, we observe that SmartMask allows for better quality outputs with layout-to-image generation methods. Project page is available at https://smartmask-gen.github.io

Recovering textures under shadows has remained a challenging problem due to the difficulty of inferring shadow-free scenes from shadow images. In this paper, we propose the use of diffusion models as they offer a promising approach to gradually refine the details of shadow regions during the diffusion process. Our method improves this process by conditioning on a learned latent feature space that inherits the characteristics of shadow-free images, thus avoiding the limitation of conventional methods that condition on degraded images only. Additionally, we propose to alleviate potential local optima during training by fusing noise features with the diffusion network. We demonstrate the effectiveness of our approach which outperforms the previous best method by 13% in terms of RMSE on the AISTD dataset. Further, we explore instance-level shadow removal, where our model outperforms the previous best method by 82% in terms of RMSE on the DESOBA dataset., Comment: project page see https://kfmei.page/shadow-diffusion/index.html

Polarized (sub)millimeter emission from dust grains in circumstellar disks was initially thought to be due to grains aligned with the magnetic field. However, higher resolution multi-wavelength observations along with improved models found that this polarization is dominated by self-scattering at shorter wavelengths (e.g., 870 $\mu$m) and by grains aligned with something other than magnetic fields at longer wavelengths (e.g., 3 mm). Nevertheless, the polarization signal is expected to depend on the underlying substructure, and observations hitherto have been unable to resolve polarization in multiple rings and gaps. HL Tau, a protoplanetary disk located 147.3 $\pm$ 0.5 pc away, is the brightest Class I or Class II disk at millimeter/submillimeter wavelengths. Here we show deep, high-resolution 870 $\mu$m polarization observations of HL Tau, resolving polarization in both the rings and gaps. We find that the gaps have polarization angles with a significant azimuthal component and a higher polarization fraction than the rings. Our models show that the disk polarization is due to both scattering and emission from aligned effectively prolate grains. The intrinsic polarization of aligned dust grains is likely over 10%, which is much higher than what was expected in low resolution observations (~1%). Asymmetries and dust features are revealed in the polarization observations that are not seen in non-polarimetric observations., Comment: Published in Nature

We propose SegGen, a highly-effective training data generation method for image segmentation, which pushes the performance limits of state-of-the-art segmentation models to a significant extent. SegGen designs and integrates two data generation strategies: MaskSyn and ImgSyn. (i) MaskSyn synthesizes new mask-image pairs via our proposed text-to-mask generation model and mask-to-image generation model, greatly improving the diversity in segmentation masks for model supervision; (ii) ImgSyn synthesizes new images based on existing masks using the mask-to-image generation model, strongly improving image diversity for model inputs. On the highly competitive ADE20K and COCO benchmarks, our data generation method markedly improves the performance of state-of-the-art segmentation models in semantic segmentation, panoptic segmentation, and instance segmentation. Notably, in terms of the ADE20K mIoU, Mask2Former R50 is largely boosted from 47.2 to 49.9 (+2.7); Mask2Former Swin-L is also significantly increased from 56.1 to 57.4 (+1.3). These promising results strongly suggest the effectiveness of our SegGen even when abundant human-annotated training data is utilized. Moreover, training with our synthetic data makes the segmentation models more robust towards unseen domains. Project website: https://seggenerator.github.io

We present ALMA observations of the Class I source Oph IRS63 in the context of the Early Planet Formation in Embedded Disks (eDisk) large program. Our ALMA observations of Oph IRS63 show a myriad of protostellar features, such as a shell-like bipolar outflow (in $^{12}$CO), an extended rotating envelope structure (in $^{13}$CO), a streamer connecting the envelope to the disk (in C$^{18}$O), and several small-scale spiral structures seen towards the edge of the dust continuum (in SO). By analyzing the velocity pattern of $^{13}$CO and C$^{18}$O, we measure a protostellar mass of $\rm M_\star = 0.5 \pm 0.2 $~$\rm M_\odot$ and confirm the presence of a disk rotating at almost Keplerian velocity that extends up to $\sim260$ au. These calculations also show that the gaseous disk is about four times larger than the dust disk, which could indicate dust evolution and radial drift. Furthermore, we model the C$^{18}$O streamer and SO spiral structures as features originating from an infalling rotating structure that continuously feeds the young protostellar disk. We compute an envelope-to-disk mass infall rate of $\sim 10^{-6}$~$\rm M_\odot \, yr^{-1}$ and compare it to the disk-to-star mass accretion rate of $\sim 10^{-8}$~$\rm M_\odot \, yr^{-1}$, from which we infer that the protostellar disk is in a mass build-up phase. At the current mass infall rate, we speculate that soon the disk will become too massive to be gravitationally stable., Comment: 26 pages and 17 figures

We present the first results from the Early Planet Formation in Embedded Disks (eDisk) ALMA Large Program toward Oph IRS43, a binary system of solar mass protostars. The 1.3 mm dust continuum observations resolve a compact disk, ~6au radius, around the northern component and show that the disk around the southern component is even smaller, <~3 au. CO, 13CO, and C18O maps reveal a large cavity in a low mass envelope that shows kinematic signatures of rotation and infall extending out to ~ 2000au. An expanding CO bubble centered on the extrapolated location of the source ~130 years ago suggests a recent outburst. Despite the small size of the disks, the overall picture is of a remarkably large and dynamically active region., Comment: Paper 10 of the ALMA eDisk Large Program. Accepted for publication in ApJ

Protostellar disks are a ubiquitous part of the star formation process and the future sites of planet formation. As part of the Early Planet Formation in Embedded Disks (eDisk) large program, we present high-angular resolution dust continuum ($\sim40\,$mas) and molecular line ($\sim150\,$mas) observations of the Class 0 protostar, IRAS 15398-3359. The dust continuum is small, compact, and centrally peaked, while more extended dust structures are found in the outflow directions. We perform a 2D Gaussian fitting to find the deconvolved size and $2\sigma$ radius of the dust disk to be $4.5\times2.8\,\mathrm{au}$ and $3.8\,\mathrm{au}$, respectively. We estimate the gas+dust disk mass assuming optically thin continuum emission to be $0.6-1.8\,M_\mathrm{jup}$, indicating a very low-mass disk. The CO isotopologues trace components of the outflows and inner envelope, while SO traces a compact, rotating disk-like component. Using several rotation curve fittings on the PV diagram of the SO emission, the lower limits of the protostellar mass and gas disk radius are $0.022\,M_\odot$ and $31.2\,\mathrm{au}$ from our Modified 2 single power-law fitting. A conservative upper limit of the protostellar mass is inferred to be $0.1\,M_\odot$. The protostellar mass-accretion rate and the specific angular momentum at the protostellar disk edge are found to be between $1.3-6.1\times10^{-6}\,M_\odot\,\mathrm{yr^{-1}}$ and $1.2-3.8\times10^{-4}\,\mathrm{km\,s^{-1}\,pc}$, respectively, with an age estimated between $0.4-7.5\times10^{4}\,$yr. At this young age with no clear substructures in the disk, planet formation would likely not yet have started. This study highlights the importance of high-resolution observations and systematic fitting procedures when deriving dynamical properties of deeply embedded Class 0 protostars., Comment: 28 pages, 16 figures. Accepted for publication in ApJ as one of the first-look papers of the eDisk ALMA Large Program

Recent advancements in deep generative models have facilitated the creation of photo-realistic images across various tasks. However, these generated images often exhibit perceptual artifacts in specific regions, necessitating manual correction. In this study, we present a comprehensive empirical examination of Perceptual Artifacts Localization (PAL) spanning diverse image synthesis endeavors. We introduce a novel dataset comprising 10,168 generated images, each annotated with per-pixel perceptual artifact labels across ten synthesis tasks. A segmentation model, trained on our proposed dataset, effectively localizes artifacts across a range of tasks. Additionally, we illustrate its proficiency in adapting to previously unseen models using minimal training samples. We further propose an innovative zoom-in inpainting pipeline that seamlessly rectifies perceptual artifacts in the generated images. Through our experimental analyses, we elucidate several practical downstream applications, such as automated artifact rectification, non-referential image quality evaluation, and abnormal region detection in images. The dataset and code are released.

Polarization is a unique tool to study the properties of dust grains of protoplanetary disks and detail the initial conditions of planet formation. Polarization around HL Tau was previously imaged using the Atacama Large Millimeter/submillimeter Array (ALMA) at Bands 3 (3.1 mm), 6 (1.3 mm), and 7 (0.87 mm), showing that the polarization orientation changes across wavelength $\lambda$. The polarization morphology at Band 7 is predominantly parallel to the disk minor axis but appears azimuthally oriented at Band 3, with the morphology at Band 6 in between the two. We present new ~0.2" (29 au) polarization observations at Q-Band (7.0 mm) using the Karl G. Jansky Very Large Array (VLA) and at Bands 4 (2.1 mm), 5 (1.5 mm), and 7 using ALMA, consolidating HL Tau's position as the protoplanetary disk with the most complete wavelength coverage in dust polarization. The polarization patterns at Bands 4 and 5 continue to follow the morphological transition with wavelength previously identified in Bands 3, 6, and 7. Based on the azimuthal variation, we decompose the polarization into contributions from scattering ($s$) and thermal emission ($t$). We find that $s$ decreases slowly with increasing $\lambda$, and $t$ increases more rapidly with $\lambda$ which are expected from optical depth effects of toroidally aligned, scattering prolate grains. The relatively weak $\lambda$ dependence of $s$ is consistent with large, porous grains. The sparse polarization detections from the Q-band image are also consistent with toroidally aligned prolate grains., Comment: 22 pages, 13 figures, submitted to MNRAS

Precise estimates of protostellar masses are crucial to characterize the formation of stars of low masses down to brown-dwarfs (BDs; M* < 0.08 Msun). The most accurate estimation of protostellar mass uses the Keplerian rotation in the circumstellar disk around the protostar. To apply the Keplerian rotation method to a protostar at the low-mass end, we have observed the Class 0 protostar IRAS 16253-2429 using the Atacama Large Millimeter/submillimeter Array (ALMA) in the 1.3 mm continuum at an angular resolution of 0.07" (10 au), and in the 12CO, C18O, 13CO (J=2-1), and SO (J_N = 6_5-5_4) molecular lines, as part of the ALMA Large Program Early Planet Formation in Embedded Disks (eDisk). The continuum emission traces a non-axisymmetric, disk-like structure perpendicular to the associated 12CO outflow. The position-velocity (PV) diagrams in the C18O and 13CO lines can be interpreted as infalling and rotating motions. In contrast, the PV diagram along the major axis of the disk-like structure in the 12CO line allows us to identify Keplerian rotation. The central stellar mass and the disk radius are estimated to be ~0.12-0.17 Msun and ~13-19 au, respectively. The SO line suggests the existence of an accretion shock at a ring (r~28 au) surrounding the disk and a streamer from the eastern side of the envelope. IRAS 16253-2429 is not a proto-BD but has a central stellar mass close to the BD mass regime, and our results provide a typical picture of such very low-mass protostars., Comment: 41 pages, 14 figures

We present high-resolution, high-sensitivity observations of the Class 0 protostar RCrA IRS5N as part of the Atacama Large Milimeter/submilimeter Array (ALMA) large program Early Planet Formation in Embedded Disks (eDisk). The 1.3 mm continuum emission reveals a flattened continuum structure around IRS5N, consistent with a protostellar disk in the early phases of evolution. The continuum emission appears smooth and shows no substructures. However, a brightness asymmetry is observed along the minor axis of the disk, suggesting the disk is optically and geometrically thick. We estimate the disk mass to be between 0.007 and 0.02 M$_{\odot}$. Furthermore, molecular emission has been detected from various species, including C$^{18}$O (2$-$1), $^{12}$CO (2$-$1), $^{13}$CO (2$-$1), and H$_2$CO (3$_{0,3}-2_{0,2}$, 3$_{2,1}-2_{2,0}$, and 3$_{2,2}-2_{2,1}$). By conducting a position-velocity analysis of the C$^{18}$O (2$-$1) emission, we find that the disk of IRS5N exhibits characteristics consistent with Keplerian rotation around a central protostar with a mass of approximately 0.3 M$_{\odot}$. Additionally, we observe dust continuum emission from the nearby binary source, IRS5a/b. The emission in $^{12}$CO toward IRS5a/b seems to emanate from IRS5b and flow into IRS5a, suggesting material transport between their mutual orbits. The lack of a detected outflow and large-scale negatives in \tlvco~observed toward IRS5N suggests that much of the flux from IRS5N is being resolved out. Due to this substantial surrounding envelope, the central IRS5N protostar is expected to be significantly more massive in the future., Comment: 25 pages, 24 figures. Accepted for publication in ApJ

We propose Subject-Conditional Relation Detection SCoRD, where conditioned on an input subject, the goal is to predict all its relations to other objects in a scene along with their locations. Based on the Open Images dataset, we propose a challenging OIv6-SCoRD benchmark such that the training and testing splits have a distribution shift in terms of the occurrence statistics of $\langle$subject, relation, object$\rangle$ triplets. To solve this problem, we propose an auto-regressive model that given a subject, it predicts its relations, objects, and object locations by casting this output as a sequence of tokens. First, we show that previous scene-graph prediction methods fail to produce as exhaustive an enumeration of relation-object pairs when conditioned on a subject on this benchmark. Particularly, we obtain a recall@3 of 83.8% for our relation-object predictions compared to the 49.75% obtained by a recent scene graph detector. Then, we show improved generalization on both relation-object and object-box predictions by leveraging during training relation-object pairs obtained automatically from textual captions and for which no object-box annotations are available. Particularly, for $\langle$subject, relation, object$\rangle$ triplets for which no object locations are available during training, we are able to obtain a recall@3 of 33.80% for relation-object pairs and 26.75% for their box locations., Comment: WACV 2024

Along with the fast evolution of deep neural networks, the hardware system is also developing rapidly. As a promising solution achieving high scalability and low manufacturing cost, multi-accelerator systems widely exist in data centers, cloud platforms, and SoCs. Thus, a challenging problem arises in multi-accelerator systems: selecting a proper combination of accelerators from available designs and searching for efficient DNN mapping strategies. To this end, we propose MARS, a novel mapping framework that can perform computation-aware accelerator selection, and apply communication-aware sharding strategies to maximize parallelism. Experimental results show that MARS can achieve 32.2% latency reduction on average for typical DNN workloads compared to the baseline, and 59.4% latency reduction on heterogeneous models compared to the corresponding state-of-the-art method., Comment: Accepted by 60th DAC

We have observed the Class 0/I protostellar system Ced110 IRS4 at an angular resolution of $0.05''$ ($\sim$10 au) as a part of the ALMA large program; Early Planet Formation in the Embedded Disks (eDisk). The 1.3 mm dust continuum emission reveals that Ced110 IRS4 is a binary system with a projected separation of $\sim$250 au. The continuum emissions associated with the main source and its companion, named Ced110 IRS4A and IRS4B respectively, exhibit disk-like shapes and likely arise from dust disks around the protostars. The continuum emission of Ced110 IRS4A has a radius of $\sim$91.7 au ($\sim0.485''$), and shows bumps along its major axis with an asymmetry. The bumps can be interpreted as an shallow, ring-like structure at a radius of $\sim$40 au ($\sim0.2''$) in the continuum emission, as demonstrated from two-dimensional intensity distribution models. A rotation curve analysis on the C$^{18}$O and $^{13}$CO $J=2$-1 lines reveals the presence of a Keplerian disk within a radius of 120 au around Ced110 IRS4A, which supports the interpretation that the dust continuum emission arises from a disk. The ring-like structure in the dust continuum emission might indicate a possible, annular substructure in the surface density of the embedded disk, although the possibility that it is an apparent structure due to the optically thick continuum emission cannot be ruled out., Comment: 32 pages, 23 figures. Accepted for publication in ApJ as one of the first-look papers of the eDisk ALMA Large Program

We present observations of the Class 0 protostar IRAS 16544-1604 in CB 68 from the ''Early Planet Formation in Embedded Disks (eDisk)'' ALMA Large program. The ALMA observations target continuum and lines at 1.3-mm with an angular resolution of $\sim$5 au. The continuum image reveals a dusty protostellar disk with a radius of $\sim$30 au seen close to edge-on, and asymmetric structures both along the major and minor axes. While the asymmetry along the minor axis can be interpreted as the effect of the dust flaring, the asymmetry along the major axis comes from a real non-axisymmetric structure. The C$^{18}$O image cubes clearly show the gas in the disk that follows a Keplerian rotation pattern around a $\sim$0.14 $M_{\odot}$ central protostar. Furthermore, there are $\sim$1500 au-scale streamer-like features of gas connecting from North-East, North-North-West, and North-West to the disk, as well as the bending outflow as seen in the $^{12}$CO (2-1) emission. At the apparent landing point of NE streamer, there are SO (6$_5$-5$_4$) and SiO (5-4) emission detected. The spatial and velocity structure of NE streamer can be interpreted as a free-falling gas with a conserved specific angular momentum, and the detection of the SO and SiO emission at the tip of the streamer implies presence of accretion shocks. Our eDisk observations have unveiled that the Class 0 protostar in CB 68 has a Keplerian rotating disk with flaring and non-axisymmetric structure associated with accretion streamers and outflows., Comment: 30 pages, 24 figures, accepted for publication in The Astrophysical Journal as one of the first-look papers of the eDisk ALMA Large Program

While dust disks around optically visible, Class II protostars are found to be vertically thin, when and how dust settles to the midplane are unclear. As part of the Atacama Large Millimeter/submillimeter Array (ALMA) large program, Early Planet Formation in Embedded Disks, we analyze the edge-on, embedded, Class I protostar IRAS 04302+2247, also nicknamed the ``Butterfly Star." With a resolution of 0.05" (8~au), the 1.3 mm continuum shows an asymmetry along the minor axis which is evidence of an optically thick and geometrically thick disk viewed nearly edge-on. There is no evidence of rings and gaps, which could be due to the lack of radial substructure or the highly inclined and optically thick view. With 0.1" (16~au) resolution, we resolve the 2D snow surfaces, i.e., the boundary region between freeze-out and sublimation, for $^{12}$CO $J$=2--1, $^{13}$CO $J$=2--1, C$^{18}$O $J$=2--1, $H_{2}$CO $J$=$3_{0,3}$--$2_{0,2}$, and SO $J$=$6_{5}$--$5_{4}$, and constrain the CO midplane snow line to $\sim 130$ au. We find Keplerian rotation around a protostar of $1.6 \pm 0.4 M_{\odot}$ using C$^{18}$O. Through forward ray-tracing using RADMC-3D, we find that the dust scale height is $\sim 6$ au at a radius of 100~au from the central star and is comparable to the gas pressure scale height. The results suggest that the dust of this Class~I source has yet to vertically settle significantly., Comment: 33 pages, 21 figures. Accepted for publication in ApJ as one of the first-look papers of the eDisk ALMA Large Program

Constraining the physical and chemical structure of young embedded disks is crucial to understanding the earliest stages of planet formation. As part of the Early Planet Formation in Embedded Disks Atacama Large Millimeter/submillimeter Array Large Program, we present high spatial resolution ($\sim$0$.\!\!^{\prime\prime}$1 or $\sim$15 au) observations of the 1.3 mm continuum and $^{13}$CO $J=$ 2-1, C$^{18}$O $J=$ 2-1, and SO $J_N=$ $6_5$-$5_4$ molecular lines toward the disk around the Class I protostar L1489 IRS. The continuum emission shows a ring-like structure at 56 au from the central protostar and a tenuous, optically thin emission extending beyond $\sim$300 au. The $^{13}$CO emission traces the warm disk surface, while the C$^{18}$O emission originates from near the disk midplane. The coincidence of the radial emission peak of C$^{18}$O with the dust ring may indicate a gap-ring structure in the gaseous disk as well. The SO emission shows a highly complex distribution, including a compact, prominent component at $\lesssim$30 au, which is likely to originate from thermally sublimated SO molecules. The compact SO emission also shows a velocity gradient along a slightly ($\sim15^\circ$) tilted direction with respect to the major axis of the dust disk, which we interpret as an inner warped disk in addition to the warp around $\sim$200 au suggested by previous work. These warped structures may be formed by a planet or companion with an inclined orbit, or by a gradual change in the angular momentum axis during gas infall., Comment: 24 pages, 12 figures. Accepted for publication in The Astrophysical Journal as one of the first-look papers of the eDisk ALMA Large Program

Studying the physical and chemical conditions of young embedded disks is crucial to constrain the initial conditions for planet formation. Here, we present Atacama Large Millimeter/submillimeter Array (ALMA) observations of dust continuum at $\sim$0.06" (8 au) resolution and molecular line emission at $\sim$0.17" (24 au) resolution toward the Class 0 protostar L1527 IRS from the Large Program eDisk (Early Planet Formation in Embedded Disks). The continuum emission is smooth without substructures, but asymmetric along both the major and minor axes of the disk as previously observed. The detected lines of $^{12}$CO, $^{13}$CO, C$^{18}$O, H$_2$CO, c-C$_3$H$_2$, SO, SiO, and DCN trace different components of the protostellar system, with a disk wind potentially visible in $^{12}$CO. The $^{13}$CO brightness temperature and the H$_2$CO line ratio confirm that the disk is too warm for CO freeze out, with the snowline located at $\sim$350 au in the envelope. Both molecules show potential evidence of a temperature increase around the disk-envelope interface. SO seems to originate predominantly in UV-irradiated regions such as the disk surface and the outflow cavity walls rather than at the disk-envelope interface as previously suggested. Finally, the continuum asymmetry along the minor axis is consistent with the inclination derived from the large-scale (100" or 14,000 au) outflow, but opposite to that based on the molecular jet and envelope emission, suggesting a misalignment in the system. Overall, these results highlight the importance of observing multiple molecular species in multiple transitions to characterize the physical and chemical environment of young disks., Comment: 27 pages, 16 figures, 2 tables, 10 pages appendix with 12 figures. Accepted for publication in ApJ as one of the first-look papers of the eDisk ALMA Large Program

We present an overview of the Large Program, ``Early Planet Formation in Embedded Disks (eDisk)'', conducted with the Atacama Large Millimeter/submillimeter Array (ALMA). The ubiquitous detections of substructures, particularly rings and gaps, in protoplanetary disks around T Tauri stars raise the possibility that at least some planet formation may have already started during the embedded stages of star formation. In order to address exactly how and when planet formation is initiated, the program focuses on searching for substructures in disks around 12 Class 0 and 7 Class I protostars in nearby ($< $200 pc) star-forming regions through 1.3 mm continuum observations at a resolution of $\sim7$ au (0.04"). The initial results show that the continuum emission, mostly arising from dust disks around the sample protostars, has relatively few distinctive substructures, such as rings and spirals, in marked contrast to Class II disks. The dramatic difference may suggest that substructures quickly develop in disks when the systems evolve from protostars to Class II sources or alternatively that high optical depth of the continuum emission could obscure internal structures. Kinematic information obtained through CO isotopologue lines and other lines reveals the presence of Keplerian disks around protostars, providing us with crucial physical parameters, in particular, the dynamical mass of the central protostars. We describe the background of the eDisk program, the sample selection and their ALMA observations, the data reduction, and also highlight representative first-look results., Comment: This is a publication of a series of eDisk ALMA large program first-look papers

Despite the progress of image segmentation for accurate visual entity segmentation, completing the diverse requirements of image editing applications for different-level region-of-interest selections remains unsolved. In this paper, we propose a new task, All-Inclusive Multi-Level Segmentation (AIMS), which segments visual regions into three levels: part, entity, and relation (two entities with some semantic relationships). We also build a unified AIMS model through multi-dataset multi-task training to address the two major challenges of annotation inconsistency and task correlation. Specifically, we propose task complementarity, association, and prompt mask encoder for three-level predictions. Extensive experiments demonstrate the effectiveness and generalization capacity of our method compared to other state-of-the-art methods on a single dataset or the concurrent work on segmenting anything. We will make our code and training model publicly available., Comment: Technical Report

In photo editing, it is common practice to remove visual distractions to improve the overall image quality and highlight the primary subject. However, manually selecting and removing these small and dense distracting regions can be a laborious and time-consuming task. In this paper, we propose an interactive distractor selection method that is optimized to achieve the task with just a single click. Our method surpasses the precision and recall achieved by the traditional method of running panoptic segmentation and then selecting the segments containing the clicks. We also showcase how a transformer-based module can be used to identify more distracting regions similar to the user's click position. Our experiments demonstrate that the model can effectively and accurately segment unknown distracting objects interactively and in groups. By significantly simplifying the photo cleaning and retouching process, our proposed model provides inspiration for exploring rare object segmentation and group selection with a single click., Comment: CVPR 2023. Project link: https://simpson-cvpr23.github.io

We present XFormer, a novel human mesh and motion capture method that achieves real-time performance on consumer CPUs given only monocular images as input. The proposed network architecture contains two branches: a keypoint branch that estimates 3D human mesh vertices given 2D keypoints, and an image branch that makes predictions directly from the RGB image features. At the core of our method is a cross-modal transformer block that allows information to flow across these two branches by modeling the attention between 2D keypoint coordinates and image spatial features. Our architecture is smartly designed, which enables us to train on various types of datasets including images with 2D/3D annotations, images with 3D pseudo labels, and motion capture datasets that do not have associated images. This effectively improves the accuracy and generalization ability of our system. Built on a lightweight backbone (MobileNetV3), our method runs blazing fast (over 30fps on a single CPU core) and still yields competitive accuracy. Furthermore, with an HRNet backbone, XFormer delivers state-of-the-art performance on Huamn3.6 and 3DPW datasets.

Diffusion-based models have achieved state-of-the-art performance on text-to-image synthesis tasks. However, one critical limitation of these models is the low fidelity of generated images with respect to the text description, such as missing objects, mismatched attributes, and mislocated objects. One key reason for such inconsistencies is the inaccurate cross-attention to text in both the spatial dimension, which controls at what pixel region an object should appear, and the temporal dimension, which controls how different levels of details are added through the denoising steps. In this paper, we propose a new text-to-image algorithm that adds explicit control over spatial-temporal cross-attention in diffusion models. We first utilize a layout predictor to predict the pixel regions for objects mentioned in the text. We then impose spatial attention control by combining the attention over the entire text description and that over the local description of the particular object in the corresponding pixel region of that object. The temporal attention control is further added by allowing the combination weights to change at each denoising step, and the combination weights are optimized to ensure high fidelity between the image and the text. Experiments show that our method generates images with higher fidelity compared to diffusion-model-based baselines without fine-tuning the diffusion model. Our code is publicly available at https://github.com/UCSB-NLP-Chang/Diffusion-SpaceTime-Attn., Comment: 20 pages, 16 figures

Recent advances in personalized image generation allow a pre-trained text-to-image model to learn a new concept from a set of images. However, existing personalization approaches usually require heavy test-time finetuning for each concept, which is time-consuming and difficult to scale. We propose InstantBooth, a novel approach built upon pre-trained text-to-image models that enables instant text-guided image personalization without any test-time finetuning. We achieve this with several major components. First, we learn the general concept of the input images by converting them to a textual token with a learnable image encoder. Second, to keep the fine details of the identity, we learn rich visual feature representation by introducing a few adapter layers to the pre-trained model. We train our components only on text-image pairs without using paired images of the same concept. Compared to test-time finetuning-based methods like DreamBooth and Textual-Inversion, our model can generate competitive results on unseen concepts concerning language-image alignment, image fidelity, and identity preservation while being 100 times faster., Comment: 13 pages