Search

Subarachnoid hemorrhage (SAH), typically due to intracranial aneurysms, demands precise imaging for effective treatment. Digital Subtraction Angiography (DSA), despite being the gold standard, broadly visualizes cerebral blood flow, potentially masking key details in areas. This study introduces an approach integrating a 3D vascular atlas with 2D DSA images to allow targeted quantitative analysis in these crucial regions, thus enhancing diagnostic accuracy during interventions. Initially, DSA data was examined to ascertain the injection site. Following this, the appropriate viewing angle was determined to align accurately with the 3D vascular atlas. Utilizing this atlas, regions corresponding to the areas indicated as perfused were selected. Concurrently, a mask representing the perfused areas was created from the DSA sequence. This mask facilitated the initial coarse alignment of the projected 3D atlas to the DSA perfused territory deformable registration techniques, ensuring a precise overlay with the DSAs perfused territories. The performance of each overlay was measured using the Structural Similarity Index Measure (SSIM). The coregistration process revealed that deformable registrations was essential to achieve precise overlays of the 3D atlas projections with the 2D DSA perfused areas. This approach enabled the extraction of targeted quantitative angiography parameters, essential for detailed vascular assessment in subarachnoid hemorrhage cases. The integration of 3D atlas registration with 2D DSA projections facilitates a more precise and targeted diagnostic process for SAH during critical interventions. This image processing strategy enhances the visualization of affected arterial territories, potentially improving the accuracy of diagnostics and supporting better informed clinical decisions at the time of intervention, Comment: 5 pages, 2 figures, to be published in SPIE Medical Imaging 2025 conference proceeds

Modern deep policy gradient methods achieve effective performance on simulated robotic tasks, but they all require large replay buffers or expensive batch updates, or both, making them incompatible for real systems with resource-limited computers. We show that these methods fail catastrophically when limited to small replay buffers or during incremental learning, where updates only use the most recent sample without batch updates or a replay buffer. We propose a novel incremental deep policy gradient method -- Action Value Gradient (AVG) and a set of normalization and scaling techniques to address the challenges of instability in incremental learning. On robotic simulation benchmarks, we show that AVG is the only incremental method that learns effectively, often achieving final performance comparable to batch policy gradient methods. This advancement enabled us to show for the first time effective deep reinforcement learning with real robots using only incremental updates, employing a robotic manipulator and a mobile robot., Comment: In The Thirty-eighth Annual Conference on Neural Information Processing Systems. Source code at https://github.com/gauthamvasan/avg and companion video at https://youtu.be/cwwuN6Hyew0

We propose a diffusion model-based approach, FloAtControlNet to generate cinemagraphs composed of animations of human clothing. We focus on human clothing like dresses, skirts and pants. The input to our model is a text prompt depicting the type of clothing and the texture of clothing like leopard, striped, or plain, and a sequence of normal maps that capture the underlying animation that we desire in the output. The backbone of our method is a normal-map conditioned ControlNet which is operated in a training-free regime. The key observation is that the underlying animation is embedded in the flow of the normal maps. We utilize the flow thus obtained to manipulate the self-attention maps of appropriate layers. Specifically, the self-attention maps of a particular layer and frame are recomputed as a linear combination of itself and the self-attention maps of the same layer and the previous frame, warped by the flow on the normal maps of the two frames. We show that manipulating the self-attention maps greatly enhances the quality of the clothing animation, making it look more natural as well as suppressing the background artifacts. Through extensive experiments, we show that the method proposed beats all baselines both qualitatively in terms of visual results and user study. Specifically, our method is able to alleviate the background flickering that exists in other diffusion model-based baselines that we consider. In addition, we show that our method beats all baselines in terms of RMSE and PSNR computed using the input normal map sequences and the normal map sequences obtained from the output RGB frames. Further, we show that well-established evaluation metrics like LPIPS, SSIM, and CLIP scores that are generally for visual quality are not necessarily suitable for capturing the subtle motions in human clothing animations.

Graphic designs are an effective medium for visual communication. They range from greeting cards to corporate flyers and beyond. Off-late, machine learning techniques are able to generate such designs, which accelerates the rate of content production. An automated way of evaluating their quality becomes critical. Towards this end, we introduce Design-o-meter, a data-driven methodology to quantify the goodness of graphic designs. Further, our approach can suggest modifications to these designs to improve its visual appeal. To the best of our knowledge, Design-o-meter is the first approach that scores and refines designs in a unified framework despite the inherent subjectivity and ambiguity of the setting. Our exhaustive quantitative and qualitative analysis of our approach against baselines adapted for the task (including recent Multimodal LLM-based approaches) brings out the efficacy of our methodology. We hope our work will usher more interest in this important and pragmatic problem setting., Comment: Accepted to WACV 2025. Project page: https://sahilg06.github.io/Design-o-meter/

This study evaluates a multimodal machine learning framework for predicting treatment outcomes in intracranial aneurysms (IAs). Combining angiographic parametric imaging (API), patient biomarkers, and disease morphology, the framework aims to enhance prognostic accuracy. Data from 340 patients were analyzed, with separate deep neural networks processing quantitative and categorical data. These networks' pre decision layers were concatenated and inputted into a final predictive network. Various data augmentation strategies, including Synthetic Minority Oversampling Technique for Nominal and Continuous data (SMOTE NC), addressed dataset imbalances. Performance metrics, evaluated through Monte Carlo cross validation, showed significant improvements with augmentation, particularly in intermediate fusion models. This study validates the framework's efficacy in accurately predicting IA treatment outcomes, demonstrating that data augmentation techniques can substantially enhance model performance., Comment: 6 pages

Here, we present the outcomes from the second Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry, which engaged participants across global hybrid locations, resulting in 34 team submissions. The submissions spanned seven key application areas and demonstrated the diverse utility of LLMs for applications in (1) molecular and material property prediction; (2) molecular and material design; (3) automation and novel interfaces; (4) scientific communication and education; (5) research data management and automation; (6) hypothesis generation and evaluation; and (7) knowledge extraction and reasoning from scientific literature. Each team submission is presented in a summary table with links to the code and as brief papers in the appendix. Beyond team results, we discuss the hackathon event and its hybrid format, which included physical hubs in Toronto, Montreal, San Francisco, Berlin, Lausanne, and Tokyo, alongside a global online hub to enable local and virtual collaboration. Overall, the event highlighted significant improvements in LLM capabilities since the previous year's hackathon, suggesting continued expansion of LLMs for applications in materials science and chemistry research. These outcomes demonstrate the dual utility of LLMs as both multipurpose models for diverse machine learning tasks and platforms for rapid prototyping custom applications in scientific research., Comment: 98 pages

In intracranial aneurysm (IA) treatment, digital subtraction angiography (DSA) monitors device-induced hemodynamic changes. Quantitative angiography (QA) provides more precise assessments but is limited by hand-injection variability. This study evaluates correction methods using in vitro phantoms that mimic diverse aneurysm morphologies and locations, addressing the 2D and temporal limitations of DSA. We used a patient-specific phantom to replicate three distinct IA morphologies at various Circle of Willis points: the middle cerebral artery (MCA), anterior communicating artery (ACA), and the internal carotid artery (ICA), each varying in size and shape. The diameters of the IA at MCA, ACA and ICA are 10.1, 10 and 7 millimeters, respectively. QA parameters for both non-stenosed and stenosed conditions were measured with 5ml and 10ml boluses over various injection durations to generate time density curves (TDCs). To address the variability in injection, several singular value decomposition (SVD) variants, standard SVD (sSVD) with Tikhonov regularization, block-circulant SVD (bSVD), and oscillation index SVD (oSVD) were applied. These methods enabled the extraction of IA impulse response function (IRF), peak height (PHIRF), area under the curve (AUCIRF), and mean transit time (MTT). We evaluated the robustness of bias-reducing methods by observing the invariance of these parameters with respect to the injection conditions, and the location and size of the aneurysm. The application of SVD variants, sSVD, bSVD, and oSVD, significantly reduced QA parameter variability due to injection techniques., Comment: arXiv admin note: text overlap with arXiv:2411.03655

We consider the problem of conditional text-to-image synthesis with diffusion models. Most recent works need to either finetune specific parts of the base diffusion model or introduce new trainable parameters, leading to deployment inflexibility due to the need for training. To address this gap in the current literature, we propose our method called TINTIN: Test-time Conditional Text-to-Image Synthesis using Diffusion Models which is a new training-free test-time only algorithm to condition text-to-image diffusion model outputs on conditioning factors such as color palettes and edge maps. In particular, we propose to interpret noise predictions during denoising as gradients of an energy-based model, leading to a flexible approach to manipulate the noise by matching predictions inferred from them to the ground truth conditioning input. This results in, to the best of our knowledge, the first approach to control model outputs with input color palettes, which we realize using a novel color distribution matching loss. We also show this test-time noise manipulation can be easily extensible to other types of conditioning, e.g., edge maps. We conduct extensive experiments using a variety of text prompts, color palettes, and edge maps and demonstrate significant improvement over the current state-of-the-art, both qualitatively and quantitatively.

This study leverages convolutional neural networks to enhance the temporal resolution of 3D angiography in intracranial aneurysms focusing on the reconstruction of volumetric contrast data from sparse and limited projections. Three patient-specific IA geometries were segmented and converted into stereolithography files to facilitate computational fluid dynamics simulations. These simulations first modeled blood flow under steady conditions with varying inlet velocities: 0.25 m/s, 0.35 m/s, and 0.45 m/s. Subsequently, 3D angiograms were simulated by labeling inlet particles to represent contrast bolus injections over durations of 0.5s, 1.0s, 1.5s, and 2.0s. The angiographic simulations were then used within a simulated cone beam C arm CT system to generate in-silico rotational DSAs, capturing projections every 10 ms over a 220-degree arc at 27 frames per second. From these simulations, both fully sampled (108 projections) and truncated projection datasets were generated the latter using a maximum of 49 projections. High fidelity volumetric images were reconstructed using a Parker weighted Feldkamp Davis Kress algorithm. A modified U Net CNN was subsequently trained on these datasets to reconstruct 3D angiographic volumes from the truncated projections. The network incorporated multiple convolutional layers with ReLU activations and Max pooling, complemented by upsampling and concatenation to preserve spatial detail. Model performance was evaluated using mean squared error (MSE). Evaluating our U net model across the test set yielded a MSE of 0.0001, indicating good agreement with ground truth reconstructions and demonstrating acceptable capabilities in capturing relevant transient angiographic features. This study confirms the feasibility of using CNNs for reconstructing 3D angiographic images from truncated projections.

This study aims to mitigate these biases and enhance QA analysis by applying a path-length correction (PLC) correction, followed by singular value decomposition (SVD)-based deconvolution, to angiograms obtained through both in-silico and in-vitro methods. We utilized DSA data from in-silico and in-vitro patient-specific intracranial aneurysm models. To remove projection bias, PLC for various views were developed by co-registering the pre-existing 3D vascular geometry mask with the DSA projections, followed by ray tracing to determine paths across 3D vessel structures. These maps were used to normalize the logarithmic angiographic images, correcting for projection-induced foreshortening across different angles. Subsequently, we focused on eliminating injection bias by analyzing the corrected angiograms under varied projection views, injection rates, and flow conditions. Regions of interest at the aneurysm dome and inlet were placed to extract Time Density Curves for the lesion and the arterial input function, respectively. Using three standard SVD methodologies, we extracted the aneurysm Impulse Response function (IRF) and its associated parameters Peak Height (PHIRF), Area Under the Curve (AUCIRF), and Mean Transit Time (MTT). Our methodology employing PLC and SVD-based deconvolution ensures reliable quantitative angiographic measurements across varying conditions, supporting consistent assessments of disease severity and treatment efficacy. This approach significantly enhances intrapatient and intraprocedural reliability in neurovascular diagnostics., Comment: 11 pages, 6figures

We consider quantum networks, where entangled photon pairs are distributed using fibre optic links from a centralized source to entangling nodes. The entanglement is then stored (via an entanglement swap) in entangling nodes' quantum memories until used in, e.g., distributed quantum computing, quantum key distribution, quantum sensing, and other applications. Due to the fibre loss, some photons are lost in transmission. Noise in the transmission link and the quantum memory also reduces fidelity. Thus, entangling nodes must keep updated records of photon-pair arrivals to each destination, and their use by the applications. This coordination requires classical information exchange between each entangled node pair. However, the same fibre link may not admit both classical and quantum transmissions, as the classical channels can generate enough noise (i.e., via spontaneous Raman scattering) to make the quantum link unusable. Here, we consider coordinating entanglement distribution using a standard Internet protocol (IP) network instead, and propose a control protocol to enable such. We analyse the increase in latency from transmission over an IP network, together with the effect of photon loss, quantum memory noise and buffer size, to determine the fidelity and rate of entangled pairs. We characterize the relationship between the latency of the non-ideal IP network and the decoherence time of the quantum memories, providing a comparison of promising quantum memory technologies.

Distributing entanglement over long distances remains a challenge due to its fragility when exposed to environmental effects. In this work, we compare various entanglement distribution protocols in a realistic noisy fiber network. We focus specifically on two schemes that only require the sending of a non-entangled carrier photon to remote nodes of the network. These protocols rely on optical CNOT gates and we vary the probability with which they can be successfully performed. Encoding our entangled states in photon polarization, we analyse the effect of depolarizing noise on the photonic states as the carrier passes through the fibers. Building a robust model of photon loss and calculating the distillable entanglement of the noisy states, we find the entanglement distribution rate. We discover that methods involving a separable carrier can reach a higher rate than the standard entanglement distribution protocol, provided that the success probability of the optical CNOT gates is sufficiently high., Comment: 12 pages, 10 figures. Comments welcome

Intraoperative 2D quantitative angiography (QA) for intracranial aneurysms (IAs) has accuracy challenges due to the variability of hand injections. Despite the success of singular value decomposition (SVD) algorithms in reducing biases in computed tomography perfusion (CTP), their application in 2D QA has not been extensively explored. This study seeks to bridge this gap by investigating the potential of SVD-based deconvolution methods in 2D QA, particularly in addressing the variability of injection durations. The study included three internal carotid aneurysm (ICA) cases. Virtual angiograms were generated using Computational Fluid Dynamics (CFD) for three physiologically relevant inlet velocities to simulate contrast media injection durations. Time-density curves (TDCs) were produced for both the inlet and aneurysm dome. Various SVD variants, including standard SVD (sSVD) with and without classical Tikhonov regularization, block-circulant SVD (bSVD), and oscillation index SVD (oSVD), were applied to virtual angiograms. The method was applied on virtual angiograms to recover the aneurysmal dome impulse response function (IRF) and extract flow related parameters such as Peak Height PHIRF, Area Under the Curve AUCIRF, and Mean transit time MTT. Furthermore, we found that SVD can effectively reduce QA parameter variability across various injection durations, enhancing the potential of QA analysis parameters in neurovascular disease diagnosis and treatment. Implementing SVD-based deconvolution techniques in QA analysis can enhance the precision and reliability of neurovascular diagnostics by effectively reducing the impact of injection duration on hemodynamic parameters.

Natural intelligence processes experience as a continuous stream, sensing, acting, and learning moment-by-moment in real time. Streaming learning, the modus operandi of classic reinforcement learning (RL) algorithms like Q-learning and TD, mimics natural learning by using the most recent sample without storing it. This approach is also ideal for resource-constrained, communication-limited, and privacy-sensitive applications. However, in deep RL, learners almost always use batch updates and replay buffers, making them computationally expensive and incompatible with streaming learning. Although the prevalence of batch deep RL is often attributed to its sample efficiency, a more critical reason for the absence of streaming deep RL is its frequent instability and failure to learn, which we refer to as stream barrier. This paper introduces the stream-x algorithms, the first class of deep RL algorithms to overcome stream barrier for both prediction and control and match sample efficiency of batch RL. Through experiments in Mujoco Gym, DM Control Suite, and Atari Games, we demonstrate stream barrier in existing algorithms and successful stable learning with our stream-x algorithms: stream Q, stream AC, and stream TD, achieving the best model-free performance in DM Control Dog environments. A set of common techniques underlies the stream-x algorithms, enabling their success with a single set of hyperparameters and allowing for easy extension to other algorithms, thereby reviving streaming RL.

Protein-ligand binding is the process by which a small molecule (drug or inhibitor) attaches to a target protein. The binding affinity, which refers to the strength of this interaction, is central to many important problems in bioinformatics such as drug design. An extensive amount of work has been devoted to predicting binding affinity over the past decades due to its significance. In this paper, we review all significant recent works, focusing on the methods, features, and benchmark datasets. We have observed a rising trend in the use of traditional machine learning and deep learning models for predicting binding affinity, accompanied by an increasing amount of data on proteins and small drug-like molecules. While prediction results are constantly improving, we also identify several open questions and potential directions that remain unexplored in the field. This paper could serve as an excellent starting point for machine learning researchers who wish to engage in the study of binding affinity, or for anyone with general interests in machine learning, drug discovery, and bioinformatics.

Accurately attributing answer text to its source document is crucial for developing a reliable question-answering system. However, attribution for long documents remains largely unexplored. Post-hoc attribution systems are designed to map answer text back to the source document, yet the granularity of this mapping has not been addressed. Furthermore, a critical question arises: What exactly should be attributed? This involves identifying the specific information units within an answer that require grounding. In this paper, we propose and investigate a novel approach to the factual decomposition of generated answers for attribution, employing template-based in-context learning. To accomplish this, we utilize the question and integrate negative sampling during few-shot in-context learning for decomposition. This approach enhances the semantic understanding of both abstractive and extractive answers. We examine the impact of answer decomposition by providing a thorough examination of various attribution approaches, ranging from retrieval-based techniques to LLM-based attributors., Comment: EMNLP 2024

We present ImPoster, a novel algorithm for generating a target image of a 'source' subject performing a 'driving' action. The inputs to our algorithm are a single pair of a source image with the subject that we wish to edit and a driving image with a subject of an arbitrary class performing the driving action, along with the text descriptions of the two images. Our approach is completely unsupervised and does not require any access to additional annotations like keypoints or pose. Our approach builds on a pretrained text-to-image latent diffusion model and learns the characteristics of the source and the driving image by finetuning the diffusion model for a small number of iterations. At inference time, ImPoster performs step-wise text prompting i.e. it denoises by first moving in the direction of the image manifold corresponding to the driving image followed by the direction of the image manifold corresponding to the text description of the desired target image. We propose a novel diffusion guidance formulation, image frequency guidance, to steer the generation towards the manifold of the source subject and the driving action at every step of the inference denoising. Our frequency guidance formulations are derived from the frequency domain properties of images. We extensively evaluate ImPoster on a diverse set of source-driving image pairs to demonstrate improvements over baselines. To the best of our knowledge, ImPoster is the first approach towards achieving both subject-driven as well as action-driven image personalization. Code and data is available at https://github.com/divyakraman/ImPosterDiffusion2024.

The Hellinger distance between quantum states is a significant measure in quantum information theory, known for its Riemannian and monotonic properties. It is also easier to compute than the Bures distance, another measure that shares these properties. In this work, we derive the mean and variance of the Hellinger distance between pairs of density matrices, where one or both matrices are random. Along the way, we also obtain exact results for the mean affinity and mean square affinity. The first two cumulants of the Hellinger distance allow us to propose an approximation for the corresponding probability density function based on the gamma distribution. Our analytical results are corroborated through Monte Carlo simulations, showing excellent agreement., Comment: 8 pages, 5 figures

We consider the problem of independently, in a disentangled fashion, controlling the outputs of text-to-image diffusion models with color and style attributes of a user-supplied reference image. We present the first training-free, test-time-only method to disentangle and condition text-to-image models on color and style attributes from reference image. To realize this, we propose two key innovations. Our first contribution is to transform the latent codes at inference time using feature transformations that make the covariance matrix of current generation follow that of the reference image, helping meaningfully transfer color. Next, we observe that there exists a natural disentanglement between color and style in the LAB image space, which we exploit to transform the self-attention feature maps of the image being generated with respect to those of the reference computed from its L channel. Both these operations happen purely at test time and can be done independently or merged. This results in a flexible method where color and style information can come from the same reference image or two different sources, and a new generation can seamlessly fuse them in either scenario., Comment: 16 pages, 17 figures

The baryon cycle is crucial for understanding galaxy formation, as gas inflows and outflows vary throughout a galaxy's lifetime and affect its star formation rate. Despite the necessity of accretion for galaxy growth at high redshifts, direct observations of inflowing gas have proven elusive especially at $z\gtrsim2$. We present spectroscopic analysis of a galaxy at redshift $z=2.45$ which exhibits signs of inflow in several ultraviolet interstellar absorption lines, with no clear outflow signatures. The absorption lines are redshifted by $\sim$250 km sec$^{-1}$ with respect to the systemic redshift, and C IV shows a prominent inverse P-Cygni profile. Simple stellar population models suggest that this galaxy has a low metallicity ($\sim$5% solar), with a very young starburst of age $\sim$4 Myr dominating the ultraviolet luminosity. The gas inflow velocity and nebular velocity dispersion suggest an approximate halo mass of order $\sim 10^{11}M_{\odot}$, a regime in which simulations predict that bursty star formation is common at this redshift. We conclude that this system is likely in the beginning of a cycle of bursty star formation, where inflow and star formation rates are high, but where supernovae and other feedback processes have not yet launched strong outflows. In this scenario, we expect the inflow-dominated phase to be observable (e.g., with net redshifted ISM absorption) for only a short timescale after a starburst onset. This result represents a promising avenue for probing the full baryon cycle, including inflows, during the formative phases of low-mass galaxies at high redshifts., Comment: Accepted for publication in ApJL, 12 pages, 4 figures

Over the past few years alone, the lensing community has discovered thousands of strong lens candidates, and spectroscopically confirmed hundreds of them. In this time of abundance, it becomes pragmatic to focus our time and resources on the few extraordinary systems, in order to most efficiently study the universe. In this paper, we present such a system: DESI-090.9854-35.9683, a cluster-scale lens at $z_{\rm l} = 0.49$, with seven observed lensed sources around the core, and additional lensed sources further out in the cluster. From the number and the textbook configuration of the lensed images, a tight constraint on the mass potential of the lens is possible. This would allow for detailed analysis on the dark and luminous matter content within galaxy clusters, as well as a probe into dark energy and high-redshift galaxies. We present our spatially resolved kinematic measurements of this system from the Very Large Telescope Multi Unit Spectroscopic Explorer, which confirm five of these source galaxies (in ascending order, at $z_{\rm s} = 0.962, 0.962, 1.166, 1.432,$ and $1.432$). With previous Hubble Space Telescope imaging in the F140W and F200LP bands, we also present a simple two power-law profile flux-based lens model that, for a cluster lens, well models the five lensed arc families with redshifts. We determine the mass to be $M(< \theta_{\rm E}) = 4.78\times10^{13} M_{\odot}$ for the primary mass potential. From the model, we extrapolate the redshift of one of the two source galaxies not yet spectroscopically confirmed to be at $z_{\rm s}=4.52^{+1.03}_{-0.71}$., Comment: 14 pages, 6 figures

While quiescent galaxies have comparable amounts of cool gas in their outer circumgalactic medium (CGM) compared to star-forming galaxies, they have significantly less interstellar gas. However, open questions remain on the processes causing galaxies to stop forming stars and stay quiescent . Theories suggest dynamical interactions with the hot corona prevent cool gas from reaching the galaxy, therefore predicting the inner regions of quiescent galaxy CGMs are devoid of cool gas. However, there is a lack of understanding of the inner regions of CGMs due to the lack of spatial information in quasar-sightline methods. We present integral-field spectroscopy probing 10--20~kpc (2.4--4.8 R\textsubscript{e}) around a massive quiescent galaxy using a gravitationally lensed star-forming galaxy. We detect absorption from Magnesium (MgII) implying large amounts of cool atomic gas (10\textsuperscript{8.4} -- 10\textsuperscript{9.3} M\textsubscript{$\odot$} with T$\sim$10\textsuperscript{4} Kelvin), in comparable amounts to star-forming galaxies. Lens modeling of Hubble imaging also reveals a diffuse asymmetric component of significant mass consistent with the spatial extent of the MgII absorption, and offset from the galaxy light profile. This study demonstrates the power of galaxy-scale gravitational lenses to not only probe the gas around galaxies, but to also independently probe the mass of the CGM due to it's gravitational effect., Comment: accepted nature communications physics

Galaxy protoclusters are sites of rapid growth, with a high density of massive galaxies driving elevated rates of star formation and accretion onto supermassive black holes. Here, we present new JWST/NIRSpec IFU observations of the J1000+0234 group at $z=4.54$, a dense region of a protocluster hosting a massive, dusty star forming galaxy (DSFG). The new data reveal two extended, high-equivalent-width (EW$_0>1000\r{A}$) [O III] nebulae that appear at both sides of the DSFG along its minor axis (namely O3-N and O3-S). On one hand, the spectrum of O3-N shows a broad and blueshifted component with a full width at half maximum (FWHM) of 1300 km/s, suggesting an outflow origin. On the other hand, O3-S stretches over 8.6 kpc, and has a velocity gradient that spans 800 km/s, but shows no evidence of a broad component. However, both sources seem to be powered by an active galactic nucleus (AGN), so we classified them as extended emission-line regions (EELRs). The strongest evidence comes from the detection of the high-ionization [Ne V] $\lambda 3427$ line toward O3-N, which paired with the lack of hard X-rays implies an obscuring column density above the Compton-thick regime. The [Ne V] line is not detected in O3-S, but we measure a He II $\lambda 4687$/H$\beta$=0.25, which is well above the expectation for star formation. Despite the remarkable alignment of O3-N and O3-S with two radio sources, we do not find evidence of shocks from a radio jet that could be powering the EELRs. We interpret this as O3-S being externally irradiated by the AGN, akin to the famous Hanny's Voorwerp object in the local Universe. In addition, classical line ratio diagnostics (e.g., [O III]/H$\beta$ vs [N II]/H$\alpha$) put the DSFG itself in the AGN region of the diagrams, and therefore suggest it to be the most probable AGN host. These results showcase the ability of JWST to unveil obscured AGN at high redshifts., Comment: 13 pages, 10 figures, and one table. Accepted for publication in A&A on November 12th 2024

Many real-world robot learning problems, such as pick-and-place or arriving at a destination, can be seen as a problem of reaching a goal state as soon as possible. These problems, when formulated as episodic reinforcement learning tasks, can easily be specified to align well with our intended goal: -1 reward every time step with termination upon reaching the goal state, called minimum-time tasks. Despite this simplicity, such formulations are often overlooked in favor of dense rewards due to their perceived difficulty and lack of informativeness. Our studies contrast the two reward paradigms, revealing that the minimum-time task specification not only facilitates learning higher-quality policies but can also surpass dense-reward-based policies on their own performance metrics. Crucially, we also identify the goal-hit rate of the initial policy as a robust early indicator for learning success in such sparse feedback settings. Finally, using four distinct real-robotic platforms, we show that it is possible to learn pixel-based policies from scratch within two to three hours using constant negative rewards., Comment: In Proceedings of Reinforcement Learning Conference 2024. For a video demo, see https://youtu.be/a6zlVUuKzBc

We consider the problem of customizing text-to-image diffusion models with user-supplied reference images. Given new prompts, the existing methods can capture the key concept from the reference images but fail to align the generated image with the prompt. In this work, we seek to address this key issue by proposing new methods that can easily be used in conjunction with existing customization methods that optimize the embeddings/weights at various intermediate stages of the text encoding process. The first contribution of this paper is a dissection of the various stages of the text encoding process leading up to the conditioning vector for text-to-image models. We take a holistic view of existing customization methods and notice that key and value outputs from this process differs substantially from their corresponding baseline (non-customized) models (e.g., baseline stable diffusion). While this difference does not impact the concept being customized, it leads to other parts of the generated image not being aligned with the prompt. Further, we also observe that these keys and values allow independent control various aspects of the final generation, enabling semantic manipulation of the output. Taken together, the features spanning these keys and values, serve as the basis for our next contribution where we fix the aforementioned issues with existing methods. We propose a new post-processing algorithm, AlignIT, that infuses the keys and values for the concept of interest while ensuring the keys and values for all other tokens in the input prompt are unchanged. Our proposed method can be plugged in directly to existing customization methods, leading to a substantial performance improvement in the alignment of the final result with the input prompt while retaining the customization quality., Comment: 10 pages, 9 figures

Attributing answer text to its source document for information-seeking questions is crucial for building trustworthy, reliable, and accountable systems. We formulate a new task of post-hoc answer attribution for long document comprehension (LDC). Owing to the lack of long-form abstractive and information-seeking LDC datasets, we refactor existing datasets to assess the strengths and weaknesses of existing retrieval-based and proposed answer decomposition and textual entailment-based optimal selection attribution systems for this task. We throw light on the limitations of existing datasets and the need for datasets to assess the actual performance of systems on this task., Comment: Accepted to *SEM 2024

Music is a universal language that can communicate emotions and feelings. It forms an essential part of the whole spectrum of creative media, ranging from movies to social media posts. Machine learning models that can synthesize music are predominantly conditioned on textual descriptions of it. Inspired by how musicians compose music not just from a movie script, but also through visualizations, we propose MeLFusion, a model that can effectively use cues from a textual description and the corresponding image to synthesize music. MeLFusion is a text-to-music diffusion model with a novel "visual synapse", which effectively infuses the semantics from the visual modality into the generated music. To facilitate research in this area, we introduce a new dataset MeLBench, and propose a new evaluation metric IMSM. Our exhaustive experimental evaluation suggests that adding visual information to the music synthesis pipeline significantly improves the quality of generated music, measured both objectively and subjectively, with a relative gain of up to 67.98% on the FAD score. We hope that our work will gather attention to this pragmatic, yet relatively under-explored research area., Comment: Accepted at CVPR 2024 as Highlight paper. Webpage: https://schowdhury671.github.io/melfusion_cvpr2024/

Accurate chemical compositions of star-forming regions are a critical diagnostic tool to characterize the star formation history and gas flows which regulate galaxy formation. However, the abundance discrepancy factor (ADF) between measurements from the "direct" optical electron temperature ($T_e$) method and from the recombination lines (RL) represents $\sim0.2$ dex systematic uncertainty in oxygen abundance. The degree of uncertainty for other elements is unknown. We conduct a comprehensive analysis of O$^{++}$ and N$^+$ ion abundances using optical and far-infrared spectra of a star-forming region within the nearby dwarf galaxy Haro 3, which exhibits a typical ADF. Assuming homogeneous conditions, the far-IR emission indicates an O abundance which is higher than the $T_e$ method and consistent with the RL value, as would be expected from temperature fluctuations, whereas the N abundance is too large to be explained by temperature fluctuations. Instead a component of highly obscured gas is likely required to explain the high far-IR to optical flux ratios. Accounting for this obscured component reduces both the IR-based metallicities and the inferred magnitude of temperature fluctuations, such that they cannot fully explain the ADF in Haro 3. Additionally, we find potential issues when predicting the RL fluxes from current atomic data. Our findings underscore the critical importance of resolving the cause of abundance discrepancies and understanding the biases between different metallicity methods. This work represents a promising methodology, and we identify further approaches to address the current dominant uncertainties., Comment: 18 pages, 7 figures, and 2 tables. Submitted to ApJ

With the enhancement in the field of generative artificial intelligence (AI), contextual question answering has become extremely relevant. Attributing model generations to the input source document is essential to ensure trustworthiness and reliability. We observe that when large language models (LLMs) are used for contextual question answering, the output answer often consists of text copied verbatim from the input prompt which is linked together with "glue text" generated by the LLM. Motivated by this, we propose that LLMs have an inherent awareness from where the text was copied, likely captured in the hidden states of the LLM. We introduce a novel method for attribution in contextual question answering, leveraging the hidden state representations of LLMs. Our approach bypasses the need for extensive model retraining and retrieval model overhead, offering granular attributions and preserving the quality of generated answers. Our experimental results demonstrate that our method performs on par or better than GPT-4 at identifying verbatim copied segments in LLM generations and in attributing these segments to their source. Importantly, our method shows robust performance across various LLM architectures, highlighting its broad applicability. Additionally, we present Verifiability-granular, an attribution dataset which has token level annotations for LLM generations in the contextual question answering setup.

Observed evolution of the total mass distribution with redshift is crucial to testing galaxy evolution theories. To measure the total mass distribution, strong gravitational lenses complement the resolved dynamical observations currently limited to $z \lesssim 0.5$. Here we present the lens models for a pilot sample of seven galaxy-scale lenses from the ASTRO3D Galaxy Evolution with Lenses (AGEL) survey. The AGEL lenses, modeled using HST/WFC3-F140W images with Gravitational Lens Efficient Explorer (GLEE) software, have deflector redshifts between $0.3 < z_{\rm defl} < 0.9$. Assuming a power-law density profile with slope $\gamma$, we measure the total density profile for the deflector galaxies via lens modeling. We also measure the stellar velocity dispersions ($\sigma_{\rm obs}$) for four lenses and obtain $\sigma_{\rm obs}$ from SDSS-BOSS for the remaining lenses to test our lens models by comparing observed and model-predicted velocity dispersions. For the seven AGEL lenses, we measure an average density profile slope of $-1.95 \pm 0.09$ and a $\gamma$--$z$ relation that does not evolve with redshift at $z<1$. Although our result is consistent with some observations and simulations, it differs from other studies at $z<1$ that suggest the $\gamma$--$z$ relation evolves with redshift. The apparent conflicts among observations and simulations may be due to a combination of 1) systematics in the lensing and dynamical modeling; 2) challenges in comparing observations with simulations; and 3) assuming a simple power-law for the total mass distribution. By providing more lenses at $z_{\rm defl} > 0.5$, the AGEL survey will provide stronger constraints on whether the mass profiles evolve with redshift as predicted by current theoretical models., Comment: Accepted for publication in the Astrophysical Journal (ApJ) 24 Pages, 8 Figures, 3 Tables

The metaverse promises a shift in the way humans interact with each other, and with their digital and physical environments. The lack of geographical boundaries and travel costs in the metaverse prompts us to ask if the fundamental laws that govern human mobility in the physical world apply. We collected data on avatar movements, along with their network mobility extracted from NFT purchases. We find that despite the absence of commuting costs, an individuals inclination to explore new locations diminishes over time, limiting movement to a small fraction of the metaverse. We also find a lack of correlation between land prices and visitation, a deviation from the patterns characterizing the physical world. Finally, we identify the scaling laws that characterize meta mobility and show that we need to add preferential selection to the existing models to explain quantitative patterns of metaverse mobility. Our ability to predict the characteristics of the emerging meta mobility network implies that the laws governing human mobility are rooted in fundamental patterns of human dynamics, rather than the nature of space and cost of movement., Comment: 4 figures

Amidst the growing interest in developing task-autonomous AI for automated mental health care, this paper addresses the ethical and practical challenges associated with the issue and proposes a structured framework that delineates levels of autonomy, outlines ethical requirements, and defines beneficial default behaviors for AI agents in the context of mental health support. We also evaluate fourteen state-of-the-art language models (ten off-the-shelf, four fine-tuned) using 16 mental health-related questionnaires designed to reflect various mental health conditions, such as psychosis, mania, depression, suicidal thoughts, and homicidal tendencies. The questionnaire design and response evaluations were conducted by mental health clinicians (M.D.s). We find that existing language models are insufficient to match the standard provided by human professionals who can navigate nuances and appreciate context. This is due to a range of issues, including overly cautious or sycophantic responses and the absence of necessary safeguards. Alarmingly, we find that most of the tested models could cause harm if accessed in mental health emergencies, failing to protect users and potentially exacerbating existing symptoms. We explore solutions to enhance the safety of current models. Before the release of increasingly task-autonomous AI systems in mental health, it is crucial to ensure that these models can reliably detect and manage symptoms of common psychiatric disorders to prevent harm to users. This involves aligning with the ethical framework and default behaviors outlined in our study. We contend that model developers are responsible for refining their systems per these guidelines to safeguard against the risks posed by current AI technologies to user mental health and safety. Trigger warning: Contains and discusses examples of sensitive mental health topics, including suicide and self-harm., Comment: Updated with fine-tuned model results to match CoLM accepted camera-ready version

We study the spatially resolved outflow properties of CSWA13, an intermediate mass ($M_*=10^{9}~\mathrm{M}_{\odot}$), gravitationally lensed star-forming galaxy at $z=1.87$. We use Keck/KCWI to map outflows in multiple rest-frame ultraviolet ISM absorption lines, along with fluorescent Si II$^*$ emission, and nebular emission from C III] tracing the local systemic velocity. The spatial structure of outflow velocity mirrors that of the nebular kinematics, which we interpret to be a signature of a young galactic wind that is pressurizing the ISM of the galaxy but is yet to burst out. From the radial extent of Si II$^*$ emission, we estimate that the outflow is largely encapsulated within $3.5$ kpc. We explore the geometry (e.g., patchiness) of the outflow by measuring the covering fraction at different velocities, finding that the maximum covering fraction is at velocities $v\simeq-150$ km$\,$s$^{-1}$. Using the outflow velocity ($v_{out}$), radius ($R$), column density ($N$), and solid angle ($\Omega$) based on the covering fraction, we measure the mass loss rate $\log\dot{m}_{out}/(\mathrm{M}_{\odot}\text{yr}^{-1}) = 1.73\pm0.23$ and mass loading factor $\log\eta = 0.04\pm0.34$ for the low-ionization outflowing gas in this galaxy. These values are relatively large and the bulk of the outflowing gas is moving with speeds less than the escape velocity of the galaxy halo, suggesting that the majority of outflowing mass will remain in the circumgalactic medium and/or recycle back into the galaxy. The results support a picture of high outflow rates transporting mass and metals into the inner circumgalactic medium, providing the gas reservoir for future star formation., Comment: Submitted to ApJ. Comments welcome (17 pages, 10 figures, 1 table)

This study introduces an innovative approach aimed at the efficient pruning of neural networks, with a particular focus on their deployment on edge devices. Our method involves the integration of the Lottery Ticket Hypothesis (LTH) with the Knowledge Distillation (KD) framework, resulting in the formulation of three distinct pruning models. These models have been developed to address scalability issue in recommender systems, whereby the complexities of deep learning models have hindered their practical deployment. With judicious application of the pruning techniques, we effectively curtail the power consumption and model dimensions without compromising on accuracy. Empirical evaluation has been performed using two real world datasets from diverse domains against two baselines. Gratifyingly, our approaches yielded a GPU computation-power reduction of up to 66.67%. Notably, our study contributes to the field of recommendation system by pioneering the application of LTH and KD., Comment: Accepted in WITS 2023 as a workshop paper

Social media advertisements are key for brand marketing, aiming to attract consumers with captivating captions and pictures or logos. While previous research has focused on generating captions for general images, incorporating brand personalities into social media captioning remains unexplored. Brand personalities are shown to be affecting consumers' behaviours and social interactions and thus are proven to be a key aspect of marketing strategies. Current open-source multimodal LLMs are not directly suited for this task. Hence, we propose a pipeline solution to assist brands in creating engaging social media captions that align with the image and the brand personalities. Our architecture is based on two parts: a the first part contains an image captioning model that takes in an image that the brand wants to post online and gives a plain English caption; b the second part takes in the generated caption along with the target brand personality and outputs a catchy personality-aligned social media caption. Along with brand personality, our system also gives users the flexibility to provide hashtags, Instagram handles, URLs, and named entities they want the caption to contain, making the captions more semantically related to the social media handles. Comparative evaluations against various baselines demonstrate the effectiveness of our approach, both qualitatively and quantitatively.