Search

Diffusion models have demonstrated remarkable efficacy across various image-to-image tasks. In this research, we introduce Imagine yourself, a state-of-the-art model designed for personalized image generation. Unlike conventional tuning-based personalization techniques, Imagine yourself operates as a tuning-free model, enabling all users to leverage a shared framework without individualized adjustments. Moreover, previous work met challenges balancing identity preservation, following complex prompts and preserving good visual quality, resulting in models having strong copy-paste effect of the reference images. Thus, they can hardly generate images following prompts that require significant changes to the reference image, \eg, changing facial expression, head and body poses, and the diversity of the generated images is low. To address these limitations, our proposed method introduces 1) a new synthetic paired data generation mechanism to encourage image diversity, 2) a fully parallel attention architecture with three text encoders and a fully trainable vision encoder to improve the text faithfulness, and 3) a novel coarse-to-fine multi-stage finetuning methodology that gradually pushes the boundary of visual quality. Our study demonstrates that Imagine yourself surpasses the state-of-the-art personalization model, exhibiting superior capabilities in identity preservation, visual quality, and text alignment. This model establishes a robust foundation for various personalization applications. Human evaluation results validate the model's SOTA superiority across all aspects (identity preservation, text faithfulness, and visual appeal) compared to the previous personalization models.

We study carbon-aware spatiotemporal workload management, which seeks to address the growing environmental impact of data centers. We formalize this as an online problem called spatiotemporal online allocation with deadline constraints ($\mathsf{SOAD}$), in which an online player completes a workload (e.g., a batch compute job) by moving and scheduling the workload across a network subject to a deadline $T$. At each time step, a service cost function is revealed, representing, e.g., the carbon intensity of servicing a workload at each location, and the player must irrevocably decide the current allocation. Furthermore, whenever the player moves the allocation, it incurs a movement cost defined by a metric space $(X,d)$ that captures, e.g., the overhead of migrating a compute job. $\mathsf{SOAD}$ formalizes the open problem of combining general metrics and deadline constraints in the online algorithms literature, unifying problems such as metrical task systems and online search. We propose a competitive algorithm for $\mathsf{SOAD}$ along with a matching lower bound that proves it is optimal. Our main algorithm, ${\rm C{\scriptsize ARBON}C{\scriptsize LIPPER}}$, is a learning-augmented algorithm that takes advantage of predictions (e.g., carbon intensity forecasts) and achieves an optimal consistency-robustness trade-off. We evaluate our proposed algorithms for carbon-aware spatiotemporal workload management on a simulated global data center network, showing that ${\rm C{\scriptsize ARBON}C{\scriptsize LIPPER}}$ significantly improves performance compared to baseline methods and delivers meaningful carbon reductions., Comment: 50 pages, 21 figures

Electrokinetic (EK) flow is a type of flow driven or manipulated by electric body forces, influenced by various factors such as electric field intensity, electric field form, frequency, electric permittivity/conductivity, fluid viscosity and etc. The diversity of dimensionless control parameters, such as the electric Rayleigh number, complicates the comparison of EK flow stability. Consequently, comparing the performance and cost of micromixers or reactors based on EK flow is challenging, posing an obstacle to their industrial and engineering applications. In this investigation, we theoretically derived a new electric Rayleigh number ($Ra_e$) that quantifies the relationship among electric body forces, fluid viscosity, and ion diffusivity, based on a tanh model of electric conductivity distribution. The calculation results indicate that the new $Ra_e$ exhibits richer variation with the control parameters and better consistency with previous experimental reports. We further conducted experimental studies on the critical electric Rayleigh number ($Ra_{ec}$) of AC EK flow of binary fluids in a divergent microchannel. The experimental variations align well with the theoretical predictions, particularly the existence of an optimal AC frequency and electric conductivity ratio, demonstrating that the tanh model can better elucidate the underlying physics of EK flow. With the new electric Rayleigh number, we found that EK flow in the designed divergent microchannel has a much smaller $Ra_{ec}$ than previously reported, indicating that EK flow is more unstable and thus more suitable for applications in micromixers or reactors in industry and engineering.

A pivotal attribute of 5G networks is their capability to cater to diverse application requirements. This is achieved by creating logically isolated virtual networks, or slices, with distinct service level agreements (SLAs) tailored to specific use cases. However, efficiently allocating resources to maintain slice SLA is challenging due to varying traffic and QoS requirements. Traditional peak traffic-based resource allocation leads to over-provisioning, as actual traffic rarely peaks. Additionally, the complex relationship between resource allocation and QoS in end-to-end slices spanning different network segments makes conventional optimization techniques impractical. Existing approaches in this domain use network models or simulations and various optimization methods but struggle with optimality, tractability, and generalizability across different slice types. In this paper, we propose MicroOpt, a novel framework that leverages a differentiable neural network-based slice model with gradient descent for resource optimization and Lagrangian decomposition for QoS constraint satisfaction. We evaluate MicroOpt against two state-of-the-art approaches using an open-source 5G testbed with real-world traffic traces. Our results demonstrate up to 21.9% improvement in resource allocation compared to these approaches across various scenarios, including different QoS thresholds and dynamic slice traffic., Comment: 12 pages, 10 figures

In the online knapsack problem, the goal is to pack items arriving online with different values and weights into a capacity-limited knapsack to maximize the total value of the accepted items. We study \textit{learning-augmented} algorithms for this problem, which aim to use machine-learned predictions to move beyond pessimistic worst-case guarantees. Existing learning-augmented algorithms for online knapsack consider relatively complicated prediction models that give an algorithm substantial information about the input, such as the total weight of items at each value. In practice, such predictions can be error-sensitive and difficult to learn. Motivated by this limitation, we introduce a family of learning-augmented algorithms for online knapsack that use \emph{succinct predictions}. In particular, the machine-learned prediction given to the algorithm is just a single value or interval that estimates the minimum value of any item accepted by an offline optimal solution. By leveraging a relaxation to online \emph{fractional} knapsack, we design algorithms that can leverage such succinct predictions in both the trusted setting (i.e., with perfect prediction) and the untrusted setting, where we prove that a simple meta-algorithm achieves a nearly optimal consistency-robustness trade-off. Empirically, we show that our algorithms significantly outperform baselines that do not use predictions and often outperform algorithms based on more complex prediction models., Comment: 29 pages, 10 figures, Submitted to NeurIPS 2024

This work proposes a novel representation of injective deformations of 3D space, which overcomes existing limitations of injective methods: inaccuracy, lack of robustness, and incompatibility with general learning and optimization frameworks. The core idea is to reduce the problem to a deep composition of multiple 2D mesh-based piecewise-linear maps. Namely, we build differentiable layers that produce mesh deformations through Tutte's embedding (guaranteed to be injective in 2D), and compose these layers over different planes to create complex 3D injective deformations of the 3D volume. We show our method provides the ability to efficiently and accurately optimize and learn complex deformations, outperforming other injective approaches. As a main application, we produce complex and artifact-free NeRF and SDF deformations.

Diamond has the known highest thermal conductivity of around \SI{2000}{\watt\per\meter\per\kelvin} and is therefore widely used for heat dissipation. In practical applications, synthetic diamond microparticles are usually assumed to have similar thermal conductivity to that of bulk diamond because the particle size is larger than theoretical phonon mean free path so that boundary scattering of heat-carrying phonons is absent. In this report, we find the thermal conductivity of diamond microparticles anomalously depends on their sizes. Thermal conductivity of diamond microparticles increases from \SI{400}{\watt\per\meter\per\kelvin} to \SI{2000}{\watt\per\meter\per\kelvin} with the size growing from \SI{20}{\micro\meter} to \SI{300}{\micro\meter}. We attribute the abnormally strong size effect to the long-range defects during the growth process based on analysis of point defects, dislocations, and thermal penetration depth dependence of thermal conductivity. Our results play a vital role in the design of diamond composites and in the improvement of thermal conductivity of synthetic diamonds.

With the increasing penetration of intermittent renewable energy sources (RESs), it becomes increasingly challenging to maintain the supply-demand balance of power systems by solely relying on the generation side. To combat the volatility led by the uncertain RESs, demand-side management by leveraging the multi-dimensional flexibility (MDF) has been recognized as an economic and efficient approach. Thus, it is important to integrate MDF into existing power systems. In this paper, we propose an enhanced day-ahead energy market, where the MDFs of aggregate loads are traded to minimize the generation cost and mitigate the volatility of locational marginal prices (LMPs) in the transmission network. We first explicitly capture the negative impact of the uncertainty from RESs on the day-ahead market by a chance-constrained economic dispatch problem (CEDP). Then, we propose a bidding mechanism for the MDF of the aggregate loads and combine this mechanism into the CEDP for the day-ahead market. Through multiple case studies, we show that MDF from load aggregators can reduce the volatility of LMPs. In addition, we identify the values of the different flexibilities in the MDF bids, which provide useful insights into the design of more complex MDF markets.

We study the design of risk-sensitive online algorithms, in which risk measures are used in the competitive analysis of randomized online algorithms. We introduce the CVaR$_\delta$-competitive ratio ($\delta$-CR) using the conditional value-at-risk of an algorithm's cost, which measures the expectation of the $(1-\delta)$-fraction of worst outcomes against the offline optimal cost, and use this measure to study three online optimization problems: continuous-time ski rental, discrete-time ski rental, and one-max search. The structure of the optimal $\delta$-CR and algorithm varies significantly between problems: we prove that the optimal $\delta$-CR for continuous-time ski rental is $2-2^{-\Theta(\frac{1}{1-\delta})}$, obtained by an algorithm described by a delay differential equation. In contrast, in discrete-time ski rental with buying cost $B$, there is an abrupt phase transition at $\delta = 1 - \Theta(\frac{1}{\log B})$, after which the classic deterministic strategy is optimal. Similarly, one-max search exhibits a phase transition at $\delta = \frac{1}{2}$, after which the classic deterministic strategy is optimal; we also obtain an algorithm that is asymptotically optimal as $\delta \downarrow 0$ that arises as the solution to a delay differential equation., Comment: Accepted for presentation at the Conference on Learning Theory (COLT) 2024. Updated with an additional reference and minor edits

We introduce and study a family of online metric problems with long-term constraints. In these problems, an online player makes decisions $\mathbf{x}_t$ in a metric space $(X,d)$ to simultaneously minimize their hitting cost $f_t(\mathbf{x}_t)$ and switching cost as determined by the metric. Over the time horizon $T$, the player must satisfy a long-term demand constraint $\sum_{t} c(\mathbf{x}_t) \geq 1$, where $c(\mathbf{x}_t)$ denotes the fraction of demand satisfied at time $t$. Such problems can find a wide array of applications to online resource allocation in sustainable energy/computing systems. We devise optimal competitive and learning-augmented algorithms for the case of bounded hitting cost gradients and weighted $\ell_1$ metrics, and further show that our proposed algorithms perform well in numerical experiments., Comment: Accepted to ICML 2024. 31 pages, 12 figures

Nonlinear Hall effect (NLHE) holds immense significance in recognizing the band geometry and its potential applications in current rectification. Recent discoveries have expanded the study from second-order to third-order nonlinear Hall effect (THE), which is governed by an intrinsic band geometric quantity called the Berry Connection Polarizability (BCP) tensor. Here we demonstrate a giant THE in a misfit layer compound, (SnS)${1.17}$(NbS$_2$)$_3$. While the THE is prohibited in individual NbS$_2$ and SnS due to the constraints imposed by the crystal symmetry and their band structures, a remarkable THE emerges when a superlattice is formed by introducing a monolayer of SnS. The angular-dependent THE and its scaling relationship indicate that the phenomenon could be correlated to the band geometry modulation, concurrently with the symmetry breaking. The resulting strength of THE is orders of magnitude higher compared to recent studies. Our work illuminates the modulation of structural and electronic geometries for novel quantum phenomena through interface engineering.

To enhance the performance and effect of AR/VR applications and visual assistance and inspection systems, visual simultaneous localization and mapping (vSLAM) is a fundamental task in computer vision and robotics. However, traditional vSLAM systems are limited by the camera's narrow field-of-view, resulting in challenges such as sparse feature distribution and lack of dense depth information. To overcome these limitations, this paper proposes a 360ORB-SLAM system for panoramic images that combines with a depth completion network. The system extracts feature points from the panoramic image, utilizes a panoramic triangulation module to generate sparse depth information, and employs a depth completion network to obtain a dense panoramic depth map. Experimental results on our novel panoramic dataset constructed based on Carla demonstrate that the proposed method achieves superior scale accuracy compared to existing monocular SLAM methods and effectively addresses the challenges of feature association and scale ambiguity. The integration of the depth completion network enhances system stability and mitigates the impact of dynamic elements on SLAM performance., Comment: 6 pages, 9 figures

Existing 3D semantic segmentation methods rely on point-wise or voxel-wise feature descriptors to output segmentation predictions. However, these descriptors are often supervised at point or voxel level, leading to segmentation models that can behave poorly at instance-level. In this paper, we proposed a novel instance-aware approach for 3D semantic segmentation. Our method combines several geometry processing tasks supervised at instance-level to promote the consistency of the learned feature representation. Specifically, our methods use shape generators and shape classifiers to perform shape reconstruction and classification tasks for each shape instance. This enforces the feature representation to faithfully encode both structural and local shape information, with an awareness of shape instances. In the experiments, our method significantly outperform existing approaches in 3D semantic segmentation on several public benchmarks, such as Waymo Open Dataset, SemanticKITTI and ScanNetV2.

We introduce Style Tailoring, a recipe to finetune Latent Diffusion Models (LDMs) in a distinct domain with high visual quality, prompt alignment and scene diversity. We choose sticker image generation as the target domain, as the images significantly differ from photorealistic samples typically generated by large-scale LDMs. We start with a competent text-to-image model, like Emu, and show that relying on prompt engineering with a photorealistic model to generate stickers leads to poor prompt alignment and scene diversity. To overcome these drawbacks, we first finetune Emu on millions of sticker-like images collected using weak supervision to elicit diversity. Next, we curate human-in-the-loop (HITL) Alignment and Style datasets from model generations, and finetune to improve prompt alignment and style alignment respectively. Sequential finetuning on these datasets poses a tradeoff between better style alignment and prompt alignment gains. To address this tradeoff, we propose a novel fine-tuning method called Style Tailoring, which jointly fits the content and style distribution and achieves best tradeoff. Evaluation results show our method improves visual quality by 14%, prompt alignment by 16.2% and scene diversity by 15.3%, compared to prompt engineering the base Emu model for stickers generation., Comment: 10 pages, 5 figures

Hydride precipitation in zirconium cladding materials can damage their integrity and durability.Service temperature and material defects have a significant effect on the dynamic growth of hydrides. In this study, we have developed a phase field model based on the assumption of elastic behaviour within a specific temperature range (613-653K). This model allows us to study the influence of temperature and interfacial effects on the morphology, stress, and average growth rate of zirconium hydride. The results suggest that changes in temperature and interfacial energy influence the aspect ratio and average growth rate of the hydride morphology. The ultimate determinant of hydride orientation is the loss of interfacial coherence, primarily induced by interfacial dislocation defects and quantifiable by the mismatch degree $q$. An escalation in interfacial coherence loss leads to a transition of hydride growth from horizontal to vertical, accompanied by the onset of redirection behaviour. Interestingly, redirection occurs at a critical mismatch level, denoted $q_c$, and remains unaffected by variations in temperature and interfacial energy. However, this redirection leads to an increase in the maximum stress, which may influence the direction of hydride crack propagation. This research highlights the importance of interfacial coherence and provides valuable insights into the morphology and growth kinetics of hydrides in zirconium alloys., Comment: 13 pages (text), 8 figures (text), 1 table (text), 6 pages (SI), 2 figures (SI)

We introduce and study online conversion with switching costs, a family of online problems that capture emerging problems at the intersection of energy and sustainability. In this problem, an online player attempts to purchase (alternatively, sell) fractional shares of an asset during a fixed time horizon with length $T$. At each time step, a cost function (alternatively, price function) is revealed, and the player must irrevocably decide an amount of asset to convert. The player also incurs a switching cost whenever their decision changes in consecutive time steps, i.e., when they increase or decrease their purchasing amount. We introduce competitive (robust) threshold-based algorithms for both the minimization and maximization variants of this problem, and show they are optimal among deterministic online algorithms. We then propose learning-augmented algorithms that take advantage of untrusted black-box advice (such as predictions from a machine learning model) to achieve significantly better average-case performance without sacrificing worst-case competitive guarantees. Finally, we empirically evaluate our proposed algorithms using a carbon-aware EV charging case study, showing that our algorithms substantially improve on baseline methods for this problem., Comment: Accepted to SIGMETRICS / Performance '24. 47 pages, 9 figures

The burgeoning field of algorithms with predictions studies the problem of using possibly imperfect machine learning predictions to improve online algorithm performance. While nearly all existing algorithms in this framework make no assumptions on prediction quality, a number of methods providing uncertainty quantification (UQ) on machine learning models have been developed in recent years, which could enable additional information about prediction quality at decision time. In this work, we investigate the problem of optimally utilizing uncertainty-quantified predictions in the design of online algorithms. In particular, we study two classic online problems, ski rental and online search, where the decision-maker is provided predictions augmented with UQ describing the likelihood of the ground truth falling within a particular range of values. We demonstrate that non-trivial modifications to algorithm design are needed to fully leverage the UQ predictions. Moreover, we consider how to utilize more general forms of UQ, proposing an online learning framework that learns to exploit UQ to make decisions in multi-instance settings.

The annotation of 3D datasets is required for semantic-segmentation and object detection in scene understanding. In this paper we present a framework for the weakly supervision of a point clouds transformer that is used for 3D object detection. The aim is to decrease the required amount of supervision needed for training, as a result of the high cost of annotating a 3D datasets. We propose an Unsupervised Voting Proposal Module, which learns randomly preset anchor points and uses voting network to select prepared anchor points of high quality. Then it distills information into student and teacher network. In terms of student network, we apply ResNet network to efficiently extract local characteristics. However, it also can lose much global information. To provide the input which incorporates the global and local information as the input of student networks, we adopt the self-attention mechanism of transformer to extract global features, and the ResNet layers to extract region proposals. The teacher network supervises the classification and regression of the student network using the pre-trained model on ImageNet. On the challenging KITTI datasets, the experimental results have achieved the highest level of average precision compared with the most recent weakly supervised 3D object detectors., Comment: International Conference on Intelligent Transportation Systems (ITSC), 2022