Search

Large Language Models (LLMs) have demonstrated promising capabilities as automatic evaluators in assessing the quality of generated natural language. However, LLMs still exhibit biases in evaluation and often struggle to generate coherent evaluations that align with human assessments. In this work, we first conduct a systematic study of the misalignment between LLM evaluators and human judgement, revealing that existing calibration methods aimed at mitigating biases are insufficient for effectively aligning LLM evaluators. Inspired by the use of preference data in RLHF, we formulate the evaluation as a ranking problem and introduce Pairwise-preference Search (PairS), an uncertainty-guided search method that employs LLMs to conduct pairwise comparisons and efficiently ranks candidate texts. PairS achieves state-of-the-art performance on representative evaluation tasks and demonstrates significant improvements over direct scoring. Furthermore, we provide insights into the role of pairwise preference in quantifying the transitivity of LLMs and demonstrate how PairS benefits from calibration.

Top-view perspective denotes a typical way in which humans read and reason over different types of maps, and it is vital for localization and navigation of humans as well as of `non-human' agents, such as the ones backed by large Vision-Language Models (VLMs). Nonetheless, spatial reasoning capabilities of modern VLMs remain unattested and underexplored. In this work, we thus study their capability to understand and reason over spatial relations from the top view. The focus on top view also enables controlled evaluations at different granularity of spatial reasoning; we clearly disentangle different abilities (e.g., recognizing particular objects versus understanding their relative positions). We introduce the TopViewRS (Top-View Reasoning in Space) dataset, consisting of 11,384 multiple-choice questions with either realistic or semantic top-view map as visual input. We then use it to study and evaluate VLMs across 4 perception and reasoning tasks with different levels of complexity. Evaluation of 10 representative open- and closed-source VLMs reveals the gap of more than 50% compared to average human performance, and it is even lower than the random baseline in some cases. Although additional experiments show that Chain-of-Thought reasoning can boost model capabilities by 5.82% on average, the overall performance of VLMs remains limited. Our findings underscore the critical need for enhanced model capability in top-view spatial reasoning and set a foundation for further research towards human-level proficiency of VLMs in real-world multimodal tasks., Comment: 9 pages, 3 figures, 3 tables (21 pages, 4 figures, 15 tables including references and appendices)

Shadow-affected images often exhibit pronounced spatial discrepancies in color and illumination, consequently degrading various vision applications including object detection and segmentation systems. To effectively eliminate shadows in real-world images while preserving intricate details and producing visually compelling outcomes, we introduce a mask-free Shadow Removal and Refinement network (ShadowRefiner) via Fast Fourier Transformer. Specifically, the Shadow Removal module in our method aims to establish effective mappings between shadow-affected and shadow-free images via spatial and frequency representation learning. To mitigate the pixel misalignment and further improve the image quality, we propose a novel Fast-Fourier Attention based Transformer (FFAT) architecture, where an innovative attention mechanism is designed for meticulous refinement. Our method wins the championship in the Perceptual Track and achieves the second best performance in the Fidelity Track of NTIRE 2024 Image Shadow Removal Challenge. Besides, comprehensive experiment result also demonstrate the compelling effectiveness of our proposed method. The code is publicly available: https://github.com/movingforward100/Shadow_R., Comment: Accepted by CVPR workshop 2024 (NTIRE 2024)

In multivariate time series forecasting, the Transformer architecture encounters two significant challenges: effectively mining features from historical sequences and avoiding overfitting during the learning of temporal dependencies. To tackle these challenges, this paper deconstructs time series forecasting into the learning of historical sequences and prediction sequences, introducing the Cross-Variable and Time Network (CVTN). This unique method divides multivariate time series forecasting into two phases: cross-variable learning for effectively mining fea tures from historical sequences, and cross-time learning to capture the temporal dependencies of prediction sequences. Separating these two phases helps avoid the impact of overfitting in cross-time learning on cross-variable learning. Exten sive experiments on various real-world datasets have confirmed its state-of-the-art (SOTA) performance. CVTN emphasizes three key dimensions in time series fore casting: the short-term and long-term nature of time series (locality and longevity), feature mining from both historical and prediction sequences, and the integration of cross-variable and cross-time learning. This approach not only advances the current state of time series forecasting but also provides a more comprehensive framework for future research in this field.

Large pre-trained models have revolutionized natural language processing (NLP) research and applications, but high training costs and limited data resources have prevented their benefits from being shared equally amongst speakers of all the world's languages. To address issues of cross-linguistic access to such models and reduce energy consumption for sustainability during large-scale model training, this study proposes an effective and energy-efficient framework called GreenPLM that uses bilingual lexicons to directly “translate” pre-trained language models of one language into another at almost no additional cost. We validate this approach in 18 languages' BERT models and show that this framework is comparable to, if not better than, other heuristics with high training costs. In addition, given lightweight continued pretraining on limited data where available, this framework outperforms the original monolingual language models in six out of seven tested languages with up to 200x less pre-training efforts. Aiming at the Leave No One Behind Principle (LNOB), our approach manages to reduce inequalities between languages and energy consumption greatly. We make our codes and models publicly available at https://github.com/qcznlp/GreenPLMs. © 2023 International Joint Conferences on Artificial Intelligence. All rights reserved.

Large pretrained language models are widely used in downstream NLP tasks via task-specific fine-tuning, but such procedures can be costly. Recently, Parameter-Efficient Fine-Tuning (PEFT) methods have achieved strong task performance while updating much fewer parameters than full model fine-tuning (FFT). However, it is non-trivial to make informed design choices on the PEFT configurations, such as their architecture, the number of tunable parameters, and even the layers in which the PEFT modules are inserted. Consequently, it is highly likely that the current, manually designed configurations are suboptimal in terms of their performance-efficiency trade-off. Inspired by advances in neural architecture search, we propose AutoPEFT for automatic PEFT configuration selection: we first design an expressive configuration search space with multiple representative PEFT modules as building blocks. Using multi-objective Bayesian optimisation in a low-cost setup, we then discover a Pareto-optimal set of configurations with strong performance-cost trade-offs across different numbers of parameters that are also highly transferable across different tasks. Empirically, on GLUE and SuperGLUE tasks, we show that AutoPEFT-discovered configurations significantly outperform existing PEFT methods and are on par or better than FFT without incurring substantial training efficiency costs., Comment: Accepted to TACL; pre-MIT Press publication version

Deep neural networks have been widely applied in dichotomous medical image segmentation (DMIS) of many anatomical structures in several modalities, achieving promising performance. However, existing networks tend to struggle with task-specific, heavy and complex designs to improve accuracy. They made little instructions to which feature channels would be more beneficial for segmentation, and that may be why the performance and universality of these segmentation models are hindered. In this study, we propose an instructive feature enhancement approach, namely IFE, to adaptively select feature channels with rich texture cues and strong discriminability to enhance raw features based on local curvature or global information entropy criteria. Being plug-and-play and applicable for diverse DMIS tasks, IFE encourages the model to focus on texture-rich features which are especially important for the ambiguous and challenging boundary identification, simultaneously achieving simplicity, universality, and certain interpretability. To evaluate the proposed IFE, we constructed the first large-scale DMIS dataset Cosmos55k, which contains 55,023 images from 7 modalities and 26 anatomical structures. Extensive experiments show that IFE can improve the performance of classic segmentation networks across different anatomies and modalities with only slight modifications. Code is available at https://github.com/yezi-66/IFE, Comment: Accepted by MICCAI 2023

In this work, two Crank-Nicolson schemes without corrections are developed for sub-diffusion equations. First, we propose a Crank-Nicolson scheme without correction for problems with regularity assumptions only on the source term. Second, since the existing Crank-Nicolson schemes have a severe reduction of convergence order for solving sub-diffusion equations with singular source terms in time, we then extend our scheme and propose a new Crank-Nicolson scheme for problems with singular source terms in time. Second-order error estimates for both the two Crank-Nicolson schemes are rigorously established by a Laplace transform technique, which are numerically verified by some numerical examples.

Haze usually leads to deteriorated images with low contrast, color shift and structural distortion. We observe that many deep learning based models exhibit exceptional performance on removing homogeneous haze, but they usually fail to address the challenge of non-homogeneous dehazing. Two main factors account for this situation. Firstly, due to the intricate and non uniform distribution of dense haze, the recovery of structural and chromatic features with high fidelity is challenging, particularly in regions with heavy haze. Secondly, the existing small scale datasets for non-homogeneous dehazing are inadequate to support reliable learning of feature mappings between hazy images and their corresponding haze-free counterparts by convolutional neural network (CNN)-based models. To tackle these two challenges, we propose a novel two branch network that leverages 2D discrete wavelete transform (DWT), fast Fourier convolution (FFC) residual block and a pretrained ConvNeXt model. Specifically, in the DWT-FFC frequency branch, our model exploits DWT to capture more high-frequency features. Moreover, by taking advantage of the large receptive field provided by FFC residual blocks, our model is able to effectively explore global contextual information and produce images with better perceptual quality. In the prior knowledge branch, an ImageNet pretrained ConvNeXt as opposed to Res2Net is adopted. This enables our model to learn more supplementary information and acquire a stronger generalization ability. The feasibility and effectiveness of the proposed method is demonstrated via extensive experiments and ablation studies. The code is available at https://github.com/zhouh115/DWT-FFC., Comment: Accepted by CVPRW 2023

The Segment Anything Model (SAM) is the first foundation model for general image segmentation. It has achieved impressive results on various natural image segmentation tasks. However, medical image segmentation (MIS) is more challenging because of the complex modalities, fine anatomical structures, uncertain and complex object boundaries, and wide-range object scales. To fully validate SAM's performance on medical data, we collected and sorted 53 open-source datasets and built a large medical segmentation dataset with 18 modalities, 84 objects, 125 object-modality paired targets, 1050K 2D images, and 6033K masks. We comprehensively analyzed different models and strategies on the so-called COSMOS 1050K dataset. Our findings mainly include the following: 1) SAM showed remarkable performance in some specific objects but was unstable, imperfect, or even totally failed in other situations. 2) SAM with the large ViT-H showed better overall performance than that with the small ViT-B. 3) SAM performed better with manual hints, especially box, than the Everything mode. 4) SAM could help human annotation with high labeling quality and less time. 5) SAM was sensitive to the randomness in the center point and tight box prompts, and may suffer from a serious performance drop. 6) SAM performed better than interactive methods with one or a few points, but will be outpaced as the number of points increases. 7) SAM's performance correlated to different factors, including boundary complexity, intensity differences, etc. 8) Finetuning the SAM on specific medical tasks could improve its average DICE performance by 4.39% and 6.68% for ViT-B and ViT-H, respectively. We hope that this comprehensive report can help researchers explore the potential of SAM applications in MIS, and guide how to appropriately use and develop SAM., Comment: Accepted by Medical Image Analysis. 23 pages, 18 figures, 8 tables

Following the standard supervised fine-tuning (SFT) paradigm, in-context learning (ICL) has become an efficient approach propelled by the recent advancements in large language models (LLMs), yielding promising performance across various tasks in few-shot data setups. However, both paradigms are prone to suffer from the critical problem of overconfidence (i.e., miscalibration), especially in such limited data setups. In this work, we deliver an in-depth analysis of the behavior across different choices of learning methods from the perspective of both performance and calibration, as well as their interplay. Through extensive controlled experiments, we find that simultaneous gains for both task performance and calibration are difficult to achieve, and the problem of miscalibration exists across all learning methods in low-resource scenarios. To address this challenging trade-off between performance and calibration, we then investigate the potential of self-ensembling techniques applied at different modeling stages (e.g., variations of in-context examples or variations in prompts or different ensembling strategies). We justify the feasibility of self-ensembling on SFT in addition to ICL, to make the predictions more calibrated and have comparable or even better performance. Our work sheds light on which learning paradigm to choose and how to enhance both task performance and calibration of LLMs., Comment: 9 pages, 4 figures, 5 tables (20 pages, 5 figures, 13 tables including references and appendices)

Achieving robust language technologies that can perform well across the world's many languages is a central goal of multilingual NLP. In this work, we take stock of and empirically analyse task performance disparities that exist between multilingual task-oriented dialogue (ToD) systems. We first define new quantitative measures of absolute and relative equivalence in system performance, capturing disparities across languages and within individual languages. Through a series of controlled experiments, we demonstrate that performance disparities depend on a number of factors: the nature of the ToD task at hand, the underlying pretrained language model, the target language, and the amount of ToD annotated data. We empirically prove the existence of the adaptation and intrinsic biases in current ToD systems: e.g., ToD systems trained for Arabic or Turkish using annotated ToD data fully parallel to English ToD data still exhibit diminished ToD task performance. Beyond providing a series of insights into the performance disparities of ToD systems in different languages, our analyses offer practical tips on how to approach ToD data collection and system development for new languages., Comment: Accepted to EMNLP 2023

Prompt-based learning has been an effective paradigm for large pretrained language models (LLM), enabling few-shot or even zero-shot learning. Black-box prompt search has received growing interest recently for its distinctive properties of gradient-free optimization, proven particularly useful and powerful for model-as-a-service usage. However, the discrete nature and the complexity of combinatorial optimization hinder the efficiency of modern black-box approaches. Despite extensive research on search algorithms, the crucial aspect of search space design and optimization has been largely overlooked. In this paper, we first conduct a sensitivity analysis by prompting LLM, revealing that only a small number of tokens exert a disproportionate amount of influence on LLM predictions. Leveraging this insight, we propose the Clustering and Pruning for Efficient Black-box Prompt Search (ClaPS), a simple black-box search method that first clusters and prunes the search space to focus exclusively on influential prompt tokens. By employing even simple search methods within the pruned search space, ClaPS achieves state-of-the-art performance across various tasks and LLMs, surpassing the performance of complex approaches while significantly reducing search costs. Our findings underscore the critical role of search space design and optimization in enhancing both the usefulness and the efficiency of black-box prompt-based learning., Comment: Findings of EMNLP 2023. 10 pages, 5 figures, 4 tables (14 pages, 5 figures, 8 tables including references and appendices)

Sequential maximization of expected improvement (EI) is one of the most widely used policies in Bayesian optimization because of its simplicity and ability to handle noisy observations. In particular, the improvement function often uses the best posterior mean as the best incumbent in noisy settings. However, the uncertainty associated with the incumbent solution is often neglected in many analytic EI-type methods: a closed-form acquisition function is derived in the noise-free setting, but then applied to the setting with noisy observations. To address this limitation, we propose a modification of EI that corrects its closed-form expression by incorporating the covariance information provided by the Gaussian Process (GP) model. This acquisition function specializes to the classical noise-free result, and we argue should replace that formula in Bayesian optimization software packages, tutorials, and textbooks. This enhanced acquisition provides good generality for noisy and noiseless settings. We show that our method achieves a sublinear convergence rate on the cumulative regret bound under heteroscedastic observation noise. Our empirical results demonstrate that our proposed acquisition function can outperform EI in the presence of noisy observations on benchmark functions for black-box optimization, as well as on parameter search for neural network model compression.

Medical image segmentation aims to delineate the anatomical or pathological structures of interest, playing a crucial role in clinical diagnosis. A substantial amount of high-quality annotated data is crucial for constructing high-precision deep segmentation models. However, medical annotation is highly cumbersome and time-consuming, especially for medical videos or 3D volumes, due to the huge labeling space and poor inter-frame consistency. Recently, a fundamental task named Moving Object Segmentation (MOS) has made significant advancements in natural images. Its objective is to delineate moving objects from the background within image sequences, requiring only minimal annotations. In this paper, we propose the first foundation model, named iMOS, for MOS in medical images. Extensive experiments on a large multi-modal medical dataset validate the effectiveness of the proposed iMOS. Specifically, with the annotation of only a small number of images in the sequence, iMOS can achieve satisfactory tracking and segmentation performance of moving objects throughout the entire sequence in bi-directions. We hope that the proposed iMOS can help accelerate the annotation speed of experts, and boost the development of medical foundation models., Comment: 5 pages, 7 figures, 3 tables. This paper has been accepted by ISBI 2024

Prompting and in-context learning (ICL) have become efficient learning paradigms for large language models (LLMs). However, LLMs suffer from prompt brittleness and various bias factors in the prompt, including but not limited to the formatting, the choice verbalizers, and the ICL examples. To address this problem that results in unexpected performance degradation, calibration methods have been developed to mitigate the effects of these biases while recovering LLM performance. In this work, we first conduct a systematic analysis of the existing calibration methods, where we both provide a unified view and reveal the failure cases. Inspired by these analyses, we propose Batch Calibration (BC), a simple yet intuitive method that controls the contextual bias from the batched input, unifies various prior approaches, and effectively addresses the aforementioned issues. BC is zero-shot, inference-only, and incurs negligible additional costs. In the few-shot setup, we further extend BC to allow it to learn the contextual bias from labeled data. We validate the effectiveness of BC with PaLM 2-(S, M, L) and CLIP models and demonstrate state-of-the-art performance over previous calibration baselines across more than 10 natural language understanding and image classification tasks., Comment: ICLR 2024. 9 pages, 9 figures, 3 tables (22 pages, 11 figures, 11 tables including references and appendices)

This work addresses a novel version of the kernel-free boundary integral (KFBI) method for solving elliptic PDEs with implicitly defined irregular boundaries and interfaces. We focus on boundary value problems and interface problems, which are reformulated into boundary integral equations and solved with the matrix-free GMRES method. In the KFBI method, evaluating boundary and volume integrals only requires solving equivalent but much simpler interface problems in a bounding box, for which fast solvers such as FFTs and geometric multigrid methods are applicable. For the simple interface problem, a correction function is introduced for both the evaluation of right-hand side correction terms and the interpolation of a non-smooth potential function. A mesh-free collocation method is proposed to compute the correction function near the interface. The new method avoids complicated derivation for derivative jumps of the solution and is easy to implement, especially for the fourth-order method in three space dimensions. Various numerical examples are presented, including challenging cases such as high-contrast coefficients, arbitrarily close interfaces and heterogeneous interface problems. The reported numerical results verify that the proposed method is both accurate and efficient.

This paper is concerned with the mean curvature flow, which describes the dynamics of a hypersurface whose normal velocity is determined by local mean curvature. We present a Cartesian grid-based method for solving mean curvature flows in two and three space dimensions. The present method embeds a closed hypersurface into a fixed Cartesian grid and decomposes it into multiple overlapping subsets. For each subset, extra tangential velocities are introduced such that marker points on the hypersurface only moves along grid lines. By utilizing an alternating direction implicit (ADI)-type time integration method, the subsets are evolved alternately by solving scalar parabolic partial differential equations on planar domains. The method removes the stiffness using a semi-implicit scheme and has no high-order stability constraint on time step size. Numerical examples in two and three space dimensions are presented to validate the proposed method.

In this paper, efficient alternating direction implicit (ADI) schemes are proposed to solve three-dimensional heat equations with irregular boundaries and interfaces. Starting from the well-known Douglas-Gunn ADI scheme, a modified ADI scheme is constructed to mitigate the issue of accuracy loss in solving problems with time-dependent boundary conditions. The unconditional stability of the new ADI scheme is also rigorously proven with the Fourier analysis. Then, by combining the ADI schemes with a 1D kernel-free boundary integral (KFBI) method, KFBI-ADI schemes are developed to solve the heat equation with irregular boundaries. In 1D sub-problems of the KFBI-ADI schemes, the KFBI discretization takes advantage of the Cartesian grid and preserves the structure of the coefficient matrix so that the fast Thomas algorithm can be applied to solve the linear system efficiently. Second-order accuracy and unconditional stability of the KFBI-ADI schemes are verified through several numerical tests for both the heat equation and a reaction-diffusion equation. For the Stefan problem, which is a free boundary problem of the heat equation, a level set method is incorporated into the ADI method to capture the time-dependent interface. Numerical examples for simulating 3D dendritic solidification phenomenons are also presented.

This paper proposes a Cartesian grid-based boundary integral method for efficiently and stably solving two representative moving interface problems, the Hele-Shaw flow and the Stefan problem. Elliptic and parabolic partial differential equations (PDEs) are reformulated into boundary integral equations and are then solved with the matrix-free generalized minimal residual (GMRES) method. The evaluation of boundary integrals is performed by solving equivalent and simple interface problems with finite difference methods, allowing the use of fast PDE solvers, such as fast Fourier transform (FFT) and geometric multigrid methods. The interface curve is evolved utilizing the $\theta-L$ variables instead of the more commonly used $x-y$ variables. This choice simplifies the preservation of mesh quality during the interface evolution. In addition, the $\theta-L$ approach enables the design of efficient and stable time-stepping schemes to remove the stiffness that arises from the curvature term. Ample numerical examples, including simulations of complex viscous fingering and dendritic solidification problems, are presented to showcase the capability of the proposed method to handle challenging moving interface problems.

Different from traditional video cameras, event cameras capture asynchronous events stream in which each event encodes pixel location, trigger time, and the polarity of the brightness changes. In this paper, we introduce a novel graph-based framework for event cameras, namely SlideGCN. Unlike some recent graph-based methods that use groups of events as input, our approach can efficiently process data event-by-event, unlock the low latency nature of events data while still maintaining the graph's structure internally. For fast graph construction, we develop a radius search algorithm, which better exploits the partial regular structure of event cloud against k-d tree based generic methods. Experiments show that our method reduces the computational complexity up to 100 times with respect to current graph-based methods while keeping state-of-the-art performance on object recognition. Moreover, we verify the superiority of event-wise processing with our method. When the state becomes stable, we can give a prediction with high confidence, thus making an early recognition. Project page: \url{https://zju3dv.github.io/slide_gcn/}., Comment: Accepted to ICCV 2021. Project Page: https://zju3dv.github.io/slide_gcn

Ultrasound (US) imaging is indispensable in clinical practice. To diagnose certain diseases, sonographers must observe corresponding dynamic anatomic structures to gather comprehensive information. However, the limited availability of specific US video cases causes teaching difficulties in identifying corresponding diseases, which potentially impacts the detection rate of such cases. The synthesis of US videos may represent a promising solution to this issue. Nevertheless, it is challenging to accurately animate the intricate motion of dynamic anatomic structures while preserving image fidelity. To address this, we present a novel online feature-decoupling framework called OnUVS for high-fidelity US video synthesis. Our highlights can be summarized by four aspects. First, we introduced anatomic information into keypoint learning through a weakly-supervised training strategy, resulting in improved preservation of anatomical integrity and motion while minimizing the labeling burden. Second, to better preserve the integrity and textural information of US images, we implemented a dual-decoder that decouples the content and textural features in the generator. Third, we adopted a multiple-feature discriminator to extract a comprehensive range of visual cues, thereby enhancing the sharpness and fine details of the generated videos. Fourth, we constrained the motion trajectories of keypoints during online learning to enhance the fluidity of generated videos. Our validation and user studies on in-house echocardiographic and pelvic floor US videos showed that OnUVS synthesizes US videos with high fidelity., Comment: 14 pages, 13 figures and 6 tables

Creating high-quality annotated data for task-oriented dialog (ToD) is known to be notoriously difficult, and the challenges are amplified when the goal is to create equitable, culturally adapted, and large-scale ToD datasets for multiple languages. Therefore, the current datasets are still very scarce and suffer from limitations such as translation-based non-native dialogs with translation artefacts, small scale, or lack of cultural adaptation, among others. In this work, we first take stock of the current landscape of multilingual ToD datasets, offering a systematic overview of their properties and limitations. Aiming to reduce all the detected limitations, we then introduce Multi3WOZ, a novel multilingual, multi-domain, multi-parallel ToD dataset. It is large-scale and offers culturally adapted dialogs in 4 languages to enable training and evaluation of multilingual and cross-lingual ToD systems. We describe a complex bottom-up data collection process that yielded the final dataset, and offer the first sets of baseline scores across different ToD-related tasks for future reference, also highlighting its challenging nature., Comment: A pre-MIT Press publication version for TACL

As technology scales into smaller feature sizes, electromigration (EM) has become a progressively severe reliability challenge due to increased interconnect current densities. It may lead to an increase in wire resistance thus resulting in functional failure of the system. In other words, the reliability of very large scale integrated (VLSI) circuits is increasingly endangered by EM. Therefore, to ensure reliable circuits, it is important to shift today’s design flow from the traditional EM verification towards an EM-aware design methodology.Since power grid wires experience the largest current flows on a chip, they are more susceptible to long-term reliability issues and functional failures. The interconnect wire in power grid networks usually contains multiple segments, which is a multi-segment wire. Generally, the wider the interconnect, the lower the current density and the greater the resistance to EM. On the other hand, during the power grid synthesis in a typical design flow, one important step is to size the wire width of the power grid stripes after the topology of the power supply network has been determined. The area of the power grids is expected to be optimized while EM and excessive IR drop constraints are met. In this dissertation, several EM-aware power grid wire sizing methods are presented to enhance EM reliability in the traditional optimization approaches and speed up the optimization process. Specifically, first, new power grid network sizing techniques based on the novel EM immortality check methods for general multi-segment interconnects have been proposed. Second, by taking EM aging effects into consideration, we have developed two EM lifetime constrained power grid optimization methods to address the overconservative EM immortality constraint. Furthermore, we have proposed a new data-driven fast EM-induced IR drop estimation framework to accelerate full-chip EM-induced IR drop analysis. Last but not least, we have presented novel approaches

As technology scales into smaller feature sizes, electromigration (EM) has become a progressively severe reliability challenge due to increased interconnect current densities. It may lead to an increase in wire resistance thus resulting in functional failure of the system. In other words, the reliability of very large scale integrated (VLSI) circuits is increasingly endangered by EM. Therefore, to ensure reliable circuits, it is important to shift today’s design flow from the traditional EM verification towards an EM-aware design methodology.Since power grid wires experience the largest current flows on a chip, they are more susceptible to long-term reliability issues and functional failures. The interconnect wire in power grid networks usually contains multiple segments, which is a multi-segment wire. Generally, the wider the interconnect, the lower the current density and the greater the resistance to EM. On the other hand, during the power grid synthesis in a typical design flow, one important step is to size the wire width of the power grid stripes after the topology of the power supply network has been determined. The area of the power grids is expected to be optimized while EM and excessive IR drop constraints are met. In this dissertation, several EM-aware power grid wire sizing methods are presented to enhance EM reliability in the traditional optimization approaches and speed up the optimization process. Specifically, first, new power grid network sizing techniques based on the novel EM immortality check methods for general multi-segment interconnects have been proposed. Second, by taking EM aging effects into consideration, we have developed two EM lifetime constrained power grid optimization methods to address the overconservative EM immortality constraint. Furthermore, we have proposed a new data-driven fast EM-induced IR drop estimation framework to accelerate full-chip EM-induced IR drop analysis. Last but not least, we have presented novel approaches

We propose the family universal $U(1)_X$ models with three right-handed neutrinos by choosing the $U(1)_X$ gauge symmetry as a linear combination of $U(1)_Y\times U(1)_{\chi}$ of $SO(10)$. To be consistent with the quantum gravity effects, we introduced a Dirac fermion $\chi$ as a dark matter candidate, which is odd under the gauged $Z_2$ symmetry after $U(1)_X$ breaking. The isospin violation dark matter with $f_n/f_p = -0.7$ can be realized naturally, and thus the LUX, PANDAX, and XENON1T experimental constraints can be evaded. Moreover, we study the masses and mixings for Higgs and gauge bosons, consider the LHC constraints on the $Z'$ mass, simulate variuous constraints from dark matter direct and indirect detection experiments, and then present the viable parameter spaces. Furthermore, we point out that the lightest sterile neutrino with mass around ${\cal O}(1-10)$ keV can be a viable DM candidate as well. To study the LHC $Z'$ mass bounds on the generic $U(1)_X$ models, we considered four kinds of Scenarios, where Scenario II with zero $U(1)_X$ charge for right-handed up-type quarks can relax the LHC $Z'$ mass bound a little bit., Comment: 19 pages, 6 figures

This dissertation presents different configurations of active Acoustic MetaMaterials (AMM) which are proposed in order to control the flow of vibration and acoustic wave propagations in various applications. Distinct among these configurations is a 1-dimensional (1D) periodic array which consists of an assembly of active acoustic unit cells which are provided with programmable piezoelectric elements. By tuning the structural properties of these cells, the 1D array can impede the wave propagation over specific frequency ranges. In order to achieve non-reciprocal acoustic wave transmission of the AMMs, three different methodologies are introduced including active control of the piezoelectric elements using virtual gyroscopic control actions, eigenstructure shaping controller, and finally spatial-temporal modulation algorithm.Theoretical models are developed to investigate the fundamentals and the underlying physical phenomena associated with all the considered three AMM configurations. Experimental prototypes of all these AMM configurations are built and tested to demonstrate their effectiveness in controlling the propagation of vibration and noise through these materials. Furthermore, the experimental results are used to validate the developed theoretical models. The developed theoretical and experimental approaches are envisioned to be valuable tools in the design of arrays of AMM for various applications which are only limited by our imagination.

Large pre-trained models have revolutionized natural language processing (NLP) research and applications, but high training costs and limited data resources have prevented their benefits from being shared equally amongst speakers of all the world's languages. To address issues of cross-linguistic access to such models and reduce energy consumption for sustainability during large-scale model training, this study proposes an effective and energy-efficient framework called GreenPLM that uses bilingual lexicons to directly "translate" pre-trained language models of one language into another at almost no additional cost. We validate this approach in 18 languages' BERT models and show that this framework is comparable to, if not better than, other heuristics with high training costs. In addition, given lightweight continued pre-training on limited data where available, this framework outperforms the original monolingual language models in six out of seven tested languages with up to 200x less pre-training efforts. Aiming at the Leave No One Behind Principle (LNOB), our approach manages to reduce inequalities between languages and energy consumption greatly. We make our codes and models publicly available here: \url{https://github.com/qcznlp/GreenPLMs}, Comment: Accepted at IJCAI 2023 AI and Social Good Track

Recently, the practical deployment of open-domain dialogue systems has been plagued by the knowledge issue of information deficiency and factual inaccuracy. To this end, we introduce PLATO-K based on two-stage dialogic learning to strengthen internal knowledge memorization and external knowledge exploitation. In the first stage, PLATO-K learns through massive dialogue corpora and memorizes essential knowledge into model parameters. In the second stage, PLATO-K mimics human beings to search for external information and to leverage the knowledge in response generation. Extensive experiments reveal that the knowledge issue is alleviated significantly in PLATO-K with such comprehensive internal and external knowledge enhancement. Compared to the existing state-of-the-art Chinese dialogue model, the overall engagingness of PLATO-K is improved remarkably by 36.2% and 49.2% on chit-chat and knowledge-intensive conversations., Comment: First four authors contributed equally to this work

Light-weight time-of-flight (ToF) depth sensors are small, cheap, low-energy and have been massively deployed on mobile devices for the purposes like autofocus, obstacle detection, etc. However, due to their specific measurements (depth distribution in a region instead of the depth value at a certain pixel) and extremely low resolution, they are insufficient for applications requiring high-fidelity depth such as 3D reconstruction. In this paper, we propose DELTAR, a novel method to empower light-weight ToF sensors with the capability of measuring high resolution and accurate depth by cooperating with a color image. As the core of DELTAR, a feature extractor customized for depth distribution and an attention-based neural architecture is proposed to fuse the information from the color and ToF domain efficiently. To evaluate our system in real-world scenarios, we design a data collection device and propose a new approach to calibrate the RGB camera and ToF sensor. Experiments show that our method produces more accurate depth than existing frameworks designed for depth completion and depth super-resolution and achieves on par performance with a commodity-level RGB-D sensor. Code and data are available at https://zju3dv.github.io/deltar/., Comment: Accepted to ECCV 2022. Project Page: https://zju3dv.github.io/deltar

Spatial-temporal Map (STMap)-based methods have shown great potential to process high-angle videos for vehicle trajectory reconstruction, which can meet the needs of various data-driven modeling and imitation learning applications. In this paper, we developed Spatial-Temporal Deep Embedding (STDE) model that imposes parity constraints at both pixel and instance levels to generate instance-aware embeddings for vehicle stripe segmentation on STMap. At pixel level, each pixel was encoded with its 8-neighbor pixels at different ranges, and this encoding is subsequently used to guide a neural network to learn the embedding mechanism. At the instance level, a discriminative loss function is designed to pull pixels belonging to the same instance closer and separate the mean value of different instances far apart in the embedding space. The output of the spatial-temporal affinity is then optimized by the mutex-watershed algorithm to obtain final clustering results. Based on segmentation metrics, our model outperformed five other baselines that have been used for STMap processing and shows robustness under the influence of shadows, static noises, and overlapping. The designed model is applied to process all public NGSIM US-101 videos to generate complete vehicle trajectories, indicating a good scalability and adaptability. Last but not least, the strengths of the scanline method with STDE and future directions were discussed. Code, STMap dataset and video trajectory are made publicly available in the online repository. GitHub Link: shorturl.at/jklT0.

High-throughput single-cell RNA sequencing (scRNA-seq) is a popular method, but it is accompanied by doublet rate problems that disturb the downstream analysis. Several computational approaches have been developed to detect doublets. However, most of these methods may yield satisfactory performance in some datasets but lack stability in others; thus, it is difficult to regard a single method as the gold standard which can be applied to all types of scenarios. It is a difficult and time-consuming task for researchers to choose the most appropriate software. We here propose Chord which implements a machine learning algorithm that integrates multiple doublet detection methods to address these issues. Chord had higher accuracy and stability than the individual approaches on different datasets containing real and synthetic data. Moreover, Chord was designed with a modular architecture port, which has high flexibility and adaptability to the incorporation of any new tools. Chord is a general solution to the doublet detection problem.

Dialogue State Tracking (DST), a crucial component of task-oriented dialogue (ToD) systems, keeps track of all important information pertaining to dialogue history: filling slots with the most probable values throughout the conversation. Existing methods generally rely on a predefined set of values and struggle to generalise to previously unseen slots in new domains. To overcome these challenges, we propose a domain-agnostic extractive question answering (QA) approach with shared weights across domains. To disentangle the complex domain information in ToDs, we train our DST with a novel domain filtering strategy by excluding out-of-domain question samples. With an independent classifier that predicts the presence of multiple domains given the context, our model tackles DST by extracting spans in active domains. Empirical results demonstrate that our model can efficiently leverage domain-agnostic QA datasets by two-stage fine-tuning while being both domain-scalable and open-vocabulary in DST. It shows strong transferability by achieving zero-shot domain-adaptation results on MultiWOZ 2.1 with an average JGA of 36.7%. It further achieves cross-lingual transfer with state-of-the-art zero-shot results, 66.2% JGA from English to German and 75.7% JGA from English to Italian on WOZ 2.0., Comment: Accepted to Findings of EACL 2023

Accurate segmentation of the blood vessels of the retina is an important step in clinical diagnosis of ophthalmic diseases. Many deep learning frameworks have come up for retinal blood vessels segmentation tasks. However, the complex vascular structure and uncertain pathological features make the blood vessel segmentation still very challenging. A novel U-shaped network named Multi-module Concatenation which is based on Atrous convolution and multi-kernel pooling is put forward to retinal vessels segmentation in this paper. The proposed network structure retains three layers the essential structure of U-Net, in which the atrous convolution combining the multi-kernel pooling blocks are designed to obtain more contextual information. The spatial attention module is concatenated with dense atrous convolution module and multi-kernel pooling module to form a multi-module concatenation. And different dilation rates are selected by cascading to acquire a larger receptive field in atrous convolution. Adequate comparative experiments are conducted on these public retinal datasets: DRIVE, STARE and CHASE_DB1. The results show that the proposed method is effective, especially for microvessels. The code will be put out at https://github.com/Rebeccala/MC-UNet, Comment: 13pages,3957

Making chatbots world aware in a conversation like a human is a crucial challenge, where the world may contain dynamic knowledge and spatiotemporal state. Several recent advances have tried to link the dialog system to a static knowledge base or search engine, but they do not contain all the world information needed for conversations. In contrast, we propose a new method to improve the dialogue system using spatiotemporal aware dynamic knowledge. We utilize service information as a way for the dialogue system to link the world. The system actively builds a request according to the dialog context and spatiotemporal state to get service information and then generates world aware responses. To implement this method, we collect DuSinc, an open-domain human-human dialogue dataset, where a participant can access the service to get the information needed for dialogue responses. Through automatic and human evaluations, we found that service information significantly improves the consistency, informativeness, factuality, and engagingness of the dialogue system, making it behave more like a human. Compared to the pre-trained models without spatiotemporal aware dynamic knowledge, the overall session-level score was improved by 60.87\%. The collection dataset and methods will be open-sourced., Comment: 13pages, 5 figures

High-throughput single-cell RNA sequencing (scRNA-seq) is a popular method, but it is accompanied by doublet rate problems that disturb the downstream analysis. Several computational approaches have been developed to detect doublets. However, most of these methods may yield satisfactory performance in some datasets but lack stability in others; thus, it is difficult to regard a single method as the gold standard which can be applied to all types of scenarios. It is a difficult and time-consuming task for researchers to choose the most appropriate software. We here propose Chord which implements a machine learning algorithm that integrates multiple doublet detection methods to address these issues. Chord had higher accuracy and stability than the individual approaches on different datasets containing real and synthetic data. Moreover, Chord was designed with a modular architecture port, which has high flexibility and adaptability to the incorporation of any new tools. Chord is a general solution to the doublet detection problem.

Low-bit width neural networks have been extensively explored for deployment on edge devices to reduce computational resources. Existing approaches have focused on gradient-based optimization in a two-stage train-and-compress setting or as a combined optimization where gradients are quantized during training. Such schemes require high-performance hardware during the training phase and usually store an equivalent number of full-precision weights apart from the quantized weights. In this paper, we explore methods of direct combinatorial optimization in the problem of risk minimization with binary weights, which can be made equivalent to a non-monotone submodular maximization under certain conditions. We employ an approximation algorithm for the cases with single and multilayer neural networks. For linear models, it has $\mathcal{O}(nd)$ time complexity where $n$ is the sample size and $d$ is the data dimension. We show that a combination of greedy coordinate descent and this novel approach can attain competitive accuracy on binary classification tasks.

Singular source terms in sub-diffusion equations may lead to the unboundedness of solutions, which will bring a severe reduction of convergence order of existing time-stepping schemes. In this work, we propose two efficient time-stepping schemes for solving sub-diffusion equations with a class of source terms mildly singular in time. One discretization is based on the Gr{\"u}nwald-Letnikov and backward Euler methods. First-order error estimate with respect to time is rigorously established for singular source terms and nonsmooth initial data. The other scheme derived from the second-order backward differentiation formula (BDF) is proved to possess second-order accuracy in time. Further, piecewise linear finite element and lumped mass finite element discretizations in space are applied and analyzed rigorously. Numerical investigations confirm our theoretical results.

Implicit neural rendering techniques have shown promising results for novel view synthesis. However, existing methods usually encode the entire scene as a whole, which is generally not aware of the object identity and limits the ability to the high-level editing tasks such as moving or adding furniture. In this paper, we present a novel neural scene rendering system, which learns an object-compositional neural radiance field and produces realistic rendering with editing capability for a clustered and real-world scene. Specifically, we design a novel two-pathway architecture, in which the scene branch encodes the scene geometry and appearance, and the object branch encodes each standalone object conditioned on learnable object activation codes. To survive the training in heavily cluttered scenes, we propose a scene-guided training strategy to solve the 3D space ambiguity in the occluded regions and learn sharp boundaries for each object. Extensive experiments demonstrate that our system not only achieves competitive performance for static scene novel-view synthesis, but also produces realistic rendering for object-level editing., Comment: Accepted to ICCV 2021. Project Page: https://zju3dv.github.io/object_nerf

Visual localization is of great importance in robotics and computer vision. Recently, scene coordinate regression based methods have shown good performance in visual localization in small static scenes. However, it still estimates camera poses from many inferior scene coordinates. To address this problem, we propose a novel visual localization framework that establishes 2D-to-3D correspondences between the query image and the 3D map with a series of learnable scene-specific landmarks. In the landmark generation stage, the 3D surfaces of the target scene are over-segmented into mosaic patches whose centers are regarded as the scene-specific landmarks. To robustly and accurately recover the scene-specific landmarks, we propose the Voting with Segmentation Network (VS-Net) to segment the pixels into different landmark patches with a segmentation branch and estimate the landmark locations within each patch with a landmark location voting branch. Since the number of landmarks in a scene may reach up to 5000, training a segmentation network with such a large number of classes is both computation and memory costly for the commonly used cross-entropy loss. We propose a novel prototype-based triplet loss with hard negative mining, which is able to train semantic segmentation networks with a large number of labels efficiently. Our proposed VS-Net is extensively tested on multiple public benchmarks and can outperform state-of-the-art visual localization methods. Code and models are available at \href{https://github.com/zju3dv/VS-Net}{https://github.com/zju3dv/VS-Net}., Comment: Project Page: https://drinkingcoder.github.io/publication/vs-net

In this work, we explore the application of PLATO-2 on various dialogue systems, including open-domain conversation, knowledge grounded dialogue, and task-oriented conversation. PLATO-2 is initially designed as an open-domain chatbot, trained via two-stage curriculum learning. In the first stage, a coarse-grained response generation model is learned to fit the simplified one-to-one mapping relationship. This model is applied to the task-oriented conversation, given that the semantic mappings tend to be deterministic in task completion. In the second stage, another fine-grained generation model and an evaluation model are further learned for diverse response generation and coherence estimation, respectively. With superior capability on capturing one-to-many mapping, such models are suitable for the open-domain conversation and knowledge grounded dialogue. For the comprehensive evaluation of PLATO-2, we have participated in multiple tasks of DSTC9, including interactive evaluation of open-domain conversation (Track3-task2), static evaluation of knowledge grounded dialogue (Track3-task1), and end-to-end task-oriented conversation (Track2-task1). PLATO-2 has obtained the 1st place in all three tasks, verifying its effectiveness as a unified framework for various dialogue systems., Comment: Presented at AAAI-21 DSTC9 Workshop. First five authors contributed equally to this work

Metamaterials with spectrally selective absorptance operating in the mid-infrared range have attracted much interest in numerous applications. However, it remains a challenge to economically fabricate scalable meta-absorbers with tailorable absorptance bands. This work demonstrates a conceptually simple and low-cost yet effective design strategy to achieve spectrally selective absorption with tailorable band positions at MIR by colloidal lithography. The strategy ingeniously uses residual diameter fluctuations of circular resonators etched through monodisperse colloidal particles for achieving superposition of multiple magnetic resonances and thereby a more than doubled absorption band, which is neglected in previous works. The proposed meta-absorber features densely packed thick aluminum resonators with a rather narrow diameter distribution and enhanced capacitive coupling among them. Moreover, the tailorability of the absorption band can be achieved by a parameterized variation in the fabrication process. As a proof of concept, infrared stealth and radiative cooling are demonstrated based on our meta-absorbers. The design and fabrication strategy create versatile metamaterials for advanced radiative thermal engineering., QC 20200915

Temperature is a fundamental parameter for all forms of lives. Natural evolution has resulted in organisms which have excellent thermoregulation capabilities in extreme climates. Bioinspired materials that mimic biological solution for thermoregulation have proven promising for passive radiative cooling. However, scalable production of artificial photonic radiators with complex structures, outstanding properties, high throughput, and low cost is still challenging. Herein, we design and demonstrate biologically inspired photonic materials for passive radiative cooling, after discovery of longicorn beetles' excellent thermoregulatory function with their dual-scale fluffs. The natural fluffs exhibit a finely structured triangular cross-section with two thermoregulatory effects which effectively reflects sunlight and emits thermal radiation, thereby decreasing the beetles' body temperature. Inspired by the finding, a photonic film consisting of a micropyramid-arrayed polymer matrix with random ceramic particles is fabricated with high throughput. The film reflects similar to 95% of solar irradiance and exhibits an infrared emissivity >0.96. The effective cooling power is found to be similar to 90.8 W center dot m(-2) and a temperature decrease of up to 5.1 degrees C is recorded under direct sunlight. Additionally, the film exhibits hydrophobicity, superior flexibility, and strong mechanical strength, which is promising for thermal management in various electronic devices and wearable products. Our work paves the way for designing and fabrication of high-performance thermal regulation materials., QC 20200914

This paper concentrates on the collaborative target tracking control of a pair of tracking vehicles with formation constraints. The proposed controller requires only distance measurements between tracking vehicles and the target. Its novelty lies in two aspects: 1) the elliptic coordinates are used to represent an arbitrary tracking formation without singularity, which can be deduced from inter-agent distances, and 2) the regulation of the tracking vehicle system obeys a binocular coordination principle, which simplifies the design of the control law by leveraging rich physical meanings of elliptic coordinates. The tracking system with the proposed controller is proven to be exponentially convergent when the target is stationary. When the target drifts with a small velocity, the desired tracking formation is achieved within a small margin proportional to the magnitude of the target's drift velocity. Simulation examples are provided to demonstrate the tracking performance of the proposed controller.

This paper concentrates on the collaborative target tracking control of a pair of tracking vehicles with formation constraints. The proposed controller requires only distance measurements between tracking vehicles and the target. Its novelty lies in two aspects: 1) the elliptic coordinates are used to represent an arbitrary tracking formation without singularity, which can be deduced from inter-agent distances, and 2) the regulation of the tracking vehicle system obeys a binocular coordination principle, which simplifies the design of the control law by leveraging rich physical meanings of elliptic coordinates. The tracking system with the proposed controller is proven to be exponentially convergent when the target is stationary. When the target drifts with a small velocity, the desired tracking formation is achieved within a small margin proportional to the magnitude of the target's drift velocity. Simulation examples are provided to demonstrate the tracking performance of the proposed controller.

In this paper, we propose a novel accuracy-reconfigurable stochastic computing (ARSC) framework for dynamic reliability and power management. Different than the existing stochastic computing works, where the accuracy versus power/energy trade-off is carried out in the design time, the new ARSC design can change accuracy or bit-width of the data in the run-time so that it can accommodate the long-term aging effects by slowing the system clock frequency at the cost of accuracy while maintaining the throughput of the computing. We validate the ARSC concept on a discrete cosine transformation (DCT) and inverse DCT designs for image compressing/decompressing applications, which are implemented on Xilinx Spartan-6 family XC6SLX45 platform. Experimental results shows that the new design can easily mitigate the long-term aging induced effects by accuracy trade-off while maintaining the throughput of the whole computing process using simple frequency scaling. We further show that one-bit precision loss for input data, which translated to 3.44dB of the accuracy loss in term of Peak Signal to Noise Ratio for images, we can sufficiently compensate the NBTI induced aging effects in 10 years while maintaining the pre-aging computing throughput of 7.19 frames per second. At the same time, we can save 74\% power consumption by 10.67dB of accuracy loss. The proposed ARSC computing framework also allows much aggressive frequency scaling, which can lead to order of magnitude power savings compared to the traditional dynamic voltage and frequency scaling (DVFS) techniques.

This paper presents a time-efficient scheme for Mars exploration by the cooperation of multiple drones and a rover. To maximize effective coverage of the Mars surface in the long run, a comprehensive framework has been developed with joint consideration for limited energy, sensor model, communication range and safety radius, which we call TIME-SC2 (TIme-efficient Mars Exploration of Simultaneous Coverage and Charging). First, we propose a multi-drone coverage control algorithm by leveraging emerging deep reinforcement learning and design a novel information map to represent dynamic system states. Second, we propose a near-optimal charging scheduling algorithm to navigate each drone to an individual charging slot, and we have proven that there always exists feasible solutions. The attractiveness of this framework not only resides on its ability to maximize exploration efficiency, but also on its high autonomy that has greatly reduced the non-exploring time. Extensive simulations have been conducted to demonstrate the remarkable performance of TIME-SC2 in terms of time-efficiency, adaptivity and flexibility.

This paper concentrates on the collaborative target tracking control of a pair of tracking vehicles with formation constraints. The proposed controller requires only distance measurements between tracking vehicles and the target. Its novelty lies in two aspects: 1) the elliptic coordinates are used to represent an arbitrary tracking formation without singularity, which can be deduced from inter-agent distances, and 2) the regulation of the tracking vehicle system obeys a binocular coordination principle, which simplifies the design of the control law by leveraging rich physical meanings of elliptic coordinates. The tracking system with the proposed controller is proven to be exponentially convergent when the target is stationary. When the target drifts with a small velocity, the desired tracking formation is achieved within a small margin proportional to the magnitude of the target's drift velocity. Simulation examples are provided to demonstrate the tracking performance of the proposed controller., Comment: 6 pages, 5 figures

We propose a $U(1)'$ model from $E_6$ which has an isospin-violation dark matter. By choosing a proper linear combination of two extra $U(1)$ gauge symmetries in $E_6$, it is natural to realize the ratio $f_n/f_p=-0.7$ so as to maximally relax the constraints from the Xenon based direct detection experiments. We study the sensitivities of the dark matter direct and indirect detection experiments, and identify the parameter spaces that can give the observed relic density. We also study the sensitivities of the future colliders with center mass energy $\sqrt{s}$= 33/50/100 TeV, and compare the different detection methods. We show that in some parameter spaces the future colliders can give much stronger limits., Comment: 16 pages, 4 figures