Search

Penalizing complexity (PC) priors is a principled framework for designing priors that reduce model complexity. PC priors penalize the Kullback-Leibler Divergence (KLD) between the distributions induced by a ``simple'' model and that of a more complex model. However, in many common cases, it is impossible to construct a prior in this way because the KLD is infinite. Various approximations are used to mitigate this problem, but the resulting priors then fail to follow the designed principles. We propose a new class of priors, the Wasserstein complexity penalization (WCP) priors, by replacing KLD with the Wasserstein distance in the PC prior framework. These priors avoid the infinite model distance issues and can be derived by following the principles exactly, making them more interpretable. Furthermore, principles and recipes to construct joint WCP priors for multiple parameters analytically and numerically are proposed and we show that they can be easily obtained, either numerically or analytically, for a general class of models. The methods are illustrated through several examples for which PC priors have previously been applied.

Generating instructional images of human daily actions from an egocentric viewpoint serves as a key step towards efficient skill transfer. In this paper, we introduce a novel problem -- egocentric action frame generation. The goal is to synthesize an image depicting an action in the user's context (i.e., action frame) by conditioning on a user prompt and an input egocentric image. Notably, existing egocentric action datasets lack the detailed annotations that describe the execution of actions. Additionally, existing diffusion-based image manipulation models are sub-optimal in controlling the state transition of an action in egocentric image pixel space because of the domain gap. To this end, we propose to Learn EGOcentric (LEGO) action frame generation via visual instruction tuning. First, we introduce a prompt enhancement scheme to generate enriched action descriptions from a visual large language model (VLLM) by visual instruction tuning. Then we propose a novel method to leverage image and text embeddings from the VLLM as additional conditioning to improve the performance of a diffusion model. We validate our model on two egocentric datasets -- Ego4D and Epic-Kitchens. Our experiments show substantial improvement over prior image manipulation models in both quantitative and qualitative evaluation. We also conduct detailed ablation studies and analysis to provide insights in our method. More details of the dataset and code are available on the website (https://bolinlai.github.io/Lego_EgoActGen/)., Comment: 34 pages

Spatial statistics is traditionally based on stationary models on $\mathbb{R^d}$ like Mat\'ern fields. The adaptation of traditional spatial statistical methods, originally designed for stationary models in Euclidean spaces, to effectively model phenomena on linear networks such as stream systems and urban road networks is challenging. The current study aims to analyze the incidence of traffic accidents on road networks using three different methodologies and compare the model performance for each methodology. Initially, we analyzed the application of spatial triangulation precisely on road networks instead of traditional continuous regions. However, this approach posed challenges in areas with complex boundaries, leading to the emergence of artificial spatial dependencies. To address this, we applied an alternative computational method to construct nonstationary barrier models. Finally, we explored a recently proposed class of Gaussian processes on compact metric graphs, the Whittle-Mat\'ern fields, defined by a fractional SPDE on the metric graph. The latter fields are a natural extension of Gaussian fields with Mat\'ern covariance functions on Euclidean domains to non-Euclidean metric graph settings. A ten-year period (2010-2019) of daily traffic-accident records from Barcelona, Spain have been used to evaluate the three models referred above. While comparing model performance we observed that the Whittle-Mat\'ern fields defined directly on the network outperformed the network triangulation and barrier models. Due to their flexibility, the Whittle-Mat\'ern fields can be applied to a wide range of environmental problems on linear networks such as spatio-temporal modeling of water contamination in stream networks or modeling air quality or accidents on urban road networks., Comment: 24 pages, 9 figures

Federated learning (FL) enables multiple participants to collaboratively train machine learning models using decentralized data sources, alleviating privacy concerns that arise from directly sharing local data. However, the lack of model privacy protection in FL becomes an unneglectable challenge, especially when people want to federally finetune models based on a proprietary large language model. In this study, we propose a novel FL training approach that accomplishes information exchange among participants via tunable soft prompts. These soft prompts, updated and transmitted between the server and clients, assume the role of the global model parameters and serve as messengers to deliver useful knowledge from the local data and global model. As the global model itself is not required to be shared and the local training is conducted based on an auxiliary model with fewer parameters than the global model, the proposed approach provides protection for the global model while reducing communication and computation costs in FL. Extensive experiments show the effectiveness of the proposed approach compared to several baselines. We have released the source code at \url{https://github.com/alibaba/FederatedScope/tree/fedsp/federatedscope/nlp/fedsp}., Comment: Accepted by EMNLP-23

Brain functional connectivity (FC), the temporal synchrony between brain networks, is essential to understand the functional organization in the brain and to identify changes due to neurological disorders, development, treatment, and other phenomena. Independent component analysis (ICA) is a matrix decomposition method used extensively for simultaneous estimation of functional brain topography and connectivity. However, estimation of FC via ICA is often sub-optimal due to the use of ad-hoc estimation methods or temporal dimension reduction prior to ICA. Bayesian ICA methods can avoid dimension reduction, produce more accurate estimates of latent variables and model parameters, and facilitate inference via posterior distributions. In this paper, we develop a novel, computationally feasible Bayesian ICA method with population-derived priors on both the spatial ICs and their temporal correlation. For the latter we consider two priors: the inverse-Wishart, which is designed for covariance matrices and has limitations for modeling correlation matrices; and a novel informative prior for correlation matrices. For both choices of prior, we derive a variational Bayes algorithm to estimate the model variables and obtain posterior variances or distributions of quantities of interest. Through extensive realistic simulation studies, we evaluate the performance of the proposed methods and compare them with existing approaches. Finally, we analyze fMRI data from over 400 healthy adults in the Human Connectome Project. We find that our Bayesian ICA algorithms produce highly accurate measures of functional connectivity and spatial brain features. Our informative prior for correlation matrices outperforms the inverse-Wishart, but comes with a higher computational burden. The proposed framework is applicable to single-subject analysis, making it potentially clinically viable., Comment: 24 pages with 5 figures, plus appendix

Generative Face Video Coding (GFVC) techniques can exploit the compact representation of facial priors and the strong inference capability of deep generative models, achieving high-quality face video communication in ultra-low bandwidth scenarios. This paper conducts a comprehensive survey on the recent advances of the GFVC techniques and standardization efforts, which could be applicable to ultra low bitrate communication, user-specified animation/filtering and metaverse-related functionalities. In particular, we generalize GFVC systems within one coding framework and summarize different GFVC algorithms with their corresponding visual representations. Moreover, we review the GFVC standardization activities that are specified with supplemental enhancement information messages. Finally, we discuss fundamental challenges and broad applications on GFVC techniques and their standardization potentials, as well as envision their future trends. The project page can be found at https://github.com/Berlin0610/Awesome-Generative-Face-Video-Coding.

In this paper, we explore FP8 low-bit data formats for efficient training of large language models (LLMs). Our key insight is that most variables, such as gradients and optimizer states, in LLM training can employ low-precision data formats without compromising model accuracy and requiring no changes to hyper-parameters. Specifically, we propose a new FP8 automatic mixed-precision framework for training LLMs. This framework offers three levels of FP8 utilization to streamline mixed-precision and distributed parallel training for LLMs. It gradually incorporates 8-bit gradients, optimizer states, and distributed learning in an incremental manner. Experiment results show that, during the training of GPT-175B model on H100 GPU platform, our FP8 mixed-precision training framework not only achieved a remarkable 39% reduction in real memory usage but also ran 75% faster than the widely adopted BF16 framework (i.e., Megatron-LM), surpassing the speed of Nvidia Transformer Engine by 37%. This largely reduces the training costs for large foundation models. Furthermore, our FP8 mixed-precision training methodology is generic. It can be seamlessly applied to other tasks such as LLM instruction tuning and reinforcement learning with human feedback, offering savings in fine-tuning expenses. Our FP8 low-precision training framework is open-sourced at {https://github.com/Azure/MS-AMP}{aka.ms/MS.AMP}.

The rotational properties of $\sim$10~m-scale asteroids are poorly understood with only a few measurements. Additionally, collisions or thermal recoil can spin their rotations to periods less than a few seconds obfuscating their study due to the observational cadence imposed by the long read-out times of charge-coupled device imagers. We present a method to measure the rotation periods of 10~m-scale asteroids using the target of opportunity capability of the Canada France Hawaii Telescope and its MegaCam imager by intentionally streaking their detections in single exposures when they are at their brightest. Periodic changes in brightness as small as $\sim$0.05 mag along the streak can be measured as short as a few seconds. Additionally, the streak photometry is taken in multiple g, r, and i filter exposures enabling the measurement of asteroid colours. The streak photometry method was tested on CFHT observations of three 10~m-scale asteroids, 2016 GE$_1$, 2016 CG$_{18}$, and 2016 EV$_{84}$. Our 3 targets are among the smallest known asteroids with measured rotation periods/colours having some of the shortest known rotation periods. We compare our rotation period and taxonomic results with independent data from the literature and discuss applications of the method to future small asteroid observations., Comment: Revised version, MNRAS:L, 13 pages, 10 figures, 3 tables

The objective of Entity Alignment (EA) is to identify equivalent entity pairs from multiple Knowledge Graphs (KGs) and create a more comprehensive and unified KG. The majority of EA methods have primarily focused on the structural modality of KGs, lacking exploration of multi-modal information. A few multi-modal EA methods have made good attempts in this field. Still, they have two shortcomings: (1) inconsistent and inefficient modality modeling that designs complex and distinct models for each modality; (2) ineffective modality fusion due to the heterogeneous nature of modalities in EA. To tackle these challenges, we propose PathFusion, consisting of two main components: (1) MSP, a unified modeling approach that simplifies the alignment process by constructing paths connecting entities and modality nodes to represent multiple modalities; (2) IRF, an iterative fusion method that effectively combines information from different modalities using the path as an information carrier. Experimental results on real-world datasets demonstrate the superiority of PathFusion over state-of-the-art methods, with 22.4%-28.9% absolute improvement on Hits@1, and 0.194-0.245 absolute improvement on MRR.

For efficient query processing, DBMS query optimizers have for decades relied on delicate cardinality estimation methods. In this work, we propose an Attention-based LEarned Cardinality Estimator (ALECE for short) for SPJ queries. The core idea is to discover the implicit relationships between queries and underlying dynamic data using attention mechanisms in ALECE's two modules that are built on top of carefully designed featurizations for data and queries. In particular, from all attributes in the database, the data-encoder module obtains organic and learnable aggregations which implicitly represent correlations among the attributes, whereas the query-analyzer module builds a bridge between the query featurizations and the data aggregations to predict the query's cardinality. We experimentally evaluate ALECE on multiple dynamic workloads. The results show that ALECE enables PostgreSQL's optimizer to achieve nearly optimal performance, clearly outperforming its built-in cardinality estimator and other alternatives., Comment: VLDB 2024

Long-period comets are planetesimal remnants constraining the environment and volatiles of the protoplanetary disc. We report the discovery of hyperbolic long-period comet C/2022 E3 (ZTF), which has a perihelion $\sim$1.11 au, an eccentricity $\gtrsim$1 and an inclination $\sim$109$^{\circ}$, from images taken with the Palomar 48-inch telescope during morning twilight on 2022 Mar 2. Additionally, we report the characterization of C/2022 E3 (ZTF) from observations taken with the Palomar 200-inch, the Palomar 60-inch, and the NASA Infrared Telescope Facility in early 2023 February to 2023 March when the comet passed within $\sim$0.28 au of the Earth and reached a visible magnitude of $\sim$5. We measure g-r = 0.70$\pm$0.01, r-i = 0.20$\pm$0.01, i-z = 0.06$\pm$0.01, z-J = 0.90$\pm$0.01, J-H = 0.38$\pm$0.01 and H-K = 0.15$\pm$0.01 colours for the comet from observations. We measure the A(0$^\circ$)f$\rho$ (0.8~$\mu$m) in a 6500~km radius from the nucleus of 1483$\pm$40~cm, and CN, C$_3$, and C$_2$ production of 5.43$\pm0.11\times$10$^{25}$~mol/s, 2.01$\pm0.04\times$10$^{24}$, and 3.08$\pm0.5\times$10$^{25}$~mol/s, similar to other long period comets. We additionally observe the appearance of jet-like structures at a scale of $\sim$4,000 km in wide-field g-band images, which may be caused by the presence of CN gas in the near-nucleus coma., Comment: Accepted for publication in MNRAS:L, 9 pages, 6 figures, 2 tables

In this letter, we envision a new metaverse communication paradigm for virtual avatar faces, and develop the semantic face compression with compact 3D facial descriptors. The fundamental principle is that the communication of virtual avatar faces primarily emphasizes the conveyance of semantic information. In light of this, the proposed scheme offers the advantages of being highly flexible, efficient and semantically meaningful. The semantic face compression, which allows the communication of the descriptors for artificial intelligence based understanding, could facilitate numerous applications without the involvement of humans in metaverse. The promise of the proposed paradigm is also demonstrated by performance comparisons with the state-of-the-art video coding standard, Versatile Video Coding. A significant improvement in terms of rate-accuracy performance has been achieved. The proposed scheme is expected to enable numerous applications, such as digital human communication based on machine analysis, and to form the cornerstone of interaction and communication in the metaverse., Comment: 5 pages, 3 figures

Traditional applications of latent class models (LCMs) often focus on scenarios where a set of unobserved classes are well-defined and easily distinguishable. However, in numerous real-world applications, these classes are weakly separated and difficult to distinguish, creating significant numerical challenges. To address these issues, we have developed an R package ddtlcm that provides comprehensive analysis and visualization tools designed to enhance the robustness and interpretability of LCMs in the presence of weak class separation, particularly useful for small sample sizes. This package implements a tree-regularized Bayesian LCM that leverages statistical strength between latent classes to make better estimates using limited data. A Shiny app has also been developed to improve user interactivity. In this paper, we showcase a typical analysis pipeline with simulated data using ddtlcm. All software has been made publicly available on CRAN and GitHub.

Before applying data analytics or machine learning to a data set, a vital step is usually the construction of an informative set of features from the data. In this paper, we present SMARTFEAT, an efficient automated feature engineering tool to assist data users, even non-experts, in constructing useful features. Leveraging the power of Foundation Models (FMs), our approach enables the creation of new features from the data, based on contextual information and open-world knowledge. Our method incorporates an intelligent operator selector that discerns a subset of operators, effectively avoiding exhaustive combinations of original features, as is typically observed in traditional automated feature engineering tools. Moreover, we address the limitations of performing data tasks through row-level interactions with FMs, which could lead to significant delays and costs due to excessive API calls. We introduce a function generator that facilitates the acquisition of efficient data transformations, such as dataframe built-in methods or lambda functions, ensuring the applicability of SMARTFEAT to generate new features for large datasets. Code repo with prompt details and datasets: (https://github.com/niceIrene/SMARTFEAT).

The immense evolution in Large Language Models (LLMs) has underscored the importance of massive, heterogeneous, and high-quality data. A data recipe is a mixture of data from different sources for training LLMs, which plays a vital role in LLMs' performance. Existing open-source tools for LLM data processing are mostly tailored for specific data recipes. To continuously uncover the potential of LLMs, incorporate data from new sources, and improve LLMs' performance, we build a new system named Data-Juicer, with which we can efficiently generate diverse data recipes, explore different possibilities in forming data mixtures, and evaluate their effects on model performance. Different from traditional data-analytics pipelines, Data-Juicer faces some unique challenges. Firstly, the possible data sources for forming data recipes are truly heterogeneous and massive with various qualities. Secondly, it is extremely expensive to precisely evaluate data recipes' impact on LLMs' performance. Thirdly, the end users of Data-Juicer, model developers, need sufficient flexibility to configure and evaluate different data recipes. Data-Juicer features a fine-grained abstraction of pipelines for constructing data recipes, with over 50 built-in operators for easy composition and extension. By incorporating visualization and auto-evaluation capabilities, Data-Juicer enables a timely feedback loop for both LLM pre-training and fine-tuning. Further, Data-Juicer is optimized and integrated with ecosystems for LLM training, evaluation, and distributed computing. The data recipes derived with Data-Juicer gain notable improvements on state-of-the-art LLMs, by up to 7.45% increase in averaged score across 16 LLM benchmarks and 17.5% higher win rate in pair-wise GPT-4 evaluations. Our system, data recipes, and tutorials are released, calling for broader data-centric research on training and understanding LLMs., Comment: 20 Pages, 10 figures, 9 tables. The system, data recipes, and demos are continuously maintained at https://github.com/alibaba/data-juicer

LLMs have demonstrated great capabilities in various NLP tasks. Different entities can further improve the performance of those LLMs on their specific downstream tasks by fine-tuning LLMs. When several entities have similar interested tasks, but their data cannot be shared because of privacy concerns regulations, federated learning (FL) is a mainstream solution to leverage the data of different entities. However, fine-tuning LLMs in federated learning settings still lacks adequate support from existing FL frameworks because it has to deal with optimizing the consumption of significant communication and computational resources, data preparation for different tasks, and distinct information protection demands. This paper first discusses these challenges of federated fine-tuning LLMs, and introduces our package FS-LLM as a main contribution, which consists of the following components: (1) we build an end-to-end benchmarking pipeline, automizing the processes of dataset preprocessing, federated fine-tuning execution, and performance evaluation on federated LLM fine-tuning; (2) we provide comprehensive federated parameter-efficient fine-tuning algorithm implementations and versatile programming interfaces for future extension in FL scenarios with low communication and computation costs, even without accessing the full model; (3) we adopt several accelerating and resource-efficient operators for fine-tuning LLMs with limited resources and the flexible pluggable sub-routines for interdisciplinary study. We conduct extensive experiments to validate the effectiveness of FS-LLM and benchmark advanced LLMs with state-of-the-art parameter-efficient fine-tuning algorithms in FL settings, which also yields valuable insights into federated fine-tuning LLMs for the research community. To facilitate further research and adoption, we release FS-LLM at https://github.com/alibaba/FederatedScope/tree/llm., Comment: Source code: https://github.com/alibaba/FederatedScope/tree/llm

Topological insulators and semimetals have been shown to possess intriguing thermoelectric properties promising for energy harvesting and cooling applications. However, thermoelectric transport associated with the Fermi arc topological surface states on topological Dirac semimetals remains less explored. In this work, we systematically examine thermoelectric transport in a series of topological Dirac semimetal Cd3As2 thin films grown by molecular beam epitaxy. Surprisingly, we find significantly enhanced Seebeck effect and anomalous Nernst effect at cryogenic temperatures when the Cd3As2 layer is thin. Combining angle-dependent quantum oscillation analysis, magnetothermoelectric measurement, transport modelling and first-principles simulation, we isolate the contributions from bulk and surface conducting channels and attribute the unusual thermoeletric properties to the topological surface states. Our analysis showcases the rich thermoelectric transport physics in quantum-confined topological Dirac semimetal thin films and suggests new routes to achieving high thermoelectric performance at cryogenic temperatures.