Search

Tabular data is the most common type of data in real-life scenarios. In this study, we propose the TabKANet model for tabular data modeling, which targets the bottlenecks in learning from numerical content. We constructed a Kolmogorov-Arnold Network (KAN) based Numerical Embedding Module and unified numerical and categorical features encoding within a Transformer architecture. TabKANet has demonstrated stable and significantly superior performance compared to Neural Networks (NNs) across multiple public datasets in binary classification, multi-class classification, and regression tasks. Its performance is comparable to or surpasses that of Gradient Boosted Decision Tree models (GBDTs). Our code is publicly available on GitHub: https://github.com/AI-thpremed/TabKANet., Comment: 13 pages,5 figures

Traversability estimation in off-road terrains is an essential procedure for autonomous navigation. However, creating reliable labels for complex interactions between the robot and the surface is still a challenging problem in learning-based costmap generation. To address this, we propose a method that predicts traversability costmaps by leveraging both visual and geometric information of the environment. To quantify the surface properties like roughness and bumpiness, we introduce a novel way of risk-aware labelling with proprioceptive information for network training. We validate our method in costmap prediction and navigation tasks for complex off-road scenarios. Our results demonstrate that our costmap prediction method excels in terms of average accuracy and MSE. The navigation results indicate that using our learned costmaps leads to safer and smoother driving, outperforming previous methods in terms of the highest success rate, lowest normalized trajectory length, lowest time cost, and highest mean stability across two scenarios.

A method for the generation of ultrarelativistic electron beams with high spin polarization is put forward, where a tightly-focused linearly-polarized ultraintense laser pulse interacts with a nonprepolarized transverse-size-tailored solid target. The radiative spin polarization and angular separation is facilitated by the standing wave formed via the incident and reflected laser pulses at the overdense plasma surface. Strong electron heating caused by transverse instability enhances photon emission in the density spikes injected into the standing wave near the surface. Two groups of electrons with opposite transverse polarization emerge, anti-aligned to the magnetic field, which are angularly separated in the standing wave due to the phase-matched oscillation of the magnetic field and the vector potential. The polarized electrons propelled into the plasma slab, are focused at the exit by the self-generated quasistatic fields. Our particle-in-cell simulations demonstrate the feasibility of highly polarized electrons with a single 10 PW laser beam, e.g. with polarization of 60% and charge of 8 pC selected at energy of 200 MeV within 15 mrad angle and 10% energy spread.

The extreme electric fields created in high-intensity laser-plasma interactions could generate energetic ions far more compactly than traditional accelerators. Despite this promise, laser-plasma accelerators have remained stagnant at maximum ion energies of 100 MeV/nucleon for the last twenty years. The central challenge is the low charge-to-mass ratio of ions, which has precluded one of the most successful approaches used for electrons: laser wakefield acceleration. Here we show that a laser pulse with a focal spot that moves transverse to the laser propagation direction enables wakefield acceleration of ions to GeV energies in underdense plasma. Three-dimensional particle-in-cell simulations demonstrate that this relativistic-intensity "transverse flying focus" can trap ions in a comoving electrostatic pocket, producing a monoenergetic collimated ion beam. With a peak intensity of $10^{20}\,$W/cm$^2$ and an acceleration distance of $0.44\,$cm, we observe a proton beam with $23.1\,$pC charge, $1.6\,$GeV peak energy, and $3.7\,$% relative energy spread. This approach allows for compact high-repetition-rate production of high-energy ions, highlighting the capability of more generalized spatio-temporal pulse shaping to address open problems in plasma physics., Comment: 11 pages, 6 figures

Fast and Safe Tracking (FaSTrack) is a modular framework that provides safety guarantees while planning and executing trajectories in real time via value functions of Hamilton-Jacobi (HJ) reachability. These value functions are computed through dynamic programming, which is notorious for being computationally inefficient. Moreover, the resulting trajectory does not adapt online to the environment, such as sudden disturbances or obstacles. DeepReach is a scalable deep learning method to HJ reachability that allows parameterization of states, which opens up possibilities for online adaptation to various controls and disturbances. In this paper, we propose Parametric FaSTrack, which uses DeepReach to approximate a value function that parameterizes the control bounds of the planning model. The new framework can smoothly trade off between the navigation speed and the tracking error (therefore maneuverability) while guaranteeing obstacle avoidance in a priori unknown environments. We demonstrate our method through two examples and a benchmark comparison with existing methods, showing the safety, efficiency, and faster solution times of the framework., Comment: 12 pages, 6 figures, 1 table, to be published in L4DC

Real-time navigation in a priori unknown environment remains a challenging task, especially when an unexpected (unmodeled) disturbance occurs. In this paper, we propose the framework Safe Returning Fast and Safe Tracking (SR-F) that merges concepts from 1) Robust Control Lyapunov-Value Functions (R-CLVF), and 2) the Fast and Safe Tracking (FaSTrack) framework. The SR-F computes an R-CLVF offline between a model of the true system and a simplified planning model. Online, a planning algorithm is used to generate a trajectory in the simplified planning space, and the R-CLVF is used to provide a tracking controller that exponentially stabilizes to the planning model. When an unexpected disturbance occurs, the proposed SR-F algorithm provides a means for the true system to recover to the planning model. We take advantage of this mechanism to induce an artificial disturbance by ``jumping'' the planning model in open environments, forcing faster navigation. Therefore, this algorithm can both reject unexpected true disturbances and accelerate navigation speed. We validate our framework using a 10D quadrotor system and show that SR-F is empirically 20\% faster than the original FaSTrack while maintaining safety., Comment: 6 pages, 4 figures, 1 table, 2 algorithms. Submitted to LCSS on 03/06

Control Lyapunov functions (CLFs) play a vital role in modern control applications, but finding them remains a problem. Recently, the control Lyapunov-value function (CLVF) and robust CLVF have been proposed as solutions for nonlinear time-invariant systems with bounded control and disturbance. However, the CLVF suffers from the ''curse of dimensionality,'' which hinders its application to practical high-dimensional systems. In this paper, we propose a method to decompose systems of a particular coupled nonlinear structure, in order to solve for the CLVF in each low-dimensional subsystem. We then reconstruct the full-dimensional CLVF and provide sufficient conditions for when this reconstruction is exact. Moreover, a point-wise optimal controller can be obtained using a quadratic program. We also show that when the exact reconstruction is impossible, the subsystems' CLVFs and their ``admissible control sets'' can be used to generate a Lipschitz continuous CLF. We provide several numerical examples to validate the theory and show computational efficiency., Comment: 7 pages, 4 figures, submitted to 63rd Conference on Decision and Control

Control Lyapunov Functions (CLFs) have been extensively used in the control community. A well-known drawback is the absence of a systematic way to construct CLFs for general nonlinear systems, and the problem can become more complex with input or state constraints. Our preliminary work on constructing Control Lyapunov Value Functions (CLVFs) using Hamilton-Jacobi (HJ) reachability analysis provides a method for finding a non-smooth CLF. In this paper, we extend our work on CLVFs to systems with bounded disturbance and define the Robust CLVF (R-CLVF). The R-CLVF naturally inherits all properties of the CLVF; i.e., it first identifies the "smallest robust control invariant set (SRCIS)" and stabilizes the system to it with a user-specified exponential rate. The region from which the exponential rate can be met is called the "region of exponential stabilizability (ROES)." We provide clearer definitions of the SRCIS and more rigorous proofs of several important theorems. Since the computation of the R-CLVF suffers from the "curse of dimensionality," we also provide two techniques (warmstart and system decomposition) that solve it, along with necessary proofs. Three numerical examples are provided, validating our definition of SRCIS, illustrating the trade-off between a faster decay rate and a smaller ROES, and demonstrating the efficiency of computation using warmstart and decomposition., Comment: 13 pages, 5 figures

Ensuring safety is important for the practical deployment of reinforcement learning (RL). Various challenges must be addressed, such as handling stochasticity in the environments, providing rigorous guarantees of persistent state-wise safety satisfaction, and avoiding overly conservative behaviors that sacrifice performance. We propose a new framework, Reachability Estimation for Safe Policy Optimization (RESPO), for safety-constrained RL in general stochastic settings. In the feasible set where there exist violation-free policies, we optimize for rewards while maintaining persistent safety. Outside this feasible set, our optimization produces the safest behavior by guaranteeing entrance into the feasible set whenever possible with the least cumulative discounted violations. We introduce a class of algorithms using our novel reachability estimation function to optimize in our proposed framework and in similar frameworks such as those concurrently handling multiple hard and soft constraints. We theoretically establish that our algorithms almost surely converge to locally optimal policies of our safe optimization framework. We evaluate the proposed methods on a diverse suite of safe RL environments from Safety Gym, PyBullet, and MuJoCo, and show the benefits in improving both reward performance and safety compared with state-of-the-art baselines., Comment: Accepted in NeurIPS 2023

Machine learning algorithms have become ubiquitous in a number of applications (e.g. image classification). However, due to the insufficient measurement of traditional metrics (e.g. the coarse-grained Accuracy of each classifier), substantial gaps are usually observed between the real-world performance of these algorithms and their scores in standardized evaluations. In this paper, inspired by the psychometric theories from human measurement, we propose a task-agnostic evaluation framework Camilla, where a multi-dimensional diagnostic metric Ability is defined for collaboratively measuring the multifaceted strength of each machine learning algorithm. Specifically, given the response logs from different algorithms to data samples, we leverage cognitive diagnosis assumptions and neural networks to learn the complex interactions among algorithms, samples and the skills (explicitly or implicitly pre-defined) of each sample. In this way, both the abilities of each algorithm on multiple skills and some of the sample factors (e.g. sample difficulty) can be simultaneously quantified. We conduct extensive experiments with hundreds of machine learning algorithms on four public datasets, and our experimental results demonstrate that Camilla not only can capture the pros and cons of each algorithm more precisely, but also outperforms state-of-the-art baselines on the metric reliability, rank consistency and rank stability.

Electron acceleration mechanisms near the counterstreaming interface of a relativistic collisionless shock (RCS) are investigated using particle-in-cell (PIC) simulations. We identify a slingshot-like injection process induced by the drifting electric field sustained by the flowing focus of backwards-moving electrons, which is distinct from the well-known stochastic acceleration. The flowing focus signifies the plasma kinetic transition from a preturbulent laminar motion to a chaotic turbulence. We find a characteristic correlation between the electron dynamics in the slingshot acceleration and the photon emission features. In particular, the integrated radiation from the RCS exhibits a counterintuitive non-monotonic dependence of the photon polarization degree on the photon energy, which originates from a polarization degradation of relatively high-energy photons emitted by the slingshot-injected electrons. Our results demonstrate the potential of photon polarization as an essential information source in exploring intricate dynamics in RCSs with relevance for earth-based plasma and astrophysical scenarios., Comment: 8 pages, 5 figures

Large multimodal language models (LMMs) have achieved significant success in general domains. However, due to the significant differences between medical images and text and general web content, the performance of LMMs in medical scenarios is limited. In ophthalmology, clinical diagnosis relies on multiple modalities of medical images, but unfortunately, multimodal ophthalmic large language models have not been explored to date. In this paper, we study and construct an ophthalmic large multimodal model. Firstly, we use fundus images as an entry point to build a disease assessment and diagnosis pipeline to achieve common ophthalmic disease diagnosis and lesion segmentation. Then, we establish a new ophthalmic multimodal instruction-following and dialogue fine-tuning dataset based on disease-related knowledge data and publicly available real-world medical dialogue. We introduce visual ability into the large language model to complete the ophthalmic large language and vision assistant (OphGLM). Our experimental results demonstrate that the OphGLM model performs exceptionally well, and it has the potential to revolutionize clinical applications in ophthalmology. The dataset, code, and models will be made publicly available at https://github.com/ML-AILab/OphGLM., Comment: OphGLM:The first ophthalmology large language-and-vision assistant based on instructions and dialogue

Although pre-trained language models~(PLMs) have recently advanced the research progress in mathematical reasoning, they are not specially designed as a capable multi-task solver, suffering from high cost for multi-task deployment (\eg a model copy for a task) and inferior performance on complex mathematical problems in practical applications. To address these issues, in this paper, we propose \textbf{JiuZhang~2.0}, a unified Chinese PLM specially for multi-task mathematical problem solving. Our idea is to maintain a moderate-sized model and employ the \emph{cross-task knowledge sharing} to improve the model capacity in a multi-task setting. Specially, we construct a Mixture-of-Experts~(MoE) architecture for modeling mathematical text, so as to capture the common mathematical knowledge across tasks. For optimizing the MoE architecture, we design \emph{multi-task continual pre-training} and \emph{multi-task fine-tuning} strategies for multi-task adaptation. These training strategies can effectively decompose the knowledge from the task data and establish the cross-task sharing via expert networks. In order to further improve the general capacity of solving different complex tasks, we leverage large language models~(LLMs) as complementary models to iteratively refine the generated solution by our PLM, via in-context learning. Extensive experiments have demonstrated the effectiveness of our model., Comment: Accepted by KDD 2023 ADS track, the 2.0 version of JiuZhang (arxiv:2206.06315v1)

Although large language models (LLMs) have achieved excellent performance in a variety of evaluation benchmarks, they still struggle in complex reasoning tasks which require specific knowledge and multi-hop reasoning. To improve the reasoning abilities, we propose ChatCoT, a tool-augmented chain-of-thought reasoning framework for chat-based LLMs (e.g., ChatGPT). In ChatCoT, we model the chain-of-thought (CoT) reasoning as multi-turn conversations, to utilize tools in a more natural way through chatting. At each turn, LLMs can either interact with tools or perform the reasoning. Our approach can effectively leverage the multi-turn conversation ability of chat-based LLMs, and integrate the thought chain following and tools manipulation in a unified way. Specially, we initialize the early turns of the conversation by the knowledge about tools, tasks, and reasoning format, and propose an iterative tool-augmented reasoning step to perform step-by-step tool-augmented reasoning. The experiment results on two complex reasoning datasets (MATH and HotpotQA) have shown the effectiveness of ChatCoT on complex reasoning tasks, achieving a 7.9% relative improvement over the state-of-the-art baseline. Our code and data are available at: \url{https://github.com/RUCAIBOX/ChatCoT}., Comment: 14 pages, working in progress, Findings of EMNLP 2023

Frequency conversion of dissipative solitons associated with the generation of broadband optical frequency combs having a tooth spacing of hundreds of giga-hertz is a topical challenge holding the key to practical applications in precision spectroscopy and data processing. The work in this direction is underpinned by fundamental problems in nonlinear and quantum optics. Here, we present the dissipative two-colour bright-bright and dark-dark solitons in a quasi-phase-matched microresonator pumped for the second-harmonic generation in the near-infrared spectral range. We also found the breather states associated with the pulse front motion and collisions. The soliton regime is found to be typical in slightly phase-mismatched resonators, while the phase-matched ones reveal broader but incoherent spectra and higher-order harmonic generation. Soliton and breather effects reported here exist for the negative tilt of the resonance line, which is possible only via the dominant contribution of second-order nonlinearity., Comment: 6 figures

Highly polarized, multi-kiloampere-current electron bunches from compact laser-plasma accelerators are desired for numerous applications. Current proposals to produce these beams suffer from intrinsic limitations to the reproducibility, charge, beam shape and final polarization degree. In this Letter, we propose colliding pulse injection as a technique for the generation of highly polarized electron bunches from pre-polarized plasma sources. Using particle-in-cell simulations, we show that colliding pulse injection enables trapping and precise control over electron spin evolution, resulting in the generation of high-current (multi-kA) electron bunches with high degrees of polarization (up to 95 % for > 2 kA). Bayesian optimization is employed to optimize the multidimensional parameter space associated with colliding pulse injection to obtain percent-level energy spread, sub-micron normalized emittance electron bunches with 90 % polarization using 100-TW class laser systems., Comment: 5 pages, 4 figures

Employing colliding-pulse injection has been shown to enable high-quality electron beams to be generated from laser-plasma accelerators. Here by leveraging test particle simulations, Hamiltonian analysis, and multidimensional particle-in-cell (PIC) simulations, we lay the theoretical framework of spin-polarized electron beam generation in the colliding-pulse injection scheme. Furthermore, we show that this scheme enables the production of quasi-monoenergetic electron beams in excess of 80\% polarization and tens pC charge with commercial 10-TW-class laser systems., Comment: 11 pages, 10 figures

3D Semantic Scene Completion (SSC) can provide dense geometric and semantic scene representations, which can be applied in the field of autonomous driving and robotic systems. It is challenging to estimate the complete geometry and semantics of a scene solely from visual images, and accurate depth information is crucial for restoring 3D geometry. In this paper, we propose the first stereo SSC method named OccDepth, which fully exploits implicit depth information from stereo images (or RGBD images) to help the recovery of 3D geometric structures. The Stereo Soft Feature Assignment (Stereo-SFA) module is proposed to better fuse 3D depth-aware features by implicitly learning the correlation between stereo images. In particular, when the input are RGBD image, a virtual stereo images can be generated through original RGB image and depth map. Besides, the Occupancy Aware Depth (OAD) module is used to obtain geometry-aware 3D features by knowledge distillation using pre-trained depth models. In addition, a reformed TartanAir benchmark, named SemanticTartanAir, is provided in this paper for further testing our OccDepth method on SSC task. Compared with the state-of-the-art RGB-inferred SSC method, extensive experiments on SemanticKITTI show that our OccDepth method achieves superior performance with improving +4.82% mIoU, of which +2.49% mIoU comes from stereo images and +2.33% mIoU comes from our proposed depth-aware method. Our code and trained models are available at https://github.com/megvii-research/OccDepth.

Free-electron radiation offers an enticing route to create light emission at arbitrary spectral regime. However, this type of light emission is generally weak, which is intrinsically limited by the weak particle-matter interaction and unavoidably impedes the development of many promising applications, such as the miniaturization of free-electron radiation sources and high-energy particle detectors. Here we reveal a mechanism to enhance the particle-matter interaction by exploiting the pseudo-Brewster effect of gain materials - presenting an enhancement of at least four orders of magnitude for the light emission. This mechanism is enabled by the emergence of an unprecedented phase diagram that maps all phenomena of free-electron radiation into three distinct phases in a gain-thickness parameter space, namely the conventional, intermediate, and Brewster phases, when an electron penetrates a dielectric slab with a modest gain and a finite thickness. Essentially, our revealed mechanism corresponds to the free-electron radiation in the Brewster phase, which also uniquely features ultrahigh directionality, always at the Brewster angle, regardless of the electron velocity. Counterintuitively, we find that the intensity of this free-electron Brewster radiation is insensitive to the Fabry-Perot resonance condition and thus the variation of slab thickness, and moreover, a weaker gain could lead to a stronger enhancement for the light emission. The scheme of free-electron Brewster radiation, especially along with its compatibility with low-energy electrons, may enable the development of high-directionality high-intensity light sources at any frequency.

Understanding mathematical questions effectively is a crucial task, which can benefit many applications, such as difficulty estimation. Researchers have drawn much attention to designing pre-training models for question representations due to the scarcity of human annotations (e.g., labeling difficulty). However, unlike general free-format texts (e.g., user comments), mathematical questions are generally designed with explicit purposes and mathematical logic, and usually consist of more complex content, such as formulas, and related mathematical knowledge (e.g., Function). Therefore, the problem of holistically representing mathematical questions remains underexplored. To this end, in this paper, we propose a novel contrastive pre-training approach for mathematical question representations, namely QuesCo, which attempts to bring questions with more similar purposes closer. Specifically, we first design two-level question augmentations, including content-level and structure-level, which generate literally diverse question pairs with similar purposes. Then, to fully exploit hierarchical information of knowledge concepts, we propose a knowledge hierarchy-aware rank strategy (KHAR), which ranks the similarities between questions in a fine-grained manner. Next, we adopt a ranking contrastive learning task to optimize our model based on the augmented and ranked questions. We conduct extensive experiments on two real-world mathematical datasets. The experimental results demonstrate the effectiveness of our model., Comment: Accepted by AAAI 2023

Transition radiation is a fundamental process of light emission and occurs whenever a charged particle moves across an inhomogeneous region. One feature of transition radiation is that it can create light emission at arbitrary frequency under any particle velocity. Therefore, transition radiation is of significant importance to both fundamental science and practical applications. In this paper, we provide a brief historical review of transition radiation and its recent development. Moreover, we pay special attention to four typical applications of transition radiation, namely the detection of high-energy particles, coherent radiation sources, beam diagnosis, and excitation of surface waves. Finally, we give an outlook for the research tendency of transition radiation, especially its flexible manipulation by exploiting artificially-engineered materials and nanostructures, such as gain materials, metamaterials, spatial-temporal materials, meta-boundaries, and layered structures with a periodic or non-periodic stacking.

When a charged particle penetrates through an optical interface, photon emissions emerge - a phenomenon known as transition radiation. Being paramount to fundamental physics, transition radiation has enabled many applications from high-energy particle identification to novel light sources. A rule of thumb in transition radiation is that the radiation intensity generally decreases with the particle velocity v; as a result, low-energy particles are not favored in practice. Here we find that there exist situations where transition radiation from particles with extremely low velocities (e.g. v/c<0.001) exhibits comparable intensity as that from high-energy particles (e.g. v/c=0.999), where c is light speed in free space. The comparable radiation intensity implies an extremely high photon extraction efficiency from low-energy particles, up to eight orders of magnitude larger than that from high-energy particles. This exotic phenomenon of low-velocity-favored transition radiation originates from the excitation of Ferrell-Berreman modes in epsilon-near-zero materials. Our findings may provide a promising route towards the design of integrated light sources based on low-energy electrons and specialized detectors for beyond-standard-model particles., Comment: 13 pages, 4 figures

Plasma current filamentation of an ultrarelativistic electron beam impinging on an overdense plasma is investigated, with emphasis on radiation-induced electron polarization. Particle-in-cell simulations provide the classification and in-depth analysis of three different regimes of the current filaments, namely, the normal filament, abnormal filament, and quenching regimes. We show that electron radiative polarization emerges during the instability along the azimuthal direction in the momentum space, which significantly varies across the regimes. We put forward an intuitive Hamiltonian model to trace the origin of the electron polarization dynamics. In particular, we discern the role of nonlinear transverse motion of plasma filaments, which induces asymmetry in radiative spin flips, yielding an accumulation of electron polarization. Our results break the conventional perception that quasi-symmetric fields are inefficient for generating radiative spin-polarized beams, suggesting the potential of electron polarization as a source of new information on laboratory and astrophysical plasma instabilities., Comment: 7 pages, 5 figures

In financial terms, an implied volatility surface can be described by its term structure, its skewness and its overall volatility level. We use a PCA variational auto-encoder model to perfectly represent these descriptors into a latent space of three dimensions. Our new encoding brings significant benefits for synthetic surface generation, in that (i) scenario generation is more interpretable; (ii) volatility extrapolation achieve better accuracy; and, (iii) we propose a solution to infer implied volatility surfaces of a stock from an index to which it belongs directly by modelling their relationship on the latent space of the encoding. All these applications, and the latter in particular, have the potential to improve risk management of financial derivatives whenever data is scarce.