Search

Large Language Models for Code (LLMs4Code) have been found to exhibit outstanding performance in the software engineering domain, especially the remarkable performance in coding tasks. However, even the most advanced LLMs4Code can inevitably contain incorrect or outdated code knowledge. Due to the high cost of training LLMs4Code, it is impractical to re-train the models for fixing these problematic code knowledge. Model editing is a new technical field for effectively and efficiently correcting erroneous knowledge in LLMs, where various model editing techniques and benchmarks have been proposed recently. Despite that, a comprehensive study that thoroughly compares and analyzes the performance of the state-of-the-art model editing techniques for adapting the knowledge within LLMs4Code across various code-related tasks is notably absent. To bridge this gap, we perform the first systematic study on applying state-of-the-art model editing approaches to repair the inaccuracy of LLMs4Code. To that end, we introduce a benchmark named CLMEEval, which consists of two datasets, i.e., CoNaLa-Edit (CNLE) with 21K+ code generation samples and CodeSearchNet-Edit (CSNE) with 16K+ code summarization samples. With the help of CLMEEval, we evaluate six advanced model editing techniques on three LLMs4Code: CodeLlama (7B), CodeQwen1.5 (7B), and Stable-Code (3B). Our findings include that the external memorization-based GRACE approach achieves the best knowledge editing effectiveness and specificity (the editing does not influence untargeted knowledge), while generalization (whether the editing can generalize to other semantically-identical inputs) is a universal challenge for existing techniques. Furthermore, building on in-depth case analysis, we introduce an enhanced version of GRACE called A-GRACE, which incorporates contrastive learning to better capture the semantics of the inputs., Comment: Accepted by ICSE2025. The code is available at: https://github.com/xpq-tech/code-llmedit.git

Long-term stability stands as a crucial requirement in data-driven medium-range global weather forecasting. Spectral bias is recognized as the primary contributor to instabilities, as data-driven methods difficult to learn small-scale dynamics. In this paper, we reveal that the universal mechanism for these instabilities is not only related to spectral bias but also to distortions brought by processing spherical data using conventional convolution. These distortions lead to a rapid amplification of errors over successive long-term iterations, resulting in a significant decline in forecast accuracy. To address this issue, a universal neural operator called the Spherical Harmonic Neural Operator (SHNO) is introduced to improve long-term iterative forecasts. SHNO uses the spherical harmonic basis to mitigate distortions for spherical data and uses gated residual spectral attention (GRSA) to correct spectral bias caused by spurious correlations across different scales. The effectiveness and merit of the proposed method have been validated through its application for spherical Shallow Water Equations (SWEs) and medium-range global weather forecasting. Our findings highlight the benefits and potential of SHNO to improve the accuracy of long-term prediction.

Here we consider the following fractional Hamiltonian system \begin{equation*} \begin{cases} \begin{aligned} (-\Delta)^{s} u&=H_v(u,v) \;\;&&\text{in}~\Omega,\\ (-\Delta)^{s} v&=H_u(u,v) &&\text{in}~\Omega,\\ u &= v = 0 &&\text{in} ~ \mathbb{R}^N\setminus\Omega, \end{aligned} \end{cases} \end{equation*} where $s\in (0,1)$, $N>2s$, $H \in C^1(\mathbb{R}^2, \mathbb{R})$ and $\Omega \subset \mathbb{R}^N$ is a smooth bounded domain. %As the problem remains unchanged if $H(u, v)$ is replaced by $H(u, v)-H(0, 0)$, hence we always assume $H(0,0)=0$. To apply the variational method for this problem, the key question is to find a suitable functional setting. Instead of usual fractional Sobolev spaces, we use the solutions space of $(-\Delta)^{s}u=f\in L^r(\Omega)$ for $r\ge 1$, for which we show the (compact) embedding properties. When $H$ has subcritical and superlinear growth, we construct two frameworks, respectively with interpolation space method and dual method, to show the existence of nontrivial solution. As byproduct, we revisit the fractional Lane-Emden system, i.e. $H(u, v)=\frac{1}{p+1}|u|^{p+1}+\frac{1}{q+1}|v|^{q+1}$, and consider the existence, uniqueness of (radial) positive solutions under subcritical assumption.

To our knowledge, this paper is the first attempt to consider the stable solution and extremal solution for the fractional $p$-Laplacian equation: $(-\Delta)_p^s u= \lambda f(u),\; u> 0 ~\text{in}~\Omega;\; u=0\;\text{in}~ \mathbb{R}^N\setminus\Omega$, where $p>1$, $s\in (0,1)$, $N>sp$, $\lambda>0$ and $\Omega$ is a bounded domain with continuous boundary. We first construct the notion of stable solution, and then we prove that when $f$ is of class $C^1$, nondecreasing and such that $f(0)>0$ and $\underset{t\to \infty}{\lim}\frac{f(t)}{t^{p-1}}=\infty$, there exists an extremal parameter $\lambda^*\in (0, \infty)$ such that a bounded minimal solution $u_\lambda$ exists if $\lambda\in (0, \lambda^*)$, and no bounded solution exists if $\lambda>\lambda^*$, no $W_0^{s,p}(\Omega)$ solution exists if in addition $f(t)^{\frac{1}{p-1}}$ is convex. Moreover, this family of minimal solutions are stable, and nondecreasing in $\lambda$, therefore the extremal function $u^*:=\underset{\lambda\to\lambda^*}{\lim}u_\lambda$ exists. For the regularity of the extremal function, we first show the $L^r$-estimates for the equation $(-\Delta)_p^su=g$ with $g\in W_0^{s, p}(\Omega)^*\cap L^q(\Omega)$, $q\ge 1$. When $f$ is a power-like nonlinearity, we derive the $W_0^{s,p}(\Omega)$ regularity of $u^*$ in all dimension and $L^{\infty}(\Omega)$ regularity of $u^*$ in some low dimensions. For more general nonlinearities, when $f$ is class of $C^2$ and such that some convexity assumptions, then $u^*\in W_0^{s,p}(\Omega)$ if $N\frac{p-2}{p-1}$, the results above can be improved as: $u^*\in W_0^{s,p}(\Omega)$ for all dimensions, and $u^*\in L^{\infty}(\Omega)$ if $N

The leading operational Global Ocean Forecasting Systems (GOFSs) use physics-driven numerical forecasting models that solve the partial differential equations with expensive computation. Recently, specifically in atmosphere weather forecasting, data-driven models have demonstrated significant potential for speeding up environmental forecasting by orders of magnitude, but there is still no data-driven GOFS that matches the forecasting accuracy of the numerical GOFSs. In this paper, we propose the first data-driven 1/12{\deg} resolution global ocean eddy-resolving forecasting model named XiHe, which is established from the 25-year France Mercator Ocean International's daily GLORYS12 reanalysis data. XiHe is a hierarchical transformer-based framework coupled with two special designs. One is the land-ocean mask mechanism for focusing exclusively on the global ocean circulation. The other is the ocean-specific block for effectively capturing both local ocean information and global teleconnection. Extensive experiments are conducted under satellite observations, in situ observations, and the IV-TT Class 4 evaluation framework of the world's leading operational GOFSs from January 2019 to December 2020. The results demonstrate that XiHe achieves stronger forecast performance in all testing variables than existing leading operational numerical GOFSs including Mercator Ocean Physical SYstem (PSY4), Global Ice Ocean Prediction System (GIOPS), BLUElinK OceanMAPS (BLK), and Forecast Ocean Assimilation Model (FOAM). Particularly, the accuracy of ocean current forecasting of XiHe out to 60 days is even better than that of PSY4 in just 10 days. Additionally, XiHe is able to forecast the large-scale circulation and the mesoscale eddies. Furthermore, it can make a 10-day forecast in only 0.35 seconds, which accelerates the forecast speed by thousands of times compared to the traditional numerical GOFSs.

The general capabilities of large language models (LLMs) make them the infrastructure for various AI applications, but updating their inner knowledge requires significant resources. Recent model editing is a promising technique for efficiently updating a small amount of knowledge of LLMs and has attracted much attention. In particular, local editing methods, which directly update model parameters, are more suitable for updating a small amount of knowledge. Local editing methods update weights by computing least squares closed-form solutions and identify edited knowledge by vector-level matching in inference, which achieve promising results. However, these methods still require a lot of time and resources to complete the computation. Moreover, vector-level matching lacks reliability, and such updates disrupt the original organization of the model's parameters. To address these issues, we propose an detachable and expandable Subject Word Embedding Altering (SWEA) framework, which finds the editing embeddings through token-level matching and adds them to the subject word embeddings in Transformer input. To get these editing embeddings, we propose optimizing then suppressing fusion method, which first optimizes learnable embedding vectors for the editing target and then suppresses the Knowledge Embedding Dimensions (KEDs) to obtain final editing embeddings. We thus propose SWEA$\oplus$OS method for editing factual knowledge in LLMs. We demonstrate the overall state-of-the-art (SOTA) performance of SWEA$\oplus$OS on the \textsc{CounterFact} and zsRE datasets. To further validate the reasoning ability of SWEA$\oplus$OS in editing knowledge, we evaluate it on the more complex \textsc{RippleEdits} benchmark. The results demonstrate that SWEA$\oplus$OS possesses SOTA reasoning ability., Comment: Under review; Our code is available at https://github.com/xpq-tech/SWEA

In this paper, we consider the existence of positive solutions with prescribed $L^2$-norm for the following nonlinear Schr\"{o}dinger equation involving potential and Sobolev critical exponent \begin{equation*} \begin{cases} -\Delta u+V(x)u=\lambda u+\mu |u|^{p-2}u+|u|^{\frac{4}{N-2}}u \;\;\text { in } \mathbb{R}^N, \\ \|u\|_2=a>0,\\ \end{cases} \end{equation*} where $N\ge 3$, $\mu>0$, $p\in [2+\frac{4}{N}, \frac{2N}{N-2})$ and $V\in C^1(\mathbb{R}^N)$. Under different assumptions on $V$, we derive two different Pohozaev identities. Based on these two cases, we respectively obtain the existence of positive solution. As far as we are aware, we did not find any works on normalized solutions with Sobolev critical growth and potential $V \not\equiv 0$. Our results extend some results of Wei and Wu [J. Funct. Anal. 283(2022)] to the potential case.

This review paper explores Multimodal Large Language Models (MLLMs), which integrate Large Language Models (LLMs) like GPT-4 to handle multimodal data such as text and vision. MLLMs demonstrate capabilities like generating image narratives and answering image-based questions, bridging the gap towards real-world human-computer interactions and hinting at a potential pathway to artificial general intelligence. However, MLLMs still face challenges in processing the semantic gap in multimodality, which may lead to erroneous generation, posing potential risks to society. Choosing the appropriate modality alignment method is crucial, as improper methods might require more parameters with limited performance improvement. This paper aims to explore modality alignment methods for LLMs and their existing capabilities. Implementing modality alignment allows LLMs to address environmental issues and enhance accessibility. The study surveys existing modal alignment methods in MLLMs into four groups: (1) Multimodal Converters that change data into something LLMs can understand; (2) Multimodal Perceivers to improve how LLMs perceive different types of data; (3) Tools Assistance for changing data into one common format, usually text; and (4) Data-Driven methods that teach LLMs to understand specific types of data in a dataset. This field is still in a phase of exploration and experimentation, and we will organize and update various existing research methods for multimodal information alignment.

We investigate the following fractional $p$-Laplacian equation \[ \begin{cases} \begin{aligned} (-\Delta)_p^s u&=\lambda |u|^{q-2}u+|u|^{p_s^*-2}u &&\text{in}~\Omega,\\ u &=0 &&\text{in}~ \mathbb{R}^n\setminus\Omega, \end{aligned} \end{cases} \] where $s\in (0,1)$, $p>q>1$, $n>sp$, $\lambda>0$, $p_s^*=\frac{np}{n-sp}$ and $\Omega$ is a bounded domain (with $C^{1, 1}$ boundary). Firstly, we get a dichotomy result for the existence of positive solution with respect to $\lambda$. For $p\ge 2$, $p-1\frac{sp(q+1)}{q+1-p}$, we provide two positive solutions for small $\lambda$. Finally, without sign constraint, for $\lambda$ sufficiently small, we show the existence of infinitely many solutions.

Advanced recommender systems usually involve multiple domains (such as scenarios or categories) for various marketing strategies, and users interact with them to satisfy diverse demands. The goal of multi-domain recommendation (MDR) is to improve the recommendation performance of all domains simultaneously. Conventional graph neural network based methods usually deal with each domain separately, or train a shared model to serve all domains. The former fails to leverage users' cross-domain behaviors, making the behavior sparseness issue a great obstacle. The latter learns shared user representation with respect to all domains, which neglects users' domain-specific preferences. In this paper we propose $\mathsf{H^3Trans}$, a hierarchical hypergraph network based correlative preference transfer framework for MDR, which represents multi-domain user-item interactions into a unified graph to help preference transfer. $\mathsf{H^3Trans}$ incorporates two hyperedge-based modules, namely dynamic item transfer (Hyper-I) and adaptive user aggregation (Hyper-U). Hyper-I extracts correlative information from multi-domain user-item feedbacks for eliminating domain discrepancy of item representations. Hyper-U aggregates users' scattered preferences in multiple domains and further exploits the high-order (not only pair-wise) connections to improve user representations. Experiments on both public and production datasets verify the superiority of $\mathsf{H^3Trans}$ for MDR., Comment: Accepted by WWW 2023 research track. The first two authors contributed equally

Predicting user response probabilities is vital for ad ranking and bidding. We hope that predictive models can produce accurate probabilistic predictions that reflect true likelihoods. Calibration techniques aim to post-process model predictions to posterior probabilities. Field-level calibration -- which performs calibration w.r.t. to a specific field value -- is fine-grained and more practical. In this paper we propose a doubly-adaptive approach AdaCalib. It learns an isotonic function family to calibrate model predictions with the guidance of posterior statistics, and field-adaptive mechanisms are designed to ensure that the posterior is appropriate for the field value to be calibrated. Experiments verify that AdaCalib achieves significant improvement on calibration performance. It has been deployed online and beats previous approach., Comment: SIGIR 2022 (short)