Search

Large Language Models (LLMs) are critical for a wide range of applications, but serving them efficiently becomes increasingly challenging as inputs become more complex. Context caching improves serving performance by exploiting inter-request dependency and reusing key-value (KV) cache across requests, thus improving time-to-first-token (TTFT). However, existing prefix-based context caching requires exact token prefix matches, limiting cache reuse in few-shot learning, multi-document QA, or retrieval-augmented generation, where prefixes may vary. In this paper, we present EPIC, an LLM serving system that introduces position-independent context caching (PIC), enabling modular KV cache reuse regardless of token chunk position (or prefix). EPIC features two key designs: AttnLink, which leverages static attention sparsity to minimize recomputation for accuracy recovery, and KVSplit, a customizable chunking method that preserves semantic coherence. Our experiments demonstrate that Epic delivers up to 8x improvements in TTFT and 7x throughput over existing systems, with negligible or no accuracy loss. By addressing the limitations of traditional caching approaches, Epic enables more scalable and efficient LLM inference.

Large language model (LLM) serving has transformed from stateless to stateful systems, utilizing techniques like context caching and disaggregated inference. These optimizations extend the lifespan and domain of the KV cache, necessitating a new architectural approach. We present MemServe, a unified system that integrates both inter-request and intra-request optimizations. MemServe introduces MemPool, an elastic memory pool managing distributed memory and KV caches across serving instances. Using MemPool APIs, MemServe combines context caching with disaggregated inference for the first time, supported by a global scheduler that enhances cache reuse through a global prompt tree-based locality-aware policy. Tests show that MemServe significantly improves job completion time and time-to-first-time.

This paper focuses on the randomized Milstein scheme for approximating solutions to stochastic Volterra integral equations with weakly singular kernels, where the drift coefficients are non-differentiable. An essential component of the error analysis involves the utilization of randomized quadrature rules for stochastic integrals to avoid the Taylor expansion in drift coefficient functions. Finally, we implement the simulation of multiple singular stochastic integral in the numerical experiment by applying the Riemann-Stieltjes integral.

Deformable image registration (DIR) is an active research topic in biomedical imaging. There is a growing interest in developing DIR methods based on deep learning (DL). A traditional DL approach to DIR is based on training a convolutional neural network (CNN) to estimate the registration field between two input images. While conceptually simple, this approach comes with a limitation that it exclusively relies on a pre-trained CNN without explicitly enforcing fidelity between the registered image and the reference. We present plug-and-play image registration network (PIRATE) as a new DIR method that addresses this issue by integrating an explicit data-fidelity penalty and a CNN prior. PIRATE pre-trains a CNN denoiser on the registration field and "plugs" it into an iterative method as a regularizer. We additionally present PIRATE+ that fine-tunes the CNN prior in PIRATE using deep equilibrium models (DEQ). PIRATE+ interprets the fixed-point iteration of PIRATE as a network with effectively infinite layers and then trains the resulting network end-to-end, enabling it to learn more task-specific information and boosting its performance. Our numerical results on OASIS and CANDI datasets show that our methods achieve state-of-the-art performance on DIR.

By using the It\^{o}-Tanaka trick, we prove the unique strong solvability as well as the gradient estimates for stochastic differential equations with irregular drifts in low regularity Lebesgue-H\"{o}lder space $L^q(0,T;{\mathcal C}_b^\alpha({\mathbb R}^d))$ with $\alpha\in(0,1)$ and $q\in (2/(1+\alpha),2$). As applications, we show the unique weak and strong solvability for stochastic transport equations driven by the low regularity drift with $q\in (4/(2+\alpha),2$) as well as the local Lipschitz estimate for stochastic strong solutions.

Recently, deep learning-based techniques have shown promising performance on various tasks related to software engineering. For these learning-based approaches to perform well, obtaining high-quality data is one fundamental and crucial issue. The comment updating task is an emerging software engineering task aiming at automatically updating the corresponding comments based on changes in source code. However, datasets for the comment updating tasks are usually crawled from committed versions in open source software repositories such as GitHub, where there is lack of quality control of comments. In this paper, we focus on cleaning existing comment updating datasets with considering some properties of the comment updating process in software development. We propose a semantic and overlapping-aware approach named CupCleaner (Comment UPdating's CLEANER) to achieve this purpose. Specifically, we calculate a score based on semantics and overlapping information of the code and comments. Based on the distribution of the scores, we filter out the data with low scores in the tail of the distribution to get rid of possible unclean data. We first conducted a human evaluation on the noise data and high-quality data identified by CupCleaner. The results show that the human ratings of the noise data identified by CupCleaner are significantly lower. Then, we applied our data cleaning approach to the training and validation sets of three existing comment updating datasets while keeping the test set unchanged. Our experimental results show that even after filtering out over 30\% of the data using CupCleaner, there is still an improvement in all performance metrics. The experimental results on the cleaned test set also suggest that CupCleaner may provide help for constructing datasets for updating-related tasks.

In this article, we consider slow-fast McKean-Vlasov stochastic differential equations driven by Brownian motions and fractional Brownian motions. We give a definition of the large deviation principle (LDP) on the product space related to Brownian motion and fractional Brownian motion, which is different from the traditional definition for LDP. Under some proper assumptions on coefficients, LDP is investigated for this type of equations by using the weak convergence method.

This paper focuses on the numerical scheme of highly nonlinear neutral multiple-delay stohchastic McKean-Vlasov equation (NMSMVE) by virtue of the stochastic particle method. First, under general assumptions, the results about propagation of chaos in $\mathcal{L}^p$ sense are shown. Then the tamed Euler-Maruyama scheme to the corresponding particle system is established and the convergence rate in $\mathcal{L}^p$ sense is obtained. Furthermore, combining these two results gives the convergence error between the objective NMSMVE and numerical approximation, which is related to the particle number and step size. Finally, two numerical examples are provided to support the finding.

Code completion has become a common practice for programmers during their daily programming activities. It aims at automatically predicting the next tokens or lines that the programmers tend to use. A good code completion tool can substantially save keystrokes and improve the programming efficiency for programmers. Recently, various techniques for code completion have been proposed for usage in practice. However, it is still unclear what are practitioners' expectations on code completion and whether existing research has met their demands. To fill the gap, we perform an empirical study by first interviewing 15 practitioners and then surveying 599 practitioners from 18 IT companies about their expectations on code completion. We then compare the practitioners' demands with current research via conducting a literature review of papers on code completion published in premier publication venues from 2012 to 2022. Based on the comparison, we highlight the directions desirable for researchers to invest efforts towards developing code completion techniques for meeting practitioners' expectations., Comment: 12 pages

Deep model-based architectures (DMBAs) are widely used in imaging inverse problems to integrate physical measurement models and learned image priors. Plug-and-play priors (PnP) and deep equilibrium models (DEQ) are two DMBA frameworks that have received significant attention. The key difference between the two is that the image prior in DEQ is trained by using a specific measurement model, while that in PnP is trained as a general image denoiser. This difference is behind a common assumption that PnP is more robust to changes in the measurement models compared to DEQ. This paper investigates the robustness of DEQ priors to changes in the measurement models. Our results on two imaging inverse problems suggest that DEQ priors trained under mismatched measurement models outperform image denoisers.

In this article, we prove the existence of weak solutions as well as the existence and uniqueness of strong solutions for McKean-Vlasov multivalued stochastic differential equations with oblique subgradients (MVMSDEswOS, for short) by means of the equations of Euler type and Skorohod's representation theorem. For this type of equation, compared with the method in [19,13], since we can't use the maximal monotony property of its constituent subdifferential operator, some different specific techniques are applied to solve our problems. Afterwards, we give an example for MVMSDEswOS with time-dependent convex constraints, which can be reduced to MVMSDEswOS. Finally, we consider an optimal control problem and establish the dynamic programming principle for the value function.