Search

We characterize, using time-frequency analysis, the continuity and compactness of the Weyl operator in global classes of ultradifferentiable functions $\mathcal{S}_\omega$, for weight functions $\omega$ in the sense of Braun, Meise and Taylor. As a consequence, we give results about the compactness of the localization operator in $\mathcal{S}_\omega$, in relation with the spaces of $\omega$-multipliers and $\omega$-convolutors of $\mathcal{S}_\omega$. Moreover, we provide several examples that complement our investigation.

Kernel orchestration is the task of mapping the computation defined in different operators of a deep neural network (DNN) to the execution of GPU kernels on modern hardware platforms. Prior approaches optimize kernel orchestration by greedily applying operator fusion, which fuses the computation of multiple operators into a single kernel, and miss a variety of optimization opportunities in kernel orchestration. This paper presents Korch, a tensor program optimizer that discovers optimal kernel orchestration strategies for tensor programs. Instead of directly fusing operators, Korch first applies operator fission to decompose tensor operators into a small set of basic tensor algebra primitives. This decomposition enables a diversity of fine-grained, inter-operator optimizations. Next, Korch optimizes kernel orchestration by formalizing it as a constrained optimization problem, leveraging an off-the-shelf binary linear programming solver to discover an optimal orchestration strategy, and generating an executable that can be directly deployed on modern GPU platforms. Evaluation on a variety of DNNs shows that Korch outperforms existing tensor program optimizers by up to 1.7x on V100 GPUs and up to 1.6x on A100 GPUs. Korch is publicly available at https://github.com/humuyan/Korch., Comment: Fix some typos in the ASPLOS version

The usefulness of time-frequency analysis methods in the study of quasicrystals was pointed out in a previous paper, where we proved that a tempered distribution $\mu$ on ${\mathbb R}^d$ whose Wigner transform is a measure supported on the cartesian product of two uniformly discrete sets in ${\mathbb R}^d$ is a Fourier quasicrystal. In this paper we go further in this direction using the matrix-Wigner transforms to detect quasicrystal structures. The results presented here cover essentially all the most important quadratic time-frequency distributions, and are obtained considering two different (disjoint) classes of matrix-Wigner transforms, discussed respectively in Theorems 1 and 2. The transforms considered in Theorem 1 include the classical Wigner transform, as well as all the time-frequency representations of matrix-Wigner type belonging to the Cohen class. On the other hand Theorem 2, which does not apply to the classical Wigner, has, as main example, the Ambiguity function. In this second case we only suppose that the support of the matrix-Wigner transform of $\mu$ is contained in the cartesian product of two discrete sets, obtaining that both the support and the spectrum of $\mu$ are uniformly discrete.

Parameter-efficient finetuning (PEFT) is a widely used technique to adapt large language models for different tasks. Service providers typically create separate systems for users to perform PEFT model finetuning and inference tasks. This is because existing systems cannot handle workloads that include a mix of inference and PEFT finetuning requests. As a result, shared GPU resources are underutilized, leading to inefficiencies. To address this problem, we present FlexLLM, the first system that can serve inference and parameter-efficient finetuning requests in the same iteration. Our system leverages the complementary nature of these two tasks and utilizes shared GPU resources to run them jointly, using a method called co-serving. To achieve this, FlexLLM introduces a novel token-level finetuning mechanism, which breaks down the finetuning computation of a sequence into smaller token-level computations and uses dependent parallelization and graph pruning, two static compilation optimizations, to minimize the memory overhead and latency for co-serving. Compared to existing systems, FlexLLM's co-serving approach reduces the activation GPU memory overhead by up to 8x, and the end-to-end GPU memory requirement of finetuning by up to 36% while maintaining a low inference latency and improving finetuning throughput. For example, under a heavy inference workload, FlexLLM can still preserve more than 80% of the peak finetuning throughput, whereas existing systems cannot make any progress with finetuning. The source code of FlexLLM is publicly available at https://github.com/flexflow/FlexFlow.

In this paper we give different estimates between Lebesgue norms of quadratic time-frequency representations. We show that, in some cases, it is not possible to have such bounds in classical $L^p$ spaces, but the Lebesgue norm needs to be suitably weighted. This leads to consider weights of polynomial type, and, more generally, of ultradifferentiable type, and this, in turn, gives rise to use as functional setting the ultradifferentiable classes. As applications of such estimates we deduce uncertainty principles both of Donoho-Stark type and of local type for representations.

We study and characterize the inclusion relations of global classes in the general weight matrix framework in terms of growth relations for the defining weight matrices. We consider the Roumieu and Beurling cases, and as a particular case we also treat the classical weight function and weight sequence cases. Moreover, we construct a weight sequence which is oscillating around any weight sequence which satisfies some minimal conditions and, in particular, around the critical weight sequence $(p!)^{1/2}$, related with the non-triviality of the classes. Finally, we also obtain comparison results both on classes defined by weight functions that can be defined by weight sequences and conversely.

We give a simple construction of the log-convex minorant of a sequence $\{M_\alpha\}_{\alpha\in\mathbb{N}_0^d}$ and consequently extend to the $d$-dimensional case the well-known formula that relates a log-convex sequence $\{M_p\}_{p\in\mathbb{N}_0}$ to its associated function $\omega_M$, that is $M_p=\sup_{t>0}t^p\exp(-\omega_M(t))$. We show that in the more dimensional anisotropic case the classical log-convex condition $M_\alpha^2\leq M_{\alpha-e_j}M_{\alpha+e_j}$ is not sufficient: convexity as a function of more variables is needed (not only coordinate-wise). We finally obtain some applications to the inclusion of spaces of rapidly decreasing ultradifferentiable functions in the matrix weighted setting.

In the rapidly evolving landscape of artificial intelligence (AI), generative large language models (LLMs) stand at the forefront, revolutionizing how we interact with our data. However, the computational intensity and memory consumption of deploying these models present substantial challenges in terms of serving efficiency, particularly in scenarios demanding low latency and high throughput. This survey addresses the imperative need for efficient LLM serving methodologies from a machine learning system (MLSys) research perspective, standing at the crux of advanced AI innovations and practical system optimizations. We provide in-depth analysis, covering a spectrum of solutions, ranging from cutting-edge algorithmic modifications to groundbreaking changes in system designs. The survey aims to provide a comprehensive understanding of the current state and future directions in efficient LLM serving, offering valuable insights for researchers and practitioners in overcoming the barriers of effective LLM deployment, thereby reshaping the future of AI.

This paper introduces SpecInfer, a system that accelerates generative large language model (LLM) serving with tree-based speculative inference and verification. The key idea behind SpecInfer is leveraging small speculative models to predict the LLM's outputs; the predictions are organized as a token tree, whose nodes each represent a candidate token sequence. The correctness of all candidate token sequences represented by a token tree is verified against the LLM in parallel using a novel tree-based parallel decoding mechanism. SpecInfer uses an LLM as a token tree verifier instead of an incremental decoder, which significantly reduces the end-to-end latency and computational requirement for serving generative LLMs while provably preserving model quality. Our evaluation shows that SpecInfer outperforms existing LLM serving systems by 1.5-2.8x for distributed LLM inference and by 2.6-3.5x for offloading-based LLM inference, while preserving the same generative performance. SpecInfer is publicly available at https://github.com/flexflow/FlexFlow/, Comment: ASPLOS'24

Given a function $f\in L^2(\mathbb R)$, we consider means and variances associated to $f$ and its Fourier transform $\hat{f}$, and explore their relations with the Wigner transform $W(f)$, obtaining a simple new proof of Shapiro's mean-dispersion principle. Uncertainty principles for orthonormal sequences in $L^2(\mathbb R)$ involving linear partial differential operators with polynomial coefficients and the Wigner distribution, or different Cohen class representations, are obtained, and an extension to the case of Riesz bases is studied., Comment: 31 pages

Greedy expansions with prescribed coefficients have been introduced by V. N. Temlyakov in the frame of Banach spaces. The idea is to choose a sequence of fixed (real) coefficients $\{c_n\}_{n=1}^\infty$ and a fixed set of elements (dictionary) of the Banach space; then, under suitable conditions on the coefficients and the dictionary, it is possible to expand all the elements of the Banach space in series that contain only the fixed coefficients and the elements of the dictionary. In Hilbert spaces the convergence of greedy algorithm with prescribed coefficients is characterized, in the sense that there are necessary and sufficient conditions on the coefficients in order that the algorithm is convergent for all the dictionaries. This paper is concerned with the question if such conditions can be weakened for particular classes of spaces or dictionaries; we prove that this is the case for finite dimensional spaces, and for some classes of dictionaries related to orthonormal sequences in infinite dimensional spaces.

Fine-grained network telemetry is becoming a modern datacenter standard and is the basis of essential applications such as congestion control, load balancing, and advanced troubleshooting. As network size increases and telemetry gets more fine-grained, there is a tremendous growth in the amount of data needed to be reported from switches to collectors to enable network-wide view. As a consequence, it is progressively hard to scale data collection systems. We introduce Direct Telemetry Access (DTA), a solution optimized for aggregating and moving hundreds of millions of reports per second from switches into queryable data structures in collectors' memory. DTA is lightweight and it is able to greatly reduce overheads at collectors. DTA is built on top of RDMA, and we propose novel and expressive reporting primitives to allow easy integration with existing state-of-the-art telemetry mechanisms such as INT or Marple. We show that DTA significantly improves telemetry collection rates. For example, when used with INT, it can collect and aggregate over 400M reports per second with a single server, improving over the Atomic MultiLog by up to $16$x., Comment: As appearing in the proceedings of ACM SIGCOMM'23

Programmable switches are driving a massive increase in fine-grained measurements. This puts significant pressure on telemetry collectors that have to process reports from many switches. Past research acknowledged this problem by either improving collectors' stack performance or by limiting the amount of data sent from switches. In this paper, we take a different and radical approach: switches are responsible for directly inserting queryable telemetry data into the collectors' memory, bypassing their CPU, and thereby improving their collection scalability. We propose to use a method we call \emph{direct telemetry access}, where switches jointly write telemetry reports directly into the same collector's memory region, without coordination. Our solution, DART, is probabilistic, trading memory redundancy and query success probability for CPU resources at collectors. We prototype DART using commodity hardware such as P4 switches and RDMA NICs and show that we get high query success rates with a reasonable memory overhead. For example, we can collect INT path tracing information on a fat tree topology without a collector's CPU involvement while achieving 99.9\% query success probability and using just 300 bytes per flow., Comment: To appear in ACM HotNets 2021

Quasicrystals are tempered distributions $\mu$ which satisfy symmetric conditions on $\mu$ and $\widehat \mu$. This suggests that techniques from time-frequency analysis could possibly be useful tools in the study of such structures. In this paper we explore this direction considering quasicrystals type conditions on time-frequency representations instead of separately on the distribution and its Fourier transform. More precisely we prove that a tempered distribution $\mu$ on ${\mathbb R}^d$ whose Wigner transform, $W(\mu)$, is supported on a product of two uniformly discrete sets in ${\mathbb R}^d$ is a quasicrystal. This result is partially extended to a generalization of the Wigner transform, called matrix-Wigner transform which is defined in terms of the Wigner transform and a linear map $T$ on ${\mathbb R}^{2d}$.

In this paper we study regularity of partial differential equations with polynomial coefficients in non isotropic Beurling spaces of ultradifferentiable functions of global type. We study the action of transformations of Gabor and Wigner type in such spaces and we prove that a suitable representation of Wigner type allows to prove regularity for classes of operators that do not have classical hypoellipticity properties.

We prove that the Hermite functions are an absolute Schauder basis for many global weighted spaces of ultradifferentiable functions in the matrix weighted setting and we determine also the corresponding coefficient spaces, thus extending previous work by Langenbruch. As a consequence we give very general conditions for these spaces to be nuclear. In particular, we obtain the corresponding results for spaces defined by weight functions.

In this paper we characterize global regularity in the sense of Shubin of twisted partial differential operators of second order in dimension $2$. These operators form a class containing the twisted Laplacian, and in bi-unique correspondence with second order ordinary differential operators with polynomial coefficients and symbol of degree $2$. This correspondence is established by a transformation of Wigner type. In this way the global regularity of twisted partial differential operators turns out to be equivalent to global regularity and injectivity of the corresponding ordinary differential operators, which can be completely characterized in terms of the asymptotic behavior of the Weyl symbol. In conclusion we observe that we have obtained a new class of globally regular partial differential operators which is disjoint from the class of hypo-elliptic operators in the sense of Shubin.

We use techniques from time-frequency analysis to show that the space $\mathcal S_\omega$ of rapidly decreasing $\omega$-ultradifferentiable functions is nuclear for every weight function $\omega(t)=o(t)$ as $t$ tends to infinity. Moreover, we prove that, for a sequence $(M_p)_p$ satisfying the classical condition $(M1)$ of Komatsu, the space of Beurling type $\mathcal S_{(M_p)}$ when defined with $L^{2}\,$norms is nuclear exactly when condition $(M2)'$ of Komatsu holds.

We use an isomorphism established by Langenbruch between some sequence spaces and weighted spaces of generalized functions to give sufficient conditions for the (Beurling type) space ${\mathcal S}_{(M_p)}$ to be nuclear. As a consequence, we obtain that for a weight function $\omega$ satisfying the mild condition: $2\omega(t)\leq \omega(Ht)+H$ for some $H>1$ and for all $t\geq0$, the space ${\mathcal S}_\omega$ in the sense of Bj\"orck is also nuclear.

We develop real Paley-Wiener theorems for classes ${\mathcal S}_\omega$ of ultradifferentiable functions and related $L^{p}$-spaces in the spirit of Bang and Andersen for the Schwartz class. We introduce results of this type for the so-called Gabor transform and give a full characterization in terms of Fourier and Wigner transforms for several variables of a Paley-Wiener theorem in this general setting, which is new in the literature. We also analyze this type of results when the support of the function is not compact using polynomials. Some examples are given.

This paper presents a proof of an uncertainty principle of Donoho-Stark type involving $\varepsilon$-concentration of localization operators. More general operators associated with time-frequency representations in the Cohen class are then considered. For these operators, which include all usual quantizations, we prove a boundedness result in the $L^p$ functional setting and a form of uncertainty principle analogous to that for localization operators., Comment: 21 pages, 1 figure

Given a non-quasianalytic subadditive weight function $\omega$ we consider the weighted Schwartz space $\mathcal{S}_\omega$ and the short-time Fourier transform on $\mathcal{S}_\omega$, $\mathcal{S}'_\omega$ and on the related modulation spaces with exponential weights. In this setting we define the $\omega$-wave front set $WF'_\omega(u)$ and the Gabor $\omega$-wave front set $WF^G_\omega(u)$ of $u\in\mathcal{S}'_{\omega}$, and we prove that they coincide. Finally we look at applications of this wave front set for operators of differential and pseudo-differential type., Comment: 35 pages