Search

The rapid proliferation of Large Language Models (LLMs) has been a driving force in the growth of cloud-based LLM services, which are now integral to advancing AI applications. However, the dynamic auto-regressive nature of LLM service, along with the need to support exceptionally long context lengths, demands the flexible allocation and release of substantial resources. This presents considerable challenges in designing cloud-based LLM service systems, where inefficient management can lead to performance degradation or resource wastage. In response to these challenges, this work introduces DistAttention, a novel distributed attention algorithm that segments the KV Cache into smaller, manageable units, enabling distributed processing and storage of the attention module. Based on that, we propose DistKV-LLM, a distributed LLM serving system that dynamically manages KV Cache and effectively orchestrates all accessible GPU and CPU memories spanning across the data center. This ensures a high-performance LLM service on the cloud, adaptable to a broad range of context lengths. Validated in a cloud environment with 32 NVIDIA A100 GPUs in configurations from 2 to 32 instances, our system exhibited 1.03-2.4x end-to-end throughput improvements and supported context lengths 2-19x longer than current state-of-the-art LLM service systems, as evidenced by extensive testing across 18 datasets with context lengths up to 1,900K.

Molecular dynamics simulations are used to calculate the theoretical structure and properties of the CaO-SiO2-Al2O3-FeO quaternary slag system with the aim of understanding the influence of the FeO content on the slag structure and properties. The radial distribution function, coordination number, diffusion coefficient, etc. are used to characterise the structural characteristics of the slag. Viscosity is one of the important physical properties in blast furnace ironmaking, having an important influence on the blast furnace oxidation metallurgy process. The results of this study indicate that, as the FeO content is increased, the network structure is destroyed, the diffusion coefficient of each atom increases, and the system viscosity decreases, indicating that the polymerisation structure of the CaO-SiO2-Al2O3-FeO slag system tends to be depolymerised. Based on these results, FeO in blast furnace slag has a great influence on the structural characteristics of CaO-SiO2-Al2O3-FeO slag., Molecular dynamics simulations are used to calculate the theoretical structure and properties of the CaO-SiO2-Al2O3-FeO quaternary slag system with the aim of understanding the influence of the FeO content on the slag structure and properties. The radial distribution function, coordination number, diffusion coefficient, etc. are used to characterise the structural characteristics of the slag. Viscosity is one of the important physical properties in blast furnace ironmaking, having an important influence on the blast furnace oxidation metallurgy process. The results of this study indicate that, as the FeO content is increased, the network structure is destroyed, the diffusion coefficient of each atom increases, and the system viscosity decreases, indicating that the polymerisation structure of the CaO-SiO2-Al2O3-FeO slag system tends to be depolymerised. Based on these results, FeO in blast furnace slag has a great influence on the structural characteristics of CaO-SiO2-Al2O3-FeO slag.

High-performance tensor programs are crucial to guarantee efficient execution of deep neural networks. However, obtaining performant tensor programs for different operators on various hardware platforms is notoriously challenging. Currently, deep learning systems rely on vendor-provided kernel libraries or various search strategies to get performant tensor programs. These approaches either require significant engineering effort to develop platform-specific optimization code or fall short of finding high-performance programs due to restricted search space and ineffective exploration strategy. We present Ansor, a tensor program generation framework for deep learning applications. Compared with existing search strategies, Ansor explores many more optimization combinations by sampling programs from a hierarchical representation of the search space. Ansor then fine-tunes the sampled programs with evolutionary search and a learned cost model to identify the best programs. Ansor can find high-performance programs that are outside the search space of existing state-of-the-art approaches. In addition, Ansor utilizes a task scheduler to simultaneously optimize multiple subgraphs in deep neural networks. We show that Ansor improves the execution performance of deep neural networks relative to the state-of-the-art on the Intel CPU, ARM CPU, and NVIDIA GPU by up to $3.8\times$, $2.6\times$, and $1.7\times$, respectively., Comment: OSDI 2020

© The Author(s), 2018. This is the author's version of the work and is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Science of The Total Environment 644 (2018): 439-451, doi:10.1016/j.scitotenv.2018.06.269., Characterized by the noticeable seasonal patterns of photosynthesis, mid-to-high latitude forests are sensitive to climate change and crucial for understanding the global carbon cycle. To monitor the seasonal cycle of the canopy photosynthesis from space, several remote sensing based indexes, such as normalized difference vegetation index (NDVI), enhanced vegetation index (EVI) and leaf area index (LAI), have been implemented within the past decades. Recently, satellite-derived sun-induced fluorescence (SIF) has shown great potentials of providing retrievals that are more related to photosynthesis process. However, the potentials of different canopy measurements have not been thoroughly assessed in the context of recent advances of new satellites and proposals of improved indexes. Here, we present a cross-site intercomparison of one emerging remote sensing based index of phenological index (PI) and two SIF datasets against the conventional indexes of NDVI, EVI and LAI to capture the seasonal cycles of canopy photosynthesis. NDVI, EVI, LAI and PI were calculated from Moderate Resolution Imaging Spectroradiometer (MODIS) measurements, while SIF were evaluated from Global Ozone Monitoring Experiment-2 (GOME-2) and Orbiting Carbon Observatory-2 (OCO-2) observations. Results indicated that GOME-2 SIF was highly correlated with gross primary productivity (GPP) and absorbed photosynthetically active radiation (APAR) during the growing seasons. Key phenological metrics captured by SIF from GOME-2 and OCO-2 matched closely with photosynthesis phenology as inferred by GPP. However, the applications of OCO-2 SIF for phenological studies may be limited only for a small range of sites (at site-level) due to a limited spatial sampling. Among the MODIS estimations, PI and NDVI provided most reliable predictions of start of growing seasons, while no indexes accurately captured the end of growing seasons., This work was supported by the Chinese Arctic and Antarctic Administration, National Natural Science Foundation of China (Grant Nos. 41676176 and 41676182), the Chinese Polar Environment Comprehensive Investigation, Assessment Program (Grant No. 312231103). This work was also supported by the Fundamental Research Funds for the 440 Central Universities, 2020-07-11

© The Author(s), 2018. This is the author's version of the work and is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Science of The Total Environment 644 (2018): 439-451, doi:10.1016/j.scitotenv.2018.06.269., Characterized by the noticeable seasonal patterns of photosynthesis, mid-to-high latitude forests are sensitive to climate change and crucial for understanding the global carbon cycle. To monitor the seasonal cycle of the canopy photosynthesis from space, several remote sensing based indexes, such as normalized difference vegetation index (NDVI), enhanced vegetation index (EVI) and leaf area index (LAI), have been implemented within the past decades. Recently, satellite-derived sun-induced fluorescence (SIF) has shown great potentials of providing retrievals that are more related to photosynthesis process. However, the potentials of different canopy measurements have not been thoroughly assessed in the context of recent advances of new satellites and proposals of improved indexes. Here, we present a cross-site intercomparison of one emerging remote sensing based index of phenological index (PI) and two SIF datasets against the conventional indexes of NDVI, EVI and LAI to capture the seasonal cycles of canopy photosynthesis. NDVI, EVI, LAI and PI were calculated from Moderate Resolution Imaging Spectroradiometer (MODIS) measurements, while SIF were evaluated from Global Ozone Monitoring Experiment-2 (GOME-2) and Orbiting Carbon Observatory-2 (OCO-2) observations. Results indicated that GOME-2 SIF was highly correlated with gross primary productivity (GPP) and absorbed photosynthetically active radiation (APAR) during the growing seasons. Key phenological metrics captured by SIF from GOME-2 and OCO-2 matched closely with photosynthesis phenology as inferred by GPP. However, the applications of OCO-2 SIF for phenological studies may be limited only for a small range of sites (at site-level) due to a limited spatial sampling. Among the MODIS estimations, PI and NDVI provided most reliable predictions of start of growing seasons, while no indexes accurately captured the end of growing seasons., This work was supported by the Chinese Arctic and Antarctic Administration, National Natural Science Foundation of China (Grant Nos. 41676176 and 41676182), the Chinese Polar Environment Comprehensive Investigation, Assessment Program (Grant No. 312231103). This work was also supported by the Fundamental Research Funds for the 440 Central Universities, 2020-07-11