Search

Data-intensive applications executing on NVM-based storage systems experience serious bottlenecks when moving data between DRAM and NVM. We advocate for the use of the long-existing but recently neglected on-chip DMA to expedite data movement with three contributions. First, we explore new latency-oriented optimization directions, driven by a comprehensive DMA study, to design a high-performance DMA module, which significantly lowers the I/O size threshold to observe benefits. Second, we propose a new data movement engine, Fastmove, that coordinates the use of the DMA along with the CPU with DDIO-aware strategies, judicious scheduling, and load splitting such that the DMA's limitations are compensated, and the overall gains are maximized. Finally, with a general kernel-based design, simple APIs, and DAX file system integration, Fastmove allows applications to transparently exploit the DMA and its new features without code change. We run three data-intensive applications MySQL, GraphWalker, and Filebench atop NOVA, ext4-DAX, and XFS-DAX, with standard benchmarks like TPC-C, and popular graph algorithms like PageRank. Across single- and multi-socket settings, compared to the conventional CPU-only NVM accesses, Fastmove introduces to TPC-C with MySQL 1.13–2.16× speedups of peak throughput, reduces the average latency by 17.7–60.8%, and saves 37.1–68.9% CPU usage spent in data movement. It also shortens the execution time of graph algorithms with GraphWalker by 39.7–53.4%, and introduces 1.01–1.48× throughput speedups for Filebench. [ABSTRACT FROM AUTHOR]

With the rapid development of computer technology, distributed systems have become an indispensable part of the field of information storage and management. In the process of large-scale data processing, it is an important issue to compress or replay files to ensure their integrity. In order to solve the problem of large-scale computing resources and data storage, the distributed file system emerged as a new system structure. The purpose of this paper to is study the key technologies of the distributed memory file system of high-performance computers is to improve the capability and efficiency of the distributed system. This paper mainly uses the experimental method and the comparative method to analyze the key technology of the distributed memory file system of the high-performance computer. Experimental results show that the maximum bandwidth value of DFMS in file memory processing can reach more than 2000, and the value becomes more stable as the file increases. [ABSTRACT FROM AUTHOR]

The imbalanced I/O load on large parallel file systems affects the parallel I/O performance of high-performance computing (HPC) applications. One of the main reasons for I/O imbalances is the lack of a global view of system-wide resource consumption. While approaches to address the problem already exist, the diversity of HPC workloads combined with different file striping patterns prevents widespread adoption of these approaches. In addition, load-balancing techniques should be transparent to client applications. To address these issues, we propose Tarazu, an end-to-end control plane where clients transparently and adaptively write to a set of selected I/O servers to achieve balanced data placement. Our control plane leverages real-time load statistics for global data placement on distributed storage servers, while our design model employs trace-based optimization techniques to minimize latency for I/O load requests between clients and servers and to handle multiple striping patterns in files. We evaluate our proposed system on an experimental cluster for two common use cases: the synthetic I/O benchmark IOR and the scientific application I/O kernel HACC-I/O. We also use a discrete-time simulator with real HPC application traces from emerging workloads running on the Summit supercomputer to validate the effectiveness and scalability of Tarazu in large-scale storage environments. The results show improvements in load balancing and read performance of up to 33% and 43%, respectively, compared to the state-of-the-art. [ABSTRACT FROM AUTHOR]

Digital service platforms provided by academic support departments in Macao assist academic staff and students in various areas such as registry, student affairs, academic activities, and research. As the number of undergraduate students increases and new departments are established, academic staff often face the challenge of dealing with paperwork that contains similar content but different formats. This situation results in redundancies and a waste of time. This paper presents our endeavors to simplify administrative procedures in higher education by automating restructured documentation and developing secondary file systems. The paper presents two case studies: Scenario One focuses on streamlining the publication system for academic staff who submit papers in different formats. At the same time, Scenario Two aims to simplify the daily paperwork process for academic staff. Both cases involve transforming the distribution of administrative documents, transitioning from a standardized form with guidelines to a customized form with concise tips. This approach allows academic staff to handle only the necessary information, which may not be available in the database or requires verification. The case studies serve to demonstrate the effectiveness of this administrative simplification. [ABSTRACT FROM AUTHOR]

Unikernels are specialized operating system (OS) kernels optimized for a single application or service, offering advantages such as rapid boot times, high performance, minimal memory usage, and enhanced security compared to general-purpose OS kernels. Unikernel applications must remain compatible with the runtime environment of general-purpose kernels, either through binary or source compatibility. As a result, many Unikernel projects have prioritized system call compatibility over performance enhancements. In this paper, we explore the design principles of Unikernel file systems and introduce a new file system tailored for Unikernels named ULFS (Ultra Lightweight File System). ULFS provides system call services akin to those of general-purpose OS kernels but achieves superior performance and security with significantly fewer system resources. Specifically, ULFS is developed as a lightweight file system embracing Unikernel design principles. It streamlines system calls, removes unnecessary locks, and omits permission checks for multiple users, utilizing a non-hypervisor architecture. This approach significantly reduces the memory footprint of the file system and enhances performance. Through measurement studies, we assess the performance and memory requirements of various file systems from major Unikernel projects. Our findings demonstrate that ULFS surpasses several existing Unikernel file systems, including Rumpvfs, Ramfs-u, Ramfs-q, 9pfs, and Hcfs. [ABSTRACT FROM AUTHOR]

The ever-growing sizes and frequent updating of mobile applications cause high network and storage cost for updating. Hence, emerging mobile systems often employ incremental update algorithms, typically HDiffPatch, to upgrade mobile applications. However, we find that existing incremental update algorithms not only generate a significant amount of redundant data accesses, but also lacks of consistency guarantees for the whole package of application. In this paper, we present a novel Consistent and Efficient Incremental Update (CEIU) mechanism for upgrading mobile applications. Firstly, CEIU reduces the memory consumption and file access of incremental updates by reusing the indexes of blocks in the old image rather than copying the blocks. Secondly, CEIU employs a two-level journaling mechanism to ensure the consistency of the whole package and subfiles of the new image. We implement the proposed mechanism in Linux kernel based on TI-LFAT file system and evaluate it with real-world applications. The experimental results show that the proposed mechanism can reduce 30%–80% memory footprints in comparison with HDiffPatch, the state-of-the-art incremental update algorithm. It also significantly reduces the recovery time when power failure or system crash occurs. [ABSTRACT FROM AUTHOR]