Search

In this paper, we present a novel technique to search for hardware architectures of accelerators optimized for end-to-end training of deep neural networks (DNNs). Our approach addresses both single-device and distributed pipeline and tensor model parallel scenarios, latter being addressed for the first time. The search optimized accelerators for training relevant metrics such as throughput/TDP under a fixed area and power constraints. However, with the proliferation of specialized architectures and complex distributed training mechanisms, the design space exploration of hardware accelerators is very large. Prior work in this space has tried to tackle this by reducing the search space to either a single accelerator execution that too only for inference, or tuning the architecture for specific layers (e.g., convolution). Instead, we take a unique heuristic-based critical path-based approach to determine the best use of available resources (power and area) either for a set of DNN workloads or each workload individually. First, we perform local search to determine the architecture for each pipeline and tensor model stage. Specifically, the system iteratively generates architectural configurations and tunes the design using a novel heuristic-based approach that prioritizes accelerator resources and scheduling to critical operators in a machine learning workload. Second, to address the complexities of distributed training, the local search selects multiple (k) designs per stage. A global search then identifies an accelerator from the top-k sets to optimize training throughput across the stages. We evaluate this work on 11 different DNN models. Compared to a recent inference-only work Spotlight, our method converges to a design in, on average, 31x less time and offers 12x higher throughput. Moreover, designs generated using our method achieve 12% throughput improvement over TPU architecture.

Training recommendation models pose significant challenges regarding resource utilization and performance. Prior research has proposed an approach that categorizes embeddings into popular and non-popular classes to reduce the training time for recommendation models. We observe that, even among the popular embeddings, certain embeddings undergo rapid training and exhibit minimal subsequent variation, resulting in saturation. Consequently, updates to these embeddings lack any contribution to model quality. This paper presents Slipstream, a software framework that identifies stale embeddings on the fly and skips their updates to enhance performance. This capability enables Slipstream to achieve substantial speedup, optimize CPU-GPU bandwidth usage, and eliminate unnecessary memory access. SlipStream showcases training time reductions of 2x, 2.4x, 1.2x, and 1.175x across real-world datasets and configurations, compared to Baseline XDL, Intel-optimized DRLM, FAE, and Hotline, respectively.

Transformers have emerged as the underpinning architecture for Large Language Models (LLMs). In generative language models, the inference process involves two primary phases: prompt processing and token generation. Token generation, which constitutes the majority of the computational workload, primarily entails vector-matrix multiplications and interactions with the Key-Value (KV) Cache. This phase is constrained by memory bandwidth due to the overhead of transferring weights and KV cache values from the memory system to the computing units. This memory bottleneck becomes particularly pronounced in applications that require long-context and extensive text generation, both of which are increasingly crucial for LLMs. This paper introduces "Keyformer", an innovative inference-time approach, to mitigate the challenges associated with KV cache size and memory bandwidth utilization. Keyformer leverages the observation that approximately 90% of the attention weight in generative inference focuses on a specific subset of tokens, referred to as "key" tokens. Keyformer retains only the key tokens in the KV cache by identifying these crucial tokens using a novel score function. This approach effectively reduces both the KV cache size and memory bandwidth usage without compromising model accuracy. We evaluate Keyformer's performance across three foundational models: GPT-J, Cerebras-GPT, and MPT, which employ various positional embedding algorithms. Our assessment encompasses a variety of tasks, with a particular emphasis on summarization and conversation tasks involving extended contexts. Keyformer's reduction of KV cache reduces inference latency by 2.1x and improves token generation throughput by 2.4x, while preserving the model's accuracy.

The smallest set of vertices needed to differentiate or categorize every other vertex in a graph is referred to as the graph's metric dimension. Finding the class of graphs for a particular given metric dimension is an NP-hard problem. This concept has applications in many different domains, including graph theory, network architecture, and facility location problems. A graph $G$ with order $n$ is known as a Toeplitz graph over the subset $S$ of consecutive collections of integers from one to $n$, and two vertices will be adjacent to each other if their absolute difference is a member of $S$. A graph $G(\mathbb{Z}_{n})$ is called a zero-divisor graph over the zero divisors of a commutative ring $\mathbb{Z}_{n}$, in which two vertices will be adjacent to each other if their product will leave the remainder zero under modulo $n$. Since the local fractional metric dimension problem is NP-hard, it is computationally difficult to identify an optimal solution or to precisely determine the minimal size of a local resolving set; in the worst case, the process takes exponential time. Different upper bound sequences of local fractional metric dimension are suggested in this article, along with a comparison analysis for certain families of Toeplitz and zero-divisor graphs. Furthermore, we note that the analyzed local fractional metric dimension upper bounds fall into three metric families: constant, limited, and unbounded., Comment: This is a preprint of a paper whose final and definite form is published Open Access in 'Symmetry', at [https://doi.org/10.3390/sym15101911]. Cite this paper as: A.S. Alali, S. Ali, M. Adnan and D.F.M. Torres, On Sharp Bounds of Local Fractional Metric Dimension for Certain Symmetrical Algebraic Structure Graphs, Symmetry 15 (2023), no. 10, Art. 1911, 25pp

In this paper, we devise a mechanism for the addition of multi-modal information with an existing pipeline for continuous sign language recognition and translation. In our procedure, we have incorporated optical flow information with RGB images to enrich the features with movement-related information. This work studies the feasibility of such modality inclusion using a cross-modal encoder. The plugin we have used is very lightweight and doesn't need to include a separate feature extractor for the new modality in an end-to-end manner. We have applied the changes in both sign language recognition and translation, improving the result in each case. We have evaluated the performance on the RWTH-PHOENIX-2014 dataset for sign language recognition and the RWTH-PHOENIX-2014T dataset for translation. On the recognition task, our approach reduced the WER by 0.9, and on the translation task, our approach increased most of the BLEU scores by ~0.6 on the test set.

Recommendation models are vital in delivering personalized user experiences by leveraging the correlation between multiple input features. However, deep learning-based recommendation models often face challenges due to evolving user behaviour and item features, leading to covariate shifts. Effective cross-feature learning is crucial to handle data distribution drift and adapting to changing user behaviour. Traditional feature interaction techniques have limitations in achieving optimal performance in this context. This work introduces Ad-Rec, an advanced network that leverages feature interaction techniques to address covariate shifts. This helps eliminate irrelevant interactions in recommendation tasks. Ad-Rec leverages masked transformers to enable the learning of higher-order cross-features while mitigating the impact of data distribution drift. Our approach improves model quality, accelerates convergence, and reduces training time, as measured by the Area Under Curve (AUC) metric. We demonstrate the scalability of Ad-Rec and its ability to achieve superior model quality through comprehensive ablation studies.

Parallel I/O refers to the ability of scientific programs to concurrently read/write from/to a single file from multiple processes executing on distributed memory platforms like compute clusters. In the HPC world, I/O becomes a significant bottleneck for many real-world scientific applications. In the last two decades, there has been significant research in improving the performance of I/O operations in scientific computing for traditional languages including C, C++, and Fortran. As a result of this, several mature and high-performance libraries including ROMIO (implementation of MPI-IO), parallel HDF5, Parallel I/O (PIO), and parallel netCDF are available today that provide efficient I/O for scientific applications. However, there is very little research done to evaluate and improve I/O performance of Java-based HPC applications. The main hindrance in the development of efficient parallel I/O Java libraries is the lack of a standard API (something equivalent to MPI-IO). Some adhoc solutions have been developed and used in proprietary applications, but there is no general-purpose solution that can be used by performance hungry applications. As part of this project, we plan to develop a Java-based parallel I/O API inspired by the MPI-IO bindings (MPI 2.0 standard document) for C, C++, and Fortran. Once the Java equivalent API of MPI-IO has been developed, we will develop a reference implementation on top of existing Java messaging libraries. Later, we will evaluate and compare performance of our reference Java Parallel I/O library with C/C++ counterparts using benchmarks and real-world applications., Comment: 10 pages

A modern-day society demands resilient, reliable, and smart urban infrastructure for effective and in telligent operations and deployment. However, unexpected, high-impact, and low-probability events such as earthquakes, tsunamis, tornadoes, and hurricanes make the design of such robust infrastructure more complex. As a result of such events, a power system infrastructure can be severely affected, leading to unprecedented events, such as blackouts. Nevertheless, the integration of smart grids into the existing framework of smart cities adds to their resilience. Therefore, designing a resilient and reliable power system network is an inevitable requirement of modern smart city infras tructure. With the deployment of the Internet of Things (IoT), smart cities infrastructures have taken a transformational turn towards introducing technologies that do not only provide ease and comfort to the citizens but are also feasible in terms of sustainability and dependability. This paper presents a holistic view of a resilient and sustainable smart city architecture that utilizes IoT, big data analytics, unmanned aerial vehicles, and smart grids through intelligent integration of renew able energy resources. In addition, the impact of disasters on the power system infrastructure is investigated and different types of optimization techniques that can be used to sustain the power flow in the network during disturbances are compared and analyzed. Furthermore, a comparative review analysis of different data-driven machine learning techniques for sustainable smart cities is performed along with the discussion on open research issues and challenges.

The spin-selective reflection to introduce chirality which can have a lot of applications in real life such as spectroscopy, optical setups, media industry etc. In this paper, a reflection based metasurface proposed to introduce the giant chiroptical effects at broadband visible and infrared (IR) regimes. The optimization and results of basic unit also termed as nanostructure are demonstrated here. The reflectance at the optimal parameters for the proposed nanostructure shows the inclusion of multiband giant chiroptical effects in reflection mode. The results show that this metasurface can elicit large spin-selective reflection coefficients with moderate chirality covering the broadband wavelength. The circular dichroism in the visible and IR regime shows its potential applicability for a lot of applications in our daily life. This work also provides a new approach to achieve giant Spin Hall Effect at broadband wavelength ranges with low loss., Comment: addition of essential technical content

Recommendation models rely on deep learning networks and large embedding tables, resulting in computationally and memory-intensive processes. These models are typically trained using hybrid CPU-GPU or GPU-only configurations. The hybrid mode combines the GPU's neural network acceleration with the CPUs' memory storage and supply for embedding tables but may incur significant CPU-to-GPU transfer time. In contrast, the GPU-only mode utilizes High Bandwidth Memory (HBM) across multiple GPUs for storing embedding tables. However, this approach is expensive and presents scaling concerns. This paper introduces Hotline, a heterogeneous acceleration pipeline that addresses these concerns. Hotline develops a data-aware and model-aware scheduling pipeline by leveraging the insight that only a few embedding entries are frequently accessed (popular). This approach utilizes CPU main memory for non-popular embeddings and GPUs' HBM for popular embeddings. To achieve this, Hotline accelerator fragments a mini-batch into popular and non-popular micro-batches. It gathers the necessary working parameters for non-popular micro-batches from the CPU, while GPUs execute popular micro-batches. The hardware accelerator dynamically coordinates the execution of popular embeddings on GPUs and non-popular embeddings from the CPU's main memory. Real-world datasets and models confirm Hotline's effectiveness, reducing average end-to-end training time by 2.2x compared to Intel-optimized CPU-GPU DLRM baseline., Comment: Accepted at The International Symposium on Computer Architecture (ISCA), 2024