Search

Today’s deep learning models face an increasing demand to handle dynamic shape tensors and computation whose shape information remains unknown at compile time and varies in a nearly infinite range at runtime. This shape dynamism brings tremendous challenges for existing compilation pipelines designed for static models which optimize tensor programs relying on exact shape values. This paper presents TSCompiler, an end-to-end compilation framework for dynamic shape models. TSCompiler first proposes a symbolic shape propagation algorithm to recover symbolic shape information at compile time to enable subsequent optimizations. TSCompiler then partitions the shape-annotated computation graph into multiple subgraphs and fine-tunes the backbone operators from the subgraph within a hardware-aligned search space to find a collection of high-performance schedules. TSCompiler can propagate the explored backbone schedule to other fusion groups within the same subgraph to generate a set of parameterized tensor programs for fused cases based on dependence analysis. At runtime, TSCompiler utilizes an occupancy-targeted cost model to select from pre-compiled tensor programs for varied tensor shapes. Extensive evaluations show that TSCompiler can achieve state-of-the-art speedups for dynamic shape models. For example, we can improve kernel efficiency by up to 3.97× on NVIDIA RTX3090, and 10.30 × on NVIDIA A100 and achieve up to five orders of magnitude speedups on end-to-end latency. [ABSTRACT FROM AUTHOR]

Training with unlabeled datasets using self-supervised models has the edge over training with labeled datasets, reducing human effort, and no need for annotated data during training. The knowledge obtained from the pre-trained self-supervised model can be transferred to an image classifier as a target task. Transferring knowledge from source to target task requires tuning a large number of hyperparameters, such as number of CNN layers, number of filters in each CNN layer, kernel size, stride, number of fully connected (FC) layers, units in each FC layer, dropouts, learning rate, and many more. An efficient process is required to automatically tune these hyperparameters to reduce the manual efforts and enhance the performance of the self-supervised model. This paper uses a pre-trained self-supervised model to transfer knowledge for image classification. We propose an efficient Bayesian optimization-based method for automatically tuning (autotuning) the hyperparameters of the self-supervised model during the knowledge transfer. The proposed autotuned image classifier consists of a few CNN layers followed by an FC layer. Finally, we use a softmax layer to obtain the probability of classes for the images. To evaluate the performance of the proposed method, state-of-the-art self-supervised models such as SimClr, SWAV, and SWAV 2x are used to extract the learned features. According to our experiments on three benchmarked datasets (CIFAR-10, CIFAR-100, and Tiny Imagenet datasets), the proposed method not only enhances the performances of the self-supervised models, but also provides state-of-the-art results. [ABSTRACT FROM AUTHOR]

PARCSIM is a parallel software simulator that allows a user to capture, through a graphical interface, matrix algorithm schemes that solve scientific problems. With this tool, the user can analyse the execution times that would be obtained by using different spatio-temporal mapping of computational tasks on available computational units, parallelism parameters and computational libraries. Furthermore, for complex problem models, the self-optimization engine incorporated in this tool analyses the huge tree of possible calculations grouping and mapping strategies in search of the choice that makes the best use of the available hardware resources. This tool also offers polyalgorithmic resolution by making automatically the best decision between different software approaches to solve a given problem on the hardware system available. This work shows the usefulness of this simulator to efficiently solve hierarchical problems constructed from previously modelled subproblems. This task is performed by reusing, in a scalable way, the optimization information of these subproblems to establish the best execution configuration for the composite problem. [ABSTRACT FROM AUTHOR]

The drug discovery process involves several tasks to be performed in vivo, in vitro and in silico. Molecular docking is a task typically performed in silico. It aims at finding the three-dimensional pose of a given molecule when it interacts with the target protein binding site. This task is often done for virtual screening a huge set of molecules to find the most promising ones, which will be forwarded to the later stages of the drug discovery process. Given the huge complexity of the problem, molecular docking cannot be solved by exploring the entire space of the ligand poses. State-of-the-art approaches face the problem by sampling the space of the ligand poses to generate results in a reasonable time budget. In this work, we improve the geometric approach to molecular docking by introducing tunable approximations. In particular, we analysed and enriched the original implementation with tunable software knobs to explore and control the performance-accuracy trade-offs. We modelled time-to-solution of the virtual screening task as a function of software knobs, input data features, and available computational resources. Therefore, the application can autotune its configuration according to a user-defined time budget. We used a Mini-App derived by LiGenDock—a state-of-the-art molecular docking application—to validate the proposed approach. We run the enhanced Mini-App on a high-performance computing system by using a very large database of pockets and ligands. The proposed approach exposes a time-to-solution interval spanning more than one order of magnitude with accuracy degradation up to 30%, more in general providing different accuracy levels according to the needs of the virtual screening campaign. [ABSTRACT FROM AUTHOR]

In this paper, an adaptive swarm learning process (SLP) algorithm for designing the optimal proportional integral and derivative (PID) parameter for a multiple-input multiple-output (MIMO) control system is proposed. The SLP algorithm is proposed to improve the performance and convergence of PID parameter autotuning by applying the swarm algorithm and the learning process. The adaptive SLP algorithm improves the stability, performance and robustness of the traditional SLP algorithm to apply it to a MIMO control system. It can update the online weights of the SLP algorithm caused by the errors in the settling time, rise time and overshoot of the system based on a stable learning rate. The gradient descent is applied to update the weights. The stable learning rate is verified based on the Lyapunov stability theorem. Additionally, simulations are performed to verify the superiority of the algorithm in terms of performance and robustness. Results that compare the adaptive SLP algorithm with the traditional SLP, a neural network (NN), the genetic algorithm (GA), the particle swarm and optimization (PSO) algorithm and the kidney-inspired algorithm (KIA) based on a two-wheel inverted pendulum system are presented. With respect to performance and robustness, the adaptive SLP algorithm provides a better response than the traditional SLP, NN, GA, PSO and KIA. [ABSTRACT FROM AUTHOR]

A hierarchical approach for autotuning linear algebra routines on heterogeneous platforms is presented. Hierarchy helps to alleviate the difficulties of tuning parallel routines for high-performance computing systems. This paper analyzes the application of the hierarchical approach at both the hardware and software levels, using the basic matrix multiplication and the Strassen multiplication as proof of concept on multicore+coprocessor nodes. In this way, the hierarchical approach allows partial delegation of the efficient exploitation of the computing units in the node to the underlying direct autotuned matrix multiplication used in the base case. [ABSTRACT FROM AUTHOR]

This paper presents an enhanced auto-optimization method to run the 3D-Fast Wavelet Transform on different computing units in a system (GPU, MIC, CPU). The proposed method automatically selects a set of parameter values (block size, number of streams and number of threads) in order to reduce the total execution time, obtaining performances close to the optimal and decreasing the number of evaluations needed. [ABSTRACT FROM AUTHOR]

Opal is a general-purpose system for modeling and solving algorithm optimization problems. Opal takes an algorithm as input, and as output it suggests parameter values that maximize some user-defined performance measure. In order to achieve this, the user provides a Python script describing how to launch the target algorithm, and defining the performance measure. Opal then models this question as a blackbox optimization problem which is then solved by a state-of-the-art direct search solver. Opal handles a wide variety of parameter types, it can exploit multiple processors in parallel at different levels, and can take advantage of a surrogate blackbox. Several features of Opal are illustrated on a problem consisting in the design of a hybrid sort strategy. [ABSTRACT FROM AUTHOR]

This paper introduces a formal model of a method for automated adjustment of parallel applications (autotuning). The software implementation of this model is described in the form of a flexible software framework for automatic generation of autotuners using an underlain term rewriting system and expert knowledge as a source of optimizing transformations. [ABSTRACT FROM AUTHOR]

In this paper, we present Patus, a code generation and auto-tuning framework for stencil computations targeted at multi- and manycore processors, such as multicore CPUs and graphics processing units. Patus, which stands for ' Parallel Auto tuned Stencils,' generates a compute kernel from a specification of the stencil operation and a strategy which describes the parallelization and optimization to be applied, and leverages the autotuning methodology to optimize strategy-specific parameters for the given hardware architecture. [ABSTRACT FROM AUTHOR]

Sustained oscillations are obtained by putting relays in the feedback loop, which provide ultimate periods and gains of processes that can be used to tune PID controllers. The ultimate periods obtained by conventional relay autotuning have relative errors up to 5% for first order plus time delay processes. To improve the estimation of ultimate periods, modified relays such as a saturation relay and a relay with preload have been available. However, these modifications lose the binary (on-off) property of the conventional relay. Here relays with multiple switching which produce pulse-widthmodulation (PWM) signals are proposed. They retain the binary property and show improved identification accuracies of ultimate parameters. [ABSTRACT FROM AUTHOR]

For the automatic tuning of PID controllers, a new identification method is proposed to estimate the frequency responses of the process from the activated process input and output. It can extract many more frequency responses as well as guarantee better accuracy compared with the previous describing function analysis algorithm. Also, the proposed method can be applied to the case that the initial part of the activated process input and output is periodic (cyclic-steady-state), whereas the previous method using the modified Fourier transform cannot incorporate the case. [ABSTRACT FROM AUTHOR]

We propose a new and simple on-line process identification method for the automatic tuning of the PID controller. It does not require a special type of test signal generators such as relay or P controller only if the signals are persistently exciting. That is, a user can choose arbitrary signal generators such as relay, a P controller, the controller itself, pulse signal and step signal generator because it needs only the measured process output and the controller output. It can incorporate nonlinearities due to actuator saturation or manual mode operation during identification work and shows a good robustness to measurement noises, nonlinearity of the process and disturbances. The proposed autotuner combined with the identification method and tuning rule using a model reduction shows good control properties compared with previous autotuning methods. [ABSTRACT FROM AUTHOR]