Search

We propose algorithms and techniques to accelerate training of deep neural networks for action recognition on a cluster of GPUs. The convergence analysis of our algorithm shows it is possible to reduce communication cost and at the same time minimize the number of iterations needed for convergence. We customize the Adam optimizer for our distributed algorithm to improve efficiency. In addition, we employ transfer-learning to further reduce training time while improving validation accuracy. For the UCF101 and HMDB51 datasets, the validation accuracies achieved are 93.1% and 67.9% respectively. With an additional end-to-end trained temporal stream, the validation accuracies achieved for UCF101 and HMDB51 are 93.47% and 81.24% respectively. As far as we know, these are the highest accuracies achieved with the two-stream approach using ResNet that does not involve computationally expensive 3D convolutions or pretraining on much larger datasets.

A high-speed and power-efficient heterogeneous octa-core CPU complex is realized in a 7nm FinFET process. On-chip sensors that handle guard band in terms of process $(V_{PGB})$, voltage, temperature $(V_{TGB})$, and aging $(V_{AGB})$ are designed for power enhancement in dynamic frequency and voltage scaling (DVFS). A process sensor (P-sensor) is used to determine per-die V PGB for Smartphone operation voltage $(V_{sov})$ with machine learning (ML). A temperature sensor (T-sensor) senses run-time local thermal and temperature inversion (T-inversion) to scale V TGB in DVFS. An aging sensor (A-sensor) converts cell degradation to corresponding V AGB for aging compensation. Lastly, a voltage sensor (V-sensor) sustains system stability against supply noise. With sensor assistance, the CPU performs up to 2.8GHz speed with 13% power improvement in a fully integrated 5G multi-mode Smartphone SoC that supports the sub-6GHz band with 2.6Gbps upload and 4.7Gbps download speeds.

This paper describes a new CPU subsystem featured in a 5G mobile SoC. The High-Performance (HP) core achieves a 3GHz clock frequency with full production yield across the fabrication range and operating environment. In contrast to previously published work [1], a third, balanced-performance (BP), gear is introduced which features a power-optimized implementation of the high-performance (HP) core. Physical implementation differences of the BP and HP cores are illustrated, while circuit techniques developed to enable full-yield 3GHz operation are detailed. A die photograph alongside a more detailed HP core CAD drawing are shown in Fig. 4.1.7 and the cluster topology, including architectural features, is summarized in Fig. 4.1.1.

Productivity from day one on supercomputers that leverage new technologies requires significant preparation. An institution that procures a novel system architecture often lacks sufficient institutional knowledge and skills to prepare for it. Thus, the "Center of Excellence" (CoE) concept has emerged to prepare for systems such as Summit and Sierra, currently the top two systems in the Top 500. This paper documents CoE experiences that prepared a workload of diverse applications and math libraries for a heterogeneous system. We describe our approach to this preparation, including our management and execution strategies, and detail our experiences with and reasons for using different programming approaches. Our early science and performance results show that the project enabled significant early seismic science with up to a l4X throughput increase over Cori. In addition to our successes, we discuss our challenges and failures so others may benefit from our experience.

Detecting actions in videos requires understanding the temporal relationships among frames. Typical action recognition approaches rely on optical flow estimation methods to convey temporal information to a CNN. Recent studies employ 3D convolutions in addition to optical flow to process the temporal information. While these models achieve slightly better results than two-stream 2D convolutional approaches, they are significantly more complex, requiring more data and time to be trained. We propose an efficient, adaptive batch size distributed training algorithm with customized optimizations for training the two 2D streams. We introduce a new 2D convolutional temporal stream that is trained end-to-end with a neural network. The flexibility to freeze some network layers from training in this temporal stream brings the possibility of ensemble learning with more than one temporal streams. Our architecture that combines three streams achieves the highest accuracies as we know of on UCF101 and HMDB51 by systems that do not pretrain on much larger datasets (e.g., Kinetics). We achieve these results while keeping our spatial and temporal streams 4.67x faster to train than the 3D convolution approaches.

Due to the additional temporal dimension, large-scale video action recognition is even more challenging than image recognition and typically takes days to train on modern GPUs even for modest-sized datasets. We propose algorithms and techniques to accelerate training of deep neural networks for action recognition on a cluster of GPUs. In terms of convergence and scaling, our distributed training algorithm with adaptive batch size is provably superior to popular asynchronous stochastic gradient descent algorithms. The convergence analysis of our algorithm shows it is possible to reduce communication cost and at the same time minimize the number of iterations needed for convergence. We customize the Adam optimizer for our distributed algorithm to improve efficiency. In addition, we employ transfer-learning to further reduce training time while improving validation accuracy. Compared with the base-line single-GPU stochastic gradient descent implementation of the two-stream training approach, our implementation achieves super-linear speedups on 16 GPUs while improving validation accuracy. For the UCFI0l and HMDB51 datasets, the validation accuracies achieved are 93.1 % and 67.9% respectively. As far as we know, these are the highest accuracies achieved with the two-stream approach that does not involve computationally expensive 3D convolutions or pretraining on much larger datasets.

Image annotation, or prediction of multiple tags for an image, is a challenging task. Most current algorithms are based on large sets of handcrafted features. Deep convolutional neural networks have recently outperformed humans in image classification, and these networks can be used to extract features highly predictive of an image's tags. In this study, we analyze semantic information in features derived from two pre-trained deep network classifiers by evaluating their performance in nearest neighbor-based approaches to tag prediction. We generally exceed performance of the manual features when using the deep features. We also find complementary information in the manual and deep features when used in combination for image annotation.

Machine learning is finding increasingly broad applications in the physical sciences. This most often involves building a model relationship between a dependent, measurable output, and an associated set of controllable, but complicated, independent inputs. We present a tutorial on current techniques in machine learning—a jumping-off point for interested researchers to advance their work. We focus on deep neural networks with an emphasis on demystifying deep learning. We begin with background ideas in machine learning and some example applications from current research in plasma physics. We discuss supervised learning techniques for modeling complicated functions, beginning with familiar regression schemes, and then advancing to more sophisticated deep learning methods. We also address unsupervised learning and techniques for reducing the dimensionality of input spaces. Along the way, we describe methods for practitioners to help ensure that their models generalize from their training data to as-yet-unseen test data. We describe classes of tasks—predicting scalars, handling images, and fitting time-series—and prepare the reader to choose an appropriate technique. We finally point out some limitations to modern machine learning and speculate on some ways that practitioners from the physical sciences may be particularly suited to help.

The heat flux distributions on divertor targets in H-mode plasmas are serious concerns for future devices. We seek to simulate the tokamak boundary plasma turbulence and heat transport in the edge localized mode-suppressed regimes. The improved BOUT++ model shows that not only Ip but also the radial electric field Er plays an important role on the turbulence behavior and sets the heat flux width. Instead of calculating Er from the pressure gradient term (diamagnetic Er), it is calculated from the plasma transport equations with the sheath potential in the scrape-off layer and the plasma density and temperature profiles inside the separatrix from the experiment. The simulation results with the new Er model have better agreement with the experiment than using the diamagnetic Er model: (1) The electromagnetic turbulence in enhanced Dα H-mode shows the characteristics of quasi-coherent modes (QCMs) and broadband turbulence. The mode spectra are in agreement with the phase contrast imaging data and almost has no change in comparison to the cases which use the diamagnetic Er model; (2) the self-consistent boundary Er is needed for the turbulence simulations to get the consistent heat flux width with the experiment; (3) the frequencies of the QCMs are proportional to Er, while the divertor heat flux widths are inversely proportional to Er; and (4) the BOUT++ turbulence simulations yield a similar heat flux width to the experimental Eich scaling law and the prediction from the Goldston heuristic drift model.

With increasing understanding and availability of nuclear technologies, and increasing persuasion of nuclear technologies by several new countries, it is increasingly becoming important to monitor the nuclear proliferation activities. There is a great need for developing technologies to automatically or semi-automatically detect nuclear proliferation activities using remote sensing. Images acquired from earth observation satellites is an important source of information in detecting proliferation activities. High-resolution remote sensing images are highly useful in verifying the correctness, as well as completeness of any nuclear program. DOE national laboratories are interested in detecting nuclear proliferation by developing advanced geospatial image mining algorithms. In this paper we describe the current understanding of geospatial image mining techniques and enumerate key gaps and identify future research needs in the context of nuclear proliferation.

The double-beam interferometric method is applied to measure the field-induced displacement of Pb(Zr, Ti)O3 thin films. The dc electric field dependence of the longitudinal piezoelectric coefficient (d 33) response of Pb(Zr, Ti)O3 thin films deposited by metal organic chemical vapor deposition (MOCVD) was measured. Experimental d 33 values were compared with coefficients calculated using a phenomenological approach and bulk parameters. Qualitative agreement was obtained between measured and calculated coefficients.