Search

Reinforcement learning augmented by the representational power of deep neural networks, has shown promising results on high-dimensional problems, such as game playing and robotic control. However, the sequential nature of these problems poses a fundamental challenge for computational efficiency. Recently, alternative approaches such as evolutionary strategies and deep neuroevolution demonstrated competitive results with faster training time on distributed CPU cores. Here, we report record training times (running at about 1 million frames per second) for Atari 2600 games using deep neuroevolution implemented on distributed FPGAs. Combined hardware implementation of the game console, image pre-processing and the neural network in an optimized pipeline, multiplied with the system level parallelism enabled the acceleration. These results are the first application demonstration on the IBM Neural Computer, which is a custom designed system that consists of 432 Xilinx FPGAs interconnected in a 3D mesh network topology. In addition to high performance, experiments also showed improvement in accuracy for all games compared to the CPU-implementation of the same algorithm., Comment: 12 pages. Submitted to ACM Journal on Emerging Technologies in Computing Systems: Special Issue on Hardware and Algorithms for Efficient Machine Learning

The IBM Neural Computer (INC) is a highly flexible, re-configurable parallel processing system that is intended as a research and development platform for emerging machine intelligence algorithms and computational neuroscience. It consists of hundreds of programmable nodes, primarily based on Xilinx's Field Programmable Gate Array (FPGA) technology. The nodes are interconnected in a scalable 3d mesh topology. We overview INC, emphasizing unique features such as flexibility and scalability both in the types of computations performed and in the available modes of communication, enabling new machine intelligence approaches and learning strategies not well suited to the matrix manipulation/SIMD libraries that GPUs are optimized for. This paper describes the architecture of the machine and applications are to be described in detail elsewhere., Comment: 8 pages, 5 figures. Submitted to IEEE Transactions on Parallel and Distributed Systems

Artificial neural networks have diverged far from their early inspiration in neurology. In spite of their technological and commercial success, they have several shortcomings, most notably the need for a large number of training examples and the resulting computation resources required for iterative learning. Here we describe an approach to neurological network simulation, both architectural and algorithmic, that adheres more closely to established biological principles and overcomes some of the shortcomings of conventional networks., Comment: 20 pages, 5 figures

Transfer learning has become the de facto standard in computer vision and natural language processing, especially where labeled data is scarce. Accuracy can be significantly improved by using pre-trained models and subsequent fine-tuning. In visual reasoning tasks, such as image question answering, transfer learning is more complex. In addition to transferring the capability to recognize visual features, we also expect to transfer the system's ability to reason. Moreover, for video data, temporal reasoning adds another dimension. In this work, we formalize these unique aspects of transfer learning and propose a theoretical framework for visual reasoning, exemplified by the well-established CLEVR and COG datasets. Furthermore, we introduce a new, end-to-end differentiable recurrent model (SAMNet), which shows state-of-the-art accuracy and better performance in transfer learning on both datasets. The improved performance of SAMNet stems from its capability to decouple the abstract multi-step reasoning from the length of the sequence and its selective attention enabling to store only the question-relevant objects in the external memory., Comment: 18 pages; more baseline comparisons; additional clarifications

In this working notes paper, we describe IBM Research AI (Almaden) team's participation in the ImageCLEF 2019 VQA-Med competition. The challenge consists of four question-answering tasks based on radiology images. The diversity of imaging modalities, organs and disease types combined with a small imbalanced training set made this a highly complex problem. To overcome these difficulties, we implemented a modular pipeline architecture that utilized transfer learning and multi-task learning. Our findings led to the development of a novel model called Supporting Facts Network (SFN). The main idea behind SFN is to cross-utilize information from upstream tasks to improve the accuracy on harder downstream ones. This approach significantly improved the scores achieved in the validation set (18 point improvement in F-1 score). Finally, we submitted four runs to the competition and were ranked seventh., Comment: Working notes from the ImageCLEF 2019 VQA-Med competition

We introduce a variant of the MAC model (Hudson and Manning, ICLR 2018) with a simplified set of equations that achieves comparable accuracy, while training faster. We evaluate both models on CLEVR and CoGenT, and show that, transfer learning with fine-tuning results in a 15 point increase in accuracy, matching the state of the art. Finally, in contrast, we demonstrate that improper fine-tuning can actually reduce a model's accuracy as well., Comment: Paper accepted for Visually Grounded Interaction and Language (ViGIL) Workshop, NIPS 2018, Montreeal, Canada

Typical neural networks with external memory do not effectively separate capacity for episodic and working memory as is required for reasoning in humans. Applying knowledge gained from psychological studies, we designed a new model called Differentiable Working Memory (DWM) in order to specifically emulate human working memory. As it shows the same functional characteristics as working memory, it robustly learns psychology inspired tasks and converges faster than comparable state-of-the-art models. Moreover, the DWM model successfully generalizes to sequences two orders of magnitude longer than the ones used in training. Our in-depth analysis shows that the behavior of DWM is interpretable and that it learns to have fine control over memory, allowing it to retain, ignore or forget information based on its relevance., Comment: 20 pages

We propose a new architecture called Memory-Augmented Encoder-Solver (MAES) that enables transfer learning to solve complex working memory tasks adapted from cognitive psychology. It uses dual recurrent neural network controllers, inside the encoder and solver, respectively, that interface with a shared memory module and is completely differentiable. We study different types of encoders in a systematic manner and demonstrate a unique advantage of multi-task learning in obtaining the best possible encoder. We show by extensive experimentation that the trained MAES models achieve task-size generalization, i.e., they are capable of handling sequential inputs 50 times longer than seen during training, with appropriately large memory modules. We demonstrate that the performance achieved by MAES far outperforms existing and well-known models such as the LSTM, NTM and DNC on the entire suite of tasks., Comment: 16 pages

A study of ripple formation on sapphire surfaces by 300-2000 eV Ar+ ion bombardment is presented. Surface characterization by in-situ synchrotron grazing incidence small angle x-ray scattering and ex-situ atomic force microscopy is performed in order to study the wavelength of ripples formed on sapphire (0001) surfaces. We find that the wavelength can be varied over a remarkably wide range-nearly two orders of magnitude-by changing the ion incidence angle. Within the linear theory regime, the ion induced viscous flow smoothing mechanism explains the general trends of the ripple wavelength at low temperature and incidence angles larger than 30. In this model, relaxation is confined to a few-nm thick damaged surface layer. The behavior at high temperature suggests relaxation by surface diffusion. However, strong smoothing is inferred from the observed ripple wavelength near normal incidence, which is not consistent with either surface diffusion or viscous flow relaxation., Comment: Revtex4, 19 pages, 10 figures with JPEG format