Search

We present AB9, a neural processor for inference acceleration. AB9 consists of a systolic tensor core (STC) neural network accelerator designed to accelerate artificial intelligence applications by exploiting the data reuse and parallelism characteristics inherent in neural networks while providing fast access to large on‐chip memory. Complementing the hardware is an intuitive and user‐friendly development environment that includes a simulator and an implementation flow that provides a high degree of programmability with a short development time. Along with a 40‐TFLOP STC that includes 32k arithmetic units and over 36 MB of on‐chip SRAM, our baseline implementation of AB9 consists of a 1‐GHz quad‐core setup with other various industry‐standard peripheral intellectual properties. The acceleration performance and power efficiency were evaluated using YOLOv2, and the results show that AB9 has superior performance and power efficiency to that of a general‐purpose graphics processing unit implementation. AB9 has been taped out in the TSMC 28‐nm process with a chip size of 17 × 23 mm2. Delivery is expected later this year.

In this demonstration, we present AB9 SoC system, a single-chip solution for AI application. It provides the reconfigurable and programmable architecture to support the general computations for a variety of neural networks. The AB9 SoC is implemented using TSMC 28-nm process technology with a chip size of 17×23 mm2 and 1GHz operating frequency.

HPL(High Performance Linpack) is the standard benchmark used to evaluate supercomputers (high-performance computing systems) around the world. HPL solves a linear system of equations, Ax=b, through a series of mathematical processes such as 2D Block-Cyclic matrix distribution and LU Decomposition. In order to achieve the best performance, optimization of key HPL parameters is essential. In this paper, we propose an optimization analysis technique of HPL parameters based on HPLinpack performance results and the efficiency graph pattern on a homogeneous system consisting of the Neoverse N1SDP(System Development Platform) from ARM. Results show the significant influence of GEMM operations on performance and the need for its acceleration were further verified through HPL function call profiling. Finally, we show the effectiveness of the proposed parameter optimization methods on a heterogeneous system. A HPL modified for heterogenous system exhibited considerably improved performance when tested on the Neoverse N1SDP and the Nvidia RTX 2060 GPU.

AI processors are extending the application area into mobile and edge devices. The requirement of low power consumption which has been an essential factor in designing processors is now becoming the most critical factor for mobile AI processors to be viable. The high performance requirement exacerbates the energy crisis caused by a large area due to a lot of processing engines required for implementing AI processors. We present the design of an AI processor targeting both CNN and MLP processing in autonomous vehicles. The proposed AI processor integrates Super-Thread-Core composed of 16384 nano cores in mesh-grid network for neural network acceleration. The performance of the processor reaches 32 Tera FLOPS enabling hyper real-time execution of CNN and MLP. Each nano core is programmable by a sequence of instructions compiled from the neural network description by the proprietary AI-Ware. The mesh-array of nano cores at the heart of neural computing accounts for most of the power consumption. The AI-ware compiler enables adaptive power gating by dynamically compiling the commands based on the temperature profile reducing 50% of total power consumption.

An implementation of Yolo-v2 image recognition and other testbenches for a deep learning accelerator is presented. This chip is the initial version of our on-going effort for a higher performance accelerator development. The accelerator is based on a systolic array and can handle convolution and max-pooling layer in a combined way or separately using 16 bit floating-point data. It also supports inner-product and LSTM layers. For demonstration and as one of the design verification testbenches, we implemented Yolo-v2 image recognition for 80 object classes. We converted the Yolo-v2 software to 16 bit floating point version and used it in the simulation and FPGA experiment during the chip development. Several other testbenches were designed and used to test various networks. In this paper, the accelerator’s architecture and the Yolo-v2 implementation on our FPGA board including camera and LCD is explained, including the software implementation. Some analyses on the measured and estimated performance is shown.

State-of-the-art neural network accelerators consist of arithmetic engines organized in a mesh structure datapath surrounded by memory blocks that provide neural data to the datapath. While server-based accelerators coupled with server-class processors are accommodated with large silicon area and consume large amounts of power, electronic control units in autonomous driving vehicles require power-optimized, ‘AI processors’ with a small footprint. An AI processor for mobile applications that integrates general-purpose processor cores with mesh-structured neural network accelerators and high speed memory while achieving high-performance with low-power and compact area constraints necessitates designing a novel AI processor architecture. We present the design of an AI processor for electronic systems in autonomous driving vehicles targeting not only CNN-based object recognition but also MLP-based in-vehicle voice recognition. The AI processor integrates Super-Thread-Cores (STC) for neural network acceleration with function-safe general purpose cores that satisfy vehicular electronics safety requirements. The STC is composed of 16384 programmable nano-cores organized in a mesh-grid structured datapath network. Designed based on thorough analysis of neural network computations, the nano-core-in-memory architecture enhances computation intensity of STC with efficient feeding of multi-dimensional activation and kernel data into the nano-cores. The quad function-safe general purpose cores ensure functional safety of Super-Thread-Core to comply with road vehicle safety standard ISO 26262. The AI processor exhibits 32 Tera FLOPS, enabling hyper real-time execution of CNN, RNN, and FCN.

We present a processing platform that implements DMR with separate clock and power sources to prevent dependent failures working with a reconfigurable cache that includes BIST with self-recovering function to detect transient faults and error prediction to prevent permanent faults. The fault tolerant processor is analyzed to be complying with the ISO26262 SOTIF for ADAS SoC which is fabricated with 28nm CMOS Technology. The single point faults metric is 99.64% with a safety mechanism.

Multiresolution Gabor filter banks are used for feature extraction in a variety of applications as Gabor filters have shown to be exceptional feature extractors with a close correspondence to the simple cells in the primary visual cortex (V1) of the brain. Yet applying the Gabor filter is a computationally intensive task. Most applications that utilize the Gabor feature space require real time results; however, the large quantity of computations involved has hindered systems from achieving real time performance. The natural solution for such compute intensive tasks is parallelization. FPGAs have emerged as attractive platforms for compute intensive signal processing applications due to their massively parallel computation resources as well as low power consumption and affordability. We present a configurable architecture for Gabor feature extraction on FPGA that enhances the resource utilization of the FPGA hardware fabric while maintaining a streaming data flow to yield exceptional performance. The increased resource utilization resulting from configurability, optimizations, and resource sharing allows for higher levels of parallelism to achieve real time feature extraction of high resolution images. Two architectures are introduced. The first is an architecture for multiresolution feature extraction with extensive resource sharing for enhanced resource utilization. The second is an architecture for many-orientation applications using a coarse to fine grain method to enhance resource utilization by reducing the number of filters applied at different orientations. Our results show that our multiresolution implementation achieves real-time performance on 2048?×?1526 images and exhibits 6X speed up over a GPU implementation while exhibiting energy efficiency with 0.4fps/W compared to the GPU that achieves 0.036fps/W.[1] The implementation for many-orientation applications using the coarse to fine grain method exhibits resource saving of at most 2 O $$ 2\sqrt{O} $$ for O number of orientations and higher, compared to a fully parallel architecture and 25× speedup compared to a GPU implementation for 16 orientations.

Video applications are becoming ubiquitous in mobile and embedded systems. Wearable video systems such as Google Glasses require capabilities for real-time video analytics and prolonged battery lifetimes. Further, the increasing resolution of image sensors in these mobile systems places an increasing demand on both the memory storage as well as the computational power. In this work, we present the Refresh Enabled Video Analytics (REVA) system, an embedded architecture for multi- object scene understanding and tackle the unique opportunities provided by real-time embedded video analytics applications for reducing the DRAM memory refresh energy. We compare our design with the existing design space and show savings of 88% in refresh power and 15% in total power, as compared to a standard DRAM refresh scheme.

A mobile robot operating in a netcentric environment can utilize offboard resources on the network to improve its local perception. One such offboard resource is a world model built and maintained by other sensor systems. In this paper we present results from research into improving the performance of Deformable Parts Model object detection algorithms by using an offboard 3D world model. Experiments were run for detecting both people and cars in 2D photographs taken in an urban environment. After generating candidate object detections, a 3D world model built from airborne Light Detection and Ranging (LIDAR) and aerial photographs was used to filter out false alarm using several types of geometric reasoning. Comparison of the baseline detection performance to the performance after false alarm filtering showed a significant decrease in false alarms for a given probability of detection.

We present a prototype system which enables users to explore the global structure for digital imagery archives as well as drill-down into individual pictures. Our search engine builds upon computer vision advances made over the past decade in low-level feature matching, large data handling and object recognition. We demonstrate hierarchical clustering among images semi-cooperatively shot around MIT, automatic linking of flickr photos and aerial frames from the Grand Canyon, and video segment identification for a TV broadcast. Moreover, our software tools incorporate visible vs infrared band selection, color content quantization and human face detection. Ongoing and future extensions of this image search system are discussed.

Multiresolution Gab or filters are used for feature extraction for a variety of applications. Most hardware implementations have focused on iterative mechanisms on fixed hardware for implementing the different levels of resolution. In contrast, we present a configurable architecture that enhances the resource utilization of the hardware fabric. Our results show that our implementation achieves real-time performance on 2048x1536 images and exhibits 6 times speed up over a GPU implementation. Further, our FPGA implementation achieves an energy-efficiency of processing 0.4 fps/W as compared to the GPU that achieves 0.036 fps/W.

In this paper we present Vortex: a reconfigurable Network-on-Chip platform suitable for implementing domain-specific hardware accelerators in a design efficient manner. Our Vortex platform provides a flexible means to compose domain-specific accelerators for streaming applications such as performance critical machine vision systems. By substituting a traditional shared-bus architecture with low latency packet-switched routers and high utility network adaptors, maximum performance is exploited with minimal regard to communication infrastructure design and validation. To highlight the utility of the Vortex platform we present a case study in which a video analytics pipeline is mapped onto a multi-FPGA system. The system meets real-time throughput requirements on 3 Megapixel 48-bit image sequences with minimal resource overhead attributed to the Vortex communication infrastructure.

Working with 2.3K+ digital images shot around MIT, we form a SIFT graph which imposes an initial topological ordering upon the quasi-random set of input urban photos. We next employ iterative bundle adjustment algorithms developed by Snavely et al to recover the MIT photos' geometrical structure. After georegistering the 2.3K+ images to an aerial ladar map, we plot recovered camera geolocations on a Google Map and conduct virtual tours through the 3D field of 2D urban photos. Similar computer vision techniques are applied to video footage collected over a rural scene by an aerial glider. 1.5K video frames are reconstructed and georegistered via the UAV's GPS track. We demonstrate how several difficult aerial video exploitation problems become tractable when a geometry-based analysis approach is adopted.

In this paper, we propose a flush optimization algorithm for fast protection switching in an Ethernet ring protection (ERP) network. To evaluate performance of our algorithm, we implement our algorithm on ERP in Ethernet switches and experiment on protection switching in an Ethernet ring network. We measure protection switching time under a link failure as increasing the number of traffic flows and compare performance with the general ERP. Experiment results show our algorithm always provides protection switching time less than 30ms even though the number of traffic flows increases.

Gabor filters are widely used to model the behavior of the V1 stage of the primary visual cortex of mammalian vision systems. Acceleration of this filter is critical for various real-time vision applications. In this work, we explore a variety of design options for the Gabor filter to tradeoff resource utilization, performance and accuracy. Our results indicate that our optimizations enable a variety of design options to meet a wide range of application constraints.

Compact cameras and robotic platforms are growing ubiquitous as performance increases and price decreases for both of these transformational technologies. To encourage rapid experimentation with imaging sensors and robots, MIT Lincoln Laboratory held a Technology Challenge in September 2010 which involved remotely characterizing a 1 km2 rural area. In this poster, we present video exploitation results from an aerial glider system fielded as part of the Technology Challenge.The aerial system's hardware setup included a Radian sailplane glider (

Biological vision systems use saliency-based visual attention mechanisms to limit higher-level vision processing on the most visually-salient subsets of an input image. Among several computational models that capture the visual-saliency in biological system, an information theoretic AIM(Attention based on Information Maximization) algorithm has been demonstrated to predict human gaze patterns better than other existing models. We present an FPGA based implementation of this computationally intensive AIM algorithm to support embedded vision applications. Our implementation provides performance of processing about 4M pixels/sec for 25 basis functions with a convolution kernel size of 21 by 21 for each of the R, G, and B color-channels, when implemented on a Virtex-6 LX240T. We also provide an optimization aimed at controlling the trade-off between power consumption and latency, and performance comparisons with a GPU implementation.

Information theoretic security, also called secrecy analysis, provides theoretical limit for secret data transmission even in wireless networking environment, and it is more focused in these days because of increasing number of private information transmission through wireless channel and its inherent broadcasting characteristic. However, many studies in secrecy analysis assume that there exists only one eavesdropper or there is no cooperation among multiple eavesdroppers, which makes research results difficult to be applied in real situations. In this paper, we conduct a game theoretic analysis of eavesdropper cooperation in MISO wireless communication system where a secret information sender, Alice, has a counter-strategy against eavesdropping. We assume that Alice has a method for preventing eavesdroppers from overhearing. It is triggered by eavesdropper detection and degrades eavesdropping channel by forwarding artificial noise. For this reason, eavesdroppers' cooperation can pose a high risk to them and we utilize game theory to investigate eavesdropper behavior based on mutual information I (X; Y eve ) between the input X from Alice and the output Y eve to an eavesdropper. We derive that non-cooperation is Nash Equilibrium in one shot eavesdropper cooperation game and show necessary conditions for eavesdropper cooperation in infinitely repeated game.

Recent work in computer vision has demonstrated the potential to automatically recover camera and scene geometry from large collections of uncooperatively-collected photos. At the same time, aerial ladar and Geographic Information System (GIS) data are becoming more readily accessible. In this paper, we present a system for fusing these data sources in order to transfer 3D and GIS information into outdoor urban imagery. Applying this system to 1000+ pictures shot of the lower Manhattan skyline and the Statue of Liberty, we present two proof-of-concept examples of geometry-based photo enhancement which are difficult to perform via conventional image processing: feature annotation and image-based querying. In these examples, high-level knowledge projects from 3D world-space into georegistered 2D image planes and/or propagates between different photos. Such automatic capabilities lay the groundwork for future real-time labeling of imagery shot in complex city environments by mobile smart phones.

Ethernet ring protection (ERP) introduced by ITU-T G.8032 Recommendation is a new class of packet switched network protection technology that can provides ring automatic protection switch for nested multiple Ethernet rings. An ERP system is required to provide network protection against link and node failures within 50ms. Whenever the protection scheme switches the ring to have a new topology, the standard requires all nodes to flush their filtering data base (FDB) resulting in congestion in the access networks. In this paper we evaluate the protection performance of the current G.8032 scheme and introduce additional novel schemes for performance enhancements in nested multi ring networks.

We extend recent automated computer vision algorithms to reconstruct the global three-dimensional structures for photos and videos shot at fixed points in outdoor city environments. Mosaics of digital stills and embedded videos are georegistered by matching a few of their 2D features with 3D counterparts in aerial ladar imagery. Once image planes are aligned with world maps, abstract urban knowledge can propagate from the latter into the former. We project geotagged annotations from a 3D map into a 2D video stream and demonstrate their tracking buildings and streets in a clip with significant panning motion. We also present an interactive tool which enables users to select city features of interest in video frames and retrieve their geocoordinates and ranges. Implications of this work for future augmented reality systems based upon mobile smart phones are discussed.

Conference agenda: http://www.aiaa.org/agenda.cfm?lumeetingid=1998, This paper analyzes task assignment for heterogeneous air vehicles using a guaranteed conflict-free assignment algorithm, the Consensus Based Bundle Algorithm (CBBA). We extend this recently proposed algorithm to handle two realistic multi- UAV operational complications. Our rst extension accounts for obstacle regions in order to generate collision free paths for UAVs. Our second extension reduces task planner sensitivity to sensor measurement noise, and thereby minimizes churning behavior in flight paths. After integrating our enhanced CBBA module with a 3D visualization and interaction software tool, we simulate multiple aircraft servicing stationary and moving ground targets. Preliminary simulation results establish that consistent, conflict-free multi-UAV path assignments can be calculated on the order of a few seconds. The enhanced CBBA consequently demonstrates signfi cant potential for real-time performance in stressing environments.

We extend recent automated computer vision algorithms to reconstruct the global three-dimensional structures for photos and videos shot at fixed points in outdoor city environments. Mosaics of digital stills and embedded videos are georegistered by matching a few of their 2D features with 3D counterparts in aerial ladar imagery. Once image planes are aligned with world maps, abstract urban knowledge can propagate from the latter into the former. We project geotagged annotations from a 3D map into a 2D video stream and demonstrate their tracking buildings and streets in a clip with significant panning motion. We also present an interactive tool which enables users to select city features of interest in video frames and retrieve their geocoordinates and ranges. Implications of this work for future augmented reality systems based upon mobile smart phones are discussed.

Motivated by the problem of uncovering networks of illicit actors in complex urban environments, we present a prototype system for intuitive navigation of vehicle track data via interacting map and network views. Our system combines 3D geospatial visualization, social network display and interactive track search software, and it provides a multi-touch interface for operators to navigate urban scenes and investigate potentially suspicious vehicle activity. We describe a case study to highlight the system's capabilities using ground truth vehicle data collected during a 2007 urban exercise. This data is most naturally viewed as tracks in space and time. But as cluttered track displays obscure potentially important actor relationships, our system provides a social network picture whose condensed format is easier to interpret. Through coordinated space-time/vehicle network searches, we demonstrate how analysts can uncover "Red" activities of tactical significance.

A prototype image processing system has recently been developed which generates, displays and analyzes threedimensional ladar data in real time. It is based upon a suite of novel algorithms that transform raw ladar data into cleaned 3D images. These algorithms perform noise reduction, ground plane identification, detector response deconvolution and illumination pattern renormalization. The system also discriminates static from dynamic objects in a scene. In order to achieve real-time throughput, we have parallelized these algorithms on a Linux cluster. We demonstrate that multiprocessor software plus Blade hardware result in a compact, real-time imagery generation adjunct to an operating ladar. Finally, we discuss several directions for future work, including automatic recognition of moving people, real-time reconnaissance from mobile platforms, and fusion of ladar plus video imagery. Such enhancements of our prototype imaging system can lead to multiple military and civilian applications of national importance.

This article presents a novel electric motor/pump system that combines an axial field, permanent magnet motor with a centrifugal pump. This system, unique because the motor permanent magnet rotor and pump impeller vanes are a single unit, provides a compact, reliable, low‐noise, and high‐power density electrically driven centrifugal pump suitable for underwater applications in which minimizing noise, vibration, and volume are major design objectives. Performance tradeoffs for the electromagnetic analysis were made by three‐dimensional finite element analysis models in conjunction with a lumped parameter magnetic circuit model.