Search

Neural saturation in Deep Neural Networks (DNNs) has been studied extensively, but remains relatively unexplored in Convolutional Neural Networks (CNNs). Understanding and alleviating the effects of convolutional kernel saturation is critical for enhancing CNN models classification accuracies. In this paper, we analyze the effect of convolutional kernel saturation in CNNs and propose a simple data augmentation technique to mitigate saturation and increase classification accuracy, by supplementing negative images to the training dataset. We hypothesize that greater semantic feature information can be extracted using negative images since they have the same structural information as standard images but differ in their data representations. Varied data representations decrease the probability of kernel saturation and thus increase the effectiveness of kernel weight updates. The two datasets selected to evaluate our hypothesis were CIFAR- 10 and STL-10 as they have similar image classes but differ in image resolutions thus making for a better understanding of the saturation phenomenon. MNIST dataset was used to highlight the ineffectiveness of the technique for linearly separable data. The ResNet CNN architecture was chosen since the skip connections in the network ensure the most important features contributing the most to classification accuracy are retained. Our results show that CNNs are indeed susceptible to convolutional kernel saturation and that supplementing negative images to the training dataset can offer a statistically significant increase in classification accuracies when compared against models trained on the original datasets. Our results present accuracy increases of 6.98% and 3.16% on the STL-10 and CIFAR-10 datasets respectively., Conference paper, 6 pages, 3 figures, 1 table

Convolutional Neural Networks (CNNs) specialize in feature extraction rather than function mapping. In doing so they form complex internal hierarchical feature representations, the complexity of which gradually increases with a corresponding increment in neural network depth. In this paper, we examine the feature extraction capabilities of CNNs using Maximum Entropy (ME) and Signal-to-Noise Ratio (SNR) to validate the idea that, CNN models should be tailored for a given task and complexity of the input data. SNR and ME measures are used as they can accurately determine in the input dataset, the relative amount of signal information to the random noise and the maximum amount of information respectively. We use two well known benchmarking datasets, MNIST and CIFAR-10 to examine the information extraction and abstraction capabilities of CNNs. Through our experiments, we examine convolutional feature extraction and abstraction capabilities in CNNs and show that the classification accuracy or performance of CNNs is greatly dependent on the amount, complexity and quality of the signal information present in the input data. Furthermore, we show the effect of information overflow and underflow on CNN classification accuracies. Our hypothesis is that the feature extraction and abstraction capabilities of convolutional layers are limited and therefore, CNN models should be tailored to the input data by using appropriately sized CNNs based on the SNR and ME measures of the input dataset., Comment: Conference paper, 6 pages, 1 table

Energy consumption in large scale data centers (LSDCs) doubled from 2000 to 2006 reaching 61 TerraWatt-hour (TWh) per year. Nearly all of the energy consumed by IT equipment dissipates as heat from the servers, creating a real problem of efficiently cooling LSDCs. Reducing the Power Usage Effectiveness (PUE) of a data center by even small fractions significantly reduces greenhouse gas emissions. We propose a novel Pulsed Variable Flow Rate (PVFR) dynamic cooling control strategy for liquid cooled LSDCs, based on the principle of pulsed power delivery instead of continuous power supply. We built a proprietary simulation software called DynaCool using model-driven design to evaluate the effectiveness of PVFR approach. An early evaluation shows that DynaCool achieves a PUE reduction of at least 15.4% over existing static and variable flow rate cooling control strategies. Assuming an adoption rate of 10%, PVFR would yield power savings of 1.88 TWh or greenhouse gas emission savings of 284,000 tons per year.