Search

Deep learning architectures comprising tens or even hundreds of convolutional and fully-connected hidden layers differ greatly from the shallow architecture of the brain. Here, we demonstrate that by increasing the relative number of filters per layer of a generalized shallow architecture, the error rates decay as a power law to zero. Additionally, a quantitative method to measure the performance of a single filter, shows that each filter identifies small clusters of possible output labels, with additional noise selected as labels outside the clusters. This average noise per filter also decays for a given generalized architecture as a power law with an increasing number of filters per layer, forming the underlying mechanism of efficient shallow learning. The results are supported by the training of the generalized LeNet-3, VGG-5, and VGG-16 on CIFAR-100 and suggest an increase in the noise power law exponent for deeper architectures. The presented underlying shallow learning mechanism calls for its further quantitative examination using various databases and shallow architectures. [ABSTRACT FROM AUTHOR]

In real-life scenarios, an input image typically comprises multiple objects, and their classification is often implemented using detection-based classification (DBC). In this approach, objects are first detected and then identified individually using a deep architecture. In this study, we demonstrate that the accuracy achieved by multilabel classification (MLC) surpasses that of DBC for a relatively small number of multilabel learning combinations. The crossover point at which DBC maximizes accuracy depends on the type of multilabel images, such as the number of multiple objects per image. The results are based on VGG-6 trained on the CIFAR-100 dataset using an upper bound for DBC accuracy, assumed under perfect detection conditions. Furthermore, we briefly discuss the potential relevance of these findings to advanced communication theory and natural language processing. The results suggest a need to reexamine the advantages of MLC over DBC using more complex datasets and deep architectures. [ABSTRACT FROM AUTHOR]

Significant variations of delays among connecting neurons cause an inevitable disadvantage of asynchronous brain dynamics compared to synchronous deep learning. However, this study demonstrates that this disadvantage can be converted into a computational advantage using a network with a single output and M multiple delays between successive layers, thereby generating a polynomial time-series outputs with M. The proposed role of delay in brain dynamics (RoDiB) model, is capable of learning increasing number of classified labels using a fixed architecture, and overcomes the inflexibility of the brain to update the learning architecture using additional neurons and connections. Moreover, the achievable accuracies of the RoDiB system are comparable with those of its counterpart tunable single delay architectures with M outputs. Further, the accuracies are significantly enhanced when the number of output labels exceeds its fully connected input size. The results are mainly obtained using simulations of VGG-6 on CIFAR datasets and also include multiple label inputs. However, currently only a small fraction of the abundant number of RoDiB outputs is utilized, thereby suggesting its potential for advanced computational power yet to be discovered. [ABSTRACT FROM AUTHOR]

The realization of classification tasks using deep learning is a primary goal of artificial intelligence; however, its possible universal behavior remains unexplored. Herein, we demonstrate a scaling behavior for the test error, ϵ , as a function of the number of classified labels, K. For trained utmost deep architectures on CIFAR-100 ϵ (K) ∝ K ρ with ρ ∼ 1 , and in case of reduced deep architectures, ρ continuously decreases until a crossover to ϵ (K) ∝ log (K) is observed for shallow architectures. A similar crossover is observed for shallow architectures, where the number of filters in the convolutional layers is proportionally increased. This unified the scaling behavior of deep and shallow architectures, which yields a reduced latency method. The dependence of Δϵ / ΔK on the trained architecture is expected to be crucial in learning scenarios involving dynamic number of labels. [ABSTRACT FROM AUTHOR]