Improving deep convolutional neural networks with mixed maxout units

The maxout units have the problem of not delivering non-max features， resulting in the insufficient of pooling operation over a subspace that is composed of several linear feature mappings,when they are applied in deep convolutional neural networks.The mixed maxout (mixout) units were proposed to deal with this constrain.Firstly,the exponential probability of the feature mappings getting from different linear transformations was computed.Then,the averaging of a subspace of different feature mappings by the exponential probability was computed.Finally,the output was randomly sampled from the max feature and the mean value by the Bernoulli distribution,leading to the better utilizing of model averaging ability of dropout.The simple models and network in network models was built to evaluate the performance of mixout units.The results show that mixout units based models have better performance.