Search

The image captioning task aims to generate descriptions from the main content of images. Recently, the Transformer with a self-attention mechanism has been widely used for the image captioning task, where the attention mechanism helps the encoder to generate image region features, and guides caption output in the decoder. However, the vanilla decoder uses a simple conventional self-attention mechanism, resulting in captions with poor semantic information and incomplete sentence logic. In this paper, we propose a novel attention block, Multi-Keys attention block, that fully enhances the relevance between explicit and implicit semantic information. Technically, the Multi-Keys attention block first concatenates the key vector and the value vector and spreads it into both the explicit channel and the implicit channel. Then, the "related value" is generated with more semantic information by applying the element-wise multiplication to them. Moreover, to perfect the sentence logic, the reverse key vector with another information flow is residually connected to the final attention result. We also apply the Multi-Keys attention block into the sentence decoder in the transformer named as Multi-Keys Transformer (MKTrans). The experiments demonstrate that our MKTrans achieves 138.6% CIDEr score on MS COCO "Karpathy" offline test split. The proposed Multi-Keys attention block and MKTrans model are proven to be more effective and superior than the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Generative Adversarial Networks (GANs) are one of the most popular and powerful models to learn the complex high dimensional distributions. However, they usually suffer from instability and generalization issues which may lead to poor generations. Most existing works focus on stabilizing the training for the discriminators of GANs while ignoring their generalization issue. In this work, we aim to improve the generalization capability of GANs by promoting the local robustness within the small neighborhood of the training samples. We prove that the robustness in the small neighborhood of the training sets can lead to better generalization. Particularly, we design a new robust method called Robust Generative Adversarial Network (RGAN) in which the generator and discriminator compete with each other in a worst-case setting within a small Wasserstein ball. The generator tries to map the worst input distribution (rather than a Gaussian distribution used in most GANs) to the real data distribution, while the discriminator attempts to distinguish the real and fake distributions with the worst perturbations. Intuitively, the proposed RGAN can learn a good generator and discriminator that can even perform well on the worst-case input points. Strictly, we have proved that RGAN can obtain a tighter generalization upper bound than the traditional GANs under mild assumptions, ensuring a theoretical superiority of RGAN over GANs. We conduct our proposed method on five different baselines (five popular GAN models). And a series of experiments on CIFAR-10, STL-10 and CelebA datasets indicate that our proposed robust frameworks outperform five baseline models substantially and consistently. [ABSTRACT FROM AUTHOR]

Great progress has been witnessed for weakly supervised semantic segmentation, which aims to segment objects without dense pixel annotations. Most approaches concentrate on generating high quality pseudo labels, which are then fed into a standard segmentation model as supervision. However, such a solution has one major limitation: noise of pseudo labels is inevitable, which is unsolvable for the standard segmentation model. In this paper, we propose a credible dual-expert learning (CDL) framework to mitigate the noise of pseudo labels. Specifically, we first observe that the model predictions with different optimization loss functions will have different credible regions; thus, it is possible to make self-corrections with multiple predictions. Based on this observation, we design a dual-expert structure to mine credible predictions, which are then processed by our noise correction module to update pseudo labels in an online way. Meanwhile, to handle the case that the dual-expert produces incredible predictions for the same region, we design a relationship transfer module to provide feature relationships, enabling our noise correction module to transfer predictions from the credible regions to such incredible regions. Considering the above designs, we propose a base CDL network and an extended CDL network to satisfy different requirements. Extensive experiments show that directly replacing our model with a conventional fully supervised segmentation model, the performances of various weakly supervised semantic segmentation pipelines were boosted, achieving new state-of-the-art performances on both PASCAL VOC 2012 and MS COCO with a clear margin. Code will be available at: https://github.com/zbf1991/CDL. [ABSTRACT FROM AUTHOR]

As additive manufacturing and electroplating technique have progressed, metal-coated lattice material has wide applications due to its lightweight nature and designability. The resin matrix coated with metallic material may enhance mechanical performances while with economic cost and additional conductivity. However, a quick evaluation of equivalent material properties of metal-coated lattices is a challenging task due to the various geometric designs and coating parameters. In this paper, a numerical prediction approach is proposed with the combination of data acquisition from Finite Element Analysis (FEA) and the Machine Learning (ML) models. Firstly, a finite element model with hybrid solid and membrane elements was adopted to simulate the metal-coated lattice structure. Based on the homogenization theory, appropriate boundary conditions were defined for the Representative Volume Element (RVE) to evaluate the effective elastic modulus. With the limited numerical results, data amplification was implemented by using Polynomial Regression (PR). Finally, different ML algorithms were investigated. Artificial Neural Network (ANN) was verified as an efficient one with better prediction accuracy 99.97% for 4 variables. The proposed approach could give a reasonable property evaluation of metal-coated lattices avoiding repetitive tests and provide a feasible reference for the lattice design. [ABSTRACT FROM AUTHOR]

The Scalable extension of the High Efficiency Video Coding (known as SHVC) combines the high compression efficiency with the possibility of encoding different resolutions of the same encoded video in a single bitstream. However, this is accompanied with a high computational complexity. In this paper, we propose an effective coding unit (CU) size decision method by restricting the CU depth range to reduce the encoding time for quality scalability in SHVC. Since the optimal depth level in the enhancement layer (EL) is highly correlated to that in the base layer (BL), we can determine the CU depth range in the EL according to the depth of the co-located CU in the BL. Based on the high correlation between the current CU and its spatio-temporal neighboring CUs, the proposed method skips some specific depth levels which are rarely used in the previous frame and neighboring CUs to further reduce the computational complexity. Experimental results demonstrate that the proposed method can efficiently reduce computational complexity while maintaining similar rate distortion (RD) performance as the original SHVC encoder. [ABSTRACT FROM AUTHOR]

Multiview video plus depth is a popular 3D video format which can provide viewers a vivid 3D feeling. However, its requirements in terms of computational complexity and transmission bandwidth are more than that of conventional 2D video. To mitigate these limitations, some works have proposed to reduce the amount of transmitted data by adopting different resolutions for different views, and consequently, the transmitted video is called mixed resolution video. In order to further reduce the transmitted data and maintain good quality at the decoder side; in this paper, we propose a down/upsampling algorithm for 3D multiview video which systematically takes into account the video encoder and decoder. At the encoder side, the rows of the two adjacent views are downsampled following an interlacing and complementary fashion, whereas, at the decoder side, the discarded pixels are recovered by fusing the virtual view pixels with the directional interpolated pixels from the complementary downsampled views. Moreover, the patterns of the texture surrounding the discarded pixels are used to aid the data fusion, so as to enhance edges recovery. Meanwhile, with the assistance of virtual views, at the decoder side, the proposed approach can effectively recover the discarded high-frequency details. The experimental results demonstrate the superior performance of the proposed framework. [ABSTRACT FROM AUTHOR]

The polyphase-based mechanism is the basis of many multiple descriptions coding schemes. Its main drawback is the inefficient exploitation of the inserted redundancy, especially when the redundancy is large. In this paper we propose a novel approach that uses mid-tread quantizers with tunable deadzone, in order to efficiently exploit the inserted redundancy. In particular, the deadzone width is selected based on the statistical distribution of the data, and the approximated level of redundancy to be inserted. The proposed approach is tailored for those codecs that use mid-tread quantizers with tunable step-size and deadzone width. This is particularly interesting given that the majority of codecs use this topology of quantization, and they rarely allow changing it. Moreover, the proposed scheme can be extended to the case of more than two descriptions. Finally, it is worth reporting that the results of the proposed approach outperform that of state-of-the-art schemes. In fact with the same side performance, the central quality can achieve up to 1 dB gain. [ABSTRACT FROM AUTHOR]

For 3-D videos, one commonly used representation method is texture videos plus depth maps for several selected viewpoints, whereas the other viewpoints are synthesized based on the available texture videos and depth maps with the depth-image-based rendering (DIBR) technique. As both the quality of the texture videos and depth maps will affect the quality of the synthesized views, bits allocation for the depth maps become indispensable. The existing bits allocation approaches are either inaccurate or requiring pre-encoding and analyzing in temporal dimension, making them unsuitable for the real-time applications. Motivated by the fact that different regions of the depth maps have different impacts on the synthesized image quality, a real-time macroblock level bits allocation approach is proposed, where different macroblocks of the depth maps are encoded with different quantization parameters and coding modes. As the bits allocation granularity is fine, the R-D performance of the proposed approach outperforms other bits allocation approaches significantly, while no additional pre-encoding delay is caused. Specifically, it can save more than 10% overall bit rate comparing with Morvan's full search approach, while maintaining the same synthesized view quality. [ABSTRACT FROM AUTHOR]

A correction to the article "One-class kernel subspace ensemble for medical image classification" that was published in the October 2015 issue is presented.