Search

The problem of knowledge-based visual question answering involves answering questions that require external knowledge in addition to the content of the image. Such knowledge typically comes in various forms, including visual, textual, and commonsense knowledge. Using more knowledge sources increases the chance of retrieving more irrelevant or noisy facts, making it challenging to comprehend the facts and find the answer. To address this challenge, we propose Multi-modal Answer Validation using External knowledge (MAVEx), where the idea is to validate a set of promising answer candidates based on answer-specific knowledge retrieval. Instead of searching for the answer in a vast collection of often irrelevant facts as most existing approaches do, MAVEx aims to learn how to extract relevant knowledge from noisy sources, which knowledge source to trust for each answer candidate, and how to validate the candidate using that source. Our multi-modal setting is the first to leverage external visual knowledge (images searched using Google), in addition to textual knowledge in the form of Wikipedia sentences and ConceptNet concepts. Our experiments with OK-VQA, a challenging knowledge-based VQA dataset, demonstrate that MAVEx achieves new state-of-the-art results. Our code is available at https://github.com/jialinwu17/MAVEX, Comment: AAAI 2022

We present the training recipe and results of scaling up PaLI-X, a multilingual vision and language model, both in terms of size of the components and the breadth of its training task mixture. Our model achieves new levels of performance on a wide-range of varied and complex tasks, including multiple image-based captioning and question-answering tasks, image-based document understanding and few-shot (in-context) learning, as well as object detection, video question answering, and video captioning. PaLI-X advances the state-of-the-art on most vision-and-language benchmarks considered (25+ of them). Finally, we observe emerging capabilities, such as complex counting and multilingual object detection, tasks that are not explicitly in the training mix.

Additional file 1.

Additional file 1. Supplementary Figures S1–S2 and Tables S1-S3. Fig. S1. Impacts on different graph neural networks. The AUC results of interaction site prediction (A). The Pearson correlation coefficient results (B) and RMSE results (C) of binding affinity prediction. Fig. S2. Impacts of different lambda values for SPE-MONN-PtsRep. The AUC results of interaction site prediction (A). The Pearson correlation coefficient results (B) and RMSE results (C) of binding affinity prediction. Table S1. Clustering results of compounds at different clustering thresholds. Table S2. Clustering results for proteins at different clustering thresholds. Table S3. Impacts of the hyperparameter α on protein representation combination. The area under receiver operator characteristic curve (AUC) is the result of interaction site prediction. The Pearson correlation coefficient and root mean square error (RMSE) are results of binding affinity prediction. NC, NP, and NN are new-compound, new-protein, both-new settings, respectively.

Additional file 4: Supplementary Table 1. primers sequences.

Additional file 3.

Additional file 2: Fig. S1. RT-qPCR shows the differential expression of different genes in esophageal cell lines. The figures show the mRNA expression of SYNM, TNXB, CFD, HSPB6 and PLIN4 in HEEC, HET-1A and ESCC cell lines (* P

World History Connected, Vol. 19 No. 3: Fall 2022

The possibility of super-AIs taking over the world has been intensively studied by numerous scholars. This paper focuses on the multi-AI competition scenario under the premise of super-AIs in power. Firstly, the article points out the defects of existing arguments supporting single-AI domination and presents arguments in favour of multi-AI competition. Then the article concludes that the multi-AI competition situation is a non-negligible possibility. Attention then turns to whether multi-AI competition is better for the overall good of humanity than a situation where a single AI is in power. After analysing the best, worst, and intermediate scenarios, the article concludes that multi-AI competition is better for humanity. Finally, considering the factors related to the formation of the best-case scenario of multiple AIs, the article gives some suggestions for current initiatives in AI development.

Most Outside-Knowledge Visual Question Answering (OK-VQA) systems employ a two-stage framework that first retrieves external knowledge given the visual question and then predicts the answer based on the retrieved content. However, the retrieved knowledge is often inadequate. Retrievals are frequently too general and fail to cover specific knowledge needed to answer the question. Also, the naturally available supervision (whether the passage contains the correct answer) is weak and does not guarantee question relevancy. To address these issues, we propose an Entity-Focused Retrieval (EnFoRe) model that provides stronger supervision during training and recognizes question-relevant entities to help retrieve more specific knowledge. Experiments show that our EnFoRe model achieves superior retrieval performance on OK-VQA, the currently largest outside-knowledge VQA dataset. We also combine the retrieved knowledge with state-of-the-art VQA models, and achieve a new state-of-the-art performance on OK-VQA., Comment: EMNLP 2022

Accurate localization of subsea pipelines is the prerequisite for its inspection, reinforcement and maintenance. Magnetic anomaly generated by the ferromagnetic pipeline immersed in the earth’s magnetic field can be used to locate the pipeline. However, since the magnetic field decays rapidly with distance, the detection distance is very limited. This paper proposes a method to improve the sensitivity and limit distance of pipeline detection and positioning through active magnetization. When the magnet is directly above the pipe, the magnetization effect is best, so a magnet array should be used to cover the pipe with wide enough coverage. Experiments have proved that by using a magnet array, with SNR = 20 dB as the judging criterion, the limit detection distance of a Φ219mm steel pipe can be increased from 0.363m to 0.554m.

Most recent state-of-the-art Visual Question Answering (VQA) systems are opaque black boxes that are only trained to fit the answer distribution given the question and visual content. As a result, these systems frequently take shortcuts, focusing on simple visual concepts or question priors. This phenomenon becomes more problematic as the questions become complex that requires more reasoning and commonsense knowledge. To address this issue, we present a novel framework that uses explanations for competing answers to help VQA systems select the correct answer. By training on human textual explanations, our framework builds better representations for the questions and visual content, and then reweights confidences in the answer candidates using either generated or retrieved explanations from the training set. We evaluate our framework on the VQA-X dataset, which has more difficult questions with human explanations, achieving new state-of-the-art results on both VQA and its explanations.

Visual Question Answering (VQA) deep-learning systems tend to capture superficial statistical correlations in the training data because of strong language priors and fail to generalize to test data with a significantly different question-answer (QA) distribution. To address this issue, we introduce a self-critical training objective that ensures that visual explanations of correct answers match the most influential image regions more than other competitive answer candidates. The influential regions are either determined from human visual/textual explanations or automatically from just significant words in the question and answer. We evaluate our approach on the VQA generalization task using the VQA-CP dataset, achieving a new state-of-the-art i.e., 49.5% using textual explanations and 48.5% using automatically annotated regions., In NeurIPS 2019

Most RNN-based image captioning models receive supervision on the output words to mimic human captions. Therefore, the hidden states can only receive noisy gradient signals via layers of back-propagation through time, leading to less accurate generated captions. Consequently, we propose a novel framework, Hidden State Guidance (HSG), that matches the hidden states in the caption decoder to those in a teacher decoder trained on an easier task of autoencoding the captions conditioned on the image. During training with the REINFORCE algorithm, the conventional rewards are sentence-based evaluation metrics equally distributed to each generated word, no matter their relevance. HSG provides a word-level reward that helps the model learn better hidden representations. Experimental results demonstrate that HSG clearly outperforms various state-of-the-art caption decoders using either raw images or detected objects as inputs.

There has long been debates on how we could interpret neural networks and understand the decisions our models make. Specifically, why deep neural networks tend to be error-prone when dealing with samples that output low softmax scores. We present an efficient approach to measure the confidence of decision-making steps by statistically investigating each unit's contribution to that decision. Instead of focusing on how the models react on datasets, we study the datasets themselves given a pre-trained model. Our approach is capable of assigning a score to each sample within a dataset that measures the frequency of occurrence of that sample's chain of activation. We demonstrate with experiments that our method could select useful samples to improve deep neural networks in a semi-supervised leaning setting.

Accurate localization of subsea pipeline is the prerequisite of pipeline maintenance, reinforcement and tracking inspection. Via finite element simulations and experiments, this paper reveals the distribution characterizations of the total magnetic flux density (MFD) and the reduced MFD near the steel pipeline, and demonstrates a method for accurately positioning subsea pipeline by using the external magnetic field. The characteristic peak position of the total and reduced MFDs’ components are all susceptible to the radial magnetization direction, except for the reduced MFD’s norm whose peak always overlaps the projection of the pipeline axis on the magnetic measurement plane. The norms of the total and reduced MFDs are used to determine the pipeline orientation and axis position, respectively. For an 8 m long steel pipe with diameter of 8 in., when the measurement plane is 1.5 times the diameter away from the pipe, the positioning error is around 2 cm.

We propose a dynamic filtering strategy with large sampling field for ConvNets (LS-DFN), where the position-specific kernels learn from not only the identical position but also multiple sampled neighbor regions. During sampling, residual learning is introduced to ease training and an attention mechanism is applied to fuse features from different samples. Such multiple samples enlarge the kernels' receptive fields significantly without requiring more parameters. While LS-DFN inherits the advantages of DFN, namely avoiding feature map blurring by position-wise kernels while keeping translation invariance, it also efficiently alleviates the overfitting issue caused by much more parameters than normal CNNs. Our model is efficient and can be trained end-to-end via standard back-propagation. We demonstrate the merits of our LS-DFN on both sparse and dense prediction tasks involving object detection, semantic segmentation, and flow estimation. Our results show LS-DFN enjoys stronger recognition abilities in object detection and semantic segmentation tasks on VOC benchmark and sharper responses in flow estimation on FlyingChairs dataset compared to strong baselines., ECCV 2018

Answering visual questions need acquire daily common knowledge and model the semantic connection among different parts in images, which is too difficult for VQA systems to learn from images with the only supervision from answers. Meanwhile, image captioning systems with beam search strategy tend to generate similar captions and fail to diversely describe images. To address the aforementioned issues, we present a system to have these two tasks compensate with each other, which is capable of jointly producing image captions and answering visual questions. In particular, we utilize question and image features to generate question-related captions and use the generated captions as additional features to provide new knowledge to the VQA system. For image captioning, our system attains more informative results in term of the relative improvements on VQA tasks as well as competitive results using automated metrics. Applying our system to the VQA tasks, our results on VQA v2 dataset achieve 65.8% using generated captions and 69.1% using annotated captions in validation set and 68.4% in the test-standard set. Further, an ensemble of 10 models results in 69.7% in the test-standard split.

We propose a novel deep supervised neural network for the task of action recognition in videos, which implicitly takes advantage of visual tracking and shares the robustness of both deep Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). In our method, a multi-branch model is proposed to suppress noise from background jitters. Specifically, we firstly extract multi-level deep features from deep CNNs and feed them into 3d-convolutional network. After that we feed those feature cubes into our novel joint LSTM module to predict labels and to generate attention regularization. We evaluate our model on two challenging datasets: UCF101 and HMDB51. The results show that our model achieves the state-of-art by only using convolutional features., 13 pages, submitted to BMVC

Composition du jury : Laura Lee Down, directrice de thèse, directrice d’études, ÉHESS et IUE, Florence ;Paul Servais, directeur de thèse, Professeur à l'Université Catholique de Louvain ;Marie Emmanuelle Chessel, directrice de recherches, CNRS – ÉHESS ;John Darwin, Professor of Global and Imperial History, University of Oxford ;Michel Dumoulin, Professeur émérite, Université Catholique de Louvain ;Tim Harper, Reader in Southeast Asian and Imperial History, University of Cambridge ;Jean-Marie ...

Silk fibroin (SF)/gelatin blend nanofibers membranes as scaffolds were fabricated successfully via electrospinning with different composition ratios in formic acid. The formation of intermolecular hydrogen bonds and the conformational transition of SF provided scaffolds with excellent mechanical properties. FTIR and DTA analysis showed the SF/gelatin nanofibers had more β-sheet structures than the pure SF nanofibers. The former's breaking tenacity increased from 0.95 up to 1.60 MPa, strain at break was 7.6%, average fiber diameter was 89.2 nm, porosity was 87%, and pore diameter was 142 nm. MTT, H&E stain, and SEM results showed that the adhesion, spreading, and proliferation of human umbilic vein endothelium cells (HUVECs) and mouse fibroblasts on the SF/gelatin nanofibers scaffolds were definitely better than that on the SF nanofibers scaffolds. The scaffolds could replace the natural ECM proteins, support long-term cell growth, form three-dimensional networks of the nanofibrous structure, and grow in the direction of fiber orientation. Our results prove that the addition of gelatin improved the mechanical and biological properties of the pure SF nanofibers, these SF/gelatin blend nanofiber membranes are desirable for the scaffolds and may be a good candidate for blood vessel engineering scaffolds. © 2008 Wiley Periodicals, Inc. J Appl Polym Sci, 2009

International audience; According to white Scoutmasters in Malaya, ‘the peculiar difficulties of locality and climate’ necessitated certain proscriptions on Scout training for Malayans. Restrictions on the physical aspects of Scouting in Malayan jungles limited opportunities for Malayans to cultivate Scouting's idealised male figure of the frontiersman. At the same time, colonial discouragement of the ‘Anglicisation’ of indigenous Scouts effectively curtailed indigenous Malayan access to those ideals of masculinity which were in many respects dominant in the colonial context. Ironically, some of these Anglicised Scouting ideals of masculinity were borrowed from ‘native cultures’ or ’martial tribes’. The exchanges and appropriations of culture and masculinities through Scouting were therefore multi-directional and more complicated than imagined. This article examines how Scouting in Malaya functioned as a vehicle for and as a product of multiple colonial exchanges. It argues that colonial Scouting in Malaya introduced but also restricted Malayan access to certain ideals of masculinity. Additionally, it argues for a more nuanced analysis of localised Malayan appropriations and negotiations of colonial/imperial concepts introduced through the Scout Movement. In particular, through considering how Malayan Scouts ‘played Indian’, this article further suggests that Scout activities of ‘make-believe’ attracted Malayan Scouts because of the liberties attached to these bodily performances of idealised Anglicised masculinities.

International audience; Tout d'abord introduit en 1910 dans les colonies anglaises de la Malaisie britannique, le scoutisme et le guidisme deviennent rapidement populaires chez les colons et les indigènes. Bien que ce phénomène reste important dans ces colonies après la période de l'impérialisme, le succès de ces mouvements de jeunesse a été peu analysé et reste méconnu. Si l'on examine les activités des scouts et des guides à l'époque coloniale, l'on est frappé de constater la prépondérance de la cuisine dans leurs activités. De plus, bien que la plupart des scouts et des guides ne soient pas Européens, leurs préparations, leurs recettes et leurs types d'activités culinaires sont notablement d'influence occidentale. Pourquoi est-ce le cas ? Pourquoi le mouvement des guides a-t-il mis l'accent sur la cuisine, sa préparation et la " domesticité " en en faisant un des programmes d'activités les plus importants et visibles ? Est-ce dérivé d'une stratégie pour introduire le guidisme et le scoutisme - par extension les valeurs occidentales - sans être ressenti comme empiétement dans la sphère privée des locaux ? Est-ce les préjugés existants des colonisateurs envers la cuisine locale et surtout leur préparation qui motiva l'introduction d'activité culinaire, a fortiori d'influence occidentale ? Ce chapitre aborde ces nombreuses questions. En s'appuyant sur de nombreuses archives, documents privés de ces mouvements et témoignages oraux des scouts et guides, en grande partie inexplorés, nous abordons les liens entre la nourriture et le guidisme. Nous explorons le rôle de ces mouvements de jeunesse dans la gestion du genre et des enjeux liés aux différences de cultures, de pratiques et de traditions.

International audience

Nanofibers could provide benefits due to high surface area for biomaterial scaffolds. In order to improve both silk fibroin and gelatin performance for potential biomedical applications, in this article the electrospinning of silk fibroin (SF)/gelatin (GT) blends with different composition ratio and concentration was performed with formic acid as a spinning solvent. The nanofibers morphologies and diameters were observed by Scanning electron microscopy (SEM). Blending with GT could improve the viscosity and spinnability of spinning solution. As-spun SF/GT blend nanofibers were more homogeneous and non-beaded than pure SF nanofibers. The diameter decreased with higher voltage at same electric field intensity. On increasing the concentrations of spinning solution, the pore diameter of nanofibers decreased, and the porosity increased. In addition, FTIR and DTA analyses showed SF/GT nanofibers had more β-sheets than pure SF nanofibers, which may be caused by an intermolecular interaction between SF and GT. Blending with GT also led to substantial enhancement in mechanical properties of SF nanofibers.

The existence of potential energy landscape in the glass transition has been theoretically proved using the recursive equation of reinforce-restraint of self-similar 8 orders of Lennard-Jones (L-J) potential fluctuations. The stability condition for fluctuation reinforce-restraint is just the Lindemann ratio that is exactly deduced as 0.1047in this paper. The origin and transfer of interface excitation comes of the balance between self-similar L-J potential fluctuation and geometric phase potential fluctuation, which also gives rise to a new attractive potential of -17/16\epsilon_i, lower than the potential well energy -\epsilon_i of i-th order of L-J potential, in the self-similar mean field of mean fields of different sizes. The delocalization energy of two-body is exactly the transfer energy of excited interface, and the delocalization path is along 8 orders of geodesics in topological analyses.

Classical thermodynamic theory still holds true in subsystem that is a percolation connected by 8 orders of self-similar 2-body-3-body coupling clusters. The fixed point, $B_2^* \equiv 3/8$, for the clusters of different size, existing in reduced second Virial coefficients has been proved by scaling theory in percolation field. It is shown that, if $B_2^* \equiv 3/8$ is combined with $B_3^* \equiv 5/8$, the potentials of 2-body-3-body coupling clusters, in critical local cluster growth phase transition, balance the kinetic energy in the glass transition. It is also proved that the glass transition corresponds to the regime in which the chemical potentials in all subsystems hold zero., Comment: 10 pages, 3 figures

The nature may be disclosed that the glass transition is only determined by the intrinsic 8 orders of instant 2-D mosaic geometric structures, without any presupposition and relevant parameter. An interface excited state on the geometric structures comes from the additional Lindemann distance increment, which is a vector with 8 orders of relaxation times, 8 orders of additional restoring force moment (ARFM), quantized energy and extra volume. Each order of anharmonic ARFM gives rise to an additional position-asymmetry on a 2-D projection plane of a reference particle, thus, in removing additional position-asymmetry, the 8 orders of 2-D clusters and hard-spheres accompanied with the 4 excited interface relaxations of the reference particle have been illustrated. Dynamical behavior comes of the slow inverse energy cascade to generate 8 orders of clusters, to thaw a solid-domain, and the fast cascade to relax tension and rearrange structure. This model provides a unified mechanism to interpret hard-sphere, compacting cluster, free volume, cage, jamming behaviors, geometrical frustration, reptation, Ising model, breaking solid lattice, percolation, cooperative migration and orientation, critical entanglement chain length and structure rearrangements. It also directly deduces a series of quantitative values for the average energy of cooperative migration in one direction, localized energy independent of temperature and the activation energy to break solid lattice. In a flexible polymer system, there are all 320 different interface excited states that have the same quantized excited energy but different interaction times, relaxation times and phases. The quantized excited energy is about 6.4 k = 0.55meV., Comment: 19 papers, 6 figures, Journal of Physics: Condens. Matter

Centralism on the basis of democracy and giving scope to democracy under the guidance of centralism—such is the dialectical relationship of unity between democracy and centralism.