Search

The problem of knowledge-based visual question answering involves answering questions that require external knowledge in addition to the content of the image. Such knowledge typically comes in various forms, including visual, textual, and commonsense knowledge. Using more knowledge sources increases the chance of retrieving more irrelevant or noisy facts, making it challenging to comprehend the facts and find the answer. To address this challenge, we propose Multi-modal Answer Validation using External knowledge (MAVEx), where the idea is to validate a set of promising answer candidates based on answer-specific knowledge retrieval. Instead of searching for the answer in a vast collection of often irrelevant facts as most existing approaches do, MAVEx aims to learn how to extract relevant knowledge from noisy sources, which knowledge source to trust for each answer candidate, and how to validate the candidate using that source. Our multi-modal setting is the first to leverage external visual knowledge (images searched using Google), in addition to textual knowledge in the form of Wikipedia sentences and ConceptNet concepts. Our experiments with OK-VQA, a challenging knowledge-based VQA dataset, demonstrate that MAVEx achieves new state-of-the-art results. Our code is available at https://github.com/jialinwu17/MAVEX, Comment: AAAI 2022

Additional file 1.

Additional file 1. Supplementary Figures S1–S2 and Tables S1-S3. Fig. S1. Impacts on different graph neural networks. The AUC results of interaction site prediction (A). The Pearson correlation coefficient results (B) and RMSE results (C) of binding affinity prediction. Fig. S2. Impacts of different lambda values for SPE-MONN-PtsRep. The AUC results of interaction site prediction (A). The Pearson correlation coefficient results (B) and RMSE results (C) of binding affinity prediction. Table S1. Clustering results of compounds at different clustering thresholds. Table S2. Clustering results for proteins at different clustering thresholds. Table S3. Impacts of the hyperparameter α on protein representation combination. The area under receiver operator characteristic curve (AUC) is the result of interaction site prediction. The Pearson correlation coefficient and root mean square error (RMSE) are results of binding affinity prediction. NC, NP, and NN are new-compound, new-protein, both-new settings, respectively.

Additional file 4: Supplementary Table 1. primers sequences.

Additional file 3.

Additional file 2: Fig. S1. RT-qPCR shows the differential expression of different genes in esophageal cell lines. The figures show the mRNA expression of SYNM, TNXB, CFD, HSPB6 and PLIN4 in HEEC, HET-1A and ESCC cell lines (* P

The possibility of super-AIs taking over the world has been intensively studied by numerous scholars. This paper focuses on the multi-AI competition scenario under the premise of super-AIs in power. Firstly, the article points out the defects of existing arguments supporting single-AI domination and presents arguments in favour of multi-AI competition. Then the article concludes that the multi-AI competition situation is a non-negligible possibility. Attention then turns to whether multi-AI competition is better for the overall good of humanity than a situation where a single AI is in power. After analysing the best, worst, and intermediate scenarios, the article concludes that multi-AI competition is better for humanity. Finally, considering the factors related to the formation of the best-case scenario of multiple AIs, the article gives some suggestions for current initiatives in AI development.

Most Outside-Knowledge Visual Question Answering (OK-VQA) systems employ a two-stage framework that first retrieves external knowledge given the visual question and then predicts the answer based on the retrieved content. However, the retrieved knowledge is often inadequate. Retrievals are frequently too general and fail to cover specific knowledge needed to answer the question. Also, the naturally available supervision (whether the passage contains the correct answer) is weak and does not guarantee question relevancy. To address these issues, we propose an Entity-Focused Retrieval (EnFoRe) model that provides stronger supervision during training and recognizes question-relevant entities to help retrieve more specific knowledge. Experiments show that our EnFoRe model achieves superior retrieval performance on OK-VQA, the currently largest outside-knowledge VQA dataset. We also combine the retrieved knowledge with state-of-the-art VQA models, and achieve a new state-of-the-art performance on OK-VQA., Comment: EMNLP 2022

Accurate localization of subsea pipelines is the prerequisite for its inspection, reinforcement and maintenance. Magnetic anomaly generated by the ferromagnetic pipeline immersed in the earth’s magnetic field can be used to locate the pipeline. However, since the magnetic field decays rapidly with distance, the detection distance is very limited. This paper proposes a method to improve the sensitivity and limit distance of pipeline detection and positioning through active magnetization. When the magnet is directly above the pipe, the magnetization effect is best, so a magnet array should be used to cover the pipe with wide enough coverage. Experiments have proved that by using a magnet array, with SNR = 20 dB as the judging criterion, the limit detection distance of a Φ219mm steel pipe can be increased from 0.363m to 0.554m.

Most recent state-of-the-art Visual Question Answering (VQA) systems are opaque black boxes that are only trained to fit the answer distribution given the question and visual content. As a result, these systems frequently take shortcuts, focusing on simple visual concepts or question priors. This phenomenon becomes more problematic as the questions become complex that requires more reasoning and commonsense knowledge. To address this issue, we present a novel framework that uses explanations for competing answers to help VQA systems select the correct answer. By training on human textual explanations, our framework builds better representations for the questions and visual content, and then reweights confidences in the answer candidates using either generated or retrieved explanations from the training set. We evaluate our framework on the VQA-X dataset, which has more difficult questions with human explanations, achieving new state-of-the-art results on both VQA and its explanations.

Most RNN-based image captioning models receive supervision on the output words to mimic human captions. Therefore, the hidden states can only receive noisy gradient signals via layers of back-propagation through time, leading to less accurate generated captions. Consequently, we propose a novel framework, Hidden State Guidance (HSG), that matches the hidden states in the caption decoder to those in a teacher decoder trained on an easier task of autoencoding the captions conditioned on the image. During training with the REINFORCE algorithm, the conventional rewards are sentence-based evaluation metrics equally distributed to each generated word, no matter their relevance. HSG provides a word-level reward that helps the model learn better hidden representations. Experimental results demonstrate that HSG clearly outperforms various state-of-the-art caption decoders using either raw images or detected objects as inputs.

Accurate localization of subsea pipeline is the prerequisite of pipeline maintenance, reinforcement and tracking inspection. Via finite element simulations and experiments, this paper reveals the distribution characterizations of the total magnetic flux density (MFD) and the reduced MFD near the steel pipeline, and demonstrates a method for accurately positioning subsea pipeline by using the external magnetic field. The characteristic peak position of the total and reduced MFDs’ components are all susceptible to the radial magnetization direction, except for the reduced MFD’s norm whose peak always overlaps the projection of the pipeline axis on the magnetic measurement plane. The norms of the total and reduced MFDs are used to determine the pipeline orientation and axis position, respectively. For an 8 m long steel pipe with diameter of 8 in., when the measurement plane is 1.5 times the diameter away from the pipe, the positioning error is around 2 cm.

Answering visual questions need acquire daily common knowledge and model the semantic connection among different parts in images, which is too difficult for VQA systems to learn from images with the only supervision from answers. Meanwhile, image captioning systems with beam search strategy tend to generate similar captions and fail to diversely describe images. To address the aforementioned issues, we present a system to have these two tasks compensate with each other, which is capable of jointly producing image captions and answering visual questions. In particular, we utilize question and image features to generate question-related captions and use the generated captions as additional features to provide new knowledge to the VQA system. For image captioning, our system attains more informative results in term of the relative improvements on VQA tasks as well as competitive results using automated metrics. Applying our system to the VQA tasks, our results on VQA v2 dataset achieve 65.8% using generated captions and 69.1% using annotated captions in validation set and 68.4% in the test-standard set. Further, an ensemble of 10 models results in 69.7% in the test-standard split.

Silk fibroin (SF)/gelatin blend nanofibers membranes as scaffolds were fabricated successfully via electrospinning with different composition ratios in formic acid. The formation of intermolecular hydrogen bonds and the conformational transition of SF provided scaffolds with excellent mechanical properties. FTIR and DTA analysis showed the SF/gelatin nanofibers had more β-sheet structures than the pure SF nanofibers. The former's breaking tenacity increased from 0.95 up to 1.60 MPa, strain at break was 7.6%, average fiber diameter was 89.2 nm, porosity was 87%, and pore diameter was 142 nm. MTT, H&E stain, and SEM results showed that the adhesion, spreading, and proliferation of human umbilic vein endothelium cells (HUVECs) and mouse fibroblasts on the SF/gelatin nanofibers scaffolds were definitely better than that on the SF nanofibers scaffolds. The scaffolds could replace the natural ECM proteins, support long-term cell growth, form three-dimensional networks of the nanofibrous structure, and grow in the direction of fiber orientation. Our results prove that the addition of gelatin improved the mechanical and biological properties of the pure SF nanofibers, these SF/gelatin blend nanofiber membranes are desirable for the scaffolds and may be a good candidate for blood vessel engineering scaffolds. © 2008 Wiley Periodicals, Inc. J Appl Polym Sci, 2009

Nanofibers could provide benefits due to high surface area for biomaterial scaffolds. In order to improve both silk fibroin and gelatin performance for potential biomedical applications, in this article the electrospinning of silk fibroin (SF)/gelatin (GT) blends with different composition ratio and concentration was performed with formic acid as a spinning solvent. The nanofibers morphologies and diameters were observed by Scanning electron microscopy (SEM). Blending with GT could improve the viscosity and spinnability of spinning solution. As-spun SF/GT blend nanofibers were more homogeneous and non-beaded than pure SF nanofibers. The diameter decreased with higher voltage at same electric field intensity. On increasing the concentrations of spinning solution, the pore diameter of nanofibers decreased, and the porosity increased. In addition, FTIR and DTA analyses showed SF/GT nanofibers had more β-sheets than pure SF nanofibers, which may be caused by an intermolecular interaction between SF and GT. Blending with GT also led to substantial enhancement in mechanical properties of SF nanofibers.

Classical thermodynamic theory still holds true in subsystem that is a percolation connected by 8 orders of self-similar 2-body-3-body coupling clusters. The fixed point, $B_2^* \equiv 3/8$, for the clusters of different size, existing in reduced second Virial coefficients has been proved by scaling theory in percolation field. It is shown that, if $B_2^* \equiv 3/8$ is combined with $B_3^* \equiv 5/8$, the potentials of 2-body-3-body coupling clusters, in critical local cluster growth phase transition, balance the kinetic energy in the glass transition. It is also proved that the glass transition corresponds to the regime in which the chemical potentials in all subsystems hold zero., Comment: 10 pages, 3 figures

The nature may be disclosed that the glass transition is only determined by the intrinsic 8 orders of instant 2-D mosaic geometric structures, without any presupposition and relevant parameter. An interface excited state on the geometric structures comes from the additional Lindemann distance increment, which is a vector with 8 orders of relaxation times, 8 orders of additional restoring force moment (ARFM), quantized energy and extra volume. Each order of anharmonic ARFM gives rise to an additional position-asymmetry on a 2-D projection plane of a reference particle, thus, in removing additional position-asymmetry, the 8 orders of 2-D clusters and hard-spheres accompanied with the 4 excited interface relaxations of the reference particle have been illustrated. Dynamical behavior comes of the slow inverse energy cascade to generate 8 orders of clusters, to thaw a solid-domain, and the fast cascade to relax tension and rearrange structure. This model provides a unified mechanism to interpret hard-sphere, compacting cluster, free volume, cage, jamming behaviors, geometrical frustration, reptation, Ising model, breaking solid lattice, percolation, cooperative migration and orientation, critical entanglement chain length and structure rearrangements. It also directly deduces a series of quantitative values for the average energy of cooperative migration in one direction, localized energy independent of temperature and the activation energy to break solid lattice. In a flexible polymer system, there are all 320 different interface excited states that have the same quantized excited energy but different interaction times, relaxation times and phases. The quantized excited energy is about 6.4 k = 0.55meV., Comment: 19 papers, 6 figures, Journal of Physics: Condens. Matter

Centralism on the basis of democracy and giving scope to democracy under the guidance of centralism—such is the dialectical relationship of unity between democracy and centralism.