Search

Recent advances in vision-language models have resulted in improved accuracy in visual question answering (VQA) tasks. However, their robustness remains limited when faced with out-of-distribution data containing unanswerable questions. In this study, we first construct a simple randomised VQA dataset, incorporating unanswerable questions from the VQA v2 dataset, to evaluate the robustness of a state-of-the-art VQA model. Our findings reveal that the model struggles to predict the "unknown" answer or provides inaccurate responses with high confidence scores for irrelevant questions. To address this issue without retraining the large backbone models, we propose Cross Modal Augmentation (CMA), a model-agnostic, test-time-only, multi-modal semantic augmentation technique. CMA generates multiple semantically-consistent but heterogeneous instances from the visual and textual inputs, which are then fed to the model, and the predictions are combined to achieve a more robust output. We demonstrate that implementing CMA enables the VQA model to provide more reliable answers in scenarios involving unanswerable questions, and show that the approach is generalisable across different categories of pre-trained vision language models. • VQA models often confidently give incorrect answers to irrelevant questions. • We enhance model robustness at test-time through multi-modal semantic augmentation. • Proposed CMA creates varied inputs for models and merges predictions for stability. • CMA variants improve VQA reliability and performance in ambiguous environments. [ABSTRACT FROM AUTHOR]

Abstract This article is concerned with inference in parametric copula setups, where both the marginals and the copula have parametric forms. For such models, two-stage maximum likelihood estimation, often referred to as inference function for margins, is used as an attractive alternative to the full maximum likelihood estimation strategy. Previous studies of the two-stage maximum likelihood estimator have largely been based on the assumption that the chosen parametric model captures the true model that generated data. We study the impact of dropping this true model assumption, both theoretically and numerically. We first show that the two-stage maximum likelihood estimator is consistent for a well-defined least false parameter value, different from the analogous least false parameter associated with the full maximum likelihood procedure. Then we demonstrate limiting normality of the full vector of estimators, with concise matrix notation for the variance matrices involved. Along with consistent estimators for these, we have built a model-robust machinery for inference in parametric copula models. The special case where the parametric model is assumed to hold corresponds to situations studied earlier in the literature, with simpler formulas for variance matrices. As a numerical illustration, we perform a set of simulations. We also analyze five-dimensional Norwegian precipitation data. We find that the variance of the copula parameter estimate can both increase and decrease, by dropping the true model assumption. In addition, we observe that the two-stage maximum likelihood estimator is still highly efficient when the true model assumption is dropped and thus the model robust asymptotic variance formulas are used. Additionally, we discover that using highly misspecified models can lead to situations where the asymptotic variance of the two-stage maximum likelihood estimator is lower than that of full maximum likelihood estimator. Our results are also used to analyze the mean squared error properties for both the full and the two-stage maximum likelihood estimators of any focus parameter. [ABSTRACT FROM AUTHOR]

In this work, we empirically study debiased Visual Question Answering (VQA) works with Adapters. Most VQA debiasing works sacrifice in-distribution (ID) performance for the sake of out-of-distribution (OOD) performance. Hence, we explore and experiment with the use of adapters to preserve the ID performance by training only a simple adapter network to debias and recreate performance. We conduct an extensive empirical study on recent well-established VQA debiasing works and show that the entirety of the debiasing information from the proposed debiasing methods can be captured and modelled using a single fully connected layer while preserving original network performance by skipping the adapters. Through our exploration, we find that different placements of adapters are required for different debiasing techniques and show the different possibilities of using adapters for debiasing through our experiments. We believe our findings in this work open up more questions to be asked and explored for the VQA community. • We empirically study using adapters to debias visual question answering models. • A single linear layer can shift a model from in-distribution to out-of-distribution performance. • Different debiasing techniques require different placement of adapters for best performance. • Debiased adapters can be used as a diagnosis tool for future works. [ABSTRACT FROM AUTHOR]

Model robustness has always been the research focus for near infrared spectroscopy technique. In this study, we compared the model performance of four different kinds of spectra (transmittance, reflectance, absorption (μ a) and reduced scattering (μ′ s)) for the prediction of fat and protein of milk. The protein content of milk in prediction set was totally out of range of fat calibration set. The same happens in the case of the fat content of milk in prediction set and protein calibration set. The aim of this present study was to investigate the effect of the difference of protein content on the robustness of four fat calibration models, and of the difference of fat content on the robustness of four protein calibration models. The single integrating sphere system was first calibrated by using 90 liquid phantoms. The detection accuracy of the calibrated system was externally validated by another 80 phantoms. It turned out that the system can measure the absorption and reduced scattering coefficients with relatively high precision in the range of 780–1073 nm, which gave mean relative errors less than 12% and 5% for μ a and μ′ s respectively. Partial least squares regression (PLSR) and stepwise multiple linear regression (SMLR) models were established, and the results showed that calibration model based on reduced scattering spectra could predict fat with the highest robustness for both regression methods, while the absorption spectra could predict protein content with the highest robustness by using SMLR. The separation of absorption and reduced scattering has great potential in improving the robustness of calibration models. Highlights • Calibrated integrating sphere system can realise high accuracy μ a , μ' s measurement. • Robustness of calibration models established by R , T , μ a , μ' s spectra were compared. • Calibration model based on μ' s spectra could predict fat with highest robustness. • Calibration model based on μ a spectra could predict protein with highest robustness. [ABSTRACT FROM AUTHOR]

One unintended consequence of the Electronic Health Records (EHR) implementation is the overuse of content-importing technology, such as copy-and-paste, that creates "bloated" notes containing large amounts of textual redundancy. Despite the rising interest in applying machine learning models to learn from real-patient data, it is unclear how the phenomenon of note bloat might affect the Natural Language Processing (NLP) models derived from these notes. Therefore, in this work we examine the impact of redundancy on deep learning-based NLP models, considering four clinical prediction tasks using a publicly available EHR database. We applied two deduplication methods to the hospital notes, identifying large quantities of redundancy, and found that removing the redundancy usually has little negative impact on downstream performances, and can in certain circumstances assist models to achieve significantly better results. We also showed it is possible to attack model predictions by simply adding note duplicates, causing changes of correct predictions made by trained models into wrong predictions. In conclusion, we demonstrated that EHR text redundancy substantively affects NLP models for clinical prediction tasks, showing that the awareness of clinical contexts and robust modeling methods are important to create effective and reliable NLP systems in healthcare contexts. [ABSTRACT FROM AUTHOR]