Search

Robustness analysis of text Classification models through adversarial attacks has gained substantial attention in recent research. This area studies the consistent behavior of text Classification models under attacks. These attacks use perturbation methods based on applying semantic and label-preserving changes to the inputs. However, the fragmented landscape of individual attack implementations, dispersed across code repositories, poses complicates the development and application of comprehensive adversarial strategies for model enhancement. To address these challenges, this paper introduces AI-Q, a Python framework specifically designed for text Classification adversarial attacks and data augmentation. One of the major strengths of our framework lies in its extensive library of perturbation methods for adversarial attacks (24 in total), and its evaluation metrics for model robustness. The framework exhibits versatility by supporting both custom models and those from the HuggingFace ecosystem, ensuring broad compatibility with leading benchmarks in the field. Beyond adversarial attacks, AI-Q can be used for data augmentation, enabling users to harness the components of adversarial attacks to increase dataset diversity. Finally, our evaluation, including human annotations, highlights the AI-Q potential for model robustness. [ABSTRACT FROM AUTHOR]

Background and Objectives: Quality assessment of bread wheat is time‐consuming and requires the determination of many complex characteristics. Because of its simplicity, protein content prediction using near‐infrared spectroscopy (NIRS) serves as the primary quality attribute in wheat trade. To enable the prediction of more complex traits, information from Raman and fluorescence spectra is added to the NIR spectra of whole grain and extracted flour. Model robustness is assessed by predictions across cultivars, locations, and years. The prediction error is corrected for the measurement error of the reference methods. Findings: Successful prediction, robustness testing, and measurement error correction were achieved for several parameters. Predicting loaf volume yielded a corrected prediction error RMSECV of 27.5 mL/100 g flour and an R² of 0.86. However, model robustness was limited due to data distribution, environmental factors, and temporal influences. Conclusions: The proposed method was proven to be suitable for applications in the wheat value chain. Furthermore, the study provides valuable insights for practical implementations. Significance and Novelty: With up to 1200 wheat samples, this is the largest study on predicting complex characteristics comprising agronomic traits; dough rheological parameters measured by Extensograph, micro‐doughLAB, and GlutoPeak; baking trial parameters like loaf volume; and specific ingredients, such as grain protein content, sugars, and minerals. [ABSTRACT FROM AUTHOR]

Recent advances in vision-language models have resulted in improved accuracy in visual question answering (VQA) tasks. However, their robustness remains limited when faced with out-of-distribution data containing unanswerable questions. In this study, we first construct a simple randomised VQA dataset, incorporating unanswerable questions from the VQA v2 dataset, to evaluate the robustness of a state-of-the-art VQA model. Our findings reveal that the model struggles to predict the "unknown" answer or provides inaccurate responses with high confidence scores for irrelevant questions. To address this issue without retraining the large backbone models, we propose Cross Modal Augmentation (CMA), a model-agnostic, test-time-only, multi-modal semantic augmentation technique. CMA generates multiple semantically-consistent but heterogeneous instances from the visual and textual inputs, which are then fed to the model, and the predictions are combined to achieve a more robust output. We demonstrate that implementing CMA enables the VQA model to provide more reliable answers in scenarios involving unanswerable questions, and show that the approach is generalisable across different categories of pre-trained vision language models. • VQA models often confidently give incorrect answers to irrelevant questions. • We enhance model robustness at test-time through multi-modal semantic augmentation. • Proposed CMA creates varied inputs for models and merges predictions for stability. • CMA variants improve VQA reliability and performance in ambiguous environments. [ABSTRACT FROM AUTHOR]

Many empirical studies have examined the relationship between urban form and canopy layer urban heat island (CUHI). However, these studies mostly have used small sample sizes and statistical methods that assume CUHI responds linearly to urban form features. Additionally, the differences in the effect of urban form on CUHI at different time instances during the diurnal cycle have not received sufficient attention. To address these issues, this study employs a relatively large sample of 1,058 microclimate sensors in Seoul, Korea, and analyzes CUHI at 9 a.m., 3 p.m., and 9 p.m. on a typical summer day. Gradient boosting decision tree (GBDT) based regression model and linear regression model and their variants with spatial information were constructed and compared for each time instance. Results show that spatially explicit GBDT models had the highest accuracy. Feature importance analysis shows that built form factors such as mean building height were more important during the afternoon, while surface fractions such as road surface fraction had greater influences during the morning and nighttime. Partial dependence plots (PDPs) show that urban form features influence CUHI in a complex nonlinear manner that varies for each time instance. PDPs were further scrutinized based on their activation and threshold effects. GBDT model findings directionally aligned with linear regression, but they provided nuanced insights into the form-CUHI relationship. These findings help planners to better understand the complexity of urban climate and intervention required to reduce CUHI magnitude across the diurnal cycle. • Nonlinearity between urban form and canyon urban heat island (CUHI) is significant. • Spatially explicit gradient boosting predicted CUHI with highest accuracy. • Nonlinearity and importance of urban form features vary during the diurnal cycle. • Policymaking regarding CUHI should consider nonlinearity and temporal variation. • Randomness in machine learning models can impact interpretability results. [ABSTRACT FROM AUTHOR]

In this work, we empirically study debiased Visual Question Answering (VQA) works with Adapters. Most VQA debiasing works sacrifice in-distribution (ID) performance for the sake of out-of-distribution (OOD) performance. Hence, we explore and experiment with the use of adapters to preserve the ID performance by training only a simple adapter network to debias and recreate performance. We conduct an extensive empirical study on recent well-established VQA debiasing works and show that the entirety of the debiasing information from the proposed debiasing methods can be captured and modelled using a single fully connected layer while preserving original network performance by skipping the adapters. Through our exploration, we find that different placements of adapters are required for different debiasing techniques and show the different possibilities of using adapters for debiasing through our experiments. We believe our findings in this work open up more questions to be asked and explored for the VQA community. • We empirically study using adapters to debias visual question answering models. • A single linear layer can shift a model from in-distribution to out-of-distribution performance. • Different debiasing techniques require different placement of adapters for best performance. • Debiased adapters can be used as a diagnosis tool for future works. [ABSTRACT FROM AUTHOR]

Watson and Holmes propose ways of investigating robustness of statistical decisions by examining certain neighbourhoods around a posterior distribution. This may partly amount to ad hoc modelling of extra uncertainty. Instead of creating neighbourhoods around the posterior a posteriori, we argue that it might be more fruitful to model a layer of extra uncertainty first, in the model building process, and then allow the data to determine how big the resulting neighbourhoods ought to be. We develop and briefly illustrate a general strategy along such lines. [ABSTRACT FROM AUTHOR]

One unintended consequence of the Electronic Health Records (EHR) implementation is the overuse of content-importing technology, such as copy-and-paste, that creates "bloated" notes containing large amounts of textual redundancy. Despite the rising interest in applying machine learning models to learn from real-patient data, it is unclear how the phenomenon of note bloat might affect the Natural Language Processing (NLP) models derived from these notes. Therefore, in this work we examine the impact of redundancy on deep learning-based NLP models, considering four clinical prediction tasks using a publicly available EHR database. We applied two deduplication methods to the hospital notes, identifying large quantities of redundancy, and found that removing the redundancy usually has little negative impact on downstream performances, and can in certain circumstances assist models to achieve significantly better results. We also showed it is possible to attack model predictions by simply adding note duplicates, causing changes of correct predictions made by trained models into wrong predictions. In conclusion, we demonstrated that EHR text redundancy substantively affects NLP models for clinical prediction tasks, showing that the awareness of clinical contexts and robust modeling methods are important to create effective and reliable NLP systems in healthcare contexts. [ABSTRACT FROM AUTHOR]

Although prospective logistic regression is the standard method of analysis for case-control data. it has been recently noted that in genetic epidemiologic studies one can use the "'retrospective'" likelihood to gain major power by incorporating various population genetics model assumptions such as Hardy-Weinberg-Equilibrium (HWE), gene-gene and gene-environment independence. In this article we review these modern methods and contrast them with the more classical approaches through two types of applications (i) association tests for typed and untyped single nucleotide polymorphisms (SNPs) and (ii) estimation of haplotype effects and haplotype-environment interactions in the presence of haplotype-phase ambiguity. We provide novel insights to existing methods by construction of various score-tests and pseudo-likelihoods. In addition, we describe a novel two-stage method for analysis of untyped SNPs that can use any flexible external algorithm for genotype imputation followed by a powerful association test based on the retrospective likelihood. We illustrate applications of the methods using simulated and real data [ABSTRACT FROM AUTHOR]

A novel molecular connectivity indix (mXtv) and electro-topological state (E-state) indexes (En) were acquired by calculating 13 substituted benzoic acids with the technique of molecule graphics. A correlative guantitative model between the 4Xpcv,E16,E40 and the regulator activity (pC) of substituted benzoic acids to plant growth was built up based on forward stepwise multiple regression tethniques. The model l~as good fitness of 0.947 and strong estimation capability. The reliable and robustness of this model was checked by Jackknifed method. [ABSTRACT FROM AUTHOR]