Search

Continual learning (CL) provides a framework for training models in ever-evolving environments. Although re-occurrence of previously seen objects or tasks is common in real-world problems, the concept of repetition in the data stream is not often considered in standard benchmarks for CL. Unlike with the rehearsal mechanism in buffer-based strategies, where sample repetition is controlled by the strategy, repetition in the data stream naturally stems from the environment. This report provides a summary of the CLVision challenge at CVPR 2023, which focused on the topic of repetition in class-incremental learning. The report initially outlines the challenge objective and then describes three solutions proposed by finalist teams that aim to effectively exploit the repetition in the stream to learn continually. The experimental results from the challenge highlight the effectiveness of ensemble-based solutions that employ multiple versions of similar modules, each trained on different but overlapping subsets of classes. This report underscores the transformative potential of taking a different perspective in CL by employing repetition in the data stream to foster innovative strategy design., Comment: Preprint; Challenge Report of the 4th Workshop on Continual Learning in Computer Vision at CVPR

Molecular modeling at the atomic level has been applied in a wide range of biological systems. The widely adopted additive force fields typically use fixed atom-centered partial charges to model electrostatic interactions. However, the additive force fields cannot accurately model polarization effects, leading to unrealistic simulations in polarization-sensitive processes. Numerous efforts have been invested in developing induced dipole-based polarizable force fields. Whether additive atomic charge models or polarizable induced dipole models are used, proper parameterization of the electrostatic term plays a key role in the force field developments. In this work, we present a Python program called PyRESP for performing atomic multipole parameterizations by reproducing ab initio electrostatic potential (ESP) around molecules. PyRESP provides parameterization schemes for several electrostatic models, including the RESP model with atomic charges for the additive force fields and the RESP-ind and RESP-perm models with additional induced and permanent dipole moments for the polarizable force fields. PyRESP is a flexible and user-friendly program that can accommodate various needs during force field parameterizations for molecular modeling of any organic molecules.

A key advantage of polarizable force fields is their ability to model the atomic polarization effects that play key roles in the atomic many-body interactions. In this work, we assessed the accuracy of the recently developed polarizable Gaussian Multipole (pGM) models in reproducing quantum mechanical (QM) interaction energies, many-body interaction energies, as well as the nonadditive and additive contributions to the many-body interactions for peptide main-chain hydrogen-bonding conformers, using glycine dipeptide oligomers as the model systems. Two types of pGM models were considered, including that with (pGM-perm) and without (pGM-ind) permanent atomic dipoles. The performances of the pGM models were compared with several widely used force fields, including two polarizable (Amoeba13 and ff12pol) and three additive (ff19SB, ff15ipq, and ff03) force fields. Encouragingly, the pGM models outperform all other force fields in terms of reproducing QM interaction energies, many-body interaction energies, as well as the nonadditive and additive contributions to the many-body interactions, as measured by the root-mean-square errors (RMSEs) and mean absolute errors (MAEs). Furthermore, we tested the robustness of the pGM models against polarizability parameterization errors by employing alternative polarizabilities that are either scaled or obtained from other force fields. The results show that the pGM models with alternative polarizabilities exhibit improved accuracy in reproducing QM many-body interaction energies as well as the nonadditive and additive contributions compared with other polarizable force fields, suggesting that the pGM models are robust against the errors in polarizability parameterizations. This work shows that the pGM models are capable of accurately modeling polarization effects and have the potential to serve as templates for developing next-generation polarizable force fields for modeling various biological systems.

Induced dipole models have proven to be effective tools for simulating electronic polarization effects in biochemical processes, yet their potential has been constrained by energy conservation issue, particularly when historical data is utilized for dipole prediction. This study identifies error outliers as the primary factor causing this failure of energy conservation and proposes a comprehensive scheme to overcome this limitation. Leveraging maximum relative errors as a convergence metric, our data demonstrates that energy conservation can be upheld even when using historical information for dipole predictions. Our study introduces the multi-order extrapolation method to quicken induction iteration and optimize the use of historical data, while also developing the preconditioned conjugate gradient with local iterations to refine the iteration process and effectively remove error outliers. This scheme further incorporates a "peek" step via Jacobi under-relaxation for optimal performance. Simulation evidence suggests that our proposed scheme can achieve energy convergence akin to that of point-charge models within a limited number of iterations, thus promising significant improvements in efficiency and accuracy.

AmberTools is a free and open-source collection of programs used to set up, run, and analyze molecular simulations. The newer features contained within AmberTools23 are briefly described in this Application note.

Target deconvolution is a crucial but costly and time-consuming task that hinders large-scale profiling for drug discovery. We present a matrix-augmented pooling strategy (MAPS) which mixes multiple drugs into samples with optimized permutation and delineates targets of each drug simultaneously with mathematical processing. We validated this strategy with thermal proteome profiling (TPP) testing of 15 drugs concurrently, increasing experimental throughput by 60x while maintaining high sensitivity and specificity. Benefiting from the lower cost and higher throughput of MAPS, we performed target deconvolution of the 15 drugs across 5 cell lines. Our profiling revealed that drug-target interactions can differ vastly in targets and binding affinity across cell lines. We further validated BRAF and CSNK2A2 as potential off-targets of bafetinib and abemaciclib, respectively. This work represents the largest thermal profiling of structurally diverse drugs across multiple cell lines to date.

Adversarial attacks have been proven to be potential threats to Deep Neural Networks (DNNs), and many methods are proposed to defend against adversarial attacks. However, while enhancing the robustness, the clean accuracy will decline to a certain extent, implying a trade-off existed between the accuracy and robustness. In this paper, to meet the trade-off problem, we theoretically explore the underlying reason for the difference of the filters' weight distribution between standard-trained and robust-trained models and then argue that this is an intrinsic property for static neural networks, thus they are difficult to fundamentally improve the accuracy and adversarial robustness at the same time. Based on this analysis, we propose a sample-wise dynamic network architecture named Adversarial Weight-Varied Network (AW-Net), which focuses on dealing with clean and adversarial examples with a "divide and rule" weight strategy. The AW-Net adaptively adjusts the network's weights based on regulation signals generated by an adversarial router, which is directly influenced by the input sample. Benefiting from the dynamic network architecture, clean and adversarial examples can be processed with different network weights, which provides the potential to enhance both accuracy and adversarial robustness. A series of experiments demonstrate that our AW-Net is architecture-friendly to handle both clean and adversarial examples and can achieve better trade-off performance than state-of-the-art robust models., Comment: Accepted by TPAMI2024

The advancement of deep learning has facilitated the integration of Artificial Intelligence (AI) into clinical practices, particularly in computer-aided diagnosis. Given the pivotal role of medical images in various diagnostic procedures, it becomes imperative to ensure the responsible and secure utilization of AI techniques. However, the unauthorized utilization of AI for image analysis raises significant concerns regarding patient privacy and potential infringement on the proprietary rights of data custodians. Consequently, the development of pragmatic and cost-effective strategies that safeguard patient privacy and uphold medical image copyrights emerges as a critical necessity. In direct response to this pressing demand, we present a pioneering solution named Medical Image Adversarial watermarking (MIAD-MARK). Our approach introduces watermarks that strategically mislead unauthorized AI diagnostic models, inducing erroneous predictions without compromising the integrity of the visual content. Importantly, our method integrates an authorization protocol tailored for legitimate users, enabling the removal of the MIAD-MARK through encryption-generated keys. Through extensive experiments, we validate the efficacy of MIAD-MARK across three prominent medical image datasets. The empirical outcomes demonstrate the substantial impact of our approach, notably reducing the accuracy of standard AI diagnostic models to a mere 8.57% under white box conditions and 45.83% in the more challenging black box scenario. Additionally, our solution effectively mitigates unauthorized exploitation of medical images even in the presence of sophisticated watermark removal networks. Notably, those AI diagnosis networks exhibit a meager average accuracy of 38.59% when applied to images protected by MIAD-MARK, underscoring the robustness of our safeguarding mechanism.

Adversarial Training (AT) has been widely proved to be an effective method to improve the adversarial robustness against adversarial examples for Deep Neural Networks (DNNs). As a variant of AT, Adversarial Robustness Distillation (ARD) has demonstrated its superior performance in improving the robustness of small student models with the guidance of large teacher models. However, both AT and ARD encounter the robust fairness problem: these models exhibit strong robustness when facing part of classes (easy class), but weak robustness when facing others (hard class). In this paper, we give an in-depth analysis of the potential factors and argue that the smoothness degree of samples' soft labels for different classes (i.e., hard class or easy class) will affect the robust fairness of DNN models from both empirical observation and theoretical analysis. Based on the above finding, we propose an Anti-Bias Soft Label Distillation (ABSLD) method to mitigate the adversarial robust fairness problem within the framework of Knowledge Distillation (KD). Specifically, ABSLD adaptively reduces the student's error risk gap between different classes to achieve fairness by adjusting the class-wise smoothness degree of samples' soft labels during the training process, and the smoothness degree of soft labels is controlled by assigning different temperatures in KD to different classes. Extensive experiments demonstrate that ABSLD outperforms state-of-the-art AT, ARD, and robust fairness methods in terms of overall performance of robustness and fairness.

Adversarial examples have attracted widespread attention in security-critical applications because of their transferability across different models. Although many methods have been proposed to boost adversarial transferability, a gap still exists between capabilities and practical demand. In this paper, we argue that the model-specific discriminative regions are a key factor causing overfitting to the source model, and thus reducing the transferability to the target model. For that, a patch-wise mask is utilized to prune the model-specific regions when calculating adversarial perturbations. To accurately localize these regions, we present a learnable approach to automatically optimize the mask. Specifically, we simulate the target models in our framework, and adjust the patch-wise mask according to the feedback of the simulated models. To improve the efficiency, the differential evolutionary (DE) algorithm is utilized to search for patch-wise masks for a specific image. During iterative attacks, the learned masks are applied to the image to drop out the patches related to model-specific regions, thus making the gradients more generic and improving the adversarial transferability. The proposed approach is a preprocessing method and can be integrated with existing methods to further boost the transferability. Extensive experiments on the ImageNet dataset demonstrate the effectiveness of our method. We incorporate the proposed approach with existing methods to perform ensemble attacks and achieve an average success rate of 93.01% against seven advanced defense methods, which can effectively enhance the state-of-the-art transfer-based attack performance.

Adversarial training is a practical approach for improving the robustness of deep neural networks against adversarial attacks. Although bringing reliable robustness, the performance toward clean examples is negatively affected after adversarial training, which means a trade-off exists between accuracy and robustness. Recently, some studies have tried to use knowledge distillation methods in adversarial training, achieving competitive performance in improving the robustness but the accuracy for clean samples is still limited. In this paper, to mitigate the accuracy-robustness trade-off, we introduce the Multi-Teacher Adversarial Robustness Distillation (MTARD) to guide the model's adversarial training process by applying a strong clean teacher and a strong robust teacher to handle the clean examples and adversarial examples, respectively. During the optimization process, to ensure that different teachers show similar knowledge scales, we design the Entropy-Based Balance algorithm to adjust the teacher's temperature and keep the teachers' information entropy consistent. Besides, to ensure that the student has a relatively consistent learning speed from multiple teachers, we propose the Normalization Loss Balance algorithm to adjust the learning weights of different types of knowledge. A series of experiments conducted on public datasets demonstrate that MTARD outperforms the state-of-the-art adversarial training and distillation methods against various adversarial attacks.

Pantetheine is ubiquitous in nature in various forms of pantetheine-containing ligands (PCLs), including coenzyme A and phosphopantetheine. Lack of scalable force field libraries for PCLs has hampered the computational studies of biological macromolecules containing PCLs. We describe here the development of the first generation Pantetheine Force Field (PFF) library that is compatible with Amber force fields; parameterized using Gasteiger, AM1-BCC, or RESP charging methods combined with gaff2 and ff14SB parameter sets. In addition, a "plug-and-play" strategy was employed to enable the systematic charging of computationally expensive molecules sharing common substructural motifs. The validation studies performed on the PFF library showed promising performance where molecular dynamics (MD) simulations results were compared with experimental data of three representative systems. The PFF library represents the first force field library capable of modeling systems containing PCLs in silico and will aid in various applications including protein engineering and drug discovery.

Polyketides are a large class of structurally and functionally diverse natural products with important bioactivities. Many polyketides are synthesized by reducing type II polyketide synthases (PKSs), containing transiently interacting standalone enzymes. During synthesis, ketoreductase (KR) catalyzes regiospecific carbonyl to hydroxyl reduction, determining the product outcome, yet little is known about what drives specific KR-substrate interactions. In this study, computational approaches were used to explore KR-substrate interactions based on previously solved apo and mimic cocrystal structures. We found five key factors guiding KR-substrate binding. First, two major substrate binding motifs were identified. Second, substrate length is the key determinant of substrate binding position. Third, two key residues in chain length specificity were confirmed. Fourth, phosphorylation of substrates is critical for binding. Finally, packing/hydrophobic effects primarily determine the binding stability. The molecular bases revealed here will help further engineering of type II PKSs and directed biosynthesis of new polyketides.

Polyketides are a large class of structurally and functionally diverse natural products with important bioactivities. Many polyketides are synthesized by reducing type II polyketide synthases (PKSs), containing transiently interacting standalone enzymes. During synthesis, ketoreductase (KR) catalyzes regiospecific carbonyl to hydroxyl reduction, determining the product outcome, yet little is known about what drives specific KR-substrate interactions. In this study, computational approaches were used to explore KR-substrate interactions based on previously solved apo and mimic cocrystal structures. We found five key factors guiding KR-substrate binding. First, two major substrate binding motifs were identified. Second, substrate length is the key determinant of substrate binding position. Third, two key residues in chain length specificity were confirmed. Fourth, phosphorylation of substrates is critical for binding. Finally, packing/hydrophobic effects primarily determine the binding stability. The molecular bases revealed here will help further engineering of type II PKSs and directed biosynthesis of new polyketides.

Various computational methodologies can be applied to enzymological studies on enzymes in the fatty acid, polyketide, and non-ribosomal peptide biosynthetic pathways. These multi-domain complexes are called fatty acid synthases, polyketide synthases, and non-ribosomal peptide synthetases. These mega-synthases biosynthesize chemically diverse and complex bioactive molecules, with the intermediates being chauffeured between catalytic partners via a carrier protein. Recent efforts have been made to engineer these systems to expand their product diversity. A major stumbling block is our poor understanding of the transient protein-protein and protein-substrate interactions between the carrier protein and its many catalytic partner domains and product intermediates. The innate reactivity of pathway intermediates in two major classes of polyketide synthases has frustrated our mechanistic understanding of these interactions during the biosynthesis of these natural products, ultimately impeding the engineering of these systems for the generation of engineered natural products. Computational techniques described in this chapter can aid data interpretation or used to generate testable models of these experimentally intractable transient interactions, thereby providing insight into key interactions that are difficult to capture otherwise, with the potential to expand the diversity in these systems.