Search

Artificial Intelligence models encoding biology and chemistry are opening new routes to high-throughput and high-quality in-silico drug development. However, their training increasingly relies on computational scale, with recent protein language models (pLM) training on hundreds of graphical processing units (GPUs). We introduce the BioNeMo Framework to facilitate the training of computational biology and chemistry AI models across hundreds of GPUs. Its modular design allows the integration of individual components, such as data loaders, into existing workflows and is open to community contributions. We detail technical features of the BioNeMo Framework through use cases such as pLM pre-training and fine-tuning. On 256 NVIDIA A100s, BioNeMo Framework trains a three billion parameter BERT-based pLM on over one trillion tokens in 4.2 days. The BioNeMo Framework is open-source and free for everyone to use.

The Internet has significantly expanded the potential for global collaboration, allowing millions of users to contribute to collective projects like Wikipedia. While prior work has assessed the success of online collaborations, most approaches are time-agnostic, evaluating success without considering its longevity. Research on the factors that ensure the long-term preservation of high-quality standards in online collaboration is scarce. In this study, we address this gap. We propose a novel metric, `Sustainable Success,' which measures the ability of collaborative efforts to maintain their quality over time. Using Wikipedia as a case study, we introduce the SustainPedia dataset, which compiles data from over 40K Wikipedia articles, including each article's sustainable success label and more than 300 explanatory features such as edit history, user experience, and team composition. Using this dataset, we develop machine learning models to predict the sustainable success of Wikipedia articles. Our best-performing model achieves a high AU-ROC score of 0.88 on average. Our analysis reveals important insights. For example, we find that the longer an article takes to be recognized as high-quality, the more likely it is to maintain that status over time (i.e., be sustainable). Additionally, user experience emerged as the most critical predictor of sustainability. Our analysis provides insights into broader collective actions beyond Wikipedia (e.g., online activism, crowdsourced open-source software), where the same social dynamics that drive success on Wikipedia might play a role. We make all data and code used for this study publicly available for further research.

Automatic detection of online hate speech serves as a crucial step in the detoxification of the online discourse. Moreover, accurate classification can promote a better understanding of the proliferation of hate as a social phenomenon. While most prior work focus on the detection of hateful utterances, we argue that focusing on the user level is as important, albeit challenging. In this paper we consider a multimodal aggregative approach for the detection of hate-mongers, taking into account the potentially hateful texts, user activity, and the user network. We evaluate our methods on three unique datasets X (Twitter), Gab, and Parler showing that a processing a user's texts in her social context significantly improves the detection of hate mongers, compared to previously used text and graph-based methods. Our method can be then used to improve the classification of coded messages, dog-whistling, and racial gas-lighting, as well as inform intervention measures. Moreover, our approach is highly efficient even for very large datasets and networks.

Studying and building datasets for dialogue tasks is both expensive and time-consuming due to the need to recruit, train, and collect data from study participants. In response, much recent work has sought to use large language models (LLMs) to simulate both human-human and human-LLM interactions, as they have been shown to generate convincingly human-like text in many settings. However, to what extent do LLM-based simulations \textit{actually} reflect human dialogues? In this work, we answer this question by generating a large-scale dataset of 100,000 paired LLM-LLM and human-LLM dialogues from the WildChat dataset and quantifying how well the LLM simulations align with their human counterparts. Overall, we find relatively low alignment between simulations and human interactions, demonstrating a systematic divergence along the multiple textual properties, including style and content. Further, in comparisons of English, Chinese, and Russian dialogues, we find that models perform similarly. Our results suggest that LLMs generally perform better when the human themself writes in a way that is more similar to the LLM's own style.

We present an innovative approach leveraging Physics-Guided Neural Networks (PGNNs) for enhancing agricultural quality assessments. Central to our methodology is the application of physics-guided inverse regression, a technique that significantly improves the model's ability to precisely predict quality metrics of crops. This approach directly addresses the challenges of scalability, speed, and practicality that traditional assessment methods face. By integrating physical principles, notably Fick`s second law of diffusion, into neural network architectures, our developed PGNN model achieves a notable advancement in enhancing both the interpretability and accuracy of assessments. Empirical validation conducted on cucumbers and mushrooms demonstrates the superior capability of our model in outperforming conventional computer vision techniques in postharvest quality evaluation. This underscores our contribution as a scalable and efficient solution to the pressing demands of global food supply challenges.

Magnetic nulls are locations where the magnetic field vanishes. Nulls are the location of magnetic reconnection, and they determine to a large degree the magnetic connectivity in a system. We describe a novel approach to understanding movement, appearance, and disappearance of nulls in magnetic fields. This approach is based on the novel concept of isotropes, or lines where the field direction is constant. These lines are streamlines of a vector field whose flux is sourced by the topological indices of nulls, and can be conceptualized as corresponding "lines of force" between nulls. We show how this topological approach can be used to generate analytical expressions for the location of nulls in the presence of external fields for dipoles and for a field defined by the Hopf fibration.

Next generation EUV sources for photolithography use light produced by laser-produced plasmas (LPP) from ablated tin droplets. A major challenge for extending the lifetime of these devices is mitigating damage caused by deposition of tin debris on the sensitive collection mirror. Especially difficult to stop are high energy (up to 10 keV) highly charged tin ions created in the plasma. Existing solutions include the use of stopping gas, electric fields, and magnetic fields. One common configuration consists of a magnetic field perpendicular to the EUV emission direction, but such a system can result in ion populations that are trapped rather than removed. We investigate a previously unconsidered mitigation geometry consisting of a magnetic null by performing full-orbit integration of the ion trajectories in an EUV system with realistic dimensions, and optimize the coil locations for the null configuration. The magnetic null prevents a fraction of ions from hitting the mirror comparable to that of the perpendicular field, but does not trap any ions due to the chaotic nature of ion trajectories that pass close to the null. This technology can potentially improve LPP-based EUV photolithography system efficiency and lifetime, and may allow for a different, more efficient formulation of buffer gas.

The containment game is a full information game for two players, initialised with a set of occupied vertices in an infinite connected graph $G$. On the $t$-th turn, the first player, called Spreader, extends the occupied set to $g(t)$ adjacent vertices, and then the second player, called Container, removes $q$ unoccupied vertices from the graph. If the spreading process continues perpetually -- Spreader wins, and otherwise -- Container wins. For $g=\infty$ this game reduces to a solitaire game for Container, known as the Firefighter Problem. On $\mathbb{Z}^2$, for $q=1/k$ and $g\equiv 1$ it is equivalent to Conway's Angel Problem. We introduce the game, and writing $q(G,g)$ for the set of $q$ values for which Container wins against a given $g(t)$, we study the minimal asymptotics of $g(t)$ such that $q(G,g)=q(G,\infty)$, i.e. for which defeating Spreader is as hard as winning the Firefighter Problem solitaire. We show, by providing explicit winning strategies, a sub-linear upper bound $g(t)=O(t^{6/7})$ and a lower bound of $g(t)=\Omega(t^{1/2})$., Comment: 34 pages, 6 figures, 1 table

Online communities develop unique characteristics, establish social norms, and exhibit distinct dynamics among their members. Activity in online communities often results in concrete ``off-line'' actions with a broad societal impact (e.g., political street protests and norms related to sexual misconduct). While community dynamics, information diffusion, and online collaborations have been widely studied in the past two decades, quantitative studies that measure the effectiveness of online communities in promoting their agenda are scarce. In this work, we study the correspondence between the effectiveness of a community, measured by its success level in a competitive online campaign, and the underlying dynamics between its members. To this end, we define a novel task: predicting the success level of online communities in Reddit's r/place - a large-scale distributed experiment that required collaboration between community members. We consider an array of definitions for success level; each is geared toward different aspects of collaborative achievement. We experiment with several hybrid models, combining various types of features. Our models significantly outperform all baseline models over all definitions of `success level'. Analysis of the results and the factors that contribute to the success of coordinated campaigns can provide a better understanding of the resilience or the vulnerability of communities to online social threats such as election interference or anti-science trends. We make all data used for this study publicly available for further research., Comment: Accepted for publication at ICWSM 2024

We study the noise sensitivity of the minimum spanning tree (MST) of the $n$-vertex complete graph when edges are assigned independent random weights. It is known that when the graph distance is rescaled by $n^{1/3}$ and vertices are given a uniform measure, the MST converges in distribution in the Gromov-Hausdorff-Prokhorov (GHP) topology. We prove that if the weight of each edge is resampled independently with probability $\varepsilon\gg n^{-1/3}$, then the pair of rescaled minimum spanning trees -- before and after the noise -- converges in distribution to independent random spaces. Conversely, if $\varepsilon\ll n^{-1/3}$, the GHP distance between the rescaled trees goes to $0$ in probability. This implies the noise sensitivity and stability for every property of the MST that corresponds to a continuity set of the random limit. The noise threshold of $n^{-1/3}$ coincides with the critical window of the Erd\H{o}s-R\'enyi random graphs. In fact, these results follow from an analog theorem we prove regarding the minimum spanning forest of critical random graphs.

Recently, haplo-identical transplantation with multiple HLA mismatches has become a viable option for system cell transplants. Haplotype sharing detection requires imputation of donor and recipient. We show that even in high-resolution typing when all alleles are known, there is a 15% error rate in haplotype phasing, and even more in low resolution typings. Similarly, in related donors, parents haplotypes should be imputed to determine what haplotype each child inherited. We propose GRAMM (GRaph bAsed FaMilly iMputation) to phase alleles in family pedigree HLA typing data, and in mother-cord blood unit pairs. We show that GRAMM has practically no phasing errors when pedigree data are available. We apply GRAMM to simulations with different typing resolutions as well as paired cord-mother typings, and show very high phasing accuracy, and improved alleles imputation accuracy. We use GRAMM to detect recombination events and show that the rate of falsely detected recombination events (False Positive Rate) in simulations is very low. We then apply recombination detection to typed families to estimate the recombination rate in Israeli and Australian population datasets. The estimated recombination rate has an upper bound of 10-20% per family (1-4% per individual). GRAMM is available at: https://gramm.math.biu.ac.il/., Comment: 23 pages (20 + 3 of Supp. Mat.), 9 figures (6 + 3 of Supp. Mat.) Submitted to NAR (Nucleic Acids Research)

Dusty plasmas are known to support a diverse range of instabilities, including both generalizations of standard plasma instabilities and ones caused by effects specific to dusty systems. It has been recently demonstrated that a novel broad class of streaming instabilities, termed resonant drag instabilities (RDIs), can be attributed to a particular resonance phenomenon, manifested by defective eigenvalues of the linearized dust/fluid system. In this work, it is demonstrated that this resonance phenomenon is not unique to RDIs and can be used as a framework to understand a wider range of instabilities, termed resonant instabilities. Particular attention is given to the filamentary ionization instability seen in laboratory dusty plasmas and to the two-stream instability. It is shown that, due to the commonalities in underlying physics between the dust-ion-acoustic two-stream instability and the acoustic RDI, these instabilities should be relevant in strongly overlapping regimes in astrophysical dusty plasmas. It is proposed that a similar overlap in the experimental accessibility of these modes (and of the filamentary instability) allows for the possibility of experimental investigation in the laboratory of complex and astrophysically relevant instability dynamics., Comment: 21 pages, 15 figures

We address the task of controlled generation of small molecules, which entails finding novel molecules with desired properties under certain constraints (e.g., similarity to a reference molecule). Here we introduce MolMIM, a probabilistic auto-encoder for small molecule drug discovery that learns an informative and clustered latent space. MolMIM is trained with Mutual Information Machine (MIM) learning, and provides a fixed length representation of variable length SMILES strings. Since encoder-decoder models can learn representations with ``holes'' of invalid samples, here we propose a novel extension to the training procedure which promotes a dense latent space, and allows the model to sample valid molecules from random perturbations of latent codes. We provide a thorough comparison of MolMIM to several variable-size and fixed-size encoder-decoder models, demonstrating MolMIM's superior generation as measured in terms of validity, uniqueness, and novelty. We then utilize CMA-ES, a naive black-box and gradient free search algorithm, over MolMIM's latent space for the task of property guided molecule optimization. We achieve state-of-the-art results in several constrained single property optimization tasks as well as in the challenging task of multi-objective optimization, improving over previous success rate SOTA by more than 5\% . We attribute the strong results to MolMIM's latent representation which clusters similar molecules in the latent space, whereas CMA-ES is often used as a baseline optimization method. We also demonstrate MolMIM to be favourable in a compute limited regime, making it an attractive model for such cases., Comment: Published at the MLDD workshop, ICLR 2023. version 2. 8 pages, 4 figures, 4 tables

Social platforms such as Gab and Parler, branded as `free-speech' networks, have seen a significant growth of their user base in recent years. This popularity is mainly attributed to the stricter moderation enforced by mainstream platforms such as Twitter, Facebook, and Reddit. In this work we provide the first large scale analysis of hate-speech on Parler. We experiment with an array of algorithms for hate-speech detection, demonstrating limitations of transfer learning in that domain, given the illusive and ever changing nature of the ways hate-speech is delivered. In order to improve classification accuracy we annotated 10K Parler posts, which we use to fine-tune a BERT classifier. Classification of individual posts is then leveraged for the classification of millions of users via label propagation over the social network. Classifying users by their propensity to disseminate hate, we find that hate mongers make 16.1\% of Parler active users, and that they have distinct characteristics comparing to other user groups. We find that hate mongers are more active, more central and express distinct levels of sentiment and convey a distinct array of emotions like anger and sadness. We further complement our analysis by comparing the trends discovered in Parler and those found in Gab. To the best of our knowledge, this is among the first works to analyze hate speech in Parler in a quantitative manner and on the user level, and the first annotated dataset to be made available to the community.

Understanding collective decision making at a large-scale, and elucidating how community organization and community dynamics shape collective behavior are at the heart of social science research. In this work we study the behavior of thousands of communities with millions of active members. We define a novel task: predicting which community will undertake an unexpected, large-scale, distributed campaign. To this end, we develop a hybrid model, combining textual cues, community meta-data, and structural properties. We show how this multi-faceted model can accurately predict large-scale collective decision-making in a distributed environment. We demonstrate the applicability of our model through Reddit's r/place - a large-scale online experiment in which millions of users, self-organized in thousands of communities, clashed and collaborated in an effort to realize their agenda. Our hybrid model achieves a high F1 prediction score of 0.826. We find that coarse meta-features are as important for prediction accuracy as fine-grained textual cues, while explicit structural features play a smaller role. Interpreting our model, we provide and support various social insights about the unique characteristics of the communities that participated in the \r/place experiment. Our results and analysis shed light on the complex social dynamics that drive collective behavior, and on the factors that propel user coordination. The scale and the unique conditions of the \rp~experiment suggest that our findings may apply in broader contexts, such as online activism, (countering) the spread of hate speech and reducing political polarization. The broader applicability of the model is demonstrated through an extensive analysis of the WallStreetBets community, their role in r/place and four years later, in the GameStop short squeeze campaign of 2021.