Search

While deep learning has revolutionized computer-aided drug discovery, the AI community has predominantly focused on model innovation and placed less emphasis on establishing best benchmarking practices. We posit that without a sound model evaluation framework, the AI community's efforts cannot reach their full potential, thereby slowing the progress and transfer of innovation into real-world drug discovery. Thus, in this paper, we seek to establish a new gold standard for small molecule drug discovery benchmarking, WelQrate. Specifically, our contributions are threefold: WelQrate Dataset Collection - we introduce a meticulously curated collection of 9 datasets spanning 5 therapeutic target classes. Our hierarchical curation pipelines, designed by drug discovery experts, go beyond the primary high-throughput screen by leveraging additional confirmatory and counter screens along with rigorous domain-driven preprocessing, such as Pan-Assay Interference Compounds (PAINS) filtering, to ensure the high-quality data in the datasets; WelQrate Evaluation Framework - we propose a standardized model evaluation framework considering high-quality datasets, featurization, 3D conformation generation, evaluation metrics, and data splits, which provides a reliable benchmarking for drug discovery experts conducting real-world virtual screening; Benchmarking - we evaluate model performance through various research questions using the WelQrate dataset collection, exploring the effects of different models, dataset quality, featurization methods, and data splitting strategies on the results. In summary, we recommend adopting our proposed WelQrate as the gold standard in small molecule drug discovery benchmarking. The WelQrate dataset collection, along with the curation codes, and experimental scripts are all publicly available at WelQrate.org., Comment: * denotes equal contribution

We introduce QuARC, Quantum Adaptive Routing using Clusters, a novel clustering-based entanglement routing protocol that leverages redundant, multi-path routing through multi-particle projective quantum measurements to enable high-throughput, low-overhead, starvation-free entanglement distribution. At its core, QuARC periodically reconfigures the underlying quantum network into clusters of different sizes, where each cluster acts as a small network that distributes entanglement across itself, and the end-to-end entanglement is established by further distributing between clusters. QuARC does not require a-priori knowledge of any physical parameters, and is able to adapt the network configuration using static topology information, and using local (within-cluster) measurements only. We present a comprehensive simulation-based evaluation that shows QuARC is robust against changes to physical network parameters, and maintains high throughput without starvation even as network sizes scale and physical parameters degrade., Comment: 12 pages, 7 figures

In this work, we propose that all BPS vortex solutions within the generalized Maxwell-Chern-Simons-Higgs (MCSH) model can be found from a single system of equations. This set of equations is derived using the BPS Lagrangian method, which is a more robust generalization of Bogomolnyi's trick. We show that the known spherically symmetric BPS vortices can be reproduced as certain limits of Bogomolnyi equations in the generalized MCSH Model. This provides us with a possible classification system using the auxiliary functions in the BPS Lagrangian. Furthermore, we also study the properties of each known vortex through the numerical approach where we found that all of the vortices behave similarly under variations of their free parameters and a system of well-separated MCSH vortices saturates the BPS bound., Comment: 25 pages, 12 figures, single column format

GPT-4o is an autoregressive omni model that accepts as input any combination of text, audio, image, and video, and generates any combination of text, audio, and image outputs. It's trained end-to-end across text, vision, and audio, meaning all inputs and outputs are processed by the same neural network. GPT-4o can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time in conversation. It matches GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages, while also being much faster and 50\% cheaper in the API. GPT-4o is especially better at vision and audio understanding compared to existing models. In line with our commitment to building AI safely and consistent with our voluntary commitments to the White House, we are sharing the GPT-4o System Card, which includes our Preparedness Framework evaluations. In this System Card, we provide a detailed look at GPT-4o's capabilities, limitations, and safety evaluations across multiple categories, focusing on speech-to-speech while also evaluating text and image capabilities, and measures we've implemented to ensure the model is safe and aligned. We also include third-party assessments on dangerous capabilities, as well as discussion of potential societal impacts of GPT-4o's text and vision capabilities.

Understanding the exact fault location in the post-event analysis is the key to improving the accuracy of outage management. Unfortunately, the fault location is not generally well documented during the restoration process, creating a big challenge for post-event analysis. By utilizing various data source systems, including outage management system (OMS) data, asset geospatial information system (GIS) data, and vehicle location data, this paper creates a novel method to pinpoint the outage location accurately to create additional insights for distribution operations and performance teams during the post-event analysis.

In the past utilities relied on in-field inspections to identify asset defects. Recently, utilities have started using drone-based inspections to enhance the field-inspection process. We consider a vast repository of drone images, providing a wealth of information about asset health and potential issues. However, making the collected imagery data useful for automated defect detection requires significant manual labeling effort. We propose a novel solution that combines synthetic asset defect images with manually labeled drone images. This solution has several benefits: improves performance of defect detection, reduces the number of hours spent on manual labeling, and enables the capability to generate realistic images of rare defects where not enough real-world data is available. We employ a workflow that combines 3D modeling tools such as Maya and Unreal Engine to create photorealistic 3D models and 2D renderings of defective assets and their surroundings. These synthetic images are then integrated into our training pipeline augmenting the real data. This study implements an end-to-end Artificial Intelligence solution to detect assets and asset defects from the combined imagery repository. The unique contribution of this research lies in the application of advanced computer vision models and the generation of photorealistic 3D renderings of defective assets, aiming to transform the asset inspection process. Our asset detection model has achieved an accuracy of 92 percent, we achieved a performance lift of 67 percent when introducing approximately 2,000 synthetic images of 2k resolution. In our tests, the defect detection model achieved an accuracy of 73 percent across two batches of images. Our analysis demonstrated that synthetic data can be successfully used in place of real-world manually labeled data to train defect detection model.

We present the theory, implementation, and benchmarking of a real-time time-dependent density functional theory (RT-TDDFT) module within the RMG code, designed to simulate the electronic response of molecular systems to external perturbations. Our method offers insights into non-equilibrium dynamics and excited states across a diverse range of systems, from small organic molecules to large metallic nanoparticles. Benchmarking results demonstrate excellent agreement with established TDDFT implementations and showcase the superior stability of our time-integration algorithm, enabling long-term simulations with minimal energy drift. The scalability and efficiency of RMG on massively parallel architectures allow for simulations of complex systems, such as plasmonic nanoparticles with thousands of atoms. Future extensions, including nuclear and spin dynamics, will broaden the applicability of this RT-TDDFT implementation, providing a powerful toolset for studies of photoactive materials, nanoscale devices, and other systems where real-time electronic dynamics is essential.

In this paper, we consider a nonminimal coupling model between gravity and nonlinear electrodynamics with cosmological constant. This cosmological model is designed to account for both the inflationary epoch of the early universe and the current phase of accelerated cosmic expansion. The nonlinear electrodynamic fields provide a mechanism for a graceful exit from the inflationary period, preventing the universe from entering an eternal inflation state. The addition of nonminimal coupling plays a crucial role in determining whether the transition from inflation to the subsequent cosmic phases results in accelerated or decelerated expansion. We compare the theoretical predictions of our model with recent observational data and other leading cosmological models, showing that our approach provides a viable and competitive explanation for key aspects of the universe's evolution. Our results suggest that this model offers a consistent and compelling framework to explain both early-time inflation and the late-time accelerated expansion of the universe, in line with current observations., Comment: 23 pages, 10 figures. Comments are welcomed

We develop a Machine Learning Inversion method for analyzing scattering functions of mechanically driven polymers and extracting the corresponding feature parameters, which include energy parameters and conformation variables. The polymer is modeled as a chain of fixed-length bonds constrained by bending energy, and it is subject to external forces such as stretching and shear. We generate a data set consisting of random combinations of energy parameters, including bending modulus, stretching, and shear force, along with Monte Carlo-calculated scattering functions and conformation variables such as end-to-end distance, radius of gyration, and the off-diagonal component of the gyration tensor. The effects of the energy parameters on the polymer are captured by the scattering function, and principal component analysis ensures the feasibility of the Machine Learning inversion. Finally, we train a Gaussian Process Regressor using part of the data set as a training set and validate the trained regressor for inversion using the rest of the data. The regressor successfully extracts the feature parameters., Comment: 7 pages, 7 figures

In this paper, we prove the uniform energy bound for the Maxwell-Higgs system in the exterior region of Reissner-Nordstr\"om black holes. By employing an integrated local energy decay (ILED) estimate in combination with the Sobolev embedding theorem on a compact Riemannian manifold, we derived $L^\infty$ bounds for the fields. These results were then used to obtain a bound for the conformal energy of the system using the Cauchy-Schwarz inequality and Hardy-type inequalities., Comment: 29 pages, no figure, comments are welcome

The African School of Fundamental Physics and Applications, also known as the African School of Physics (ASP), was initiated in 2010, as a three-week biennial event, to offer additional training in fundamental and applied physics to African students with a minimum of three-year university education. Since its inception, ASP has grown to be much more than a school. ASP has become a series of activities and events to support academic development of African students, teachers and faculties. We report on the eighth African School of Physics, ASP2024, organized in Morocco, on April 15--19 and July 7--21, 2024. ASP2024 included programs for university students, high school teachers and high school pupils., Comment: 23 pages, 24 figures. arXiv admin note: substantial text overlap with arXiv:2302.13940

We develop off-lattice simulations of semiflexible polymer chains subjected to applied mechanical forces using Markov Chain Monte Carlo. Our approach models the polymer as a chain of fixed-length bonds, with configurations updated through adaptive non-local Monte Carlo moves. This proposed method enables precise calculation of a polymer's response to a wide range of mechanical forces, which traditional on-lattice models cannot achieve. Our approach has shown excellent agreement with theoretical predictions of persistence length and end-to-end distance in quiescent states, as well as stretching distances under tension. Moreover, our model eliminates the orientational bias present in on-lattice models, which significantly impacts calculations such as the scattering function, a crucial technique for revealing polymer conformation., Comment: 16 pages, 7 figures

Carbon dioxide levels below the soil surface are an important measurement relating to plant health, especially for plants such as perennial grasses in northern climates where ice encasement can occur over winter. In such cases, the CO$_2$ levels can build up and become toxic. This is likely a significant contributor to turfgrass death over winter; however, there is an insufficient amount of data regarding this phenomenon in large part due to the lack of effective sensors. Many off the shelf CO$_2$ sensors exist, but they are not sufficiently hardened for in ground deployment over winter. As a result, the only options currently available are very costly automated gas samplers or manual sampling at intervals with laboratory testing -- a process that results in a limited number of data points and is labor intensive. To combat this problem we have taken an established NDIR CO$_2$ sensor and hardened it for use in winter and ice encased environments to allow for continuous automated sampling of subsurface CO$_2$ levels to better understand ice encasement damage in perennial grass systems.

In this work we explore the relevance of dropout for modern language models, particularly in the context of models on the scale of <100M parameters. We explore it's relevance firstly in the regime of improving the sample efficiency of models given small, high quality datasets, and secondly in the regime of improving the quality of its fit on larger datasets where models may underfit. We find that concordant with conventional wisdom, dropout remains effective in the overfitting scenario, and that furthermore it may have some relevance for improving the fit of models even in the case of excess data, as suggested by previous research. In the process we find that the existing explanation for the mechanism behind this performance gain is not applicable in the case of language modelling., Comment: 6 pages, 3 figures, For code base see https://github.com/LeonGuertler/SuperTinyLanguageModels

Exploration is a key part of many video games. We investigate the using an exploratory agent to provide feedback on the design of procedurally generated game levels, 5 engaging levels and 5 unengaging levels. We expand upon a framework introduced in previous research which models motivations for exploration and introduce a fitness function for evaluating an environment's potential for exploration. Our study showed that our exploratory agent can clearly distinguish between engaging and unengaging levels. The findings suggest that our agent has the potential to serve as an effective tool for assessing procedurally generated levels, in terms of exploration. This work contributes to the growing field of AI-driven game design by offering new insights into how game environments can be evaluated and optimised for player exploration., Comment: 9 Pages, 9 figures, 2 tables, work in progress

In healthcare, medical image segmentation is crucial for accurate disease diagnosis and the development of effective treatment strategies. Early detection can significantly aid in managing diseases and potentially prevent their progression. Machine learning, particularly deep convolutional neural networks, has emerged as a promising approach to addressing segmentation challenges. Traditional methods like U-Net use encoding blocks for local representation modeling and decoding blocks to uncover semantic relationships. However, these models often struggle with multi-scale objects exhibiting significant variations in texture and shape, and they frequently fail to capture long-range dependencies in the input data. Transformers designed for sequence-to-sequence predictions have been proposed as alternatives, utilizing global self-attention mechanisms. Yet, they can sometimes lack precise localization due to insufficient granular details. To overcome these limitations, we introduce TransDAE: a novel approach that reimagines the self-attention mechanism to include both spatial and channel-wise associations across the entire feature space, while maintaining computational efficiency. Additionally, TransDAE enhances the skip connection pathway with an inter-scale interaction module, promoting feature reuse and improving localization accuracy. Remarkably, TransDAE outperforms existing state-of-the-art methods on the Synaps multi-organ dataset, even without relying on pre-trained weights.

This paper provides the background, methodology and validation for constructing a representation of the European high-voltage grid, including and above 200 kV, based on public data provided by OpenStreetMap. The model-independent grid dataset is published under the Open Data Commons Open Database (ODbL 1.0) licence and can be used for large-scale electricity as well as energy system modelling. The dataset and workflow are provided as part of PyPSA-Eur -- an open-source, sector-coupled optimisation model of the European energy system. By integrating with the codebase for initiatives such as PyPSA-Earth, the value of open and maintainable high-voltage grid data extends to the global context. By accessing the latest data through the the Overpass turbo API, the dataset can be easily reconstructed and updated within minutes. To assess the data quality, this paper further compares the dataset with official statistics and representative model runs using PyPSA-Eur based on different electricity grid representations., Comment: 20 pages, 15 figures, 8 tables. For associated prebuilt electricity network, see https://doi.org/10.5281/zenodo.13358976

In this paper, we investigate models of radiating gravastars with both isotropic and anisotropic interiors, incorporating various types of thin shell matter. For the isotropic interior case, we consider a thin spherical shell characterized by an equation of state in which its pressure is proportional to its mass density, enclosing a de Sitter spacetime and surrounded by Vaidya exterior spacetime. Our analysis reveals that stable gravastars can form under specific scenarios of radiative mechanisms and for certain thin shell matter types. In addition, we also show and discuss in brief the possibility of existence of stable radiating anti-de Sitter gravastar formation. For the anisotropic interior, we use an anisotropic dark energy model with a Tolman-Matese-Whitman (TMW) mass function. We explore several thin shell matter types: standard, dark energy, and repulsive phantom. Our findings indicate that stable gravastars can also emerge in this context, particularly with standard and repulsive phantom thin shells. Furthermore, our results suggest that the density of black holes is consistently higher than that of gravastars and normal stars, regardless of the type of matter in the thin shell. This observation supports the notion that gravastars and black holes are distinct entities, reinforcing the theoretical distinction between these two types of compact objects., Comment: 25 pages, 21 figures. Comments are welcome

Budget allocation of marketplace levers, such as incentives for drivers and promotions for riders, has long been a technical and business challenge at Uber; understanding lever budget changes' impact and estimating cost efficiency to achieve predefined budgets is crucial, with the goal of optimal allocations that maximize business value; we introduce an end-to-end machine learning and optimization procedure to automate budget decision-making for cities, relying on feature store, model training and serving, optimizers, and backtesting; proposing state-of-the-art deep learning (DL) estimator based on S-Learner and a novel tensor B-Spline regression model, we solve high-dimensional optimization with ADMM and primal-dual interior point convex optimization, substantially improving Uber's resource allocation efficiency., Comment: To be published in the 2nd Workshop on Causal Inference and Machine Learning in Practice, KDD 2024, August 25 to 29, 2024, Barcelona, Spain, 10 pages

We study the Cauchy problem of higher dimensional Einstein-Maxwell-Higgs system in the framework of Bondi coordinates. As a first step, the problem is reduced to a single first-order integro-differential equation by defining a generalized ansatz function. Then, we employ contraction mapping to show that there exists the unique fixed point of the problem. For a given small initial data, we prove the existence of a global classical solution. Finally, by introducing local mass and local charge functions in higher dimensions, we also show the completeness property of the spacetimes., Comment: 45 pages, no figure. arXiv admin note: text overlap with arXiv:2304.09449

We investigate and quantify the distinction between rectifiable and purely unrectifiable 1-sets in the plane. That is, given that purely unrectifiable 1-sets always have null intersections with Lipschitz images, we ask whether these sets intersect with Lipschitz images at a dimension that is close to one. In an answer to this question, we show that one-dimensional attractors of iterated function systems that satisfy the open set condition have subsets of dimension arbitrarily close to one that can be covered by Lipschitz graphs. Moreover, the Lipschitz constant of such graphs depends explicitly on the difference between the dimension of the original set and the subset that intersects with the graph., Comment: 47 pages, 8 figures, 1 table

Five dimensional super conformal field theories can be studied using their geometric realisation as a limit of $M$-theory on a metrically conical Calabi-Yau threefold. We utilise this framework to investigate the phases of such theories that arise by varying the couplings away from the conformal point. We demonstrate that many 5d SCFTs, including strongly coupled gauge theories, have couplings giving rise to massive, confining vacua with confining strings and corresponding unbroken 1-form symmetries. The simplest examples arise by considering the parameter space of {\it complete} Ricci flat metrics on discrete quotients of the standard conifold singularity. Varying other couplings produces coupled 5d SCFTs interacting via massive BPS instanton particle states., Comment: 10 pages, one figure

Rutile Germanium Dioxide (r-GeO$_2$) has been identified as an ultrawide bandgap (UWBG) semiconductor recently, featuring a bandgap of 4.68 eV, comparable to Ga$_2$O$_3$ but offering bipolar dopability, higher electron mobility, higher thermal conductivity, and higher Baliga's figure of merit (BFOM).These superior properties position GeO$_2$ as a promising material for various semiconductor applications. However, the epitaxial growth of r-GeO$_2$, particularly in its most advantageous rutile polymorph, is still at an early stage. This work explores the growth of r-GeO$_2$ using metal-organic chemical vapor deposition (MOCVD) on an r-TiO$_2$ (001) substrate, utilizing tetraethyl germane (TEGe) as the precursor. Our investigations reveal that higher growth temperatures significantly enhance crystalline quality, achieving a full width at half maximum (FWHM) of 0.181 degree at 925 degree C, compared to 0.54 degree at 840 degree C and amorphous structures at 725 degree C. Additionally, we found that longer growth durations increase surface roughness due to the formation of faceted crystals. Meanwhile, adjusting the susceptor rotation speed from 300 RPM to 170 RPM plays a crucial role in optimizing crystalline quality, effectively reducing surface roughness by approximately 15 times. This study offers a foundational guide for optimizing MOCVD growth conditions of r-GeO$_2$ films, emphasizing the crucial need for precise control over deposition temperature and rotation speed to enhance adatom mobility and effectively minimize the boundary layer thickness., Comment: 20 pages, 5 figures, 3 tables

In this article, we explore the connection between the heating phase of periodically driven CFTs and the Modular Hamiltonian of a subregion in the vacuum state. We show that the heating phase Hamiltonian corresponds to the Modular Hamiltonian, with the fixed points mapping to the endpoints of the subregion. In the bulk dual, we find that these fixed points correspond to the Ryu-Takayanagi surface of the AdS-Rindler wedge. Consequently, the entanglement entropy associated to the boundary interval within two fixed points exactly matches with the Rindler entropy of AdS-Rindler. We observe the emergent Virasoro algebra in the boundary quantization of the Modular Hamiltonian has a striking similarity with the emergent near Horizon Virasoro algebra. This is a consequence of the fact that while obtaining the boundary Virasoro algebra, a cut-off with conformal boundary condition around the fixed point is introduced, which in the bulk is related to a stretched horizon, with an emergent two-dimensional conformal symmetry. We also argue that as one tunes the parameter space of Floquet Hamiltonians to transition from the non-heating to the heating phase the operator algebra type changes from Von Neumann type $I$ to $III_1$ factor, providing a non-equilibrium analogue of the Hawking-Page transition., Comment: 22 pages, 3 figures, references added and typos corrected

We explore the physics of scrambling in the moving mirror models, in which a two-dimensional CFT is subjected to a time-dependent boundary condition. It is well-known that by choosing an appropriate mirror profile, one can model quantum aspects of black holes in two-dimensions, ranging from Hawking radiation in an eternal black hole (for an "escaping mirror") to the recent realization of Page curve in evaporating black holes (for a "kink mirror"). We explore a class of OTOCs in the presence of such a boundary and explicitly demonstrate the following primary aspects: First, we show that the dynamical CFT data directly affect an OTOC and maximally chaotic scrambling occurs for the escaping mirror for a large-$c$ CFT with identity block dominance. We further show that the exponential growth of OTOC associated with the physics of scrambling yields a power-law growth in the model for evaporating black holes which demonstrates a unitary dynamics in terms of a Page curve. We also demonstrate that, by tuning a parameter, one can naturally interpolate between an exponential growth associated to scrambling and a power-law growth in unitary dynamics. Our work explicitly exhibits the role of higher-point functions in CFT dynamics as well as the distinction between scrambling and Page curve. We also discuss several future possibilities based on this class of models., Comment: 30 pages, 6 figures

Sequence modeling plays a vital role across various domains, with recurrent neural networks being historically the predominant method of performing these tasks. However, the emergence of transformers has altered this paradigm due to their superior performance. Built upon these advances, transformers have conjoined CNNs as two leading foundational models for learning visual representations. However, transformers are hindered by the $\mathcal{O}(N^2)$ complexity of their attention mechanisms, while CNNs lack global receptive fields and dynamic weight allocation. State Space Models (SSMs), specifically the \textit{\textbf{Mamba}} model with selection mechanisms and hardware-aware architecture, have garnered immense interest lately in sequential modeling and visual representation learning, challenging the dominance of transformers by providing infinite context lengths and offering substantial efficiency maintaining linear complexity in the input sequence. Capitalizing on the advances in computer vision, medical imaging has heralded a new epoch with Mamba models. Intending to help researchers navigate the surge, this survey seeks to offer an encyclopedic review of Mamba models in medical imaging. Specifically, we start with a comprehensive theoretical review forming the basis of SSMs, including Mamba architecture and its alternatives for sequence modeling paradigms in this context. Next, we offer a structured classification of Mamba models in the medical field and introduce a diverse categorization scheme based on their application, imaging modalities, and targeted organs. Finally, we summarize key challenges, discuss different future research directions of the SSMs in the medical domain, and propose several directions to fulfill the demands of this field. In addition, we have compiled the studies discussed in this paper along with their open-source implementations on our GitHub repository., Comment: This is the first version of our survey, and the paper is currently under review

Outlier Features (OFs) are neurons whose activation magnitudes significantly exceed the average over a neural network's (NN) width. They are well known to emerge during standard transformer training and have the undesirable effect of hindering quantisation in afflicted models. Despite their practical importance, little is known behind why OFs emerge during training, nor how one can minimise them. Our work focuses on the above questions, first identifying several quantitative metrics, such as the kurtosis over neuron activation norms, to measure OFs. With these metrics, we study how architectural and optimisation choices influence OFs, and provide practical insights to minimise OFs during training. As highlights, we introduce a novel unnormalised transformer block, the Outlier Protected block, and present a previously unknown benefit of non-diagonal preconditioning optimisers, finding both approaches to significantly reduce OFs and improve quantisation without compromising convergence speed, at scales of up to 7B parameters. Notably, our combination of OP block and non-diagonal preconditioner (SOAP) achieves 14.87 int8 weight-and-activation perplexity (from 14.71 in standard precision), compared to 63.4 int8 perplexity (from 16.00) with a default OF-prone combination of Pre-Norm model and Adam, when quantising OPT-125m models post-training. Overall, our findings shed new light on our understanding of, our ability to prevent, and the complexity of this important aspect of NN training dynamics., Comment: NeurIPS 2024 camera ready

The rapid growth in the size of deep learning models strains the capabilities of traditional dense computation paradigms. Leveraging sparse computation has become increasingly popular for training and deploying large-scale models, but existing deep learning frameworks lack extensive support for sparse operations. To bridge this gap, we introduce Scorch, a library that seamlessly integrates efficient sparse tensor computation into the PyTorch ecosystem, with an initial focus on inference workloads on CPUs. Scorch provides a flexible and intuitive interface for sparse tensors, supporting diverse sparse data structures. Scorch introduces a compiler stack that automates key optimizations, including automatic loop ordering, tiling, and format inference. Combined with a runtime that adapts its execution to both dense and sparse data, Scorch delivers substantial speedups over hand-written PyTorch Sparse (torch.sparse) operations without sacrificing usability. More importantly, Scorch enables efficient computation of complex sparse operations that lack hand-optimized PyTorch implementations. This flexibility is crucial for exploring novel sparse architectures. We demonstrate Scorch's ease of use and performance gains on diverse deep learning models across multiple domains. With only minimal code changes, Scorch achieves 1.05-5.78x speedups over PyTorch Sparse on end-to-end tasks. Scorch's seamless integration and performance gains make it a valuable addition to the PyTorch ecosystem. We believe Scorch will enable wider exploration of sparsity as a tool for scaling deep learning and inform the development of other sparse libraries., Comment: 25 pages, 8 figures

The rapid advancement of large language models (LLMs) has led to significant improvements in natural language processing but also poses challenges due to their high computational and energy demands. This paper introduces a series of research efforts focused on Super Tiny Language Models (STLMs), which aim to deliver high performance with significantly reduced parameter counts. We explore innovative techniques such as byte-level tokenization with a pooling mechanism, weight tying, and efficient training strategies. These methods aim to significantly reduce reduce the parameter count compared to traditional models -- in future works, we aim to build on these in a way that maintains and improves upon the performance of base transformer models. This series of papers will explore into various subproblems, including tokenizer-free models, self-play based training, and alternative training objectives. We will target models with 10M, 50M, and 100M parameters. Our ultimate goal is to make high-performance language models more accessible and practical for a wide range of applications., Comment: 11 pages, 4 figures

The prevalence and harms of online misinformation is a perennial concern for internet platforms, institutions and society at large. Over time, information shared online has become more media-heavy and misinformation has readily adapted to these new modalities. The rise of generative AI-based tools, which provide widely-accessible methods for synthesizing realistic audio, images, video and human-like text, have amplified these concerns. Despite intense public interest and significant press coverage, quantitative information on the prevalence and modality of media-based misinformation remains scarce. Here, we present the results of a two-year study using human raters to annotate online media-based misinformation, mostly focusing on images, based on claims assessed in a large sample of publicly-accessible fact checks with the ClaimReview markup. We present an image typology, designed to capture aspects of the image and manipulation relevant to the image's role in the misinformation claim. We visualize the distribution of these types over time. We show the rise of generative AI-based content in misinformation claims, and that its commonality is a relatively recent phenomenon, occurring significantly after heavy press coverage. We also show "simple" methods dominated historically, particularly context manipulations, and continued to hold a majority as of the end of data collection in November 2023. The dataset, Annotated Misinformation, Media-Based (AMMeBa), is publicly-available, and we hope that these data will serve as both a means of evaluating mitigation methods in a realistic setting and as a first-of-its-kind census of the types and modalities of online misinformation., Comment: Grammar, spelling corrections. Minor rewording and clarification of one sentence. 24 pages, 31 figures

We consider a spherically symmetric homogeneous perfect fluid undergoing a gravitational collapse to singularity in the framework of higher-dimensional Rastall gravity in the cases of vanishing and nonvanishing cosmological constants. The possible final states of the collapse in any finite dimension are black hole and naked singularity, hence violating the cosmic censorship conjecture, but the naked singularity formation becomes less favored when the dimension is increased, such that the conjecture is fully restored in the limit of very high dimensions. We find that there are two physically distinct solutions for the collapse evolution in the case of nonzero cosmological constant: trigonometric and exponential solutions. The effective energy density of the fluid is decreasing (increasing) in the former (latter) when the magnitude of the cosmological constant is increased, which implies that the former undergoes a slower collapse than the latter. Furthermore, we find that a temporary trapped surface is possible to emerge in the case of trigonometric solution in the naked singularity region only. Therefore, faraway observers with observational time shorter than the collapse duration may conclude that a black hole is formed, although the collapse will eventually lead to a naked singularity formation., Comment: 13 pages, 6 figures, comments are welcome

Artificial Intelligence (AI) approaches are increasingly being applied to more and more domains of Science, Engineering, Chemistry, and Industries to not only improve efficiencies and enhance productivity, but also enable new capabilities. The new opportunities range from automated molecule design and screening, properties prediction, gaining insights of chemical reactions, to computer-aided design, predictive maintenance of systems, robotics, and autonomous vehicles. This review focuses on the new applications of AI in manufacturing and healthcare. For the Manufacturing Industries, we focus on AI and algorithms for (1) Battery, (2) Flow Chemistry, (3) Additive Manufacturing, (4) Sensors, and (5) Machine Vision. For Healthcare applications, we focus on: (1) Medical Vision (2) Diagnosis, (3) Protein Design, and (4) Drug Discovery. In the end, related topics are discussed, including physics integrated machine learning, model explainability, security, and governance during model deployment.

We discuss a collider probe of a dark sector model in which the dark matter is charged under a new, hidden $U(1)$ gauge group. In particular, we look for the so-called Darkstrahlung process, in which the final states dark matter radiates a new $Z'$ gauge boson and it manifests as dilepton resonances. This work emphasizes the potential of dilepton final states with missing transverse energy in probing the darkstrahlung process. We recast the ATLAS Run 2 search for dilepton resonances in association with missing energy in addition to the ATLAS and CMS searches for sleptons. We find that the recasted searches put strong constraints on the coupling between the dark matter and the $Z'$. Moreover, we evaluate refined search strategies for $Z'$ production and propose an analysis employing constraints on the lepton invariant mass and higher missing energy cut related to the darkstrahlung process. Simulation outcomes indicate substantial enhancements, particularly by a factor of 6 in regions featuring lower $Z'$ masses. Finally, we also discuss the case when the $Z'$ has a long lifetime, resulting in displaced decay of the boson., Comment: 21 pages, 14 figures, Published Version

Ulcerative Colitis (UC) is a chronic inflammatory bowel disease decreasing life quality through symptoms such as bloody diarrhoea and abdominal pain. Endoscopy is a cornerstone of diagnosis and monitoring of UC. The Mayo endoscopic subscore (MES) index is the standard for measuring UC severity during endoscopic evaluation. However, the MES is subject to high inter-observer variability leading to misdiagnosis and suboptimal treatment. We propose using a machine-learning based MES classification system to support the endoscopic process and to mitigate the observer-variability. The system runs real-time in the clinic and augments doctors' decision-making during the endoscopy. This project report outlines the process of designing, creating and evaluating our system. We describe our initial evaluation, which is a combination of a standard non-clinical model test and a first clinical test of the system on a real patient.

With the increasing emphasis on machine learning and artificial intelligence to drive knowledge discovery in the agricultural sciences, spatial internet of things (IoT) technologies have become increasingly important for collecting real-time, high resolution data for these models. However, managing large fleets of devices while maintaining high data quality remains an ongoing challenge as scientists iterate from prototype to mature end-to-end applications. Here, we provide a set of case studies using the framework of technology readiness levels for an open source spatial IoT system. The spatial IoT systems underwent 3 major and 14 minor system versions, had over 2,727 devices manufactured both in academic and commercial contexts, and are either in active or planned deployment across four continents. Our results show the evolution of a generalizable, open source spatial IoT system designed for agricultural scientists, and provide a model for academic researchers to overcome the challenges that exist in going from one-off prototypes to thousands of internet-connected devices., Comment: 20 pages, 5 figures, 1 table

In this work, we explore the inflationary dynamics induced by small fluctuations on the Skyrme brane, characterized by a time-dependent perturbative function $\tilde{\phi}$. In the low-energy regime, the model successfully reproduces standard inflation, with a potential term dictated by the Skyrmion at the brane. Gravity localization is achieved at the brane, and the lowest energy scale is established at the asymptotic boundary. The model demonstrates the capability to emulate standard inflation dynamics, resembling $\tilde{\phi}^4$ potential characteristics under certain conditions. At higher energy levels, the behaviour of $\tilde{\phi}$ is contingent upon the Skyrme term coupling constant $\lambda$, influencing reheating phases. The wave-like nature of fluctuations allows for energy transfer, resulting in a possibly lower reheating temperature. We also discuss the prospect of $\lambda$ changing sign during inflation, presenting a non-standard coupling dependent on the matter field., Comment: 14 pages, 3 figures, single column format, the analysis of dynamical system are revised for both high and low energy cases, accepted in Annals of Physics

We propose a fresh take on understanding the mechanisms of neural networks by analyzing the rich directional structure of optimization trajectories, represented by their pointwise parameters. Towards this end, we introduce some natural notions of the complexity of optimization trajectories, both qualitative and quantitative, which hallmark the directional nature of optimization in neural networks: when is there redundancy, and when exploration. We use them to reveal the inherent nuance and interplay involved between various optimization choices, such as momentum and weight decay. Further, the trajectory perspective helps us see the effect of scale on regularizing the directional nature of trajectories, and as a by-product, we also observe an intriguing heterogeneity of Q,K,V dynamics in the middle attention layers in LLMs and which is homogenized by scale. Importantly, we put the significant directional redundancy observed to the test by demonstrating that training only scalar batchnorm parameters some while into training matches the performance of training the entire network, which thus exhibits the potential of hybrid optimization schemes that are geared towards efficiency., Comment: Preprint, 57 pages

With a 676% growth rate in HIV incidence between 2010 and 2021, the HIV/AIDS epidemic in the Philippines is the one that is spreading the quickest in the western Pacific. Although the full effects of COVID-19 on HIV services and development are still unknown, it is predicted that such disruptions could lead to a significant increase in HIV casualties. Therefore, the nation needs some modeling and forecasting techniques to foresee the spread pattern and enhance the governments prevention, treatment, testing, and care program. In this study, the researcher uses Multilayer Perceptron Neural Network to forecast time series during the period when the COVID-19 pandemic strikes the nation, using statistics taken from the HIV/AIDS and ART Registry of the Philippines. After training, validation, and testing of data, the study finds that the predicted cumulative cases in the nation by 2030 will reach 145,273. Additionally, there is very little difference between observed and anticipated HIV epidemic levels, as evidenced by reduced RMSE, MAE, and MAPE values as well as a greater coefficient of determination. Further research revealed that the Philippines seems far from achieving Sustainable Development Goal 3 of Project 2030 due to an increase in the nations rate of new HIV infections. Despite the detrimental effects of COVID-19 spread on HIV/AIDS efforts nationwide, the Philippine government, under the Marcos administration, must continue to adhere to the United Nations 90-90-90 targets by enhancing its ART program and ensuring that all vital health services are readily accessible and available., Comment: 14 pages, 9 figures, Published with International Journal of Emerging Technology and Advanced Engineering (IJETAE)

Inspired by considerations in $M$-theory, we prove the equivalence between the moduli spaces of (suitably complexified) torsion free $G_{2}$-structures on 7-manifolds which are families of hyperK\"ahler ALE 4-manifolds fibered over compact flat 3-manifolds $T^{3}/K$ and certain moduli spaces of flat $G^{\mathbb{C}}_{ADE}$ connections on $T^{3}/K$.

The vision of 2030STEM is to address systemic barriers in institutional structures and funding mechanisms required to achieve full inclusion in Science, Technology, Engineering, and Mathematics (STEM) and accelerate leadership pathways for individuals from underrepresented populations across STEM sectors. 2030STEM takes a systems-level approach to create a community of practice that can test, learn and promote programs and policies that affirm and value cultural identities in STEM. To achieve parity and full representation in the STEM workforce, a variety of changes are needed across academia and STEM professional industries (e.g., business, finance, biotech, government) to accelerate underrepresented groups into positions of leadership throughout the STEM ecosystem. Through a series of subject matter interviews, roundtables, and curated analysis four major themes have surfaced, which, if implemented, could exponentially accelerate the creation of critical pathways to leadership, break down pre-existing barriers and biases, intentionally elevate the voices, value, and research of underrepresented groups in STEM, and implement new structural strategies at scale. This white paper provides a summary of innovative new practices designed to accelerate inclusion, including expanding on known global toolkits, new funding strategies, and the structural changes required throughout various STEM professions to propel pathways to leadership for underrepresented groups., Comment: 13 pages

Graph neural networks based on iterative one-hop message passing have been shown to struggle in harnessing the information from distant nodes effectively. Conversely, graph transformers allow each node to attend to all other nodes directly, but lack graph inductive bias and have to rely on ad-hoc positional encoding. In this paper, we propose a new architecture to reconcile these challenges. Our approach stems from the recent breakthroughs in long-range modeling provided by deep state-space models: for a given target node, our model aggregates other nodes by their shortest distances to the target and uses a linear RNN to encode the sequence of hop representations. The linear RNN is parameterized in a particular diagonal form for stable long-range signal propagation and is theoretically expressive enough to encode the neighborhood hierarchy. With no need for positional encoding, we empirically show that the performance of our model is comparable to or better than that of state-of-the-art graph transformers on various benchmarks, with a significantly reduced computational cost. Our code is open-source at https://github.com/skeletondyh/GRED., Comment: ICML 2024

Accurate and automated segmentation of intervertebral discs (IVDs) in medical images is crucial for assessing spine-related disorders, such as osteoporosis, vertebral fractures, or IVD herniation. We present HCA-Net, a novel contextual attention network architecture for semantic labeling of IVDs, with a special focus on exploiting prior geometric information. Our approach excels at processing features across different scales and effectively consolidating them to capture the intricate spatial relationships within the spinal cord. To achieve this, HCA-Net models IVD labeling as a pose estimation problem, aiming to minimize the discrepancy between each predicted IVD location and its corresponding actual joint location. In addition, we introduce a skeletal loss term to reinforce the model's geometric dependence on the spine. This loss function is designed to constrain the model's predictions to a range that matches the general structure of the human vertebral skeleton. As a result, the network learns to reduce the occurrence of false predictions and adaptively improves the accuracy of IVD location estimation. Through extensive experimental evaluation on multi-center spine datasets, our approach consistently outperforms previous state-of-the-art methods on both MRI T1w and T2w modalities. The codebase is accessible to the public on \href{https://github.com/xmindflow/HCA-Net}{GitHub}.

The Fractions Skill Score (FSS) is a widely used metric for assessing forecast skill, with applications ranging from precipitation to volcanic ash forecasts. By evaluating the fraction of grid squares exceeding a threshold in a neighbourhood, the intuition is that it can avoid the pitfalls of pixel-wise comparisons and identify length scales at which a forecast has skill. The FSS is typically interpreted relative to a `useful' criterion, which provides a measure of whether a forecast is considered skilful enough. However, this useful criterion is problematic, as there can be considerable skill for forecasts that do not meet it, and it is not derived in a way that provides obvious meaning. We therefore provide a new derivation of a useful criterion, as the score that a random forecast achieves, which provides a more robust and meaningful way to assess the FSS. Through illustrative examples we show that this new criterion considerably changes the length scales at which a forecast would be regarded as skilful, and reveals subtleties in how the FSS should be interpreted.

A simple design recipe for deep Transformers is to compose identical building blocks. But standard transformer blocks are far from simple, interweaving attention and MLP sub-blocks with skip connections & normalisation layers in precise arrangements. This complexity leads to brittle architectures, where seemingly minor changes can significantly reduce training speed, or render models untrainable. In this work, we ask to what extent the standard transformer block can be simplified? Combining signal propagation theory and empirical observations, we motivate modifications that allow many block components to be removed with no loss of training speed, including skip connections, projection or value parameters, sequential sub-blocks and normalisation layers. In experiments on both autoregressive decoder-only and BERT encoder-only models, our simplified transformers emulate the per-update training speed and performance of standard transformers, while enjoying 15% faster training throughput, and using 15% fewer parameters., Comment: ICLR 2024

Characterizing rectifiability of Radon measures in Euclidean space has led to fundamental contributions to geometric measure theory. Conditions involving existence of principal values of certain singular integrals \cite{mattila1995rectifiable} and the existence of densities with respect to Euclidean balls \cite{preiss1987geometry} have given rise to major breakthroughs. We explore similar questions in a rough elliptic setting where Euclidean balls $B(a,r)$ are replaced by ellipses $B_{\Lambda}(a,r)$ whose eccentricity and principal axes depend on $a$. Precisely, given $\Lambda : \mathbb{R}^{n} \to GL(n,\mathbb{R})$, we first consider the family of ellipses $B_{\Lambda}(a,r) = a + \Lambda(a) B(0,r)$ and show that almost everywhere existence of the principal values $$ \lim_{\epsilon \downarrow 0} \int_{\mathbb{R}^{n} \setminus B_{\Lambda}(a,\epsilon)} \frac{\Lambda(a)^{-1}(y-a)}{|\Lambda(a)^{-1}(y-a)|^{m+1}} d \mu(y) \in (0, \infty) $$ implies rectifiability of the measure $\mu$ under a positive lower density condition. Second we characterize rectifiability in terms of the almost everywhere existence of $$ \theta^{m}_{\Lambda(a)}(\mu,a) = \lim_{r \downarrow 0} \frac{\mu(B_{\Lambda}(a,r))}{r^{m}} \in (0, \infty).$$

Foundation models, large-scale, pre-trained deep-learning models adapted to a wide range of downstream tasks have gained significant interest lately in various deep-learning problems undergoing a paradigm shift with the rise of these models. Trained on large-scale dataset to bridge the gap between different modalities, foundation models facilitate contextual reasoning, generalization, and prompt capabilities at test time. The predictions of these models can be adjusted for new tasks by augmenting the model input with task-specific hints called prompts without requiring extensive labeled data and retraining. Capitalizing on the advances in computer vision, medical imaging has also marked a growing interest in these models. To assist researchers in navigating this direction, this survey intends to provide a comprehensive overview of foundation models in the domain of medical imaging. Specifically, we initiate our exploration by providing an exposition of the fundamental concepts forming the basis of foundation models. Subsequently, we offer a methodical taxonomy of foundation models within the medical domain, proposing a classification system primarily structured around training strategies, while also incorporating additional facets such as application domains, imaging modalities, specific organs of interest, and the algorithms integral to these models. Furthermore, we emphasize the practical use case of some selected approaches and then discuss the opportunities, applications, and future directions of these large-scale pre-trained models, for analyzing medical images. In the same vein, we address the prevailing challenges and research pathways associated with foundational models in medical imaging. These encompass the areas of interpretability, data management, computational requirements, and the nuanced issue of contextual comprehension., Comment: The paper is currently in the process of being prepared for submission to MIA

Fundamental understanding of macroscopic properties of polymer nanocomposites (PNCs) remains difficult due to the complex interplay of microscopic dynamics and structure, namely interfacial layer relaxations and three-dimensional nanoparticle arrangements. The effect of surface modification by alkyl methoxysilanes at different grafting densities has been studied in PNCs made of poly(2-vinylpyridine) and spherical 20 nm silica nanoparticles (NPs). The segmental dynamics has been probed by broadband dielectric spectroscopy, and the filler structure by small-angle X-ray scattering and reverse Monte Carlo simulations. By combining the particle configurations with the interfacial layer properties, it is shown how surface modification tunes the attractive polymer-particle interactions: bare NPs slow down the polymer interfacial layer dynamics over a thickness of ca. 5 nm, while grafting screens these interactions. Our analysis of interparticle spacing and segmental dynamics provides unprecedented insight into the effect of surface modification on the main characteristics of PNCs: particle interactions and polymer interfacial layers.

Both the dispersion state of nanoparticles (NPs) within polymer nanocomposites (PNCs) and the dynamical state of the polymer altered by the presence of the NP/polymer interfaces have a strong impact on the macroscopic properties of PNCs. In particular, mechanical properties are strongly affected by percolation of hard phases, which may be NP networks, dynamically modified polymer regions, or combinations of both. In this article, the impact on dispersion and dynamics of surface modification of the NPs by short monomethoxysilanes with eight carbons in the alkyl part (C8) is studied. As a function of grafting density and particle content, polymer dynamics is followed by broadband dielectric spectroscopy and analyzed by an interfacial layer model, whereas the particle dispersion is investigated by small-angle X-ray scattering and analyzed by reverse Monte Carlo simulations. NP dispersions are found to be destabilized only at the highest grafting. The interfacial layer formalism allows the clear identification of the volume fraction of interfacial polymer, with its characteristic time. The strongest dynamical slow-down in the polymer is found for unmodified NPs, while grafting weakens this effect progressively. The combination of all three techniques enables a unique measurement of the true thickness of the interfacial layer, which is ca. 5 nm. Finally, the comparison between longer (C18) and shorter (C8) grafts provides unprecedented insight into the efficacy and tunability of surface modification. It is shown that C8-grafting allows for a more progressive tuning, which goes beyond a pure mass effect.

The supersymmetric baby-Skyrme model is an interesting field theoretical model, and its BPS states have been studied using the usual methods. Here, we propose a novel method to rigorously obtain both topologically stable BPS and non-BPS states in the $\mathcal{N}=2$ baby Skyrme Model. It is observed that the BPS states found using this novel method coincide with the BPS states found using the usual methods. However, we are also able to obtain the non-BPS states, which break all of the supersymmetry of the theory. Furthermore, there exists a one-parameter family of non-BPS solutions that are connected to the half-BPS solutions, where half of the supersymmetry is restored when the parameter is set to zero. The proposed method is very general, and we expect that it might be useful for investigating the topologically stable non-BPS states of other theories. Thus, this method could possibly have wide applications for the study of non-BPS states in supersymmetric theories., Comment: 13 pages, 3 figures, abstract and title changed

In this paper, we prove that the initial value problem for the mass-critical defocusing nonlinear Schr\"odinger equation on the three-dimensional hyperbolic space $\mathbb{H}^3$ is globally well-posed and scatters for data with radial symmetry in the critical space $L^2 (\mathbb{H}^3)$., Comment: arXiv admin note: substantial text overlap with arXiv:1008.1237 by other authors