Search

With the recent advancements in generative modeling, the realism of deepfake content has been increasing at a steady pace, even reaching the point where people often fail to detect manipulated media content online, thus being deceived into various kinds of scams. In this paper, we survey deepfake generation and detection techniques, including the most recent developments in the field, such as diffusion models and Neural Radiance Fields. Our literature review covers all deepfake media types, comprising image, video, audio and multimodal (audio-visual) content. We identify various kinds of deepfakes, according to the procedure used to alter or generate the fake content. We further construct a taxonomy of deepfake generation and detection methods, illustrating the important groups of methods and the domains where these methods are applied. Next, we gather datasets used for deepfake detection and provide updated rankings of the best performing deepfake detectors on the most popular datasets. In addition, we develop a novel multimodal benchmark to evaluate deepfake detectors on out-of-distribution content. The results indicate that state-of-the-art detectors fail to generalize to deepfake content generated by unseen deepfake generators. Finally, we propose future directions to obtain robust and powerful deepfake detectors. Our project page and new benchmark are available at https://github.com/CroitoruAlin/biodeep.

Path planning is a complex problem for many practical applications, particularly in robotics. Existing algorithms, however, are exhaustive in nature and become increasingly complex when additional side constraints are incorporated alongside distance minimization. In this paper, a novel approach using vision language models (VLMs) is proposed for enabling path planning in complex wireless-aware environments. To this end, insights from a digital twin (DT) with real-world wireless ray tracing data are explored in order to guarantee an average path gain threshold while minimizing the trajectory length. First, traditional approaches such as A* are compared to several wireless-aware extensions, and an optimal iterative dynamic programming approach (DP-WA*) is derived, which fully takes into account all path gains and distance metrics within the DT. On the basis of these baselines, the role of VLMs as an alternative assistant for path planning is investigated, and a strategic chain-of-thought tasking (SCoTT) approach is proposed. SCoTT divides the complex planning task into several subproblems and solves each with advanced CoT prompting. Results show that SCoTT achieves very close average path gains compared to DP-WA* while at the same time yielding consistently shorter path lengths. The results also show that VLMs can be used to accelerate DP-WA* by efficiently reducing the algorithm's search space and thus saving up to 62\% in execution time. This work underscores the potential of VLMs in future digital systems as capable assistants for solving complex tasks, while enhancing user interaction and accelerating rapid prototyping under diverse wireless constraints.

Multi-task large language models (MTLLMs) are important for many applications at the wireless edge, where users demand specialized models to handle multiple tasks efficiently. However, training MTLLMs is complex and exhaustive, particularly when tasks are subject to change. Recently, the concept of model fusion via task vectors has emerged as an efficient approach for combining fine-tuning parameters to produce an MTLLM. In this paper, the problem of enabling edge users to collaboratively craft such MTTLMs via tasks vectors is studied, under the assumption of worst-case adversarial attacks. To this end, first the influence of adversarial noise to multi-task model fusion is investigated and a relationship between the so-called weight disentanglement error and the mean squared error (MSE) is derived. Using hypothesis testing, it is directly shown that the MSE increases interference between task vectors, thereby rendering model fusion ineffective. Then, a novel resilient MTLLM fusion (R-MTLLMF) is proposed, which leverages insights about the LLM architecture and fine-tuning process to safeguard task vector aggregation under adversarial noise by realigning the MTLLM. The proposed R-MTLLMF is then compared for both worst-case and ideal transmission scenarios to study the impact of the wireless channel. Extensive model fusion experiments with vision LLMs demonstrate R-MTLLMF's effectiveness, achieving close-to-baseline performance across eight different tasks in ideal noise scenarios and significantly outperforming unprotected model fusion in worst-case scenarios. The results further advocate for additional physical layer protection for a holistic approach to resilience, from both a wireless and LLM perspective.

Associative memory models, such as Hopfield networks and their modern variants, have garnered renewed interest due to advancements in memory capacity and connections with self-attention in transformers. In this work, we introduce a unified framework-Hopfield-Fenchel-Young networks-which generalizes these models to a broader family of energy functions. Our energies are formulated as the difference between two Fenchel-Young losses: one, parameterized by a generalized entropy, defines the Hopfield scoring mechanism, while the other applies a post-transformation to the Hopfield output. By utilizing Tsallis and norm entropies, we derive end-to-end differentiable update rules that enable sparse transformations, uncovering new connections between loss margins, sparsity, and exact retrieval of single memory patterns. We further extend this framework to structured Hopfield networks using the SparseMAP transformation, allowing the retrieval of pattern associations rather than a single pattern. Our framework unifies and extends traditional and modern Hopfield networks and provides an energy minimization perspective for widely used post-transformations like $\ell_2$-normalization and layer normalization-all through suitable choices of Fenchel-Young losses and by using convex analysis as a building block. Finally, we validate our Hopfield-Fenchel-Young networks on diverse memory recall tasks, including free and sequential recall. Experiments on simulated data, image retrieval, multiple instance learning, and text rationalization demonstrate the effectiveness of our approach., Comment: 49 pages, 14 figures. arXiv admin note: text overlap with arXiv:2402.13725

Pointers are a powerful, but dangerous feature provided by the C and C++ programming languages, and incorrect use of pointers is a common source of bugs and security vulnerabilities. Making secure software is crucial, as vulnerabilities exploited by malicious actors not only lead to monetary losses, but possibly loss of human lives. Fixing these vulnerabilities is costly if they are found at the end of development, and the cost will be even higher if found after deployment. That is why it is desirable to find the bugs as early in the development process as possible. We propose a framework that can statically find use-after-free bugs at compile-time and report the errors to the users. It works by tracking the lifetime of objects and memory locations pointers might point to and, using this information, a possibly invalid dereferencing of a pointer can be detected. The framework was tested on over 100 handwritten small tests, as well as 5 real-world projects, and has shown good results detecting errors, while at the same time highlighting some scenarios where false positive reports may occur. Based on the results, it was concluded that our framework achieved its goals, as it is able to detect multiple patterns of use-after-free bugs, and correctly report the errors to the programmer., Comment: In Proceedings FROM 2024, arXiv:2410.23020

GPT-4o is an autoregressive omni model that accepts as input any combination of text, audio, image, and video, and generates any combination of text, audio, and image outputs. It's trained end-to-end across text, vision, and audio, meaning all inputs and outputs are processed by the same neural network. GPT-4o can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time in conversation. It matches GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages, while also being much faster and 50\% cheaper in the API. GPT-4o is especially better at vision and audio understanding compared to existing models. In line with our commitment to building AI safely and consistent with our voluntary commitments to the White House, we are sharing the GPT-4o System Card, which includes our Preparedness Framework evaluations. In this System Card, we provide a detailed look at GPT-4o's capabilities, limitations, and safety evaluations across multiple categories, focusing on speech-to-speech while also evaluating text and image capabilities, and measures we've implemented to ensure the model is safe and aligned. We also include third-party assessments on dangerous capabilities, as well as discussion of potential societal impacts of GPT-4o's text and vision capabilities.

Speech recognition performance varies by language, domain, and speaker characteristics such as accent, and fine-tuning a model on any of these categories may lead to catastrophic forgetting. $k$ nearest neighbor search ($k$NN), first proposed for neural sequence decoders for natural language generation (NLG) and machine translation (MT), is a non-parametric method that can instead adapt by building an external datastore that can then be searched during inference time, without training the underlying model. We show that Whisper, a transformer end-to-end speech model, benefits from $k$NN. We investigate the differences between the speech and text setups. We discuss implications for speaker adaptation, and analyze improvements by gender, accent, and age., Comment: 6 pages incl. appendix, 2 figures, 6 tables

The stochastic sewing lemma recently introduced by Le~(2020) allows to construct a unique limit process from a doubly indexed stochastic process that satisfies some regularity. This lemma is stated in a given probability space on which these processes are defined. The present paper develops a version of this lemma for probability measures: from a doubly indexed family of maps on the set of probability measures that have a suitable probabilistic representation, we are able to construct a limit flow of maps on the probability measures. This result complements and improves the existing result coming from the classical sewing lemma. It is applied to the case of law-dependent jump SDEs for which we obtain weak existence result as well as the uniqueness of the marginal laws.

Document structure editing involves manipulating localized textual, visual, and layout components in document images based on the user's requests. Past works have shown that multimodal grounding of user requests in the document image and identifying the accurate structural components and their associated attributes remain key challenges for this task. To address these, we introduce the DocEdit-v2, a novel framework that performs end-to-end document editing by leveraging Large Multimodal Models (LMMs). It consists of three novel components: (1) Doc2Command, which simultaneously localizes edit regions of interest (RoI) and disambiguates user edit requests into edit commands; (2) LLM-based Command Reformulation prompting to tailor edit commands originally intended for specialized software into edit instructions suitable for generalist LMMs. (3) Moreover, DocEdit-v2 processes these outputs via Large Multimodal Models like GPT-4V and Gemini, to parse the document layout, execute edits on grounded Region of Interest (RoI), and generate the edited document image. Extensive experiments on the DocEdit dataset show that DocEdit-v2 significantly outperforms strong baselines on edit command generation (2-33%), RoI bounding box detection (12-31%), and overall document editing (1-12\%) tasks., Comment: EMNLP 2024 (Main)

Training interatomic potentials for spin-polarized systems continues to be a difficult task for the molecular modeling community. In this note, a proof-of-concept, random initial spin committee approach is proposed for obtaining the ground state of spin-polarized systems with a controllable degree of accuracy. The approach is tested on two toy models of elemental sulfur where the exact optimal spin configuration can be known. Machine-learning potentials are trained on the resulting data, and increasingly accurate fits with respect to the ground state are achieved, marking a step towards machine-learning force fields for general bulk spin-polarized systems., Comment: The supporting data will be uploaded at https://doi.org/10.17863/CAM.112212 after the publication is accepted

We introduce LDAdam, a memory-efficient optimizer for training large models, that performs adaptive optimization steps within lower dimensional subspaces, while consistently exploring the full parameter space during training. This strategy keeps the optimizer's memory footprint to a fraction of the model size. LDAdam relies on a new projection-aware update rule for the optimizer states that allows for transitioning between subspaces, i.e., estimation of the statistics of the projected gradients. To mitigate the errors due to low-rank projection, LDAdam integrates a new generalized error feedback mechanism, which explicitly accounts for both gradient and optimizer state compression. We prove the convergence of LDAdam under standard assumptions, and show that LDAdam allows for accurate and efficient fine-tuning and pre-training of language models., Comment: 36 pages

Designing adaptive documents that are visually appealing across various devices and for diverse viewers is a challenging task. This is due to the wide variety of devices and different viewer requirements and preferences. Alterations to a document's content, style, or layout often necessitate numerous adjustments, potentially leading to a complete layout redesign. We introduce FlexDoc, a framework for creating and consuming documents that seamlessly adapt to different devices, author, and viewer preferences and interactions. It eliminates the need for manually creating multiple document layouts, as FlexDoc enables authors to define desired document properties using templates and employs both discrete and continuous optimization in a novel comprehensive optimization process, which leverages automatic text summarization and image carving techniques to adapt both layout and content during consumption dynamically. Furthermore, we demonstrate FlexDoc in multiple real-world application scenarios, such as news readers and academic papers.

Large language models (LLM) are increasingly strong contenders in machine translation. We study document-level translation, where some words cannot be translated without context from outside the sentence. We investigate the ability of prominent LLMs to utilize context by analyzing models' robustness to perturbed and randomized document context. We find that LLMs' improved document-translation performance is not always reflected in pronoun translation performance. We highlight the need for context-aware finetuning of LLMs with a focus on relevant parts of the context to improve their reliability for document-level translation., Comment: 4 pages, 2 figures, 2 tables

The emergence of Segment Anything (SAM) sparked research interest in the field of interactive segmentation, especially in the context of image editing tasks and speeding up data annotation. Unlike common semantic segmentation, interactive segmentation methods allow users to directly influence their output through prompts (e.g. clicks). However, click patterns in real-world interactive segmentation scenarios remain largely unexplored. Most methods rely on the assumption that users would click in the center of the largest erroneous area. Nevertheless, recent studies show that this is not always the case. Thus, methods may have poor performance in real-world deployment despite high metrics in a baseline benchmark. To accurately simulate real-user clicks, we conducted a large crowdsourcing study of click patterns in an interactive segmentation scenario and collected 475K real-user clicks. Drawing on ideas from saliency tasks, we develop a clickability model that enables sampling clicks, which closely resemble actual user inputs. Using our model and dataset, we propose RClicks benchmark for a comprehensive comparison of existing interactive segmentation methods on realistic clicks. Specifically, we evaluate not only the average quality of methods, but also the robustness w.r.t. click patterns. According to our benchmark, in real-world usage interactive segmentation models may perform worse than it has been reported in the baseline benchmark, and most of the methods are not robust. We believe that RClicks is a significant step towards creating interactive segmentation methods that provide the best user experience in real-world cases., Comment: Accepted by NeurIPS 2024

Quantum Software (QSW) uses the principles of quantum mechanics, specifically programming quantum bits (qubits) that manipulate quantum gates, to implement quantum computing systems. QSW has become a specialized field of software development, requiring specific notations, languages, patterns, and tools for mapping the behavior of qubits and the structure of quantum gates to components and connectors of QSW architectures. To support declarative modeling of QSW, we aim to enable architecture-driven development, where software engineers can design, program, and evaluate quantum software systems by abstracting complex details through high-level components and connectors. We introduce QADL (Quantum Architecture Description Language), which provides a specification language, design space, and execution environment for architecting QSW. Inspired by classical ADLs, QADL offers (1) a graphical interface to specify and design QSW components, (2) a parser for syntactical correctness, and (3) an execution environment by integrating QADL with IBM Qiskit. The initial evaluation of QADL is based on usability assessments by a team of quantum physicists and software engineers, using quantum algorithms such as Quantum Teleportation and Grover's Search. QADL offers a pioneering specification language and environment for QSW architecture. A demo is available at https://youtu.be/xaplHH_3NtQ., Comment: 6 pages, 8 figures, conference

Our Galaxy - the Milky Way - has certain features of the structure and evolution. The morphological, photometric, kinematic, and chemodynamical properties are usually considered in the search for the Milky Way galaxies-analogues (MWAs). The discovery of MWA galaxies with a larger number of simultaneous selection parameters and more stringent constraints on a given parameter yields a sample of MWA galaxies with properties closer to the true properties of the Milky Way. So, in general, such MW parameters as the morphological type, luminosity, color indices, structural parameters (size, bar, bulge, thin and thick disks, inner ring, halo), bulge to total ratio, stellar mass, star formation rate, metallicity, and rotation velocity were used in various combinations for comparison with other galaxies. However, the offset of some MW features in the multiparameter space of MWAs features should be significant. The paper aims to give a brief overview of the problematics and to present our approach for studying Milky Way and MWAs matching characteristics (this project is supported by the National Research Fund of Ukraine). We propose to enlarge as much as possible the number of Milky Way features and compile various samples of MWAs in our comoving cosmological volume for their further optimization. Such features can include 3Dkinematics of star's movement in certain regions, low oxygen content on the periphery, low nuclear activity, and the lack of significant merging over the past 10 Gyrs (isolation criterion). This approach will make it possible to widely formulate the necessary and sufficient conditions for the detection of MWA galaxies as well as to reveal other MW multiwavelength features., Comment: 7 pages

Upon almost-every realisation of the Brownian continuum random tree (CRT), it is possible to define a canonical diffusion process or `Brownian motion'. The main result of this article establishes that the cover time of the Brownian motion on the Brownian CRT (i.e.\ the time taken by the process in question to visit the entire state space) is equal to the infimum over the times at which the associated local times are strictly positive everywhere. The proof of this result depends on the recursive self-similarity of the Brownian CRT and a novel version of the first Ray-Knight theorem for trees, which is of independent interest. As a consequence, we obtain that the suitably-rescaled cover times of simple random walks on critical, finite variance Galton-Watson trees converge in distribution with respect to their annealed laws to the cover time of Brownian motion on the Brownian CRT. Other families of graphs that have the Brownian CRT as a scaling limit are also covered. Additionally, we partially confirm a 1991 conjecture of David Aldous regarding related cover-and-return times., Comment: 55 pages, 4 figures

The first step in the formulation and study of the Riemann Hypothesis is the analytic continuation of the Riemann Zeta Function (RZF) in the full Complex Plane with a pole at $s=1$. In the current work, we study the analytic continuation of two random versions of RZF using, for $Re s>1$, the Euler representation of ZF in terms of the product of functions over primes. In the first case, we substitute in the Euler product pseudo-prime numbers from the famous Cram\'er Model. In the second case, we use pseudo-primes with local symmetries. We show that in the Cram\'er case analytic continuation is possible $\mathbb{P}$-a.s. for $Res>1/2$, but not through the critical line $Re s=1/2.$ In the second case, we show that the analytic continuation is possible in a larger domain. We also study for the Cram\'er pseudo-primes several problems from Additive Number Theory., Comment: 12 pages, no figures

We address the challenge of efficient auto-regressive generation in sequence prediction models by introducing FutureFill - a method for fast generation that applies to any sequence prediction algorithm based on convolutional operators. Our approach reduces the generation time requirement from quadratic to quasilinear relative to the context length. Additionally, FutureFill requires a prefill cache sized only by the number of tokens generated, which is smaller than the cache requirements for standard convolutional and attention-based models. We validate our theoretical findings with experimental evidence demonstrating correctness and efficiency gains in a synthetic generation task.

In [7], Caravenna, Sun and Zygouras gave general criteria for the partition functions of binary valued spin systems with a relevant random field perturbation to have non-trivial continuum and weak disorder limits. In this work, we show how these criteria can be extended to non-binary valued spin systems., Comment: 16 pages

Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development.

This paper highlights the significance of natural language processing (NLP) within artificial intelligence, underscoring its pivotal role in comprehending and modeling human language. Recent advancements in NLP, particularly in conversational bots, have garnered substantial attention and adoption among developers. This paper explores advanced methodologies for attaining smaller and more efficient NLP models. Specifically, we employ three key approaches: (1) training a Transformer-based neural network to detect offensive language, (2) employing data augmentation and knowledge distillation techniques to increase performance, and (3) incorporating multi-task learning with knowledge distillation and teacher annealing using diverse datasets to enhance efficiency. The culmination of these methods has yielded demonstrably improved outcomes., Comment: Accepted by NLDB2024

In this article we obtain certain inequalities involving the entropy of a positive integer and divergence of two positive integers, respectively the entropy of an ideal and divergence of two ideals of a ring of algebraic integers., Comment: 16 pages, 1 figure. Key words and phrases. entropy, divergence, numbers, ideals, ramification theory in algebraic number fields

This paper derives an achievable rate-distortion (R-D) region for the state-dependent discrete memoryless multiple access channel (SD-DMMAC), where the generalized feedback and causal side information are present at encoders, and the decoder performs the joint task of message decoding and state estimation. The Markov coding and backward-forward two-stage decoding schemes are adopted in the proof. This scenario is shown to be capable of modeling various integrated sensing and communication (ISAC) applications, including the monostatic-uplink system and multi-modal sensor networks, which are then studied as examples., Comment: Accepted by IEEE Information Theory Workshop 2024

We introduce the AIM 2024 UHD-IQA Challenge, a competition to advance the No-Reference Image Quality Assessment (NR-IQA) task for modern, high-resolution photos. The challenge is based on the recently released UHD-IQA Benchmark Database, which comprises 6,073 UHD-1 (4K) images annotated with perceptual quality ratings from expert raters. Unlike previous NR-IQA datasets, UHD-IQA focuses on highly aesthetic photos of superior technical quality, reflecting the ever-increasing standards of digital photography. This challenge aims to develop efficient and effective NR-IQA models. Participants are tasked with creating novel architectures and training strategies to achieve high predictive performance on UHD-1 images within a computational budget of 50G MACs. This enables model deployment on edge devices and scalable processing of extensive image collections. Winners are determined based on a combination of performance metrics, including correlation measures (SRCC, PLCC, KRCC), absolute error metrics (MAE, RMSE), and computational efficiency (G MACs). To excel in this challenge, participants leverage techniques like knowledge distillation, low-precision inference, and multi-scale training. By pushing the boundaries of NR-IQA for high-resolution photos, the UHD-IQA Challenge aims to stimulate the development of practical models that can keep pace with the rapidly evolving landscape of digital photography. The innovative solutions emerging from this competition will have implications for various applications, from photo curation and enhancement to image compression., Comment: ECCV 2024 - Advances in Image Manipulation (AIM). arXiv admin note: text overlap with arXiv:2401.10511 by other authors

In this note, we give an exact formula for a general family of rational zeta series involving the coefficient $\zeta(2n)$ in terms of Hurwitz zeta values. This formula generalizes two formulas from a previous paper of the first author. Our method will involve derivatives polynomials for the cotangent function.

This paper presents a novel deep neural network-based architecture tailored for Speech Emotion Recognition (SER). The architecture capitalises on dense interconnections among multiple layers of bidirectional dilated convolutions. A linear kernel dynamically fuses the outputs of these layers to yield the final emotion class prediction. This innovative architecture is denoted as TBDM-Net: Temporally-Aware Bi-directional Dense Multi-Scale Network. We conduct a comprehensive performance evaluation of TBDM-Net, including an ablation study, across six widely-acknowledged SER datasets for unimodal speech emotion recognition. Additionally, we explore the influence of gender-informed emotion prediction by appending either golden or predicted gender labels to the architecture's inputs or predictions. The implementation of TBDM-Net is accessible at: https://github.com/adrianastan/tbdm-net, Comment: In Proceedings of 2024 IEEE International Workshop on Machine Learning for Signal Processing, London, UK

Despite the immense potential of AI-powered medical devices to revolutionize healthcare, concerns regarding their safety in life-critical applications remain. While the European regulatory framework provides a comprehensive approach to medical device software development, it falls short in addressing AI-specific considerations. This article proposes a model to bridge this gap by extending the general idea of AI lifecycle with regulatory activities relevant to AI-enabled medical systems.

The rising footprint of machine learning has led to a focus on imposing \emph{model sparsity} as a means of reducing computational and memory costs. For deep neural networks (DNNs), the state-of-the-art accuracy-vs-sparsity is achieved by heuristics inspired by the classical Optimal Brain Surgeon (OBS) framework~\citep{lecun90brain, hassibi1992second, hassibi1993optimal}, which leverages loss curvature information to make better pruning decisions. Yet, these results still lack a solid theoretical understanding, and it is unclear whether they can be improved by leveraging connections to the wealth of work on sparse recovery algorithms. In this paper, we draw new connections between these two areas and present new sparse recovery algorithms inspired by the OBS framework that comes with theoretical guarantees under reasonable assumptions and have strong practical performance. Specifically, our work starts from the observation that we can leverage curvature information in OBS-like fashion upon the projection step of classic iterative sparse recovery algorithms such as IHT. We show for the first time that this leads both to improved convergence bounds under standard assumptions. Furthermore, we present extensions of this approach to the practical task of obtaining accurate sparse DNNs, and validate it experimentally at scale for Transformer-based models on vision and language tasks.

We analyzed the Hartree-Fock approximation for an electron system. The interaction between particles is modeled by a non-Coulombian potential. We analyzed both the three-dimensional and two-dimensional systems. We obtained accurate analytical results for the particle energy, the particle velocity, the ground state energy of the system as well as the momentum dependent density of states. The previous classical results for the Coulombian case were reobtained as particular cases., Comment: 23 pages, 3 figures

Restoring limb motor function in individuals with spinal cord injury (SCI), stroke, or amputation remains a critical challenge, one which affects millions worldwide. Recent studies show through surface electromyography (EMG) that spared motor neurons can still be voluntarily controlled, even without visible limb movement . These signals can be decoded and used for motor intent estimation; however, current wearable solutions lack the necessary hardware and software for intuitive interfacing of the spared degrees of freedom after neural injuries. To address these limitations, we developed a wireless, high-density EMG bracelet, coupled with a novel software framework, MyoGestic. Our system allows rapid and tailored adaptability of machine learning models to the needs of the users, facilitating real-time decoding of multiple spared distinctive degrees of freedom. In our study, we successfully decoded the motor intent from two participants with SCI, two with spinal stroke , and three amputees in real-time, achieving several controllable degrees of freedom within minutes after wearing the EMG bracelet. We provide a proof-of-concept that these decoded signals can be used to control a digitally rendered hand, a wearable orthosis, a prosthesis, or a 2D cursor. Our framework promotes a participant-centered approach, allowing immediate feedback integration, thus enhancing the iterative development of myocontrol algorithms. The proposed open-source software framework, MyoGestic, allows researchers and patients to focus on the augmentation and training of the spared degrees of freedom after neural lesions, thus potentially bridging the gap between research and clinical application and advancing the development of intuitive EMG interfaces for diverse neural lesions., Comment: 23 pages, 8 figures

Quantum computing proposes a revolutionary paradigm that can radically transform numerous scientific and industrial application domains. To realize this promise, new capabilities need software solutions that are able to effectively harness its power. However, developers face significant challenges when developing quantum software due to the high computational demands of simulating quantum computers on classical systems. In this paper, we investigate the potential of using an accessible and cost-efficient manner remote computational capabilities to improve the experience of quantum software developers.

In this work, we survey recent studies on masked image modeling (MIM), an approach that emerged as a powerful self-supervised learning technique in computer vision. The MIM task involves masking some information, e.g. pixels, patches, or even latent representations, and training a model, usually an autoencoder, to predicting the missing information by using the context available in the visible part of the input. We identify and formalize two categories of approaches on how to implement MIM as a pretext task, one based on reconstruction and one based on contrastive learning. Then, we construct a taxonomy and review the most prominent papers in recent years. We complement the manually constructed taxonomy with a dendrogram obtained by applying a hierarchical clustering algorithm. We further identify relevant clusters via manually inspecting the resulting dendrogram. Our review also includes datasets that are commonly used in MIM research. We aggregate the performance results of various masked image modeling methods on the most popular datasets, to facilitate the comparison of competing methods. Finally, we identify research gaps and propose several interesting directions of future work.

We introduce Imagen 3, a latent diffusion model that generates high quality images from text prompts. We describe our quality and responsibility evaluations. Imagen 3 is preferred over other state-of-the-art (SOTA) models at the time of evaluation. In addition, we discuss issues around safety and representation, as well as methods we used to minimize the potential harm of our models.

We prove the equivalence between the integral central limit theorem and the local central limit theorem for two-body potentials with long-range interactions on the lattice $\mathbb{Z}^d$ for $d\ge 1$. The spin space can be an arbitrary, possibly unbounded subset of the real axis with a suitable a-priori measure. For general unbounded spins, our method works at high-enough temperature, but for bounded spins our results hold for every temperature. Our proof relies on the control of the integrated characteristic function, which is achieved by dividing the integration into three different regions, following a standard approach proposed forty years ago by Campanino, Del Grosso and Tirozzi. The bounds required in the different regions are obtained through cluster-expansion techniques. For bounded spins, the arbitrariness of the temperature is achieved through a decimation ("dilution") technique, also introduced in the later reference.

We present a study of the scattering of a monochromatic helical laser beam, described by a Laguerre-Gauss solution of the Maxwell equations, on an electron sheet, initially at rest in the focal plane of the laser; the interaction is described in the framework of a local plane wave approximation. We calculate the scattered electromagnetic field observed in an arbitrary point at a large distance from the laser focal spot, by adding coherently the contributions of each electron in the electron sheet. Due to the interference effects, the radiation is emitted only into the forward direction, within a narrow cone, and it has a spatial structure that we analyze theoretically and numerically. For circularly polarized incident fields, the structure is also helical, with a helical index which depends on the helical index of the incident radiation and the harmonic order. These structures can be observed experimentally, as each harmonic order is emitted with a different frequency, and within cones of different opening angles. Our findings are in agreement with experimental results in the literature which demonstrate the generation of OAM carrying photons by radiation scattering on electrons.

We are releasing a new suite of security benchmarks for LLMs, CYBERSECEVAL 3, to continue the conversation on empirically measuring LLM cybersecurity risks and capabilities. CYBERSECEVAL 3 assesses 8 different risks across two broad categories: risk to third parties, and risk to application developers and end users. Compared to previous work, we add new areas focused on offensive security capabilities: automated social engineering, scaling manual offensive cyber operations, and autonomous offensive cyber operations. In this paper we discuss applying these benchmarks to the Llama 3 models and a suite of contemporaneous state-of-the-art LLMs, enabling us to contextualize risks both with and without mitigations in place.

Learning good representations involves capturing the diverse ways in which data samples relate. Contrastive loss - an objective matching related samples - underlies methods from self-supervised to multimodal learning. Contrastive losses, however, can be viewed more broadly as modifying a similarity graph to indicate how samples should relate in the embedding space. This view reveals a shortcoming in contrastive learning: the similarity graph is binary, as only one sample is the related positive sample. Crucially, similarities \textit{across} samples are ignored. Based on this observation, we revise the standard contrastive loss to explicitly encode how a sample relates to others. We experiment with this new objective, called $\mathbb{X}$-Sample Contrastive, to train vision models based on similarities in class or text caption descriptions. Our study spans three scales: ImageNet-1k with 1 million, CC3M with 3 million, and CC12M with 12 million samples. The representations learned via our objective outperform both contrastive self-supervised and vision-language models trained on the same data across a range of tasks. When training on CC12M, we outperform CLIP by $0.6\%$ on both ImageNet and ImageNet Real. Our objective appears to work particularly well in lower-data regimes, with gains over CLIP of $16.8\%$ on ImageNet and $18.1\%$ on ImageNet Real when training with CC3M. Finally, our objective seems to encourage the model to learn representations that separate objects from their attributes and backgrounds, with gains of $3.3$-$5.6$\% over CLIP on ImageNet9. We hope the proposed solution takes a small step towards developing richer learning objectives for understanding sample relations in foundation models.

The control system in surgical robots must ensure patient safety and real time control. As such, all the uncertainties which could appear should be considered into an extended model of the plant. After such an uncertain plant is formed, an adequate controller which ensures a minimum set of performances for each situation should be computed. As such, the continuous-time robust control paradigm is suitable for such scenarios. However, the problem is generally solved only for linear and time invariant plants. The main focus of the current paper is to include m-link serial surgical robots into Robust Control Framework by considering all nonlinearities as uncertainties. Moreover, the paper studies an incipient problem of numerical implementation of such control structures.

Context: The emergence of quantum computing proposes a revolutionary paradigm that can radically transform numerous scientific and industrial application domains. The ability of quantum computers to scale computations beyond what the current computers are capable of implies better performance and efficiency for certain algorithmic tasks. Objective: However, to benefit from such improvement, quantum computers must be integrated with existing software systems, a process that is not straightforward. In this paper, we propose a unified execution model that addresses the challenges that emerge from building hybrid classical-quantum applications at scale. Method: Following the Design Science Research methodology, we proposed a convention for mapping quantum resources and artifacts to Kubernetes concepts. Then, in an experimental Kubernetes cluster, we conducted experiments for scheduling and executing quantum tasks on both quantum simulators and hardware. Results: The experimental results demonstrate that the proposed platform Qubernetes (or Kubernetes for quantum) exposes the quantum computation tasks and hardware capabilities following established cloud-native principles, allowing seamless integration into the larger Kubernetes ecosystem. Conclusion: The quantum computing potential cannot be realised without seamless integration into classical computing. By validating that it is practical to execute quantum tasks in a Kubernetes infrastructure, we pave the way for leveraging the existing Kubernetes ecosystem as an enabler for hybrid classical-quantum computing.

Research in the field of language models is rapidly evolving, with many open models being released to the public. Openly available pretraining corpora usually focus on only a handful of languages, with many others either missing completely or extremely underrepresented. In this report, we introduce FuLG, a hundred-fifty-billion-token Romanian corpus extracted from CommonCrawl. We present our methodology for filtering FuLG and compare it via ablation studies against existing Romanian corpora.

The European Committee for Future Accelerators (ECFA) Early-Career Researcher (ECR) panel, which represents the interests of the ECR community to ECFA, presents in this document its initiatives and activities in the year 2023. This report summarises the process of the first big turnover in the panel composition at the start of 2023 and reports on the activities of the active working groups - either pursued from before or newly established. The overarching goal of the ECFA-ECR panel is to better understand and support the diverse interests of early-career researchers in the ECFA community and beyond., Comment: Editors: Jan-Hendrik Arling, Cecilia Borca, Armin Ilg, Arnau Morancho Tarda, Holly Pacey, Marko Pesut, Elisabetta Spadaro Norella and Marta Urbaniak. arXiv admin note: substantial text overlap with arXiv:2212.11238

Split federated learning (SFL) is a compute-efficient paradigm in distributed machine learning (ML), where components of large ML models are outsourced to remote servers. A significant challenge in SFL, particularly when deployed over wireless channels, is the susceptibility of transmitted model parameters to adversarial jamming that could jeopardize the learning process. This is particularly pronounced for word embedding parameters in large language models (LLMs), which are crucial for language understanding. In this paper, rigorous insights are provided into the influence of jamming LLM word embeddings in SFL by deriving an expression for the ML training loss divergence and showing that it is upper-bounded by the mean squared error (MSE). Based on this analysis, a physical layer framework is developed for resilient SFL with LLMs (R-SFLLM) over wireless networks. R-SFLLM leverages wireless sensing data to gather information on the jamming directions-of-arrival (DoAs) for the purpose of devising a novel, sensing-assisted anti-jamming strategy while jointly optimizing beamforming, user scheduling, and resource allocation. Extensive experiments using BERT and RoBERTa models demonstrate R-SFLLM's effectiveness, achieving close-to-baseline performance across various natural language processing (NLP) tasks and datasets. The proposed methodology further introduces an adversarial training component, where controlled noise exposure significantly enhances the LLM's resilience to perturbed parameters during training. The results show that more noise-sensitive models, such as RoBERTa, benefit from this feature, especially when resource allocation is unfair. It is also shown that worst-case jamming in particular translates into worst-case model outcomes, thereby necessitating the need for jamming-resilient SFL protocols.

Smectic liquid crystals can be viewed as model systems for lamellar structures for which there has been extensive theoretical development. We demonstrate that a nonlinear energy description is required with respect to the usual Landau-de Gennes elasticity in order to explain the observed layer spacing of highly curved smectic layers. Using X-ray diffraction we have quantitatively determined the dilation of bent layers distorted by antagonistic anchoring (as high as 1.8% of dilation for the most bent smectic layers) and accurately described it by the minimal nonlinear expression for energy. We observe a 1{\deg} tilt of planar layers that are connected to the curved layers. This value is consistent with simple energetic calculations, demonstrating how the bending energy impacts the overall structure of a thin distorted smectic film. Finally, we show that combined X-ray measurements and theoretical modeling allow for the quantitative determination of the number of curved smectic layers and of the resulting thickness of the dilated region with unprecedented precision., Comment: 6 pages with 6 figures

We prove finite-time vorticity blowup for smooth solutions of the 2D compressible Euler equations with smooth, localized, and non-vacuous initial data. The vorticity blowup occurs at the time of the first singularity, and is accompanied by an axisymmetric implosion in which the swirl velocity enjoys full stability, as opposed to finite co-dimension stability.

Understanding the rules underlying organismal development is a major unsolved problem in biology. Each cell in a developing organism responds to signals in its local environment by dividing, excreting, consuming, or reorganizing, yet how these individual actions coordinate over a macroscopic number of cells to grow complex structures with exquisite functionality is unknown. Here we use recent advances in automatic differentiation to discover local interaction rules and genetic networks that yield emergent, systems-level characteristics in a model of development. We consider a growing tissue with cellular interactions mediated by morphogen diffusion, differential cell adhesion and mechanical stress. Each cell has an internal genetic network that is used to make decisions based on the cell's local environment. We show that one can simultaneously learn parameters governing the cell interactions and the genetic network for complex developmental scenarios, including the symmetry breaking of an embryo from an initial cell by creation of emergent chemical gradients, homogenization of growth via mechanical stress, programmed growth into a pre-specified shape, and the ability to repair from damage. When combined with recent experimental advances measuring spatio-temporal dynamics and gene expression of cells in a growing tissue, the methodology outlined here offers a promising path to unravelling the cellular bases of development., Comment: 8 pages, 6 figures

Language models trained on large amounts of data are known to produce inappropriate content in some cases and require careful tuning to be used in the real world. We revisit the reward augmented decoding (RAD) approach to control the generation from a language model using the scores from a task-specific reward model. We investigate the training objective of RAD, and reformulate it as a task of learning a reward matrix. We show that RAD is designed to support high flexibility when representing the reward matrices, which leads to a higher computational costs during decoding. However, we demonstrate that RAD does not use its full flexibility. Motivated by this, we propose a simpler but more efficient low-rank parametrization of the reward model enabling fast and effective guided decoding. For the detoxification and sentiment control tasks, we show that our low-rank reward model performs on par with the more flexible RAD parametrization, while requiring only a single reward model call per generated token.

ULTRASAT (ULtra-violet TRansient Astronomy SATellite) is a wide-angle space telescope that will perform a deep time-resolved all-sky survey in the near-ultraviolet (NUV) spectrum. The science objectives are the detection of counterparts to short-lived transient astronomical events such as gravitational wave sources and supernovae. The mission is led by the Weizmann Institute of Science and is planned for launch in 2026 in collaboration with the Israeli Space Agency and NASA. DESY will provide the UV camera, composed by the detector assembly located in the telescope focal plane and the remote electronics unit. The camera is composed out of four back-metallized CMOS Image Sensors (CIS) manufactured in the 4T, dual gain Tower process. As part of the radiation qualification of the camera, Single Event Effect (SEE) testing has been performed by irradiating the sensor with heavy ions at the RADEF, Jyvaskyla facility. Preliminary results of both Single Event Upset (SEU) and Single Event Latch-up (SEL) occurrence rate in the sensor are presented. Additionally, an in-orbit SEE rate simulation has been performed in order to gain preliminary knowledge about the expected effect of SEE on the mission.