Search

We tackle the recently introduced benchmark for whole-body humanoid control HumanoidBench using MuJoCo MPC. We find that sparse reward functions of HumanoidBench yield undesirable and unrealistic behaviors when optimized; therefore, we propose a set of regularization terms that stabilize the robot behavior across tasks. Current evaluations on a subset of tasks demonstrate that our proposed reward function allows achieving the highest HumanoidBench scores while maintaining realistic posture and smooth control signals. Our code is publicly available and will become a part of MuJoCo MPC, enabling rapid prototyping of robot behaviors., Comment: 3 pages, 3 figures, submitted to IEEE Conference on Robotics and Automation (ICRA@40)

For classical knots, there is a concept of (semi)meander diagrams; in this short note we generalize this concept to virtual knots and prove that the classes of meander and semimeander diagrams are universal (this was known for classical knots). We also introduce a new class of invariants for virtual knots -- virtual $k$-arc crossing numbers., Comment: 6 pages, 2 figures

We consider noncompact open $SL(2, \mathbb{C})$ spin chain and construct eigenfunctions of $B$-element of monodromy matrix for the simplest case of the chain with one site. The reflection operator appearing in this construction can be used to express eigenfunction for $n$ sites in terms of the eigenfunction for $n-1$ sites, this general result is briefly announced. We prove orthogonality and completeness of constructed eigenfunctions in the case of one site, express them in terms of the hypergeometric function of the complex field and derive the equation for the reflection operator with the general $SL(2,\mathbb{C})$-invariant $\mathbb{R}$-operator.

Deep Reinforcement Learning (RL) is well known for being highly sensitive to hyperparameters, requiring practitioners substantial efforts to optimize them for the problem at hand. In recent years, the field of automated Reinforcement Learning (AutoRL) has grown in popularity by trying to address this issue. However, these approaches typically hinge on additional samples to select well-performing hyperparameters, hindering sample-efficiency and practicality in RL. Furthermore, most AutoRL methods are heavily based on already existing AutoML methods, which were originally developed neglecting the additional challenges inherent to RL due to its non-stationarities. In this work, we propose a new approach for AutoRL, called Adaptive $Q$-Network (AdaQN), that is tailored to RL to take into account the non-stationarity of the optimization procedure without requiring additional samples. AdaQN learns several $Q$-functions, each one trained with different hyperparameters, which are updated online using the $Q$-function with the smallest approximation error as a shared target. Our selection scheme simultaneously handles different hyperparameters while coping with the non-stationarity induced by the RL optimization procedure and being orthogonal to any critic-based RL algorithm. We demonstrate that AdaQN is theoretically sound and empirically validate it in MuJoCo control problems, showing benefits in sample-efficiency, overall performance, training stability, and robustness to stochasticity., Comment: Preprint

This paper explores active sensing strategies that employ vision-based tactile sensors for robotic perception and classification of fabric textures. We formalize the active sampling problem in the context of tactile fabric recognition and provide an implementation of information-theoretic exploration strategies based on minimizing predictive entropy and variance of probabilistic models. Through ablation studies and human experiments, we investigate which components are crucial for quick and reliable texture recognition. Along with the active sampling strategies, we evaluate neural network architectures, representations of uncertainty, influence of data augmentation, and dataset variability. By evaluating our method on a previously published Active Clothing Perception Dataset and on a real robotic system, we establish that the choice of the active exploration strategy has only a minor influence on the recognition accuracy, whereas data augmentation and dropout rate play a significantly larger role. In a comparison study, while humans achieve 66.9% recognition accuracy, our best approach reaches 90.0% in under 5 touches, highlighting that vision-based tactile sensors are highly effective for fabric texture recognition., Comment: 7 pages, 9 figures, accepted at 2024 IEEE International Conference on Robotics and Automation (ICRA)

We prove that all general smooth Fano threefolds of Picard rank $3$ and degree $14$ are K-stable, where the generality condition is stated explicitly., Comment: arXiv admin note: text overlap with arXiv:2206.12208

The vast majority of Reinforcement Learning methods is largely impacted by the computation effort and data requirements needed to obtain effective estimates of action-value functions, which in turn determine the quality of the overall performance and the sample-efficiency of the learning procedure. Typically, action-value functions are estimated through an iterative scheme that alternates the application of an empirical approximation of the Bellman operator and a subsequent projection step onto a considered function space. It has been observed that this scheme can be potentially generalized to carry out multiple iterations of the Bellman operator at once, benefiting the underlying learning algorithm. However, till now, it has been challenging to effectively implement this idea, especially in high-dimensional problems. In this paper, we introduce iterated $Q$-Network (iQN), a novel principled approach that enables multiple consecutive Bellman updates by learning a tailored sequence of action-value functions where each serves as the target for the next. We show that iQN is theoretically grounded and that it can be seamlessly used in value-based and actor-critic methods. We empirically demonstrate the advantages of iQN in Atari $2600$ games and MuJoCo continuous control problems., Comment: Preprint

In the rapidly evolving domain of Recommender Systems (RecSys), new algorithms frequently claim state-of-the-art performance based on evaluations over a limited set of arbitrarily selected datasets. However, this approach may fail to holistically reflect their effectiveness due to the significant impact of dataset characteristics on algorithm performance. Addressing this deficiency, this paper introduces a novel benchmarking methodology to facilitate a fair and robust comparison of RecSys algorithms, thereby advancing evaluation practices. By utilizing a diverse set of $30$ open datasets, including two introduced in this work, and evaluating $11$ collaborative filtering algorithms across $9$ metrics, we critically examine the influence of dataset characteristics on algorithm performance. We further investigate the feasibility of aggregating outcomes from multiple datasets into a unified ranking. Through rigorous experimental analysis, we validate the reliability of our methodology under the variability of datasets, offering a benchmarking strategy that balances quality and computational demands. This methodology enables a fair yet effective means of evaluating RecSys algorithms, providing valuable guidance for future research endeavors., Comment: 8 pages with 11 figures

Living materials adapt their shape to signals from the environment, yet the impact of shape changes on signal processing and the associated feedback dynamics remain unclear. We derive coarse-grained equations for droplets that adjust their interfacial tension in response to signals exchanged at contact surfaces, from the microscopic biophysics of adhesion and signaling. We find that droplet pairs exhibit symmetry-breaking, excitability, and oscillations. The underlying critical points reveal novel mechanisms for physical signal processing through shape adaptation in soft active materials.

The article identified 42 cognitive architectures for creating general artificial intelligence (AGI) and proposed a set of interrelated functional blocks that an agent approaching AGI in its capabilities should possess. Since the required set of blocks is not found in any of the existing architectures, the article proposes a new cognitive architecture for intelligent systems approaching AGI in their capabilities. As one of the key solutions within the framework of the architecture, a universal method of knowledge representation is proposed, which allows combining various non-formalized, partially and fully formalized methods of knowledge representation in a single knowledge base, such as texts in natural languages, images, audio and video recordings, graphs, algorithms, databases, neural networks, knowledge graphs, ontologies, frames, essence-property-relation models, production systems, predicate calculus models, conceptual models, and others. To combine and structure various fragments of knowledge, archigraph models are used, constructed as a development of annotated metagraphs. As components, the cognitive architecture being developed includes machine consciousness, machine subconsciousness, blocks of interaction with the external environment, a goal management block, an emotional control system, a block of social interaction, a block of reflection, an ethics block and a worldview block, a learning block, a monitoring block, blocks of statement and solving problems, self-organization and meta learning block.

Comet nuclei in the outer Solar system are constantly irradiated by cosmic rays at low temperatures. Accumulated high concentrations of radicals can undergo fast recombination with significant heating of cometary surface layers. We present the model of comet activity at large heliocentric distances caused by the recombination of radicals. We found that the considered mechanism can cause activity of comets in distant regions of the Solar system, even at the Oort cloud distances. Outbursts in distant comet reservoirs can be a new source of dust and ice particles contributing to the recently discovered anomalous diffuse light in the cosmic extragalactic background optic light. The orbits of small-radii comets in the Oort cloud are highly influenced by cometary outbursts. This effect may account for the observed decrease in the number of small-radius long-period comets., Comment: 16 pages, 8 figures

Approximate value iteration (AVI) is a family of algorithms for reinforcement learning (RL) that aims to obtain an approximation of the optimal value function. Generally, AVI algorithms implement an iterated procedure where each step consists of (i) an application of the Bellman operator and (ii) a projection step into a considered function space. Notoriously, the Bellman operator leverages transition samples, which strongly determine its behavior, as uninformative samples can result in negligible updates or long detours, whose detrimental effects are further exacerbated by the computationally intensive projection step. To address these issues, we propose a novel alternative approach based on learning an approximate version of the Bellman operator rather than estimating it through samples as in AVI approaches. This way, we are able to (i) generalize across transition samples and (ii) avoid the computationally intensive projection step. For this reason, we call our novel operator projected Bellman operator (PBO). We formulate an optimization problem to learn PBO for generic sequential decision-making problems, and we theoretically analyze its properties in two representative classes of RL problems. Furthermore, we theoretically study our approach under the lens of AVI and devise algorithmic implementations to learn PBO in offline and online settings by leveraging neural network parameterizations. Finally, we empirically showcase the benefits of PBO w.r.t. the regular Bellman operator on several RL problems., Comment: Proceedings of the National Conference on Artificial Intelligence (AAAI-24)

It is known that each knot has a semimeander diagram (i. e. a diagram composed of two smooth simple arcs), however the number of crossings in such a diagram can only be roughly estimated. In the present paper we provide a new estimate of the complexity of the semimeander diagrams. We prove that for each knot $K$ with more than 10 crossings, there exists a semimeander diagram with no more than $0.31 \cdot 1.558^{\operatorname{cr}(K)}$ crossings, where $\operatorname{cr}(K)$ is the crossing number of $K$. As a corollary, we provide new estimates of the complexity of other meander-like types of knot diagrams, such as meander diagrams and potholders. We also describe an efficient algorithm for constructing a semimeander diagram from a given one., Comment: 11 pages, 12 figures

We present a novel information-theoretic framework, termed as TURBO, designed to systematically analyse and generalise auto-encoding methods. We start by examining the principles of information bottleneck and bottleneck-based networks in the auto-encoding setting and identifying their inherent limitations, which become more prominent for data with multiple relevant, physics-related representations. The TURBO framework is then introduced, providing a comprehensive derivation of its core concept consisting of the maximisation of mutual information between various data representations expressed in two directions reflecting the information flows. We illustrate that numerous prevalent neural network models are encompassed within this framework. The paper underscores the insufficiency of the information bottleneck concept in elucidating all such models, thereby establishing TURBO as a preferable theoretical reference. The introduction of TURBO contributes to a richer understanding of data representation and the structure of neural network models, enabling more efficient and versatile applications.

Copy detection patterns (CDP) present an efficient technique for product protection against counterfeiting. However, the complexity of studying CDP production variability often results in time-consuming and costly procedures, limiting CDP scalability. Recent advancements in computer modelling, notably the concept of a "digital twin" for printing-imaging channels, allow for enhanced scalability and the optimization of authentication systems. Yet, the development of an accurate digital twin is far from trivial. This paper extends previous research which modelled a printing-imaging channel using a machine learning-based digital twin for CDP. This model, built upon an information-theoretic framework known as "Turbo", demonstrated superior performance over traditional generative models such as CycleGAN and pix2pix. However, the emerging field of Denoising Diffusion Probabilistic Models (DDPM) presents a potential advancement in generative models due to its ability to stochastically model the inherent randomness of the printing-imaging process, and its impressive performance in image-to-image translation tasks. This study aims at comparing the capabilities of the Turbo framework and DDPM on the same CDP datasets, with the goal of establishing the real-world benefits of DDPM models for digital twin applications in CDP security. Furthermore, the paper seeks to evaluate the generative potential of the studied models in the context of mobile phone data acquisition. Despite the increased complexity of DDPM methods when compared to traditional approaches, our study highlights their advantages and explores their potential for future applications., Comment: Paper accepted at the IEEE International Workshop on Information Forensics and Security (WIFS) 2023

In these notes we review the technique of Baxter Q-operators in the Ruijsenaars-Sutherland hyperbolic systems in the cases of one and two particles. Using these operators we show in particular that eigenfunctions of these systems admit two dual integral representations and prove their orthogonality and completeness.

We introduce and study a new family of commuting Baxter operators in the Ruijsenaars hyperbolic system, different from that considered by us earlier. Using a degeneration of Rains integral identity we verify the commutativity between the two families of Baxter operators and explore this fact for the proof of the coupling constant symmetry of the wave function. We also establish a connection between new Baxter operators and Noumi-Sano difference operators.

In the previous paper we showed that the wave functions of the quantum Ruijsenaars hyperbolic system diagonalize Baxter Q-operators. Using this property and duality relation we prove orthogonality and completeness relations for the wave functions or, equivalently, unitarity of the corresponding integral transform.

Cellular Potts models are broadly applied across developmental biology and cancer research. We overcome limitations of the traditional approach, which reinterprets a modified Metropolis sampling as ad hoc dynamics, by introducing a physical timescale through Poissonian kinetics and by applying principles of stochastic thermodynamics to separate thermal and relaxation effects from athermal noise and nonconservative forces. Our method accurately describes cell-sorting dynamics in mouse-embryo development and identifies the distinct contributions of nonequilibrium processes, e.g. cell growth and active fluctuations., Comment: 7 Figures