Search

We introduce StableMaterials, a novel approach for generating photorealistic physical-based rendering (PBR) materials that integrate semi-supervised learning with Latent Diffusion Models (LDMs). Our method employs adversarial training to distill knowledge from existing large-scale image generation models, minimizing the reliance on annotated data and enhancing the diversity in generation. This distillation approach aligns the distribution of the generated materials with that of image textures from an SDXL model, enabling the generation of novel materials that are not present in the initial training dataset. Furthermore, we employ a diffusion-based refiner model to improve the visual quality of the samples and achieve high-resolution generation. Finally, we distill a latent consistency model for fast generation in just four steps and propose a new tileability technique that removes visual artifacts typically associated with fewer diffusion steps. We detail the architecture and training process of StableMaterials, the integration of semi-supervised training within existing LDM frameworks and show the advantages of our approach. Comparative evaluations with state-of-the-art methods show the effectiveness of StableMaterials, highlighting its potential applications in computer graphics and beyond. StableMaterials is publicly available at https://gvecchio.com/stablematerials.

We study the formation and the subsequent dynamics of shock waves in repulsive one-dimensional Bose gases during the free expansion of a density hump. By building coherent Fermi states for interacting Bethe fermions, we define a quantum fluctuating initial state expressed in terms of universal quantities, namely the density and the Luttinger parameter. In the integrable case, this fluctuating state is then evolved by generalized hydrodynamics (GHD) and, differently from non-fluctuating initial states, it develops density ripples on top of the hydrodynamic mean value. Our analysis gives a general theory of quantum ripples and wave breaking in integrable and quasi-integrable one-dimensional liquids and clarifies the role of the interaction strength. In particular, for strongly/intermediately interacting bosons, we find quantum ripples originating from low-energy modes at the Fermi surface interfering when transported by GHD. In the low coupling limit, near the quasicondensate regime, we find instead that density ripples have a semi-classical nature, and their description requires information on the curvature of the Fermi surface., Comment: 19 pages, 7 figures

We introduce MatSynth, a dataset of 4,000+ CC0 ultra-high resolution PBR materials. Materials are crucial components of virtual relightable assets, defining the interaction of light at the surface of geometries. Given their importance, significant research effort was dedicated to their representation, creation and acquisition. However, in the past 6 years, most research in material acquisiton or generation relied either on the same unique dataset, or on company-owned huge library of procedural materials. With this dataset we propose a significantly larger, more diverse, and higher resolution set of materials than previously publicly available. We carefully discuss the data collection process and demonstrate the benefits of this dataset on material acquisition and generation applications. The complete data further contains metadata with each material's origin, license, category, tags, creation method and, when available, descriptions and physical size, as well as 3M+ renderings of the augmented materials, in 1K, under various environment lightings. The MatSynth dataset is released through the project page at: https://www.gvecchio.com/matsynth.

Material reconstruction from a photograph is a key component of 3D content creation democratization. We propose to formulate this ill-posed problem as a controlled synthesis one, leveraging the recent progress in generative deep networks. We present ControlMat, a method which, given a single photograph with uncontrolled illumination as input, conditions a diffusion model to generate plausible, tileable, high-resolution physically-based digital materials. We carefully analyze the behavior of diffusion models for multi-channel outputs, adapt the sampling process to fuse multi-scale information and introduce rolled diffusion to enable both tileability and patched diffusion for high-resolution outputs. Our generative approach further permits exploration of a variety of materials which could correspond to the input image, mitigating the unknown lighting conditions. We show that our approach outperforms recent inference and latent-space-optimization methods, and carefully validate our diffusion process design choices. Supplemental materials and additional details are available at: https://gvecchio.com/controlmat/.

Creating high-quality materials in computer graphics is a challenging and time-consuming task, which requires great expertise. To simplify this process, we introduce MatFuse, a unified approach that harnesses the generative power of diffusion models for creation and editing of 3D materials. Our method integrates multiple sources of conditioning, including color palettes, sketches, text, and pictures, enhancing creative possibilities and granting fine-grained control over material synthesis. Additionally, MatFuse enables map-level material editing capabilities through latent manipulation by means of a multi-encoder compression model which learns a disentangled latent representation for each map. We demonstrate the effectiveness of MatFuse under multiple conditioning settings and explore the potential of material editing. Finally, we assess the quality of the generated materials both quantitatively in terms of CLIP-IQA and FID scores and qualitatively by conducting a user study. Source code for training MatFuse and supplemental materials are publicly available at https://gvecchio.com/matfuse.

Polygonal meshes have become the standard for discretely approximating 3D shapes, thanks to their efficiency and high flexibility in capturing non-uniform shapes. This non-uniformity, however, leads to irregularity in the mesh structure, making tasks like segmentation of 3D meshes particularly challenging. Semantic segmentation of 3D mesh has been typically addressed through CNN-based approaches, leading to good accuracy. Recently, transformers have gained enough momentum both in NLP and computer vision fields, achieving performance at least on par with CNN models, supporting the long-sought architecture universalism. Following this trend, we propose a transformer-based method for semantic segmentation of 3D mesh motivated by a better modeling of the graph structure of meshes, by means of global attention mechanisms. In order to address the limitations of standard transformer architectures in modeling relative positions of non-sequential data, as in the case of 3D meshes, as well as in capturing the local context, we perform positional encoding by means the Laplacian eigenvectors of the adjacency matrix, replacing the traditional sinusoidal positional encodings, and by introducing clustering-based features into the self-attention and cross-attention operators. Experimental results, carried out on three sets of the Shape COSEG Dataset, on the human segmentation dataset proposed in Maron et al., 2017 and on the ShapeNet benchmark, show how the proposed approach yields state-of-the-art performance on semantic segmentation of 3D meshes.

We present the first exact theory and analytical formulas for the large-scale phase fluctuations in the sine-Gordon model, valid in all regimes of the field theory, for arbitrary temperatures and interaction strengths. Our result is based on the Ballistic Fluctuation Theory combined with Generalized Hydrodynamics, and can be seen as an exact ``dressing" of the phenomenological soliton-gas picture first introduced by Sachdev and Young [S. Sachdev and A. P. Young, PRL 78, 2220 (1997)], to the modes of Generalised Hydrodynamics. The resulting physics of phase fluctuations in the sine-Gordon model is qualitatively different, as the stable quasi-particles of integrability give coherent ballistic propagation instead of diffusive spreading. We provide extensive numerical checks of our analytical predictions within the classical regime of the field theory by using Monte Carlo methods. We discuss how our results are of ready applicability to experiments on tunnel-coupled quasicondensates., Comment: 8+14 pages; 4+2 Figures

The large-scale behaviour of entanglement entropy in finite-density states, in and out of equilibrium, can be understood using the physical picture of particle pairs. However, the full theoretical origin of this picture is not fully established yet. In this work, we clarify this picture by investigating entanglement entropy using its connection with the large-deviation theory for thermodynamic and hydrodynamic fluctuations. We apply the universal framework of Ballistic Fluctuation Theory (BFT), based the Euler hydrodynamics of the model, to correlation functions of \emph{branch-point twist fields}, the starting point for computing R\'enyi entanglement entropies within the replica approach. Focusing on free fermionic systems in order to illustrate the ideas, we show that both the equilibrium behavior and the dynamics of R\'enyi entanglement entropies can be fully derived from the BFT. In particular, we emphasise that long-range correlations develop after quantum quenches, and accounting for these explain the structure of the entanglement growth. We further show that this growth is related to fluctuations of charge transport, generalising to quantum quenches the relation between charge fluctuations and entanglement observed earlier. The general ideas we introduce suggest that the large-scale behaviour of entanglement has its origin within hydrodynamic fluctuations., Comment: 53 pages, 2 figures

The development of autonomous agents which can interact with other agents to accomplish a given task is a core area of research in artificial intelligence and machine learning. Towards this goal, the Autonomous Agents Research Group develops novel machine learning algorithms for autonomous systems control, with a specific focus on deep reinforcement learning and multi-agent reinforcement learning. Research problems include scalable learning of coordinated agent policies and inter-agent communication; reasoning about the behaviours, goals, and composition of other agents from limited observations; and sample-efficient learning based on intrinsic motivation, curriculum learning, causal inference, and representation learning. This article provides a broad overview of the ongoing research portfolio of the group and discusses open problems for future directions., Comment: Published in AI Communications Special Issue on Multi-Agent Systems Research in the UK

We present MIDGARD, an open-source simulation platform for autonomous robot navigation in outdoor unstructured environments. MIDGARD is designed to enable the training of autonomous agents (e.g., unmanned ground vehicles) in photorealistic 3D environments, and to support the generalization skills of learning-based agents through the variability in training scenarios. MIDGARD's main features include a configurable, extensible, and difficulty-driven procedural landscape generation pipeline, with fast and photorealistic scene rendering based on Unreal Engine. Additionally, MIDGARD has built-in support for OpenAI Gym, a programming interface for feature extension (e.g., integrating new types of sensors, customizing exposing internal simulation variables), and a variety of simulated agent sensors (e.g., RGB, depth and instance/semantic segmentation). We evaluate MIDGARD's capabilities as a benchmarking tool for robot navigation utilizing a set of state-of-the-art reinforcement learning algorithms. The results demonstrate MIDGARD's suitability as a simulation and training environment, as well as the effectiveness of our procedural generation approach in controlling scene difficulty, which directly reflects on accuracy metrics. MIDGARD build, source code and documentation are available at https://midgardsim.org/.

By the hydrodynamic linear response theory, dynamical correlation functions decay as power laws along certain velocities, determined by the flux Jacobian. Such correlations are obtained by hydrodynamic projections, and physically, they are due to propagating "sound waves" or generalisation thereof, transporting conserved quantities between the observables. However, some observables do not emit sound waves, such as order parameters associated to symmetry breaking. In these cases correlation functions decay exponentially everywhere, a behaviour not captured by the hydrodynamic linear response theory. Focussing on spin-spin correlation functions in the XX quantum chain, we first review how hydrodynamic linear response works, emphasising that the necessary fluid cell averaging washes out oscillatory effects. We then show how, beyond linear response, Euler hydrodynamics can still predict the exponential decay of correlation functions of order parameters. This is done by accounting for the large-scale fluctuations of domain walls, via the recently developed ballistic fluctuation theory. We use the framework of generalised hydrodynamics, which is particularly simple in this model due to its free fermion description. In particular, this reproduces, by elementary calculations, the exponential decay in the celebrated formulae by A.R. Its, A.G. Izergin, V.E. Korepin, N.A. Slavnov (1993) and by X. Jie (1998), which were originally obtained by intricate Fredholm determinant analysis; and gives a new formula in a parameter domain where no result was obtained before. We confirm the results by numerical simulations., Comment: v1: 40 pages, 5 figures. v2: 43 pages, one appendix added, typos corrected, references added. v3: 44 pages, moved appendix A in main text, typos corrected, few comments removed/added for clarification, references added

In this paper we present SurfaceNet, an approach for estimating spatially-varying bidirectional reflectance distribution function (SVBRDF) material properties from a single image. We pose the problem as an image translation task and propose a novel patch-based generative adversarial network (GAN) that is able to produce high-quality, high-resolution surface reflectance maps. The employment of the GAN paradigm has a twofold objective: 1) allowing the model to recover finer details than standard translation models; 2) reducing the domain shift between synthetic and real data distributions in an unsupervised way. An extensive evaluation, carried out on a public benchmark of synthetic and real images under different illumination conditions, shows that SurfaceNet largely outperforms existing SVBRDF reconstruction methods, both quantitatively and qualitatively. Furthermore, SurfaceNet exhibits a remarkable ability in generating high-quality maps from real samples without any supervision at training time.

We consider 1D integrable systems supporting ballistic propagation of excitations, perturbed by a localised defect that breaks most conservation laws and induces chaotic dynamics. Focusing on classical systems, we study an out-of-equilibrium protocol engineered activating the defect in an initially homogeneous and far from the equilibrium state. We find that large enough defects induce full thermalisation at their center, but nonetheless the outgoing flow of carriers emerging from the defect is non-thermal due to a generalization of the celebrated Boundary Thermal Resistance effect, occurring at the edges of the chaotic region. Our results are obtained combining ab-initio numerical simulations for relatively small-sized defects, with the solution of the Boltzmann equation, which becomes exact in the scaling limit of large, but weak defects., Comment: 21 pages, 7 figures

We study the out-of-equilibrium properties of a classical integrable non-relativistic theory, with a time evolution initially prepared with a finite energy density in the thermodynamic limit. The theory considered here is the Non-Linear Schrodinger equation which describes the dynamics of the one-dimensional interacting Bose gas in the regime of high occupation numbers. The main emphasis is on the determination of the late-time Generalised Gibbs Ensemble (GGE), which can be efficiently semi-numerically computed on arbitrary initial states, completely solving the famous quench problem in the classical regime. We take advantage of known results in the quantum model and the semiclassical limit to achieve new exact results for the momenta of the density operator on arbitrary GGEs, which we successfully compare with ab-initio numerical simulations. Furthermore, we determine the whole probability distribution of the density operator (full counting statistics), whose exact expression is still out of reach in the quantum model., Comment: 42 pages, 9 figures, resubmission to SciPost Physics