Search

Radiance field methods represent the state of the art in reconstructing complex scenes from multi-view photos. However, these reconstructions often suffer from one or both of the following limitations: First, they typically represent scenes in low dynamic range (LDR), which restricts their use to evenly lit environments and hinders immersive viewing experiences. Secondly, their reliance on a pinhole camera model, assuming all scene elements are in focus in the input images, presents practical challenges and complicates refocusing during novel-view synthesis. Addressing these limitations, we present a lightweight method based on 3D Gaussian Splatting that utilizes multi-view LDR images of a scene with varying exposure times, apertures, and focus distances as input to reconstruct a high-dynamic-range (HDR) radiance field. By incorporating analytical convolutions of Gaussians based on a thin-lens camera model as well as a tonemapping module, our reconstructions enable the rendering of HDR content with flexible refocusing capabilities. We demonstrate that our combined treatment of HDR and depth of field facilitates real-time cinematic rendering, outperforming the state of the art.

Estimating perceptual attributes of materials directly from images is a challenging task due to their complex, not fully-understood interactions with external factors, such as geometry and lighting. Supervised deep learning models have recently been shown to outperform traditional approaches, but rely on large datasets of human-annotated images for accurate perception predictions. Obtaining reliable annotations is a costly endeavor, aggravated by the limited ability of these models to generalise to different aspects of appearance. In this work, we show how a much smaller set of human annotations ("strong labels") can be effectively augmented with automatically derived "weak labels" in the context of learning a low-dimensional image-computable gloss metric. We evaluate three alternative weak labels for predicting human gloss perception from limited annotated data. Incorporating weak labels enhances our gloss prediction beyond the current state of the art. Moreover, it enables a substantial reduction in human annotation costs without sacrificing accuracy, whether working with rendered images or real photographs., Comment: Computer Graphics Forum (Eurographics 2024)

We analyze how the presence of closed timelike curves (CTCs) characterizing a time machine can be discerned by placing a local particle detector in a region of spacetime which is causally disconnected from the CTCs. Our study shows that not only can the detector tell if there are CTCs, but also that the detector can separate topological from geometrical information and distinguish periodic spacetimes without CTCs (like the Einstein cylinder), curvature, and spacetimes with topological identifications that enable time-machines., Comment: 20 pages, 3 figures; v2: fixed typos and updated to match published version

Entertaining the possibility of time travel will invariably challenge dearly held concepts of fundamental physics. It becomes relatively easy to construct multiple logical contradictions using differing starting points from various well-established fields of physics. Sometimes, the interpretation is that only a full theory of quantum gravity will be able to settle these logical contradictions. Even then, it remains unclear if the multitude of problems could be overcome. Yet as definitive as this seems to the notion of time travel in physics, such a recourse to quantum gravity comes with its own, long-standing challenge to most of these counter-arguments to time travel: These arguments rely on time, while quantum gravity is (in)famously stuck with and dealing with the problem of time. One attempt to answer this problem within the canonical framework resulted in the Page-Wootters formalism, and its recent gauge-theoretic re-interpretation - as an emergent notion of time. Herein, we will begin a programme to study toy models implementing the Hamiltonian constraint in quantum theory, with an aim towards understanding what an emergent notion of time can tell us about the (im)possibility of time travel., Comment: v1: 19 pages, 2 figures; v2: 20 pages, 2 figures, added references, minor text changes, no physics changes. Closely matches published version, but retains authors' preferred style choices

We investigate the primordial power spectrum of cosmological tensor perturbations in the dressed metric approach to Loop Quantum Cosmology. We compute the background-dependent effective mass that affects their propagation using the effective description of Loop Quantum Cosmology and show that this mass can be approximated in different cosmological epochs by appropriate analytic functions. Moreover, in each of those epochs we can analytically solve the propagation of the perturbations, then obtaining the general solution globally by continuity requirements. On the other hand, since there are regimes far away from slow roll in the considered background evolution, the Bunch-Davies state does not provide a privileged choice of vacuum that would pick out a specific solution for the perturbations. Instead, we select the state of these perturbations by a recently proposed criterion that removes unwanted oscillations in the power spectrum. We compute the spectrum of this vacuum and compare it with other spectra obtained in the literature, especially with one corresponding to the hybrid approach to Loop Quantum Cosmology. Finally, we notice that the same type of background dynamics is found in a phenomenological approach to quantum gravity based on thermodynamics, allowing in this case a free value for the tantamount of the critical density. Extending the dressed metric proposal to this phenomenological model, one might expect a similar form for the associated primordial power spectrum., Comment: 19 pages, 8 figures

The thermodynamics of local causal horizons has been shown to imply gravitational dynamics. In this essay, we discuss the principles underlying this observation, and its significance in our understanding of (quantum) gravity. We also show why the local thermodynamic methods cannot by themselves recover general relativity. Instead, they lead to the so-called Weyl transverse gravity. Because of this, local thermodynamic approaches avoid huge vacuum energy contributions to the cosmological constant. They even suggest a possible source for its small observed value. We also outline a way in which thermodynamics allows us to study low energy quantum gravitational effects. We arrive at quantum corrections to the gravitational equations which are suppressed by the Planck length squared., Comment: 10 pages. Essay received Honorable Mention at the Gravity Research Foundation 2023 Awards for Essays on Gravitation. Matches the version accepted in IJMPD

In this work we present a derivation of modified Raychaudhuri and Friedmann equations from a phenomenological model of quantum gravity based on the thermodynamics of spacetime. Starting from general gravitational equations of motion which encode low-energy quantum gravity effects, we found its particular solution for homogenous and isotropic universes with standard matter content, obtaining a modified Raychaudhuri equation. Then, we imposed local energy conservation and used a perturbative treatment to derive a modified Friedmann equation. The modified evolution in the early universe we obtained suggests a replacement of the Big Bang singularity by a regular bounce. Lastly, we also briefly discuss the range of validity of the perturbative approach and its results., Comment: 8 pages, 2 figures, text matches the version published in PLB

In this work, we derive the linearised equations of quadratic gravity from entanglement equilibrium of local causal diamonds. Rather than starting from the Wald entropy prescription (which depends on the gravitational Lagrangian), we employ a model independent approach based on the logarithmic corrections to horizon entanglement entropy. In this way, we are able to show the emergence of linearised quadratic gravity from entanglement equilibrium without using any a priori knowledge about gravitational dynamics. If the logarithmic correction to entropy has a negative sign, as predicted by replica trick calculations of entanglement entropy, we find that the quadratic gravity correction terms have the sign necessary to avoid tachyonic instabilities of the theory., Comment: 8 pages, 1 figure. Matches the version accepted in PRD

Most in-the-wild images are stored in Low Dynamic Range (LDR) form, serving as a partial observation of the High Dynamic Range (HDR) visual world. Despite limited dynamic range, these LDR images are often captured with different exposures, implicitly containing information about the underlying HDR image distribution. Inspired by this intuition, in this work we present, to the best of our knowledge, the first method for learning a generative model of HDR images from in-the-wild LDR image collections in a fully unsupervised manner. The key idea is to train a generative adversarial network (GAN) to generate HDR images which, when projected to LDR under various exposures, are indistinguishable from real LDR images. The projection from HDR to LDR is achieved via a camera model that captures the stochasticity in exposure and camera response function. Experiments show that our method GlowGAN can synthesize photorealistic HDR images in many challenging cases such as landscapes, lightning, or windows, where previous supervised generative models produce overexposed images. We further demonstrate the new application of unsupervised inverse tone mapping (ITM) enabled by GlowGAN. Our ITM method does not need HDR images or paired multi-exposure images for training, yet it reconstructs more plausible information for overexposed regions than state-of-the-art supervised learning models trained on such data.

Gravitational theories invariant under transverse diffeomorphisms and Weyl transformations have the same classical solutions as the corresponding fully diffeomorphism invariant theories. However, they solve some of the problems related to the cosmological constant and in principle allow local energy non-conservation. In the present work, we obtain the Noether charge formalism for these theories. We first derive expressions for the Noether currents and charges corresponding to transverse diffeomorphisms and Weyl transformations, showing that the latter vanish identically. We then use these results to obtain an expression for a perturbation of a Hamiltonian corresponding to evolution along a transverse diffeomorphism generator. From this expression, we derive the first law of black hole mechanics, identifying the total energy, the total angular momentum, the Wald entropy, and the contributions of the cosmological constant perturbations and energy non-conservation. Lastly, we extend our formalism to derive the first law of causal diamonds., Comment: 20 pages

This registered report introduces the largest, and for the first time, reproducible experimental survey on biomedical sentence similarity with the following aims: (1) to elucidate the state of the art of the problem; (2) to solve some reproducibility problems preventing the evaluation of most of current methods; (3) to evaluate several unexplored sentence similarity methods; (4) to evaluate an unexplored benchmark, called Corpus-Transcriptional-Regulation; (5) to carry out a study on the impact of the pre-processing stages and Named Entity Recognition (NER) tools on the performance of the sentence similarity methods; and finally, (6) to bridge the lack of reproducibility resources for methods and experiments in this line of research. Our experimental survey is based on a single software platform that is provided with a detailed reproducibility protocol and dataset as supplementary material to allow the exact replication of all our experiments. In addition, we introduce a new aggregated string-based sentence similarity method, called LiBlock, together with eight variants of current ontology-based methods and a new pre-trained word embedding model trained on the full-text articles in the PMC-BioC corpus. Our experiments show that our novel string-based measure sets the new state of the art on the sentence similarity task in the biomedical domain and significantly outperforms all the methods evaluated herein, except one ontology-based method. Likewise, our experiments confirm that the pre-processing stages, and the choice of the NER tool, have a significant impact on the performance of the sentence similarity methods. We also detail some drawbacks and limitations of current methods, and warn on the need of refining the current benchmarks. Finally, a noticeable finding is that our new string-based method significantly outperforms all state-of-the-art Machine Learning models evaluated herein., Comment: 48 pages, 4 figures, 17 tables, three appendices not included in pre-print version

Weyl transverse gravity is a gravitational theory that is invariant under transverse diffeomorphisms and Weyl transformations. It is characterised by having the same classical solutions as general relativity while solving some of its issues with the cosmological constant. In this work, we first find the Noether currents and charges corresponding to local symmetries of Weyl transverse gravity as well as a prescription for the symplectic form. We then employ these results to derive the first law of black hole mechanics in Weyl transverse gravity (both in vacuum and in the presence of a perfect fluid), identifying the total energy, the total angular momentum, and the Wald entropy of black holes. We further obtain the first law and Smarr formula for Schwarzschild-anti-de Sitter and pure de Sitter spacetimes, discussing the contributions of the varying cosmological constant, which naturally appear in Weyl transverse gravity. Lastly, we derive the first law of causal diamonds in vacuum., Comment: 36 pages

In this review we discuss emergence of unimodular gravity (or, more precisely, Weyl transverse gravity) from thermodynamics of spacetime. By analyzing three different ways to obtain gravitational equations of motion by thermodynamic arguments, we show that the results point to unimodular rather than fully diffeomorphism invariant theories and that this is true even for modified gravity. The unimodular character of dynamics is especially evident from the status of cosmological constant and energy-momentum conservation., Comment: 19 pages, 2 figures. Review prepared for a Special Issue of the International Journal of Geometric Methods in Modern Physics dedicated to the conference Geometric Foundations of Gravity 2021. Version accepted by IJGMMP

High Dynamic Range (HDR) content is becoming ubiquitous due to the rapid development of capture technologies. Nevertheless, the dynamic range of common display devices is still limited, therefore tone mapping (TM) remains a key challenge for image visualization. Recent work has demonstrated that neural networks can achieve remarkable performance in this task when compared to traditional methods, however, the quality of the results of these learning-based methods is limited by the training data. Most existing works use as training set a curated selection of best-performing results from existing traditional tone mapping operators (often guided by a quality metric), therefore, the quality of newly generated results is fundamentally limited by the performance of such operators. This quality might be even further limited by the pool of HDR content that is used for training. In this work we propose a learning-based self-supervised tone mapping operator that is trained at test time specifically for each HDR image and does not need any data labeling. The key novelty of our approach is a carefully designed loss function built upon fundamental knowledge on contrast perception that allows for directly comparing the content in the HDR and tone mapped images. We achieve this goal by reformulating classic VGG feature maps into feature contrast maps that normalize local feature differences by their average magnitude in a local neighborhood, allowing our loss to account for contrast masking effects. We perform extensive ablation studies and exploration of parameters and demonstrate that our solution outperforms existing approaches with a single set of fixed parameters, as confirmed by both objective and subjective metrics.

We study the relationship between the quantization of a massless scalar field on the two-dimensional Einstein cylinder and in a spacetime with a time machine. We find that the latter picks out a unique prescription for the state of the zero mode in the Einstein cylinder. We show how this choice arises from the computation of the vacuum Wightman function and the vacuum renormalized stress-energy tensor in the time-machine geometry. Finally, we relate the previously proposed regularization of the zero mode state as a squeezed state with the time-machine warp parameter, thus demonstrating that the quantization in the latter regularizes the quantization in an Einstein cylinder., Comment: 17 pages, 2 Appendices, RevTex4-2; v2: accepted to JHEP, updated to match the upcoming published version. Added one figure for the conformal diagram

Motivated by exploring the interface between thermodynamics of spacetime and quantum gravity effects, we develop a heuristic derivation of Hawking temperature and Bekenstein entropy from the existence of a minimal resolvable area. Moreover, we find leading order quantum gravity corrections to them that are in qualitative agreement with results obtained by other methods, both heuristic and rigorous. In this way, we recover, as a particular case, the corrections heuristically obtained from the existence of minimal length. We also show that the size of minimal area is constrained from above by well understood results of semiclassical black hole physics, specifically by the entropy content of Hawking radiation. The minimal area derivation we introduce is also applied to finding the Unruh temperature associated with causal diamonds and to establish a new relation between this temperature and the entropy of the causal diamond's horizon., Comment: 11 pages, 3 figures. Version accepted to Physical Review D

The Belinski-Khalatnikov-Lifshitz (BKL) conjecture predicts a chaotic alternation of Kasner epochs in the evolution of generic classical spacetimes towards a spacelike singularity. As a first step towards understanding the full quantum BKL scenario, we analyze a vacuum Bianchi II model with local rotational symmetry, which presents just one Kasner transition. During the Kasner epochs, the quantum state is coherent and it is thus characterized by constant values of the different quantum fluctuations, correlations and higher-order moments. By computing the constants of motion of the system we provide, for any peaked semiclassical state, the explicit analytical transition rules that relate the parametrization of the asymptotic coherent state before and after the transition. In particular, we obtain the modification of the transition rules for the classical variables due to quantum back-reaction effects. This analysis is performed by considering a high-order truncation in moments (the full computations are performed up to fifth-order, which corresponds to neglecting terms of an order $\hbar^3$), providing a solid estimate about the quantum modifications to the classical model. Finally, in order to understand the dynamics of the state during the transition, we perform some numerical simulations for an initial Gaussian state, that show that the initial and final equilibrium values of the quantum variables are connected by strong and rapid oscillations., Comment: Version accepted in PRD. Minor typos corrected

Understanding and modeling the dynamics of human gaze behavior in 360$^\circ$ environments is a key challenge in computer vision and virtual reality. Generative adversarial approaches could alleviate this challenge by generating a large number of possible scanpaths for unseen images. Existing methods for scanpath generation, however, do not adequately predict realistic scanpaths for 360$^\circ$ images. We present ScanGAN360, a new generative adversarial approach to address this challenging problem. Our network generator is tailored to the specifics of 360$^\circ$ images representing immersive environments. Specifically, we accomplish this by leveraging the use of a spherical adaptation of dynamic-time warping as a loss function and proposing a novel parameterization of 360$^\circ$ scanpaths. The quality of our scanpaths outperforms competing approaches by a large margin and is almost on par with the human baseline. ScanGAN360 thus allows fast simulation of large numbers of virtual observers, whose behavior mimics real users, enabling a better understanding of gaze behavior and novel applications in virtual scene design.

Virtual reality (VR) is rapidly growing, with the potential to change the way we create and consume content. In VR, users integrate multimodal sensory information they receive, to create a unified perception of the virtual world. In this survey, we review the body of work addressing multimodality in VR, and its role and benefits in user experience, together with different applications that leverage multimodality in many disciplines. These works thus encompass several fields of research, and demonstrate that multimodality plays a fundamental role in VR; enhancing the experience, improving overall performance, and yielding unprecedented abilities in skill and knowledge transfer., Comment: 35 pages (24 pages not including references), 10 figures, 4 tables

Observing and recognizing materials is a fundamental part of our daily life. Under typical viewing conditions, we are capable of effortlessly identifying the objects that surround us and recognizing the materials they are made of. Nevertheless, understanding the underlying perceptual processes that take place to accurately discern the visual properties of an object is a long-standing problem. In this work, we perform a comprehensive and systematic analysis of how the interplay of geometry, illumination, and their spatial frequencies affects human performance on material recognition tasks. We carry out large-scale behavioral experiments where participants are asked to recognize different reference materials among a pool of candidate samples. In the different experiments, we carefully sample the information in the frequency domain of the stimuli. From our analysis, we find significant first-order interactions between the geometry and the illumination, of both the reference and the candidates. In addition, we observe that simple image statistics and higher-order image histograms do not correlate with human performance. Therefore, we perform a high-level comparison of highly non-linear statistics by training a deep neural network on material recognition tasks. Our results show that such models can accurately classify materials, which suggests that they are capable of defining a meaningful representation of material appearance from labeled proximal image data. Last, we find preliminary evidence that these highly non-linear models and humans may use similar high-level factors for material recognition tasks., Comment: 15 pages, 16 figures, Accepted to the Journal of Vision, 2021