Search

Images as an artistic medium often rely on specific camera angles and lens distortions to convey ideas or emotions; however, such precise control is missing in current text-to-image models. We propose an efficient and general solution that allows precise control over the camera when generating both photographic and artistic images. Unlike prior methods that rely on predefined shots, we rely solely on four simple extrinsic and intrinsic camera parameters, removing the need for pre-existing geometry, reference 3D objects, and multi-view data. We also present a novel dataset with more than 57,000 images, along with their text prompts and ground-truth camera parameters. Our evaluation shows precise camera control in text-to-image generation, surpassing traditional prompt engineering approaches. Our data, model, and code are publicly available at https://graphics.unizar.es/projects/PreciseCam2024.

This paper aims to provide a comparison between texts produced by French and Italian politicians on polarizing issues, such as immigration and the European Union, and their chatbot counterparts created with ChatGPT 3.5. In this study, we focus on implicit communication, in particular on presuppositions and their functions in discourse, which have been considered in the literature as a potential linguistic feature of manipulation. This study also aims to contribute to the emerging literature on the pragmatic competences of Large Language Models., Comment: Published: 2024-07-04

We present an analysis method to search for exotic low-mass field (ELF) bursts generated during large energy astrophysical events such as supernovae, binary black hole or binary neutron star mergers, and fast radio bursts using the Global Network of Optical Magnetometers for Exotic physics searches (GNOME). In our model, the associated gravitational waves or electromagnetic signals herald the arrival of the ELF burst that interacts via coupling to the spin of fermions in the magnetometers. This enables GNOME to serve as a tool for multi-messenger astronomy. The algorithm employs a model-agnostic excess-power method to identify network-wide candidate events to be subjected to a model-dependent generalized likelihood-ratio test to determine their statistical significance. We perform the first search with this technique on GNOME data coincident with the binary black hole merger S200311bg detected by LIGO/Virgo on the 11th of March 2020 and find no significant events. We place the first lab-based limits on combinations of ELF production and coupling parameters.

Generative models have enabled intuitive image creation and manipulation using natural language. In particular, diffusion models have recently shown remarkable results for natural image editing. In this work, we propose to apply diffusion techniques to edit textures, a specific class of images that are an essential part of 3D content creation pipelines. We analyze existing editing methods and show that they are not directly applicable to textures, since their common underlying approach, manipulating attention maps, is unsuitable for the texture domain. To address this, we propose a novel approach that instead manipulates CLIP image embeddings to condition the diffusion generation. We define editing directions using simple text prompts (e.g., "aged wood" to "new wood") and map these to CLIP image embedding space using a texture prior, with a sampling-based approach that gives us identity-preserving directions in CLIP space. To further improve identity preservation, we project these directions to a CLIP subspace that minimizes identity variations resulting from entangled texture attributes. Our editing pipeline facilitates the creation of arbitrary sliders using natural language prompts only, with no ground-truth annotated data necessary., Comment: SIGGRAPH 2024 Conference Proceedings

Estimating perceptual attributes of materials directly from images is a challenging task due to their complex, not fully-understood interactions with external factors, such as geometry and lighting. Supervised deep learning models have recently been shown to outperform traditional approaches, but rely on large datasets of human-annotated images for accurate perception predictions. Obtaining reliable annotations is a costly endeavor, aggravated by the limited ability of these models to generalise to different aspects of appearance. In this work, we show how a much smaller set of human annotations ("strong labels") can be effectively augmented with automatically derived "weak labels" in the context of learning a low-dimensional image-computable gloss metric. We evaluate three alternative weak labels for predicting human gloss perception from limited annotated data. Incorporating weak labels enhances our gloss prediction beyond the current state of the art. Moreover, it enables a substantial reduction in human annotation costs without sacrificing accuracy, whether working with rendered images or real photographs., Comment: Computer Graphics Forum (Eurographics 2024)

We introduce text2fabric, a novel dataset that links free-text descriptions to various fabric materials. The dataset comprises 15,000 natural language descriptions associated to 3,000 corresponding images of fabric materials. Traditionally, material descriptions come in the form of tags/keywords, which limits their expressivity, induces pre-existing knowledge of the appropriate vocabulary, and ultimately leads to a chopped description system. Therefore, we study the use of free-text as a more appropriate way to describe material appearance, taking the use case of fabrics as a common item that non-experts may often deal with. Based on the analysis of the dataset, we identify a compact lexicon, set of attributes and key structure that emerge from the descriptions. This allows us to accurately understand how people describe fabrics and draw directions for generalization to other types of materials. We also show that our dataset enables specializing large vision-language models such as CLIP, creating a meaningful latent space for fabric appearance, and significantly improving applications such as fine-grained material retrieval and automatic captioning.

Numerous observations suggest that there exist undiscovered beyond-the-Standard-Model particles and fields. Because of their unknown nature, these exotic particles and fields could interact with Standard Model particles in many different ways and assume a variety of possible configurations. Here we present an overview of the Global Network of Optical Magnetometers for Exotic physics searches (GNOME), our ongoing experimental program designed to test a wide range of exotic physics scenarios. The GNOME experiment utilizes a worldwide network of shielded atomic magnetometers (and, more recently, comagnetometers) to search for spatially and temporally correlated signals due to torques on atomic spins from exotic fields of astrophysical origin. We survey the temporal characteristics of a variety of possible signals currently under investigation such as those from topological defect dark matter (axion-like particle domain walls), axion-like particle stars, solitons of complex-valued scalar fields (Q-balls), stochastic fluctuations of bosonic dark matter fields, a solar axion-like particle halo, and bursts of ultralight bosonic fields produced by cataclysmic astrophysical events such as binary black hole mergers., Comment: 22 pages, 12 figures, submitted to Annalen der Physik

The COHERENT collaboration observed coherent elastic neutrino nucleus scattering using a 14.6 kg cesium-iodide (CsI) detector in 2017 and recently published the updated results before decommissioning the detector. Here, we present the legacy determination of the weak mixing angle and of the average neutron rms radius of $^{133}\mathrm{Cs}$ and $^{127}\mathrm{I}$ obtained with the full CsI dataset, also exploiting the combination with the atomic parity violation (APV) experimental result, that allows us to achieve a precision as low as $\sim$4.5% and to disentangle the contributions of the $^{133}\mathrm{Cs}$ and $^{127}\mathrm{I}$ nuclei. Interestingly, we show that the COHERENT CsI data show a 6$\sigma$ evidence of the nuclear structure suppression of the full coherence. Moreover, we derive a data-driven APV+COHERENT measurement of the low-energy weak mixing angle with a percent uncertainty, independent of the value of the average neutron rms radius of $^{133}\mathrm{Cs}$ and $^{127}\mathrm{I}$, that is allowed to vary freely in the fit. Additionally, we extensively discuss the impact of using two different determinations of the theoretical parity non-conserving amplitude in the APV fit. Our findings show that the particular choice can make a significant difference, up to 6.5% on $R_n$(Cs) and 11% on the weak mixing angle. Finally, in light of the recent announcement of a future deployment of a 10 kg and a $\sim$700 kg cryogenic CsI detectors, we provide future prospects for these measurements, comparing them with other competitive experiments that are foreseen in the near future., Comment: 21 pages, 10 figures, matches journal version

Just like in humans vision plays a fundamental role in guiding adaptive locomotion, when designing the control strategy for a walking assistive technology, Computer Vision may bring substantial improvements when performing an environment-based assistance modulation. In this work, we developed a hip exosuit controller able to distinguish among three different walking terrains through the use of an RGB camera and to adapt the assistance accordingly. The system was tested with seven healthy participants walking throughout an overground path comprising of staircases and level ground. Subjects performed the task with the exosuit disabled (Exo Off), constant assistance profile (Vision Off ), and with assistance modulation (Vision On). Our results showed that the controller was able to promptly classify in real-time the path in front of the user with an overall accuracy per class above the 85%, and to perform assistance modulation accordingly. Evaluation related to the effects on the user showed that Vision On was able to outperform the other two conditions: we obtained significantly higher metabolic savings than Exo Off, with a peak of about -20% when climbing up the staircase and about -16% in the overall path, and than Vision Off when ascending or descending stairs. Such advancements in the field may yield to a step forward for the exploitation of lightweight walking assistive technologies in real-life scenarios.

Bimanual object manipulation involves multiple visuo-haptic sensory feedbacks arising from the interaction with the environment that are managed from the central nervous system and consequently translated in motor commands. Kinematic strategies that occur during bimanual coupled tasks are still a scientific debate despite modern advances in haptics and robotics. Current technologies may have the potential to provide realistic scenarios involving the entire upper limb extremities during multi-joint movements but are not yet exploited to their full potential. The present study explores how hands dynamically interact when manipulating a shared object through the use of two impedance-controlled exoskeletons programmed to simulate bimanually coupled manipulation of virtual objects. We enrolled twenty-six participants (2 groups: right-handed and left-handed) who were requested to use both hands to grab simulated objects across the robot workspace and place them in specific locations. The virtual objects were rendered with different dynamic proprieties and textures influencing the manipulation strategies to complete the tasks. Results revealed that the roles of hands are related to the movement direction, the haptic features, and the handedness preference. Outcomes suggested that the haptic feedback affects bimanual strategies depending on the movement direction. However, left-handers show better control of the force applied between the two hands, probably due to environmental pressures for right-handed manipulations.

Correlative light electron microscopy (CLEM) requires the availability of robust probes which are visible both in light and electron microscopy. Here we demonstrate a CLEM approach using small gold nanoparticles as a single probe. Individual gold nanoparticles bound to the epidermal growth factor protein were located with nanometric precision background-free in human cancer cells by light microscopy using resonant four-wave-mixing (FWM), and were correlatively mapped with high accuracy to the corresponding transmission electron microscopy images. We used nanoparticles of 10 nm and 5 nm radius, and show a correlation accuracy below 60 nm over an area larger than 10 um size, without the need for additional fiducial markers. Correlation accuracy was improved to below 40 nm by reducing systematic errors, while the localisation precision is below 10 nm. Polarisation-resolved FWM correlates with nanoparticle shapes, promising for multiplexing by shape recognition in future applications. Owing to the photostability of gold nanoparticles and the applicability of FWM microscopy to living cells, FWM-CLEM opens up a powerful alternative to fluorescence-based methods.

Human visual attention is a complex phenomenon that has been studied for decades. Within it, the particular problem of scanpath prediction poses a challenge, particularly due to the inter- and intra-observer variability, among other reasons. Besides, most existing approaches to scanpath prediction have focused on optimizing the prediction of a gaze point given the previous ones. In this work, we present a probabilistic time-evolving approach to scanpath prediction, based on Bayesian deep learning. We optimize our model using a novel spatio-temporal loss function based on a combination of Kullback-Leibler divergence and dynamic time warping, jointly considering the spatial and temporal dimensions of scanpaths. Our scanpath prediction framework yields results that outperform those of current state-of-the-art approaches, and are almost on par with the human baseline, suggesting that our model is able to generate scanpaths whose behavior closely resembles those of the real ones., Comment: Under submission

Low-energy electron microscopy (LEEM) taken as intensity-voltage (I-V) curves provides hyperspectral images of surfaces, which can be used to identify the surface type, but are difficult to analyze. Here, we demonstrate the use of an algorithm for factorizing the data into spectra and concentrations of characteristic components (FSC3) for identifying distinct physical surface phases. Importantly, FSC3 is an unsupervised and fast algorithm. As example data we use experiments on the growth of praseodymium oxide or ruthenium oxide on ruthenium single crystal substrates, both featuring a complex distribution of coexisting surface components, varying in both chemical composition and crystallographic structure. With the factorization result a sparse sampling method is demonstrated, reducing the measurement time by 1-2 orders of magnitude, relevant for dynamic surface studies. The FSC3 concentrations are providing the features for a support vector machine (SVM) based supervised classification of the types. Here, specific surface regions which have been identified structurally, via their diffraction pattern, as well as chemically by complementary spectro-microscopic techniques, are used as training sets. A reliable classification is demonstrated on both exemplary LEEM I-V datasets., Comment: 13 pages, 7 figures

Ultralight bosonic dark matter (UBDM) can be described by a classical wave-like field oscillating near the Compton frequency of the bosons. If a measurement scheme for the direct detection of UBDM interactions is sensitive to a signature quadratic in the field, then there is a near-zero-frequency (dc) component of the signal. Thus, a detector with a given finite bandwidth can be used to search for bosons with Compton frequencies many orders of magnitude larger than its bandwidth. This opens the possibility of a detection scheme analogous to Hanbury Brown and Twiss intensity interferometry. Assuming that the UBDM is virialized in the galactic gravitational potential, the random velocities produce slight deviations from the Compton frequency. These result in stochastic fluctuations of the intensity on a time scale determined by the spread in kinetic energies. In order to mitigate ubiquitous local low-frequency noise, a network of sensors can be used to search for the stochastic intensity fluctuations by measuring cross-correlation between the sensors. This method is inherently broadband, since a large range of Compton frequencies will yield near-zero-frequency components within the sensor bandwidth that can be searched for simultaneously. Measurements with existing sensor networks have sufficient sensitivity to search experimentally unexplored parameter space., Comment: 11 pages, 4 figures. Supplementary material included after the references

The Global Network of Optical Magnetometers for Exotic physics searches (GNOME) conducts an experimental search for certain forms of dark matter based on their spatiotemporal signatures imprinted on a global array of synchronized atomic magnetometers. The experiment described here looks for a gradient coupling of axion-like particles (ALPs) with proton spins as a signature of locally dense dark matter objects such as domain walls. In this work, stochastic optimization with machine learning is proposed for use in a search for ALP domain walls based on GNOME data. The validity and reliability of this method were verified using binary classification. The projected sensitivity of this new analysis method for ALP domain-wall crossing events is presented.

We present a single-image data-driven method to automatically relight images with full-body humans in them. Our framework is based on a realistic scene decomposition leveraging precomputed radiance transfer (PRT) and spherical harmonics (SH) lighting. In contrast to previous work, we lift the assumptions on Lambertian materials and explicitly model diffuse and specular reflectance in our data. Moreover, we introduce an additional light-dependent residual term that accounts for errors in the PRT-based image reconstruction. We propose a new deep learning architecture, tailored to the decomposition performed in PRT, that is trained using a combination of L1, logarithmic, and rendering losses. Our model outperforms the state of the art for full-body human relighting both with synthetic images and photographs., Comment: 11 pages, 12 figures

Wearable robots are undergoing a disruptive transition, from the rigid machines that populated the science-fiction world in the early eighties to lightweight robotic apparel, hardly distinguishable from our daily clothes. In less than a decade of development, soft robotic suits have achieved important results in human motor assistance and augmentation. In this paper, we start by giving a definition of soft robotic suits and proposing a taxonomy to classify existing systems. We then critically review the modes of actuation, the physical human-robot interface and the intention-detection strategies of state of the art soft robotic suits, highlighting the advantages and limitations of different approaches. Finally, we discuss the impact of this new technology on human movements, for both augmenting human function and supporting motor impairments, and identify areas that are in need of further development., Comment: Accepted as a Survey Paper on IEEE Transaction on Robotics

Understanding and modeling the dynamics of human gaze behavior in 360$^\circ$ environments is a key challenge in computer vision and virtual reality. Generative adversarial approaches could alleviate this challenge by generating a large number of possible scanpaths for unseen images. Existing methods for scanpath generation, however, do not adequately predict realistic scanpaths for 360$^\circ$ images. We present ScanGAN360, a new generative adversarial approach to address this challenging problem. Our network generator is tailored to the specifics of 360$^\circ$ images representing immersive environments. Specifically, we accomplish this by leveraging the use of a spherical adaptation of dynamic-time warping as a loss function and proposing a novel parameterization of 360$^\circ$ scanpaths. The quality of our scanpaths outperforms competing approaches by a large margin and is almost on par with the human baseline. ScanGAN360 thus allows fast simulation of large numbers of virtual observers, whose behavior mimics real users, enabling a better understanding of gaze behavior and novel applications in virtual scene design.

Ultralight bosons such as axion-like particles are viable candidates for dark matter. They can form stable, macroscopic field configurations in the form of topological defects that could concentrate the dark matter density into many distinct, compact spatial regions that are small compared to the galaxy but much larger than the Earth. Here, we report the results of a search for transient signals from axion-like particle domain walls with the Global Network of Optical Magnetometers for Exotic physics searches (GNOME). We search the data, consisting of correlated measurements from optical atomic magnetometers located in laboratories all over the world, for patterns of signals propagating through the network consistent with domain walls. The analysis of data from a continuous month-long operation of the GNOME finds no statistically significant signals, thus placing experimental constraints on such dark matter scenarios., Comment: 25 pages, 6 figures, 2 tables

Despite their widespread use in cell biology, fluorescence lifetime imaging microscopy (FLIM) data-sets are challenging to analyse, because each spatial position can contain a superposition of multiple fluorescent components. Here, we present a data analysis method employing all information in the available photon budget, as well as being fast. The method, called uFLIM, determines spatial distributions and temporal dynamics of multiple fluorescent components with no prior knowledge. It goes significantly beyond current approaches which either assume the functional dependence of the dynamics, e.g. an exponential decay, or require dynamics to be known, or calibrated. Its efficient non-negative matrix factorization algorithm allows for real-time data processing. We validate in silico that uFLIM is capable to disentangle the spatial distribution and spectral properties of five fluorescing probes, from only two excitation and detection channels and a photon budget of 100 detected photons per pixel. By adapting the method to data exhibiting Forster resonant energy transfer (FRET), we retrieve the spatial and transfer rate distribution of the bound species, without constrains on donor and acceptor dynamics.

Virtual reality (VR) is rapidly growing, with the potential to change the way we create and consume content. In VR, users integrate multimodal sensory information they receive, to create a unified perception of the virtual world. In this survey, we review the body of work addressing multimodality in VR, and its role and benefits in user experience, together with different applications that leverage multimodality in many disciplines. These works thus encompass several fields of research, and demonstrate that multimodality plays a fundamental role in VR; enhancing the experience, improving overall performance, and yielding unprecedented abilities in skill and knowledge transfer., Comment: 35 pages (24 pages not including references), 10 figures, 4 tables

Observing and recognizing materials is a fundamental part of our daily life. Under typical viewing conditions, we are capable of effortlessly identifying the objects that surround us and recognizing the materials they are made of. Nevertheless, understanding the underlying perceptual processes that take place to accurately discern the visual properties of an object is a long-standing problem. In this work, we perform a comprehensive and systematic analysis of how the interplay of geometry, illumination, and their spatial frequencies affects human performance on material recognition tasks. We carry out large-scale behavioral experiments where participants are asked to recognize different reference materials among a pool of candidate samples. In the different experiments, we carefully sample the information in the frequency domain of the stimuli. From our analysis, we find significant first-order interactions between the geometry and the illumination, of both the reference and the candidates. In addition, we observe that simple image statistics and higher-order image histograms do not correlate with human performance. Therefore, we perform a high-level comparison of highly non-linear statistics by training a deep neural network on material recognition tasks. Our results show that such models can accurately classify materials, which suggests that they are capable of defining a meaningful representation of material appearance from labeled proximal image data. Last, we find preliminary evidence that these highly non-linear models and humans may use similar high-level factors for material recognition tasks., Comment: 15 pages, 16 figures, Accepted to the Journal of Vision, 2021

The Global Network of Optical Magnetometers for Exotic physics searches (GNOME) is a network of time-synchronized, geographically separated, optically pumped atomic magnetometers that is being used to search for correlated transient signals heralding exotic physics. GNOME is sensitive to exotic couplings of atomic spins to certain classes of dark matter candidates, such as axions. This work presents a data analysis procedure to search for axion dark matter in the form of topological defects: specifically, walls separating domains of discrete degenerate vacua in the axion field. An axion domain wall crossing the Earth creates a distinctive signal pattern in the network that can be distinguished from random noise. The reliability of the analysis procedure and the sensitivity of the GNOME to domain-wall crossings is studied using simulated data.

We present a model to measure the similarity in appearance between different materials, which correlates with human similarity judgments. We first create a database of 9,000 rendered images depicting objects with varying materials, shape and illumination. We then gather data on perceived similarity from crowdsourced experiments; our analysis of over 114,840 answers suggests that indeed a shared perception of appearance similarity exists. We feed this data to a deep learning architecture with a novel loss function, which learns a feature space for materials that correlates with such perceived appearance similarity. Our evaluation shows that our model outperforms existing metrics. Last, we demonstrate several applications enabled by our metric, including appearance-based search for material suggestions, database visualization, clustering and summarization, and gamut mapping., Comment: 12 pages, 17 figures

Spontaneous emission from excitonic transitions in InAs/GaAs quantum dots embedded in photonic crystal waveguides at 5K into non-guided and guided modes is determined by direct hyperspectral imaging. This enables measurement of the absolute coupling efficiency into the guided modes, the beta-factor, directly, without assumptions on decay rates used previously. Notably, we found beta-factors above 90% over a wide spectral range of 40meV in the fast light regime, reaching a maximum of (99 $\pm$ 1)%. We measure the directional emission of the circularly polarized transitions in a magnetic field into counter-propagating guided modes, to deduce the mode circularity at the quantum dot sites. We find that points of high directionality, up to 97%, correlate with a reduced beta-factor, consistent with their positions away from the mode field antinode. By comparison with calibrated finite-difference time-domain simulations, we use the emission energy, mode circularity and beta-factor to estimate the quantum dot position inside the photonic crystal waveguide unit cell.

We measure the coherent nonlinear response of excitons in a single-layer of molybdenum disulphide embedded in hexagonal boron nitride, forming a $h$-BN/MoS$_2$/$h$-BN heterostructure. Using four-wave mixing microscopy and imaging, we correlate the exciton homogeneous and inhomogeneous broadenings. We find that the exciton dynamics is governed by microscopic disorder on top of the ideal crystal properties. Analyzing the exciton ultra-fast density dynamics using amplitude and phase of the response, we investigate the relaxation pathways of the resonantly driven exciton population. The surface protection via encapsulation provides stable monolayer samples with low disorder, avoiding surface contaminations and the resulting exciton broadening and modifications of the dynamics. We identify areas localized to a few microns where the optical response is totally dominated by homogeneous broadening. Across the sample of tens of micrometers, weak inhomogeneous broadening and strain effects are observed, attributed to the remaining interaction with the $h$-BN and imperfections in the encapsulation process.

Fully-inorganic cesium lead halide perovskite nanocrystals (NCs) have shown to exhibit outstanding optical properties such as wide spectral tunability, high quantum yield, high oscillator strength as well as blinking-free single photon emission and low spectral diffusion. Here, we report measurements of the coherent and incoherent exciton dynamics on the 100 fs to 10 ns timescale, determining dephasing and density decay rates in these NCs. The experiments are performed on CsPbBr$_{2}$Cl NCs using transient resonant three-pulse four-wave mixing (FWM) in heterodyne detection at temperatures ranging from 5 K to 50 K. We found a low-temperature exciton dephasing time of 24.5$\pm$1.0 ps, inferred from the decay of the photon-echo amplitude at 5 K, corresponding to a homogeneous linewidth (FWHM) of 54$\pm$5 {\mu}eV. Furthermore, oscillations in the photon-echo signal on a picosecond timescale are observed and attributed to coherent coupling of the exciton to a quantized phonon mode with 3.45 meV energy.

The Global Network of Optical Magnetometers to search for Exotic physics (GNOME) is a network of geographically separated, time-synchronized, optically pumped atomic magnetometers that is being used to search for correlated transient signals heralding exotic physics. The GNOME is sensitive to nuclear- and electron-spin couplings to exotic fields from astrophysical sources such as compact dark-matter objects (for example, axion stars and domain walls). Properties of the GNOME sensors such as sensitivity, bandwidth, and noise characteristics are studied in the present work, and features of the network's operation (e.g., data acquisition, format, storage, and diagnostics) are described. Characterization of the GNOME is a key prerequisite to searches for and identification of exotic physics signatures., Comment: 45 pages, 16 figures, 2 tables

Many different techniques for measuring material appearance have been proposed in the last few years. These have produced large public datasets, which have been used for accurate, data-driven appearance modeling. However, although these datasets have allowed us to reach an unprecedented level of realism in visual appearance, editing the captured data remains a challenge. In this paper, we present an intuitive control space for predictable editing of captured BRDF data, which allows for artistic creation of plausible novel material appearances, bypassing the difficulty of acquiring novel samples. We first synthesize novel materials, extending the existing MERL dataset up to 400 mathematically valid BRDFs. We then design a large-scale experiment, gathering 56,000 subjective ratings on the high-level perceptual attributes that best describe our extended dataset of materials. Using these ratings, we build and train networks of radial basis functions to act as functionals mapping the perceptual attributes to an underlying PCA-based representation of BRDFs. We show that our functionals are excellent predictors of the perceived attributes of appearance. Our control space enables many applications, including intuitive material editing of a wide range of visual properties, guidance for gamut mapping, analysis of the correlation between perceptual attributes, or novel appearance similarity metrics. Moreover, our methodology can be used to derive functionals applicable to classic analytic BRDF representations. We release our code and dataset publicly, in order to support and encourage further research in this direction.

Video capture is limited by the trade-off between spatial and temporal resolution: when capturing videos of high temporal resolution, the spatial resolution decreases due to bandwidth limitations in the capture system. Achieving both high spatial and temporal resolution is only possible with highly specialized and very expensive hardware, and even then the same basic trade-off remains. The recent introduction of compressive sensing and sparse reconstruction techniques allows for the capture of single-shot high-speed video, by coding the temporal information in a single frame, and then reconstructing the full video sequence from this single coded image and a trained dictionary of image patches. In this paper, we first analyze this approach, and find insights that help improve the quality of the reconstructed videos. We then introduce a novel technique, based on convolutional sparse coding (CSC), and show how it outperforms the state-of-the-art, patch-based approach in terms of flexibility and efficiency, due to the convolutional nature of its filter banks. The key idea for CSC high-speed video acquisition is extending the basic formulation by imposing an additional constraint in the temporal dimension, which enforces sparsity of the first-order derivatives over time.