Search

We introduce the task of spotting temporally precise, fine-grained events in video (detecting the precise moment in time events occur). Precise spotting requires models to reason globally about the full-time scale of actions and locally to identify subtle frame-to-frame appearance and motion differences that identify events during these actions. Surprisingly, we find that top performing solutions to prior video understanding tasks such as action detection and segmentation do not simultaneously meet both requirements. In response, we propose E2E-Spot, a compact, end-to-end model that performs well on the precise spotting task and can be trained quickly on a single GPU. We demonstrate that E2E-Spot significantly outperforms recent baselines adapted from the video action detection, segmentation, and spotting literature to the precise spotting task. Finally, we contribute new annotations and splits to several fine-grained sports action datasets to make these datasets suitable for future work on precise spotting., Comment: ECCV 2022; Website URL: https://jhong93.github.io/projects/spot.html

Stein discrepancies have emerged as a powerful statistical tool, being applied to fundamental statistical problems including parameter inference, goodness-of-fit testing, and sampling. The canonical Stein discrepancies require the derivatives of a statistical model to be computed, and in return provide theoretical guarantees of convergence detection and control. However, for complex statistical models, the stable numerical computation of derivatives can require bespoke algorithmic development and render Stein discrepancies impractical. This paper focuses on posterior approximation using Stein discrepancies, and introduces a collection of non-canonical Stein discrepancies that are gradient free, meaning that derivatives of the statistical model are not required. Sufficient conditions for convergence detection and control are established, and applications to sampling and variational inference are presented.

Rigged puppets are one of the most prevalent representations to create 2D character animations. Creating these puppets requires partitioning characters into independently moving parts. In this work, we present a method to automatically identify such articulated parts from a small set of character poses shown in a sprite sheet, which is an illustration of the character that artists often draw before puppet creation. Our method is trained to infer articulated parts, e.g. head, torso and limbs, that can be re-assembled to best reconstruct the given poses. Our results demonstrate significantly better performance than alternatives qualitatively and quantitatively.Our project page https://zhan-xu.github.io/parts/ includes our code and data.

We present ASSET, a neural architecture for automatically modifying an input high-resolution image according to a user's edits on its semantic segmentation map. Our architecture is based on a transformer with a novel attention mechanism. Our key idea is to sparsify the transformer's attention matrix at high resolutions, guided by dense attention extracted at lower image resolutions. While previous attention mechanisms are computationally too expensive for handling high-resolution images or are overly constrained within specific image regions hampering long-range interactions, our novel attention mechanism is both computationally efficient and effective. Our sparsified attention mechanism is able to capture long-range interactions and context, leading to synthesizing interesting phenomena in scenes, such as reflections of landscapes onto water or flora consistent with the rest of the landscape, that were not possible to generate reliably with previous convnets and transformer approaches. We present qualitative and quantitative results, along with user studies, demonstrating the effectiveness of our method., Comment: SIGGRAPH 2022 - Journal Track

Approaches for single-view reconstruction typically rely on viewpoint annotations, silhouettes, the absence of background, multiple views of the same instance, a template shape, or symmetry. We avoid all such supervision and assumptions by explicitly leveraging the consistency between images of different object instances. As a result, our method can learn from large collections of unlabelled images depicting the same object category. Our main contributions are two ways for leveraging cross-instance consistency: (i) progressive conditioning, a training strategy to gradually specialize the model from category to instances in a curriculum learning fashion; and (ii) neighbor reconstruction, a loss enforcing consistency between instances having similar shape or texture. Also critical to the success of our method are: our structured autoencoding architecture decomposing an image into explicit shape, texture, pose, and background; an adapted formulation of differential rendering; and a new optimization scheme alternating between 3D and pose learning. We compare our approach, UNICORN, both on the diverse synthetic ShapeNet dataset - the classical benchmark for methods requiring multiple views as supervision - and on standard real-image benchmarks (Pascal3D+ Car, CUB) for which most methods require known templates and silhouette annotations. We also showcase applicability to more challenging real-world collections (CompCars, LSUN), where silhouettes are not available and images are not cropped around the object., Comment: ECCV 2022. Project webpage with code and videos: http://imagine.enpc.fr/~monniert/UNICORN/

Sequential algorithms are popular for experimental design, enabling emulation, optimisation and inference to be efficiently performed. For most of these applications bespoke software has been developed, but the approach is general and many of the actual computations performed in such software are identical. Motivated by the diverse problems that can in principle be solved with common code, this paper presents GaussED, a simple probabilistic programming language coupled to a powerful experimental design engine, which together automate sequential experimental design for approximating a (possibly nonlinear) quantity of interest in Gaussian processes models. Using a handful of commands, GaussED can be used to: solve linear partial differential equations, perform tomographic reconstruction from integral data and implement Bayesian optimisation with gradient data.

This paper introduces a model for producing stylized line drawings from 3D shapes. The model takes a 3D shape and a viewpoint as input, and outputs a drawing with textured strokes, with variations in stroke thickness, deformation, and color learned from an artist's style. The model is fully differentiable. We train its parameters from a single training drawing of another 3D shape. We show that, in contrast to previous image-based methods, the use of a geometric representation of 3D shape and 2D strokes allows the model to transfer important aspects of shape and texture style while preserving contours. Our method outputs the resulting drawing in a vector representation, enabling richer downstream analysis or editing in interactive applications., Comment: Accepted to ICCV 2021

We introduce an exact mapping of Clifford (stabilizer) random tensor networks (RTNs) and monitored quantum circuits, onto a statistical mechanics model. With Haar unitaries, the fundamental degrees of freedom ('spins') are permutations because all operators commuting with the action of the unitaries on a tensor product arise from permutations of the tensor factors ('Schur-Weyl duality'). For unitaries restricted to the smaller Clifford group, the set of commuting operators, the 'commutant', forming the new 'spin' degrees of freedom, will be larger. We use the recent full characterization of this commutant by Gross et al., Comm. Math. Phys. 385, 1325 (2021), to construct the Clifford statistical mechanics models for on-site Hilbert space dimensions which are powers of a prime number $p$. We show that the Boltzmann weights are invariant under a symmetry group involving orthogonal matrices with entries in the finite number field ${\bf F}_p$. This implies that the symmetry group, and consequently all universal properties of entanglement transitions in Clifford circuits and RTNs will in general depend on, and only on the prime $p$. We show that Clifford monitored circuits with on-site Hilbert space dimension $d=p^M$ are described by percolation in the limits $d \to \infty$ at (a) $p=$ fixed but $M\to \infty$, and at (b) $M= 1$ but $p \to \infty$. In the limit (a) we calculate the effective central charge, and in the limit (b) we derive the following universal minimal cut entanglement entropy $S_A =(\sqrt{3}/\pi)\ln p \ln L_A$ for $d=p$ large at the transition. We verify those predictions numerically, and present extensive numerical results for critical exponents at the transition in monitored Clifford circuits for prime number on-site Hilbert space dimension $d=p$ for a variety of different values of $p$, and find that they approach percolation values at large $p$., Comment: 23 pages, 5 figures. Abstract shortened to meet arxiv requirements, see pdf for full abstract

Human pose is a useful feature for fine-grained sports action understanding. However, pose estimators are often unreliable when run on sports video due to domain shift and factors such as motion blur and occlusions. This leads to poor accuracy when downstream tasks, such as action recognition, depend on pose. End-to-end learning circumvents pose, but requires more labels to generalize. We introduce Video Pose Distillation (VPD), a weakly-supervised technique to learn features for new video domains, such as individual sports that challenge pose estimation. Under VPD, a student network learns to extract robust pose features from RGB frames in the sports video, such that, whenever pose is considered reliable, the features match the output of a pretrained teacher pose detector. Our strategy retains the best of both pose and end-to-end worlds, exploiting the rich visual patterns in raw video frames, while learning features that agree with the athletes' pose and motion in the target video domain to avoid over-fitting to patterns unrelated to athletes' motion. VPD features improve performance on few-shot, fine-grained action recognition, retrieval, and detection tasks in four real-world sports video datasets, without requiring additional ground-truth pose annotations., Comment: ICCV 2021 (poster)

Generating font glyphs of consistent style from one or a few reference glyphs, i.e., font completion, is an important task in topographical design. As the problem is more well-defined than general image style transfer tasks, thus it has received interest from both vision and machine learning communities. Existing approaches address this problem as a direct image-to-image translation task. In this work, we innovate to explore the generation of font glyphs as 2D graphic objects with the graph as an intermediate representation, so that more intrinsic graphic properties of font styles can be captured. Specifically, we formulate a cross-modality cycled image-to-image model structure with a graph constructor between an image encoder and an image renderer. The novel graph constructor maps a glyph's latent code to its graph representation that matches expert knowledge, which is trained to help the translation task. Our model generates improved results than both image-to-image baseline and previous state-of-the-art methods for glyph completion. Furthermore, the graph representation output by our model also provides an intuitive interface for users to do local editing and manipulation. Our proposed cross-modality cycled representation learning has the potential to be applied to other domains with prior knowledge from different data modalities. Our code is available at https://github.com/VITA-Group/Font_Completion_Graph., Comment: submitted to IEEE Transactions on Multimedia (TMM)

We explore a class of "open" quantum circuit models with local decoherence ("noise") and local projective measurements, each respecting a global Z_2 symmetry. The model supports a spin glass phase where the Z_2 symmetry is spontaneously broken (not possible in an equilibrium 1d system), a paramagnetic phase characterized by a divergent susceptibility, and an intermediate "trivial" phase. All three phases are also stable to Z_2-symmetric local unitary gates, and the dynamical phase transitions between the phases are in the percolation universality class. The open circuit dynamics can be purified by explicitly introducing a bath with its own "scrambling" dynamics, as in [Bao, Choi, Altman, arXiv:2102.09164], which does not change any of the universal physics. Within the spin glass phase the circuit dynamics can be interpreted as a quantum repetition code, with each stabilizer of the code measured stochastically at a finite rate, and the decoherences as effective bit-flip errors. Motivated by the geometry of the spin glass phase, we devise a novel decoding algorithm for recovering an arbitrary initial qubit state in the code space, assuming knowledge of the history of the measurement outcomes, and the ability of performing local Pauli measurements and gates on the final state. For a circuit with L^d qubits running for time T, the time needed to execute the decoder scales as O(L^d T) (with dimensionality d). With this decoder in hand, we find that the information of the initial encoded qubit state can be retained (and then recovered) for a time logarithmic in L for a 1d circuit, and for a time at least linear in L in 2d below a finite error threshold. For both the repetition and toric codes, we compare and contrast our decoding algorithm with earlier algorithms that map the error model to the random bond Ising model., Comment: 32+epsilon pages, 22 figures. Published version

We investigate the problem of training generative models on a very sparse collection of 3D models. We use geometrically motivated energies to augment and thus boost a sparse collection of example (training) models. We analyze the Hessian of the as-rigid-as-possible (ARAP) energy to sample from and project to the underlying (local) shape space, and use the augmented dataset to train a variational autoencoder (VAE). We iterate the process of building latent spaces of VAE and augmenting the associated dataset, to progressively reveal a richer and more expressive generative space for creating geometrically and semantically valid samples. Our framework allows us to train generative 3D models even with a small set of good quality 3D models, which are typically hard to curate. We extensively evaluate our method against a set of strong baselines, provide ablation studies and demonstrate application towards establishing shape correspondences. We present multiple examples of interesting and meaningful shape variations even when starting from as few as 3-10 training shapes., Comment: CVPR 2022

We explore a class of random tensor network models with "stabilizer" local tensors which we name Random Stabilizer Tensor Networks (RSTNs). For RSTNs defined on a two-dimensional square lattice, we perform extensive numerical studies of entanglement phase transitions between volume-law and area-law entangled phases of the one-dimensional boundary states. These transitions occur when either (a) the bond dimension $D$ of the constituent tensors is varied, or (b) the tensor network is subject to random breaking of bulk bonds, implemented by forced measurements. In the absence of broken bonds, we find that the RSTN supports a volume-law entangled boundary state with bond dimension $D\geq3$ where $D$ is a prime number, and an area-law entangled boundary state for $D=2$. Upon breaking bonds at random in the bulk with probability $p$, there exists a critical measurement rate $p_c$ for each $D\geq 3$ above which the boundary state becomes area-law entangled. To explore the conformal invariance at these entanglement transitions for different prime $D$, we consider tensor networks on a finite rectangular geometry with a variety of boundary conditions, and extract universal operator scaling dimensions via extensive numerical calculations of the entanglement entropy, mutual information and mutual negativity at their respective critical points. Our results at large $D$ approach known universal data of percolation conformal field theory, while showing clear discrepancies at smaller $D$, suggesting a distinct entanglement transition universality class for each prime $D$. We further study universal entanglement properties in the volume-law phase and demonstrate quantitative agreement with the recently proposed description in terms of a directed polymer in a random environment., Comment: Updated to published version

Fonts are ubiquitous across documents and come in a variety of styles. They are either represented in a native vector format or rasterized to produce fixed resolution images. In the first case, the non-standard representation prevents benefiting from latest network architectures for neural representations; while, in the latter case, the rasterized representation, when encoded via networks, results in loss of data fidelity, as font-specific discontinuities like edges and corners are difficult to represent using neural networks. Based on the observation that complex fonts can be represented by a superposition of a set of simpler occupancy functions, we introduce \textit{multi-implicits} to represent fonts as a permutation-invariant set of learned implict functions, without losing features (e.g., edges and corners). However, while multi-implicits locally preserve font features, obtaining supervision in the form of ground truth multi-channel signals is a problem in itself. Instead, we propose how to train such a representation with only local supervision, while the proposed neural architecture directly finds globally consistent multi-implicits for font families. We extensively evaluate the proposed representation for various tasks including reconstruction, interpolation, and synthesis to demonstrate clear advantages with existing alternatives. Additionally, the representation naturally enables glyph completion, wherein a single characteristic font is used to synthesize a whole font family in the target style.

Monitored quantum dynamics reveal quantum state trajectories which exhibit a rich phenomenology of entanglement structures, including a transition from a weakly-monitored volume law entangled phase to a strongly-monitored area law phase. For one-dimensional hybrid circuits with both random unitary dynamics and interspersed measurements, we combine analytic mappings to an effective statistical mechanics model with extensive numerical simulations on hybrid Clifford circuits to demonstrate that the universal entanglement properties of the volume law phase can be quantitatively described by a fluctuating entanglement domain wall that is equivalent to a "directed polymer in a random environment" (DPRE). This relationship improves upon a qualitative "mean-field" statistical mechanics of the volume-law-entangled phase [1, 2]. For the Clifford circuit in various geometries, we obtain agreement between the subleading entanglement entropies and error correcting properties of the volume-law phase (which quantify its stability to projective measurements) with predictions of the DPRE. We further demonstrate that depolarizing noise in the hybrid dynamics near the final circuit time can drive a continuous phase transition to a non-error correcting volume law phase that is not immune to the disentangling action of projective measurements. We observe this transition in hybrid Clifford dynamics, and obtain quantitative agreement with critical exponents for a "pinning" phase transition of the DPRE in the presence of an attractive interface., Comment: 22 pages, 18 figures

Artists and video game designers often construct 2D animations using libraries of sprites -- textured patches of objects and characters. We propose a deep learning approach that decomposes sprite-based video animations into a disentangled representation of recurring graphic elements in a self-supervised manner. By jointly learning a dictionary of possibly transparent patches and training a network that places them onto a canvas, we deconstruct sprite-based content into a sparse, consistent, and explicit representation that can be easily used in downstream tasks, like editing or analysis. Our framework offers a promising approach for discovering recurring visual patterns in image collections without supervision., Comment: Accepted to NeurIPS 2021

We introduce CharacterGAN, a generative model that can be trained on only a few samples (8 - 15) of a given character. Our model generates novel poses based on keypoint locations, which can be modified in real time while providing interactive feedback, allowing for intuitive reposing and animation. Since we only have very limited training samples, one of the key challenges lies in how to address (dis)occlusions, e.g. when a hand moves behind or in front of a body. To address this, we introduce a novel layering approach which explicitly splits the input keypoints into different layers which are processed independently. These layers represent different parts of the character and provide a strong implicit bias that helps to obtain realistic results even with strong (dis)occlusions. To combine the features of individual layers we use an adaptive scaling approach conditioned on all keypoints. Finally, we introduce a mask connectivity constraint to reduce distortion artifacts that occur with extreme out-of-distribution poses at test time. We show that our approach outperforms recent baselines and creates realistic animations for diverse characters. We also show that our model can handle discrete state changes, for example a profile facing left or right, that the different layers do indeed learn features specific for the respective keypoints in those layers, and that our model scales to larger datasets when more data is available., Comment: Best Paper WACV 2022. Code available at https://github.com/tohinz/CharacterGAN

We introduce a deep generative network for 3D shape detailization, akin to stylization with the style being geometric details. We address the challenge of creating large varieties of high-resolution and detailed 3D geometry from a small set of exemplars by treating the problem as that of geometric detail transfer. Given a low-resolution coarse voxel shape, our network refines it, via voxel upsampling, into a higher-resolution shape enriched with geometric details. The output shape preserves the overall structure (or content) of the input, while its detail generation is conditioned on an input "style code" corresponding to a detailed exemplar. Our 3D detailization via conditional refinement is realized by a generative adversarial network, coined DECOR-GAN. The network utilizes a 3D CNN generator for upsampling coarse voxels and a 3D PatchGAN discriminator to enforce local patches of the generated model to be similar to those in the training detailed shapes. During testing, a style code is fed into the generator to condition the refinement. We demonstrate that our method can refine a coarse shape into a variety of detailed shapes with different styles. The generated results are evaluated in terms of content preservation, plausibility, and diversity. Comprehensive ablation studies are conducted to validate our network designs. Code is available at https://github.com/czq142857/DECOR-GAN., Comment: CVPR 2021 (oral). Code: https://github.com/czq142857/DECOR-GAN

We present a deep neural network to predict structural similarity between 2D layouts by leveraging Graph Matching Networks (GMN). Our network, coined LayoutGMN, learns the layout metric via neural graph matching, using an attention-based GMN designed under a triplet network setting. To train our network, we utilize weak labels obtained by pixel-wise Intersection-over-Union (IoUs) to define the triplet loss. Importantly, LayoutGMN is built with a structural bias which can effectively compensate for the lack of structure awareness in IoUs. We demonstrate this on two prominent forms of layouts, viz., floorplans and UI designs, via retrieval experiments on large-scale datasets. In particular, retrieval results by our network better match human judgement of structural layout similarity compared to both IoUs and other baselines including a state-of-the-art method based on graph neural networks and image convolution. In addition, LayoutGMN is the first deep model to offer both metric learning of structural layout similarity and structural matching between layout elements.

We propose entanglement negativity as a fine-grained probe of measurement-induced criticality. We motivate this proposal in stabilizer states, where for two disjoint subregions, comparing their "mutual negativity" and their mutual information leads to a precise distinction between bipartite and multipartite entanglement. In a measurement-only stabilizer circuit that maps exactly to two-dimensional critical percolation, we show that the mutual information and the mutual negativity are governed by boundary conformal fields of different scaling dimensions at long distances. We then consider a class of "hybrid" circuit models obtained by perturbing the measurement-only circuit with unitary gates of progressive levels of complexity. While other critical exponents vary appreciably for different choices of unitary gate ensembles at their respective critical points, the mutual negativity has scaling dimension 3 across remarkably many of the hybrid circuits, which is notably different from that in percolation. We contrast our results with limiting cases where a geometrical minimal-cut picture is available.

Measure transport underpins several recent algorithms for posterior approximation in the Bayesian context, wherein a transport map is sought to minimise the Kullback--Leibler divergence (KLD) from the posterior to the approximation. The KLD is a strong mode of convergence, requiring absolute continuity of measures and placing restrictions on which transport maps can be permitted. Here we propose to minimise a kernel Stein discrepancy (KSD) instead, requiring only that the set of transport maps is dense in an $L^2$ sense and demonstrating how this condition can be validated. The consistency of the associated posterior approximation is established and empirical results suggest that KSD is competitive and more flexible alternative to KLD for measure transport.

We introduce COALESCE, the first data-driven framework for component-based shape assembly which employs deep learning to synthesize part connections. To handle geometric and topological mismatches between parts, we remove the mismatched portions via erosion, and rely on a joint synthesis step, which is learned from data, to fill the gap and arrive at a natural and plausible part joint. Given a set of input parts extracted from different objects, COALESCE automatically aligns them and synthesizes plausible joints to connect the parts into a coherent 3D object represented by a mesh. The joint synthesis network, designed to focus on joint regions, reconstructs the surface between the parts by predicting an implicit shape representation that agrees with existing parts, while generating a smooth and topologically meaningful connection. We employ test-time optimization to further ensure that the synthesized joint region closely aligns with the input parts to create realistic component assemblies from diverse input parts. We demonstrate that our method significantly outperforms prior approaches including baseline deep models for 3D shape synthesis, as well as state-of-the-art methods for shape completion., Comment: 20 pages: paper + supplementary

We propose a novel neural architecture for representing 3D surfaces, which harnesses two complementary shape representations: (i) an explicit representation via an atlas, i.e., embeddings of 2D domains into 3D; (ii) an implicit-function representation, i.e., a scalar function over the 3D volume, with its levels denoting surfaces. We make these two representations synergistic by introducing novel consistency losses that ensure that the surface created from the atlas aligns with the level-set of the implicit function. Our hybrid architecture outputs results which are superior to the output of the two equivalent single-representation networks, yielding smoother explicit surfaces with more accurate normals, and a more accurate implicit occupancy function. Additionally, our surface reconstruction step can directly leverage the explicit atlas-based representation. This process is computationally efficient, and can be directly used by differentiable rasterizers, enabling training our hybrid representation with image-based losses., Comment: ECCV 2020

People often create art by following an artistic workflow involving multiple stages that inform the overall design. If an artist wishes to modify an earlier decision, significant work may be required to propagate this new decision forward to the final artwork. Motivated by the above observations, we propose a generative model that follows a given artistic workflow, enabling both multi-stage image generation as well as multi-stage image editing of an existing piece of art. Furthermore, for the editing scenario, we introduce an optimization process along with learning-based regularization to ensure the edited image produced by the model closely aligns with the originally provided image. Qualitative and quantitative results on three different artistic datasets demonstrate the effectiveness of the proposed framework on both image generation and editing tasks., Comment: ECCV 2020. Code: https://github.com/hytseng0509/ArtEditing

We study stabilizer quantum error correcting codes (QECC) generated under hybrid dynamics of local Clifford unitaries and local Pauli measurements in one dimension. Building upon 1) a general formula relating the error-susceptibility of a subregion to its entanglement properties, and 2) a previously established mapping between entanglement entropies and domain wall free energies of an underlying spin model, we propose a statistical mechanical description of the QECC in terms of "entanglement domain walls". Free energies of such domain walls generically feature a leading volume law term coming from its "surface energy", and a sub-volume law correction coming from thermodynamic entropies of its transverse fluctuations. These are most easily accounted for by capillary-wave theory of liquid-gas interfaces, which we use as an illustrative tool. We show that the information-theoretic decoupling criterion corresponds to a geometric decoupling of domain walls, which further leads to the identification of the "contiguous code distance" of the QECC as the crossover length scale at which the energy and entropy of the domain wall are comparable. The contiguous code distance thus diverges with the system size as the subleading entropic term of the free energy, protecting a finite code rate against local undetectable errors. We support these correspondences with numerical evidence, where we find capillary-wave theory describes many qualitative features of the QECC; we also discuss when and why it fails to do so., Comment: 22 pages, 8 figures. v2: fixed .bbl file, new footnote 12. v3: updated references and acknowledgements. v4: fixed typos; published version

We present random quantum circuit models for non-unitary quantum dynamics of free fermions in one spatial dimension. Numerical simulations reveal that the dynamics tends towards steady states with logarithmic violations of the entanglement area law and power law correlation functions. Moreover, starting with a short-range entangled many-body state, the dynamical evolution of entanglement and correlations quantitatively agrees with the predictions of two-dimensional conformal field theory with a space-like time direction. We argue that this behavior is generic in non-unitary free quantum dynamics with time-dependent randomness, and show that the emergent conformal dynamics of two-point functions arises out of a simple "nonlinear master equation".

We establish the emergence of a conformal field theory (CFT) in a (1+1)-dimensional hybrid quantum circuit right at the measurement-driven entanglement transition by revealing space-time conformal covariance of entanglement entropies and mutual information for various subregions at different circuit depths. While the evolution takes place in real time, the spacetime manifold of the circuit appears to host a Euclidean field theory with imaginary time. Throughout the paper we investigate Clifford circuits with several different boundary conditions by injecting physical qubits at the spatial and/or temporal boundaries, all giving consistent characterizations of the underlying "Clifford CFT." We emphasize (super)universal results that are consequences solely of the conformal invariance and do not depend crucially on the precise nature of the CFT. Among these are the infinite entangling speed as a consequence of measurement-induced quantum nonlocality and the critical purification dynamics of a mixed initial state., Comment: 31 pages, 16 figures. v2: updated references and acknowledgements. v3: new Appendix on "localizable entanglement". Published version

Recently there has been an interest in the potential of learning generative models from a single image, as opposed to from a large dataset. This task is of practical significance, as it means that generative models can be used in domains where collecting a large dataset is not feasible. However, training a model capable of generating realistic images from only a single sample is a difficult problem. In this work, we conduct a number of experiments to understand the challenges of training these methods and propose some best practices that we found allowed us to generate improved results over previous work in this space. One key piece is that unlike prior single image generation methods, we concurrently train several stages in a sequential multi-stage manner, allowing us to learn models with fewer stages of increasing image resolution. Compared to a recent state of the art baseline, our model is up to six times faster to train, has fewer parameters, and can better capture the global structure of images., Comment: WACV 2021. Code and supplementary material available at https://github.com/tohinz/ConSinGAN

Bayesian cubature (BC) is a popular inferential perspective on the cubature of expensive integrands, wherein the integrand is emulated using a stochastic process model. Several approaches have been put forward to encode sequential adaptation (i.e. dependence on previous integrand evaluations) into this framework. However, these proposals have been limited to either estimating the parameters of a stationary covariance model or focusing computational resources on regions where large values are taken by the integrand. In contrast, many classical adaptive cubature methods focus computational resources on spatial regions in which local error estimates are largest. The contributions of this work are three-fold: First, we present a theoretical result that suggests there does not exist a direct Bayesian analogue of the classical adaptive trapezoidal method. Then we put forward a novel BC method that has empirically similar behaviour to the adaptive trapezoidal method. Finally we present evidence that the novel method provides improved cubature performance, relative to standard BC, in a detailed empirical assessment.

We introduce UprightNet, a learning-based approach for estimating 2DoF camera orientation from a single RGB image of an indoor scene. Unlike recent methods that leverage deep learning to perform black-box regression from image to orientation parameters, we propose an end-to-end framework that incorporates explicit geometric reasoning. In particular, we design a network that predicts two representations of scene geometry, in both the local camera and global reference coordinate systems, and solves for the camera orientation as the rotation that best aligns these two predictions via a differentiable least squares module. This network can be trained end-to-end, and can be supervised with both ground truth camera poses and intermediate representations of surface geometry. We evaluate UprightNet on the single-image camera orientation task on synthetic and real datasets, and show significant improvements over prior state-of-the-art approaches.

We propose to represent shapes as the deformation and combination of learnable elementary 3D structures, which are primitives resulting from training over a collection of shape. We demonstrate that the learned elementary 3D structures lead to clear improvements in 3D shape generation and matching. More precisely, we present two complementary approaches for learning elementary structures: (i) patch deformation learning and (ii) point translation learning. Both approaches can be extended to abstract structures of higher dimensions for improved results. We evaluate our method on two tasks: reconstructing ShapeNet objects and estimating dense correspondences between human scans (FAUST inter challenge). We show 16% improvement over surface deformation approaches for shape reconstruction and outperform FAUST inter challenge state of the art by 6%.

We propose a self-supervised approach to deep surface deformation. Given a pair of shapes, our algorithm directly predicts a parametric transformation from one shape to the other respecting correspondences. Our insight is to use cycle-consistency to define a notion of good correspondences in groups of objects and use it as a supervisory signal to train our network. Our method does not rely on a template, assume near isometric deformations or rely on point-correspondence supervision. We demonstrate the efficacy of our approach by using it to transfer segmentation across shapes. We show, on Shapenet, that our approach is competitive with comparable state-of-the-art methods when annotated training data is readily available, but outperforms them by a large margin in the few-shot segmentation scenario.

We introduce a procedure to systematically search for a local unitary transformation that maps a wavefunction with a non-trivial sign structure into a positive-real form. The transformation is parametrized as a quantum circuit compiled into a set of one and two qubit gates. We design a cost function that maximizes the average sign of the output state and removes its complex phases. The optimization of the gates is performed through automatic differentiation algorithms, widely used in the machine learning community. We provide numerical evidence for significant improvements in the average sign for a two-leg triangular Heisenberg ladder with next-to-nearest neighbour and ring-exchange interactions. This model exhibits phases where the sign structure can be removed by simple local one-qubit unitaries, but also an exotic Bose-metal phase whose sign structure induces "Bose surfaces" with a fermionic character and a higher entanglement that requires deeper circuits., Comment: 9 pages, 5 figures

Many tasks in graphics and vision demand machinery for converting shapes into consistent representations with sparse sets of parameters; these representations facilitate rendering, editing, and storage. When the source data is noisy or ambiguous, however, artists and engineers often manually construct such representations, a tedious and potentially time-consuming process. While advances in deep learning have been successfully applied to noisy geometric data, the task of generating parametric shapes has so far been difficult for these methods. Hence, we propose a new framework for predicting parametric shape primitives using deep learning. We use distance fields to transition between shape parameters like control points and input data on a pixel grid. We demonstrate efficacy on 2D and 3D tasks, including font vectorization and surface abstraction., Comment: Accepted to CVPR 2020

We treat shape co-segmentation as a representation learning problem and introduce BAE-NET, a branched autoencoder network, for the task. The unsupervised BAE-NET is trained with a collection of un-segmented shapes, using a shape reconstruction loss, without any ground-truth labels. Specifically, the network takes an input shape and encodes it using a convolutional neural network, whereas the decoder concatenates the resulting feature code with a point coordinate and outputs a value indicating whether the point is inside/outside the shape. Importantly, the decoder is branched: each branch learns a compact representation for one commonly recurring part of the shape collection, e.g., airplane wings. By complementing the shape reconstruction loss with a label loss, BAE-NET is easily tuned for one-shot learning. We show unsupervised, weakly supervised, and one-shot learning results by BAE-NET, demonstrating that using only a couple of exemplars, our network can generally outperform state-of-the-art supervised methods trained on hundreds of segmented shapes. Code is available at https://github.com/czq142857/BAE-NET., Comment: Accepted to ICCV 2019. Code: https://github.com/czq142857/BAE-NET Supplementary material: https://www.sfu.ca/~zhiqinc/imseg/sup.pdf