Search

We study rational agents with different perception capabilities in strategic games. We focus on a class of one-shot limited-perception games. These games extend simultaneous-move normal-form games by presenting each player with an individualized perception of all players' payoff functions. The accuracy of a player's perception is determined by the player's capability level. Capability levels are countable and totally ordered, with a higher level corresponding to a more accurate perception. We study the rational behavior of players in these games and formalize relevant equilibria conditions. In contrast to equilibria in conventional bimatrix games, which can be represented by a pair of mixed strategies, in our limited perception games a higher-order response function captures how the lower-capability player uses their (less accurate) perception of the payoff function to reason about the (more accurate) possible perceptions of the higher-capability opponent. This response function characterizes, for each possible perception of the higher-capability player (from the perspective of the lower-capability player), the best response of the higher capability player for that perception. Since the domain of the response function can be exponentially large or even infinite, finding one equilibrium may be computationally intractable or even undecidable. Nevertheless, we show that for any $\epsilon$, there exists an $\epsilon$-equilibrium with a compact, tractable representation whose size is independent of the size of the response function's domain. We further identify classes of zero-sum limited-perception games in which finding an equilibrium becomes a (typically tractable) nonsmooth convex optimization problem.

The interplay of three light-matter interaction processes - rotating and counter-rotating interactions and two-photon relaxation of the light field - is a topic of interest in quantum optics and quantum information processing. In this work, we theoretically investigate the three light-matter interaction processes using the anisotropic Rabi model, which accounts for different strengths of rotating and counter-rotating interactions and the unique occurrence of photon escape exclusively in pairs. By numerically solving the Lindblad master equation, we analyze the excitation-relaxation dynamics and derive a non-Hermitian effective Hamiltonian to gain further physical insights. To explore the individual effects of these interactions, we examine three analytically tractable limits of the effective Hamiltonian. Our analysis reveals that the three competitive light-matter interaction processes exhibit sensitivity to parity, leading to intriguing phenomena in both transient and steady states. Particularly interesting dynamical patterns resembling quantum phase transitions emerge when these three interaction terms compete. This work deepens the understanding of ultrastrong light-matter interaction in open quantum systems and offers valuable insights into cavity-based quantum computations., Comment: 13 pages, 8 figures

We present the Trust Region Adversarial Functional Subdifferential (TRAFS) algorithm for constrained optimization of nonsmooth convex Lipschitz functions. Unlike previous methods that assume a subgradient oracle model, we work with the functional subdifferential defined as a set of subgradients that simultaneously captures sufficient local information for effective minimization while being easy to compute for a wide range of functions. In each iteration, TRAFS finds the best step vector in an $\ell_2$-bounded trust region by considering the worst bound given by the functional subdifferential. TRAFS finds an approximate solution with an absolute error up to $\epsilon$ in $\mathcal{O}\left( \epsilon^{-1}\right)$ or $\mathcal{O}\left(\epsilon^{-0.5} \right)$ iterations depending on whether the objective function is strongly convex, compared to the previously best-known bounds of $\mathcal{O}\left(\epsilon^{-2}\right)$ and $\mathcal{O}\left(\epsilon^{-1}\right)$ in these settings. TRAFS makes faster progress if the functional subdifferential satisfies a locally quadratic property; as a corollary, TRAFS achieves linear convergence (i.e., $\mathcal{O}\left(\log \epsilon^{-1}\right)$) for strongly convex smooth functions. In the numerical experiments, TRAFS is on average 39.1x faster and solves twice as many problems compared to the second-best method.

We take a formal approach to the explainability problem of machine learning systems. We argue against the practice of interpreting black-box models via attributing scores to input components due to inherently conflicting goals of attribution-based interpretation. We prove that no attribution algorithm satisfies specificity, additivity, completeness, and baseline invariance. We then formalize the concept, sound explanation, that has been informally adopted in prior work. A sound explanation entails providing sufficient information to causally explain the predictions made by a system. Finally, we present the application of feature selection as a sound explanation for cancer prediction models to cultivate trust among clinicians.

$L_0$ regularization of neural networks is a fundamental problem. In addition to regularizing models for better generalizability, $L_0$ regularization also applies to selecting input features and training sparse neural networks. There is a large body of research on related topics, some with quite complicated methods. In this paper, we show that a straightforward formulation, BinMask, which multiplies weights with deterministic binary masks and uses the identity straight-through estimator for backpropagation, is an effective $L_0$ regularizer. We evaluate BinMask on three tasks: feature selection, network sparsification, and model regularization. Despite its simplicity, BinMask achieves competitive performance on all the benchmarks without task-specific tuning compared to methods designed for each task. Our results suggest that decoupling weights from mask optimization, which has been widely adopted by previous work, is a key component for effective $L_0$ regularization.

Regression-based methods have shown high efficiency and effectiveness for multi-view human mesh recovery. The key components of a typical regressor lie in the feature extraction of input views and the fusion of multi-view features. In this paper, we present Pixel-aligned Feedback Fusion (PaFF) for accurate yet efficient human mesh recovery from multi-view images. PaFF is an iterative regression framework that performs feature extraction and fusion alternately. At each iteration, PaFF extracts pixel-aligned feedback features from each input view according to the reprojection of the current estimation and fuses them together with respect to each vertex of the downsampled mesh. In this way, our regressor can not only perceive the misalignment status of each view from the feedback features but also correct the mesh parameters more effectively based on the feature fusion on mesh vertices. Additionally, our regressor disentangles the global orientation and translation of the body mesh from the estimation of mesh parameters such that the camera parameters of input views can be better utilized in the regression process. The efficacy of our method is validated in the Human3.6M dataset via comprehensive ablation experiments, where PaFF achieves 33.02 MPJPE and brings significant improvements over the previous best solutions by more than 29%. The project page with code and video results can be found at https://kairobo.github.io/PaFF/., Comment: Project Page: https://kairobo.github.io/PaFF/

In this work, we explore the PT-symmetric quantum Rabi model, which describes a PT-symmetric qubit coupled to a quantized light field. By employing the adiabatic approximation (AA), we are able to solve this model analytically in the parameter regime of interest and analyze various physical aspects. We investigate the static and dynamic properties of the model, using both the AA and numerical diagonalization. Our analysis reveals a multitude of exceptional points (EPs) that are closely connected with the exactly solvable points in the Hermitian counterpart of the model. Intriguingly, these EPs vanish and revive depending on the light-matter coupling strength. Furthermore, we discuss the time evolution of physical observables under the non-Hermitian Hamiltonian. Rich and exotic behaviors are observed in both strong and ultra-strong coupling regimes. Our work extends the theory of PT symmetry into the full quantum light-matter interaction regime and provides insights that can be readily enlarged to a broad class of quantum optical systems.

We present a new class of strategic games, mixed capability games, as a foundation for studying how different player capabilities impact the dynamics and outcomes of strategic games. We analyze the impact of different player capabilities via a capability transfer function that characterizes the payoff of each player at equilibrium given capabilities for all players in the game. In this paper, we model a player's capability as the size of the strategy space available to that player. We analyze a mixed capability variant of the Gold and Mines Game recently proposed by Yang et al. and derive its capability transfer function in closed form.

We study the impact of player capability on social welfare in congestion games. We introduce a new game, the Distance-bounded Network Congestion game (DNC), as the basis of our study. DNC is a symmetric network congestion game with a bound on the number of edges each player can use. We show that DNC is PLS-complete in contrast to standard symmetric network congestion games which are in P. To model different player capabilities, we propose using programs in a Domain-Specific Language (DSL) to compactly represent player strategies. We define a player's capability as the maximum size of the programs they can use. We introduce two variants of DNC with accompanying DSLs representing the strategy spaces. We propose four capability preference properties to characterize the impact of player capability on social welfare at equilibrium. We then establish necessary and sufficient conditions for the four properties in the context of our DNC variants. Finally, we study a specific game where we derive exact expressions of the social welfare in terms of the capability bound. This provides examples where the social welfare at equilibrium increases, stays the same, or decreases as players become more capable.

This paper presents a method called sampling-computation-optimization (SCO) to design batch Bayesian optimization. SCO does not construct new high-dimensional acquisition functions but samples from the existing one-site acquisition function to obtain several candidate samples. To reduce the uncertainty of the sampling, the general discrepancy is computed to compare these samples. Finally, the genetic algorithm and switch algorithm are used to optimize the design. Several strategies are used to reduce the computational burden in the SCO. From the numerical results, the SCO designs were less uncertain than those of other sampling-based methods. As for application in batch Bayesian optimization, SCO can find a better solution when compared with other batch methods in the same dimension and batch size. In addition, it is also flexible and can be adapted to different one-site methods. Finally, a complex experimental case is given to illustrate the application value and scenario of SCO method.

Deep neural networks are an attractive tool for compressing the control policy lookup tables in systems such as the Airborne Collision Avoidance System (ACAS). It is vital to ensure the safety of such neural controllers via verification techniques. The problem of analyzing ACAS Xu networks has motivated many successful neural network verifiers. These verifiers typically analyze the internal computation of neural networks to decide whether a property regarding the input/output holds. The intrinsic complexity of neural network computation renders such verifiers slow to run and vulnerable to floating-point error. This paper revisits the original problem of verifying ACAS Xu networks. The networks take low-dimensional sensory inputs with training data provided by a precomputed lookup table. We propose to prepend an input quantization layer to the network. Quantization allows efficient verification via input state enumeration, whose complexity is bounded by the size of the quantization space. Quantization is equivalent to nearest-neighbor interpolation at run time, which has been shown to provide acceptable accuracy for ACAS in simulation. Moreover, our technique can deliver exact verification results immune to floating-point error if we directly enumerate the network outputs on the target inference implementation or on an accurate simulation of the target implementation., Comment: SAS 2021

Solving nonlinear systems is an important problem. Numerical continuation methods efficiently solve certain nonlinear systems. The Asymptotic Numerical Method (ANM) is a powerful continuation method that usually converges faster than Newtonian methods. ANM explores the landscape of the function by following a parameterized solution curve approximated with a high-order power series. Although ANM has successfully solved a few graphics and engineering problems, prior to our work, applying ANM to new problems required significant effort because the standard ANM assumes quadratic functions, while manually deriving the power series expansion for nonquadratic systems is a tedious and challenging task. This paper presents a novel solver, SANM, that applies ANM to solve symbolically represented nonlinear systems. SANM solves such systems in a fully automated manner. SANM also extends ANM to support many nonquadratic operators, including intricate ones such as singular value decomposition. Furthermore, SANM generalizes ANM to support the implicit homotopy form. Moreover, SANM achieves high computing performance via optimized system design and implementation. We deploy SANM to solve forward and inverse elastic force equilibrium problems and controlled mesh deformation problems with a few constitutive models. Our results show that SANM converges faster than Newtonian solvers, requires little programming effort for new problems, and delivers comparable or better performance than a hand-coded, specialized ANM solver. While we demonstrate on mesh deformation problems, SANM is generic and potentially applicable to many tasks., Comment: SIGGRAPH 2021

In order to study the Yamabe changing-sign problem, Bahri and Xu proposed a conjecture which is a universal inequality for $p$ points in $\mathbb R^m$. They have verified the conjecture for $p\leq3$. In this paper, we first simplify this conjecture by giving two sufficient and necessary conditions inductively. Then we prove the conjecture for the basic case $m=1$ with arbitrary $p$. In addition, for the cases when $p=4,5$ and $m\geq2$, we manage to reduce them to the basic case $m=1$ and thus prove them as well., Comment: Accepted by Acta Math. Sin. (Engl. Ser.)

One of the core problems in large-scale recommendations is to retrieve top relevant candidates accurately and efficiently, preferably in sub-linear time. Previous approaches are mostly based on a two-step procedure: first learn an inner-product model, and then use some approximate nearest neighbor (ANN) search algorithm to find top candidates. In this paper, we present Deep Retrieval (DR), to learn a retrievable structure directly with user-item interaction data (e.g. clicks) without resorting to the Euclidean space assumption in ANN algorithms. DR's structure encodes all candidate items into a discrete latent space. Those latent codes for the candidates are model parameters and learnt together with other neural network parameters to maximize the same objective function. With the model learnt, a beam search over the structure is performed to retrieve the top candidates for reranking. Empirically, we first demonstrate that DR, with sub-linear computational complexity, can achieve almost the same accuracy as the brute-force baseline on two public datasets. Moreover, we show that, in a live production recommendation system, a deployed DR approach significantly outperforms a well-tuned ANN baseline in terms of engagement metrics. To the best of our knowledge, DR is among the first non-ANN algorithms successfully deployed at the scale of hundreds of millions of items for industrial recommendation systems., Comment: 9 pages, 6 figures

Concerned with the reliability of neural networks, researchers have developed verification techniques to prove their robustness. Most verifiers work with real-valued networks. Unfortunately, the exact (complete and sound) verifiers face scalability challenges and provide no correctness guarantees due to floating point errors. We argue that Binarized Neural Networks (BNNs) provide comparable robustness and allow exact and significantly more efficient verification. We present a new system, EEV, for efficient and exact verification of BNNs. EEV consists of two parts: (i) a novel SAT solver that speeds up BNN verification by natively handling the reified cardinality constraints arising in BNN encodings; and (ii) strategies to train solver-friendly robust BNNs by inducing balanced layer-wise sparsity and low cardinality bounds, and adaptively cancelling the gradients. We demonstrate the effectiveness of EEV by presenting the first exact verification results for L-inf-bounded adversarial robustness of nontrivial convolutional BNNs on the MNIST and CIFAR10 datasets. Compared to exact verification of real-valued networks of the same architectures on the same tasks, EEV verifies BNNs hundreds to thousands of times faster, while delivering comparable verifiable accuracy in most cases., Comment: To be published in NeurIPS 2020

Researchers have developed neural network verification algorithms motivated by the need to characterize the robustness of deep neural networks. The verifiers aspire to answer whether a neural network guarantees certain properties with respect to all inputs in a space. However, many verifiers inaccurately model floating point arithmetic but do not thoroughly discuss the consequences. We show that the negligence of floating point error leads to unsound verification that can be systematically exploited in practice. For a pretrained neural network, we present a method that efficiently searches inputs as witnesses for the incorrectness of robustness claims made by a complete verifier. We also present a method to construct neural network architectures and weights that induce wrong results of an incomplete verifier. Our results highlight that, to achieve practically reliable verification of neural networks, any verification system must accurately (or conservatively) model the effects of any floating point computations in the network inference or verification system., Comment: SAS 2021

Sliced Latin hypercube designs (SLHDs) are widely used in computer experiments with both quantitative and qualitative factors and in batches. Optimal SLHDs achieve better space-filling property on the whole experimental region. However, most existing methods for constructing optimal SLHDs have restriction on the run sizes. In this paper, we propose a new method for constructing SLHDs with arbitrary run sizes, and a new combined space-filling measurement describing the space-filling property for both the whole design and its slices. Furthermore, we develop general algorithms to search the optimal SLHD with arbitrary run sizes under the proposed measurement. Examples are presented to illustrate that effectiveness of the proposed methods.

Dimensionality reduction (DR) methods are commonly used for analyzing and visualizing multidimensional data. However, when data is a live streaming feed, conventional DR methods cannot be directly used because of their computational complexity and inability to preserve the projected data positions at previous time points. In addition, the problem becomes even more challenging when the dynamic data records have a varying number of dimensions as often found in real-world applications. This paper presents an incremental DR solution. We enhance an existing incremental PCA method in several ways to ensure its usability for visualizing streaming multidimensional data. First, we use geometric transformation and animation methods to help preserve a viewer's mental map when visualizing the incremental results. Second, to handle data dimension variants, we use an optimization method to estimate the projected data positions, and also convey the resulting uncertainty in the visualization. We demonstrate the effectiveness of our design with two case studies using real-world datasets., Comment: This is the author's version of the article that has been published in IEEE Transactions on Visualization and Computer Graphics. The final version of this record is available at: 10.1109/TVCG.2019.2934433

The improvements in recent CNN-based object detection works, from R-CNN [11], Fast/Faster R-CNN [10, 31] to recent Mask R-CNN [14] and RetinaNet [24], mainly come from new network, new framework, or novel loss design. But mini-batch size, a key factor in the training, has not been well studied. In this paper, we propose a Large MiniBatch Object Detector (MegDet) to enable the training with much larger mini-batch size than before (e.g. from 16 to 256), so that we can effectively utilize multiple GPUs (up to 128 in our experiments) to significantly shorten the training time. Technically, we suggest a learning rate policy and Cross-GPU Batch Normalization, which together allow us to successfully train a large mini-batch detector in much less time (e.g., from 33 hours to 4 hours), and achieve even better accuracy. The MegDet is the backbone of our submission (mmAP 52.5%) to COCO 2017 Challenge, where we won the 1st place of Detection task.

Theano is a Python library that allows to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements. Theano is being actively and continuously developed since 2008, multiple frameworks have been built on top of it and it has been used to produce many state-of-the-art machine learning models. The present article is structured as follows. Section I provides an overview of the Theano software and its community. Section II presents the principal features of Theano and how to use them, and compares them with other similar projects. Section III focuses on recently-introduced functionalities and improvements. Section IV compares the performance of Theano against Torch7 and TensorFlow on several machine learning models. Section V discusses current limitations of Theano and potential ways of improving it., Comment: 19 pages, 5 figures