Search

Periodically driven systems offer a perfect breeding ground for out-of-equilibrium engineering of topological boundary states at zero energy ($0$-mode), as well as finite energy ($\pi$-mode), with the latter having no static analog. The Floquet operator and the effective Floquet Hamiltonian, which encapsulate the stroboscopic features of the driven system, capture both spectral and localization properties of the $0$- and $\pi$-modes but sometimes fail to provide complete topological characterization, especially when $0$- and $\pi$-modes coexist. In this work, we utilize the spectral localizer, a powerful local probe that can provide numerically efficient, spatially local, and energy-resolved topological characterization. In particular, we apply the spectral localizer to the effective Floquet Hamiltonian for driven one- and two-dimensional topological systems with no or limited symmetries and are able to assign topological invariants, or local markers, that characterize the $0$- and the $\pi$-boundary modes individually and unambiguously. Due to the spatial resolution, we also demonstrate that the extracted topological invariants are suitable for studying driven disordered systems and can even capture disorder-induced phase transitions., Comment: 11 pages, 6 figures; Comments are welcome

In the ever-evolving landscape of Deep Neural Networks (DNN) hardware acceleration, unlocking the true potential of systolic array accelerators has long been hindered by the daunting challenges of expertise and time investment. Large Language Models (LLMs) offer a promising solution for automating code generation which is key to unlocking unprecedented efficiency and performance in various domains, including hardware descriptive code. The generative power of LLMs can enable the effective utilization of preexisting designs and dedicated hardware generators. However, the successful application of LLMs to hardware accelerator design is contingent upon the availability of specialized datasets tailored for this purpose. To bridge this gap, we introduce the Systolic Array-based Accelerator Data Set (SA-DS). SA-DS comprises a diverse collection of spatial array designs following the standardized Berkeley's Gemmini accelerator generator template, enabling design reuse, adaptation, and customization. SA-DS is intended to spark LLM-centered research on DNN hardware accelerator architecture. We envision that SA-DS provides a framework that will shape the course of DNN hardware acceleration research for generations to come. SA-DS is open-sourced under the permissive MIT license at https://github.com/ACADLab/SA-DS.git}{https://github.com/ACADLab/SA-DS., Comment: 4 pages, 5 Figures

We consider non-Hermitian (NH) analog of a second-order topological insulator, protected by chiral symmetry, in the presence of second-nearest neighbor hopping elements to theoretically investigate the interplay between long-range and topological order away from Hermiticity. In addition to the four zero-energy corner modes present in the first nearest neighbor hopping model, we uncover that the second nearest neighbor hopping introduces another topological phase with sixteen zero-energy corner modes. Importantly, the NH effects are manifested in altering the Hermitian phase boundaries for both the models. While comparing the complex energy spectrum under open boundary conditions, and bi-orthogonalized quadrupolar winding number (QWN) in real space, we resolve the apparent anomaly in the bulk boundary correspondence of the NH system as compared to the Hermitian counterpart by incorporating the effect of non-Bloch form of momentum into the mass term. The above invariant is also capable of capturing the phase boundaries between the two different topological phases where the degeneracy of the corner modes is evident, as exclusively observed for the second nearest neighbor model., Comment: 9 pages, 5 figures

Condensed matter systems are continuously subjected to dissipation, which often has adverse effects on quantum phenomena. We focus on the impact of dissipation on a superconducting Rashba nanowire. We reveal that the system can still host Majorana zero-modes (MZMs) with a finite lifetime in the presence of dissipation. Most interestingly, dissipation can also generate two kinds of dissipative boundary states: four robust zero-modes (RZMs) and two MZMs, in the regime where the non-dissipative system is topologically trivial. The MZMs appear via bulk gap closing and are topologically characterized by a winding number. The RZMs are not associated with any bulk states and possess no winding number, but their emergence is instead tied to exceptional points. Further, we confirm the stability of the dissipation-induced RZMs and MZMs in the presence of random disorder. Our study paves the way for both realizing and stabilizing MZMs in an experimental setup, driven by dissipation.

We propose a Safe Adversarial Trained Actor Critic (SATAC) algorithm for offline reinforcement learning (RL) with general function approximation in the presence of limited data coverage. SATAC operates as a two-player Stackelberg game featuring a refined objective function. The actor (leader player) optimizes the policy against two adversarially trained value critics (follower players), who focus on scenarios where the actor's performance is inferior to the behavior policy. Our framework provides both theoretical guarantees and a robust deep-RL implementation. Theoretically, we demonstrate that when the actor employs a no-regret optimization oracle, SATAC achieves two guarantees: (i) For the first time in the offline RL setting, we establish that SATAC can produce a policy that outperforms the behavior policy while maintaining the same level of safety, which is critical to designing an algorithm for offline RL. (ii) We demonstrate that the algorithm guarantees policy improvement across a broad range of hyperparameters, indicating its practical robustness. Additionally, we offer a practical version of SATAC and compare it with existing state-of-the-art offline safe-RL algorithms in continuous control environments. SATAC outperforms all baselines across a range of tasks, thus validating the theoretical performance.

The presence of random disorder in a metallic system accounts for the localization of extended states in general. On the contrary, the presence of disorder can induce topological phases hosting metallic boundary states out of a non-topological system, giving birth to the topological Anderson insulator phase. In this context, we theoretically investigate the generation of an out of equilibrium higher-order topological Anderson phase in the presence of disorder potential in a time-periodic dynamical background. In particular, the time-dependent drive and the disorder potential concomitantly render the generation of Floquet higher-order topological Anderson insulator~(FHOTAI) phase, while the clean, undriven system is topologically trivial. We showcase the generation of FHOTAI hosting both $0$- and $\pi$-modes. Most importantly, we develop the real space topological invariant -- a winding number based on chiral symmetry to characterize the Floquet $0$ -- and $\pi$-modes distinctly. This chiral winding number serves the purpose of the indicator for the topological phase transition in the presence of drive as well as disorder and appropriately characterizes the FHOTAI., Comment: 4.5 Pages, 4 PDF Figures (main text) + 5 Pages, 4 PDF Figures (Supplementary Material), Comments are welcome

We put forth a theoretical framework for engineering a two-dimensional (2D) second-order topological superconductor (SOTSC) by utilizing a heterostructure: incorporating noncollinear magnetic textures between an $s$-wave superconductor and a 2D quantum spin Hall insulator. It stabilizes the higher order topological superconducting phase, resulting in Majorana corner modes (MCMs) at four corners of a 2D domain. The calculated non-zero quadrupole moment characterizes the bulk topology. Subsequently, through a unitary transformation, an effective low-energy Hamiltonian reveals the effects of magnetic textures, resulting in an effective in-plane Zeeman field and spin-orbit coupling. This approach provides a qualitative depiction of the topological phase, substantiated by numerical validation within exact real-space model. Analytically calculated effective pairings in the bulk illuminate the microscopic behavior of the SOTSC. The comprehension of MCM emergence is supported by a low-energy edge theory, which is attributed to the interplay between effective pairings of $(p_x + p_y)$-type and $(p_x + i p_y)$-type. Our extensive study paves the way for practically attaining the SOTSC phase by integrating noncollinear magnetic textures., Comment: This is the published version

Fairness plays a crucial role in various multi-agent systems (e.g., communication networks, financial markets, etc.). Many multi-agent dynamical interactions can be cast as Markov Decision Processes (MDPs). While existing research has focused on studying fairness in known environments, the exploration of fairness in such systems for unknown environments remains open. In this paper, we propose a Reinforcement Learning (RL) approach to achieve fairness in multi-agent finite-horizon episodic MDPs. Instead of maximizing the sum of individual agents' value functions, we introduce a fairness function that ensures equitable rewards across agents. Since the classical Bellman's equation does not hold when the sum of individual value functions is not maximized, we cannot use traditional approaches. Instead, in order to explore, we maintain a confidence bound of the unknown environment and then propose an online convex optimization based approach to obtain a policy constrained to this confidence region. We show that such an approach achieves sub-linear regret in terms of the number of episodes. Additionally, we provide a probably approximately correct (PAC) guarantee based on the obtained regret bound. We also propose an offline RL algorithm and bound the optimality gap with respect to the optimal fair solution. To mitigate computational complexity, we introduce a policy-gradient type method for the fair objective. Simulation experiments also demonstrate the efficacy of our approach.

We theoretically explore the Floquet generation of Majorana end modes~(MEMs) (both regular $0$- and anomalous $\pi$-modes) implementing a periodic sinusoidal modulation in chemical potential in an experimentally feasible setup based on a one-dimensional chain of magnetic impurity atoms having spin spiral configuration (out-of-plane N\'eel-type) fabricated on the surface of most common bulk $s$-wave superconductor. We obtain a rich phase diagram in the parameter space, highlighting the possibility of generating multiple $0$-/$\pi$-MEMs localized at the end of the chain. We also study the real-time evolution of these emergent MEMs, especially when they start to appear in the time domain. These MEMs are topologically characterized by employing the dynamical winding number. We observe that the existing perturbative analysis is unable to explain the numerical findings, indicating the complex mechanism behind the formation of the Floquet Shiba minigap, which is characteristically distinct from other setup e.g. Rashba nanowire model. We also discuss the possible experimental parameters in connection to our model. Our work paves the way to realize the Floquet MEMs in a magnet-superconductor heterostructure., Comment: This is the published version

We study model-free reinforcement learning (RL) algorithms in episodic non-stationary constrained Markov Decision Processes (CMDPs), in which an agent aims to maximize the expected cumulative reward subject to a cumulative constraint on the expected utility (cost). In the non-stationary environment, reward, utility functions, and transition kernels can vary arbitrarily over time as long as the cumulative variations do not exceed certain variation budgets. We propose the first model-free, simulator-free RL algorithms with sublinear regret and zero constraint violation for non-stationary CMDPs in both tabular and linear function approximation settings with provable performance guarantees. Our results on regret bound and constraint violation for the tabular case match the corresponding best results for stationary CMDPs when the total budget is known. Additionally, we present a general framework for addressing the well-known challenges associated with analyzing non-stationary CMDPs, without requiring prior knowledge of the variation budget. We apply the approach for both tabular and linear approximation settings.

We report on a realistic and rather general scheme where noncollinear magnetic textures proximitized with the most common $s$-wave superconductor can appear as the alternative to $p$-wave superconductor{--}the prime proposal to realize two-dimensional (2D) Kitaev model for topological superconductors (TSCs) hosting Majorana flat edge mode (MFEM). A general minimal Hamiltonian suitable for magnet/superconductor heterostructures reveals robust MFEM within the gap of Shiba bands due to the emergence of an effective ``$p_x+p_y$"-type $p$-wave pairing, spatially localized at the edges of a 2D magnetic domain of spin-spiral. We finally verify this concept by considering Mn (Cr) monolayer grown on a $s$-wave superconducting substrate, Nb(110) under strain (Nb(001)). In both 2D cases, the antiferromagnetic spin-spiral solutions exhibit robust MFEM at certain domain edges that is beyond the scope of the trivial extension of 1D spin-chain model in 2D. This approach, particularly when the MFEM appears in the TSC phase for such heterostructure materials, offers a perspective to extend the realm of the TSC in 2D., Comment: This is the published version

Autonomous vehicles will be an integral part of ride-sharing services in the future. This setting differs from traditional ride-sharing marketplaces because of the absence of the supply side (drivers). However, it has far-reaching consequences because in addition to pricing, players now have to make decisions on how to distribute fleets across network locations and re-balance vehicles in order to serve future demand. In this paper, we explore a duopoly setting in the ride-sharing marketplace where the players have fully autonomous fleets. Each ride-service provider (RSP)'s prices depend on the prices and the supply of the other player. We formulate their decision-making problems using a game-theoretic setup where each player seeks to find the optimal prices and supplies at each node while considering the decisions of the other player. This leads to a scenario where the players' optimization problems are coupled and it is challenging to find the equilibrium. We characterize the types of demand functions (e.g.: linear) for which this game admits an exact potential function and can be solved efficiently. For other types of demand functions, we propose an iterative algorithm to compute the equilibrium. We conclude by providing numerical insights into how different kinds of equilibria would play out in the market when the players are asymmetric. Our numerical evaluations also provide insights into how the regulator needs to consider network effects while deciding regulation in order to avoid unfavorable outcomes., Comment: Submitted to MobiHoc'23

We consider a multi-agent episodic MDP setup where an agent (leader) takes action at each step of the episode followed by another agent (follower). The state evolution and rewards depend on the joint action pair of the leader and the follower. Such type of interactions can find applications in many domains such as smart grids, mechanism design, security, and policymaking. We are interested in how to learn policies for both the players with provable performance guarantee under a bandit feedback setting. We focus on a setup where both the leader and followers are {\em non-myopic}, i.e., they both seek to maximize their rewards over the entire episode and consider a linear MDP which can model continuous state-space which is very common in many RL applications. We propose a {\em model-free} RL algorithm and show that $\tilde{\mathcal{O}}(\sqrt{d^3H^3T})$ regret bounds can be achieved for both the leader and the follower, where $d$ is the dimension of the feature mapping, $H$ is the length of the episode, and $T$ is the total number of steps under the bandit feedback information setup. Thus, our result holds even when the number of states becomes infinite. The algorithm relies on {\em novel} adaptation of the LSVI-UCB algorithm. Specifically, we replace the standard greedy policy (as the best response) with the soft-max policy for both the leader and the follower. This turns out to be key in establishing uniform concentration bound for the value functions. To the best of our knowledge, this is the first sub-linear regret bound guarantee for the Markov games with non-myopic followers with function approximation., Comment: arXiv admin note: text overlap with arXiv:2206.11889

We propose a practically feasible time-periodic sinusoidal drive protocol in onsite mass term to generate the two-dimensional~(2D) Floquet second-order topological superconductor, hosting both the regular $0$- and anomalous $\pi$-Majorana corner modes~(MCMs) while starting from a static 2D topological insulator/$d$-wave superconductor heterostructure setup. We theoretically study the local density spectra and the time dynamics of MCMs in the presence of such drive. The dynamical MCMs are topologically characterized by employing the average quadrupolar motion. Furthermore, we employ the Floquet perturbation theory~(FPT) in the strong driving amplitude limit to provide analytical insight into the problem. We compare our exact (numerical), and the FPT results in terms of the eigenvalue spectra and the time dynamics of the MCMs. We emphasize that the agreement between the exact numerical and the FPT results are more prominent in the higher frequency regime for close to the $0$-quasi-energy mode., Comment: This is the published version

We theoretically investigate a practically realizable Floquet topological superconductor model, based on a one-dimensional Rashba nanowire and proximity induced $s$-wave superconductivity in the presence of a Zeeman field. The driven system hosts regular $0$- and anomalous $\pi$-Majorana end modes~(MEMs). By tuning the chemical potential and the frequency of the drive, we illustrate the generation of multiple MEMs in our theoretical set up. We utilize the chiral symmetry operator to topologically characterize these MEMs via a dynamical winding number constructed out of the periodized evolution operator. Interestingly, the robustness of the $0$- and $\pi$-MEMs is established in the presence of on-site time-independent random disorder potential. We employ the twisted boundary condition to define the dynamical topological invariant for this translational-symmetry broken system. The interplay between the Floquet driving and the weak disorder can stabilize the MEMs giving rise to a quantized value of the dynamical winding number for a finite range of drive parameters. This observation might be experimentally helpful in scrutinizing the topological nature of the Floquet MEMs. We showcase another driving protocol namely, a periodic kick in the chemical potential to study the generation of Floquet MEMs in our setup. Our work paves a realistic way to engineer multiple MEMs in a driven system., Comment: This is the published version

To fully utilize the abundant spectrum resources in millimeter wave (mmWave), Beam Alignment (BA) is necessary for large antenna arrays to achieve large array gains. In practical dynamic wireless environments, channel modeling is challenging due to time-varying and multipath effects. In this paper, we formulate the beam alignment problem as a non-stationary online learning problem with the objective to maximize the received signal strength under interference constraint. In particular, we employ the non-stationary kernelized bandit to leverage the correlation among beams and model the complex beamforming and multipath channel functions. Furthermore, to mitigate interference to other user equipment, we leverage the primal-dual method to design a constrained UCB-type kernelized bandit algorithm. Our theoretical analysis indicates that the proposed algorithm can adaptively adjust the beam in time-varying environments, such that both the cumulative regret of the received signal and constraint violations have sublinear bounds with respect to time. This result is of independent interest for applications such as adaptive pricing and news ranking. In addition, the algorithm assumes the channel is a black-box function and does not require any prior knowledge for dynamic channel modeling, and thus is applicable in a variety of scenarios. We further show that if the information about the channel variation is known, the algorithm will have better theoretical guarantees and performance. Finally, we conduct simulations to highlight the effectiveness of the proposed algorithm.

We study the constrained reinforcement learning problem, in which an agent aims to maximize the expected cumulative reward subject to a constraint on the expected total value of a utility function. In contrast to existing model-based approaches or model-free methods accompanied with a `simulator', we aim to develop the first model-free, simulator-free algorithm that achieves a sublinear regret and a sublinear constraint violation even in large-scale systems. To this end, we consider the episodic constrained Markov decision processes with linear function approximation, where the transition dynamics and the reward function can be represented as a linear function of some known feature mapping. We show that $\tilde{\mathcal{O}}(\sqrt{d^3H^3T})$ regret and $\tilde{\mathcal{O}}(\sqrt{d^3H^3T})$ constraint violation bounds can be achieved, where $d$ is the dimension of the feature mapping, $H$ is the length of the episode, and $T$ is the total number of steps. Our bounds are attained without explicitly estimating the unknown transition model or requiring a simulator, and they depend on the state space only through the dimension of the feature mapping. Hence our bounds hold even when the number of states goes to infinity. Our main results are achieved via novel adaptations of the standard LSVI-UCB algorithms. In particular, we first introduce primal-dual optimization into the LSVI-UCB algorithm to balance between regret and constraint violation. More importantly, we replace the standard greedy selection with respect to the state-action function in LSVI-UCB with a soft-max policy. This turns out to be key in establishing uniform concentration for the constrained case via its approximation-smoothness trade-off. We also show that one can achieve an even zero constraint violation while still maintaining the same order with respect to $T$., Comment: Accepted and Published at the 36th Neural Information Processing Systems (NeurIPS'22). Section J (where different episodes may start from different states) is added in this version

Being motivated by intriguing phenomena such as the breakdown of conventional bulk boundary correspondence and emergence of skin modes in the context of non-Hermitian (NH) topological insulators, we here propose a NH second-order topological superconductor (SOTSC) model that hosts Majorana zero modes (MZMs). Employing the non-Bloch form of NH Hamiltonian, we topologically characterize the above modes by biorthogonal nested polarization and resolve the apparent breakdown of the bulk boundary correspondence. Unlike the Hermitian SOTSC, we notice that the MZMs inhabit only one corner out of four in the two-dimensional NH SOTSC. We extend the static MZMs into the realm of Floquet drive. We find anomalous $\pi$-mode following low-frequency mass-kick in addition to the regular $0$-mode that is usually engineered in a high-frequency regime. We further characterize the regular $0$-mode with biorthogonal Floquet nested polarization. Our proposal is not limited to the $d$-wave superconductivity only and can be realized in the experiment with strongly correlated optical lattice platforms., Comment: Main text: 9 pages and 4 figures, SM: 4 pages and 5 figures

We propose a three-step periodic drive protocol to engineer two-dimensional~(2D) Floquet quadrupole superconductors and three-dimensional~(3D) Floquet octupole superconductors hosting zero-dimensional Majorana corner modes~(MCMs), based on unconventional $d$-wave superconductivity. Remarkably, the driven system conceives four phases with only $0$ MCMs, no MCMs, only anomalous $\pi$ MCMs, and both regular $0$ and anomalous $\pi$ MCMs. To circumvent the subtle issue of characterizing $0$ and $\pi$ MCMs separately, we employ the periodized evolution operator to architect the dynamical invariants, namely quadrupole and octupole motion in 2D and 3D, respectively, that can distinguish different higher order topological phases unambiguously.Our study paves the way for the realization of dynamical quadrupolar and octupolar topological superconductors., Comment: This is the published version

After extensive investigation on the Floquet second-order topological insulator (FSOTI) in two dimension (2D), here we propose two driving schemes to systematically engineer the hierarchy of Floquet first-order topological insulator, FSOTI, and Floquet third-order topological insulator in three dimension (3D). Our driving protocols allow these Floquet phases to showcase regular $0$, anomalous $\pi$, and hybrid $0$-$\pi$-modes in a unified phase diagram, obtained for both 2D and 3D systems, while staring from the lower order topological or non-topological phases. Both the step drive and the mass kick protocols exhibit the analogous structure of the evolution operator around the high symmetry points. These eventually enable us to understand the Floquet phase diagrams analytically and the Floquet higher order modes numerically based on finite size systems. The number of $0$ and $\pi$-modes can be tuned irrespective of the frequency in the step drive scheme while we observe frequency driven topological phase transitions for the mass kick protocol. We topologically characterize some of these higher order Floquet phases (harboring either $0$ or anomalous $\pi$ mode) by suitable topological invariant in 2D and 3D cases., Comment: This is the published version

We consider an intersection zone where autonomous vehicles (AVs) and human-driven vehicles (HDVs) can be present. As a new vehicle arrives, the traffic controller needs to decide and impose an optimal sequence of the vehicles that will exit the intersection zone. The traffic controller can send information regarding the time at which an AV can cross the intersection; however, the traffic controller can not communicate with the HDVs, rather the HDVs can only be controlled using the traffic lights. We formulate the problem as an integer constrained non-linear optimization problem where the traffic-intersection controller only communicates with a subset of the AVs. Since the number of possible combinations increases exponentially with the number of vehicles in the system, we relax the original problem and proposes an algorithm that gives the optimal solution of the relaxed problem and yet only scales linearly with the number of vehicles in the system. The numerical evaluation shows that our algorithm outperforms the First-In-First-Out (FIFO) algorithm.

We consider a mixed autonomy scenario where the traffic intersection controller decides whether the traffic light will be green or red at each lane for multiple traffic-light blocks. The objective of the traffic intersection controller is to minimize the queue length at each lane and maximize the outflow of vehicles over each block. We consider that the traffic intersection controller informs the autonomous vehicle (AV) whether the traffic light will be green or red for the future traffic-light block. Thus, the AV can adapt its dynamics by solving an optimal control problem. We model the decision process of the traffic intersection controller as a deterministic delay Markov decision process owing to the delayed action by the traffic controller. We propose Reinforcement-learning based algorithm to obtain the optimal policy. We show - empirically - that our algorithm converges and reduces the energy costs of AVs drastically as the traffic controller communicates with the AVs., Comment: Accepted for Publication at 24th IEEE International Conference on Intelligent Transportation (ITSC'2021)

After exploring much on two-dimensional higher-order topological superconductors (HOTSCs) hosting Majorana corner modes (MCMs) only, we propose a simple fermionic model based on a three-dimensional topological insulator proximized with $s$-wave superconductor to realize Majorana hinge modes (MHMs) followed by MCMs under the application of appropriate Wilson-Dirac perturbations. We interestingly find that the second-order topological superconductor, hosting MHMs, appears above a threshold value of the first type perturbation while the third-order topological superconducting phase, supporting MCMs, immediately arises incorporating infinitesimal perturbation of the second kind. Thus, a hierarchy of HOTSC phases can be realized in a single three-dimensional model. Additionally, the application of bulk magnetic field is found to be instrumental in manipulating the number of MHMs, leaving the number for MCMs unaltered. We analytically understand these above-mentioned numerical findings by resorting to the low energy model. We further characterize these topological phases with a distinct structure of the Wannier spectra. From the practical point of view, we manifest quantized transport signatures of these higher-order modes. Finally, we construct Floquet engineering to generate the hierarchy of HOTSC phases by kicking the same perturbations as considered in their static counterpart., Comment: This is the published version

Autonomous vehicles (AVs) are attractive for ride service providers (RSPs) in part because they eliminate the need to compete for human drivers. We investigate a scenario where two RSPs with AVs compete for customers. We model the problem as a game where the RSPs select prices for each origin-destination pair over multiple time periods in an underlying graph representing the customers' desired trips. Each RSP also decides the number of AVs to be stationed at each node at each time period to serve the customers' demands. The number of customers who avail of the service of an RSP depends on the price selected by the RSP and its competitor. Since the strategy choices available to an RSP depends on its competitor, we seek to compute a Generalized Nash equilibrium (GNE). We show that there may be multiple GNEs. However, when an RSP selects prices in order to deter its competitor when it is not serving a source-destination pair, the game has a potential function and admits a unique GNE. We also compare the competitive prices with a monopoly price where only one RSP is in the market. Numerically, we show that if a network consists of two equal-size spatial clusters of demand where the demand between clusters is low, the RSPs may partition the market, i.e, one cluster is served by only one RSP. Hence, the competitive price may become close to the monopoly price., Comment: Submitted in IEEE Conference on Decision and Control

We consider a multi-agent Markov strategic interaction over an infinite horizon where agents can be of multiple types. We model the strategic interaction as a mean-field game in the asymptotic limit when the number of agents of each type becomes infinite. Each agent has a private state; the state evolves depending on the distribution of the state of the agents of different types and the action of the agent. Each agent wants to maximize the discounted sum of rewards over the infinite horizon which depends on the state of the agent and the distribution of the state of the leaders and followers. We seek to characterize and compute a stationary multi-type Mean field equilibrium (MMFE) in the above game. We characterize the conditions under which a stationary MMFE exists. Finally, we propose Reinforcement learning (RL) based algorithm using policy gradient approach to find the stationary MMFE when the agents are unaware of the dynamics. We, numerically, evaluate how such kind of interaction can model the cyber attacks among defenders and adversaries, and show how RL based algorithm can converge to an equilibrium., Comment: 9 pages

We theoretically investigate the Floquet generation of second-order topological superconducting (SOTSC) phase in the high-temperature platform both in two-dimension (2D) and three-dimension (3D). Starting from a $d$-wave superconducting pairing gap, we periodically kick the mass term to engineer the dynamical SOTSC phase within a specific range of the strength of the drive. Under such dynamical breaking of time-reversal symmetry (TRS), we show the emergence of the \textit{weak} SOTSC phase, harboring eight corner modes \ie two zero-energy Majorana per corner, with vanishing Floquet quadrupole moment. On the other hand, our study interestingly indicates that upon the introduction of an explicit TRS breaking Zeeman field, the \textit{weak} SOTSC phase can be transformed into \textit{strong} SOTSC phase, hosting one zero-energy Majorana mode per corner, with quantized quadrupole moment. We also compute the Floquet Wannier spectra that further establishes the \textit{weak} and \textit{strong} nature of these phases. We numerically verify our protocol computing the exact Floquet operator in open boundary condition and then analytically validate our findings with the low energy effective theory (in the high-frequency limit). The above protocol is applicable for 3D as well where we find one dimensional (1D) hinge mode in the SOTSC phase. We then show that these corner modes are robust against moderate disorder and the topological invariants continue to exhibit quantized nature until disorder becomes substantially strong. The existence of zero-energy Majorana modes in these higher-order phases is guaranteed by the anti-unitary spectral symmetry., Comment: This is the published version

Users can now give back energies to the grid using distributed resources. Proper incentive mechanisms are required for such users, also known as prosumers, in order to maximize the sell-back amount while maintaining the retailer's profit. However, all the existing literature considers expected utility theory (EUT) where they assume that prosumers maximize their expected payoff. We consider prospect theory (PT) which models the behavior of humans in the face of uncertainty in a better manner. We show that in a day-ahead contract pricing mechanism, the actual optimal value of contract and the sell-back amount may be smaller compared to the one computed by the EUT. We also propose a lottery-based mechanism and show that such a mechanism can increase the sell-back amount while increasing the retailer's savings compared to day-ahead contract pricing., Comment: Accepted in ECC'21

We theoretically investigate the Floquet generation of second-order topological superconducting (SOTSC) phase, hosting Majorana corner modes (MCMs), considering a quantum spin Hall insulator (QSHI) with proximity induced superconducting $s$-wave pairing in it. Our dynamical prescription consists of the periodic kick in time-reversal symmetry breaking in-plane magnetic field and four-fold rotational symmetry breaking mass term while these Floquet MCMs are preserved by anti-unitary particle-hole symmetry. The first driving protocol always leads to four zero energy MCMs (i.e. one Majorana per corner) as a sign of a {\it{strong}} SOTSC phase. Interestingly, the second protocol can result in a {\it{weak}} SOTSC phase, harbouring eight zero energy MCMs (two Majorana states per corner), in addition to the {\it{strong}} SOTSC phase. We characterize the topological nature of these phases by Floquet quadrupolar moment and Floquet Wannier spectrum. We believe that relying on the recent experimental advancement in the driven systems and proximity induced superconductivity, our schemes may be possible to test in the future., Comment: This is the Published Version

We consider a scenario where a retailer can set different prices for different consumers in a smart grid. The retailer's objective is to maximize the revenue, minimize the operating cost, and maximize the consumer's welfare. The retailer wants to optimize a convex combination of the above objectives using price signals specific to each consumer. However, variability in unit prices across consumers is bounded by a parameter $\eta$, hence limiting the discrimination. We formulate the pricing problem as a Stackelberg game where the retailer is the leader and consumers are followers. Since the retailer's optimization problem turns out to be non-convex, we convexify it via relaxations. We provide performance guarantees for the relaxations in the asymptotic sense (when number of consumers tends to $\infty$). Further, we show that despite the variability in pricing, the pricing scheme proposed by our model is fair as higher prices are charged to consumers who have higher willingness for demand. We extend our analysis to the scenario where consumers can feed energy back to the grid via net-metering. We show that our pricing policy promotes fairness even in this scenario as prosumers who contribute more to the grid, are given large cuts on buying rates. The policy is also found to incentivize more prosumers to invest in renewable energy, thus encouraging sustainability., Comment: Under review

We theoretically investigate a periodically driven semimetal based on a square lattice. The possibility of engineering both Floquet Topological Insulator featuring Floquet edge states and Floquet higher order topological insulating phase, accommodating topological corner modes has been demonstrated starting from the semimetal phase, based on Floquet Hamiltonian picture. Topological phase transition takes place in the bulk quasi-energy spectrum with the variation of the drive amplitude where Chern number changes sign from $+1$ to $-1$. This can be attributed to broken time-reversal invariance ($\mathcal{T}$) due to circularly polarized light. When the discrete four-fold rotational symmetry ($\mathcal{C}_4$) is also broken by adding a Wilson mass term along with broken $\mathcal{T}$, higher order topological insulator (HOTI), hosting in-gap modes at all the corners, can be realized. The Floquet quadrupolar moment, calculated with the Floquet states, exhibits a quantized value of $ 0.5$ (modulo 1) identifying the HOTI phase. We also show the emergence of the {\it{dressed corner modes}} at quasi-energy $\omega/2$ (remnants of zero modes in the quasi-static high frequency limit), where $\omega$ is the driving frequency, in the intermediate frequency regime., Comment: This is the published version

Stochastic games provide a framework for interactions among multiple agents and enable a myriad of applications. In these games, agents decide on actions simultaneously, the state of every agent moves to the next state, and each agent receives a reward. However, finding an equilibrium (if exists) in this game is often difficult when the number of agents becomes large. This paper focuses on finding a mean-field equilibrium (MFE) in an action coupled stochastic game setting in an episodic framework. It is assumed that the impact of the other agents' can be assumed by the empirical distribution of the mean of the actions. All agents know the action distribution and employ lower-myopic best response dynamics to choose the optimal oblivious strategy. This paper proposes a posterior sampling based approach for reinforcement learning in the mean-field game, where each agent samples a transition probability from the previous transitions. We show that the policy and action distributions converge to the optimal oblivious strategy and the limiting distribution, respectively, which constitute an MFE.

The success of modern ride-sharing platforms crucially depends on the profit of the ride-sharing fleet operating companies, and how efficiently the resources are managed. Further, ride-sharing allows sharing costs and, hence, reduces the congestion and emission by making better use of vehicle capacities. In this work, we develop a distributed model-free, DeepPool, that uses deep Q-network (DQN) techniques to learn optimal dispatch policies by interacting with the environment. Further, DeepPool efficiently incorporates travel demand statistics and deep learning models to manage dispatching vehicles for improved ride sharing services. Using real-world dataset of taxi trip records in New York City, DeepPool performs better than other strategies, proposed in the literature, that do not consider ride sharing or do not dispatch the vehicles to regions where the future demand is anticipated. Finally, DeepPool can adapt rapidly to dynamic environments since it is implemented in a distributed manner in which each vehicle solves its own DQN individually without coordination.

The Citizens Broadband Radio Service (CBRS) recently adopted in the U.S. enables two tiers of commercial users to share spectrum with a third tier of incumbent users. This sharing can be further assisted by Environmental Sensing Capability operators (ESCs), that monitor the spectrum occupancy to determine when use of the spectrum will not harm incumbents. Two key aspects of this framework that impact how firms may compete are the differences in information provided by different ESCs and the different tiers in which a firm may access the spectrum. We develop a game theoretic model that captures both of these features and analyze it to gain insight into their impact. Specifically, we consider a priority access (PA) tier firm has access to the both licensed band and unlicensed band, and a general authorized access (GAA) tier firm has access only to the unlicensed band. The PA tier and GAA tier firms compete for users. Our analysis reveals that the amount of unlicensed and licensed bandwidth in the CBRS must be chosen judiciously in order to maximize the social welfare. We also show that a limited amount of unlicensed access of PA tier firm is beneficial to the user's surplus as well as to the social welfare., Comment: Submitted to MobiHoc'2019

We propose a robust scheme for streaming 360-degree immersive videos to maximize the quality of experience (QoE). Our streaming approach introduces a holistic analytical framework built upon the formal method of stochastic optimization. We propose a robust algorithm which provides a streaming rate such that the video quality degrades below that rate with very low probability even in presence of uncertain head movement, and bandwidth. It assumes the knowledge of the viewing probability of different portions (tiles) of a panoramic scene. Such probabilities can be easily derived from crowdsourced measurements performed by 360 video content providers. We then propose efficient methods to solve the problem at runtime while achieving a bounded optimality gap (in terms of the QoE). We implemented our proposed approaches using emulation. Using real users' head movement traces and real cellular bandwidth traces, we show that our algorithms significantly outperform the baseline algorithms by at least in $30\%$ in the QoE metric. Our algorithm gives a streaming rate which is $50\%$ higher compared to the baseline algorithms when the prediction error is high., Comment: arXiv admin note: text overlap with arXiv:1704.08215

Amazon EC2 provides two most popular pricing schemes--i) the {\em costly} on-demand instance where the job is guaranteed to be completed, and ii) the {\em cheap} spot instance where a job may be interrupted. We consider a user can select a combination of on-demand and spot instances to finish a task. Thus he needs to find the optimal bidding price for the spot-instance, and the portion of the job to be run on the on-demand instance. We formulate the problem as an optimization problem and seek to find the optimal solution. We consider three bidding strategies: one-time requests with expected guarantee and one-time requests with penalty for incomplete job and violating the deadline, and persistent requests. Even without a penalty on incomplete jobs, the optimization problem turns out to be non-convex. Nevertheless, we show that the portion of the job to be run on the on-demand instance is at most half. If the job has a higher execution time or smaller deadline, the bidding price is higher and vice versa. Additionally, the user never selects the on-demand instance if the execution time is smaller than the deadline. The numerical results illustrate the sensitivity of the effective portfolio to several of the parameters involved in the model. Our empirical analysis on the Amazon EC2 data shows that our strategies can be employed on the real instances, where the expected total cost of the proposed scheme decreases over 45\% compared to the baseline strategy.

We consider users which may have renewable energy harvesting devices, or distributed generators. Such users can behave as consumer or producer (hence, we denote them as prosumers) at different time instances. A prosumer may sell the energy to other prosumers in exchange of money. We consider a demand response model, where the price of conventional energy depends on the total demand of all the prosumers at a certain time. A prosumer depending on its own utility has to select the amount of energy it wants to buy either from the grid or from other prosumers, or the amount of excess energy it wants to sell to other prosumers. However, the strategy, and the payoff of a prosumer inherently depends on the strategy of other prosumers as a prosumer can only buy if the other prosumers are willing to sell. We formulate the problem as a coupled constrained game, and seek to obtain the generalized Nash equilibrium. We show that the game is a concave potential game and show that there exists a unique generalized Nash equilibrium. We consider that a platform will set the price for distributed interchange of energy among the prosumers in order to minimize the consumption of the conventional energy. We propose a distributed algorithm where the platform sets a price to each prosumer, and then each prosumer at a certain time only optimizes its own payoff. The prosumer then updates the price depending on the supply and demand for each prosumer. We show that the algorithm converges to an optimal generalized Nash equilibrium. The distributed algorithm also provides an optimal price for the exchange market., Comment: 9 pages, 4 figures

In cloud storage, the digital data is stored in logical storage pools, backed by heterogeneous physical storage media and computing infrastructure that are managed by a Cloud Service Provider (CSP). To balance the tradeoff between service performance and cost, CSPs often employ different storage tiers, for instance, cold storage and hot storage. Storing data in hot storage incurs high storage cost yet delivers low access latency, whereas cold storage is able to inexpensively store massive amounts of data and thus provides lower cost with higher latency. In this paper, we address a major challenge confronting the CSPs utilizing such tiered storage architecture - how to maximize their overall profit over a variety of storage tiers that offer distinct characteristics, as well as file placement and access request scheduling policies. To this end, we propose a scheme where the CSP offers a two-stage auction process for (a) requesting storage capacity, and (b) requesting accesses with latency requirements. Our two-stage bidding scheme provides a hybrid storage and access optimization framework with the objective of maximizing the CSP's total net profit over four dimensions: file acceptance decision, placement of accepted files, file access decision and access request scheduling policy. The proposed optimization is a mixed-integer nonlinear program that is hard to solve. We propose an efficient heuristic to relax the integer optimization and to solve the resulting nonlinear stochastic programs. The algorithm is evaluated under different scenarios and with different storage system parameters, and insightful numerical results are reported by comparing the proposed approach with other profit-maximization models. We see a profit increase of over 60% of our proposed method compared to other schemes in certain simulation scenarios., Comment: 14 pages

In the 360-degree immersive video, a user only views a part of the entire raw video frame based on her viewing direction. However, today's 360-degree video players always fetch the entire panoramic view regardless of users' head movement, leading to significant bandwidth waste that can be potentially avoided. In this paper, we propose a novel adaptive streaming scheme for 360-degree videos. The basic idea is to fetch the invisible portion of a video at the lowest quality based on users' head movement prediction and to adaptively decide the video playback quality for the visible portion based on bandwidth prediction. Doing both in a robust manner requires overcome a series of challenges, such as jointly considering the spatial and temporal domains, tolerating prediction errors, and achieving low complexity. To overcome these challenges, we first define quality of experience (QoE) metrics for adaptive 360-degree video streaming. We then formulate an optimization problem and solve it at a low complexity. The algorithm strategically leverages both future bandwidth and the distribution of users' head positions to determine the quality level of each tile (i.e., a sub-area of a raw frame). We further provide theoretical proof showing that our algorithm achieves optimality under practical assumptions. Numerical results show that our proposed algorithms significantly boost the user QoE by at least 20\% compared to baseline algorithms.

The paper considers a bidirectional power flow model of the electric vehicles (EVs) in a charging station. The EVs can inject energies by discharging via a Vehicle-to-Grid (V2G) service which can enhance the profits of the charging station. However, frequent charging and discharging degrade battery life. A proper compensation needs to be paid to the users to participate in the V2G service. We propose a menu-based pricing scheme, where the charging station selects a price for each arriving user for the amount of battery utilization, the total energy, and the time (deadline) that the EV will stay. The user can accept one of the contracts or rejects all depending on their utilities. The charging station can serve users using a combination of the renewable energy and the conventional energy bought from the grid. We show that though there exists a profit maximizing price which maximizes the social welfare, it provides no surplus to the users if the charging station is aware of the utilities of the users. If the charging station is not aware of the exact utilities, the social welfare maximizing price may not maximize the expected profit. In fact, it can give a zero profit. We propose a pricing strategy which provides a guaranteed fixed profit to the charging station and it also maximizes the expected profit for a wide range of utility functions. Our analysis shows that when the harvested renewable energy is small the users have higher incentives for the V2G service. We, numerically, show that the charging station's profit and the user's surplus both increase as V2G service is efficiently utilized by the pricing mechanism., Comment: Submitted in IEEE Transactions on Automatic Control. arXiv admin note: substantial text overlap with arXiv:1609.09037

We propose an online pricing mechanism for electric vehicle (EV) charging. A charging station decides prices for each arriving EV depending on the energy and the time within which the EV will be served (i.e. deadline). The user selects either one of the contracts by paying the prescribed price or rejects all depending on their surpluses. The charging station can serve users using renewable energy and conventional energy. Users may select longer deadlines as they may have to pay less because of the less amount of conventional energy, however, they have to wait a longer period. We consider a {\em myopic} charging station and show that there exists a pricing mechanism which jointly maximizes the social welfare and the profit of the charging station when the charging station knows the utilities of the users. However, when the charging station does not know the utilities of the users, the social welfare pricing strategy may not maximize the expected profit of the charging station and even the profit may be $0$. We propose a fixed profit pricing strategy which provides a guaranteed fixed profit to the charging station and can maximize the profit in practice. We empirically show that our proposed mechanism reduces the peak-demand and utilizes the limited charging spots in a charging station efficiently., Comment: 31 pages, 11 figures, Accepted to IEEE Transactions on Smart Grid

In a secondary spectrum market primaries set prices for their unused channels to the secondaries. The payoff of a primary depends on the availability of unused channels of its competitors. We consider a model were a primary can acquire its competitor's channel state information (C-CSI) at a cost. We formulate a game between two primaries where each primary decides whether to acquire C-CSI or not and then selects its price based on that. We first characterize the Nash Equilibrium (NE) of this game for a symmetric model where the C-CSI is perfect. We show that the payoff of a primary is independent of the C-CSI acquisition cost. We then generalize our analysis to allow for imperfect estimation and cases where the two primaries have different C-CSI costs or different channel availabilities. Our results show interestingly that the payoff of a primary increases when there is estimation error. We also show that surprisingly, the expected payoff of a primary may decrease when the C-CSI acquisition cost decreases when primaries have different availabilities., Comment: Shorter version will be presented in ISIT'16. Submitted to JSAC Special Issue on Spectrum Sharing, 2016. Impacts of Estimation errors, different C-CSI acquisition costs and different availabilities are now considered

The economic model of the Internet of Things (IoT) consists of end users, advertisers and three different kinds of providers--IoT service provider (IoTSP), Wireless service provider (WSP) and cloud service provider (CSP). We investigate three different kinds of interactions among the providers. First, we consider that the IoTSP prices a bundled service to the end-users, and the WSP and CSP pay the IoTSP (push model). Next, we consider the model where the end-users independently pay the each provider (pull model). Finally, we consider a hybrid model of the above two where the IoTSP and WSP quote their prices to the end-users, but the CSP quotes its price to the IoTSP. We characterize and quantify the impact of the advertisement revenue on the equilibrium pricing strategy and payoff of providers, and corresponding demands of end users in each of the above interaction models. Our analysis reveals that the demand of end-users, and the payoffs of the providers are non decreasing functions of the advertisement revenue. For sufficiently high advertisement revenue, the IoTSP will offer its service free of cost in each interaction model. However, the payoffs of the providers, and the demand of end-users vary across different interaction models. Our analysis shows that the demand of end-users, and the payoff of the WSP are the highest in the pull (push, resp.) model in the low (high, resp.) advertisement revenue regime. The payoff of the IoTSP is always higher in the pull model irrespective of the advertisement revenue. The payoff of the CSP is the highest in the hybrid model in the low advertisement revenue regime. However, in the high advertisement revenue regime the payoff of the CSP in the hybrid model or in the push model can be higher depending on the equilibrium chosen in the push model., Comment: Submitted in IEEE Transactions on Automatic Control

We investigate a spectrum oligopoly market where each primary seeks to sell secondary access to its channel at multiple locations. Transmission qualities of a channel evolve randomly. Each primary needs to select a price and a set of non-interfering locations (which is an independent set in the conflict graph of the region) at which to offer its channel without knowing the transmission qualities of the channels of its competitors. We formulate the above problem as a non-cooperative game. We consider two scenarios-i) when the region is small, ii) when the region is large. In the first setting, we focus on a class of conflict graphs, known as mean valid graphs which commonly arise when the region is small. We explicitly compute a symmetric Nash equilibrium (NE); the NE is threshold type in that primaries only choose independent set whose cardinality is greater than a certain threshold. The threshold on the cardinality increases with increase in quality of the channel on sale. We show that the symmetric NE strategy profile is unique in a special class of conflict graphs (linear graph). In the second setting, we consider node symmetric conflict graphs which arises when the number of locations is large (potentially, infinite). We explicitly compute a symmetric NE that randomizes equally among the maximum independent sets at a given channel state vector. In the NE a primary only selects the maximum independent set at a given channel state vector. We show that the two symmetric NEs computed in two settings exhibit important structural difference., Comment: Added several results from the other versions. Submitted to IEEE Transactions on Information Theory. 63 pages one column, 22 figures

We investigate a spectrum oligopoly market where primaries lease their channels to secondaries in lieu of financial remuneration. Transmission quality of a channel evolves randomly. Each primary has to select the price it would quote without knowing the transmission qualities of its competitors' channels. Each secondary buys a channel depending on the price and the transmission quality a channel offers. We formulate the price selection problem as a non co-operative game with primaries as players. In the one-shot game, we show that there exists a unique symmetric Nash Equilibrium(NE) strategy profile and explicitly compute it. Our analysis reveals that under the NE strategy profile a primary prices its channel to render high quality channel more preferable to the secondary; this negates the popular belief that prices ought to be selected to render channels equally preferable to the secondary regardless of their qualities. We show the loss of revenue in the asymptotic limit due to the non co-operation of primaries. In the repeated version of the game, we characterize a subgame perfect NE where a primary can attain a payoff arbitrarily close to the payoff it would obtain when primaries co-operate., Comment: Accepted for publication in IEEE/ACM Transactions on Networking. 41 pages single column format.Conference version is available at arXiv:1305.3351

We investigate a spectrum oligopoly where primary users allow secondary access in lieu of financial remuneration. Transmission qualities of the licensed bands fluctuate randomly. Each primary needs to select the price of its channel with the knowledge of its own channel state but not that of its competitors. Secondaries choose among the channels available on sale based on their states and prices. We formulate the price selection as a non-cooperative game and prove that a symmetric Nash equilibrium (NE) strategy profile exists uniquely. We explicitly compute this strategy profile and analytically and numerically evaluate its efficiency. Our structural results provide certain key insights about the unique symmetric NE., Comment: Presented in ISIT' 2013, Istanbul Version 2 contains some modified versions of proofs of version 1. In IEEE Proceedings of International Symposium on Information Theory, 2013