Search

We present SupMario, a characterization framework designed to thoroughly analyze, model, and optimize CXL memory performance. SupMario is based on extensive evaluation of 265 workloads spanning 4 real CXL devices within 7 memory latency configurations across 4 processor platforms. SupMario uncovers many key insights, including detailed workload performance at sub-us memory latencies (140-410 ns), CXL tail latencies, CPU tolerance to CXL latencies, CXL performance root-cause analysis and precise performance prediction models. In particular, SupMario performance models rely solely on 12 CPU performance counters and accurately fit over 99% and 91%-94% workloads with a 10% misprediction target for NUMA and CXL memory, respectively. We demonstrate the practical utility of SupMario characterization findings, models, and insights by applying them to popular CXL memory management schemes, such as page interleaving and tiering policies, to identify system inefficiencies during runtime. We introduce a novel ``bestshot'' page interleaving policy and a regulated page tiering policy (Alto) tailored for memory bandwidth- and latency-sensitive workloads. In bandwidth bound scenarios, our ``best-shot'' interleaving, guided by our novel performance prediction model, achieves close-to optimal scenarios by exploiting the aggregate system and CXL/NUMA memory bandwidth. For latency sensitive workloads, Alto, driven by our key insight of utilizing ``amortized'' memory latency to regulate unnecessary page migrations, achieves up to 177% improvement over state-of-the-art memory tiering systems like TPP, as demonstrated through extensive evaluation with 8 real-world applications.

Video analytics is widely used in contemporary systems and services. At the forefront of video analytics are video queries that users develop to find objects of particular interest. Building upon the insight that video objects (e.g., human, animals, cars, etc.), the center of video analytics, are similar in spirit to objects modeled by traditional object-oriented languages, we propose to develop an object-oriented approach to video analytics. This approach, named VQPy, consists of a frontend$\unicode{x2015}$a Python variant with constructs that make it easy for users to express video objects and their interactions$\unicode{x2015}$as well as an extensible backend that can automatically construct and optimize pipelines based on video objects. We have implemented and open-sourced VQPy, which has been productized in Cisco as part of its DeepVision framework., Comment: MLSys'24

In this paper, we formulate the adaptive learning problem---the problem of how to find an individualized learning plan (called policy) that chooses the most appropriate learning materials based on learner's latent traits---faced in adaptive learning systems as a Markov decision process (MDP). We assume latent traits to be continuous with an unknown transition model. We apply a model-free deep reinforcement learning algorithm---the deep Q-learning algorithm---that can effectively find the optimal learning policy from data on learners' learning process without knowing the actual transition model of the learners' continuous latent traits. To efficiently utilize available data, we also develop a transition model estimator that emulates the learner's learning process using neural networks. The transition model estimator can be used in the deep Q-learning algorithm so that it can more efficiently discover the optimal learning policy for a learner. Numerical simulation studies verify that the proposed algorithm is very efficient in finding a good learning policy, especially with the aid of a transition model estimator, it can find the optimal learning policy after training using a small number of learners.

In this letter, we address the problem of controlling energy storage systems (ESSs) for arbitrage in real-time electricity markets under price uncertainty. We first formulate this problem as a Markov decision process, and then develop a deep reinforcement learning based algorithm to learn a stochastic control policy that maps a set of available information processed by a recurrent neural network to ESSs' charging/discharging actions. Finally, we verify the effectiveness of our algorithm using real-time electricity prices from PJM.

In this paper, we develop a data-driven voltage regulation framework for distributed energy resources (DERs) in a balanced radial power distribution system. The objective is to determine optimal DER power injections that minimize the voltage deviations from a desirable voltage range without knowing a complete power distribution system model a priori. The nonlinear relationship between the voltage magnitudes and the power injections in the power distribution system is approximated by a linear model, the parameters of which---referred to as the voltage sensitivities---can be computed directly using information on the topology and the line parameters. Assuming the knowledge of feasible topology configurations and distribution line resistance-to-reactance ratios, the true topology configuration and corresponding line parameters can be estimated effectively using a few sets of measurements on voltage magnitudes and power injections. Using the estimated voltage sensitivities, the optimal DER power injections can be readily determined by solving a convex optimization problem. The proposed framework is intrinsically adaptive to changes in system conditions such as unknown topology reconfiguration due to its data-driven nature. The effectiveness and efficiency of the proposed framework is validated via numerical simulations on the IEEE 123-bus distribution test feeder., Comment: arXiv admin note: text overlap with arXiv:1711.04159

This paper reports an initial work on power system oscillation damping improvement using a data-driven online optimization method. An online oscillation damping optimization mod-el is proposed and formulated in a form solvable by the data-driven method. Key issues in the online optimization procedures, including the damping sensitivity identification method, its compatibility with the dispatch plans, as well as other practical issues in real large-scale system are discussed. Simulation results based on the 2-area 4-machine system, and the NETS-NYPS 68-bus system verify the feasibility and efficiency of the proposed method. The results also show the capability of the proposed method to bridge the gap between online data analysis and complex optimization for power system dynamics.

Price responsiveness is a major feature of end use customers (EUCs) that participate in demand response (DR) programs, and has been conventionally modeled with static demand functions, which take the electricity price as the input and the aggregate energy consumption as the output. This, however, neglects the inherent temporal correlation of the EUC behaviors, and may result in large errors when predicting the actual responses of EUCs in real-time pricing (RTP) programs. In this paper, we propose a dynamical DR model so as to capture the temporal behavior of the EUCs. The states in the proposed dynamical DR model can be explicitly chosen, in which case the model can be represented by a linear function or a multi-layer feedforward neural network, or implicitly chosen, in which case the model can be represented by a recurrent neural network or a long short-term memory unit network. In both cases, the dynamical DR model can be learned from historical price and energy consumption data. Numerical simulation illustrated how the states are chosen and also showed the proposed dynamical DR model significantly outperforms the static ones., Comment: Accepted to IEEE ISGT NA 2019

E-learning systems are capable of providing more adaptive and efficient learning experiences for students than the traditional classroom setting. A key component of such systems is the learning strategy, the algorithm that designs the learning paths for students based on information such as the students' current progresses, their skills, learning materials, and etc. In this paper, we address the problem of finding the optimal learning strategy for an E-learning system. To this end, we first develop a model for students' hierarchical skills in the E-learning system. Based on the hierarchical skill model and the classical cognitive diagnosis model, we further develop a framework to model various proficiency levels of hierarchical skills. The optimal learning strategy on top of the hierarchical structure is found by applying a model-free reinforcement learning method, which does not require information on students' learning transition process. The effectiveness of the proposed framework is demonstrated via numerical experiments.

In this paper, we address the problem of setting the tap positions of load tap changers (LTCs) for voltage regulation in radial power distribution systems under uncertain load dynamics. The objective is to find a policy to determine the tap positions that only uses measurements of voltage magnitudes and topology information so as to minimize the voltage deviation across the system. We formulate this problem as a Markov decision process (MDP), and propose a batch reinforcement learning (RL) algorithm to solve it. By taking advantage of a linearized power flow model, we propose an effective algorithm to estimate the voltage magnitudes under different tap settings, which allows the RL algorithm to explore the state and action spaces freely offline without impacting the system operation. To circumvent the "curse of dimensionality" resulted from the large state and action spaces, we propose a sequential learning algorithm to learn an action-value function for each LTC, based on which the optimal tap positions can be directly determined. The effectiveness of the proposed algorithm is validated via numerical simulations on the IEEE 13-bus and 123-bus distribution test feeders.

In this paper, we propose a framework for coordinating distributed energy resources (DERs) connected to a power distribution system, the model of which is not completely known, so that they collectively provide a specified amount of active power to the bulk power system as quantified by the power exchange between both systems at the bus interconnecting them, while respecting distribution line capacity limits. The proposed framework consists of (i) a linear time-varying input-output (IO) system model that represents the relation between the DER active power injections (inputs), and the total active power exchanged between the distribution and bulk power systems (output); (ii) an estimator that aims to estimate the IO model parameters, and (iii) a controller that determines the optimal DER active power injections so the power exchanged between both systems equals to the specified amount at a minimum generating cost. We formulate the estimation problem as a quadratic program with box constraints and solve it using the projected gradient descent algorithm. To resolve the potential issue of collinearity in the measurements used by the estimator, we introduce random perturbations in the DER active power injections during the estimation process. Using the estimated IO model, the optimal DER coordination problem to be solved by the controller can be formulated as a convex optimization problem, which can be solved easily. The effectiveness of the framework is validated via numerical simulations using the IEEE 123-bus distribution test feeder., Comment: Accepted to IEEE Transactions on Power Systems

In this paper, we address the problem of coordinating a set of distributed energy resources (DERs) to regulate voltage in power distribution systems to desired levels. To this end, we formulate the voltage control problem as an optimization problem, the objective of which is to determine the optimal DER power injections that minimize the voltage deviations from desirable voltage levels subject to a set of constraints. The nonlinear relationship between the voltage magnitudes and the nodal power injections is approximated by a linear model, the parameters of which can be estimated in real-time efficiently using measurements. In particular, the parameter estimation requires much fewer data by exploiting the structural characteristics of the power distribution system. As such, the voltage control framework is intrinsically adaptive to parameter changes. Numerical studies on the IEEE 37-bus power distribution test feeder validated the effectiveness of the propose framework.

This paper is concerned with the problem of coordinating a set of distributed energy resources (DERs) in a lossy power distribution system to provide frequency regulation services to a bulk power grid with the explicit consideration of system losses. To this end, we formulate the problem as an optimization problem, the objective of which is to minimize some cost function subject to a set of constraints. The formulation requires knowledge of incremental total system losses, which we approximate using the so-called loss factors (LFs) that explicitly capture the impacts of both active and reactive power injections on system losses. The LFs are estimated recursively using power injection measurements; thus, they are adaptive to various phenomena that impact the power system operation. Numerical simulation on a 33-bus distribution test feeder validated the effectiveness of the proposed framework.

The demand response provides an opportunity for load serving entities (LSEs) that operate retail electricity markets (REMs) to strategically purchase energy and provide reserves in wholesale electricity markets (WEMs). This paper concerns with the problem of simultaneously determining the optimal energy bids and reserve offers an LSE submits to the WEM as well as the optimal energy and reserve prices it sets in the REM so as to maximize its profit. To this end, we explicitly model the tri-layer market structure that consists of a WEM, a REM, and a set of end user customers, so as to capture the coupling between the bidding problem and the pricing problem. Based on the tri-layer market model, we then formulate the joint bidding and pricing problem as a bi-level programming problem and further transform it into a single-level mixed integer linear programming problem, which can be solved efficiently. Numerical studies using the IEEE test cases are presented to illustrate the application of the proposed methodology as well as to reveal several interesting characteristics of the LSE's profit-seeking behavior., Comment: Accepted to IEEE Transactions on Power Systems