Search

This study aims at figuring out the challenges faced by business-English majored students at a Faculty of Foreign Languages of a university in Ho Chi Minh City, Vietnam when they translate business contracts from English into Vietnamese and vice versa. The study also proposes the techniques that may be employed to tackle these difficulties. The questionnaire and they are used in the study. The study results revealed the challenges that business-majored students face including vocabulary, grammar, teaching material, style, term equivalents, and cultural differences between the source and target texts. Techniques may be adapted to tackle these difficulties as reading all samples of business contracts in both English and Vietnamese, being familiar with professional knowledge of business English, availability of more business contract translation-specialized books, exercises and websites, and participation of teachers and experts on workshops related to business contract translation. Most importantly, students or those who would like to learn business contract translation should apply themselves to their studies of business contract translation and expose themselves to different business contracts texts and systems.

Prediction markets are useful for estimating probabilities of claims whose truth will be revealed at some fixed time -- this includes questions about the values of real-world events (i.e. statistical uncertainty), and questions about the values of primitive recursive functions (i.e. logical or algorithmic uncertainty). However, they cannot be directly applied to questions without a fixed resolution criterion, and real-world applications of prediction markets to such questions often amount to predicting not whether a sentence is true, but whether it will be proven. Such questions could be represented by countable unions or intersections of more basic events, or as First-Order-Logic sentences on the Arithmetical Hierarchy (or even beyond FOL, as hyperarithmetical sentences). In this paper, we propose an approach to betting on such events via options, or equivalently as bets on the outcome of a "verification-falsification game". Our work thus acts as an alternative to the existing framework of Garrabrant induction for logical uncertainty, and relates to the stance known as constructivism in the philosophy of mathematics; furthermore it has broader implications for philosophy and mathematical logic., Comment: 15 pages, 4 figures

Reward allocation, also known as the credit assignment problem, has been an important topic in economics, engineering, and machine learning. An important concept in reward allocation is the core, which is the set of stable allocations where no agent has the motivation to deviate from the grand coalition. In previous works, computing the core requires either knowledge of the reward function in deterministic games or the reward distribution in stochastic games. However, this is unrealistic, as the reward function or distribution is often only partially known and may be subject to uncertainty. In this paper, we consider the core learning problem in stochastic cooperative games, where the reward distribution is unknown. Our goal is to learn the expected core, that is, the set of allocations that are stable in expectation, given an oracle that returns a stochastic reward for an enquired coalition each round. Within the class of strictly convex games, we present an algorithm named \texttt{Common-Points-Picking} that returns a point in the expected core given a polynomial number of samples, with high probability. To analyse the algorithm, we develop a new extension of the separation hyperplane theorem for multiple convex sets.

High-dimensional linear bandits with low-dimensional structure have received considerable attention in recent studies due to their practical significance. The most common structure in the literature is sparsity. However, it may not be available in practice. Symmetry, where the reward is invariant under certain groups of transformations on the set of arms, is another important inductive bias in the high-dimensional case that covers many standard structures, including sparsity. In this work, we study high-dimensional symmetric linear bandits where the symmetry is hidden from the learner, and the correct symmetry needs to be learned in an online setting. We examine the structure of a collection of hidden symmetry and provide a method based on model selection within the collection of low-dimensional subspaces. Our algorithm achieves a regret bound of $ O(d_0^{1/3} T^{2/3} \log(d))$, where $d$ is the ambient dimension which is potentially very large, and $d_0$ is the dimension of the true low-dimensional subspace such that $d_0 \ll d$. With an extra assumption on well-separated models, we can further improve the regret to $ O(d_0\sqrt{T\log(d)} )$.

Non-native English speakers (NNESs) often face challenges in achieving fluency in their written English. AI paraphrasing tools have the potential to improve their writing by suggesting more fluent paraphrases to their original sentences. Yet, the effectiveness of these tools depends on the user's ability to accurately assess and select context-appropriate suggestions, which is a significant challenge for those with limited English proficiency. This paper explores how NNESs utilize a paraphrasing tool augmented with information aids designed to facilitate the assessment of paraphrased suggestions. Through a formative study with 15 NNESs, we identify their specific needs when paraphrasing with AI, leading to the design of a paraphrasing tool integrated with five types of information aids, termed "support features." A user study with 22 NNESs demonstrates their heavy reliance on the paraphrasing functionality throughout the writing process, where they leverage the support features to assess and select suggestions efficiently and comprehensively. When equipped with the support features, NNESs experience enhanced writing experience in efficiency, confidence, and trust. Our findings contribute to the HCI community by (i) identifying the distinct needs of NNESs in AI paraphrasing tools, (ii) elucidating how NNESs use paraphrasing tools with support features, and (iii) offering design implications for the development of more effective AI paraphrasing tools tailored to NNESs' requirements.

This study determines the cause of soil erosion in red soils in sugarcane fields, especially even with the use of subsoiling fissures, and to compare the effectiveness of a novel artificial linear-macropore with the insertion of fibrous material into the fractures. Four column treatments (tillage, subsoiling, linear-macropore with plant residue fillings, and no-tillage-with-mulching) were established. A subsoiler was used to break up hard soil layers to enhance infiltration, whereas mulching reduced the impact of raindrops on the soil. Sugarcane residue was inserted in the empty fissure to reinforce the structure, making linear macropore. Simulated rainfall with 20 mmh−1 was applied to the soil surface for 6 h per day for two days. Surface runoff, soil erosion, and drainage were measured during each run. Erosion was minimal (1/7 reduction), and bottom drainage was observed in the linear-macropore and no-tillage-with-mulching plots. Conversely, due to the formation of an impermeable layer or surface crust, high erosion (0.282 t-C ha−1 yr−1) and decreased drainage levels were detected in the subsoiling and tillage plots. Moreover, the aboveground protrusion of fibrous material at the linear-macropore maintained infiltration, even following crust formation. Field application of these four management strategies revealed the effectiveness of linear-macropore and mulching in reducing surface flow. Linear-macropore application maintains appropriate levels of infiltration, and insertion of plant residue fillings reinforces the macropore structure while also avoiding clogging. Hence, the linear-macropore scheme may represent an effective strategy for reducing surface runoff and red soil erosion.

We study online learning problems in which the learner has extra knowledge about the adversary's behaviour, i.e., in game-theoretic settings where opponents typically follow some no-external regret learning algorithms. Under this assumption, we propose two new online learning algorithms, Accurate Follow the Regularized Leader (AFTRL) and Prod-Best Response (Prod-BR), that intensively exploit this extra knowledge while maintaining the no-regret property in the worst-case scenario of having inaccurate extra information. Specifically, AFTRL achieves $O(1)$ external regret or $O(1)$ \emph{forward regret} against no-external regret adversary in comparison with $O(\sqrt{T})$ \emph{dynamic regret} of Prod-BR. To the best of our knowledge, our algorithm is the first to consider forward regret that achieves $O(1)$ regret against strategic adversaries. When playing zero-sum games with Accurate Multiplicative Weights Update (AMWU), a special case of AFTRL, we achieve \emph{last round convergence} to the Nash Equilibrium. We also provide numerical experiments to further support our theoretical results. In particular, we demonstrate that our methods achieve significantly better regret bounds and rate of last round convergence, compared to the state of the art (e.g., Multiplicative Weights Update (MWU) and its optimistic counterpart, OMWU).

The construction field has an important and meaningful role in the economy of any nation, especially developing nations. However, construction is one of the most dangerous fields and has the highest rate of accidents, including deaths and disabling injuries in the world. The construction field uses many tower cranes, especiallyfor constructing multi-storey buildings, factories, or in urban areas, and requires an increasing number of tower cranes. Tower cranes certainly contribute to the high rate of construction injuries and fatalities, in which tower crane installation and dismantling take a high rate. Accidents during tower crane installation and dismantling can kill people (workers and citizens) as well as delays in construction schedules of project and/or damage to buildings and machines under construction. As a result, the cost of costruction projects can increase. In this paper, safety risk factors during tower crane installation and dismantling on construction sites in Vietnam are estimated by showing their likelihood of occurrence, degree of influence, and risk levels. A suitable-structured questionnaire was produced and sent to get data that had their likelihood of occurrence and degree of influence by applying a five-point Likert scale. The results showed that “Time constraints are requested by investor, principal contractor or employer” is the most likely factor with a mean value of 3.60 and “Break of a wire rope occurring on dismantling” has the highest degree of influence, with a mean value of 4.18. The result also showed that there are 15/21 factors with a moderate risk level that is acceptable but, requires suitable controls to maintain a safe working condition of tower crane installation and dismantling. The results of the paper may help managers as well as practicians with good understandings of how to advance the safety of tower crane installation and dismantling on construction sites in Vietnam.

Modeling and simulations of pandemic dynamics play an essential role in understanding and addressing the spreading of highly infectious diseases such as COVID-19. In this work, we propose a novel deep learning architecture named Attention-based Multiresolution Graph Neural Networks (ATMGNN) that learns to combine the spatial graph information, i.e. geographical data, with the temporal information, i.e. timeseries data of number of COVID-19 cases, to predict the future dynamics of the pandemic. The key innovation is that our method can capture the multiscale structures of the spatial graph via a learning to cluster algorithm in a data-driven manner. This allows our architecture to learn to pick up either local or global signals of a pandemic, and model both the long-range spatial and temporal dependencies. Importantly, we collected and assembled a new dataset for New Zealand. We established a comprehensive benchmark of statistical methods, temporal architectures, graph neural networks along with our spatio-temporal model. We also incorporated socioeconomic cross-sectional data to further enhance our prediction. Our proposed model have shown highly robust predictions and outperformed all other baselines in various metrics for our new dataset of New Zealand along with existing datasets of England, France, Italy and Spain. For a future work, we plan to extend our work for real-time prediction and global scale. Our data and source code are publicly available at https://github.com/HySonLab/pandemic_tgnn

This paper explores Large Batch Training techniques using layer-wise adaptive scaling ratio (LARS) across diverse settings, uncovering insights. LARS algorithms with warm-up tend to be trapped in sharp minimizers early on due to redundant ratio scaling. Additionally, a fixed steep decline in the latter phase restricts deep neural networks from effectively navigating early-phase sharp minimizers. Building on these findings, we propose Time Varying LARS (TVLARS), a novel algorithm that replaces warm-up with a configurable sigmoid-like function for robust training in the initial phase. TVLARS promotes gradient exploration early on, surpassing sharp optimizers and gradually transitioning to LARS for robustness in later phases. Extensive experiments demonstrate that TVLARS consistently outperforms LARS and LAMB in most cases, with up to 2\% improvement in classification scenarios. Notably, in all self-supervised learning cases, TVLARS dominates LARS and LAMB with performance improvements of up to 10\%.

Despite a surge in interest in GNN development, homogeneity in benchmarking datasets still presents a fundamental issue to GNN research. GraphWorld is a recent solution which uses the Stochastic Block Model (SBM) to generate diverse populations of synthetic graphs for benchmarking any GNN task. Despite its success, the SBM imposed fundamental limitations on the kinds of graph structure GraphWorld could create. In this work we examine how two additional synthetic graph generators can improve GraphWorld's evaluation; LFR, a well-established model in the graph clustering literature and CABAM, a recent adaptation of the Barabasi-Albert model tailored for GNN benchmarking. By integrating these generators, we significantly expand the coverage of graph space within the GraphWorld framework while preserving key graph properties observed in real-world networks. To demonstrate their effectiveness, we generate 300,000 graphs to benchmark 11 GNN models on a node classification task. We find GNN performance variations in response to homophily, degree distribution and feature signal. Based on these findings, we classify models by their sensitivity to the new generators under these properties. Additionally, we release the extensions made to GraphWorld on the GitHub repository, offering further evaluation of GNN performance on new graphs., Comment: Accepted to Workshop on Graph Learning Benchmarks at KDD 2023

This paper addresses the load restoration problem after power outage events. Our primary proposed methodology is using multi-agent deep reinforcement learning to optimize the load restoration process in distribution systems, modeled as networked microgrids, via determining the optimal operational sequence of circuit breakers (switches). An innovative invalid action masking technique is incorporated into the multi-agent method to handle both the physical constraints in the restoration process and the curse of dimensionality as the action space of operational decisions grows exponentially with the number of circuit breakers. The features of our proposed method include centralized training for multi-agents to overcome non-stationary environment problems, decentralized execution to ease the deployment, and zero constraint violations to prevent harmful actions. Our simulations are performed in OpenDSS and Python environments to demonstrate the effectiveness of the proposed approach using the IEEE 13, 123, and 8500-node distribution test feeders. The results show that the proposed algorithm can achieve a significantly better learning curve and stability than the conventional methods., Comment: 12 pages, 19 figures, journal under review

Mosquito-borne diseases (MBDs), including those caused by flaviviruses, remain human health problems for developing and urbanizing economies. This cross-sectional study examined risks of flavivirus exposure through a survey regarding knowledge and practices of pig farmers, and serological analysis of pigs in peri-urban and suburban Hanoi city. A total of 636 pig sera from 179 pig farms in 4 districts, namely, Chuong My, Dan Phuong, Ha Dong, and Bac Tu Liem, were analyzed by a competitive ELISA designed for flavivirus antibody detection. The results indicated a low level of awareness about MBDs among pig farmers, and a high seroprevalence in pigs at 88.5% (95%CI = 85.8-90.9%). Moreover, common practices of pig owners to prevent mosquitoes at home and farm did not show a significant reduction in flavivirus exposure in pigs. At animal level, significant associations between seropositive pigs and the farms with more than 60 pigs, and the district location were found. Farm-level multivariable analysis did not identify significant risk factors for flavivirus exposure. The study suggests that improving awareness of pig owners about MBDs in Hanoi city may be warranted to reduce the risk for MBD flavivirus infections in both humans and pigs.

In rapid urbanizing cities, the high density of livestock populations in close vicinity to humans, and high mosquito vector abundance have provided suitable conditions for transmission of mosquito-borne zoonotic flaviviruses, that cause several million cases of human disease annually. This thesis explores the urban livestock-keeping in Hanoi and its potential influence on the epidemiology of flavivirus infections. Four major components were investigated: livestock development of the city, knowledge and practices of local people on mosquito-borne diseases (MBDs), mosquito vector abundance, and risk factors for flavivirus exposure in animals. The analyses of the livestock development policy, animal data and key informant interviews showed that the livestock production system of Hanoi had increased in line with the governmental policy during 2014 to 2018. However, some major challenges were identified such as the high proportion of small-scale livestock farms, animal diseases, and a slow progress of translocation of livestock farms out of the urban areas. A cross-sectional study was conducted in 513 households in six districts of Hanoi (two urban, two peripheral, and two peri-urban). Knowledge and practices were evaluated; mosquitoes and larvae inside and outside the houses were collected and identified; blood samples of pigs and dogs were tested by a West Nile virus competitive enzyme-linked immunosorbent assay (cELISA), a kit that allowed for detection of antibodies against several flaviviruses of different animal species. JEV-specific antibodies were confirmed by plaque reduction neutralization test (PRNT). Logistic regression models using seropositivity of households as the outcome were built to identify significant risk factors. A low level of knowledge and preventive practices against zoonotic MBDs in community was indicated in the study. Among 12,861 adult mosquitoes and 2,427 larvae collected, Culex mosquitoes were the most abundant (93.01%), followed by Anoph, Formas, grant number 2016-00364

Symmetry arises in many optimization and decision-making problems, and has attracted considerable attention from the optimization community: By utilizing the existence of such symmetries, the process of searching for optimal solutions can be improved significantly. Despite its success in (offline) optimization, the utilization of symmetries has not been well examined within the online optimization settings, especially in the bandit literature. As such, in this paper we study the invariant Lipschitz bandit setting, a subclass of the Lipschitz bandits where the reward function and the set of arms are preserved under a group of transformations. We introduce an algorithm named \texttt{UniformMesh-N}, which naturally integrates side observations using group orbits into the \texttt{UniformMesh} algorithm (\cite{Kleinberg2005_UniformMesh}), which uniformly discretizes the set of arms. Using the side-observation approach, we prove an improved regret upper bound, which depends on the cardinality of the group, given that the group is finite. We also prove a matching regret's lower bound for the invariant Lipschitz bandit class (up to logarithmic factors). We hope that our work will ignite further investigation of symmetry in bandit theory and sequential decision-making theory in general.

Motivated by cognitive radios, stochastic Multi-Player Multi-Armed Bandits has been extensively studied in recent years. In this setting, each player pulls an arm, and receives a reward corresponding to the arm if there is no collision, namely the arm was selected by one single player. Otherwise, the player receives no reward if collision occurs. In this paper, we consider the presence of malicious players (or attackers) who obstruct the cooperative players (or defenders) from maximizing their rewards, by deliberately colliding with them. We provide the first decentralized and robust algorithm RESYNC for defenders whose performance deteriorates gracefully as $\tilde{O}(C)$ as the number of collisions $C$ from the attackers increases. We show that this algorithm is order-optimal by proving a lower bound which scales as $\Omega(C)$. This algorithm is agnostic to the algorithm used by the attackers and agnostic to the number of collisions $C$ faced from attackers.

In real-world applications, Federated Learning (FL) meets two challenges: (1) scalability, especially when applied to massive IoT networks; and (2) how to be robust against an environment with heterogeneous data. Realizing the first problem, we aim to design a novel FL framework named Full-stack FL (F2L). More specifically, F2L utilizes a hierarchical network architecture, making extending the FL network accessible without reconstructing the whole network system. Moreover, leveraging the advantages of hierarchical network design, we propose a new label-driven knowledge distillation (LKD) technique at the global server to address the second problem. As opposed to current knowledge distillation techniques, LKD is capable of training a student model, which consists of good knowledge from all teachers' models. Therefore, our proposed algorithm can effectively extract the knowledge of the regions' data distribution (i.e., the regional aggregated models) to reduce the divergence between clients' models when operating under the FL system with non-independent identically distributed data. Extensive experiment results reveal that: (i) our F2L method can significantly improve the overall FL efficiency in all global distillations, and (ii) F2L rapidly achieves convergence as global distillation stages occur instead of increasing on each communication cycle., Comment: 28 pages, 5 figures, 10 tables

Networking of microgrids can provide the operational flexibility needed for the increasing number of DERs deployed at the distribution level and supporting end-use demand when there is loss of the bulk power system. But, networked microgrids are vulnerable to cyber-physical attacks and faults due to the complex interconnections. As such, it is necessary to design resilient control systems to support the operations of networked microgrids in responses to cyber-physical attacks and faults. This paper introduces a resilient communication scheme for interconnecting multiple microgrids to support critical demand, in which the interconnection decision can be made distributedly by each microgrid controller even in the presence of cyberattacks to some communication links or microgrid controllers. This scheme blends a randomized peer-to-peer communication network for exchanging information among controllers and resilient consensus algorithms for achieving reliable interconnection agreement. The network of 6 microgrids divided from a modified 123-node test distribution feeder is used to demonstrate the effectiveness of the proposed resilient communication scheme.

To understand the security threats to reinforcement learning (RL) algorithms, this paper studies poisoning attacks to manipulate \emph{any} order-optimal learning algorithm towards a targeted policy in episodic RL and examines the potential damage of two natural types of poisoning attacks, i.e., the manipulation of \emph{reward} and \emph{action}. We discover that the effect of attacks crucially depend on whether the rewards are bounded or unbounded. In bounded reward settings, we show that only reward manipulation or only action manipulation cannot guarantee a successful attack. However, by combining reward and action manipulation, the adversary can manipulate any order-optimal learning algorithm to follow any targeted policy with $\tilde{\Theta}(\sqrt{T})$ total attack cost, which is order-optimal, without any knowledge of the underlying MDP. In contrast, in unbounded reward settings, we show that reward manipulation attacks are sufficient for an adversary to successfully manipulate any order-optimal learning algorithm to follow any targeted policy using $\tilde{O}(\sqrt{T})$ amount of contamination. Our results reveal useful insights about what can or cannot be achieved by poisoning attacks, and are set to spur more works on the design of robust RL algorithms., Comment: Accepted at International Joint Conferences on Artificial Intelligence (IJCAI) 2022

High penetration of wind power with conventional grid following controls for inverter-based wind turbine generators (WTGs) weakens the power grid, challenging the power system stability. Grid-forming (GFM) controls are emerging technologies that can address such stability issues. Numerous methodologies of GFM inverters have been developed in the literature; however, their applications for WTGs have not been thoroughly explored. As WTGs need to incorporate multiple control functions to operate reliably in different operational regions, the GFM control should be appropriately developed for the WTGs. This paper presents a review of GFM controls for WTGs, which covers the latest developments in GFM controls and includes multi-loop and single-loop GFM, virtual synchronous machine-based GFM, and virtual inertia control-based GFM. A comparison study for these GFM-based WTGs regarding normal and abnormal operating conditions together with black-start capability is then performed. The control parameters of these GFM types are properly designed and optimized to enable a fair comparison. In addition, the challenges of applying these GFM controls to wind turbines are discussed, which include the impact of DC-link voltage control strategy and the current saturation algorithm on the GFM control performance, black-start capability, and autonomous operation capability. Finally, recommendations and future developments of GFM-based wind turbines to increase the power system reliability are presented.

We propose a novel method for code summarization utilizing Heterogeneous Code Representations (HCRs) and our specially designed HierarchyNet. HCRs effectively capture essential code features at lexical, syntactic, and semantic levels by abstracting coarse-grained code elements and incorporating fine-grained program elements in a hierarchical structure. Our HierarchyNet method processes each layer of the HCR separately through a unique combination of the Heterogeneous Graph Transformer, a Tree-based CNN, and a Transformer Encoder. This approach preserves dependencies between code elements and captures relations through a novel Hierarchical-Aware Cross Attention layer. Our method surpasses current state-of-the-art techniques, such as PA-Former, CAST, and NeuralCodeSum.

In this paper, we introduce Temporal Multiresolution Graph Neural Networks (TMGNN), the first architecture that both learns to construct the multiscale and multiresolution graph structures and incorporates the time-series signals to capture the temporal changes of the dynamic graphs. We have applied our proposed model to the task of predicting future spreading of epidemic and pandemic based on the historical time-series data collected from the actual COVID-19 pandemic and chickenpox epidemic in several European countries, and have obtained competitive results in comparison to other previous state-of-the-art temporal architectures and graph learning algorithms. We have shown that capturing the multiscale and multiresolution structures of graphs is important to extract either local or global information that play a critical role in understanding the dynamic of a global pandemic such as COVID-19 which started from a local city and spread to the whole world. Our work brings a promising research direction in forecasting and mitigating future epidemics and pandemics.

Improving model accuracy is one of the most frequently addressed issues in bankruptcy prediction. Several previous studies employed artificial neural networks (ANNs) to improve the accuracy at which construction company bankruptcy can be predicted. However, most of these studies use the sample-matching technique and all of the available company quarters or company years in the dataset, resulting in sample selection biases and between-class imbalances. This study integrates a back propagation neural network (BPNN) with the synthetic minority over-sampling technique (SMOTE) and the use of all of the available company-year samples during the sample period to improve the accuracy at which bankruptcy in construction companies can be predicted. In addition to eliminating sample selection biases during the sample matching and between-class imbalance, these methods also achieve the high accuracy rates. Furthermore, the approach used in this study shows optimal over-sampling times, neurons of the hidden layer, and learning rate, all of which are major parameters in the BPNN and SMOTE-BPNN models. The traditional BPNN model is provided as a benchmark for evaluating the predictive abilities of the SMOTE-BPNN model. The empirical results of this paper show that the SMOTE-BPNN model outperforms the traditional BPNN.

The need to understand chemistry through data-driven analysis from a comprehensive collection of reaction data has long been recognized, and the development of chemical reaction databases has been conducted for decades, including commercial databases such as Reaxys, Scifinder, or Web of Science and open-source database like the Open Reaction Database. As a result, exploiting information in this large-scale database could accelerate the process of addressing challenging chemistry problems. Recent developments in big data and machine learning have revolutionized approaches for analyzing chemical data, and successful applications of these techniques have significantly advanced the research in chemical reactions, including reaction classification, product prediction, yield estimation, and synthesis planning. In recent years, homogeneous nickel catalysis has been extensively studied in academia and industry. Remarkable advances have been made for many valuable reactions by using nickel as an effective catalyst, owing to its unique reactivity and cost efficiency. It would be beneficial to prepare a dataset to analyze this field from a fundamental point of view and provide more insight into the nickel-catalyzed reactions. We collected and analyzed a comprehensive dataset containing important papers in the field of C-O bond activation cross-coupling reactions and developed data mining tools to accelerate similar studies in the future.

Background: Dengue virus and Japanese encephalitis virus are two common flaviviruses that are spread widely by Aedes and Culex mosquitoes. Livestock keeping is vital for cities; however, it can pose the risk of increasing the mosquito population. Our study explored how livestock keeping in and around a large city is associated with the presence of mosquitoes and the risk of them spreading flaviviruses. Methods: An entomological study was conducted in 6 districts with 233 households with livestock, and 280 households without livestock, in Hanoi city. BG-Sentinel traps and CDC light traps were used to collect mosquitoes close to animal farms and human habitats. Adult mosquitoes were counted, identified to species level, and grouped into 385 pools, which were screened for flaviviruses using a pan-flavivirus qPCR protocol and sequencing. Results: A total of 12,861 adult mosquitoes were collected at the 513 households, with 5 different genera collected, of which the Culex genus was the most abundant. Our study found that there was a positive association between livestock keeping and the size of the mosquito population-most predominantly between pig rearing and Culex species (p < 0.001). One pool of Cx. tritaeniorhynchus, collected in a peri-urban district, was found to be positive for Japanese encephalitis virus. Conclusions: The risk of flavivirus transmission in urban areas of Hanoi city due to the spread of Culex and Aedes mosquitoes could be facilitated by livestock keeping.

Diseases caused by flaviviruses, including dengue fever and Japanese encephalitis, are major health problems in Vietnam. This cross-sectional study explored the feasibility of domestic dogs as sentinels to better understand risks of mosquito-borne diseases in Hanoi city. A total of 475 dogs serum samples from 221 households in six districts of Hanoi were analyzed by a competitive enzyme-linked immunosorbent assay (cELISA) for antibodies to the pr-E protein of West Nile virus and other flaviviruses due to cross-reactivity. The overall flavivirus seroprevalence in the dog population was 70.7% (95% CI = 66.4-74.8%). At the animal level, significant associations between seropositive dogs and district location, age, breed and keeping practice were determined. At the household level, the major risk factors were rural and peri-urban locations, presence of pigs, coil burning and households without mosquito-borne disease experience (p < 0.05). Mosquito control by using larvicides or electric traps could lower seropositivity, but other measures did not contribute to significant risk mitigation of flavivirus exposure in dogs. These results will support better control of mosquito-borne diseases in Hanoi, and they indicate that dogs can be used as sentinels for flavivirus exposure.

Mosquito-borne diseases (MBDs) are causing high morbidity and mortality for humans. Urban livestock keeping is still common in cities around the world. The animals may serve as reservoirs for zoonotic MBDs, which increase the risks for humans. Here we assess the knowledge and practices related to MBDs in households with livestock and without livestock and explore the perceptions of the health care sector about MBDs and livestock keeping in Hanoi city of Vietnam in a cross-sectional study. A quantitative survey was conducted including 513 households with and without livestock-keeping in six districts and complemented with qualitative surveys with four health staff from Hanoi Center of Disease Control and three district health centers. The quantitative survey indicated that the participants possessed basic knowledge on MBDs with an average score of 18.3 out of 35, of which non-livestock-keeping households had a better knowledge than households keeping livestock (p<0.05). Both household categories had low score, 3.5 out of 11, regarding preventive practices against MBDs. The negative binomial model showed that occupation and location of living were factors associated to the knowledge on MBDs. Farmers were likely to have better preventive practices as compared to office workers (p<0.05). Those who had better knowledge also had more adequate preventive practices against MBDs (p<0.001). The qualitative survey revealed that livestock keeping was determined as increasing risks of MBDs due to the increase of mosquito population. It is recommended that community campaigns to raise the awareness and change behavior on MBDs should be organized based on collaboration between the health sector and the veterinary sector for households with and without livestock living in central urban and peri-urban areas. Further studies are needed to confirm the association between urban livestock keeping and potential increasing risks of MBDs such as dengue and Japanese encephalitis.

Under voltage load shedding has been considered as a standard approach to recover the voltage stability of the electric power grid under emergency conditions, yet this scheme usually trips a massive amount of load inefficiently. Reinforcement learning (RL) has been adopted as a promising approach to circumvent the issues; however, RL approach usually cannot guarantee the safety of the systems under control. In this paper, we discuss a couple of novel safe RL approaches, namely constrained optimization approach and Barrier function-based approach, that can safely recover voltage under emergency events. This method is general and can be applied to other safety-critical control problems. Numerical simulations on the 39-bus IEEE benchmark are performed to demonstrate the effectiveness of the proposed safe RL emergency control., Comment: Workshop on Safe and Robust Control of Uncertain Systems at the 35th Conference on Neural Information Processing Systems (NeurIPS) 2021. arXiv admin note: substantial text overlap with arXiv:2103.14186, arXiv:2011.09664, arXiv:2006.12667

Iterated coopetitive games capture the situation when one must efficiently balance between cooperation and competition with the other agents over time in order to win the game (e.g., to become the player with highest total utility). Achieving this balance is typically very challenging or even impossible when explicit communication is not feasible (e.g., negotiation or bargaining are not allowed). In this paper we investigate how an agent can achieve this balance to win in iterated coopetitive polymatrix games, without explicit communication. In particular, we consider a 3-player repeated game setting in which our agent is allowed to (slightly) manipulate the underlying game matrices of the other agents for which she pays a manipulation cost, while the other agents satisfy weak behavioural assumptions. We first propose a payoff matrix manipulation scheme and sequence of strategies for our agent that provably guarantees that the utility of any opponent would converge to a value we desire. We then use this scheme to design winning policies for our agent. We also prove that these winning policies can be found in polynomial running time. We then turn to demonstrate the efficiency of our framework in several concrete coopetitive polymatrix games, and prove that the manipulation costs needed to win are bounded above by small budgets. For instance, in the social distancing game, a polymatrix version of the lemonade stand coopetitive game, we showcase a policy with an infinitesimally small manipulation cost per round, along with a provable guarantee that, using this policy leads our agent to win in the long-run. Note that our findings can be trivially extended to $n$-player game settings as well (with $n > 3$).

Socialbots are software-driven user accounts on social platforms, acting autonomously (mimicking human behavior), with the aims to influence the opinions of other users or spread targeted misinformation for particular goals. As socialbots undermine the ecosystem of social platforms, they are often considered harmful. As such, there have been several computational efforts to auto-detect the socialbots. However, to our best knowledge, the adversarial nature of these socialbots has not yet been studied. This begs a question "can adversaries, controlling socialbots, exploit AI techniques to their advantage?" To this question, we successfully demonstrate that indeed it is possible for adversaries to exploit computational learning mechanism such as reinforcement learning (RL) to maximize the influence of socialbots while avoiding being detected. We first formulate the adversarial socialbot learning as a cooperative game between two functional hierarchical RL agents. While one agent curates a sequence of activities that can avoid the detection, the other agent aims to maximize network influence by selectively connecting with right users. Our proposed policy networks train with a vast amount of synthetic graphs and generalize better than baselines on unseen real-life graphs both in terms of maximizing network influence (up to +18%) and sustainable stealthiness (up to +40% undetectability) under a strong bot detector (with 90% detection accuracy). During inference, the complexity of our approach scales linearly, independent of a network's structure and the virality of news. This makes our approach a practical adversarial attack when deployed in a real-life setting., Comment: Accepted to The ACM Web Conference 2022

We study a novel setting in Online Markov Decision Processes (OMDPs) where the loss function is chosen by a non-oblivious strategic adversary who follows a no-external regret algorithm. In this setting, we first demonstrate that MDP-Expert, an existing algorithm that works well with oblivious adversaries can still apply and achieve a policy regret bound of $\mathcal{O}(\sqrt{T \log(L)}+\tau^2\sqrt{ T \log(|A|)})$ where $L$ is the size of adversary's pure strategy set and $|A|$ denotes the size of agent's action space. Considering real-world games where the support size of a NE is small, we further propose a new algorithm: MDP-Online Oracle Expert (MDP-OOE), that achieves a policy regret bound of $\mathcal{O}(\sqrt{T\log(L)}+\tau^2\sqrt{ T k \log(k)})$ where $k$ depends only on the support size of the NE. MDP-OOE leverages the key benefit of Double Oracle in game theory and thus can solve games with prohibitively large action space. Finally, to better understand the learning dynamics of no-regret methods, under the same setting of no-external regret adversary in OMDPs, we introduce an algorithm that achieves last-round convergence result to a NE. To our best knowledge, this is first work leading to the last iteration result in OMDPs., Comment: Accepted at Autonomous Agents and Multi-Agent Systems (2023)

We consider a sequential blocked matching (SBM) model where strategic agents repeatedly report ordinal preferences over a set of services to a central planner. The planner's goal is to elicit agents' true preferences and design a policy that matches services to agents in order to maximize the expected social welfare with the added constraint that each matched service can be \emph{blocked} or unavailable for a number of time periods. Naturally, SBM models the repeated allocation of reusable services to a set of agents where each allocated service becomes unavailable for a fixed duration. We first consider the offline SBM setting, where the strategic agents are aware of their true preferences. We measure the performance of any policy by \emph{distortion}, the worst-case multiplicative approximation guaranteed by any policy. For the setting with $s$ services, we establish lower bounds of $\Omega(s)$ and $\Omega(\sqrt{s})$ on the distortions of any deterministic and randomised mechanisms, respectively. We complement these results by providing approximately truthful, measured by \emph{incentive ratio}, deterministic and randomised policies based on random serial dictatorship which match our lower bounds. Our results show that there is a significant improvement if one considers the class of randomised policies. Finally, we consider the online SBM setting with bandit feedback where each agent is initially unaware of her true preferences, and the planner must facilitate each agent in the learning of their preferences through the matching of services over time. We design an approximately truthful mechanism based on the Explore-then-Commit paradigm, which achieves logarithmic dynamic approximate regret., Comment: AAAI-2022 Version

Inference for Deep Neural Networks is increasingly being executed locally on mobile and embedded platforms due to its advantages in latency, privacy and connectivity. Since modern System on Chips typically execute a combination of different and dynamic workloads concurrently, it is challenging to consistently meet inference time/energy budget at runtime because of the local computing resources available to the DNNs vary considerably. To address this challenge, a variety of dynamic DNNs were proposed. However, these works have significant memory overhead, limited runtime recoverable compression rate and narrow dynamic ranges of performance scaling. In this paper, we present a dynamic DNN using incremental training and group convolution pruning. The channels of the DNN convolution layer are divided into groups, which are then trained incrementally. At runtime, following groups can be pruned for inference time/energy reduction or added back for accuracy recovery without model retraining. In addition, we combine task mapping and Dynamic Voltage Frequency Scaling (DVFS) with our dynamic DNN to deliver finer trade-off between accuracy and time/power/energy over a wider dynamic range. We illustrate the approach by modifying AlexNet for the CIFAR10 image dataset and evaluate our work on two heterogeneous hardware platforms: Odroid XU3 (ARM big.LITTLE CPUs) and Nvidia Jetson Nano (CPU and GPU). Compared to the existing works, our approach can provide up to 2.36x (energy) and 2.73x (time) wider dynamic range with a 2.4x smaller memory footprint at the same compression rate. It achieved 10.6x (energy) and 41.6x (time) wider dynamic range by combining with task mapping and DVFS., Comment: Accepted at ACM/IEEE Workshop on Machine Learning for CAD (MLCAD) 2019

Machine learning inference is increasingly being executed locally on mobile and embedded platforms, due to the clear advantages in latency, privacy and connectivity. In this paper, we present approaches for online resource management in heterogeneous multi-core systems and show how they can be applied to optimise the performance of machine learning workloads. Performance can be defined using platform-dependent (e.g. speed, energy) and platform-independent (accuracy, confidence) metrics. In particular, we show how a Deep Neural Network (DNN) can be dynamically scalable to trade-off these various performance metrics. Achieving consistent performance when executing on different platforms is necessary yet challenging, due to the different resources provided and their capability, and their time-varying availability when executing alongside other workloads. Managing the interface between available hardware resources (often numerous and heterogeneous in nature), software requirements, and user experience is increasingly complex., Comment: Accepted at DATE 2020

Under voltage load shedding has been considered as a standard and effective measure to recover the voltage stability of the electric power grid under emergency and severe conditions. However, this scheme usually trips a massive amount of load which can be unnecessary and harmful to customers. Recently, deep reinforcement learning (RL) has been regarded and adopted as a promising approach that can significantly reduce the amount of load shedding. However, like most existing machine learning (ML)-based control techniques, RL control usually cannot guarantee the safety of the systems under control. In this paper, we introduce a novel safe RL method for emergency load shedding of power systems, that can enhance the safe voltage recovery of the electric power grid after experiencing faults. Unlike the standard RL method, the safe RL method has a reward function consisting of a Barrier function that goes to minus infinity when the system state goes to the safety bounds. Consequently, the optimal control policy, that maximizes the reward function, can render the power system to avoid the safety bounds. This method is general and can be applied to other safety-critical control problems. Numerical simulations on the 39-bus IEEE benchmark is performed to demonstrate the effectiveness of the proposed safe RL emergency control, as well as its adaptive capability to faults not seen in the training., Comment: IEEE CDC 2021. arXiv admin note: substantial text overlap with arXiv:2011.09664

We study bandit algorithms under data poisoning attacks in a bounded reward setting. We consider a strong attacker model in which the attacker can observe both the selected actions and their corresponding rewards and can contaminate the rewards with additive noise. We show that any bandit algorithm with regret $O(\log T)$ can be forced to suffer a regret $\Omega(T)$ with an expected amount of contamination $O(\log T)$. This amount of contamination is also necessary, as we prove that there exists an $O(\log T)$ regret bandit algorithm, specifically the classical UCB, that requires $\Omega(\log T)$ amount of contamination to suffer regret $\Omega(T)$. To combat such attacks, our second main contribution is to propose verification based mechanisms, which use limited verification to access a limited number of uncontaminated rewards. In particular, for the case of unlimited verifications, we show that with $O(\log T)$ expected number of verifications, a simple modified version of the ETC type bandit algorithm can restore the order optimal $O(\log T)$ regret irrespective of the amount of contamination used by the attacker. We also provide a UCB-like verification scheme, called Secure-UCB, that also enjoys full recovery from any attacks, also with $O(\log T)$ expected number of verifications. To derive a matching lower bound on the number of verifications, we prove that for any order-optimal bandit algorithm, this number of verifications $\Omega(\log T)$ is necessary to recover the order-optimal regret. On the other hand, when the number of verifications is bounded above by a budget $B$, we propose a novel algorithm, Secure-BARBAR, which provably achieves $O(\min\{C,T/\sqrt{B} \})$ regret with high probability against weak attackers where $C$ is the total amount of contamination by the attacker, which breaks the known $\Omega(C)$ lower bound of the non-verified setting if $C$ is large., Comment: Accepted to AAAI 2022

This paper presents a novel hierarchical deep reinforcement learning (DRL) based design for the voltage control of power grids. DRL agents are trained for fast, and adaptive selection of control actions such that the voltage recovery criterion can be met following disturbances. Existing voltage control techniques suffer from the issues of speed of operation, optimal coordination between different locations, and scalability. We exploit the area-wise division structure of the power system to propose a hierarchical DRL design that can be scaled to the larger grid models. We employ an enhanced augmented random search algorithm that is tailored for the voltage control problem in a two-level architecture. We train area-wise decentralized RL agents to compute lower-level policies for the individual areas, and concurrently train a higher-level DRL agent that uses the updates of the lower-level policies to efficiently coordinate the control actions taken by the lower-level agents. Numerical experiments on the IEEE benchmark 39-bus model with 3 areas demonstrate the advantages and various intricacies of the proposed hierarchical approach., Comment: 8 pages, 13 figures

In this work, we study sequential choice bandits with feedback. We propose bandit algorithms for a platform that personalizes users' experience to maximize its rewards. For each action directed to a given user, the platform is given a positive reward, which is a non-decreasing function of the action, if this action is below the user's threshold. Users are equipped with a patience budget, and actions that are above the threshold decrease the user's patience. When all patience is lost, the user abandons the platform. The platform attempts to learn the thresholds of the users in order to maximize its rewards, based on two different feedback models describing the information pattern available to the platform at each action. We define a notion of regret by determining the best action to be taken when the platform knows that the user's threshold is in a given interval. We then propose bandit algorithms for the two feedback models and show that upper and lower bounds on the regret are of the order of $\tilde{O}(N^{2/3})$ and $\tilde\Omega(N^{2/3})$, respectively, where $N$ is the total number of users. Finally, we show that the waiting time of any user before receiving a personalized experience is uniform in $N$.

In the updated NDC, Vietnam has allocated mitigation targets to energy, agriculture, industrial processes and product use (IPPU), land use, land use and forestry change (LULUCF) and waste in the period of 2021-2030. The establishment of a measurement, reporting and verification system (MRV) at national and sectoral levels is necessary to track the progress towards national and sectoral mitigation targets. However, at present very few studies on MRV indicators and remain fragmented. To meet that urgent need, this study was implemented to develop MRV indicators for mitigation actions to support the policy makers in tracking the NDC implementation. In this paper, a set of 85 MRV indicators divided into two categories: (i) 12 outcome indicators to track the national and sectoral mitigation targets, (ii) 72 progress indicators to track the implementation of mitigation options (including 40 for energy, 14 for agriculture, 7 for LULUCF, 4 for IPPU and 7 for waste) were developed based on relevant studies and expert consultation. The paper also tested the application of progress indicators in several mitigation options in the energy sector. The result showed that in energy sector, in 2014 the two mitigation options that have highest progress include: High efficiency residential refrigerators (0.85) and Introduction of CNG buses (0.75). Freight transport shift from road to railway and Cleaner cooking fuels had average values of progress indicators, 0.53 and 0.6, with respectively. The indicator values of the other tested mitigation options were all below 0.4 and hence greater efforts is needed to reach the mitigation targets.

Dengue is a growing problem in Hanoi, with cyclical epidemics of increasing frequency and magnitude. In June 2019, we conducted a cross-sectional survey using mixed methods to investigate how inhabitants of Hanoi perceive and respond to the risk of mosquito-borne diseases (MBD). A total of 117 participants recruited using a stratified random sampling method were interviewed in three districts of Hanoi. Knowledge and practices (KP) regarding MBDs were assessed using a pre-tested questionnaire. Inferential statistics were used to identify factors associated with KP scores and describe the relationship between variables. Additionally, a "risk-mapping" exercise was conducted in a subsample through semi-structured interviews and analyzed qualitatively and quantitatively using the System Effects platform. Factors significantly associated with knowledge scores were education and family history of MBDs. While knowledge and practice scores were found to be positively correlated in the statistical analysis, this was not corroborated by our observations on the field. The results also revealed gaps in knowledge about MBDs and vectors and highlighted a general feeling of powerlessness which prevented the adoption of protective behaviors. Therefore, educational interventions which provide concrete tools to empower communities should have a positive impact on improving vector control.

The rapid urban growth of Hanoi city requires a livestock production system that ensures both food security and the livelihoods of dwellers. This paper reviews the existing policies for livestock production of Hanoi city and the changes in livestock population between 2014 and 2018 and identifies major challenges for livestock development of the city. While a remarkable increase of the livestock population in recent years is evident, the dominance of small-scale farms, the presence of animal diseases, the slow progress of transiting farms out of urban areas, as well as the lack of analysis of climate change and gender impacts are major challenges that could affect the livestock development of Hanoi.

We investigate a repeated two-player zero-sum game setting where the column player is also a designer of the system, and has full control on the design of the payoff matrix. In addition, the row player uses a no-regret algorithm to efficiently learn how to adapt their strategy to the column player's behaviour over time in order to achieve good total payoff. The goal of the column player is to guide her opponent to pick a mixed strategy which is favourable for the system designer. Therefore, she needs to: (i) design an appropriate payoff matrix $A$ whose unique minimax solution contains the desired mixed strategy of the row player; and (ii) strategically interact with the row player during a sequence of plays in order to guide her opponent to converge to that desired behaviour. To design such a payoff matrix, we propose a novel solution that provably has a unique minimax solution with the desired behaviour. We also investigate a relaxation of this problem where uniqueness is not required, but all the minimax solutions have the same mixed strategy for the row player. Finally, we propose a new game playing algorithm for the system designer and prove that it can guide the row player, who may play a \emph{stable} no-regret algorithm, to converge to a minimax solution., Comment: Accepted at International Foundation for Autonomous Agents and Multiagent Systems (AAMAS 2022)

Persistence diagrams concisely represent the topology of a point cloud whilst having strong theoretical guarantees, but the question of how to best integrate this information into machine learning workflows remains open. In this paper we extend the ubiquitous Fuzzy c-Means (FCM) clustering algorithm to the space of persistence diagrams, enabling unsupervised learning that automatically captures the topological structure of data without the topological prior knowledge or additional processing of persistence diagrams that many other techniques require. We give theoretical convergence guarantees that correspond to the Euclidean case, and empirically demonstrate the capability of our algorithm to capture topological information via the fuzzy RAND index. We end with experiments on two datasets that utilise both the topological and fuzzy nature of our algorithm: pre-trained model selection in machine learning and lattices structures from materials science. As pre-trained models can perform well on multiple tasks, selecting the best model is a naturally fuzzy problem; we show that fuzzy clustering persistence diagrams allows for model selection using the topology of decision boundaries. In materials science, we classify transformed lattice structure datasets for the first time, whilst the probabilistic membership values let us rank candidate lattices in a scenario where further investigation requires expensive laboratory time and expertise., Comment: Version 4

This paper considers repeated games in which one player has more information about the game than the other players. In particular, we investigate repeated two-player zero-sum games where only the column player knows the payoff matrix A of the game. Suppose that while repeatedly playing this game, the row player chooses her strategy at each round by using a no-regret algorithm to minimize her (pseudo) regret. We develop a no-instant-regret algorithm for the column player to exhibit last round convergence to a minimax equilibrium. We show that our algorithm is efficient against a large set of popular no-regret algorithms of the row player, including the multiplicative weight update algorithm, the online mirror descent method/follow-the-regularized-leader, the linear multiplicative weight update algorithm, and the optimistic multiplicative weight update., Comment: Accepted at International Conference on Algorithmic Learning Theory, PMLR 132:553-577, 2021

Distributed Constraint Optimization Problems (DCOPs) are an important framework for modeling coordinated decision-making problems in multi-agent systems with a set of discrete variables. Later works have extended DCOPs to model problems with a set of continuous variables, named Functional DCOPs (F-DCOPs). In this paper, we combine both of these frameworks into the Mixed Integer Functional DCOP (MIF-DCOP) framework that can deal with problems regardless of their variables' type. We then propose a novel algorithm $-$ Distributed Parallel Simulated Annealing (DPSA), where agents cooperatively learn the optimal parameter configuration for the algorithm while also solving the given problem using the learned knowledge. Finally, we empirically evaluate our approach in DCOP, F-DCOP, and MIF-DCOP settings and show that DPSA produces solutions of significantly better quality than the state-of-the-art non-exact algorithms in their corresponding settings., Comment: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence Main track. Pages 268-275

In a purely inverter-based microgrid, both grid-forming (GFM) and grid-following (GFL) inverters will have a crucial role to play in frequency/voltage regulation and maintaining power sharing through their grid support capabilities. Consequently, the coordination between these two technologies becomes increasingly important for optimal system performance. However, the existing work does not consider GFL's potential to participate in a secondary control in coordination with GFM, thus not able to utilize the full capability of inverter resources. In this paper, we show that it is possible to fully coordinate the GFL and GFM inverters to achieve accurate power sharing, frequency/voltage regulation, and circulating var mitigation in networked microgrids even without the support of any synchronous generators or the bulk power system. We use the leader-follower consensus framework to develop a GFM-GFL coordination control. The effectiveness of the proposed coordination is verified under different disturbances and communication degradation. In addition, we find that the proposed fully-coordinated secondary control outperforms other approaches such as un-coordinated and partially-coordinated secondary controls, in aspects of load sharing and frequency and voltage regulation. Overall, this study emphasizes the need and benefits of GFL-GFM coordination in microgrids.

The paradigm shift in the electric power grid necessitates a revisit of existing control methods to ensure the grid's security and resilience. In particular, the increased uncertainties and rapidly changing operational conditions in power systems have revealed outstanding issues in terms of either speed, adaptiveness, or scalability of the existing control methods for power systems. On the other hand, the availability of massive real-time data can provide a clearer picture of what is happening in the grid. Recently, deep reinforcement learning(RL) has been regarded and adopted as a promising approach leveraging massive data for fast and adaptive grid control. However, like most existing machine learning (ML)-basedcontrol techniques, RL control usually cannot guarantee the safety of the systems under control. In this paper, we introduce a novel method for safe RL-based load shedding of power systems that can enhance the safe voltage recovery of the electric power grid after experiencing faults. Numerical simulations on the 39-bus IEEE benchmark is performed to demonstrate the effectiveness of the proposed safe RL emergency control, as well as its adaptive capability to faults not seen in the training., Comment: arXiv admin note: text overlap with arXiv:2006.12667

This paper discusses learning a structured feedback control to obtain sufficient robustness to exogenous inputs for linear dynamic systems with unknown state matrix. The structural constraint on the controller is necessary for many cyber-physical systems, and our approach presents a design for any generic structure, paving the way for distributed learning control. The ideas from reinforcement learning (RL) in conjunction with control-theoretic sufficient stability and performance guarantees are used to develop the methodology. First, a model-based framework is formulated using dynamic programming to embed the structural constraint in the linear quadratic regulator (LQR) setting along with sufficient robustness conditions. Thereafter, we translate these conditions to a data-driven learning-based framework - robust structured reinforcement learning (RSRL) that enjoys the control-theoretic guarantees on stability and convergence. We validate our theoretical results with a simulation on a multi-agent network with 6 agents., Comment: 16 pages, 4 figures. arXiv admin note: substantial text overlap with arXiv:2011.01128

This paper delves into designing stabilizing feedback control gains for continuous linear systems with unknown state matrix, in which the control is subject to a general structural constraint. We bring forth the ideas from reinforcement learning (RL) in conjunction with sufficient stability and performance guarantees in order to design these structured gains using the trajectory measurements of states and controls. We first formulate a model-based framework using dynamic programming (DP) to embed the structural constraint to the Linear Quadratic Regulator (LQR) gain computation in the continuous-time setting. Subsequently, we transform this LQR formulation into a policy iteration RL algorithm that can alleviate the requirement of known state matrix in conjunction with maintaining the feedback gain structure. Theoretical guarantees are provided for stability and convergence of the structured RL (SRL) algorithm. The introduced RL framework is general and can be applied to any control structure. A special control structure enabled by this RL framework is distributed learning control which is necessary for many large-scale cyber-physical systems. As such, we validate our theoretical results with numerical simulations on a multi-agent networked linear time-invariant (LTI) dynamic system., Comment: 13 pages, 7 figures