Dynamics analysis of a novel hybrid deep clustering for unsupervised learning by reinforcement of multi-agent to energy saving in intelligent buildings.

[Display omitted] • Clustering based hybrid network structure is used to tackle an extremely large state-action space. • Converting the TS inference into hybrid layers enables HDCMARL to deal with the continuous actions space. • Clustering structure generated by novel TSF rules for systemizing multi-agent policy. • Quasi-Newton algorithm is well tuning the parameters and weights of policy for storing at hybrid layers. • The investigation for the HDCMARL performance saving more than 32% of HVAC energy. The heating, ventilating and air conditioning (HVAC) systems energy demand can be reduced by manipulating indoor conditions within the comfort range, which relates to control performance and, simultaneously, achieves peak load shifting toward off-peak hours. Reinforcement learning (RL) is considered a promising technique to solve this problem without an analytical approach, but it has been unable to overcome the awkwardness of an extremely large action space in the real world; it would be quite hard to converge to a set point. The core of the problem with RL is its state space and action space of multi-agent action for building and HVAC systems that have an extremely large amount of training data sets. This makes it difficult to create weights layers accurately of the black-box model. Despite the efforts of past works carried out on deep RL, there are still drawback issues that have not been dealt with as part of the basic elements of large action space and the large-scale nonlinearity due to high thermal inertia. The hybrid deep clustering of multi-agent reinforcement learning (HDCMARL) has the ability to overcome these challenges since the hybrid deep clustering approach has a higher capacity for learning the representation of large space and massive data. The framework of RL agents is a greedy iterative trained and organized as a hybrid layer clustering structure to be able to deal with a non-convex, non-linear and non-separable objective function. The parameters of the hybrid layer are optimized by using the Quasi-Newton (QN) algorithm for fast response signals of agents. That is to say, the main motivation is that the state and action space of multi-agent actions for building HVAC controls are exploding, and the proposed method can overcome this challenge and achieve 32% better performance in energy savings and 21% better performance in thermal comfort than PID. [ABSTRACT FROM AUTHOR]