Search

Recommender systems help Internet users quickly find information they may be interested in from an extremely large amount of resources. Recent studies have shown that incorporating auxiliary social trust relationship information into the recommender system improves the accuracy of recommendations. Most existing research only considers explicit trust relationships, which result in sub-optimal recommendation performance. In this research, we present a trust model which analyses user trustworthiness based on user’s behaviours on the social networks. The proposed trust model increases the density of trust relationships by considering explicit and implicit social trust relationships and also reflects a more fine-grained and realistic trust level between users. This improved social trust information is then incorporated into TrustSVD, a matrix factorisation–based social recommendation method. By analysing the prediction result using a real-world data set, Douban-600k from the Douban Movie website, we found that our proposed method provides more accurate predictions compared with SVD++ and traditional TrustSVD, improving users’ experiences.

Load forecasting has been deeply studied because of its critical role in Smart Grid. In current Smart Grid, there are various types of customers with different energy consumption patterns. Customer’s energy consumption patterns are referred to as customer behaviors. It would significantly benefit load forecasting in a grid if customer behaviors could be taken into account. This paper proposes an innovative method that aggregates different types of customers by their identified behaviors, and then predicts the load of each customer cluster, so as to improve load forecasting accuracy of the whole grid. Sparse Continuous Conditional Random Fields (sCCRF) is proposed to effectively identify different customer behaviors through learning. A hierarchical clustering process is then introduced to aggregate customers according to the identified behaviors. Within each customer cluster, a representative sCCRF is fine-tuned to predict the load of its cluster. The final load of the whole grid is obtained by summing the loads of each cluster. The proposed method for load forecasting in Smart Grid has two major advantages. 1) Learning customer behaviors not only improves the prediction accuracy but also has a low computational cost. 2) sCCRF can effectively model the load forecasting problem of one customer, and simultaneously select key features to identify its energy consumption pattern. Experiments conducted from different perspectives demonstrate the advantages of the proposed load forecasting method. Further discussion is provided, indicating that the approach of learning customer behaviors can be extended as a general framework to facilitate decision making in other market domains.

Agent-based scheduling refers to applying intelligent agents to autonomously allocate resources to jobs. Decentralized agent-based scheduling approaches have achieved good performance in open and dynamic environments because the relationships of agents are flexible. For new jobs and resources and unexpected events, decentralized agents can respond adaptively and flexibly. Besides, decentralized approaches are easy to be extended because there is no central control agent that limits the scalability. However, decentralized approaches might have low efficiency in large-scale environments because behaviors of agents may be self-interested and competitive, due to their local views during decision making. When interacting with a large number of agents, each agent may spend a considerable amount of time on failed attempts before reaching the final agreements with other agents. To improve the efficiency of decentralized agent-based scheduling approaches in large-scale environments, and to keep the flexibility and adaptability of decentralized agents for the decision-making on scheduling, this paper provides a new agent-based adaptive mechanism for efficient job scheduling. A new type of agent named host agent is introduced to coordinate self-interested behaviors of agents without participating in the decision making of agents during job scheduling. The proposed mechanism was developed in JADE and tested in open and large-scale environments. The experimental results indicate that the proposed mechanism is effective and efficient in open and large-scale environments.

A broker in an open e-marketplace enables buyers and sellers to do business with each other. Although a broker plays an important role in e-marketplaces, theory and guidelines for matching between buyers and sellers in multi-attribute exchanges are limited. Therefore, a challenge for a broker’s responsibility is how to maximize a buyer’s total satisfaction degree as its goals under the consideration of trade-off between a buyer’s buying quantity and price paid to a seller, and other attributes. To solve this challenge, this paper proposes an economic model-based matching approach between a buyer’s requirements and a seller’s offers. The major contributions of this paper are that (i) a broker can model a seller’s price policy as per a buyer’s buying quantity through communication between a broker and a seller; (ii) due to each buyer’s different quantity demand, a broker models a buyer’s satisfaction degree as per a buyer’s buying quantity based on communication between a broker and a buyer; and (iii) to carry out a broker’s matching processes, an objective function and a set of constraints are generated to help a broker to maximize a buyer’s total satisfaction degree. Experimental results demonstrate the good performance of the proposed approach.

Smart Grid markets are dynamic and complex, and brokers are widely introduced to better manage the markets. However, brokers face great challenges, including the varying energy demands of consumers, the changing prices in the markets, and the competitions between each other. This paper proposes an intelligent broker model based on hybrid learning (including unsupervised, supervised and reinforcement learning), which generates smart trading strategies to adapt to the dynamics and complexity of Smart Grid markets. The proposed broker model comprises three interconnected modules. Customer demand prediction module predicts short-term demands of various consumers with a data-driven method. Wholesale market module employs a Markov Decision Process for the one-day-ahead power auction based on the predicted demand. Retail market module introduces independent reinforcement learning processes to optimize prices for different types of consumers to compete with other brokers in the retail market. We evaluate the proposed broker model on Power TAC platform. The experimental results show that our broker is not only is competitive in making profit, but also maintains a good supply-demand balance. In addition, we also discover two empirical laws in the competitive power market environment, which are: 1. profit margin shrinks when there are fierce competitions in markets; 2. the imbalance rate of supply demand increases when the market environment is more competitive.

Emergency incidents happen frequently and cause huge loss of both lives and properties. However, in the traditional emergency response cycle, the involvement of human operators and resource dispatchers make bad decisions in some cases and the communications between different departments are in low efficiencies. Thus, Effective and rapid emergency response is very necessary. This paper discusses some existing approaches which can assist the emergency responses then propose a multi-agent-based system to model and simulate the emergency response cycle in real life in order to improve the efficiency of the emergency response.

Reinforcement learning agents can be helped by the knowledge transferred from experienced agents. This paper studies the problem of how an experienced agent helps another agent learn when they have different learning goals by action transfer. This problem is motivated by the widely existing situations where agents have different learning goals and only action transfer is available to agents. To tackle the problem, we propose an approach to facilitate the transfer of actions that are right to a learning agent’s goal. Experimental results show the effectiveness of the proposed approach in transferring right actions to an agent and helping the agent learn to reach a different goal.

Social conventions have been used as an efficient mechanism to facilitate coordination among agents. Establishing a convention in a decentralised manner has attracted much attention in the literature. Existing techniques on convention emergence are not robust. These techniques may establish sub-conventions under particular network structures. The emergence of sub-conventions indicates that agents in a society fail to conform to a single convention. As a result, the coordination among these agents is negatively affected. In this paper, we propose a strategy to avoid sub-conventions under diverse network structures. The proposed strategy requires agents to only have local views. We prove that a convention can be established using the proposed strategy. We also give empirical studies on the speed of convention emergence with various experimental settings

Action advice is an important mechanism to improve the learning speed of multiple agents. To do so, an advisor agent suggests actions to an advisee agent. In the current advising approaches, the advisor’s advice is always applicable based on the assumption that the advisor and advisee have the same objective, and the environment is stable. However, in many real-world applications, the advisor and advisee may have different objectives, and the environment may be dynamic. This would make the advisor’s advice not always applicable. In this paper, we propose an approach where the advisor and advisee jointly determine the applicability of advice by considering the different objectives and dynamic changes in the environment. The proposed approach is evaluated in various robot navigation domains. The evaluation results show that the proposed approach can determine the applicability of advice. The multi-agent learning speed can also be improved benefiting from determined applicable advice.

Regressions on Sequential Data (RSD) are widely used in different disciplines. This paper proposes DeepRSD, which utilizes several different neural networks to result in an effective end-to-end learning method for RSD problems. There have been several variants of deep Recurrent Neural Networks (RNNs) in classification problems. The main functional part of DeepRSD is the stacked bi-directional RNNs, which is the most suitable deep RNN model for sequential data. We explore several conditions to ensure a plausible training of DeepRSD. More importantly, we propose an alternative dropout to improve its generalization. We apply DeepRSD to two different real-world problems and achieve state-of-the-art performances. Through comparisons with state-of-the-art methods, we conclude that DeepRSD can be a competitive method for RSD problems.

Social norms such as social rules and conventions play a pivotal role in sustaining system order by regulating and controlling individual behaviors toward a global consensus in large-scale distributed systems. Systematic studies of efficient mechanisms that can facilitate the emergence of social norms enable us to build and design robust distributed systems, such as electronic institutions and norm-governed sensor networks. This paper studies the emergence of social norms via learning from repeated local interactions in networked multiagent systems. A collective learning framework, which imitates the opinion aggregation process in human decision making, is proposed to study the impact of agent local collective behaviors on the emergence of social norms in a number of different situations. In the framework, each agent interacts repeatedly with all of its neighbors. At each step, an agent first takes a best-response action toward each of its neighbors and then combines all of these actions into a final action using ensemble learning methods. Extensive experiments are carried out to evaluate the framework with respect to different network topologies, learning strategies, numbers of actions, influences of nonlearning agents, and so on. Experimental results reveal some significant insights into the manipulation and control of norm emergence in networked multiagent systems achieved through local collective behaviors.

In traditional policy generation models, the preferences over polices are often represented by qualitative orderings due to the difficulty of acquisition of accurate utility. Thus, it is difficult to evaluate agreements in such models so that players cannot adjust their strategies during a policy generation process. To this end, this paper introduces a negotiation-based model for policy generation, which contains two evaluation methods, both from the perspectives of concessional utilities and consistency, to guide players to make decisions flexibly. The first method is used to model humans’ reasoning about how to calculate concessional utilities from uncertain preference information of policies based on fuzzy reasoning, while the second method is used to measure similarity between an ideal agreement and an offer based on a prioritised consistency degree. The experimental results show the difference between the evaluation methods and confirm that the proposed model and evaluation methods can help players achieve better agreements than an existing model.

Electronic Commerce has been a significant commercial phenomenon in recent years, and brings more benefits to people by comparison with the traditional market in aspects of cost, convenience and efficiency. The use of agent technology in e-markets for automatic bargains between buyers and sellers further increases the advantages of the e-market. However, most of existing agent-based bargain strategies assume a fixed number of negotiation participants, which may fail to enlarge agents' profits or to lead a bargain to a success when these strategies are applied in the e-market-based agent negotiations straightway. Problems such as unexpected changes on negotiation participants, possible changes on agents' expected negotiation outcomes, and unexpected switching in-between the buyer's and seller's markets need to be considered in order to guarantee agents' benefits and the success of negotiations. This paper proposes a novel agent negotiation model to help agents to perform a more effective bargain in e-markets by considering the objectiveness of the e-markets and the subjectiveness of the agents. The e-market situation by considering the number of bargain participants is proposed to reflect the objectiveness of the e-markets, and the agents' negotiation attitudes is introduced to indicate agents' responses to possible changes of the e-markets. Both the objectiveness of e-markets and the subjectiveness of agents are taken into account in negotiation procedures such as offer evaluation, negotiation decision making and counter-offer generation. Experimental results on a simulated e-market illustrate the benefits and efficiency of the proposed negotiation model in handling agents bargain problem in complex and dynamic e-market environments.

Bilateral agent negotiation is considered as a fundamental research issue in autonomous agent negotiation, and was studied well by researchers. Generally, a predefined negotiation decision function and utility function are used to generate an offer in each negotiation round according to a negotiator's negotiation strategy, preference, and restrictions. However, such a negotiation procedure may not work well when the negotiator's utility function is nonlinear, and the unique offer is difficult to be generated. That is because if the negotiator's utility function is non-monotonic, the negotiator may find several offers that come with the same utility at the same time; and if the negotiator's utility function is discrete, the negotiator may not find an offer to satisfy its expected utility exactly. In order to solve such a problem, we propose a novel negotiation model in this paper. Firstly, a 3D model is introduced to illustrate the relationships between an agent's utility function, negotiation decision function and offer generation function. Then two negotiation mechanisms are proposed to handle two types of nonlinear utility functions respectively, i.e. a multiple offer mechanism is introduced to handle non-monotonic utility functions, and an approximating offer mechanism is introduced to handle discrete utility functions. Lastly, a combined negotiation mechanism is proposed to handle nonlinear utility functions in general situations by considering both the non-monotonic and discrete. The experimental results demonstrate the effectiveness and efficiency of the proposed negotiation model.

A broker in a market enables buyers and sellers to do business with each other and can provide many value-adding functions that cannot be replaced by direct buyer-seller dealings. Recently, some research has focused on this issue. However, broker modelling based on buyer’s membership functions to carry out a matching process between buyer’s requirements in fuzzy preference information and seller’s offers is still sparse. Thus, this paper proposes membership function based matching approach of buyers and sellers through a broker in open e-marketplace. The major contributions of this paper are that (i) a proposed framework is applicable to help a broker to carry out the matching process between buyers and sellers; (ii) a proposed method is to determine buyer’s attribute weight with soft constraints by using association rule mining; and (iii) an objective optimization function and a set of constraints are built to help a broker to maximize buyer’s total utility. Experimental results demonstrate the good performance of the proposed approach in terms of satisfying buyer’s requirements and maximizing buyer’s total utility.

In metropolitan regions, emergency events request urgent response within a short time limit in order to minimise the damage and the number of fatality. Most of these events require different resources that are usually distributed over a large area. How to efficiently allocate the distributed resources to an event is a challenging research issue. Traditional centralised resource allocation approaches have difficulties to find out the best resource assignment within the event’s time limits by considering the dynamics of the metropolitan environment and the event itself. In this paper, a multiagent-based decentralised resource allocation approach using domain transportation theory is proposed to handle an emergency event with multiple tasks. Experimental results indicates that the proposed approach can effectively generate the optimal resource allocation plans by considering multiple factors of an emergency event.

Concurrent multiple negotiation (CMN) provides a mechanism for an agent to simultaneously conduct more than one negotiation. There may exist different interdependency relationships among these negotiations and these interdependency relationships can impact the outcomes of these negotiations. The outcomes of these concurrent negotiations contribute together for the agent to achieve an overall negotiation goal. Handling a CMN while considering interdependency relationships among multiple negotiations is a challenging research problem. This paper: 1) comprehensively highlights research problems of negotiations at concurrent negotiation level; 2) provides a graph-based CMN model with consideration of the interdependency relationships; and 3) proposes a colored Petri net-based negotiation protocol for conducting CMNs. With the proposed protocol, a CMN can be efficiently and concurrently processed and negotiation agreements can be efficiently achieved. Experimental results indicate the effectiveness and efficiency of the proposed protocol in terms of the negotiation success rate, the negotiation time and the negotiation outcome.

Learning is an important capability of humans and plays a vital role in human society for forming beliefs and opinions. In this paper, we investigate how learning affects the dynamics of opinion formation in social networks. A novel learning model is proposed, in which agents can dynamically adapt their learning behaviours in order to facilitate the formation of consensus among them, and thus establish a consistent social norm in the whole population more efficiently. In the model, agents adapt their opinions through trail-and-error interactions with others. By exploiting historical interaction experience, a guiding opinion, which is considered to be the most successful opinion in the neighbourhood, can be generated based on the principle of evolutionary game theory. Then, depending on the consistency between its own opinion and the guiding opinion, a focal agent can realize whether its opinion complies with the social norm (i.e., the majority opinion that has been adopted) in the population, and adapt its behaviours accordingly. The highlight of the model lies in that it captures the essential features of people’s adaptive learning behaviours during the evolution and formation of opinions. Experimental results show that the proposed model can facilitate the formation of consensus among agents, and some critical factors such as size of opinion space and network topology can have significant influences on opinion dynamics.

Continuous Conditional Random Fields (CCRF) has been widely applied to various research domains as an efficient approach for structural regression. In previous studies, the weights of CCRF are constrained to be positive from a theoretical perspective. This paper extends the definition domains of weights of CCRF and thus introduces L1 norm to regularize CCRF, which enables CCRF to perform feature selection. We provide a plausible learning method for L1-Regularized CCRF (L1-CCRF) and verify its effectiveness. Moreover, we demonstrate that the proposed L1-CCRF performs well in selecting key features related to the various customers' power usages in Smart Grid.

In multi-issue negotiation, agents’ preferences are extremely important factors for reaching mutual beneficial agreements. However, agents would usually keeping their preferences in secret in order to avoid be exploited by their opponents during a negotiation. Thus, preference modelling has become an important research direction in the area of agent-based negotiation. In this paper, a bilateral multi-issue negotiation approach is proposed to help both negotiation agents to maximise their utilities under a setting that the opponent agent’s preference is private information. In the proposed approach, Bayesian learning is employed to analyse the opponent’s historical offers and approximately predicate the opponent’s preference over negotiation issues. Besides, a counter-offer proposition algorithm is integrated in our approach to help agents to generate mutual beneficial offers based on the preference learning result. Also, the experimental results indicate the good performance of the proposed approach in aspects of utility gain and negotiation efficiency.

In metropolitan regions, emergency events could happen concurrently at different places with different severities, types, deadlines and resource requirements. Due to the complexity, unpredictability, dynamic natures and potential resource contention problems among these events, traditional resource allocation approaches may have difficulties to efficiently and effectively deploy resources to these emergency events concurrently, which may result in a considerable increase in fatalities. In this paper, an multi-agent based decentralised resource allocation approach is proposed to coordinate and allocate resources to multiple concurrent emergency events. Besides, an emergency resource deployment simulation system based on GoogleMaps is developed for testing the proposed approach in a virtual metropolitan environment.

Concurrent Multiple Negotiation (CMN) mechanism is necessary for agents to achieve agreements in multiple negotiations, and it has become a very important research topic in multi-agent systems in recent years. However, in the open and dynamic negotiation environment, negotiations may be dynamically and concurrently initialized or terminated during the process of other existing negotiations. Therefore, how to process dynamic CMN becomes a serious challenge in agent negotiation research. The motivation of this paper is to propose an adaptive mechanism for handling dynamic CMN by considering the possible changes of concurrent negotiations. First, a formal mechanism for modeling and representing dynamic CMN is presented. Then, a novel Colored Petri Net-based CMN protocol for processing CMN with unexpected negotiation changes is presented. We also demonstrate the performance and procedure of the proposed approach in handling the dynamism of negotiations in CMN, and the experimental results show that the proposed approach can effectively handle unexpected changes in the CMN dynamically, and successfully lead the CMN to agreements.

This paper investigates how norm emergence can be facilitated by agents' adaptive learning behaviors in networked multiagent systems. A general learning framework is proposed, in which agents can dynamically adapt their learning behaviors through social learning of their individual learning experience. Extensive verification of the proposed framework is conducted in a variety of situations, using comprehensive evaluation criteria of efficiency, effectiveness and efficacy. Experimental results show that the adaptive learning framework is robust and efficient for evolving stable norms among agents.

SUMMARY Multiagent learning provides a promising paradigm to study how autonomous agents learn to achieve coordinated behavior in multiagent systems. In multiagent learning, the concurrency of multiple distributed learning processes makes the environment nonstationary for each individual learner. Developing an efficient learning approach to coordinate agents’ behavior in this dynamic environment is a difficult problem especially when agents do not know the domain structure and at the same time have only local observability of the environment. In this paper, a coordinated learning approach is proposed to enable agents to learn where and how to coordinate their behavior in loosely coupled multiagent systems where the sparse interactions of agents constrain coordination to some specific parts of the environment. In the proposed approach, an agent first collects statistical information to detect those states where coordination is most necessary by considering not only the potential contributions from all the domain states but also the direct causes of the miscoordination in a conflicting state. The agent then learns to coordinate its behavior with others through its local observability of the environment according to different scenarios of state transitions. To handle the uncertainties caused by agents’ local observability, an optimistic estimation mechanism is introduced to guide the learning process of the agents. Empirical studies show that the proposed approach can achieve a better performance by improving the average agent reward compared with an uncoordinated learning approach and by reducing the computational complexity significantly compared with a centralized learning approach. Copyright © 2012 John Wiley & Sons, Ltd.

Outages and faults in interconnected power systems may cause cascading sequences of events, and catastrophic failures of power systems. How to efficiently manage power systems and restore the systems from faults is a challenging research issue in power engineering. Multi-agent systems are employed to address such a challenge in recent years. A centralized coordination strategy was firstly introduced to manage agents in a power system. Such a strategy usually adopts a single central coordinator to control the whole system for system management, maintenance, and restoration purposes. However, disadvantages such as deficiencies in robustness, openness, and flexibility prevent this strategy from extensive online applications. Consequently, a decentralized coordination strategy was proposed to overcome such limitations. But the decentralized coordination strategy cannot efficiently provide a global solution when serious faults spread out in a power system. In this paper, a conceptual multi-agent system design is introduced to express our proposal in power system modeling. A novel dynamic team forming mechanism is proposed to dynamically manage agents in power system with a flexible coordination structure, so as to balance the effectiveness and efficiency of the introduced multi-agent system. The results from simulations of case studies indicate the performance of the proposed multi-agent model.

Electronic Commerce has been a very significant commercial phenomenon in recent years, and autonomous agents are widely adopted by business or individuals in electronic marketplaces to fulfill time consuming tasks in trading. Agent negotiation mechanisms are usually applied between conflicted agents in order to reach a mutually beneficial agreement. Prediction of trading agents' strategies and behaviours in negotiation is a very significant research topic in agent negotiation. By employing the prediction results on opponents' possible strategies and behaviours during a negotiation, trading agents can plan and perform corresponding strategies in order to maximize their own profits. Significant achievements have been made on this topic. However, most existing approaches are based on machine learning mechanisms, which may fail to capture opponents' behaviours in open and dynamic electronic marketplaces. In this paper, two agent behaviour expectation approaches are introduced to help trading agents to capture opponents' potential behaviours during a negotiation in complex e-marketplaces. i The regression analysis approach focuses on illustrating the main trends of opponents' trading behaviours; ii the vector analysis approach pays more attention to identifying opponents' detailed negotiation strategies. The experimental results show the efficiency and efficacy of the two proposed approaches in open and dynamic negotiation environments.

In Agent Based Scheduling and Planning Systems, autonomous agents are used to execute scheduling/planning tasks on behalves of represented enterprises. As application domains become more and more complex, agents are required to handle a number of changing and uncertain factors. This makes it necessary to embed state prediction mechanisms in Agent Based Scheduling and Planning Systems. In this paper, a Colored Petri Net based approach for supporting automated scheduling and planning is introduced. In the approach, we adopt an augmentation Colored Petri Net model which can not only analyse future states of a system but also estimate the success probability of reaching a particular future state. By using augmentation Colored Petri Nets to model relative dynamic factors in scheduling/planning problems, agents can predict the probable future states of a system and corresponding risks of reaching those states. The proposed approach can enable agents to make more rational and accurate decisions in complex scheduling and planning problems.

One of the crucial issues of automated negotiation in multi-agent systems is how to reach an agreement when a negotiation environment becomes open and dynamic. Even though some strategies have been proposed by researchers, most of them can only work within a static negotiation environment. In this paper, we present a model for designing a strategy for agents that makes adjustable rates of concession by negotiating according to the changes of environments with uncertain and dynamic outside options. This proposal is based on the market-driven agents (MDAs) model, and is guided by four factors in order to determine the degree of concession. These factors are trading opportunity, trading competition, trading time and strategy, and eagerness. The contribution of this paper is extending the MDAs model to an open and dynamic negotiation environment by considering both the current and potential changes of the environment.

A broker acts as a middleman between buyers and sellers in the trading processes to achieve its profit as well as to satisfy buyer's requirements based on seller's offers. This paper proposes a broker-based optimal matching approach in the markets. The major contributions of this paper include (1) an abstract model of a broker agent, that is applicable to a broad range of market types, (2) predicting buyers and sellers' behavior by using Bayes' rule so that a broker can identify an appropriate allocation of items between buyers and sellers, and (3) an objective function and a set of constraints to help a broker to maximize its profit under consideration of buyer and seller's total satisfaction. Experimental results demonstrate the good performance of the proposed approach in terms of satisfying buyer's requirements and maximizing broker's profit.

Social dilemmas have attracted extensive interest in the research of multiagent systems in order to study the emergence of cooperative behaviors among selfish agents. Understanding how agents can achieve cooperation in social dilemmas through learning from local experience is a critical problem that has motivated researchers for decades. This paper investigates the possibility of exploiting emotions in agent learning in order to facilitate the emergence of cooperation in social dilemmas. In particular, the spatial version of social dilemmas is considered to study the impact of local interactions on the emergence of cooperation in the whole system. A double-layered emotional multiagent reinforcement learning framework is proposed to endow agents with internal cognitive and emotional capabilities that can drive these agents to learn cooperative behaviors. Experimental results reveal that various network topologies and agent heterogeneities have significant impacts on agent learning behaviors in the proposed framework, and under certain circumstances, high levels of cooperation can be achieved among the agents.

Multiagent learning (MAL) is a promising technique for agents to learn efficient coordinated behaviors in multiagent systems (MASs). In MAL, concurrent multiple distributed learning processes can make the learning environment nonstationary for each individual learner. Developing an efficient learning approach to coordinate agents’ behaviors in this dynamic environment is a difficult problem, especially when agents do not know the domain structure and have only local observability of the environment. In this paper, a coordinated MAL approach is proposed to enable agents to learn efficient coordinated behaviors by exploiting agent independence in loosely coupled MASs. The main feature of the proposed approach is to explicitly quantify and dynamically adapt agent independence during learning so that agents can make a trade-off between a single-agent learning process and a coordinated learning process for an efficient decision making. The proposed approach is employed to solve two-robot navigation problems in different scales of domains. Experimental results show that agents using the proposed approach can learn to act in concert or independently in different areas of the environment, which results in great computational savings and near optimal performance.

In this paper, a hierarchical learning framework is proposed for emergence of social norms in networked multiagent systems. This framework features a bottom level of agents and several levels of supervisors. Agents in the bottom level interact with each other using reinforcement learning methods, and report their information to their supervisors after each interaction. Supervisors then aggregate the reported information and produce guide policies by exchanging information with other supervisors. The guide policies are then passed down to the subordinate agents in order to adjust their learning behaviors heuristically. Experiments are carried out to explore the efficiency of norm emergence under the proposed framework, and results verify that learning from local interactions integrating hierarchical supervision can be an effective mechanism for emergence of social norms.