Search

Introduction: Expressing emotions play a special role in daily communication, and one of the most essential methods in detecting emotions is to detect facial emotional states. Therefore, one of the crucial aspects of the natural human-machine interaction is the recognition of facial expressions and the creation of feedback, according to the perceived emotion. Methods: To implement each part of this model, two main steps have been introduced. The first step is reading the video and converting it to images and preprocessing on them. The next step is to use the combination of 3D convolutional neural network (3DCNN) and learning automata (LA) to classify and detect the rate of facial emotional recognition. The reason for choosing 3DCNN in our model is that no dimension is removed from the images, and considering the temporal information in dynamic images leads to more efficient and better classification. In addition, the training of the 3DCNN network in calculating the backpropagation error is adjusted by LA so that both the efficiency of the proposed model is increased, and the working memory part of the SOAR model can be implemented. Results and discussion: Due to the importance of the topic, this article presents an efficient method for recognizing emotional states from facial images based on a mixed deep learning and cognitive model called SOAR. Among the objectives of the proposed model, it is possible to mention providing a model for learning the time order of frames in the movie and providing a model for better display of visual features, increasing the recognition rate. The accuracy of recognition rate of facial emotional states in the proposed model is 85.3%. To compare the effectiveness of the proposed model with other models, this model has been compared with competing models. By examining the results, we found that the proposed model has a better performance than other models. [ABSTRACT FROM AUTHOR]

IntroductionExpressing emotions play a special role in daily communication, and one of the most essential methods in detecting emotions is to detect facial emotional states. Therefore, one of the crucial aspects of the natural human–machine interaction is the recognition of facial expressions and the creation of feedback, according to the perceived emotion.MethodsTo implement each part of this model, two main steps have been introduced. The first step is reading the video and converting it to images and preprocessing on them. The next step is to use the combination of 3D convolutional neural network (3DCNN) and learning automata (LA) to classify and detect the rate of facial emotional recognition. The reason for choosing 3DCNN in our model is that no dimension is removed from the images, and considering the temporal information in dynamic images leads to more efficient and better classification. In addition, the training of the 3DCNN network in calculating the backpropagation error is adjusted by LA so that both the efficiency of the proposed model is increased, and the working memory part of the SOAR model can be implemented.Results and discussionDue to the importance of the topic, this article presents an efficient method for recognizing emotional states from facial images based on a mixed deep learning and cognitive model called SOAR. Among the objectives of the proposed model, it is possible to mention providing a model for learning the time order of frames in the movie and providing a model for better display of visual features, increasing the recognition rate. The accuracy of recognition rate of facial emotional states in the proposed model is 85.3%. To compare the effectiveness of the proposed model with other models, this model has been compared with competing models. By examining the results, we found that the proposed model has a better performance than other models.

Clustering, an unsupervised machine learning technique, plays a crucial role in partitioning unlabeled data into meaningful groups. K-means, known for its simplicity, has gained popularity as a clustering method. However, both K-means and the LAC algorithm, which utilize learning automata, are sensitive to the selection of initial points. To overcome this limitation, we propose an enhanced LAC algorithm based on the K-Harmonic means approach. We evaluate its performance on seven datasets and demonstrate its superiority over other representative algorithms. Moreover, we tailor this algorithm to address the controller placement problem in software-defined networks, a critical field in this context. To optimize relevant parameters such as switch–controller delay, intercontroller delay, and load balancing, we leverage learning automata. In our comparative analysis conducted in Python, we benchmark our algorithm against spectral, K-means, and LAC algorithms on four different network topologies. The results unequivocally show that our proposed algorithm outperforms the others, achieving a significant improvement ranging from 3 to 11 percent. This research contributes to the advancement of clustering techniques and their practical application in software-defined networks. [ABSTRACT FROM AUTHOR]

It is critical to increasing the network throughput on the internet of things with short-range nodes. Nodes prevent to cooperate with other nodes are known as selfish nodes. The proposed method for discovering the selfish node is based on genetic algorithm and learning automata. It consists of three phases of setup and clustering, the best routing selection based on genetic algorithm, and finally, the learning and update phase. The clustering algorithm implemented in the first phase. In the second phase, the neighbor node selected for forwarding the packet in which has a high value of fitness function. In the third phase, each node monitors its neighbor nodes and uses the learning automata system to identify the selfish nodes. The results of the simulation has shown the detection accuracy of selfish nodes in comparison with the existing methods average 10 %, and the false positive rate has decreased by 5 %.

It is critical to increasing the network throughput on the internet of things with short-range nodes. Nodes prevent to cooperate with other nodes are known as selfish nodes. The proposed method for discovering the selfish node is based on genetic algorithm and learning automata. It consists of three phases of setup and clustering, the best routing selection based on genetic algorithm, and finally, the learning and update phase. The clustering algorithm implemented in the first phase. In the second phase, the neighbor node selected for forwarding the packet in which has a high value of fitness function. In the third phase, each node monitors its neighbor nodes and uses the learning automata system to identify the selfish nodes. The results of the simulation has shown the detection accuracy of selfish nodes in comparison with the existing methods average 10 %, and the false positive rate has decreased by 5 %. [ABSTRACT FROM AUTHOR]

Clustering, an unsupervised machine learning technique, plays a crucial role in partitioning unlabeled data into meaningful groups. K-means, known for its simplicity, has gained popularity as a clustering method. However, both K-means and the LAC algorithm, which utilize learning automata, are sensitive to the selection of initial points. To overcome this limitation, we propose an enhanced LAC algorithm based on the K-Harmonic means approach. We evaluate its performance on seven datasets and demonstrate its superiority over other representative algorithms. Moreover, we tailor this algorithm to address the controller placement problem in software-defined networks, a critical field in this context. To optimize relevant parameters such as switch–controller delay, intercontroller delay, and load balancing, we leverage learning automata. In our comparative analysis conducted in Python, we benchmark our algorithm against spectral, K-means, and LAC algorithms on four different network topologies. The results unequivocally show that our proposed algorithm outperforms the others, achieving a significant improvement ranging from 3 to 11 percent. This research contributes to the advancement of clustering techniques and their practical application in software-defined networks.

Sensor fusion has attracted a lot of research attention during the few last years. Recently, a new research direction has emerged dealing with sensor fusion without knowledge of the ground truth. In this article, we present a novel solution to the latter pertinent problem. In contrast to the first reported solutions to this problem, we present a solution that does not involve any assumption on the group average reliability which makes our results more general than previous works. We devise a strategic game where we show that a perfect partitioning of the sensors into reliable and unreliable groups corresponds to a Nash equilibrium of the game. Furthermore, we give sound theoretical results that prove that those equilibria are indeed the unique Nash equilibria of the game. We then propose a solution involving a team of learning automata (LA) to unveil the identity of each sensor, whether it is reliable or unreliable, using game-theoretic learning. The experimental results show the accuracy of our solution and its ability to deal with settings that are unsolvable by legacy works. [ABSTRACT FROM AUTHOR]

The evolution of cloud computing facilitates applications with varying demands to operate in a virtualized environment. For instance, applications like the healthcare system, video streaming, Internet of Things (IoT) that are moving to the cloud, demand responses within a particular time limit, i.e., deadline. However, the cloud computing system consumes a considerable amount of electric energy while providing services to these type of applications, which in turn contribute to the high operational cost. Specifically, it becomes cumbersome to offer services to deadline sensitive task while minimizing energy consumption. In this regard, efficient task scheduling is an attractive way to cut down energy usage while ensuring satisfactory services for cloud users. In this paper, the task scheduling problem is considered as a bi-objective minimization problem which includes minimization of energy consumption and makespan. First, we proposed a novel learning automata-based scheduling framework for deadline sensitive tasks in the cloud. Learning automata (LA) is an adaptive decision-making unit that helps the scheduler to select the best responses. Later, the LA-based Scheduling (LAS) algorithm is introduced which exploits the heterogeneity of tasks and virtual machines (VMs) while ensuring the timing requirements of the tasks. Extensive simulation is carried out to designate the effectiveness and applicability of LAS for deadline sensitive task scheduling in the heterogeneous cloud environment. [ABSTRACT FROM AUTHOR]

In this paper, the Transitivity Pursuit-Enhanced Object Migration Automata (TPEOMA) is used to capture the dependence of elements in a hierarchical Singly-Linked-Lists on Singly-Linked-Lists (SLLs-on-SLLs) "adaptive" data structure. In doing so, the TPEOMA-enhanced hierarchical SLLs-on-SLLs learns the probability distribution of elements in a Non-stationary Environment. In this framework, we divide the hierarchical Singly-Linked-Lists on Singly-Linked-Lists (SLLs-on-SLLs) into an outer and inner list context. The inner-list context is itself a SLLs containing sub-elements of the list, while the outer-list context contains these sublist partitions as its primitive elements. The elements belonging to a particular sublist partition are determined using the TPEOMA reinforcement learning scheme from the theory of Learning Automata. The idea of Transitivity builds on the Pursuit concept that injects a noise filter into the EOMA to filter divergent queries from the Environment, thereby increasing the likelihood of training the Automaton to approximate the "true" distribution of the Environment. The Transitivity phenomenon can infer "dependent" query pairs from non-accessed elements in the transitivity relation based on the statistical distribution of the queried elements. The TPEOMA-enhanced hierarchical SLLs-on-SLLs schemes results in superior performances to the MTF and TR schemes as well as to the EOMA-enhanced hierarchical SLLs-on-SLLs schemes in NSEs. [ABSTRACT FROM AUTHOR]

This paper concentrates on a host of problems with characteristics similar to those that are related to moving elevators within a building. These are referred to as Elevator-like problems (ELPs), and their common phenomena will be expanded on in the body of the paper. We shall resolve ELPs using a subfield of AI, namely the field of learning automata (LA). Rather than working with the well-established mathematical formulations of the field, our intention is to use these tools to tackle ELPs, and in particular, those that deal with single "elevators" moving between "floors". ELPs have not been tackled before using AI. In a simplified domain, the ELP involves the problem of optimizing the scheduling of elevators. In particular, we are concerned with determining the elevators' optimal "parking" location. In our case, the objective is to find the optimal parking floors for the single elevator scenario, so as to minimize the passengers' average waiting time (AWT). Apart from proposing benchmark solutions, we have provided two different novel LA-based solutions for the single-elevator scenario as the multi-elevator setting is more complicated. The first solution is based on the well-known L RI scheme, and the second solution incorporates the Pursuit concept to improve the performance and the convergence speed of the first solution, leading to the P L RI scheme. The simulation results presented demonstrate that our solutions performed better than those used in modern-day elevators, and provided results that are near-optimal, yielding a performance increase of up to 90%. The solutions presented for real elevators are directly applicable for the entire family of ELPs. [ABSTRACT FROM AUTHOR]

Learning automata (LA), which intellectually explores its optimal state by interacting with an external environment continuously, is encountered widely in artificial intelligence. In the evaluation of LA, it has always been a key issue how to tradeoff between “accuracy” and “speed,” which substantially touches on parameter tuning. A latest issue in the design of LA methodology involves bearing a parameter-free property, thus removing the tremendous expenses brought by parameter tuning. Nevertheless, the currently existing parameter-free LA schemes generally maintain a Monte-Carlo technique, which helps avoid the tuning process at the cost of more computations. This paper examines a new measurement of parameter-free LA schemes based on statistics which overcome the difficulties found in other counterparts. Specifically, it has innovatively disengaged from the dependance on Monte-Carlo methods. Of greater significance, the learning mechanisms operating in the common stationary environments are likewise extended to the nonstationary environments. Simulations confirm the effectiveness and efficiency of the proposed algorithm, especially its low computation consumption as well as the strong tracking capability to abrupt environmental changes. [ABSTRACT FROM AUTHOR]

In wireless sensor networks (WSNs), optimal energy utilization is one of the most crucial issues which needs special attention. It has been observed from the existing literature that communication among sensor nodes (SNs) consumes more energy than computation. Therefore, an efficient mechanism needs to be designed for energy conservation during communication among different SNs. To address these gaps, we propose a learning automata-based multilevel heterogeneous routing (LA-MHR) scheme for WSNs. In an LA-MHR, S-model-based LA is used for cluster heads (CHs) selection. A base station (BS) is used to allocate the cognitive radio spectrum to selected CHs. Moreover, single-hop communication among different SNs is used as multihop communication. It suffers from energy holes problem in WSNs. Based upon the initial energy of SNs, these are divided into intermediate, advanced, super-intermediate, and super-advanced categories. The performance of LA-MHR is evaluated by varying the locations of BS and heterogeneity parameters of SNs. Extensive simulations are performed to evaluate the performance of LA-MHR. Performance evaluation results show that both the network lifetime and stability of LA-MHR are increased by more than 10% as compared to other competing preexisting protocols such as EHE-LEACH, E-SEP, LA-EEHSC, and MCR. [ABSTRACT FROM AUTHOR]

Border surveillance is an application of wireless sensor networks (WSNs) and it is mainly used for security purpose. The present issue of implementing this application is a barrier coverage in WSNs. Due to energy constraints, minimizing energy consumption of sensors is a critical challenge. This paper presents a new distributed approach to the border surveillance in WSNs in order to maximize the number of barriers and minimize energy consumption. We use a learning automata scheme in our approach to solve this problem. We present distributed border surveillance (DBS) algorithm that aims to find the minimum possible number of nodes in each barrier to monitor the network borders. In DBS approach, learning automaton assists to find the best nodes to assure barrier coverage at any moment. The algorithm was implemented through computer simulations and compared to the existing competing algorithms. The simulation results show that the DBS scheme outperformed some state-of-art barrier-coverage algorithms. [ABSTRACT FROM AUTHOR]

Handoff and cabling costs management plays an important role in the design of cellular mobile networks. Efficient assigning of cells to switches can have a significant impact on handoff and cabling cost. Assignment of cells to switches problem (ACTSP) in cellular mobile network is NP-hard problem and consequently cannot be solved by exact methods. In this paper a new memetic algorithm which is obtained from the combination of learning automata (LA) and local search is proposed for solving the ACTSP in which the learning automata keeps the history of the local search process and manages the problem’s constraints. The proposed algorithm represents chromosome as object migration automata (OMAs), whose states represent the history of the local search process. Each state in an OMA has two attributes: the value of the gene (allele), and the degree of association with those values. The local search changes the degree of association between genes and their values. To show the superiority of the proposed algorithm several computer experiments have been conducted. The obtained results confirm the efficiency of proposed algorithm in comparison with the existing algorithms such as genetic algorithm, memetic algorithm, and a hybrid Hopfield network-genetic algorithm. [ABSTRACT FROM AUTHOR]

Due to the growing interest for multimedia contents by mobile users, designing bandwidth and delay-efficient distributed algorithms for data searching over wireless (possibly, mobile) 'ad hoc' Peer-to-Peer (P2P) content Delivery Networks (CDNs) is a topic of current interest. This is mainly due to the limited computing-plus-communication resources featuring state-of-the-art wireless P2P CDNs. In principle, an effective means to cope with this limitation is to empower traditional P2P CDNs by distributed Fog nodes. Motivated by this consideration, the goal of this paper is twofold. First, we propose and describe the main building blocks of a hybrid (e.g., mixed infrastructure and 'ad hoc') Fog-supported P2P architecture for wireless content delivery, namely, the Fog-Caching P2P architecture. It exploits the topological (possibly, time varying) information locally available at the serving Fog nodes, in order to speed up the data searching operations performed by the served peers. Second, we propose a bandwidth and delay-efficient, distributed and adaptive probabilistic search algorithm, that relies on the learning automata paradigm, e.g., the Fog-supported Learning Automata Adaptive Probabilistic Search (FLAPS) algorithm. The main feature of the FLAPS algorithm is the exploitation of the local topology information provided by the serving Fog nodes and the current status of the collaborating peers, in order to run a suitably distributed reinforcement algorithm for the adaptive discovery of peer-to-peer and peer-to-fog minimum-hop routes. The performance of the proposed FLAPS algorithm is numerically evaluated in terms of Success Rate, Hit-per-Query, Message-per-Query, Response Delay and Message Duplication Factor over a number of randomly generated benchmark CDN topologies. Furthermore, in order to corroborate the actual effectiveness of the FLAPS algorithm, extensive performance comparisons are carried out with some state-of-the-art searching algorithms, namely the Adaptive Probabilistic Search, Improved Adaptive Probabilistic Search and the Random Walk algorithms. [ABSTRACT FROM AUTHOR]

A fault-tolerant routing system is highly needed in autonomous unmanned vehicle (AUxV)-based networks because of the various constraints due to adversarial situations present in the environment of AUxVs. Critical consequences might be resulting even with a minor fault in the system software/hardware in these types of systems due to the involved hazard of their applications such as search and rescue, threat surveillance, and chemical and biohazard sampling. The architecture, capability, application, and power of the internal systems are the parameters that vary among the AUxV network member nodes. Fault-tolerant system design is a key aspect for AUxVs, and heterogeneity is to be considered while designing the fault-tolerant systems' AUxVs. This paper describes an approach to fault-tolerant AUxV networks by designing a routing method. The proposed algorithm for AUxVs is based on the cross-layer design and the learning automata (LA), referred as Unmanned Vehicle Network with LA-based Routing using Cross-Layer Design (ULARC). The optimal path between the source and the destination is obtained using the theory of LA. The effectiveness of the proposed strategy is proved by the proof of convergence and is shown in the Appendix. [ABSTRACT FROM PUBLISHER]

The purpose of this paper is to propose a solution to an extremely pertinent problem, namely, that of identifying unreliable sensors (in a domain of reliable and unreliable ones) without any knowledge of the ground truth. This fascinating paradox can be formulated in simple terms as trying to identify stochastic liars without any additional information about the truth. Though apparently impossible, we will show that it is feasible to solve the problem, a claim that is counter-intuitive in and of itself. One aspect of our contribution is to show how redundancy can be introduced, and how it can be effectively utilized in resolving this paradox. Legacy work and the reported literature (for example, in the so-called weighted majority algorithm) have merely addressed assessing the reliability of a sensor by comparing its reading to the ground truth either in an online or an offline manner. Unfortunately, the fundamental assumption of revealing the ground truth cannot be always guaranteed (or even expected) in many real life scenarios. While some extensions of the Condorcet jury theorem [9] can lead to a probabilistic guarantee on the quality of the fused process, they do not provide a solution to the unreliable sensor identification problem. The essence of our approach involves studying the agreement of each sensor with the rest of the sensors, and not comparing the reading of the individual sensors with the ground truth—as advocated in the literature. Under some mild conditions on the reliability of the sensors, we can prove that we can, indeed, filter out the unreliable ones. Our approach leverages the power of the theory of learning automata (LA) so as to gradually learn the identity of the reliable and unreliable sensors. To achieve this, we resort to a team of LA, where a distinct automaton is associated with each sensor. The solution provided here has been subjected to rigorous experimental tests, and the results presented are, in our opinion, both novel and conclusive. [ABSTRACT FROM PUBLISHER]

Plug-in Hybrid Electric Vehicles (PHEVs) can be one of the cost-effective options of modern intelligent transportation systems in smart grid (SG) which can balance the demand and supply by temporarily storing the electrical energy in their batteries. In this paper, we propose a new context-aware layered architecture for demand side management using vehicular cyber-physical system (VCPS) with cloud support. We have used the concept of Bayesian coalition game and learning automata for an intelligent context-aware data collection and processing using a new payoff function for the players in the coalition game. In the proposed scheme, vehicles are assumed as the players which sense the SG environment during their mobility and collect information from it. The players in the game perform actions such as alert generation, and information dissemination. For each action, players receive a feedback from the environment according to which they update their action probability vector. The performance of the proposed scheme shows that there is a reduction in energy shortage by 30%, and information processing delay of 10%–15%. In addition, there is an increase of 15% in energy sold back to the grid using the proposed scheme. The results obtained demonstrate the effectiveness of the proposed scheme. [ABSTRACT FROM PUBLISHER]

Generally speaking, research in the field of estimation involves designing strong estimators, i.e., those which converge with probability 1, as the number of samples increases indefinitely. But when the underlying distribution is nonstationary, one should rather seek for weak estimators, i.e., those which can unlearn when the distribution has changed. One such estimator, the so-called stochastic learning weak estimator (SLWE) was based on the principles of continuous stochastic learning automata (LA). A problem that has been unsolved has been that of designing such weak estimators in the context of systems with finite memory, which is what we investigate here. In this paper, we propose a new family of stochastic discretized weak estimators which can track time-varying binomial1 distributions. As opposed to the SLWE, our proposed estimator is discretized,2 i.e., the estimate can assume only a finite number of values. By virtue of discretization, our estimator realizes extremely fast adjustments of the running estimates by executing jumps, and it is thus able to robustly, and very quickly, track changes in the parameters of the distribution after a switch has occurred. The design principle of our strategy is based on a solution for the stochastic search on the line problem. In order to achieve efficient estimation, we have to first infer (or rather simulate) an Artificial Oracle which informs the LA whether to go right or left, which is then utilized to infer whether we are to increase the current estimate or to decrease it. This paper briefly reports pioneering and conclusive experimental results that demonstrate the ability of the proposed estimator to cope with nonstationary environments.1

The extension to multinomial and to other distributions is straightforward. [ABSTRACT FROM PUBLISHER]

Research on multiobjective optimization problems becomes one of the hottest topics of intelligent computation. In order to improve the search efficiency of an evolutionary algorithm and maintain the diversity of solutions, in this paper, the learning automata (LA) is first used for quantization orthogonal crossover (QOX), and a new fitness function based on decomposition is proposed to achieve these two purposes. Based on these, an orthogonal evolutionary algorithm with LA for complex multiobjective optimization problems with continuous variables is proposed. The experimental results show that in continuous states, the proposed algorithm is able to achieve accurate Pareto-optimal sets and wide Pareto-optimal fronts efficiently. Moreover, the comparison with the several existing well-known algorithms: nondominated sorting genetic algorithm II, decomposition-based multiobjective evolutionary algorithm, decomposition-based multiobjective evolutionary algorithm with an ensemble of neighborhood sizes, multiobjective optimization by LA, and multiobjective immune algorithm with nondominated neighbor-based selection, on 15 multiobjective benchmark problems, shows that the proposed algorithm is able to find more accurate and evenly distributed Pareto-optimal fronts than the compared ones. [ABSTRACT FROM PUBLISHER]

Community structure is an important feature in complex networks which has great significant for organization of networks. The community detection is the process of partitioning the network into some communities in such a way that there exist many connections in the communities and few connections between them. In this paper a Michigan memetic algorithm; called MLAMA-Net; is proposed for solving the community detection problem. The proposed algorithm is an evolutionary algorithm in which each chromosome represents a part of the solution and the whole population represents the solution. In the proposed algorithm, the population of chromosomes is a network of chromosomes which is isomorphic to the input network. Each node has a chromosome and a learning automaton (LA). The chromosome represents the community of corresponding node and saves the histories of exploration. The learning automaton represents a meme and saves the histories of the exploitation. The proposed algorithm is a distributed algorithm in which each chromosome locally evolves by evolutionary operators and improves by a local search. By interacting with both the evolutionary operators and local search, our algorithm effectively detects the community structure in complex networks and solves the resolution limit problem of modularity optimization. To show the superiority of our proposed algorithm over the some well-known algorithms, several computer experiments have been conducted. The obtained results show MLAMA-Net is effective and efficient at detecting the community structure in complex networks. [ABSTRACT FROM AUTHOR]

Vertex coloring problem is a combinatorial optimization problem in graph theory in which a color is assigned to each vertex of graph such that no two adjacent vertices have the same color. In this paper a new hybrid algorithm which is obtained from combination of cellular learning automata (CLA) and memetic algorithm (MA) is proposed for solving the vertex coloring problem. CLA is an effective probabilistic learning model combining cellular automata and learning automaton (LA). Irregular CLA (ICLA) is a generalization of CLA in which the restriction of rectangular grid structure in CLA is removed. The proposed algorithm is based on the irregular open CLA (IOCLA) that is an extension of ICLA in which the evolution of CLA is influenced by both local and global environments. Similar to other IOCLA-based algorithms, in the proposed algorithm, local environment is constituted by neighboring LAs of any cell and the global environment consists of a pool of memes in which each meme corresponds to a certain local search method. Each meme is represented by a set of LAs from which the history of the corresponding local search method can be extracted. To show the superiority of the proposed algorithm over some well-known algorithms, several computer experiments have been conducted. The results show that the proposed algorithm performs better than other methods in terms of running time of algorithm and the required number of colors. [ABSTRACT FROM AUTHOR]

This paper considers the stochastic point location (SPL) problem as a learning mechanism trying to locate a point on a real line via interacting with a random environment. Compared to the stochastic environment in the literatures that confines the learning mechanism to moving in two directions, i.e., left or right, this paper introduces a general triple level stochastic environment which not only tells the learning mechanism to go left or right, but also informs it to stay unmoved. It is easy to understand, as we will prove in this paper, that the environment reported in the previous literatures is just a special case of the triple level environment. And a new learning algorithm, named as random walk-based triple level learning algorithm, is proposed to locate an unknown point under this new type of environment. In order to examine the performance of this algorithm, we divided the triple level SPL problems into four distinguished scenarios by the properties of the unknown point and the stochastic environment, and proved that even under the triple level nonstationary environment and the convergence condition having not being satisfied for some time, which are rarely considered in existing SPL problems, the proposed learning algorithm is still working properly whenever the unknown point is static or evolving with time. Extensive experiments validate our theoretical analyses and demonstrate that the proposed learning algorithms are quite effective and efficient. [ABSTRACT FROM PUBLISHER]

A novel cooperative framework of learning automata (LA) is presented in this paper. In a system, where different LA can be integrated via the proposed framework, current State of Learning (SoL) index is advocated to evaluate the learning status of individual LA. Based on that learning status, individual LA will adaptively choose an appropriate interaction strategy at cooperative learning phase. Theoretical analysis demonstrated that this index is able to preserve the ε -optimal feature of independent learning automata. Experimental simulations validated that this cooperative framework is effective in improving learning speed of a variety of LA embedded in this framework. Then, we also present an application of the proposed cooperative framework in tutorial-like systems. Compared with existing method, our cooperative framework based method outperforms in both speed and accuracy. [ABSTRACT FROM AUTHOR]

A learning automaton (LA) is a powerful tool for reinforcement learning. Its action probability vector plays two roles: 1) deciding when it converges, i.e., total computing budget it has used, and 2) allocating computing budget among actions to identify the optimal one. These two intertwined roles lead to a problem: the computing budget mostly goes to the currently estimated optimal action due to its high action probability regardless whether such budget allocation can help identify the true optimal one or not. This work proposes a new class of LA that avoids the use of its action probability vector for computing budget allocation. Instead we use such vector only to determine if it converges and then employ optimal computing budget allocation to accomplish the allocation of computing budget in a way that maximizes the probability of identifying the true optimal actions. \varepsilon -optimality is proven. Simulations verify its advantages over existing algorithms. [ABSTRACT FROM AUTHOR]

Combing a genetic algorithm (GA) with a local search method produces a type of evolutionary algorithm known as a memetic algorithm (MA). Combining a GA with a learning automaton (LA) produces an MA named GALA, where the LA provides the local search function. GALA represents chromosomes as object migration automata (OMAs), whose states represent the history of the local search process. Each state in an OMA has two attributes: the value of the gene (allele), and the degree of association with those values. The local search changes the degree of association between genes and their values. In GALA a chromosome's fitness is computed using only the value of the genes. GALA is a Lamarckian learning model as it passes on the learned traits acquired by its local search method to offspring by a modification of the genotype. Herein we introduce a modified GALA (MGALA) that behaves according to a Baldwinian learning model. In MGALA the fitness function is computed using a chromosome's fitness and the history of the local search recorded by the OMA states. In addition, in MGALA the learned traits are not passed to the offspring. Unlike GALA, MGALA uses all the information recorded in an OMA representation of the chromosome, i.e., the degree of association between genes and their alleles, and the value of a gene, to compute the fitness of genes. We used MGALA to solve two problems: object partitioning and graph isomorphism. MGALA outperformed GALA, a canonical MA, and an OMA-based method using computer simulations, in terms of solution quality and rate of convergence. [ABSTRACT FROM AUTHOR]

Wireless mesh networks have emerged as a new technology for providing cost-effective broadband Internet access to users living in different communities across the globe. However, due to changes in a network topology across different paths, it is a challenging task to handle heavy data traffic in a multichannel environment. To address this issue, we propose a new collaborative-learning-automata-based channel assignment with topology preservation in this paper. In the proposed scheme, learning automata (LA) are deployed at the nearest mesh routers to collaborate with each other for information sharing and data transmission while learning from an environment. For each performed action, the LA get a reward or a penalty from the environment. Based on the inputs from the environment, the LA update their action probability vector and then decide the next action. The performance of the proposed scheme is evaluated with respect to various metrics such as throughput, data delivery ratio, switching and buffering delays, effective transmission, and effective channel utilization. [ABSTRACT FROM PUBLISHER]

In vehicular sensor networks (VSNs), an increase in the density of the vehicles on road and route jamming in the network causes delay in receiving the emergency alerts, which results in overall system performance degradation. In order to address this issue in VSNs deployed in dense urban regions, in this paper, we propose collaborative learning automata-based routing algorithm for sending information to the intended destination with minimum delay and maximum throughput. The learning automata (LA) stationed at the nearest access points (APs) in the network learn from their past experience and make routing decisions quickly. The proposed strategy consists of dividing the whole region into different clusters, based on which an optimized path is selected using collaborative LA having input parameters as vehicle density, distance from the nearest service unit, and delay. A theoretical expression for density estimation is derived, which is used for the selection of the “best” path by LA. The performance of the proposed scheme is evaluated with respect to metrics such as packet delivery delay (network delay), packet delivery ratio with varying node (vehicle) speed, transmission range, density of vehicle, and number of road side units/APs). The results obtained show that the proposed scheme performs better than the benchmark chosen in this study, as there is a 30% reduction in network delay and a 20% increase in packet delivery ratio. [ABSTRACT FROM PUBLISHER]

The Worldwide Interoperability for Microwave Access (WiMAX) family of standards have introduced a flexible, efficient, and robust wireless interface. Among other interesting features, WiMAX access networks bring into play a flexible determination of the ratio between the downlink and uplink directions, allowing a relation width from 3 : 1 to 1 : 1, respectively. However, this promising feature is not properly utilized, since hitherto scheduling and mapping schemes proposed neglect it. In this paper, this challenging issue is effectively addressed by proposing an adaptive model that attempts to adequately adjust the downlink-to-uplink subframe width ratio according to the current traffic conditions. In the context of a mobile WiMAX wireless access network, the base station is enhanced with an error-aware learning automaton in order to be able to identify the magnitude of the incoming and outgoing traffic flows and, in turn, to suitably define the ratio on a frame-by-frame basis. The model designed is extensively evaluated under realistic and dynamic scenarios, and the results indicate that its performance is clearly improved compared with schemes having predefined and fixed ratio values. [ABSTRACT FROM PUBLISHER]

In a mobile video surveillance system (MVSS), an efficient approach is required, so that captured video can be transmitted to its final destination under tight constraints of delay and accuracy. This paper presents a new intelligent MVSS using the concepts of Bayesian coalition game and learning automata (LA). These LA are assumed to be the players in a game and are deployed on vehicles. Coalition among players is formed using the Bayesian Coalition Game Theory. To decrease the delay that occurred during transmission of captured video to the nearest access points, the best path is chosen based on a new metric called Path Score, which is computed by each player in the game. For each action performed by the automata, their actions may be rewarded or penalized by a value, which is defined as a sequence, with respect to the inputs provided from the stochastic environment. According to the reward or penalty received from the environment, the automata update their action probability vector. After 15 iterations, a Nash equilibrium is achieved in the game by defining a twice-differentiable function in Banach spaces, and convergence of sequence is proved using the Cauchy convergence theorem. The performance of the proposed scheme is found to be better in comparison to the other state-of-the-art schemes, with respect to various performance evaluation metrics. [ABSTRACT FROM PUBLISHER]

Wireless mesh networks (WMNs) consist of static nodes that usually have one or more radios or media. Optimal channel assignment (CA) for nodes is a challenging problem in WMNs. CA aims to minimize interference in the overall network and thus increase the total capacity of the network. This paper proposes a new method for solving the CA problem that comparatively performs more efficient than existing methods. The link layer in the TCP/IP model is a descriptive realm of networking protocols that operates on the local network link in routers discovery and neighboring hosts. TCP/IP employs the link-layer protocol (LLP) that is included among the hybrid states in CA methods, and learning automata are used to complete the algorithm with an intelligent method for suitable CA. We call this algorithm LLLA, which are short for LLP and learning automata. Our simulation results show that LLLA performs more efficient than ad hoc on-demand distance vector (AODV) types with respect to parameters such as packet drop, end-to-end delay, average goodput, jitter in special applications, and energy usage. [ABSTRACT FROM AUTHOR]

A cyber-physical system (CPS) consists of a group of interlinked networked objects. CPSs are characteristically different from the traditional internetworked systems (e.g., Internet) due to the operating environment and their applications, which impose high reliability constraints on them. One of the fundamental considerations for enabling this technology is the medium access control (MAC). The traditional IEEE 802.11-based MAC layer solutions need to be revamped. In order to increase the efficiency of data transmission, we introduce a new approach, which argues the replacement of the traditional backoff algorithm of 802.11. The proposed approach attempts to address some of the obligations posed by CPS networks. At the core of the proposed solution is the introduction of the adaptability factor to the traditional backoff procedure, which can help CPS networks operate under diverse environmental conditions. In order to survive the high degrees of dynamism of these networks, each of the nodes is equipped with a learning automaton, which adapts to the changing environment and provides the node with the most optimized value. The consistent outperformance of the proposed approach when compared with similar existing approaches under different simulation scenarios exhibits its efficacy. The critical parameters, such as packet delivery ratio and delay, are improved by up to 40% and 27%, respectively, over the traditional protocols. [ABSTRACT FROM PUBLISHER]

An update scheme of the state probability vector of actions is critical for learning automata (LA). The most popular is the pursuit scheme that pursues the estimated optimal action and penalizes others. This paper proposes a reverse philosophy that leads to last-position elimination-based learning automata (LELA). The action graded last in terms of the estimated performance is penalized by decreasing its state probability and is eliminated when its state probability becomes zero. All active actions, that is, actions with nonzero state probability, equally share the penalized state probability from the last-position action at each iteration. The proposed LELA is characterized by the relaxed convergence condition for the optimal action, the accelerated step size of the state probability update scheme for the estimated optimal action, and the enriched sampling for the estimated nonoptimal actions. The proof of the $\epsilon$ -optimal property for the proposed algorithm is presented. Last-position elimination is a widespread philosophy in the real world and has proved to be also helpful for the update scheme of the learning automaton via the simulations of well-known benchmark environments. In the simulations, two versions of the LELA, using different selection strategies of the last action, are compared with the classical pursuit algorithms Discretized Pursuit Reward-Inaction ({\rm DP}_{\rm RI}) and Discretized Generalized Pursuit Algorithm (DGPA). Simulation results show that the proposed schemes achieve significantly faster convergence and higher accuracy than the classical ones. Specifically, the proposed schemes reduce the interval to find the best parameter for a specific environment in the classical pursuit algorithms. Thus, they can have their parameter tuning easier to perform and can save much more time when applied to a practical case. Furthermore, the convergence curves and the corresponding variance coefficient curves of the contenders are illustrated to characterize their essential differences and verify the analysis results of the proposed algorithms. [ABSTRACT FROM PUBLISHER]

In this paper, a new structure for cooperative learning automata called extended learning automata (eDLA) is introduced. Based on the new structure, an iterative randomized heuristic algorithm using sampling is proposed for finding an optimal subgraph in a stochastic edge-weighted graph. Stochastic graphs are graphs in which the weights of edges have an unknown probability distribution. The proposed algorithm uses an eDLA to find a policy that leads to a subgraph that satisfy some restrictions such as minimum or maximum weight (length). At each stage of the proposed algorithm, the eDLA determines which edges should be sampled. The proposed eDLA-based sampling method may reduce unnecessary samples and hence decrease the time required for finding an optimal subgraph. It is shown that the proposed method converges to an optimal solution, the probability of which can be made arbitrarily close to 1 by using a sufficiently small learning parameter. A new variance-aware threshold value is also proposed that can significantly improve the convergence rate of the proposed eDLA-based algorithm. It is further shown that our algorithm is competitive in terms of the quality of the solution. [ABSTRACT FROM AUTHOR]

This paper focuses on resource discovery problem for Grid. Grid is a devices and services environment that has evolved with the goal of resource sharing. Grid resource discovery encompasses locating and retrieving computational resources. Existing resource discovery solutions are not well adapted to the dynamicity and heterogeneity of Grid. Query propagation is a novel approach that forwards an unsupported query from its resident peer to an adjacent peer. The concept of next generation intelligent Grid environments needs intelligent modules for resource discovery. Learning automaton is a stochastic tool with learning ability which simply adapts to the progressive environmental changes. The proposed method utilizes a distributed learning automata (DLA) which is a network of learning automata (LA). Here, multiple DLA are used for forwarding domain-specific queries. Different Grid scales are utilized for evaluation of the proposed method. Results demonstrate that the resource discovery based on DLA optimizes resource utilization, maximizes throughput, minimizes response time and avoids overload. Moreover, the algorithm is also scalable, fully distributed and failure-free. [ABSTRACT FROM AUTHOR]

Intrusion detection is one of the most important applications of wireless sensor networks. When mobile objects are entering into the boundary of a sensor field or are moving cross the sensor field, they should be detected by the scattered sensor nodes before they pierce through the field of sensor (barrier coverage). In this paper, we propose an energy efficient scheduling method based on learning automata, in which each node is equipped with a learning automaton, which helps the node to select best node to guarantee barrier coverage, at any given time. To apply our method, we used coverage graph of deployed networks and learning automata of each node operates based on nodes that located in adjacency of current node. Our algorithm tries to select minimum number of required nodes to monitor barriers in deployed network. To investigate the efficiency of the proposed barrier coverage algorithm several computer simulation experiments are conducted. Numerical results show the superiority of the proposed method over the existing methods in term of the network lifetime and our proposed algorithm can operate very close to optimal method. [ABSTRACT FROM AUTHOR]

This paper presents a Learning Automata (LA)-based QoS (LAQ) framework capable of addressing some of the challenges and demands of various cloud applications. The proposed LAQ framework ensures that the computing resources are used in an efficient manner and are not over- or under-utilized by the consumer applications. Service provisioning can only be guaranteed by continuously monitoring the resource and quantifying various QoS metrics, so that services can be delivered in an on-demand basis with certain levels of guarantee. The proposed framework helps in ensuring guarantees with these metrics in order to provide QoS-enabled cloud services. The performance of the proposed system is evaluated with and without LA, and it is shown that the LA-based solution improves the performance of the system in terms of response time and speed up. [ABSTRACT FROM PUBLISHER]

In wireless sensor networks, when each target is covered by multiple sensors, we can schedule sensor nodes to monitor deployed targets in order to improve lifetime of network. In this paper, we propose an efficient scheduling method based on learning automata, in which each node is equipped with a learning automaton, which helps the node to select its proper state (active or sleep), at any given time. To study the performance of the proposed method, computer simulations are conducted. Results of these simulations show that the proposed scheduling method can better prolong the lifetime of the network in comparison to similar existing methods. [ABSTRACT FROM AUTHOR]