Search

This paper outlines a method of non-stationary reinforcement-based learning for feature selection. The method was developed for the Watershed REstoration using Spatio-Temporal Optimization of REsources (WRESTORE) system, which is a decision support system used for wetland design on the Eagle Creek Watershed, northwest of Indianapolis, Indiana. Our results show measurable impact for maximizing reward efficiently for the feature selection task. This work describes the existing WRESTORE system, provides an overview of related work in reinforcement learning and dimensionality reduction, and shows the impact of our work in the multi-objective optimization process of WRESTORE. The contribution of this work is the application of an RL-based feature selection technique in interactive optimization of watershed design.

Machine learning algorithms and traditional data mining process usually require a large volume of data to train the algorithm-specific models, with little or no user feedback during the model building process. Such a “big data” based automatic learning strategy is sometimes unrealistic for applications where data collection or processing is very expensive or difficult, such as in clinical trials. Furthermore, expert knowledge can be very valuable in the model building process in some fields such as biomedical sciences. In this paper, we propose a new visual analytics approach to interactive machine learning and visual data mining. In this approach, multi-dimensional data visualization techniques are employed to facilitate user interactions with the machine learning and mining process. This allows dynamic user feedback in different forms, such as data selection, data labeling, and data correction, to enhance the efficiency of model building. In particular, this approach can significantly reduce the amount of data required for training an accurate model, and therefore can be highly impactful for applications where large amount of data is hard to obtain. The proposed approach is tested on two application problems: the handwriting recognition (classification) problem and the human cognitive score prediction (regression) problem. Both experiments show that visualization supported interactive machine learning and data mining can achieve the same accuracy as an automatic process can with much smaller training data sets.

Determining the optimal design of a watershed is a highly subjective process which involves the consideration of many distinct factors by several different stakeholder groups. We describe additional functionality for our watershed planning system, called WRESTORE (Watershed REstoration Using Spatio-Temporal Optimization of REsources) (http://wrestore.iupui.edu), where stakeholders can collaboratively optimize best management practices on to the watershed. WRESTORE utilizes the USDA's public domain Soil and Water Assessment Tool hydrologic model for watershed simulations. Reinforcement learning and interactive genetic algorithms are applied for the search process. The new functionality described is a user modeling component that develops a computational model of a user's decision-making, based on real-time user-provided ratings for a subset of possible designs. The user modeling task utilizes neural network approaches, such as deep learning. We believe the originality of our approach centers on integrating user models in to the hydrological decision support process. This paper thus has three objectives: (i) outline current work in user modeling and watershed design, (ii) describe our system for interactive optimization of watershed design, and (iii) describe our work on implementing accurate and stable user predictive models to boost optimization performance.

Database Selection is the problem of choosing, from a finite number of databases, the one that contains the most relevant information pertaining to a query. Previous approaches to this problem consisted of deterministic search techniques in conjunction with efficient pruning of search spaces, based on vector space model and ranking. In this article, we propose a new probabilistic search method, based on a reinforcement learning algorithm, to solve the database selection problem, where an agent learns to map situations to action by means of receiving reward and penalties for the action taken and trying to maximize its rewards. We use reinforcement algorithm with user feedback to learn a policy, which maps a particular topic or particular interest to a set of databases. Reinforcement learning is an attractive approach to this problem due to its ability to generate optimal solutions for both stationary and non-stationary/dynamically changing environments(queries). Experiments with simple queries show that reinforcement learning has potential to be considered as an efficient approach for database selection.

Although large-scale classification studies of genetic sequence data are in progress around the world, very few studies compare different classification approaches, e.g. unsupervised and supervised, in terms of objective criteria such as classification accuracy and computational complexity. In this paper, we study such criteria for both unsupervised and supervised classification of a relatively large sequence data set. The unsupervised approach involves use of different sequence alignment algorithms (e.g., Smith-Waterman, FASTA and BLAST) followed by clustering using the Maximin algorithm. The supervised approach uses a suitable numeric encoding (relative frequencies of tuples of nucleotides followed by principal component analysis) which is fed to a Multi-layer Backpropagation Neural Network. Classification experiments conducted on IBM-SP parallel computers show that FASTA with unsupervised Maximin leads to best trade-off between accuracy and speed among all methods, followed by supervised neural networks as the second best approach. Finally, the different classifiers are applied to the problem of cross-species homology detection.

Today a massive amount of information available on the WWW often makes searching for information of interest a long and tedious task. Chasing hyperlinks to find relevant information may be daunting. To overcome such a problem, a learning system, cognizant of a user's interests, can be employed to automatically search for and retrieve relevant information by following appropriate hyperlinks. In this paper, we describe the design of such a learning system for automated Web navigation using adaptive dynamic programming methods. To improve the performance of the learning system, we introduce the notion of multiple model-based learning agents operating in parallel, and describe methods for combining their models. Experimental results on the WWW navigation problem are presented to indicate that combining multiple learning agents, relying on user feedback, is a promising direction to improve learning speed in automated WWW navigation.

In this paper, two supervised neural networks are used to estimate the forces developed during milling. These two Artificial Neural Networks (ANNs) are compared based on a cost function that relates the size of the training data to the accuracy of the model. Training experiments are screened based on design of experiments. Verification experiments are conducted to evaluate these two models. It is shown that the Radial Basis Network model is superior in this particular case. Orthogonal design and specifically equally spaced dimensioning showed to be a good way to select the training experiments.

User modelling is one of the prominent research fields in information retrieval systems. In this paper, we model user's preferences and search criteria using an NN (Neural Network) to solve a multiobjective optimization problem specific to environmental planning systems. We argue that some NP hard problems cannot be solved alone either by a human or by a computer. Human participation in automated search is one way of combining human intuition with algorithmic search to solve such problems. However, even humans have some limitations for participation in that they cannot participate in search completely because of human fatigue. To overcome this, in our approach, an NN tries to model the user's rating criteria and preferences to help the user in rating large set of designs. Although training an NN with limited data is not always feasible, there are many situations where a simple modelling technique (e.g., linear/quadratic mapping) works better if the learning data set is small. In this paper we attempt to get more accuracy of the NN by generating data using other linear/non-linear techniques that fills the gap created by lack of sufficient training data. Also, we provided the architectural design of an HPC based framework we have proposed and compared the performance of the NN with fuzzy logic and other linear/non-linear user modelling techniques for the environmental resources optimization problem.

Mining frequent patterns from a hidden dataset is an important task with 43 various real-life applications. In this research, we propose a solution to this problem that is based on Markov Chain Monte Carlo (MCMC) sampling of frequent patterns. Instead of returning all the frequent patterns, the proposed paradigm returns a small set of randomly selected patterns so that the clandestinity of the dataset can be maintained. Our solution also allows interactive sampling, so that the sampled patterns can fulfill the user's requirement effectively. We show experimental results from several real life datasets to validate the capability and usefulness of our solution; in particular, we show examples that by using our proposed solution, an eCommerce marketplace can allow pattern mining on user session data without disclosing the data to the public; such a mining paradigm helps the sellers of the marketplace, which eventually boost the marketplace's own revenue.

The problem of selecting a subset of sensors in a distributed object tracking environment that optimizes an objective function consisting of a trade-off between data accuracy and energy consumption is known to be NP-hard. The problem is exacerbated because of the uncertainty and dynamic nature of either sensor characteristics or the environment or both. We propose, for the first time, a novel framework based on a reinforcement learning approach, to deal with the problems of computational complexity, dynamic nature and uncertainty for sensor subset selection. Our proposed sensor subset selection approach is completely decentralized and sensors do not need to know even the presence of other sensors in the system. This makes our approach extremely scalable and easy to implement in a distributed system. To the best of our knowledge, this is the first application of reinforcement learning to the domain of sensor subset selection.

Objectives: Biological research literature, as in many other domains of human endeavor, represents a rich, ever growing source of knowledge. An important form of such biological knowledge constitutes associations among biological entities such as genes, proteins, diseases, drugs and chemicals, etc. There has been a considerable amount of recent research in extraction of various kinds of binary associations (e.g., gene-gene, gene-protein, protein-protein, etc.) using different text mining approaches. However, an important aspect of such associations (e.g., ''gene A activates protein B'') is identifying the context in which such associations occur (e.g., ''gene A activates protein B in the context of disease C in organ D under the influence of chemical E''). Such contexts can be represented appropriately by a multi-way relationship involving more than two objects (e.g., objects A, B, C, D, E) rather than usual binary relationship (objects A and B). Methods: Such multi-way relations naturally lead to a hyper-graph representation of the knowledge rather than a binary graph. The hyper-graph based multi-way knowledge extraction from biological text literature represents a computationally difficult problem (due to its combinatorial nature) which has not received much attention from the Bioinformatics research community. In this paper, we describe and compare two different approaches to such multi-way hyper-graph extraction: one based on an exhaustive enumeration of all multi-way hyper-edges and the other based on an extension of the well-known A Priori algorithm for structured data to the case unstructured textual data. We also present a representative graph based approach towards visualizing these genetic association hyper-graphs. Results: Two case studies are conducted for two biomedical problems (related to the diseases of lung cancer and colorectal cancer respectively), illustrating that the latter approach (using the text-based A Priori method) identifies the same hyper-edges as the former approach (the exhaustive method), but at a much less computational cost. The extracted hyper-relations are presented in the paper as cognition-rich representative graphs, representing the corresponding hyper-graphs. Conclusions: The text-based A Priori algorithm is a practical, useful method to extract hyper-graphs representing multi-way associations among biological objects. These hyper-graphs and their visualization using representative graphs can provide important contextual information for understanding gene-gene associations relevant to specific diseases.

In this paper, we introduce reinforcement learning as a methodology to solve complex multi-criteria optimization problems for ground water monitoring. Multiple analytical criteria are used to assess design decisions and human feedback is simulated by adding random noise. Different learning automata based reinforcement learning methods as well as a genetic algorithm based method are used in experimental studies, which demonstrate the efficiency of reinforcement learning approaches.

There has been a considerable amount of recent research in extraction of various kinds of binary associations (e.g., gene-gene, gene-protein, protein-protein, etc) using different text mining approaches. However, an important aspect of such associations is identifying the context in which such associations occur (e.g., "gene A activates protein B in the context of disease C in organ D under the influence of chemical E"). Such contexts can be represented appropriately by a multi-way relationship involving more than two objects rather than usual binary relationships. Such multi-way relations naturally lead to a hyper-graph representation of the knowledge. The hyper-graph based knowledge extraction from biological literature represents a computationally difficult problem due to its combinatorial nature. In this paper, we compare two different approaches to such hyper-graph extraction: one based on an exhaustive enumeration of all hyper-edges and the other based on an extension of the well-known A Priori algorithm.

Classification of genetic sequence data available in public and private databases is an important problem in using, understanding, retrieving, filtering and correlating such large volumes of information. Although a significant amount of research effort is being spent internationally on this problem, very few studies exist that compare different classification approaches in terms of an objective and quantitative classification performance criterion. In this paper, we present experimental studies for classification of genetic sequences using both unsupervised and supervised approaches, focusing on both computational effort as well as a suitably defined classification performance measure. The results indicate that both unsupervised classification using the Maximin algorithm combined with FASTA sequence alignment algorithm and supervised classification using artificial neural network have good classification performance, with the unsupervised classification performs better and the supervised classification performs faster. A trade-off between the quality of classification and the computational efforts exists. The utilization of these classifiers for retrieval, filtering and correlation of genetic information as well as prediction of functions and structures will be logical future directions for further research.

In recent years, systems consisting of multiple modular neural networks have attracted substantial interest in the neural networks community because of various advantages they offer over a single large monolithic network. In this paper, we propose two basic feature decomposition models (namely, parallel model and tandem model) in which each of the neural network modules processes a disjoint subset of the input features. A novel feature decomposition algorithm is introduced to partition the input space into disjoint subsets solely based on the available training data. Under certain assumptions, the approximation error due to decomposition can be proved to be bounded by any desired small value over a compact set. Finally, the performance of feature decomposition networks is compared with that of a monolithic network in real-world bench-mark pattern recognition and modeling problems.

A nonstationary environment is one in which the suitability of the strategies available to a learning element changes with time. Since the optimal action in such a case is not fixed, the learning problem (i.e., the determination of the optimal strategy) becomes considerably difficult. In this paper, a two-level approach is presented for a learning automaton operating in a nonstationary environment. The lower level consists of a standard absolutely expedient learning algorithm for stationary environments. The higher level on the other hand is a tracking algorithm, based on Bayesian decision theory, for detecting changes in the environment and reinitializing the lower level algorithm in a suitable manner. Simulation studies empirically demonstrate the clear superiority of the two-level approach over the single-level learning in nonstationary environments.