Search

The use of Association Rule Mining techniques in diverse contexts and domains has resulted in the creation of numerous interestingness measures. This, in turn, has motivated researchers to come up with various classification schemes for these measures. One popular approach to classify the objective measures is to assess the set of mathematical properties they satisfy in order to help practitioners select the right measure for a given problem. In this research, we discuss the insufficiency of the existing properties in literature to capture certain behaviors of interestingness measures. This motivates us to present a novel approach to analyze and classify measures. We refer to this as a rate of change analysis (RCA). In this analysis a measure is described by how it varies if there is a unit change in the frequency count $(f_{11},f_{10},f_{01},f_{00})$, for different pre-existing states of the frequency counts. More formally, we look at the first partial derivative of the measure with respect to the various frequency count variables. We then use this analysis to define two new properties, Unit-Null Asymptotic Invariance (UNAI) and Unit-Null Zero Rate (UNZR). UNAI looks at the asymptotic effect of adding frequency patterns, while UNZR looks at the initial effect of adding frequency patterns when they do not pre-exist in the dataset. We present a comprehensive analysis of 50 interestingness measures and classify them in accordance with the two properties. We also present empirical studies, involving both synthetic and real-world datasets, which are used to cluster various measures according to the rule ranking patterns of the measures. The study concludes with the observation that classification of measures using the empirical clusters share significant similarities to the classification of measures done through the properties presented in this research.

Data from Global Positioning Systems (GPS) and fare-meters in For-Hire vehicles (FHVs) have been used for various applications – both in research as well as organizational decision-making. The utility of such exercises largely depend on the accuracy of the data. This study looks at an environment where the data is partially mislabeled. Specifically, we take a common real-world setting where vehicle operators choose to render transportation services to customers without the use of a fare-meter, often by negotiating a fixed rate with the customer. This practice, which to different degrees, has been observed and documented across urban areas in the world, leads to various undesirable effects. In this study, we seek to identify cases of such behavior in the dataset. Typically, a supervised learning classifier could be built to predict the occupancy status from GPS traces, which can then be used, to look for anomalies between the predicted and stated behaviors. However, in our case the training dataset also contains instances of incorrect tagging. We address this problem by casting it as one of learning from Positive and Unlabeled instances (PU Learning) . This is owing to the fact that we observe the phenomenon of one-sided label noise, where trips tagged ‘vacant’ by the taximeter could be truly vacant or occupied, whereas trips tagged ‘occupied’ are expected to be occupied in reality as well. To support this novel formulation, we apply three state-of-the-art PU Learning algorithms on a real-world trajectory data set from an organization plying 170 active vehicles over a period of two months. We compare these to the baselines of standard supervised learning. Validation is carried out by the organization through alternate channels of investigation which is not indicated in the data set. The results show that the PU Learners provide a significant improvement in classification across a range of metrics when compared to the baseline approaches. This translates to a significant increase in identifying or reclassifying the mislabeled rides.

This research studies the use of predetermined experimental plans in a live setting with a finite implementation horizon. In this context, we seek to determine the optimal experimental budget in different environments using a Bayesian framework. We derive theoretical results on the optimal allocation of resources to treatments with the objective of minimizing cumulative regret, a metric commonly used in online statistical learning. Our base case studies a setting with two treatments assuming Gaussian priors for the treatment means and noise distributions. We extend our study through analytical and semi-analytical techniques which explore worst-case bounds, the presence of unequal prior distributions, and the generalization to k treatments. We determine theoretical limits for the experimental budget across all possible scenarios. The optimal level of experimentation that is recommended by this study varies extensively and depends on the experimental environment as well as the number of available units. This highlights the importance of such an approach which incorporates these factors to determine the budget.

There is a growing interest in understanding, as opposed to predicting, the repayment behavior of customers of financial institutions that provide loans. This study proposes a modified Hidden Markov Model (HMM) based clustering, which clusters repayment sequences across selected subsets of the HMM parameters. We demonstrate that different implementations of this modified adaptation helps us gain an in-depth understanding of various drivers that are hard to directly observe but nevertheless govern repayment. These include drivers such as the ability to repay (financial health of the customer) or the intention to repay independent of the ability (willful defaulting and unintentional delinquency). Algorithmically, we achieve this partially constrained HMM clustering (PC-HMM) by placing constraints on the expectation-maximization (EM) algorithm where a subset of parameters are used to cluster the repayments via the estimation process, while the other parameters are learned globally across all repayments. We compare our approach with three other baselines on a real-world loan repayment data set. We use an exogenous variable to validate and benchmark the clusters. We conclude our study with the observation that the ability to cluster on selective parameters, in conjunction with the structural construct of HMMs, enables the discovery of substantially more meaningful business insights than the baselines.

A designed experiment is typically followed by a statistical analysis of the results, using which the preferred settings of the inputs are selected for operation. In this paper, we motivate real-world scenarios, where it could be advantageous to succeed the experiment with continued exploration upon deployment in the online context. We propose the use of Linear Bandits to conduct sequential experiments in the online setting. The linear bandit algorithms, which utilize results from the designed experiment as an initial seed, are then used to select a treatment combination in each step or trial. Specifically, the study analyzes two linear bandit algorithms and compares them to three different baselines. The two linear bandit algorithms are OFUL, which is shown in literature to have one of the best theoretical performances, and LGUCBand, a novel contribution of this research, which uses the statistical concept of upper confidence bands for linear models. The baselines are different designs and data analyses without any form of online experimentation. The results are compared using simulations of a model built on meta-data from published experiments on real engineering applications. An analytical derivation of the default baselines is also an important contribution of this research and is intended to provide theoretical validation of the simulation results. The results indicate that, across different environments, substantial long-term improvements can be made by following designed experiments with linear bandits, with minimal short term costs.