Search

The Multi-Armed Bandit (MAB) is a fundamental model capturing the dilemma between exploration and exploitation in sequential decision making. At every time step, the decision maker selects a set of arms and observes a reward from each of the chosen arms. In this paper, we present a variant of the problem, which we call the Scaling MAB (S-MAB): The goal of the decision maker is not only to maximize the cumulative rewards, i.e., choosing the arms with the highest expected reward, but also to decide how many arms to select so that, in expectation, the cost of selecting arms does not exceed the rewards. This problem is relevant to many real-world applications, e.g., online advertising, financial investments or data stream monitoring. We propose an extension of Thompson Sampling, which has strong theoretical guarantees and is reported to perform well in practice. Our extension dynamically controls the number of arms to draw. Furthermore, we combine the proposed method with ADWIN, a state-of-the-art change detector, to deal with non-static environments. We illustrate the benefits of our contribution via a real-world use case on predictive maintenance.

Machine learning algorithms have become increasingly common and have affect many aspects of our life. However, because the objective of most of the standard, off-the-shelf machine learning algorithms is to maximize the prediction performance, the results produced by these algorithms could be discriminatory. The discrimination issue has gain the interest from both academic researchers and practitioners to develop machine learning algorithms that are fair. Even then, most such algorithms focus on decreasing the disparity in predictions of successful outcomes. However, these algorithms tend to ignore the strategic behavior of prediction subpopulations, resulting in disparity in the behavior of prediction subjects at equilibrium. One exception is those algorithms that use equalized odds as a fairness criterion which can decrease disparity in behavior. However, they cannot be used in many practical settings. We propose a new class of fair machine learning algorithms that alleviate disparity in prediction results, disparity in behavior of prediction subjects, and does not need to account for the sensitive variable explicitly. Our algorithm also complies with the notion of equal treatment and explainable AI, and can be applied to a wide variety of prediction tasks. We demonstrate the theoretical performance of our algorithm in the asymptotic scenario. In addition, we show the practical performance of the proposed algorithm by comparing its performance with that of other ordinary off-the-shelf algorithms and that of existing fair machine learning algorithms available in the IBM Fairness 360 suite.

Stock return predictability is an important research theme as it reflects our economic and social organization, and significant efforts are made to explain the dynamism therein. Statistics of strong explanative power, called "factor", have been proposed to summarize the essence of predictive stock returns. The challenge here is to make a multi-factor investment strategy that is consistent over a reasonably long period based on supervised machine learning. Although machine learning methods are increasingly popular in stock return prediction, an inference of the stock return is highly elusive, and naive use of complex machine learning methods easily overfits the current data and results in poor performance on future data. We propose a principled stock return prediction framework that we call Ranked Information Coefficient Neural Network (RIC-NN) that alleviates the overfitting. RIC-NN addresses the difficulty that arises in nonconvex machine learning: Namely, initialization and the stopping of the training model and the transfer among several different tasks (markets). RIC-NN is a deep learning approach and includes the following three novel ideas: (1) nonlinear multi-factor approach, (2) stopping criteria with ranked information coefficient (rank IC), and (3) deep transfer learning among multiple regions. Experimental comparison with the stocks in the Morgan Stanley Capital International indices shows that RIC-NN outperforms not only off-the-shelf machine learning methods but also the average return of major equity investment funds in the last fourteen years.

With the increasingly popular use of algorithmic-based decision support systems, the issue of discrimination in machine learning has attracted increasing concerns. Essentially, when an algorithm is applied to prediction tasks that are related to human, a prediction algorithm with the objective to solely optimize its predictive power often pick up biases in the training data and use them in the process. Consequently, the prediction results may discriminate one (or more) group of subjects because of the biases captured in the prediction process. In response to this issue, fairness-aware machine learning, which describes machine learning algorithms that can identify biases in the data and suppress their influence in the prediction process, is developed. This paper studies the impact of two most popular types of fairness-aware machine learning, the demographic parity (DP) and the equalized odds (EO), on welfare of stakeholders and society. We demonstrate that the EO is the only type of fairness-aware machine learning that can remove group-level disparity. However, such fairness comes with a non-negligible cost to the firm and society.

We present a method for estimating road safety using large-scale and heterogeneous vehicle recorder data. Our assamption in this study is that occurrences of traffic accidents on crossings could be estimated by their road shape, driving behaviors over them, and photos taken by dash cams. Utilizing 3 years dataset from 1,500 drivers in a fleet delivery company, we try to detect risky crossings based on supervised machine learning. Results show that our method provides good performance, and can extract potentially risky crossings.

Emerging patterns are patterns whose support significantly differs between two databases. We study the problem of listing emerging patterns with a multiple testing guarantee. Recently, Terada et al., proposed the Limitless Arity Multiple-testing Procedure (LAMP) that controls the family-wise error rate (FWER) in statistical association mining. LAMP reduces the number of "untestable" hypotheses without compromising its statistical power. Still, FWER is restrictive, and as a result, its statistical power is inherently unsatisfying when the number of patterns is large. On the other hand, the false discovery rate (FDR) is less restrictive than FWER, and thus controlling FDR yields a larger number of significant patterns. We propose two emerging pattern mining methods: the first one controls FWER, and the second one controls FDR. The effectiveness of the methods is verified in computer simulations with real-world datasets.

Contents displayed on web portals (e.g., news articles at Yahoo.com) are usually adaptively selected from a dynamic set of candidate items, and the attractiveness of each item decays over time. The goal of those websites is to maximize the engagement of users (usually measured by their clicks) on the selected items. We formulate this kind of applications as a new variant of bandit problems where new arms are dynamically added into the candidate set and the expected reward of each arm decays as the round proceeds. For this new problem, a direct application of the algorithms designed for stochastic MAB (e.g., UCB) will lead to over-estimation of the rewards of old arms, and thus cause a misidentification of the optimal arm. To tackle this challenge, we propose a new algorithm that can adaptively estimate the temporal dynamics in the rewards of the arms, and effectively identify the best arm at a given time point on this basis. When the temporal dynamics are represented by a set of features, the proposed algorithm is able to enjoy a sub-linear regret. Our experiments verify the effectiveness of the proposed algorithm.