Search

We study how web publishers should set their floor prices in order to maximize expected revenues when they have access to two selling mechanisms, namely an ad exchange and header bidding, in order to sell impressions on the real-time bidding market. We consider the publisher’s problem under incomplete information, propose bandit-type algorithms, and show that their regret – the performance loss compared to the optimal algorithm – is sub-linear in the time horizon. Experiments illustrate the effectiveness of our algorithms.

Fill-and-finish is among the most commonly outsourced operations in biopharmaceutical manufacturing and involves several challenges. For example, fill-operations have a random production yield, as biopharmaceutical drugs might lose their quality or stability during these operations. In addition, biopharmaceuticals are fragile molecules that need specialized equipment with limited capacity, and the associated production quantities are often strictly regulated. The non-stationary nature of the biopharmaceutical demand and limitations in forecasts add another layer of challenge in production planning. Furthermore, most companies tend to “freeze” their production decisions for a limited period of time, in which they do not react to changes in the manufacturing system. Using such freeze periods helps to improve stability in planning, but comes at a price of reduced flexibility. To address these challenges, we develop a finite-horizon, discounted-cost Markov decision model, and optimize the production decisions in biopharmaceutical fill-and-finish operations. We characterize the structural properties of optimal cost and policies, and propose a new, zone-based decision-making approach for these operations. More specifically, we show that the state space can be partitioned into decision zones that provide guidelines for optimal production policies. We illustrate the use of the model with an industry case study.

This paper studies the problem of allocating semiconductor wafers to customer orders with the objective of minimizing the overallocation prior to assembly. It is an important problem for back-end semiconductor manufacturing as overallocation may have severe impact on operational performance due to excess inventory and unnecessarily occupied manufacturing equipment. In practice, a wafer can contain dies from several different die classes, making the wafer-allocation problem more challenging. As a novel contribution of this work, we explicitly consider the existence of multiple die classes on a wafer in the wafer-allocation problem. An integer linear programming formulation of the class-constrained wafer allocation problem is provided. The formulation is further extended to be more flexible by allowing the dies from different classes on the same wafer to be allocated to distinct customer orders. A real-world case study from the back-end assembly and test facility of a semiconductor manufacturer is presented. Experiments with real-world data show that the proposed method significantly reduces the overallocation performance in current practice and allows planners to quantify the value of flexibility in wafer allocation.

Non-transparent shipping processes of transporting goods with special handling needs (special cargoes) have resulted in inefficiency in the airfreight industry. Special cargo ontology elicits, structures, and stores domain knowledge and represents the domain concepts and relationship between them in a machine-readable format. In this paper, we proposed an ontology population pipeline for the special cargo domain, and as part of the ontology population task, we investigated how to build an efficient information extraction model from low-resource domains based on available domain data for industry use cases. For this purpose, a model is designed for extracting and classifying instances of different relation types between each concept pair. The model is based on a relation representation learning approach built upon a Hierarchical Attention-based Multi-task architecture in the special cargo domain. The results of experiments show that the model could represent the complex semantic information of the domain, and tasks initialized with these representations achieve promising results.

Deep Reinforcement Learning (RL) has achieved high success in solving routing problems. However, state-of-the-art deep RL approaches require a considerable amount of data before they reach reasonable performance. This may be acceptable for small problems, but as instances grow bigger, this fact severely limits the applicability of these methods to many real-world instances. In this work, we study a setting where the agent can access data from previously handcrafted heuristics for the Traveling Salesman Problem. In our setting, the agent has access to demonstrations from 2-opt improvement policies. Our goal is to learn policies that can surpass the quality of the demonstrations while requiring fewer samples than pure RL. In this study, we propose to first learn policies with Imitation Learning (IL), leveraging a small set of demonstration data to accelerate policy learning. Afterward, we combine on policy and value approximation updates to improve performance over the expert's performance. We show that our method learns good policies in a shorter time and using less data than classical policy gradient, which does not incorporate demonstration data into RL. Moreover, in terms of solution quality, it performs similarly to other state-of-the-art deep RL approaches.

In Prognostics and Health Management (PHM) sufficient prior observed degradation data is usually critical for Remaining Useful Lifetime (RUL) prediction. Most previous data-driven prediction methods assume that training (source) and testing (target) condition monitoring data have similar distributions. However, due to different operating conditions, fault modes, noise and equipment updates distribution shift exists across different data domains. This shift reduces the performance of predictive models previously built to specific conditions when no observed run-to-failure data is available for retraining. To address this issue, this paper proposes a new data-driven approach for domain adaptation in prognostics using Long Short-Term Neural Networks (LSTM). We use a time window approach to extract temporal information from time-series data in a source domain with observed RUL values and a target domain containing only sensor information. We propose a Domain Adversarial Neural Network (DANN) approach to learn domain-invariant features that can be used to predict the RUL in the target domain. The experimental results show that the proposed method can provide more reliable RUL predictions under datasets with different operating conditions and fault modes. These results suggest that the proposed method offers a promising approach to performing domain adaptation in practical PHM applications.

We consider randomly failing high-precision machine tools in a discrete manufacturing setting. Before a tool fails, it goes through a defective phase where it can continue processing new products. However, the products processed by a defective tool do not necessarily generate the same reward obtained from the ones processed by a normal tool. The defective phase of the tool is not visible and can only be detected by a costly inspection. The tool can be retired from production to avoid a tool failure and save its salvage value; however, doing so too early causes not fully using the production potential of the tool. We build a Markov decision model and study when it is the right moment to inspect or retire a tool with the objective of maximizing the total expected reward obtained from an individual tool. The structure of the optimal policy is characterized. The implementation of our model by using the real-world maintenance logs at the Philips shaver factory shows that the value of the optimal policy can be substantial compared to the policy currently used in practice.

Stochastic simulation is a commonly used tool by practitioners for evaluating the performance of inventory policies. A typical inventory simulation starts with the determination of the best-fit input models (e.g. probability distribution function of the demand random variable) and then obtains a performance measure estimate under these input models. However, this sequential approach ignores the uncertainty around the input models, leading to inaccurate performance measures, especially when there is limited historical input data. In this paper, we take an alternative approach and propose a simulation replication algorithm that jointly estimates the input models and the performance measure, leading to a credible interval for the performance measure under input-model uncertainty. Our approach builds on a nonparametric Bayesian input model and frees the inventory manager from making any restrictive assumptions on the functional form of the input models. Focusing on a single-product inventory simulation, we show that the proposed method improves the estimation of the service levels when compared to the traditional practice of using the best-fit or the empirical distribution as the unknown demand distribution.

We study a variant of the display-ad allocation problem where an online publisher needs to decide which subset of advertisement slots should be used in order to fulfill guaranteed contracts and which subset should be sold on supply side platforms (SSPs) in order to maximize the expected revenue. Our modeling approach also takes the uncertainty associated with the sale of an impression by an SSP into account. The way that information is revealed over time allows us to model the display-ad allocation problem as a two-stage stochastic program. We refer to our model as the Stochastic Programming (SP) model. Numerical experiments show that the SP model performs well in most cases. We compare the solutions of the SP model with the solutions of an allocation policy (the Priority Assignment (PA) heuristic) that is used in practice. We find that the performance gap between the PA heuristic and the SP model depends on the fraction of total impressions that need to be allocated to the guaranteed contracts. The results suggest that the benefit of using the SP model is highest in periods where the website traffic is high compared with the targets for the guaranteed contracts.

In this work we study a dynamic pricing problem with demand censoring and limited price changes. In our problem there is a seller of a single product that aims to maximize revenue over a finite sales horizon. The seller does not know the form of the mean demand function but does have some limited knowledge. We assume that the seller has a hypothesis set of mean demand functions and that the true mean demand function is an element of this set. Furthermore, the seller faces a business constraint on the number of price changes that is allowed during the sales horizon. More specifically, the number of price changes that the seller is allowed to make is bounded above by a finite integer. We furthermore assume that the seller can only observe the sales (minimum between realized demand and available inventory) and thus that demand is censored. In each period the seller can replenish his inventory to a particular level. The objective of the seller is to set the best price and inventory level in each period of the sales horizon in order to maximize his profit. The profit is determined by the revenue of the sales minus holding costs and costs for lost sales (unsatisfied demand). In determining the best price and inventory level the seller faces and exploration-exploitation trade-off. The seller has to experiment with different prices and inventory levels in order to learn from historical sales data which contains information about market responses to offered prices. On the other hand, the seller also needs to exploit what it has learned and set prices and inventory levels that are optimal given the information collected so far. We propose a heuristic policy for this problem and study its performance using numerical experiments. The results are promising and indicate that the growth rate of regret of the policy is sub-linear with respect to the sales horizon.

One of the main mechanisms that online publishers use in online advertising in order to sell their advertisement space is the real-time bidding (RTB) mechanism. In RTB the publisher sells advertisement space via a second-price auction. Publishers can set a reserve price for their inventory in the second-price auction. In this paper we consider an online publisher that sells advertisement space and propose a method for learning optimal reserve prices in second-price auctions. We study a limited information setting where the values of the bids are not revealed and no historical information about the values of the bids is available. Our proposed method leverages the dynamics of particles in particle swarm optimization (PSO) to set reserve prices and is suitable for non-stationary environments. We also show that, taking the gap between the winning bid and second highest bid into account leads to better decisions for the reserve prices. Experiments using real-life ad auction data show that the proposed method outperforms popular bandit algorithms.

In this paper we consider an online publisher that sells advertisement space and propose a method for learning optimal reserve prices in second-price auctions. We study a limited information setting where the values of the bids are not revealed and no historical information about the values of the bids is available. Our proposed method combines an adaptive search procedure with a fuzzy logic pricing step to set reserve prices and is suitable for non-stationary environments. In the fuzzy logic pricing step, we take the gap between the winning bid and second highest bid into account and show that this leads to better decisions for the reserve prices. Experiments using real-life ad auction data show that the proposed method outperforms popular bandit algorithms.

In this paper we consider an online publisher that sells advertisement space and propose a method for learning optimal reserve prices in second-price auctions. We study a limited information setting where the values of the bids are not revealed and no historical information about the values of the bids is available. Our proposed method is based on the principle of Thompson sampling combined with a particle filter to approximate and sample from the posterior distribution. Our method is suitable for non-stationary environments, and we show that, when the distribution of the winning bid suffers from estimation uncertainty, taking the gap between the winning bid and second highest bid into account leads to better decisions for the reserve prices. Experiments using real-life ad auction data show that the proposed method outperforms popular bandit algorithms.

Parking, matching, scheduling, and routing are common problems in train maintenance. In particular, train units are commonly maintained and cleaned at dedicated shunting yards. The planning problem that results from such situations is referred to as the Train Unit Shunting Problem (TUSP). This problem involves matching arriving train units to service tasks and determining the schedule for departing trains. The TUSP is an important problem as it is used to determine the capacity of shunting yards and arises as a sub-problem of more general scheduling and planning problems. In this paper, we consider the case of the Dutch Railways (NS) TUSP. As the TUSP is complex, NS currently uses a local search (LS) heuristic to determine if an instance of the TUSP has a feasible solution. Given the number of shunting yards and the size of the planning problems, improving the evaluation speed of the LS brings significant computational gain. In this work, we use a machine learning approach that complements the LS and accelerates the search process. We use a Deep Graph Convolutional Neural Network (DGCNN) model to predict the feasibility of solutions obtained during the run of the LS heuristic. We use this model to decide whether to continue or abort the search process. In this way, the computation time is used more efficiently as it is spent on instances that are more likely to be feasible. Using simulations based on real-life instances of the TUSP, we show how our approach improves upon the previous method on prediction accuracy and leads to computational gains for the decision-making process.

This paper provides a risk assessment framework to select shipping lanes for pharmaceutical products. The main categories of risks are determined through an algorithm based on yes/no decisions. Then, according to the risk categories, a Failure Mode and Effects Analysis (FMEA) table is proposed for risk assessment of pharmaceutical product shipments and logistics. The evaluations are based on Intuitionistic Fuzzy Numbers (IFNs) to be able to account for the uncertainty in the experts’ judgments. By using an intuitionistic fuzzy hybrid TOPSIS (Technique for Order Preference by Similarity to Ideal Solution) approach, the evaluated risks of each shipment lane can be scored and prioritized. The proposed TOPSIS-based FMEA approach in the intuitionistic fuzzy environment provides an opportunity to aggregate the risk assessments of different experts in a practically efficient way. Different from the earlier literature, we address risk identification and risk assessment under uncertainty as the two key challenges in group decision making. Our method further provides a framework that integrates the categorization and evaluation of risks with subsequent decision making. A case study of shipping lane selection in the context of air cargo distribution of pharmaceutical products demonstrates a potential implementation of the proposed approach.

A popular method in machine learning for supervised classification is a decision tree. In this work we propose a new framework to learn fuzzy decision trees using mathematical programming. More specifically, we encode the problem of constructing fuzzy decision trees using a Mixed Integer Linear Programming (MIP) model, which can be solved by any optimization solver. We compare the performance of our method with the performance of off-the-shelf decision tree algorithm CART and Fuzzy Inference Systems (FIS) using benchmark data-sets. Our initial results are promising and show the advantages of using non-crisp boundaries for improving classification accuracy on testing data.

We provide a new method for solving a very general model of an assemble-to-order system: multiple products, multiple components that may be demanded in different quantities by different products, batch production, random lead times, and lost sales, modeled as a Markov decision process under the discounted cost criterion. A control policy specifies when a batch of components should be produced and whether an arriving demand for each product should be satisfied. As optimal solutions for our model are computationally intractable for even moderately sized systems, we approximate the optimal cost function by reformulating it on an aggregate state space and restricting each aggregate state to be represented by its extreme original states. Our aggregation drastically reduces the value iteration computational burden. We derive an upper bound on the distance between aggregate and optimal solutions. This guarantees that the value iteration algorithm for the original problem initialized with the aggregate solution converges to the optimal solution. We also establish the optimality of a lattice-dependent base-stock and rationing policy in the aggregate problem when certain product and component characteristics are incorporated into the aggregation/disaggregation schemes. This enables us to further alleviate the value iteration computational burden in the aggregate problem by eliminating suboptimal actions. Leveraging all of our results, we can solve the aggregate problem for systems of up to 22 components, with an average distance of 11.09% from the optimal cost in systems of up to 4 components (for which we could solve the original problem to optimality). The e-companion is available at https://doi.org/10.1287/opre.2017.1710 .

This paper addresses batch scheduling at a back-end semiconductor plant of Nexperia. This complex manufacturing environment is characterized by a large product and batch size variety, numerous parallel machines with large capacity differences, sequence and machine dependent setup times and machine eligibility constraints. A hybrid genetic algorithm is proposed to improve the scheduling process, the main features of which are a local search enhanced crossover mechanism, two additional fast local search procedures and a user-controlled multi-objective fitness function. Testing with real-life production data shows that this multi-objective approach can strike the desired balance between production time, setup time and tardiness, yielding high-quality practically feasible production schedules.

Many inventory control problems require the satisfaction of service-level criteria such as order fill rate, line-item fill rate, and dollar fill rate that are joint across random customer orders consisting of products with correlated demands. This paper introduces a multi-product, joint service-level model for setting inventory targets subject to such criteria under multivariate demand uncertainty. Unlike the extant service-constrained inventory models meeting target service levels on average, we satisfy the target joint service level with a guaranteed probability of beta in every period. We do this by combining computationally tractable approximations for service-level constraints with simulation-based procedures that are easy to implement. Numerical analysis demonstrates the effectiveness of our procedures in comparison to the widely-used heuristics, and provides new insights on managing multi-product inventory with a probabilistic guarantee on joint service-level satisfaction. In particular, as the number of products in the system increases, the beta-guaranteed solution consistently delivers the target service level with only a small percent increase in cost in comparison to the satisfaction of the target service level on average. We further consider positively correlated demands, incomplete orders, and customer segmentation, and obtain inventory targets that deliver beta-guaranteed joint service levels with reduced cost.

We consider a single-product discrete-time inventory model with intermittent demand. In every period, either zero demand or a positive demand is observed with an unknown probability. The distribution of the positive demand is assumed to be from the location-scale family with unknown mean and variance. The functional form of the optimal inventory target is available but it is a function of the unknown intermittent demand parameters that must be estimated from a limited amount of historical demand data. We first quantify the expected cost associated with implementing the optimal inventory policy using the point estimates of the unknown parameters by ignoring the uncertainty around them. We then minimize this expected cost with respect to a threshold variable that factors the statistical estimation errors of the unknown parameters into the inventory decision. We find that the use of an optimized threshold leads to a significant reduction in the a priori expected cost of the decision maker.

Data envelopment analysis (DEA) has proven to be a useful tool for assessing efficiency or productivity of organizations, which is of vital practical importance in managerial decision making. DEA provides a significant amount of information from which analysts and managers derive insights and guidelines to promote their existing performances. Regarding to this fact, effective and methodologic analysis and interpretation of DEA results are very critical. The main objective of this study is then to develop a general decision support system (DSS) framework to analyze the results of basic DEA models. The paper formally shows how the results of DEA models should be structured so that these solutions can be examined and interpreted by analysts through information visualization and data mining techniques effectively. An innovative and convenient DEA solver, SmartDEA, is designed and developed in accordance with the proposed analysis framework. The developed software provides DEA results which are consistent with the framework and are ready-to-analyze with data mining tools, thanks to their specially designed table-based structures. The developed framework is tested and applied in a real world project for benchmarking the vendors of a leading Turkish automotive company. The results show the effectiveness and the efficacy of the proposed framework. (C) 2012 Elsevier Ltd. All rights reserved.

We consider a repeated newsvendor setting where the parameters of the demand distribution are unknown, and we study the problem of setting inventory targets using only a limited amount of historical demand data. We assume that the demand process is autocorrelated and represented by an Autoregressive-To-Anything time series. We represent the marginal demand distribution with the highly flexible Johnson translation system that captures a wide variety of distributional shapes. Using a simulation-based sampling algorithm, we quantify the expected cost due to parameter uncertainty as a function of the length of the historical demand data, the critical fractile, the parameters of the marginal demand distribution, and the autocorrelation of the demand process. We determine the improved inventory-target estimate accounting for this parameter uncertainty via sample-path optimization.

In this paper, a taxonomy of supply chain and logistics innovations was developed and presented. The taxonomy was based on an extensive literature survey of both theoretical research and case studies. The primary goals are to provide guidelines for choosing the most appropriate innovations for a company, and help companies in positioning themselves in the supply of chain innovations landscape. To this end, the three dimensions of supply chain innovations, namely the goals, supply chain attributes, and innovation attributes were identified and classified. The taxonomy allows for the efficient representation of critical supply chain innovations information, and serves the mentioned goals, which are fundamental to companies in a multitude of industries.

Most of the literature on inventory management assumes that the demand distribution and the values of its parameters are known with certainty. In this paper, we consider a repeated newsvendor setting where this is not the case and study the problem of setting inventory targets when there is a limited amount of historical demand data. Consequently, we achieve the following objectives: (1) to quantify the inaccuracy in the inventory-target estimation as a function of the length of the historical demand data, the critical fractile, and the shape parameters of the demand distribution; and (2) to determine the inventory target that minimizes the expected cost and accounts for the uncertainty around the demand parameters estimated from limited historical data. We achieve these objectives by using the concept of expected total operating cost and representing the demand distribution with the highly flexible Johnson translation system. Our procedures require no restrictive assumptions about the first four moments of the demand random variables, and they can be easily implemented in practical settings with reduced expected total operating costs.

We consider an assemble-to-order system with multiple products, multiple components which may be demanded in different quantities by different products, batch ordering of components, random lead times, and lost sales. We model the system as an infinite-horizon Markov decision process under the discounted cost criterion. A control policy specifies when a batch of components should be produced (i.e., inventory replenishment) and whether an arriving demand for each product should be satisfied (i.e., inventory allocation). As optimal solutions for such problems are computationally intractable for even moderate sized systems, we approximate the optimal cost function by reducing the state space of the original problem via a novel aggregation technique that uses knowledge of products' component requirements and components' replenishment batch sizes.We establish that a lattice-dependent base-stock and lattice-dependent rationing policy is the optimal inventory replenishment and allocation policy for the aggregate problem under a disaggregation rule that disaggregates each aggregate state into its two extreme original states. This rule drastically reduces the per iteration computational complexity of the value iteration algorithm for the aggregate problem (without sacrificing much accuracy, according to our numerical experiments). We further alleviate the value iteration computational burden by eliminating suboptimal actions based on our optimal policy structure.For systems in which there is a product that has fulfillment priority over all other products at optimality, we are able to derive finite error bound for the cost function of the aggregate problem. With these bounds we show that the value iteration algorithm in the original problem that starts with the aggregate solution converges to the optimal cost function. Numerical experiments indicate that such an algorithm has distinct computational advantage over the standard value iteration method in the original problem.

Recent works using deep learning to solve the Traveling Salesman Problem (TSP) have focused on learning construction heuristics. Such approaches find TSP solutions of good quality but require additional procedures such as beam search and sampling to improve solutions and achieve state-of-the-art performance. However, few studies have focused on improvement heuristics, where a given solution is improved until reaching a near-optimal one. In this work, we propose to learn a local search heuristic based on 2-opt operators via deep reinforcement learning. We propose a policy gradient algorithm to learn a stochastic policy that selects 2-opt operations given a current solution. Moreover, we introduce a policy neural network that leverages a pointing attention mechanism, which unlike previous works, can be easily extended to more general k-opt moves. Our results show that the learned policies can improve even over random initial solutions and approach near-optimal solutions at a faster rate than previous state-of-the-art deep learning methods., To appear in Proceedings Machine Learning Research - ACML 2020