Search

Deploying deep reinforcement learning (DRL) in real-world inventory management presents challenges, including dynamic environments and uncertain problem parameters, e.g. demand and lead time distributions. These challenges highlight a research gap, suggesting a need for a unifying framework to model and solve sequential decision-making under parameter uncertainty. We address this by exploring an underexplored area of DRL for inventory management: training generally capable agents (GCAs) under zero-shot generalization (ZSG). Here, GCAs are advanced DRL policies designed to handle a broad range of sampled problem instances with diverse inventory challenges. ZSG refers to the ability to successfully apply learned policies to unseen instances with unknown parameters without retraining. We propose a unifying Super-Markov Decision Process formulation and the Train, then Estimate and Decide (TED) framework to train and deploy a GCA tailored to inventory management applications. The TED framework consists of three phases: training a GCA on varied problem instances, continuously estimating problem parameters during deployment, and making decisions based on these estimates. Applied to periodic review inventory problems with lost sales, cyclic demand patterns, and stochastic lead times, our trained agent, the Generally Capable Lost Sales Network (GC-LSN) consistently outperforms well-known traditional policies when problem parameters are known. Moreover, under conditions where demand and/or lead time distributions are initially unknown and must be estimated, we benchmark against online learning methods that provide worst-case performance guarantees. Our GC-LSN policy, paired with the Kaplan-Meier estimator, is demonstrated to complement these methods by providing superior empirical performance.

Process pattern discovery methods (PPDMs) aim at identifying patterns of interest to users. Existing PPDMs typically are unsupervised and focus on a single dimension of interest, such as discovering frequent patterns. We present an interactive multi interest driven framework for process pattern discovery aimed at identifying patterns that are optimal according to a multi-dimensional analysis goal. The proposed approach is iterative and interactive, thus taking experts knowledge into account during the discovery process. The paper focuses on a concrete analysis goal, i.e., deriving process patterns that affect the process outcome. We evaluate the approach on real world event logs in both interactive and fully automated settings. The approach extracted meaningful patterns validated by expert knowledge in the interactive setting. Patterns extracted in the automated settings consistently led to prediction performance comparable to or better than patterns derived considering single interest dimensions without requiring user defined thresholds., Comment: 16 pages, 5 figures, To appear in the preceedings of 21st International Conference on Business Process Management (BPM), 11-15 September 2023, Utrecht, the Netherlands

Dynamic task assignment involves assigning arriving tasks to a limited number of resources in order to minimize the overall cost of the assignments. To achieve optimal task assignment, it is necessary to model the assignment problem first. While there exist separate formalisms, specifically Markov Decision Processes and (Colored) Petri Nets, to model, execute, and solve different aspects of the problem, there is no integrated modeling technique. To address this gap, this paper proposes Action-Evolution Petri Nets (A-E PN) as a framework for modeling and solving dynamic task assignment problems. A-E PN provides a unified modeling technique that can represent all elements of dynamic task assignment problems. Moreover, A-E PN models are executable, which means they can be used to learn close-to-optimal assignment policies through Reinforcement Learning (RL) without additional modeling effort. To evaluate the framework, we define a taxonomy of archetypical assignment problems. We show for three cases that A-E PN can be used to learn close-to-optimal assignment policies. Our results suggest that A-E PN can be used to model and solve a broad range of dynamic task assignment problems.

In this paper we propose a Deep Reinforcement Learning approach to solve a multimodal transportation planning problem, in which containers must be assigned to a truck or to trains that will transport them to their destination. While traditional planning methods work "offline" (i.e., they take decisions for a batch of containers before the transportation starts), the proposed approach is "online", in that it can take decisions for individual containers, while transportation is being executed. Planning transportation online helps to effectively respond to unforeseen events that may affect the original transportation plan, thus supporting companies in lowering transportation costs. We implemented different container selection heuristics within the proposed Deep Reinforcement Learning algorithm and we evaluated its performance for each heuristic using data that simulate a realistic scenario, designed on the basis of a real case study at a logistics company. The experimental results revealed that the proposed method was able to learn effective patterns of container assignment. It outperformed tested competitors in terms of total transportation costs and utilization of train capacity by 20.48% to 55.32% for the cost and by 7.51% to 20.54% for the capacity. Furthermore, it obtained results within 2.7% for the cost and 0.72% for the capacity of the optimal solution generated by an Integer Linear Programming solver in an offline setting.

Predictive business process monitoring focuses on predicting future characteristics of a running process using event logs. The foresight into process execution promises great potentials for efficient operations, better resource management, and effective customer services. Deep learning-based approaches have been widely adopted in process mining to address the limitations of classical algorithms for solving multiple problems, especially the next event and remaining-time prediction tasks. Nevertheless, designing a deep neural architecture that performs competitively across various tasks is challenging as existing methods fail to capture long-range dependencies in the input sequences and perform poorly for lengthy process traces. In this paper, we propose ProcessTransformer, an approach for learning high-level representations from event logs with an attention-based network. Our model incorporates long-range memory and relies on a self-attention mechanism to establish dependencies between a multitude of event sequences and corresponding outputs. We evaluate the applicability of our technique on nine real event logs. We demonstrate that the transformer-based model outperforms several baselines of prior techniques by obtaining on average above 80% accuracy for the task of predicting the next activity. Our method also perform competitively, compared to baselines, for the tasks of predicting event time and remaining time of a running case

Problem Definition: Are traditional deep reinforcement learning (DRL) algorithms, developed for a broad range of purposes including game-play and robotics, the most suitable machine learning algorithms for applications in inventory control? To what extent would DRL algorithms tailored to the unique characteristics of inventory control problems provide superior performance compared to DRL and traditional benchmarks? Methodology/results: We propose and study Deep Controlled Learning (DCL), a new DRL framework based on approximate policy iteration specifically designed to tackle inventory problems. Comparative evaluations reveal that DCL outperforms existing state-of-the-art heuristics in lost sales inventory control, perishable inventory systems, and inventory systems with random lead times, achieving lower average costs across all test instances and maintaining an optimality gap of no more than 0.1\%. Notably, the same hyperparameter set is utilized across all experiments, underscoring the robustness and generalizability of the proposed method. Managerial implications: These substantial performance and robustness improvements pave the way for the effective application of tailored DRL algorithms to inventory management problems, empowering decision-makers to optimize stock levels, minimize costs, and enhance responsiveness across various industries.

In e-commerce markets, on time delivery is of great importance to customer satisfaction. In this paper, we present a Deep Reinforcement Learning (DRL) approach for deciding how and when orders should be batched and picked in a warehouse to minimize the number of tardy orders. In particular, the technique facilitates making decisions on whether an order should be picked individually (pick-by-order) or picked in a batch with other orders (pick-by-batch), and if so with which other orders. We approach the problem by formulating it as a semi-Markov decision process and develop a vector-based state representation that includes the characteristics of the warehouse system. This allows us to create a deep reinforcement learning solution that learns a strategy by interacting with the environment and solve the problem with a proximal policy optimization algorithm. We evaluate the performance of the proposed DRL approach by comparing it with several batching and sequencing heuristics in different problem settings. The results show that the DRL approach is able to develop a strategy that produces consistent, good solutions and performs better than the proposed heuristics., Comment: Preprint

Transportation planning depends on predictions of the travel times between loading and unloading locations. While accurate techniques exist for making deterministic predictions of travel times based on real-world data, making stochastic predictions remains an open issue. This paper aims to fill this gap by showing how floating car data from TomTom can be used to make stochastic predictions of travel times. It also shows how these predictions are affected by choices that can be made with respect to the level of aggregation of the data in space and time, and by choices regarding the dependence between travel times on different parts of the route.

Typical legacy information systems store data in relational databases. Process mining is a research discipline that analyzes this data to obtain insights into processes. Many different process mining techniques can be applied to data. In current techniques, an XES event log serves as a basis for analysis. However, because of the static characteristic of an XES event log, we need to create one XES file for each process mining question, which leads to overhead and inflexibility. As an alternative, people attempt to perform process mining directly on the data source using so-called intermediate structures. In previous work, we investigated methods to build intermediate structures on source data by executing a basic SQL query on the database. However, the nested form in the SQL query can cause performance issues on the database side. Therefore, in this paper, we propose a native SQL operator for direct process discovery on relational databases. We define a native operator for the simplest form of the intermediate structure, called the "directly follows relation". This approach has been evaluated with big event data and the experimental results show that it performs faster than the state-of-the-art of database approaches., Comment: 12 pages

In this position paper, we present our vision on the future of the logistics business domain and the use of information technology (IT) in this domain. The vision is based on extensive experience with Dutch and European logistics in various contexts and from various perspectives. We expect that the vision also holds for logistics outside Europe. We build our vision in a number of steps. First, we make an inventory of the most important trends in the logistics domain - we call these mega-trends. Next, we do the same for the information technology domain, restricted to technologies that have relevance for logistics. Then, we introduce logistics meta-concepts that we use to describe our vision and relate them to business engineering. We use these three ingredients to analyze leading concepts that we currently observe in the logistics domain. Next, we consolidate all elements into a model that represents our vision of the integrated future of logistics and IT. We elaborate on the role of data platforms and open standards in this integrated vision., Comment: 22 pages, 7 figures, 3 tables

The execution logs that are used for process mining in practice are often obtained by querying an operational database and storing the result in a flat file. Consequently, the data processing power of the database system cannot be used anymore for this information, leading to constrained flexibility in the definition of mining patterns and limited execution performance in mining large logs. Enabling process mining directly on a database - instead of via intermediate storage in a flat file - therefore provides additional flexibility and efficiency. To help facilitate this ideal of in-database process mining, this paper formally defines a database operator that extracts the 'directly follows' relation from an operational database. This operator can both be used to do in-database process mining and to flexibly evaluate process mining related queries, such as: "which employee most frequently changes the 'amount' attribute of a case from one task to the next". We define the operator using the well-known relational algebra that forms the formal underpinning of relational databases. We formally prove equivalence properties of the operator that are useful for query optimization and present time-complexity properties of the operator. By doing so this paper formally defines the necessary relational algebraic elements of a 'directly follows' operator, which are required for implementation of such an operator in a DBMS.

Business process models describe the way of working in an organization. Typically, business process models distinguish between the normal flow of work and exceptions to that normal flow. However, they often present an idealized view. This means that unexpected exceptions - exceptions that are not modelled in the business process model - can also occur in practice. This has an effect on the efficiency of the organization, because information systems are not developed to handle unexpected exceptions. This paper studies the relation between the occurrence of exceptions and operational performance. It does this by analyzing the execution logs of business processes from five organizations, classifying execution paths as normal or exceptional. Subsequently, it analyzes the differences between normal and exceptional paths. The results show that exceptions are related to worse operational performance in terms of a longer throughput time and that unexpected exceptions relate to a stronger increase in throughput time than expected exceptions.

This paper studies the relation between activity on Twitter and sales. While research exists into the relation between Tweets and movie and book sales, this paper shows that the same relations do not hold for products that receive less attention on social media. For such products, classification of Tweets is far more important to determine a relation. Also, for such products advanced statistical relations, in addition to correlation, are required to relate Twitter activity and sales. In a case study that involves Tweets and sales from a company in four countries, the paper shows how, by classifying Tweets, such relations can be identified. In particular, the paper shows evidence that positive Tweets by persons (as opposed to companies) can be used to forecast sales and that peaks in positive Tweets by persons are strongly related to an increase in sales. These results can be used to improve sales forecasts and to increase sales in marketing campaigns.