Search

Nowadays distributed computing environments, large amounts of data are generated from different resources with a high velocity, rendering the data difficult to capture, manage, and process within existing relational databases. Hadoop is a tool to store and process large datasets in a parallel manner across a cluster of machines in a distributed environment. Hadoop brings many benefits like flexibility, scalability, and high fault tolerance; however, it faces some challenges in terms of data access time, I/O operation, and duplicate computations resulting in extra overhead, resource wastage, and poor performance. Many researchers have utilized caching mechanisms to tackle these challenges. For example, they have presented approaches to improve data access time, enhance data locality rate, remove repetitive calculations, reduce the number of I/O operations, decrease the job execution time, and increase resource efficiency. In the current study, we provide a comprehensive overview of caching strategies to improve Hadoop performance. Additionally, a novel classification is introduced based on cache utilization. Using this classification, we analyze the impact on Hadoop performance and discuss the advantages and disadvantages of each group. Finally, a novel hybrid approach called Hybrid Intelligent Cache (HIC) that combines the benefits of two methods from different groups, H-SVM-LRU and CLQLMRS, is presented. Experimental results show that our hybrid method achieves an average improvement of 31.2% in job execution time.

Modern applications can generate a large amount of data from different sources with high velocity, a combination that is difficult to store and process via traditional tools. Hadoop is one framework that is used for the parallel processing of a large amount of data in a distributed environment, however, various challenges can lead to poor performance. Two particular issues that can limit performance are the high access time for I/O operations and the recomputation of intermediate data. The combination of these two issues can result in resource wastage. In recent years, there have been attempts to overcome these problems by using caching mechanisms. Due to cache space limitations, it is crucial to use this space efficiently and avoid cache pollution (the cache contains data that is not used in the future). We propose Hadoop-oriented SVM-LRU (HSVM- LRU) to improve Hadoop performance. For this purpose, we use an intelligent cache replacement algorithm, SVM-LRU, that combines the well-known LRU mechanism with a machine learning algorithm, SVM, to classify cached data into two groups based on their future usage. Experimental results show a significant decrease in execution time as a result of an increased cache hit ratio, leading to a positive impact on Hadoop performance.

Thermal-aware workload distribution is a common approach in the literature for power consumption optimization in data centers. However, data centers also have other operational costs such as the cost of equipment maintenance and replacement. It has been shown that server reliability depends on frequency of their temperature variations, arising from workload transitions due to dynamic demands. In this work, we formulate a nonlinear optimization problem that considers the cost of workload transitions in addition to IT and cooling power consumption. To approximate the solution, we first linearize the problem; the result is a mixed integer programming problem. A modified heuristic is then proposed to approximate the solution of the linear problem. Finally, a Model Predictive Control (MPC) approach is integrated with the proposed heuristics for automatic workload reconfiguration when future demand is not known exactly, but predictions are available. Numerical results show that the proposed schemes are attractive in different settings.

With the current high levels of energy consumption of data centers, reducing power consumption by even a small percentage is beneficial. We propose a framework for thermal-aware workload distribution in a data center to reduce cooling power consumption. The framework includes linearization of the general optimization problem and proposing a heuristic to approximate the solution for the resulting Integer Linear Programming (ILP) problems. We first define a general nonlinear power optimization problem including several cooling parameters, heat recirculation effects, and constraints on server temperatures. We propose to study a linearized version of the problem, which is easier to analyze. As an energy saving scenario and as a proof of concept for our approach, we also consider the possibility that the red-line temperature for idle servers is higher than that for busy servers. For the resulting ILP problem, we propose a heuristic for intelligent rounding of the fractional solution. Through numerical simulations, we compare our heuristics with two baseline algorithms. We also evaluate the performance of the solution of the linearized system on the original system. The results show that the proposed approach can reduce the cooling power consumption by more than 30 percent compared to the case of continuous utilizations and a single red-line temperature.

Epidemics are a serious public health threat, and the resources for mitigating their effects are typically limited. Decision-makers face challenges in forecasting the supply and demand for these resources as prior information about the disease is often not available, the behaviour of the disease can periodically change (either naturally or as a result of public health policies) and can differ by geographical region. Randomized controlled trials (RCTs) using scarce resources such as blood products as a randomized intervention are affected by epidemics. In this work, we discuss a model that is suitable for short-term real-time supply and demand forecasting during emerging outbreaks. We consider a case study of demand forecasting and allocating scarce quantities of COVID-19 Convalescent Plasma (CCP) in an international multi-site RCT involving multiple hospital hubs across Canada (excluding Qu\'ebec). We propose a data-driven mixed-integer programming (MIP) resource allocation model that assigns available resources to maximize a notion of fairness among the resource-demanding entities. Numerical results from applying our MIP model to the case study suggest that our approach can help balance the supply and demand of limited products such as CCP and minimize the unmet demand ratios of the demand entities. We analyze the sensitivity of our model to different allocation settings and show that our model assigns equitable allocations across the entities.

Platelet products are both expensive and have very short shelf lives. As usage rates for platelets are highly variable, the effective management of platelet demand and supply is very important yet challenging. The primary goal of this paper is to present an efficient forecasting model for platelet demand at Canadian Blood Services (CBS). To accomplish this goal, four different demand forecasting methods, ARIMA (Auto Regressive Moving Average), Prophet, lasso regression (least absolute shrinkage and selection operator) and LSTM (Long Short-Term Memory) networks are utilized and evaluated. We use a large clinical dataset for a centralized blood distribution centre for four hospitals in Hamilton, Ontario, spanning from 2010 to 2018 and consisting of daily platelet transfusions along with information such as the product specifications, the recipients' characteristics, and the recipients' laboratory test results. This study is the first to utilize different methods from statistical time series models to data-driven regression and a machine learning technique for platelet transfusion using clinical predictors and with different amounts of data. We find that the multivariate approaches have the highest accuracy in general, however, if sufficient data are available, a simpler time series approach such as ARIMA appears to be sufficient. We also comment on the approach to choose clinical indicators (inputs) for the multivariate models.

For a single server system, Shortest Remaining Processing Time (SRPT) is an optimal size-based policy. In this paper, we discuss scheduling a single-server system when exact information about the jobs' processing times is not available. When the SRPT policy uses estimated processing times, the underestimation of large jobs can significantly degrade performance. We propose a simple heuristic, Size Estimate Hedging (SEH), that only uses estimated processing times for scheduling decisions. A job's priority is increased dynamically according to an SRPT rule until it is determined that it is underestimated, at which time the priority is frozen. Numerical results suggest that SEH has desirable performance for estimation error variance that is consistent with what is seen in practice., Comment: For a published version of this paper refer to Quantitative Evaluation of Systems: 18th International Conference, https: //doi.org/10.1007/978-3-030-85172-9_9

Blood transfusion is one of the most crucial and commonly administered therapeutics worldwide. The need for more accurate and efficient ways to manage blood demand and supply is an increasing concern. Building a technology-based, robust blood demand and supply chain that can achieve the goals of reducing ordering frequency, inventory level, wastage and shortage, while maintaining the safety of blood usage, is essential in modern healthcare systems. In this study, we summarize the key challenges in current demand and supply management for red blood cells (RBCs). We combine ideas from statistical time series modeling, machine learning, and operations research in developing an ordering decision strategy for RBCs, through integrating a hybrid demand forecasting model using clinical predictors and a data-driven multi-period inventory problem considering inventory and reorder constraints. We have applied the integrated ordering strategy to the blood inventory management system in Hamilton, Ontario using a large clinical database from 2008 to 2018. The proposed hybrid demand forecasting model provides robust and accurate predictions, and identifies important clinical predictors for short-term RBC demand forecasting. Compared with the actual historical data, our integrated ordering strategy reduces the inventory level by 40% and decreases the ordering frequency by 60%, with low incidence of shortages and wastage due to expiration. If implemented successfully, our proposed strategy can achieve significant cost savings for healthcare systems and blood suppliers. The proposed ordering strategy is generalizable to other blood products or even other perishable products., Comment: This file includes 30 pages with 10 figures in total. This work has been submitted to Operations Research for Health Care for publication in August 2020

We study certain properties of the function space of autocorrelation functions of Unit Continuous Time Markov Chains (CTMCs). It is shown that under particular conditions, the $L^p$ norm of the autocorrelation function of arbitrary finite state space CTMCs is infinite. Several interesting inferences are made for point processes associated with CTMCs/ Discrete Time Markov Chains (DTMCs).